2024-09-13T16:32:43.143 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T16:32:43.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T16:32:43.265 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830 branch: wip-rishabh-testing-20240910.095512-reef description: fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} email: null first_in_suite: false job_id: '7903830' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rishabh-testing-20240910.095512-reef ceph: cephfs: fs: - name: a - name: b session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - Replacing daemon mds - SLOW_OPS - slow request sha1: 50bbe7f395997dcd871813949c49addbbf630361 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: cephfs_name: a client.1: cephfs_name: b install: ceph: flavor: default sha1: 50bbe7f395997dcd871813949c49addbbf630361 kclient: client.0: cephfs_name: a client.1: cephfs_name: b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.095512-reef sha1: 50bbe7f395997dcd871813949c49addbbf630361 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 1152 sha1: 50bbe7f395997dcd871813949c49addbbf630361 sleep_before_teardown: 0 subset: 984/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.095512-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 50bbe7f395997dcd871813949c49addbbf630361 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAldJo9viHmxc+L/pgE39hKns4NRV82SNIJRD6Yzol1BeZX+walspT7a2bVrHaVZlvrpQ8cAdB+DlureXkgcnuk= smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAemyebAsO1/zC3pTAUCIvemeQVaFR/9Hzmok7w7gTXMEhnJdbpZj4oerKNCOTCmRXbGcMa1uaGanA/o270Mn0I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - mds_thrash: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:36:41 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T16:32:43.265 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa; will attempt to use it 2024-09-13T16:32:43.266 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa/tasks 2024-09-13T16:32:43.266 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T16:32:43.268 INFO:teuthology.task.internal:Checking packages... 2024-09-13T16:32:43.290 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '50bbe7f395997dcd871813949c49addbbf630361' 2024-09-13T16:32:43.290 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T16:32:43.290 INFO:teuthology.packaging:ref: None 2024-09-13T16:32:43.290 INFO:teuthology.packaging:tag: None 2024-09-13T16:32:43.290 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.095512-reef 2024-09-13T16:32:43.291 INFO:teuthology.packaging:sha1: 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:32:43.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rishabh-testing-20240910.095512-reef 2024-09-13T16:32:43.445 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-850-g50bbe7f3-1jammy 2024-09-13T16:32:43.447 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T16:32:43.454 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T16:32:43.454 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T16:32:43.477 INFO:teuthology.task.internal:Saving configuration 2024-09-13T16:32:43.495 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T16:32:43.502 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T16:32:43.533 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 16:26:14.888507', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIL37vUzfjk7Ak78eBsBe/3I7cXdAnz6yawrEWH+DCtPi'} 2024-09-13T16:32:43.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 16:26:14.886374', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG2spz2Azi9OGQkoIQBWnmsFu3z8JpRSOdWn3MpxmOey'} 2024-09-13T16:32:43.564 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T16:32:43.571 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-13T16:32:43.571 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-09-13T16:32:43.571 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T16:32:43.650 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f240a94fbe0>, signals=[15]) 2024-09-13T16:32:43.650 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T16:32:43.656 INFO:teuthology.task.internal:Opening connections... 2024-09-13T16:32:43.656 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T16:32:43.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:32:43.754 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T16:32:43.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:32:43.840 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T16:32:43.847 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-09-13T16:32:43.853 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-09-13T16:32:43.853 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Ubuntu" 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="22.04" 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_CODENAME=jammy 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:ID=ubuntu 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE=debian 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T16:32:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:UBUNTU_CODENAME=jammy 2024-09-13T16:32:43.902 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-09-13T16:32:43.929 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-09-13T16:32:43.933 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-09-13T16:32:43.934 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Ubuntu" 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="22.04" 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_CODENAME=jammy 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:ID=ubuntu 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE=debian 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T16:32:43.980 INFO:teuthology.orchestra.run.smithi084.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T16:32:43.981 INFO:teuthology.orchestra.run.smithi084.stdout:UBUNTU_CODENAME=jammy 2024-09-13T16:32:43.981 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-09-13T16:32:44.011 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T16:32:44.020 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T16:32:44.026 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T16:32:44.027 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-09-13T16:32:44.029 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2024-09-13T16:32:44.033 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T16:32:44.040 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T16:32:44.040 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-09-13T16:32:44.077 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2024-09-13T16:32:44.092 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T16:32:44.167 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T16:32:44.181 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T16:32:44.181 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T16:32:44.182 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T16:32:44.182 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T16:32:44.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:32:44.187 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-13T16:32:44.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:32:44.188 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-09-13T16:32:44.236 INFO:teuthology.orchestra.run.smithi039.stdout:5.15.0-56-generic 2024-09-13T16:32:44.236 INFO:teuthology.task.kernel:Running kernel on smithi039: 5.15.0-56-generic 2024-09-13T16:32:44.236 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get clean 2024-09-13T16:32:44.237 INFO:teuthology.orchestra.run.smithi084.stdout:5.15.0-56-generic 2024-09-13T16:32:44.237 INFO:teuthology.task.kernel:Running kernel on smithi084: 5.15.0-56-generic 2024-09-13T16:32:44.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo apt-get clean 2024-09-13T16:32:44.344 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get update 2024-09-13T16:32:44.381 DEBUG:teuthology.orchestra.run.smithi084:> sudo apt-get update 2024-09-13T16:32:44.484 INFO:teuthology.orchestra.run.smithi039.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T16:32:44.542 INFO:teuthology.orchestra.run.smithi084.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T16:32:44.588 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T16:32:44.651 INFO:teuthology.orchestra.run.smithi084.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T16:32:44.669 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T16:32:44.708 INFO:teuthology.orchestra.run.smithi039.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T16:32:44.733 INFO:teuthology.orchestra.run.smithi084.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T16:32:44.747 INFO:teuthology.orchestra.run.smithi084.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T16:32:44.809 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T16:32:44.814 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T16:32:44.817 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T16:32:44.817 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T16:32:44.837 INFO:teuthology.orchestra.run.smithi039.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T16:32:44.837 INFO:teuthology.orchestra.run.smithi039.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T16:32:44.841 INFO:teuthology.orchestra.run.smithi039.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T16:32:44.841 INFO:teuthology.orchestra.run.smithi039.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T16:32:44.845 INFO:teuthology.orchestra.run.smithi084.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T16:32:44.849 INFO:teuthology.orchestra.run.smithi039.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T16:32:44.854 INFO:teuthology.orchestra.run.smithi039.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T16:32:44.855 INFO:teuthology.orchestra.run.smithi039.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T16:32:44.856 INFO:teuthology.orchestra.run.smithi039.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T16:32:44.856 INFO:teuthology.orchestra.run.smithi039.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T16:32:44.870 INFO:teuthology.orchestra.run.smithi039.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T16:32:44.870 INFO:teuthology.orchestra.run.smithi039.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T16:32:44.883 INFO:teuthology.orchestra.run.smithi084.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T16:32:44.885 INFO:teuthology.orchestra.run.smithi084.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T16:32:44.886 INFO:teuthology.orchestra.run.smithi084.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T16:32:44.886 INFO:teuthology.orchestra.run.smithi084.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T16:32:44.919 INFO:teuthology.orchestra.run.smithi084.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T16:32:44.925 INFO:teuthology.orchestra.run.smithi084.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T16:32:44.925 INFO:teuthology.orchestra.run.smithi084.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T16:32:44.931 INFO:teuthology.orchestra.run.smithi084.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T16:32:44.937 INFO:teuthology.orchestra.run.smithi084.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T16:32:44.938 INFO:teuthology.orchestra.run.smithi084.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T16:32:44.938 INFO:teuthology.orchestra.run.smithi084.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T16:32:44.939 INFO:teuthology.orchestra.run.smithi084.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T16:32:44.965 INFO:teuthology.orchestra.run.smithi084.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T16:32:44.965 INFO:teuthology.orchestra.run.smithi084.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T16:32:45.025 INFO:teuthology.orchestra.run.smithi039.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T16:32:45.090 INFO:teuthology.orchestra.run.smithi084.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T16:32:45.164 INFO:teuthology.orchestra.run.smithi039.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T16:32:45.230 INFO:teuthology.orchestra.run.smithi084.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T16:32:45.481 INFO:teuthology.orchestra.run.smithi039.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T16:32:45.516 INFO:teuthology.orchestra.run.smithi039.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T16:32:45.531 INFO:teuthology.orchestra.run.smithi039.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T16:32:45.531 INFO:teuthology.orchestra.run.smithi039.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T16:32:45.533 INFO:teuthology.orchestra.run.smithi039.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T16:32:45.548 INFO:teuthology.orchestra.run.smithi084.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T16:32:45.583 INFO:teuthology.orchestra.run.smithi084.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T16:32:45.598 INFO:teuthology.orchestra.run.smithi084.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T16:32:45.598 INFO:teuthology.orchestra.run.smithi084.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T16:32:45.600 INFO:teuthology.orchestra.run.smithi084.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T16:32:45.601 INFO:teuthology.orchestra.run.smithi039.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T16:32:45.618 INFO:teuthology.orchestra.run.smithi039.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T16:32:45.618 INFO:teuthology.orchestra.run.smithi039.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T16:32:45.639 INFO:teuthology.orchestra.run.smithi039.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T16:32:45.669 INFO:teuthology.orchestra.run.smithi039.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T16:32:45.669 INFO:teuthology.orchestra.run.smithi084.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T16:32:45.674 INFO:teuthology.orchestra.run.smithi039.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T16:32:45.676 INFO:teuthology.orchestra.run.smithi039.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T16:32:45.677 INFO:teuthology.orchestra.run.smithi039.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T16:32:45.678 INFO:teuthology.orchestra.run.smithi039.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T16:32:45.683 INFO:teuthology.orchestra.run.smithi084.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T16:32:45.684 INFO:teuthology.orchestra.run.smithi084.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T16:32:45.718 INFO:teuthology.orchestra.run.smithi084.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T16:32:45.726 INFO:teuthology.orchestra.run.smithi039.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T16:32:45.726 INFO:teuthology.orchestra.run.smithi039.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T16:32:45.728 INFO:teuthology.orchestra.run.smithi039.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T16:32:45.729 INFO:teuthology.orchestra.run.smithi039.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T16:32:45.737 INFO:teuthology.orchestra.run.smithi084.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T16:32:45.742 INFO:teuthology.orchestra.run.smithi084.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T16:32:45.743 INFO:teuthology.orchestra.run.smithi084.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T16:32:45.744 INFO:teuthology.orchestra.run.smithi084.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T16:32:45.745 INFO:teuthology.orchestra.run.smithi084.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T16:32:45.797 INFO:teuthology.orchestra.run.smithi084.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T16:32:45.797 INFO:teuthology.orchestra.run.smithi084.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T16:32:45.799 INFO:teuthology.orchestra.run.smithi084.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T16:32:45.808 INFO:teuthology.orchestra.run.smithi039.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T16:32:45.808 INFO:teuthology.orchestra.run.smithi039.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T16:32:45.808 INFO:teuthology.orchestra.run.smithi039.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T16:32:45.809 INFO:teuthology.orchestra.run.smithi039.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T16:32:45.809 INFO:teuthology.orchestra.run.smithi039.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T16:32:45.827 INFO:teuthology.orchestra.run.smithi084.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T16:32:45.828 INFO:teuthology.orchestra.run.smithi084.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T16:32:45.828 INFO:teuthology.orchestra.run.smithi084.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T16:32:45.829 INFO:teuthology.orchestra.run.smithi084.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T16:32:45.829 INFO:teuthology.orchestra.run.smithi084.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T16:32:45.879 INFO:teuthology.orchestra.run.smithi084.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T16:32:51.215 INFO:teuthology.orchestra.run.smithi084.stdout:Fetched 16.0 MB in 2s (7,493 kB/s) 2024-09-13T16:32:51.443 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 16.0 MB in 2s (7,131 kB/s) 2024-09-13T16:32:52.614 INFO:teuthology.orchestra.run.smithi084.stdout:Reading package lists... 2024-09-13T16:32:52.631 DEBUG:teuthology.orchestra.run.smithi084:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T16:32:52.743 INFO:teuthology.orchestra.run.smithi084.stdout:Reading package lists... 2024-09-13T16:32:52.846 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-13T16:32:52.863 DEBUG:teuthology.orchestra.run.smithi039:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T16:32:52.931 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-13T16:32:52.955 INFO:teuthology.orchestra.run.smithi084.stdout:Building dependency tree... 2024-09-13T16:32:52.956 INFO:teuthology.orchestra.run.smithi084.stdout:Reading state information... 2024-09-13T16:32:53.128 INFO:teuthology.orchestra.run.smithi039.stdout:Building dependency tree... 2024-09-13T16:32:53.128 INFO:teuthology.orchestra.run.smithi039.stdout:Reading state information... 2024-09-13T16:32:53.144 INFO:teuthology.orchestra.run.smithi084.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:32:53.144 INFO:teuthology.orchestra.run.smithi084.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T16:32:53.145 INFO:teuthology.orchestra.run.smithi084.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T16:32:53.145 INFO:teuthology.orchestra.run.smithi084.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T16:32:53.145 INFO:teuthology.orchestra.run.smithi084.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T16:32:53.146 INFO:teuthology.orchestra.run.smithi084.stdout: python2.7-minimal 2024-09-13T16:32:53.146 INFO:teuthology.orchestra.run.smithi084.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:32:53.147 INFO:teuthology.orchestra.run.smithi084.stdout:The following additional packages will be installed: 2024-09-13T16:32:53.147 INFO:teuthology.orchestra.run.smithi084.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T16:32:53.147 INFO:teuthology.orchestra.run.smithi084.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T16:32:53.147 INFO:teuthology.orchestra.run.smithi084.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T16:32:53.149 INFO:teuthology.orchestra.run.smithi084.stdout:Suggested packages: 2024-09-13T16:32:53.149 INFO:teuthology.orchestra.run.smithi084.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T16:32:53.149 INFO:teuthology.orchestra.run.smithi084.stdout:Recommended packages: 2024-09-13T16:32:53.149 INFO:teuthology.orchestra.run.smithi084.stdout: thermald 2024-09-13T16:32:53.204 INFO:teuthology.orchestra.run.smithi084.stdout:The following NEW packages will be installed: 2024-09-13T16:32:53.204 INFO:teuthology.orchestra.run.smithi084.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T16:32:53.204 INFO:teuthology.orchestra.run.smithi084.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T16:32:53.205 INFO:teuthology.orchestra.run.smithi084.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T16:32:53.206 INFO:teuthology.orchestra.run.smithi084.stdout:The following packages will be upgraded: 2024-09-13T16:32:53.206 INFO:teuthology.orchestra.run.smithi084.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T16:32:53.259 INFO:teuthology.orchestra.run.smithi084.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T16:32:53.259 INFO:teuthology.orchestra.run.smithi084.stdout:Need to get 113 MB of archives. 2024-09-13T16:32:53.259 INFO:teuthology.orchestra.run.smithi084.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T16:32:53.259 INFO:teuthology.orchestra.run.smithi084.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T16:32:53.301 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:32:53.301 INFO:teuthology.orchestra.run.smithi039.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T16:32:53.301 INFO:teuthology.orchestra.run.smithi039.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T16:32:53.302 INFO:teuthology.orchestra.run.smithi039.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T16:32:53.302 INFO:teuthology.orchestra.run.smithi039.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T16:32:53.302 INFO:teuthology.orchestra.run.smithi039.stdout: python2.7-minimal 2024-09-13T16:32:53.302 INFO:teuthology.orchestra.run.smithi039.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:32:53.303 INFO:teuthology.orchestra.run.smithi039.stdout:The following additional packages will be installed: 2024-09-13T16:32:53.303 INFO:teuthology.orchestra.run.smithi039.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T16:32:53.303 INFO:teuthology.orchestra.run.smithi039.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T16:32:53.303 INFO:teuthology.orchestra.run.smithi039.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T16:32:53.304 INFO:teuthology.orchestra.run.smithi039.stdout:Suggested packages: 2024-09-13T16:32:53.304 INFO:teuthology.orchestra.run.smithi039.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T16:32:53.304 INFO:teuthology.orchestra.run.smithi039.stdout:Recommended packages: 2024-09-13T16:32:53.305 INFO:teuthology.orchestra.run.smithi039.stdout: thermald 2024-09-13T16:32:53.345 INFO:teuthology.orchestra.run.smithi039.stdout:The following NEW packages will be installed: 2024-09-13T16:32:53.346 INFO:teuthology.orchestra.run.smithi039.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T16:32:53.346 INFO:teuthology.orchestra.run.smithi039.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T16:32:53.346 INFO:teuthology.orchestra.run.smithi039.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T16:32:53.347 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages will be upgraded: 2024-09-13T16:32:53.347 INFO:teuthology.orchestra.run.smithi039.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T16:32:53.527 INFO:teuthology.orchestra.run.smithi039.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T16:32:53.527 INFO:teuthology.orchestra.run.smithi039.stdout:Need to get 113 MB of archives. 2024-09-13T16:32:53.527 INFO:teuthology.orchestra.run.smithi039.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T16:32:53.527 INFO:teuthology.orchestra.run.smithi039.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T16:32:53.732 INFO:teuthology.orchestra.run.smithi084.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T16:32:53.830 INFO:teuthology.orchestra.run.smithi084.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T16:32:54.641 INFO:teuthology.orchestra.run.smithi084.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T16:32:54.641 INFO:teuthology.orchestra.run.smithi084.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T16:32:54.641 INFO:teuthology.orchestra.run.smithi084.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T16:32:54.762 INFO:teuthology.orchestra.run.smithi084.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T16:32:54.861 INFO:teuthology.orchestra.run.smithi084.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T16:32:54.911 INFO:teuthology.orchestra.run.smithi039.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T16:32:55.261 INFO:teuthology.orchestra.run.smithi084.stdout:Fetched 113 MB in 2s (68.8 MB/s) 2024-09-13T16:32:55.272 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T16:32:55.452 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T16:32:57.552 INFO:teuthology.orchestra.run.smithi039.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T16:32:57.552 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T16:32:57.553 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T16:32:57.564 INFO:teuthology.orchestra.run.smithi084.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T16:32:57.569 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T16:32:57.757 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:32:57.923 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T16:32:58.008 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T16:32:58.436 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 113 MB in 5s (24.3 MB/s) 2024-09-13T16:32:58.605 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T16:32:59.826 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T16:32:59.847 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T16:32:59.930 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:00.437 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T16:33:00.457 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T16:33:00.496 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:00.572 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T16:33:00.576 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T16:33:00.644 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:02.944 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T16:33:02.965 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T16:33:03.056 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:03.464 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T16:33:03.484 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T16:33:03.522 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:07.202 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T16:33:07.327 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T16:33:07.684 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T16:33:07.805 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T16:33:08.141 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T16:33:08.163 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T16:33:08.208 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T16:33:09.420 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T16:33:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T16:33:09.863 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T16:33:09.991 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T16:33:10.278 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T16:33:10.301 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T16:33:10.336 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T16:33:14.224 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T16:33:14.261 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T16:33:14.324 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:15.623 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T16:33:15.643 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T16:33:15.681 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:17.018 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T16:33:17.143 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T16:33:17.510 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T16:33:17.640 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:17.783 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T16:33:17.910 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:18.756 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T16:33:18.853 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T16:33:19.162 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T16:33:19.228 INFO:teuthology.orchestra.run.smithi084.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T16:33:19.228 INFO:teuthology.orchestra.run.smithi084.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T16:33:19.228 INFO:teuthology.orchestra.run.smithi084.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T16:33:19.228 INFO:teuthology.orchestra.run.smithi084.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T16:33:19.284 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:19.410 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T16:33:19.413 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:19.537 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:20.878 INFO:teuthology.orchestra.run.smithi039.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T16:33:20.878 INFO:teuthology.orchestra.run.smithi039.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T16:33:20.878 INFO:teuthology.orchestra.run.smithi039.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T16:33:20.878 INFO:teuthology.orchestra.run.smithi039.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T16:33:21.050 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:21.222 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:22.467 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:22.624 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T16:33:22.775 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T16:33:22.884 INFO:teuthology.orchestra.run.smithi084.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:22.943 INFO:teuthology.orchestra.run.smithi084.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T16:33:22.943 INFO:teuthology.orchestra.run.smithi084.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T16:33:23.911 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T16:33:24.045 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T16:33:24.180 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T16:33:24.251 INFO:teuthology.orchestra.run.smithi039.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T16:33:24.252 INFO:teuthology.orchestra.run.smithi039.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T16:33:37.112 INFO:teuthology.orchestra.run.smithi084.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T16:33:37.112 INFO:teuthology.orchestra.run.smithi084.stdout:Sourcing file `/etc/default/grub' 2024-09-13T16:33:37.124 INFO:teuthology.orchestra.run.smithi084.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T16:33:37.144 INFO:teuthology.orchestra.run.smithi084.stdout:Generating grub configuration file ... 2024-09-13T16:33:37.483 INFO:teuthology.orchestra.run.smithi084.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T16:33:37.497 INFO:teuthology.orchestra.run.smithi084.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T16:33:37.810 INFO:teuthology.orchestra.run.smithi084.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T16:33:37.815 INFO:teuthology.orchestra.run.smithi084.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T16:33:37.891 INFO:teuthology.orchestra.run.smithi084.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T16:33:37.897 INFO:teuthology.orchestra.run.smithi084.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T16:33:38.091 INFO:teuthology.orchestra.run.smithi084.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T16:33:38.091 INFO:teuthology.orchestra.run.smithi084.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T16:33:38.091 INFO:teuthology.orchestra.run.smithi084.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T16:33:38.122 INFO:teuthology.orchestra.run.smithi084.stdout:done 2024-09-13T16:33:38.530 INFO:teuthology.orchestra.run.smithi039.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T16:33:38.530 INFO:teuthology.orchestra.run.smithi039.stdout:Sourcing file `/etc/default/grub' 2024-09-13T16:33:38.541 INFO:teuthology.orchestra.run.smithi039.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T16:33:38.568 INFO:teuthology.orchestra.run.smithi039.stdout:Generating grub configuration file ... 2024-09-13T16:33:38.606 DEBUG:teuthology.orchestra.run.smithi084:> dpkg -s linux-image-generic 2024-09-13T16:33:38.634 INFO:teuthology.orchestra.run.smithi084.stdout:Package: linux-image-generic 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Status: install ok installed 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Priority: optional 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Section: kernel 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Installed-Size: 21 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: amd64 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Source: linux-meta 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Version: 5.15.0.121.121 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Recommends: thermald 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout:Description: Generic Linux kernel image 2024-09-13T16:33:38.635 INFO:teuthology.orchestra.run.smithi084.stdout: This package will always depend on the latest generic kernel image 2024-09-13T16:33:38.636 INFO:teuthology.orchestra.run.smithi084.stdout: available. 2024-09-13T16:33:38.636 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T16:33:38.636 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T16:33:38.636 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T16:33:38.636 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T16:33:38.636 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi084.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T16:33:38.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo apt-get clean 2024-09-13T16:33:38.820 DEBUG:teuthology.orchestra.run.smithi084:> sudo apt-get update 2024-09-13T16:33:38.834 INFO:teuthology.orchestra.run.smithi039.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T16:33:38.861 INFO:teuthology.orchestra.run.smithi039.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T16:33:38.948 INFO:teuthology.orchestra.run.smithi084.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T16:33:38.957 INFO:teuthology.orchestra.run.smithi084.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T16:33:38.974 INFO:teuthology.orchestra.run.smithi084.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T16:33:39.069 INFO:teuthology.orchestra.run.smithi084.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T16:33:39.214 INFO:teuthology.orchestra.run.smithi039.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T16:33:39.221 INFO:teuthology.orchestra.run.smithi039.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T16:33:39.300 INFO:teuthology.orchestra.run.smithi039.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T16:33:39.306 INFO:teuthology.orchestra.run.smithi039.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T16:33:39.471 INFO:teuthology.orchestra.run.smithi039.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T16:33:39.472 INFO:teuthology.orchestra.run.smithi039.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T16:33:39.472 INFO:teuthology.orchestra.run.smithi039.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T16:33:39.526 INFO:teuthology.orchestra.run.smithi039.stdout:done 2024-09-13T16:33:40.131 DEBUG:teuthology.orchestra.run.smithi039:> dpkg -s linux-image-generic 2024-09-13T16:33:40.150 INFO:teuthology.orchestra.run.smithi039.stdout:Package: linux-image-generic 2024-09-13T16:33:40.150 INFO:teuthology.orchestra.run.smithi039.stdout:Status: install ok installed 2024-09-13T16:33:40.150 INFO:teuthology.orchestra.run.smithi039.stdout:Priority: optional 2024-09-13T16:33:40.150 INFO:teuthology.orchestra.run.smithi039.stdout:Section: kernel 2024-09-13T16:33:40.150 INFO:teuthology.orchestra.run.smithi039.stdout:Installed-Size: 21 2024-09-13T16:33:40.150 INFO:teuthology.orchestra.run.smithi039.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T16:33:40.150 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: amd64 2024-09-13T16:33:40.151 INFO:teuthology.orchestra.run.smithi039.stdout:Source: linux-meta 2024-09-13T16:33:40.151 INFO:teuthology.orchestra.run.smithi039.stdout:Version: 5.15.0.121.121 2024-09-13T16:33:40.151 INFO:teuthology.orchestra.run.smithi039.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T16:33:40.151 INFO:teuthology.orchestra.run.smithi039.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T16:33:40.151 INFO:teuthology.orchestra.run.smithi039.stdout:Recommends: thermald 2024-09-13T16:33:40.151 INFO:teuthology.orchestra.run.smithi039.stdout:Description: Generic Linux kernel image 2024-09-13T16:33:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: This package will always depend on the latest generic kernel image 2024-09-13T16:33:40.151 INFO:teuthology.orchestra.run.smithi039.stdout: available. 2024-09-13T16:33:40.151 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T16:33:40.151 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T16:33:40.152 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T16:33:40.152 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T16:33:40.152 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi039.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T16:33:40.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get clean 2024-09-13T16:33:40.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get update 2024-09-13T16:33:40.365 INFO:teuthology.orchestra.run.smithi084.stdout:Reading package lists... 2024-09-13T16:33:40.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T16:33:40.444 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T16:33:40.444 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T16:33:40.448 INFO:teuthology.orchestra.run.smithi084.stdout:Reading package lists... 2024-09-13T16:33:40.452 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T16:33:40.469 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T16:33:40.675 INFO:teuthology.orchestra.run.smithi084.stdout:Building dependency tree... 2024-09-13T16:33:40.675 INFO:teuthology.orchestra.run.smithi084.stdout:Reading state information... 2024-09-13T16:33:40.865 INFO:teuthology.orchestra.run.smithi084.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T16:33:40.865 INFO:teuthology.orchestra.run.smithi084.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:33:40.866 INFO:teuthology.orchestra.run.smithi084.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T16:33:40.866 INFO:teuthology.orchestra.run.smithi084.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T16:33:40.866 INFO:teuthology.orchestra.run.smithi084.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T16:33:40.866 INFO:teuthology.orchestra.run.smithi084.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T16:33:40.866 INFO:teuthology.orchestra.run.smithi084.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T16:33:40.866 INFO:teuthology.orchestra.run.smithi084.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:33:40.909 INFO:teuthology.orchestra.run.smithi084.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T16:33:40.911 DEBUG:teuthology.orchestra.run.smithi084:> dpkg -s linux-image-generic 2024-09-13T16:33:40.926 INFO:teuthology.orchestra.run.smithi084.stdout:Package: linux-image-generic 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Status: install ok installed 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Priority: optional 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Section: kernel 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Installed-Size: 21 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: amd64 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Source: linux-meta 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Version: 5.15.0.121.121 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Recommends: thermald 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout:Description: Generic Linux kernel image 2024-09-13T16:33:40.927 INFO:teuthology.orchestra.run.smithi084.stdout: This package will always depend on the latest generic kernel image 2024-09-13T16:33:40.928 INFO:teuthology.orchestra.run.smithi084.stdout: available. 2024-09-13T16:33:40.928 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-09-13T16:33:40.975 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.njXIvwIzym 2024-09-13T16:33:40.975 DEBUG:teuthology.orchestra.run.smithi084:> sudo cp /boot/grub/grub.cfg /tmp/tmp.njXIvwIzym 2024-09-13T16:33:41.031 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.njXIvwIzym 2024-09-13T16:33:41.148 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.njXIvwIzym is 10KB 2024-09-13T16:33:41.161 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.njXIvwIzym 2024-09-13T16:33:41.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T16:33:41.219 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:33:41.219 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T16:33:41.219 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T16:33:41.286 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T16:33:41.286 DEBUG:teuthology.orchestra.run.smithi084:> sudo update-grub 2024-09-13T16:33:41.889 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-13T16:33:41.907 DEBUG:teuthology.orchestra.run.smithi039:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T16:33:41.973 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-13T16:33:42.188 INFO:teuthology.orchestra.run.smithi039.stdout:Building dependency tree... 2024-09-13T16:33:42.188 INFO:teuthology.orchestra.run.smithi039.stdout:Reading state information... 2024-09-13T16:33:42.391 INFO:teuthology.orchestra.run.smithi039.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T16:33:42.391 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:33:42.391 INFO:teuthology.orchestra.run.smithi039.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T16:33:42.391 INFO:teuthology.orchestra.run.smithi039.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T16:33:42.391 INFO:teuthology.orchestra.run.smithi039.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T16:33:42.392 INFO:teuthology.orchestra.run.smithi039.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T16:33:42.392 INFO:teuthology.orchestra.run.smithi039.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T16:33:42.392 INFO:teuthology.orchestra.run.smithi039.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:33:42.437 INFO:teuthology.orchestra.run.smithi039.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T16:33:42.439 DEBUG:teuthology.orchestra.run.smithi039:> dpkg -s linux-image-generic 2024-09-13T16:33:42.455 INFO:teuthology.orchestra.run.smithi039.stdout:Package: linux-image-generic 2024-09-13T16:33:42.455 INFO:teuthology.orchestra.run.smithi039.stdout:Status: install ok installed 2024-09-13T16:33:42.455 INFO:teuthology.orchestra.run.smithi039.stdout:Priority: optional 2024-09-13T16:33:42.455 INFO:teuthology.orchestra.run.smithi039.stdout:Section: kernel 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:Installed-Size: 21 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: amd64 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:Source: linux-meta 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:Version: 5.15.0.121.121 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:Recommends: thermald 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout:Description: Generic Linux kernel image 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout: This package will always depend on the latest generic kernel image 2024-09-13T16:33:42.456 INFO:teuthology.orchestra.run.smithi039.stdout: available. 2024-09-13T16:33:42.457 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-13T16:33:42.503 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.pWjAXSmNR4 2024-09-13T16:33:42.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /boot/grub/grub.cfg /tmp/tmp.pWjAXSmNR4 2024-09-13T16:33:42.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.pWjAXSmNR4 2024-09-13T16:33:42.618 INFO:teuthology.orchestra.run.smithi084.stderr:Sourcing file `/etc/default/grub' 2024-09-13T16:33:42.619 INFO:teuthology.orchestra.run.smithi084.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T16:33:42.628 INFO:teuthology.orchestra.run.smithi084.stderr:Generating grub configuration file ... 2024-09-13T16:33:42.681 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.pWjAXSmNR4 is 10KB 2024-09-13T16:33:42.694 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.pWjAXSmNR4 2024-09-13T16:33:42.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T16:33:42.754 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:33:42.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T16:33:42.755 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T16:33:42.812 INFO:teuthology.orchestra.run.smithi084.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T16:33:42.818 INFO:teuthology.orchestra.run.smithi084.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T16:33:42.822 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T16:33:42.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo update-grub 2024-09-13T16:33:43.078 INFO:teuthology.orchestra.run.smithi084.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T16:33:43.083 INFO:teuthology.orchestra.run.smithi084.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T16:33:43.127 INFO:teuthology.orchestra.run.smithi084.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T16:33:43.132 INFO:teuthology.orchestra.run.smithi084.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T16:33:43.260 INFO:teuthology.orchestra.run.smithi084.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T16:33:43.260 INFO:teuthology.orchestra.run.smithi084.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T16:33:43.260 INFO:teuthology.orchestra.run.smithi084.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T16:33:43.275 INFO:teuthology.orchestra.run.smithi084.stderr:done 2024-09-13T16:33:43.277 DEBUG:teuthology.orchestra.run.smithi084:> sudo shutdown -r now 2024-09-13T16:33:43.953 INFO:teuthology.orchestra.run.smithi039.stderr:Sourcing file `/etc/default/grub' 2024-09-13T16:33:43.955 INFO:teuthology.orchestra.run.smithi039.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T16:33:43.957 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2024-09-13T16:33:44.247 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T16:33:44.255 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T16:33:44.499 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T16:33:44.505 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T16:33:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T16:33:44.552 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T16:33:44.714 INFO:teuthology.orchestra.run.smithi039.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T16:33:44.715 INFO:teuthology.orchestra.run.smithi039.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T16:33:44.715 INFO:teuthology.orchestra.run.smithi039.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T16:33:44.754 INFO:teuthology.orchestra.run.smithi039.stderr:done 2024-09-13T16:33:44.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo shutdown -r now 2024-09-13T16:34:13.308 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T16:34:13.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-13T16:34:13.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:34:14.766 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T16:34:14.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-13T16:34:14.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:34:31.869 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-09-13T16:34:33.149 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-13T16:34:40.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-13T16:34:40.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:34:42.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-13T16:34:42.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:34:43.965 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-09-13T16:34:45.249 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-13T16:34:55.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-13T16:34:55.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:34:57.251 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-13T16:34:57.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:35:00.317 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-13T16:35:14.365 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-09-13T16:35:15.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-13T16:35:15.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:35:29.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-13T16:35:29.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:35:29.673 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-09-13T16:35:30.239 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-13T16:35:30.239 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-13T16:35:30.240 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-09-13T16:35:30.286 INFO:teuthology.orchestra.run.smithi084.stdout:5.15.0-121-generic 2024-09-13T16:35:30.286 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T16:35:30.286 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T16:35:30.286 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T16:35:31.288 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T16:35:31.288 DEBUG:teuthology.orchestra.run.smithi084:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T16:35:31.348 INFO:teuthology.orchestra.run.smithi084.stdout:ttyS1 2024-09-13T16:35:31.372 DEBUG:teuthology.parallel:result is None 2024-09-13T16:36:15.330 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T16:36:33.331 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-13T16:36:33.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:36:33.613 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-13T16:36:34.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-13T16:36:34.247 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-13T16:36:34.247 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-13T16:36:34.293 INFO:teuthology.orchestra.run.smithi039.stdout:5.15.0-121-generic 2024-09-13T16:36:34.293 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T16:36:34.293 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T16:36:34.293 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T16:36:35.294 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T16:36:35.295 DEBUG:teuthology.orchestra.run.smithi039:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T16:36:35.373 INFO:teuthology.orchestra.run.smithi039.stdout:ttyS1 2024-09-13T16:36:35.407 DEBUG:teuthology.parallel:result is None 2024-09-13T16:36:35.407 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T16:36:35.414 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T16:36:35.414 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T16:36:35.416 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T16:36:35.422 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T16:36:35.428 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T16:36:35.436 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T16:36:35.436 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T16:36:35.466 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T16:36:35.517 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T16:36:35.523 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T16:36:35.524 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T16:36:35.542 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T16:36:35.571 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:36:35.577 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:36:35.579 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:36:35.585 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T16:36:35.587 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T16:36:35.594 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T16:36:35.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T16:36:35.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T16:36:35.647 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T16:36:35.658 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T16:36:35.658 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T16:36:35.682 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T16:36:35.694 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:36:35.733 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:36:35.782 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:36:35.782 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T16:36:35.848 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T16:36:35.853 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T16:36:35.902 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:36:35.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T16:36:35.961 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-09-13T16:36:35.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2024-09-13T16:36:36.036 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T16:36:36.044 INFO:teuthology.task.internal:Starting timer... 2024-09-13T16:36:36.044 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T16:36:36.054 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T16:36:36.098 DEBUG:teuthology.task.selinux:Excluding smithi039: OS 'ubuntu' does not support SELinux 2024-09-13T16:36:36.099 DEBUG:teuthology.task.selinux:Excluding smithi084: OS 'ubuntu' does not support SELinux 2024-09-13T16:36:36.099 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T16:36:36.099 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T16:36:36.099 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T16:36:36.099 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T16:36:36.107 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-13T16:36:36.110 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-13T16:36:36.292 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T16:36:36.303 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T16:36:36.304 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi084.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T16:44:16.753 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi084.front.sepia.ceph.com')] 2024-09-13T16:44:16.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-13T16:44:16.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:44:16.875 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-13T16:44:16.953 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-13T16:44:16.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-13T16:44:16.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T16:44:17.049 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-09-13T16:44:17.125 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-13T16:44:17.125 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T16:44:17.136 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T16:44:17.136 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T16:44:17.136 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:44:17.139 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T16:44:17.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T16:44:17.165 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T16:44:17.165 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: Command line: ntpd -gq 2024-09-13T16:44:17.165 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: ---------------------------------------------------- 2024-09-13T16:44:17.166 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T16:44:17.166 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T16:44:17.166 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: corporation. Support and training for ntp-4 are 2024-09-13T16:44:17.166 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: available at https://www.nwtime.org/support 2024-09-13T16:44:17.166 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: ---------------------------------------------------- 2024-09-13T16:44:17.166 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: proto: precision = 0.064 usec (-24) 2024-09-13T16:44:17.166 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: basedate set to 2022-02-04 2024-09-13T16:44:17.167 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: gps base set to 2022-02-06 (week 2196) 2024-09-13T16:44:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T16:44:17.167 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T16:44:17.167 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T16:44:17.168 INFO:teuthology.orchestra.run.smithi039.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T16:44:17.168 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T16:44:17.169 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T16:44:17.169 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T16:44:17.169 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: Listen normally on 3 ens1f0 172.21.15.39:123 2024-09-13T16:44:17.169 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: Listen normally on 4 lo [::1]:123 2024-09-13T16:44:17.169 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1370%4]:123 2024-09-13T16:44:17.169 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:17 ntpd[18439]: Listening on routing socket on fd #22 for interface updates 2024-09-13T16:44:17.195 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T16:44:17.195 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: Command line: ntpd -gq 2024-09-13T16:44:17.195 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: ---------------------------------------------------- 2024-09-13T16:44:17.195 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T16:44:17.195 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T16:44:17.195 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: corporation. Support and training for ntp-4 are 2024-09-13T16:44:17.195 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: available at https://www.nwtime.org/support 2024-09-13T16:44:17.195 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: ---------------------------------------------------- 2024-09-13T16:44:17.196 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: proto: precision = 0.044 usec (-24) 2024-09-13T16:44:17.196 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: basedate set to 2022-02-04 2024-09-13T16:44:17.196 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: gps base set to 2022-02-06 (week 2196) 2024-09-13T16:44:17.197 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T16:44:17.197 INFO:teuthology.orchestra.run.smithi084.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T16:44:17.197 INFO:teuthology.orchestra.run.smithi084.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T16:44:17.198 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T16:44:17.198 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T16:44:17.199 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T16:44:17.199 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T16:44:17.199 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: Listen normally on 3 enp3s0f1 172.21.15.84:123 2024-09-13T16:44:17.199 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: Listen normally on 4 lo [::1]:123 2024-09-13T16:44:17.199 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d751%5]:123 2024-09-13T16:44:17.199 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:17 ntpd[18489]: Listening on routing socket on fd #22 for interface updates 2024-09-13T16:44:18.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:18 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:18.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:18 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:18.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:18 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:18.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:18 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:18.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:18 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:18.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:18 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:18.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:18 ntpd[18439]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:18.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:18 ntpd[18439]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:18.195 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:18 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:18.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:18 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:18.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:18 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:18.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:18 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:18.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:18 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:18.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:18 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:19.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:19 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:19.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:19 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:19.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:19 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:19.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:19 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:19.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:19 ntpd[18439]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:19.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:19 ntpd[18439]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:20 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:20 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:20 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:20 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:20 ntpd[18439]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:20 ntpd[18439]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:20.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:20.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:20.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:20.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:20.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:20.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:20.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:20.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:20.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:20.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:20.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:20.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:20 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:21.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:21 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:21.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:21 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:21.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:21 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:21.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:21 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:21.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:21 ntpd[18439]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:21.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:21 ntpd[18439]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:21.195 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:21 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:21.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:21 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:21.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:21 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:21.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:21 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:21.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:21 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:21.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:21 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:22.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:22.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:22.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:22.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:22.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:22.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:22.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:22.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:22.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:22.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:22.168 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:22.168 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:22 ntpd[18439]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:22.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:22.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:22.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:22.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:22.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:22.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:22.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:22.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:22.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:22.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:22.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:22.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:22.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:22.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:22 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:23.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:23 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:23.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:23 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:23.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:23 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:23.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:23 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:23.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:23 ntpd[18439]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:23.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:23 ntpd[18439]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:23.195 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:23 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:23.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:23 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:23.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:23 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:23.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:23 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:23.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:23 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:23.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:23 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:24.166 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:24.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:24.167 INFO:teuthology.orchestra.run.smithi039.stdout:13 Sep 16:44:24 ntpd[18439]: ntpd: time slew -0.000462 s 2024-09-13T16:44:24.167 INFO:teuthology.orchestra.run.smithi039.stdout:ntpd: time slew -0.000462s 2024-09-13T16:44:24.167 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T16:44:24.168 INFO:teuthology.orchestra.run.smithi039.stderr:13 Sep 16:44:24 ntpd[18439]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T16:44:24.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:24.196 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T16:44:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T16:44:24.198 INFO:teuthology.orchestra.run.smithi084.stdout:13 Sep 16:44:24 ntpd[18489]: ntpd: time slew -0.001478 s 2024-09-13T16:44:24.198 INFO:teuthology.orchestra.run.smithi084.stdout:ntpd: time slew -0.001478s 2024-09-13T16:44:24.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T16:44:24.198 INFO:teuthology.orchestra.run.smithi084.stderr:13 Sep 16:44:24 ntpd[18489]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T16:44:24.218 INFO:teuthology.orchestra.run.smithi039.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T16:44:24.218 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================== 2024-09-13T16:44:24.218 INFO:teuthology.orchestra.run.smithi039.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T16:44:24.218 INFO:teuthology.orchestra.run.smithi039.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T16:44:24.218 INFO:teuthology.orchestra.run.smithi039.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T16:44:24.218 INFO:teuthology.orchestra.run.smithi039.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T16:44:24.251 INFO:teuthology.orchestra.run.smithi084.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T16:44:24.251 INFO:teuthology.orchestra.run.smithi084.stdout:============================================================================== 2024-09-13T16:44:24.251 INFO:teuthology.orchestra.run.smithi084.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T16:44:24.251 INFO:teuthology.orchestra.run.smithi084.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T16:44:24.251 INFO:teuthology.orchestra.run.smithi084.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T16:44:24.251 INFO:teuthology.orchestra.run.smithi084.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T16:44:24.252 INFO:teuthology.run_tasks:Running task install... 2024-09-13T16:44:24.260 DEBUG:teuthology.task.install:project ceph 2024-09-13T16:44:24.261 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}} 2024-09-13T16:44:24.261 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'} 2024-09-13T16:44:24.261 INFO:teuthology.task.install:Using flavor: default 2024-09-13T16:44:24.267 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T16:44:24.267 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T16:44:24.268 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-key list | grep Ceph 2024-09-13T16:44:24.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo apt-key list | grep Ceph 2024-09-13T16:44:24.327 INFO:teuthology.orchestra.run.smithi039.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T16:44:24.350 INFO:teuthology.orchestra.run.smithi084.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T16:44:24.354 INFO:teuthology.orchestra.run.smithi039.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T16:44:24.354 INFO:teuthology.orchestra.run.smithi039.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T16:44:24.355 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T16:44:24.355 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T16:44:24.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:44:24.378 INFO:teuthology.orchestra.run.smithi084.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T16:44:24.379 INFO:teuthology.orchestra.run.smithi084.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T16:44:24.379 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T16:44:24.379 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T16:44:24.379 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:44:24.532 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default/ 2024-09-13T16:44:24.533 INFO:teuthology.task.install.deb:Package version is 18.2.4-850-g50bbe7f3-1jammy 2024-09-13T16:44:24.558 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default/ 2024-09-13T16:44:24.558 INFO:teuthology.task.install.deb:Package version is 18.2.4-850-g50bbe7f3-1jammy 2024-09-13T16:44:24.685 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:44:24.686 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T16:44:24.701 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get update 2024-09-13T16:44:24.707 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:44:24.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T16:44:24.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo apt-get update 2024-09-13T16:44:24.886 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T16:44:24.949 INFO:teuthology.orchestra.run.smithi039.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T16:44:24.973 INFO:teuthology.orchestra.run.smithi084.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T16:44:24.974 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-13T16:44:24.999 INFO:teuthology.orchestra.run.smithi084.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-13T16:44:25.002 INFO:teuthology.orchestra.run.smithi039.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T16:44:25.002 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T16:44:25.029 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-13T16:44:25.036 INFO:teuthology.orchestra.run.smithi084.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T16:44:25.037 INFO:teuthology.orchestra.run.smithi084.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T16:44:25.037 INFO:teuthology.orchestra.run.smithi084.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T16:44:25.061 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,423 B] 2024-09-13T16:44:25.062 INFO:teuthology.orchestra.run.smithi084.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,423 B] 2024-09-13T16:44:25.088 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T16:44:25.096 INFO:teuthology.orchestra.run.smithi084.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-13T16:44:25.112 INFO:teuthology.orchestra.run.smithi084.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T16:44:25.174 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T16:44:25.200 INFO:teuthology.orchestra.run.smithi084.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T16:44:25.325 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 28.8 kB in 0s (65.7 kB/s) 2024-09-13T16:44:25.401 INFO:teuthology.orchestra.run.smithi084.stdout:Fetched 28.8 kB in 0s (58.0 kB/s) 2024-09-13T16:44:26.384 INFO:teuthology.orchestra.run.smithi084.stdout:Reading package lists... 2024-09-13T16:44:26.385 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-13T16:44:26.401 DEBUG:teuthology.orchestra.run.smithi084:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-850-g50bbe7f3-1jammy cephadm=18.2.4-850-g50bbe7f3-1jammy ceph-mds=18.2.4-850-g50bbe7f3-1jammy ceph-mgr=18.2.4-850-g50bbe7f3-1jammy ceph-common=18.2.4-850-g50bbe7f3-1jammy ceph-fuse=18.2.4-850-g50bbe7f3-1jammy ceph-test=18.2.4-850-g50bbe7f3-1jammy ceph-volume=18.2.4-850-g50bbe7f3-1jammy radosgw=18.2.4-850-g50bbe7f3-1jammy python3-rados=18.2.4-850-g50bbe7f3-1jammy python3-rgw=18.2.4-850-g50bbe7f3-1jammy python3-cephfs=18.2.4-850-g50bbe7f3-1jammy python3-rbd=18.2.4-850-g50bbe7f3-1jammy libcephfs2=18.2.4-850-g50bbe7f3-1jammy libcephfs-dev=18.2.4-850-g50bbe7f3-1jammy librados2=18.2.4-850-g50bbe7f3-1jammy librbd1=18.2.4-850-g50bbe7f3-1jammy rbd-fuse=18.2.4-850-g50bbe7f3-1jammy python3-cephfs=18.2.4-850-g50bbe7f3-1jammy cephfs-shell=18.2.4-850-g50bbe7f3-1jammy cephfs-top=18.2.4-850-g50bbe7f3-1jammy cephfs-mirror=18.2.4-850-g50bbe7f3-1jammy 2024-09-13T16:44:26.402 DEBUG:teuthology.orchestra.run.smithi039:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-850-g50bbe7f3-1jammy cephadm=18.2.4-850-g50bbe7f3-1jammy ceph-mds=18.2.4-850-g50bbe7f3-1jammy ceph-mgr=18.2.4-850-g50bbe7f3-1jammy ceph-common=18.2.4-850-g50bbe7f3-1jammy ceph-fuse=18.2.4-850-g50bbe7f3-1jammy ceph-test=18.2.4-850-g50bbe7f3-1jammy ceph-volume=18.2.4-850-g50bbe7f3-1jammy radosgw=18.2.4-850-g50bbe7f3-1jammy python3-rados=18.2.4-850-g50bbe7f3-1jammy python3-rgw=18.2.4-850-g50bbe7f3-1jammy python3-cephfs=18.2.4-850-g50bbe7f3-1jammy python3-rbd=18.2.4-850-g50bbe7f3-1jammy libcephfs2=18.2.4-850-g50bbe7f3-1jammy libcephfs-dev=18.2.4-850-g50bbe7f3-1jammy librados2=18.2.4-850-g50bbe7f3-1jammy librbd1=18.2.4-850-g50bbe7f3-1jammy rbd-fuse=18.2.4-850-g50bbe7f3-1jammy python3-cephfs=18.2.4-850-g50bbe7f3-1jammy cephfs-shell=18.2.4-850-g50bbe7f3-1jammy cephfs-top=18.2.4-850-g50bbe7f3-1jammy cephfs-mirror=18.2.4-850-g50bbe7f3-1jammy 2024-09-13T16:44:26.466 INFO:teuthology.orchestra.run.smithi084.stdout:Reading package lists... 2024-09-13T16:44:26.467 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-13T16:44:26.673 INFO:teuthology.orchestra.run.smithi084.stdout:Building dependency tree... 2024-09-13T16:44:26.673 INFO:teuthology.orchestra.run.smithi084.stdout:Reading state information... 2024-09-13T16:44:26.681 INFO:teuthology.orchestra.run.smithi039.stdout:Building dependency tree... 2024-09-13T16:44:26.682 INFO:teuthology.orchestra.run.smithi039.stdout:Reading state information... 2024-09-13T16:44:26.878 INFO:teuthology.orchestra.run.smithi084.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:44:26.879 INFO:teuthology.orchestra.run.smithi084.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T16:44:26.879 INFO:teuthology.orchestra.run.smithi084.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T16:44:26.879 INFO:teuthology.orchestra.run.smithi084.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T16:44:26.879 INFO:teuthology.orchestra.run.smithi084.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T16:44:26.879 INFO:teuthology.orchestra.run.smithi084.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T16:44:26.879 INFO:teuthology.orchestra.run.smithi084.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:44:26.880 INFO:teuthology.orchestra.run.smithi084.stdout:The following additional packages will be installed: 2024-09-13T16:44:26.880 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T16:44:26.880 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T16:44:26.880 INFO:teuthology.orchestra.run.smithi084.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T16:44:26.880 INFO:teuthology.orchestra.run.smithi084.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T16:44:26.881 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-13T16:44:26.881 INFO:teuthology.orchestra.run.smithi084.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-13T16:44:26.882 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-13T16:44:26.883 INFO:teuthology.orchestra.run.smithi084.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T16:44:26.883 INFO:teuthology.orchestra.run.smithi084.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-13T16:44:26.883 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-13T16:44:26.883 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-13T16:44:26.883 INFO:teuthology.orchestra.run.smithi084.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T16:44:26.883 INFO:teuthology.orchestra.run.smithi084.stdout:Suggested packages: 2024-09-13T16:44:26.883 INFO:teuthology.orchestra.run.smithi084.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-13T16:44:26.884 INFO:teuthology.orchestra.run.smithi084.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-13T16:44:26.884 INFO:teuthology.orchestra.run.smithi084.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-13T16:44:26.884 INFO:teuthology.orchestra.run.smithi084.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-13T16:44:26.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-13T16:44:26.884 INFO:teuthology.orchestra.run.smithi084.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-13T16:44:26.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-watchdog 2024-09-13T16:44:26.884 INFO:teuthology.orchestra.run.smithi084.stdout:Recommended packages: 2024-09-13T16:44:26.884 INFO:teuthology.orchestra.run.smithi084.stdout: btrfs-tools 2024-09-13T16:44:26.897 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:44:26.898 INFO:teuthology.orchestra.run.smithi039.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T16:44:26.898 INFO:teuthology.orchestra.run.smithi039.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T16:44:26.898 INFO:teuthology.orchestra.run.smithi039.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T16:44:26.898 INFO:teuthology.orchestra.run.smithi039.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T16:44:26.898 INFO:teuthology.orchestra.run.smithi039.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T16:44:26.898 INFO:teuthology.orchestra.run.smithi039.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:44:26.899 INFO:teuthology.orchestra.run.smithi039.stdout:The following additional packages will be installed: 2024-09-13T16:44:26.899 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T16:44:26.900 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T16:44:26.900 INFO:teuthology.orchestra.run.smithi039.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T16:44:26.900 INFO:teuthology.orchestra.run.smithi039.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T16:44:26.900 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-13T16:44:26.901 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout:Suggested packages: 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-13T16:44:26.902 INFO:teuthology.orchestra.run.smithi039.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-13T16:44:26.903 INFO:teuthology.orchestra.run.smithi039.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-13T16:44:26.903 INFO:teuthology.orchestra.run.smithi039.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-13T16:44:26.903 INFO:teuthology.orchestra.run.smithi039.stdout: python3-watchdog 2024-09-13T16:44:26.903 INFO:teuthology.orchestra.run.smithi039.stdout:Recommended packages: 2024-09-13T16:44:26.903 INFO:teuthology.orchestra.run.smithi039.stdout: btrfs-tools 2024-09-13T16:44:26.990 INFO:teuthology.orchestra.run.smithi084.stdout:The following NEW packages will be installed: 2024-09-13T16:44:26.990 INFO:teuthology.orchestra.run.smithi084.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T16:44:26.990 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T16:44:26.991 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T16:44:26.991 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-13T16:44:26.991 INFO:teuthology.orchestra.run.smithi084.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T16:44:26.991 INFO:teuthology.orchestra.run.smithi084.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T16:44:26.992 INFO:teuthology.orchestra.run.smithi084.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T16:44:26.993 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T16:44:26.993 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T16:44:26.993 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T16:44:26.993 INFO:teuthology.orchestra.run.smithi084.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T16:44:26.993 INFO:teuthology.orchestra.run.smithi084.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T16:44:26.993 INFO:teuthology.orchestra.run.smithi084.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T16:44:26.993 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T16:44:26.993 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T16:44:26.993 INFO:teuthology.orchestra.run.smithi084.stdout: socat xmlstarlet 2024-09-13T16:44:27.016 INFO:teuthology.orchestra.run.smithi039.stdout:The following NEW packages will be installed: 2024-09-13T16:44:27.016 INFO:teuthology.orchestra.run.smithi039.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T16:44:27.016 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T16:44:27.016 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T16:44:27.016 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-13T16:44:27.017 INFO:teuthology.orchestra.run.smithi039.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T16:44:27.017 INFO:teuthology.orchestra.run.smithi039.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-13T16:44:27.017 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-13T16:44:27.017 INFO:teuthology.orchestra.run.smithi039.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-13T16:44:27.017 INFO:teuthology.orchestra.run.smithi039.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T16:44:27.018 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T16:44:27.019 INFO:teuthology.orchestra.run.smithi039.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T16:44:27.019 INFO:teuthology.orchestra.run.smithi039.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T16:44:27.019 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T16:44:27.019 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T16:44:27.019 INFO:teuthology.orchestra.run.smithi039.stdout: socat xmlstarlet 2024-09-13T16:44:27.057 INFO:teuthology.orchestra.run.smithi084.stdout:0 upgraded, 115 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T16:44:27.057 INFO:teuthology.orchestra.run.smithi084.stdout:Need to get 163 MB of archives. 2024-09-13T16:44:27.058 INFO:teuthology.orchestra.run.smithi084.stdout:After this operation, 747 MB of additional disk space will be used. 2024-09-13T16:44:27.058 INFO:teuthology.orchestra.run.smithi084.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-13T16:44:27.078 INFO:teuthology.orchestra.run.smithi084.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T16:44:27.083 INFO:teuthology.orchestra.run.smithi039.stdout:0 upgraded, 115 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T16:44:27.083 INFO:teuthology.orchestra.run.smithi039.stdout:Need to get 163 MB of archives. 2024-09-13T16:44:27.083 INFO:teuthology.orchestra.run.smithi039.stdout:After this operation, 747 MB of additional disk space will be used. 2024-09-13T16:44:27.083 INFO:teuthology.orchestra.run.smithi039.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-13T16:44:27.089 INFO:teuthology.orchestra.run.smithi084.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-13T16:44:27.104 INFO:teuthology.orchestra.run.smithi039.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T16:44:27.115 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-13T16:44:27.138 INFO:teuthology.orchestra.run.smithi084.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-13T16:44:27.148 INFO:teuthology.orchestra.run.smithi084.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-850-g50bbe7f3-1jammy [3,451 kB] 2024-09-13T16:44:27.148 INFO:teuthology.orchestra.run.smithi084.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T16:44:27.163 INFO:teuthology.orchestra.run.smithi039.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-13T16:44:27.167 INFO:teuthology.orchestra.run.smithi084.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T16:44:27.171 INFO:teuthology.orchestra.run.smithi084.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T16:44:27.175 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T16:44:27.177 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-850-g50bbe7f3-1jammy [3,451 kB] 2024-09-13T16:44:27.182 INFO:teuthology.orchestra.run.smithi084.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T16:44:27.193 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T16:44:27.196 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T16:44:27.210 INFO:teuthology.orchestra.run.smithi039.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T16:44:27.236 INFO:teuthology.orchestra.run.smithi084.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T16:44:27.241 INFO:teuthology.orchestra.run.smithi084.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T16:44:27.253 INFO:teuthology.orchestra.run.smithi084.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T16:44:27.257 INFO:teuthology.orchestra.run.smithi084.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T16:44:27.257 INFO:teuthology.orchestra.run.smithi084.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T16:44:27.261 INFO:teuthology.orchestra.run.smithi039.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T16:44:27.266 INFO:teuthology.orchestra.run.smithi084.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T16:44:27.266 INFO:teuthology.orchestra.run.smithi039.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T16:44:27.266 INFO:teuthology.orchestra.run.smithi084.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T16:44:27.267 INFO:teuthology.orchestra.run.smithi084.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T16:44:27.268 INFO:teuthology.orchestra.run.smithi084.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T16:44:27.268 INFO:teuthology.orchestra.run.smithi084.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T16:44:27.278 INFO:teuthology.orchestra.run.smithi039.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T16:44:27.282 INFO:teuthology.orchestra.run.smithi039.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T16:44:27.283 INFO:teuthology.orchestra.run.smithi039.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T16:44:27.283 INFO:teuthology.orchestra.run.smithi084.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T16:44:27.284 INFO:teuthology.orchestra.run.smithi084.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T16:44:27.284 INFO:teuthology.orchestra.run.smithi084.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T16:44:27.284 INFO:teuthology.orchestra.run.smithi084.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T16:44:27.292 INFO:teuthology.orchestra.run.smithi039.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T16:44:27.292 INFO:teuthology.orchestra.run.smithi039.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T16:44:27.292 INFO:teuthology.orchestra.run.smithi039.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T16:44:27.294 INFO:teuthology.orchestra.run.smithi039.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T16:44:27.294 INFO:teuthology.orchestra.run.smithi039.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T16:44:27.305 INFO:teuthology.orchestra.run.smithi084.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T16:44:27.308 INFO:teuthology.orchestra.run.smithi039.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T16:44:27.308 INFO:teuthology.orchestra.run.smithi039.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T16:44:27.308 INFO:teuthology.orchestra.run.smithi084.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T16:44:27.309 INFO:teuthology.orchestra.run.smithi039.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T16:44:27.309 INFO:teuthology.orchestra.run.smithi084.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T16:44:27.310 INFO:teuthology.orchestra.run.smithi084.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T16:44:27.311 INFO:teuthology.orchestra.run.smithi084.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T16:44:27.312 INFO:teuthology.orchestra.run.smithi084.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T16:44:27.326 INFO:teuthology.orchestra.run.smithi084.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T16:44:27.326 INFO:teuthology.orchestra.run.smithi084.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T16:44:27.328 INFO:teuthology.orchestra.run.smithi084.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T16:44:27.329 INFO:teuthology.orchestra.run.smithi039.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T16:44:27.329 INFO:teuthology.orchestra.run.smithi084.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T16:44:27.329 INFO:teuthology.orchestra.run.smithi039.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T16:44:27.332 INFO:teuthology.orchestra.run.smithi039.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T16:44:27.332 INFO:teuthology.orchestra.run.smithi039.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T16:44:27.333 INFO:teuthology.orchestra.run.smithi039.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T16:44:27.333 INFO:teuthology.orchestra.run.smithi039.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T16:44:27.334 INFO:teuthology.orchestra.run.smithi039.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T16:44:27.346 INFO:teuthology.orchestra.run.smithi084.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T16:44:27.351 INFO:teuthology.orchestra.run.smithi039.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T16:44:27.351 INFO:teuthology.orchestra.run.smithi039.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T16:44:27.352 INFO:teuthology.orchestra.run.smithi039.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T16:44:27.354 INFO:teuthology.orchestra.run.smithi084.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T16:44:27.354 INFO:teuthology.orchestra.run.smithi084.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T16:44:27.355 INFO:teuthology.orchestra.run.smithi084.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T16:44:27.356 INFO:teuthology.orchestra.run.smithi084.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T16:44:27.358 INFO:teuthology.orchestra.run.smithi084.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T16:44:27.367 INFO:teuthology.orchestra.run.smithi084.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T16:44:27.368 INFO:teuthology.orchestra.run.smithi084.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T16:44:27.370 INFO:teuthology.orchestra.run.smithi084.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T16:44:27.371 INFO:teuthology.orchestra.run.smithi084.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T16:44:27.372 INFO:teuthology.orchestra.run.smithi039.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T16:44:27.372 INFO:teuthology.orchestra.run.smithi039.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T16:44:27.378 INFO:teuthology.orchestra.run.smithi039.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T16:44:27.378 INFO:teuthology.orchestra.run.smithi039.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T16:44:27.378 INFO:teuthology.orchestra.run.smithi039.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T16:44:27.379 INFO:teuthology.orchestra.run.smithi039.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T16:44:27.381 INFO:teuthology.orchestra.run.smithi039.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T16:44:27.388 INFO:teuthology.orchestra.run.smithi084.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T16:44:27.389 INFO:teuthology.orchestra.run.smithi084.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T16:44:27.390 INFO:teuthology.orchestra.run.smithi084.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T16:44:27.393 INFO:teuthology.orchestra.run.smithi039.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T16:44:27.393 INFO:teuthology.orchestra.run.smithi084.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T16:44:27.394 INFO:teuthology.orchestra.run.smithi039.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T16:44:27.396 INFO:teuthology.orchestra.run.smithi039.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T16:44:27.416 INFO:teuthology.orchestra.run.smithi039.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T16:44:27.419 INFO:teuthology.orchestra.run.smithi039.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T16:44:27.419 INFO:teuthology.orchestra.run.smithi039.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T16:44:27.420 INFO:teuthology.orchestra.run.smithi039.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T16:44:27.423 INFO:teuthology.orchestra.run.smithi039.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T16:44:27.452 INFO:teuthology.orchestra.run.smithi084.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-850-g50bbe7f3-1jammy [3,108 kB] 2024-09-13T16:44:27.459 INFO:teuthology.orchestra.run.smithi084.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T16:44:27.460 INFO:teuthology.orchestra.run.smithi084.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T16:44:27.461 INFO:teuthology.orchestra.run.smithi084.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T16:44:27.462 INFO:teuthology.orchestra.run.smithi084.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T16:44:27.462 INFO:teuthology.orchestra.run.smithi084.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T16:44:27.463 INFO:teuthology.orchestra.run.smithi084.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T16:44:27.463 INFO:teuthology.orchestra.run.smithi084.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T16:44:27.465 INFO:teuthology.orchestra.run.smithi084.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T16:44:27.465 INFO:teuthology.orchestra.run.smithi084.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T16:44:27.479 INFO:teuthology.orchestra.run.smithi039.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T16:44:27.480 INFO:teuthology.orchestra.run.smithi039.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T16:44:27.481 INFO:teuthology.orchestra.run.smithi039.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T16:44:27.482 INFO:teuthology.orchestra.run.smithi039.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T16:44:27.482 INFO:teuthology.orchestra.run.smithi084.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T16:44:27.483 INFO:teuthology.orchestra.run.smithi039.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T16:44:27.483 INFO:teuthology.orchestra.run.smithi039.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T16:44:27.483 INFO:teuthology.orchestra.run.smithi039.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T16:44:27.486 INFO:teuthology.orchestra.run.smithi039.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T16:44:27.486 INFO:teuthology.orchestra.run.smithi039.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T16:44:27.499 INFO:teuthology.orchestra.run.smithi084.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T16:44:27.501 INFO:teuthology.orchestra.run.smithi084.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T16:44:27.502 INFO:teuthology.orchestra.run.smithi084.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T16:44:27.507 INFO:teuthology.orchestra.run.smithi039.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T16:44:27.508 INFO:teuthology.orchestra.run.smithi039.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-850-g50bbe7f3-1jammy [3,108 kB] 2024-09-13T16:44:27.517 INFO:teuthology.orchestra.run.smithi084.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T16:44:27.517 INFO:teuthology.orchestra.run.smithi084.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T16:44:27.517 INFO:teuthology.orchestra.run.smithi084.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T16:44:27.518 INFO:teuthology.orchestra.run.smithi084.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T16:44:27.520 INFO:teuthology.orchestra.run.smithi084.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T16:44:27.520 INFO:teuthology.orchestra.run.smithi084.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T16:44:27.527 INFO:teuthology.orchestra.run.smithi039.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T16:44:27.529 INFO:teuthology.orchestra.run.smithi084.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T16:44:27.529 INFO:teuthology.orchestra.run.smithi039.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T16:44:27.529 INFO:teuthology.orchestra.run.smithi039.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T16:44:27.530 INFO:teuthology.orchestra.run.smithi084.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T16:44:27.533 INFO:teuthology.orchestra.run.smithi084.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T16:44:27.551 INFO:teuthology.orchestra.run.smithi039.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T16:44:27.551 INFO:teuthology.orchestra.run.smithi039.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T16:44:27.552 INFO:teuthology.orchestra.run.smithi084.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T16:44:27.552 INFO:teuthology.orchestra.run.smithi039.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T16:44:27.552 INFO:teuthology.orchestra.run.smithi039.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T16:44:27.553 INFO:teuthology.orchestra.run.smithi039.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T16:44:27.553 INFO:teuthology.orchestra.run.smithi084.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T16:44:27.554 INFO:teuthology.orchestra.run.smithi039.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T16:44:27.554 INFO:teuthology.orchestra.run.smithi039.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T16:44:27.554 INFO:teuthology.orchestra.run.smithi084.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T16:44:27.557 INFO:teuthology.orchestra.run.smithi084.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T16:44:27.559 INFO:teuthology.orchestra.run.smithi084.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-13T16:44:27.560 INFO:teuthology.orchestra.run.smithi084.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T16:44:27.563 INFO:teuthology.orchestra.run.smithi084.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T16:44:27.576 INFO:teuthology.orchestra.run.smithi039.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T16:44:27.577 INFO:teuthology.orchestra.run.smithi084.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T16:44:27.578 INFO:teuthology.orchestra.run.smithi084.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T16:44:27.579 INFO:teuthology.orchestra.run.smithi084.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T16:44:27.579 INFO:teuthology.orchestra.run.smithi039.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T16:44:27.600 INFO:teuthology.orchestra.run.smithi039.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T16:44:27.602 INFO:teuthology.orchestra.run.smithi039.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T16:44:27.602 INFO:teuthology.orchestra.run.smithi084.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T16:44:27.603 INFO:teuthology.orchestra.run.smithi039.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T16:44:27.603 INFO:teuthology.orchestra.run.smithi084.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T16:44:27.603 INFO:teuthology.orchestra.run.smithi084.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T16:44:27.606 INFO:teuthology.orchestra.run.smithi039.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T16:44:27.610 INFO:teuthology.orchestra.run.smithi039.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-13T16:44:27.610 INFO:teuthology.orchestra.run.smithi039.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T16:44:27.615 INFO:teuthology.orchestra.run.smithi039.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T16:44:27.618 INFO:teuthology.orchestra.run.smithi039.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T16:44:27.622 INFO:teuthology.orchestra.run.smithi084.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-850-g50bbe7f3-1jammy [684 kB] 2024-09-13T16:44:27.623 INFO:teuthology.orchestra.run.smithi039.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T16:44:27.625 INFO:teuthology.orchestra.run.smithi039.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T16:44:27.644 INFO:teuthology.orchestra.run.smithi039.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T16:44:27.645 INFO:teuthology.orchestra.run.smithi039.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T16:44:27.645 INFO:teuthology.orchestra.run.smithi039.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T16:44:27.664 INFO:teuthology.orchestra.run.smithi084.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-850-g50bbe7f3-1jammy [356 kB] 2024-09-13T16:44:27.671 INFO:teuthology.orchestra.run.smithi084.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-850-g50bbe7f3-1jammy [32.8 kB] 2024-09-13T16:44:27.672 INFO:teuthology.orchestra.run.smithi084.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-850-g50bbe7f3-1jammy [180 kB] 2024-09-13T16:44:27.676 INFO:teuthology.orchestra.run.smithi084.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-850-g50bbe7f3-1jammy [63.8 kB] 2024-09-13T16:44:27.678 INFO:teuthology.orchestra.run.smithi084.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-850-g50bbe7f3-1jammy [334 kB] 2024-09-13T16:44:27.689 INFO:teuthology.orchestra.run.smithi039.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-850-g50bbe7f3-1jammy [684 kB] 2024-09-13T16:44:27.712 INFO:teuthology.orchestra.run.smithi084.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-850-g50bbe7f3-1jammy [5,933 kB] 2024-09-13T16:44:27.733 INFO:teuthology.orchestra.run.smithi039.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-850-g50bbe7f3-1jammy [356 kB] 2024-09-13T16:44:27.741 INFO:teuthology.orchestra.run.smithi039.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-850-g50bbe7f3-1jammy [32.8 kB] 2024-09-13T16:44:27.742 INFO:teuthology.orchestra.run.smithi039.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-850-g50bbe7f3-1jammy [180 kB] 2024-09-13T16:44:27.747 INFO:teuthology.orchestra.run.smithi039.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-850-g50bbe7f3-1jammy [63.8 kB] 2024-09-13T16:44:27.748 INFO:teuthology.orchestra.run.smithi039.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-850-g50bbe7f3-1jammy [334 kB] 2024-09-13T16:44:27.783 INFO:teuthology.orchestra.run.smithi039.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-850-g50bbe7f3-1jammy [5,933 kB] 2024-09-13T16:44:28.022 INFO:teuthology.orchestra.run.smithi084.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-850-g50bbe7f3-1jammy [112 kB] 2024-09-13T16:44:28.024 INFO:teuthology.orchestra.run.smithi084.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-850-g50bbe7f3-1jammy [445 kB] 2024-09-13T16:44:28.033 INFO:teuthology.orchestra.run.smithi084.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-850-g50bbe7f3-1jammy [22.1 MB] 2024-09-13T16:44:28.109 INFO:teuthology.orchestra.run.smithi039.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-850-g50bbe7f3-1jammy [112 kB] 2024-09-13T16:44:28.112 INFO:teuthology.orchestra.run.smithi039.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-850-g50bbe7f3-1jammy [445 kB] 2024-09-13T16:44:28.122 INFO:teuthology.orchestra.run.smithi039.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-850-g50bbe7f3-1jammy [22.1 MB] 2024-09-13T16:44:29.106 INFO:teuthology.orchestra.run.smithi084.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-850-g50bbe7f3-1jammy [5,005 kB] 2024-09-13T16:44:29.205 INFO:teuthology.orchestra.run.smithi084.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-850-g50bbe7f3-1jammy [241 kB] 2024-09-13T16:44:29.210 INFO:teuthology.orchestra.run.smithi084.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-850-g50bbe7f3-1jammy [130 kB] 2024-09-13T16:44:29.213 INFO:teuthology.orchestra.run.smithi084.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-850-g50bbe7f3-1jammy [1,495 kB] 2024-09-13T16:44:29.241 INFO:teuthology.orchestra.run.smithi039.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-850-g50bbe7f3-1jammy [5,005 kB] 2024-09-13T16:44:29.265 INFO:teuthology.orchestra.run.smithi084.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-850-g50bbe7f3-1jammy [6,085 kB] 2024-09-13T16:44:29.420 INFO:teuthology.orchestra.run.smithi039.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-850-g50bbe7f3-1jammy [241 kB] 2024-09-13T16:44:29.425 INFO:teuthology.orchestra.run.smithi039.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-850-g50bbe7f3-1jammy [130 kB] 2024-09-13T16:44:29.428 INFO:teuthology.orchestra.run.smithi039.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-850-g50bbe7f3-1jammy [1,495 kB] 2024-09-13T16:44:29.494 INFO:teuthology.orchestra.run.smithi084.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-850-g50bbe7f3-1jammy [22.4 MB] 2024-09-13T16:44:29.509 INFO:teuthology.orchestra.run.smithi039.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-850-g50bbe7f3-1jammy [6,085 kB] 2024-09-13T16:44:29.801 INFO:teuthology.orchestra.run.smithi039.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-850-g50bbe7f3-1jammy [22.4 MB] 2024-09-13T16:44:30.412 INFO:teuthology.orchestra.run.smithi084.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-850-g50bbe7f3-1jammy [14.2 kB] 2024-09-13T16:44:30.412 INFO:teuthology.orchestra.run.smithi084.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-850-g50bbe7f3-1jammy [835 kB] 2024-09-13T16:44:30.429 INFO:teuthology.orchestra.run.smithi084.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-850-g50bbe7f3-1jammy [2,159 kB] 2024-09-13T16:44:30.466 INFO:teuthology.orchestra.run.smithi084.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-850-g50bbe7f3-1jammy [240 kB] 2024-09-13T16:44:30.470 INFO:teuthology.orchestra.run.smithi084.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-850-g50bbe7f3-1jammy [145 kB] 2024-09-13T16:44:30.472 INFO:teuthology.orchestra.run.smithi084.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-850-g50bbe7f3-1jammy [2,233 kB] 2024-09-13T16:44:30.569 INFO:teuthology.orchestra.run.smithi084.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-850-g50bbe7f3-1jammy [8,625 kB] 2024-09-13T16:44:30.885 INFO:teuthology.orchestra.run.smithi084.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-850-g50bbe7f3-1jammy [14.3 kB] 2024-09-13T16:44:30.886 INFO:teuthology.orchestra.run.smithi084.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-850-g50bbe7f3-1jammy [42.8 MB] 2024-09-13T16:44:30.924 INFO:teuthology.orchestra.run.smithi039.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-850-g50bbe7f3-1jammy [14.2 kB] 2024-09-13T16:44:30.925 INFO:teuthology.orchestra.run.smithi039.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-850-g50bbe7f3-1jammy [835 kB] 2024-09-13T16:44:30.941 INFO:teuthology.orchestra.run.smithi039.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-850-g50bbe7f3-1jammy [2,159 kB] 2024-09-13T16:44:30.982 INFO:teuthology.orchestra.run.smithi039.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-850-g50bbe7f3-1jammy [240 kB] 2024-09-13T16:44:30.985 INFO:teuthology.orchestra.run.smithi039.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-850-g50bbe7f3-1jammy [145 kB] 2024-09-13T16:44:30.988 INFO:teuthology.orchestra.run.smithi039.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-850-g50bbe7f3-1jammy [2,233 kB] 2024-09-13T16:44:31.094 INFO:teuthology.orchestra.run.smithi039.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-850-g50bbe7f3-1jammy [8,625 kB] 2024-09-13T16:44:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-850-g50bbe7f3-1jammy [14.3 kB] 2024-09-13T16:44:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-850-g50bbe7f3-1jammy [42.8 MB] 2024-09-13T16:44:32.712 INFO:teuthology.orchestra.run.smithi084.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-850-g50bbe7f3-1jammy [122 kB] 2024-09-13T16:44:32.714 INFO:teuthology.orchestra.run.smithi084.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.4-850-g50bbe7f3-1jammy [233 kB] 2024-09-13T16:44:32.719 INFO:teuthology.orchestra.run.smithi084.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.4-850-g50bbe7f3-1jammy [34.0 kB] 2024-09-13T16:44:32.720 INFO:teuthology.orchestra.run.smithi084.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.4-850-g50bbe7f3-1jammy [23.5 kB] 2024-09-13T16:44:32.721 INFO:teuthology.orchestra.run.smithi084.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-850-g50bbe7f3-1jammy [38.3 kB] 2024-09-13T16:44:32.722 INFO:teuthology.orchestra.run.smithi084.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-850-g50bbe7f3-1jammy [11.7 MB] 2024-09-13T16:44:33.076 INFO:teuthology.orchestra.run.smithi084.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-850-g50bbe7f3-1jammy [93.1 kB] 2024-09-13T16:44:33.351 INFO:teuthology.orchestra.run.smithi039.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-850-g50bbe7f3-1jammy [122 kB] 2024-09-13T16:44:33.353 INFO:teuthology.orchestra.run.smithi039.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.4-850-g50bbe7f3-1jammy [233 kB] 2024-09-13T16:44:33.358 INFO:teuthology.orchestra.run.smithi039.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.4-850-g50bbe7f3-1jammy [34.0 kB] 2024-09-13T16:44:33.359 INFO:teuthology.orchestra.run.smithi039.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.4-850-g50bbe7f3-1jammy [23.5 kB] 2024-09-13T16:44:33.360 INFO:teuthology.orchestra.run.smithi039.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-850-g50bbe7f3-1jammy [38.3 kB] 2024-09-13T16:44:33.360 INFO:teuthology.orchestra.run.smithi039.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-850-g50bbe7f3-1jammy [11.7 MB] 2024-09-13T16:44:33.606 INFO:teuthology.orchestra.run.smithi039.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-850-g50bbe7f3-1jammy [93.1 kB] 2024-09-13T16:44:33.646 INFO:teuthology.orchestra.run.smithi084.stdout:Fetched 163 MB in 6s (26.8 MB/s) 2024-09-13T16:44:33.779 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T16:44:33.872 INFO:teuthology.orchestra.run.smithi084.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T16:44:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-13T16:44:33.919 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-13T16:44:34.203 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 163 MB in 7s (24.8 MB/s) 2024-09-13T16:44:34.213 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T16:44:34.240 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T16:44:34.280 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T16:44:34.320 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T16:44:34.420 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T16:44:34.426 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-13T16:44:34.460 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-13T16:44:34.716 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T16:44:34.761 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T16:44:34.762 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-13T16:44:34.765 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T16:44:34.783 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-13T16:44:34.804 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T16:44:35.077 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-13T16:44:35.092 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-13T16:44:35.140 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T16:44:35.160 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T16:44:35.167 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-13T16:44:35.207 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-13T16:44:35.455 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T16:44:35.481 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T16:44:35.521 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T16:44:35.535 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-13T16:44:35.561 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-13T16:44:35.643 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T16:44:35.875 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T16:44:35.901 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T16:44:35.941 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T16:44:35.946 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T16:44:35.973 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T16:44:36.013 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T16:44:36.270 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T16:44:36.296 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T16:44:36.328 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T16:44:36.358 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T16:44:36.385 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T16:44:36.424 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T16:44:36.769 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T16:44:36.781 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T16:44:36.796 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T16:44:36.808 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T16:44:36.835 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T16:44:36.856 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:44:37.239 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T16:44:37.259 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T16:44:37.266 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T16:44:37.286 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T16:44:37.315 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:44:37.325 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:44:37.629 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T16:44:37.655 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T16:44:37.687 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:44:37.726 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T16:44:37.753 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T16:44:37.792 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:44:38.032 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T16:44:38.058 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T16:44:38.098 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T16:44:38.137 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T16:44:38.164 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T16:44:38.203 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:44:38.418 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package librados2. 2024-09-13T16:44:38.444 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../011-librados2_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:38.484 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking librados2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:38.531 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T16:44:38.558 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T16:44:38.589 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T16:44:38.909 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librados2. 2024-09-13T16:44:38.936 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../011-librados2_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:38.945 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package librbd1. 2024-09-13T16:44:38.972 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../012-librbd1_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:38.975 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librados2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:39.012 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking librbd1 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:39.406 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T16:44:39.429 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librbd1. 2024-09-13T16:44:39.432 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../013-libcephfs2_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:39.455 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../012-librbd1_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:39.464 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libcephfs2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:39.495 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librbd1 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:39.751 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-rados. 2024-09-13T16:44:39.766 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../014-python3-rados_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:39.866 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-rados (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:39.923 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T16:44:39.949 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../013-libcephfs2_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:39.989 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libcephfs2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:40.153 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T16:44:40.180 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:44:40.220 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-ceph-argparse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:40.492 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rados. 2024-09-13T16:44:40.505 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T16:44:40.518 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../014-python3-rados_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:40.520 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../016-python3-cephfs_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:40.558 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rados (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:40.563 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-cephfs (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:40.858 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T16:44:40.878 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T16:44:40.884 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:44:40.904 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:44:40.925 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-ceph-common (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:40.944 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-ceph-argparse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:41.221 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T16:44:41.244 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T16:44:41.248 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../016-python3-cephfs_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:41.271 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T16:44:41.288 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cephfs (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:41.311 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T16:44:41.538 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T16:44:41.553 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../019-python3-rbd_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:41.588 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-rbd (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:41.599 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T16:44:41.626 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:44:41.666 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-ceph-common (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:41.941 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T16:44:41.952 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T16:44:41.967 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T16:44:41.979 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T16:44:42.008 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T16:44:42.010 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T16:44:42.263 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T16:44:42.290 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../019-python3-rbd_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:42.327 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package librgw2. 2024-09-13T16:44:42.329 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rbd (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:42.353 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../021-librgw2_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:42.393 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking librgw2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:42.699 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T16:44:42.726 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T16:44:42.765 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T16:44:42.888 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T16:44:42.903 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../022-python3-rgw_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:42.938 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-rgw (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:43.094 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librgw2. 2024-09-13T16:44:43.120 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../021-librgw2_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:43.160 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librgw2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:43.215 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T16:44:43.242 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T16:44:43.282 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T16:44:43.568 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T16:44:43.594 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../024-libradosstriper1_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:43.595 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T16:44:43.612 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../022-python3-rgw_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:43.635 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libradosstriper1 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:43.645 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rgw (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:43.982 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T16:44:44.009 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T16:44:44.048 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T16:44:44.071 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-common. 2024-09-13T16:44:44.097 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../025-ceph-common_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:44.138 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-common (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:44.376 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T16:44:44.403 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../024-libradosstriper1_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:44.443 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libradosstriper1 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:44.796 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-common. 2024-09-13T16:44:44.823 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../025-ceph-common_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:44.854 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-common (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:45.502 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-base. 2024-09-13T16:44:45.529 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../026-ceph-base_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:45.578 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-base (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:46.133 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T16:44:46.136 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-base. 2024-09-13T16:44:46.158 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T16:44:46.162 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../026-ceph-base_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:46.197 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T16:44:46.211 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-base (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:46.450 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T16:44:46.477 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T16:44:46.508 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T16:44:46.803 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T16:44:46.823 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T16:44:46.830 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T16:44:46.849 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T16:44:46.870 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T16:44:46.889 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T16:44:47.156 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T16:44:47.182 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T16:44:47.192 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T16:44:47.219 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T16:44:47.222 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T16:44:47.259 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T16:44:47.559 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T16:44:47.570 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T16:44:47.576 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T16:44:47.597 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T16:44:47.617 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T16:44:47.636 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T16:44:47.920 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T16:44:47.940 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T16:44:47.946 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T16:44:47.966 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T16:44:48.003 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T16:44:48.006 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T16:44:48.256 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-portend. 2024-09-13T16:44:48.282 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2024-09-13T16:44:48.314 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T16:44:48.351 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T16:44:48.377 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T16:44:48.417 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T16:44:48.617 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T16:44:48.643 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T16:44:48.683 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T16:44:48.720 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T16:44:48.747 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T16:44:48.786 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T16:44:48.969 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T16:44:48.984 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T16:44:49.028 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T16:44:49.098 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-portend. 2024-09-13T16:44:49.124 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2024-09-13T16:44:49.164 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T16:44:49.373 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T16:44:49.399 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T16:44:49.439 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T16:44:49.484 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T16:44:49.511 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T16:44:49.550 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T16:44:49.727 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T16:44:49.752 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T16:44:49.792 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T16:44:49.879 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T16:44:49.895 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T16:44:49.937 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T16:44:50.253 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T16:44:50.276 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T16:44:50.280 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T16:44:50.303 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T16:44:50.320 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T16:44:50.342 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T16:44:50.581 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-mako. 2024-09-13T16:44:50.596 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T16:44:50.631 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T16:44:50.776 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T16:44:50.803 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T16:44:50.862 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T16:44:50.934 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T16:44:50.960 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T16:44:51.001 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T16:44:51.165 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T16:44:51.192 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T16:44:51.232 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T16:44:51.320 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T16:44:51.346 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T16:44:51.387 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T16:44:51.501 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-mako. 2024-09-13T16:44:51.528 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T16:44:51.559 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T16:44:51.681 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-webob. 2024-09-13T16:44:51.707 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T16:44:51.747 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T16:44:51.863 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T16:44:51.890 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T16:44:51.929 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T16:44:52.067 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T16:44:52.082 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T16:44:52.120 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T16:44:52.190 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T16:44:52.206 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T16:44:52.240 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T16:44:52.403 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T16:44:52.430 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T16:44:52.462 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T16:44:52.535 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-webob. 2024-09-13T16:44:52.562 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T16:44:52.602 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T16:44:52.748 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-paste. 2024-09-13T16:44:52.774 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T16:44:52.815 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T16:44:52.972 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T16:44:52.988 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T16:44:53.033 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T16:44:53.109 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T16:44:53.124 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T16:44:53.159 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T16:44:53.341 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T16:44:53.368 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T16:44:53.416 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T16:44:53.445 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T16:44:53.471 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T16:44:53.511 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T16:44:53.719 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-paste. 2024-09-13T16:44:53.746 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T16:44:53.786 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T16:44:53.798 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T16:44:53.824 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T16:44:53.864 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T16:44:54.106 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T16:44:54.123 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T16:44:54.164 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T16:44:54.176 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T16:44:54.202 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T16:44:54.242 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T16:44:54.459 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T16:44:54.486 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T16:44:54.525 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T16:44:54.597 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T16:44:54.623 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T16:44:54.688 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T16:44:54.845 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T16:44:54.872 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T16:44:54.920 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T16:44:54.991 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-idna. 2024-09-13T16:44:55.018 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../051-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T16:44:55.049 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T16:44:55.274 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T16:44:55.300 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T16:44:55.340 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T16:44:55.361 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T16:44:55.387 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T16:44:55.436 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T16:44:55.703 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T16:44:55.730 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T16:44:55.778 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T16:44:55.780 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-requests. 2024-09-13T16:44:55.807 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T16:44:55.847 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T16:44:56.090 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-idna. 2024-09-13T16:44:56.116 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../051-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T16:44:56.125 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T16:44:56.151 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T16:44:56.156 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T16:44:56.191 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T16:44:56.468 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T16:44:56.469 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T16:44:56.495 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T16:44:56.496 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:44:56.536 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-mgr-modules-core (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:56.559 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T16:44:56.907 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T16:44:56.921 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-requests. 2024-09-13T16:44:56.922 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:56.948 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T16:44:56.957 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:56.996 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T16:44:57.278 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T16:44:57.287 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../057-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T16:44:57.317 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T16:44:57.327 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T16:44:57.342 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T16:44:57.382 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T16:44:57.629 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T16:44:57.656 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../058-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T16:44:57.677 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T16:44:57.695 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T16:44:57.704 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:44:57.743 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-modules-core (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:58.024 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T16:44:58.051 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../059-ceph-mgr_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:58.090 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-mgr (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:58.107 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T16:44:58.134 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:58.173 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:58.493 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T16:44:58.494 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T16:44:58.520 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../057-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T16:44:58.521 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../060-ceph-mon_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:58.560 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-mon (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:58.560 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T16:44:58.846 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T16:44:58.873 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../058-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T16:44:58.912 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T16:44:59.105 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T16:44:59.132 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../061-ceph-osd_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:59.163 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-osd (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:59.216 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T16:44:59.243 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../059-ceph-mgr_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:59.274 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:44:59.677 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T16:44:59.704 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../060-ceph-mon_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:44:59.744 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mon (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:01.178 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph. 2024-09-13T16:45:01.204 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../062-ceph_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:01.244 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:01.341 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T16:45:01.368 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../061-ceph-osd_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-osd (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:01.530 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T16:45:01.557 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../063-ceph-fuse_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:01.596 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-fuse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:01.967 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T16:45:01.993 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../064-ceph-mds_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:02.033 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-mds (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:02.394 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph. 2024-09-13T16:45:02.411 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package cephadm. 2024-09-13T16:45:02.421 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../062-ceph_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:02.438 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../065-cephadm_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:02.460 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:02.478 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking cephadm (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:02.780 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T16:45:02.789 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T16:45:02.807 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../063-ceph-fuse_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:02.816 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../066-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T16:45:02.846 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-fuse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:02.855 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T16:45:03.167 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T16:45:03.182 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../067-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T16:45:03.192 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T16:45:03.219 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../064-ceph-mds_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:03.225 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T16:45:03.258 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mds (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:03.620 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package cephadm. 2024-09-13T16:45:03.647 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../065-cephadm_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:03.678 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking cephadm (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:03.998 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T16:45:04.025 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../066-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T16:45:04.064 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T16:45:04.147 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-babel. 2024-09-13T16:45:04.174 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../068-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T16:45:04.213 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T16:45:04.393 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T16:45:04.420 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../067-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T16:45:04.459 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T16:45:04.524 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T16:45:04.551 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../069-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T16:45:04.591 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T16:45:04.961 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T16:45:04.988 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../070-ceph-mgr-cephadm_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:05.027 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-mgr-cephadm (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:05.356 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T16:45:05.383 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T16:45:05.439 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T16:45:05.506 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-babel. 2024-09-13T16:45:05.534 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../068-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T16:45:05.572 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T16:45:05.734 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-py. 2024-09-13T16:45:05.761 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2024-09-13T16:45:05.800 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T16:45:05.901 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T16:45:05.928 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../069-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T16:45:05.968 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T16:45:06.146 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T16:45:06.173 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T16:45:06.220 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T16:45:06.329 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T16:45:06.357 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../070-ceph-mgr-cephadm_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:06.396 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-cephadm (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:06.506 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-toml. 2024-09-13T16:45:06.533 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2024-09-13T16:45:06.565 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T16:45:06.708 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T16:45:06.735 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T16:45:06.834 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T16:45:06.841 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T16:45:06.861 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T16:45:06.901 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T16:45:07.128 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-py. 2024-09-13T16:45:07.155 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2024-09-13T16:45:07.194 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T16:45:07.313 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T16:45:07.340 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T16:45:07.371 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T16:45:07.523 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T16:45:07.550 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T16:45:07.589 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T16:45:07.641 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-routes. 2024-09-13T16:45:07.667 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T16:45:07.707 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T16:45:07.893 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-toml. 2024-09-13T16:45:07.920 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2024-09-13T16:45:07.967 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T16:45:08.027 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T16:45:08.054 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:08.093 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-mgr-dashboard (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:08.270 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T16:45:08.298 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T16:45:08.328 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T16:45:08.657 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T16:45:08.685 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T16:45:08.724 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T16:45:08.985 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-routes. 2024-09-13T16:45:09.013 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T16:45:09.043 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T16:45:09.282 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T16:45:09.309 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T16:45:09.348 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T16:45:09.355 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T16:45:09.383 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:09.422 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-dashboard (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:09.768 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T16:45:09.795 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T16:45:09.826 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T16:45:10.163 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T16:45:10.190 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T16:45:10.233 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T16:45:10.507 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T16:45:10.534 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T16:45:10.565 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T16:45:10.677 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T16:45:10.694 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T16:45:10.735 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T16:45:11.248 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T16:45:11.275 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T16:45:11.314 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T16:45:11.567 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T16:45:11.595 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:11.633 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:11.910 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T16:45:11.927 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T16:45:11.976 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T16:45:12.254 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T16:45:12.282 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T16:45:12.312 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T16:45:12.629 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T16:45:12.655 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T16:45:12.695 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T16:45:12.998 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T16:45:13.025 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T16:45:13.064 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T16:45:13.132 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T16:45:13.160 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:13.198 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:13.393 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T16:45:13.420 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T16:45:13.459 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T16:45:13.746 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T16:45:13.773 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-13T16:45:13.812 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T16:45:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T16:45:14.104 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T16:45:14.132 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T16:45:14.135 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T16:45:14.159 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T16:45:14.199 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T16:45:14.421 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T16:45:14.449 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T16:45:14.487 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T16:45:14.502 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T16:45:14.529 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T16:45:14.568 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T16:45:14.825 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T16:45:14.852 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T16:45:14.880 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T16:45:14.891 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T16:45:14.906 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T16:45:14.946 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T16:45:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T16:45:15.255 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-13T16:45:15.258 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T16:45:15.285 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T16:45:15.294 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T16:45:15.324 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T16:45:15.565 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T16:45:15.592 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T16:45:15.631 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T16:45:15.926 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T16:45:15.954 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T16:45:15.971 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T16:45:15.993 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T16:45:15.999 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:16.038 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:16.305 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T16:45:16.333 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T16:45:16.340 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T16:45:16.357 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T16:45:16.371 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T16:45:16.390 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T16:45:16.691 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T16:45:16.710 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T16:45:16.719 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T16:45:16.737 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T16:45:16.757 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T16:45:16.777 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T16:45:17.037 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package jq. 2024-09-13T16:45:17.053 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T16:45:17.096 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T16:45:17.455 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T16:45:17.474 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:17.490 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package socat. 2024-09-13T16:45:17.513 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:17.517 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T16:45:17.557 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T16:45:17.827 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T16:45:17.833 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T16:45:17.854 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T16:45:17.861 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T16:45:17.885 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T16:45:17.899 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T16:45:18.245 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T16:45:18.247 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-test. 2024-09-13T16:45:18.272 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T16:45:18.275 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../098-ceph-test_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:18.311 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T16:45:18.313 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-test (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:18.539 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package jq. 2024-09-13T16:45:18.567 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T16:45:18.597 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T16:45:18.833 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package socat. 2024-09-13T16:45:18.861 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T16:45:18.899 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T16:45:19.254 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T16:45:19.284 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T16:45:19.320 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T16:45:19.700 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-test. 2024-09-13T16:45:19.716 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../098-ceph-test_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:19.758 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-test (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:20.201 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T16:45:20.229 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../099-ceph-volume_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:20.267 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ceph-volume (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:20.639 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T16:45:20.666 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:20.705 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking cephfs-mirror (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:20.991 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T16:45:21.018 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2024-09-13T16:45:21.058 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-13T16:45:21.260 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T16:45:21.276 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../102-cephfs-shell_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:21.310 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking cephfs-shell (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:21.563 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T16:45:21.590 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../103-cephfs-top_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:21.629 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking cephfs-top (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:21.687 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T16:45:21.715 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../099-ceph-volume_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:21.753 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-volume (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:21.882 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T16:45:21.909 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:21.949 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libcephfs-dev (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:22.125 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T16:45:22.153 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:22.191 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking cephfs-mirror (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:22.285 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T16:45:22.301 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T16:45:22.343 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T16:45:22.520 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T16:45:22.547 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2024-09-13T16:45:22.586 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-13T16:45:22.691 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T16:45:22.718 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T16:45:22.757 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T16:45:22.848 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T16:45:22.876 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../102-cephfs-shell_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:22.914 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking cephfs-shell (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:23.150 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T16:45:23.177 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T16:45:23.178 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../103-cephfs-top_18.2.4-850-g50bbe7f3-1jammy_all.deb ... 2024-09-13T16:45:23.204 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T16:45:23.208 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking cephfs-top (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:23.243 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T16:45:23.470 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T16:45:23.498 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:23.536 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libcephfs-dev (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:23.571 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T16:45:23.598 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T16:45:23.637 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T16:45:23.914 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T16:45:23.942 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T16:45:23.965 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T16:45:23.981 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T16:45:23.993 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T16:45:24.032 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T16:45:24.320 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T16:45:24.336 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T16:45:24.377 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T16:45:24.503 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T16:45:24.530 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T16:45:24.569 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T16:45:24.781 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T16:45:24.809 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T16:45:24.855 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T16:45:24.872 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T16:45:24.899 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T16:45:24.938 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T16:45:25.184 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T16:45:25.212 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T16:45:25.216 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T16:45:25.243 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T16:45:25.250 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T16:45:25.282 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T16:45:25.612 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T16:45:25.640 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T16:45:25.687 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T16:45:26.003 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package radosgw. 2024-09-13T16:45:26.031 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../113-radosgw_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:26.069 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking radosgw (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:26.174 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T16:45:26.191 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T16:45:26.224 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T16:45:26.535 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T16:45:26.564 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T16:45:26.610 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T16:45:26.863 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T16:45:26.880 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T16:45:26.921 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T16:45:27.089 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T16:45:27.104 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../114-rbd-fuse_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:27.138 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking rbd-fuse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:27.401 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T16:45:27.651 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T16:45:27.777 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T16:45:27.984 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package radosgw. 2024-09-13T16:45:28.012 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../113-radosgw_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:28.040 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T16:45:28.050 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking radosgw (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:28.363 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T16:45:28.781 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T16:45:28.809 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../114-rbd-fuse_18.2.4-850-g50bbe7f3-1jammy_amd64.deb ... 2024-09-13T16:45:28.831 INFO:teuthology.orchestra.run.smithi084.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T16:45:28.847 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking rbd-fuse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:28.896 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up cephadm (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:29.160 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T16:45:29.426 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T16:45:29.437 INFO:teuthology.orchestra.run.smithi084.stdout:Adding system user cephadm....done 2024-09-13T16:45:29.552 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T16:45:29.586 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T16:45:29.824 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T16:45:29.862 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T16:45:30.105 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T16:45:30.162 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T16:45:30.231 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T16:45:30.482 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T16:45:30.581 INFO:teuthology.orchestra.run.smithi039.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T16:45:30.646 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up cephadm (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:30.783 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T16:45:30.901 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T16:45:31.186 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T16:45:31.212 INFO:teuthology.orchestra.run.smithi039.stdout:Adding system user cephadm....done 2024-09-13T16:45:31.368 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T16:45:31.546 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T16:45:31.645 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T16:45:31.913 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T16:45:31.930 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-13T16:45:32.039 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T16:45:32.190 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T16:45:32.290 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T16:45:32.466 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T16:45:32.566 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T16:45:32.701 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T16:45:32.867 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T16:45:32.977 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T16:45:33.160 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T16:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-ceph-argparse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T16:45:33.537 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T16:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-13T16:45:33.805 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-13T16:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T16:45:34.089 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T16:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T16:45:34.233 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T16:45:34.366 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T16:45:34.593 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T16:45:34.625 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T16:45:34.720 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T16:45:34.859 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T16:45:34.986 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T16:45:35.144 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-ceph-argparse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:35.245 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T16:45:35.411 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T16:45:35.538 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-13T16:45:35.555 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T16:45:35.797 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T16:45:36.056 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T16:45:36.182 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T16:45:36.190 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T16:45:36.442 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T16:45:36.542 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T16:45:36.567 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T16:45:36.667 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T16:45:36.693 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T16:45:36.985 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T16:45:36.994 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T16:45:37.261 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T16:45:37.396 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T16:45:37.571 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T16:45:37.638 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T16:45:38.006 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T16:45:38.281 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T16:45:38.291 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-13T16:45:38.549 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T16:45:38.601 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T16:45:38.675 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T16:45:38.817 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T16:45:38.952 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T16:45:39.135 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T16:45:39.211 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T16:45:39.337 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-ceph-common (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:39.528 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T16:45:39.630 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T16:45:39.762 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T16:45:39.939 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T16:45:40.214 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T16:45:40.515 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-13T16:45:40.833 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T16:45:41.095 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:45:41.176 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T16:45:41.221 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T16:45:41.427 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T16:45:41.472 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T16:45:41.552 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-ceph-common (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:41.615 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T16:45:41.741 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T16:45:41.846 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T16:45:42.050 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T16:45:42.172 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T16:45:42.618 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T16:45:42.877 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:45:42.978 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T16:45:43.096 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T16:45:43.389 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T16:45:43.410 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:45:43.544 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T16:45:43.640 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T16:45:43.846 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T16:45:43.901 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T16:45:43.963 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T16:45:44.089 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T16:45:44.150 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T16:45:44.402 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T16:45:44.415 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T16:45:44.658 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T16:45:44.661 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T16:45:44.770 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T16:45:44.893 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:45:45.019 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T16:45:45.054 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T16:45:45.144 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T16:45:45.189 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T16:45:45.479 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T16:45:45.481 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T16:45:45.739 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T16:45:45.785 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T16:45:46.015 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T16:45:46.036 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T16:45:46.282 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T16:45:46.433 INFO:teuthology.orchestra.run.smithi084.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T16:45:46.533 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T16:45:46.672 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T16:45:46.784 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T16:45:46.798 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T16:45:46.910 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T16:45:47.177 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T16:45:47.295 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T16:45:47.421 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T16:45:47.433 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T16:45:47.693 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:45:47.722 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T16:45:47.819 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T16:45:48.023 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T16:45:48.153 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T16:45:48.282 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T16:45:48.455 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T16:45:48.696 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T16:45:48.722 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T16:45:48.864 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T16:45:48.969 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T16:45:49.095 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T16:45:49.140 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T16:45:49.408 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T16:45:49.496 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T16:45:49.676 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T16:45:49.756 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T16:45:50.007 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T16:45:50.027 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T16:45:50.133 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T16:45:50.303 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T16:45:50.517 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T16:45:50.680 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T16:45:50.806 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up librados2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:50.835 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T16:45:50.932 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up librgw2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:51.094 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T16:45:51.135 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libsqlite3-mod-ceph (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:51.204 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T16:45:51.236 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libcephfs2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:51.362 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libradosstriper1 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:51.471 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T16:45:51.488 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T16:45:51.756 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T16:45:51.765 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up librbd1 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:51.891 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-mgr-modules-core (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:52.017 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-fuse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:52.049 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T16:45:52.237 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T16:45:52.238 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T16:45:52.442 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T16:45:52.735 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T16:45:52.735 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libcephfs-dev (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:52.844 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-rados (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:52.962 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up cephfs-top (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:53.079 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T16:45:53.088 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T16:45:53.205 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librados2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:53.331 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librgw2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:53.558 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libsqlite3-mod-ceph (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:53.701 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libcephfs2 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:53.827 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libradosstriper1 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:53.962 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T16:45:54.239 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librbd1 (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:54.259 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-rbd (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:54.364 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-modules-core (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:54.384 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up rbd-fuse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:54.490 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-fuse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:54.510 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-rgw (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:54.620 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up python3-cephfs (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T16:45:54.709 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T16:45:54.721 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up cephfs-shell (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:54.838 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-common (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:55.183 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libcephfs-dev (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:55.309 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rados (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:55.390 INFO:teuthology.orchestra.run.smithi084.stdout:Adding group ceph....done 2024-09-13T16:45:55.435 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up cephfs-top (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:55.561 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T16:45:55.757 INFO:teuthology.orchestra.run.smithi084.stdout:Adding system user ceph....done 2024-09-13T16:45:55.943 INFO:teuthology.orchestra.run.smithi084.stdout:Setting system user ceph properties....done 2024-09-13T16:45:55.957 INFO:teuthology.orchestra.run.smithi084.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T16:45:56.092 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T16:45:56.418 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T16:45:56.799 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rbd (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:56.931 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up cephfs-mirror (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:56.941 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up rbd-fuse (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:57.093 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rgw (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:57.142 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:45:57.142 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:45:57.219 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cephfs (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:57.345 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up cephfs-shell (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:57.471 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-common (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:57.665 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-test (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:57.791 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up radosgw (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:58.039 INFO:teuthology.orchestra.run.smithi039.stdout:Adding group ceph....done 2024-09-13T16:45:58.293 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:45:58.294 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:45:58.440 INFO:teuthology.orchestra.run.smithi039.stdout:Adding system user ceph....done 2024-09-13T16:45:58.593 INFO:teuthology.orchestra.run.smithi039.stdout:Setting system user ceph properties....done 2024-09-13T16:45:58.613 INFO:teuthology.orchestra.run.smithi039.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T16:45:58.758 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T16:45:58.793 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-base (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:59.113 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T16:45:59.131 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T16:45:59.586 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-mds (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:59.589 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up cephfs-mirror (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:45:59.801 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T16:45:59.801 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T16:45:59.807 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:45:59.807 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T16:46:00.298 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-test (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:00.304 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-mgr (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:00.424 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up radosgw (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:00.513 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T16:46:00.514 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T16:46:00.899 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:46:00.899 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T16:46:00.997 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-osd (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:01.279 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T16:46:01.279 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T16:46:01.359 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-base (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:01.731 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T16:46:01.807 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-mgr-k8sevents (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:01.933 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:02.067 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-mon (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:02.203 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mds (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:02.280 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T16:46:02.281 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T16:46:02.410 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T16:46:02.410 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T16:46:02.785 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-mgr-cephadm (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:02.927 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:02.938 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:03.104 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-mgr-dashboard (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:03.169 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T16:46:03.169 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T16:46:03.255 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ceph-volume (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:03.608 INFO:teuthology.orchestra.run.smithi084.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T16:46:03.865 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-osd (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:04.164 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T16:46:04.164 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T16:46:04.650 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-k8sevents (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:04.742 INFO:teuthology.orchestra.run.smithi084.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T16:46:04.776 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:04.927 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mon (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:05.146 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T16:46:05.146 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T16:46:05.219 INFO:teuthology.orchestra.run.smithi084.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T16:46:05.222 DEBUG:teuthology.orchestra.run.smithi084:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T16:46:05.276 INFO:teuthology.orchestra.run.smithi084.stdout:Reading package lists... 2024-09-13T16:46:05.488 INFO:teuthology.orchestra.run.smithi084.stdout:Building dependency tree... 2024-09-13T16:46:05.488 INFO:teuthology.orchestra.run.smithi084.stdout:Reading state information... 2024-09-13T16:46:05.611 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-cephadm (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:05.674 INFO:teuthology.orchestra.run.smithi084.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-13T16:46:05.674 INFO:teuthology.orchestra.run.smithi084.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-13T16:46:05.675 INFO:teuthology.orchestra.run.smithi084.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-13T16:46:05.675 INFO:teuthology.orchestra.run.smithi084.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:46:05.675 INFO:teuthology.orchestra.run.smithi084.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T16:46:05.675 INFO:teuthology.orchestra.run.smithi084.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T16:46:05.675 INFO:teuthology.orchestra.run.smithi084.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T16:46:05.675 INFO:teuthology.orchestra.run.smithi084.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T16:46:05.675 INFO:teuthology.orchestra.run.smithi084.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T16:46:05.676 INFO:teuthology.orchestra.run.smithi084.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:46:05.676 INFO:teuthology.orchestra.run.smithi084.stdout:The following additional packages will be installed: 2024-09-13T16:46:05.676 INFO:teuthology.orchestra.run.smithi084.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-13T16:46:05.676 INFO:teuthology.orchestra.run.smithi084.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-13T16:46:05.677 INFO:teuthology.orchestra.run.smithi084.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-13T16:46:05.677 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-13T16:46:05.677 INFO:teuthology.orchestra.run.smithi084.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T16:46:05.678 INFO:teuthology.orchestra.run.smithi084.stdout:Suggested packages: 2024-09-13T16:46:05.678 INFO:teuthology.orchestra.run.smithi084.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-13T16:46:05.678 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-13T16:46:05.678 INFO:teuthology.orchestra.run.smithi084.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-13T16:46:05.720 INFO:teuthology.orchestra.run.smithi084.stdout:The following NEW packages will be installed: 2024-09-13T16:46:05.721 INFO:teuthology.orchestra.run.smithi084.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-13T16:46:05.721 INFO:teuthology.orchestra.run.smithi084.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-13T16:46:05.721 INFO:teuthology.orchestra.run.smithi084.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-13T16:46:05.721 INFO:teuthology.orchestra.run.smithi084.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-13T16:46:05.721 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-13T16:46:05.722 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-13T16:46:05.722 INFO:teuthology.orchestra.run.smithi084.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T16:46:05.723 INFO:teuthology.orchestra.run.smithi084.stdout:The following packages will be upgraded: 2024-09-13T16:46:05.724 INFO:teuthology.orchestra.run.smithi084.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-13T16:46:05.796 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:05.904 INFO:teuthology.orchestra.run.smithi084.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-13T16:46:05.904 INFO:teuthology.orchestra.run.smithi084.stdout:Need to get 55.8 MB of archives. 2024-09-13T16:46:05.904 INFO:teuthology.orchestra.run.smithi084.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-13T16:46:05.904 INFO:teuthology.orchestra.run.smithi084.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-13T16:46:05.922 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-dashboard (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:06.073 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-volume (18.2.4-850-g50bbe7f3-1jammy) ... 2024-09-13T16:46:06.434 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T16:46:06.528 INFO:teuthology.orchestra.run.smithi084.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-13T16:46:06.544 INFO:teuthology.orchestra.run.smithi084.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-13T16:46:06.668 INFO:teuthology.orchestra.run.smithi084.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-13T16:46:06.669 INFO:teuthology.orchestra.run.smithi084.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-13T16:46:06.670 INFO:teuthology.orchestra.run.smithi084.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-13T16:46:06.698 INFO:teuthology.orchestra.run.smithi084.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-13T16:46:06.700 INFO:teuthology.orchestra.run.smithi084.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-13T16:46:06.703 INFO:teuthology.orchestra.run.smithi084.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-13T16:46:06.706 INFO:teuthology.orchestra.run.smithi084.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-13T16:46:06.707 INFO:teuthology.orchestra.run.smithi084.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-13T16:46:06.709 INFO:teuthology.orchestra.run.smithi084.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-13T16:46:06.772 INFO:teuthology.orchestra.run.smithi084.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-13T16:46:06.772 INFO:teuthology.orchestra.run.smithi084.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-13T16:46:06.773 INFO:teuthology.orchestra.run.smithi084.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-13T16:46:06.853 INFO:teuthology.orchestra.run.smithi084.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-13T16:46:07.510 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T16:46:07.638 INFO:teuthology.orchestra.run.smithi084.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-13T16:46:07.643 INFO:teuthology.orchestra.run.smithi084.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-13T16:46:07.643 INFO:teuthology.orchestra.run.smithi084.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-13T16:46:07.650 INFO:teuthology.orchestra.run.smithi084.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-13T16:46:07.650 INFO:teuthology.orchestra.run.smithi084.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-13T16:46:07.661 INFO:teuthology.orchestra.run.smithi084.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-13T16:46:07.664 INFO:teuthology.orchestra.run.smithi084.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-13T16:46:07.681 INFO:teuthology.orchestra.run.smithi084.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-13T16:46:07.682 INFO:teuthology.orchestra.run.smithi084.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-13T16:46:07.749 INFO:teuthology.orchestra.run.smithi084.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-13T16:46:07.767 INFO:teuthology.orchestra.run.smithi084.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-13T16:46:07.790 INFO:teuthology.orchestra.run.smithi084.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-13T16:46:07.856 INFO:teuthology.orchestra.run.smithi084.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-13T16:46:07.861 INFO:teuthology.orchestra.run.smithi084.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-13T16:46:07.861 INFO:teuthology.orchestra.run.smithi084.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-13T16:46:07.862 INFO:teuthology.orchestra.run.smithi084.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-13T16:46:07.863 INFO:teuthology.orchestra.run.smithi084.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-13T16:46:07.889 INFO:teuthology.orchestra.run.smithi084.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-13T16:46:07.892 INFO:teuthology.orchestra.run.smithi084.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-13T16:46:07.988 INFO:teuthology.orchestra.run.smithi039.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T16:46:07.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T16:46:08.050 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2024-09-13T16:46:08.264 INFO:teuthology.orchestra.run.smithi039.stdout:Building dependency tree... 2024-09-13T16:46:08.265 INFO:teuthology.orchestra.run.smithi039.stdout:Reading state information... 2024-09-13T16:46:08.443 INFO:teuthology.orchestra.run.smithi084.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-13T16:46:08.443 INFO:teuthology.orchestra.run.smithi084.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-13T16:46:08.444 INFO:teuthology.orchestra.run.smithi084.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-13T16:46:08.444 INFO:teuthology.orchestra.run.smithi084.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-13T16:46:08.444 INFO:teuthology.orchestra.run.smithi084.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-13T16:46:08.445 INFO:teuthology.orchestra.run.smithi084.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-13T16:46:08.473 INFO:teuthology.orchestra.run.smithi084.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-13T16:46:08.473 INFO:teuthology.orchestra.run.smithi084.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-13T16:46:08.478 INFO:teuthology.orchestra.run.smithi039.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-13T16:46:08.478 INFO:teuthology.orchestra.run.smithi039.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-13T16:46:08.478 INFO:teuthology.orchestra.run.smithi039.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-13T16:46:08.479 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T16:46:08.479 INFO:teuthology.orchestra.run.smithi039.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T16:46:08.479 INFO:teuthology.orchestra.run.smithi039.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T16:46:08.479 INFO:teuthology.orchestra.run.smithi039.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T16:46:08.479 INFO:teuthology.orchestra.run.smithi039.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T16:46:08.480 INFO:teuthology.orchestra.run.smithi039.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T16:46:08.480 INFO:teuthology.orchestra.run.smithi039.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T16:46:08.480 INFO:teuthology.orchestra.run.smithi039.stdout:The following additional packages will be installed: 2024-09-13T16:46:08.480 INFO:teuthology.orchestra.run.smithi039.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-13T16:46:08.481 INFO:teuthology.orchestra.run.smithi039.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-13T16:46:08.481 INFO:teuthology.orchestra.run.smithi039.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-13T16:46:08.481 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-13T16:46:08.481 INFO:teuthology.orchestra.run.smithi039.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T16:46:08.483 INFO:teuthology.orchestra.run.smithi039.stdout:Suggested packages: 2024-09-13T16:46:08.483 INFO:teuthology.orchestra.run.smithi039.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-13T16:46:08.483 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-13T16:46:08.483 INFO:teuthology.orchestra.run.smithi039.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-13T16:46:08.527 INFO:teuthology.orchestra.run.smithi039.stdout:The following NEW packages will be installed: 2024-09-13T16:46:08.528 INFO:teuthology.orchestra.run.smithi039.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-13T16:46:08.528 INFO:teuthology.orchestra.run.smithi039.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-13T16:46:08.529 INFO:teuthology.orchestra.run.smithi039.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-13T16:46:08.529 INFO:teuthology.orchestra.run.smithi039.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-13T16:46:08.529 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-13T16:46:08.529 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-13T16:46:08.529 INFO:teuthology.orchestra.run.smithi039.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T16:46:08.530 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages will be upgraded: 2024-09-13T16:46:08.531 INFO:teuthology.orchestra.run.smithi039.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-13T16:46:08.584 INFO:teuthology.orchestra.run.smithi039.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-13T16:46:08.585 INFO:teuthology.orchestra.run.smithi039.stdout:Need to get 55.8 MB of archives. 2024-09-13T16:46:08.585 INFO:teuthology.orchestra.run.smithi039.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-13T16:46:08.585 INFO:teuthology.orchestra.run.smithi039.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-13T16:46:08.753 INFO:teuthology.orchestra.run.smithi039.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-13T16:46:08.760 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-13T16:46:08.802 INFO:teuthology.orchestra.run.smithi039.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-13T16:46:08.802 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-13T16:46:08.804 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-13T16:46:08.816 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-13T16:46:08.817 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-13T16:46:08.819 INFO:teuthology.orchestra.run.smithi039.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-13T16:46:08.821 INFO:teuthology.orchestra.run.smithi039.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-13T16:46:08.822 INFO:teuthology.orchestra.run.smithi039.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-13T16:46:08.823 INFO:teuthology.orchestra.run.smithi039.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-13T16:46:08.824 INFO:teuthology.orchestra.run.smithi039.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-13T16:46:08.842 INFO:teuthology.orchestra.run.smithi039.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-13T16:46:08.843 INFO:teuthology.orchestra.run.smithi039.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-13T16:46:08.845 INFO:teuthology.orchestra.run.smithi039.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-13T16:46:08.884 INFO:teuthology.orchestra.run.smithi084.stdout:Preconfiguring packages ... 2024-09-13T16:46:09.046 INFO:teuthology.orchestra.run.smithi084.stdout:Fetched 55.8 MB in 3s (20.3 MB/s) 2024-09-13T16:46:09.193 INFO:teuthology.orchestra.run.smithi039.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-13T16:46:09.196 INFO:teuthology.orchestra.run.smithi039.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-13T16:46:09.196 INFO:teuthology.orchestra.run.smithi039.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-13T16:46:09.198 INFO:teuthology.orchestra.run.smithi039.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-13T16:46:09.199 INFO:teuthology.orchestra.run.smithi039.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-13T16:46:09.204 INFO:teuthology.orchestra.run.smithi039.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-13T16:46:09.205 INFO:teuthology.orchestra.run.smithi039.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-13T16:46:09.213 INFO:teuthology.orchestra.run.smithi039.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-13T16:46:09.213 INFO:teuthology.orchestra.run.smithi039.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-13T16:46:09.236 INFO:teuthology.orchestra.run.smithi039.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-13T16:46:09.236 INFO:teuthology.orchestra.run.smithi039.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-13T16:46:09.246 INFO:teuthology.orchestra.run.smithi039.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-13T16:46:09.266 INFO:teuthology.orchestra.run.smithi039.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-13T16:46:09.270 INFO:teuthology.orchestra.run.smithi039.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-13T16:46:09.270 INFO:teuthology.orchestra.run.smithi039.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-13T16:46:09.271 INFO:teuthology.orchestra.run.smithi039.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-13T16:46:09.271 INFO:teuthology.orchestra.run.smithi039.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-13T16:46:09.281 INFO:teuthology.orchestra.run.smithi039.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-13T16:46:09.283 INFO:teuthology.orchestra.run.smithi039.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-13T16:46:09.370 INFO:teuthology.orchestra.run.smithi084.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173231 files and directories currently installed.) 2024-09-13T16:46:09.376 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T16:46:09.485 INFO:teuthology.orchestra.run.smithi039.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-13T16:46:09.485 INFO:teuthology.orchestra.run.smithi039.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-13T16:46:09.485 INFO:teuthology.orchestra.run.smithi039.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-13T16:46:09.485 INFO:teuthology.orchestra.run.smithi039.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-13T16:46:09.486 INFO:teuthology.orchestra.run.smithi039.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-13T16:46:09.486 INFO:teuthology.orchestra.run.smithi039.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-13T16:46:09.497 INFO:teuthology.orchestra.run.smithi039.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-13T16:46:09.498 INFO:teuthology.orchestra.run.smithi039.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-13T16:46:09.523 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T16:46:09.923 INFO:teuthology.orchestra.run.smithi039.stdout:Preconfiguring packages ... 2024-09-13T16:46:10.082 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 55.8 MB in 1s (58.1 MB/s) 2024-09-13T16:46:10.150 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-13T16:46:10.407 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173231 files and directories currently installed.) 2024-09-13T16:46:10.413 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T16:46:10.566 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T16:46:11.127 INFO:teuthology.orchestra.run.smithi084.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-13T16:46:11.203 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-13T16:46:11.402 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package flex. 2024-09-13T16:46:11.477 INFO:teuthology.orchestra.run.smithi084.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173231 files and directories currently installed.) 2024-09-13T16:46:11.482 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-13T16:46:11.525 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-13T16:46:12.153 INFO:teuthology.orchestra.run.smithi039.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-13T16:46:12.345 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T16:46:12.421 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package flex. 2024-09-13T16:46:12.525 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173231 files and directories currently installed.) 2024-09-13T16:46:12.531 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-13T16:46:12.577 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-13T16:46:12.667 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-13T16:46:13.130 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T16:46:13.215 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T16:46:13.308 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-13T16:46:13.726 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T16:46:14.217 INFO:teuthology.orchestra.run.smithi084.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173304 files and directories currently installed.) 2024-09-13T16:46:14.222 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T16:46:14.308 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173304 files and directories currently installed.) 2024-09-13T16:46:14.313 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T16:46:14.381 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T16:46:14.441 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T16:46:14.807 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T16:46:14.826 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T16:46:14.946 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T16:46:14.956 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T16:46:15.272 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T16:46:15.291 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T16:46:15.621 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package bison. 2024-09-13T16:46:15.623 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package bison. 2024-09-13T16:46:15.725 INFO:teuthology.orchestra.run.smithi084.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173304 files and directories currently installed.) 2024-09-13T16:46:15.727 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173304 files and directories currently installed.) 2024-09-13T16:46:15.730 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-13T16:46:15.733 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-13T16:46:15.773 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T16:46:15.776 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T16:46:16.201 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T16:46:16.208 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T16:46:16.229 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-13T16:46:16.235 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-13T16:46:16.268 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T16:46:16.274 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T16:46:16.585 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package dump. 2024-09-13T16:46:16.612 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package dump. 2024-09-13T16:46:16.612 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-13T16:46:16.640 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-13T16:46:16.652 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking dump (0.4b47-3) ... 2024-09-13T16:46:16.678 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking dump (0.4b47-3) ... 2024-09-13T16:46:16.922 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package indent. 2024-09-13T16:46:16.949 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package indent. 2024-09-13T16:46:16.949 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:16.977 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:16.988 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T16:46:17.015 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T16:46:17.250 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T16:46:17.265 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-13T16:46:17.308 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T16:46:17.310 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T16:46:17.338 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-13T16:46:17.376 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T16:46:17.670 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T16:46:17.697 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-13T16:46:17.714 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T16:46:17.736 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T16:46:17.742 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-13T16:46:17.780 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T16:46:17.981 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T16:46:17.996 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-13T16:46:18.016 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T16:46:18.030 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T16:46:18.033 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-13T16:46:18.066 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T16:46:18.367 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-13T16:46:18.395 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-13T16:46:18.403 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-13T16:46:18.431 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-13T16:46:18.433 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T16:46:18.469 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T16:46:18.686 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-13T16:46:18.714 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-13T16:46:18.724 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-13T16:46:18.741 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-13T16:46:18.753 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T16:46:18.782 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T16:46:19.005 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libjson-perl. 2024-09-13T16:46:19.027 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libjson-perl. 2024-09-13T16:46:19.033 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-13T16:46:19.055 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-13T16:46:19.072 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-13T16:46:19.093 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-13T16:46:19.358 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-13T16:46:19.388 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-13T16:46:19.388 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-13T16:46:19.405 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-13T16:46:19.446 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T16:46:19.467 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T16:46:21.116 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T16:46:21.144 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:46:21.174 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T16:46:21.220 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T16:46:21.247 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:46:21.286 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T16:46:21.402 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T16:46:21.430 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:46:21.468 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T16:46:21.547 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T16:46:21.563 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T16:46:21.605 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T16:46:21.788 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T16:46:21.816 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T16:46:21.854 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T16:46:21.908 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T16:46:21.936 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T16:46:21.966 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T16:46:22.199 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T16:46:22.227 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-13T16:46:22.265 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T16:46:22.303 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T16:46:22.330 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-13T16:46:22.369 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T16:46:22.543 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T16:46:22.572 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T16:46:22.610 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T16:46:22.697 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T16:46:22.724 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T16:46:22.763 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T16:46:22.963 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T16:46:22.991 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:23.029 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:23.100 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T16:46:23.127 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:23.158 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:23.477 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T16:46:23.505 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:23.544 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T16:46:23.558 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T16:46:23.586 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:23.624 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T16:46:23.846 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T16:46:23.862 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:23.905 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T16:46:23.918 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T16:46:23.935 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:23.968 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T16:46:24.149 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T16:46:24.164 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T16:46:24.207 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T16:46:24.230 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T16:46:24.247 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T16:46:24.288 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T16:46:24.901 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T16:46:24.929 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-13T16:46:24.967 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T16:46:25.030 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T16:46:25.057 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-13T16:46:25.096 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T16:46:25.270 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package modemmanager. 2024-09-13T16:46:25.299 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T16:46:25.328 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T16:46:25.408 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package modemmanager. 2024-09-13T16:46:25.435 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T16:46:25.474 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T16:46:25.969 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package network-manager. 2024-09-13T16:46:25.997 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T16:46:26.044 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-13T16:46:26.089 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package network-manager. 2024-09-13T16:46:26.116 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T16:46:26.164 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-13T16:46:26.684 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ppp. 2024-09-13T16:46:26.712 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-13T16:46:26.762 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package ppp. 2024-09-13T16:46:26.790 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-13T16:46:27.023 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T16:46:27.072 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T16:46:27.347 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T16:46:27.363 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-13T16:46:27.375 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T16:46:27.403 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-13T16:46:27.405 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-13T16:46:27.442 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-13T16:46:27.641 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T16:46:27.658 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-13T16:46:27.691 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-13T16:46:27.694 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T16:46:27.711 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-13T16:46:27.753 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-13T16:46:27.977 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T16:46:28.006 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-13T16:46:28.031 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T16:46:28.044 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-client-common (238) ... 2024-09-13T16:46:28.058 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-13T16:46:28.122 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking postgresql-client-common (238) ... 2024-09-13T16:46:28.348 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-13T16:46:28.376 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:28.414 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:28.434 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-13T16:46:28.461 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:28.501 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:29.011 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T16:46:29.039 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-13T16:46:29.047 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T16:46:29.074 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-13T16:46:29.096 INFO:teuthology.orchestra.run.smithi039.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T16:46:29.125 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-common (238) ... 2024-09-13T16:46:29.131 INFO:teuthology.orchestra.run.smithi084.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T16:46:29.161 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking postgresql-common (238) ... 2024-09-13T16:46:29.612 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package postgresql-14. 2024-09-13T16:46:29.639 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:29.681 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:29.694 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-14. 2024-09-13T16:46:29.724 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T16:46:29.764 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:30.924 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package postgresql. 2024-09-13T16:46:30.941 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-13T16:46:30.982 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking postgresql (14+238) ... 2024-09-13T16:46:30.988 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql. 2024-09-13T16:46:31.006 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-13T16:46:31.046 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql (14+238) ... 2024-09-13T16:46:31.202 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T16:46:31.230 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-13T16:46:31.259 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking postgresql-client (14+238) ... 2024-09-13T16:46:31.290 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T16:46:31.319 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-13T16:46:31.365 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-client (14+238) ... 2024-09-13T16:46:31.478 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T16:46:31.507 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-13T16:46:31.545 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-13T16:46:31.609 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T16:46:31.638 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-13T16:46:31.676 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-13T16:46:31.889 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package tcl8.6. 2024-09-13T16:46:31.917 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-13T16:46:31.928 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package tcl8.6. 2024-09-13T16:46:31.955 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T16:46:31.957 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-13T16:46:31.986 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T16:46:32.233 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package tcl. 2024-09-13T16:46:32.255 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package tcl. 2024-09-13T16:46:32.261 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-13T16:46:32.284 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-13T16:46:32.305 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-13T16:46:32.327 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-13T16:46:32.569 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T16:46:32.597 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-13T16:46:32.627 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T16:46:32.700 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T16:46:32.729 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-13T16:46:32.766 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T16:46:32.981 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T16:46:32.997 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-13T16:46:33.039 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-13T16:46:33.121 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T16:46:33.138 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-13T16:46:33.179 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-13T16:46:33.473 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T16:46:33.502 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-13T16:46:33.534 INFO:teuthology.orchestra.run.smithi084.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T16:46:33.540 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T16:46:33.563 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-13T16:46:33.609 INFO:teuthology.orchestra.run.smithi084.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T16:46:33.808 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-client-common (238) ... 2024-09-13T16:46:33.896 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up postgresql-client-common (238) ... 2024-09-13T16:46:33.996 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-13T16:46:34.106 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-13T16:46:34.122 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T16:46:34.248 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:34.249 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T16:46:34.375 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:34.501 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T16:46:34.602 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T16:46:34.637 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T16:46:34.703 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:34.942 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T16:46:35.068 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:35.206 INFO:teuthology.orchestra.run.smithi084.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T16:46:35.592 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T16:46:35.612 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T16:46:35.698 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T16:46:36.039 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T16:46:36.039 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T16:46:36.094 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T16:46:36.094 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T16:46:36.569 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T16:46:36.601 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T16:46:36.679 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T16:46:36.727 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T16:46:36.805 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T16:46:36.853 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T16:46:36.930 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-13T16:46:36.953 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-13T16:46:37.056 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T16:46:37.071 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T16:46:37.182 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T16:46:37.197 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T16:46:37.308 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T16:46:37.323 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T16:46:37.434 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T16:46:37.449 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T16:46:38.006 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T16:46:38.079 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T16:46:38.132 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up dump (0.4b47-3) ... 2024-09-13T16:46:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up dump (0.4b47-3) ... 2024-09-13T16:46:38.225 INFO:teuthology.orchestra.run.smithi084.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T16:46:38.291 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T16:46:38.299 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T16:46:38.365 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T16:46:38.499 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T16:46:38.501 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T16:46:38.602 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-13T16:46:38.625 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-13T16:46:38.703 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-13T16:46:38.751 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-13T16:46:38.804 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T16:46:38.877 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T16:46:38.910 INFO:teuthology.orchestra.run.smithi084.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-13T16:46:38.960 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-13T16:46:38.980 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T16:46:39.020 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T16:46:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T16:46:39.121 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T16:46:39.282 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T16:46:39.297 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T16:46:39.408 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T16:46:39.423 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T16:46:39.534 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-13T16:46:39.550 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-13T16:46:39.677 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up postgresql-client (14+238) ... 2024-09-13T16:46:39.778 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-13T16:46:40.171 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-client (14+238) ... 2024-09-13T16:46:40.245 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:40.246 INFO:teuthology.orchestra.run.smithi084.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T16:46:40.246 INFO:teuthology.orchestra.run.smithi084.stdout:which means they are currently configured by ifupdown: 2024-09-13T16:46:40.246 INFO:teuthology.orchestra.run.smithi084.stdout:- enp3s0f1 2024-09-13T16:46:40.246 INFO:teuthology.orchestra.run.smithi084.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T16:46:40.246 INFO:teuthology.orchestra.run.smithi084.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T16:46:40.246 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:40.319 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-13T16:46:40.363 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T16:46:40.700 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T16:46:40.816 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:40.816 INFO:teuthology.orchestra.run.smithi039.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T16:46:40.816 INFO:teuthology.orchestra.run.smithi039.stdout:which means they are currently configured by ifupdown: 2024-09-13T16:46:40.816 INFO:teuthology.orchestra.run.smithi039.stdout:- ens1f0 2024-09-13T16:46:40.816 INFO:teuthology.orchestra.run.smithi039.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T16:46:40.817 INFO:teuthology.orchestra.run.smithi039.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T16:46:40.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:40.930 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T16:46:41.030 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T16:46:41.255 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T16:46:41.556 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T16:46:41.571 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T16:46:41.682 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T16:46:41.783 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-13T16:46:41.909 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up postgresql-common (238) ... 2024-09-13T16:46:42.097 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T16:46:42.356 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T16:46:42.581 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-13T16:46:42.759 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-common (238) ... 2024-09-13T16:46:42.838 INFO:teuthology.orchestra.run.smithi084.stdout:Adding user postgres to group ssl-cert 2024-09-13T16:46:43.039 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:43.039 INFO:teuthology.orchestra.run.smithi084.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T16:46:43.243 INFO:teuthology.orchestra.run.smithi084.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T16:46:43.243 INFO:teuthology.orchestra.run.smithi084.stdout:Removing obsolete dictionary files: 2024-09-13T16:46:43.646 INFO:teuthology.orchestra.run.smithi039.stdout:Adding user postgres to group ssl-cert 2024-09-13T16:46:43.790 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T16:46:43.882 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T16:46:44.079 INFO:teuthology.orchestra.run.smithi039.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T16:46:44.080 INFO:teuthology.orchestra.run.smithi039.stdout:Removing obsolete dictionary files: 2024-09-13T16:46:44.168 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:44.479 INFO:teuthology.orchestra.run.smithi084.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-13T16:46:44.489 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-13T16:46:44.525 INFO:teuthology.orchestra.run.smithi084.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T16:46:44.525 INFO:teuthology.orchestra.run.smithi084.stdout:This user must also own the server process. 2024-09-13T16:46:44.525 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:44.525 INFO:teuthology.orchestra.run.smithi084.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T16:46:44.525 INFO:teuthology.orchestra.run.smithi084.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T16:46:44.525 INFO:teuthology.orchestra.run.smithi084.stdout:The default text search configuration will be set to "english". 2024-09-13T16:46:44.526 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:44.526 INFO:teuthology.orchestra.run.smithi084.stdout:Data page checksums are disabled. 2024-09-13T16:46:44.526 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:44.526 INFO:teuthology.orchestra.run.smithi084.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-13T16:46:44.526 INFO:teuthology.orchestra.run.smithi084.stdout:creating subdirectories ... ok 2024-09-13T16:46:44.526 INFO:teuthology.orchestra.run.smithi084.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T16:46:44.540 INFO:teuthology.orchestra.run.smithi084.stdout:selecting default max_connections ... 100 2024-09-13T16:46:44.561 INFO:teuthology.orchestra.run.smithi084.stdout:selecting default shared_buffers ... 128MB 2024-09-13T16:46:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:selecting default time zone ... Etc/UTC 2024-09-13T16:46:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:creating configuration files ... ok 2024-09-13T16:46:44.673 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T16:46:44.688 INFO:teuthology.orchestra.run.smithi084.stdout:running bootstrap script ... ok 2024-09-13T16:46:45.077 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T16:46:45.312 INFO:teuthology.orchestra.run.smithi084.stdout:performing post-bootstrap initialization ... ok 2024-09-13T16:46:45.416 INFO:teuthology.orchestra.run.smithi039.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-13T16:46:45.426 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout:This user must also own the server process. 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout:The default text search configuration will be set to "english". 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout:Data page checksums are disabled. 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-13T16:46:45.475 INFO:teuthology.orchestra.run.smithi039.stdout:creating subdirectories ... ok 2024-09-13T16:46:45.476 INFO:teuthology.orchestra.run.smithi039.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T16:46:45.489 INFO:teuthology.orchestra.run.smithi039.stdout:selecting default max_connections ... 100 2024-09-13T16:46:45.506 INFO:teuthology.orchestra.run.smithi039.stdout:selecting default shared_buffers ... 128MB 2024-09-13T16:46:45.516 INFO:teuthology.orchestra.run.smithi039.stdout:selecting default time zone ... Etc/UTC 2024-09-13T16:46:45.517 INFO:teuthology.orchestra.run.smithi039.stdout:creating configuration files ... ok 2024-09-13T16:46:45.634 INFO:teuthology.orchestra.run.smithi039.stdout:running bootstrap script ... ok 2024-09-13T16:46:45.963 INFO:teuthology.orchestra.run.smithi084.stdout:syncing data to disk ... ok 2024-09-13T16:46:46.260 INFO:teuthology.orchestra.run.smithi039.stdout:performing post-bootstrap initialization ... ok 2024-09-13T16:46:46.584 INFO:teuthology.orchestra.run.smithi084.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T16:46:46.829 INFO:teuthology.orchestra.run.smithi039.stdout:syncing data to disk ... ok 2024-09-13T16:46:47.212 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T16:46:49.340 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T16:46:49.458 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-13T16:46:49.607 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up postgresql (14+238) ... 2024-09-13T16:46:49.735 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T16:46:49.861 INFO:teuthology.orchestra.run.smithi084.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T16:46:49.991 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T16:46:50.109 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-13T16:46:50.116 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T16:46:50.116 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T16:46:50.260 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql (14+238) ... 2024-09-13T16:46:50.386 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T16:46:50.512 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T16:46:50.688 INFO:teuthology.orchestra.run.smithi084.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T16:46:50.798 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T16:46:50.799 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T16:46:50.906 INFO:teuthology.orchestra.run.smithi084.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T16:46:51.191 INFO:teuthology.orchestra.run.smithi084.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T16:46:51.347 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T16:46:51.351 INFO:teuthology.orchestra.run.smithi084.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T16:46:51.532 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T16:46:51.868 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T16:46:52.012 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T16:46:52.629 INFO:teuthology.orchestra.run.smithi084.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-13T16:46:52.728 INFO:teuthology.orchestra.run.smithi084.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T16:46:53.170 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-13T16:46:53.270 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T16:46:53.358 INFO:teuthology.orchestra.run.smithi084.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T16:46:53.365 DEBUG:teuthology.parallel:result is None 2024-09-13T16:46:53.944 INFO:teuthology.orchestra.run.smithi039.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T16:46:53.949 DEBUG:teuthology.parallel:result is None 2024-09-13T16:46:53.949 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:46:54.095 DEBUG:teuthology.orchestra.run.smithi039:> dpkg-query -W -f '${Version}' ceph 2024-09-13T16:46:54.115 INFO:teuthology.orchestra.run.smithi039.stdout:18.2.4-850-g50bbe7f3-1jammy 2024-09-13T16:46:54.116 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-850-g50bbe7f3-1jammy 2024-09-13T16:46:54.116 INFO:teuthology.task.install:The correct ceph version 18.2.4-850-g50bbe7f3-1jammy is installed. 2024-09-13T16:46:54.118 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:46:54.267 DEBUG:teuthology.orchestra.run.smithi084:> dpkg-query -W -f '${Version}' ceph 2024-09-13T16:46:54.287 INFO:teuthology.orchestra.run.smithi084.stdout:18.2.4-850-g50bbe7f3-1jammy 2024-09-13T16:46:54.287 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-850-g50bbe7f3-1jammy 2024-09-13T16:46:54.287 INFO:teuthology.task.install:The correct ceph version 18.2.4-850-g50bbe7f3-1jammy is installed. 2024-09-13T16:46:54.288 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T16:46:54.289 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:54.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T16:46:54.303 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:46:54.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T16:46:54.343 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T16:46:54.343 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:54.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T16:46:54.359 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T16:46:54.414 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:46:54.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T16:46:54.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T16:46:54.481 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T16:46:54.481 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:54.482 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T16:46:54.495 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T16:46:54.551 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:46:54.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T16:46:54.564 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T16:46:54.617 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T16:46:54.618 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:54.618 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T16:46:54.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T16:46:54.689 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:46:54.689 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T16:46:54.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T16:46:54.758 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T16:46:54.835 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T16:46:54.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /var/log/ceph 2024-09-13T16:46:54.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /var/log/ceph 2024-09-13T16:46:54.850 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T16:46:54.850 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T16:46:54.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T16:46:54.905 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T16:46:54.905 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T16:46:54.952 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T16:46:54.970 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-13T16:46:54.970 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-13T16:46:54.971 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-13T16:46:54.971 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:54.971 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T16:46:55.011 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T16:46:55.087 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:46:55.088 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T16:46:55.093 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T16:46:55.171 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T16:46:55.172 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T16:46:55.172 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830', 'branch': 'wip-rishabh-testing-20240910.095512-reef', 'description': 'fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}}', 'email': None, 'first_in_suite': False, 'job_id': '7903830', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.095512-reef'}, 'ceph': {'cephfs': {'fs': [{'name': 'a'}, {'name': 'b'}], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'ceph-fuse': {'client.0': {'cephfs_name': 'a'}, 'client.1': {'cephfs_name': 'b'}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}}, 'kclient': {'client.0': {'cephfs_name': 'a'}, 'client.1': {'cephfs_name': 'b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.095512-reef', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 1152, 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'sleep_before_teardown': 0, 'subset': '984/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.095512-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi039.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAldJo9viHmxc+L/pgE39hKns4NRV82SNIJRD6Yzol1BeZX+walspT7a2bVrHaVZlvrpQ8cAdB+DlureXkgcnuk=', 'smithi084.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAemyebAsO1/zC3pTAUCIvemeQVaFR/9Hzmok7w7gTXMEhnJdbpZj4oerKNCOTCmRXbGcMa1uaGanA/o270Mn0I='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'fs': [{'name': 'a'}, {'name': 'b'}], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds', 'SLOW_OPS', 'slow request'], 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['suites/ffsb.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:36:41', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T16:46:55.173 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T16:46:55.175 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T16:46:55.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T16:46:55.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T16:46:55.271 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:55.271 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T16:46:55.283 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T16:46:55.283 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-09-13T16:46:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T16:46:55.331 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:46:55.331 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 861 Links: 1 2024-09-13T16:46:55.331 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:46:55.331 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-13 16:40:42.041928219 +0000 2024-09-13T16:46:55.331 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-13 16:40:41.793930434 +0000 2024-09-13T16:46:55.331 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-13 16:40:41.793930434 +0000 2024-09-13T16:46:55.332 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-09-13T16:46:55.332 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T16:46:55.385 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-13T16:46:55.385 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-13T16:46:55.386 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000520749 s, 983 kB/s 2024-09-13T16:46:55.387 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T16:46:55.436 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-09-13T16:46:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T16:46:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:46:55.483 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-09-13T16:46:55.483 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:46:55.483 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-13 16:40:42.529923859 +0000 2024-09-13T16:46:55.483 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-13 16:40:42.289926003 +0000 2024-09-13T16:46:55.483 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-13 16:40:42.289926003 +0000 2024-09-13T16:46:55.483 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-09-13T16:46:55.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T16:46:55.538 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-13T16:46:55.538 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-13T16:46:55.538 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000507952 s, 1.0 MB/s 2024-09-13T16:46:55.539 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T16:46:55.588 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-09-13T16:46:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T16:46:55.635 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:46:55.635 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 890 Links: 1 2024-09-13T16:46:55.635 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:46:55.635 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-13 16:40:43.061919108 +0000 2024-09-13T16:46:55.636 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-13 16:40:42.789921537 +0000 2024-09-13T16:46:55.636 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-13 16:40:42.789921537 +0000 2024-09-13T16:46:55.636 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-09-13T16:46:55.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T16:46:55.689 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-13T16:46:55.690 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-13T16:46:55.690 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000505696 s, 1.0 MB/s 2024-09-13T16:46:55.691 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T16:46:55.740 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-09-13T16:46:55.786 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T16:46:55.786 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:46:55.787 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 913 Links: 1 2024-09-13T16:46:55.787 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:46:55.787 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-13 16:40:43.577914498 +0000 2024-09-13T16:46:55.787 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-13 16:40:43.317916821 +0000 2024-09-13T16:46:55.787 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-13 16:40:43.317916821 +0000 2024-09-13T16:46:55.787 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-09-13T16:46:55.787 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T16:46:55.841 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-13T16:46:55.841 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-13T16:46:55.841 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000447425 s, 1.1 MB/s 2024-09-13T16:46:55.842 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T16:46:55.892 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:46:55.892 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:46:55.892 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T16:46:55.899 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T16:46:55.899 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2024-09-13T16:46:55.948 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T16:46:55.948 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:46:55.948 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-09-13T16:46:55.948 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:46:55.948 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-13 16:40:36.407061621 +0000 2024-09-13T16:46:55.948 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-13 16:40:36.139064015 +0000 2024-09-13T16:46:55.948 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-13 16:40:36.139064015 +0000 2024-09-13T16:46:55.948 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2024-09-13T16:46:55.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T16:46:56.002 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-13T16:46:56.002 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-13T16:46:56.002 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.00045395 s, 1.1 MB/s 2024-09-13T16:46:56.003 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T16:46:56.052 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2024-09-13T16:46:56.099 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T16:46:56.117 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:46:56.117 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 907 Links: 1 2024-09-13T16:46:56.117 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:46:56.117 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-13 16:40:36.883057369 +0000 2024-09-13T16:46:56.117 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-13 16:40:36.623059691 +0000 2024-09-13T16:46:56.117 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-13 16:40:36.623059691 +0000 2024-09-13T16:46:56.117 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2024-09-13T16:46:56.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T16:46:56.153 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-13T16:46:56.153 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-13T16:46:56.153 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000446817 s, 1.1 MB/s 2024-09-13T16:46:56.155 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T16:46:56.204 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2024-09-13T16:46:56.251 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T16:46:56.251 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:46:56.251 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 924 Links: 1 2024-09-13T16:46:56.251 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:46:56.251 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-13 16:40:37.103055403 +0000 2024-09-13T16:46:56.251 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-13 16:40:37.099055439 +0000 2024-09-13T16:46:56.251 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-13 16:40:37.099055439 +0000 2024-09-13T16:46:56.252 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2024-09-13T16:46:56.252 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T16:46:56.305 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-13T16:46:56.306 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-13T16:46:56.306 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000309206 s, 1.7 MB/s 2024-09-13T16:46:56.307 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T16:46:56.356 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2024-09-13T16:46:56.403 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T16:46:56.403 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T16:46:56.403 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 943 Links: 1 2024-09-13T16:46:56.403 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:46:56.403 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-13 16:40:37.831048901 +0000 2024-09-13T16:46:56.403 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-13 16:40:37.599050973 +0000 2024-09-13T16:46:56.403 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-13 16:40:37.599050973 +0000 2024-09-13T16:46:56.403 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2024-09-13T16:46:56.404 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T16:46:56.458 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-13T16:46:56.458 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-13T16:46:56.458 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.00038842 s, 1.3 MB/s 2024-09-13T16:46:56.459 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T16:46:56.508 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:46:56.509 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T16:46:56.509 INFO:tasks.ceph:Generating config... 2024-09-13T16:46:56.510 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T16:46:56.510 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-09-13T16:46:56.510 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T16:46:56.510 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T16:46:56.510 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T16:46:56.510 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T16:46:56.510 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T16:46:56.510 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T16:46:56.510 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T16:46:56.511 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T16:46:56.512 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T16:46:56.512 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T16:46:56.512 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T16:46:56.512 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T16:46:56.512 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T16:46:56.512 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T16:46:56.512 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T16:46:56.512 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T16:46:56.512 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T16:46:56.512 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T16:46:56.512 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T16:46:56.512 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] filestore flush min = 0 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T16:46:56.513 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T16:46:56.515 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T16:46:56.516 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T16:46:56.550 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T16:46:56.565 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T16:46:56.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T16:46:56.694 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.39'), ('mon.b', '172.21.15.84'), ('mon.c', '[v2:172.21.15.84:3301,v1:172.21.15.84:6790]')] 2024-09-13T16:46:56.695 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.39,172.21.15.84,[v2:172.21.15.84:3301,v1:172.21.15.84:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T16:46:56.695 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:56.696 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T16:46:56.743 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.39 --add b 172.21.15.84 --addv c '[v2:172.21.15.84:3301,v1:172.21.15.84:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:46:56.829 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:46:56.829 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: generated fsid b9195ac8-73ed-4db1-a594-cde2ff5ccb0f 2024-09-13T16:46:56.829 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = pacific 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 0 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:fsid b9195ac8-73ed-4db1-a594-cde2ff5ccb0f 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:last_changed 2024-09-13T16:46:56.830680+0000 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-09-13T16:46:56.830680+0000 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:min_mon_release 16 (pacific) 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:election_strategy: 1 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:2: [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] mon.c 2024-09-13T16:46:56.830 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T16:46:56.831 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T16:46:56.879 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b9195ac8-73ed-4db1-a594-cde2ff5ccb0f... 2024-09-13T16:46:56.880 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:46:56.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T16:46:56.954 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-09-13T16:46:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2024-09-13T16:46:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:46:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2024-09-13T16:46:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.954 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2024-09-13T16:46:56.955 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:46:56.956 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,172.21.15.84,[v2:172.21.15.84:3301,v1:172.21.15.84:6790]" 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = b9195ac8-73ed-4db1-a594-cde2ff5ccb0f 2024-09-13T16:46:56.957 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.958 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2024-09-13T16:46:56.959 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: bdev async discard = True 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: bdev enable discard = True 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore allocator = bitmap 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: filestore flush min = 0 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:46:56.960 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: osd blocked scrub grace period = 3600 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: osd max markdown count = 1000 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: osd op complaint time = 180 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:46:56.961 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: debug client = 20 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:46:56.962 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: mon op complaint time = 120 2024-09-13T16:46:56.963 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on pool no app = False 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: client mount timeout = 600 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: client_shutdown_timeout = 120 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: debug client = 20 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: rados mon op timeout = 900 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout: rados osd op timeout = 900 2024-09-13T16:46:56.964 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout:[mds] 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: debug mds = 20 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: debug mds balancer = 20 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: mds bal fragment size max = 10000 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: mds bal merge size = 5 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: mds bal split bits = 3 2024-09-13T16:46:56.965 INFO:teuthology.orchestra.run.smithi039.stdout: mds bal split size = 100 2024-09-13T16:46:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: mds debug frag = True 2024-09-13T16:46:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: mds debug scatterstat = True 2024-09-13T16:46:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: mds op complaint time = 180 2024-09-13T16:46:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: mds verify scatter = True 2024-09-13T16:46:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: osd op complaint time = 180 2024-09-13T16:46:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: rados mon op timeout = 900 2024-09-13T16:46:56.966 INFO:teuthology.orchestra.run.smithi039.stdout: rados osd op timeout = 900 2024-09-13T16:46:56.967 INFO:teuthology.orchestra.run.smithi084.stdout:[global] 2024-09-13T16:46:56.967 INFO:teuthology.orchestra.run.smithi084.stdout: chdir = "" 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: auth supported = cephx 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: filestore xattr use omap = true 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: mon clock drift allowed = 1.000 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: osd crush chooseleaf type = 0 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: auth debug = true 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on old message = true 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on bug = true 2024-09-13T16:46:56.968 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: mon pg warn max object skew = 0 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default size = 2 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow primary affinity = true 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow pg remap = true 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on legacy crush tunables = false 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on crush straw calc version zero = false 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on no sortbitwise = false 2024-09-13T16:46:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on osd down out interval zero = false 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on too few osds = false 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: mon_allow_pool_size_one = true 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: osd default data pool replay window = 5 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: mon allow pool delete = true 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: mon cluster log file level = debug 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: debug asserts on shutdown = true 2024-09-13T16:46:56.970 INFO:teuthology.orchestra.run.smithi084.stdout: mon health detail to clog = false 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: mon host = "172.21.15.39,172.21.15.84,[v2:172.21.15.84:3301,v1:172.21.15.84:6790]" 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: fsid = b9195ac8-73ed-4db1-a594-cde2ff5ccb0f 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout:[osd] 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: osd journal size = 100 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub load threshold = 5.0 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub max interval = 600 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: osd mclock profile = high_recovery_ops 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: osd recover clone overlap = true 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: osd recovery max chunk = 1048576 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug shutdown = true 2024-09-13T16:46:56.971 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug op order = true 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug verify stray on activate = true 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: osd open classes on start = true 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug pg log writeout = true 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: osd deep scrub update digest min age = 30 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: osd map max advance = 10 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: journal zero on create = true 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: filestore ondisk finisher threads = 3 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: filestore apply finisher threads = 3 2024-09-13T16:46:56.972 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: bdev debug aio = true 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug misdirected ops = true 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: bdev async discard = True 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: bdev enable discard = True 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore allocator = bitmap 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore block size = 96636764160 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore fsck on mount = True 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluefs = 1/20 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluestore = 1/20 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: debug osd = 20 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: debug rocksdb = 4/10 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: filestore flush min = 0 2024-09-13T16:46:56.973 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd full ratio = 0.9 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd nearfull ratio = 0.8 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: osd blocked scrub grace period = 3600 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: osd failsafe full ratio = 0.95 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: osd max markdown count = 1000 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: osd objectstore = bluestore 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr] 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: debug mgr = 20 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2024-09-13T16:46:56.974 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: mgr/telemetry/nag = false 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: debug client = 20 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout:[mon] 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: debug paxos = 20 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: mon data avail warn = 5 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: mon mgr mkfs grace = 240 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2024-09-13T16:46:56.975 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd reporter subtree level = osd 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd prime pg temp = true 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: # 1m isn't quite enough 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: mon_down_mkfs_grace = 2m 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.976 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_filestore_osds = false 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: mon op complaint time = 120 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on pool no app = False 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout:[client] 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: rgw cache enabled = true 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable ops log = true 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable usage log = true 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: client mount timeout = 600 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: client_shutdown_timeout = 120 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: debug client = 20 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2024-09-13T16:46:56.977 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.a] 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.b] 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.c] 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout:[mds] 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds = 20 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds balancer = 20 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout: mds bal fragment size max = 10000 2024-09-13T16:46:56.978 INFO:teuthology.orchestra.run.smithi084.stdout: mds bal merge size = 5 2024-09-13T16:46:56.979 INFO:teuthology.orchestra.run.smithi084.stdout: mds bal split bits = 3 2024-09-13T16:46:56.979 INFO:teuthology.orchestra.run.smithi084.stdout: mds bal split size = 100 2024-09-13T16:46:56.979 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug frag = True 2024-09-13T16:46:56.979 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug scatterstat = True 2024-09-13T16:46:56.979 INFO:teuthology.orchestra.run.smithi084.stdout: mds op complaint time = 180 2024-09-13T16:46:56.979 INFO:teuthology.orchestra.run.smithi084.stdout: mds verify scatter = True 2024-09-13T16:46:56.979 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2024-09-13T16:46:56.979 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2024-09-13T16:46:56.979 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2024-09-13T16:46:56.979 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T16:46:56.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T16:46:57.060 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T16:46:57.060 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:57.060 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T16:46:57.065 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:57.066 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T16:46:57.110 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T16:46:57.110 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:57.110 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T16:46:57.110 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T16:46:57.176 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:46:57.177 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:46:57.227 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T16:46:57.227 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:46:57.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T16:46:57.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T16:46:57.250 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:46:57.250 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:46:57.299 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T16:46:57.299 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T16:46:57.300 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T16:46:57.346 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T16:46:57.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T16:46:57.392 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T16:46:57.407 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T16:46:57.407 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T16:46:57.449 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T16:46:57.452 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-13T16:46:57.507 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T16:46:57.593 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T16:46:57.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-13T16:46:57.652 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T16:46:57.697 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T16:46:57.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-13T16:46:57.755 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T16:46:57.836 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T16:46:57.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-13T16:46:57.895 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T16:46:57.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T16:46:57.934 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T16:46:57.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T16:46:57.979 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T16:46:57.990 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T16:46:57.990 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T16:46:57.990 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T16:46:58.003 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:46:58.003 INFO:tasks.ceph:role: osd.0 2024-09-13T16:46:58.003 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T16:46:58.003 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T16:46:58.087 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:46:58.087 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:46:58.087 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:46:58.087 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T16:46:58.087 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:46:58.087 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:46:58.087 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:46:58.088 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:46:58.088 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:46:58.088 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:46:59.380 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-13T16:46:59.383 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-13T16:46:59.383 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T16:46:59.445 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T16:46:59.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:46:59.497 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:46:59.497 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T16:46:59.549 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:46:59.549 INFO:tasks.ceph:role: osd.1 2024-09-13T16:46:59.549 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T16:46:59.550 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:46:59.630 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:47:00.919 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-13T16:47:00.921 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-13T16:47:00.921 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T16:47:00.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T16:47:01.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:47:01.037 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:47:01.037 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T16:47:01.090 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:47:01.090 INFO:tasks.ceph:role: osd.2 2024-09-13T16:47:01.090 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T16:47:01.091 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T16:47:01.174 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:47:01.174 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:47:01.174 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:47:01.174 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T16:47:01.174 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:47:01.174 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:47:01.174 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:47:01.174 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:47:01.174 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:47:01.175 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:47:02.459 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-13T16:47:02.461 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-13T16:47:02.462 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T16:47:02.526 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T16:47:02.583 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:47:02.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:47:02.584 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T16:47:02.638 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T16:47:02.638 INFO:tasks.ceph:role: osd.3 2024-09-13T16:47:02.638 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T16:47:02.639 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T16:47:02.723 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:47:02.723 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:47:02.723 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:47:02.723 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T16:47:02.723 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:47:02.723 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:47:02.723 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:47:02.723 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:47:02.724 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:47:02.724 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:47:04.024 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-13T16:47:04.027 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-13T16:47:04.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T16:47:04.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T16:47:04.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:47:04.143 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:47:04.143 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:04.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:04.266+0000 7ffa7b4067c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T16:47:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:04.266+0000 7ffa7b4067c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T16:47:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:04.266+0000 7ffa7b4067c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T16:47:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:04.266+0000 7ffa7b4067c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T16:47:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:04.266+0000 7ffa7b4067c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T16:47:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:04.266+0000 7ffa7b4067c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T16:47:07.266 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T16:47:07.281 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:07.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:07.358+0000 7f99fb47f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T16:47:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:07.362+0000 7f99fb47f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T16:47:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:07.362+0000 7f99fb47f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T16:47:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:07.362+0000 7f99fb47f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T16:47:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:07.362+0000 7f99fb47f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T16:47:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:07.362+0000 7f99fb47f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T16:47:10.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T16:47:10.396 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:10.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:10.482+0000 7f6e70a7f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T16:47:10.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:10.482+0000 7f6e70a7f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T16:47:10.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:10.482+0000 7f6e70a7f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T16:47:10.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:10.482+0000 7f6e70a7f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T16:47:10.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:10.482+0000 7f6e70a7f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T16:47:10.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:10.482+0000 7f6e70a7f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T16:47:13.491 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T16:47:13.547 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:13.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:13.630+0000 7febb295d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T16:47:13.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:13.630+0000 7febb295d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T16:47:13.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:13.630+0000 7febb295d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T16:47:13.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:13.630+0000 7febb295d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T16:47:13.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:13.630+0000 7febb295d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T16:47:13.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:13.630+0000 7febb295d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T16:47:16.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T16:47:16.663 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T16:47:16.678 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:47:16.679 INFO:tasks.ceph:role: osd.4 2024-09-13T16:47:16.679 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T16:47:16.679 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:47:16.743 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:47:17.219 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2024-09-13T16:47:17.221 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2024-09-13T16:47:17.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T16:47:17.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T16:47:17.359 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:47:17.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:47:17.360 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T16:47:17.414 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:47:17.414 INFO:tasks.ceph:role: osd.5 2024-09-13T16:47:17.414 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T16:47:17.414 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T16:47:17.474 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:47:17.475 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:47:17.475 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:47:17.475 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T16:47:17.475 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:47:17.475 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:47:17.475 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:47:17.475 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:47:17.475 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:47:17.475 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:47:17.912 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2024-09-13T16:47:17.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2024-09-13T16:47:17.915 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T16:47:17.978 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T16:47:18.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:47:18.032 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:47:18.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T16:47:18.085 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:47:18.085 INFO:tasks.ceph:role: osd.6 2024-09-13T16:47:18.085 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T16:47:18.085 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T16:47:18.145 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:47:18.146 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:47:18.146 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:47:18.146 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T16:47:18.146 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:47:18.146 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:47:18.146 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:47:18.146 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:47:18.146 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:47:18.146 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:47:18.596 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2024-09-13T16:47:18.598 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2024-09-13T16:47:18.598 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T16:47:18.682 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T16:47:18.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:47:18.732 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:47:18.732 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T16:47:18.787 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T16:47:18.787 INFO:tasks.ceph:role: osd.7 2024-09-13T16:47:18.787 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T16:47:18.787 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T16:47:18.856 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T16:47:18.856 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T16:47:18.856 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T16:47:18.856 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T16:47:18.856 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T16:47:18.856 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2024-09-13T16:47:18.856 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T16:47:18.856 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T16:47:18.857 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T16:47:18.857 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T16:47:19.321 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2024-09-13T16:47:19.323 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2024-09-13T16:47:19.323 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T16:47:19.455 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T16:47:19.467 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T16:47:19.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:47:19.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:19.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:19.572+0000 7f34bc58c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T16:47:19.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:19.580+0000 7f34bc58c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T16:47:19.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:19.580+0000 7f34bc58c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T16:47:19.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:19.580+0000 7f34bc58c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T16:47:19.583 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:19.580+0000 7f34bc58c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T16:47:19.588 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:19.584+0000 7f34bc58c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T16:47:22.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T16:47:22.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:22.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:22.660+0000 7fd80aff17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T16:47:22.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:22.660+0000 7fd80aff17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T16:47:22.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:22.660+0000 7fd80aff17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T16:47:22.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:22.660+0000 7fd80aff17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T16:47:22.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:22.660+0000 7fd80aff17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T16:47:22.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:22.660+0000 7fd80aff17c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T16:47:25.174 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:47:25.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:47:25.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T16:47:25.657 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:25.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:25.732+0000 7f373d3737c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T16:47:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:25.732+0000 7f373d3737c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T16:47:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:25.732+0000 7f373d3737c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T16:47:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:25.732+0000 7f373d3737c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T16:47:25.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:25.732+0000 7f373d3737c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T16:47:25.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:25.732+0000 7f373d3737c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T16:47:28.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T16:47:28.772 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:28.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:28.852+0000 7fbe622b37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T16:47:28.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:28.852+0000 7fbe622b37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T16:47:28.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:28.852+0000 7fbe622b37c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T16:47:28.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:28.852+0000 7fbe622b37c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T16:47:28.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:28.852+0000 7fbe622b37c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T16:47:28.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:28.852+0000 7fbe622b37c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T16:47:31.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T16:47:31.884 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T16:47:31.884 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:31.884 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T16:47:31.899 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:31.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-13T16:47:31.957 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:31.958 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-13T16:47:32.013 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:32.014 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T16:47:32.070 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:32.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T16:47:32.129 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:32.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T16:47:32.185 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:32.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T16:47:32.243 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:32.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T16:47:32.256 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:32.256 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-13T16:47:32.311 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:32.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-13T16:47:32.367 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:32.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T16:47:32.422 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:32.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T16:47:32.478 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:32.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T16:47:32.534 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:32.534 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T16:47:32.589 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:32.589 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T16:47:32.595 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:32.595 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T16:47:32.636 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T16:47:32.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T16:47:32.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T16:47:32.681 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.y] 2024-09-13T16:47:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCBbORmwX2wFBAADyM0iuLusC3BYbp60JuZ+g== 2024-09-13T16:47:32.681 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2024-09-13T16:47:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCBbORmwh7bGhAALNkSpxy8Iq4iFD/2FYHtOA== 2024-09-13T16:47:32.681 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2024-09-13T16:47:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCBbORm4lJuIxAAKCgcg7E2WQD3Ybk02Hnuvw== 2024-09-13T16:47:32.681 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.0] 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCIbORmEPH7DxAAnPvHDypS0PoUb2lFUx9ocQ== 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.1] 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCLbORmmpSoFRAAwYek2S0BNWt8OSshIxXyIQ== 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.2] 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCObORmovXjHBAAUPAyWg0411wpUu/5IXWisQ== 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.3] 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCRbORmUTykJRAA6hwbvWm9dJd2LZJ1j8asaA== 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.x] 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCBbORmzaxuFxAAGdrPZKYsJpjD3IjKIe+9OA== 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCBbORmK+KiKRAAcWPkEVOjZ2ylcvzpkyQoiQ== 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCBbORmC/znMRAAAJxqISmDO3dyLavNlVAefg== 2024-09-13T16:47:32.682 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.4] 2024-09-13T16:47:32.683 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCXbORm7HB2IhAAbnID2D0tv+bRnqVUe+sWHw== 2024-09-13T16:47:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.5] 2024-09-13T16:47:32.683 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCabORmHcqkJxAAz1CPeCSG6RNIGuYqKqawAw== 2024-09-13T16:47:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.6] 2024-09-13T16:47:32.683 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCdbORmmifuKxAA19xnw1svlizMZvRs0t2qfQ== 2024-09-13T16:47:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.7] 2024-09-13T16:47:32.683 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCgbORmgp4hMxAA1cETX6RU00uXhfj5MIZYuA== 2024-09-13T16:47:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-13T16:47:32.683 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCBbORmn3i+NxAAzameGqyN430kC23LSuO4/Q== 2024-09-13T16:47:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:[client.1] 2024-09-13T16:47:32.684 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCBbORmyHtsOhAA9WQ8jUPYIh/su6DGi4RuHQ== 2024-09-13T16:47:32.689 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.y] 2024-09-13T16:47:32.689 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCBbORmwX2wFBAADyM0iuLusC3BYbp60JuZ+g== 2024-09-13T16:47:32.689 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2024-09-13T16:47:32.689 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCBbORmwh7bGhAALNkSpxy8Iq4iFD/2FYHtOA== 2024-09-13T16:47:32.689 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2024-09-13T16:47:32.689 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCBbORm4lJuIxAAKCgcg7E2WQD3Ybk02Hnuvw== 2024-09-13T16:47:32.689 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.0] 2024-09-13T16:47:32.689 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCIbORmEPH7DxAAnPvHDypS0PoUb2lFUx9ocQ== 2024-09-13T16:47:32.689 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.1] 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCLbORmmpSoFRAAwYek2S0BNWt8OSshIxXyIQ== 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.2] 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCObORmovXjHBAAUPAyWg0411wpUu/5IXWisQ== 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.3] 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCRbORmUTykJRAA6hwbvWm9dJd2LZJ1j8asaA== 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.x] 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCBbORmzaxuFxAAGdrPZKYsJpjD3IjKIe+9OA== 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCBbORmK+KiKRAAcWPkEVOjZ2ylcvzpkyQoiQ== 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCBbORmC/znMRAAAJxqISmDO3dyLavNlVAefg== 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.4] 2024-09-13T16:47:32.690 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCXbORm7HB2IhAAbnID2D0tv+bRnqVUe+sWHw== 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.5] 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCabORmHcqkJxAAz1CPeCSG6RNIGuYqKqawAw== 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.6] 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCdbORmmifuKxAA19xnw1svlizMZvRs0t2qfQ== 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.7] 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCgbORmgp4hMxAA1cETX6RU00uXhfj5MIZYuA== 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCBbORmn3i+NxAAzameGqyN430kC23LSuO4/Q== 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout:[client.1] 2024-09-13T16:47:32.691 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCBbORmyHtsOhAA9WQ8jUPYIh/su6DGi4RuHQ== 2024-09-13T16:47:32.692 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T16:47:32.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T16:47:32.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:47:32.820 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:47:32.869 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:47:32.872 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:47:32.914 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:32.916 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:32.992 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.003 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.073 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.163 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T16:47:33.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T16:47:33.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:47:33.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:47:33.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:47:33.386 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T16:47:33.468 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.471 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.552 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.555 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.720 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T16:47:33.806 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:47:33.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:47:33.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:47:33.897 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T16:47:33.979 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T16:47:33.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T16:47:33.997 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T16:47:34.094 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T16:47:34.105 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T16:47:34.119 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T16:47:34.213 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T16:47:34.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T16:47:34.281 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T16:47:34.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T16:47:34.386 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:34.389 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T16:47:34.434 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T16:47:34.435 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T16:47:34.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T16:47:34.437 INFO:tasks.ceph.mon.a:Started 2024-09-13T16:47:34.437 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T16:47:34.437 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T16:47:34.480 INFO:tasks.ceph.mon.b:Started 2024-09-13T16:47:34.480 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T16:47:34.480 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T16:47:34.482 INFO:tasks.ceph.mon.c:Started 2024-09-13T16:47:34.482 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T16:47:34.483 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T16:47:34.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T16:47:34.485 INFO:tasks.ceph.mgr.y:Started 2024-09-13T16:47:34.485 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T16:47:34.486 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T16:47:34.488 INFO:tasks.ceph.mgr.x:Started 2024-09-13T16:47:34.488 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T16:47:34.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T16:47:34.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b44a4640 1 Processor -- start 2024-09-13T16:47:34.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b44a4640 1 -- start start 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b44a4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 0x7f58ac0c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b44a4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 0x7f58ac151c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b44a4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac0c4730 con 0x7f58ac0c4e80 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b44a4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac0c0ba0 con 0x7f58ac14f5a0 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b44a4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac0c0e60 con 0x7f58ac0c58f0 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b2240640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 0x7f58ac0c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b2240640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 0x7f58ac0c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45106/0 (socket says 172.21.15.39:45106) 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b2240640 1 -- 172.21.15.39:0/2873892381 learned_addr learned my addr 172.21.15.39:0/2873892381 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b2240640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 msgr2=0x7f58ac0c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T16:47:34.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b2240640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 msgr2=0x7f58ac0c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T16:47:34.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b2240640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 0x7f58ac0c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T16:47:34.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b2240640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 0x7f58ac0c53b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:47:34.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b1a3f640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 0x7f58ac151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b1a3f640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 msgr2=0x7f58ac151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T16:47:34.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b1a3f640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 msgr2=0x7f58ac151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T16:47:34.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b1a3f640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 0x7f58ac151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T16:47:34.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.610+0000 7f58b1a3f640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 0x7f58ac151c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T16:47:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b2240640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 0x7f58ac0c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b1a3f640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 0x7f58ac151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b1a3f640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 msgr2=0x7f58ac151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T16:47:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b1a3f640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 msgr2=0x7f58ac151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T16:47:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b1a3f640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 0x7f58ac151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T16:47:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b1a3f640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 0x7f58ac151c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T16:47:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b2240640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 msgr2=0x7f58ac0c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T16:47:34.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b2240640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 msgr2=0x7f58ac0c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T16:47:34.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b2240640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 0x7f58ac0c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T16:47:34.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.814+0000 7f58b2240640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 0x7f58ac0c53b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T16:47:34.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.850+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154321929 0 0) 0x7f58ac0c4730 con 0x7f58ac0c4e80 2024-09-13T16:47:34.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.850+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5894003530 con 0x7f58ac0c4e80 2024-09-13T16:47:34.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1195370680 0 0) 0x7f5894003530 con 0x7f58ac0c4e80 2024-09-13T16:47:34.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 msgr2=0x7f58ac151c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T16:47:34.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0c58f0 0x7f58ac151c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 msgr2=0x7f58ac0c53b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 --2- 172.21.15.39:0/2873892381 >> v2:172.21.15.39:3300/0 conn(0x7f58ac14f5a0 0x7f58ac0c53b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac155c10 con 0x7f58ac0c4e80 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f589c001a00 con 0x7f58ac0c4e80 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 >> v1:172.21.15.84:6789/0 conn(0x7f58ac0c4e80 legacy=0x7f58ac069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58940039d0 0x7f5894003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58940047d0 0x7f5894004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5894005190 0x7f589400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0c0ba0 con 0x7f58940047d0 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0c0e60 con 0x7f5894005190 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589400c0c0 con 0x7f58940039d0 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b2240640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58940039d0 0x7f5894003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b1a3f640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58940047d0 0x7f5894004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b2a41640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5894005190 0x7f589400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b2240640 1 -- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5894005190 msgr2=0x7f589400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:34.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b2240640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5894005190 0x7f589400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b2240640 1 -- 172.21.15.39:0/2873892381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58940047d0 msgr2=0x7f5894004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:34.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b2240640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58940047d0 0x7f5894004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b2240640 1 -- 172.21.15.39:0/2873892381 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5894003530 con 0x7f58940039d0 2024-09-13T16:47:34.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b2a41640 1 -- 172.21.15.39:0/2873892381 reap_dead start 2024-09-13T16:47:34.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.854+0000 7f58b1a3f640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58940047d0 0x7f5894004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:34.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.858+0000 7f58b2240640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58940039d0 0x7f5894003df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f58a400e900 tx=0x7f58a401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f424e30572fca03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:34.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40a5850 con 0x7f58940039d0 2024-09-13T16:47:34.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a40a9070 con 0x7f58940039d0 2024-09-13T16:47:34.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b123e640 1 -- 172.21.15.39:0/2873892381 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40a5c30 con 0x7f58940039d0 2024-09-13T16:47:34.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b44a4640 1 -- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58940039d0 msgr2=0x7f5894003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:34.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b44a4640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58940039d0 0x7f5894003df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f58a400e900 tx=0x7f58a401c950 comp rx=0 tx=0).stop 2024-09-13T16:47:34.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b44a4640 1 -- 172.21.15.39:0/2873892381 shutdown_connections 2024-09-13T16:47:34.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b44a4640 1 --2- 172.21.15.39:0/2873892381 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58940039d0 0x7f5894003df0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b44a4640 1 -- 172.21.15.39:0/2873892381 >> 172.21.15.39:0/2873892381 conn(0x7f58ac0c32a0 msgr2=0x7f58ac0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:34.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b44a4640 1 -- 172.21.15.39:0/2873892381 shutdown_connections 2024-09-13T16:47:34.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.862+0000 7f58b44a4640 1 -- 172.21.15.39:0/2873892381 wait complete. 2024-09-13T16:47:34.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b44a4640 1 Processor -- start 2024-09-13T16:47:34.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b44a4640 1 -- start start 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b44a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58940039d0 0x7f58ac14ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b44a4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac070320 0x7f58ac1de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b44a4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac14b6b0 0x7f58ac1ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b44a4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac148d60 con 0x7f58ac14b6b0 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b44a4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac1e9e40 con 0x7f58940039d0 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b44a4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0720f0 con 0x7f58ac070320 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b2a41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac14b6b0 0x7f58ac1ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b2a41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac14b6b0 0x7f58ac1ea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45144/0 (socket says 172.21.15.39:45144) 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b2a41640 1 -- 172.21.15.39:0/3121888722 learned_addr learned my addr 172.21.15.39:0/3121888722 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b2240640 1 --2- 172.21.15.39:0/3121888722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58940039d0 0x7f58ac14ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.866+0000 7f58b1a3f640 1 --2- 172.21.15.39:0/3121888722 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac070320 0x7f58ac1de7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58b2a41640 1 -- 172.21.15.39:0/3121888722 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac070320 msgr2=0x7f58ac1de7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58b2a41640 1 --2- 172.21.15.39:0/3121888722 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac070320 0x7f58ac1de7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58b2a41640 1 -- 172.21.15.39:0/3121888722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58940039d0 msgr2=0x7f58ac14ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58b2a41640 1 --2- 172.21.15.39:0/3121888722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58940039d0 0x7f58ac14ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58b2a41640 1 -- 172.21.15.39:0/3121888722 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac14b280 con 0x7f58ac14b6b0 2024-09-13T16:47:34.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58b2a41640 1 --2- 172.21.15.39:0/3121888722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac14b6b0 0x7f58ac1ea2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f58ac072730 tx=0x7f589c00fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58aaffd640 1 -- 172.21.15.39:0/3121888722 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c006e30 con 0x7f58ac14b6b0 2024-09-13T16:47:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58aaffd640 1 -- 172.21.15.39:0/3121888722 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c003980 con 0x7f58ac14b6b0 2024-09-13T16:47:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58aaffd640 1 -- 172.21.15.39:0/3121888722 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c003d30 con 0x7f58ac14b6b0 2024-09-13T16:47:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58b44a4640 1 -- 172.21.15.39:0/3121888722 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac148a80 con 0x7f58ac14b6b0 2024-09-13T16:47:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58b44a4640 1 -- 172.21.15.39:0/3121888722 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac1decf0 con 0x7f58ac14b6b0 2024-09-13T16:47:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58aaffd640 1 -- 172.21.15.39:0/3121888722 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f589c00daa0 con 0x7f58ac14b6b0 2024-09-13T16:47:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58aaffd640 1 -- 172.21.15.39:0/3121888722 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f589c005620 con 0x7f58ac14b6b0 2024-09-13T16:47:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.870+0000 7f58b44a4640 1 -- 172.21.15.39:0/3121888722 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac071c70 con 0x7f58ac14b6b0 2024-09-13T16:47:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.874+0000 7f58aaffd640 1 -- 172.21.15.39:0/3121888722 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f58ac071c70 con 0x7f58ac14b6b0 2024-09-13T16:47:34.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.938+0000 7f58b44a4640 1 -- 172.21.15.39:0/3121888722 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f58ac072230 con 0x7f58ac14b6b0 2024-09-13T16:47:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58aaffd640 1 -- 172.21.15.39:0/3121888722 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f58ac072230 con 0x7f58ac14b6b0 2024-09-13T16:47:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-09-13T16:47:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58b44a4640 1 -- 172.21.15.39:0/3121888722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac14b6b0 msgr2=0x7f58ac1ea2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58b44a4640 1 --2- 172.21.15.39:0/3121888722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac14b6b0 0x7f58ac1ea2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f58ac072730 tx=0x7f589c00fee0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58b44a4640 1 -- 172.21.15.39:0/3121888722 shutdown_connections 2024-09-13T16:47:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58b44a4640 1 --2- 172.21.15.39:0/3121888722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac14b6b0 0x7f58ac1ea2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58b44a4640 1 --2- 172.21.15.39:0/3121888722 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac070320 0x7f58ac1de7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58b44a4640 1 --2- 172.21.15.39:0/3121888722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58940039d0 0x7f58ac14ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58b44a4640 1 -- 172.21.15.39:0/3121888722 >> 172.21.15.39:0/3121888722 conn(0x7f58ac0c32a0 msgr2=0x7f58ac0bfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58b44a4640 1 -- 172.21.15.39:0/3121888722 shutdown_connections 2024-09-13T16:47:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:34.950+0000 7f58b44a4640 1 -- 172.21.15.39:0/3121888722 wait complete. 2024-09-13T16:47:34.971 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T16:47:34.971 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T16:47:34.971 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:34.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T16:47:34.983 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:34.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T16:47:35.040 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:35.038+0000 7f4574214180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T16:47:35.041 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:35.041 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T16:47:35.094 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:47:35.094 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T16:47:35.149 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:35.149 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T16:47:35.164 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:35.164 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T16:47:35.216 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:35.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T16:47:35.269 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:47:35.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T16:47:35.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3 0 2024-09-13T16:47:35.400 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:35.396+0000 7f79ad3ae180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T16:47:35.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3e670640 1 Processor -- start 2024-09-13T16:47:35.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3e670640 1 -- start start 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3e670640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0c3806b1f0 0x7f0c38069e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3e670640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0c38150b90 0x7f0c38150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3e670640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c380c3e70 con 0x7f0c380c1b80 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3e670640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c380c2f30 con 0x7f0c3806b1f0 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3e670640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c380c1980 con 0x7f0c38150b90 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3cc0d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0c38150b90 0x7f0c38150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3cc0d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0c38150b90 0x7f0c38150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39040/0 (socket says 172.21.15.84:39040) 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c3cc0d640 1 -- 172.21.15.84:0/4212998228 learned_addr learned my addr 172.21.15.84:0/4212998228 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.488+0000 7f0c37fff640 1 --2- 172.21.15.84:0/4212998228 >> v2:172.21.15.39:3300/0 conn(0x7f0c3806b1f0 0x7f0c38069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3cc0d640 1 -- 172.21.15.84:0/4212998228 >> v2:172.21.15.39:3300/0 conn(0x7f0c3806b1f0 msgr2=0x7f0c38069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:35.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3cc0d640 1 --2- 172.21.15.84:0/4212998228 >> v2:172.21.15.39:3300/0 conn(0x7f0c3806b1f0 0x7f0c38069e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3cc0d640 1 -- 172.21.15.84:0/4212998228 >> v1:172.21.15.39:6789/0 conn(0x7f0c380c1b80 legacy=0x7f0c3806a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3cc0d640 1 -- 172.21.15.84:0/4212998228 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c380c11a0 con 0x7f0c38150b90 2024-09-13T16:47:35.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3cc0d640 1 --2- 172.21.15.84:0/4212998228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c38150b90 0x7f0c38150fe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0c3806e5f0 tx=0x7f0c2801e670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6baef13e26062c9f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:35.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c36ffd640 1 -- 172.21.15.84:0/4212998228 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c280ac5f0 con 0x7f0c38150b90 2024-09-13T16:47:35.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c36ffd640 1 -- 172.21.15.84:0/4212998228 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c280ac750 con 0x7f0c38150b90 2024-09-13T16:47:35.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c36ffd640 1 -- 172.21.15.84:0/4212998228 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c280acae0 con 0x7f0c38150b90 2024-09-13T16:47:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3e670640 1 -- 172.21.15.84:0/4212998228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c38150b90 msgr2=0x7f0c38150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3e670640 1 --2- 172.21.15.84:0/4212998228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c38150b90 0x7f0c38150fe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0c3806e5f0 tx=0x7f0c2801e670 comp rx=0 tx=0).stop 2024-09-13T16:47:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3e670640 1 -- 172.21.15.84:0/4212998228 shutdown_connections 2024-09-13T16:47:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3e670640 1 --2- 172.21.15.84:0/4212998228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c38150b90 0x7f0c38150fe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3e670640 1 --2- 172.21.15.84:0/4212998228 >> v2:172.21.15.39:3300/0 conn(0x7f0c3806b1f0 0x7f0c38069e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3e670640 1 -- 172.21.15.84:0/4212998228 >> 172.21.15.84:0/4212998228 conn(0x7f0c380c32a0 msgr2=0x7f0c380c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3e670640 1 -- 172.21.15.84:0/4212998228 shutdown_connections 2024-09-13T16:47:35.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.492+0000 7f0c3e670640 1 -- 172.21.15.84:0/4212998228 wait complete. 2024-09-13T16:47:35.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 Processor -- start 2024-09-13T16:47:35.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 -- start start 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3806b1f0 0x7f0c3814bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0c380c1b80 0x7f0c380bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38150b90 0x7f0c381eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c381ed2a0 con 0x7f0c38150b90 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3814e2d0 con 0x7f0c3806b1f0 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3814dff0 con 0x7f0c380c1b80 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3806b1f0 0x7f0c3814bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c377fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0c380c1b80 0x7f0c380bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c37fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3806b1f0 0x7f0c3814bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39054/0 (socket says 172.21.15.84:39054) 2024-09-13T16:47:35.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c37fff640 1 -- 172.21.15.84:0/2546500725 learned_addr learned my addr 172.21.15.84:0/2546500725 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3cc0d640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38150b90 0x7f0c381eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c37fff640 1 -- 172.21.15.84:0/2546500725 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0c380c1b80 msgr2=0x7f0c380bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c37fff640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0c380c1b80 0x7f0c380bfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c37fff640 1 -- 172.21.15.84:0/2546500725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38150b90 msgr2=0x7f0c381eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c37fff640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38150b90 0x7f0c381eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c37fff640 1 -- 172.21.15.84:0/2546500725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c280094d0 con 0x7f0c3806b1f0 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3cc0d640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38150b90 0x7f0c381eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c377fe640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0c380c1b80 0x7f0c380bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c37fff640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3806b1f0 0x7f0c3814bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009db0 tx=0x7f0c3000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c34ff9640 1 -- 172.21.15.84:0/2546500725 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30014480 con 0x7f0c3806b1f0 2024-09-13T16:47:35.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c34ff9640 1 -- 172.21.15.84:0/2546500725 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3000f040 con 0x7f0c3806b1f0 2024-09-13T16:47:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 -- 172.21.15.84:0/2546500725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c380ab940 con 0x7f0c3806b1f0 2024-09-13T16:47:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c34ff9640 1 -- 172.21.15.84:0/2546500725 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30014800 con 0x7f0c3806b1f0 2024-09-13T16:47:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 -- 172.21.15.84:0/2546500725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c381ed770 con 0x7f0c3806b1f0 2024-09-13T16:47:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c34ff9640 1 -- 172.21.15.84:0/2546500725 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0c30014c60 con 0x7f0c3806b1f0 2024-09-13T16:47:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c34ff9640 1 -- 172.21.15.84:0/2546500725 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0c3001ddb0 con 0x7f0c3806b1f0 2024-09-13T16:47:35.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.496+0000 7f0c3e670640 1 -- 172.21.15.84:0/2546500725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c380abb30 con 0x7f0c3806b1f0 2024-09-13T16:47:35.506 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:35.506+0000 7f4574214180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T16:47:35.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.508+0000 7f0c34ff9640 1 -- 172.21.15.84:0/2546500725 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f0c3003be30 con 0x7f0c3806b1f0 2024-09-13T16:47:35.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.580+0000 7f0c3e670640 1 -- 172.21.15.84:0/2546500725 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3", "id": 0} v 0) v1 -- 0x7f0c38066820 con 0x7f0c3806b1f0 2024-09-13T16:47:35.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.588+0000 7f0c34ff9640 1 -- 172.21.15.84:0/2546500725 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0c3002a030 con 0x7f0c3806b1f0 2024-09-13T16:47:35.593 INFO:teuthology.orchestra.run.smithi084.stdout:0 2024-09-13T16:47:35.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.592+0000 7f0c3e670640 1 -- 172.21.15.84:0/2546500725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3806b1f0 msgr2=0x7f0c3814bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.592+0000 7f0c3e670640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3806b1f0 0x7f0c3814bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009db0 tx=0x7f0c3000c490 comp rx=0 tx=0).stop 2024-09-13T16:47:35.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.592+0000 7f0c3e670640 1 -- 172.21.15.84:0/2546500725 shutdown_connections 2024-09-13T16:47:35.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.592+0000 7f0c3e670640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38150b90 0x7f0c381eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.592+0000 7f0c3e670640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0c380c1b80 0x7f0c380bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.592+0000 7f0c3e670640 1 --2- 172.21.15.84:0/2546500725 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c3806b1f0 0x7f0c3814bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.592+0000 7f0c3e670640 1 -- 172.21.15.84:0/2546500725 >> 172.21.15.84:0/2546500725 conn(0x7f0c380c32a0 msgr2=0x7f0c3806dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:35.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.592+0000 7f0c3e670640 1 -- 172.21.15.84:0/2546500725 shutdown_connections 2024-09-13T16:47:35.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.592+0000 7f0c3e670640 1 -- 172.21.15.84:0/2546500725 wait complete. 2024-09-13T16:47:35.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new 069d6f12-1c43-468a-871e-4f53f7755e1d 1 2024-09-13T16:47:35.700 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:35.698+0000 7f4574214180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T16:47:35.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d0e34c640 1 Processor -- start 2024-09-13T16:47:35.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d0e34c640 1 -- start start 2024-09-13T16:47:35.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d0e34c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9d0814bf30 0x7f9d0806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d0e34c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d08155160 0x7f9d081520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d0e34c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d08151bf0 con 0x7f9d0806b360 2024-09-13T16:47:35.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d0e34c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d08155660 con 0x7f9d0814bf30 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d0e34c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08155940 con 0x7f9d08155160 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d07fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9d0814bf30 0x7f9d0806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d077fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d08155160 0x7f9d081520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d07fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9d0814bf30 0x7f9d0806ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39056/0 (socket says 172.21.15.84:39056) 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d07fff640 1 -- 172.21.15.84:0/1403217992 learned_addr learned my addr 172.21.15.84:0/1403217992 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d07fff640 1 -- 172.21.15.84:0/1403217992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d08155160 msgr2=0x7f9d081520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d07fff640 1 --2- 172.21.15.84:0/1403217992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d08155160 0x7f9d081520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d07fff640 1 -- 172.21.15.84:0/1403217992 >> v1:172.21.15.39:6789/0 conn(0x7f9d0806b360 legacy=0x7f9d08152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d07fff640 1 -- 172.21.15.84:0/1403217992 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d08155c20 con 0x7f9d0814bf30 2024-09-13T16:47:35.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.768+0000 7f9d077fe640 1 --2- 172.21.15.84:0/1403217992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d08155160 0x7f9d081520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:35.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d07fff640 1 --2- 172.21.15.84:0/1403217992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d0814bf30 0x7f9d0806ae20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80098a0 tx=0x7f9cf801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68da032fdea82fac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:35.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d06ffd640 1 -- 172.21.15.84:0/1403217992 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80ab670 con 0x7f9d0814bf30 2024-09-13T16:47:35.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d06ffd640 1 -- 172.21.15.84:0/1403217992 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf80ab7d0 con 0x7f9d0814bf30 2024-09-13T16:47:35.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d06ffd640 1 -- 172.21.15.84:0/1403217992 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80abb60 con 0x7f9d0814bf30 2024-09-13T16:47:35.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1403217992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d0814bf30 msgr2=0x7f9d0806ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d0e34c640 1 --2- 172.21.15.84:0/1403217992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d0814bf30 0x7f9d0806ae20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80098a0 tx=0x7f9cf801c4d0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1403217992 shutdown_connections 2024-09-13T16:47:35.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d0e34c640 1 --2- 172.21.15.84:0/1403217992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d08155160 0x7f9d081520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d0e34c640 1 --2- 172.21.15.84:0/1403217992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d0814bf30 0x7f9d0806ae20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1403217992 >> 172.21.15.84:0/1403217992 conn(0x7f9d080c32a0 msgr2=0x7f9d080c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:35.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1403217992 shutdown_connections 2024-09-13T16:47:35.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.772+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1403217992 wait complete. 2024-09-13T16:47:35.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 Processor -- start 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 -- start start 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d0806b360 0x7f9d08071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0814bf30 0x7f9d08142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d08155160 0x7f9d08143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08153960 con 0x7f9d0814bf30 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d081532f0 con 0x7f9d08155160 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08145490 con 0x7f9d0806b360 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d07fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d0806b360 0x7f9d08071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d07fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d0806b360 0x7f9d08071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.84:40796/0 (socket says 172.21.15.84:40796) 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0c8e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d08155160 0x7f9d08143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d07fff640 1 -- 172.21.15.84:0/1381624450 learned_addr learned my addr 172.21.15.84:0/1381624450 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d077fe640 1 --2- 172.21.15.84:0/1381624450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0814bf30 0x7f9d08142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0c8e9640 1 -- 172.21.15.84:0/1381624450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d0806b360 msgr2=0x7f9d08071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0c8e9640 1 --2- 172.21.15.84:0/1381624450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d0806b360 0x7f9d08071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0c8e9640 1 -- 172.21.15.84:0/1381624450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0814bf30 msgr2=0x7f9d08142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0c8e9640 1 --2- 172.21.15.84:0/1381624450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0814bf30 0x7f9d08142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0c8e9640 1 -- 172.21.15.84:0/1381624450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc009420 con 0x7f9d08155160 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d07fff640 1 --2- 172.21.15.84:0/1381624450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d0806b360 0x7f9d08071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0c8e9640 1 --2- 172.21.15.84:0/1381624450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d08155160 0x7f9d08143180 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c780 tx=0x7f9d0000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d04ff9640 1 -- 172.21.15.84:0/1381624450 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000a840 con 0x7f9d08155160 2024-09-13T16:47:35.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d04ff9640 1 -- 172.21.15.84:0/1381624450 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0000a9a0 con 0x7f9d08155160 2024-09-13T16:47:35.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1381624450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf80094d0 con 0x7f9d08155160 2024-09-13T16:47:35.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d04ff9640 1 -- 172.21.15.84:0/1381624450 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000ad90 con 0x7f9d08155160 2024-09-13T16:47:35.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1381624450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d08144ed0 con 0x7f9d08155160 2024-09-13T16:47:35.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d04ff9640 1 -- 172.21.15.84:0/1381624450 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9d00004c20 con 0x7f9d08155160 2024-09-13T16:47:35.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d04ff9640 1 -- 172.21.15.84:0/1381624450 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f9d00012420 con 0x7f9d08155160 2024-09-13T16:47:35.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.776+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1381624450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d08153ad0 con 0x7f9d08155160 2024-09-13T16:47:35.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.780+0000 7f9d04ff9640 1 -- 172.21.15.84:0/1381624450 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f9d00039e30 con 0x7f9d08155160 2024-09-13T16:47:35.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.848+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1381624450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd new", "uuid": "069d6f12-1c43-468a-871e-4f53f7755e1d", "id": 1} v 0) v1 -- 0x7f9d0806ae20 con 0x7f9d08155160 2024-09-13T16:47:35.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.856+0000 7f9d04ff9640 1 -- 172.21.15.84:0/1381624450 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "069d6f12-1c43-468a-871e-4f53f7755e1d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9d00028070 con 0x7f9d08155160 2024-09-13T16:47:35.862 INFO:teuthology.orchestra.run.smithi084.stdout:1 2024-09-13T16:47:35.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.860+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1381624450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d08155160 msgr2=0x7f9d08143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.860+0000 7f9d0e34c640 1 --2- 172.21.15.84:0/1381624450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d08155160 0x7f9d08143180 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c780 tx=0x7f9d0000cc40 comp rx=0 tx=0).stop 2024-09-13T16:47:35.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.860+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1381624450 shutdown_connections 2024-09-13T16:47:35.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.860+0000 7f9d0e34c640 1 --2- 172.21.15.84:0/1381624450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d08155160 0x7f9d08143180 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.860+0000 7f9d0e34c640 1 --2- 172.21.15.84:0/1381624450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d0814bf30 0x7f9d08142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.860+0000 7f9d0e34c640 1 --2- 172.21.15.84:0/1381624450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d0806b360 0x7f9d08071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.860+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1381624450 >> 172.21.15.84:0/1381624450 conn(0x7f9d080c32a0 msgr2=0x7f9d080c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.860+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1381624450 shutdown_connections 2024-09-13T16:47:35.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.860+0000 7f9d0e34c640 1 -- 172.21.15.84:0/1381624450 wait complete. 2024-09-13T16:47:35.867 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:35.864+0000 7f79ad3ae180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T16:47:35.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new cb3c693e-e427-46c1-97a3-fd7723d4993b 2 2024-09-13T16:47:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.980+0000 7f8751e67640 1 Processor -- start 2024-09-13T16:47:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f8751e67640 1 -- start start 2024-09-13T16:47:35.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f8751e67640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f874c0c73e0 0x7f874c0c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f8751e67640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f874c0a71c0 0x7f874c0c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f8751e67640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874c0c62f0 con 0x7f874c0c3e80 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f8751e67640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c06dd90 con 0x7f874c0a71c0 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f8751e67640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c06f040 con 0x7f874c0c73e0 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f874affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f874c0c73e0 0x7f874c0c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f874affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f874c0c73e0 0x7f874c0c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39068/0 (socket says 172.21.15.84:39068) 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f874affd640 1 -- 172.21.15.84:0/3092308799 learned_addr learned my addr 172.21.15.84:0/3092308799 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f874bfff640 1 --2- 172.21.15.84:0/3092308799 >> v2:172.21.15.39:3300/0 conn(0x7f874c0a71c0 0x7f874c0c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f874affd640 1 -- 172.21.15.84:0/3092308799 >> v2:172.21.15.39:3300/0 conn(0x7f874c0a71c0 msgr2=0x7f874c0c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f874affd640 1 --2- 172.21.15.84:0/3092308799 >> v2:172.21.15.39:3300/0 conn(0x7f874c0a71c0 0x7f874c0c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f874affd640 1 -- 172.21.15.84:0/3092308799 >> v1:172.21.15.39:6789/0 conn(0x7f874c0c3e80 legacy=0x7f874c0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f874affd640 1 -- 172.21.15.84:0/3092308799 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c0aa460 con 0x7f874c0c73e0 2024-09-13T16:47:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.984+0000 7f874affd640 1 --2- 172.21.15.84:0/3092308799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c73e0 0x7f874c0c5870 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f874c0c1f50 tx=0x7f874001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7adbdea6067f1440 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874a7fc640 1 -- 172.21.15.84:0/3092308799 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400aa5f0 con 0x7f874c0c73e0 2024-09-13T16:47:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874a7fc640 1 -- 172.21.15.84:0/3092308799 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87400aa750 con 0x7f874c0c73e0 2024-09-13T16:47:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874a7fc640 1 -- 172.21.15.84:0/3092308799 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400aaae0 con 0x7f874c0c73e0 2024-09-13T16:47:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- 172.21.15.84:0/3092308799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c73e0 msgr2=0x7f874c0c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 --2- 172.21.15.84:0/3092308799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c73e0 0x7f874c0c5870 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f874c0c1f50 tx=0x7f874001c490 comp rx=0 tx=0).stop 2024-09-13T16:47:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- 172.21.15.84:0/3092308799 shutdown_connections 2024-09-13T16:47:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 --2- 172.21.15.84:0/3092308799 >> v2:172.21.15.39:3300/0 conn(0x7f874c0a71c0 0x7f874c0c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 --2- 172.21.15.84:0/3092308799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c73e0 0x7f874c0c5870 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- 172.21.15.84:0/3092308799 >> 172.21.15.84:0/3092308799 conn(0x7f874c0a9720 msgr2=0x7f874c0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- 172.21.15.84:0/3092308799 shutdown_connections 2024-09-13T16:47:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- 172.21.15.84:0/3092308799 wait complete. 2024-09-13T16:47:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 Processor -- start 2024-09-13T16:47:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- start start 2024-09-13T16:47:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f874c0a71c0 0x7f874c0acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c3e80 0x7f874c0ab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c0c73e0 0x7f874c1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1ea530 con 0x7f874c0c73e0 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1ea7f0 con 0x7f874c0c3e80 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c14c330 con 0x7f874c0a71c0 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c3e80 0x7f874c0ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874b7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f874c0a71c0 0x7f874c0acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c3e80 0x7f874c0ab180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39082/0 (socket says 172.21.15.84:39082) 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874affd640 1 -- 172.21.15.84:0/2899293336 learned_addr learned my addr 172.21.15.84:0/2899293336 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874bfff640 1 --2- 172.21.15.84:0/2899293336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c0c73e0 0x7f874c1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874b7fe640 1 -- 172.21.15.84:0/2899293336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c3e80 msgr2=0x7f874c0ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874b7fe640 1 --2- 172.21.15.84:0/2899293336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c3e80 0x7f874c0ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874b7fe640 1 -- 172.21.15.84:0/2899293336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c0c73e0 msgr2=0x7f874c1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874b7fe640 1 --2- 172.21.15.84:0/2899293336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c0c73e0 0x7f874c1522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874b7fe640 1 -- 172.21.15.84:0/2899293336 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400094d0 con 0x7f874c0a71c0 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874affd640 1 --2- 172.21.15.84:0/2899293336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c3e80 0x7f874c0ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f874b7fe640 1 --2- 172.21.15.84:0/2899293336 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f874c0a71c0 0x7f874c0acd10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f873c00c3c0 tx=0x7f873c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8733fff640 1 -- 172.21.15.84:0/2899293336 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c0052d0 con 0x7f874c0a71c0 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8733fff640 1 -- 172.21.15.84:0/2899293336 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873c005430 con 0x7f874c0a71c0 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8733fff640 1 -- 172.21.15.84:0/2899293336 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c007260 con 0x7f874c0a71c0 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- 172.21.15.84:0/2899293336 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c14bdd0 con 0x7f874c0a71c0 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8751e67640 1 -- 172.21.15.84:0/2899293336 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c0ab6c0 con 0x7f874c0a71c0 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8733fff640 1 -- 172.21.15.84:0/2899293336 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f873c0067e0 con 0x7f874c0a71c0 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.988+0000 7f8733fff640 1 -- 172.21.15.84:0/2899293336 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f873c003d40 con 0x7f874c0a71c0 2024-09-13T16:47:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.992+0000 7f8751e67640 1 -- 172.21.15.84:0/2899293336 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c14bc30 con 0x7f874c0a71c0 2024-09-13T16:47:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:35.992+0000 7f8733fff640 1 -- 172.21.15.84:0/2899293336 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f874c14bc30 con 0x7f874c0a71c0 2024-09-13T16:47:36.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.056+0000 7f8751e67640 1 -- 172.21.15.84:0/2899293336 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cb3c693e-e427-46c1-97a3-fd7723d4993b", "id": 2} v 0) v1 -- 0x7f874c14c010 con 0x7f874c0a71c0 2024-09-13T16:47:36.067 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:36.064+0000 7f79ad3ae180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T16:47:36.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.068+0000 7f8733fff640 1 -- 172.21.15.84:0/2899293336 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb3c693e-e427-46c1-97a3-fd7723d4993b", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f874c14c010 con 0x7f874c0a71c0 2024-09-13T16:47:36.074 INFO:teuthology.orchestra.run.smithi084.stdout:2 2024-09-13T16:47:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.072+0000 7f8751e67640 1 -- 172.21.15.84:0/2899293336 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f874c0a71c0 msgr2=0x7f874c0acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.072+0000 7f8751e67640 1 --2- 172.21.15.84:0/2899293336 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f874c0a71c0 0x7f874c0acd10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f873c00c3c0 tx=0x7f873c00c880 comp rx=0 tx=0).stop 2024-09-13T16:47:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.072+0000 7f8751e67640 1 -- 172.21.15.84:0/2899293336 shutdown_connections 2024-09-13T16:47:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.072+0000 7f8751e67640 1 --2- 172.21.15.84:0/2899293336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f874c0c3e80 0x7f874c0ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.072+0000 7f8751e67640 1 --2- 172.21.15.84:0/2899293336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f874c0c73e0 0x7f874c1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.072+0000 7f8751e67640 1 --2- 172.21.15.84:0/2899293336 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f874c0a71c0 0x7f874c0acd10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.072+0000 7f8751e67640 1 -- 172.21.15.84:0/2899293336 >> 172.21.15.84:0/2899293336 conn(0x7f874c0a9720 msgr2=0x7f874c0c5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.072+0000 7f8751e67640 1 -- 172.21.15.84:0/2899293336 shutdown_connections 2024-09-13T16:47:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.072+0000 7f8751e67640 1 -- 172.21.15.84:0/2899293336 wait complete. 2024-09-13T16:47:36.091 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new 441b35f8-eab6-409d-a03c-6dca2d9f5671 3 2024-09-13T16:47:36.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79aa075640 1 Processor -- start 2024-09-13T16:47:36.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79aa075640 1 -- start start 2024-09-13T16:47:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79aa075640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f79a40aefa0 0x7f79a4157d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79aa075640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c7e10 0x7f79a4070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79aa075640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a40c77e0 con 0x7f79a40bcfb0 2024-09-13T16:47:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79aa075640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a41582c0 con 0x7f79a40aefa0 2024-09-13T16:47:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79aa075640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4157330 con 0x7f79a40c7e10 2024-09-13T16:47:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79a2ffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c7e10 0x7f79a4070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79a37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f79a40aefa0 0x7f79a4157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79a2ffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c7e10 0x7f79a4070430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.84:40814/0 (socket says 172.21.15.84:40814) 2024-09-13T16:47:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79a2ffd640 1 -- 172.21.15.84:0/1622277333 learned_addr learned my addr 172.21.15.84:0/1622277333 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79a2ffd640 1 -- 172.21.15.84:0/1622277333 >> v2:172.21.15.84:3300/0 conn(0x7f79a40aefa0 msgr2=0x7f79a4157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79a2ffd640 1 --2- 172.21.15.84:0/1622277333 >> v2:172.21.15.84:3300/0 conn(0x7f79a40aefa0 0x7f79a4157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79a2ffd640 1 -- 172.21.15.84:0/1622277333 >> v1:172.21.15.39:6789/0 conn(0x7f79a40bcfb0 legacy=0x7f79a40c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:36.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.192+0000 7f79a2ffd640 1 -- 172.21.15.84:0/1622277333 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a41578f0 con 0x7f79a40c7e10 2024-09-13T16:47:36.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79a2ffd640 1 --2- 172.21.15.84:0/1622277333 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c7e10 0x7f79a4070430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7998009870 tx=0x7f799801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53329ff7f47f81a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:36.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79a27fc640 1 -- 172.21.15.84:0/1622277333 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79980ab600 con 0x7f79a40c7e10 2024-09-13T16:47:36.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79a27fc640 1 -- 172.21.15.84:0/1622277333 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79980a7050 con 0x7f79a40c7e10 2024-09-13T16:47:36.200 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79a27fc640 1 -- 172.21.15.84:0/1622277333 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79980ab980 con 0x7f79a40c7e10 2024-09-13T16:47:36.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79aa075640 1 -- 172.21.15.84:0/1622277333 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c7e10 msgr2=0x7f79a4070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79aa075640 1 --2- 172.21.15.84:0/1622277333 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c7e10 0x7f79a4070430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7998009870 tx=0x7f799801c490 comp rx=0 tx=0).stop 2024-09-13T16:47:36.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79aa075640 1 -- 172.21.15.84:0/1622277333 shutdown_connections 2024-09-13T16:47:36.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79aa075640 1 --2- 172.21.15.84:0/1622277333 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c7e10 0x7f79a4070430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79aa075640 1 --2- 172.21.15.84:0/1622277333 >> v2:172.21.15.84:3300/0 conn(0x7f79a40aefa0 0x7f79a4157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79aa075640 1 -- 172.21.15.84:0/1622277333 >> 172.21.15.84:0/1622277333 conn(0x7f79a4069e10 msgr2=0x7f79a406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:36.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79aa075640 1 -- 172.21.15.84:0/1622277333 shutdown_connections 2024-09-13T16:47:36.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.196+0000 7f79aa075640 1 -- 172.21.15.84:0/1622277333 wait complete. 2024-09-13T16:47:36.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 Processor -- start 2024-09-13T16:47:36.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 -- start start 2024-09-13T16:47:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79a40aefa0 0x7f79a40c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79a40bcfb0 0x7f79a41ed480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c99e0 0x7f79a41ed9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41ecff0 con 0x7f79a40aefa0 2024-09-13T16:47:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a40c3cd0 con 0x7f79a40bcfb0 2024-09-13T16:47:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41546a0 con 0x7f79a40c99e0 2024-09-13T16:47:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79a40bcfb0 0x7f79a41ed480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79a40bcfb0 0x7f79a41ed480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39098/0 (socket says 172.21.15.84:39098) 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c99e0 0x7f79a41ed9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a2ffd640 1 -- 172.21.15.84:0/2390444800 learned_addr learned my addr 172.21.15.84:0/2390444800 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a2ffd640 1 -- 172.21.15.84:0/2390444800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c99e0 msgr2=0x7f79a41ed9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a2ffd640 1 --2- 172.21.15.84:0/2390444800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c99e0 0x7f79a41ed9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a2ffd640 1 -- 172.21.15.84:0/2390444800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79a40aefa0 msgr2=0x7f79a40c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a2ffd640 1 --2- 172.21.15.84:0/2390444800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79a40aefa0 0x7f79a40c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a2ffd640 1 -- 172.21.15.84:0/2390444800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79980094d0 con 0x7f79a40bcfb0 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a3fff640 1 --2- 172.21.15.84:0/2390444800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c99e0 0x7f79a41ed9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79a2ffd640 1 --2- 172.21.15.84:0/2390444800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79a40bcfb0 0x7f79a41ed480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f79980ae040 tx=0x7f7998019230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f798bfff640 1 -- 172.21.15.84:0/2390444800 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79980b6420 con 0x7f79a40bcfb0 2024-09-13T16:47:36.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f798bfff640 1 -- 172.21.15.84:0/2390444800 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79980b6580 con 0x7f79a40bcfb0 2024-09-13T16:47:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f798bfff640 1 -- 172.21.15.84:0/2390444800 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79980b6930 con 0x7f79a40bcfb0 2024-09-13T16:47:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 -- 172.21.15.84:0/2390444800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a4154c40 con 0x7f79a40bcfb0 2024-09-13T16:47:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 -- 172.21.15.84:0/2390444800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a4154960 con 0x7f79a40bcfb0 2024-09-13T16:47:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f798bfff640 1 -- 172.21.15.84:0/2390444800 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f79980b2040 con 0x7f79a40bcfb0 2024-09-13T16:47:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f798bfff640 1 -- 172.21.15.84:0/2390444800 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f79980b43f0 con 0x7f79a40bcfb0 2024-09-13T16:47:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.200+0000 7f79aa075640 1 -- 172.21.15.84:0/2390444800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a4157d80 con 0x7f79a40bcfb0 2024-09-13T16:47:36.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.204+0000 7f798bfff640 1 -- 172.21.15.84:0/2390444800 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f79980dee30 con 0x7f79a40bcfb0 2024-09-13T16:47:36.273 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:36.270+0000 7f4574214180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T16:47:36.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.272+0000 7f79aa075640 1 -- 172.21.15.84:0/2390444800 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd new", "uuid": "441b35f8-eab6-409d-a03c-6dca2d9f5671", "id": 3} v 0) v1 -- 0x7f79a40c6e00 con 0x7f79a40bcfb0 2024-09-13T16:47:36.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.280+0000 7f798bfff640 1 -- 172.21.15.84:0/2390444800 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "441b35f8-eab6-409d-a03c-6dca2d9f5671", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f79980cd030 con 0x7f79a40bcfb0 2024-09-13T16:47:36.285 INFO:teuthology.orchestra.run.smithi084.stdout:3 2024-09-13T16:47:36.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.280+0000 7f79aa075640 1 -- 172.21.15.84:0/2390444800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79a40bcfb0 msgr2=0x7f79a41ed480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.280+0000 7f79aa075640 1 --2- 172.21.15.84:0/2390444800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79a40bcfb0 0x7f79a41ed480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f79980ae040 tx=0x7f7998019230 comp rx=0 tx=0).stop 2024-09-13T16:47:36.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.280+0000 7f79aa075640 1 -- 172.21.15.84:0/2390444800 shutdown_connections 2024-09-13T16:47:36.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.280+0000 7f79aa075640 1 --2- 172.21.15.84:0/2390444800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79a40bcfb0 0x7f79a41ed480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.280+0000 7f79aa075640 1 --2- 172.21.15.84:0/2390444800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79a40c99e0 0x7f79a41ed9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.280+0000 7f79aa075640 1 --2- 172.21.15.84:0/2390444800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79a40aefa0 0x7f79a40c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.280+0000 7f79aa075640 1 -- 172.21.15.84:0/2390444800 >> 172.21.15.84:0/2390444800 conn(0x7f79a4069e10 msgr2=0x7f79a406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:36.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.284+0000 7f79aa075640 1 -- 172.21.15.84:0/2390444800 shutdown_connections 2024-09-13T16:47:36.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.284+0000 7f79aa075640 1 -- 172.21.15.84:0/2390444800 wait complete. 2024-09-13T16:47:36.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new 7387ca7b-a2ad-4477-aead-26e0ee232e36 4 2024-09-13T16:47:36.329 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:36.326+0000 7f4574214180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T16:47:36.384 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:36.382+0000 7f4574214180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T16:47:36.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f41f11640 1 Processor -- start 2024-09-13T16:47:36.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f41f11640 1 -- start start 2024-09-13T16:47:36.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f41f11640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c14bf30 0x7f2f3c06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f41f11640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c151bf0 con 0x7f2f3c155160 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f41f11640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c155660 con 0x7f2f3c06b360 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f41f11640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c155940 con 0x7f2f3c14bf30 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c14bf30 0x7f2f3c06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f3bfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2f3c06b360 0x7f2f3c15c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35796/0 (socket says 172.21.15.84:35796) 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f3bfff640 1 -- 172.21.15.84:0/1178184909 learned_addr learned my addr 172.21.15.84:0/1178184909 (peer_addr_for_me v1:172.21.15.84:0/0) 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f3b7fe640 1 -- 172.21.15.84:0/1178184909 >> v1:172.21.15.84:6789/0 conn(0x7f2f3c06b360 legacy=0x7f2f3c15c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f3b7fe640 1 -- 172.21.15.84:0/1178184909 >> v1:172.21.15.39:6789/0 conn(0x7f2f3c155160 legacy=0x7f2f3c1520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f3a7fc640 1 -- 172.21.15.84:0/1178184909 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495403674 0 0) 0x7f2f3c155660 con 0x7f2f3c06b360 2024-09-13T16:47:36.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.408+0000 7f2f3b7fe640 1 -- 172.21.15.84:0/1178184909 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c155c20 con 0x7f2f3c14bf30 2024-09-13T16:47:36.415 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f3b7fe640 1 --2- 172.21.15.84:0/1178184909 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c14bf30 0x7f2f3c06ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009820 tx=0x7f2f2c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba42ae3e767325d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:36.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f3a7fc640 1 -- 172.21.15.84:0/1178184909 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0a7070 con 0x7f2f3c14bf30 2024-09-13T16:47:36.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f3a7fc640 1 -- 172.21.15.84:0/1178184909 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c0ab430 con 0x7f2f3c14bf30 2024-09-13T16:47:36.416 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f3a7fc640 1 -- 172.21.15.84:0/1178184909 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0ab7c0 con 0x7f2f3c14bf30 2024-09-13T16:47:36.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f41f11640 1 -- 172.21.15.84:0/1178184909 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c14bf30 msgr2=0x7f2f3c06ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f41f11640 1 --2- 172.21.15.84:0/1178184909 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c14bf30 0x7f2f3c06ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009820 tx=0x7f2f2c01c450 comp rx=0 tx=0).stop 2024-09-13T16:47:36.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f41f11640 1 -- 172.21.15.84:0/1178184909 shutdown_connections 2024-09-13T16:47:36.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f41f11640 1 --2- 172.21.15.84:0/1178184909 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c14bf30 0x7f2f3c06ae20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f41f11640 1 -- 172.21.15.84:0/1178184909 >> 172.21.15.84:0/1178184909 conn(0x7f2f3c0c32a0 msgr2=0x7f2f3c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:36.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f41f11640 1 -- 172.21.15.84:0/1178184909 shutdown_connections 2024-09-13T16:47:36.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f41f11640 1 -- 172.21.15.84:0/1178184909 wait complete. 2024-09-13T16:47:36.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f41f11640 1 Processor -- start 2024-09-13T16:47:36.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.412+0000 7f2f41f11640 1 -- start start 2024-09-13T16:47:36.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f41f11640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c06b360 0x7f2f3c152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f41f11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f3c14bf30 0x7f2f3c0bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f41f11640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f3c155160 0x7f2f3c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f41f11640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c14a630 con 0x7f2f3c155160 2024-09-13T16:47:36.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f41f11640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c153290 con 0x7f2f3c14bf30 2024-09-13T16:47:36.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f41f11640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c145b10 con 0x7f2f3c06b360 2024-09-13T16:47:36.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c06b360 0x7f2f3c152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3b7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c06b360 0x7f2f3c152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.84:40846/0 (socket says 172.21.15.84:40846) 2024-09-13T16:47:36.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3b7fe640 1 -- 172.21.15.84:0/3402014560 learned_addr learned my addr 172.21.15.84:0/3402014560 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3affd640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f3c14bf30 0x7f2f3c0bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3bfff640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f3c155160 0x7f2f3c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3b7fe640 1 -- 172.21.15.84:0/3402014560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f3c14bf30 msgr2=0x7f2f3c0bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3b7fe640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f3c14bf30 0x7f2f3c0bfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3b7fe640 1 -- 172.21.15.84:0/3402014560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f3c155160 msgr2=0x7f2f3c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3b7fe640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f3c155160 0x7f2f3c071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3b7fe640 1 -- 172.21.15.84:0/3402014560 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c009420 con 0x7f2f3c06b360 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3affd640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f3c14bf30 0x7f2f3c0bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f3b7fe640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c06b360 0x7f2f3c152d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0ad040 tx=0x7f2f2c0191f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f33fff640 1 -- 172.21.15.84:0/3402014560 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0b4800 con 0x7f2f3c06b360 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f33fff640 1 -- 172.21.15.84:0/3402014560 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c0b4980 con 0x7f2f3c06b360 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f41f11640 1 -- 172.21.15.84:0/3402014560 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c1497c0 con 0x7f2f3c06b360 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f33fff640 1 -- 172.21.15.84:0/3402014560 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0b4d30 con 0x7f2f3c06b360 2024-09-13T16:47:36.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f41f11640 1 -- 172.21.15.84:0/3402014560 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c1ed860 con 0x7f2f3c06b360 2024-09-13T16:47:36.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f33fff640 1 -- 172.21.15.84:0/3402014560 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f2f2c0c1760 con 0x7f2f3c06b360 2024-09-13T16:47:36.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f33fff640 1 -- 172.21.15.84:0/3402014560 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2f2c0c0a40 con 0x7f2f3c06b360 2024-09-13T16:47:36.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.416+0000 7f2f41f11640 1 -- 172.21.15.84:0/3402014560 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c149ca0 con 0x7f2f3c06b360 2024-09-13T16:47:36.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.420+0000 7f2f33fff640 1 -- 172.21.15.84:0/3402014560 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f2f2c0c0c50 con 0x7f2f3c06b360 2024-09-13T16:47:36.465 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:36.462+0000 7f4574214180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T16:47:36.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.480+0000 7f2f41f11640 1 -- 172.21.15.84:0/3402014560 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7387ca7b-a2ad-4477-aead-26e0ee232e36", "id": 4} v 0) v1 -- 0x7f2f3c06ae20 con 0x7f2f3c06b360 2024-09-13T16:47:36.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f33fff640 1 -- 172.21.15.84:0/3402014560 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7387ca7b-a2ad-4477-aead-26e0ee232e36", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2f2c0c0df0 con 0x7f2f3c06b360 2024-09-13T16:47:36.492 INFO:teuthology.orchestra.run.smithi084.stdout:4 2024-09-13T16:47:36.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f41f11640 1 -- 172.21.15.84:0/3402014560 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c06b360 msgr2=0x7f2f3c152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f41f11640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c06b360 0x7f2f3c152d50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0ad040 tx=0x7f2f2c0191f0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f41f11640 1 -- 172.21.15.84:0/3402014560 shutdown_connections 2024-09-13T16:47:36.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f41f11640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f3c14bf30 0x7f2f3c0bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f41f11640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f3c155160 0x7f2f3c071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f41f11640 1 --2- 172.21.15.84:0/3402014560 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f3c06b360 0x7f2f3c152d50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.493 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f41f11640 1 -- 172.21.15.84:0/3402014560 >> 172.21.15.84:0/3402014560 conn(0x7f2f3c0c32a0 msgr2=0x7f2f3c150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:36.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f41f11640 1 -- 172.21.15.84:0/3402014560 shutdown_connections 2024-09-13T16:47:36.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.488+0000 7f2f41f11640 1 -- 172.21.15.84:0/3402014560 wait complete. 2024-09-13T16:47:36.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new 68838edd-ff7e-4147-9d5d-8f2103121f21 5 2024-09-13T16:47:36.519 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:36.518+0000 7f4574214180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T16:47:36.600 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:36.598+0000 7f4574214180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T16:47:36.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f652458c640 1 Processor -- start 2024-09-13T16:47:36.627 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f652458c640 1 -- start start 2024-09-13T16:47:36.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f652458c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f651c14acf0 0x7f651c151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f652458c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c0c4710 0x7f651c06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f652458c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c154ab0 con 0x7f651c0c4b70 2024-09-13T16:47:36.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f652458c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c154d90 con 0x7f651c14acf0 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f652458c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c150d50 con 0x7f651c0c4710 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6522328640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f651c14acf0 0x7f651c151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6522328640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f651c14acf0 0x7f651c151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39112/0 (socket says 172.21.15.84:39112) 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6522328640 1 -- 172.21.15.84:0/4100599006 learned_addr learned my addr 172.21.15.84:0/4100599006 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6521b27640 1 --2- 172.21.15.84:0/4100599006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c0c4710 0x7f651c06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6522328640 1 -- 172.21.15.84:0/4100599006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c0c4710 msgr2=0x7f651c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6522328640 1 --2- 172.21.15.84:0/4100599006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c0c4710 0x7f651c06ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6522328640 1 -- 172.21.15.84:0/4100599006 >> v1:172.21.15.84:6789/0 conn(0x7f651c0c4b70 legacy=0x7f651c153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6522328640 1 -- 172.21.15.84:0/4100599006 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c151030 con 0x7f651c14acf0 2024-09-13T16:47:36.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6521b27640 1 --2- 172.21.15.84:0/4100599006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c0c4710 0x7f651c06ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:36.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6522328640 1 --2- 172.21.15.84:0/4100599006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c14acf0 0x7f651c151550 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6518009870 tx=0x7f651801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=949b7fdb8762abed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:36.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.624+0000 7f6521326640 1 -- 172.21.15.84:0/4100599006 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65180aa5f0 con 0x7f651c14acf0 2024-09-13T16:47:36.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6521326640 1 -- 172.21.15.84:0/4100599006 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65180aa750 con 0x7f651c14acf0 2024-09-13T16:47:36.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6521326640 1 -- 172.21.15.84:0/4100599006 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65180aaae0 con 0x7f651c14acf0 2024-09-13T16:47:36.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 -- 172.21.15.84:0/4100599006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c14acf0 msgr2=0x7f651c151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 --2- 172.21.15.84:0/4100599006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c14acf0 0x7f651c151550 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6518009870 tx=0x7f651801c470 comp rx=0 tx=0).stop 2024-09-13T16:47:36.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 -- 172.21.15.84:0/4100599006 shutdown_connections 2024-09-13T16:47:36.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 --2- 172.21.15.84:0/4100599006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c0c4710 0x7f651c06ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 --2- 172.21.15.84:0/4100599006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c14acf0 0x7f651c151550 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 -- 172.21.15.84:0/4100599006 >> 172.21.15.84:0/4100599006 conn(0x7f651c0c32a0 msgr2=0x7f651c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:36.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 -- 172.21.15.84:0/4100599006 shutdown_connections 2024-09-13T16:47:36.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 -- 172.21.15.84:0/4100599006 wait complete. 2024-09-13T16:47:36.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 Processor -- start 2024-09-13T16:47:36.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 -- start start 2024-09-13T16:47:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c0c4710 0x7f651c1479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0c4b70 0x7f651c14e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c14acf0 0x7f651c14cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0ab570 con 0x7f651c0c4b70 2024-09-13T16:47:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0ab810 con 0x7f651c0c4710 2024-09-13T16:47:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f652458c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0abaf0 con 0x7f651c14acf0 2024-09-13T16:47:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522328640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c0c4710 0x7f651c1479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522b29640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c14acf0 0x7f651c14cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522b29640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c14acf0 0x7f651c14cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.84:40866/0 (socket says 172.21.15.84:40866) 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522b29640 1 -- 172.21.15.84:0/2215296540 learned_addr learned my addr 172.21.15.84:0/2215296540 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522328640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c0c4710 0x7f651c1479f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39126/0 (socket says 172.21.15.84:39126) 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6521b27640 1 --2- 172.21.15.84:0/2215296540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0c4b70 0x7f651c14e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522b29640 1 -- 172.21.15.84:0/2215296540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c0c4710 msgr2=0x7f651c1479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522b29640 1 --2- 172.21.15.84:0/2215296540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c0c4710 0x7f651c1479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522b29640 1 -- 172.21.15.84:0/2215296540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0c4b70 msgr2=0x7f651c14e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522b29640 1 --2- 172.21.15.84:0/2215296540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0c4b70 0x7f651c14e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522b29640 1 -- 172.21.15.84:0/2215296540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c009420 con 0x7f651c14acf0 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522328640 1 --2- 172.21.15.84:0/2215296540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c0c4710 0x7f651c1479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:36.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.628+0000 7f6522b29640 1 --2- 172.21.15.84:0/2215296540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c14acf0 0x7f651c14cd50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f651400c6c0 tx=0x7f651400cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:36.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.632+0000 7f6512ffd640 1 -- 172.21.15.84:0/2215296540 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514003b80 con 0x7f651c14acf0 2024-09-13T16:47:36.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.632+0000 7f6512ffd640 1 -- 172.21.15.84:0/2215296540 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6514003ce0 con 0x7f651c14acf0 2024-09-13T16:47:36.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.632+0000 7f652458c640 1 -- 172.21.15.84:0/2215296540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65180094d0 con 0x7f651c14acf0 2024-09-13T16:47:36.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.632+0000 7f6512ffd640 1 -- 172.21.15.84:0/2215296540 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140052f0 con 0x7f651c14acf0 2024-09-13T16:47:36.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.632+0000 7f652458c640 1 -- 172.21.15.84:0/2215296540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c154880 con 0x7f651c14acf0 2024-09-13T16:47:36.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.632+0000 7f6512ffd640 1 -- 172.21.15.84:0/2215296540 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f651400a640 con 0x7f651c14acf0 2024-09-13T16:47:36.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.632+0000 7f6512ffd640 1 -- 172.21.15.84:0/2215296540 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f65140129a0 con 0x7f651c14acf0 2024-09-13T16:47:36.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.632+0000 7f652458c640 1 -- 172.21.15.84:0/2215296540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c0c0f10 con 0x7f651c14acf0 2024-09-13T16:47:36.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.632+0000 7f6512ffd640 1 -- 172.21.15.84:0/2215296540 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f6514012c80 con 0x7f651c14acf0 2024-09-13T16:47:36.639 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:36.636+0000 7f79ad3ae180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T16:47:36.654 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:36.654+0000 7f4574214180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T16:47:36.694 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:36.688+0000 7f79ad3ae180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T16:47:36.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.700+0000 7f652458c640 1 -- 172.21.15.84:0/2215296540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "osd new", "uuid": "68838edd-ff7e-4147-9d5d-8f2103121f21", "id": 5} v 0) v1 -- 0x7f651c0ac010 con 0x7f651c14acf0 2024-09-13T16:47:36.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f6512ffd640 1 -- 172.21.15.84:0/2215296540 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68838edd-ff7e-4147-9d5d-8f2103121f21", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f651c0ac010 con 0x7f651c14acf0 2024-09-13T16:47:36.712 INFO:teuthology.orchestra.run.smithi084.stdout:5 2024-09-13T16:47:36.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f652458c640 1 -- 172.21.15.84:0/2215296540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c14acf0 msgr2=0x7f651c14cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f652458c640 1 --2- 172.21.15.84:0/2215296540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c14acf0 0x7f651c14cd50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f651400c6c0 tx=0x7f651400cb80 comp rx=0 tx=0).stop 2024-09-13T16:47:36.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f652458c640 1 -- 172.21.15.84:0/2215296540 shutdown_connections 2024-09-13T16:47:36.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f652458c640 1 --2- 172.21.15.84:0/2215296540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f651c14acf0 0x7f651c14cd50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f652458c640 1 --2- 172.21.15.84:0/2215296540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0c4b70 0x7f651c14e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f652458c640 1 --2- 172.21.15.84:0/2215296540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f651c0c4710 0x7f651c1479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f652458c640 1 -- 172.21.15.84:0/2215296540 >> 172.21.15.84:0/2215296540 conn(0x7f651c0c32a0 msgr2=0x7f651c070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:36.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f652458c640 1 -- 172.21.15.84:0/2215296540 shutdown_connections 2024-09-13T16:47:36.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.708+0000 7f652458c640 1 -- 172.21.15.84:0/2215296540 wait complete. 2024-09-13T16:47:36.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new f67525eb-b919-4f97-b0bf-9f45ffe8f709 6 2024-09-13T16:47:36.750 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:36.744+0000 7f79ad3ae180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T16:47:36.812 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:36.810+0000 7f4574214180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T16:47:36.830 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:36.828+0000 7f79ad3ae180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T16:47:36.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.840+0000 7f12d0362640 1 Processor -- start 2024-09-13T16:47:36.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.840+0000 7f12d0362640 1 -- start start 2024-09-13T16:47:36.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.840+0000 7f12d0362640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c80c4e80 0x7f12c8151c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.840+0000 7f12d0362640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f12c81521a0 0x7f12c815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.840+0000 7f12d0362640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c80c4730 con 0x7f12c814f5a0 2024-09-13T16:47:36.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.840+0000 7f12d0362640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c80c0ba0 con 0x7f12c81521a0 2024-09-13T16:47:36.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.840+0000 7f12d0362640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c8152650 con 0x7f12c80c4e80 2024-09-13T16:47:36.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd8fd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c80c4e80 0x7f12c8151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd8fd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c80c4e80 0x7f12c8151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.84:40878/0 (socket says 172.21.15.84:40878) 2024-09-13T16:47:36.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd8fd640 1 -- 172.21.15.84:0/1893916604 learned_addr learned my addr 172.21.15.84:0/1893916604 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:36.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12ce0fe640 1 --1- 172.21.15.84:0/1893916604 >> v1:172.21.15.84:6789/0 conn(0x7f12c814f5a0 0x7f12c80c53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:35810/0 (socket says 172.21.15.84:35810) 2024-09-13T16:47:36.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12ce8ff640 1 --2- 172.21.15.84:0/1893916604 >> v2:172.21.15.39:3300/0 conn(0x7f12c81521a0 0x7f12c815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd8fd640 1 -- 172.21.15.84:0/1893916604 >> v2:172.21.15.39:3300/0 conn(0x7f12c81521a0 msgr2=0x7f12c815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:36.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd8fd640 1 --2- 172.21.15.84:0/1893916604 >> v2:172.21.15.39:3300/0 conn(0x7f12c81521a0 0x7f12c815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd8fd640 1 -- 172.21.15.84:0/1893916604 >> v1:172.21.15.84:6789/0 conn(0x7f12c814f5a0 legacy=0x7f12c80c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd8fd640 1 -- 172.21.15.84:0/1893916604 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c80c0e60 con 0x7f12c80c4e80 2024-09-13T16:47:36.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12ce8ff640 1 --2- 172.21.15.84:0/1893916604 >> v2:172.21.15.39:3300/0 conn(0x7f12c81521a0 0x7f12c815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:47:36.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd8fd640 1 --2- 172.21.15.84:0/1893916604 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c80c4e80 0x7f12c8151c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009870 tx=0x7f12b8023160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d4fe5899bb21da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:36.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd0fc640 1 -- 172.21.15.84:0/1893916604 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80ace60 con 0x7f12c80c4e80 2024-09-13T16:47:36.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd0fc640 1 -- 172.21.15.84:0/1893916604 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b80b4440 con 0x7f12c80c4e80 2024-09-13T16:47:36.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12cd0fc640 1 -- 172.21.15.84:0/1893916604 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80b47d0 con 0x7f12c80c4e80 2024-09-13T16:47:36.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12d0362640 1 -- 172.21.15.84:0/1893916604 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c80c4e80 msgr2=0x7f12c8151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.844+0000 7f12d0362640 1 --2- 172.21.15.84:0/1893916604 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c80c4e80 0x7f12c8151c60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009870 tx=0x7f12b8023160 comp rx=0 tx=0).stop 2024-09-13T16:47:36.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- 172.21.15.84:0/1893916604 shutdown_connections 2024-09-13T16:47:36.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 --2- 172.21.15.84:0/1893916604 >> v2:172.21.15.39:3300/0 conn(0x7f12c81521a0 0x7f12c815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 --2- 172.21.15.84:0/1893916604 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c80c4e80 0x7f12c8151c60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- 172.21.15.84:0/1893916604 >> 172.21.15.84:0/1893916604 conn(0x7f12c80c32a0 msgr2=0x7f12c80c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:36.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- 172.21.15.84:0/1893916604 shutdown_connections 2024-09-13T16:47:36.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- 172.21.15.84:0/1893916604 wait complete. 2024-09-13T16:47:36.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 Processor -- start 2024-09-13T16:47:36.852 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- start start 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c80c4e80 0x7f12c80abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c814f5a0 0x7f12c81e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c81521a0 0x7f12c81e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c8149080 con 0x7f12c80c4e80 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c814c4e0 con 0x7f12c814f5a0 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c8148b40 con 0x7f12c81521a0 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12ce8ff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c81521a0 0x7f12c81e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12cd8fd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c814f5a0 0x7f12c81e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12ce8ff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c81521a0 0x7f12c81e91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.84:40880/0 (socket says 172.21.15.84:40880) 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12ce8ff640 1 -- 172.21.15.84:0/2733202553 learned_addr learned my addr 172.21.15.84:0/2733202553 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:36.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12ce0fe640 1 --2- 172.21.15.84:0/2733202553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c80c4e80 0x7f12c80abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12cd8fd640 1 -- 172.21.15.84:0/2733202553 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c81521a0 msgr2=0x7f12c81e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12cd8fd640 1 --2- 172.21.15.84:0/2733202553 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c81521a0 0x7f12c81e91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12cd8fd640 1 -- 172.21.15.84:0/2733202553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c80c4e80 msgr2=0x7f12c80abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12cd8fd640 1 --2- 172.21.15.84:0/2733202553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c80c4e80 0x7f12c80abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12cd8fd640 1 -- 172.21.15.84:0/2733202553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80094d0 con 0x7f12c814f5a0 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12ce8ff640 1 --2- 172.21.15.84:0/2733202553 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c81521a0 0x7f12c81e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12cd8fd640 1 --2- 172.21.15.84:0/2733202553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c814f5a0 0x7f12c81e8cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f12b80b3000 tx=0x7f12b801ff00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12beffd640 1 -- 172.21.15.84:0/2733202553 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80bf670 con 0x7f12c814f5a0 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12beffd640 1 -- 172.21.15.84:0/2733202553 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b80bf7d0 con 0x7f12c814f5a0 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- 172.21.15.84:0/2733202553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c8148da0 con 0x7f12c814f5a0 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12beffd640 1 -- 172.21.15.84:0/2733202553 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80bfb80 con 0x7f12c814f5a0 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- 172.21.15.84:0/2733202553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c81e9840 con 0x7f12c814f5a0 2024-09-13T16:47:36.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12beffd640 1 -- 172.21.15.84:0/2733202553 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f12b80bb070 con 0x7f12c814f5a0 2024-09-13T16:47:36.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12beffd640 1 -- 172.21.15.84:0/2733202553 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f12b80bd420 con 0x7f12c814f5a0 2024-09-13T16:47:36.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.848+0000 7f12d0362640 1 -- 172.21.15.84:0/2733202553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c80ac7b0 con 0x7f12c814f5a0 2024-09-13T16:47:36.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.852+0000 7f12beffd640 1 -- 172.21.15.84:0/2733202553 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f12c80ac7b0 con 0x7f12c814f5a0 2024-09-13T16:47:36.875 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:36.874+0000 7f4574214180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T16:47:36.885 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:36.880+0000 7f79ad3ae180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T16:47:36.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.916+0000 7f12d0362640 1 -- 172.21.15.84:0/2733202553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f67525eb-b919-4f97-b0bf-9f45ffe8f709", "id": 6} v 0) v1 -- 0x7f12c81e9a30 con 0x7f12c814f5a0 2024-09-13T16:47:36.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12beffd640 1 -- 172.21.15.84:0/2733202553 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f67525eb-b919-4f97-b0bf-9f45ffe8f709", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f12b80b9070 con 0x7f12c814f5a0 2024-09-13T16:47:36.927 INFO:teuthology.orchestra.run.smithi084.stdout:6 2024-09-13T16:47:36.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12d0362640 1 -- 172.21.15.84:0/2733202553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c814f5a0 msgr2=0x7f12c81e8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:36.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12d0362640 1 --2- 172.21.15.84:0/2733202553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c814f5a0 0x7f12c81e8cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f12b80b3000 tx=0x7f12b801ff00 comp rx=0 tx=0).stop 2024-09-13T16:47:36.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12d0362640 1 -- 172.21.15.84:0/2733202553 shutdown_connections 2024-09-13T16:47:36.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12d0362640 1 --2- 172.21.15.84:0/2733202553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12c814f5a0 0x7f12c81e8cb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12d0362640 1 --2- 172.21.15.84:0/2733202553 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12c81521a0 0x7f12c81e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12d0362640 1 --2- 172.21.15.84:0/2733202553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c80c4e80 0x7f12c80abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:36.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12d0362640 1 -- 172.21.15.84:0/2733202553 >> 172.21.15.84:0/2733202553 conn(0x7f12c80c32a0 msgr2=0x7f12c80c20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:36.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12d0362640 1 -- 172.21.15.84:0/2733202553 shutdown_connections 2024-09-13T16:47:36.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:36.924+0000 7f12d0362640 1 -- 172.21.15.84:0/2733202553 wait complete. 2024-09-13T16:47:36.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd new d525e6b6-f481-4193-9f63-888925559396 7 2024-09-13T16:47:36.965 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:36.960+0000 7f79ad3ae180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T16:47:37.018 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:37.012+0000 7f79ad3ae180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T16:47:37.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76f82f640 1 Processor -- start 2024-09-13T16:47:37.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76f82f640 1 -- start start 2024-09-13T16:47:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76f82f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa7680c2690 0x7fa76806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76f82f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa7680c6290 0x7fa7681577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76f82f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa768157330 con 0x7fa7680c0be0 2024-09-13T16:47:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76f82f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7680c4a20 con 0x7fa7680c6290 2024-09-13T16:47:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76f82f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7680c4740 con 0x7fa7680c2690 2024-09-13T16:47:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76d5cb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa7680c2690 0x7fa76806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76d5cb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa7680c2690 0x7fa76806ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39150/0 (socket says 172.21.15.84:39150) 2024-09-13T16:47:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76d5cb640 1 -- 172.21.15.84:0/2860146625 learned_addr learned my addr 172.21.15.84:0/2860146625 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:37.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76d5cb640 1 -- 172.21.15.84:0/2860146625 >> v2:172.21.15.39:3300/0 conn(0x7fa7680c6290 msgr2=0x7fa7681577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:37.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76cdca640 1 --2- 172.21.15.84:0/2860146625 >> v2:172.21.15.39:3300/0 conn(0x7fa7680c6290 0x7fa7681577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76d5cb640 1 --2- 172.21.15.84:0/2860146625 >> v2:172.21.15.39:3300/0 conn(0x7fa7680c6290 0x7fa7681577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76d5cb640 1 -- 172.21.15.84:0/2860146625 >> v1:172.21.15.39:6789/0 conn(0x7fa7680c0be0 legacy=0x7fa768065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:37.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.072+0000 7fa76d5cb640 1 -- 172.21.15.84:0/2860146625 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768154880 con 0x7fa7680c2690 2024-09-13T16:47:37.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76d5cb640 1 --2- 172.21.15.84:0/2860146625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 0x7fa76806ae20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa768066d20 tx=0x7fa76401ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfef5408f331b7ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:37.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa75ffff640 1 -- 172.21.15.84:0/2860146625 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640ab640 con 0x7fa7680c2690 2024-09-13T16:47:37.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa75ffff640 1 -- 172.21.15.84:0/2860146625 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7640a7070 con 0x7fa7680c2690 2024-09-13T16:47:37.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa75ffff640 1 -- 172.21.15.84:0/2860146625 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640ab9c0 con 0x7fa7680c2690 2024-09-13T16:47:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 -- 172.21.15.84:0/2860146625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 msgr2=0x7fa76806ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 --2- 172.21.15.84:0/2860146625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 0x7fa76806ae20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa768066d20 tx=0x7fa76401ce80 comp rx=0 tx=0).stop 2024-09-13T16:47:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 -- 172.21.15.84:0/2860146625 shutdown_connections 2024-09-13T16:47:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 --2- 172.21.15.84:0/2860146625 >> v2:172.21.15.39:3300/0 conn(0x7fa7680c6290 0x7fa7681577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 --2- 172.21.15.84:0/2860146625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 0x7fa76806ae20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 -- 172.21.15.84:0/2860146625 >> 172.21.15.84:0/2860146625 conn(0x7fa7680c32a0 msgr2=0x7fa7680c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 -- 172.21.15.84:0/2860146625 shutdown_connections 2024-09-13T16:47:37.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 -- 172.21.15.84:0/2860146625 wait complete. 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 Processor -- start 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.076+0000 7fa76f82f640 1 -- start start 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76f82f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa7680c0be0 0x7fa768071a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76f82f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 0x7fa7681ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76f82f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7680c6290 0x7fa7681550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76f82f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa768157050 con 0x7fa7680c6290 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76f82f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7681eccf0 con 0x7fa7680c2690 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76f82f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa768071fa0 con 0x7fa7680c0be0 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76cdca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 0x7fa7681ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76d5cb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa7680c0be0 0x7fa768071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76cdca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 0x7fa7681ed700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39158/0 (socket says 172.21.15.84:39158) 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76cdca640 1 -- 172.21.15.84:0/3333127965 learned_addr learned my addr 172.21.15.84:0/3333127965 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76ddcc640 1 --2- 172.21.15.84:0/3333127965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7680c6290 0x7fa7681550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76cdca640 1 -- 172.21.15.84:0/3333127965 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa7680c0be0 msgr2=0x7fa768071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76cdca640 1 --2- 172.21.15.84:0/3333127965 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa7680c0be0 0x7fa768071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76cdca640 1 -- 172.21.15.84:0/3333127965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7680c6290 msgr2=0x7fa7681550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76cdca640 1 --2- 172.21.15.84:0/3333127965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7680c6290 0x7fa7681550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76cdca640 1 -- 172.21.15.84:0/3333127965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7640094d0 con 0x7fa7680c2690 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76ddcc640 1 --2- 172.21.15.84:0/3333127965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7680c6290 0x7fa7681550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76cdca640 1 --2- 172.21.15.84:0/3333127965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 0x7fa7681ed700 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c7f0 tx=0x7fa75800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa75dffb640 1 -- 172.21.15.84:0/3333127965 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758004220 con 0x7fa7680c2690 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa75dffb640 1 -- 172.21.15.84:0/3333127965 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7580043a0 con 0x7fa7680c2690 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76f82f640 1 -- 172.21.15.84:0/3333127965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7681edc40 con 0x7fa7680c2690 2024-09-13T16:47:37.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa75dffb640 1 -- 172.21.15.84:0/3333127965 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758004510 con 0x7fa7680c2690 2024-09-13T16:47:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76f82f640 1 -- 172.21.15.84:0/3333127965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa768156a90 con 0x7fa7680c2690 2024-09-13T16:47:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa75dffb640 1 -- 172.21.15.84:0/3333127965 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fa758003860 con 0x7fa7680c2690 2024-09-13T16:47:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa75dffb640 1 -- 172.21.15.84:0/3333127965 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fa758007bc0 con 0x7fa7680c2690 2024-09-13T16:47:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.080+0000 7fa76f82f640 1 -- 172.21.15.84:0/3333127965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76806ae20 con 0x7fa7680c2690 2024-09-13T16:47:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.084+0000 7fa75dffb640 1 -- 172.21.15.84:0/3333127965 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fa75801e060 con 0x7fa7680c2690 2024-09-13T16:47:37.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.160+0000 7fa76f82f640 1 -- 172.21.15.84:0/3333127965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d525e6b6-f481-4193-9f63-888925559396", "id": 7} v 0) v1 -- 0x7fa7680c0470 con 0x7fa7680c2690 2024-09-13T16:47:37.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa75dffb640 1 -- 172.21.15.84:0/3333127965 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d525e6b6-f481-4193-9f63-888925559396", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa758015070 con 0x7fa7680c2690 2024-09-13T16:47:37.172 INFO:teuthology.orchestra.run.smithi084.stdout:7 2024-09-13T16:47:37.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa76f82f640 1 -- 172.21.15.84:0/3333127965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 msgr2=0x7fa7681ed700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa76f82f640 1 --2- 172.21.15.84:0/3333127965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 0x7fa7681ed700 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c7f0 tx=0x7fa75800ccb0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa76f82f640 1 -- 172.21.15.84:0/3333127965 shutdown_connections 2024-09-13T16:47:37.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa76f82f640 1 --2- 172.21.15.84:0/3333127965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7680c2690 0x7fa7681ed700 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa76f82f640 1 --2- 172.21.15.84:0/3333127965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7680c6290 0x7fa7681550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa76f82f640 1 --2- 172.21.15.84:0/3333127965 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa7680c0be0 0x7fa768071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa76f82f640 1 -- 172.21.15.84:0/3333127965 >> 172.21.15.84:0/3333127965 conn(0x7fa7680c32a0 msgr2=0x7fa768065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:37.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa76f82f640 1 -- 172.21.15.84:0/3333127965 shutdown_connections 2024-09-13T16:47:37.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-09-13T16:47:37.168+0000 7fa76f82f640 1 -- 172.21.15.84:0/3333127965 wait complete. 2024-09-13T16:47:37.188 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T16:47:37.188 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T16:47:37.191 INFO:tasks.ceph.osd.0:Started 2024-09-13T16:47:37.191 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T16:47:37.191 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T16:47:37.192 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:37.188+0000 7f79ad3ae180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T16:47:37.193 INFO:tasks.ceph.osd.1:Started 2024-09-13T16:47:37.194 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T16:47:37.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T16:47:37.196 INFO:tasks.ceph.osd.2:Started 2024-09-13T16:47:37.196 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T16:47:37.197 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T16:47:37.199 INFO:tasks.ceph.osd.3:Started 2024-09-13T16:47:37.199 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T16:47:37.199 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T16:47:37.201 INFO:tasks.ceph.osd.4:Started 2024-09-13T16:47:37.201 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T16:47:37.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T16:47:37.203 INFO:tasks.ceph.osd.5:Started 2024-09-13T16:47:37.203 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T16:47:37.204 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T16:47:37.206 INFO:tasks.ceph.osd.6:Started 2024-09-13T16:47:37.206 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T16:47:37.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T16:47:37.208 INFO:tasks.ceph.osd.7:Started 2024-09-13T16:47:37.208 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:47:37.261 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:37.256+0000 7f79ad3ae180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T16:47:37.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb983c0a640 1 Processor -- start 2024-09-13T16:47:37.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb983c0a640 1 -- start start 2024-09-13T16:47:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb983c0a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c155910 0x7fb97c0c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb983c0a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c14f920 con 0x7fb97c0c1ab0 2024-09-13T16:47:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb983c0a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c0c1780 con 0x7fb97c14f540 2024-09-13T16:47:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb983c0a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c0709c0 con 0x7fb97c155910 2024-09-13T16:47:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9811a5640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb97c0c1ab0 0x7fb97c155200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38030/0 (socket says 172.21.15.39:38030) 2024-09-13T16:47:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9811a5640 1 -- 172.21.15.39:0/3436062103 learned_addr learned my addr 172.21.15.39:0/3436062103 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:47:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9821a7640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c155910 0x7fb97c0c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211596697 0 0) 0x7fb97c14f920 con 0x7fb97c0c1ab0 2024-09-13T16:47:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb968003530 con 0x7fb97c0c1ab0 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1923546829 0 0) 0x7fb968003530 con 0x7fb97c0c1ab0 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c155910 msgr2=0x7fb97c0c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c155910 0x7fb97c0c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 >> v1:172.21.15.84:6789/0 conn(0x7fb97c14f540 legacy=0x7fb97c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c071580 con 0x7fb97c0c1ab0 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fb96c001680 con 0x7fb97c0c1ab0 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 >> v1:172.21.15.39:6789/0 conn(0x7fb97c0c1ab0 legacy=0x7fb97c155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9680039b0 0x7fb968003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb968004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c155910 0x7fb968005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96800ba40 con 0x7fb9680047d0 2024-09-13T16:47:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96800bb80 con 0x7fb9680039b0 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96800bd40 con 0x7fb97c155910 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.378+0000 7fb9811a5640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb968004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9819a6640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9680039b0 0x7fb968003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9821a7640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c155910 0x7fb968005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9811a5640 1 -- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c155910 msgr2=0x7fb968005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9811a5640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c155910 0x7fb968005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9811a5640 1 -- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9680039b0 msgr2=0x7fb968003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9811a5640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9680039b0 0x7fb968003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9811a5640 1 -- 172.21.15.39:0/3436062103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb968003530 con 0x7fb9680047d0 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9821a7640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c155910 0x7fb968005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:47:37.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9819a6640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9680039b0 0x7fb968003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:47:37.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9811a5640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb968004c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00c210 tx=0x7fb96c01f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1e1c199517e78f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c005ae0 con 0x7fb9680047d0 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c0ab070 con 0x7fb9680047d0 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9809a4640 1 -- 172.21.15.39:0/3436062103 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c007070 con 0x7fb9680047d0 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb983c0a640 1 -- 172.21.15.39:0/3436062103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 msgr2=0x7fb968004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb983c0a640 1 --2- 172.21.15.39:0/3436062103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb968004c50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00c210 tx=0x7fb96c01f370 comp rx=0 tx=0).stop 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb9821a7640 1 -- 172.21.15.39:0/3436062103 reap_dead start 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb983c0a640 1 -- 172.21.15.39:0/3436062103 shutdown_connections 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb983c0a640 1 -- 172.21.15.39:0/3436062103 >> 172.21.15.39:0/3436062103 conn(0x7fb97c0c32a0 msgr2=0x7fb97c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb983c0a640 1 -- 172.21.15.39:0/3436062103 shutdown_connections 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.382+0000 7fb983c0a640 1 -- 172.21.15.39:0/3436062103 wait complete. 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 Processor -- start 2024-09-13T16:47:37.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 -- start start 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb97c0bcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97c1ddde0 0x7fb97c1dc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c0bc0e0 0x7fb97c0bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1dbcb0 con 0x7fb9680047d0 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1dca00 con 0x7fb97c1ddde0 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1e8640 con 0x7fb97c0bc0e0 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9819a6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb97c0bcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9819a6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb97c0bcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45196/0 (socket says 172.21.15.39:45196) 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9819a6640 1 -- 172.21.15.39:0/4144258077 learned_addr learned my addr 172.21.15.39:0/4144258077 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9811a5640 1 --2- 172.21.15.39:0/4144258077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97c1ddde0 0x7fb97c1dc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9821a7640 1 --2- 172.21.15.39:0/4144258077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c0bc0e0 0x7fb97c0bfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9819a6640 1 -- 172.21.15.39:0/4144258077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c0bc0e0 msgr2=0x7fb97c0bfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9819a6640 1 --2- 172.21.15.39:0/4144258077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c0bc0e0 0x7fb97c0bfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9819a6640 1 -- 172.21.15.39:0/4144258077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97c1ddde0 msgr2=0x7fb97c1dc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9819a6640 1 --2- 172.21.15.39:0/4144258077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97c1ddde0 0x7fb97c1dc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9819a6640 1 -- 172.21.15.39:0/4144258077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c1e82f0 con 0x7fb9680047d0 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9819a6640 1 --2- 172.21.15.39:0/4144258077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb97c0bcf50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb97800e4d0 tx=0x7fb97800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9727fc640 1 -- 172.21.15.39:0/4144258077 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9780052e0 con 0x7fb9680047d0 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9727fc640 1 -- 172.21.15.39:0/4144258077 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb978005440 con 0x7fb9680047d0 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9727fc640 1 -- 172.21.15.39:0/4144258077 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978003820 con 0x7fb9680047d0 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 -- 172.21.15.39:0/4144258077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c1db4d0 con 0x7fb9680047d0 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 -- 172.21.15.39:0/4144258077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c1dc6e0 con 0x7fb9680047d0 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9727fc640 1 -- 172.21.15.39:0/4144258077 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb978003c80 con 0x7fb9680047d0 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb9727fc640 1 -- 172.21.15.39:0/4144258077 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb97801dcd0 con 0x7fb9680047d0 2024-09-13T16:47:37.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.386+0000 7fb983c0a640 1 -- 172.21.15.39:0/4144258077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c0619c0 con 0x7fb9680047d0 2024-09-13T16:47:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.398+0000 7fb9727fc640 1 -- 172.21.15.39:0/4144258077 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fb97801d460 con 0x7fb9680047d0 2024-09-13T16:47:37.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.450+0000 7fb983c0a640 1 -- 172.21.15.39:0/4144258077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb97c1db710 con 0x7fb9680047d0 2024-09-13T16:47:37.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb9727fc640 1 -- 172.21.15.39:0/4144258077 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb97801d600 con 0x7fb9680047d0 2024-09-13T16:47:37.455 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:47:37.455 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:47:37.169194+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:47:37.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb983c0a640 1 -- 172.21.15.39:0/4144258077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 msgr2=0x7fb97c0bcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:37.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb983c0a640 1 --2- 172.21.15.39:0/4144258077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb97c0bcf50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb97800e4d0 tx=0x7fb97800e990 comp rx=0 tx=0).stop 2024-09-13T16:47:37.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb983c0a640 1 -- 172.21.15.39:0/4144258077 shutdown_connections 2024-09-13T16:47:37.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb983c0a640 1 --2- 172.21.15.39:0/4144258077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97c1ddde0 0x7fb97c1dc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb983c0a640 1 --2- 172.21.15.39:0/4144258077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb97c0bc0e0 0x7fb97c0bfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb983c0a640 1 --2- 172.21.15.39:0/4144258077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9680047d0 0x7fb97c0bcf50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:37.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb983c0a640 1 -- 172.21.15.39:0/4144258077 >> 172.21.15.39:0/4144258077 conn(0x7fb97c0c32a0 msgr2=0x7fb97c0c5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:37.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb983c0a640 1 -- 172.21.15.39:0/4144258077 shutdown_connections 2024-09-13T16:47:37.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:37.454+0000 7fb983c0a640 1 -- 172.21.15.39:0/4144258077 wait complete. 2024-09-13T16:47:37.470 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T16:47:37.471 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T16:47:37.532 INFO:tasks.ceph.mgr.y.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T16:47:37.532 INFO:tasks.ceph.mgr.y.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T16:47:37.532 INFO:tasks.ceph.mgr.y.smithi039.stderr: from numpy import show_config as show_numpy_config 2024-09-13T16:47:37.593 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:37.590+0000 7f4574214180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T16:47:37.765 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:37.762+0000 7f4574214180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:47:37.808 INFO:tasks.ceph.osd.4.smithi084.stderr:2024-09-13T16:47:37.804+0000 7f0b6a44d7c0 -1 Falling back to public interface 2024-09-13T16:47:37.823 INFO:tasks.ceph.osd.5.smithi084.stderr:2024-09-13T16:47:37.820+0000 7f98c2e877c0 -1 Falling back to public interface 2024-09-13T16:47:37.823 INFO:tasks.ceph.osd.6.smithi084.stderr:2024-09-13T16:47:37.820+0000 7f2964fd87c0 -1 Falling back to public interface 2024-09-13T16:47:37.844 INFO:tasks.ceph.osd.7.smithi084.stderr:2024-09-13T16:47:37.840+0000 7fdc9b8187c0 -1 Falling back to public interface 2024-09-13T16:47:37.844 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-13T16:47:37.842+0000 7f094a4f47c0 -1 Falling back to public interface 2024-09-13T16:47:37.862 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-13T16:47:37.862+0000 7f7611c417c0 -1 Falling back to public interface 2024-09-13T16:47:37.867 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-13T16:47:37.866+0000 7f8dfe1647c0 -1 Falling back to public interface 2024-09-13T16:47:37.882 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-13T16:47:37.882+0000 7fc9528dc7c0 -1 Falling back to public interface 2024-09-13T16:47:37.921 INFO:tasks.ceph.mgr.x.smithi084.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T16:47:37.921 INFO:tasks.ceph.mgr.x.smithi084.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T16:47:37.921 INFO:tasks.ceph.mgr.x.smithi084.stderr: from numpy import show_config as show_numpy_config 2024-09-13T16:47:37.933 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:37.930+0000 7f4574214180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T16:47:37.995 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:37.992+0000 7f79ad3ae180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T16:47:38.004 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:38.002+0000 7f4574214180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T16:47:38.154 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:38.154+0000 7f4574214180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:47:38.166 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:38.160+0000 7f79ad3ae180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:47:38.277 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T16:47:38.316 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:38.312+0000 7f79ad3ae180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T16:47:38.378 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:38.376+0000 7f79ad3ae180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T16:47:38.389 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.386+0000 7fe4768c9640 1 Processor -- start 2024-09-13T16:47:38.389 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.386+0000 7fe4768c9640 1 -- start start 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe47014e6a0 0x7fe4700708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe4700c15e0 0x7fe4700c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4700c21a0 con 0x7fe4700a60b0 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe470151b80 con 0x7fe47014e6a0 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4701517e0 con 0x7fe4700c15e0 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe474e66640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe4700a60b0 0x7fe4700c25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38044/0 (socket says 172.21.15.39:38044) 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe474e66640 1 -- 172.21.15.39:0/997449778 learned_addr learned my addr 172.21.15.39:0/997449778 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe47014e6a0 0x7fe4700708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46f7fe640 1 --2- 172.21.15.39:0/997449778 >> v2:172.21.15.84:3300/0 conn(0x7fe4700c15e0 0x7fe4700c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46ffff640 1 -- 172.21.15.39:0/997449778 >> v2:172.21.15.84:3300/0 conn(0x7fe4700c15e0 msgr2=0x7fe4700c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:38.391 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46ffff640 1 --2- 172.21.15.39:0/997449778 >> v2:172.21.15.84:3300/0 conn(0x7fe4700c15e0 0x7fe4700c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:38.392 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46ffff640 1 -- 172.21.15.39:0/997449778 >> v1:172.21.15.39:6789/0 conn(0x7fe4700a60b0 legacy=0x7fe4700c25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:38.392 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46ffff640 1 -- 172.21.15.39:0/997449778 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe470071060 con 0x7fe47014e6a0 2024-09-13T16:47:38.392 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46effd640 1 -- 172.21.15.39:0/997449778 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797724883 0 0) 0x7fe4700c21a0 con 0x7fe4700a60b0 2024-09-13T16:47:38.393 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46ffff640 1 --2- 172.21.15.39:0/997449778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe47014e6a0 0x7fe4700708e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe4700c1e00 tx=0x7fe46001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9ea4b8f428b4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:38.393 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46effd640 1 -- 172.21.15.39:0/997449778 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4600ab600 con 0x7fe47014e6a0 2024-09-13T16:47:38.393 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46effd640 1 -- 172.21.15.39:0/997449778 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4600a7050 con 0x7fe47014e6a0 2024-09-13T16:47:38.393 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe46effd640 1 -- 172.21.15.39:0/997449778 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4600ab980 con 0x7fe47014e6a0 2024-09-13T16:47:38.394 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 -- 172.21.15.39:0/997449778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe47014e6a0 msgr2=0x7fe4700708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:38.394 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 --2- 172.21.15.39:0/997449778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe47014e6a0 0x7fe4700708e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe4700c1e00 tx=0x7fe46001c360 comp rx=0 tx=0).stop 2024-09-13T16:47:38.394 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 -- 172.21.15.39:0/997449778 shutdown_connections 2024-09-13T16:47:38.394 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 --2- 172.21.15.39:0/997449778 >> v2:172.21.15.84:3300/0 conn(0x7fe4700c15e0 0x7fe4700c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:38.394 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 --2- 172.21.15.39:0/997449778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe47014e6a0 0x7fe4700708e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:38.394 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 -- 172.21.15.39:0/997449778 >> 172.21.15.39:0/997449778 conn(0x7fe4700a8630 msgr2=0x7fe4700a8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:38.394 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 -- 172.21.15.39:0/997449778 shutdown_connections 2024-09-13T16:47:38.394 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.390+0000 7fe4768c9640 1 -- 172.21.15.39:0/997449778 wait complete. 2024-09-13T16:47:38.396 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 Processor -- start 2024-09-13T16:47:38.396 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 -- start start 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4700a60b0 0x7fe47006dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4700c15e0 0x7fe4701475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe47014e6a0 0x7fe470147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470148570 con 0x7fe4700c15e0 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700abfd0 con 0x7fe4700a60b0 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700aabe0 con 0x7fe47014e6a0 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4700c15e0 0x7fe4701475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4700c15e0 0x7fe4701475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45216/0 (socket says 172.21.15.39:45216) 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46f7fe640 1 -- 172.21.15.39:0/3912719241 learned_addr learned my addr 172.21.15.39:0/3912719241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:38.397 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46ffff640 1 --2- 172.21.15.39:0/3912719241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4700a60b0 0x7fe47006dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe474e66640 1 --2- 172.21.15.39:0/3912719241 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe47014e6a0 0x7fe470147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46f7fe640 1 -- 172.21.15.39:0/3912719241 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe47014e6a0 msgr2=0x7fe470147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46f7fe640 1 --2- 172.21.15.39:0/3912719241 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe47014e6a0 0x7fe470147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46f7fe640 1 -- 172.21.15.39:0/3912719241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4700a60b0 msgr2=0x7fe47006dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46f7fe640 1 --2- 172.21.15.39:0/3912719241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4700a60b0 0x7fe47006dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46f7fe640 1 -- 172.21.15.39:0/3912719241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe460009420 con 0x7fe4700c15e0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46f7fe640 1 --2- 172.21.15.39:0/3912719241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4700c15e0 0x7fe4701475f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe46400d5f0 tx=0x7fe46400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46cff9640 1 -- 172.21.15.39:0/3912719241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4640125f0 con 0x7fe4700c15e0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46cff9640 1 -- 172.21.15.39:0/3912719241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe464012750 con 0x7fe4700c15e0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 -- 172.21.15.39:0/3912719241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4700ab810 con 0x7fe4700c15e0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46cff9640 1 -- 172.21.15.39:0/3912719241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464012ae0 con 0x7fe4700c15e0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 -- 172.21.15.39:0/3912719241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4700abab0 con 0x7fe4700c15e0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46cff9640 1 -- 172.21.15.39:0/3912719241 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe46401e460 con 0x7fe4700c15e0 2024-09-13T16:47:38.398 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe46cff9640 1 -- 172.21.15.39:0/3912719241 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe464014ba0 con 0x7fe4700c15e0 2024-09-13T16:47:38.399 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.394+0000 7fe4768c9640 1 -- 172.21.15.39:0/3912719241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47006e730 con 0x7fe4700c15e0 2024-09-13T16:47:38.399 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.398+0000 7fe46cff9640 1 -- 172.21.15.39:0/3912719241 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fe464014e60 con 0x7fe4700c15e0 2024-09-13T16:47:38.455 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 -- 172.21.15.39:0/3912719241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4700c4780 con 0x7fe4700c15e0 2024-09-13T16:47:38.455 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe46cff9640 1 -- 172.21.15.39:0/3912719241 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe4640144d0 con 0x7fe4700c15e0 2024-09-13T16:47:38.456 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-13T16:47:38.456 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":10,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:47:37.169194+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:47:38.457 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 -- 172.21.15.39:0/3912719241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4700c15e0 msgr2=0x7fe4701475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:38.457 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 --2- 172.21.15.39:0/3912719241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4700c15e0 0x7fe4701475f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe46400d5f0 tx=0x7fe46400dab0 comp rx=0 tx=0).stop 2024-09-13T16:47:38.457 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 -- 172.21.15.39:0/3912719241 shutdown_connections 2024-09-13T16:47:38.457 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 --2- 172.21.15.39:0/3912719241 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe47014e6a0 0x7fe470147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:38.457 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 --2- 172.21.15.39:0/3912719241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4700c15e0 0x7fe4701475f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:38.457 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 --2- 172.21.15.39:0/3912719241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4700a60b0 0x7fe47006dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:38.457 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 -- 172.21.15.39:0/3912719241 >> 172.21.15.39:0/3912719241 conn(0x7fe4700a8630 msgr2=0x7fe4700c25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:38.457 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 -- 172.21.15.39:0/3912719241 shutdown_connections 2024-09-13T16:47:38.457 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:38.454+0000 7fe4768c9640 1 -- 172.21.15.39:0/3912719241 wait complete. 2024-09-13T16:47:38.470 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T16:47:38.513 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:38.508+0000 7f79ad3ae180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T16:47:38.809 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:38.806+0000 7f4574214180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T16:47:38.888 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:38.886+0000 7f4574214180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T16:47:39.076 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:39.074+0000 7f4574214180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T16:47:39.127 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:39.124+0000 7f79ad3ae180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T16:47:39.152 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:39.150+0000 7f4574214180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T16:47:39.202 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:39.196+0000 7f79ad3ae180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T16:47:39.381 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:39.376+0000 7f79ad3ae180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T16:47:39.455 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:39.452+0000 7f79ad3ae180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T16:47:39.460 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:39.458+0000 7f4574214180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T16:47:39.524 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:39.522+0000 7f4574214180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T16:47:39.751 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:39.748+0000 7f79ad3ae180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T16:47:39.814 INFO:tasks.ceph.mgr.x.smithi084.stderr:2024-09-13T16:47:39.808+0000 7f79ad3ae180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T16:47:40.567 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:40.566+0000 7f451b6c9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T16:47:40.568 INFO:tasks.ceph.mgr.y.smithi039.stderr:2024-09-13T16:47:40.566+0000 7f451b6c9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T16:47:41.694 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-13T16:47:41.694+0000 7fc9528dc7c0 -1 osd.3 0 log_to_monitors true 2024-09-13T16:47:41.722 INFO:tasks.ceph.osd.4.smithi084.stderr:2024-09-13T16:47:41.720+0000 7f0b6a44d7c0 -1 osd.4 0 log_to_monitors true 2024-09-13T16:47:41.729 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-13T16:47:41.726+0000 7f8dfe1647c0 -1 osd.2 0 log_to_monitors true 2024-09-13T16:47:41.744 INFO:tasks.ceph.osd.6.smithi084.stderr:2024-09-13T16:47:41.740+0000 7f2964fd87c0 -1 osd.6 0 log_to_monitors true 2024-09-13T16:47:41.753 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-13T16:47:41.750+0000 7f7611c417c0 -1 osd.1 0 log_to_monitors true 2024-09-13T16:47:41.761 INFO:tasks.ceph.osd.5.smithi084.stderr:2024-09-13T16:47:41.756+0000 7f98c2e877c0 -1 osd.5 0 log_to_monitors true 2024-09-13T16:47:41.839 INFO:tasks.ceph.osd.7.smithi084.stderr:2024-09-13T16:47:41.836+0000 7fdc9b8187c0 -1 osd.7 0 log_to_monitors true 2024-09-13T16:47:41.844 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-13T16:47:41.842+0000 7f094a4f47c0 -1 osd.0 0 log_to_monitors true 2024-09-13T16:47:45.277 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T16:47:45.476 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58ed56640 1 Processor -- start 2024-09-13T16:47:45.476 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58ed56640 1 -- start start 2024-09-13T16:47:45.477 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58ed56640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc5880c8120 0x7fc5880c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:45.477 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58ed56640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc58815c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:45.477 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58ed56640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58815bdb0 con 0x7fc588158870 2024-09-13T16:47:45.477 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58ed56640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc58815bab0 con 0x7fc5880c8120 2024-09-13T16:47:45.478 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58ed56640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880c2b70 con 0x7fc5880c2400 2024-09-13T16:47:45.478 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58caf2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc5880c8120 0x7fc5880c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:45.478 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58caf2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc5880c8120 0x7fc5880c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50742/0 (socket says 172.21.15.39:50742) 2024-09-13T16:47:45.479 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.474+0000 7fc58caf2640 1 -- 172.21.15.39:0/2471945395 learned_addr learned my addr 172.21.15.39:0/2471945395 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:45.479 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.478+0000 7fc587fff640 1 --2- 172.21.15.39:0/2471945395 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc58815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:45.479 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.478+0000 7fc58caf2640 1 -- 172.21.15.39:0/2471945395 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 msgr2=0x7fc58815c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:45.479 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.478+0000 7fc58caf2640 1 --2- 172.21.15.39:0/2471945395 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc58815c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.479 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.478+0000 7fc58caf2640 1 -- 172.21.15.39:0/2471945395 >> v1:172.21.15.39:6789/0 conn(0x7fc588158870 legacy=0x7fc588158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:45.479 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.478+0000 7fc58caf2640 1 -- 172.21.15.39:0/2471945395 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5880c2e50 con 0x7fc5880c8120 2024-09-13T16:47:45.482 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc58caf2640 1 --2- 172.21.15.39:0/2471945395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5880c8120 0x7fc5880c5900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc580009870 tx=0x7fc58001c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e37cb6c894185d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:45.483 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc587fff640 1 --2- 172.21.15.39:0/2471945395 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc58815c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:47:45.483 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc5877fe640 1 -- 172.21.15.39:0/2471945395 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5800aa5f0 con 0x7fc5880c8120 2024-09-13T16:47:45.484 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc5877fe640 1 -- 172.21.15.39:0/2471945395 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5800aa750 con 0x7fc5880c8120 2024-09-13T16:47:45.484 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc58ed56640 1 -- 172.21.15.39:0/2471945395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5880c8120 msgr2=0x7fc5880c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:45.484 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc58ed56640 1 --2- 172.21.15.39:0/2471945395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5880c8120 0x7fc5880c5900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc580009870 tx=0x7fc58001c580 comp rx=0 tx=0).stop 2024-09-13T16:47:45.485 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc58ed56640 1 -- 172.21.15.39:0/2471945395 shutdown_connections 2024-09-13T16:47:45.485 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc58ed56640 1 --2- 172.21.15.39:0/2471945395 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc58815c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.485 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc58ed56640 1 --2- 172.21.15.39:0/2471945395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5880c8120 0x7fc5880c5900 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.485 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc58ed56640 1 -- 172.21.15.39:0/2471945395 >> 172.21.15.39:0/2471945395 conn(0x7fc588060dd0 msgr2=0x7fc5880611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:45.485 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc58ed56640 1 -- 172.21.15.39:0/2471945395 shutdown_connections 2024-09-13T16:47:45.485 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.482+0000 7fc58ed56640 1 -- 172.21.15.39:0/2471945395 wait complete. 2024-09-13T16:47:45.487 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58ed56640 1 Processor -- start 2024-09-13T16:47:45.488 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58ed56640 1 -- start start 2024-09-13T16:47:45.491 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58ed56640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc5880ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:45.491 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58ed56640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5880c8120 0x7fc5881564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:45.491 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58ed56640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc588158870 0x7fc5881f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58ed56640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880abef0 con 0x7fc5880c8120 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58ed56640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc588158010 con 0x7fc588158870 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58ed56640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881f51f0 con 0x7fc5880c2400 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58caf2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc5880ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc587fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5880c8120 0x7fc5881564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58caf2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc5880ac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:53802/0 (socket says 172.21.15.39:53802) 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58caf2640 1 -- 172.21.15.39:0/2895380105 learned_addr learned my addr 172.21.15.39:0/2895380105 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58caf2640 1 -- 172.21.15.39:0/2895380105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc588158870 msgr2=0x7fc5881f4560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58caf2640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc588158870 0x7fc5881f4560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58caf2640 1 -- 172.21.15.39:0/2895380105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5880c8120 msgr2=0x7fc5881564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58caf2640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5880c8120 0x7fc5881564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58caf2640 1 -- 172.21.15.39:0/2895380105 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5800094d0 con 0x7fc5880c2400 2024-09-13T16:47:45.492 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.486+0000 7fc58caf2640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc5880ac5d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc5800ac040 tx=0x7fc58001adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:45.493 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc5857fa640 1 -- 172.21.15.39:0/2895380105 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5800aa5f0 con 0x7fc5880c2400 2024-09-13T16:47:45.493 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58806ee90 con 0x7fc5880c2400 2024-09-13T16:47:45.493 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc5857fa640 1 -- 172.21.15.39:0/2895380105 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5800b1c40 con 0x7fc5880c2400 2024-09-13T16:47:45.493 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc5857fa640 1 -- 172.21.15.39:0/2895380105 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5800aa5f0 con 0x7fc5880c2400 2024-09-13T16:47:45.493 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5881576e0 con 0x7fc5880c2400 2024-09-13T16:47:45.493 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5880720c0 con 0x7fc5880c2400 2024-09-13T16:47:45.494 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc5857fa640 1 -- 172.21.15.39:0/2895380105 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7fc5800b33f0 con 0x7fc5880c2400 2024-09-13T16:47:45.494 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc5857fa640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5680666d0 0x7fc568068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:45.494 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc5857fa640 1 -- 172.21.15.39:0/2895380105 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fc5800a7070 con 0x7fc5880c2400 2024-09-13T16:47:45.494 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.490+0000 7fc587fff640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5680666d0 0x7fc568068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:45.494 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.494+0000 7fc587fff640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5680666d0 0x7fc568068b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc5880ab2b0 tx=0x7fc5780077c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:45.501 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.498+0000 7fc5857fa640 1 -- 172.21.15.39:0/2895380105 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc5800ec040 con 0x7fc5880c2400 2024-09-13T16:47:45.655 INFO:tasks.ceph.osd.4.smithi084.stderr:2024-09-13T16:47:45.652+0000 7f0b65bcf640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T16:47:45.679 INFO:tasks.ceph.osd.4.smithi084.stderr:2024-09-13T16:47:45.676+0000 7f0b5e214640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:47:45.699 INFO:tasks.ceph.osd.5.smithi084.stderr:2024-09-13T16:47:45.696+0000 7f98be609640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T16:47:45.731 INFO:tasks.ceph.osd.5.smithi084.stderr:2024-09-13T16:47:45.728+0000 7f98b6c4e640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:47:45.736 INFO:tasks.ceph.osd.7.smithi084.stderr:2024-09-13T16:47:45.732+0000 7fdc96f9a640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T16:47:45.754 INFO:tasks.ceph.osd.6.smithi084.stderr:2024-09-13T16:47:45.752+0000 7f296075a640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T16:47:45.767 INFO:tasks.ceph.osd.7.smithi084.stderr:2024-09-13T16:47:45.764+0000 7fdc8f5df640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:47:45.774 INFO:tasks.ceph.osd.6.smithi084.stderr:2024-09-13T16:47:45.772+0000 7f2958d9f640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:47:45.808 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-13T16:47:45.806+0000 7f0945c76640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T16:47:45.830 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-13T16:47:45.830+0000 7f093e2bb640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:47:45.866 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.866+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc58806f070 con 0x7fc5880c2400 2024-09-13T16:47:45.867 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.866+0000 7fc5857fa640 1 -- 172.21.15.39:0/2895380105 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc5800c3030 con 0x7fc5880c2400 2024-09-13T16:47:45.867 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-13T16:47:45.868 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":12,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:47:43.569881+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:47:45.870 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5680666d0 msgr2=0x7fc568068b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5680666d0 0x7fc568068b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc5880ab2b0 tx=0x7fc5780077c0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 msgr2=0x7fc5880ac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc5880ac5d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc5800ac040 tx=0x7fc58001adc0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 shutdown_connections 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5680666d0 0x7fc568068b90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc588158870 0x7fc5881f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5880c8120 0x7fc5881564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 --2- 172.21.15.39:0/2895380105 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5880c2400 0x7fc5880ac5d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 >> 172.21.15.39:0/2895380105 conn(0x7fc588060dd0 msgr2=0x7fc5880611b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 shutdown_connections 2024-09-13T16:47:45.871 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc58ed56640 1 -- 172.21.15.39:0/2895380105 wait complete. 2024-09-13T16:47:45.872 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-13T16:47:45.870+0000 7fc94e05e640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T16:47:45.890 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T16:47:45.902 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-13T16:47:45.902+0000 7fc9466a3640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:47:45.917 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-13T16:47:45.918+0000 7f8dfb121640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T16:47:45.927 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-13T16:47:45.926+0000 7f760d3c3640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T16:47:45.954 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-13T16:47:45.954+0000 7f8df1f2b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:47:45.970 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-13T16:47:45.970+0000 7f7605a08640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T16:47:52.697 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T16:47:52.825 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.821+0000 7f30e7b15640 1 Processor -- start 2024-09-13T16:47:52.825 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.821+0000 7f30e7b15640 1 -- start start 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e7b15640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f30e0155850 0x7f30e0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e7b15640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 0x7f30e0158e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e7b15640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e0159340 con 0x7f30e00c5fa0 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e7b15640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e00ac4d0 con 0x7f30e0155850 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e7b15640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00abd10 con 0x7f30e00bcd40 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e58b1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f30e0155850 0x7f30e0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e58b1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f30e0155850 0x7f30e0155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49764/0 (socket says 172.21.15.39:49764) 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e58b1640 1 -- 172.21.15.39:0/143836204 learned_addr learned my addr 172.21.15.39:0/143836204 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e50b0640 1 --2- 172.21.15.39:0/143836204 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 0x7f30e0158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e58b1640 1 -- 172.21.15.39:0/143836204 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 msgr2=0x7f30e0158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:52.827 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e58b1640 1 --2- 172.21.15.39:0/143836204 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 0x7f30e0158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:52.828 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e58b1640 1 -- 172.21.15.39:0/143836204 >> v1:172.21.15.84:6789/0 conn(0x7f30e00c5fa0 legacy=0x7f30e006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:52.828 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e58b1640 1 -- 172.21.15.39:0/143836204 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00c43f0 con 0x7f30e0155850 2024-09-13T16:47:52.830 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.825+0000 7f30e58b1640 1 --2- 172.21.15.39:0/143836204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 0x7f30e0155c30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009870 tx=0x7f30dc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c24793e6c96b825 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:52.830 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e48af640 1 -- 172.21.15.39:0/143836204 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0ac640 con 0x7f30e0155850 2024-09-13T16:47:52.830 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e48af640 1 -- 172.21.15.39:0/143836204 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc0ac7a0 con 0x7f30e0155850 2024-09-13T16:47:52.830 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e48af640 1 -- 172.21.15.39:0/143836204 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0acb30 con 0x7f30e0155850 2024-09-13T16:47:52.831 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- 172.21.15.39:0/143836204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 msgr2=0x7f30e0155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:52.831 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 --2- 172.21.15.39:0/143836204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 0x7f30e0155c30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009870 tx=0x7f30dc01c4d0 comp rx=0 tx=0).stop 2024-09-13T16:47:52.831 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- 172.21.15.39:0/143836204 shutdown_connections 2024-09-13T16:47:52.831 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 --2- 172.21.15.39:0/143836204 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 0x7f30e0158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:52.831 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 --2- 172.21.15.39:0/143836204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 0x7f30e0155c30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:52.831 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- 172.21.15.39:0/143836204 >> 172.21.15.39:0/143836204 conn(0x7f30e0060fa0 msgr2=0x7f30e00ab560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:52.831 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- 172.21.15.39:0/143836204 shutdown_connections 2024-09-13T16:47:52.832 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- 172.21.15.39:0/143836204 wait complete. 2024-09-13T16:47:52.832 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 Processor -- start 2024-09-13T16:47:52.832 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- start start 2024-09-13T16:47:52.834 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 0x7f30e015c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:52.834 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30e00c5fa0 0x7f30e00c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:52.834 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 0x7f30e0155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:52.834 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e01f1040 con 0x7f30e0155850 2024-09-13T16:47:52.834 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e015a320 con 0x7f30e00c5fa0 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e015a600 con 0x7f30e00bcd40 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e60b2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 0x7f30e0155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e60b2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 0x7f30e0155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49776/0 (socket says 172.21.15.39:49776) 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e60b2640 1 -- 172.21.15.39:0/1001334323 learned_addr learned my addr 172.21.15.39:0/1001334323 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e58b1640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 0x7f30e015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e50b0640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30e00c5fa0 0x7f30e00c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e60b2640 1 -- 172.21.15.39:0/1001334323 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 msgr2=0x7f30e015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e60b2640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 0x7f30e015c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e60b2640 1 -- 172.21.15.39:0/1001334323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30e00c5fa0 msgr2=0x7f30e00c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e60b2640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30e00c5fa0 0x7f30e00c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e60b2640 1 -- 172.21.15.39:0/1001334323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0094d0 con 0x7f30e0155850 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e60b2640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 0x7f30e0155040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f30d800c770 tx=0x7f30d800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:52.835 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30d67fc640 1 -- 172.21.15.39:0/1001334323 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8003b80 con 0x7f30e0155850 2024-09-13T16:47:52.836 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30d67fc640 1 -- 172.21.15.39:0/1001334323 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d8003ce0 con 0x7f30e0155850 2024-09-13T16:47:52.836 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30d67fc640 1 -- 172.21.15.39:0/1001334323 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d800aa90 con 0x7f30e0155850 2024-09-13T16:47:52.836 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.829+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e015a8e0 con 0x7f30e0155850 2024-09-13T16:47:52.836 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.833+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e00a9e90 con 0x7f30e0155850 2024-09-13T16:47:52.837 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.833+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e01f1240 con 0x7f30e0155850 2024-09-13T16:47:52.837 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.833+0000 7f30d67fc640 1 -- 172.21.15.39:0/1001334323 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f30d800abf0 con 0x7f30e0155850 2024-09-13T16:47:52.837 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.833+0000 7f30d67fc640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30c4066690 0x7f30c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:52.837 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.833+0000 7f30e58b1640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30c4066690 0x7f30c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:52.837 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.833+0000 7f30d67fc640 1 -- 172.21.15.39:0/1001334323 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f30d8083bd0 con 0x7f30e0155850 2024-09-13T16:47:52.837 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.833+0000 7f30e58b1640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30c4066690 0x7f30c4068b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0ae000 tx=0x7f30dc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:52.841 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.837+0000 7f30d67fc640 1 -- 172.21.15.39:0/1001334323 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f30d80512b0 con 0x7f30e0155850 2024-09-13T16:47:53.001 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:52.997+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30e0158e00 con 0x7f30e0155850 2024-09-13T16:47:53.003 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30d67fc640 1 -- 172.21.15.39:0/1001334323 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7f30d80548d0 con 0x7f30e0155850 2024-09-13T16:47:53.003 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-13T16:47:53.003 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":16,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:47:49.714865+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:47:53.005 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30c4066690 msgr2=0x7f30c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.005 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30c4066690 0x7f30c4068b50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0ae000 tx=0x7f30dc0a7000 comp rx=0 tx=0).stop 2024-09-13T16:47:53.005 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 msgr2=0x7f30e0155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.005 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 0x7f30e0155040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f30d800c770 tx=0x7f30d800cc30 comp rx=0 tx=0).stop 2024-09-13T16:47:53.005 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 shutdown_connections 2024-09-13T16:47:53.005 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30c4066690 0x7f30c4068b50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.005 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30e00c5fa0 0x7f30e00c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.006 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30e0155850 0x7f30e0155040 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.006 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 --2- 172.21.15.39:0/1001334323 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30e00bcd40 0x7f30e015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.006 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 >> 172.21.15.39:0/1001334323 conn(0x7f30e0060fa0 msgr2=0x7f30e00c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:53.006 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 shutdown_connections 2024-09-13T16:47:53.006 INFO:teuthology.misc.health.smithi039.stderr:2024-09-13T16:47:53.001+0000 7f30e7b15640 1 -- 172.21.15.39:0/1001334323 wait complete. 2024-09-13T16:47:53.020 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T16:47:53.020 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T16:47:53.020 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:47:53.020 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:47:53.022 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:47:53.022 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T16:47:53.023 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T16:47:53.025 INFO:tasks.ceph.mds.c:Started 2024-09-13T16:47:53.025 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:47:53.025 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:47:53.027 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:47:53.028 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:47:53.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:47:53.030 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:47:53.030 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-13T16:47:53.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:47:53.103 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T16:47:53.104 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T16:47:53.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda8fb0640 1 Processor -- start 2024-09-13T16:47:53.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda8fb0640 1 -- start start 2024-09-13T16:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda8fb0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcda40c6290 0x7fcda41577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda8fb0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcda40c0be0 0x7fcda4157d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda8fb0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda4157330 con 0x7fcda40c2690 2024-09-13T16:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda8fb0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda40c4a20 con 0x7fcda40c6290 2024-09-13T16:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda8fb0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda4158240 con 0x7fcda40c0be0 2024-09-13T16:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda1d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcda40c6290 0x7fcda41577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda259c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcda40c2690 0x7fcda406ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42632/0 (socket says 172.21.15.39:42632) 2024-09-13T16:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda259c640 1 -- 172.21.15.39:0/3127398392 learned_addr learned my addr 172.21.15.39:0/3127398392 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda1d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcda40c6290 0x7fcda41577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49816/0 (socket says 172.21.15.39:49816) 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda1d9b640 1 -- 172.21.15.39:0/3127398392 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcda40c0be0 msgr2=0x7fcda4157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda2d9d640 1 --2- 172.21.15.39:0/3127398392 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcda40c0be0 0x7fcda4157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda159a640 1 -- 172.21.15.39:0/3127398392 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411542186 0 0) 0x7fcda4157330 con 0x7fcda40c2690 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda1d9b640 1 --2- 172.21.15.39:0/3127398392 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcda40c0be0 0x7fcda4157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda1d9b640 1 -- 172.21.15.39:0/3127398392 >> v1:172.21.15.39:6789/0 conn(0x7fcda40c2690 legacy=0x7fcda406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.145+0000 7fcda1d9b640 1 -- 172.21.15.39:0/3127398392 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda40c4740 con 0x7fcda40c6290 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda1d9b640 1 --2- 172.21.15.39:0/3127398392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 0x7fcda41577c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd980097c0 tx=0x7fcd9801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169c5b348bde9385 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda159a640 1 -- 172.21.15.39:0/3127398392 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd980ac640 con 0x7fcda40c6290 2024-09-13T16:47:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda159a640 1 -- 172.21.15.39:0/3127398392 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd980ac7a0 con 0x7fcda40c6290 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda159a640 1 -- 172.21.15.39:0/3127398392 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd980acb30 con 0x7fcda40c6290 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3127398392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 msgr2=0x7fcda41577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 --2- 172.21.15.39:0/3127398392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 0x7fcda41577c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd980097c0 tx=0x7fcd9801c3e0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3127398392 shutdown_connections 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 --2- 172.21.15.39:0/3127398392 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcda40c0be0 0x7fcda4157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 --2- 172.21.15.39:0/3127398392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 0x7fcda41577c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3127398392 >> 172.21.15.39:0/3127398392 conn(0x7fcda40c32a0 msgr2=0x7fcda40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3127398392 shutdown_connections 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3127398392 wait complete. 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 Processor -- start 2024-09-13T16:47:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 -- start start 2024-09-13T16:47:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcda40c0be0 0x7fcda4152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcda40c2690 0x7fcda415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 0x7fcda415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda40abb20 con 0x7fcda40c6290 2024-09-13T16:47:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda4158ae0 con 0x7fcda40c0be0 2024-09-13T16:47:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda8fb0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda4153ab0 con 0x7fcda40c2690 2024-09-13T16:47:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 0x7fcda415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 0x7fcda415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49824/0 (socket says 172.21.15.39:49824) 2024-09-13T16:47:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.149+0000 7fcda2d9d640 1 -- 172.21.15.39:0/3285013626 learned_addr learned my addr 172.21.15.39:0/3285013626 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda259c640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcda40c0be0 0x7fcda4152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda2d9d640 1 -- 172.21.15.39:0/3285013626 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcda40c2690 msgr2=0x7fcda415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda2d9d640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcda40c2690 0x7fcda415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda2d9d640 1 -- 172.21.15.39:0/3285013626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcda40c0be0 msgr2=0x7fcda4152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda2d9d640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcda40c0be0 0x7fcda4152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda2d9d640 1 -- 172.21.15.39:0/3285013626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd98009420 con 0x7fcda40c6290 2024-09-13T16:47:53.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda2d9d640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 0x7fcda415beb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00cc80 tx=0x7fcd9c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:53.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcd92ffd640 1 -- 172.21.15.39:0/3285013626 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c00ce30 con 0x7fcda40c6290 2024-09-13T16:47:53.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcd92ffd640 1 -- 172.21.15.39:0/3285013626 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c0039a0 con 0x7fcda40c6290 2024-09-13T16:47:53.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda4153810 con 0x7fcda40c6290 2024-09-13T16:47:53.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcd92ffd640 1 -- 172.21.15.39:0/3285013626 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c003d50 con 0x7fcda40c6290 2024-09-13T16:47:53.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda40ac340 con 0x7fcda40c6290 2024-09-13T16:47:53.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda40abc60 con 0x7fcda40c6290 2024-09-13T16:47:53.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcd92ffd640 1 -- 172.21.15.39:0/3285013626 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcd9c003eb0 con 0x7fcda40c6290 2024-09-13T16:47:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcd92ffd640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd84066690 0x7fcd84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda259c640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd84066690 0x7fcd84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcd92ffd640 1 -- 172.21.15.39:0/3285013626 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcd9c084350 con 0x7fcda40c6290 2024-09-13T16:47:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.153+0000 7fcda259c640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd84066690 0x7fcd84068b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd94000f70 tx=0x7fcd94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.157+0000 7fcd92ffd640 1 -- 172.21.15.39:0/3285013626 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcda40abc60 con 0x7fcda40c6290 2024-09-13T16:47:53.208 INFO:tasks.ceph.mds.b.smithi084.stdout:starting mds.b at 2024-09-13T16:47:53.211 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T16:47:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.317+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcda40abc60 con 0x7fcda40c6290 2024-09-13T16:47:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.317+0000 7fcd92ffd640 1 -- 172.21.15.39:0/3285013626 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7fcda40abc60 con 0x7fcda40c6290 2024-09-13T16:47:53.322 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:47:53.322 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":16,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:47:49.714865+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:47:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd84066690 msgr2=0x7fcd84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd84066690 0x7fcd84068b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd94000f70 tx=0x7fcd94009040 comp rx=0 tx=0).stop 2024-09-13T16:47:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 msgr2=0x7fcda415beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 0x7fcda415beb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00cc80 tx=0x7fcd9c00a5c0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 shutdown_connections 2024-09-13T16:47:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd84066690 0x7fcd84068b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcda40c6290 0x7fcda415beb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcda40c2690 0x7fcda415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 --2- 172.21.15.39:0/3285013626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcda40c0be0 0x7fcda4152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 >> 172.21.15.39:0/3285013626 conn(0x7fcda40c32a0 msgr2=0x7fcda4156930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:53.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 shutdown_connections 2024-09-13T16:47:53.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.321+0000 7fcda8fb0640 1 -- 172.21.15.39:0/3285013626 wait complete. 2024-09-13T16:47:53.340 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:47:53.340 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:47:53.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f633788b640 1 Processor -- start 2024-09-13T16:47:53.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f633788b640 1 -- start start 2024-09-13T16:47:53.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f633788b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6330071f00 0x7f63300c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f633788b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f633006edf0 0x7f6330155660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f633788b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63300c7190 0x7f6330155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f633788b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633006de90 con 0x7f633006edf0 2024-09-13T16:47:53.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f633788b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633006f2f0 con 0x7f6330071f00 2024-09-13T16:47:53.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f633788b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63300c6170 con 0x7f63300c7190 2024-09-13T16:47:53.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6334e26640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f633006edf0 0x7f6330155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6334e26640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f633006edf0 0x7f6330155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49840/0 (socket says 172.21.15.39:49840) 2024-09-13T16:47:53.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6334e26640 1 -- 172.21.15.39:0/1160843978 learned_addr learned my addr 172.21.15.39:0/1160843978 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:53.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6335627640 1 --2- 172.21.15.39:0/1160843978 >> v2:172.21.15.84:3300/0 conn(0x7f6330071f00 0x7f63300c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6335e28640 1 --2- 172.21.15.39:0/1160843978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63300c7190 0x7f6330155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6334e26640 1 -- 172.21.15.39:0/1160843978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63300c7190 msgr2=0x7f6330155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:53.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6334e26640 1 --2- 172.21.15.39:0/1160843978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63300c7190 0x7f6330155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6334e26640 1 -- 172.21.15.39:0/1160843978 >> v2:172.21.15.84:3300/0 conn(0x7f6330071f00 msgr2=0x7f63300c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:53.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6334e26640 1 --2- 172.21.15.39:0/1160843978 >> v2:172.21.15.84:3300/0 conn(0x7f6330071f00 0x7f63300c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.457+0000 7f6334e26640 1 -- 172.21.15.39:0/1160843978 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63300c5e20 con 0x7f633006edf0 2024-09-13T16:47:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f6334e26640 1 --2- 172.21.15.39:0/1160843978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633006edf0 0x7f6330155660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f63300c3860 tx=0x7f632001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b08cd09b191966 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f6327fff640 1 -- 172.21.15.39:0/1160843978 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63200ac640 con 0x7f633006edf0 2024-09-13T16:47:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f6327fff640 1 -- 172.21.15.39:0/1160843978 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63200ac7a0 con 0x7f633006edf0 2024-09-13T16:47:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f6327fff640 1 -- 172.21.15.39:0/1160843978 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63200acb30 con 0x7f633006edf0 2024-09-13T16:47:53.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 -- 172.21.15.39:0/1160843978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633006edf0 msgr2=0x7f6330155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 --2- 172.21.15.39:0/1160843978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633006edf0 0x7f6330155660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f63300c3860 tx=0x7f632001c400 comp rx=0 tx=0).stop 2024-09-13T16:47:53.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 -- 172.21.15.39:0/1160843978 shutdown_connections 2024-09-13T16:47:53.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 --2- 172.21.15.39:0/1160843978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63300c7190 0x7f6330155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 --2- 172.21.15.39:0/1160843978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633006edf0 0x7f6330155660 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 --2- 172.21.15.39:0/1160843978 >> v2:172.21.15.84:3300/0 conn(0x7f6330071f00 0x7f63300c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 -- 172.21.15.39:0/1160843978 >> 172.21.15.39:0/1160843978 conn(0x7f633006c020 msgr2=0x7f633006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:53.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 -- 172.21.15.39:0/1160843978 shutdown_connections 2024-09-13T16:47:53.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 -- 172.21.15.39:0/1160843978 wait complete. 2024-09-13T16:47:53.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 Processor -- start 2024-09-13T16:47:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.461+0000 7f633788b640 1 -- start start 2024-09-13T16:47:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f633788b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f633006edf0 0x7f6330152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f633788b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6330071f00 0x7f63300be860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f633788b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63300c7190 0x7f63300beda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f633788b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63301499b0 con 0x7f63300c7190 2024-09-13T16:47:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f633788b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63300c01a0 con 0x7f633006edf0 2024-09-13T16:47:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f633788b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63300c0860 con 0x7f6330071f00 2024-09-13T16:47:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335e28640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63300c7190 0x7f63300beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335e28640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63300c7190 0x7f63300beda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49844/0 (socket says 172.21.15.39:49844) 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335e28640 1 -- 172.21.15.39:0/985267314 learned_addr learned my addr 172.21.15.39:0/985267314 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335627640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f633006edf0 0x7f6330152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6334e26640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6330071f00 0x7f63300be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335e28640 1 -- 172.21.15.39:0/985267314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6330071f00 msgr2=0x7f63300be860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335e28640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6330071f00 0x7f63300be860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335e28640 1 -- 172.21.15.39:0/985267314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f633006edf0 msgr2=0x7f6330152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335e28640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f633006edf0 0x7f6330152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335e28640 1 -- 172.21.15.39:0/985267314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63200094d0 con 0x7f63300c7190 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335627640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f633006edf0 0x7f6330152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335e28640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63300c7190 0x7f63300beda0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6328007d20 tx=0x7f632800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6325ffb640 1 -- 172.21.15.39:0/985267314 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328014420 con 0x7f63300c7190 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6325ffb640 1 -- 172.21.15.39:0/985267314 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63280145a0 con 0x7f63300c7190 2024-09-13T16:47:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63300c0b00 con 0x7f63300c7190 2024-09-13T16:47:53.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6325ffb640 1 -- 172.21.15.39:0/985267314 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328014950 con 0x7f63300c7190 2024-09-13T16:47:53.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63300bfe30 con 0x7f63300c7190 2024-09-13T16:47:53.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63300ace40 con 0x7f63300c7190 2024-09-13T16:47:53.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6325ffb640 1 -- 172.21.15.39:0/985267314 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6328014ab0 con 0x7f63300c7190 2024-09-13T16:47:53.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6325ffb640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6314066690 0x7f6314068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6325ffb640 1 -- 172.21.15.39:0/985267314 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6328084720 con 0x7f63300c7190 2024-09-13T16:47:53.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.465+0000 7f6335627640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6314066690 0x7f6314068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.469+0000 7f6335627640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6314066690 0x7f6314068b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f632c004660 tx=0x7f632c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:53.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.473+0000 7f6325ffb640 1 -- 172.21.15.39:0/985267314 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6328051e70 con 0x7f63300c7190 2024-09-13T16:47:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.629+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63300c4090 con 0x7f63300c7190 2024-09-13T16:47:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.629+0000 7f6325ffb640 1 -- 172.21.15.39:0/985267314 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6328055490 con 0x7f63300c7190 2024-09-13T16:47:53.632 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6314066690 msgr2=0x7f6314068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6314066690 0x7f6314068b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f632c004660 tx=0x7f632c008040 comp rx=0 tx=0).stop 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63300c7190 msgr2=0x7f63300beda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63300c7190 0x7f63300beda0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6328007d20 tx=0x7f632800e4b0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 shutdown_connections 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6314066690 0x7f6314068b50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63300c7190 0x7f63300beda0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6330071f00 0x7f63300be860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 --2- 172.21.15.39:0/985267314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f633006edf0 0x7f6330152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 >> 172.21.15.39:0/985267314 conn(0x7f633006c020 msgr2=0x7f63301538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 shutdown_connections 2024-09-13T16:47:53.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.633+0000 7f633788b640 1 -- 172.21.15.39:0/985267314 wait complete. 2024-09-13T16:47:53.650 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:47:53.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.757+0000 7f7634919640 1 Processor -- start 2024-09-13T16:47:53.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.757+0000 7f7634919640 1 -- start start 2024-09-13T16:47:53.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f7634919640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7630069e10 0x7f763006a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f7634919640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76300c2170 con 0x7f763014f3d0 2024-09-13T16:47:53.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f7634919640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763014f7b0 con 0x7f76300c0ba0 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f7634919640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763006a960 con 0x7f7630069e10 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762e59c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f763014f3d0 0x7f7630071a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42638/0 (socket says 172.21.15.39:42638) 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762e59c640 1 -- 172.21.15.39:0/1182695495 learned_addr learned my addr 172.21.15.39:0/1182695495 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f7627fff640 1 --2- 172.21.15.39:0/1182695495 >> v2:172.21.15.84:3300/0 conn(0x7f7630069e10 0x7f763006a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120608387 0 0) 0x7f76300c2170 con 0x7f763014f3d0 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f761c003530 con 0x7f763014f3d0 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 61467 0 0) 0x7f761c003530 con 0x7f763014f3d0 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 >> v2:172.21.15.84:3300/0 conn(0x7f7630069e10 msgr2=0x7f763006a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 --2- 172.21.15.39:0/1182695495 >> v2:172.21.15.84:3300/0 conn(0x7f7630069e10 0x7f763006a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 >> v1:172.21.15.84:6789/0 conn(0x7f76300c0ba0 legacy=0x7f76300c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76301526e0 con 0x7f763014f3d0 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f7620001680 con 0x7f763014f3d0 2024-09-13T16:47:53.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 >> v1:172.21.15.39:6789/0 conn(0x7f763014f3d0 legacy=0x7f7630071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f761c003980 0x7f761c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f761c0047d0 0x7f761c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761c005190 0x7f761c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763006a960 con 0x7f761c005190 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c00c030 con 0x7f761c0047d0 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761c00c1f0 con 0x7f761c003980 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762ed9d640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761c005190 0x7f761c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f7627fff640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f761c0047d0 0x7f761c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762e59c640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f761c003980 0x7f761c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762ed9d640 1 -- 172.21.15.39:0/1182695495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f761c003980 msgr2=0x7f761c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762ed9d640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f761c003980 0x7f761c003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762ed9d640 1 -- 172.21.15.39:0/1182695495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f761c0047d0 msgr2=0x7f761c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:53.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762ed9d640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f761c0047d0 0x7f761c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762ed9d640 1 -- 172.21.15.39:0/1182695495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c003530 con 0x7f761c005190 2024-09-13T16:47:53.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762ed9d640 1 -- 172.21.15.39:0/1182695495 reap_dead start 2024-09-13T16:47:53.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762ed9d640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761c005190 0x7f761c00b990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7628001e70 tx=0x7f762801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c624333bce3e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:53.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628003b80 con 0x7f761c005190 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7628003d20 con 0x7f761c005190 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.761+0000 7f762dd9b640 1 -- 172.21.15.39:0/1182695495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628005260 con 0x7f761c005190 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- 172.21.15.39:0/1182695495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761c005190 msgr2=0x7f761c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761c005190 0x7f761c00b990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7628001e70 tx=0x7f762801c780 comp rx=0 tx=0).stop 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- 172.21.15.39:0/1182695495 shutdown_connections 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 --2- 172.21.15.39:0/1182695495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f761c005190 0x7f761c00b990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- 172.21.15.39:0/1182695495 >> 172.21.15.39:0/1182695495 conn(0x7f76300a8a60 msgr2=0x7f76300c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- 172.21.15.39:0/1182695495 shutdown_connections 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- 172.21.15.39:0/1182695495 wait complete. 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 Processor -- start 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- start start 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f761c005190 0x7f76301e6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76301e6bd0 0x7f7630144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7630144b70 0x7f76301e9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76300aabc0 con 0x7f76301e6bd0 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630071730 con 0x7f761c005190 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76300c55b0 con 0x7f7630144b70 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7627fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76301e6bd0 0x7f7630144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7627fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76301e6bd0 0x7f7630144630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49854/0 (socket says 172.21.15.39:49854) 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7627fff640 1 -- 172.21.15.39:0/2152319928 learned_addr learned my addr 172.21.15.39:0/2152319928 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f762e59c640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f761c005190 0x7f76301e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f762ed9d640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7630144b70 0x7f76301e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7627fff640 1 -- 172.21.15.39:0/2152319928 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7630144b70 msgr2=0x7f76301e9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7627fff640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7630144b70 0x7f76301e9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7627fff640 1 -- 172.21.15.39:0/2152319928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f761c005190 msgr2=0x7f76301e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7627fff640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f761c005190 0x7f76301e6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7627fff640 1 -- 172.21.15.39:0/2152319928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76300a9e00 con 0x7f76301e6bd0 2024-09-13T16:47:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7627fff640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76301e6bd0 0x7f7630144630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7618011560 tx=0x7f7618011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7626ffd640 1 -- 172.21.15.39:0/2152319928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618015930 con 0x7f76301e6bd0 2024-09-13T16:47:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7626ffd640 1 -- 172.21.15.39:0/2152319928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618015a90 con 0x7f76301e6bd0 2024-09-13T16:47:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76300aa0e0 con 0x7f76301e6bd0 2024-09-13T16:47:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7626ffd640 1 -- 172.21.15.39:0/2152319928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618015e60 con 0x7f76301e6bd0 2024-09-13T16:47:53.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76300aa6a0 con 0x7f76301e6bd0 2024-09-13T16:47:53.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.765+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763006a240 con 0x7f76301e6bd0 2024-09-13T16:47:53.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.769+0000 7f7626ffd640 1 -- 172.21.15.39:0/2152319928 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f761800f460 con 0x7f76301e6bd0 2024-09-13T16:47:53.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.769+0000 7f7626ffd640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7610066690 0x7f7610068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:53.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.769+0000 7f7626ffd640 1 -- 172.21.15.39:0/2152319928 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76180845e0 con 0x7f76301e6bd0 2024-09-13T16:47:53.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.769+0000 7f762e59c640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7610066690 0x7f7610068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:53.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.769+0000 7f762e59c640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7610066690 0x7f7610068b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7620001ec0 tx=0x7f7620009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:53.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.773+0000 7f7626ffd640 1 -- 172.21.15.39:0/2152319928 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7618051d30 con 0x7f76301e6bd0 2024-09-13T16:47:53.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.961+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7630061370 con 0x7f76301e6bd0 2024-09-13T16:47:53.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7626ffd640 1 -- 172.21.15.39:0/2152319928 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7f7618055350 con 0x7f76301e6bd0 2024-09-13T16:47:53.967 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:47:53.967 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T16:47:53.967 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2024-09-13T16:47:53.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7610066690 msgr2=0x7f7610068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7610066690 0x7f7610068b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7620001ec0 tx=0x7f7620009040 comp rx=0 tx=0).stop 2024-09-13T16:47:53.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76301e6bd0 msgr2=0x7f7630144630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:53.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76301e6bd0 0x7f7630144630 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7618011560 tx=0x7f7618011a20 comp rx=0 tx=0).stop 2024-09-13T16:47:53.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 shutdown_connections 2024-09-13T16:47:53.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7610066690 0x7f7610068b50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7630144b70 0x7f76301e9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76301e6bd0 0x7f7630144630 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 --2- 172.21.15.39:0/2152319928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f761c005190 0x7f76301e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:53.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 >> 172.21.15.39:0/2152319928 conn(0x7f76300a8a60 msgr2=0x7f76300c60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:53.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 shutdown_connections 2024-09-13T16:47:53.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:53.965+0000 7f7634919640 1 -- 172.21.15.39:0/2152319928 wait complete. 2024-09-13T16:47:55.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:47:55.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:47:55.986 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:47:56.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f72228da640 1 Processor -- start 2024-09-13T16:47:56.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f72228da640 1 -- start start 2024-09-13T16:47:56.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f72228da640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f721c151500 0x7f721c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f72228da640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f721c071dd0 0x7f721c154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f72228da640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c0c5a30 con 0x7f721c0c47c0 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f72228da640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c1576a0 con 0x7f721c151500 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f72228da640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c157980 con 0x7f721c071dd0 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f721bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f721c151500 0x7f721c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f721bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f721c151500 0x7f721c0c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49866/0 (socket says 172.21.15.39:49866) 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f721bfff640 1 -- 172.21.15.39:0/2037359203 learned_addr learned my addr 172.21.15.39:0/2037359203 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f7220e77640 1 --1- 172.21.15.39:0/2037359203 >> v1:172.21.15.39:6789/0 conn(0x7f721c0c47c0 0x7f721c1545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42646/0 (socket says 172.21.15.39:42646) 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f721bfff640 1 -- 172.21.15.39:0/2037359203 >> v2:172.21.15.84:3300/0 conn(0x7f721c071dd0 msgr2=0x7f721c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.113+0000 7f721b7fe640 1 --2- 172.21.15.39:0/2037359203 >> v2:172.21.15.84:3300/0 conn(0x7f721c071dd0 0x7f721c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f721bfff640 1 --2- 172.21.15.39:0/2037359203 >> v2:172.21.15.84:3300/0 conn(0x7f721c071dd0 0x7f721c154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f721bfff640 1 -- 172.21.15.39:0/2037359203 >> v1:172.21.15.39:6789/0 conn(0x7f721c0c47c0 legacy=0x7f721c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f721bfff640 1 -- 172.21.15.39:0/2037359203 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c157c60 con 0x7f721c151500 2024-09-13T16:47:56.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f721affd640 1 -- 172.21.15.39:0/2037359203 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157080245 0 0) 0x7f721c0c5a30 con 0x7f721c0c47c0 2024-09-13T16:47:56.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f721bfff640 1 --2- 172.21.15.39:0/2037359203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c151500 0x7f721c0c4280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7214009870 tx=0x7f721401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a185eb62b82c24d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:56.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f721affd640 1 -- 172.21.15.39:0/2037359203 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140ab680 con 0x7f721c151500 2024-09-13T16:47:56.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f721affd640 1 -- 172.21.15.39:0/2037359203 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72140ab7e0 con 0x7f721c151500 2024-09-13T16:47:56.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f721affd640 1 -- 172.21.15.39:0/2037359203 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140abb70 con 0x7f721c151500 2024-09-13T16:47:56.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f72228da640 1 -- 172.21.15.39:0/2037359203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c151500 msgr2=0x7f721c0c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f72228da640 1 --2- 172.21.15.39:0/2037359203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c151500 0x7f721c0c4280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7214009870 tx=0x7f721401c460 comp rx=0 tx=0).stop 2024-09-13T16:47:56.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f72228da640 1 -- 172.21.15.39:0/2037359203 shutdown_connections 2024-09-13T16:47:56.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f72228da640 1 --2- 172.21.15.39:0/2037359203 >> v2:172.21.15.84:3300/0 conn(0x7f721c071dd0 0x7f721c154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f72228da640 1 --2- 172.21.15.39:0/2037359203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c151500 0x7f721c0c4280 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f72228da640 1 -- 172.21.15.39:0/2037359203 >> 172.21.15.39:0/2037359203 conn(0x7f721c0ab530 msgr2=0x7f721c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:56.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f72228da640 1 -- 172.21.15.39:0/2037359203 shutdown_connections 2024-09-13T16:47:56.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.117+0000 7f72228da640 1 -- 172.21.15.39:0/2037359203 wait complete. 2024-09-13T16:47:56.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 Processor -- start 2024-09-13T16:47:56.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 -- start start 2024-09-13T16:47:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f721c071dd0 0x7f721c0c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c0c47c0 0x7f721c148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f721c151500 0x7f721c14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1f0480 con 0x7f721c0c47c0 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c068420 con 0x7f721c071dd0 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0ca3c0 con 0x7f721c151500 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c0c47c0 0x7f721c148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c0c47c0 0x7f721c148580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49870/0 (socket says 172.21.15.39:49870) 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721b7fe640 1 -- 172.21.15.39:0/3694947169 learned_addr learned my addr 172.21.15.39:0/3694947169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721bfff640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f721c071dd0 0x7f721c0c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f7220e77640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f721c151500 0x7f721c14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721b7fe640 1 -- 172.21.15.39:0/3694947169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f721c151500 msgr2=0x7f721c14c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721b7fe640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f721c151500 0x7f721c14c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721b7fe640 1 -- 172.21.15.39:0/3694947169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f721c071dd0 msgr2=0x7f721c0c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721b7fe640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f721c071dd0 0x7f721c0c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721b7fe640 1 -- 172.21.15.39:0/3694947169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140094d0 con 0x7f721c0c47c0 2024-09-13T16:47:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721bfff640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f721c071dd0 0x7f721c0c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:47:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721b7fe640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c0c47c0 0x7f721c148580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f721000c7f0 tx=0x7f721000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f7218ff9640 1 -- 172.21.15.39:0/3694947169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210004230 con 0x7f721c0c47c0 2024-09-13T16:47:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f7218ff9640 1 -- 172.21.15.39:0/3694947169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7210004390 con 0x7f721c0c47c0 2024-09-13T16:47:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c1efdc0 con 0x7f721c0c47c0 2024-09-13T16:47:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f7218ff9640 1 -- 172.21.15.39:0/3694947169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210003860 con 0x7f721c0c47c0 2024-09-13T16:47:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c148ac0 con 0x7f721c0c47c0 2024-09-13T16:47:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c1f06f0 con 0x7f721c0c47c0 2024-09-13T16:47:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f7218ff9640 1 -- 172.21.15.39:0/3694947169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7210003bd0 con 0x7f721c0c47c0 2024-09-13T16:47:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f7218ff9640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7208066690 0x7f7208068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f7218ff9640 1 -- 172.21.15.39:0/3694947169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7210083ed0 con 0x7f721c0c47c0 2024-09-13T16:47:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.121+0000 7f721bfff640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7208066690 0x7f7208068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.125+0000 7f721bfff640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7208066690 0x7f7208068b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72140af000 tx=0x7f7214008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.129+0000 7f7218ff9640 1 -- 172.21.15.39:0/3694947169 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7210051620 con 0x7f721c0c47c0 2024-09-13T16:47:56.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.313+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f721c1efc20 con 0x7f721c0c47c0 2024-09-13T16:47:56.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.313+0000 7f7218ff9640 1 -- 172.21.15.39:0/3694947169 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f721c1efc20 con 0x7f721c0c47c0 2024-09-13T16:47:56.317 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:47:56.317 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T16:47:56.317 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 3 2024-09-13T16:47:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7208066690 msgr2=0x7f7208068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7208066690 0x7f7208068b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72140af000 tx=0x7f7214008040 comp rx=0 tx=0).stop 2024-09-13T16:47:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c0c47c0 msgr2=0x7f721c148580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c0c47c0 0x7f721c148580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f721000c7f0 tx=0x7f721000ccb0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 shutdown_connections 2024-09-13T16:47:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7208066690 0x7f7208068b50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f721c151500 0x7f721c14c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f721c0c47c0 0x7f721c148580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 --2- 172.21.15.39:0/3694947169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f721c071dd0 0x7f721c0c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 >> 172.21.15.39:0/3694947169 conn(0x7f721c0ab530 msgr2=0x7f721c0c6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:56.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 shutdown_connections 2024-09-13T16:47:56.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.317+0000 7f72228da640 1 -- 172.21.15.39:0/3694947169 wait complete. 2024-09-13T16:47:56.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:47:56.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.441+0000 7fc150f89640 1 Processor -- start 2024-09-13T16:47:56.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.441+0000 7fc150f89640 1 -- start start 2024-09-13T16:47:56.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.441+0000 7fc150f89640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc14c0c5330 0x7fc14c0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.441+0000 7fc150f89640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c06c090 0x7fc14c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.441+0000 7fc150f89640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14c1546c0 con 0x7fc14c150f80 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.441+0000 7fc150f89640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14c0c19a0 con 0x7fc14c0c5330 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.441+0000 7fc150f89640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c0c49f0 con 0x7fc14c06c090 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc141d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc14c0c5330 0x7fc14c0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc141d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc14c0c5330 0x7fc14c0c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49880/0 (socket says 172.21.15.39:49880) 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc141d9b640 1 -- 172.21.15.39:0/2841976368 learned_addr learned my addr 172.21.15.39:0/2841976368 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc14ad9d640 1 --2- 172.21.15.39:0/2841976368 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c06c090 0x7fc14c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc141d9b640 1 -- 172.21.15.39:0/2841976368 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c06c090 msgr2=0x7fc14c06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc141d9b640 1 --2- 172.21.15.39:0/2841976368 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c06c090 0x7fc14c06c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc141d9b640 1 -- 172.21.15.39:0/2841976368 >> v1:172.21.15.84:6789/0 conn(0x7fc14c150f80 legacy=0x7fc14c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:56.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc141d9b640 1 -- 172.21.15.39:0/2841976368 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c1543e0 con 0x7fc14c0c5330 2024-09-13T16:47:56.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc141d9b640 1 --2- 172.21.15.39:0/2841976368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c0c5330 0x7fc14c0c1210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1340098a0 tx=0x7fc13401e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6318b2d9c8aaad43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:56.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc149d9b640 1 -- 172.21.15.39:0/2841976368 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1340ab660 con 0x7fc14c0c5330 2024-09-13T16:47:56.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc149d9b640 1 -- 172.21.15.39:0/2841976368 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1340ab7c0 con 0x7fc14c0c5330 2024-09-13T16:47:56.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc149d9b640 1 -- 172.21.15.39:0/2841976368 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1340abb50 con 0x7fc14c0c5330 2024-09-13T16:47:56.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc150f89640 1 -- 172.21.15.39:0/2841976368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c0c5330 msgr2=0x7fc14c0c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc150f89640 1 --2- 172.21.15.39:0/2841976368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c0c5330 0x7fc14c0c1210 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1340098a0 tx=0x7fc13401e650 comp rx=0 tx=0).stop 2024-09-13T16:47:56.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc150f89640 1 -- 172.21.15.39:0/2841976368 shutdown_connections 2024-09-13T16:47:56.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc150f89640 1 --2- 172.21.15.39:0/2841976368 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c06c090 0x7fc14c06c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc150f89640 1 --2- 172.21.15.39:0/2841976368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c0c5330 0x7fc14c0c1210 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc150f89640 1 -- 172.21.15.39:0/2841976368 >> 172.21.15.39:0/2841976368 conn(0x7fc14c069e10 msgr2=0x7fc14c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:56.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc150f89640 1 -- 172.21.15.39:0/2841976368 shutdown_connections 2024-09-13T16:47:56.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.445+0000 7fc150f89640 1 -- 172.21.15.39:0/2841976368 wait complete. 2024-09-13T16:47:56.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 Processor -- start 2024-09-13T16:47:56.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 -- start start 2024-09-13T16:47:56.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc14c06c090 0x7fc14c14a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c0c5330 0x7fc14c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c150f80 0x7fc14c06f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c070d00 con 0x7fc14c150f80 2024-09-13T16:47:56.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c071d70 con 0x7fc14c06c090 2024-09-13T16:47:56.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c149e60 con 0x7fc14c0c5330 2024-09-13T16:47:56.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c150f80 0x7fc14c06f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c150f80 0x7fc14c06f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49892/0 (socket says 172.21.15.39:49892) 2024-09-13T16:47:56.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14ad9d640 1 -- 172.21.15.39:0/3510783169 learned_addr learned my addr 172.21.15.39:0/3510783169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:56.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14a59c640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc14c06c090 0x7fc14c14a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc141d9b640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c0c5330 0x7fc14c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14ad9d640 1 -- 172.21.15.39:0/3510783169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c0c5330 msgr2=0x7fc14c0707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14ad9d640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c0c5330 0x7fc14c0707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14ad9d640 1 -- 172.21.15.39:0/3510783169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc14c06c090 msgr2=0x7fc14c14a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14ad9d640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc14c06c090 0x7fc14c14a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14ad9d640 1 -- 172.21.15.39:0/3510783169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340094d0 con 0x7fc14c150f80 2024-09-13T16:47:56.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc14ad9d640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c150f80 0x7fc14c06f4d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc144007e60 tx=0x7fc14400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:56.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc1437fe640 1 -- 172.21.15.39:0/3510783169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc144014420 con 0x7fc14c150f80 2024-09-13T16:47:56.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc1437fe640 1 -- 172.21.15.39:0/3510783169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1440145a0 con 0x7fc14c150f80 2024-09-13T16:47:56.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14c159d90 con 0x7fc14c150f80 2024-09-13T16:47:56.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc1437fe640 1 -- 172.21.15.39:0/3510783169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc144014950 con 0x7fc14c150f80 2024-09-13T16:47:56.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c1473f0 con 0x7fc14c150f80 2024-09-13T16:47:56.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14c14aad0 con 0x7fc14c150f80 2024-09-13T16:47:56.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.449+0000 7fc1437fe640 1 -- 172.21.15.39:0/3510783169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc144014ab0 con 0x7fc14c150f80 2024-09-13T16:47:56.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.453+0000 7fc1437fe640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc12c066690 0x7fc12c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.453+0000 7fc1437fe640 1 -- 172.21.15.39:0/3510783169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc144084750 con 0x7fc14c150f80 2024-09-13T16:47:56.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.453+0000 7fc14a59c640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc12c066690 0x7fc12c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.453+0000 7fc14a59c640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc12c066690 0x7fc12c068b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00b810 tx=0x7fc13c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:56.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.457+0000 7fc1437fe640 1 -- 172.21.15.39:0/3510783169 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc144051e00 con 0x7fc14c150f80 2024-09-13T16:47:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.617+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc14c072150 con 0x7fc14c150f80 2024-09-13T16:47:56.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.625+0000 7fc1437fe640 1 -- 172.21.15.39:0/3510783169 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12802 (secure 0 0 0) 0x7fc144055420 con 0x7fc14c150f80 2024-09-13T16:47:56.629 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:47:56.629 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":16,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:47:49.714865+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:47:56.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc12c066690 msgr2=0x7fc12c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc12c066690 0x7fc12c068b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00b810 tx=0x7fc13c008040 comp rx=0 tx=0).stop 2024-09-13T16:47:56.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c150f80 msgr2=0x7fc14c06f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c150f80 0x7fc14c06f4d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc144007e60 tx=0x7fc14400e4b0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 shutdown_connections 2024-09-13T16:47:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc12c066690 0x7fc12c068b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c150f80 0x7fc14c06f4d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc14c0c5330 0x7fc14c0707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 --2- 172.21.15.39:0/3510783169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc14c06c090 0x7fc14c14a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 >> 172.21.15.39:0/3510783169 conn(0x7fc14c069e10 msgr2=0x7fc14c14b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 shutdown_connections 2024-09-13T16:47:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.629+0000 7fc150f89640 1 -- 172.21.15.39:0/3510783169 wait complete. 2024-09-13T16:47:56.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:47:56.648 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:47:56.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69edde9640 1 Processor -- start 2024-09-13T16:47:56.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69edde9640 1 -- start start 2024-09-13T16:47:56.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69edde9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f69e8151880 0x7f69e81542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69edde9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f69e80c46a0 0x7f69e8154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69edde9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e8157910 con 0x7f69e806b080 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69edde9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e8153ed0 con 0x7f69e80c46a0 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69edde9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e8158490 con 0x7f69e8151880 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69e7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f69e80c46a0 0x7f69e8154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69e7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f69e80c46a0 0x7f69e8154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49898/0 (socket says 172.21.15.39:49898) 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69e7fff640 1 -- 172.21.15.39:0/1208022995 learned_addr learned my addr 172.21.15.39:0/1208022995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/1208022995 >> v2:172.21.15.84:3300/0 conn(0x7f69e8151880 0x7f69e81542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69e7fff640 1 -- 172.21.15.39:0/1208022995 >> v2:172.21.15.84:3300/0 conn(0x7f69e8151880 msgr2=0x7f69e81542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69e7fff640 1 --2- 172.21.15.39:0/1208022995 >> v2:172.21.15.84:3300/0 conn(0x7f69e8151880 0x7f69e81542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69e7fff640 1 -- 172.21.15.39:0/1208022995 >> v1:172.21.15.84:6789/0 conn(0x7f69e806b080 legacy=0x7f69e8151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.765+0000 7f69e7fff640 1 -- 172.21.15.39:0/1208022995 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e8157bf0 con 0x7f69e80c46a0 2024-09-13T16:47:56.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e7fff640 1 --2- 172.21.15.39:0/1208022995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e80c46a0 0x7f69e8154820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69e80aca90 tx=0x7f69e001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b609b696974092eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:56.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e67fc640 1 -- 172.21.15.39:0/1208022995 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00a5680 con 0x7f69e80c46a0 2024-09-13T16:47:56.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e67fc640 1 -- 172.21.15.39:0/1208022995 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e00a57e0 con 0x7f69e80c46a0 2024-09-13T16:47:56.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e67fc640 1 -- 172.21.15.39:0/1208022995 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00a5b70 con 0x7f69e80c46a0 2024-09-13T16:47:56.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 -- 172.21.15.39:0/1208022995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e80c46a0 msgr2=0x7f69e8154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 --2- 172.21.15.39:0/1208022995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e80c46a0 0x7f69e8154820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69e80aca90 tx=0x7f69e001c8d0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 -- 172.21.15.39:0/1208022995 shutdown_connections 2024-09-13T16:47:56.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 --2- 172.21.15.39:0/1208022995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e80c46a0 0x7f69e8154820 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 --2- 172.21.15.39:0/1208022995 >> v2:172.21.15.84:3300/0 conn(0x7f69e8151880 0x7f69e81542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 -- 172.21.15.39:0/1208022995 >> 172.21.15.39:0/1208022995 conn(0x7f69e80c32a0 msgr2=0x7f69e80c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:56.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 -- 172.21.15.39:0/1208022995 shutdown_connections 2024-09-13T16:47:56.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 -- 172.21.15.39:0/1208022995 wait complete. 2024-09-13T16:47:56.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 Processor -- start 2024-09-13T16:47:56.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 -- start start 2024-09-13T16:47:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69e806b080 0x7f69e8148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69e80c46a0 0x7f69e8148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8151880 0x7f69e814f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e8072400 con 0x7f69e8151880 2024-09-13T16:47:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80707c0 con 0x7f69e806b080 2024-09-13T16:47:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69edde9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e814f110 con 0x7f69e80c46a0 2024-09-13T16:47:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8151880 0x7f69e814f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8151880 0x7f69e814f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49902/0 (socket says 172.21.15.39:49902) 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e7fff640 1 -- 172.21.15.39:0/1241154072 learned_addr learned my addr 172.21.15.39:0/1241154072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e77fe640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69e806b080 0x7f69e8148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69e80c46a0 0x7f69e8148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e7fff640 1 -- 172.21.15.39:0/1241154072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69e80c46a0 msgr2=0x7f69e8148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e7fff640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69e80c46a0 0x7f69e8148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e7fff640 1 -- 172.21.15.39:0/1241154072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69e806b080 msgr2=0x7f69e8148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e7fff640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69e806b080 0x7f69e8148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e7fff640 1 -- 172.21.15.39:0/1241154072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e0009b30 con 0x7f69e8151880 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.769+0000 7f69e6ffd640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69e80c46a0 0x7f69e8148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69e7fff640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8151880 0x7f69e814f870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69e00194e0 tx=0x7f69e0019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69cffff640 1 -- 172.21.15.39:0/1241154072 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00b6640 con 0x7f69e8151880 2024-09-13T16:47:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69cffff640 1 -- 172.21.15.39:0/1241154072 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e00b67e0 con 0x7f69e8151880 2024-09-13T16:47:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e8071e40 con 0x7f69e8151880 2024-09-13T16:47:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69cffff640 1 -- 172.21.15.39:0/1241154072 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00b6b90 con 0x7f69e8151880 2024-09-13T16:47:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e814eda0 con 0x7f69e8151880 2024-09-13T16:47:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e814ec00 con 0x7f69e8151880 2024-09-13T16:47:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69cffff640 1 -- 172.21.15.39:0/1241154072 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69e00b6cf0 con 0x7f69e8151880 2024-09-13T16:47:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69cffff640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69d4066690 0x7f69d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69cffff640 1 -- 172.21.15.39:0/1241154072 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f69e011f110 con 0x7f69e8151880 2024-09-13T16:47:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69e77fe640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69d4066690 0x7f69d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.773+0000 7f69e77fe640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69d4066690 0x7f69d4068b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69e814e450 tx=0x7f69d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:56.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.777+0000 7f69cffff640 1 -- 172.21.15.39:0/1241154072 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69e814ec00 con 0x7f69e8151880 2024-09-13T16:47:56.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.937+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69e814ec00 con 0x7f69e8151880 2024-09-13T16:47:56.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.937+0000 7f69cffff640 1 -- 172.21.15.39:0/1241154072 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69e814ec00 con 0x7f69e8151880 2024-09-13T16:47:56.941 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69d4066690 msgr2=0x7f69d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69d4066690 0x7f69d4068b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f69e814e450 tx=0x7f69d8009040 comp rx=0 tx=0).stop 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8151880 msgr2=0x7f69e814f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8151880 0x7f69e814f870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69e00194e0 tx=0x7f69e0019670 comp rx=0 tx=0).stop 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 shutdown_connections 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69d4066690 0x7f69d4068b50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69e8151880 0x7f69e814f870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69e80c46a0 0x7f69e8148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 --2- 172.21.15.39:0/1241154072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69e806b080 0x7f69e8148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 >> 172.21.15.39:0/1241154072 conn(0x7f69e80c32a0 msgr2=0x7f69e8065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 shutdown_connections 2024-09-13T16:47:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:56.941+0000 7f69edde9640 1 -- 172.21.15.39:0/1241154072 wait complete. 2024-09-13T16:47:56.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T16:47:57.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe2939ff640 1 Processor -- start 2024-09-13T16:47:57.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe2939ff640 1 -- start start 2024-09-13T16:47:57.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe2939ff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe28c071dd0 0x7fe28c154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe2939ff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe28c0c7800 0x7fe28c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe2939ff640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28c0c5a30 con 0x7fe28c151500 2024-09-13T16:47:57.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe2939ff640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28c1576a0 con 0x7fe28c0c7800 2024-09-13T16:47:57.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe2939ff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28c157980 con 0x7fe28c071dd0 2024-09-13T16:47:57.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe291f9c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe28c0c7800 0x7fe28c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe291f9c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe28c0c7800 0x7fe28c1545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49910/0 (socket says 172.21.15.39:49910) 2024-09-13T16:47:57.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe291f9c640 1 -- 172.21.15.39:0/1789587578 learned_addr learned my addr 172.21.15.39:0/1789587578 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:57.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe290f9a640 1 --2- 172.21.15.39:0/1789587578 >> v2:172.21.15.84:3300/0 conn(0x7fe28c071dd0 0x7fe28c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe291f9c640 1 -- 172.21.15.39:0/1789587578 >> v2:172.21.15.84:3300/0 conn(0x7fe28c071dd0 msgr2=0x7fe28c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:57.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe291f9c640 1 --2- 172.21.15.39:0/1789587578 >> v2:172.21.15.84:3300/0 conn(0x7fe28c071dd0 0x7fe28c154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe291f9c640 1 -- 172.21.15.39:0/1789587578 >> v1:172.21.15.39:6789/0 conn(0x7fe28c151500 legacy=0x7fe28c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe27ffff640 1 -- 172.21.15.39:0/1789587578 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232279895 0 0) 0x7fe28c0c5a30 con 0x7fe28c151500 2024-09-13T16:47:57.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.069+0000 7fe291f9c640 1 -- 172.21.15.39:0/1789587578 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c157c60 con 0x7fe28c0c7800 2024-09-13T16:47:57.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe291f9c640 1 --2- 172.21.15.39:0/1789587578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0c7800 0x7fe28c1545b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe284009e50 tx=0x7fe28401d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902ee474ac78069d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:57.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe27ffff640 1 -- 172.21.15.39:0/1789587578 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2840a5810 con 0x7fe28c0c7800 2024-09-13T16:47:57.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe27ffff640 1 -- 172.21.15.39:0/1789587578 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2840a5970 con 0x7fe28c0c7800 2024-09-13T16:47:57.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe27ffff640 1 -- 172.21.15.39:0/1789587578 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2840a5d00 con 0x7fe28c0c7800 2024-09-13T16:47:57.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- 172.21.15.39:0/1789587578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0c7800 msgr2=0x7fe28c1545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 --2- 172.21.15.39:0/1789587578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0c7800 0x7fe28c1545b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe284009e50 tx=0x7fe28401d390 comp rx=0 tx=0).stop 2024-09-13T16:47:57.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- 172.21.15.39:0/1789587578 shutdown_connections 2024-09-13T16:47:57.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 --2- 172.21.15.39:0/1789587578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0c7800 0x7fe28c1545b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 --2- 172.21.15.39:0/1789587578 >> v2:172.21.15.84:3300/0 conn(0x7fe28c071dd0 0x7fe28c154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- 172.21.15.39:0/1789587578 >> 172.21.15.39:0/1789587578 conn(0x7fe28c0ab530 msgr2=0x7fe28c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:57.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- 172.21.15.39:0/1789587578 shutdown_connections 2024-09-13T16:47:57.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- 172.21.15.39:0/1789587578 wait complete. 2024-09-13T16:47:57.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 Processor -- start 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- start start 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c071dd0 0x7fe28c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe28c0c7800 0x7fe28c149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe28c151500 0x7fe28c0c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c148fd0 con 0x7fe28c071dd0 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c14b8b0 con 0x7fe28c0c7800 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c14e810 con 0x7fe28c151500 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe29179b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c071dd0 0x7fe28c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe29179b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c071dd0 0x7fe28c1559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49924/0 (socket says 172.21.15.39:49924) 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe29179b640 1 -- 172.21.15.39:0/2529210232 learned_addr learned my addr 172.21.15.39:0/2529210232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe290f9a640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe28c0c7800 0x7fe28c149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe29179b640 1 -- 172.21.15.39:0/2529210232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe28c151500 msgr2=0x7fe28c0c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe29179b640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe28c151500 0x7fe28c0c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe29179b640 1 -- 172.21.15.39:0/2529210232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe28c0c7800 msgr2=0x7fe28c149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe29179b640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe28c0c7800 0x7fe28c149460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe29179b640 1 -- 172.21.15.39:0/2529210232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe284009b30 con 0x7fe28c071dd0 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe29179b640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c071dd0 0x7fe28c1559f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c3f0 tx=0x7fe28800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe27dffb640 1 -- 172.21.15.39:0/2529210232 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880052d0 con 0x7fe28c071dd0 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe27dffb640 1 -- 172.21.15.39:0/2529210232 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288005450 con 0x7fe28c071dd0 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c14bb70 con 0x7fe28c071dd0 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe27dffb640 1 -- 172.21.15.39:0/2529210232 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288003860 con 0x7fe28c071dd0 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.073+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c148c60 con 0x7fe28c071dd0 2024-09-13T16:47:57.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.077+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28c0c4280 con 0x7fe28c071dd0 2024-09-13T16:47:57.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.077+0000 7fe27dffb640 1 -- 172.21.15.39:0/2529210232 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2880039c0 con 0x7fe28c071dd0 2024-09-13T16:47:57.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.077+0000 7fe27dffb640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2700666c0 0x7fe270068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.077+0000 7fe27dffb640 1 -- 172.21.15.39:0/2529210232 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe288083410 con 0x7fe28c071dd0 2024-09-13T16:47:57.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.077+0000 7fe290f9a640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2700666c0 0x7fe270068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.077+0000 7fe290f9a640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2700666c0 0x7fe270068b80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0c73f0 tx=0x7fe280008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:57.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.081+0000 7fe27dffb640 1 -- 172.21.15.39:0/2529210232 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe288050ad0 con 0x7fe28c071dd0 2024-09-13T16:47:57.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.233+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe28c071bb0 con 0x7fe28c071dd0 2024-09-13T16:47:57.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe27dffb640 1 -- 172.21.15.39:0/2529210232 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe288017060 con 0x7fe28c071dd0 2024-09-13T16:47:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:47:57.239 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-13T16:47:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-13T16:47:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-13T16:47:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-13T16:47:57.239 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-13T16:47:57.239 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-13T16:47:57.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2700666c0 msgr2=0x7fe270068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2700666c0 0x7fe270068b80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe28c0c73f0 tx=0x7fe280008040 comp rx=0 tx=0).stop 2024-09-13T16:47:57.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c071dd0 msgr2=0x7fe28c1559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c071dd0 0x7fe28c1559f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c3f0 tx=0x7fe28800c8b0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 shutdown_connections 2024-09-13T16:47:57.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2700666c0 0x7fe270068b80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe28c0c7800 0x7fe28c149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe28c151500 0x7fe28c0c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 --2- 172.21.15.39:0/2529210232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c071dd0 0x7fe28c1559f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 >> 172.21.15.39:0/2529210232 conn(0x7fe28c0ab530 msgr2=0x7fe28c0ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:57.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 shutdown_connections 2024-09-13T16:47:57.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.237+0000 7fe2939ff640 1 -- 172.21.15.39:0/2529210232 wait complete. 2024-09-13T16:47:57.256 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'a' 2024-09-13T16:47:57.256 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create a_metadata --pg_num_min 64 2024-09-13T16:47:57.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99fb4ff640 1 Processor -- start 2024-09-13T16:47:57.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99fb4ff640 1 -- start start 2024-09-13T16:47:57.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99fb4ff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99f40aa2d0 0x7f99f40aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99fb4ff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f4151cf0 0x7f99f4152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99fb4ff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f4155d20 con 0x7f99f40a4d60 2024-09-13T16:47:57.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99fb4ff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f4155b30 con 0x7f99f40aa2d0 2024-09-13T16:47:57.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99fb4ff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40c4750 con 0x7f99f4151cf0 2024-09-13T16:47:57.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f9a9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f4151cf0 0x7f99f4152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f929b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99f40aa2d0 0x7f99f40aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f9a9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f4151cf0 0x7f99f4152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:52358/0 (socket says 172.21.15.39:52358) 2024-09-13T16:47:57.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f8a9a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f99f40a4d60 0x7f99f414ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:46216/0 (socket says 172.21.15.39:46216) 2024-09-13T16:47:57.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f9a9c640 1 -- 172.21.15.39:0/2285155429 learned_addr learned my addr 172.21.15.39:0/2285155429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:57.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f929b640 1 -- 172.21.15.39:0/2285155429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f4151cf0 msgr2=0x7f99f4152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:57.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f929b640 1 --2- 172.21.15.39:0/2285155429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f4151cf0 0x7f99f4152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f929b640 1 -- 172.21.15.39:0/2285155429 >> v1:172.21.15.84:6789/0 conn(0x7f99f40a4d60 legacy=0x7f99f414ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f929b640 1 -- 172.21.15.39:0/2285155429 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f4155fc0 con 0x7f99f40aa2d0 2024-09-13T16:47:57.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.369+0000 7f99f9a9c640 1 --2- 172.21.15.39:0/2285155429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f4151cf0 0x7f99f4152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:47:57.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f929b640 1 --2- 172.21.15.39:0/2285155429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99f40aa2d0 0x7f99f40aa6b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99e80097c0 tx=0x7f99e801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1407b3983b2149c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:57.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99e7fff640 1 -- 172.21.15.39:0/2285155429 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80ab600 con 0x7f99f40aa2d0 2024-09-13T16:47:57.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99e7fff640 1 -- 172.21.15.39:0/2285155429 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e80a7050 con 0x7f99f40aa2d0 2024-09-13T16:47:57.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99e7fff640 1 -- 172.21.15.39:0/2285155429 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80ab980 con 0x7f99f40aa2d0 2024-09-13T16:47:57.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 -- 172.21.15.39:0/2285155429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99f40aa2d0 msgr2=0x7f99f40aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 --2- 172.21.15.39:0/2285155429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99f40aa2d0 0x7f99f40aa6b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99e80097c0 tx=0x7f99e801c3e0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 -- 172.21.15.39:0/2285155429 shutdown_connections 2024-09-13T16:47:57.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 --2- 172.21.15.39:0/2285155429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f4151cf0 0x7f99f4152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 --2- 172.21.15.39:0/2285155429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99f40aa2d0 0x7f99f40aa6b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 -- 172.21.15.39:0/2285155429 >> 172.21.15.39:0/2285155429 conn(0x7f99f40c5790 msgr2=0x7f99f40c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:57.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 -- 172.21.15.39:0/2285155429 shutdown_connections 2024-09-13T16:47:57.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 -- 172.21.15.39:0/2285155429 wait complete. 2024-09-13T16:47:57.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 Processor -- start 2024-09-13T16:47:57.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 -- start start 2024-09-13T16:47:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f40a4d60 0x7f99f41477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f40aa2d0 0x7f99f414d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99f4151cf0 0x7f99f41efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f4147d20 con 0x7f99f40a4d60 2024-09-13T16:47:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f406ca60 con 0x7f99f4151cf0 2024-09-13T16:47:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99fb4ff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40c2e10 con 0x7f99f40aa2d0 2024-09-13T16:47:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f929b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f40a4d60 0x7f99f41477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f929b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f40a4d60 0x7f99f41477e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49930/0 (socket says 172.21.15.39:49930) 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f929b640 1 -- 172.21.15.39:0/4141332906 learned_addr learned my addr 172.21.15.39:0/4141332906 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f9a9c640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99f4151cf0 0x7f99f41efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f8a9a640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f40aa2d0 0x7f99f414d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f929b640 1 -- 172.21.15.39:0/4141332906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f40aa2d0 msgr2=0x7f99f414d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f929b640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f40aa2d0 0x7f99f414d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f929b640 1 -- 172.21.15.39:0/4141332906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99f4151cf0 msgr2=0x7f99f41efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f929b640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99f4151cf0 0x7f99f41efbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f929b640 1 -- 172.21.15.39:0/4141332906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec009b30 con 0x7f99f40a4d60 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.373+0000 7f99f8a9a640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f40aa2d0 0x7f99f414d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99f929b640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f40a4d60 0x7f99f41477e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f99e801a010 tx=0x7f99e8019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99e5ffb640 1 -- 172.21.15.39:0/4141332906 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80b4410 con 0x7f99f40a4d60 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99e5ffb640 1 -- 172.21.15.39:0/4141332906 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e80b4570 con 0x7f99f40a4d60 2024-09-13T16:47:57.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e8009420 con 0x7f99f40a4d60 2024-09-13T16:47:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99e5ffb640 1 -- 172.21.15.39:0/4141332906 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80b4920 con 0x7f99f40a4d60 2024-09-13T16:47:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f40c2850 con 0x7f99f40a4d60 2024-09-13T16:47:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f40aa6b0 con 0x7f99f40a4d60 2024-09-13T16:47:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99e5ffb640 1 -- 172.21.15.39:0/4141332906 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f99e80b2040 con 0x7f99f40a4d60 2024-09-13T16:47:57.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99e5ffb640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99d80666c0 0x7f99d8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99e5ffb640 1 -- 172.21.15.39:0/4141332906 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f99e811ddb0 con 0x7f99f40a4d60 2024-09-13T16:47:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99f8a9a640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99d80666c0 0x7f99d8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.377+0000 7f99f8a9a640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99d80666c0 0x7f99d8068b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99f4071ee0 tx=0x7f99f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:57.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.381+0000 7f99e5ffb640 1 -- 172.21.15.39:0/4141332906 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f99e80eb500 con 0x7f99f40a4d60 2024-09-13T16:47:57.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.537+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "a_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f99f40a86f0 con 0x7f99f40a4d60 2024-09-13T16:47:57.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.633+0000 7f99e5ffb640 1 -- 172.21.15.39:0/4141332906 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a_metadata", "pg_num_min": 64}]=0 pool 'a_metadata' created v17) v1 ==== 128+0+0 (secure 0 0 0) 0x7f99e80c20a0 con 0x7f99f40a4d60 2024-09-13T16:47:57.638 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'a_metadata' created 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99d80666c0 msgr2=0x7f99d8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99d80666c0 0x7f99d8068b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99f4071ee0 tx=0x7f99f0009040 comp rx=0 tx=0).stop 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f40a4d60 msgr2=0x7f99f41477e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f40a4d60 0x7f99f41477e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f99e801a010 tx=0x7f99e8019180 comp rx=0 tx=0).stop 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 shutdown_connections 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99d80666c0 0x7f99d8068b80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99f4151cf0 0x7f99f41efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99f40aa2d0 0x7f99f414d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 --2- 172.21.15.39:0/4141332906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f40a4d60 0x7f99f41477e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 >> 172.21.15.39:0/4141332906 conn(0x7f99f40c5790 msgr2=0x7f99f4150260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 shutdown_connections 2024-09-13T16:47:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.641+0000 7f99fb4ff640 1 -- 172.21.15.39:0/4141332906 wait complete. 2024-09-13T16:47:57.662 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create a_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T16:47:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a8ec5640 1 Processor -- start 2024-09-13T16:47:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a8ec5640 1 -- start start 2024-09-13T16:47:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a8ec5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3a414bf30 0x7fc3a406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a8ec5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc3a406b360 0x7fc3a4152800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a8ec5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a4151bf0 con 0x7fc3a4155160 2024-09-13T16:47:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a8ec5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a4155660 con 0x7fc3a406b360 2024-09-13T16:47:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a8ec5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a4155940 con 0x7fc3a414bf30 2024-09-13T16:47:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a2d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc3a406b360 0x7fc3a4152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a2d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc3a406b360 0x7fc3a4152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49942/0 (socket says 172.21.15.39:49942) 2024-09-13T16:47:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a1d9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc3a4155160 0x7fc3a41520f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42660/0 (socket says 172.21.15.39:42660) 2024-09-13T16:47:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a2d9d640 1 -- 172.21.15.39:0/917298121 learned_addr learned my addr 172.21.15.39:0/917298121 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a259c640 1 --2- 172.21.15.39:0/917298121 >> v2:172.21.15.84:3300/0 conn(0x7fc3a414bf30 0x7fc3a406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a2d9d640 1 -- 172.21.15.39:0/917298121 >> v2:172.21.15.84:3300/0 conn(0x7fc3a414bf30 msgr2=0x7fc3a406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a2d9d640 1 --2- 172.21.15.39:0/917298121 >> v2:172.21.15.84:3300/0 conn(0x7fc3a414bf30 0x7fc3a406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a2d9d640 1 -- 172.21.15.39:0/917298121 >> v1:172.21.15.39:6789/0 conn(0x7fc3a4155160 legacy=0x7fc3a41520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a2d9d640 1 -- 172.21.15.39:0/917298121 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4155c20 con 0x7fc3a406b360 2024-09-13T16:47:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.789+0000 7fc3a159a640 1 -- 172.21.15.39:0/917298121 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320117005 0 0) 0x7fc3a4151bf0 con 0x7fc3a4155160 2024-09-13T16:47:57.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a2d9d640 1 --2- 172.21.15.39:0/917298121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a406b360 0x7fc3a4152800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4066cf0 tx=0x7fc39401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecffed2f788627f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a159a640 1 -- 172.21.15.39:0/917298121 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940aa430 con 0x7fc3a406b360 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a159a640 1 -- 172.21.15.39:0/917298121 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3940aa590 con 0x7fc3a406b360 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a159a640 1 -- 172.21.15.39:0/917298121 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940aa920 con 0x7fc3a406b360 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/917298121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a406b360 msgr2=0x7fc3a4152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 --2- 172.21.15.39:0/917298121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a406b360 0x7fc3a4152800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4066cf0 tx=0x7fc39401c4f0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/917298121 shutdown_connections 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 --2- 172.21.15.39:0/917298121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a406b360 0x7fc3a4152800 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 --2- 172.21.15.39:0/917298121 >> v2:172.21.15.84:3300/0 conn(0x7fc3a414bf30 0x7fc3a406ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/917298121 >> 172.21.15.39:0/917298121 conn(0x7fc3a40c32a0 msgr2=0x7fc3a40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/917298121 shutdown_connections 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/917298121 wait complete. 2024-09-13T16:47:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 Processor -- start 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 -- start start 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a406b360 0x7fc3a40bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a414bf30 0x7fc3a40ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3a4155160 0x7fc3a41eca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a414b820 con 0x7fc3a414bf30 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a4142c30 con 0x7fc3a406b360 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a8ec5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a41ec560 con 0x7fc3a4155160 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a414bf30 0x7fc3a40ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a414bf30 0x7fc3a40ab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49954/0 (socket says 172.21.15.39:49954) 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a1d9b640 1 -- 172.21.15.39:0/1002135433 learned_addr learned my addr 172.21.15.39:0/1002135433 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a259c640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a406b360 0x7fc3a40bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a1d9b640 1 -- 172.21.15.39:0/1002135433 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3a4155160 msgr2=0x7fc3a41eca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a1d9b640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3a4155160 0x7fc3a41eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a1d9b640 1 -- 172.21.15.39:0/1002135433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a406b360 msgr2=0x7fc3a40bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a1d9b640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a406b360 0x7fc3a40bfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.793+0000 7fc3a1d9b640 1 -- 172.21.15.39:0/1002135433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3940094d0 con 0x7fc3a414bf30 2024-09-13T16:47:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc3a1d9b640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a414bf30 0x7fc3a40ab990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3980011f0 tx=0x7fc39800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc392ffd640 1 -- 172.21.15.39:0/1002135433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3980074e0 con 0x7fc3a414bf30 2024-09-13T16:47:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc392ffd640 1 -- 172.21.15.39:0/1002135433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc398007640 con 0x7fc3a414bf30 2024-09-13T16:47:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a4142ed0 con 0x7fc3a414bf30 2024-09-13T16:47:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc392ffd640 1 -- 172.21.15.39:0/1002135433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398004a10 con 0x7fc3a414bf30 2024-09-13T16:47:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a4141870 con 0x7fc3a414bf30 2024-09-13T16:47:57.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a41ec720 con 0x7fc3a414bf30 2024-09-13T16:47:57.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc392ffd640 1 -- 172.21.15.39:0/1002135433 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc398004b70 con 0x7fc3a414bf30 2024-09-13T16:47:57.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc392ffd640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3840666c0 0x7fc384068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:57.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc392ffd640 1 -- 172.21.15.39:0/1002135433 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5222+0+0 (secure 0 0 0) 0x7fc398083460 con 0x7fc3a414bf30 2024-09-13T16:47:57.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc3a259c640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3840666c0 0x7fc384068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:57.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.797+0000 7fc3a259c640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3840666c0 0x7fc384068b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009f30 tx=0x7fc39c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:57.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.801+0000 7fc392ffd640 1 -- 172.21.15.39:0/1002135433 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc398050a40 con 0x7fc3a414bf30 2024-09-13T16:47:57.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:57.957+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "a_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc3a406ae20 con 0x7fc3a414bf30 2024-09-13T16:47:58.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.637+0000 7fc392ffd640 1 -- 172.21.15.39:0/1002135433 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "a_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'a_data' created v18) v1 ==== 160+0+0 (secure 0 0 0) 0x7fc398054060 con 0x7fc3a414bf30 2024-09-13T16:47:58.642 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'a_data' created 2024-09-13T16:47:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3840666c0 msgr2=0x7fc384068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3840666c0 0x7fc384068b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009f30 tx=0x7fc39c00b040 comp rx=0 tx=0).stop 2024-09-13T16:47:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a414bf30 msgr2=0x7fc3a40ab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a414bf30 0x7fc3a40ab990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3980011f0 tx=0x7fc39800c9b0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 shutdown_connections 2024-09-13T16:47:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3840666c0 0x7fc384068b80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3a4155160 0x7fc3a41eca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a414bf30 0x7fc3a40ab990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 --2- 172.21.15.39:0/1002135433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a406b360 0x7fc3a40bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 >> 172.21.15.39:0/1002135433 conn(0x7fc3a40c32a0 msgr2=0x7fc3a40c4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 shutdown_connections 2024-09-13T16:47:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.645+0000 7fc3a8ec5640 1 -- 172.21.15.39:0/1002135433 wait complete. 2024-09-13T16:47:58.681 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new a a_metadata a_data 2024-09-13T16:47:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.801+0000 7fbb3982a640 1 Processor -- start 2024-09-13T16:47:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb3982a640 1 -- start start 2024-09-13T16:47:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb3982a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb341515b0 0x7fbb3406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb3982a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbb3406b360 0x7fbb34154480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb3982a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb340c6570 con 0x7fbb340c0670 2024-09-13T16:47:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb3982a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb340a61c0 con 0x7fbb3406b360 2024-09-13T16:47:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb3982a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c46a0 con 0x7fbb341515b0 2024-09-13T16:47:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb327fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbb340c0670 0x7fbb34153d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42674/0 (socket says 172.21.15.39:42674) 2024-09-13T16:47:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb327fc640 1 -- 172.21.15.39:0/4195701703 learned_addr learned my addr 172.21.15.39:0/4195701703 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:47:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb337fe640 1 --2- 172.21.15.39:0/4195701703 >> v2:172.21.15.84:3300/0 conn(0x7fbb3406b360 0x7fbb34154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb32ffd640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb341515b0 0x7fbb3406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801516145 0 0) 0x7fbb340c6570 con 0x7fbb340c0670 2024-09-13T16:47:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb1c003530 con 0x7fbb340c0670 2024-09-13T16:47:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 340162248 0 0) 0x7fbb1c003530 con 0x7fbb340c0670 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb341515b0 msgr2=0x7fbb3406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb341515b0 0x7fbb3406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 >> v2:172.21.15.84:3300/0 conn(0x7fbb3406b360 msgr2=0x7fbb34154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 --2- 172.21.15.39:0/4195701703 >> v2:172.21.15.84:3300/0 conn(0x7fbb3406b360 0x7fbb34154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340c4980 con 0x7fbb340c0670 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fbb28001680 con 0x7fbb340c0670 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 >> v1:172.21.15.39:6789/0 conn(0x7fbb340c0670 legacy=0x7fbb34153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb1c003980 0x7fbb1c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c0047d0 0x7fbb1c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb341515b0 0x7fbb1c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340a61c0 con 0x7fbb1c003980 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c46a0 con 0x7fbb1c0047d0 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c00ba10 con 0x7fbb341515b0 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb32ffd640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb1c003980 0x7fbb1c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb327fc640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c0047d0 0x7fbb1c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb337fe640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb341515b0 0x7fbb1c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb32ffd640 1 -- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb341515b0 msgr2=0x7fbb1c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb32ffd640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb341515b0 0x7fbb1c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb32ffd640 1 -- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c0047d0 msgr2=0x7fbb1c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb32ffd640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c0047d0 0x7fbb1c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.805+0000 7fbb32ffd640 1 -- 172.21.15.39:0/4195701703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c003530 con 0x7fbb1c003980 2024-09-13T16:47:58.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb32ffd640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb1c003980 0x7fbb1c003da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009ae0 tx=0x7fbb2401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb47cf5ee1e25bc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:58.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb240a5610 con 0x7fbb1c003980 2024-09-13T16:47:58.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb240a5770 con 0x7fbb1c003980 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb31ffb640 1 -- 172.21.15.39:0/4195701703 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb240a5b20 con 0x7fbb1c003980 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb3982a640 1 -- 172.21.15.39:0/4195701703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb1c003980 msgr2=0x7fbb1c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb3982a640 1 --2- 172.21.15.39:0/4195701703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb1c003980 0x7fbb1c003da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009ae0 tx=0x7fbb2401ca40 comp rx=0 tx=0).stop 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb337fe640 1 -- 172.21.15.39:0/4195701703 reap_dead start 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb3982a640 1 -- 172.21.15.39:0/4195701703 shutdown_connections 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb3982a640 1 -- 172.21.15.39:0/4195701703 >> 172.21.15.39:0/4195701703 conn(0x7fbb340c32a0 msgr2=0x7fbb340c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb3982a640 1 -- 172.21.15.39:0/4195701703 shutdown_connections 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb3982a640 1 -- 172.21.15.39:0/4195701703 wait complete. 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb3982a640 1 Processor -- start 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.809+0000 7fbb3982a640 1 -- start start 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb3982a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c003980 0x7fbb341edb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb3982a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb340a78a0 0x7fbb3414cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:58.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb3982a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb340a6c90 0x7fbb3414aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb3982a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340a80d0 con 0x7fbb340a6c90 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb3982a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34153240 con 0x7fbb1c003980 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb3982a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34157460 con 0x7fbb340a78a0 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb340a6c90 0x7fbb3414aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb340a6c90 0x7fbb3414aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49980/0 (socket says 172.21.15.39:49980) 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb337fe640 1 -- 172.21.15.39:0/181939188 learned_addr learned my addr 172.21.15.39:0/181939188 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb32ffd640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c003980 0x7fbb341edb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb327fc640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb340a78a0 0x7fbb3414cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb337fe640 1 -- 172.21.15.39:0/181939188 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb340a78a0 msgr2=0x7fbb3414cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb337fe640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb340a78a0 0x7fbb3414cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb337fe640 1 -- 172.21.15.39:0/181939188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c003980 msgr2=0x7fbb341edb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb337fe640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c003980 0x7fbb341edb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:58.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb337fe640 1 -- 172.21.15.39:0/181939188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34157740 con 0x7fbb340a6c90 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb32ffd640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c003980 0x7fbb341edb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb337fe640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb340a6c90 0x7fbb3414aca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb340c3270 tx=0x7fbb2c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb1bfff640 1 -- 172.21.15.39:0/181939188 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00e070 con 0x7fbb340a6c90 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb1bfff640 1 -- 172.21.15.39:0/181939188 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c011ed0 con 0x7fbb340a6c90 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb340a6490 con 0x7fbb340a6c90 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb1bfff640 1 -- 172.21.15.39:0/181939188 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0039a0 con 0x7fbb340a6c90 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3414d430 con 0x7fbb340a6c90 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb34153d70 con 0x7fbb340a6c90 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb1bfff640 1 -- 172.21.15.39:0/181939188 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb2c003b00 con 0x7fbb340a6c90 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb1bfff640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb200666c0 0x7fbb20068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb1bfff640 1 -- 172.21.15.39:0/181939188 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5599+0+0 (secure 0 0 0) 0x7fbb2c084740 con 0x7fbb340a6c90 2024-09-13T16:47:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb32ffd640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb200666c0 0x7fbb20068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:58.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.813+0000 7fbb32ffd640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb200666c0 0x7fbb20068b80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb240af000 tx=0x7fbb240046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:58.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:58.817+0000 7fbb1bfff640 1 -- 172.21.15.39:0/181939188 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbb2c051b20 con 0x7fbb340a6c90 2024-09-13T16:47:59.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.017+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "a", "metadata": "a_metadata", "data": "a_data"} v 0) v1 -- 0x7fbb340a6940 con 0x7fbb340a6c90 2024-09-13T16:47:59.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.025+0000 7fbb1bfff640 1 -- 172.21.15.39:0/181939188 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "a", "metadata": "a_metadata", "data": "a_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 157+0+0 (secure 0 0 0) 0x7fbb2c055140 con 0x7fbb340a6c90 2024-09-13T16:47:59.031 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T16:47:59.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb200666c0 msgr2=0x7fbb20068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb200666c0 0x7fbb20068b80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb240af000 tx=0x7fbb240046d0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb340a6c90 msgr2=0x7fbb3414aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb340a6c90 0x7fbb3414aca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb340c3270 tx=0x7fbb2c0115a0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 shutdown_connections 2024-09-13T16:47:59.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb200666c0 0x7fbb20068b80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb340a6c90 0x7fbb3414aca0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb340a78a0 0x7fbb3414cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 --2- 172.21.15.39:0/181939188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb1c003980 0x7fbb341edb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.033+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 >> 172.21.15.39:0/181939188 conn(0x7fbb340c32a0 msgr2=0x7fbb340c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:59.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.061+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 shutdown_connections 2024-09-13T16:47:59.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.061+0000 7fbb3982a640 1 -- 172.21.15.39:0/181939188 wait complete. 2024-09-13T16:47:59.081 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:47:59.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0eaf3f640 1 Processor -- start 2024-09-13T16:47:59.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0eaf3f640 1 -- start start 2024-09-13T16:47:59.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0eaf3f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e40c20d0 0x7fe0e40c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0eaf3f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe0e40c1790 0x7fe0e4157800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0eaf3f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e4157370 con 0x7fe0e406d370 2024-09-13T16:47:59.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0eaf3f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e40c04a0 con 0x7fe0e40c1790 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0eaf3f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e4153df0 con 0x7fe0e40c20d0 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e94dc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe0e406d370 0x7fe0e4157d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42690/0 (socket says 172.21.15.39:42690) 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e94dc640 1 -- 172.21.15.39:0/1213748795 learned_addr learned my addr 172.21.15.39:0/1213748795 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e3fff640 1 --2- 172.21.15.39:0/1213748795 >> v2:172.21.15.84:3300/0 conn(0x7fe0e40c1790 0x7fe0e4157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e8cdb640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e40c20d0 0x7fe0e40c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045950303 0 0) 0x7fe0e4157370 con 0x7fe0e406d370 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0d0003530 con 0x7fe0e406d370 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 987257835 0 0) 0x7fe0d0003530 con 0x7fe0e406d370 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e40c20d0 msgr2=0x7fe0e40c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e40c20d0 0x7fe0e40c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 >> v2:172.21.15.84:3300/0 conn(0x7fe0e40c1790 msgr2=0x7fe0e4157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 --2- 172.21.15.39:0/1213748795 >> v2:172.21.15.84:3300/0 conn(0x7fe0e40c1790 0x7fe0e4157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e406c700 con 0x7fe0e406d370 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fe0dc001680 con 0x7fe0e406d370 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 >> v1:172.21.15.39:6789/0 conn(0x7fe0e406d370 legacy=0x7fe0e4157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e40c20d0 0x7fe0d0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0d0003f00 0x7fe0d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0d0004d10 0x7fe0d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e4153df0 con 0x7fe0d0004d10 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d000b990 con 0x7fe0d0003f00 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d000bad0 con 0x7fe0e40c20d0 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e94dc640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0d0004d10 0x7fe0d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e3fff640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0d0003f00 0x7fe0d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e8cdb640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e40c20d0 0x7fe0d0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e94dc640 1 -- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e40c20d0 msgr2=0x7fe0d0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e94dc640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e40c20d0 0x7fe0d0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e94dc640 1 -- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0d0003f00 msgr2=0x7fe0d00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e94dc640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0d0003f00 0x7fe0d00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.193+0000 7fe0e94dc640 1 -- 172.21.15.39:0/1213748795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d0003530 con 0x7fe0d0004d10 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0e94dc640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0d0004d10 0x7fe0d0005190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0e406d820 tx=0x7fe0dc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e13ce6a545c97fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0a5930 con 0x7fe0d0004d10 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc0a5a90 con 0x7fe0d0004d10 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0e37fe640 1 -- 172.21.15.39:0/1213748795 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0a5e40 con 0x7fe0d0004d10 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/1213748795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0d0004d10 msgr2=0x7fe0d0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0eaf3f640 1 --2- 172.21.15.39:0/1213748795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0d0004d10 0x7fe0d0005190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0e406d820 tx=0x7fe0dc01cc60 comp rx=0 tx=0).stop 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0e94dc640 1 -- 172.21.15.39:0/1213748795 reap_dead start 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/1213748795 shutdown_connections 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/1213748795 >> 172.21.15.39:0/1213748795 conn(0x7fe0e4069e10 msgr2=0x7fe0e406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/1213748795 shutdown_connections 2024-09-13T16:47:59.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/1213748795 wait complete. 2024-09-13T16:47:59.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0eaf3f640 1 Processor -- start 2024-09-13T16:47:59.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.197+0000 7fe0eaf3f640 1 -- start start 2024-09-13T16:47:59.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0eaf3f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e4156d20 0x7fe0e406fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0eaf3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e414a8e0 0x7fe0e414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0eaf3f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e4070f40 0x7fe0e40713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0eaf3f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e41dd900 con 0x7fe0e4070f40 2024-09-13T16:47:59.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0eaf3f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e41ed690 con 0x7fe0e414a8e0 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0eaf3f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e41ed970 con 0x7fe0e4156d20 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e94dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e4070f40 0x7fe0e40713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e414a8e0 0x7fe0e414ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e94dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e4070f40 0x7fe0e40713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49992/0 (socket says 172.21.15.39:49992) 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e94dc640 1 -- 172.21.15.39:0/3862459134 learned_addr learned my addr 172.21.15.39:0/3862459134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e8cdb640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e4156d20 0x7fe0e406fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e94dc640 1 -- 172.21.15.39:0/3862459134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e4156d20 msgr2=0x7fe0e406fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e94dc640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e4156d20 0x7fe0e406fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e94dc640 1 -- 172.21.15.39:0/3862459134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e414a8e0 msgr2=0x7fe0e414ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e94dc640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e414a8e0 0x7fe0e414ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e94dc640 1 -- 172.21.15.39:0/3862459134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e4070740 con 0x7fe0e4070f40 2024-09-13T16:47:59.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e94dc640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e4070f40 0x7fe0e40713c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc01d500 tx=0x7fe0d0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e17fa640 1 -- 172.21.15.39:0/3862459134 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0a5930 con 0x7fe0e4070f40 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e17fa640 1 -- 172.21.15.39:0/3862459134 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc000e50 con 0x7fe0e4070f40 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e17fa640 1 -- 172.21.15.39:0/3862459134 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0072f0 con 0x7fe0e4070f40 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e4070a20 con 0x7fe0e4070f40 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e414d0d0 con 0x7fe0e4070f40 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e41ed7d0 con 0x7fe0e4070f40 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e17fa640 1 -- 172.21.15.39:0/3862459134 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe0dc007450 con 0x7fe0e4070f40 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e17fa640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0c4066690 0x7fe0c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e17fa640 1 -- 172.21.15.39:0/3862459134 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5689+0+0 (secure 0 0 0) 0x7fe0dc0b4030 con 0x7fe0e4070f40 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.201+0000 7fe0e8cdb640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0c4066690 0x7fe0c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.205+0000 7fe0e8cdb640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0c4066690 0x7fe0c4068b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40ab9c0 tx=0x7fe0d4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:59.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.209+0000 7fe0e17fa640 1 -- 172.21.15.39:0/3862459134 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe0e41ed7d0 con 0x7fe0e4070f40 2024-09-13T16:47:59.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.361+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0e41ed7d0 con 0x7fe0e4070f40 2024-09-13T16:47:59.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.365+0000 7fe0e17fa640 1 -- 172.21.15.39:0/3862459134 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16450 (secure 0 0 0) 0x7fe0e41ed7d0 con 0x7fe0e4070f40 2024-09-13T16:47:59.369 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:47:59.369 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:47:59.022565+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:47:59.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0c4066690 msgr2=0x7fe0c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0c4066690 0x7fe0c4068b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40ab9c0 tx=0x7fe0d4009190 comp rx=0 tx=0).stop 2024-09-13T16:47:59.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e4070f40 msgr2=0x7fe0e40713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e4070f40 0x7fe0e40713c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc01d500 tx=0x7fe0d0005800 comp rx=0 tx=0).stop 2024-09-13T16:47:59.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 shutdown_connections 2024-09-13T16:47:59.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0c4066690 0x7fe0c4068b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0e414a8e0 0x7fe0e414ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e4070f40 0x7fe0e40713c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 --2- 172.21.15.39:0/3862459134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0e4156d20 0x7fe0e406fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 >> 172.21.15.39:0/3862459134 conn(0x7fe0e4069e10 msgr2=0x7fe0e4157700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:59.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 shutdown_connections 2024-09-13T16:47:59.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.369+0000 7fe0eaf3f640 1 -- 172.21.15.39:0/3862459134 wait complete. 2024-09-13T16:47:59.388 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:47:59.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8ef015b640 1 Processor -- start 2024-09-13T16:47:59.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8ef015b640 1 -- start start 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8ef015b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8ee81556d0 0x7f8ee80c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8ef015b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee80c7610 0x7f8ee8151b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8ef015b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee806feb0 con 0x7f8ee80a72b0 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8ef015b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee80aae50 con 0x7f8ee81556d0 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8ef015b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee8155ab0 con 0x7f8ee80c7610 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eee6f8640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8ee80a72b0 0x7f8ee8071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42702/0 (socket says 172.21.15.39:42702) 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eee6f8640 1 -- 172.21.15.39:0/2722831086 learned_addr learned my addr 172.21.15.39:0/2722831086 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eedef7640 1 --2- 172.21.15.39:0/2722831086 >> v2:172.21.15.84:3300/0 conn(0x7f8ee81556d0 0x7f8ee80c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eed6f6640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee80c7610 0x7f8ee8151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973960261 0 0) 0x7f8ee806feb0 con 0x7f8ee80a72b0 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed4003530 con 0x7f8ee80a72b0 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 299695772 0 0) 0x7f8ed4003530 con 0x7f8ee80a72b0 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee80c7610 msgr2=0x7f8ee8151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee80c7610 0x7f8ee8151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 >> v2:172.21.15.84:3300/0 conn(0x7f8ee81556d0 msgr2=0x7f8ee80c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 --2- 172.21.15.39:0/2722831086 >> v2:172.21.15.84:3300/0 conn(0x7f8ee81556d0 0x7f8ee80c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80c3c20 con 0x7f8ee80a72b0 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f8ee0001a00 con 0x7f8ee80a72b0 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eed6f6640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee80c7610 0x7f8ee8151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:47:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 >> v1:172.21.15.39:6789/0 conn(0x7f8ee80a72b0 legacy=0x7f8ee8071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ed4003980 0x7f8ed4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed40042a0 0x7f8ed40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ed400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed400bed0 con 0x7f8ed4004f90 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed400c010 con 0x7f8ed40042a0 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed400c150 con 0x7f8ed4003980 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eee6f8640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ed400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eed6f6640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed40042a0 0x7f8ed40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eedef7640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ed4003980 0x7f8ed4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eee6f8640 1 -- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ed4003980 msgr2=0x7f8ed4003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eee6f8640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ed4003980 0x7f8ed4003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eee6f8640 1 -- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed40042a0 msgr2=0x7f8ed40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eee6f8640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ed40042a0 0x7f8ed40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.497+0000 7f8eee6f8640 1 -- 172.21.15.39:0/2722831086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed4003530 con 0x7f8ed4004f90 2024-09-13T16:47:59.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eee6f8640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ed400b990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000a830 tx=0x7f8ee001d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b818296f9432d2c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:59.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00ad630 con 0x7f8ed4004f90 2024-09-13T16:47:59.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee00a8070 con 0x7f8ed4004f90 2024-09-13T16:47:59.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eecef5640 1 -- 172.21.15.39:0/2722831086 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00ada30 con 0x7f8ed4004f90 2024-09-13T16:47:59.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 -- 172.21.15.39:0/2722831086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 msgr2=0x7f8ed400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 --2- 172.21.15.39:0/2722831086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ed400b990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000a830 tx=0x7f8ee001d300 comp rx=0 tx=0).stop 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eee6f8640 1 -- 172.21.15.39:0/2722831086 reap_dead start 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 -- 172.21.15.39:0/2722831086 shutdown_connections 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 -- 172.21.15.39:0/2722831086 >> 172.21.15.39:0/2722831086 conn(0x7f8ee80a97b0 msgr2=0x7f8ee80a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 -- 172.21.15.39:0/2722831086 shutdown_connections 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 -- 172.21.15.39:0/2722831086 wait complete. 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 Processor -- start 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 -- start start 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ee806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee8150ca0 0x7f8ee814e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee814ed10 0x7f8ee81eb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee81eaf60 con 0x7f8ed4004f90 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80c2090 con 0x7f8ee814ed10 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8ef015b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee806eac0 con 0x7f8ee8150ca0 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eedef7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ee806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eedef7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ee806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50006/0 (socket says 172.21.15.39:50006) 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eedef7640 1 -- 172.21.15.39:0/1472319930 learned_addr learned my addr 172.21.15.39:0/1472319930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eed6f6640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee8150ca0 0x7f8ee814e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eee6f8640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee814ed10 0x7f8ee81eb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eedef7640 1 -- 172.21.15.39:0/1472319930 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee8150ca0 msgr2=0x7f8ee814e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eedef7640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee8150ca0 0x7f8ee814e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eedef7640 1 -- 172.21.15.39:0/1472319930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee814ed10 msgr2=0x7f8ee81eb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eedef7640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee814ed10 0x7f8ee81eb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eedef7640 1 -- 172.21.15.39:0/1472319930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee806ed10 con 0x7f8ed4004f90 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eed6f6640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee8150ca0 0x7f8ee814e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.501+0000 7f8eedef7640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ee806dbe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4011690 tx=0x7f8ee4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8ede7fc640 1 -- 172.21.15.39:0/1472319930 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400f960 con 0x7f8ed4004f90 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8ede7fc640 1 -- 172.21.15.39:0/1472319930 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee400fac0 con 0x7f8ed4004f90 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee806eff0 con 0x7f8ed4004f90 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8ede7fc640 1 -- 172.21.15.39:0/1472319930 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400fe30 con 0x7f8ed4004f90 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee806e7d0 con 0x7f8ed4004f90 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee8066ac0 con 0x7f8ed4004f90 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8ede7fc640 1 -- 172.21.15.39:0/1472319930 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ee4015460 con 0x7f8ed4004f90 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8ede7fc640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ecc0666c0 0x7f8ecc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8ede7fc640 1 -- 172.21.15.39:0/1472319930 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f8ee4084970 con 0x7f8ed4004f90 2024-09-13T16:47:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8eed6f6640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ecc0666c0 0x7f8ecc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.505+0000 7f8eed6f6640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ecc0666c0 0x7f8ecc068b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80c1c80 tx=0x7f8ed8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.509+0000 7f8ede7fc640 1 -- 172.21.15.39:0/1472319930 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8ee4051d80 con 0x7f8ed4004f90 2024-09-13T16:47:59.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.665+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ee806fe10 con 0x7f8ed4004f90 2024-09-13T16:47:59.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.669+0000 7f8ede7fc640 1 -- 172.21.15.39:0/1472319930 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16450 (secure 0 0 0) 0x7f8ee40553a0 con 0x7f8ed4004f90 2024-09-13T16:47:59.672 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:47:59.672 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:47:59.022565+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:47:59.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.669+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ecc0666c0 msgr2=0x7f8ecc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.669+0000 7f8ef015b640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ecc0666c0 0x7f8ecc068b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80c1c80 tx=0x7f8ed8008040 comp rx=0 tx=0).stop 2024-09-13T16:47:59.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.669+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 msgr2=0x7f8ee806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.669+0000 7f8ef015b640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ee806dbe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4011690 tx=0x7f8ee4011b50 comp rx=0 tx=0).stop 2024-09-13T16:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.673+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 shutdown_connections 2024-09-13T16:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.673+0000 7f8ef015b640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ecc0666c0 0x7f8ecc068b80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.673+0000 7f8ef015b640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ee814ed10 0x7f8ee81eb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.673+0000 7f8ef015b640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ee8150ca0 0x7f8ee814e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.673+0000 7f8ef015b640 1 --2- 172.21.15.39:0/1472319930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed4004f90 0x7f8ee806dbe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.673+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 >> 172.21.15.39:0/1472319930 conn(0x7f8ee80a97b0 msgr2=0x7f8ee80c5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.673+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 shutdown_connections 2024-09-13T16:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.673+0000 7f8ef015b640 1 -- 172.21.15.39:0/1472319930 wait complete. 2024-09-13T16:47:59.690 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a standby_count_wanted 0 2024-09-13T16:47:59.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a68b3640 1 Processor -- start 2024-09-13T16:47:59.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a68b3640 1 -- start start 2024-09-13T16:47:59.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a68b3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 0x7fb7a0152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a68b3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a0155d20 con 0x7fb7a00a4d60 2024-09-13T16:47:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a68b3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a00c4750 con 0x7fb7a00aa2d0 2024-09-13T16:47:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a68b3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a0155b30 con 0x7fb7a0151cf0 2024-09-13T16:47:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb7a00a4d60 0x7fb7a014ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42718/0 (socket says 172.21.15.39:42718) 2024-09-13T16:47:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79f7fe640 1 -- 172.21.15.39:0/3731181697 learned_addr learned my addr 172.21.15.39:0/3731181697 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:47:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626751947 0 0) 0x7fb7a0155d20 con 0x7fb7a00a4d60 2024-09-13T16:47:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a4e50640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 0x7fb7a0152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb788003530 con 0x7fb7a00a4d60 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1466095401 0 0) 0x7fb788003530 con 0x7fb7a00a4d60 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 msgr2=0x7fb7a0152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 0x7fb7a0152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 >> v1:172.21.15.84:6789/0 conn(0x7fb7a00aa2d0 legacy=0x7fb7a00aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a0155fc0 con 0x7fb7a00a4d60 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fb790001680 con 0x7fb7a00a4d60 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a4e50640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 0x7fb7a0152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 >> v1:172.21.15.39:6789/0 conn(0x7fb7a00a4d60 legacy=0x7fb7a014ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 0x7fb7880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb788003fa0 0x7fb7880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb788005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78800bb50 con 0x7fb788004d10 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78800bcc0 con 0x7fb788003fa0 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78800be80 con 0x7fb7a0151cf0 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a4e50640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb788005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79ffff640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 0x7fb7880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79f7fe640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb788003fa0 0x7fb7880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a4e50640 1 -- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 msgr2=0x7fb7880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a4e50640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 0x7fb7880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a4e50640 1 -- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb788003fa0 msgr2=0x7fb7880047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:47:59.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a4e50640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb788003fa0 0x7fb7880047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb7a4e50640 1 -- 172.21.15.39:0/3731181697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb788003530 con 0x7fb788004d10 2024-09-13T16:47:59.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.805+0000 7fb79ffff640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a0151cf0 0x7fb7880039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:47:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a4e50640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb788005190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb7980073e0 tx=0x7fb79801cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a442e2b2b55994c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7980a5cc0 con 0x7fb788004d10 2024-09-13T16:47:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7980a5e20 con 0x7fb788004d10 2024-09-13T16:47:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79effd640 1 -- 172.21.15.39:0/3731181697 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7980ad640 con 0x7fb788004d10 2024-09-13T16:47:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 -- 172.21.15.39:0/3731181697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 msgr2=0x7fb788005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 --2- 172.21.15.39:0/3731181697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb788005190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb7980073e0 tx=0x7fb79801cec0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a4e50640 1 -- 172.21.15.39:0/3731181697 reap_dead start 2024-09-13T16:47:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 -- 172.21.15.39:0/3731181697 shutdown_connections 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 -- 172.21.15.39:0/3731181697 >> 172.21.15.39:0/3731181697 conn(0x7fb7a00c5790 msgr2=0x7fb7a00c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 -- 172.21.15.39:0/3731181697 shutdown_connections 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 -- 172.21.15.39:0/3731181697 wait complete. 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 Processor -- start 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 -- start start 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb7a00c2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7a01448a0 0x7fb7a0144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a00c1380 0x7fb7a00c1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a0146a60 con 0x7fb788004d10 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a01e8d80 con 0x7fb7a01448a0 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a68b3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00c03d0 con 0x7fb7a00c1380 2024-09-13T16:47:59.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb7a00c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb7a00c2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50016/0 (socket says 172.21.15.39:50016) 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79ffff640 1 -- 172.21.15.39:0/660296580 learned_addr learned my addr 172.21.15.39:0/660296580 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79f7fe640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7a01448a0 0x7fb7a0144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb7a4e50640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a00c1380 0x7fb7a00c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79ffff640 1 -- 172.21.15.39:0/660296580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a00c1380 msgr2=0x7fb7a00c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79ffff640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a00c1380 0x7fb7a00c1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79ffff640 1 -- 172.21.15.39:0/660296580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7a01448a0 msgr2=0x7fb7a0144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79ffff640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7a01448a0 0x7fb7a0144d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.809+0000 7fb79ffff640 1 -- 172.21.15.39:0/660296580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a00c0930 con 0x7fb788004d10 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb79ffff640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb7a00c2740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb79400e7b0 tx=0x7fb79400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb79cff9640 1 -- 172.21.15.39:0/660296580 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940050d0 con 0x7fb788004d10 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb79cff9640 1 -- 172.21.15.39:0/660296580 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794005230 con 0x7fb788004d10 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb79cff9640 1 -- 172.21.15.39:0/660296580 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79400c640 con 0x7fb788004d10 2024-09-13T16:47:59.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a00c0ef0 con 0x7fb788004d10 2024-09-13T16:47:59.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a00c0b80 con 0x7fb788004d10 2024-09-13T16:47:59.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a0155b30 con 0x7fb788004d10 2024-09-13T16:47:59.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb79cff9640 1 -- 172.21.15.39:0/660296580 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb79400c7a0 con 0x7fb788004d10 2024-09-13T16:47:59.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb79cff9640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb78c0666c0 0x7fb78c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:47:59.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb79f7fe640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb78c0666c0 0x7fb78c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:47:59.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb79cff9640 1 -- 172.21.15.39:0/660296580 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5689+0+0 (secure 0 0 0) 0x7fb794083ec0 con 0x7fb788004d10 2024-09-13T16:47:59.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.813+0000 7fb79f7fe640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb78c0666c0 0x7fb78c068b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb7a014ef20 tx=0x7fb790009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:47:59.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:47:59.817+0000 7fb79cff9640 1 -- 172.21.15.39:0/660296580 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb794051390 con 0x7fb788004d10 2024-09-13T16:48:00.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.013+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb7a00c0090 con 0x7fb788004d10 2024-09-13T16:48:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.669+0000 7fb79cff9640 1 -- 172.21.15.39:0/660296580 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 113+0+0 (secure 0 0 0) 0x7fb7940549b0 con 0x7fb788004d10 2024-09-13T16:48:00.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb78c0666c0 msgr2=0x7fb78c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:00.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb78c0666c0 0x7fb78c068b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb7a014ef20 tx=0x7fb790009040 comp rx=0 tx=0).stop 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 msgr2=0x7fb7a00c2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb7a00c2740 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb79400e7b0 tx=0x7fb79400ec70 comp rx=0 tx=0).stop 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 shutdown_connections 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb78c0666c0 0x7fb78c068b80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7a01448a0 0x7fb7a0144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7a00c1380 0x7fb7a00c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 --2- 172.21.15.39:0/660296580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb788004d10 0x7fb7a00c2740 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 >> 172.21.15.39:0/660296580 conn(0x7fb7a00c5790 msgr2=0x7fb7a0151160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 shutdown_connections 2024-09-13T16:48:00.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.673+0000 7fb7a68b3640 1 -- 172.21.15.39:0/660296580 wait complete. 2024-09-13T16:48:00.700 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-09-13T16:48:00.700 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a allow_standby_replay false 2024-09-13T16:48:00.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e5666a640 1 Processor -- start 2024-09-13T16:48:00.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e5666a640 1 -- start start 2024-09-13T16:48:00.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e5666a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5e500a4d60 0x7f5e5014ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:00.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e5666a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5e5014f4b0 0x7f5e50151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:00.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e5666a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e50155d20 con 0x7f5e500aa2d0 2024-09-13T16:48:00.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e5666a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e50155b30 con 0x7f5e500a4d60 2024-09-13T16:48:00.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e5666a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e500aadc0 con 0x7f5e5014f4b0 2024-09-13T16:48:00.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e4f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5e500a4d60 0x7f5e5014ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:00.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e4ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5e500aa2d0 0x7f5e500aa6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38620/0 (socket says 172.21.15.39:38620) 2024-09-13T16:48:00.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e4ffff640 1 -- 172.21.15.39:0/2152601026 learned_addr learned my addr 172.21.15.39:0/2152601026 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:00.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e54c07640 1 --2- 172.21.15.39:0/2152601026 >> v2:172.21.15.84:3300/0 conn(0x7f5e5014f4b0 0x7f5e50151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:00.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e4effd640 1 -- 172.21.15.39:0/2152601026 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135876370 0 0) 0x7f5e50155d20 con 0x7f5e500aa2d0 2024-09-13T16:48:00.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e4effd640 1 -- 172.21.15.39:0/2152601026 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e38003530 con 0x7f5e500aa2d0 2024-09-13T16:48:00.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e4f7fe640 1 -- 172.21.15.39:0/2152601026 >> v2:172.21.15.84:3300/0 conn(0x7f5e5014f4b0 msgr2=0x7f5e50151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:00.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e4f7fe640 1 --2- 172.21.15.39:0/2152601026 >> v2:172.21.15.84:3300/0 conn(0x7f5e5014f4b0 0x7f5e50151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e4f7fe640 1 -- 172.21.15.39:0/2152601026 >> v1:172.21.15.39:6789/0 conn(0x7f5e500aa2d0 legacy=0x7f5e500aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:00.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.833+0000 7f5e4f7fe640 1 -- 172.21.15.39:0/2152601026 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e500c4750 con 0x7f5e500a4d60 2024-09-13T16:48:00.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e4f7fe640 1 --2- 172.21.15.39:0/2152601026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500a4d60 0x7f5e5014ef70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e500c6640 tx=0x7f5e4401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f62d8cde784d9dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:00.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e4effd640 1 -- 172.21.15.39:0/2152601026 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50152a40 con 0x7f5e500a4d60 2024-09-13T16:48:00.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e4effd640 1 -- 172.21.15.39:0/2152601026 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e440a7050 con 0x7f5e500a4d60 2024-09-13T16:48:00.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e4effd640 1 -- 172.21.15.39:0/2152601026 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440ab820 con 0x7f5e500a4d60 2024-09-13T16:48:00.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 -- 172.21.15.39:0/2152601026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500a4d60 msgr2=0x7f5e5014ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:00.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 --2- 172.21.15.39:0/2152601026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500a4d60 0x7f5e5014ef70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e500c6640 tx=0x7f5e4401c3a0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 -- 172.21.15.39:0/2152601026 shutdown_connections 2024-09-13T16:48:00.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 --2- 172.21.15.39:0/2152601026 >> v2:172.21.15.84:3300/0 conn(0x7f5e5014f4b0 0x7f5e50151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 --2- 172.21.15.39:0/2152601026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500a4d60 0x7f5e5014ef70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 -- 172.21.15.39:0/2152601026 >> 172.21.15.39:0/2152601026 conn(0x7f5e500c5790 msgr2=0x7f5e500c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:00.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 -- 172.21.15.39:0/2152601026 shutdown_connections 2024-09-13T16:48:00.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 -- 172.21.15.39:0/2152601026 wait complete. 2024-09-13T16:48:00.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 Processor -- start 2024-09-13T16:48:00.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.837+0000 7f5e5666a640 1 -- start start 2024-09-13T16:48:00.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e5666a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e500a4d60 0x7f5e501433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:00.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e5666a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500aa2d0 0x7f5e50153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:00.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e5666a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e5014f4b0 0x7f5e50153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:00.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e5666a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e501514e0 con 0x7f5e500aa2d0 2024-09-13T16:48:00.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e5666a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50143d10 con 0x7f5e500a4d60 2024-09-13T16:48:00.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e5666a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50148fa0 con 0x7f5e5014f4b0 2024-09-13T16:48:00.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500aa2d0 0x7f5e50153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500aa2d0 0x7f5e50153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41048/0 (socket says 172.21.15.39:41048) 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4f7fe640 1 -- 172.21.15.39:0/589392272 learned_addr learned my addr 172.21.15.39:0/589392272 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4ffff640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e500a4d60 0x7f5e501433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e54c07640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e5014f4b0 0x7f5e50153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4f7fe640 1 -- 172.21.15.39:0/589392272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e5014f4b0 msgr2=0x7f5e50153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4f7fe640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e5014f4b0 0x7f5e50153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4f7fe640 1 -- 172.21.15.39:0/589392272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e500a4d60 msgr2=0x7f5e501433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4f7fe640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e500a4d60 0x7f5e501433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4f7fe640 1 -- 172.21.15.39:0/589392272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e44009420 con 0x7f5e500aa2d0 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e54c07640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e5014f4b0 0x7f5e50153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4f7fe640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500aa2d0 0x7f5e50153980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e440ae040 tx=0x7f5e44019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:00.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4cff9640 1 -- 172.21.15.39:0/589392272 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440b4430 con 0x7f5e500aa2d0 2024-09-13T16:48:00.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4cff9640 1 -- 172.21.15.39:0/589392272 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e440b4590 con 0x7f5e500aa2d0 2024-09-13T16:48:00.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e50151200 con 0x7f5e500aa2d0 2024-09-13T16:48:00.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4cff9640 1 -- 172.21.15.39:0/589392272 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e440b4940 con 0x7f5e500aa2d0 2024-09-13T16:48:00.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e50142bd0 con 0x7f5e500aa2d0 2024-09-13T16:48:00.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e50142dc0 con 0x7f5e500aa2d0 2024-09-13T16:48:00.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4cff9640 1 -- 172.21.15.39:0/589392272 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e440b2040 con 0x7f5e500aa2d0 2024-09-13T16:48:00.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4cff9640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e3c066690 0x7f5e3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:00.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4cff9640 1 -- 172.21.15.39:0/589392272 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f5e4411e030 con 0x7f5e500aa2d0 2024-09-13T16:48:00.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4ffff640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e3c066690 0x7f5e3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:00.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.841+0000 7f5e4ffff640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e3c066690 0x7f5e3c068b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e400011f0 tx=0x7f5e40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:00.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:00.845+0000 7f5e4cff9640 1 -- 172.21.15.39:0/589392272 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e440eb320 con 0x7f5e500aa2d0 2024-09-13T16:48:01.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.033+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5e50142750 con 0x7f5e500aa2d0 2024-09-13T16:48:01.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.673+0000 7f5e4cff9640 1 -- 172.21.15.39:0/589392272 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 117+0+0 (secure 0 0 0) 0x7f5e50142750 con 0x7f5e500aa2d0 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e3c066690 msgr2=0x7f5e3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e3c066690 0x7f5e3c068b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e400011f0 tx=0x7f5e40009040 comp rx=0 tx=0).stop 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500aa2d0 msgr2=0x7f5e50153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500aa2d0 0x7f5e50153980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e440ae040 tx=0x7f5e44019140 comp rx=0 tx=0).stop 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 shutdown_connections 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e3c066690 0x7f5e3c068b50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e5014f4b0 0x7f5e50153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e500aa2d0 0x7f5e50153980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 --2- 172.21.15.39:0/589392272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e500a4d60 0x7f5e501433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 >> 172.21.15.39:0/589392272 conn(0x7f5e500c5790 msgr2=0x7f5e500aa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:01.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 shutdown_connections 2024-09-13T16:48:01.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.677+0000 7f5e5666a640 1 -- 172.21.15.39:0/589392272 wait complete. 2024-09-13T16:48:01.702 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set a session_timeout 300 2024-09-13T16:48:01.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.829+0000 7f3eb3118640 1 Processor -- start 2024-09-13T16:48:01.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.829+0000 7f3eb3118640 1 -- start start 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb3118640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0aa2d0 0x7f3eac0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb3118640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac155d20 con 0x7f3eac151cf0 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb3118640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac0c4750 con 0x7f3eac0a4d60 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb3118640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac155b30 con 0x7f3eac0aa2d0 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb16b5640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f3eac151cf0 0x7f3eac152180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38632/0 (socket says 172.21.15.39:38632) 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb16b5640 1 -- 172.21.15.39:0/2275664425 learned_addr learned my addr 172.21.15.39:0/2275664425 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544709798 0 0) 0x7f3eac155d20 con 0x7f3eac151cf0 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e98003530 con 0x7f3eac151cf0 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb0eb4640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0aa2d0 0x7f3eac0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524791916 0 0) 0x7f3e98003530 con 0x7f3eac151cf0 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0aa2d0 msgr2=0x7f3eac0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:01.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0aa2d0 0x7f3eac0aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 >> v1:172.21.15.84:6789/0 conn(0x7f3eac0a4d60 legacy=0x7f3eac14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac155fc0 con 0x7f3eac151cf0 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f3ea4001680 con 0x7f3eac151cf0 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 >> v1:172.21.15.39:6789/0 conn(0x7f3eac151cf0 legacy=0x7f3eac152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e980039b0 0x7f3e98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0aa2d0 0x7f3e980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3e98005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9800bb40 con 0x7f3e98004d10 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9800bcb0 con 0x7f3e980039b0 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9800be70 con 0x7f3eac0aa2d0 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb16b5640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3e98005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb0eb4640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e980039b0 0x7f3e98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eabfff640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0aa2d0 0x7f3e980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb16b5640 1 -- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0aa2d0 msgr2=0x7f3e980047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb16b5640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0aa2d0 0x7f3e980047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb16b5640 1 -- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e980039b0 msgr2=0x7f3e98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb16b5640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e980039b0 0x7f3e98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.833+0000 7f3eb16b5640 1 -- 172.21.15.39:0/2275664425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98003530 con 0x7f3e98004d10 2024-09-13T16:48:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb16b5640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3e98005190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c1c0 tx=0x7f3ea401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99734773d8e56c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4005ae0 con 0x7f3e98004d10 2024-09-13T16:48:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea40ab070 con 0x7f3e98004d10 2024-09-13T16:48:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eab7fe640 1 -- 172.21.15.39:0/2275664425 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4007070 con 0x7f3e98004d10 2024-09-13T16:48:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- 172.21.15.39:0/2275664425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 msgr2=0x7f3e98005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 --2- 172.21.15.39:0/2275664425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3e98005190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c1c0 tx=0x7f3ea401f3b0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb16b5640 1 -- 172.21.15.39:0/2275664425 reap_dead start 2024-09-13T16:48:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- 172.21.15.39:0/2275664425 shutdown_connections 2024-09-13T16:48:01.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- 172.21.15.39:0/2275664425 >> 172.21.15.39:0/2275664425 conn(0x7f3eac0c5790 msgr2=0x7f3eac0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:01.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- 172.21.15.39:0/2275664425 shutdown_connections 2024-09-13T16:48:01.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- 172.21.15.39:0/2275664425 wait complete. 2024-09-13T16:48:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 Processor -- start 2024-09-13T16:48:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- start start 2024-09-13T16:48:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3eac14efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac1470a0 0x7f3eac146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0c2bb0 0x7f3eac146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0c2560 con 0x7f3e98004d10 2024-09-13T16:48:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1ed060 con 0x7f3eac1470a0 2024-09-13T16:48:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac14b2f0 con 0x7f3eac0c2bb0 2024-09-13T16:48:01.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb0eb4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3eac14efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb0eb4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3eac14efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41062/0 (socket says 172.21.15.39:41062) 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb0eb4640 1 -- 172.21.15.39:0/1654985714 learned_addr learned my addr 172.21.15.39:0/1654985714 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eabfff640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac1470a0 0x7f3eac146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb16b5640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0c2bb0 0x7f3eac146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb0eb4640 1 -- 172.21.15.39:0/1654985714 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0c2bb0 msgr2=0x7f3eac146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb0eb4640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0c2bb0 0x7f3eac146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb0eb4640 1 -- 172.21.15.39:0/1654985714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac1470a0 msgr2=0x7f3eac146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb0eb4640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac1470a0 0x7f3eac146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb0eb4640 1 -- 172.21.15.39:0/1654985714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac0c11e0 con 0x7f3e98004d10 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb0eb4640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3eac14efa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009cb0 tx=0x7f3e9c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3ea97fa640 1 -- 172.21.15.39:0/1654985714 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c014610 con 0x7f3e98004d10 2024-09-13T16:48:01.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3ea97fa640 1 -- 172.21.15.39:0/1654985714 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c00f050 con 0x7f3e98004d10 2024-09-13T16:48:01.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3ea97fa640 1 -- 172.21.15.39:0/1654985714 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c014990 con 0x7f3e98004d10 2024-09-13T16:48:01.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac1ed590 con 0x7f3e98004d10 2024-09-13T16:48:01.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.837+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac0c1ea0 con 0x7f3e98004d10 2024-09-13T16:48:01.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.841+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac152180 con 0x7f3e98004d10 2024-09-13T16:48:01.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.841+0000 7f3ea97fa640 1 -- 172.21.15.39:0/1654985714 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e9c014af0 con 0x7f3e98004d10 2024-09-13T16:48:01.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.841+0000 7f3ea97fa640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c0666c0 0x7f3e8c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:01.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.841+0000 7f3ea97fa640 1 -- 172.21.15.39:0/1654985714 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f3e9c0849a0 con 0x7f3e98004d10 2024-09-13T16:48:01.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.841+0000 7f3eabfff640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c0666c0 0x7f3e8c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:01.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.841+0000 7f3eabfff640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c0666c0 0x7f3e8c068b80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0c15e0 tx=0x7f3e94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:01.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:01.845+0000 7f3ea97fa640 1 -- 172.21.15.39:0/1654985714 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3e9c051db0 con 0x7f3e98004d10 2024-09-13T16:48:02.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.041+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "a", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f3eac1518f0 con 0x7f3e98004d10 2024-09-13T16:48:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3ea97fa640 1 -- 172.21.15.39:0/1654985714 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "a", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 110+0+0 (secure 0 0 0) 0x7f3e9c0553d0 con 0x7f3e98004d10 2024-09-13T16:48:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c0666c0 msgr2=0x7f3e8c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c0666c0 0x7f3e8c068b80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0c15e0 tx=0x7f3e94009040 comp rx=0 tx=0).stop 2024-09-13T16:48:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 msgr2=0x7f3eac14efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3eac14efa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009cb0 tx=0x7f3e9c007700 comp rx=0 tx=0).stop 2024-09-13T16:48:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 shutdown_connections 2024-09-13T16:48:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c0666c0 0x7f3e8c068b80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac1470a0 0x7f3eac146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac0c2bb0 0x7f3eac146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:02.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 --2- 172.21.15.39:0/1654985714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98004d10 0x7f3eac14efa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:02.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 >> 172.21.15.39:0/1654985714 conn(0x7f3eac0c5790 msgr2=0x7f3eac14e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:02.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 shutdown_connections 2024-09-13T16:48:02.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.693+0000 7f3eb3118640 1 -- 172.21.15.39:0/1654985714 wait complete. 2024-09-13T16:48:02.762 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'a' 2024-09-13T16:48:02.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:02.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.889+0000 7fe9adf15640 1 Processor -- start 2024-09-13T16:48:02.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.889+0000 7fe9adf15640 1 -- start start 2024-09-13T16:48:02.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9adf15640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9a806c870 0x7fe9a80aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:02.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9adf15640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9a80c40b0 0x7fe9a815c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:02.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9adf15640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a8152a80 con 0x7fe9a80aabc0 2024-09-13T16:48:02.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9adf15640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a80c30f0 con 0x7fe9a806c870 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9adf15640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a8151e00 con 0x7fe9a80c40b0 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a6ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9a806c870 0x7fe9a80aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a77fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe9a80aabc0 0x7fe9a80c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38646/0 (socket says 172.21.15.39:38646) 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a77fe640 1 -- 172.21.15.39:0/79763425 learned_addr learned my addr 172.21.15.39:0/79763425 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a7fff640 1 --2- 172.21.15.39:0/79763425 >> v2:172.21.15.84:3300/0 conn(0x7fe9a80c40b0 0x7fe9a815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a67fc640 1 -- 172.21.15.39:0/79763425 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654689965 0 0) 0x7fe998001120 con 0x7fe9a80aabc0 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a67fc640 1 -- 172.21.15.39:0/79763425 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe994003530 con 0x7fe9a80aabc0 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a6ffd640 1 -- 172.21.15.39:0/79763425 >> v2:172.21.15.84:3300/0 conn(0x7fe9a80c40b0 msgr2=0x7fe9a815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a6ffd640 1 --2- 172.21.15.39:0/79763425 >> v2:172.21.15.84:3300/0 conn(0x7fe9a80c40b0 0x7fe9a815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a6ffd640 1 -- 172.21.15.39:0/79763425 >> v1:172.21.15.39:6789/0 conn(0x7fe9a80aabc0 legacy=0x7fe9a80c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:02.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a6ffd640 1 -- 172.21.15.39:0/79763425 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a81520e0 con 0x7fe9a806c870 2024-09-13T16:48:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a6ffd640 1 --2- 172.21.15.39:0/79763425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a806c870 0x7fe9a80aafa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9a806f220 tx=0x7fe99001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c90ae5bab0c8e15 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a67fc640 1 -- 172.21.15.39:0/79763425 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900a5930 con 0x7fe9a806c870 2024-09-13T16:48:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a67fc640 1 -- 172.21.15.39:0/79763425 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9900a5a90 con 0x7fe9a806c870 2024-09-13T16:48:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.893+0000 7fe9a67fc640 1 -- 172.21.15.39:0/79763425 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900a5e20 con 0x7fe9a806c870 2024-09-13T16:48:02.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- 172.21.15.39:0/79763425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a806c870 msgr2=0x7fe9a80aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:02.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 --2- 172.21.15.39:0/79763425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a806c870 0x7fe9a80aafa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9a806f220 tx=0x7fe99001c870 comp rx=0 tx=0).stop 2024-09-13T16:48:02.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- 172.21.15.39:0/79763425 shutdown_connections 2024-09-13T16:48:02.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 --2- 172.21.15.39:0/79763425 >> v2:172.21.15.84:3300/0 conn(0x7fe9a80c40b0 0x7fe9a815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:02.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 --2- 172.21.15.39:0/79763425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a806c870 0x7fe9a80aafa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:02.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- 172.21.15.39:0/79763425 >> 172.21.15.39:0/79763425 conn(0x7fe9a8069e10 msgr2=0x7fe9a806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:02.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- 172.21.15.39:0/79763425 shutdown_connections 2024-09-13T16:48:02.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- 172.21.15.39:0/79763425 wait complete. 2024-09-13T16:48:02.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 Processor -- start 2024-09-13T16:48:02.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- start start 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9a806c870 0x7fe9a8148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9a80aabc0 0x7fe9a8141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80c40b0 0x7fe9a8141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a81ec5a0 con 0x7fe9a80c40b0 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a814b600 con 0x7fe9a80aabc0 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a81421d0 con 0x7fe9a806c870 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80c40b0 0x7fe9a8141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80c40b0 0x7fe9a8141c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41082/0 (socket says 172.21.15.39:41082) 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a7fff640 1 -- 172.21.15.39:0/3583028711 learned_addr learned my addr 172.21.15.39:0/3583028711 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a6ffd640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9a80aabc0 0x7fe9a8141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:02.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a77fe640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9a806c870 0x7fe9a8148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a7fff640 1 -- 172.21.15.39:0/3583028711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9a806c870 msgr2=0x7fe9a8148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a7fff640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9a806c870 0x7fe9a8148770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a7fff640 1 -- 172.21.15.39:0/3583028711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9a80aabc0 msgr2=0x7fe9a8141750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a7fff640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9a80aabc0 0x7fe9a8141750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a7fff640 1 -- 172.21.15.39:0/3583028711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe990009a70 con 0x7fe9a80c40b0 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9a7fff640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80c40b0 0x7fe9a8141c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000d590 tx=0x7fe9a000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe99ffff640 1 -- 172.21.15.39:0/3583028711 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00125f0 con 0x7fe9a80c40b0 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe99ffff640 1 -- 172.21.15.39:0/3583028711 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a0012770 con 0x7fe9a80c40b0 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a80bfb70 con 0x7fe9a80c40b0 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe99ffff640 1 -- 172.21.15.39:0/3583028711 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0012b20 con 0x7fe9a80c40b0 2024-09-13T16:48:02.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.897+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a80bf2d0 con 0x7fe9a80c40b0 2024-09-13T16:48:02.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.901+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a81efb10 con 0x7fe9a80c40b0 2024-09-13T16:48:02.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.901+0000 7fe99ffff640 1 -- 172.21.15.39:0/3583028711 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe9a0012c80 con 0x7fe9a80c40b0 2024-09-13T16:48:02.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.901+0000 7fe99ffff640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe988066690 0x7fe988068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.901+0000 7fe99ffff640 1 -- 172.21.15.39:0/3583028711 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7fe9a0084f50 con 0x7fe9a80c40b0 2024-09-13T16:48:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.901+0000 7fe9a77fe640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe988066690 0x7fe988068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.901+0000 7fe9a77fe640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe988066690 0x7fe988068b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe998001280 tx=0x7fe998009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:02.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:02.905+0000 7fe99ffff640 1 -- 172.21.15.39:0/3583028711 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe9a0052340 con 0x7fe9a80c40b0 2024-09-13T16:48:03.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9a81efb10 con 0x7fe9a80c40b0 2024-09-13T16:48:03.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe99ffff640 1 -- 172.21.15.39:0/3583028711 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4721 (secure 0 0 0) 0x7fe9a81efb10 con 0x7fe9a80c40b0 2024-09-13T16:48:03.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:03.100 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T16:48:03.101 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-13T16:48:03.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe988066690 msgr2=0x7fe988068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe988066690 0x7fe988068b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe998001280 tx=0x7fe998009040 comp rx=0 tx=0).stop 2024-09-13T16:48:03.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80c40b0 msgr2=0x7fe9a8141c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80c40b0 0x7fe9a8141c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000d590 tx=0x7fe9a000da50 comp rx=0 tx=0).stop 2024-09-13T16:48:03.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 shutdown_connections 2024-09-13T16:48:03.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe988066690 0x7fe988068b50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9a80aabc0 0x7fe9a8141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80c40b0 0x7fe9a8141c90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 --2- 172.21.15.39:0/3583028711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9a806c870 0x7fe9a8148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 >> 172.21.15.39:0/3583028711 conn(0x7fe9a8069e10 msgr2=0x7fe9a80c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:03.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 shutdown_connections 2024-09-13T16:48:03.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.097+0000 7fe9adf15640 1 -- 172.21.15.39:0/3583028711 wait complete. 2024-09-13T16:48:03.117 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.276+0000 7f3291151640 1 Processor -- start 2024-09-13T16:48:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.276+0000 7f3291151640 1 -- start start 2024-09-13T16:48:03.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.276+0000 7f3291151640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f328c0c4740 0x7f328c0c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.276+0000 7f3291151640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c0c2780 con 0x7f328c14e0c0 2024-09-13T16:48:03.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.276+0000 7f3291151640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c0c2a60 con 0x7f328c0a4e70 2024-09-13T16:48:03.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.276+0000 7f3291151640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0c2d40 con 0x7f328c0c4740 2024-09-13T16:48:03.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.276+0000 7f328ad9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f328c14e0c0 0x7f328c0c1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38654/0 (socket says 172.21.15.39:38654) 2024-09-13T16:48:03.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.276+0000 7f328ad9d640 1 -- 172.21.15.39:0/1937935618 learned_addr learned my addr 172.21.15.39:0/1937935618 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:03.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.276+0000 7f328b59e640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f328c0c4740 0x7f328c0c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217157853 0 0) 0x7f328c0c2780 con 0x7f328c14e0c0 2024-09-13T16:48:03.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3274003530 con 0x7f328c14e0c0 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593442578 0 0) 0x7f328c0c2a60 con 0x7f328c0a4e70 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f328c0c2780 con 0x7f328c0a4e70 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2948680816 0 0) 0x7f3274003530 con 0x7f328c14e0c0 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f328c0c4740 msgr2=0x7f328c0c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f328c0c4740 0x7f328c0c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 >> v1:172.21.15.84:6789/0 conn(0x7f328c0a4e70 legacy=0x7f328c06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0c3020 con 0x7f328c14e0c0 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f327c001a00 con 0x7f328c14e0c0 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 >> v1:172.21.15.39:6789/0 conn(0x7f328c14e0c0 legacy=0x7f328c0c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f328c0c4740 0x7f3274003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32740040f0 0x7f32740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3274004d10 0x7f3274005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0c2d40 con 0x7f32740040f0 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327400bb50 con 0x7f3274004d10 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327400bd10 con 0x7f328c0c4740 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328a59c640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32740040f0 0x7f32740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328b59e640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3274004d10 0x7f3274005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328ad9d640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f328c0c4740 0x7f3274003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328a59c640 1 -- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f328c0c4740 msgr2=0x7f3274003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328a59c640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f328c0c4740 0x7f3274003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328a59c640 1 -- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3274004d10 msgr2=0x7f3274005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328a59c640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3274004d10 0x7f3274005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328a59c640 1 -- 172.21.15.39:0/1937935618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3274003530 con 0x7f32740040f0 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328b59e640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3274004d10 0x7f3274005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328ad9d640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f328c0c4740 0x7f3274003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328a59c640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32740040f0 0x7f32740047d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32800018d0 tx=0x7f328001c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc26a018c71a327a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32800052d0 con 0x7f32740040f0 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32800a7050 con 0x7f32740040f0 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3289d9b640 1 -- 172.21.15.39:0/1937935618 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280003600 con 0x7f32740040f0 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3291151640 1 -- 172.21.15.39:0/1937935618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32740040f0 msgr2=0x7f32740047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3291151640 1 --2- 172.21.15.39:0/1937935618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32740040f0 0x7f32740047d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32800018d0 tx=0x7f328001c2c0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f328b59e640 1 -- 172.21.15.39:0/1937935618 reap_dead start 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3291151640 1 -- 172.21.15.39:0/1937935618 shutdown_connections 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3291151640 1 -- 172.21.15.39:0/1937935618 >> 172.21.15.39:0/1937935618 conn(0x7f328c069e10 msgr2=0x7f328c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3291151640 1 -- 172.21.15.39:0/1937935618 shutdown_connections 2024-09-13T16:48:03.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.280+0000 7f3291151640 1 -- 172.21.15.39:0/1937935618 wait complete. 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 Processor -- start 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 -- start start 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f32740040f0 0x7f328c0bf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f328c0bf860 0x7f328c0c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f328c0c1a20 0x7f328c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c141d40 con 0x7f328c0bf860 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c14cbd0 con 0x7f328c0c1a20 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c141640 con 0x7f32740040f0 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f328c0bf860 0x7f328c0c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f328c0bf860 0x7f328c0c14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41096/0 (socket says 172.21.15.39:41096) 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328a59c640 1 -- 172.21.15.39:0/1070496764 learned_addr learned my addr 172.21.15.39:0/1070496764 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328b59e640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f328c0c1a20 0x7f328c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328ad9d640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f32740040f0 0x7f328c0bf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328a59c640 1 -- 172.21.15.39:0/1070496764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f32740040f0 msgr2=0x7f328c0bf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328a59c640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f32740040f0 0x7f328c0bf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328a59c640 1 -- 172.21.15.39:0/1070496764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f328c0c1a20 msgr2=0x7f328c0a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328a59c640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f328c0c1a20 0x7f328c0a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328a59c640 1 -- 172.21.15.39:0/1070496764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c15ffe0 con 0x7f328c0bf860 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328b59e640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f328c0c1a20 0x7f328c0a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f328a59c640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f328c0bf860 0x7f328c0c14e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32800ad040 tx=0x7f3274005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f32737fe640 1 -- 172.21.15.39:0/1070496764 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32800069e0 con 0x7f328c0bf860 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f32737fe640 1 -- 172.21.15.39:0/1070496764 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3280006b40 con 0x7f328c0bf860 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c141890 con 0x7f328c0bf860 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f32737fe640 1 -- 172.21.15.39:0/1070496764 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280007030 con 0x7f328c0bf860 2024-09-13T16:48:03.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c1441b0 con 0x7f328c0bf860 2024-09-13T16:48:03.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c160220 con 0x7f328c0bf860 2024-09-13T16:48:03.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.284+0000 7f32737fe640 1 -- 172.21.15.39:0/1070496764 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3280007190 con 0x7f328c0bf860 2024-09-13T16:48:03.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.288+0000 7f32737fe640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f32780666c0 0x7f3278068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.288+0000 7f32737fe640 1 -- 172.21.15.39:0/1070496764 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f328011e440 con 0x7f328c0bf860 2024-09-13T16:48:03.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.288+0000 7f328ad9d640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f32780666c0 0x7f3278068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.288+0000 7f328ad9d640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f32780666c0 0x7f3278068b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f327c000f90 tx=0x7f327c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:03.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.292+0000 7f32737fe640 1 -- 172.21.15.39:0/1070496764 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f328c160220 con 0x7f328c0bf860 2024-09-13T16:48:03.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.444+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f328c1f0c50 con 0x7f328c0bf860 2024-09-13T16:48:03.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f32737fe640 1 -- 172.21.15.39:0/1070496764 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f328c1f0c50 con 0x7f328c0bf860 2024-09-13T16:48:03.452 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:03.453 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":20,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:00.025477+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:03.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f32780666c0 msgr2=0x7f3278068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f32780666c0 0x7f3278068b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f327c000f90 tx=0x7f327c009040 comp rx=0 tx=0).stop 2024-09-13T16:48:03.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f328c0bf860 msgr2=0x7f328c0c14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f328c0bf860 0x7f328c0c14e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32800ad040 tx=0x7f3274005800 comp rx=0 tx=0).stop 2024-09-13T16:48:03.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 shutdown_connections 2024-09-13T16:48:03.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f32780666c0 0x7f3278068b80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f328c0c1a20 0x7f328c0a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f328c0bf860 0x7f328c0c14e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 --2- 172.21.15.39:0/1070496764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f32740040f0 0x7f328c0bf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 >> 172.21.15.39:0/1070496764 conn(0x7f328c069e10 msgr2=0x7f328c14c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:03.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 shutdown_connections 2024-09-13T16:48:03.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.448+0000 7f3291151640 1 -- 172.21.15.39:0/1070496764 wait complete. 2024-09-13T16:48:03.470 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T16:48:03.470 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:03.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42c043c640 1 Processor -- start 2024-09-13T16:48:03.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42c043c640 1 -- start start 2024-09-13T16:48:03.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42c043c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f42b80a4e70 0x7f42b8151460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42c043c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42b80c4530 0x7f42b80c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42c043c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b8154100 con 0x7f42b806d020 2024-09-13T16:48:03.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42c043c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b80a8c50 con 0x7f42b80c4530 2024-09-13T16:48:03.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42c043c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b81519a0 con 0x7f42b80a4e70 2024-09-13T16:48:03.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42be9d9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42b80c4530 0x7f42b80c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42be9d9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42b80c4530 0x7f42b80c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41104/0 (socket says 172.21.15.39:41104) 2024-09-13T16:48:03.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42be9d9640 1 -- 172.21.15.39:0/1135734701 learned_addr learned my addr 172.21.15.39:0/1135734701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:03.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42bd9d7640 1 --2- 172.21.15.39:0/1135734701 >> v2:172.21.15.84:3300/0 conn(0x7f42b80a4e70 0x7f42b8151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42be9d9640 1 -- 172.21.15.39:0/1135734701 >> v2:172.21.15.84:3300/0 conn(0x7f42b80a4e70 msgr2=0x7f42b8151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:03.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42be9d9640 1 --2- 172.21.15.39:0/1135734701 >> v2:172.21.15.84:3300/0 conn(0x7f42b80a4e70 0x7f42b8151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42be9d9640 1 -- 172.21.15.39:0/1135734701 >> v1:172.21.15.84:6789/0 conn(0x7f42b806d020 legacy=0x7f42b814db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:03.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.588+0000 7f42be9d9640 1 -- 172.21.15.39:0/1135734701 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b81543e0 con 0x7f42b80c4530 2024-09-13T16:48:03.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42be9d9640 1 --2- 172.21.15.39:0/1135734701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b80c4530 0x7f42b80c4930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f42b80c3920 tx=0x7f42b001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52001f39c42e174 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:03.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42bd1d6640 1 -- 172.21.15.39:0/1135734701 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b00ab600 con 0x7f42b80c4530 2024-09-13T16:48:03.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42bd1d6640 1 -- 172.21.15.39:0/1135734701 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b00a7050 con 0x7f42b80c4530 2024-09-13T16:48:03.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42bd1d6640 1 -- 172.21.15.39:0/1135734701 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b00ab980 con 0x7f42b80c4530 2024-09-13T16:48:03.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 -- 172.21.15.39:0/1135734701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b80c4530 msgr2=0x7f42b80c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 --2- 172.21.15.39:0/1135734701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b80c4530 0x7f42b80c4930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f42b80c3920 tx=0x7f42b001c420 comp rx=0 tx=0).stop 2024-09-13T16:48:03.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 -- 172.21.15.39:0/1135734701 shutdown_connections 2024-09-13T16:48:03.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 --2- 172.21.15.39:0/1135734701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b80c4530 0x7f42b80c4930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 --2- 172.21.15.39:0/1135734701 >> v2:172.21.15.84:3300/0 conn(0x7f42b80a4e70 0x7f42b8151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 -- 172.21.15.39:0/1135734701 >> 172.21.15.39:0/1135734701 conn(0x7f42b8069e10 msgr2=0x7f42b806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:03.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 -- 172.21.15.39:0/1135734701 shutdown_connections 2024-09-13T16:48:03.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 -- 172.21.15.39:0/1135734701 wait complete. 2024-09-13T16:48:03.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 Processor -- start 2024-09-13T16:48:03.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.592+0000 7f42c043c640 1 -- start start 2024-09-13T16:48:03.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42c043c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b806d020 0x7f42b80a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42c043c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42b80a4e70 0x7f42b81e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42c043c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b80c4530 0x7f42b80aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42c043c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b81e8ff0 con 0x7f42b806d020 2024-09-13T16:48:03.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42c043c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b8145e90 con 0x7f42b80c4530 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42c043c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b8145b40 con 0x7f42b80a4e70 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be1d8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b806d020 0x7f42b80a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be1d8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b806d020 0x7f42b80a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41108/0 (socket says 172.21.15.39:41108) 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be1d8640 1 -- 172.21.15.39:0/356426627 learned_addr learned my addr 172.21.15.39:0/356426627 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be9d9640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b80c4530 0x7f42b80aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42bd9d7640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42b80a4e70 0x7f42b81e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be1d8640 1 -- 172.21.15.39:0/356426627 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42b80a4e70 msgr2=0x7f42b81e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be1d8640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42b80a4e70 0x7f42b81e8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be1d8640 1 -- 172.21.15.39:0/356426627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b80c4530 msgr2=0x7f42b80aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be1d8640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b80c4530 0x7f42b80aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be1d8640 1 -- 172.21.15.39:0/356426627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b00094d0 con 0x7f42b806d020 2024-09-13T16:48:03.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42bd9d7640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42b80a4e70 0x7f42b81e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42be1d8640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b806d020 0x7f42b80a6890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c7f0 tx=0x7f42a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42b6ffd640 1 -- 172.21.15.39:0/356426627 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a800aa50 con 0x7f42b806d020 2024-09-13T16:48:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42b6ffd640 1 -- 172.21.15.39:0/356426627 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a800abb0 con 0x7f42b806d020 2024-09-13T16:48:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42c043c640 1 -- 172.21.15.39:0/356426627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b8146e80 con 0x7f42b806d020 2024-09-13T16:48:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42b6ffd640 1 -- 172.21.15.39:0/356426627 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8012420 con 0x7f42b806d020 2024-09-13T16:48:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42c043c640 1 -- 172.21.15.39:0/356426627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b8147440 con 0x7f42b806d020 2024-09-13T16:48:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42b4ff9640 1 -- 172.21.15.39:0/356426627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c005260 con 0x7f42b806d020 2024-09-13T16:48:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42b6ffd640 1 -- 172.21.15.39:0/356426627 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f42a8012580 con 0x7f42b806d020 2024-09-13T16:48:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42b6ffd640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42a4066690 0x7f42a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42bd9d7640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42a4066690 0x7f42a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.596+0000 7f42b6ffd640 1 -- 172.21.15.39:0/356426627 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f42a8084420 con 0x7f42b806d020 2024-09-13T16:48:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.600+0000 7f42bd9d7640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42a4066690 0x7f42a4068b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42b815a370 tx=0x7f42ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:03.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.604+0000 7f42b6ffd640 1 -- 172.21.15.39:0/356426627 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f42a8051790 con 0x7f42b806d020 2024-09-13T16:48:03.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.756+0000 7f42b4ff9640 1 -- 172.21.15.39:0/356426627 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f429c000bd0 con 0x7f42a4066690 2024-09-13T16:48:03.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.764+0000 7f42b6ffd640 1 -- 172.21.15.39:0/356426627 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264630 (secure 0 0 0) 0x7f429c000bd0 con 0x7f42a4066690 2024-09-13T16:48:03.771 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:03.775 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:03.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 -- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42a4066690 msgr2=0x7f42a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42a4066690 0x7f42a4068b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42b815a370 tx=0x7f42ac008040 comp rx=0 tx=0).stop 2024-09-13T16:48:03.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 -- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b806d020 msgr2=0x7f42b80a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b806d020 0x7f42b80a6890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f42a800c7f0 tx=0x7f42a800ccb0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 -- 172.21.15.39:0/356426627 shutdown_connections 2024-09-13T16:48:03.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42a4066690 0x7f42a4068b50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b80c4530 0x7f42b80aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42b80a4e70 0x7f42b81e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 --2- 172.21.15.39:0/356426627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b806d020 0x7f42b80a6890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 -- 172.21.15.39:0/356426627 >> 172.21.15.39:0/356426627 conn(0x7f42b8069e10 msgr2=0x7f42b814e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:03.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 -- 172.21.15.39:0/356426627 shutdown_connections 2024-09-13T16:48:03.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.772+0000 7f42b4ff9640 1 -- 172.21.15.39:0/356426627 wait complete. 2024-09-13T16:48:03.791 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T16:48:03.558915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235368,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739352,"statfs":{"total":773094113280,"available":772853096448,"internally_reserved":0,"allocated":2891776,"data_stored":1752004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.926209"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264239+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:00.264239+0000","last_peered":"2024-09-13T16:48:00.264239+0000","last_clean":"2024-09-13T16:48:00.264239+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:00.264239+0000","last_undegraded":"2024-09-13T16:48:00.264239+0000","last_fullsized":"2024-09-13T16:48:00.264239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035349+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:00.035349+0000","last_peered":"2024-09-13T16:48:00.035349+0000","last_clean":"2024-09-13T16:48:00.035349+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:00.035349+0000","last_undegraded":"2024-09-13T16:48:00.035349+0000","last_fullsized":"2024-09-13T16:48:00.035349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266436+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:00.266436+0000","last_peered":"2024-09-13T16:48:00.266436+0000","last_clean":"2024-09-13T16:48:00.266436+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:00.266436+0000","last_undegraded":"2024-09-13T16:48:00.266436+0000","last_fullsized":"2024-09-13T16:48:00.266436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036261+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:00.036261+0000","last_peered":"2024-09-13T16:48:00.036261+0000","last_clean":"2024-09-13T16:48:00.036261+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:00.036261+0000","last_undegraded":"2024-09-13T16:48:00.036261+0000","last_fullsized":"2024-09-13T16:48:00.036261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263858+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:00.263858+0000","last_peered":"2024-09-13T16:48:00.263858+0000","last_clean":"2024-09-13T16:48:00.263858+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:00.263858+0000","last_undegraded":"2024-09-13T16:48:00.263858+0000","last_fullsized":"2024-09-13T16:48:00.263858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035241+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:00.035241+0000","last_peered":"2024-09-13T16:48:00.035241+0000","last_clean":"2024-09-13T16:48:00.035241+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:00.035241+0000","last_undegraded":"2024-09-13T16:48:00.035241+0000","last_fullsized":"2024-09-13T16:48:00.035241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263156+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:00.263156+0000","last_peered":"2024-09-13T16:48:00.263156+0000","last_clean":"2024-09-13T16:48:00.263156+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:00.263156+0000","last_undegraded":"2024-09-13T16:48:00.263156+0000","last_fullsized":"2024-09-13T16:48:00.263156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264056+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:00.264056+0000","last_peered":"2024-09-13T16:48:00.264056+0000","last_clean":"2024-09-13T16:48:00.264056+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:00.264056+0000","last_undegraded":"2024-09-13T16:48:00.264056+0000","last_fullsized":"2024-09-13T16:48:00.264056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264307+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:00.264307+0000","last_peered":"2024-09-13T16:48:00.264307+0000","last_clean":"2024-09-13T16:48:00.264307+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:00.264307+0000","last_undegraded":"2024-09-13T16:48:00.264307+0000","last_fullsized":"2024-09-13T16:48:00.264307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266206+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:00.266206+0000","last_peered":"2024-09-13T16:48:00.266206+0000","last_clean":"2024-09-13T16:48:00.266206+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:00.266206+0000","last_undegraded":"2024-09-13T16:48:00.266206+0000","last_fullsized":"2024-09-13T16:48:00.266206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265417+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:00.265417+0000","last_peered":"2024-09-13T16:48:00.265417+0000","last_clean":"2024-09-13T16:48:00.265417+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:00.265417+0000","last_undegraded":"2024-09-13T16:48:00.265417+0000","last_fullsized":"2024-09-13T16:48:00.265417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035298+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:00.035298+0000","last_peered":"2024-09-13T16:48:00.035298+0000","last_clean":"2024-09-13T16:48:00.035298+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:00.035298+0000","last_undegraded":"2024-09-13T16:48:00.035298+0000","last_fullsized":"2024-09-13T16:48:00.035298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265724+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:00.265724+0000","last_peered":"2024-09-13T16:48:00.265724+0000","last_clean":"2024-09-13T16:48:00.265724+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:00.265724+0000","last_undegraded":"2024-09-13T16:48:00.265724+0000","last_fullsized":"2024-09-13T16:48:00.265724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264662+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:00.264662+0000","last_peered":"2024-09-13T16:48:00.264662+0000","last_clean":"2024-09-13T16:48:00.264662+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:00.264662+0000","last_undegraded":"2024-09-13T16:48:00.264662+0000","last_fullsized":"2024-09-13T16:48:00.264662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035096+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:00.035096+0000","last_peered":"2024-09-13T16:48:00.035096+0000","last_clean":"2024-09-13T16:48:00.035096+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:00.035096+0000","last_undegraded":"2024-09-13T16:48:00.035096+0000","last_fullsized":"2024-09-13T16:48:00.035096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035358+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:00.035358+0000","last_peered":"2024-09-13T16:48:00.035358+0000","last_clean":"2024-09-13T16:48:00.035358+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:00.035358+0000","last_undegraded":"2024-09-13T16:48:00.035358+0000","last_fullsized":"2024-09-13T16:48:00.035358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036971+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:00.036971+0000","last_peered":"2024-09-13T16:48:00.036971+0000","last_clean":"2024-09-13T16:48:00.036971+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:00.036971+0000","last_undegraded":"2024-09-13T16:48:00.036971+0000","last_fullsized":"2024-09-13T16:48:00.036971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265064+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:00.265064+0000","last_peered":"2024-09-13T16:48:00.265064+0000","last_clean":"2024-09-13T16:48:00.265064+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:00.265064+0000","last_undegraded":"2024-09-13T16:48:00.265064+0000","last_fullsized":"2024-09-13T16:48:00.265064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263944+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:00.263944+0000","last_peered":"2024-09-13T16:48:00.263944+0000","last_clean":"2024-09-13T16:48:00.263944+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:00.263944+0000","last_undegraded":"2024-09-13T16:48:00.263944+0000","last_fullsized":"2024-09-13T16:48:00.263944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265671+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:00.265671+0000","last_peered":"2024-09-13T16:48:00.265671+0000","last_clean":"2024-09-13T16:48:00.265671+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:00.265671+0000","last_undegraded":"2024-09-13T16:48:00.265671+0000","last_fullsized":"2024-09-13T16:48:00.265671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266724+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:00.266724+0000","last_peered":"2024-09-13T16:48:00.266724+0000","last_clean":"2024-09-13T16:48:00.266724+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:00.266724+0000","last_undegraded":"2024-09-13T16:48:00.266724+0000","last_fullsized":"2024-09-13T16:48:00.266724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035473+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:00.035473+0000","last_peered":"2024-09-13T16:48:00.035473+0000","last_clean":"2024-09-13T16:48:00.035473+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:00.035473+0000","last_undegraded":"2024-09-13T16:48:00.035473+0000","last_fullsized":"2024-09-13T16:48:00.035473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265723+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:00.265723+0000","last_peered":"2024-09-13T16:48:00.265723+0000","last_clean":"2024-09-13T16:48:00.265723+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:00.265723+0000","last_undegraded":"2024-09-13T16:48:00.265723+0000","last_fullsized":"2024-09-13T16:48:00.265723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.034990+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:00.034990+0000","last_peered":"2024-09-13T16:48:00.034990+0000","last_clean":"2024-09-13T16:48:00.034990+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:00.034990+0000","last_undegraded":"2024-09-13T16:48:00.034990+0000","last_fullsized":"2024-09-13T16:48:00.034990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265455+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:00.265455+0000","last_peered":"2024-09-13T16:48:00.265455+0000","last_clean":"2024-09-13T16:48:00.265455+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:00.265455+0000","last_undegraded":"2024-09-13T16:48:00.265455+0000","last_fullsized":"2024-09-13T16:48:00.265455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266331+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:00.266331+0000","last_peered":"2024-09-13T16:48:00.266331+0000","last_clean":"2024-09-13T16:48:00.266331+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:00.266331+0000","last_undegraded":"2024-09-13T16:48:00.266331+0000","last_fullsized":"2024-09-13T16:48:00.266331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035334+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:00.035334+0000","last_peered":"2024-09-13T16:48:00.035334+0000","last_clean":"2024-09-13T16:48:00.035334+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:00.035334+0000","last_undegraded":"2024-09-13T16:48:00.035334+0000","last_fullsized":"2024-09-13T16:48:00.035334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266080+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:00.266080+0000","last_peered":"2024-09-13T16:48:00.266080+0000","last_clean":"2024-09-13T16:48:00.266080+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:00.266080+0000","last_undegraded":"2024-09-13T16:48:00.266080+0000","last_fullsized":"2024-09-13T16:48:00.266080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266881+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:00.266881+0000","last_peered":"2024-09-13T16:48:00.266881+0000","last_clean":"2024-09-13T16:48:00.266881+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:00.266881+0000","last_undegraded":"2024-09-13T16:48:00.266881+0000","last_fullsized":"2024-09-13T16:48:00.266881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035793+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:00.035793+0000","last_peered":"2024-09-13T16:48:00.035793+0000","last_clean":"2024-09-13T16:48:00.035793+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:00.035793+0000","last_undegraded":"2024-09-13T16:48:00.035793+0000","last_fullsized":"2024-09-13T16:48:00.035793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265446+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:00.265446+0000","last_peered":"2024-09-13T16:48:00.265446+0000","last_clean":"2024-09-13T16:48:00.265446+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:00.265446+0000","last_undegraded":"2024-09-13T16:48:00.265446+0000","last_fullsized":"2024-09-13T16:48:00.265446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265168+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:00.265168+0000","last_peered":"2024-09-13T16:48:00.265168+0000","last_clean":"2024-09-13T16:48:00.265168+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:00.265168+0000","last_undegraded":"2024-09-13T16:48:00.265168+0000","last_fullsized":"2024-09-13T16:48:00.265168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.034332+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:00.034332+0000","last_peered":"2024-09-13T16:48:00.034332+0000","last_clean":"2024-09-13T16:48:00.034332+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:00.034332+0000","last_undegraded":"2024-09-13T16:48:00.034332+0000","last_fullsized":"2024-09-13T16:48:00.034332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266324+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:00.266324+0000","last_peered":"2024-09-13T16:48:00.266324+0000","last_clean":"2024-09-13T16:48:00.266324+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:00.266324+0000","last_undegraded":"2024-09-13T16:48:00.266324+0000","last_fullsized":"2024-09-13T16:48:00.266324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.267013+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:00.267013+0000","last_peered":"2024-09-13T16:48:00.267013+0000","last_clean":"2024-09-13T16:48:00.267013+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:00.267013+0000","last_undegraded":"2024-09-13T16:48:00.267013+0000","last_fullsized":"2024-09-13T16:48:00.267013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265664+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:00.265664+0000","last_peered":"2024-09-13T16:48:00.265664+0000","last_clean":"2024-09-13T16:48:00.265664+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:00.265664+0000","last_undegraded":"2024-09-13T16:48:00.265664+0000","last_fullsized":"2024-09-13T16:48:00.265664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266251+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:00.266251+0000","last_peered":"2024-09-13T16:48:00.266251+0000","last_clean":"2024-09-13T16:48:00.266251+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:00.266251+0000","last_undegraded":"2024-09-13T16:48:00.266251+0000","last_fullsized":"2024-09-13T16:48:00.266251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266591+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:00.266591+0000","last_peered":"2024-09-13T16:48:00.266591+0000","last_clean":"2024-09-13T16:48:00.266591+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:00.266591+0000","last_undegraded":"2024-09-13T16:48:00.266591+0000","last_fullsized":"2024-09-13T16:48:00.266591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035119+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:00.035119+0000","last_peered":"2024-09-13T16:48:00.035119+0000","last_clean":"2024-09-13T16:48:00.035119+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:00.035119+0000","last_undegraded":"2024-09-13T16:48:00.035119+0000","last_fullsized":"2024-09-13T16:48:00.035119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.034977+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:00.034977+0000","last_peered":"2024-09-13T16:48:00.034977+0000","last_clean":"2024-09-13T16:48:00.034977+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:00.034977+0000","last_undegraded":"2024-09-13T16:48:00.034977+0000","last_fullsized":"2024-09-13T16:48:00.034977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265774+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:00.265774+0000","last_peered":"2024-09-13T16:48:00.265774+0000","last_clean":"2024-09-13T16:48:00.265774+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:00.265774+0000","last_undegraded":"2024-09-13T16:48:00.265774+0000","last_fullsized":"2024-09-13T16:48:00.265774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036116+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:00.036116+0000","last_peered":"2024-09-13T16:48:00.036116+0000","last_clean":"2024-09-13T16:48:00.036116+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:00.036116+0000","last_undegraded":"2024-09-13T16:48:00.036116+0000","last_fullsized":"2024-09-13T16:48:00.036116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036301+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:00.036301+0000","last_peered":"2024-09-13T16:48:00.036301+0000","last_clean":"2024-09-13T16:48:00.036301+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:00.036301+0000","last_undegraded":"2024-09-13T16:48:00.036301+0000","last_fullsized":"2024-09-13T16:48:00.036301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266561+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:00.266561+0000","last_peered":"2024-09-13T16:48:00.266561+0000","last_clean":"2024-09-13T16:48:00.266561+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:00.266561+0000","last_undegraded":"2024-09-13T16:48:00.266561+0000","last_fullsized":"2024-09-13T16:48:00.266561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035523+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:00.035523+0000","last_peered":"2024-09-13T16:48:00.035523+0000","last_clean":"2024-09-13T16:48:00.035523+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:00.035523+0000","last_undegraded":"2024-09-13T16:48:00.035523+0000","last_fullsized":"2024-09-13T16:48:00.035523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035368+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:00.035368+0000","last_peered":"2024-09-13T16:48:00.035368+0000","last_clean":"2024-09-13T16:48:00.035368+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:00.035368+0000","last_undegraded":"2024-09-13T16:48:00.035368+0000","last_fullsized":"2024-09-13T16:48:00.035368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265791+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:00.265791+0000","last_peered":"2024-09-13T16:48:00.265791+0000","last_clean":"2024-09-13T16:48:00.265791+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:00.265791+0000","last_undegraded":"2024-09-13T16:48:00.265791+0000","last_fullsized":"2024-09-13T16:48:00.265791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263372+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:00.263372+0000","last_peered":"2024-09-13T16:48:00.263372+0000","last_clean":"2024-09-13T16:48:00.263372+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:00.263372+0000","last_undegraded":"2024-09-13T16:48:00.263372+0000","last_fullsized":"2024-09-13T16:48:00.263372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.267125+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:00.267125+0000","last_peered":"2024-09-13T16:48:00.267125+0000","last_clean":"2024-09-13T16:48:00.267125+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:00.267125+0000","last_undegraded":"2024-09-13T16:48:00.267125+0000","last_fullsized":"2024-09-13T16:48:00.267125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264931+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:00.264931+0000","last_peered":"2024-09-13T16:48:00.264931+0000","last_clean":"2024-09-13T16:48:00.264931+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:00.264931+0000","last_undegraded":"2024-09-13T16:48:00.264931+0000","last_fullsized":"2024-09-13T16:48:00.264931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035706+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:00.035706+0000","last_peered":"2024-09-13T16:48:00.035706+0000","last_clean":"2024-09-13T16:48:00.035706+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:00.035706+0000","last_undegraded":"2024-09-13T16:48:00.035706+0000","last_fullsized":"2024-09-13T16:48:00.035706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263925+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:00.263925+0000","last_peered":"2024-09-13T16:48:00.263925+0000","last_clean":"2024-09-13T16:48:00.263925+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:00.263925+0000","last_undegraded":"2024-09-13T16:48:00.263925+0000","last_fullsized":"2024-09-13T16:48:00.263925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265326+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:00.265326+0000","last_peered":"2024-09-13T16:48:00.265326+0000","last_clean":"2024-09-13T16:48:00.265326+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:00.265326+0000","last_undegraded":"2024-09-13T16:48:00.265326+0000","last_fullsized":"2024-09-13T16:48:00.265326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265297+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:00.265297+0000","last_peered":"2024-09-13T16:48:00.265297+0000","last_clean":"2024-09-13T16:48:00.265297+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:00.265297+0000","last_undegraded":"2024-09-13T16:48:00.265297+0000","last_fullsized":"2024-09-13T16:48:00.265297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264752+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:00.264752+0000","last_peered":"2024-09-13T16:48:00.264752+0000","last_clean":"2024-09-13T16:48:00.264752+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:00.264752+0000","last_undegraded":"2024-09-13T16:48:00.264752+0000","last_fullsized":"2024-09-13T16:48:00.264752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263286+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:00.263286+0000","last_peered":"2024-09-13T16:48:00.263286+0000","last_clean":"2024-09-13T16:48:00.263286+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:00.263286+0000","last_undegraded":"2024-09-13T16:48:00.263286+0000","last_fullsized":"2024-09-13T16:48:00.263286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035289+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:00.035289+0000","last_peered":"2024-09-13T16:48:00.035289+0000","last_clean":"2024-09-13T16:48:00.035289+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:00.035289+0000","last_undegraded":"2024-09-13T16:48:00.035289+0000","last_fullsized":"2024-09-13T16:48:00.035289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264352+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:00.264352+0000","last_peered":"2024-09-13T16:48:00.264352+0000","last_clean":"2024-09-13T16:48:00.264352+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:00.264352+0000","last_undegraded":"2024-09-13T16:48:00.264352+0000","last_fullsized":"2024-09-13T16:48:00.264352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265322+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:00.265322+0000","last_peered":"2024-09-13T16:48:00.265322+0000","last_clean":"2024-09-13T16:48:00.265322+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:00.265322+0000","last_undegraded":"2024-09-13T16:48:00.265322+0000","last_fullsized":"2024-09-13T16:48:00.265322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264475+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:00.264475+0000","last_peered":"2024-09-13T16:48:00.264475+0000","last_clean":"2024-09-13T16:48:00.264475+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:00.264475+0000","last_undegraded":"2024-09-13T16:48:00.264475+0000","last_fullsized":"2024-09-13T16:48:00.264475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035614+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:00.035614+0000","last_peered":"2024-09-13T16:48:00.035614+0000","last_clean":"2024-09-13T16:48:00.035614+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:00.035614+0000","last_undegraded":"2024-09-13T16:48:00.035614+0000","last_fullsized":"2024-09-13T16:48:00.035614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036751+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:00.036751+0000","last_peered":"2024-09-13T16:48:00.036751+0000","last_clean":"2024-09-13T16:48:00.036751+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:00.036751+0000","last_undegraded":"2024-09-13T16:48:00.036751+0000","last_fullsized":"2024-09-13T16:48:00.036751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265183+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:00.265183+0000","last_peered":"2024-09-13T16:48:00.265183+0000","last_clean":"2024-09-13T16:48:00.265183+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:00.265183+0000","last_undegraded":"2024-09-13T16:48:00.265183+0000","last_fullsized":"2024-09-13T16:48:00.265183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263837+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:00.263837+0000","last_peered":"2024-09-13T16:48:00.263837+0000","last_clean":"2024-09-13T16:48:00.263837+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:00.263837+0000","last_undegraded":"2024-09-13T16:48:00.263837+0000","last_fullsized":"2024-09-13T16:48:00.263837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T16:47:59.040456+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:47:59.040456+0000","last_peered":"2024-09-13T16:47:59.040456+0000","last_clean":"2024-09-13T16:47:59.040456+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:47:59.040456+0000","last_undegraded":"2024-09-13T16:47:59.040456+0000","last_fullsized":"2024-09-13T16:47:59.040456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035766+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:00.035766+0000","last_peered":"2024-09-13T16:48:00.035766+0000","last_clean":"2024-09-13T16:48:00.035766+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:00.035766+0000","last_undegraded":"2024-09-13T16:48:00.035766+0000","last_fullsized":"2024-09-13T16:48:00.035766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265192+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:00.265192+0000","last_peered":"2024-09-13T16:48:00.265192+0000","last_clean":"2024-09-13T16:48:00.265192+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:00.265192+0000","last_undegraded":"2024-09-13T16:48:00.265192+0000","last_fullsized":"2024-09-13T16:48:00.265192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263224+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:00.263224+0000","last_peered":"2024-09-13T16:48:00.263224+0000","last_clean":"2024-09-13T16:48:00.263224+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:00.263224+0000","last_undegraded":"2024-09-13T16:48:00.263224+0000","last_fullsized":"2024-09-13T16:48:00.263224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264480+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:00.264480+0000","last_peered":"2024-09-13T16:48:00.264480+0000","last_clean":"2024-09-13T16:48:00.264480+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:00.264480+0000","last_undegraded":"2024-09-13T16:48:00.264480+0000","last_fullsized":"2024-09-13T16:48:00.264480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265666+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:00.265666+0000","last_peered":"2024-09-13T16:48:00.265666+0000","last_clean":"2024-09-13T16:48:00.265666+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:00.265666+0000","last_undegraded":"2024-09-13T16:48:00.265666+0000","last_fullsized":"2024-09-13T16:48:00.265666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035453+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:00.035453+0000","last_peered":"2024-09-13T16:48:00.035453+0000","last_clean":"2024-09-13T16:48:00.035453+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:00.035453+0000","last_undegraded":"2024-09-13T16:48:00.035453+0000","last_fullsized":"2024-09-13T16:48:00.035453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035808+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:00.035808+0000","last_peered":"2024-09-13T16:48:00.035808+0000","last_clean":"2024-09-13T16:48:00.035808+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:00.035808+0000","last_undegraded":"2024-09-13T16:48:00.035808+0000","last_fullsized":"2024-09-13T16:48:00.035808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035023+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:00.035023+0000","last_peered":"2024-09-13T16:48:00.035023+0000","last_clean":"2024-09-13T16:48:00.035023+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:00.035023+0000","last_undegraded":"2024-09-13T16:48:00.035023+0000","last_fullsized":"2024-09-13T16:48:00.035023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035263+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:00.035263+0000","last_peered":"2024-09-13T16:48:00.035263+0000","last_clean":"2024-09-13T16:48:00.035263+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:00.035263+0000","last_undegraded":"2024-09-13T16:48:00.035263+0000","last_fullsized":"2024-09-13T16:48:00.035263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036318+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:00.036318+0000","last_peered":"2024-09-13T16:48:00.036318+0000","last_clean":"2024-09-13T16:48:00.036318+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:00.036318+0000","last_undegraded":"2024-09-13T16:48:00.036318+0000","last_fullsized":"2024-09-13T16:48:00.036318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264034+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:00.264034+0000","last_peered":"2024-09-13T16:48:00.264034+0000","last_clean":"2024-09-13T16:48:00.264034+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:00.264034+0000","last_undegraded":"2024-09-13T16:48:00.264034+0000","last_fullsized":"2024-09-13T16:48:00.264034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035147+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:00.035147+0000","last_peered":"2024-09-13T16:48:00.035147+0000","last_clean":"2024-09-13T16:48:00.035147+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:00.035147+0000","last_undegraded":"2024-09-13T16:48:00.035147+0000","last_fullsized":"2024-09-13T16:48:00.035147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035553+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:00.035553+0000","last_peered":"2024-09-13T16:48:00.035553+0000","last_clean":"2024-09-13T16:48:00.035553+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:00.035553+0000","last_undegraded":"2024-09-13T16:48:00.035553+0000","last_fullsized":"2024-09-13T16:48:00.035553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265727+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:00.265727+0000","last_peered":"2024-09-13T16:48:00.265727+0000","last_clean":"2024-09-13T16:48:00.265727+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:00.265727+0000","last_undegraded":"2024-09-13T16:48:00.265727+0000","last_fullsized":"2024-09-13T16:48:00.265727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265936+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:00.265936+0000","last_peered":"2024-09-13T16:48:00.265936+0000","last_clean":"2024-09-13T16:48:00.265936+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:00.265936+0000","last_undegraded":"2024-09-13T16:48:00.265936+0000","last_fullsized":"2024-09-13T16:48:00.265936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266671+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:00.266671+0000","last_peered":"2024-09-13T16:48:00.266671+0000","last_clean":"2024-09-13T16:48:00.266671+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:00.266671+0000","last_undegraded":"2024-09-13T16:48:00.266671+0000","last_fullsized":"2024-09-13T16:48:00.266671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263191+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:00.263191+0000","last_peered":"2024-09-13T16:48:00.263191+0000","last_clean":"2024-09-13T16:48:00.263191+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:00.263191+0000","last_undegraded":"2024-09-13T16:48:00.263191+0000","last_fullsized":"2024-09-13T16:48:00.263191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.034061+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:00.034061+0000","last_peered":"2024-09-13T16:48:00.034061+0000","last_clean":"2024-09-13T16:48:00.034061+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:00.034061+0000","last_undegraded":"2024-09-13T16:48:00.034061+0000","last_fullsized":"2024-09-13T16:48:00.034061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035089+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:00.035089+0000","last_peered":"2024-09-13T16:48:00.035089+0000","last_clean":"2024-09-13T16:48:00.035089+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:00.035089+0000","last_undegraded":"2024-09-13T16:48:00.035089+0000","last_fullsized":"2024-09-13T16:48:00.035089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266826+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:00.266826+0000","last_peered":"2024-09-13T16:48:00.266826+0000","last_clean":"2024-09-13T16:48:00.266826+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:00.266826+0000","last_undegraded":"2024-09-13T16:48:00.266826+0000","last_fullsized":"2024-09-13T16:48:00.266826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035268+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:00.035268+0000","last_peered":"2024-09-13T16:48:00.035268+0000","last_clean":"2024-09-13T16:48:00.035268+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:00.035268+0000","last_undegraded":"2024-09-13T16:48:00.035268+0000","last_fullsized":"2024-09-13T16:48:00.035268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266055+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:00.266055+0000","last_peered":"2024-09-13T16:48:00.266055+0000","last_clean":"2024-09-13T16:48:00.266055+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:00.266055+0000","last_undegraded":"2024-09-13T16:48:00.266055+0000","last_fullsized":"2024-09-13T16:48:00.266055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264081+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:00.264081+0000","last_peered":"2024-09-13T16:48:00.264081+0000","last_clean":"2024-09-13T16:48:00.264081+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:00.264081+0000","last_undegraded":"2024-09-13T16:48:00.264081+0000","last_fullsized":"2024-09-13T16:48:00.264081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265573+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:00.265573+0000","last_peered":"2024-09-13T16:48:00.265573+0000","last_clean":"2024-09-13T16:48:00.265573+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:00.265573+0000","last_undegraded":"2024-09-13T16:48:00.265573+0000","last_fullsized":"2024-09-13T16:48:00.265573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036257+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:00.036257+0000","last_peered":"2024-09-13T16:48:00.036257+0000","last_clean":"2024-09-13T16:48:00.036257+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:00.036257+0000","last_undegraded":"2024-09-13T16:48:00.036257+0000","last_fullsized":"2024-09-13T16:48:00.036257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263576+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:00.263576+0000","last_peered":"2024-09-13T16:48:00.263576+0000","last_clean":"2024-09-13T16:48:00.263576+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:00.263576+0000","last_undegraded":"2024-09-13T16:48:00.263576+0000","last_fullsized":"2024-09-13T16:48:00.263576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036516+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:00.036516+0000","last_peered":"2024-09-13T16:48:00.036516+0000","last_clean":"2024-09-13T16:48:00.036516+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:00.036516+0000","last_undegraded":"2024-09-13T16:48:00.036516+0000","last_fullsized":"2024-09-13T16:48:00.036516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263920+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:00.263920+0000","last_peered":"2024-09-13T16:48:00.263920+0000","last_clean":"2024-09-13T16:48:00.263920+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:00.263920+0000","last_undegraded":"2024-09-13T16:48:00.263920+0000","last_fullsized":"2024-09-13T16:48:00.263920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266290+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:00.266290+0000","last_peered":"2024-09-13T16:48:00.266290+0000","last_clean":"2024-09-13T16:48:00.266290+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:00.266290+0000","last_undegraded":"2024-09-13T16:48:00.266290+0000","last_fullsized":"2024-09-13T16:48:00.266290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266408+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:00.266408+0000","last_peered":"2024-09-13T16:48:00.266408+0000","last_clean":"2024-09-13T16:48:00.266408+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:00.266408+0000","last_undegraded":"2024-09-13T16:48:00.266408+0000","last_fullsized":"2024-09-13T16:48:00.266408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266689+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:00.266689+0000","last_peered":"2024-09-13T16:48:00.266689+0000","last_clean":"2024-09-13T16:48:00.266689+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:00.266689+0000","last_undegraded":"2024-09-13T16:48:00.266689+0000","last_fullsized":"2024-09-13T16:48:00.266689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266883+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:00.266883+0000","last_peered":"2024-09-13T16:48:00.266883+0000","last_clean":"2024-09-13T16:48:00.266883+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:00.266883+0000","last_undegraded":"2024-09-13T16:48:00.266883+0000","last_fullsized":"2024-09-13T16:48:00.266883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":72382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":70818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4630000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":212992,"data_stored":70978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7110000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":661824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:03.796 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d963a640 1 Processor -- start 2024-09-13T16:48:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d963a640 1 -- start start 2024-09-13T16:48:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d963a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29d4150f80 0x7f29d406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d963a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d41546c0 con 0x7f29d40c1310 2024-09-13T16:48:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d963a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d4151430 con 0x7f29d40c5330 2024-09-13T16:48:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d963a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d40c19a0 con 0x7f29d4150f80 2024-09-13T16:48:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29d4150f80 0x7f29d406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29d4150f80 0x7f29d406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41114/0 (socket says 172.21.15.39:41114) 2024-09-13T16:48:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d37fe640 1 -- 172.21.15.39:0/1804380292 learned_addr learned my addr 172.21.15.39:0/1804380292 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d37fe640 1 -- 172.21.15.39:0/1804380292 >> v1:172.21.15.84:6789/0 conn(0x7f29d40c5330 legacy=0x7f29d4153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d37fe640 1 -- 172.21.15.39:0/1804380292 >> v1:172.21.15.39:6789/0 conn(0x7f29d40c1310 legacy=0x7f29d40c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d37fe640 1 -- 172.21.15.39:0/1804380292 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40c49f0 con 0x7f29d4150f80 2024-09-13T16:48:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d27fc640 1 -- 172.21.15.39:0/1804380292 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806610465 0 0) 0x7f29d41546c0 con 0x7f29d40c1310 2024-09-13T16:48:03.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.956+0000 7f29d37fe640 1 --2- 172.21.15.39:0/1804380292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 0x7f29d406c090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009e80 tx=0x7f29cc01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a558fe3655f6c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:03.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d27fc640 1 -- 172.21.15.39:0/1804380292 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d406cde0 con 0x7f29d4150f80 2024-09-13T16:48:03.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d27fc640 1 -- 172.21.15.39:0/1804380292 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc0a5730 con 0x7f29d4150f80 2024-09-13T16:48:03.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d27fc640 1 -- 172.21.15.39:0/1804380292 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0a5ac0 con 0x7f29d4150f80 2024-09-13T16:48:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- 172.21.15.39:0/1804380292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 msgr2=0x7f29d406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 --2- 172.21.15.39:0/1804380292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 0x7f29d406c090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009e80 tx=0x7f29cc01c980 comp rx=0 tx=0).stop 2024-09-13T16:48:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- 172.21.15.39:0/1804380292 shutdown_connections 2024-09-13T16:48:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 --2- 172.21.15.39:0/1804380292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 0x7f29d406c090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- 172.21.15.39:0/1804380292 >> 172.21.15.39:0/1804380292 conn(0x7f29d4069e10 msgr2=0x7f29d406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:03.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- 172.21.15.39:0/1804380292 shutdown_connections 2024-09-13T16:48:03.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- 172.21.15.39:0/1804380292 wait complete. 2024-09-13T16:48:03.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 Processor -- start 2024-09-13T16:48:03.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- start start 2024-09-13T16:48:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d40c1310 0x7f29d414b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d40c5330 0x7f29d4147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 0x7f29d4148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d4148550 con 0x7f29d4150f80 2024-09-13T16:48:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40a67a0 con 0x7f29d40c1310 2024-09-13T16:48:03.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40a5420 con 0x7f29d40c5330 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 0x7f29d4148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 0x7f29d4148010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41128/0 (socket says 172.21.15.39:41128) 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d37fe640 1 -- 172.21.15.39:0/3408850254 learned_addr learned my addr 172.21.15.39:0/3408850254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d2ffd640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d40c1310 0x7f29d414b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29cbfff640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d40c5330 0x7f29d4147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d37fe640 1 -- 172.21.15.39:0/3408850254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d40c5330 msgr2=0x7f29d4147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d37fe640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d40c5330 0x7f29d4147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d37fe640 1 -- 172.21.15.39:0/3408850254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d40c1310 msgr2=0x7f29d414b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d37fe640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d40c1310 0x7f29d414b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d37fe640 1 -- 172.21.15.39:0/3408850254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc009b30 con 0x7f29d4150f80 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d2ffd640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d40c1310 0x7f29d414b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d37fe640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 0x7f29d4148010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0af040 tx=0x7f29cc019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:03.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29cb7fe640 1 -- 172.21.15.39:0/3408850254 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc003ba0 con 0x7f29d4150f80 2024-09-13T16:48:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29cb7fe640 1 -- 172.21.15.39:0/3408850254 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc003d00 con 0x7f29d4150f80 2024-09-13T16:48:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d40a5700 con 0x7f29d4150f80 2024-09-13T16:48:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29cb7fe640 1 -- 172.21.15.39:0/3408850254 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc004230 con 0x7f29d4150f80 2024-09-13T16:48:03.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.960+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d41e9cc0 con 0x7f29d4150f80 2024-09-13T16:48:03.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.964+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d406c090 con 0x7f29d4150f80 2024-09-13T16:48:03.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.964+0000 7f29cb7fe640 1 -- 172.21.15.39:0/3408850254 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29cc004390 con 0x7f29d4150f80 2024-09-13T16:48:03.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.964+0000 7f29cb7fe640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:03.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.964+0000 7f29cb7fe640 1 -- 172.21.15.39:0/3408850254 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f29cc11f3a0 con 0x7f29d4150f80 2024-09-13T16:48:03.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.964+0000 7f29d2ffd640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:03.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.964+0000 7f29d2ffd640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29c4000f70 tx=0x7f29c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:03.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:03.968+0000 7f29cb7fe640 1 -- 172.21.15.39:0/3408850254 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f29cc0ec8d0 con 0x7f29d4150f80 2024-09-13T16:48:04.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.116+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29d415ec90 con 0x7f29b4066690 2024-09-13T16:48:04.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.128+0000 7f29cb7fe640 1 -- 172.21.15.39:0/3408850254 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264630 (secure 0 0 0) 0x7f29c4001c90 con 0x7f29b4066690 2024-09-13T16:48:04.134 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:04.136 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 msgr2=0x7f29b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29c4000f70 tx=0x7f29c4009040 comp rx=0 tx=0).stop 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 msgr2=0x7f29d4148010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 0x7f29d4148010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0af040 tx=0x7f29cc019720 comp rx=0 tx=0).stop 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 shutdown_connections 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4150f80 0x7f29d4148010 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d40c5330 0x7f29d4147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 --2- 172.21.15.39:0/3408850254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d40c1310 0x7f29d414b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 >> 172.21.15.39:0/3408850254 conn(0x7f29d4069e10 msgr2=0x7f29d415c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 shutdown_connections 2024-09-13T16:48:04.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.132+0000 7f29d963a640 1 -- 172.21.15.39:0/3408850254 wait complete. 2024-09-13T16:48:04.143 INFO:tasks.ceph.mds.b.smithi084.stderr:2024-09-13T16:48:04.143+0000 7f5252c5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:48:04.152 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T16:48:03.558915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235368,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739352,"statfs":{"total":773094113280,"available":772853096448,"internally_reserved":0,"allocated":2891776,"data_stored":1752004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.926209"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264239+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:00.264239+0000","last_peered":"2024-09-13T16:48:00.264239+0000","last_clean":"2024-09-13T16:48:00.264239+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:00.264239+0000","last_undegraded":"2024-09-13T16:48:00.264239+0000","last_fullsized":"2024-09-13T16:48:00.264239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035349+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:00.035349+0000","last_peered":"2024-09-13T16:48:00.035349+0000","last_clean":"2024-09-13T16:48:00.035349+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:00.035349+0000","last_undegraded":"2024-09-13T16:48:00.035349+0000","last_fullsized":"2024-09-13T16:48:00.035349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266436+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:00.266436+0000","last_peered":"2024-09-13T16:48:00.266436+0000","last_clean":"2024-09-13T16:48:00.266436+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:00.266436+0000","last_undegraded":"2024-09-13T16:48:00.266436+0000","last_fullsized":"2024-09-13T16:48:00.266436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036261+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:00.036261+0000","last_peered":"2024-09-13T16:48:00.036261+0000","last_clean":"2024-09-13T16:48:00.036261+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:00.036261+0000","last_undegraded":"2024-09-13T16:48:00.036261+0000","last_fullsized":"2024-09-13T16:48:00.036261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263858+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:00.263858+0000","last_peered":"2024-09-13T16:48:00.263858+0000","last_clean":"2024-09-13T16:48:00.263858+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:00.263858+0000","last_undegraded":"2024-09-13T16:48:00.263858+0000","last_fullsized":"2024-09-13T16:48:00.263858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035241+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:00.035241+0000","last_peered":"2024-09-13T16:48:00.035241+0000","last_clean":"2024-09-13T16:48:00.035241+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:00.035241+0000","last_undegraded":"2024-09-13T16:48:00.035241+0000","last_fullsized":"2024-09-13T16:48:00.035241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263156+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:00.263156+0000","last_peered":"2024-09-13T16:48:00.263156+0000","last_clean":"2024-09-13T16:48:00.263156+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:00.263156+0000","last_undegraded":"2024-09-13T16:48:00.263156+0000","last_fullsized":"2024-09-13T16:48:00.263156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264056+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:00.264056+0000","last_peered":"2024-09-13T16:48:00.264056+0000","last_clean":"2024-09-13T16:48:00.264056+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:00.264056+0000","last_undegraded":"2024-09-13T16:48:00.264056+0000","last_fullsized":"2024-09-13T16:48:00.264056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264307+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:00.264307+0000","last_peered":"2024-09-13T16:48:00.264307+0000","last_clean":"2024-09-13T16:48:00.264307+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:00.264307+0000","last_undegraded":"2024-09-13T16:48:00.264307+0000","last_fullsized":"2024-09-13T16:48:00.264307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266206+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:00.266206+0000","last_peered":"2024-09-13T16:48:00.266206+0000","last_clean":"2024-09-13T16:48:00.266206+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:00.266206+0000","last_undegraded":"2024-09-13T16:48:00.266206+0000","last_fullsized":"2024-09-13T16:48:00.266206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265417+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:00.265417+0000","last_peered":"2024-09-13T16:48:00.265417+0000","last_clean":"2024-09-13T16:48:00.265417+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:00.265417+0000","last_undegraded":"2024-09-13T16:48:00.265417+0000","last_fullsized":"2024-09-13T16:48:00.265417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035298+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:00.035298+0000","last_peered":"2024-09-13T16:48:00.035298+0000","last_clean":"2024-09-13T16:48:00.035298+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:00.035298+0000","last_undegraded":"2024-09-13T16:48:00.035298+0000","last_fullsized":"2024-09-13T16:48:00.035298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265724+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:00.265724+0000","last_peered":"2024-09-13T16:48:00.265724+0000","last_clean":"2024-09-13T16:48:00.265724+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:00.265724+0000","last_undegraded":"2024-09-13T16:48:00.265724+0000","last_fullsized":"2024-09-13T16:48:00.265724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264662+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:00.264662+0000","last_peered":"2024-09-13T16:48:00.264662+0000","last_clean":"2024-09-13T16:48:00.264662+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:00.264662+0000","last_undegraded":"2024-09-13T16:48:00.264662+0000","last_fullsized":"2024-09-13T16:48:00.264662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035096+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:00.035096+0000","last_peered":"2024-09-13T16:48:00.035096+0000","last_clean":"2024-09-13T16:48:00.035096+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:00.035096+0000","last_undegraded":"2024-09-13T16:48:00.035096+0000","last_fullsized":"2024-09-13T16:48:00.035096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035358+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:00.035358+0000","last_peered":"2024-09-13T16:48:00.035358+0000","last_clean":"2024-09-13T16:48:00.035358+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:00.035358+0000","last_undegraded":"2024-09-13T16:48:00.035358+0000","last_fullsized":"2024-09-13T16:48:00.035358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036971+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:00.036971+0000","last_peered":"2024-09-13T16:48:00.036971+0000","last_clean":"2024-09-13T16:48:00.036971+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:00.036971+0000","last_undegraded":"2024-09-13T16:48:00.036971+0000","last_fullsized":"2024-09-13T16:48:00.036971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265064+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:00.265064+0000","last_peered":"2024-09-13T16:48:00.265064+0000","last_clean":"2024-09-13T16:48:00.265064+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:00.265064+0000","last_undegraded":"2024-09-13T16:48:00.265064+0000","last_fullsized":"2024-09-13T16:48:00.265064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263944+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:00.263944+0000","last_peered":"2024-09-13T16:48:00.263944+0000","last_clean":"2024-09-13T16:48:00.263944+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:00.263944+0000","last_undegraded":"2024-09-13T16:48:00.263944+0000","last_fullsized":"2024-09-13T16:48:00.263944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265671+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:00.265671+0000","last_peered":"2024-09-13T16:48:00.265671+0000","last_clean":"2024-09-13T16:48:00.265671+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:00.265671+0000","last_undegraded":"2024-09-13T16:48:00.265671+0000","last_fullsized":"2024-09-13T16:48:00.265671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266724+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:00.266724+0000","last_peered":"2024-09-13T16:48:00.266724+0000","last_clean":"2024-09-13T16:48:00.266724+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:00.266724+0000","last_undegraded":"2024-09-13T16:48:00.266724+0000","last_fullsized":"2024-09-13T16:48:00.266724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035473+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:00.035473+0000","last_peered":"2024-09-13T16:48:00.035473+0000","last_clean":"2024-09-13T16:48:00.035473+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:00.035473+0000","last_undegraded":"2024-09-13T16:48:00.035473+0000","last_fullsized":"2024-09-13T16:48:00.035473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265723+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:00.265723+0000","last_peered":"2024-09-13T16:48:00.265723+0000","last_clean":"2024-09-13T16:48:00.265723+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:00.265723+0000","last_undegraded":"2024-09-13T16:48:00.265723+0000","last_fullsized":"2024-09-13T16:48:00.265723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.034990+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:00.034990+0000","last_peered":"2024-09-13T16:48:00.034990+0000","last_clean":"2024-09-13T16:48:00.034990+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:00.034990+0000","last_undegraded":"2024-09-13T16:48:00.034990+0000","last_fullsized":"2024-09-13T16:48:00.034990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265455+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:00.265455+0000","last_peered":"2024-09-13T16:48:00.265455+0000","last_clean":"2024-09-13T16:48:00.265455+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:00.265455+0000","last_undegraded":"2024-09-13T16:48:00.265455+0000","last_fullsized":"2024-09-13T16:48:00.265455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266331+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:00.266331+0000","last_peered":"2024-09-13T16:48:00.266331+0000","last_clean":"2024-09-13T16:48:00.266331+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:00.266331+0000","last_undegraded":"2024-09-13T16:48:00.266331+0000","last_fullsized":"2024-09-13T16:48:00.266331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035334+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:00.035334+0000","last_peered":"2024-09-13T16:48:00.035334+0000","last_clean":"2024-09-13T16:48:00.035334+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:00.035334+0000","last_undegraded":"2024-09-13T16:48:00.035334+0000","last_fullsized":"2024-09-13T16:48:00.035334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266080+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:00.266080+0000","last_peered":"2024-09-13T16:48:00.266080+0000","last_clean":"2024-09-13T16:48:00.266080+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:00.266080+0000","last_undegraded":"2024-09-13T16:48:00.266080+0000","last_fullsized":"2024-09-13T16:48:00.266080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266881+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:00.266881+0000","last_peered":"2024-09-13T16:48:00.266881+0000","last_clean":"2024-09-13T16:48:00.266881+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:00.266881+0000","last_undegraded":"2024-09-13T16:48:00.266881+0000","last_fullsized":"2024-09-13T16:48:00.266881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035793+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:00.035793+0000","last_peered":"2024-09-13T16:48:00.035793+0000","last_clean":"2024-09-13T16:48:00.035793+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:00.035793+0000","last_undegraded":"2024-09-13T16:48:00.035793+0000","last_fullsized":"2024-09-13T16:48:00.035793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265446+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:00.265446+0000","last_peered":"2024-09-13T16:48:00.265446+0000","last_clean":"2024-09-13T16:48:00.265446+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:00.265446+0000","last_undegraded":"2024-09-13T16:48:00.265446+0000","last_fullsized":"2024-09-13T16:48:00.265446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265168+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:00.265168+0000","last_peered":"2024-09-13T16:48:00.265168+0000","last_clean":"2024-09-13T16:48:00.265168+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:00.265168+0000","last_undegraded":"2024-09-13T16:48:00.265168+0000","last_fullsized":"2024-09-13T16:48:00.265168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.034332+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:00.034332+0000","last_peered":"2024-09-13T16:48:00.034332+0000","last_clean":"2024-09-13T16:48:00.034332+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:00.034332+0000","last_undegraded":"2024-09-13T16:48:00.034332+0000","last_fullsized":"2024-09-13T16:48:00.034332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266324+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:00.266324+0000","last_peered":"2024-09-13T16:48:00.266324+0000","last_clean":"2024-09-13T16:48:00.266324+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:00.266324+0000","last_undegraded":"2024-09-13T16:48:00.266324+0000","last_fullsized":"2024-09-13T16:48:00.266324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.267013+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:00.267013+0000","last_peered":"2024-09-13T16:48:00.267013+0000","last_clean":"2024-09-13T16:48:00.267013+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:00.267013+0000","last_undegraded":"2024-09-13T16:48:00.267013+0000","last_fullsized":"2024-09-13T16:48:00.267013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265664+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:00.265664+0000","last_peered":"2024-09-13T16:48:00.265664+0000","last_clean":"2024-09-13T16:48:00.265664+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:00.265664+0000","last_undegraded":"2024-09-13T16:48:00.265664+0000","last_fullsized":"2024-09-13T16:48:00.265664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266251+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:00.266251+0000","last_peered":"2024-09-13T16:48:00.266251+0000","last_clean":"2024-09-13T16:48:00.266251+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:00.266251+0000","last_undegraded":"2024-09-13T16:48:00.266251+0000","last_fullsized":"2024-09-13T16:48:00.266251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266591+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:00.266591+0000","last_peered":"2024-09-13T16:48:00.266591+0000","last_clean":"2024-09-13T16:48:00.266591+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:00.266591+0000","last_undegraded":"2024-09-13T16:48:00.266591+0000","last_fullsized":"2024-09-13T16:48:00.266591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035119+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:00.035119+0000","last_peered":"2024-09-13T16:48:00.035119+0000","last_clean":"2024-09-13T16:48:00.035119+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:00.035119+0000","last_undegraded":"2024-09-13T16:48:00.035119+0000","last_fullsized":"2024-09-13T16:48:00.035119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.034977+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:00.034977+0000","last_peered":"2024-09-13T16:48:00.034977+0000","last_clean":"2024-09-13T16:48:00.034977+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:00.034977+0000","last_undegraded":"2024-09-13T16:48:00.034977+0000","last_fullsized":"2024-09-13T16:48:00.034977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265774+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:00.265774+0000","last_peered":"2024-09-13T16:48:00.265774+0000","last_clean":"2024-09-13T16:48:00.265774+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:00.265774+0000","last_undegraded":"2024-09-13T16:48:00.265774+0000","last_fullsized":"2024-09-13T16:48:00.265774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036116+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:00.036116+0000","last_peered":"2024-09-13T16:48:00.036116+0000","last_clean":"2024-09-13T16:48:00.036116+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:00.036116+0000","last_undegraded":"2024-09-13T16:48:00.036116+0000","last_fullsized":"2024-09-13T16:48:00.036116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036301+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:00.036301+0000","last_peered":"2024-09-13T16:48:00.036301+0000","last_clean":"2024-09-13T16:48:00.036301+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:00.036301+0000","last_undegraded":"2024-09-13T16:48:00.036301+0000","last_fullsized":"2024-09-13T16:48:00.036301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266561+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:00.266561+0000","last_peered":"2024-09-13T16:48:00.266561+0000","last_clean":"2024-09-13T16:48:00.266561+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:00.266561+0000","last_undegraded":"2024-09-13T16:48:00.266561+0000","last_fullsized":"2024-09-13T16:48:00.266561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035523+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:00.035523+0000","last_peered":"2024-09-13T16:48:00.035523+0000","last_clean":"2024-09-13T16:48:00.035523+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:00.035523+0000","last_undegraded":"2024-09-13T16:48:00.035523+0000","last_fullsized":"2024-09-13T16:48:00.035523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035368+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:00.035368+0000","last_peered":"2024-09-13T16:48:00.035368+0000","last_clean":"2024-09-13T16:48:00.035368+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:00.035368+0000","last_undegraded":"2024-09-13T16:48:00.035368+0000","last_fullsized":"2024-09-13T16:48:00.035368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265791+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:00.265791+0000","last_peered":"2024-09-13T16:48:00.265791+0000","last_clean":"2024-09-13T16:48:00.265791+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:00.265791+0000","last_undegraded":"2024-09-13T16:48:00.265791+0000","last_fullsized":"2024-09-13T16:48:00.265791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263372+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:00.263372+0000","last_peered":"2024-09-13T16:48:00.263372+0000","last_clean":"2024-09-13T16:48:00.263372+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:00.263372+0000","last_undegraded":"2024-09-13T16:48:00.263372+0000","last_fullsized":"2024-09-13T16:48:00.263372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.267125+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:00.267125+0000","last_peered":"2024-09-13T16:48:00.267125+0000","last_clean":"2024-09-13T16:48:00.267125+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:00.267125+0000","last_undegraded":"2024-09-13T16:48:00.267125+0000","last_fullsized":"2024-09-13T16:48:00.267125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264931+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:00.264931+0000","last_peered":"2024-09-13T16:48:00.264931+0000","last_clean":"2024-09-13T16:48:00.264931+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:00.264931+0000","last_undegraded":"2024-09-13T16:48:00.264931+0000","last_fullsized":"2024-09-13T16:48:00.264931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035706+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:00.035706+0000","last_peered":"2024-09-13T16:48:00.035706+0000","last_clean":"2024-09-13T16:48:00.035706+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:00.035706+0000","last_undegraded":"2024-09-13T16:48:00.035706+0000","last_fullsized":"2024-09-13T16:48:00.035706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263925+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:00.263925+0000","last_peered":"2024-09-13T16:48:00.263925+0000","last_clean":"2024-09-13T16:48:00.263925+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:00.263925+0000","last_undegraded":"2024-09-13T16:48:00.263925+0000","last_fullsized":"2024-09-13T16:48:00.263925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265326+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:00.265326+0000","last_peered":"2024-09-13T16:48:00.265326+0000","last_clean":"2024-09-13T16:48:00.265326+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:00.265326+0000","last_undegraded":"2024-09-13T16:48:00.265326+0000","last_fullsized":"2024-09-13T16:48:00.265326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265297+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:00.265297+0000","last_peered":"2024-09-13T16:48:00.265297+0000","last_clean":"2024-09-13T16:48:00.265297+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:00.265297+0000","last_undegraded":"2024-09-13T16:48:00.265297+0000","last_fullsized":"2024-09-13T16:48:00.265297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264752+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:00.264752+0000","last_peered":"2024-09-13T16:48:00.264752+0000","last_clean":"2024-09-13T16:48:00.264752+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:00.264752+0000","last_undegraded":"2024-09-13T16:48:00.264752+0000","last_fullsized":"2024-09-13T16:48:00.264752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263286+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:00.263286+0000","last_peered":"2024-09-13T16:48:00.263286+0000","last_clean":"2024-09-13T16:48:00.263286+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:00.263286+0000","last_undegraded":"2024-09-13T16:48:00.263286+0000","last_fullsized":"2024-09-13T16:48:00.263286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035289+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:00.035289+0000","last_peered":"2024-09-13T16:48:00.035289+0000","last_clean":"2024-09-13T16:48:00.035289+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:00.035289+0000","last_undegraded":"2024-09-13T16:48:00.035289+0000","last_fullsized":"2024-09-13T16:48:00.035289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264352+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:00.264352+0000","last_peered":"2024-09-13T16:48:00.264352+0000","last_clean":"2024-09-13T16:48:00.264352+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:00.264352+0000","last_undegraded":"2024-09-13T16:48:00.264352+0000","last_fullsized":"2024-09-13T16:48:00.264352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265322+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:00.265322+0000","last_peered":"2024-09-13T16:48:00.265322+0000","last_clean":"2024-09-13T16:48:00.265322+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:00.265322+0000","last_undegraded":"2024-09-13T16:48:00.265322+0000","last_fullsized":"2024-09-13T16:48:00.265322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264475+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:00.264475+0000","last_peered":"2024-09-13T16:48:00.264475+0000","last_clean":"2024-09-13T16:48:00.264475+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:00.264475+0000","last_undegraded":"2024-09-13T16:48:00.264475+0000","last_fullsized":"2024-09-13T16:48:00.264475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035614+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:00.035614+0000","last_peered":"2024-09-13T16:48:00.035614+0000","last_clean":"2024-09-13T16:48:00.035614+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:00.035614+0000","last_undegraded":"2024-09-13T16:48:00.035614+0000","last_fullsized":"2024-09-13T16:48:00.035614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036751+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:00.036751+0000","last_peered":"2024-09-13T16:48:00.036751+0000","last_clean":"2024-09-13T16:48:00.036751+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:00.036751+0000","last_undegraded":"2024-09-13T16:48:00.036751+0000","last_fullsized":"2024-09-13T16:48:00.036751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265183+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:00.265183+0000","last_peered":"2024-09-13T16:48:00.265183+0000","last_clean":"2024-09-13T16:48:00.265183+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:00.265183+0000","last_undegraded":"2024-09-13T16:48:00.265183+0000","last_fullsized":"2024-09-13T16:48:00.265183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263837+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:00.263837+0000","last_peered":"2024-09-13T16:48:00.263837+0000","last_clean":"2024-09-13T16:48:00.263837+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:00.263837+0000","last_undegraded":"2024-09-13T16:48:00.263837+0000","last_fullsized":"2024-09-13T16:48:00.263837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T16:47:59.040456+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:47:59.040456+0000","last_peered":"2024-09-13T16:47:59.040456+0000","last_clean":"2024-09-13T16:47:59.040456+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:47:59.040456+0000","last_undegraded":"2024-09-13T16:47:59.040456+0000","last_fullsized":"2024-09-13T16:47:59.040456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035766+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:00.035766+0000","last_peered":"2024-09-13T16:48:00.035766+0000","last_clean":"2024-09-13T16:48:00.035766+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:00.035766+0000","last_undegraded":"2024-09-13T16:48:00.035766+0000","last_fullsized":"2024-09-13T16:48:00.035766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265192+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:00.265192+0000","last_peered":"2024-09-13T16:48:00.265192+0000","last_clean":"2024-09-13T16:48:00.265192+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:00.265192+0000","last_undegraded":"2024-09-13T16:48:00.265192+0000","last_fullsized":"2024-09-13T16:48:00.265192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263224+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:00.263224+0000","last_peered":"2024-09-13T16:48:00.263224+0000","last_clean":"2024-09-13T16:48:00.263224+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:00.263224+0000","last_undegraded":"2024-09-13T16:48:00.263224+0000","last_fullsized":"2024-09-13T16:48:00.263224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264480+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:00.264480+0000","last_peered":"2024-09-13T16:48:00.264480+0000","last_clean":"2024-09-13T16:48:00.264480+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:00.264480+0000","last_undegraded":"2024-09-13T16:48:00.264480+0000","last_fullsized":"2024-09-13T16:48:00.264480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265666+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:00.265666+0000","last_peered":"2024-09-13T16:48:00.265666+0000","last_clean":"2024-09-13T16:48:00.265666+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:00.265666+0000","last_undegraded":"2024-09-13T16:48:00.265666+0000","last_fullsized":"2024-09-13T16:48:00.265666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035453+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:00.035453+0000","last_peered":"2024-09-13T16:48:00.035453+0000","last_clean":"2024-09-13T16:48:00.035453+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:00.035453+0000","last_undegraded":"2024-09-13T16:48:00.035453+0000","last_fullsized":"2024-09-13T16:48:00.035453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035808+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:00.035808+0000","last_peered":"2024-09-13T16:48:00.035808+0000","last_clean":"2024-09-13T16:48:00.035808+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:00.035808+0000","last_undegraded":"2024-09-13T16:48:00.035808+0000","last_fullsized":"2024-09-13T16:48:00.035808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035023+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:00.035023+0000","last_peered":"2024-09-13T16:48:00.035023+0000","last_clean":"2024-09-13T16:48:00.035023+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:00.035023+0000","last_undegraded":"2024-09-13T16:48:00.035023+0000","last_fullsized":"2024-09-13T16:48:00.035023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035263+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:00.035263+0000","last_peered":"2024-09-13T16:48:00.035263+0000","last_clean":"2024-09-13T16:48:00.035263+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:00.035263+0000","last_undegraded":"2024-09-13T16:48:00.035263+0000","last_fullsized":"2024-09-13T16:48:00.035263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036318+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:00.036318+0000","last_peered":"2024-09-13T16:48:00.036318+0000","last_clean":"2024-09-13T16:48:00.036318+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:00.036318+0000","last_undegraded":"2024-09-13T16:48:00.036318+0000","last_fullsized":"2024-09-13T16:48:00.036318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264034+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:00.264034+0000","last_peered":"2024-09-13T16:48:00.264034+0000","last_clean":"2024-09-13T16:48:00.264034+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:00.264034+0000","last_undegraded":"2024-09-13T16:48:00.264034+0000","last_fullsized":"2024-09-13T16:48:00.264034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035147+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:00.035147+0000","last_peered":"2024-09-13T16:48:00.035147+0000","last_clean":"2024-09-13T16:48:00.035147+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:00.035147+0000","last_undegraded":"2024-09-13T16:48:00.035147+0000","last_fullsized":"2024-09-13T16:48:00.035147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035553+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:00.035553+0000","last_peered":"2024-09-13T16:48:00.035553+0000","last_clean":"2024-09-13T16:48:00.035553+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:00.035553+0000","last_undegraded":"2024-09-13T16:48:00.035553+0000","last_fullsized":"2024-09-13T16:48:00.035553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265727+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:00.265727+0000","last_peered":"2024-09-13T16:48:00.265727+0000","last_clean":"2024-09-13T16:48:00.265727+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:00.265727+0000","last_undegraded":"2024-09-13T16:48:00.265727+0000","last_fullsized":"2024-09-13T16:48:00.265727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265936+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:00.265936+0000","last_peered":"2024-09-13T16:48:00.265936+0000","last_clean":"2024-09-13T16:48:00.265936+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:00.265936+0000","last_undegraded":"2024-09-13T16:48:00.265936+0000","last_fullsized":"2024-09-13T16:48:00.265936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266671+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:00.266671+0000","last_peered":"2024-09-13T16:48:00.266671+0000","last_clean":"2024-09-13T16:48:00.266671+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:00.266671+0000","last_undegraded":"2024-09-13T16:48:00.266671+0000","last_fullsized":"2024-09-13T16:48:00.266671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263191+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:00.263191+0000","last_peered":"2024-09-13T16:48:00.263191+0000","last_clean":"2024-09-13T16:48:00.263191+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:00.263191+0000","last_undegraded":"2024-09-13T16:48:00.263191+0000","last_fullsized":"2024-09-13T16:48:00.263191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.034061+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:00.034061+0000","last_peered":"2024-09-13T16:48:00.034061+0000","last_clean":"2024-09-13T16:48:00.034061+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:00.034061+0000","last_undegraded":"2024-09-13T16:48:00.034061+0000","last_fullsized":"2024-09-13T16:48:00.034061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035089+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:00.035089+0000","last_peered":"2024-09-13T16:48:00.035089+0000","last_clean":"2024-09-13T16:48:00.035089+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:00.035089+0000","last_undegraded":"2024-09-13T16:48:00.035089+0000","last_fullsized":"2024-09-13T16:48:00.035089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266826+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:00.266826+0000","last_peered":"2024-09-13T16:48:00.266826+0000","last_clean":"2024-09-13T16:48:00.266826+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:00.266826+0000","last_undegraded":"2024-09-13T16:48:00.266826+0000","last_fullsized":"2024-09-13T16:48:00.266826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.035268+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:00.035268+0000","last_peered":"2024-09-13T16:48:00.035268+0000","last_clean":"2024-09-13T16:48:00.035268+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:00.035268+0000","last_undegraded":"2024-09-13T16:48:00.035268+0000","last_fullsized":"2024-09-13T16:48:00.035268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266055+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:00.266055+0000","last_peered":"2024-09-13T16:48:00.266055+0000","last_clean":"2024-09-13T16:48:00.266055+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:00.266055+0000","last_undegraded":"2024-09-13T16:48:00.266055+0000","last_fullsized":"2024-09-13T16:48:00.266055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.264081+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:00.264081+0000","last_peered":"2024-09-13T16:48:00.264081+0000","last_clean":"2024-09-13T16:48:00.264081+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:00.264081+0000","last_undegraded":"2024-09-13T16:48:00.264081+0000","last_fullsized":"2024-09-13T16:48:00.264081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.265573+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:00.265573+0000","last_peered":"2024-09-13T16:48:00.265573+0000","last_clean":"2024-09-13T16:48:00.265573+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:00.265573+0000","last_undegraded":"2024-09-13T16:48:00.265573+0000","last_fullsized":"2024-09-13T16:48:00.265573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036257+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:00.036257+0000","last_peered":"2024-09-13T16:48:00.036257+0000","last_clean":"2024-09-13T16:48:00.036257+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:00.036257+0000","last_undegraded":"2024-09-13T16:48:00.036257+0000","last_fullsized":"2024-09-13T16:48:00.036257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263576+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:00.263576+0000","last_peered":"2024-09-13T16:48:00.263576+0000","last_clean":"2024-09-13T16:48:00.263576+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:00.263576+0000","last_undegraded":"2024-09-13T16:48:00.263576+0000","last_fullsized":"2024-09-13T16:48:00.263576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.036516+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:00.036516+0000","last_peered":"2024-09-13T16:48:00.036516+0000","last_clean":"2024-09-13T16:48:00.036516+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:00.036516+0000","last_undegraded":"2024-09-13T16:48:00.036516+0000","last_fullsized":"2024-09-13T16:48:00.036516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263920+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:00.263920+0000","last_peered":"2024-09-13T16:48:00.263920+0000","last_clean":"2024-09-13T16:48:00.263920+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:00.263920+0000","last_undegraded":"2024-09-13T16:48:00.263920+0000","last_fullsized":"2024-09-13T16:48:00.263920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266290+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:00.266290+0000","last_peered":"2024-09-13T16:48:00.266290+0000","last_clean":"2024-09-13T16:48:00.266290+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:00.266290+0000","last_undegraded":"2024-09-13T16:48:00.266290+0000","last_fullsized":"2024-09-13T16:48:00.266290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266408+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:00.266408+0000","last_peered":"2024-09-13T16:48:00.266408+0000","last_clean":"2024-09-13T16:48:00.266408+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:00.266408+0000","last_undegraded":"2024-09-13T16:48:00.266408+0000","last_fullsized":"2024-09-13T16:48:00.266408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266689+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:00.266689+0000","last_peered":"2024-09-13T16:48:00.266689+0000","last_clean":"2024-09-13T16:48:00.266689+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:00.266689+0000","last_undegraded":"2024-09-13T16:48:00.266689+0000","last_fullsized":"2024-09-13T16:48:00.266689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.266883+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:00.266883+0000","last_peered":"2024-09-13T16:48:00.266883+0000","last_clean":"2024-09-13T16:48:00.266883+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:00.266883+0000","last_undegraded":"2024-09-13T16:48:00.266883+0000","last_fullsized":"2024-09-13T16:48:00.266883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":72382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":204800,"data_stored":70818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4630000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":212992,"data_stored":70978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7110000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":661824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:04.157 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T16:48:04.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.300+0000 7f324fab1640 1 Processor -- start 2024-09-13T16:48:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.300+0000 7f324fab1640 1 -- start start 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f324814e0c0 0x7f32480a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f32480c4740 0x7f32480c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32480c2780 con 0x7f32480c1f50 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32480c2a60 con 0x7f324814e0c0 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32480c2d40 con 0x7f32480c4740 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324d84d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f324814e0c0 0x7f32480a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324d84d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f324814e0c0 0x7f32480a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41134/0 (socket says 172.21.15.39:41134) 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324d84d640 1 -- 172.21.15.39:0/3560017367 learned_addr learned my addr 172.21.15.39:0/3560017367 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324e04e640 1 --2- 172.21.15.39:0/3560017367 >> v2:172.21.15.84:3300/0 conn(0x7f32480c4740 0x7f32480c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324d84d640 1 -- 172.21.15.39:0/3560017367 >> v2:172.21.15.84:3300/0 conn(0x7f32480c4740 msgr2=0x7f32480c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324d84d640 1 --2- 172.21.15.39:0/3560017367 >> v2:172.21.15.84:3300/0 conn(0x7f32480c4740 0x7f32480c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324d84d640 1 -- 172.21.15.39:0/3560017367 >> v1:172.21.15.39:6789/0 conn(0x7f32480c1f50 legacy=0x7f324806cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324d84d640 1 -- 172.21.15.39:0/3560017367 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32480c3020 con 0x7f324814e0c0 2024-09-13T16:48:04.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324c84b640 1 -- 172.21.15.39:0/3560017367 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293405759 0 0) 0x7f32480c2780 con 0x7f32480c1f50 2024-09-13T16:48:04.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324e04e640 1 --2- 172.21.15.39:0/3560017367 >> v2:172.21.15.84:3300/0 conn(0x7f32480c4740 0x7f32480c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:04.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324d84d640 1 --2- 172.21.15.39:0/3560017367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f324814e0c0 0x7f32480a4e70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32480c3920 tx=0x7f324401eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e4003840098d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:04.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324c84b640 1 -- 172.21.15.39:0/3560017367 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440a7a10 con 0x7f324814e0c0 2024-09-13T16:48:04.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324c84b640 1 -- 172.21.15.39:0/3560017367 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32440a7b70 con 0x7f324814e0c0 2024-09-13T16:48:04.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324c84b640 1 -- 172.21.15.39:0/3560017367 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440af440 con 0x7f324814e0c0 2024-09-13T16:48:04.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 -- 172.21.15.39:0/3560017367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f324814e0c0 msgr2=0x7f32480a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 --2- 172.21.15.39:0/3560017367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f324814e0c0 0x7f32480a4e70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32480c3920 tx=0x7f324401eb30 comp rx=0 tx=0).stop 2024-09-13T16:48:04.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 -- 172.21.15.39:0/3560017367 shutdown_connections 2024-09-13T16:48:04.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 --2- 172.21.15.39:0/3560017367 >> v2:172.21.15.84:3300/0 conn(0x7f32480c4740 0x7f32480c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 --2- 172.21.15.39:0/3560017367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f324814e0c0 0x7f32480a4e70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 -- 172.21.15.39:0/3560017367 >> 172.21.15.39:0/3560017367 conn(0x7f3248069e10 msgr2=0x7f324806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:04.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 -- 172.21.15.39:0/3560017367 shutdown_connections 2024-09-13T16:48:04.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.304+0000 7f324fab1640 1 -- 172.21.15.39:0/3560017367 wait complete. 2024-09-13T16:48:04.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 Processor -- start 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 -- start start 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32480c1f50 0x7f32481ebb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32480c4740 0x7f3248147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f324814e0c0 0x7f32481475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3248145b70 con 0x7f32480c1f50 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3248072290 con 0x7f32480c4740 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324806f400 con 0x7f324814e0c0 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d84d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32480c1f50 0x7f32481ebb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d84d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32480c1f50 0x7f32481ebb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41136/0 (socket says 172.21.15.39:41136) 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d84d640 1 -- 172.21.15.39:0/2744202832 learned_addr learned my addr 172.21.15.39:0/2744202832 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d84d640 1 -- 172.21.15.39:0/2744202832 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f324814e0c0 msgr2=0x7f32481475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d84d640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f324814e0c0 0x7f32481475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d84d640 1 -- 172.21.15.39:0/2744202832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32480c4740 msgr2=0x7f3248147070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d04c640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32480c4740 0x7f3248147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d84d640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32480c4740 0x7f3248147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d84d640 1 -- 172.21.15.39:0/2744202832 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324400b770 con 0x7f32480c1f50 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324d84d640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32480c1f50 0x7f32481ebb90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f324400b2c0 tx=0x7f324401b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f323e7fc640 1 -- 172.21.15.39:0/2744202832 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440ba640 con 0x7f32480c1f50 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f323e7fc640 1 -- 172.21.15.39:0/2744202832 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32440ba7a0 con 0x7f32480c1f50 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3248146000 con 0x7f32480c1f50 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f323e7fc640 1 -- 172.21.15.39:0/2744202832 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440bab50 con 0x7f32480c1f50 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32481ec470 con 0x7f32480c1f50 2024-09-13T16:48:04.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324806c880 con 0x7f32480c1f50 2024-09-13T16:48:04.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.308+0000 7f323e7fc640 1 -- 172.21.15.39:0/2744202832 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f32440b6040 con 0x7f32480c1f50 2024-09-13T16:48:04.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.312+0000 7f323e7fc640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f322c066690 0x7f322c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.312+0000 7f323e7fc640 1 -- 172.21.15.39:0/2744202832 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f3244121670 con 0x7f32480c1f50 2024-09-13T16:48:04.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.312+0000 7f324d04c640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f322c066690 0x7f322c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.312+0000 7f324d04c640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f322c066690 0x7f322c068b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f324814d890 tx=0x7f3238009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:04.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.316+0000 7f323e7fc640 1 -- 172.21.15.39:0/2744202832 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f32440eea80 con 0x7f32480c1f50 2024-09-13T16:48:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.468+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32480a5190 con 0x7f32480c1f50 2024-09-13T16:48:04.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f323e7fc640 1 -- 172.21.15.39:0/2744202832 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f32440f20a0 con 0x7f32480c1f50 2024-09-13T16:48:04.475 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:04.475 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":20,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:00.025477+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:04.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f322c066690 msgr2=0x7f322c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f322c066690 0x7f322c068b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f324814d890 tx=0x7f3238009040 comp rx=0 tx=0).stop 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32480c1f50 msgr2=0x7f32481ebb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32480c1f50 0x7f32481ebb90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f324400b2c0 tx=0x7f324401b8d0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 shutdown_connections 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f322c066690 0x7f322c068b50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32480c4740 0x7f3248147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f324814e0c0 0x7f32481475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 --2- 172.21.15.39:0/2744202832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32480c1f50 0x7f32481ebb90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 >> 172.21.15.39:0/2744202832 conn(0x7f3248069e10 msgr2=0x7f324814e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 shutdown_connections 2024-09-13T16:48:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.472+0000 7f324fab1640 1 -- 172.21.15.39:0/2744202832 wait complete. 2024-09-13T16:48:04.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:48:04.493 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:48:04.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c88b7640 1 Processor -- start 2024-09-13T16:48:04.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c88b7640 1 -- start start 2024-09-13T16:48:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c88b7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c2200 0x7ff0c4150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c88b7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff0c40c4720 0x7ff0c4151530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c88b7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c40c06c0 con 0x7ff0c40c4080 2024-09-13T16:48:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c88b7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c40c2fa0 con 0x7ff0c40c4720 2024-09-13T16:48:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c88b7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40c0090 con 0x7ff0c40c2200 2024-09-13T16:48:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c359e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff0c40c4720 0x7ff0c4151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c359e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff0c40c4720 0x7ff0c4151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41142/0 (socket says 172.21.15.39:41142) 2024-09-13T16:48:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c359e640 1 -- 172.21.15.39:0/433371405 learned_addr learned my addr 172.21.15.39:0/433371405 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c2d9d640 1 --2- 172.21.15.39:0/433371405 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c2200 0x7ff0c4150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c359e640 1 -- 172.21.15.39:0/433371405 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c2200 msgr2=0x7ff0c4150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c359e640 1 --2- 172.21.15.39:0/433371405 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c2200 0x7ff0c4150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c359e640 1 -- 172.21.15.39:0/433371405 >> v1:172.21.15.84:6789/0 conn(0x7ff0c40c4080 legacy=0x7ff0c4157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.616+0000 7ff0c359e640 1 -- 172.21.15.39:0/433371405 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40c4d90 con 0x7ff0c40c4720 2024-09-13T16:48:04.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c359e640 1 --2- 172.21.15.39:0/433371405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c4720 0x7ff0c4151530 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009e50 tx=0x7ff0bc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad747f93c0f81a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:04.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c1d9b640 1 -- 172.21.15.39:0/433371405 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0a5880 con 0x7ff0c40c4720 2024-09-13T16:48:04.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c1d9b640 1 -- 172.21.15.39:0/433371405 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0a59e0 con 0x7ff0c40c4720 2024-09-13T16:48:04.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c1d9b640 1 -- 172.21.15.39:0/433371405 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0a5d70 con 0x7ff0c40c4720 2024-09-13T16:48:04.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- 172.21.15.39:0/433371405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c4720 msgr2=0x7ff0c4151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 --2- 172.21.15.39:0/433371405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c4720 0x7ff0c4151530 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009e50 tx=0x7ff0bc01c9a0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- 172.21.15.39:0/433371405 shutdown_connections 2024-09-13T16:48:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 --2- 172.21.15.39:0/433371405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c4720 0x7ff0c4151530 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 --2- 172.21.15.39:0/433371405 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c2200 0x7ff0c4150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- 172.21.15.39:0/433371405 >> 172.21.15.39:0/433371405 conn(0x7ff0c40c5510 msgr2=0x7ff0c40c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- 172.21.15.39:0/433371405 shutdown_connections 2024-09-13T16:48:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- 172.21.15.39:0/433371405 wait complete. 2024-09-13T16:48:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 Processor -- start 2024-09-13T16:48:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- start start 2024-09-13T16:48:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c2200 0x7ff0c41ef960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0c40c4080 0x7ff0c406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c4720 0x7ff0c4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c414dd90 con 0x7ff0c40c2200 2024-09-13T16:48:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c414e070 con 0x7ff0c40c4080 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4071500 con 0x7ff0c40c4720 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c2200 0x7ff0c41ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c2200 0x7ff0c41ef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41154/0 (socket says 172.21.15.39:41154) 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c2d9d640 1 -- 172.21.15.39:0/3803321116 learned_addr learned my addr 172.21.15.39:0/3803321116 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c259c640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0c40c4080 0x7ff0c406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c359e640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c4720 0x7ff0c4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c2d9d640 1 -- 172.21.15.39:0/3803321116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c4720 msgr2=0x7ff0c4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c2d9d640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c4720 0x7ff0c4071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c2d9d640 1 -- 172.21.15.39:0/3803321116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0c40c4080 msgr2=0x7ff0c406c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c2d9d640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0c40c4080 0x7ff0c406c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c2d9d640 1 -- 172.21.15.39:0/3803321116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc009b30 con 0x7ff0c40c2200 2024-09-13T16:48:04.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c259c640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0c40c4080 0x7ff0c406c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:04.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c2d9d640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c2200 0x7ff0c41ef960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400d5f0 tx=0x7ff0b400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:04.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0ab7fe640 1 -- 172.21.15.39:0/3803321116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40125f0 con 0x7ff0c40c2200 2024-09-13T16:48:04.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0ab7fe640 1 -- 172.21.15.39:0/3803321116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b4012750 con 0x7ff0c40c2200 2024-09-13T16:48:04.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0ab7fe640 1 -- 172.21.15.39:0/3803321116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4012ae0 con 0x7ff0c40c2200 2024-09-13T16:48:04.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c40717e0 con 0x7ff0c40c2200 2024-09-13T16:48:04.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.620+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c406cce0 con 0x7ff0c40c2200 2024-09-13T16:48:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.624+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c4157260 con 0x7ff0c40c2200 2024-09-13T16:48:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.624+0000 7ff0ab7fe640 1 -- 172.21.15.39:0/3803321116 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff0b4012c40 con 0x7ff0c40c2200 2024-09-13T16:48:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.624+0000 7ff0ab7fe640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0b0066690 0x7ff0b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.624+0000 7ff0ab7fe640 1 -- 172.21.15.39:0/3803321116 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7ff0b4084590 con 0x7ff0c40c2200 2024-09-13T16:48:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.624+0000 7ff0c259c640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0b0066690 0x7ff0b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.624+0000 7ff0c259c640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0b0066690 0x7ff0b0068b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0c406d1b0 tx=0x7ff0b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:04.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.628+0000 7ff0ab7fe640 1 -- 172.21.15.39:0/3803321116 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff0b40519a0 con 0x7ff0c40c2200 2024-09-13T16:48:04.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.780+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0c4150980 con 0x7ff0c40c2200 2024-09-13T16:48:04.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.780+0000 7ff0ab7fe640 1 -- 172.21.15.39:0/3803321116 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0b4007380 con 0x7ff0c40c2200 2024-09-13T16:48:04.786 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:48:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0b0066690 msgr2=0x7ff0b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0b0066690 0x7ff0b0068b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0c406d1b0 tx=0x7ff0b8009040 comp rx=0 tx=0).stop 2024-09-13T16:48:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c2200 msgr2=0x7ff0c41ef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c2200 0x7ff0c41ef960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400d5f0 tx=0x7ff0b400dab0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 shutdown_connections 2024-09-13T16:48:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0b0066690 0x7ff0b0068b50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff0c40c4080 0x7ff0c406c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff0c40c4720 0x7ff0c4071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 --2- 172.21.15.39:0/3803321116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0c40c2200 0x7ff0c41ef960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 >> 172.21.15.39:0/3803321116 conn(0x7ff0c40c5510 msgr2=0x7ff0c415dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 shutdown_connections 2024-09-13T16:48:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.784+0000 7ff0c88b7640 1 -- 172.21.15.39:0/3803321116 wait complete. 2024-09-13T16:48:04.804 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T16:48:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432fbcd640 1 Processor -- start 2024-09-13T16:48:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432fbcd640 1 -- start start 2024-09-13T16:48:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432fbcd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43280aa2d0 0x7f43280aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432fbcd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280a4d60 0x7f432814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432fbcd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f432814f4b0 0x7f4328151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432fbcd640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4328155b30 con 0x7f432814f4b0 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432fbcd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4328155d20 con 0x7f43280aa2d0 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432fbcd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280c4750 con 0x7f43280a4d60 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432e16a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f432814f4b0 0x7f4328151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432e16a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f432814f4b0 0x7f4328151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41168/0 (socket says 172.21.15.39:41168) 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432e16a640 1 -- 172.21.15.39:0/2829127262 learned_addr learned my addr 172.21.15.39:0/2829127262 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432d969640 1 --2- 172.21.15.39:0/2829127262 >> v2:172.21.15.84:3300/0 conn(0x7f43280aa2d0 0x7f43280aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432d168640 1 --2- 172.21.15.39:0/2829127262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280a4d60 0x7f432814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432e16a640 1 -- 172.21.15.39:0/2829127262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280a4d60 msgr2=0x7f432814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432e16a640 1 --2- 172.21.15.39:0/2829127262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280a4d60 0x7f432814ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432e16a640 1 -- 172.21.15.39:0/2829127262 >> v2:172.21.15.84:3300/0 conn(0x7f43280aa2d0 msgr2=0x7f43280aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432e16a640 1 --2- 172.21.15.39:0/2829127262 >> v2:172.21.15.84:3300/0 conn(0x7f43280aa2d0 0x7f43280aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432e16a640 1 -- 172.21.15.39:0/2829127262 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4328155fc0 con 0x7f432814f4b0 2024-09-13T16:48:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432d168640 1 --2- 172.21.15.39:0/2829127262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280a4d60 0x7f432814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.920+0000 7f432e16a640 1 --2- 172.21.15.39:0/2829127262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f432814f4b0 0x7f4328151cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f432806d4d0 tx=0x7f431801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93632b8e59f6e56b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432c967640 1 -- 172.21.15.39:0/2829127262 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328152a40 con 0x7f432814f4b0 2024-09-13T16:48:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432c967640 1 -- 172.21.15.39:0/2829127262 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43180a7050 con 0x7f432814f4b0 2024-09-13T16:48:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432c967640 1 -- 172.21.15.39:0/2829127262 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180ab820 con 0x7f432814f4b0 2024-09-13T16:48:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- 172.21.15.39:0/2829127262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f432814f4b0 msgr2=0x7f4328151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 --2- 172.21.15.39:0/2829127262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f432814f4b0 0x7f4328151cf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f432806d4d0 tx=0x7f431801c460 comp rx=0 tx=0).stop 2024-09-13T16:48:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- 172.21.15.39:0/2829127262 shutdown_connections 2024-09-13T16:48:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 --2- 172.21.15.39:0/2829127262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f432814f4b0 0x7f4328151cf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 --2- 172.21.15.39:0/2829127262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280a4d60 0x7f432814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 --2- 172.21.15.39:0/2829127262 >> v2:172.21.15.84:3300/0 conn(0x7f43280aa2d0 0x7f43280aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- 172.21.15.39:0/2829127262 >> 172.21.15.39:0/2829127262 conn(0x7f43280c5790 msgr2=0x7f43280c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:04.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- 172.21.15.39:0/2829127262 shutdown_connections 2024-09-13T16:48:04.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- 172.21.15.39:0/2829127262 wait complete. 2024-09-13T16:48:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 Processor -- start 2024-09-13T16:48:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- start start 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43280a4d60 0x7f43281ec860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280aa2d0 0x7f43280c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f432814f4b0 0x7f43281416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432806f5c0 con 0x7f43280a4d60 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43281ecda0 con 0x7f432814f4b0 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280a8690 con 0x7f43280aa2d0 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d969640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43280a4d60 0x7f43281ec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d969640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43280a4d60 0x7f43281ec860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41176/0 (socket says 172.21.15.39:41176) 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d969640 1 -- 172.21.15.39:0/3770525320 learned_addr learned my addr 172.21.15.39:0/3770525320 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432e16a640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f432814f4b0 0x7f43281416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d168640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280aa2d0 0x7f43280c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d969640 1 -- 172.21.15.39:0/3770525320 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280aa2d0 msgr2=0x7f43280c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d969640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280aa2d0 0x7f43280c2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d969640 1 -- 172.21.15.39:0/3770525320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f432814f4b0 msgr2=0x7f43281416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d969640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f432814f4b0 0x7f43281416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d969640 1 -- 172.21.15.39:0/3770525320 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180094d0 con 0x7f43280a4d60 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432d969640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43280a4d60 0x7f43281ec860 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f432000d800 tx=0x7f432000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f431e7fc640 1 -- 172.21.15.39:0/3770525320 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432000b6c0 con 0x7f43280a4d60 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f431e7fc640 1 -- 172.21.15.39:0/3770525320 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432000b820 con 0x7f43280a4d60 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432814e280 con 0x7f43280a4d60 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f431e7fc640 1 -- 172.21.15.39:0/3770525320 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432000bb90 con 0x7f43280a4d60 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.924+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43280c0230 con 0x7f43280a4d60 2024-09-13T16:48:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.928+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43280aa6b0 con 0x7f43280a4d60 2024-09-13T16:48:04.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.928+0000 7f431e7fc640 1 -- 172.21.15.39:0/3770525320 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f432000bcf0 con 0x7f43280a4d60 2024-09-13T16:48:04.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.928+0000 7f431e7fc640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f430c066690 0x7f430c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.928+0000 7f431e7fc640 1 -- 172.21.15.39:0/3770525320 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f43200846c0 con 0x7f43280a4d60 2024-09-13T16:48:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.928+0000 7f432d168640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f430c066690 0x7f430c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.928+0000 7f432d168640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f430c066690 0x7f430c068b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f432806d300 tx=0x7f4324008460 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:04.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:04.932+0000 7f431e7fc640 1 -- 172.21.15.39:0/3770525320 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4320051a40 con 0x7f43280a4d60 2024-09-13T16:48:05.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.084+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43280c53a0 con 0x7f43280a4d60 2024-09-13T16:48:05.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.084+0000 7f431e7fc640 1 -- 172.21.15.39:0/3770525320 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v20) v1 ==== 99+0+11 (secure 0 0 0) 0x7f4320055060 con 0x7f43280a4d60 2024-09-13T16:48:05.090 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f430c066690 msgr2=0x7f430c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f430c066690 0x7f430c068b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f432806d300 tx=0x7f4324008460 comp rx=0 tx=0).stop 2024-09-13T16:48:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43280a4d60 msgr2=0x7f43281ec860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43280a4d60 0x7f43281ec860 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f432000d800 tx=0x7f432000dcc0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 shutdown_connections 2024-09-13T16:48:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f430c066690 0x7f430c068b50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f432814f4b0 0x7f43281416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f43280aa2d0 0x7f43280c2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 --2- 172.21.15.39:0/3770525320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43280a4d60 0x7f43281ec860 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 >> 172.21.15.39:0/3770525320 conn(0x7f43280c5790 msgr2=0x7f4328149960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 shutdown_connections 2024-09-13T16:48:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.088+0000 7f432fbcd640 1 -- 172.21.15.39:0/3770525320 wait complete. 2024-09-13T16:48:05.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T16:48:05.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f007d54c640 1 Processor -- start 2024-09-13T16:48:05.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f007d54c640 1 -- start start 2024-09-13T16:48:05.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f007d54c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0078151cf0 0x7f0078152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f007d54c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0078155d20 con 0x7f00780a4d60 2024-09-13T16:48:05.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f007d54c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00780c4750 con 0x7f00780aa2d0 2024-09-13T16:48:05.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f007d54c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0078155b30 con 0x7f0078151cf0 2024-09-13T16:48:05.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f00767fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f00780a4d60 0x7f007814ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38678/0 (socket says 172.21.15.39:38678) 2024-09-13T16:48:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f00767fc640 1 -- 172.21.15.39:0/630719170 learned_addr learned my addr 172.21.15.39:0/630719170 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f00777fe640 1 --2- 172.21.15.39:0/630719170 >> v2:172.21.15.84:3300/0 conn(0x7f0078151cf0 0x7f0078152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482196247 0 0) 0x7f0078155d20 con 0x7f00780a4d60 2024-09-13T16:48:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0060003530 con 0x7f00780a4d60 2024-09-13T16:48:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3288244272 0 0) 0x7f0060003530 con 0x7f00780a4d60 2024-09-13T16:48:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 >> v2:172.21.15.84:3300/0 conn(0x7f0078151cf0 msgr2=0x7f0078152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 --2- 172.21.15.39:0/630719170 >> v2:172.21.15.84:3300/0 conn(0x7f0078151cf0 0x7f0078152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 >> v1:172.21.15.84:6789/0 conn(0x7f00780aa2d0 legacy=0x7f00780aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0078155fc0 con 0x7f00780a4d60 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f006c001680 con 0x7f00780a4d60 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 >> v1:172.21.15.39:6789/0 conn(0x7f00780a4d60 legacy=0x7f007814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003980 0x7f0060003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00600047d0 0x7f0060004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0060005190 0x7f006000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006000c100 con 0x7f0060003980 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006000c270 con 0x7f0060005190 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006000c430 con 0x7f00600047d0 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0076ffd640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003980 0x7f0060003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f00777fe640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0060005190 0x7f006000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f00767fc640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00600047d0 0x7f0060004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0076ffd640 1 -- 172.21.15.39:0/630719170 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00600047d0 msgr2=0x7f0060004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:05.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0076ffd640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00600047d0 0x7f0060004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0076ffd640 1 -- 172.21.15.39:0/630719170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0060005190 msgr2=0x7f006000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:05.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0076ffd640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0060005190 0x7f006000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f0076ffd640 1 -- 172.21.15.39:0/630719170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060003530 con 0x7f0060003980 2024-09-13T16:48:05.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.228+0000 7f00777fe640 1 -- 172.21.15.39:0/630719170 reap_dead start 2024-09-13T16:48:05.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f0076ffd640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003980 0x7f0060003da0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0068001780 tx=0x7f006801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fdc64655d221a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:05.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00680052d0 con 0x7f0060003980 2024-09-13T16:48:05.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00680a7050 con 0x7f0060003980 2024-09-13T16:48:05.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f0075ffb640 1 -- 172.21.15.39:0/630719170 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068003600 con 0x7f0060003980 2024-09-13T16:48:05.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f007d54c640 1 -- 172.21.15.39:0/630719170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003980 msgr2=0x7f0060003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f007d54c640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003980 0x7f0060003da0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0068001780 tx=0x7f006801c3d0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f007d54c640 1 -- 172.21.15.39:0/630719170 shutdown_connections 2024-09-13T16:48:05.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f007d54c640 1 --2- 172.21.15.39:0/630719170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003980 0x7f0060003da0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f007d54c640 1 -- 172.21.15.39:0/630719170 >> 172.21.15.39:0/630719170 conn(0x7f00780c5790 msgr2=0x7f00780c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:05.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f007d54c640 1 -- 172.21.15.39:0/630719170 shutdown_connections 2024-09-13T16:48:05.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f007d54c640 1 -- 172.21.15.39:0/630719170 wait complete. 2024-09-13T16:48:05.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f007d54c640 1 Processor -- start 2024-09-13T16:48:05.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.232+0000 7f007d54c640 1 -- start start 2024-09-13T16:48:05.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f007d54c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0060003980 0x7f00780c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f007d54c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00781e9a90 0x7f00780a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f007d54c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00780a6d30 0x7f00780a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f007d54c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0078071bc0 con 0x7f00781e9a90 2024-09-13T16:48:05.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f007d54c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780a7400 con 0x7f0060003980 2024-09-13T16:48:05.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f007d54c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780a86f0 con 0x7f00780a6d30 2024-09-13T16:48:05.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00767fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00781e9a90 0x7f00780a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00767fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00781e9a90 0x7f00780a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41206/0 (socket says 172.21.15.39:41206) 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00767fc640 1 -- 172.21.15.39:0/1947399627 learned_addr learned my addr 172.21.15.39:0/1947399627 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f0076ffd640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0060003980 0x7f00780c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00777fe640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00780a6d30 0x7f00780a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00767fc640 1 -- 172.21.15.39:0/1947399627 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00780a6d30 msgr2=0x7f00780a78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00767fc640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00780a6d30 0x7f00780a78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00767fc640 1 -- 172.21.15.39:0/1947399627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0060003980 msgr2=0x7f00780c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00767fc640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0060003980 0x7f00780c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00767fc640 1 -- 172.21.15.39:0/1947399627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0078071e10 con 0x7f00781e9a90 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f00767fc640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00781e9a90 0x7f00780a67f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d7d0 tx=0x7f006c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f005f7fe640 1 -- 172.21.15.39:0/1947399627 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0074e0 con 0x7f00781e9a90 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f005f7fe640 1 -- 172.21.15.39:0/1947399627 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c007640 con 0x7f00781e9a90 2024-09-13T16:48:05.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00780720f0 con 0x7f00781e9a90 2024-09-13T16:48:05.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f005f7fe640 1 -- 172.21.15.39:0/1947399627 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c004a30 con 0x7f00781e9a90 2024-09-13T16:48:05.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00780a5c30 con 0x7f00781e9a90 2024-09-13T16:48:05.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007814f5a0 con 0x7f00781e9a90 2024-09-13T16:48:05.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f005f7fe640 1 -- 172.21.15.39:0/1947399627 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f006c004b90 con 0x7f00781e9a90 2024-09-13T16:48:05.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f005f7fe640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00640666c0 0x7f0064068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f005f7fe640 1 -- 172.21.15.39:0/1947399627 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f006c083d20 con 0x7f00781e9a90 2024-09-13T16:48:05.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f0076ffd640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00640666c0 0x7f0064068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.236+0000 7f0076ffd640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00640666c0 0x7f0064068b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0068001750 tx=0x7f00680011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:05.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.240+0000 7f005f7fe640 1 -- 172.21.15.39:0/1947399627 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f006c051130 con 0x7f00781e9a90 2024-09-13T16:48:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.392+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f0078148b70 con 0x7f00781e9a90 2024-09-13T16:48:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f005f7fe640 1 -- 172.21.15.39:0/1947399627 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v20) v1 ==== 95+0+11 (secure 0 0 0) 0x7f006c0190a0 con 0x7f00781e9a90 2024-09-13T16:48:05.399 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00640666c0 msgr2=0x7f0064068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00640666c0 0x7f0064068b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0068001750 tx=0x7f00680011a0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00781e9a90 msgr2=0x7f00780a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00781e9a90 0x7f00780a67f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f006c00d7d0 tx=0x7f006c00dc90 comp rx=0 tx=0).stop 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 shutdown_connections 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00640666c0 0x7f0064068b80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00780a6d30 0x7f00780a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00781e9a90 0x7f00780a67f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 --2- 172.21.15.39:0/1947399627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0060003980 0x7f00780c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 >> 172.21.15.39:0/1947399627 conn(0x7f00780c5790 msgr2=0x7f007814c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 shutdown_connections 2024-09-13T16:48:05.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.396+0000 7f007d54c640 1 -- 172.21.15.39:0/1947399627 wait complete. 2024-09-13T16:48:05.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T16:48:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b8215640 1 Processor -- start 2024-09-13T16:48:05.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b8215640 1 -- start start 2024-09-13T16:48:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b8215640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc1b0153df0 0x7fc1b00a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b8215640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1b0157740 0x7fc1b0157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b8215640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b00c2260 con 0x7fc1b00c1e80 2024-09-13T16:48:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b8215640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b0070a60 con 0x7fc1b0153df0 2024-09-13T16:48:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b8215640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0154280 con 0x7fc1b0157740 2024-09-13T16:48:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b57b0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc1b0153df0 0x7fc1b00a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b57b0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc1b0153df0 0x7fc1b00a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41208/0 (socket says 172.21.15.39:41208) 2024-09-13T16:48:05.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b57b0640 1 -- 172.21.15.39:0/2495257890 learned_addr learned my addr 172.21.15.39:0/2495257890 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b5fb1640 1 --1- 172.21.15.39:0/2495257890 >> v1:172.21.15.39:6789/0 conn(0x7fc1b00c1e80 0x7fc1b00c4b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38692/0 (socket says 172.21.15.39:38692) 2024-09-13T16:48:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b67b2640 1 --2- 172.21.15.39:0/2495257890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1b0157740 0x7fc1b0157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b57b0640 1 -- 172.21.15.39:0/2495257890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1b0157740 msgr2=0x7fc1b0157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b57b0640 1 --2- 172.21.15.39:0/2495257890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1b0157740 0x7fc1b0157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b57b0640 1 -- 172.21.15.39:0/2495257890 >> v1:172.21.15.39:6789/0 conn(0x7fc1b00c1e80 legacy=0x7fc1b00c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b57b0640 1 -- 172.21.15.39:0/2495257890 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b0154560 con 0x7fc1b0153df0 2024-09-13T16:48:05.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.536+0000 7fc1b4faf640 1 -- 172.21.15.39:0/2495257890 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230887499 0 0) 0x7fc1b00c2260 con 0x7fc1b00c1e80 2024-09-13T16:48:05.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 --2- 172.21.15.39:0/2495257890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0153df0 0x7fc1b00a5110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009870 tx=0x7fc1a001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0c2586fcb843a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b4faf640 1 -- 172.21.15.39:0/2495257890 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00a95f0 con 0x7fc1b0153df0 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b4faf640 1 -- 172.21.15.39:0/2495257890 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a00a9750 con 0x7fc1b0153df0 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b4faf640 1 -- 172.21.15.39:0/2495257890 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00a9ae0 con 0x7fc1b0153df0 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- 172.21.15.39:0/2495257890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0153df0 msgr2=0x7fc1b00a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 --2- 172.21.15.39:0/2495257890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0153df0 0x7fc1b00a5110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009870 tx=0x7fc1a001c530 comp rx=0 tx=0).stop 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- 172.21.15.39:0/2495257890 shutdown_connections 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 --2- 172.21.15.39:0/2495257890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1b0157740 0x7fc1b0157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 --2- 172.21.15.39:0/2495257890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0153df0 0x7fc1b00a5110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- 172.21.15.39:0/2495257890 >> 172.21.15.39:0/2495257890 conn(0x7fc1b00a7790 msgr2=0x7fc1b00a7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- 172.21.15.39:0/2495257890 shutdown_connections 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- 172.21.15.39:0/2495257890 wait complete. 2024-09-13T16:48:05.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 Processor -- start 2024-09-13T16:48:05.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- start start 2024-09-13T16:48:05.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1b00c1e80 0x7fc1b01522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0157740 0x7fc1b00a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1b0156df0 0x7fc1b0144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0145590 con 0x7fc1b0157740 2024-09-13T16:48:05.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b006cc20 con 0x7fc1b0156df0 2024-09-13T16:48:05.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b006cf00 con 0x7fc1b00c1e80 2024-09-13T16:48:05.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0157740 0x7fc1b00a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0157740 0x7fc1b00a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41210/0 (socket says 172.21.15.39:41210) 2024-09-13T16:48:05.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 -- 172.21.15.39:0/1334640126 learned_addr learned my addr 172.21.15.39:0/1334640126 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:05.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 -- 172.21.15.39:0/1334640126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1b00c1e80 msgr2=0x7fc1b01522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1b00c1e80 0x7fc1b01522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 -- 172.21.15.39:0/1334640126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1b0156df0 msgr2=0x7fc1b0144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1b0156df0 0x7fc1b0144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 -- 172.21.15.39:0/1334640126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00094d0 con 0x7fc1b0157740 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b57b0640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0157740 0x7fc1b00a6dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1a001abe0 tx=0x7fc1a00192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1a67fc640 1 -- 172.21.15.39:0/1334640126 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00b1800 con 0x7fc1b0157740 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1a67fc640 1 -- 172.21.15.39:0/1334640126 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a00b1960 con 0x7fc1b0157740 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b00a68a0 con 0x7fc1b0157740 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1a67fc640 1 -- 172.21.15.39:0/1334640126 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00b1d10 con 0x7fc1b0157740 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.540+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b01457e0 con 0x7fc1b0157740 2024-09-13T16:48:05.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.544+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b00a6a90 con 0x7fc1b0157740 2024-09-13T16:48:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.544+0000 7fc1a67fc640 1 -- 172.21.15.39:0/1334640126 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc1a00c43f0 con 0x7fc1b0157740 2024-09-13T16:48:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.544+0000 7fc1a67fc640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc194066690 0x7fc194068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.544+0000 7fc1a67fc640 1 -- 172.21.15.39:0/1334640126 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7fc1a011ecc0 con 0x7fc1b0157740 2024-09-13T16:48:05.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.544+0000 7fc1b5fb1640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc194066690 0x7fc194068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.544+0000 7fc1b5fb1640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc194066690 0x7fc194068b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80015a0 tx=0x7fc1a8007970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:05.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.548+0000 7fc1a67fc640 1 -- 172.21.15.39:0/1334640126 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc1a00ec040 con 0x7fc1b0157740 2024-09-13T16:48:05.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.704+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc1b00a5110 con 0x7fc1b0157740 2024-09-13T16:48:05.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.704+0000 7fc1a67fc640 1 -- 172.21.15.39:0/1334640126 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+197 (secure 0 0 0) 0x7fc1a00ef660 con 0x7fc1b0157740 2024-09-13T16:48:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:05.709 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-13T16:48:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-13T16:48:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-13T16:48:05.709 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 2, 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "a_metadata" 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 3, 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "a_data" 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-13T16:48:05.710 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-13T16:48:05.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc194066690 msgr2=0x7fc194068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc194066690 0x7fc194068b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80015a0 tx=0x7fc1a8007970 comp rx=0 tx=0).stop 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0157740 msgr2=0x7fc1b00a6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0157740 0x7fc1b00a6dc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1a001abe0 tx=0x7fc1a00192d0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 shutdown_connections 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc194066690 0x7fc194068b50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1b0156df0 0x7fc1b0144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1b0157740 0x7fc1b00a6dc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 --2- 172.21.15.39:0/1334640126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1b00c1e80 0x7fc1b01522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 >> 172.21.15.39:0/1334640126 conn(0x7fc1b00a7790 msgr2=0x7fc1b00a7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 shutdown_connections 2024-09-13T16:48:05.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.708+0000 7fc1b8215640 1 -- 172.21.15.39:0/1334640126 wait complete. 2024-09-13T16:48:05.727 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'b' 2024-09-13T16:48:05.727 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create b_metadata --pg_num_min 64 2024-09-13T16:48:05.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f788f70e640 1 Processor -- start 2024-09-13T16:48:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f788f70e640 1 -- start start 2024-09-13T16:48:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f788f70e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7888151cf0 0x7f7888152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f788f70e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7888155d20 con 0x7f78880aa2d0 2024-09-13T16:48:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f788f70e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78880c4750 con 0x7f78880a4d60 2024-09-13T16:48:05.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f788f70e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7888155b30 con 0x7f7888151cf0 2024-09-13T16:48:05.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f788d4aa640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f78880aa2d0 0x7f78880aa6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38700/0 (socket says 172.21.15.39:38700) 2024-09-13T16:48:05.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f788d4aa640 1 -- 172.21.15.39:0/201544805 learned_addr learned my addr 172.21.15.39:0/201544805 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:05.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f788dcab640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7888151cf0 0x7f7888152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113677901 0 0) 0x7f7888155d20 con 0x7f78880aa2d0 2024-09-13T16:48:05.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7874003530 con 0x7f78880aa2d0 2024-09-13T16:48:05.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1702866045 0 0) 0x7f7874003530 con 0x7f78880aa2d0 2024-09-13T16:48:05.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7888151cf0 msgr2=0x7f7888152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:05.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7888151cf0 0x7f7888152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 >> v1:172.21.15.84:6789/0 conn(0x7f78880a4d60 legacy=0x7f788814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7888155fc0 con 0x7f78880aa2d0 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f7884001680 con 0x7f78880aa2d0 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 >> v1:172.21.15.39:6789/0 conn(0x7f78880aa2d0 legacy=0x7f78880aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740039b0 0x7f7874003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78740047d0 0x7f7874004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7888151cf0 0x7f78740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787400bac0 con 0x7f78740039b0 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787400bc30 con 0x7f78740047d0 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.848+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787400bdf0 con 0x7f7888151cf0 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f788d4aa640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740039b0 0x7f7874003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f788dcab640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7888151cf0 0x7f78740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f788cca9640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78740047d0 0x7f7874004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f788d4aa640 1 -- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7888151cf0 msgr2=0x7f78740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:05.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f788d4aa640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7888151cf0 0x7f78740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f788d4aa640 1 -- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78740047d0 msgr2=0x7f7874004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:05.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f788d4aa640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78740047d0 0x7f7874004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f788d4aa640 1 -- 172.21.15.39:0/201544805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7874003530 con 0x7f78740039b0 2024-09-13T16:48:05.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f788d4aa640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740039b0 0x7f7874003dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f788400c1c0 tx=0x7f788401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52798ed58b3be569 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:05.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884005ae0 con 0x7f78740039b0 2024-09-13T16:48:05.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7884005c40 con 0x7f78740039b0 2024-09-13T16:48:05.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.852+0000 7f787ffff640 1 -- 172.21.15.39:0/201544805 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884007070 con 0x7f78740039b0 2024-09-13T16:48:05.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 -- 172.21.15.39:0/201544805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740039b0 msgr2=0x7f7874003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 --2- 172.21.15.39:0/201544805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740039b0 0x7f7874003dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f788400c1c0 tx=0x7f788401cc40 comp rx=0 tx=0).stop 2024-09-13T16:48:05.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788dcab640 1 -- 172.21.15.39:0/201544805 reap_dead start 2024-09-13T16:48:05.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 -- 172.21.15.39:0/201544805 shutdown_connections 2024-09-13T16:48:05.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 -- 172.21.15.39:0/201544805 >> 172.21.15.39:0/201544805 conn(0x7f78880c5790 msgr2=0x7f78880c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:05.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 -- 172.21.15.39:0/201544805 shutdown_connections 2024-09-13T16:48:05.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 -- 172.21.15.39:0/201544805 wait complete. 2024-09-13T16:48:05.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 Processor -- start 2024-09-13T16:48:05.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 -- start start 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78740039b0 0x7f7888148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7888150c30 0x7f78880a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78880a7c40 0x7f78881ed900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880c2c30 con 0x7f7888150c30 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78881ed470 con 0x7f78880a7c40 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788f70e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78880c2390 con 0x7f78740039b0 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788cca9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7888150c30 0x7f78880a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788cca9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7888150c30 0x7f78880a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41234/0 (socket says 172.21.15.39:41234) 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788cca9640 1 -- 172.21.15.39:0/1209365776 learned_addr learned my addr 172.21.15.39:0/1209365776 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:05.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788d4aa640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78740039b0 0x7f7888148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788dcab640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78880a7c40 0x7f78881ed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788cca9640 1 -- 172.21.15.39:0/1209365776 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78740039b0 msgr2=0x7f7888148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788cca9640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78740039b0 0x7f7888148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788cca9640 1 -- 172.21.15.39:0/1209365776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78880a7c40 msgr2=0x7f78881ed900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788cca9640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78880a7c40 0x7f78881ed900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788cca9640 1 -- 172.21.15.39:0/1209365776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880c2670 con 0x7f7888150c30 2024-09-13T16:48:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f788cca9640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7888150c30 0x7f78880a5250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f787800e4d0 tx=0x7f787800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:05.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.856+0000 7f787dffb640 1 -- 172.21.15.39:0/1209365776 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78780052e0 con 0x7f7888150c30 2024-09-13T16:48:05.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f787dffb640 1 -- 172.21.15.39:0/1209365776 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7878005440 con 0x7f7888150c30 2024-09-13T16:48:05.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f787dffb640 1 -- 172.21.15.39:0/1209365776 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878003820 con 0x7f7888150c30 2024-09-13T16:48:05.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78880c2950 con 0x7f7888150c30 2024-09-13T16:48:05.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7888147040 con 0x7f7888150c30 2024-09-13T16:48:05.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788814ef70 con 0x7f7888150c30 2024-09-13T16:48:05.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f787dffb640 1 -- 172.21.15.39:0/1209365776 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7878003980 con 0x7f7888150c30 2024-09-13T16:48:05.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f787dffb640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f786c0666c0 0x7f786c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:05.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f787dffb640 1 -- 172.21.15.39:0/1209365776 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5689+0+0 (secure 0 0 0) 0x7f7878083e60 con 0x7f7888150c30 2024-09-13T16:48:05.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f788d4aa640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f786c0666c0 0x7f786c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:05.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.860+0000 7f788d4aa640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f786c0666c0 0x7f786c068b80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78840ab000 tx=0x7f78840a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:05.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:05.864+0000 7f787dffb640 1 -- 172.21.15.39:0/1209365776 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f78780511f0 con 0x7f7888150c30 2024-09-13T16:48:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.020+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f78880a8820 con 0x7f7888150c30 2024-09-13T16:48:06.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.728+0000 7f787dffb640 1 -- 172.21.15.39:0/1209365776 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b_metadata", "pg_num_min": 64}]=0 pool 'b_metadata' created v21) v1 ==== 128+0+0 (secure 0 0 0) 0x7f7878054810 con 0x7f7888150c30 2024-09-13T16:48:06.733 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'b_metadata' created 2024-09-13T16:48:06.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f786c0666c0 msgr2=0x7f786c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:06.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f786c0666c0 0x7f786c068b80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78840ab000 tx=0x7f78840a7070 comp rx=0 tx=0).stop 2024-09-13T16:48:06.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7888150c30 msgr2=0x7f78880a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:06.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7888150c30 0x7f78880a5250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f787800e4d0 tx=0x7f787800e990 comp rx=0 tx=0).stop 2024-09-13T16:48:06.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 shutdown_connections 2024-09-13T16:48:06.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f786c0666c0 0x7f786c068b80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78880a7c40 0x7f78881ed900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7888150c30 0x7f78880a5250 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 --2- 172.21.15.39:0/1209365776 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78740039b0 0x7f7888148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 >> 172.21.15.39:0/1209365776 conn(0x7f78880c5790 msgr2=0x7f7888150400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:06.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 shutdown_connections 2024-09-13T16:48:06.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.736+0000 7f788f70e640 1 -- 172.21.15.39:0/1209365776 wait complete. 2024-09-13T16:48:06.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create b_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T16:48:06.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f8661cb2640 1 Processor -- start 2024-09-13T16:48:06.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f8661cb2640 1 -- start start 2024-09-13T16:48:06.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f8661cb2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c0bcf90 0x7f865c154240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f8661cb2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f865c157370 0x7f865c154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f8661cb2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c155270 con 0x7f865c1514d0 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f8661cb2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c153d80 con 0x7f865c157370 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f8661cb2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c157870 con 0x7f865c0bcf90 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f865bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f865c157370 0x7f865c154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f865bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f865c157370 0x7f865c154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41248/0 (socket says 172.21.15.39:41248) 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f865bfff640 1 -- 172.21.15.39:0/3605638451 learned_addr learned my addr 172.21.15.39:0/3605638451 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f865affd640 1 --2- 172.21.15.39:0/3605638451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c0bcf90 0x7f865c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f865bfff640 1 -- 172.21.15.39:0/3605638451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c0bcf90 msgr2=0x7f865c154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f865bfff640 1 --2- 172.21.15.39:0/3605638451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c0bcf90 0x7f865c154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f865bfff640 1 -- 172.21.15.39:0/3605638451 >> v1:172.21.15.84:6789/0 conn(0x7f865c1514d0 legacy=0x7f865c0c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.888+0000 7f865bfff640 1 -- 172.21.15.39:0/3605638451 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c157b50 con 0x7f865c157370 2024-09-13T16:48:06.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f865bfff640 1 --2- 172.21.15.39:0/3605638451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 0x7f865c154780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8654009870 tx=0x7f865401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84f2a7929a0ebcf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:06.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f865a7fc640 1 -- 172.21.15.39:0/3605638451 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540ac620 con 0x7f865c157370 2024-09-13T16:48:06.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f865a7fc640 1 -- 172.21.15.39:0/3605638451 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86540ac780 con 0x7f865c157370 2024-09-13T16:48:06.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f865a7fc640 1 -- 172.21.15.39:0/3605638451 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540acb10 con 0x7f865c157370 2024-09-13T16:48:06.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 -- 172.21.15.39:0/3605638451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 msgr2=0x7f865c154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:06.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 --2- 172.21.15.39:0/3605638451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 0x7f865c154780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8654009870 tx=0x7f865401c4d0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 -- 172.21.15.39:0/3605638451 shutdown_connections 2024-09-13T16:48:06.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 --2- 172.21.15.39:0/3605638451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 0x7f865c154780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 --2- 172.21.15.39:0/3605638451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c0bcf90 0x7f865c154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 -- 172.21.15.39:0/3605638451 >> 172.21.15.39:0/3605638451 conn(0x7f865c06a210 msgr2=0x7f865c06a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:06.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 -- 172.21.15.39:0/3605638451 shutdown_connections 2024-09-13T16:48:06.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 -- 172.21.15.39:0/3605638451 wait complete. 2024-09-13T16:48:06.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 Processor -- start 2024-09-13T16:48:06.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 -- start start 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f865c0bcf90 0x7f865c1480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c1514d0 0x7f865c14c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 0x7f865c0a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c150510 con 0x7f865c157370 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c148aa0 con 0x7f865c0bcf90 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f8661cb2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c1487c0 con 0x7f865c1514d0 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.892+0000 7f865bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 0x7f865c0a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 0x7f865c0a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41264/0 (socket says 172.21.15.39:41264) 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865bfff640 1 -- 172.21.15.39:0/2831866727 learned_addr learned my addr 172.21.15.39:0/2831866727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865b7fe640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f865c0bcf90 0x7f865c1480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865affd640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c1514d0 0x7f865c14c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:06.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865bfff640 1 -- 172.21.15.39:0/2831866727 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c1514d0 msgr2=0x7f865c14c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865bfff640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c1514d0 0x7f865c14c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865bfff640 1 -- 172.21.15.39:0/2831866727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f865c0bcf90 msgr2=0x7f865c1480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865bfff640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f865c0bcf90 0x7f865c1480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865bfff640 1 -- 172.21.15.39:0/2831866727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540094d0 con 0x7f865c157370 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865affd640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c1514d0 0x7f865c14c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865bfff640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 0x7f865c0a9910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8654009870 tx=0x7f8654019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f8643fff640 1 -- 172.21.15.39:0/2831866727 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540b5440 con 0x7f865c157370 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f8643fff640 1 -- 172.21.15.39:0/2831866727 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86540b55a0 con 0x7f865c157370 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c1efbd0 con 0x7f865c157370 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f8643fff640 1 -- 172.21.15.39:0/2831866727 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540b5910 con 0x7f865c157370 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c1f0190 con 0x7f865c157370 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c1f0380 con 0x7f865c157370 2024-09-13T16:48:06.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f8643fff640 1 -- 172.21.15.39:0/2831866727 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f86540b3070 con 0x7f865c157370 2024-09-13T16:48:06.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f8643fff640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8648066690 0x7f8648068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:06.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f8643fff640 1 -- 172.21.15.39:0/2831866727 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6054+0+0 (secure 0 0 0) 0x7f865411e2f0 con 0x7f865c157370 2024-09-13T16:48:06.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865b7fe640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8648066690 0x7f8648068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:06.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.896+0000 7f865b7fe640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8648066690 0x7f8648068b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f864c000f70 tx=0x7f864c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:06.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:06.900+0000 7f8643fff640 1 -- 172.21.15.39:0/2831866727 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f86540eb510 con 0x7f865c157370 2024-09-13T16:48:07.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.068+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f865c0c41f0 con 0x7f865c157370 2024-09-13T16:48:07.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.748+0000 7f8643fff640 1 -- 172.21.15.39:0/2831866727 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'b_data' created v22) v1 ==== 160+0+0 (secure 0 0 0) 0x7f86540eeb30 con 0x7f865c157370 2024-09-13T16:48:07.758 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'b_data' created 2024-09-13T16:48:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8648066690 msgr2=0x7f8648068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8648066690 0x7f8648068b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f864c000f70 tx=0x7f864c009040 comp rx=0 tx=0).stop 2024-09-13T16:48:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 msgr2=0x7f865c0a9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 0x7f865c0a9910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8654009870 tx=0x7f8654019270 comp rx=0 tx=0).stop 2024-09-13T16:48:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 shutdown_connections 2024-09-13T16:48:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8648066690 0x7f8648068b50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865c157370 0x7f865c0a9910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f865c1514d0 0x7f865c14c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 --2- 172.21.15.39:0/2831866727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f865c0bcf90 0x7f865c1480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.764+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 >> 172.21.15.39:0/2831866727 conn(0x7f865c06a210 msgr2=0x7f865c152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.768+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 shutdown_connections 2024-09-13T16:48:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.768+0000 7f8661cb2640 1 -- 172.21.15.39:0/2831866727 wait complete. 2024-09-13T16:48:07.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new b b_metadata b_data 2024-09-13T16:48:07.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.916+0000 7fe6034f6640 1 Processor -- start 2024-09-13T16:48:07.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe6034f6640 1 -- start start 2024-09-13T16:48:07.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe6034f6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5fc0aa2d0 0x7fe5fc0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:07.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe6034f6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0a4d60 0x7fe5fc14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:07.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe6034f6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5fc155d20 con 0x7fe5fc14f4b0 2024-09-13T16:48:07.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe6034f6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5fc155b30 con 0x7fe5fc0aa2d0 2024-09-13T16:48:07.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe6034f6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0c4750 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe600a91640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0a4d60 0x7fe5fc14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe601292640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe5fc0aa2d0 0x7fe5fc0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe601a93640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe5fc14f4b0 0x7fe5fc151cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:37912/0 (socket says 172.21.15.39:37912) 2024-09-13T16:48:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe601a93640 1 -- 172.21.15.39:0/2293641892 learned_addr learned my addr 172.21.15.39:0/2293641892 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe5f3fff640 1 -- 172.21.15.39:0/2293641892 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846228743 0 0) 0x7fe5fc155d20 con 0x7fe5fc14f4b0 2024-09-13T16:48:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe5f3fff640 1 -- 172.21.15.39:0/2293641892 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e8003530 con 0x7fe5fc14f4b0 2024-09-13T16:48:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe600a91640 1 -- 172.21.15.39:0/2293641892 >> v2:172.21.15.84:3300/0 conn(0x7fe5fc0aa2d0 msgr2=0x7fe5fc0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe600a91640 1 --2- 172.21.15.39:0/2293641892 >> v2:172.21.15.84:3300/0 conn(0x7fe5fc0aa2d0 0x7fe5fc0aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe600a91640 1 -- 172.21.15.39:0/2293641892 >> v1:172.21.15.84:6789/0 conn(0x7fe5fc14f4b0 legacy=0x7fe5fc151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe600a91640 1 -- 172.21.15.39:0/2293641892 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc155fc0 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.920+0000 7fe601292640 1 --2- 172.21.15.39:0/2293641892 >> v2:172.21.15.84:3300/0 conn(0x7fe5fc0aa2d0 0x7fe5fc0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:07.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe600a91640 1 --2- 172.21.15.39:0/2293641892 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0a4d60 0x7fe5fc14ef70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80097c0 tx=0x7fe5f801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9d37cf550163935 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:07.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe5f3fff640 1 -- 172.21.15.39:0/2293641892 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80ab600 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe5f3fff640 1 -- 172.21.15.39:0/2293641892 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f80a7050 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe5f3fff640 1 -- 172.21.15.39:0/2293641892 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80ab960 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 -- 172.21.15.39:0/2293641892 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0a4d60 msgr2=0x7fe5fc14ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:07.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 --2- 172.21.15.39:0/2293641892 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0a4d60 0x7fe5fc14ef70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80097c0 tx=0x7fe5f801c3e0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 -- 172.21.15.39:0/2293641892 shutdown_connections 2024-09-13T16:48:07.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 --2- 172.21.15.39:0/2293641892 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0a4d60 0x7fe5fc14ef70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 --2- 172.21.15.39:0/2293641892 >> v2:172.21.15.84:3300/0 conn(0x7fe5fc0aa2d0 0x7fe5fc0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 -- 172.21.15.39:0/2293641892 >> 172.21.15.39:0/2293641892 conn(0x7fe5fc0c5790 msgr2=0x7fe5fc0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:07.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 -- 172.21.15.39:0/2293641892 shutdown_connections 2024-09-13T16:48:07.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 -- 172.21.15.39:0/2293641892 wait complete. 2024-09-13T16:48:07.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 Processor -- start 2024-09-13T16:48:07.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.924+0000 7fe6034f6640 1 -- start start 2024-09-13T16:48:07.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe6034f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5fc0a4d60 0x7fe5fc145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:07.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe6034f6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0aa2d0 0x7fe5fc0bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:07.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe6034f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5fc14f4b0 0x7fe5fc06c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:07.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe6034f6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc151710 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe6034f6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0c2750 con 0x7fe5fc14f4b0 2024-09-13T16:48:07.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe6034f6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc0c2d10 con 0x7fe5fc0aa2d0 2024-09-13T16:48:07.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601292640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5fc0a4d60 0x7fe5fc145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601292640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5fc0a4d60 0x7fe5fc145cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41276/0 (socket says 172.21.15.39:41276) 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601292640 1 -- 172.21.15.39:0/2613611246 learned_addr learned my addr 172.21.15.39:0/2613611246 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe600a91640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0aa2d0 0x7fe5fc0bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601a93640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5fc14f4b0 0x7fe5fc06c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601292640 1 -- 172.21.15.39:0/2613611246 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0aa2d0 msgr2=0x7fe5fc0bebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601292640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0aa2d0 0x7fe5fc0bebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601292640 1 -- 172.21.15.39:0/2613611246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5fc14f4b0 msgr2=0x7fe5fc06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601292640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5fc14f4b0 0x7fe5fc06c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601292640 1 -- 172.21.15.39:0/2613611246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec009420 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe601292640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5fc0a4d60 0x7fe5fc145cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00ebe0 tx=0x7fe5ec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe5f1ffb640 1 -- 172.21.15.39:0/2613611246 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00eee0 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe5f1ffb640 1 -- 172.21.15.39:0/2613611246 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec00cc30 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f8009420 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe5f1ffb640 1 -- 172.21.15.39:0/2613611246 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec013470 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5fc1ed050 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5fc14ef70 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe5f1ffb640 1 -- 172.21.15.39:0/2613611246 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe5ec0135d0 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe5f1ffb640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5e00666c0 0x7fe5e0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:07.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe5f1ffb640 1 -- 172.21.15.39:0/2613611246 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6431+0+0 (secure 0 0 0) 0x7fe5ec0846e0 con 0x7fe5fc0a4d60 2024-09-13T16:48:07.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.928+0000 7fe600a91640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5e00666c0 0x7fe5e0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:07.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.932+0000 7fe600a91640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5e00666c0 0x7fe5e0068b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80ad000 tx=0x7fe5f80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:07.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:07.936+0000 7fe5f1ffb640 1 -- 172.21.15.39:0/2613611246 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe5ec051790 con 0x7fe5fc0a4d60 2024-09-13T16:48:08.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.128+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "b", "metadata": "b_metadata", "data": "b_data"} v 0) v1 -- 0x7fe5fc0a87f0 con 0x7fe5fc0a4d60 2024-09-13T16:48:08.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.140+0000 7fe5f1ffb640 1 -- 172.21.15.39:0/2613611246 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "b", "metadata": "b_metadata", "data": "b_data"}]=0 new fs with metadata pool 4 and data pool 5 v9) v1 ==== 157+0+0 (secure 0 0 0) 0x7fe5ec054db0 con 0x7fe5fc0a4d60 2024-09-13T16:48:08.146 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 4 and data pool 5 2024-09-13T16:48:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5e00666c0 msgr2=0x7fe5e0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5e00666c0 0x7fe5e0068b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80ad000 tx=0x7fe5f80a6000 comp rx=0 tx=0).stop 2024-09-13T16:48:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5fc0a4d60 msgr2=0x7fe5fc145cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5fc0a4d60 0x7fe5fc145cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00ebe0 tx=0x7fe5ec00c520 comp rx=0 tx=0).stop 2024-09-13T16:48:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 shutdown_connections 2024-09-13T16:48:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5e00666c0 0x7fe5e0068b80 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80ad000 tx=0x7fe5f80a6000 comp rx=0 tx=0).stop 2024-09-13T16:48:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5fc14f4b0 0x7fe5fc06c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5fc0aa2d0 0x7fe5fc0bebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 --2- 172.21.15.39:0/2613611246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5fc0a4d60 0x7fe5fc145cd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 >> 172.21.15.39:0/2613611246 conn(0x7fe5fc0c5790 msgr2=0x7fe5fc14a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 shutdown_connections 2024-09-13T16:48:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.148+0000 7fe6034f6640 1 -- 172.21.15.39:0/2613611246 wait complete. 2024-09-13T16:48:08.180 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f7800f9f640 1 Processor -- start 2024-09-13T16:48:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f7800f9f640 1 -- start start 2024-09-13T16:48:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f7800f9f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0c20d0 0x7f77fc0c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f7800f9f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77fc157370 con 0x7f77fc06d370 2024-09-13T16:48:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f7800f9f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77fc153df0 con 0x7f77fc0c1790 2024-09-13T16:48:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f7800f9f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0c04a0 con 0x7f77fc0c20d0 2024-09-13T16:48:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fad9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f77fc06d370 0x7f77fc15e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:38716/0 (socket says 172.21.15.39:38716) 2024-09-13T16:48:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fad9d640 1 -- 172.21.15.39:0/828647477 learned_addr learned my addr 172.21.15.39:0/828647477 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fa59c640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0c20d0 0x7f77fc0c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444398606 0 0) 0x7f77fc157370 con 0x7f77fc06d370 2024-09-13T16:48:08.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77e4003530 con 0x7f77fc06d370 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765334304 0 0) 0x7f77fc153df0 con 0x7f77fc0c1790 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77fc157370 con 0x7f77fc0c1790 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2370044526 0 0) 0x7f77e4003530 con 0x7f77fc06d370 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0c20d0 msgr2=0x7f77fc0c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0c20d0 0x7f77fc0c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 >> v1:172.21.15.84:6789/0 conn(0x7f77fc0c1790 legacy=0x7f77fc157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc06c700 con 0x7f77fc06d370 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f77f4001a00 con 0x7f77fc06d370 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 >> v1:172.21.15.39:6789/0 conn(0x7f77fc06d370 legacy=0x7f77fc15e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0c20d0 0x7f77e4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77e40040f0 0x7f77e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77e4004d10 0x7f77e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e400baa0 con 0x7f77e4004d10 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e400bc10 con 0x7f77e40040f0 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e400bdd0 con 0x7f77fc0c20d0 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fad9d640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77e4004d10 0x7f77e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fa59c640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0c20d0 0x7f77e4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77f9d9b640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77e40040f0 0x7f77e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fad9d640 1 -- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0c20d0 msgr2=0x7f77e4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fad9d640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0c20d0 0x7f77e4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fad9d640 1 -- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77e40040f0 msgr2=0x7f77e40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fad9d640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77e40040f0 0x7f77e40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.340+0000 7f77fad9d640 1 -- 172.21.15.39:0/828647477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4003530 con 0x7f77e4004d10 2024-09-13T16:48:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f77fad9d640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77e4004d10 0x7f77e4005190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77f400a6e0 tx=0x7f77f401f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5794fbd88eb49a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40ac5f0 con 0x7f77e4004d10 2024-09-13T16:48:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f40ac750 con 0x7f77e4004d10 2024-09-13T16:48:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f77f959a640 1 -- 172.21.15.39:0/828647477 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40acb80 con 0x7f77e4004d10 2024-09-13T16:48:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f7800f9f640 1 -- 172.21.15.39:0/828647477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77e4004d10 msgr2=0x7f77e4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f7800f9f640 1 --2- 172.21.15.39:0/828647477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77e4004d10 0x7f77e4005190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77f400a6e0 tx=0x7f77f401f940 comp rx=0 tx=0).stop 2024-09-13T16:48:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f77fad9d640 1 -- 172.21.15.39:0/828647477 reap_dead start 2024-09-13T16:48:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f7800f9f640 1 -- 172.21.15.39:0/828647477 shutdown_connections 2024-09-13T16:48:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f7800f9f640 1 -- 172.21.15.39:0/828647477 >> 172.21.15.39:0/828647477 conn(0x7f77fc069e10 msgr2=0x7f77fc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f7800f9f640 1 -- 172.21.15.39:0/828647477 shutdown_connections 2024-09-13T16:48:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.344+0000 7f7800f9f640 1 -- 172.21.15.39:0/828647477 wait complete. 2024-09-13T16:48:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 Processor -- start 2024-09-13T16:48:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 -- start start 2024-09-13T16:48:08.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77fc0716a0 0x7f77fc14db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc152040 0x7f77fc14e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0a9c40 0x7f77fc0a8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc1dd3e0 con 0x7f77fc0716a0 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc1dd9a0 con 0x7f77fc152040 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0ab5c0 con 0x7f77fc0a9c40 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77fc0716a0 0x7f77fc14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77fc0716a0 0x7f77fc14db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41304/0 (socket says 172.21.15.39:41304) 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fa59c640 1 -- 172.21.15.39:0/3545737555 learned_addr learned my addr 172.21.15.39:0/3545737555 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fad9d640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0a9c40 0x7f77fc0a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77f9d9b640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc152040 0x7f77fc14e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fa59c640 1 -- 172.21.15.39:0/3545737555 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0a9c40 msgr2=0x7f77fc0a8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fa59c640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0a9c40 0x7f77fc0a8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fa59c640 1 -- 172.21.15.39:0/3545737555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc152040 msgr2=0x7f77fc14e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fa59c640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc152040 0x7f77fc14e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fa59c640 1 -- 172.21.15.39:0/3545737555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc0ab8a0 con 0x7f77fc0716a0 2024-09-13T16:48:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77f9d9b640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc152040 0x7f77fc14e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77fa59c640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77fc0716a0 0x7f77fc14db10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0119d0 tx=0x7f77ec011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77eaffd640 1 -- 172.21.15.39:0/3545737555 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec00fb50 con 0x7f77fc0716a0 2024-09-13T16:48:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77eaffd640 1 -- 172.21.15.39:0/3545737555 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec00fcb0 con 0x7f77fc0716a0 2024-09-13T16:48:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77fc1dd630 con 0x7f77fc0716a0 2024-09-13T16:48:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77eaffd640 1 -- 172.21.15.39:0/3545737555 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec014440 con 0x7f77fc0716a0 2024-09-13T16:48:08.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77fc0ab250 con 0x7f77fc0716a0 2024-09-13T16:48:08.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77fc151850 con 0x7f77fc0716a0 2024-09-13T16:48:08.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77eaffd640 1 -- 172.21.15.39:0/3545737555 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f77ec0145a0 con 0x7f77fc0716a0 2024-09-13T16:48:08.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77eaffd640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77dc0666c0 0x7f77dc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77f9d9b640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77dc0666c0 0x7f77dc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.348+0000 7f77eaffd640 1 -- 172.21.15.39:0/3545737555 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f77ec084cc0 con 0x7f77fc0716a0 2024-09-13T16:48:08.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.352+0000 7f77f9d9b640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77dc0666c0 0x7f77dc068b80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f77fc157230 tx=0x7f77f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.356+0000 7f77eaffd640 1 -- 172.21.15.39:0/3545737555 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f77ec051e50 con 0x7f77fc0716a0 2024-09-13T16:48:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.516+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77fc072570 con 0x7f77fc0716a0 2024-09-13T16:48:08.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.516+0000 7f77eaffd640 1 -- 172.21.15.39:0/3545737555 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+20142 (secure 0 0 0) 0x7f77ec055470 con 0x7f77fc0716a0 2024-09-13T16:48:08.521 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:08.521 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":23,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:08.134987+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77dc0666c0 msgr2=0x7f77dc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77dc0666c0 0x7f77dc068b80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f77fc157230 tx=0x7f77f0009040 comp rx=0 tx=0).stop 2024-09-13T16:48:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77fc0716a0 msgr2=0x7f77fc14db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77fc0716a0 0x7f77fc14db10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0119d0 tx=0x7f77ec011e90 comp rx=0 tx=0).stop 2024-09-13T16:48:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 shutdown_connections 2024-09-13T16:48:08.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77dc0666c0 0x7f77dc068b80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77fc152040 0x7f77fc14e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f77fc0a9c40 0x7f77fc0a8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 --2- 172.21.15.39:0/3545737555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f77fc0716a0 0x7f77fc14db10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 >> 172.21.15.39:0/3545737555 conn(0x7f77fc069e10 msgr2=0x7f77fc157700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 shutdown_connections 2024-09-13T16:48:08.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.520+0000 7f7800f9f640 1 -- 172.21.15.39:0/3545737555 wait complete. 2024-09-13T16:48:08.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:08.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.660+0000 7f2fddfee640 1 Processor -- start 2024-09-13T16:48:08.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.660+0000 7f2fddfee640 1 -- start start 2024-09-13T16:48:08.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fddfee640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 0x7f2fd80c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fddfee640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fd8071dd0 0x7f2fd8154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fddfee640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd80c5a30 con 0x7f2fd80c47c0 2024-09-13T16:48:08.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fddfee640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd81576a0 con 0x7f2fd8071dd0 2024-09-13T16:48:08.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fddfee640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8157980 con 0x7f2fd8151500 2024-09-13T16:48:08.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 0x7f2fd80c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fd6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fd8071dd0 0x7f2fd8154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fd6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fd8071dd0 0x7f2fd8154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:56350/0 (socket says 172.21.15.39:56350) 2024-09-13T16:48:08.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fd6ffd640 1 -- 172.21.15.39:0/3662262565 learned_addr learned my addr 172.21.15.39:0/3662262565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:08.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fd6ffd640 1 -- 172.21.15.39:0/3662262565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 msgr2=0x7f2fd80c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:08.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fd6ffd640 1 --2- 172.21.15.39:0/3662262565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 0x7f2fd80c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fd6ffd640 1 -- 172.21.15.39:0/3662262565 >> v1:172.21.15.84:6789/0 conn(0x7f2fd80c47c0 legacy=0x7f2fd81545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.664+0000 7f2fd6ffd640 1 -- 172.21.15.39:0/3662262565 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd8157c60 con 0x7f2fd8071dd0 2024-09-13T16:48:08.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fd6ffd640 1 --2- 172.21.15.39:0/3662262565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8071dd0 0x7f2fd8154070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009870 tx=0x7f2fcc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b26ad6dc3ed7b571 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:08.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fd67fc640 1 -- 172.21.15.39:0/3662262565 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0aa620 con 0x7f2fd8071dd0 2024-09-13T16:48:08.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fd67fc640 1 -- 172.21.15.39:0/3662262565 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc0aa780 con 0x7f2fd8071dd0 2024-09-13T16:48:08.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fd67fc640 1 -- 172.21.15.39:0/3662262565 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0aab10 con 0x7f2fd8071dd0 2024-09-13T16:48:08.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 -- 172.21.15.39:0/3662262565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8071dd0 msgr2=0x7f2fd8154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 --2- 172.21.15.39:0/3662262565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8071dd0 0x7f2fd8154070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009870 tx=0x7f2fcc01c4a0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 -- 172.21.15.39:0/3662262565 shutdown_connections 2024-09-13T16:48:08.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 --2- 172.21.15.39:0/3662262565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd8071dd0 0x7f2fd8154070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 --2- 172.21.15.39:0/3662262565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 0x7f2fd80c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 -- 172.21.15.39:0/3662262565 >> 172.21.15.39:0/3662262565 conn(0x7f2fd80ab530 msgr2=0x7f2fd80ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:08.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 -- 172.21.15.39:0/3662262565 shutdown_connections 2024-09-13T16:48:08.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 -- 172.21.15.39:0/3662262565 wait complete. 2024-09-13T16:48:08.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 Processor -- start 2024-09-13T16:48:08.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 -- start start 2024-09-13T16:48:08.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd8071dd0 0x7f2fd8147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd80c47c0 0x7f2fd814ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 0x7f2fd814fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd814a9d0 con 0x7f2fd8071dd0 2024-09-13T16:48:08.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd814b630 con 0x7f2fd80c47c0 2024-09-13T16:48:08.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fddfee640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd814bc40 con 0x7f2fd8151500 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd8071dd0 0x7f2fd8147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd8071dd0 0x7f2fd8147620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41310/0 (socket says 172.21.15.39:41310) 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fd77fe640 1 -- 172.21.15.39:0/740433555 learned_addr learned my addr 172.21.15.39:0/740433555 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fd6ffd640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd80c47c0 0x7f2fd814ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.668+0000 7f2fd7fff640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 0x7f2fd814fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fd77fe640 1 -- 172.21.15.39:0/740433555 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 msgr2=0x7f2fd814fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fd77fe640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 0x7f2fd814fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fd77fe640 1 -- 172.21.15.39:0/740433555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd80c47c0 msgr2=0x7f2fd814ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fd77fe640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd80c47c0 0x7f2fd814ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fd77fe640 1 -- 172.21.15.39:0/740433555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc0094d0 con 0x7f2fd8071dd0 2024-09-13T16:48:08.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fd77fe640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd8071dd0 0x7f2fd8147620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800e870 tx=0x7f2fc800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:08.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fbffff640 1 -- 172.21.15.39:0/740433555 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8009510 con 0x7f2fd8071dd0 2024-09-13T16:48:08.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fbffff640 1 -- 172.21.15.39:0/740433555 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc800eec0 con 0x7f2fd8071dd0 2024-09-13T16:48:08.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fbffff640 1 -- 172.21.15.39:0/740433555 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800cb40 con 0x7f2fd8071dd0 2024-09-13T16:48:08.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd814bf20 con 0x7f2fd8071dd0 2024-09-13T16:48:08.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd814b3a0 con 0x7f2fd8071dd0 2024-09-13T16:48:08.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd80c4280 con 0x7f2fd8071dd0 2024-09-13T16:48:08.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fbffff640 1 -- 172.21.15.39:0/740433555 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2fc800cca0 con 0x7f2fd8071dd0 2024-09-13T16:48:08.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fbffff640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fc4066690 0x7f2fc4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fbffff640 1 -- 172.21.15.39:0/740433555 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f2fc8084c00 con 0x7f2fd8071dd0 2024-09-13T16:48:08.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fd6ffd640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fc4066690 0x7f2fc4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.672+0000 7f2fd6ffd640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fc4066690 0x7f2fc4068b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0ae000 tx=0x7f2fcc0b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:08.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.676+0000 7f2fbffff640 1 -- 172.21.15.39:0/740433555 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2fc8051e60 con 0x7f2fd8071dd0 2024-09-13T16:48:08.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.836+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fd8071bb0 con 0x7f2fd8071dd0 2024-09-13T16:48:08.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.840+0000 7f2fbffff640 1 -- 172.21.15.39:0/740433555 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+20142 (secure 0 0 0) 0x7f2fc8055480 con 0x7f2fd8071dd0 2024-09-13T16:48:08.845 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:08.845 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":23,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:08.134987+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:08.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.840+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fc4066690 msgr2=0x7f2fc4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.840+0000 7f2fddfee640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fc4066690 0x7f2fc4068b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0ae000 tx=0x7f2fcc0b5040 comp rx=0 tx=0).stop 2024-09-13T16:48:08.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd8071dd0 msgr2=0x7f2fd8147620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd8071dd0 0x7f2fd8147620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800e870 tx=0x7f2fc800ec70 comp rx=0 tx=0).stop 2024-09-13T16:48:08.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 shutdown_connections 2024-09-13T16:48:08.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fc4066690 0x7f2fc4068b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd80c47c0 0x7f2fd814ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd8151500 0x7f2fd814fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 --2- 172.21.15.39:0/740433555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd8071dd0 0x7f2fd8147620 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 >> 172.21.15.39:0/740433555 conn(0x7f2fd80ab530 msgr2=0x7f2fd80ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:08.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 shutdown_connections 2024-09-13T16:48:08.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.844+0000 7f2fddfee640 1 -- 172.21.15.39:0/740433555 wait complete. 2024-09-13T16:48:08.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b standby_count_wanted 0 2024-09-13T16:48:08.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5347fc9640 1 Processor -- start 2024-09-13T16:48:08.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5347fc9640 1 -- start start 2024-09-13T16:48:08.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5347fc9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f53401515b0 0x7f53400c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5347fc9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53400c0b80 0x7f53400c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:08.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5347fc9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5340158070 con 0x7f5340151990 2024-09-13T16:48:08.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5347fc9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53400c17b0 con 0x7f53401515b0 2024-09-13T16:48:08.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5347fc9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5340158350 con 0x7f53400c0b80 2024-09-13T16:48:08.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5345d65640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f53401515b0 0x7f53400c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5345d65640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f53401515b0 0x7f53400c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41320/0 (socket says 172.21.15.39:41320) 2024-09-13T16:48:08.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5345d65640 1 -- 172.21.15.39:0/1823907292 learned_addr learned my addr 172.21.15.39:0/1823907292 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:08.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5345564640 1 --2- 172.21.15.39:0/1823907292 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53400c0b80 0x7f53400c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:08.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5345d65640 1 -- 172.21.15.39:0/1823907292 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53400c0b80 msgr2=0x7f53400c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:08.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5345d65640 1 --2- 172.21.15.39:0/1823907292 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53400c0b80 0x7f53400c21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:08.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5345d65640 1 -- 172.21.15.39:0/1823907292 >> v1:172.21.15.84:6789/0 conn(0x7f5340151990 legacy=0x7f53401540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:08.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.992+0000 7f5345d65640 1 -- 172.21.15.39:0/1823907292 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53400c3690 con 0x7f53401515b0 2024-09-13T16:48:09.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5345d65640 1 --2- 172.21.15.39:0/1823907292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53401515b0 0x7f53400c51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f533c009870 tx=0x7f533c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c291f2336d2373cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:09.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5344d63640 1 -- 172.21.15.39:0/1823907292 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0ac630 con 0x7f53401515b0 2024-09-13T16:48:09.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5344d63640 1 -- 172.21.15.39:0/1823907292 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c0ac790 con 0x7f53401515b0 2024-09-13T16:48:09.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5344d63640 1 -- 172.21.15.39:0/1823907292 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0acb20 con 0x7f53401515b0 2024-09-13T16:48:09.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 -- 172.21.15.39:0/1823907292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53401515b0 msgr2=0x7f53400c51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:09.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 --2- 172.21.15.39:0/1823907292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53401515b0 0x7f53400c51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f533c009870 tx=0x7f533c01c410 comp rx=0 tx=0).stop 2024-09-13T16:48:09.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 -- 172.21.15.39:0/1823907292 shutdown_connections 2024-09-13T16:48:09.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 --2- 172.21.15.39:0/1823907292 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53400c0b80 0x7f53400c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:09.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 --2- 172.21.15.39:0/1823907292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53401515b0 0x7f53400c51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:09.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 -- 172.21.15.39:0/1823907292 >> 172.21.15.39:0/1823907292 conn(0x7f53400c5790 msgr2=0x7f53400c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:09.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 -- 172.21.15.39:0/1823907292 shutdown_connections 2024-09-13T16:48:09.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 -- 172.21.15.39:0/1823907292 wait complete. 2024-09-13T16:48:09.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 Processor -- start 2024-09-13T16:48:09.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:08.996+0000 7f5347fc9640 1 -- start start 2024-09-13T16:48:09.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5347fc9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53400c0b80 0x7f53401563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:09.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5347fc9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53401515b0 0x7f53400aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:09.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5347fc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5340151990 0x7f53401f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5347fc9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53400a94f0 con 0x7f53400c0b80 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5347fc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534014e170 con 0x7f5340151990 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5347fc9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53401efce0 con 0x7f53401515b0 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345d65640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53400c0b80 0x7f53401563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345d65640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53400c0b80 0x7f53401563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41334/0 (socket says 172.21.15.39:41334) 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345d65640 1 -- 172.21.15.39:0/1008194896 learned_addr learned my addr 172.21.15.39:0/1008194896 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345d65640 1 -- 172.21.15.39:0/1008194896 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53401515b0 msgr2=0x7f53400aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345d65640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53401515b0 0x7f53400aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345d65640 1 -- 172.21.15.39:0/1008194896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5340151990 msgr2=0x7f53401f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5346566640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5340151990 0x7f53401f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345d65640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5340151990 0x7f53401f01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345d65640 1 -- 172.21.15.39:0/1008194896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c0094d0 con 0x7f53400c0b80 2024-09-13T16:48:09.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5346566640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5340151990 0x7f53401f01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:09.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345d65640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53400c0b80 0x7f53401563f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f533c0af040 tx=0x7f533c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:09.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f53367fc640 1 -- 172.21.15.39:0/1008194896 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0b7670 con 0x7f53400c0b80 2024-09-13T16:48:09.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f53367fc640 1 -- 172.21.15.39:0/1008194896 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c0b77d0 con 0x7f53400c0b80 2024-09-13T16:48:09.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534014e380 con 0x7f53400c0b80 2024-09-13T16:48:09.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f53367fc640 1 -- 172.21.15.39:0/1008194896 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0b7b80 con 0x7f53400c0b80 2024-09-13T16:48:09.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534014e940 con 0x7f53400c0b80 2024-09-13T16:48:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53401efeb0 con 0x7f53400c0b80 2024-09-13T16:48:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f53367fc640 1 -- 172.21.15.39:0/1008194896 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f533c0b3070 con 0x7f53400c0b80 2024-09-13T16:48:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f53367fc640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f53367fc640 1 -- 172.21.15.39:0/1008194896 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f533c11eba0 con 0x7f53400c0b80 2024-09-13T16:48:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.000+0000 7f5345564640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:09.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.004+0000 7f5345564640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5330006aa0 tx=0x7f5330006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:09.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.004+0000 7f53367fc640 1 -- 172.21.15.39:0/1008194896 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f533c0ebbe0 con 0x7f53400c0b80 2024-09-13T16:48:09.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:09.208+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f53400c21e0 con 0x7f53400c0b80 2024-09-13T16:48:10.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.160+0000 7f53367fc640 1 -- 172.21.15.39:0/1008194896 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 113+0+0 (secure 0 0 0) 0x7f533c0a8020 con 0x7f53400c0b80 2024-09-13T16:48:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 msgr2=0x7f5324068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5330006aa0 tx=0x7f5330006a30 comp rx=0 tx=0).stop 2024-09-13T16:48:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53400c0b80 msgr2=0x7f53401563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53400c0b80 0x7f53401563f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f533c0af040 tx=0x7f533c0191b0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 shutdown_connections 2024-09-13T16:48:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5340151990 0x7f53401f01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53401515b0 0x7f53400aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 --2- 172.21.15.39:0/1008194896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53400c0b80 0x7f53401563f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 >> 172.21.15.39:0/1008194896 conn(0x7f53400c5790 msgr2=0x7f53400c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:10.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 shutdown_connections 2024-09-13T16:48:10.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.164+0000 7f5347fc9640 1 -- 172.21.15.39:0/1008194896 wait complete. 2024-09-13T16:48:10.190 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-09-13T16:48:10.190 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b allow_standby_replay false 2024-09-13T16:48:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.316+0000 7f6719b9c640 1 Processor -- start 2024-09-13T16:48:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.316+0000 7f6719b9c640 1 -- start start 2024-09-13T16:48:10.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6719b9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714151070 0x7f67141514c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:10.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6719b9c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67140c1d70 con 0x7f67141540e0 2024-09-13T16:48:10.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6719b9c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6714150b70 con 0x7f671414e530 2024-09-13T16:48:10.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6719b9c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6714154570 con 0x7f6714151070 2024-09-13T16:48:10.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6712ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f67141540e0 0x7f67140c47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33080/0 (socket says 172.21.15.39:33080) 2024-09-13T16:48:10.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6712ffd640 1 -- 172.21.15.39:0/1397057678 learned_addr learned my addr 172.21.15.39:0/1397057678 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:10.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6713fff640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714151070 0x7f67141514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764010180 0 0) 0x7f67140c1d70 con 0x7f67141540e0 2024-09-13T16:48:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6700005740 con 0x7f67141540e0 2024-09-13T16:48:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 425013738 0 0) 0x7f6700005740 con 0x7f67141540e0 2024-09-13T16:48:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714151070 msgr2=0x7f67141514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714151070 0x7f67141514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 >> v1:172.21.15.84:6789/0 conn(0x7f671414e530 legacy=0x7f67140c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6714154850 con 0x7f67141540e0 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f66fc001a00 con 0x7f67141540e0 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 >> v1:172.21.15.39:6789/0 conn(0x7f67141540e0 legacy=0x7f67140c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714151070 0x7f6700005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6700006140 0x7f67000069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6700006f20 0x7f67000073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6714154570 con 0x7f6700006140 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670000dd30 con 0x7f6700006f20 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670000def0 con 0x7f6714151070 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6712ffd640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6700006140 0x7f67000069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f67137fe640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714151070 0x7f6700005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6713fff640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6700006f20 0x7f67000073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:10.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6712ffd640 1 -- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714151070 msgr2=0x7f6700005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6712ffd640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714151070 0x7f6700005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6712ffd640 1 -- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6700006f20 msgr2=0x7f67000073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6712ffd640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6700006f20 0x7f67000073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6712ffd640 1 -- 172.21.15.39:0/1397057678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700005740 con 0x7f6700006140 2024-09-13T16:48:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.320+0000 7f6713fff640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6700006f20 0x7f67000073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6712ffd640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6700006140 0x7f67000069e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00a830 tx=0x7f66fc01d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af593d075a4e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0072d0 con 0x7f6700006140 2024-09-13T16:48:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc0a8070 con 0x7f6700006140 2024-09-13T16:48:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67127fc640 1 -- 172.21.15.39:0/1397057678 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0076b0 con 0x7f6700006140 2024-09-13T16:48:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 -- 172.21.15.39:0/1397057678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6700006140 msgr2=0x7f67000069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 --2- 172.21.15.39:0/1397057678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6700006140 0x7f67000069e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00a830 tx=0x7f66fc01d330 comp rx=0 tx=0).stop 2024-09-13T16:48:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6713fff640 1 -- 172.21.15.39:0/1397057678 reap_dead start 2024-09-13T16:48:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 -- 172.21.15.39:0/1397057678 shutdown_connections 2024-09-13T16:48:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 -- 172.21.15.39:0/1397057678 >> 172.21.15.39:0/1397057678 conn(0x7f67140c5790 msgr2=0x7f67140c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 -- 172.21.15.39:0/1397057678 shutdown_connections 2024-09-13T16:48:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 -- 172.21.15.39:0/1397057678 wait complete. 2024-09-13T16:48:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 Processor -- start 2024-09-13T16:48:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 -- start start 2024-09-13T16:48:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67140bee20 0x7f6714145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714152bc0 0x7f6714145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6714151cd0 0x7f6714070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671406fbf0 con 0x7f67140bee20 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6714153990 con 0x7f6714151cd0 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6719b9c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67141530f0 con 0x7f6714152bc0 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67137fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67140bee20 0x7f6714145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67137fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67140bee20 0x7f6714145940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34166/0 (socket says 172.21.15.39:34166) 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67137fe640 1 -- 172.21.15.39:0/1898695952 learned_addr learned my addr 172.21.15.39:0/1898695952 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6712ffd640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714152bc0 0x7f6714145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f6713fff640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6714151cd0 0x7f6714070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67137fe640 1 -- 172.21.15.39:0/1898695952 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714152bc0 msgr2=0x7f6714145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67137fe640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714152bc0 0x7f6714145e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67137fe640 1 -- 172.21.15.39:0/1898695952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6714151cd0 msgr2=0x7f6714070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67137fe640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6714151cd0 0x7f6714070920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.324+0000 7f67137fe640 1 -- 172.21.15.39:0/1898695952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67141533d0 con 0x7f67140bee20 2024-09-13T16:48:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f67137fe640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67140bee20 0x7f6714145940 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f670400f640 tx=0x7f670400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f670bfff640 1 -- 172.21.15.39:0/1898695952 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040052e0 con 0x7f67140bee20 2024-09-13T16:48:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f670bfff640 1 -- 172.21.15.39:0/1898695952 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6704005440 con 0x7f67140bee20 2024-09-13T16:48:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f670bfff640 1 -- 172.21.15.39:0/1898695952 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704003820 con 0x7f67140bee20 2024-09-13T16:48:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67141536b0 con 0x7f67140bee20 2024-09-13T16:48:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67141463c0 con 0x7f67140bee20 2024-09-13T16:48:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714070240 con 0x7f67140bee20 2024-09-13T16:48:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f670bfff640 1 -- 172.21.15.39:0/1898695952 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6704003980 con 0x7f67140bee20 2024-09-13T16:48:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f670bfff640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f66f4066690 0x7f66f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f670bfff640 1 -- 172.21.15.39:0/1898695952 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f6704084450 con 0x7f67140bee20 2024-09-13T16:48:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f6712ffd640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f66f4066690 0x7f66f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:10.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.328+0000 7f6712ffd640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f66f4066690 0x7f66f4068b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00a830 tx=0x7f66fc001d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.332+0000 7f670bfff640 1 -- 172.21.15.39:0/1898695952 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6704051400 con 0x7f67140bee20 2024-09-13T16:48:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:10.528+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f67140c1660 con 0x7f67140bee20 2024-09-13T16:48:11.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.160+0000 7f670bfff640 1 -- 172.21.15.39:0/1898695952 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 117+0+0 (secure 0 0 0) 0x7f6704054a20 con 0x7f67140bee20 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f66f4066690 msgr2=0x7f66f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f66f4066690 0x7f66f4068b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00a830 tx=0x7f66fc001d90 comp rx=0 tx=0).stop 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67140bee20 msgr2=0x7f6714145940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67140bee20 0x7f6714145940 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f670400f640 tx=0x7f670400fb00 comp rx=0 tx=0).stop 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 shutdown_connections 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f66f4066690 0x7f66f4068b50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6714151cd0 0x7f6714070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6714152bc0 0x7f6714145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 --2- 172.21.15.39:0/1898695952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67140bee20 0x7f6714145940 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 >> 172.21.15.39:0/1898695952 conn(0x7f67140c5790 msgr2=0x7f67140aa490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 shutdown_connections 2024-09-13T16:48:11.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.164+0000 7f6719b9c640 1 -- 172.21.15.39:0/1898695952 wait complete. 2024-09-13T16:48:11.188 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set b session_timeout 300 2024-09-13T16:48:11.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc4764640 1 Processor -- start 2024-09-13T16:48:11.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc4764640 1 -- start start 2024-09-13T16:48:11.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc4764640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8bbc14e530 0x7f8bbc0c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:11.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc4764640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc150b70 con 0x7f8bbc1540e0 2024-09-13T16:48:11.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc4764640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc154570 con 0x7f8bbc151070 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc4764640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bbc151b90 con 0x7f8bbc14e530 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc1cff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8bbc1540e0 0x7f8bbc0c47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33092/0 (socket says 172.21.15.39:33092) 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc1cff640 1 -- 172.21.15.39:0/314086638 learned_addr learned my addr 172.21.15.39:0/314086638 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2500640 1 --2- 172.21.15.39:0/314086638 >> v2:172.21.15.84:3300/0 conn(0x7f8bbc14e530 0x7f8bbc0c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167689814 0 0) 0x7f8bbc150b70 con 0x7f8bbc1540e0 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba8003530 con 0x7f8bbc1540e0 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2821545312 0 0) 0x7f8ba8003530 con 0x7f8bbc1540e0 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 >> v2:172.21.15.84:3300/0 conn(0x7f8bbc14e530 msgr2=0x7f8bbc0c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 --2- 172.21.15.39:0/314086638 >> v2:172.21.15.84:3300/0 conn(0x7f8bbc14e530 0x7f8bbc0c1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 >> v1:172.21.15.84:6789/0 conn(0x7f8bbc151070 legacy=0x7f8bbc151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc154850 con 0x7f8bbc1540e0 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f8bac001680 con 0x7f8bbc1540e0 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2500640 1 --2- 172.21.15.39:0/314086638 >> v2:172.21.15.84:3300/0 conn(0x7f8bbc14e530 0x7f8bbc0c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 >> v1:172.21.15.39:6789/0 conn(0x7f8bbc1540e0 legacy=0x7f8bbc0c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba80039d0 0x7f8ba8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba80047d0 0x7f8ba8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba80051e0 0x7f8ba800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc151b90 con 0x7f8ba80051e0 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba800c130 con 0x7f8ba80039d0 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba800c2f0 con 0x7f8ba80047d0 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2d01640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba80051e0 0x7f8ba800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2500640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba80039d0 0x7f8ba8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc1cff640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba80047d0 0x7f8ba8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2500640 1 -- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba80047d0 msgr2=0x7f8ba8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2500640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba80047d0 0x7f8ba8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2500640 1 -- 172.21.15.39:0/314086638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba80051e0 msgr2=0x7f8ba800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2500640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba80051e0 0x7f8ba800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2500640 1 -- 172.21.15.39:0/314086638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8003530 con 0x7f8ba80039d0 2024-09-13T16:48:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2d01640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba80051e0 0x7f8ba800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.320+0000 7f8bc2d01640 1 -- 172.21.15.39:0/314086638 reap_dead start 2024-09-13T16:48:11.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc2500640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba80039d0 0x7f8ba8003df0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009da0 tx=0x7f8bb801ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404d451020ae6f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:11.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80ae600 con 0x7f8ba80039d0 2024-09-13T16:48:11.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb80aa050 con 0x7f8ba80039d0 2024-09-13T16:48:11.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc14fe640 1 -- 172.21.15.39:0/314086638 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80ae9a0 con 0x7f8ba80039d0 2024-09-13T16:48:11.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc4764640 1 -- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba80039d0 msgr2=0x7f8ba8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:11.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc4764640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba80039d0 0x7f8ba8003df0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009da0 tx=0x7f8bb801ef70 comp rx=0 tx=0).stop 2024-09-13T16:48:11.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc4764640 1 -- 172.21.15.39:0/314086638 shutdown_connections 2024-09-13T16:48:11.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc4764640 1 --2- 172.21.15.39:0/314086638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba80039d0 0x7f8ba8003df0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc4764640 1 -- 172.21.15.39:0/314086638 >> 172.21.15.39:0/314086638 conn(0x7f8bbc0c5790 msgr2=0x7f8bbc0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:11.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc4764640 1 -- 172.21.15.39:0/314086638 shutdown_connections 2024-09-13T16:48:11.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.324+0000 7f8bc4764640 1 -- 172.21.15.39:0/314086638 wait complete. 2024-09-13T16:48:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 Processor -- start 2024-09-13T16:48:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 -- start start 2024-09-13T16:48:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba80039d0 0x7f8bbc0a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc0a4b30 0x7f8bbc149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c4d10 0x7f8bbc1f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0c3710 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc1533d0 con 0x7f8bbc0a4b30 2024-09-13T16:48:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc150b30 con 0x7f8ba80039d0 2024-09-13T16:48:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2d01640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c4d10 0x7f8bbc1f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2d01640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c4d10 0x7f8bbc1f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34180/0 (socket says 172.21.15.39:34180) 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2d01640 1 -- 172.21.15.39:0/1093846495 learned_addr learned my addr 172.21.15.39:0/1093846495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2500640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba80039d0 0x7f8bbc0a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc1cff640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc0a4b30 0x7f8bbc149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2d01640 1 -- 172.21.15.39:0/1093846495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba80039d0 msgr2=0x7f8bbc0a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2d01640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba80039d0 0x7f8bbc0a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2d01640 1 -- 172.21.15.39:0/1093846495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc0a4b30 msgr2=0x7f8bbc149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2d01640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc0a4b30 0x7f8bbc149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2d01640 1 -- 172.21.15.39:0/1093846495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc1535b0 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2d01640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c4d10 0x7f8bbc1f0f70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4007fd0 tx=0x7f8bb400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bb2ffd640 1 -- 172.21.15.39:0/1093846495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4005070 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bb2ffd640 1 -- 172.21.15.39:0/1093846495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb40051f0 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc153890 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bb2ffd640 1 -- 172.21.15.39:0/1093846495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4007030 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc0c4510 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc153db0 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bb2ffd640 1 -- 172.21.15.39:0/1093846495 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8bb4007190 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bb2ffd640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba00666c0 0x7f8ba0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bb2ffd640 1 -- 172.21.15.39:0/1093846495 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f8bb4083460 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.328+0000 7f8bc2500640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba00666c0 0x7f8ba0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.332+0000 7f8bc2500640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba00666c0 0x7f8ba0068b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80b0000 tx=0x7f8bb8004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.336+0000 7f8bb2ffd640 1 -- 172.21.15.39:0/1093846495 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8bb40504a0 con 0x7f8bbc0c4d10 2024-09-13T16:48:11.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:11.524+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "b", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f8bbc151810 con 0x7f8bbc0c4d10 2024-09-13T16:48:12.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.168+0000 7f8bb2ffd640 1 -- 172.21.15.39:0/1093846495 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "b", "var": "session_timeout", "val": "300"}]=0 v14) v1 ==== 110+0+0 (secure 0 0 0) 0x7f8bb4053ac0 con 0x7f8bbc0c4d10 2024-09-13T16:48:12.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba00666c0 msgr2=0x7f8ba0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba00666c0 0x7f8ba0068b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80b0000 tx=0x7f8bb8004ac0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c4d10 msgr2=0x7f8bbc1f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c4d10 0x7f8bbc1f0f70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4007fd0 tx=0x7f8bb400fc00 comp rx=0 tx=0).stop 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 shutdown_connections 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba00666c0 0x7f8ba0068b80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc0a4b30 0x7f8bbc149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c4d10 0x7f8bbc1f0f70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 --2- 172.21.15.39:0/1093846495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba80039d0 0x7f8bbc0a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 >> 172.21.15.39:0/1093846495 conn(0x7f8bbc0c5790 msgr2=0x7f8bbc0c0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 shutdown_connections 2024-09-13T16:48:12.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.176+0000 7f8bc4764640 1 -- 172.21.15.39:0/1093846495 wait complete. 2024-09-13T16:48:12.204 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'b' 2024-09-13T16:48:12.204 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:12.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb79d006640 1 Processor -- start 2024-09-13T16:48:12.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb79d006640 1 -- start start 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb79d006640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb7980aabc0 0x7fb7980aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb79d006640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb7980c40b0 0x7fb79815c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb79d006640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb798152a80 con 0x7fb79806c870 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb79d006640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980c30f0 con 0x7fb7980aabc0 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb79d006640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb798151e00 con 0x7fb7980c40b0 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb796d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb7980aabc0 0x7fb7980aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb796d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb7980aabc0 0x7fb7980aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34194/0 (socket says 172.21.15.39:34194) 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb796d9d640 1 -- 172.21.15.39:0/4018451594 learned_addr learned my addr 172.21.15.39:0/4018451594 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb78ffff640 1 --1- 172.21.15.39:0/4018451594 >> v1:172.21.15.39:6789/0 conn(0x7fb79806c870 0x7fb7980c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33100/0 (socket says 172.21.15.39:33100) 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb79759e640 1 --2- 172.21.15.39:0/4018451594 >> v2:172.21.15.84:3300/0 conn(0x7fb7980c40b0 0x7fb79815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb796d9d640 1 -- 172.21.15.39:0/4018451594 >> v2:172.21.15.84:3300/0 conn(0x7fb7980c40b0 msgr2=0x7fb79815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:12.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb796d9d640 1 --2- 172.21.15.39:0/4018451594 >> v2:172.21.15.84:3300/0 conn(0x7fb7980c40b0 0x7fb79815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb796d9d640 1 -- 172.21.15.39:0/4018451594 >> v1:172.21.15.39:6789/0 conn(0x7fb79806c870 legacy=0x7fb7980c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.332+0000 7fb79659c640 1 -- 172.21.15.39:0/4018451594 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479005773 0 0) 0x7fb780001120 con 0x7fb79806c870 2024-09-13T16:48:12.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb796d9d640 1 -- 172.21.15.39:0/4018451594 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7981520e0 con 0x7fb7980aabc0 2024-09-13T16:48:12.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb796d9d640 1 --2- 172.21.15.39:0/4018451594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980aabc0 0x7fb7980aafa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb79806f220 tx=0x7fb78801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd9e79b8e81e3ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:12.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79659c640 1 -- 172.21.15.39:0/4018451594 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880ab660 con 0x7fb7980aabc0 2024-09-13T16:48:12.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79659c640 1 -- 172.21.15.39:0/4018451594 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7880ab7c0 con 0x7fb7980aabc0 2024-09-13T16:48:12.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79659c640 1 -- 172.21.15.39:0/4018451594 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880abb50 con 0x7fb7980aabc0 2024-09-13T16:48:12.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79d006640 1 -- 172.21.15.39:0/4018451594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980aabc0 msgr2=0x7fb7980aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79d006640 1 --2- 172.21.15.39:0/4018451594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980aabc0 0x7fb7980aafa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb79806f220 tx=0x7fb78801c460 comp rx=0 tx=0).stop 2024-09-13T16:48:12.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79d006640 1 -- 172.21.15.39:0/4018451594 shutdown_connections 2024-09-13T16:48:12.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79d006640 1 --2- 172.21.15.39:0/4018451594 >> v2:172.21.15.84:3300/0 conn(0x7fb7980c40b0 0x7fb79815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79d006640 1 --2- 172.21.15.39:0/4018451594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980aabc0 0x7fb7980aafa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79d006640 1 -- 172.21.15.39:0/4018451594 >> 172.21.15.39:0/4018451594 conn(0x7fb798069e10 msgr2=0x7fb79806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:12.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79d006640 1 -- 172.21.15.39:0/4018451594 shutdown_connections 2024-09-13T16:48:12.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.336+0000 7fb79d006640 1 -- 172.21.15.39:0/4018451594 wait complete. 2024-09-13T16:48:12.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 Processor -- start 2024-09-13T16:48:12.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 -- start start 2024-09-13T16:48:12.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb79806c870 0x7fb79814be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7980aabc0 0x7fb7981482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980c40b0 0x7fb798148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980a66a0 con 0x7fb7980c40b0 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798148d60 con 0x7fb79806c870 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb798071f30 con 0x7fb7980aabc0 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980c40b0 0x7fb798148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980c40b0 0x7fb798148820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34204/0 (socket says 172.21.15.39:34204) 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79759e640 1 -- 172.21.15.39:0/831277104 learned_addr learned my addr 172.21.15.39:0/831277104 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb796d9d640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb79806c870 0x7fb79814be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb78ffff640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7980aabc0 0x7fb7981482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79759e640 1 -- 172.21.15.39:0/831277104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7980aabc0 msgr2=0x7fb7981482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79759e640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7980aabc0 0x7fb7981482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79759e640 1 -- 172.21.15.39:0/831277104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb79806c870 msgr2=0x7fb79814be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79759e640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb79806c870 0x7fb79814be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79759e640 1 -- 172.21.15.39:0/831277104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7880094d0 con 0x7fb7980c40b0 2024-09-13T16:48:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79759e640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980c40b0 0x7fb798148820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb790007e60 tx=0x7fb79000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb78f7fe640 1 -- 172.21.15.39:0/831277104 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790014420 con 0x7fb7980c40b0 2024-09-13T16:48:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb78f7fe640 1 -- 172.21.15.39:0/831277104 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7900145a0 con 0x7fb7980c40b0 2024-09-13T16:48:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb798072210 con 0x7fb7980c40b0 2024-09-13T16:48:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb78f7fe640 1 -- 172.21.15.39:0/831277104 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790014950 con 0x7fb7980c40b0 2024-09-13T16:48:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7980a6b90 con 0x7fb7980c40b0 2024-09-13T16:48:12.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.340+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7980a6db0 con 0x7fb7980c40b0 2024-09-13T16:48:12.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.344+0000 7fb78f7fe640 1 -- 172.21.15.39:0/831277104 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb790014ab0 con 0x7fb7980c40b0 2024-09-13T16:48:12.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.344+0000 7fb78f7fe640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb7780666c0 0x7fb778068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.344+0000 7fb78f7fe640 1 -- 172.21.15.39:0/831277104 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb790084dd0 con 0x7fb7980c40b0 2024-09-13T16:48:12.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.344+0000 7fb796d9d640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb7780666c0 0x7fb778068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.344+0000 7fb796d9d640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb7780666c0 0x7fb778068b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7880af000 tx=0x7fb788008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:12.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.348+0000 7fb78f7fe640 1 -- 172.21.15.39:0/831277104 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb790051e10 con 0x7fb7980c40b0 2024-09-13T16:48:12.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.540+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7980aafa0 con 0x7fb7980c40b0 2024-09-13T16:48:12.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.540+0000 7fb78f7fe640 1 -- 172.21.15.39:0/831277104 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7fb790086020 con 0x7fb7980c40b0 2024-09-13T16:48:12.546 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:12.547 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:48:12.547 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:48:12.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb7780666c0 msgr2=0x7fb778068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb7780666c0 0x7fb778068b80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7880af000 tx=0x7fb788008040 comp rx=0 tx=0).stop 2024-09-13T16:48:12.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980c40b0 msgr2=0x7fb798148820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980c40b0 0x7fb798148820 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb790007e60 tx=0x7fb79000e4b0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 shutdown_connections 2024-09-13T16:48:12.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb7780666c0 0x7fb778068b80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7980c40b0 0x7fb798148820 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb7980aabc0 0x7fb7981482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 --2- 172.21.15.39:0/831277104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb79806c870 0x7fb79814be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 >> 172.21.15.39:0/831277104 conn(0x7fb798069e10 msgr2=0x7fb79806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 shutdown_connections 2024-09-13T16:48:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.544+0000 7fb79d006640 1 -- 172.21.15.39:0/831277104 wait complete. 2024-09-13T16:48:12.566 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:12.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.676+0000 7f6fe0cf6640 1 Processor -- start 2024-09-13T16:48:12.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.676+0000 7f6fe0cf6640 1 -- start start 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6fdc0aabc0 0x7f6fdc0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6fdc0c40b0 0x7f6fdc15c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc152a80 con 0x7f6fdc06c870 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fdc0c30f0 con 0x7f6fdc0c40b0 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fdc151e00 con 0x7f6fdc0aabc0 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fdad9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6fdc0c40b0 0x7f6fdc15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fdad9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6fdc0c40b0 0x7f6fdc15c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34210/0 (socket says 172.21.15.39:34210) 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fdad9d640 1 -- 172.21.15.39:0/2275999067 learned_addr learned my addr 172.21.15.39:0/2275999067 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fda59c640 1 --2- 172.21.15.39:0/2275999067 >> v2:172.21.15.84:3300/0 conn(0x7f6fdc0aabc0 0x7f6fdc0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fdad9d640 1 -- 172.21.15.39:0/2275999067 >> v2:172.21.15.84:3300/0 conn(0x7f6fdc0aabc0 msgr2=0x7f6fdc0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:12.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fdad9d640 1 --2- 172.21.15.39:0/2275999067 >> v2:172.21.15.84:3300/0 conn(0x7f6fdc0aabc0 0x7f6fdc0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fdad9d640 1 -- 172.21.15.39:0/2275999067 >> v1:172.21.15.84:6789/0 conn(0x7f6fdc06c870 legacy=0x7f6fdc0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:12.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fdad9d640 1 -- 172.21.15.39:0/2275999067 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc1520e0 con 0x7f6fdc0c40b0 2024-09-13T16:48:12.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fdad9d640 1 --2- 172.21.15.39:0/2275999067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc0c40b0 0x7f6fdc15c470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0c33a0 tx=0x7f6fd401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8464793daff8ff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fd959a640 1 -- 172.21.15.39:0/2275999067 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40a57b0 con 0x7f6fdc0c40b0 2024-09-13T16:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fd959a640 1 -- 172.21.15.39:0/2275999067 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd40a5910 con 0x7f6fdc0c40b0 2024-09-13T16:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fd959a640 1 -- 172.21.15.39:0/2275999067 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd40a5ca0 con 0x7f6fdc0c40b0 2024-09-13T16:48:12.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/2275999067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc0c40b0 msgr2=0x7f6fdc15c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 --2- 172.21.15.39:0/2275999067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc0c40b0 0x7f6fdc15c470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0c33a0 tx=0x7f6fd401c8f0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/2275999067 shutdown_connections 2024-09-13T16:48:12.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 --2- 172.21.15.39:0/2275999067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc0c40b0 0x7f6fdc15c470 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 --2- 172.21.15.39:0/2275999067 >> v2:172.21.15.84:3300/0 conn(0x7f6fdc0aabc0 0x7f6fdc0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/2275999067 >> 172.21.15.39:0/2275999067 conn(0x7f6fdc069e10 msgr2=0x7f6fdc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:12.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/2275999067 shutdown_connections 2024-09-13T16:48:12.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.680+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/2275999067 wait complete. 2024-09-13T16:48:12.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 Processor -- start 2024-09-13T16:48:12.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 -- start start 2024-09-13T16:48:12.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc06c870 0x7f6fdc1ec740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6fdc0aabc0 0x7f6fdc1474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fdc0c40b0 0x7f6fdc1f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc1438e0 con 0x7f6fdc06c870 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc1efcb0 con 0x7f6fdc0c40b0 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc142d20 con 0x7f6fdc0aabc0 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fda59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc06c870 0x7f6fdc1ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fda59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc06c870 0x7f6fdc1ec740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34224/0 (socket says 172.21.15.39:34224) 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fda59c640 1 -- 172.21.15.39:0/1527154177 learned_addr learned my addr 172.21.15.39:0/1527154177 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fdad9d640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fdc0c40b0 0x7f6fdc1f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fd9d9b640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6fdc0aabc0 0x7f6fdc1474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fda59c640 1 -- 172.21.15.39:0/1527154177 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6fdc0aabc0 msgr2=0x7f6fdc1474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fda59c640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6fdc0aabc0 0x7f6fdc1474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fda59c640 1 -- 172.21.15.39:0/1527154177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fdc0c40b0 msgr2=0x7f6fdc1f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fda59c640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fdc0c40b0 0x7f6fdc1f0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fda59c640 1 -- 172.21.15.39:0/1527154177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd4009b30 con 0x7f6fdc06c870 2024-09-13T16:48:12.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fda59c640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc06c870 0x7f6fdc1ec740 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000c7f0 tx=0x7f6fd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fcaffd640 1 -- 172.21.15.39:0/1527154177 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0004220 con 0x7f6fdc06c870 2024-09-13T16:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fcaffd640 1 -- 172.21.15.39:0/1527154177 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd00043a0 con 0x7f6fdc06c870 2024-09-13T16:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc143000 con 0x7f6fdc06c870 2024-09-13T16:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fcaffd640 1 -- 172.21.15.39:0/1527154177 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0004510 con 0x7f6fdc06c870 2024-09-13T16:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc147a20 con 0x7f6fdc06c870 2024-09-13T16:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc143240 con 0x7f6fdc06c870 2024-09-13T16:48:12.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.684+0000 7f6fcaffd640 1 -- 172.21.15.39:0/1527154177 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6fd0003a70 con 0x7f6fdc06c870 2024-09-13T16:48:12.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.688+0000 7f6fcaffd640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fbc066690 0x7f6fbc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.688+0000 7f6fcaffd640 1 -- 172.21.15.39:0/1527154177 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f6fd0084400 con 0x7f6fdc06c870 2024-09-13T16:48:12.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.688+0000 7f6fd9d9b640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fbc066690 0x7f6fbc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.688+0000 7f6fd9d9b640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fbc066690 0x7f6fbc068b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc151a50 tx=0x7f6fcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:12.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.692+0000 7f6fcaffd640 1 -- 172.21.15.39:0/1527154177 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6fd00514d0 con 0x7f6fdc06c870 2024-09-13T16:48:12.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.844+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fdc06d530 con 0x7f6fdc06c870 2024-09-13T16:48:12.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.848+0000 7f6fcaffd640 1 -- 172.21.15.39:0/1527154177 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7f6fd0054af0 con 0x7f6fdc06c870 2024-09-13T16:48:12.853 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:12.853 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:12.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fbc066690 msgr2=0x7f6fbc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fbc066690 0x7f6fbc068b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc151a50 tx=0x7f6fcc008040 comp rx=0 tx=0).stop 2024-09-13T16:48:12.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc06c870 msgr2=0x7f6fdc1ec740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc06c870 0x7f6fdc1ec740 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000c7f0 tx=0x7f6fd000ccb0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 shutdown_connections 2024-09-13T16:48:12.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fbc066690 0x7f6fbc068b50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6fdc0c40b0 0x7f6fdc1f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6fdc0aabc0 0x7f6fdc1474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 --2- 172.21.15.39:0/1527154177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fdc06c870 0x7f6fdc1ec740 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 >> 172.21.15.39:0/1527154177 conn(0x7f6fdc069e10 msgr2=0x7f6fdc14ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:12.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 shutdown_connections 2024-09-13T16:48:12.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.852+0000 7f6fe0cf6640 1 -- 172.21.15.39:0/1527154177 wait complete. 2024-09-13T16:48:12.870 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T16:48:12.870 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:12.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9e50e640 1 Processor -- start 2024-09-13T16:48:12.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9e50e640 1 -- start start 2024-09-13T16:48:12.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9e50e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b980abc50 0x7f5b9814f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9e50e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b980ab340 0x7f5b9814f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9e50e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b98156270 con 0x7f5b9806cbc0 2024-09-13T16:48:12.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9e50e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b98061530 con 0x7f5b980ab340 2024-09-13T16:48:12.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9e50e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b981554b0 con 0x7f5b980abc50 2024-09-13T16:48:12.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9caab640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5b9806cbc0 0x7f5b9806d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33110/0 (socket says 172.21.15.39:33110) 2024-09-13T16:48:12.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9caab640 1 -- 172.21.15.39:0/685258595 learned_addr learned my addr 172.21.15.39:0/685258595 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:12.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b97fff640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b980abc50 0x7f5b9814f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b977fe640 1 --2- 172.21.15.39:0/685258595 >> v2:172.21.15.84:3300/0 conn(0x7f5b980ab340 0x7f5b9814f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965023678 0 0) 0x7f5b98156270 con 0x7f5b9806cbc0 2024-09-13T16:48:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b84003530 con 0x7f5b9806cbc0 2024-09-13T16:48:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 829272184 0 0) 0x7f5b84003530 con 0x7f5b9806cbc0 2024-09-13T16:48:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b980abc50 msgr2=0x7f5b9814f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b980abc50 0x7f5b9814f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 >> v2:172.21.15.84:3300/0 conn(0x7f5b980ab340 msgr2=0x7f5b9814f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 --2- 172.21.15.39:0/685258595 >> v2:172.21.15.84:3300/0 conn(0x7f5b980ab340 0x7f5b9814f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b98155790 con 0x7f5b9806cbc0 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f5b88001a00 con 0x7f5b9806cbc0 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 >> v1:172.21.15.39:6789/0 conn(0x7f5b9806cbc0 legacy=0x7f5b9806d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b84003980 0x7f5b84003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b840047d0 0x7f5b84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b980abc50 0x7f5b84005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98061530 con 0x7f5b84003980 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b981554b0 con 0x7f5b840047d0 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8400ba10 con 0x7f5b980abc50 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b97fff640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b84003980 0x7f5b84003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b9caab640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b980abc50 0x7f5b84005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:12.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b97fff640 1 -- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b980abc50 msgr2=0x7f5b84005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:12.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b97fff640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b980abc50 0x7f5b84005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b97fff640 1 -- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b840047d0 msgr2=0x7f5b84004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:12.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b97fff640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b840047d0 0x7f5b84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:12.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.984+0000 7f5b97fff640 1 -- 172.21.15.39:0/685258595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b84003530 con 0x7f5b84003980 2024-09-13T16:48:12.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b97fff640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b84003980 0x7f5b84003da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009ae0 tx=0x7f5b9001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c347e8fa41b87f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:12.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900a5520 con 0x7f5b84003980 2024-09-13T16:48:12.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b900a5680 con 0x7f5b84003980 2024-09-13T16:48:12.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b96ffd640 1 -- 172.21.15.39:0/685258595 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900a5a30 con 0x7f5b84003980 2024-09-13T16:48:12.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b9e50e640 1 -- 172.21.15.39:0/685258595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b84003980 msgr2=0x7f5b84003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:12.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b9e50e640 1 --2- 172.21.15.39:0/685258595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b84003980 0x7f5b84003da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5b90009ae0 tx=0x7f5b9001ca40 comp rx=0 tx=0).stop 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b9caab640 1 -- 172.21.15.39:0/685258595 reap_dead start 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b9e50e640 1 -- 172.21.15.39:0/685258595 shutdown_connections 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b9e50e640 1 -- 172.21.15.39:0/685258595 >> 172.21.15.39:0/685258595 conn(0x7f5b98069e10 msgr2=0x7f5b9806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b9e50e640 1 -- 172.21.15.39:0/685258595 shutdown_connections 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b9e50e640 1 -- 172.21.15.39:0/685258595 wait complete. 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b9e50e640 1 Processor -- start 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.988+0000 7f5b9e50e640 1 -- start start 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9e50e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b84003980 0x7f5b98152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9e50e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b981454a0 0x7f5b981458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9e50e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9815a240 0x7f5b9815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:12.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9e50e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98147fc0 con 0x7f5b9815a240 2024-09-13T16:48:13.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9e50e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98148320 con 0x7f5b981454a0 2024-09-13T16:48:13.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9e50e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98152c80 con 0x7f5b84003980 2024-09-13T16:48:13.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9caab640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9815a240 0x7f5b9815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9caab640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9815a240 0x7f5b9815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34248/0 (socket says 172.21.15.39:34248) 2024-09-13T16:48:13.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9caab640 1 -- 172.21.15.39:0/418919997 learned_addr learned my addr 172.21.15.39:0/418919997 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:13.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b977fe640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b981454a0 0x7f5b981458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b97fff640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b84003980 0x7f5b98152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9caab640 1 -- 172.21.15.39:0/418919997 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b84003980 msgr2=0x7f5b98152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9caab640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b84003980 0x7f5b98152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9caab640 1 -- 172.21.15.39:0/418919997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b981454a0 msgr2=0x7f5b981458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9caab640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b981454a0 0x7f5b981458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9caab640 1 -- 172.21.15.39:0/418919997 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b981522f0 con 0x7f5b9815a240 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9caab640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9815a240 0x7f5b9815a680 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b98072500 tx=0x7f5b8800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b94ff9640 1 -- 172.21.15.39:0/418919997 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800ee20 con 0x7f5b9815a240 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b94ff9640 1 -- 172.21.15.39:0/418919997 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b88001960 con 0x7f5b9815a240 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b98147440 con 0x7f5b9815a240 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b94ff9640 1 -- 172.21.15.39:0/418919997 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88013660 con 0x7f5b9815a240 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b98147a00 con 0x7f5b9815a240 2024-09-13T16:48:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b980ac030 con 0x7f5b9815a240 2024-09-13T16:48:13.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b94ff9640 1 -- 172.21.15.39:0/418919997 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b880137c0 con 0x7f5b9815a240 2024-09-13T16:48:13.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b94ff9640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b780666c0 0x7f5b78068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b97fff640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b780666c0 0x7f5b78068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b94ff9640 1 -- 172.21.15.39:0/418919997 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f5b880848a0 con 0x7f5b9815a240 2024-09-13T16:48:13.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.992+0000 7f5b97fff640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b780666c0 0x7f5b78068b80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5b900af000 tx=0x7f5b900046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:13.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:12.996+0000 7f5b94ff9640 1 -- 172.21.15.39:0/418919997 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b88051970 con 0x7f5b9815a240 2024-09-13T16:48:13.138 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T16:48:13.132+0000 7fcdae1dc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:48:13.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.152+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b980a9b70 con 0x7f5b780666c0 2024-09-13T16:48:13.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.172+0000 7f5b94ff9640 1 -- 172.21.15.39:0/418919997 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+492189 (secure 0 0 0) 0x7f5b980a9b70 con 0x7f5b780666c0 2024-09-13T16:48:13.178 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:13.186 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:13.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b780666c0 msgr2=0x7f5b78068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b780666c0 0x7f5b78068b80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5b900af000 tx=0x7f5b900046d0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9815a240 msgr2=0x7f5b9815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9815a240 0x7f5b9815a680 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b98072500 tx=0x7f5b8800e530 comp rx=0 tx=0).stop 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 shutdown_connections 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b780666c0 0x7f5b78068b80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b981454a0 0x7f5b981458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b9815a240 0x7f5b9815a680 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 --2- 172.21.15.39:0/418919997 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b84003980 0x7f5b98152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 >> 172.21.15.39:0/418919997 conn(0x7f5b98069e10 msgr2=0x7f5b9806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 shutdown_connections 2024-09-13T16:48:13.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.180+0000 7f5b9e50e640 1 -- 172.21.15.39:0/418919997 wait complete. 2024-09-13T16:48:13.201 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-13T16:48:11.561214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235428,"kb_used_data":2884,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739292,"statfs":{"total":773094113280,"available":772853035008,"internally_reserved":0,"allocated":2953216,"data_stored":1785199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.832843"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597573+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:02.597573+0000","last_peered":"2024-09-13T16:48:02.597573+0000","last_clean":"2024-09-13T16:48:02.597573+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:02.597573+0000","last_undegraded":"2024-09-13T16:48:02.597573+0000","last_fullsized":"2024-09-13T16:48:02.597573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590824+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:02.590824+0000","last_peered":"2024-09-13T16:48:02.590824+0000","last_clean":"2024-09-13T16:48:02.590824+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:02.590824+0000","last_undegraded":"2024-09-13T16:48:02.590824+0000","last_fullsized":"2024-09-13T16:48:02.590824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738264+0000","last_change":"2024-09-13T16:48:06.738264+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738264+0000","last_undegraded":"2024-09-13T16:48:06.738264+0000","last_fullsized":"2024-09-13T16:48:06.738264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590848+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:02.590848+0000","last_peered":"2024-09-13T16:48:02.590848+0000","last_clean":"2024-09-13T16:48:02.590848+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:02.590848+0000","last_undegraded":"2024-09-13T16:48:02.590848+0000","last_fullsized":"2024-09-13T16:48:02.590848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597512+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:02.597512+0000","last_peered":"2024-09-13T16:48:02.597512+0000","last_clean":"2024-09-13T16:48:02.597512+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:02.597512+0000","last_undegraded":"2024-09-13T16:48:02.597512+0000","last_fullsized":"2024-09-13T16:48:02.597512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734168+0000","last_change":"2024-09-13T16:48:06.734168+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734168+0000","last_undegraded":"2024-09-13T16:48:06.734168+0000","last_fullsized":"2024-09-13T16:48:06.734168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597829+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:02.597829+0000","last_peered":"2024-09-13T16:48:02.597829+0000","last_clean":"2024-09-13T16:48:02.597829+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:02.597829+0000","last_undegraded":"2024-09-13T16:48:02.597829+0000","last_fullsized":"2024-09-13T16:48:02.597829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593935+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:02.593935+0000","last_peered":"2024-09-13T16:48:02.593935+0000","last_clean":"2024-09-13T16:48:02.593935+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:02.593935+0000","last_undegraded":"2024-09-13T16:48:02.593935+0000","last_fullsized":"2024-09-13T16:48:02.593935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736405+0000","last_change":"2024-09-13T16:48:06.736405+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736405+0000","last_undegraded":"2024-09-13T16:48:06.736405+0000","last_fullsized":"2024-09-13T16:48:06.736405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597754+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:02.597754+0000","last_peered":"2024-09-13T16:48:02.597754+0000","last_clean":"2024-09-13T16:48:02.597754+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:02.597754+0000","last_undegraded":"2024-09-13T16:48:02.597754+0000","last_fullsized":"2024-09-13T16:48:02.597754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593979+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:02.593979+0000","last_peered":"2024-09-13T16:48:02.593979+0000","last_clean":"2024-09-13T16:48:02.593979+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:02.593979+0000","last_undegraded":"2024-09-13T16:48:02.593979+0000","last_fullsized":"2024-09-13T16:48:02.593979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734117+0000","last_change":"2024-09-13T16:48:06.734117+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734117+0000","last_undegraded":"2024-09-13T16:48:06.734117+0000","last_fullsized":"2024-09-13T16:48:06.734117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593280+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:02.593280+0000","last_peered":"2024-09-13T16:48:02.593280+0000","last_clean":"2024-09-13T16:48:02.593280+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:02.593280+0000","last_undegraded":"2024-09-13T16:48:02.593280+0000","last_fullsized":"2024-09-13T16:48:02.593280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597729+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:02.597729+0000","last_peered":"2024-09-13T16:48:02.597729+0000","last_clean":"2024-09-13T16:48:02.597729+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:02.597729+0000","last_undegraded":"2024-09-13T16:48:02.597729+0000","last_fullsized":"2024-09-13T16:48:02.597729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.735578+0000","last_change":"2024-09-13T16:48:06.735578+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.735578+0000","last_undegraded":"2024-09-13T16:48:06.735578+0000","last_fullsized":"2024-09-13T16:48:06.735578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590720+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:02.590720+0000","last_peered":"2024-09-13T16:48:02.590720+0000","last_clean":"2024-09-13T16:48:02.590720+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:02.590720+0000","last_undegraded":"2024-09-13T16:48:02.590720+0000","last_fullsized":"2024-09-13T16:48:02.590720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593524+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:02.593524+0000","last_peered":"2024-09-13T16:48:02.593524+0000","last_clean":"2024-09-13T16:48:02.593524+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:02.593524+0000","last_undegraded":"2024-09-13T16:48:02.593524+0000","last_fullsized":"2024-09-13T16:48:02.593524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734251+0000","last_change":"2024-09-13T16:48:06.734251+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734251+0000","last_undegraded":"2024-09-13T16:48:06.734251+0000","last_fullsized":"2024-09-13T16:48:06.734251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590600+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:02.590600+0000","last_peered":"2024-09-13T16:48:02.590600+0000","last_clean":"2024-09-13T16:48:02.590600+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:02.590600+0000","last_undegraded":"2024-09-13T16:48:02.590600+0000","last_fullsized":"2024-09-13T16:48:02.590600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593478+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:02.593478+0000","last_peered":"2024-09-13T16:48:02.593478+0000","last_clean":"2024-09-13T16:48:02.593478+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:02.593478+0000","last_undegraded":"2024-09-13T16:48:02.593478+0000","last_fullsized":"2024-09-13T16:48:02.593478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736068+0000","last_change":"2024-09-13T16:48:06.736068+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736068+0000","last_undegraded":"2024-09-13T16:48:06.736068+0000","last_fullsized":"2024-09-13T16:48:06.736068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589876+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:02.589876+0000","last_peered":"2024-09-13T16:48:02.589876+0000","last_clean":"2024-09-13T16:48:02.589876+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:02.589876+0000","last_undegraded":"2024-09-13T16:48:02.589876+0000","last_fullsized":"2024-09-13T16:48:02.589876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597549+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:02.597549+0000","last_peered":"2024-09-13T16:48:02.597549+0000","last_clean":"2024-09-13T16:48:02.597549+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:02.597549+0000","last_undegraded":"2024-09-13T16:48:02.597549+0000","last_fullsized":"2024-09-13T16:48:02.597549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738308+0000","last_change":"2024-09-13T16:48:06.738308+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738308+0000","last_undegraded":"2024-09-13T16:48:06.738308+0000","last_fullsized":"2024-09-13T16:48:06.738308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597488+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:02.597488+0000","last_peered":"2024-09-13T16:48:02.597488+0000","last_clean":"2024-09-13T16:48:02.597488+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:02.597488+0000","last_undegraded":"2024-09-13T16:48:02.597488+0000","last_fullsized":"2024-09-13T16:48:02.597488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593499+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:02.593499+0000","last_peered":"2024-09-13T16:48:02.593499+0000","last_clean":"2024-09-13T16:48:02.593499+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:02.593499+0000","last_undegraded":"2024-09-13T16:48:02.593499+0000","last_fullsized":"2024-09-13T16:48:02.593499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736950+0000","last_change":"2024-09-13T16:48:06.736950+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736950+0000","last_undegraded":"2024-09-13T16:48:06.736950+0000","last_fullsized":"2024-09-13T16:48:06.736950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590892+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:02.590892+0000","last_peered":"2024-09-13T16:48:02.590892+0000","last_clean":"2024-09-13T16:48:02.590892+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:02.590892+0000","last_undegraded":"2024-09-13T16:48:02.590892+0000","last_fullsized":"2024-09-13T16:48:02.590892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:03.216479+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:03.216479+0000","last_peered":"2024-09-13T16:48:03.216479+0000","last_clean":"2024-09-13T16:48:03.216479+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:03.216479+0000","last_undegraded":"2024-09-13T16:48:03.216479+0000","last_fullsized":"2024-09-13T16:48:03.216479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738588+0000","last_change":"2024-09-13T16:48:06.738588+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738588+0000","last_undegraded":"2024-09-13T16:48:06.738588+0000","last_fullsized":"2024-09-13T16:48:06.738588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593591+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:02.593591+0000","last_peered":"2024-09-13T16:48:02.593591+0000","last_clean":"2024-09-13T16:48:02.593591+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:02.593591+0000","last_undegraded":"2024-09-13T16:48:02.593591+0000","last_fullsized":"2024-09-13T16:48:02.593591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597874+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:02.597874+0000","last_peered":"2024-09-13T16:48:02.597874+0000","last_clean":"2024-09-13T16:48:02.597874+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:02.597874+0000","last_undegraded":"2024-09-13T16:48:02.597874+0000","last_fullsized":"2024-09-13T16:48:02.597874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.798573+0000","last_change":"2024-09-13T16:48:06.798573+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.798573+0000","last_undegraded":"2024-09-13T16:48:06.798573+0000","last_fullsized":"2024-09-13T16:48:06.798573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594000+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:02.594000+0000","last_peered":"2024-09-13T16:48:02.594000+0000","last_clean":"2024-09-13T16:48:02.594000+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:02.594000+0000","last_undegraded":"2024-09-13T16:48:02.594000+0000","last_fullsized":"2024-09-13T16:48:02.594000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593637+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:02.593637+0000","last_peered":"2024-09-13T16:48:02.593637+0000","last_clean":"2024-09-13T16:48:02.593637+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:02.593637+0000","last_undegraded":"2024-09-13T16:48:02.593637+0000","last_fullsized":"2024-09-13T16:48:02.593637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734599+0000","last_change":"2024-09-13T16:48:06.734599+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734599+0000","last_undegraded":"2024-09-13T16:48:06.734599+0000","last_fullsized":"2024-09-13T16:48:06.734599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589940+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:02.589940+0000","last_peered":"2024-09-13T16:48:02.589940+0000","last_clean":"2024-09-13T16:48:02.589940+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:02.589940+0000","last_undegraded":"2024-09-13T16:48:02.589940+0000","last_fullsized":"2024-09-13T16:48:02.589940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593847+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:02.593847+0000","last_peered":"2024-09-13T16:48:02.593847+0000","last_clean":"2024-09-13T16:48:02.593847+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:02.593847+0000","last_undegraded":"2024-09-13T16:48:02.593847+0000","last_fullsized":"2024-09-13T16:48:02.593847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734532+0000","last_change":"2024-09-13T16:48:06.734532+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734532+0000","last_undegraded":"2024-09-13T16:48:06.734532+0000","last_fullsized":"2024-09-13T16:48:06.734532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593308+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:02.593308+0000","last_peered":"2024-09-13T16:48:02.593308+0000","last_clean":"2024-09-13T16:48:02.593308+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:02.593308+0000","last_undegraded":"2024-09-13T16:48:02.593308+0000","last_fullsized":"2024-09-13T16:48:02.593308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594186+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:02.594186+0000","last_peered":"2024-09-13T16:48:02.594186+0000","last_clean":"2024-09-13T16:48:02.594186+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:02.594186+0000","last_undegraded":"2024-09-13T16:48:02.594186+0000","last_fullsized":"2024-09-13T16:48:02.594186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.735058+0000","last_change":"2024-09-13T16:48:06.735058+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.735058+0000","last_undegraded":"2024-09-13T16:48:06.735058+0000","last_fullsized":"2024-09-13T16:48:06.735058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597220+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:02.597220+0000","last_peered":"2024-09-13T16:48:02.597220+0000","last_clean":"2024-09-13T16:48:02.597220+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:02.597220+0000","last_undegraded":"2024-09-13T16:48:02.597220+0000","last_fullsized":"2024-09-13T16:48:02.597220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597578+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:02.597578+0000","last_peered":"2024-09-13T16:48:02.597578+0000","last_clean":"2024-09-13T16:48:02.597578+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:02.597578+0000","last_undegraded":"2024-09-13T16:48:02.597578+0000","last_fullsized":"2024-09-13T16:48:02.597578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.799225+0000","last_change":"2024-09-13T16:48:06.799225+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.799225+0000","last_undegraded":"2024-09-13T16:48:06.799225+0000","last_fullsized":"2024-09-13T16:48:06.799225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590676+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:02.590676+0000","last_peered":"2024-09-13T16:48:02.590676+0000","last_clean":"2024-09-13T16:48:02.590676+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:02.590676+0000","last_undegraded":"2024-09-13T16:48:02.590676+0000","last_fullsized":"2024-09-13T16:48:02.590676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593871+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:02.593871+0000","last_peered":"2024-09-13T16:48:02.593871+0000","last_clean":"2024-09-13T16:48:02.593871+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:02.593871+0000","last_undegraded":"2024-09-13T16:48:02.593871+0000","last_fullsized":"2024-09-13T16:48:02.593871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263443+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:00.263443+0000","last_peered":"2024-09-13T16:48:00.263443+0000","last_clean":"2024-09-13T16:48:00.263443+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:00.263443+0000","last_undegraded":"2024-09-13T16:48:00.263443+0000","last_fullsized":"2024-09-13T16:48:00.263443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.740802+0000","last_change":"2024-09-13T16:48:06.740802+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.740802+0000","last_undegraded":"2024-09-13T16:48:06.740802+0000","last_fullsized":"2024-09-13T16:48:06.740802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597803+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:02.597803+0000","last_peered":"2024-09-13T16:48:02.597803+0000","last_clean":"2024-09-13T16:48:02.597803+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:02.597803+0000","last_undegraded":"2024-09-13T16:48:02.597803+0000","last_fullsized":"2024-09-13T16:48:02.597803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594240+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:02.594240+0000","last_peered":"2024-09-13T16:48:02.594240+0000","last_clean":"2024-09-13T16:48:02.594240+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:02.594240+0000","last_undegraded":"2024-09-13T16:48:02.594240+0000","last_fullsized":"2024-09-13T16:48:02.594240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.741059+0000","last_change":"2024-09-13T16:48:06.741059+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.741059+0000","last_undegraded":"2024-09-13T16:48:06.741059+0000","last_fullsized":"2024-09-13T16:48:06.741059+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593957+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:02.593957+0000","last_peered":"2024-09-13T16:48:02.593957+0000","last_clean":"2024-09-13T16:48:02.593957+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:02.593957+0000","last_undegraded":"2024-09-13T16:48:02.593957+0000","last_fullsized":"2024-09-13T16:48:02.593957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593802+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:02.593802+0000","last_peered":"2024-09-13T16:48:02.593802+0000","last_clean":"2024-09-13T16:48:02.593802+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:02.593802+0000","last_undegraded":"2024-09-13T16:48:02.593802+0000","last_fullsized":"2024-09-13T16:48:02.593802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.799237+0000","last_change":"2024-09-13T16:48:06.799237+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.799237+0000","last_undegraded":"2024-09-13T16:48:06.799237+0000","last_fullsized":"2024-09-13T16:48:06.799237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597984+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:02.597984+0000","last_peered":"2024-09-13T16:48:02.597984+0000","last_clean":"2024-09-13T16:48:02.597984+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:02.597984+0000","last_undegraded":"2024-09-13T16:48:02.597984+0000","last_fullsized":"2024-09-13T16:48:02.597984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594334+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:02.594334+0000","last_peered":"2024-09-13T16:48:02.594334+0000","last_clean":"2024-09-13T16:48:02.594334+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:02.594334+0000","last_undegraded":"2024-09-13T16:48:02.594334+0000","last_fullsized":"2024-09-13T16:48:02.594334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736498+0000","last_change":"2024-09-13T16:48:06.736498+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736498+0000","last_undegraded":"2024-09-13T16:48:06.736498+0000","last_fullsized":"2024-09-13T16:48:06.736498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597908+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:02.597908+0000","last_peered":"2024-09-13T16:48:02.597908+0000","last_clean":"2024-09-13T16:48:02.597908+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:02.597908+0000","last_undegraded":"2024-09-13T16:48:02.597908+0000","last_fullsized":"2024-09-13T16:48:02.597908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590065+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:02.590065+0000","last_peered":"2024-09-13T16:48:02.590065+0000","last_clean":"2024-09-13T16:48:02.590065+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:02.590065+0000","last_undegraded":"2024-09-13T16:48:02.590065+0000","last_fullsized":"2024-09-13T16:48:02.590065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738166+0000","last_change":"2024-09-13T16:48:06.738166+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738166+0000","last_undegraded":"2024-09-13T16:48:06.738166+0000","last_fullsized":"2024-09-13T16:48:06.738166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597646+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:02.597646+0000","last_peered":"2024-09-13T16:48:02.597646+0000","last_clean":"2024-09-13T16:48:02.597646+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:02.597646+0000","last_undegraded":"2024-09-13T16:48:02.597646+0000","last_fullsized":"2024-09-13T16:48:02.597646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594083+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:02.594083+0000","last_peered":"2024-09-13T16:48:02.594083+0000","last_clean":"2024-09-13T16:48:02.594083+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:02.594083+0000","last_undegraded":"2024-09-13T16:48:02.594083+0000","last_fullsized":"2024-09-13T16:48:02.594083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.798551+0000","last_change":"2024-09-13T16:48:06.798551+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.798551+0000","last_undegraded":"2024-09-13T16:48:06.798551+0000","last_fullsized":"2024-09-13T16:48:06.798551+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593229+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:02.593229+0000","last_peered":"2024-09-13T16:48:02.593229+0000","last_clean":"2024-09-13T16:48:02.593229+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:02.593229+0000","last_undegraded":"2024-09-13T16:48:02.593229+0000","last_fullsized":"2024-09-13T16:48:02.593229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593257+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:02.593257+0000","last_peered":"2024-09-13T16:48:02.593257+0000","last_clean":"2024-09-13T16:48:02.593257+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:02.593257+0000","last_undegraded":"2024-09-13T16:48:02.593257+0000","last_fullsized":"2024-09-13T16:48:02.593257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.739047+0000","last_change":"2024-09-13T16:48:06.739047+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.739047+0000","last_undegraded":"2024-09-13T16:48:06.739047+0000","last_fullsized":"2024-09-13T16:48:06.739047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597706+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:02.597706+0000","last_peered":"2024-09-13T16:48:02.597706+0000","last_clean":"2024-09-13T16:48:02.597706+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:02.597706+0000","last_undegraded":"2024-09-13T16:48:02.597706+0000","last_fullsized":"2024-09-13T16:48:02.597706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593823+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:02.593823+0000","last_peered":"2024-09-13T16:48:02.593823+0000","last_clean":"2024-09-13T16:48:02.593823+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:02.593823+0000","last_undegraded":"2024-09-13T16:48:02.593823+0000","last_fullsized":"2024-09-13T16:48:02.593823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.739744+0000","last_change":"2024-09-13T16:48:06.739744+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.739744+0000","last_undegraded":"2024-09-13T16:48:06.739744+0000","last_fullsized":"2024-09-13T16:48:06.739744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597339+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:02.597339+0000","last_peered":"2024-09-13T16:48:02.597339+0000","last_clean":"2024-09-13T16:48:02.597339+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:02.597339+0000","last_undegraded":"2024-09-13T16:48:02.597339+0000","last_fullsized":"2024-09-13T16:48:02.597339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597439+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:02.597439+0000","last_peered":"2024-09-13T16:48:02.597439+0000","last_clean":"2024-09-13T16:48:02.597439+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:02.597439+0000","last_undegraded":"2024-09-13T16:48:02.597439+0000","last_fullsized":"2024-09-13T16:48:02.597439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734245+0000","last_change":"2024-09-13T16:48:06.734245+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734245+0000","last_undegraded":"2024-09-13T16:48:06.734245+0000","last_fullsized":"2024-09-13T16:48:06.734245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597783+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:02.597783+0000","last_peered":"2024-09-13T16:48:02.597783+0000","last_clean":"2024-09-13T16:48:02.597783+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:02.597783+0000","last_undegraded":"2024-09-13T16:48:02.597783+0000","last_fullsized":"2024-09-13T16:48:02.597783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593373+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:02.593373+0000","last_peered":"2024-09-13T16:48:02.593373+0000","last_clean":"2024-09-13T16:48:02.593373+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:02.593373+0000","last_undegraded":"2024-09-13T16:48:02.593373+0000","last_fullsized":"2024-09-13T16:48:02.593373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736301+0000","last_change":"2024-09-13T16:48:06.736301+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736301+0000","last_undegraded":"2024-09-13T16:48:06.736301+0000","last_fullsized":"2024-09-13T16:48:06.736301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589987+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:02.589987+0000","last_peered":"2024-09-13T16:48:02.589987+0000","last_clean":"2024-09-13T16:48:02.589987+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:02.589987+0000","last_undegraded":"2024-09-13T16:48:02.589987+0000","last_fullsized":"2024-09-13T16:48:02.589987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593727+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:02.593727+0000","last_peered":"2024-09-13T16:48:02.593727+0000","last_clean":"2024-09-13T16:48:02.593727+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:02.593727+0000","last_undegraded":"2024-09-13T16:48:02.593727+0000","last_fullsized":"2024-09-13T16:48:02.593727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.735294+0000","last_change":"2024-09-13T16:48:06.735294+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.735294+0000","last_undegraded":"2024-09-13T16:48:06.735294+0000","last_fullsized":"2024-09-13T16:48:06.735294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597550+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:02.597550+0000","last_peered":"2024-09-13T16:48:02.597550+0000","last_clean":"2024-09-13T16:48:02.597550+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:02.597550+0000","last_undegraded":"2024-09-13T16:48:02.597550+0000","last_fullsized":"2024-09-13T16:48:02.597550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594051+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:02.594051+0000","last_peered":"2024-09-13T16:48:02.594051+0000","last_clean":"2024-09-13T16:48:02.594051+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:02.594051+0000","last_undegraded":"2024-09-13T16:48:02.594051+0000","last_fullsized":"2024-09-13T16:48:02.594051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.741644+0000","last_change":"2024-09-13T16:48:06.741644+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.741644+0000","last_undegraded":"2024-09-13T16:48:06.741644+0000","last_fullsized":"2024-09-13T16:48:06.741644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594026+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:02.594026+0000","last_peered":"2024-09-13T16:48:02.594026+0000","last_clean":"2024-09-13T16:48:02.594026+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:02.594026+0000","last_undegraded":"2024-09-13T16:48:02.594026+0000","last_fullsized":"2024-09-13T16:48:02.594026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593473+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:02.593473+0000","last_peered":"2024-09-13T16:48:02.593473+0000","last_clean":"2024-09-13T16:48:02.593473+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:02.593473+0000","last_undegraded":"2024-09-13T16:48:02.593473+0000","last_fullsized":"2024-09-13T16:48:02.593473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.739066+0000","last_change":"2024-09-13T16:48:06.739066+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.739066+0000","last_undegraded":"2024-09-13T16:48:06.739066+0000","last_fullsized":"2024-09-13T16:48:06.739066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.799964+0000","last_change":"2024-09-13T16:48:06.799964+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.799964+0000","last_undegraded":"2024-09-13T16:48:06.799964+0000","last_fullsized":"2024-09-13T16:48:06.799964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593364+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:02.593364+0000","last_peered":"2024-09-13T16:48:02.593364+0000","last_clean":"2024-09-13T16:48:02.593364+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:02.593364+0000","last_undegraded":"2024-09-13T16:48:02.593364+0000","last_fullsized":"2024-09-13T16:48:02.593364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593710+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:02.593710+0000","last_peered":"2024-09-13T16:48:02.593710+0000","last_clean":"2024-09-13T16:48:02.593710+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:02.593710+0000","last_undegraded":"2024-09-13T16:48:02.593710+0000","last_fullsized":"2024-09-13T16:48:02.593710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.740672+0000","last_change":"2024-09-13T16:48:06.740672+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.740672+0000","last_undegraded":"2024-09-13T16:48:06.740672+0000","last_fullsized":"2024-09-13T16:48:06.740672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590575+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:02.590575+0000","last_peered":"2024-09-13T16:48:02.590575+0000","last_clean":"2024-09-13T16:48:02.590575+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:02.590575+0000","last_undegraded":"2024-09-13T16:48:02.590575+0000","last_fullsized":"2024-09-13T16:48:02.590575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590651+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:02.590651+0000","last_peered":"2024-09-13T16:48:02.590651+0000","last_clean":"2024-09-13T16:48:02.590651+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:02.590651+0000","last_undegraded":"2024-09-13T16:48:02.590651+0000","last_fullsized":"2024-09-13T16:48:02.590651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593304+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:02.593304+0000","last_peered":"2024-09-13T16:48:02.593304+0000","last_clean":"2024-09-13T16:48:02.593304+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:02.593304+0000","last_undegraded":"2024-09-13T16:48:02.593304+0000","last_fullsized":"2024-09-13T16:48:02.593304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597416+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:02.597416+0000","last_peered":"2024-09-13T16:48:02.597416+0000","last_clean":"2024-09-13T16:48:02.597416+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:02.597416+0000","last_undegraded":"2024-09-13T16:48:02.597416+0000","last_fullsized":"2024-09-13T16:48:02.597416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738471+0000","last_change":"2024-09-13T16:48:06.738471+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738471+0000","last_undegraded":"2024-09-13T16:48:06.738471+0000","last_fullsized":"2024-09-13T16:48:06.738471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590769+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:02.590769+0000","last_peered":"2024-09-13T16:48:02.590769+0000","last_clean":"2024-09-13T16:48:02.590769+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:02.590769+0000","last_undegraded":"2024-09-13T16:48:02.590769+0000","last_fullsized":"2024-09-13T16:48:02.590769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589916+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:02.589916+0000","last_peered":"2024-09-13T16:48:02.589916+0000","last_clean":"2024-09-13T16:48:02.589916+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:02.589916+0000","last_undegraded":"2024-09-13T16:48:02.589916+0000","last_fullsized":"2024-09-13T16:48:02.589916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.739561+0000","last_change":"2024-09-13T16:48:06.739561+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.739561+0000","last_undegraded":"2024-09-13T16:48:06.739561+0000","last_fullsized":"2024-09-13T16:48:06.739561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593189+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:02.593189+0000","last_peered":"2024-09-13T16:48:02.593189+0000","last_clean":"2024-09-13T16:48:02.593189+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:02.593189+0000","last_undegraded":"2024-09-13T16:48:02.593189+0000","last_fullsized":"2024-09-13T16:48:02.593189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594144+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:02.594144+0000","last_peered":"2024-09-13T16:48:02.594144+0000","last_clean":"2024-09-13T16:48:02.594144+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:02.594144+0000","last_undegraded":"2024-09-13T16:48:02.594144+0000","last_fullsized":"2024-09-13T16:48:02.594144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593798+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:02.593798+0000","last_peered":"2024-09-13T16:48:02.593798+0000","last_clean":"2024-09-13T16:48:02.593798+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:02.593798+0000","last_undegraded":"2024-09-13T16:48:02.593798+0000","last_fullsized":"2024-09-13T16:48:02.593798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597394+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:02.597394+0000","last_peered":"2024-09-13T16:48:02.597394+0000","last_clean":"2024-09-13T16:48:02.597394+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:02.597394+0000","last_undegraded":"2024-09-13T16:48:02.597394+0000","last_fullsized":"2024-09-13T16:48:02.597394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590798+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:02.590798+0000","last_peered":"2024-09-13T16:48:02.590798+0000","last_clean":"2024-09-13T16:48:02.590798+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:02.590798+0000","last_undegraded":"2024-09-13T16:48:02.590798+0000","last_fullsized":"2024-09-13T16:48:02.590798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593678+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:02.593678+0000","last_peered":"2024-09-13T16:48:02.593678+0000","last_clean":"2024-09-13T16:48:02.593678+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:02.593678+0000","last_undegraded":"2024-09-13T16:48:02.593678+0000","last_fullsized":"2024-09-13T16:48:02.593678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593451+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:02.593451+0000","last_peered":"2024-09-13T16:48:02.593451+0000","last_clean":"2024-09-13T16:48:02.593451+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:02.593451+0000","last_undegraded":"2024-09-13T16:48:02.593451+0000","last_fullsized":"2024-09-13T16:48:02.593451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590871+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:02.590871+0000","last_peered":"2024-09-13T16:48:02.590871+0000","last_clean":"2024-09-13T16:48:02.590871+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:02.590871+0000","last_undegraded":"2024-09-13T16:48:02.590871+0000","last_fullsized":"2024-09-13T16:48:02.590871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593682+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:02.593682+0000","last_peered":"2024-09-13T16:48:02.593682+0000","last_clean":"2024-09-13T16:48:02.593682+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:02.593682+0000","last_undegraded":"2024-09-13T16:48:02.593682+0000","last_fullsized":"2024-09-13T16:48:02.593682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590624+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:02.590624+0000","last_peered":"2024-09-13T16:48:02.590624+0000","last_clean":"2024-09-13T16:48:02.590624+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:02.590624+0000","last_undegraded":"2024-09-13T16:48:02.590624+0000","last_fullsized":"2024-09-13T16:48:02.590624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593760+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:02.593760+0000","last_peered":"2024-09-13T16:48:02.593760+0000","last_clean":"2024-09-13T16:48:02.593760+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:02.593760+0000","last_undegraded":"2024-09-13T16:48:02.593760+0000","last_fullsized":"2024-09-13T16:48:02.593760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597369+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:02.597369+0000","last_peered":"2024-09-13T16:48:02.597369+0000","last_clean":"2024-09-13T16:48:02.597369+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:02.597369+0000","last_undegraded":"2024-09-13T16:48:02.597369+0000","last_fullsized":"2024-09-13T16:48:02.597369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593328+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:02.593328+0000","last_peered":"2024-09-13T16:48:02.593328+0000","last_clean":"2024-09-13T16:48:02.593328+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:02.593328+0000","last_undegraded":"2024-09-13T16:48:02.593328+0000","last_fullsized":"2024-09-13T16:48:02.593328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597853+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:02.597853+0000","last_peered":"2024-09-13T16:48:02.597853+0000","last_clean":"2024-09-13T16:48:02.597853+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:02.597853+0000","last_undegraded":"2024-09-13T16:48:02.597853+0000","last_fullsized":"2024-09-13T16:48:02.597853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594286+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:02.594286+0000","last_peered":"2024-09-13T16:48:02.594286+0000","last_clean":"2024-09-13T16:48:02.594286+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:02.594286+0000","last_undegraded":"2024-09-13T16:48:02.594286+0000","last_fullsized":"2024-09-13T16:48:02.594286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594076+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:02.594076+0000","last_peered":"2024-09-13T16:48:02.594076+0000","last_clean":"2024-09-13T16:48:02.594076+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:02.594076+0000","last_undegraded":"2024-09-13T16:48:02.594076+0000","last_fullsized":"2024-09-13T16:48:02.594076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593650+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:02.593650+0000","last_peered":"2024-09-13T16:48:02.593650+0000","last_clean":"2024-09-13T16:48:02.593650+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:02.593650+0000","last_undegraded":"2024-09-13T16:48:02.593650+0000","last_fullsized":"2024-09-13T16:48:02.593650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593413+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:02.593413+0000","last_peered":"2024-09-13T16:48:02.593413+0000","last_clean":"2024-09-13T16:48:02.593413+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:02.593413+0000","last_undegraded":"2024-09-13T16:48:02.593413+0000","last_fullsized":"2024-09-13T16:48:02.593413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597286+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:02.597286+0000","last_peered":"2024-09-13T16:48:02.597286+0000","last_clean":"2024-09-13T16:48:02.597286+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:02.597286+0000","last_undegraded":"2024-09-13T16:48:02.597286+0000","last_fullsized":"2024-09-13T16:48:02.597286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590698+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:02.590698+0000","last_peered":"2024-09-13T16:48:02.590698+0000","last_clean":"2024-09-13T16:48:02.590698+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:02.590698+0000","last_undegraded":"2024-09-13T16:48:02.590698+0000","last_fullsized":"2024-09-13T16:48:02.590698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589962+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:02.589962+0000","last_peered":"2024-09-13T16:48:02.589962+0000","last_clean":"2024-09-13T16:48:02.589962+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:02.589962+0000","last_undegraded":"2024-09-13T16:48:02.589962+0000","last_fullsized":"2024-09-13T16:48:02.589962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593397+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:02.593397+0000","last_peered":"2024-09-13T16:48:02.593397+0000","last_clean":"2024-09-13T16:48:02.593397+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:02.593397+0000","last_undegraded":"2024-09-13T16:48:02.593397+0000","last_fullsized":"2024-09-13T16:48:02.593397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593423+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:02.593423+0000","last_peered":"2024-09-13T16:48:02.593423+0000","last_clean":"2024-09-13T16:48:02.593423+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:02.593423+0000","last_undegraded":"2024-09-13T16:48:02.593423+0000","last_fullsized":"2024-09-13T16:48:02.593423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597957+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:02.597957+0000","last_peered":"2024-09-13T16:48:02.597957+0000","last_clean":"2024-09-13T16:48:02.597957+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:02.597957+0000","last_undegraded":"2024-09-13T16:48:02.597957+0000","last_fullsized":"2024-09-13T16:48:02.597957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590539+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:02.590539+0000","last_peered":"2024-09-13T16:48:02.590539+0000","last_clean":"2024-09-13T16:48:02.590539+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:02.590539+0000","last_undegraded":"2024-09-13T16:48:02.590539+0000","last_fullsized":"2024-09-13T16:48:02.590539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597624+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:02.597624+0000","last_peered":"2024-09-13T16:48:02.597624+0000","last_clean":"2024-09-13T16:48:02.597624+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:02.597624+0000","last_undegraded":"2024-09-13T16:48:02.597624+0000","last_fullsized":"2024-09-13T16:48:02.597624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593732+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:02.593732+0000","last_peered":"2024-09-13T16:48:02.593732+0000","last_clean":"2024-09-13T16:48:02.593732+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:02.593732+0000","last_undegraded":"2024-09-13T16:48:02.593732+0000","last_fullsized":"2024-09-13T16:48:02.593732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597669+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:02.597669+0000","last_peered":"2024-09-13T16:48:02.597669+0000","last_clean":"2024-09-13T16:48:02.597669+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:02.597669+0000","last_undegraded":"2024-09-13T16:48:02.597669+0000","last_fullsized":"2024-09-13T16:48:02.597669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593894+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:02.593894+0000","last_peered":"2024-09-13T16:48:02.593894+0000","last_clean":"2024-09-13T16:48:02.593894+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:02.593894+0000","last_undegraded":"2024-09-13T16:48:02.593894+0000","last_fullsized":"2024-09-13T16:48:02.593894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597466+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:02.597466+0000","last_peered":"2024-09-13T16:48:02.597466+0000","last_clean":"2024-09-13T16:48:02.597466+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:02.597466+0000","last_undegraded":"2024-09-13T16:48:02.597466+0000","last_fullsized":"2024-09-13T16:48:02.597466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593852+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:02.593852+0000","last_peered":"2024-09-13T16:48:02.593852+0000","last_clean":"2024-09-13T16:48:02.593852+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:02.593852+0000","last_undegraded":"2024-09-13T16:48:02.593852+0000","last_fullsized":"2024-09-13T16:48:02.593852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597930+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:02.597930+0000","last_peered":"2024-09-13T16:48:02.597930+0000","last_clean":"2024-09-13T16:48:02.597930+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:02.597930+0000","last_undegraded":"2024-09-13T16:48:02.597930+0000","last_fullsized":"2024-09-13T16:48:02.597930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":668407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":77547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":79021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":77457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":212992,"data_stored":70978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":661824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:13.212 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.360+0000 7f4d2cf2c640 1 Processor -- start 2024-09-13T16:48:13.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.360+0000 7f4d2cf2c640 1 -- start start 2024-09-13T16:48:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d2cf2c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4d2814e0c0 0x7f4d280a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d2cf2c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d2806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d2cf2c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d280c2780 con 0x7f4d2806d300 2024-09-13T16:48:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d2cf2c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d280c2a60 con 0x7f4d2814e0c0 2024-09-13T16:48:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d2cf2c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d280c2d40 con 0x7f4d280c1f50 2024-09-13T16:48:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d25d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d2806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d2659c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4d2814e0c0 0x7f4d280a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d25d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d2806cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45238/0 (socket says 172.21.15.39:45238) 2024-09-13T16:48:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d25d9b640 1 -- 172.21.15.39:0/2313200774 learned_addr learned my addr 172.21.15.39:0/2313200774 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d26d9d640 1 --1- 172.21.15.39:0/2313200774 >> v1:172.21.15.84:6789/0 conn(0x7f4d2806d300 0x7f4d280c4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:44244/0 (socket says 172.21.15.39:44244) 2024-09-13T16:48:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d25d9b640 1 -- 172.21.15.39:0/2313200774 >> v2:172.21.15.84:3300/0 conn(0x7f4d2814e0c0 msgr2=0x7f4d280a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d25d9b640 1 --2- 172.21.15.39:0/2313200774 >> v2:172.21.15.84:3300/0 conn(0x7f4d2814e0c0 0x7f4d280a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d25d9b640 1 -- 172.21.15.39:0/2313200774 >> v1:172.21.15.84:6789/0 conn(0x7f4d2806d300 legacy=0x7f4d280c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d25d9b640 1 -- 172.21.15.39:0/2313200774 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280c3020 con 0x7f4d280c1f50 2024-09-13T16:48:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d2659c640 1 --2- 172.21.15.39:0/2313200774 >> v2:172.21.15.84:3300/0 conn(0x7f4d2814e0c0 0x7f4d280a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.364+0000 7f4d25d9b640 1 --2- 172.21.15.39:0/2313200774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d2806cdc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009e50 tx=0x7f4d1801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f64bac2c6afe5923 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2559a640 1 -- 172.21.15.39:0/2313200774 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d180a5750 con 0x7f4d280c1f50 2024-09-13T16:48:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2559a640 1 -- 172.21.15.39:0/2313200774 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d180a58b0 con 0x7f4d280c1f50 2024-09-13T16:48:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2559a640 1 -- 172.21.15.39:0/2313200774 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d180a5c40 con 0x7f4d280c1f50 2024-09-13T16:48:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/2313200774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 msgr2=0x7f4d2806cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 --2- 172.21.15.39:0/2313200774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d2806cdc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009e50 tx=0x7f4d1801c970 comp rx=0 tx=0).stop 2024-09-13T16:48:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/2313200774 shutdown_connections 2024-09-13T16:48:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 --2- 172.21.15.39:0/2313200774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d2806cdc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 --2- 172.21.15.39:0/2313200774 >> v2:172.21.15.84:3300/0 conn(0x7f4d2814e0c0 0x7f4d280a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/2313200774 >> 172.21.15.39:0/2313200774 conn(0x7f4d28069e10 msgr2=0x7f4d2806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/2313200774 shutdown_connections 2024-09-13T16:48:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/2313200774 wait complete. 2024-09-13T16:48:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 Processor -- start 2024-09-13T16:48:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- start start 2024-09-13T16:48:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2806d300 0x7f4d2814c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d280aad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d2814e0c0 0x7f4d2815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d28144130 con 0x7f4d2806d300 2024-09-13T16:48:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d28145a00 con 0x7f4d2814e0c0 2024-09-13T16:48:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d28151dc0 con 0x7f4d280c1f50 2024-09-13T16:48:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2806d300 0x7f4d2814c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2806d300 0x7f4d2814c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34264/0 (socket says 172.21.15.39:34264) 2024-09-13T16:48:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2659c640 1 -- 172.21.15.39:0/3353501308 learned_addr learned my addr 172.21.15.39:0/3353501308 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d25d9b640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d280aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d26d9d640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d2814e0c0 0x7f4d2815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2659c640 1 -- 172.21.15.39:0/3353501308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 msgr2=0x7f4d280aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2659c640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d280aad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2659c640 1 -- 172.21.15.39:0/3353501308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d2814e0c0 msgr2=0x7f4d2815a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2659c640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d2814e0c0 0x7f4d2815a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2659c640 1 -- 172.21.15.39:0/3353501308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c009420 con 0x7f4d2806d300 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d25d9b640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d280aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2659c640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2806d300 0x7f4d2814c300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c00e9b0 tx=0x7f4d1c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d16ffd640 1 -- 172.21.15.39:0/3353501308 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00ca00 con 0x7f4d2806d300 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d16ffd640 1 -- 172.21.15.39:0/3353501308 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c00cb60 con 0x7f4d2806d300 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d16ffd640 1 -- 172.21.15.39:0/3353501308 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c013470 con 0x7f4d2806d300 2024-09-13T16:48:13.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d18009b30 con 0x7f4d2806d300 2024-09-13T16:48:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.368+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2814d1f0 con 0x7f4d2806d300 2024-09-13T16:48:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.372+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2806cdc0 con 0x7f4d2806d300 2024-09-13T16:48:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.372+0000 7f4d16ffd640 1 -- 172.21.15.39:0/3353501308 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4d1c0135d0 con 0x7f4d2806d300 2024-09-13T16:48:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.372+0000 7f4d16ffd640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d08066690 0x7f4d08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.372+0000 7f4d16ffd640 1 -- 172.21.15.39:0/3353501308 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f4d1c084750 con 0x7f4d2806d300 2024-09-13T16:48:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.372+0000 7f4d25d9b640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d08066690 0x7f4d08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.372+0000 7f4d25d9b640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d08066690 0x7f4d08068b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4d180ae000 tx=0x7f4d180a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.376+0000 7f4d16ffd640 1 -- 172.21.15.39:0/3353501308 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4d1c051780 con 0x7f4d2806d300 2024-09-13T16:48:13.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.528+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d2815dae0 con 0x7f4d08066690 2024-09-13T16:48:13.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.544+0000 7f4d16ffd640 1 -- 172.21.15.39:0/3353501308 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+492189 (secure 0 0 0) 0x7f4d180bc090 con 0x7f4d08066690 2024-09-13T16:48:13.553 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:13.556 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:13.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d08066690 msgr2=0x7f4d08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d08066690 0x7f4d08068b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4d180ae000 tx=0x7f4d180a7070 comp rx=0 tx=0).stop 2024-09-13T16:48:13.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2806d300 msgr2=0x7f4d2814c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2806d300 0x7f4d2814c300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c00e9b0 tx=0x7f4d1c00ee70 comp rx=0 tx=0).stop 2024-09-13T16:48:13.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 shutdown_connections 2024-09-13T16:48:13.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d08066690 0x7f4d08068b50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d2814e0c0 0x7f4d2815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d280c1f50 0x7f4d280aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 --2- 172.21.15.39:0/3353501308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2806d300 0x7f4d2814c300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 >> 172.21.15.39:0/3353501308 conn(0x7f4d28069e10 msgr2=0x7f4d280bf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:13.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 shutdown_connections 2024-09-13T16:48:13.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.548+0000 7f4d2cf2c640 1 -- 172.21.15.39:0/3353501308 wait complete. 2024-09-13T16:48:13.569 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-13T16:48:11.561214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235428,"kb_used_data":2884,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739292,"statfs":{"total":773094113280,"available":772853035008,"internally_reserved":0,"allocated":2953216,"data_stored":1785199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.832843"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597573+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:02.597573+0000","last_peered":"2024-09-13T16:48:02.597573+0000","last_clean":"2024-09-13T16:48:02.597573+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:02.597573+0000","last_undegraded":"2024-09-13T16:48:02.597573+0000","last_fullsized":"2024-09-13T16:48:02.597573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590824+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:02.590824+0000","last_peered":"2024-09-13T16:48:02.590824+0000","last_clean":"2024-09-13T16:48:02.590824+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:02.590824+0000","last_undegraded":"2024-09-13T16:48:02.590824+0000","last_fullsized":"2024-09-13T16:48:02.590824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738264+0000","last_change":"2024-09-13T16:48:06.738264+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738264+0000","last_undegraded":"2024-09-13T16:48:06.738264+0000","last_fullsized":"2024-09-13T16:48:06.738264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590848+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:02.590848+0000","last_peered":"2024-09-13T16:48:02.590848+0000","last_clean":"2024-09-13T16:48:02.590848+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:02.590848+0000","last_undegraded":"2024-09-13T16:48:02.590848+0000","last_fullsized":"2024-09-13T16:48:02.590848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597512+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:02.597512+0000","last_peered":"2024-09-13T16:48:02.597512+0000","last_clean":"2024-09-13T16:48:02.597512+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:02.597512+0000","last_undegraded":"2024-09-13T16:48:02.597512+0000","last_fullsized":"2024-09-13T16:48:02.597512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734168+0000","last_change":"2024-09-13T16:48:06.734168+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734168+0000","last_undegraded":"2024-09-13T16:48:06.734168+0000","last_fullsized":"2024-09-13T16:48:06.734168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597829+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:02.597829+0000","last_peered":"2024-09-13T16:48:02.597829+0000","last_clean":"2024-09-13T16:48:02.597829+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:02.597829+0000","last_undegraded":"2024-09-13T16:48:02.597829+0000","last_fullsized":"2024-09-13T16:48:02.597829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593935+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:02.593935+0000","last_peered":"2024-09-13T16:48:02.593935+0000","last_clean":"2024-09-13T16:48:02.593935+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:02.593935+0000","last_undegraded":"2024-09-13T16:48:02.593935+0000","last_fullsized":"2024-09-13T16:48:02.593935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736405+0000","last_change":"2024-09-13T16:48:06.736405+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736405+0000","last_undegraded":"2024-09-13T16:48:06.736405+0000","last_fullsized":"2024-09-13T16:48:06.736405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597754+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:02.597754+0000","last_peered":"2024-09-13T16:48:02.597754+0000","last_clean":"2024-09-13T16:48:02.597754+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:02.597754+0000","last_undegraded":"2024-09-13T16:48:02.597754+0000","last_fullsized":"2024-09-13T16:48:02.597754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593979+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:02.593979+0000","last_peered":"2024-09-13T16:48:02.593979+0000","last_clean":"2024-09-13T16:48:02.593979+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:02.593979+0000","last_undegraded":"2024-09-13T16:48:02.593979+0000","last_fullsized":"2024-09-13T16:48:02.593979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734117+0000","last_change":"2024-09-13T16:48:06.734117+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734117+0000","last_undegraded":"2024-09-13T16:48:06.734117+0000","last_fullsized":"2024-09-13T16:48:06.734117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593280+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:02.593280+0000","last_peered":"2024-09-13T16:48:02.593280+0000","last_clean":"2024-09-13T16:48:02.593280+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:02.593280+0000","last_undegraded":"2024-09-13T16:48:02.593280+0000","last_fullsized":"2024-09-13T16:48:02.593280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597729+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:02.597729+0000","last_peered":"2024-09-13T16:48:02.597729+0000","last_clean":"2024-09-13T16:48:02.597729+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:02.597729+0000","last_undegraded":"2024-09-13T16:48:02.597729+0000","last_fullsized":"2024-09-13T16:48:02.597729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.735578+0000","last_change":"2024-09-13T16:48:06.735578+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.735578+0000","last_undegraded":"2024-09-13T16:48:06.735578+0000","last_fullsized":"2024-09-13T16:48:06.735578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590720+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:02.590720+0000","last_peered":"2024-09-13T16:48:02.590720+0000","last_clean":"2024-09-13T16:48:02.590720+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:02.590720+0000","last_undegraded":"2024-09-13T16:48:02.590720+0000","last_fullsized":"2024-09-13T16:48:02.590720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593524+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:02.593524+0000","last_peered":"2024-09-13T16:48:02.593524+0000","last_clean":"2024-09-13T16:48:02.593524+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:02.593524+0000","last_undegraded":"2024-09-13T16:48:02.593524+0000","last_fullsized":"2024-09-13T16:48:02.593524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734251+0000","last_change":"2024-09-13T16:48:06.734251+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734251+0000","last_undegraded":"2024-09-13T16:48:06.734251+0000","last_fullsized":"2024-09-13T16:48:06.734251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590600+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:02.590600+0000","last_peered":"2024-09-13T16:48:02.590600+0000","last_clean":"2024-09-13T16:48:02.590600+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:02.590600+0000","last_undegraded":"2024-09-13T16:48:02.590600+0000","last_fullsized":"2024-09-13T16:48:02.590600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593478+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:02.593478+0000","last_peered":"2024-09-13T16:48:02.593478+0000","last_clean":"2024-09-13T16:48:02.593478+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:02.593478+0000","last_undegraded":"2024-09-13T16:48:02.593478+0000","last_fullsized":"2024-09-13T16:48:02.593478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736068+0000","last_change":"2024-09-13T16:48:06.736068+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736068+0000","last_undegraded":"2024-09-13T16:48:06.736068+0000","last_fullsized":"2024-09-13T16:48:06.736068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589876+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:02.589876+0000","last_peered":"2024-09-13T16:48:02.589876+0000","last_clean":"2024-09-13T16:48:02.589876+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:02.589876+0000","last_undegraded":"2024-09-13T16:48:02.589876+0000","last_fullsized":"2024-09-13T16:48:02.589876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597549+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:02.597549+0000","last_peered":"2024-09-13T16:48:02.597549+0000","last_clean":"2024-09-13T16:48:02.597549+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:02.597549+0000","last_undegraded":"2024-09-13T16:48:02.597549+0000","last_fullsized":"2024-09-13T16:48:02.597549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738308+0000","last_change":"2024-09-13T16:48:06.738308+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738308+0000","last_undegraded":"2024-09-13T16:48:06.738308+0000","last_fullsized":"2024-09-13T16:48:06.738308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597488+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:02.597488+0000","last_peered":"2024-09-13T16:48:02.597488+0000","last_clean":"2024-09-13T16:48:02.597488+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:02.597488+0000","last_undegraded":"2024-09-13T16:48:02.597488+0000","last_fullsized":"2024-09-13T16:48:02.597488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593499+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:02.593499+0000","last_peered":"2024-09-13T16:48:02.593499+0000","last_clean":"2024-09-13T16:48:02.593499+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:02.593499+0000","last_undegraded":"2024-09-13T16:48:02.593499+0000","last_fullsized":"2024-09-13T16:48:02.593499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736950+0000","last_change":"2024-09-13T16:48:06.736950+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736950+0000","last_undegraded":"2024-09-13T16:48:06.736950+0000","last_fullsized":"2024-09-13T16:48:06.736950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590892+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:02.590892+0000","last_peered":"2024-09-13T16:48:02.590892+0000","last_clean":"2024-09-13T16:48:02.590892+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:02.590892+0000","last_undegraded":"2024-09-13T16:48:02.590892+0000","last_fullsized":"2024-09-13T16:48:02.590892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:03.216479+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:03.216479+0000","last_peered":"2024-09-13T16:48:03.216479+0000","last_clean":"2024-09-13T16:48:03.216479+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:03.216479+0000","last_undegraded":"2024-09-13T16:48:03.216479+0000","last_fullsized":"2024-09-13T16:48:03.216479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738588+0000","last_change":"2024-09-13T16:48:06.738588+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738588+0000","last_undegraded":"2024-09-13T16:48:06.738588+0000","last_fullsized":"2024-09-13T16:48:06.738588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593591+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:02.593591+0000","last_peered":"2024-09-13T16:48:02.593591+0000","last_clean":"2024-09-13T16:48:02.593591+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:02.593591+0000","last_undegraded":"2024-09-13T16:48:02.593591+0000","last_fullsized":"2024-09-13T16:48:02.593591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597874+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:02.597874+0000","last_peered":"2024-09-13T16:48:02.597874+0000","last_clean":"2024-09-13T16:48:02.597874+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:02.597874+0000","last_undegraded":"2024-09-13T16:48:02.597874+0000","last_fullsized":"2024-09-13T16:48:02.597874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.798573+0000","last_change":"2024-09-13T16:48:06.798573+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.798573+0000","last_undegraded":"2024-09-13T16:48:06.798573+0000","last_fullsized":"2024-09-13T16:48:06.798573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594000+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:02.594000+0000","last_peered":"2024-09-13T16:48:02.594000+0000","last_clean":"2024-09-13T16:48:02.594000+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:02.594000+0000","last_undegraded":"2024-09-13T16:48:02.594000+0000","last_fullsized":"2024-09-13T16:48:02.594000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593637+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:02.593637+0000","last_peered":"2024-09-13T16:48:02.593637+0000","last_clean":"2024-09-13T16:48:02.593637+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:02.593637+0000","last_undegraded":"2024-09-13T16:48:02.593637+0000","last_fullsized":"2024-09-13T16:48:02.593637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734599+0000","last_change":"2024-09-13T16:48:06.734599+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734599+0000","last_undegraded":"2024-09-13T16:48:06.734599+0000","last_fullsized":"2024-09-13T16:48:06.734599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589940+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:02.589940+0000","last_peered":"2024-09-13T16:48:02.589940+0000","last_clean":"2024-09-13T16:48:02.589940+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:02.589940+0000","last_undegraded":"2024-09-13T16:48:02.589940+0000","last_fullsized":"2024-09-13T16:48:02.589940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593847+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:02.593847+0000","last_peered":"2024-09-13T16:48:02.593847+0000","last_clean":"2024-09-13T16:48:02.593847+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:02.593847+0000","last_undegraded":"2024-09-13T16:48:02.593847+0000","last_fullsized":"2024-09-13T16:48:02.593847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734532+0000","last_change":"2024-09-13T16:48:06.734532+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734532+0000","last_undegraded":"2024-09-13T16:48:06.734532+0000","last_fullsized":"2024-09-13T16:48:06.734532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593308+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:02.593308+0000","last_peered":"2024-09-13T16:48:02.593308+0000","last_clean":"2024-09-13T16:48:02.593308+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:02.593308+0000","last_undegraded":"2024-09-13T16:48:02.593308+0000","last_fullsized":"2024-09-13T16:48:02.593308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594186+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:02.594186+0000","last_peered":"2024-09-13T16:48:02.594186+0000","last_clean":"2024-09-13T16:48:02.594186+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:02.594186+0000","last_undegraded":"2024-09-13T16:48:02.594186+0000","last_fullsized":"2024-09-13T16:48:02.594186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.735058+0000","last_change":"2024-09-13T16:48:06.735058+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.735058+0000","last_undegraded":"2024-09-13T16:48:06.735058+0000","last_fullsized":"2024-09-13T16:48:06.735058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597220+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:02.597220+0000","last_peered":"2024-09-13T16:48:02.597220+0000","last_clean":"2024-09-13T16:48:02.597220+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:02.597220+0000","last_undegraded":"2024-09-13T16:48:02.597220+0000","last_fullsized":"2024-09-13T16:48:02.597220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597578+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:02.597578+0000","last_peered":"2024-09-13T16:48:02.597578+0000","last_clean":"2024-09-13T16:48:02.597578+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:02.597578+0000","last_undegraded":"2024-09-13T16:48:02.597578+0000","last_fullsized":"2024-09-13T16:48:02.597578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.799225+0000","last_change":"2024-09-13T16:48:06.799225+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.799225+0000","last_undegraded":"2024-09-13T16:48:06.799225+0000","last_fullsized":"2024-09-13T16:48:06.799225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590676+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:02.590676+0000","last_peered":"2024-09-13T16:48:02.590676+0000","last_clean":"2024-09-13T16:48:02.590676+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:02.590676+0000","last_undegraded":"2024-09-13T16:48:02.590676+0000","last_fullsized":"2024-09-13T16:48:02.590676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593871+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:02.593871+0000","last_peered":"2024-09-13T16:48:02.593871+0000","last_clean":"2024-09-13T16:48:02.593871+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:02.593871+0000","last_undegraded":"2024-09-13T16:48:02.593871+0000","last_fullsized":"2024-09-13T16:48:02.593871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:00.263443+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:00.263443+0000","last_peered":"2024-09-13T16:48:00.263443+0000","last_clean":"2024-09-13T16:48:00.263443+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:00.263443+0000","last_undegraded":"2024-09-13T16:48:00.263443+0000","last_fullsized":"2024-09-13T16:48:00.263443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.740802+0000","last_change":"2024-09-13T16:48:06.740802+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.740802+0000","last_undegraded":"2024-09-13T16:48:06.740802+0000","last_fullsized":"2024-09-13T16:48:06.740802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597803+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:02.597803+0000","last_peered":"2024-09-13T16:48:02.597803+0000","last_clean":"2024-09-13T16:48:02.597803+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:02.597803+0000","last_undegraded":"2024-09-13T16:48:02.597803+0000","last_fullsized":"2024-09-13T16:48:02.597803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594240+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:02.594240+0000","last_peered":"2024-09-13T16:48:02.594240+0000","last_clean":"2024-09-13T16:48:02.594240+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:02.594240+0000","last_undegraded":"2024-09-13T16:48:02.594240+0000","last_fullsized":"2024-09-13T16:48:02.594240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.741059+0000","last_change":"2024-09-13T16:48:06.741059+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.741059+0000","last_undegraded":"2024-09-13T16:48:06.741059+0000","last_fullsized":"2024-09-13T16:48:06.741059+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593957+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:02.593957+0000","last_peered":"2024-09-13T16:48:02.593957+0000","last_clean":"2024-09-13T16:48:02.593957+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:02.593957+0000","last_undegraded":"2024-09-13T16:48:02.593957+0000","last_fullsized":"2024-09-13T16:48:02.593957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593802+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:02.593802+0000","last_peered":"2024-09-13T16:48:02.593802+0000","last_clean":"2024-09-13T16:48:02.593802+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:02.593802+0000","last_undegraded":"2024-09-13T16:48:02.593802+0000","last_fullsized":"2024-09-13T16:48:02.593802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.799237+0000","last_change":"2024-09-13T16:48:06.799237+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.799237+0000","last_undegraded":"2024-09-13T16:48:06.799237+0000","last_fullsized":"2024-09-13T16:48:06.799237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597984+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:02.597984+0000","last_peered":"2024-09-13T16:48:02.597984+0000","last_clean":"2024-09-13T16:48:02.597984+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:02.597984+0000","last_undegraded":"2024-09-13T16:48:02.597984+0000","last_fullsized":"2024-09-13T16:48:02.597984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594334+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:02.594334+0000","last_peered":"2024-09-13T16:48:02.594334+0000","last_clean":"2024-09-13T16:48:02.594334+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:02.594334+0000","last_undegraded":"2024-09-13T16:48:02.594334+0000","last_fullsized":"2024-09-13T16:48:02.594334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736498+0000","last_change":"2024-09-13T16:48:06.736498+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736498+0000","last_undegraded":"2024-09-13T16:48:06.736498+0000","last_fullsized":"2024-09-13T16:48:06.736498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597908+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:02.597908+0000","last_peered":"2024-09-13T16:48:02.597908+0000","last_clean":"2024-09-13T16:48:02.597908+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:02.597908+0000","last_undegraded":"2024-09-13T16:48:02.597908+0000","last_fullsized":"2024-09-13T16:48:02.597908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590065+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:02.590065+0000","last_peered":"2024-09-13T16:48:02.590065+0000","last_clean":"2024-09-13T16:48:02.590065+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:02.590065+0000","last_undegraded":"2024-09-13T16:48:02.590065+0000","last_fullsized":"2024-09-13T16:48:02.590065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738166+0000","last_change":"2024-09-13T16:48:06.738166+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738166+0000","last_undegraded":"2024-09-13T16:48:06.738166+0000","last_fullsized":"2024-09-13T16:48:06.738166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597646+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:02.597646+0000","last_peered":"2024-09-13T16:48:02.597646+0000","last_clean":"2024-09-13T16:48:02.597646+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:02.597646+0000","last_undegraded":"2024-09-13T16:48:02.597646+0000","last_fullsized":"2024-09-13T16:48:02.597646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594083+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:02.594083+0000","last_peered":"2024-09-13T16:48:02.594083+0000","last_clean":"2024-09-13T16:48:02.594083+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:02.594083+0000","last_undegraded":"2024-09-13T16:48:02.594083+0000","last_fullsized":"2024-09-13T16:48:02.594083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.798551+0000","last_change":"2024-09-13T16:48:06.798551+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.798551+0000","last_undegraded":"2024-09-13T16:48:06.798551+0000","last_fullsized":"2024-09-13T16:48:06.798551+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593229+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:02.593229+0000","last_peered":"2024-09-13T16:48:02.593229+0000","last_clean":"2024-09-13T16:48:02.593229+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:02.593229+0000","last_undegraded":"2024-09-13T16:48:02.593229+0000","last_fullsized":"2024-09-13T16:48:02.593229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593257+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:02.593257+0000","last_peered":"2024-09-13T16:48:02.593257+0000","last_clean":"2024-09-13T16:48:02.593257+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:02.593257+0000","last_undegraded":"2024-09-13T16:48:02.593257+0000","last_fullsized":"2024-09-13T16:48:02.593257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.739047+0000","last_change":"2024-09-13T16:48:06.739047+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.739047+0000","last_undegraded":"2024-09-13T16:48:06.739047+0000","last_fullsized":"2024-09-13T16:48:06.739047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597706+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:02.597706+0000","last_peered":"2024-09-13T16:48:02.597706+0000","last_clean":"2024-09-13T16:48:02.597706+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:02.597706+0000","last_undegraded":"2024-09-13T16:48:02.597706+0000","last_fullsized":"2024-09-13T16:48:02.597706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593823+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:02.593823+0000","last_peered":"2024-09-13T16:48:02.593823+0000","last_clean":"2024-09-13T16:48:02.593823+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:02.593823+0000","last_undegraded":"2024-09-13T16:48:02.593823+0000","last_fullsized":"2024-09-13T16:48:02.593823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.739744+0000","last_change":"2024-09-13T16:48:06.739744+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.739744+0000","last_undegraded":"2024-09-13T16:48:06.739744+0000","last_fullsized":"2024-09-13T16:48:06.739744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597339+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:02.597339+0000","last_peered":"2024-09-13T16:48:02.597339+0000","last_clean":"2024-09-13T16:48:02.597339+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:02.597339+0000","last_undegraded":"2024-09-13T16:48:02.597339+0000","last_fullsized":"2024-09-13T16:48:02.597339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597439+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:02.597439+0000","last_peered":"2024-09-13T16:48:02.597439+0000","last_clean":"2024-09-13T16:48:02.597439+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:02.597439+0000","last_undegraded":"2024-09-13T16:48:02.597439+0000","last_fullsized":"2024-09-13T16:48:02.597439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.734245+0000","last_change":"2024-09-13T16:48:06.734245+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.734245+0000","last_undegraded":"2024-09-13T16:48:06.734245+0000","last_fullsized":"2024-09-13T16:48:06.734245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597783+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:02.597783+0000","last_peered":"2024-09-13T16:48:02.597783+0000","last_clean":"2024-09-13T16:48:02.597783+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:02.597783+0000","last_undegraded":"2024-09-13T16:48:02.597783+0000","last_fullsized":"2024-09-13T16:48:02.597783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593373+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:02.593373+0000","last_peered":"2024-09-13T16:48:02.593373+0000","last_clean":"2024-09-13T16:48:02.593373+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:02.593373+0000","last_undegraded":"2024-09-13T16:48:02.593373+0000","last_fullsized":"2024-09-13T16:48:02.593373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.736301+0000","last_change":"2024-09-13T16:48:06.736301+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.736301+0000","last_undegraded":"2024-09-13T16:48:06.736301+0000","last_fullsized":"2024-09-13T16:48:06.736301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589987+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:02.589987+0000","last_peered":"2024-09-13T16:48:02.589987+0000","last_clean":"2024-09-13T16:48:02.589987+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:02.589987+0000","last_undegraded":"2024-09-13T16:48:02.589987+0000","last_fullsized":"2024-09-13T16:48:02.589987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593727+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:02.593727+0000","last_peered":"2024-09-13T16:48:02.593727+0000","last_clean":"2024-09-13T16:48:02.593727+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:02.593727+0000","last_undegraded":"2024-09-13T16:48:02.593727+0000","last_fullsized":"2024-09-13T16:48:02.593727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.735294+0000","last_change":"2024-09-13T16:48:06.735294+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.735294+0000","last_undegraded":"2024-09-13T16:48:06.735294+0000","last_fullsized":"2024-09-13T16:48:06.735294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597550+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:02.597550+0000","last_peered":"2024-09-13T16:48:02.597550+0000","last_clean":"2024-09-13T16:48:02.597550+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:02.597550+0000","last_undegraded":"2024-09-13T16:48:02.597550+0000","last_fullsized":"2024-09-13T16:48:02.597550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594051+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:02.594051+0000","last_peered":"2024-09-13T16:48:02.594051+0000","last_clean":"2024-09-13T16:48:02.594051+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:02.594051+0000","last_undegraded":"2024-09-13T16:48:02.594051+0000","last_fullsized":"2024-09-13T16:48:02.594051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.741644+0000","last_change":"2024-09-13T16:48:06.741644+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.741644+0000","last_undegraded":"2024-09-13T16:48:06.741644+0000","last_fullsized":"2024-09-13T16:48:06.741644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594026+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:02.594026+0000","last_peered":"2024-09-13T16:48:02.594026+0000","last_clean":"2024-09-13T16:48:02.594026+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:02.594026+0000","last_undegraded":"2024-09-13T16:48:02.594026+0000","last_fullsized":"2024-09-13T16:48:02.594026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593473+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:02.593473+0000","last_peered":"2024-09-13T16:48:02.593473+0000","last_clean":"2024-09-13T16:48:02.593473+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:02.593473+0000","last_undegraded":"2024-09-13T16:48:02.593473+0000","last_fullsized":"2024-09-13T16:48:02.593473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.739066+0000","last_change":"2024-09-13T16:48:06.739066+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.739066+0000","last_undegraded":"2024-09-13T16:48:06.739066+0000","last_fullsized":"2024-09-13T16:48:06.739066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.799964+0000","last_change":"2024-09-13T16:48:06.799964+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.799964+0000","last_undegraded":"2024-09-13T16:48:06.799964+0000","last_fullsized":"2024-09-13T16:48:06.799964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593364+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:02.593364+0000","last_peered":"2024-09-13T16:48:02.593364+0000","last_clean":"2024-09-13T16:48:02.593364+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:02.593364+0000","last_undegraded":"2024-09-13T16:48:02.593364+0000","last_fullsized":"2024-09-13T16:48:02.593364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593710+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:02.593710+0000","last_peered":"2024-09-13T16:48:02.593710+0000","last_clean":"2024-09-13T16:48:02.593710+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:02.593710+0000","last_undegraded":"2024-09-13T16:48:02.593710+0000","last_fullsized":"2024-09-13T16:48:02.593710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.740672+0000","last_change":"2024-09-13T16:48:06.740672+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.740672+0000","last_undegraded":"2024-09-13T16:48:06.740672+0000","last_fullsized":"2024-09-13T16:48:06.740672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590575+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:02.590575+0000","last_peered":"2024-09-13T16:48:02.590575+0000","last_clean":"2024-09-13T16:48:02.590575+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:02.590575+0000","last_undegraded":"2024-09-13T16:48:02.590575+0000","last_fullsized":"2024-09-13T16:48:02.590575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590651+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:02.590651+0000","last_peered":"2024-09-13T16:48:02.590651+0000","last_clean":"2024-09-13T16:48:02.590651+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:02.590651+0000","last_undegraded":"2024-09-13T16:48:02.590651+0000","last_fullsized":"2024-09-13T16:48:02.590651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593304+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:02.593304+0000","last_peered":"2024-09-13T16:48:02.593304+0000","last_clean":"2024-09-13T16:48:02.593304+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:02.593304+0000","last_undegraded":"2024-09-13T16:48:02.593304+0000","last_fullsized":"2024-09-13T16:48:02.593304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597416+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:02.597416+0000","last_peered":"2024-09-13T16:48:02.597416+0000","last_clean":"2024-09-13T16:48:02.597416+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:02.597416+0000","last_undegraded":"2024-09-13T16:48:02.597416+0000","last_fullsized":"2024-09-13T16:48:02.597416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.738471+0000","last_change":"2024-09-13T16:48:06.738471+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.738471+0000","last_undegraded":"2024-09-13T16:48:06.738471+0000","last_fullsized":"2024-09-13T16:48:06.738471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590769+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:02.590769+0000","last_peered":"2024-09-13T16:48:02.590769+0000","last_clean":"2024-09-13T16:48:02.590769+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:02.590769+0000","last_undegraded":"2024-09-13T16:48:02.590769+0000","last_fullsized":"2024-09-13T16:48:02.590769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589916+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:02.589916+0000","last_peered":"2024-09-13T16:48:02.589916+0000","last_clean":"2024-09-13T16:48:02.589916+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:02.589916+0000","last_undegraded":"2024-09-13T16:48:02.589916+0000","last_fullsized":"2024-09-13T16:48:02.589916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":21,"state":"creating+peering","last_fresh":"2024-09-13T16:48:06.739561+0000","last_change":"2024-09-13T16:48:06.739561+0000","last_active":"2024-09-13T16:48:06.713484+0000","last_peered":"2024-09-13T16:48:06.713484+0000","last_clean":"2024-09-13T16:48:06.713484+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:06.739561+0000","last_undegraded":"2024-09-13T16:48:06.739561+0000","last_fullsized":"2024-09-13T16:48:06.739561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593189+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:02.593189+0000","last_peered":"2024-09-13T16:48:02.593189+0000","last_clean":"2024-09-13T16:48:02.593189+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:02.593189+0000","last_undegraded":"2024-09-13T16:48:02.593189+0000","last_fullsized":"2024-09-13T16:48:02.593189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594144+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:02.594144+0000","last_peered":"2024-09-13T16:48:02.594144+0000","last_clean":"2024-09-13T16:48:02.594144+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:02.594144+0000","last_undegraded":"2024-09-13T16:48:02.594144+0000","last_fullsized":"2024-09-13T16:48:02.594144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593798+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:02.593798+0000","last_peered":"2024-09-13T16:48:02.593798+0000","last_clean":"2024-09-13T16:48:02.593798+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:02.593798+0000","last_undegraded":"2024-09-13T16:48:02.593798+0000","last_fullsized":"2024-09-13T16:48:02.593798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597394+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:02.597394+0000","last_peered":"2024-09-13T16:48:02.597394+0000","last_clean":"2024-09-13T16:48:02.597394+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:02.597394+0000","last_undegraded":"2024-09-13T16:48:02.597394+0000","last_fullsized":"2024-09-13T16:48:02.597394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590798+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:02.590798+0000","last_peered":"2024-09-13T16:48:02.590798+0000","last_clean":"2024-09-13T16:48:02.590798+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:02.590798+0000","last_undegraded":"2024-09-13T16:48:02.590798+0000","last_fullsized":"2024-09-13T16:48:02.590798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593678+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:02.593678+0000","last_peered":"2024-09-13T16:48:02.593678+0000","last_clean":"2024-09-13T16:48:02.593678+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:02.593678+0000","last_undegraded":"2024-09-13T16:48:02.593678+0000","last_fullsized":"2024-09-13T16:48:02.593678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593451+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:02.593451+0000","last_peered":"2024-09-13T16:48:02.593451+0000","last_clean":"2024-09-13T16:48:02.593451+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:02.593451+0000","last_undegraded":"2024-09-13T16:48:02.593451+0000","last_fullsized":"2024-09-13T16:48:02.593451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590871+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:02.590871+0000","last_peered":"2024-09-13T16:48:02.590871+0000","last_clean":"2024-09-13T16:48:02.590871+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:02.590871+0000","last_undegraded":"2024-09-13T16:48:02.590871+0000","last_fullsized":"2024-09-13T16:48:02.590871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593682+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:02.593682+0000","last_peered":"2024-09-13T16:48:02.593682+0000","last_clean":"2024-09-13T16:48:02.593682+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:02.593682+0000","last_undegraded":"2024-09-13T16:48:02.593682+0000","last_fullsized":"2024-09-13T16:48:02.593682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590624+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:02.590624+0000","last_peered":"2024-09-13T16:48:02.590624+0000","last_clean":"2024-09-13T16:48:02.590624+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:02.590624+0000","last_undegraded":"2024-09-13T16:48:02.590624+0000","last_fullsized":"2024-09-13T16:48:02.590624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593760+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:02.593760+0000","last_peered":"2024-09-13T16:48:02.593760+0000","last_clean":"2024-09-13T16:48:02.593760+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:02.593760+0000","last_undegraded":"2024-09-13T16:48:02.593760+0000","last_fullsized":"2024-09-13T16:48:02.593760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597369+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:02.597369+0000","last_peered":"2024-09-13T16:48:02.597369+0000","last_clean":"2024-09-13T16:48:02.597369+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:02.597369+0000","last_undegraded":"2024-09-13T16:48:02.597369+0000","last_fullsized":"2024-09-13T16:48:02.597369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593328+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:02.593328+0000","last_peered":"2024-09-13T16:48:02.593328+0000","last_clean":"2024-09-13T16:48:02.593328+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:02.593328+0000","last_undegraded":"2024-09-13T16:48:02.593328+0000","last_fullsized":"2024-09-13T16:48:02.593328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597853+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:02.597853+0000","last_peered":"2024-09-13T16:48:02.597853+0000","last_clean":"2024-09-13T16:48:02.597853+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:02.597853+0000","last_undegraded":"2024-09-13T16:48:02.597853+0000","last_fullsized":"2024-09-13T16:48:02.597853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594286+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:02.594286+0000","last_peered":"2024-09-13T16:48:02.594286+0000","last_clean":"2024-09-13T16:48:02.594286+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:02.594286+0000","last_undegraded":"2024-09-13T16:48:02.594286+0000","last_fullsized":"2024-09-13T16:48:02.594286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.594076+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:02.594076+0000","last_peered":"2024-09-13T16:48:02.594076+0000","last_clean":"2024-09-13T16:48:02.594076+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:02.594076+0000","last_undegraded":"2024-09-13T16:48:02.594076+0000","last_fullsized":"2024-09-13T16:48:02.594076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593650+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:02.593650+0000","last_peered":"2024-09-13T16:48:02.593650+0000","last_clean":"2024-09-13T16:48:02.593650+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:02.593650+0000","last_undegraded":"2024-09-13T16:48:02.593650+0000","last_fullsized":"2024-09-13T16:48:02.593650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593413+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:02.593413+0000","last_peered":"2024-09-13T16:48:02.593413+0000","last_clean":"2024-09-13T16:48:02.593413+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:02.593413+0000","last_undegraded":"2024-09-13T16:48:02.593413+0000","last_fullsized":"2024-09-13T16:48:02.593413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597286+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:02.597286+0000","last_peered":"2024-09-13T16:48:02.597286+0000","last_clean":"2024-09-13T16:48:02.597286+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:02.597286+0000","last_undegraded":"2024-09-13T16:48:02.597286+0000","last_fullsized":"2024-09-13T16:48:02.597286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590698+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:02.590698+0000","last_peered":"2024-09-13T16:48:02.590698+0000","last_clean":"2024-09-13T16:48:02.590698+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:02.590698+0000","last_undegraded":"2024-09-13T16:48:02.590698+0000","last_fullsized":"2024-09-13T16:48:02.590698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.589962+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:02.589962+0000","last_peered":"2024-09-13T16:48:02.589962+0000","last_clean":"2024-09-13T16:48:02.589962+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:02.589962+0000","last_undegraded":"2024-09-13T16:48:02.589962+0000","last_fullsized":"2024-09-13T16:48:02.589962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593397+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:02.593397+0000","last_peered":"2024-09-13T16:48:02.593397+0000","last_clean":"2024-09-13T16:48:02.593397+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:02.593397+0000","last_undegraded":"2024-09-13T16:48:02.593397+0000","last_fullsized":"2024-09-13T16:48:02.593397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593423+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:02.593423+0000","last_peered":"2024-09-13T16:48:02.593423+0000","last_clean":"2024-09-13T16:48:02.593423+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:02.593423+0000","last_undegraded":"2024-09-13T16:48:02.593423+0000","last_fullsized":"2024-09-13T16:48:02.593423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597957+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:02.597957+0000","last_peered":"2024-09-13T16:48:02.597957+0000","last_clean":"2024-09-13T16:48:02.597957+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:02.597957+0000","last_undegraded":"2024-09-13T16:48:02.597957+0000","last_fullsized":"2024-09-13T16:48:02.597957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.590539+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:02.590539+0000","last_peered":"2024-09-13T16:48:02.590539+0000","last_clean":"2024-09-13T16:48:02.590539+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:02.590539+0000","last_undegraded":"2024-09-13T16:48:02.590539+0000","last_fullsized":"2024-09-13T16:48:02.590539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597624+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:02.597624+0000","last_peered":"2024-09-13T16:48:02.597624+0000","last_clean":"2024-09-13T16:48:02.597624+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:02.597624+0000","last_undegraded":"2024-09-13T16:48:02.597624+0000","last_fullsized":"2024-09-13T16:48:02.597624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593732+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:02.593732+0000","last_peered":"2024-09-13T16:48:02.593732+0000","last_clean":"2024-09-13T16:48:02.593732+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:02.593732+0000","last_undegraded":"2024-09-13T16:48:02.593732+0000","last_fullsized":"2024-09-13T16:48:02.593732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597669+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:02.597669+0000","last_peered":"2024-09-13T16:48:02.597669+0000","last_clean":"2024-09-13T16:48:02.597669+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:02.597669+0000","last_undegraded":"2024-09-13T16:48:02.597669+0000","last_fullsized":"2024-09-13T16:48:02.597669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593894+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:02.593894+0000","last_peered":"2024-09-13T16:48:02.593894+0000","last_clean":"2024-09-13T16:48:02.593894+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:02.593894+0000","last_undegraded":"2024-09-13T16:48:02.593894+0000","last_fullsized":"2024-09-13T16:48:02.593894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597466+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:02.597466+0000","last_peered":"2024-09-13T16:48:02.597466+0000","last_clean":"2024-09-13T16:48:02.597466+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:02.597466+0000","last_undegraded":"2024-09-13T16:48:02.597466+0000","last_fullsized":"2024-09-13T16:48:02.597466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.593852+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:02.593852+0000","last_peered":"2024-09-13T16:48:02.593852+0000","last_clean":"2024-09-13T16:48:02.593852+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:02.593852+0000","last_undegraded":"2024-09-13T16:48:02.593852+0000","last_fullsized":"2024-09-13T16:48:02.593852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T16:48:07.726913+0000","last_change":"2024-09-13T16:48:07.726913+0000","last_active":"2024-09-13T16:48:07.726913+0000","last_peered":"2024-09-13T16:48:07.726913+0000","last_clean":"2024-09-13T16:48:07.726913+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T16:48:07.726913+0000","last_undegraded":"2024-09-13T16:48:07.726913+0000","last_fullsized":"2024-09-13T16:48:07.726913+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T16:48:02.597930+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:02.597930+0000","last_peered":"2024-09-13T16:48:02.597930+0000","last_clean":"2024-09-13T16:48:02.597930+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:02.597930+0000","last_undegraded":"2024-09-13T16:48:02.597930+0000","last_fullsized":"2024-09-13T16:48:02.597930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":668407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":77547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":229376,"data_stored":79021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":217088,"data_stored":77457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":212992,"data_stored":70978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":811008,"data_stored":661824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:13.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:13.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00c65a7640 1 Processor -- start 2024-09-13T16:48:13.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00c65a7640 1 -- start start 2024-09-13T16:48:13.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00c65a7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00c0151500 0x7f00c00c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00c65a7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 0x7f00c0154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00c65a7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f00c00c47c0 0x7f00c01545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00c65a7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c00c5a30 con 0x7f00c0151500 2024-09-13T16:48:13.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00c65a7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c01576a0 con 0x7f00c00c47c0 2024-09-13T16:48:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00c65a7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c0157980 con 0x7f00c0071dd0 2024-09-13T16:48:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00bffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00c0151500 0x7f00c00c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00bffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00c0151500 0x7f00c00c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34266/0 (socket says 172.21.15.39:34266) 2024-09-13T16:48:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00bffff640 1 -- 172.21.15.39:0/1621813407 learned_addr learned my addr 172.21.15.39:0/1621813407 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00bf7fe640 1 --2- 172.21.15.39:0/1621813407 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 0x7f00c0154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00c4b44640 1 --2- 172.21.15.39:0/1621813407 >> v2:172.21.15.84:3300/0 conn(0x7f00c00c47c0 0x7f00c01545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00bffff640 1 -- 172.21.15.39:0/1621813407 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 msgr2=0x7f00c0154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00bffff640 1 --2- 172.21.15.39:0/1621813407 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 0x7f00c0154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00bffff640 1 -- 172.21.15.39:0/1621813407 >> v2:172.21.15.84:3300/0 conn(0x7f00c00c47c0 msgr2=0x7f00c01545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:13.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00bffff640 1 --2- 172.21.15.39:0/1621813407 >> v2:172.21.15.84:3300/0 conn(0x7f00c00c47c0 0x7f00c01545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.732+0000 7f00bffff640 1 -- 172.21.15.39:0/1621813407 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0157c60 con 0x7f00c0151500 2024-09-13T16:48:13.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00bffff640 1 --2- 172.21.15.39:0/1621813407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 0x7f00c00c4280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f00c00c6260 tx=0x7f00b001cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab9aba79c8780f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:13.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00beffd640 1 -- 172.21.15.39:0/1621813407 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00ab640 con 0x7f00c0151500 2024-09-13T16:48:13.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00beffd640 1 -- 172.21.15.39:0/1621813407 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b00a7070 con 0x7f00c0151500 2024-09-13T16:48:13.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00beffd640 1 -- 172.21.15.39:0/1621813407 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00ab9c0 con 0x7f00c0151500 2024-09-13T16:48:13.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 -- 172.21.15.39:0/1621813407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 msgr2=0x7f00c00c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 --2- 172.21.15.39:0/1621813407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 0x7f00c00c4280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f00c00c6260 tx=0x7f00b001cec0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 -- 172.21.15.39:0/1621813407 shutdown_connections 2024-09-13T16:48:13.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 --2- 172.21.15.39:0/1621813407 >> v2:172.21.15.84:3300/0 conn(0x7f00c00c47c0 0x7f00c01545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 --2- 172.21.15.39:0/1621813407 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 0x7f00c0154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 --2- 172.21.15.39:0/1621813407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 0x7f00c00c4280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 -- 172.21.15.39:0/1621813407 >> 172.21.15.39:0/1621813407 conn(0x7f00c00ab530 msgr2=0x7f00c00ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:13.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 -- 172.21.15.39:0/1621813407 shutdown_connections 2024-09-13T16:48:13.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 -- 172.21.15.39:0/1621813407 wait complete. 2024-09-13T16:48:13.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 Processor -- start 2024-09-13T16:48:13.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 -- start start 2024-09-13T16:48:13.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 0x7f00c0151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00c00c47c0 0x7f00c01501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 0x7f00c00a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01ec960 con 0x7f00c0151500 2024-09-13T16:48:13.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c00ad080 con 0x7f00c00c47c0 2024-09-13T16:48:13.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c65a7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c00aaad0 con 0x7f00c0071dd0 2024-09-13T16:48:13.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c4b44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 0x7f00c00a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c4b44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 0x7f00c00a9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34268/0 (socket says 172.21.15.39:34268) 2024-09-13T16:48:13.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00c4b44640 1 -- 172.21.15.39:0/292082823 learned_addr learned my addr 172.21.15.39:0/292082823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:13.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00bf7fe640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00c00c47c0 0x7f00c01501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.736+0000 7f00bffff640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 0x7f00c0151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00c4b44640 1 -- 172.21.15.39:0/292082823 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 msgr2=0x7f00c0151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00c4b44640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 0x7f00c0151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00c4b44640 1 -- 172.21.15.39:0/292082823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00c00c47c0 msgr2=0x7f00c01501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00c4b44640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00c00c47c0 0x7f00c01501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00c4b44640 1 -- 172.21.15.39:0/292082823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00094d0 con 0x7f00c0151500 2024-09-13T16:48:13.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00c4b44640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 0x7f00c00a9e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f00b800efc0 tx=0x7f00b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:13.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00bcff9640 1 -- 172.21.15.39:0/292082823 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8014480 con 0x7f00c0151500 2024-09-13T16:48:13.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00bcff9640 1 -- 172.21.15.39:0/292082823 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b800f040 con 0x7f00c0151500 2024-09-13T16:48:13.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c01ec680 con 0x7f00c0151500 2024-09-13T16:48:13.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00bcff9640 1 -- 172.21.15.39:0/292082823 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8014820 con 0x7f00c0151500 2024-09-13T16:48:13.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c00a9920 con 0x7f00c0151500 2024-09-13T16:48:13.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c00a94a0 con 0x7f00c0151500 2024-09-13T16:48:13.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00bcff9640 1 -- 172.21.15.39:0/292082823 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00b8014980 con 0x7f00c0151500 2024-09-13T16:48:13.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00bcff9640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00ac0666c0 0x7f00ac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:13.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00bcff9640 1 -- 172.21.15.39:0/292082823 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f00b8085550 con 0x7f00c0151500 2024-09-13T16:48:13.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00bffff640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00ac0666c0 0x7f00ac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:13.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.740+0000 7f00bffff640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00ac0666c0 0x7f00ac068b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f00b00af000 tx=0x7f00b0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:13.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.744+0000 7f00bcff9640 1 -- 172.21.15.39:0/292082823 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f00b8052700 con 0x7f00c0151500 2024-09-13T16:48:13.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.900+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00c01518e0 con 0x7f00ac0666c0 2024-09-13T16:48:13.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.932+0000 7f00bcff9640 1 -- 172.21.15.39:0/292082823 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+503124 (secure 0 0 0) 0x7f00b00ab600 con 0x7f00ac0666c0 2024-09-13T16:48:13.938 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:13.944 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:13.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00ac0666c0 msgr2=0x7f00ac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00ac0666c0 0x7f00ac068b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f00b00af000 tx=0x7f00b0004740 comp rx=0 tx=0).stop 2024-09-13T16:48:13.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 msgr2=0x7f00c00a9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:13.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 0x7f00c00a9e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f00b800efc0 tx=0x7f00b800c490 comp rx=0 tx=0).stop 2024-09-13T16:48:13.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 shutdown_connections 2024-09-13T16:48:13.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f00ac0666c0 0x7f00ac068b80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00c00c47c0 0x7f00c01501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00c0151500 0x7f00c00a9e40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 --2- 172.21.15.39:0/292082823 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00c0071dd0 0x7f00c0151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:13.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 >> 172.21.15.39:0/292082823 conn(0x7f00c00ab530 msgr2=0x7f00c00c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:13.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 shutdown_connections 2024-09-13T16:48:13.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:13.940+0000 7f00c65a7640 1 -- 172.21.15.39:0/292082823 wait complete. 2024-09-13T16:48:13.961 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-13T16:48:13.561783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595092,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":248,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194,"ondisk_log_size":194,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":375,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235816,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738904,"statfs":{"total":773094113280,"available":772852637696,"internally_reserved":0,"allocated":3350528,"data_stored":1983984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001967"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174677+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T16:48:09.174677+0000","last_peered":"2024-09-13T16:48:09.174677+0000","last_clean":"2024-09-13T16:48:09.174677+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T16:48:09.174677+0000","last_undegraded":"2024-09-13T16:48:09.174677+0000","last_fullsized":"2024-09-13T16:48:09.174677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163864+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:08.163864+0000","last_peered":"2024-09-13T16:48:08.163864+0000","last_clean":"2024-09-13T16:48:08.163864+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:08.163864+0000","last_undegraded":"2024-09-13T16:48:08.163864+0000","last_fullsized":"2024-09-13T16:48:08.163864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146105+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:08.146105+0000","last_peered":"2024-09-13T16:48:08.146105+0000","last_clean":"2024-09-13T16:48:08.146105+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:08.146105+0000","last_undegraded":"2024-09-13T16:48:08.146105+0000","last_fullsized":"2024-09-13T16:48:08.146105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143176+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T16:48:08.143176+0000","last_peered":"2024-09-13T16:48:08.143176+0000","last_clean":"2024-09-13T16:48:08.143176+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T16:48:08.143176+0000","last_undegraded":"2024-09-13T16:48:08.143176+0000","last_fullsized":"2024-09-13T16:48:08.143176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173274+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T16:48:09.173274+0000","last_peered":"2024-09-13T16:48:09.173274+0000","last_clean":"2024-09-13T16:48:09.173274+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T16:48:09.173274+0000","last_undegraded":"2024-09-13T16:48:09.173274+0000","last_fullsized":"2024-09-13T16:48:09.173274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146246+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:08.146246+0000","last_peered":"2024-09-13T16:48:08.146246+0000","last_clean":"2024-09-13T16:48:08.146246+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:08.146246+0000","last_undegraded":"2024-09-13T16:48:08.146246+0000","last_fullsized":"2024-09-13T16:48:08.146246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163480+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:08.163480+0000","last_peered":"2024-09-13T16:48:08.163480+0000","last_clean":"2024-09-13T16:48:08.163480+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:08.163480+0000","last_undegraded":"2024-09-13T16:48:08.163480+0000","last_fullsized":"2024-09-13T16:48:08.163480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184157+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T16:48:08.184157+0000","last_peered":"2024-09-13T16:48:08.184157+0000","last_clean":"2024-09-13T16:48:08.184157+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T16:48:08.184157+0000","last_undegraded":"2024-09-13T16:48:08.184157+0000","last_fullsized":"2024-09-13T16:48:08.184157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173949+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T16:48:09.173949+0000","last_peered":"2024-09-13T16:48:09.173949+0000","last_clean":"2024-09-13T16:48:09.173949+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T16:48:09.173949+0000","last_undegraded":"2024-09-13T16:48:09.173949+0000","last_fullsized":"2024-09-13T16:48:09.173949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178923+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:08.178923+0000","last_peered":"2024-09-13T16:48:08.178923+0000","last_clean":"2024-09-13T16:48:08.178923+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:08.178923+0000","last_undegraded":"2024-09-13T16:48:08.178923+0000","last_fullsized":"2024-09-13T16:48:08.178923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151847+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:08.151847+0000","last_peered":"2024-09-13T16:48:08.151847+0000","last_clean":"2024-09-13T16:48:08.151847+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:08.151847+0000","last_undegraded":"2024-09-13T16:48:08.151847+0000","last_fullsized":"2024-09-13T16:48:08.151847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208404+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T16:48:08.208404+0000","last_peered":"2024-09-13T16:48:08.208404+0000","last_clean":"2024-09-13T16:48:08.208404+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T16:48:08.208404+0000","last_undegraded":"2024-09-13T16:48:08.208404+0000","last_fullsized":"2024-09-13T16:48:08.208404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159591+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T16:48:09.159591+0000","last_peered":"2024-09-13T16:48:09.159591+0000","last_clean":"2024-09-13T16:48:09.159591+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T16:48:09.159591+0000","last_undegraded":"2024-09-13T16:48:09.159591+0000","last_fullsized":"2024-09-13T16:48:09.159591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165397+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:08.165397+0000","last_peered":"2024-09-13T16:48:08.165397+0000","last_clean":"2024-09-13T16:48:08.165397+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:08.165397+0000","last_undegraded":"2024-09-13T16:48:08.165397+0000","last_fullsized":"2024-09-13T16:48:08.165397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153028+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:08.153028+0000","last_peered":"2024-09-13T16:48:08.153028+0000","last_clean":"2024-09-13T16:48:08.153028+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:08.153028+0000","last_undegraded":"2024-09-13T16:48:08.153028+0000","last_fullsized":"2024-09-13T16:48:08.153028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.203658+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T16:48:08.203658+0000","last_peered":"2024-09-13T16:48:08.203658+0000","last_clean":"2024-09-13T16:48:08.203658+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T16:48:08.203658+0000","last_undegraded":"2024-09-13T16:48:08.203658+0000","last_fullsized":"2024-09-13T16:48:08.203658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173632+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T16:48:09.173632+0000","last_peered":"2024-09-13T16:48:09.173632+0000","last_clean":"2024-09-13T16:48:09.173632+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T16:48:09.173632+0000","last_undegraded":"2024-09-13T16:48:09.173632+0000","last_fullsized":"2024-09-13T16:48:09.173632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146884+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:08.146884+0000","last_peered":"2024-09-13T16:48:08.146884+0000","last_clean":"2024-09-13T16:48:08.146884+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:08.146884+0000","last_undegraded":"2024-09-13T16:48:08.146884+0000","last_fullsized":"2024-09-13T16:48:08.146884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160788+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:08.160788+0000","last_peered":"2024-09-13T16:48:08.160788+0000","last_clean":"2024-09-13T16:48:08.160788+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:08.160788+0000","last_undegraded":"2024-09-13T16:48:08.160788+0000","last_fullsized":"2024-09-13T16:48:08.160788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.206623+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T16:48:08.206623+0000","last_peered":"2024-09-13T16:48:08.206623+0000","last_clean":"2024-09-13T16:48:08.206623+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T16:48:08.206623+0000","last_undegraded":"2024-09-13T16:48:08.206623+0000","last_fullsized":"2024-09-13T16:48:08.206623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159080+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T16:48:09.159080+0000","last_peered":"2024-09-13T16:48:09.159080+0000","last_clean":"2024-09-13T16:48:09.159080+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T16:48:09.159080+0000","last_undegraded":"2024-09-13T16:48:09.159080+0000","last_fullsized":"2024-09-13T16:48:09.159080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144345+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:08.144345+0000","last_peered":"2024-09-13T16:48:08.144345+0000","last_clean":"2024-09-13T16:48:08.144345+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:08.144345+0000","last_undegraded":"2024-09-13T16:48:08.144345+0000","last_fullsized":"2024-09-13T16:48:08.144345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147467+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:08.147467+0000","last_peered":"2024-09-13T16:48:08.147467+0000","last_clean":"2024-09-13T16:48:08.147467+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:08.147467+0000","last_undegraded":"2024-09-13T16:48:08.147467+0000","last_fullsized":"2024-09-13T16:48:08.147467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184081+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T16:48:08.184081+0000","last_peered":"2024-09-13T16:48:08.184081+0000","last_clean":"2024-09-13T16:48:08.184081+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T16:48:08.184081+0000","last_undegraded":"2024-09-13T16:48:08.184081+0000","last_fullsized":"2024-09-13T16:48:08.184081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143476+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:08.143476+0000","last_peered":"2024-09-13T16:48:08.143476+0000","last_clean":"2024-09-13T16:48:08.143476+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:08.143476+0000","last_undegraded":"2024-09-13T16:48:08.143476+0000","last_fullsized":"2024-09-13T16:48:08.143476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147565+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:08.147565+0000","last_peered":"2024-09-13T16:48:08.147565+0000","last_clean":"2024-09-13T16:48:08.147565+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:08.147565+0000","last_undegraded":"2024-09-13T16:48:08.147565+0000","last_fullsized":"2024-09-13T16:48:08.147565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174007+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T16:48:08.174007+0000","last_peered":"2024-09-13T16:48:08.174007+0000","last_clean":"2024-09-13T16:48:08.174007+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T16:48:08.174007+0000","last_undegraded":"2024-09-13T16:48:08.174007+0000","last_fullsized":"2024-09-13T16:48:08.174007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162482+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T16:48:09.162482+0000","last_peered":"2024-09-13T16:48:09.162482+0000","last_clean":"2024-09-13T16:48:09.162482+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T16:48:09.162482+0000","last_undegraded":"2024-09-13T16:48:09.162482+0000","last_fullsized":"2024-09-13T16:48:09.162482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186454+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:08.186454+0000","last_peered":"2024-09-13T16:48:08.186454+0000","last_clean":"2024-09-13T16:48:08.186454+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:08.186454+0000","last_undegraded":"2024-09-13T16:48:08.186454+0000","last_fullsized":"2024-09-13T16:48:08.186454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160985+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:08.160985+0000","last_peered":"2024-09-13T16:48:08.160985+0000","last_clean":"2024-09-13T16:48:08.160985+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:08.160985+0000","last_undegraded":"2024-09-13T16:48:08.160985+0000","last_fullsized":"2024-09-13T16:48:08.160985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208189+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T16:48:08.208189+0000","last_peered":"2024-09-13T16:48:08.208189+0000","last_clean":"2024-09-13T16:48:08.208189+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T16:48:08.208189+0000","last_undegraded":"2024-09-13T16:48:08.208189+0000","last_fullsized":"2024-09-13T16:48:08.208189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176059+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T16:48:09.176059+0000","last_peered":"2024-09-13T16:48:09.176059+0000","last_clean":"2024-09-13T16:48:09.176059+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T16:48:09.176059+0000","last_undegraded":"2024-09-13T16:48:09.176059+0000","last_fullsized":"2024-09-13T16:48:09.176059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163768+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:08.163768+0000","last_peered":"2024-09-13T16:48:08.163768+0000","last_clean":"2024-09-13T16:48:08.163768+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:08.163768+0000","last_undegraded":"2024-09-13T16:48:08.163768+0000","last_fullsized":"2024-09-13T16:48:08.163768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148721+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:08.148721+0000","last_peered":"2024-09-13T16:48:08.148721+0000","last_clean":"2024-09-13T16:48:08.148721+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:08.148721+0000","last_undegraded":"2024-09-13T16:48:08.148721+0000","last_fullsized":"2024-09-13T16:48:08.148721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185371+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T16:48:08.185371+0000","last_peered":"2024-09-13T16:48:08.185371+0000","last_clean":"2024-09-13T16:48:08.185371+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T16:48:08.185371+0000","last_undegraded":"2024-09-13T16:48:08.185371+0000","last_fullsized":"2024-09-13T16:48:08.185371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174416+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T16:48:09.174416+0000","last_peered":"2024-09-13T16:48:09.174416+0000","last_clean":"2024-09-13T16:48:09.174416+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T16:48:09.174416+0000","last_undegraded":"2024-09-13T16:48:09.174416+0000","last_fullsized":"2024-09-13T16:48:09.174416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145183+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:08.145183+0000","last_peered":"2024-09-13T16:48:08.145183+0000","last_clean":"2024-09-13T16:48:08.145183+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:08.145183+0000","last_undegraded":"2024-09-13T16:48:08.145183+0000","last_fullsized":"2024-09-13T16:48:08.145183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152363+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:08.152363+0000","last_peered":"2024-09-13T16:48:08.152363+0000","last_clean":"2024-09-13T16:48:08.152363+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:08.152363+0000","last_undegraded":"2024-09-13T16:48:08.152363+0000","last_fullsized":"2024-09-13T16:48:08.152363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143237+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T16:48:08.143237+0000","last_peered":"2024-09-13T16:48:08.143237+0000","last_clean":"2024-09-13T16:48:08.143237+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T16:48:08.143237+0000","last_undegraded":"2024-09-13T16:48:08.143237+0000","last_fullsized":"2024-09-13T16:48:08.143237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158858+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T16:48:09.158858+0000","last_peered":"2024-09-13T16:48:09.158858+0000","last_clean":"2024-09-13T16:48:09.158858+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T16:48:09.158858+0000","last_undegraded":"2024-09-13T16:48:09.158858+0000","last_fullsized":"2024-09-13T16:48:09.158858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148446+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:08.148446+0000","last_peered":"2024-09-13T16:48:08.148446+0000","last_clean":"2024-09-13T16:48:08.148446+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:08.148446+0000","last_undegraded":"2024-09-13T16:48:08.148446+0000","last_fullsized":"2024-09-13T16:48:08.148446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179426+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:08.179426+0000","last_peered":"2024-09-13T16:48:08.179426+0000","last_clean":"2024-09-13T16:48:08.179426+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:08.179426+0000","last_undegraded":"2024-09-13T16:48:08.179426+0000","last_fullsized":"2024-09-13T16:48:08.179426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161207+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T16:48:08.161207+0000","last_peered":"2024-09-13T16:48:08.161207+0000","last_clean":"2024-09-13T16:48:08.161207+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T16:48:08.161207+0000","last_undegraded":"2024-09-13T16:48:08.161207+0000","last_fullsized":"2024-09-13T16:48:08.161207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173006+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T16:48:09.173006+0000","last_peered":"2024-09-13T16:48:09.173006+0000","last_clean":"2024-09-13T16:48:09.173006+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T16:48:09.173006+0000","last_undegraded":"2024-09-13T16:48:09.173006+0000","last_fullsized":"2024-09-13T16:48:09.173006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153222+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:08.153222+0000","last_peered":"2024-09-13T16:48:08.153222+0000","last_clean":"2024-09-13T16:48:08.153222+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:08.153222+0000","last_undegraded":"2024-09-13T16:48:08.153222+0000","last_fullsized":"2024-09-13T16:48:08.153222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148522+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:08.148522+0000","last_peered":"2024-09-13T16:48:08.148522+0000","last_clean":"2024-09-13T16:48:08.148522+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:08.148522+0000","last_undegraded":"2024-09-13T16:48:08.148522+0000","last_fullsized":"2024-09-13T16:48:08.148522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146684+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T16:48:08.146684+0000","last_peered":"2024-09-13T16:48:08.146684+0000","last_clean":"2024-09-13T16:48:08.146684+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T16:48:08.146684+0000","last_undegraded":"2024-09-13T16:48:08.146684+0000","last_fullsized":"2024-09-13T16:48:08.146684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174867+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T16:48:09.174867+0000","last_peered":"2024-09-13T16:48:09.174867+0000","last_clean":"2024-09-13T16:48:09.174867+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T16:48:09.174867+0000","last_undegraded":"2024-09-13T16:48:09.174867+0000","last_fullsized":"2024-09-13T16:48:09.174867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185855+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:08.185855+0000","last_peered":"2024-09-13T16:48:08.185855+0000","last_clean":"2024-09-13T16:48:08.185855+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:08.185855+0000","last_undegraded":"2024-09-13T16:48:08.185855+0000","last_fullsized":"2024-09-13T16:48:08.185855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149466+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:08.149466+0000","last_peered":"2024-09-13T16:48:08.149466+0000","last_clean":"2024-09-13T16:48:08.149466+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:08.149466+0000","last_undegraded":"2024-09-13T16:48:08.149466+0000","last_fullsized":"2024-09-13T16:48:08.149466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148372+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T16:48:08.148372+0000","last_peered":"2024-09-13T16:48:08.148372+0000","last_clean":"2024-09-13T16:48:08.148372+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T16:48:08.148372+0000","last_undegraded":"2024-09-13T16:48:08.148372+0000","last_fullsized":"2024-09-13T16:48:08.148372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159186+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T16:48:09.159186+0000","last_peered":"2024-09-13T16:48:09.159186+0000","last_clean":"2024-09-13T16:48:09.159186+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T16:48:09.159186+0000","last_undegraded":"2024-09-13T16:48:09.159186+0000","last_fullsized":"2024-09-13T16:48:09.159186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146860+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:08.146860+0000","last_peered":"2024-09-13T16:48:08.146860+0000","last_clean":"2024-09-13T16:48:08.146860+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:08.146860+0000","last_undegraded":"2024-09-13T16:48:08.146860+0000","last_fullsized":"2024-09-13T16:48:08.146860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154111+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:08.154111+0000","last_peered":"2024-09-13T16:48:08.154111+0000","last_clean":"2024-09-13T16:48:08.154111+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:08.154111+0000","last_undegraded":"2024-09-13T16:48:08.154111+0000","last_fullsized":"2024-09-13T16:48:08.154111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175388+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T16:48:08.175388+0000","last_peered":"2024-09-13T16:48:08.175388+0000","last_clean":"2024-09-13T16:48:08.175388+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T16:48:08.175388+0000","last_undegraded":"2024-09-13T16:48:08.175388+0000","last_fullsized":"2024-09-13T16:48:08.175388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162556+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T16:48:09.162556+0000","last_peered":"2024-09-13T16:48:09.162556+0000","last_clean":"2024-09-13T16:48:09.162556+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T16:48:09.162556+0000","last_undegraded":"2024-09-13T16:48:09.162556+0000","last_fullsized":"2024-09-13T16:48:09.162556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160990+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:08.160990+0000","last_peered":"2024-09-13T16:48:08.160990+0000","last_clean":"2024-09-13T16:48:08.160990+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:08.160990+0000","last_undegraded":"2024-09-13T16:48:08.160990+0000","last_fullsized":"2024-09-13T16:48:08.160990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161254+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:08.161254+0000","last_peered":"2024-09-13T16:48:08.161254+0000","last_clean":"2024-09-13T16:48:08.161254+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:08.161254+0000","last_undegraded":"2024-09-13T16:48:08.161254+0000","last_fullsized":"2024-09-13T16:48:08.161254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163476+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T16:48:08.163476+0000","last_peered":"2024-09-13T16:48:08.163476+0000","last_clean":"2024-09-13T16:48:08.163476+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T16:48:08.163476+0000","last_undegraded":"2024-09-13T16:48:08.163476+0000","last_fullsized":"2024-09-13T16:48:08.163476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175463+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T16:48:09.175463+0000","last_peered":"2024-09-13T16:48:09.175463+0000","last_clean":"2024-09-13T16:48:09.175463+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T16:48:09.175463+0000","last_undegraded":"2024-09-13T16:48:09.175463+0000","last_fullsized":"2024-09-13T16:48:09.175463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143792+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:08.143792+0000","last_peered":"2024-09-13T16:48:08.143792+0000","last_clean":"2024-09-13T16:48:08.143792+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:08.143792+0000","last_undegraded":"2024-09-13T16:48:08.143792+0000","last_fullsized":"2024-09-13T16:48:08.143792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148284+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:08.148284+0000","last_peered":"2024-09-13T16:48:08.148284+0000","last_clean":"2024-09-13T16:48:08.148284+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:08.148284+0000","last_undegraded":"2024-09-13T16:48:08.148284+0000","last_fullsized":"2024-09-13T16:48:08.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161478+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:08.161478+0000","last_peered":"2024-09-13T16:48:08.161478+0000","last_clean":"2024-09-13T16:48:08.161478+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:08.161478+0000","last_undegraded":"2024-09-13T16:48:08.161478+0000","last_fullsized":"2024-09-13T16:48:08.161478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148905+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T16:48:08.148905+0000","last_peered":"2024-09-13T16:48:08.148905+0000","last_clean":"2024-09-13T16:48:08.148905+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T16:48:08.148905+0000","last_undegraded":"2024-09-13T16:48:08.148905+0000","last_fullsized":"2024-09-13T16:48:08.148905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158934+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T16:48:09.158934+0000","last_peered":"2024-09-13T16:48:09.158934+0000","last_clean":"2024-09-13T16:48:09.158934+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T16:48:09.158934+0000","last_undegraded":"2024-09-13T16:48:09.158934+0000","last_fullsized":"2024-09-13T16:48:09.158934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178617+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:08.178617+0000","last_peered":"2024-09-13T16:48:08.178617+0000","last_clean":"2024-09-13T16:48:08.178617+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:08.178617+0000","last_undegraded":"2024-09-13T16:48:08.178617+0000","last_fullsized":"2024-09-13T16:48:08.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155913+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:08.155913+0000","last_peered":"2024-09-13T16:48:08.155913+0000","last_clean":"2024-09-13T16:48:08.155913+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:08.155913+0000","last_undegraded":"2024-09-13T16:48:08.155913+0000","last_fullsized":"2024-09-13T16:48:08.155913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146319+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T16:48:08.146319+0000","last_peered":"2024-09-13T16:48:08.146319+0000","last_clean":"2024-09-13T16:48:08.146319+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T16:48:08.146319+0000","last_undegraded":"2024-09-13T16:48:08.146319+0000","last_fullsized":"2024-09-13T16:48:08.146319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158609+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T16:48:09.158609+0000","last_peered":"2024-09-13T16:48:09.158609+0000","last_clean":"2024-09-13T16:48:09.158609+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T16:48:09.158609+0000","last_undegraded":"2024-09-13T16:48:09.158609+0000","last_fullsized":"2024-09-13T16:48:09.158609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152057+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:08.152057+0000","last_peered":"2024-09-13T16:48:08.152057+0000","last_clean":"2024-09-13T16:48:08.152057+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:08.152057+0000","last_undegraded":"2024-09-13T16:48:08.152057+0000","last_fullsized":"2024-09-13T16:48:08.152057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149487+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:08.149487+0000","last_peered":"2024-09-13T16:48:08.149487+0000","last_clean":"2024-09-13T16:48:08.149487+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:08.149487+0000","last_undegraded":"2024-09-13T16:48:08.149487+0000","last_fullsized":"2024-09-13T16:48:08.149487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175317+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T16:48:08.175317+0000","last_peered":"2024-09-13T16:48:08.175317+0000","last_clean":"2024-09-13T16:48:08.175317+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T16:48:08.175317+0000","last_undegraded":"2024-09-13T16:48:08.175317+0000","last_fullsized":"2024-09-13T16:48:08.175317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173209+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T16:48:09.173209+0000","last_peered":"2024-09-13T16:48:09.173209+0000","last_clean":"2024-09-13T16:48:09.173209+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T16:48:09.173209+0000","last_undegraded":"2024-09-13T16:48:09.173209+0000","last_fullsized":"2024-09-13T16:48:09.173209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.177954+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:08.177954+0000","last_peered":"2024-09-13T16:48:08.177954+0000","last_clean":"2024-09-13T16:48:08.177954+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:08.177954+0000","last_undegraded":"2024-09-13T16:48:08.177954+0000","last_fullsized":"2024-09-13T16:48:08.177954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155936+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:08.155936+0000","last_peered":"2024-09-13T16:48:08.155936+0000","last_clean":"2024-09-13T16:48:08.155936+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:08.155936+0000","last_undegraded":"2024-09-13T16:48:08.155936+0000","last_fullsized":"2024-09-13T16:48:08.155936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158692+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T16:48:08.158692+0000","last_peered":"2024-09-13T16:48:08.158692+0000","last_clean":"2024-09-13T16:48:08.158692+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T16:48:08.158692+0000","last_undegraded":"2024-09-13T16:48:08.158692+0000","last_fullsized":"2024-09-13T16:48:08.158692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158034+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T16:48:09.158034+0000","last_peered":"2024-09-13T16:48:09.158034+0000","last_clean":"2024-09-13T16:48:09.158034+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T16:48:09.158034+0000","last_undegraded":"2024-09-13T16:48:09.158034+0000","last_fullsized":"2024-09-13T16:48:09.158034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160811+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:08.160811+0000","last_peered":"2024-09-13T16:48:08.160811+0000","last_clean":"2024-09-13T16:48:08.160811+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:08.160811+0000","last_undegraded":"2024-09-13T16:48:08.160811+0000","last_fullsized":"2024-09-13T16:48:08.160811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.192267+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:08.192267+0000","last_peered":"2024-09-13T16:48:08.192267+0000","last_clean":"2024-09-13T16:48:08.192267+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:08.192267+0000","last_undegraded":"2024-09-13T16:48:08.192267+0000","last_fullsized":"2024-09-13T16:48:08.192267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185117+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T16:48:08.185117+0000","last_peered":"2024-09-13T16:48:08.185117+0000","last_clean":"2024-09-13T16:48:08.185117+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T16:48:08.185117+0000","last_undegraded":"2024-09-13T16:48:08.185117+0000","last_fullsized":"2024-09-13T16:48:08.185117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177888+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T16:48:09.177888+0000","last_peered":"2024-09-13T16:48:09.177888+0000","last_clean":"2024-09-13T16:48:09.177888+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T16:48:09.177888+0000","last_undegraded":"2024-09-13T16:48:09.177888+0000","last_fullsized":"2024-09-13T16:48:09.177888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167237+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:08.167237+0000","last_peered":"2024-09-13T16:48:08.167237+0000","last_clean":"2024-09-13T16:48:08.167237+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:08.167237+0000","last_undegraded":"2024-09-13T16:48:08.167237+0000","last_fullsized":"2024-09-13T16:48:08.167237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156043+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:08.156043+0000","last_peered":"2024-09-13T16:48:08.156043+0000","last_clean":"2024-09-13T16:48:08.156043+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:08.156043+0000","last_undegraded":"2024-09-13T16:48:08.156043+0000","last_fullsized":"2024-09-13T16:48:08.156043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163690+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T16:48:08.163690+0000","last_peered":"2024-09-13T16:48:08.163690+0000","last_clean":"2024-09-13T16:48:08.163690+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T16:48:08.163690+0000","last_undegraded":"2024-09-13T16:48:08.163690+0000","last_fullsized":"2024-09-13T16:48:08.163690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162212+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T16:48:09.162212+0000","last_peered":"2024-09-13T16:48:09.162212+0000","last_clean":"2024-09-13T16:48:09.162212+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T16:48:09.162212+0000","last_undegraded":"2024-09-13T16:48:09.162212+0000","last_fullsized":"2024-09-13T16:48:09.162212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144923+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:08.144923+0000","last_peered":"2024-09-13T16:48:08.144923+0000","last_clean":"2024-09-13T16:48:08.144923+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:08.144923+0000","last_undegraded":"2024-09-13T16:48:08.144923+0000","last_fullsized":"2024-09-13T16:48:08.144923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145599+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:08.145599+0000","last_peered":"2024-09-13T16:48:08.145599+0000","last_clean":"2024-09-13T16:48:08.145599+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:08.145599+0000","last_undegraded":"2024-09-13T16:48:08.145599+0000","last_fullsized":"2024-09-13T16:48:08.145599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145022+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T16:48:08.145022+0000","last_peered":"2024-09-13T16:48:08.145022+0000","last_clean":"2024-09-13T16:48:08.145022+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T16:48:08.145022+0000","last_undegraded":"2024-09-13T16:48:08.145022+0000","last_fullsized":"2024-09-13T16:48:08.145022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161391+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T16:48:09.161391+0000","last_peered":"2024-09-13T16:48:09.161391+0000","last_clean":"2024-09-13T16:48:09.161391+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T16:48:09.161391+0000","last_undegraded":"2024-09-13T16:48:09.161391+0000","last_fullsized":"2024-09-13T16:48:09.161391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160556+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:08.160556+0000","last_peered":"2024-09-13T16:48:08.160556+0000","last_clean":"2024-09-13T16:48:08.160556+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:08.160556+0000","last_undegraded":"2024-09-13T16:48:08.160556+0000","last_fullsized":"2024-09-13T16:48:08.160556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149235+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:08.149235+0000","last_peered":"2024-09-13T16:48:08.149235+0000","last_clean":"2024-09-13T16:48:08.149235+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:08.149235+0000","last_undegraded":"2024-09-13T16:48:08.149235+0000","last_fullsized":"2024-09-13T16:48:08.149235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.209853+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T16:48:08.209853+0000","last_peered":"2024-09-13T16:48:08.209853+0000","last_clean":"2024-09-13T16:48:08.209853+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T16:48:08.209853+0000","last_undegraded":"2024-09-13T16:48:08.209853+0000","last_fullsized":"2024-09-13T16:48:08.209853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161110+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:08.161110+0000","last_peered":"2024-09-13T16:48:08.161110+0000","last_clean":"2024-09-13T16:48:08.161110+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:08.161110+0000","last_undegraded":"2024-09-13T16:48:08.161110+0000","last_fullsized":"2024-09-13T16:48:08.161110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161421+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:08.161421+0000","last_peered":"2024-09-13T16:48:08.161421+0000","last_clean":"2024-09-13T16:48:08.161421+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:08.161421+0000","last_undegraded":"2024-09-13T16:48:08.161421+0000","last_fullsized":"2024-09-13T16:48:08.161421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.188902+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T16:48:08.188902+0000","last_peered":"2024-09-13T16:48:08.188902+0000","last_clean":"2024-09-13T16:48:08.188902+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T16:48:08.188902+0000","last_undegraded":"2024-09-13T16:48:08.188902+0000","last_fullsized":"2024-09-13T16:48:08.188902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165383+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:08.165383+0000","last_peered":"2024-09-13T16:48:08.165383+0000","last_clean":"2024-09-13T16:48:08.165383+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:08.165383+0000","last_undegraded":"2024-09-13T16:48:08.165383+0000","last_fullsized":"2024-09-13T16:48:08.165383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148543+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:08.148543+0000","last_peered":"2024-09-13T16:48:08.148543+0000","last_clean":"2024-09-13T16:48:08.148543+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:08.148543+0000","last_undegraded":"2024-09-13T16:48:08.148543+0000","last_fullsized":"2024-09-13T16:48:08.148543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146582+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T16:48:08.146582+0000","last_peered":"2024-09-13T16:48:08.146582+0000","last_clean":"2024-09-13T16:48:08.146582+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T16:48:08.146582+0000","last_undegraded":"2024-09-13T16:48:08.146582+0000","last_fullsized":"2024-09-13T16:48:08.146582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173477+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T16:48:09.173477+0000","last_peered":"2024-09-13T16:48:09.173477+0000","last_clean":"2024-09-13T16:48:09.173477+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T16:48:09.173477+0000","last_undegraded":"2024-09-13T16:48:09.173477+0000","last_fullsized":"2024-09-13T16:48:09.173477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.190852+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:08.190852+0000","last_peered":"2024-09-13T16:48:08.190852+0000","last_clean":"2024-09-13T16:48:08.190852+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:08.190852+0000","last_undegraded":"2024-09-13T16:48:08.190852+0000","last_fullsized":"2024-09-13T16:48:08.190852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149403+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:08.149403+0000","last_peered":"2024-09-13T16:48:08.149403+0000","last_clean":"2024-09-13T16:48:08.149403+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:08.149403+0000","last_undegraded":"2024-09-13T16:48:08.149403+0000","last_fullsized":"2024-09-13T16:48:08.149403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208558+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T16:48:08.208558+0000","last_peered":"2024-09-13T16:48:08.208558+0000","last_clean":"2024-09-13T16:48:08.208558+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T16:48:08.208558+0000","last_undegraded":"2024-09-13T16:48:08.208558+0000","last_fullsized":"2024-09-13T16:48:08.208558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158601+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T16:48:09.158601+0000","last_peered":"2024-09-13T16:48:09.158601+0000","last_clean":"2024-09-13T16:48:09.158601+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T16:48:09.158601+0000","last_undegraded":"2024-09-13T16:48:09.158601+0000","last_fullsized":"2024-09-13T16:48:09.158601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163596+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:08.163596+0000","last_peered":"2024-09-13T16:48:08.163596+0000","last_clean":"2024-09-13T16:48:08.163596+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:08.163596+0000","last_undegraded":"2024-09-13T16:48:08.163596+0000","last_fullsized":"2024-09-13T16:48:08.163596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156189+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:08.156189+0000","last_peered":"2024-09-13T16:48:08.156189+0000","last_clean":"2024-09-13T16:48:08.156189+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:08.156189+0000","last_undegraded":"2024-09-13T16:48:08.156189+0000","last_fullsized":"2024-09-13T16:48:08.156189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174258+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T16:48:08.174258+0000","last_peered":"2024-09-13T16:48:08.174258+0000","last_clean":"2024-09-13T16:48:08.174258+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T16:48:08.174258+0000","last_undegraded":"2024-09-13T16:48:08.174258+0000","last_fullsized":"2024-09-13T16:48:08.174258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173305+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T16:48:09.173305+0000","last_peered":"2024-09-13T16:48:09.173305+0000","last_clean":"2024-09-13T16:48:09.173305+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T16:48:09.173305+0000","last_undegraded":"2024-09-13T16:48:09.173305+0000","last_fullsized":"2024-09-13T16:48:09.173305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155963+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:08.155963+0000","last_peered":"2024-09-13T16:48:08.155963+0000","last_clean":"2024-09-13T16:48:08.155963+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:08.155963+0000","last_undegraded":"2024-09-13T16:48:08.155963+0000","last_fullsized":"2024-09-13T16:48:08.155963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148500+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:08.148500+0000","last_peered":"2024-09-13T16:48:08.148500+0000","last_clean":"2024-09-13T16:48:08.148500+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:08.148500+0000","last_undegraded":"2024-09-13T16:48:08.148500+0000","last_fullsized":"2024-09-13T16:48:08.148500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143144+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T16:48:08.143144+0000","last_peered":"2024-09-13T16:48:08.143144+0000","last_clean":"2024-09-13T16:48:08.143144+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T16:48:08.143144+0000","last_undegraded":"2024-09-13T16:48:08.143144+0000","last_fullsized":"2024-09-13T16:48:08.143144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161066+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T16:48:09.161066+0000","last_peered":"2024-09-13T16:48:09.161066+0000","last_clean":"2024-09-13T16:48:09.161066+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T16:48:09.161066+0000","last_undegraded":"2024-09-13T16:48:09.161066+0000","last_fullsized":"2024-09-13T16:48:09.161066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186757+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T16:48:08.186757+0000","last_peered":"2024-09-13T16:48:08.186757+0000","last_clean":"2024-09-13T16:48:08.186757+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T16:48:08.186757+0000","last_undegraded":"2024-09-13T16:48:08.186757+0000","last_fullsized":"2024-09-13T16:48:08.186757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147291+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:08.147291+0000","last_peered":"2024-09-13T16:48:08.147291+0000","last_clean":"2024-09-13T16:48:08.147291+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:08.147291+0000","last_undegraded":"2024-09-13T16:48:08.147291+0000","last_fullsized":"2024-09-13T16:48:08.147291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158941+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:08.158941+0000","last_peered":"2024-09-13T16:48:08.158941+0000","last_clean":"2024-09-13T16:48:08.158941+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:08.158941+0000","last_undegraded":"2024-09-13T16:48:08.158941+0000","last_fullsized":"2024-09-13T16:48:08.158941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175068+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T16:48:09.175068+0000","last_peered":"2024-09-13T16:48:09.175068+0000","last_clean":"2024-09-13T16:48:09.175068+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T16:48:09.175068+0000","last_undegraded":"2024-09-13T16:48:09.175068+0000","last_fullsized":"2024-09-13T16:48:09.175068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.182997+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T16:48:08.182997+0000","last_peered":"2024-09-13T16:48:08.182997+0000","last_clean":"2024-09-13T16:48:08.182997+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T16:48:08.182997+0000","last_undegraded":"2024-09-13T16:48:08.182997+0000","last_fullsized":"2024-09-13T16:48:08.182997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143381+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:08.143381+0000","last_peered":"2024-09-13T16:48:08.143381+0000","last_clean":"2024-09-13T16:48:08.143381+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:08.143381+0000","last_undegraded":"2024-09-13T16:48:08.143381+0000","last_fullsized":"2024-09-13T16:48:08.143381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143921+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:08.143921+0000","last_peered":"2024-09-13T16:48:08.143921+0000","last_clean":"2024-09-13T16:48:08.143921+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:08.143921+0000","last_undegraded":"2024-09-13T16:48:08.143921+0000","last_fullsized":"2024-09-13T16:48:08.143921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174477+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T16:48:09.174477+0000","last_peered":"2024-09-13T16:48:09.174477+0000","last_clean":"2024-09-13T16:48:09.174477+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T16:48:09.174477+0000","last_undegraded":"2024-09-13T16:48:09.174477+0000","last_fullsized":"2024-09-13T16:48:09.174477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147114+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:08.147114+0000","last_peered":"2024-09-13T16:48:08.147114+0000","last_clean":"2024-09-13T16:48:08.147114+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:08.147114+0000","last_undegraded":"2024-09-13T16:48:08.147114+0000","last_fullsized":"2024-09-13T16:48:08.147114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161410+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:08.161410+0000","last_peered":"2024-09-13T16:48:08.161410+0000","last_clean":"2024-09-13T16:48:08.161410+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:08.161410+0000","last_undegraded":"2024-09-13T16:48:08.161410+0000","last_fullsized":"2024-09-13T16:48:08.161410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170387+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T16:48:08.170387+0000","last_peered":"2024-09-13T16:48:08.170387+0000","last_clean":"2024-09-13T16:48:08.170387+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T16:48:08.170387+0000","last_undegraded":"2024-09-13T16:48:08.170387+0000","last_fullsized":"2024-09-13T16:48:08.170387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175603+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T16:48:09.175603+0000","last_peered":"2024-09-13T16:48:09.175603+0000","last_clean":"2024-09-13T16:48:09.175603+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T16:48:09.175603+0000","last_undegraded":"2024-09-13T16:48:09.175603+0000","last_fullsized":"2024-09-13T16:48:09.175603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144443+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:08.144443+0000","last_peered":"2024-09-13T16:48:08.144443+0000","last_clean":"2024-09-13T16:48:08.144443+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:08.144443+0000","last_undegraded":"2024-09-13T16:48:08.144443+0000","last_fullsized":"2024-09-13T16:48:08.144443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185002+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:08.185002+0000","last_peered":"2024-09-13T16:48:08.185002+0000","last_clean":"2024-09-13T16:48:08.185002+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:08.185002+0000","last_undegraded":"2024-09-13T16:48:08.185002+0000","last_fullsized":"2024-09-13T16:48:08.185002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170142+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T16:48:08.170142+0000","last_peered":"2024-09-13T16:48:08.170142+0000","last_clean":"2024-09-13T16:48:08.170142+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T16:48:08.170142+0000","last_undegraded":"2024-09-13T16:48:08.170142+0000","last_fullsized":"2024-09-13T16:48:08.170142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177615+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T16:48:09.177615+0000","last_peered":"2024-09-13T16:48:09.177615+0000","last_clean":"2024-09-13T16:48:09.177615+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T16:48:09.177615+0000","last_undegraded":"2024-09-13T16:48:09.177615+0000","last_fullsized":"2024-09-13T16:48:09.177615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144121+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:08.144121+0000","last_peered":"2024-09-13T16:48:08.144121+0000","last_clean":"2024-09-13T16:48:08.144121+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:08.144121+0000","last_undegraded":"2024-09-13T16:48:08.144121+0000","last_fullsized":"2024-09-13T16:48:08.144121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174330+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T16:48:09.174330+0000","last_peered":"2024-09-13T16:48:09.174330+0000","last_clean":"2024-09-13T16:48:09.174330+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T16:48:09.174330+0000","last_undegraded":"2024-09-13T16:48:09.174330+0000","last_fullsized":"2024-09-13T16:48:09.174330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153996+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:08.153996+0000","last_peered":"2024-09-13T16:48:08.153996+0000","last_clean":"2024-09-13T16:48:08.153996+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:08.153996+0000","last_undegraded":"2024-09-13T16:48:08.153996+0000","last_fullsized":"2024-09-13T16:48:08.153996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161029+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T16:48:09.161029+0000","last_peered":"2024-09-13T16:48:09.161029+0000","last_clean":"2024-09-13T16:48:09.161029+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T16:48:09.161029+0000","last_undegraded":"2024-09-13T16:48:09.161029+0000","last_fullsized":"2024-09-13T16:48:09.161029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154547+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:08.154547+0000","last_peered":"2024-09-13T16:48:08.154547+0000","last_clean":"2024-09-13T16:48:08.154547+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:08.154547+0000","last_undegraded":"2024-09-13T16:48:08.154547+0000","last_fullsized":"2024-09-13T16:48:08.154547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174304+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T16:48:09.174304+0000","last_peered":"2024-09-13T16:48:09.174304+0000","last_clean":"2024-09-13T16:48:09.174304+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T16:48:09.174304+0000","last_undegraded":"2024-09-13T16:48:09.174304+0000","last_fullsized":"2024-09-13T16:48:09.174304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161339+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:08.161339+0000","last_peered":"2024-09-13T16:48:08.161339+0000","last_clean":"2024-09-13T16:48:08.161339+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:08.161339+0000","last_undegraded":"2024-09-13T16:48:08.161339+0000","last_fullsized":"2024-09-13T16:48:08.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173686+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T16:48:09.173686+0000","last_peered":"2024-09-13T16:48:09.173686+0000","last_clean":"2024-09-13T16:48:09.173686+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T16:48:09.173686+0000","last_undegraded":"2024-09-13T16:48:09.173686+0000","last_fullsized":"2024-09-13T16:48:09.173686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145104+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:08.145104+0000","last_peered":"2024-09-13T16:48:08.145104+0000","last_clean":"2024-09-13T16:48:08.145104+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:08.145104+0000","last_undegraded":"2024-09-13T16:48:08.145104+0000","last_fullsized":"2024-09-13T16:48:08.145104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175359+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T16:48:09.175359+0000","last_peered":"2024-09-13T16:48:09.175359+0000","last_clean":"2024-09-13T16:48:09.175359+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T16:48:09.175359+0000","last_undegraded":"2024-09-13T16:48:09.175359+0000","last_fullsized":"2024-09-13T16:48:09.175359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149320+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:08.149320+0000","last_peered":"2024-09-13T16:48:08.149320+0000","last_clean":"2024-09-13T16:48:08.149320+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:08.149320+0000","last_undegraded":"2024-09-13T16:48:08.149320+0000","last_fullsized":"2024-09-13T16:48:08.149320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173205+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T16:48:09.173205+0000","last_peered":"2024-09-13T16:48:09.173205+0000","last_clean":"2024-09-13T16:48:09.173205+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T16:48:09.173205+0000","last_undegraded":"2024-09-13T16:48:09.173205+0000","last_fullsized":"2024-09-13T16:48:09.173205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149001+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:08.149001+0000","last_peered":"2024-09-13T16:48:08.149001+0000","last_clean":"2024-09-13T16:48:08.149001+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:08.149001+0000","last_undegraded":"2024-09-13T16:48:08.149001+0000","last_fullsized":"2024-09-13T16:48:08.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158758+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T16:48:09.158758+0000","last_peered":"2024-09-13T16:48:09.158758+0000","last_clean":"2024-09-13T16:48:09.158758+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T16:48:09.158758+0000","last_undegraded":"2024-09-13T16:48:09.158758+0000","last_fullsized":"2024-09-13T16:48:09.158758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145105+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:08.145105+0000","last_peered":"2024-09-13T16:48:08.145105+0000","last_clean":"2024-09-13T16:48:08.145105+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:08.145105+0000","last_undegraded":"2024-09-13T16:48:08.145105+0000","last_fullsized":"2024-09-13T16:48:08.145105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177538+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T16:48:09.177538+0000","last_peered":"2024-09-13T16:48:09.177538+0000","last_clean":"2024-09-13T16:48:09.177538+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T16:48:09.177538+0000","last_undegraded":"2024-09-13T16:48:09.177538+0000","last_fullsized":"2024-09-13T16:48:09.177538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158985+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:08.158985+0000","last_peered":"2024-09-13T16:48:08.158985+0000","last_clean":"2024-09-13T16:48:08.158985+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:08.158985+0000","last_undegraded":"2024-09-13T16:48:08.158985+0000","last_fullsized":"2024-09-13T16:48:08.158985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177461+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T16:48:09.177461+0000","last_peered":"2024-09-13T16:48:09.177461+0000","last_clean":"2024-09-13T16:48:09.177461+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T16:48:09.177461+0000","last_undegraded":"2024-09-13T16:48:09.177461+0000","last_fullsized":"2024-09-13T16:48:09.177461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143902+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:08.143902+0000","last_peered":"2024-09-13T16:48:08.143902+0000","last_clean":"2024-09-13T16:48:08.143902+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:08.143902+0000","last_undegraded":"2024-09-13T16:48:08.143902+0000","last_fullsized":"2024-09-13T16:48:08.143902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159127+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T16:48:09.159127+0000","last_peered":"2024-09-13T16:48:09.159127+0000","last_clean":"2024-09-13T16:48:09.159127+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T16:48:09.159127+0000","last_undegraded":"2024-09-13T16:48:09.159127+0000","last_fullsized":"2024-09-13T16:48:09.159127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148988+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:08.148988+0000","last_peered":"2024-09-13T16:48:08.148988+0000","last_clean":"2024-09-13T16:48:08.148988+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:08.148988+0000","last_undegraded":"2024-09-13T16:48:08.148988+0000","last_fullsized":"2024-09-13T16:48:08.148988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177190+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T16:48:09.177190+0000","last_peered":"2024-09-13T16:48:09.177190+0000","last_clean":"2024-09-13T16:48:09.177190+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T16:48:09.177190+0000","last_undegraded":"2024-09-13T16:48:09.177190+0000","last_fullsized":"2024-09-13T16:48:09.177190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161327+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:08.161327+0000","last_peered":"2024-09-13T16:48:08.161327+0000","last_clean":"2024-09-13T16:48:08.161327+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:08.161327+0000","last_undegraded":"2024-09-13T16:48:08.161327+0000","last_fullsized":"2024-09-13T16:48:08.161327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148225+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:08.148225+0000","last_peered":"2024-09-13T16:48:08.148225+0000","last_clean":"2024-09-13T16:48:08.148225+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:08.148225+0000","last_undegraded":"2024-09-13T16:48:08.148225+0000","last_fullsized":"2024-09-13T16:48:08.148225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175099+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T16:48:09.175099+0000","last_peered":"2024-09-13T16:48:09.175099+0000","last_clean":"2024-09-13T16:48:09.175099+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T16:48:09.175099+0000","last_undegraded":"2024-09-13T16:48:09.175099+0000","last_fullsized":"2024-09-13T16:48:09.175099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179166+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:08.179166+0000","last_peered":"2024-09-13T16:48:08.179166+0000","last_clean":"2024-09-13T16:48:08.179166+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:08.179166+0000","last_undegraded":"2024-09-13T16:48:08.179166+0000","last_fullsized":"2024-09-13T16:48:08.179166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158364+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T16:48:09.158364+0000","last_peered":"2024-09-13T16:48:09.158364+0000","last_clean":"2024-09-13T16:48:09.158364+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T16:48:09.158364+0000","last_undegraded":"2024-09-13T16:48:09.158364+0000","last_fullsized":"2024-09-13T16:48:09.158364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155807+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:08.155807+0000","last_peered":"2024-09-13T16:48:08.155807+0000","last_clean":"2024-09-13T16:48:08.155807+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:08.155807+0000","last_undegraded":"2024-09-13T16:48:08.155807+0000","last_fullsized":"2024-09-13T16:48:08.155807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.172410+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T16:48:09.172410+0000","last_peered":"2024-09-13T16:48:09.172410+0000","last_clean":"2024-09-13T16:48:09.172410+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T16:48:09.172410+0000","last_undegraded":"2024-09-13T16:48:09.172410+0000","last_fullsized":"2024-09-13T16:48:09.172410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.157093+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:08.157093+0000","last_peered":"2024-09-13T16:48:08.157093+0000","last_clean":"2024-09-13T16:48:08.157093+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:08.157093+0000","last_undegraded":"2024-09-13T16:48:08.157093+0000","last_fullsized":"2024-09-13T16:48:08.157093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173535+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T16:48:09.173535+0000","last_peered":"2024-09-13T16:48:09.173535+0000","last_clean":"2024-09-13T16:48:09.173535+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T16:48:09.173535+0000","last_undegraded":"2024-09-13T16:48:09.173535+0000","last_fullsized":"2024-09-13T16:48:09.173535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158715+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:08.158715+0000","last_peered":"2024-09-13T16:48:08.158715+0000","last_clean":"2024-09-13T16:48:08.158715+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:08.158715+0000","last_undegraded":"2024-09-13T16:48:08.158715+0000","last_fullsized":"2024-09-13T16:48:08.158715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175097+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T16:48:09.175097+0000","last_peered":"2024-09-13T16:48:09.175097+0000","last_clean":"2024-09-13T16:48:09.175097+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T16:48:09.175097+0000","last_undegraded":"2024-09-13T16:48:09.175097+0000","last_fullsized":"2024-09-13T16:48:09.175097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147419+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:08.147419+0000","last_peered":"2024-09-13T16:48:08.147419+0000","last_clean":"2024-09-13T16:48:08.147419+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:08.147419+0000","last_undegraded":"2024-09-13T16:48:08.147419+0000","last_fullsized":"2024-09-13T16:48:08.147419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159429+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T16:48:09.159429+0000","last_peered":"2024-09-13T16:48:09.159429+0000","last_clean":"2024-09-13T16:48:09.159429+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T16:48:09.159429+0000","last_undegraded":"2024-09-13T16:48:09.159429+0000","last_fullsized":"2024-09-13T16:48:09.159429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161028+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:08.161028+0000","last_peered":"2024-09-13T16:48:08.161028+0000","last_clean":"2024-09-13T16:48:08.161028+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:08.161028+0000","last_undegraded":"2024-09-13T16:48:08.161028+0000","last_fullsized":"2024-09-13T16:48:08.161028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177095+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T16:48:09.177095+0000","last_peered":"2024-09-13T16:48:09.177095+0000","last_clean":"2024-09-13T16:48:09.177095+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T16:48:09.177095+0000","last_undegraded":"2024-09-13T16:48:09.177095+0000","last_fullsized":"2024-09-13T16:48:09.177095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143741+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:08.143741+0000","last_peered":"2024-09-13T16:48:08.143741+0000","last_clean":"2024-09-13T16:48:08.143741+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:08.143741+0000","last_undegraded":"2024-09-13T16:48:08.143741+0000","last_fullsized":"2024-09-13T16:48:08.143741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176397+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T16:48:09.176397+0000","last_peered":"2024-09-13T16:48:09.176397+0000","last_clean":"2024-09-13T16:48:09.176397+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T16:48:09.176397+0000","last_undegraded":"2024-09-13T16:48:09.176397+0000","last_fullsized":"2024-09-13T16:48:09.176397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186343+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:08.186343+0000","last_peered":"2024-09-13T16:48:08.186343+0000","last_clean":"2024-09-13T16:48:08.186343+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:08.186343+0000","last_undegraded":"2024-09-13T16:48:08.186343+0000","last_fullsized":"2024-09-13T16:48:08.186343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158832+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T16:48:09.158832+0000","last_peered":"2024-09-13T16:48:09.158832+0000","last_clean":"2024-09-13T16:48:09.158832+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T16:48:09.158832+0000","last_undegraded":"2024-09-13T16:48:09.158832+0000","last_fullsized":"2024-09-13T16:48:09.158832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149361+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:08.149361+0000","last_peered":"2024-09-13T16:48:08.149361+0000","last_clean":"2024-09-13T16:48:08.149361+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:08.149361+0000","last_undegraded":"2024-09-13T16:48:08.149361+0000","last_fullsized":"2024-09-13T16:48:08.149361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173784+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T16:48:09.173784+0000","last_peered":"2024-09-13T16:48:09.173784+0000","last_clean":"2024-09-13T16:48:09.173784+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T16:48:09.173784+0000","last_undegraded":"2024-09-13T16:48:09.173784+0000","last_fullsized":"2024-09-13T16:48:09.173784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148791+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:08.148791+0000","last_peered":"2024-09-13T16:48:08.148791+0000","last_clean":"2024-09-13T16:48:08.148791+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:08.148791+0000","last_undegraded":"2024-09-13T16:48:08.148791+0000","last_fullsized":"2024-09-13T16:48:08.148791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159787+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T16:48:09.159787+0000","last_peered":"2024-09-13T16:48:09.159787+0000","last_clean":"2024-09-13T16:48:09.159787+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T16:48:09.159787+0000","last_undegraded":"2024-09-13T16:48:09.159787+0000","last_fullsized":"2024-09-13T16:48:09.159787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166109+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:08.166109+0000","last_peered":"2024-09-13T16:48:08.166109+0000","last_clean":"2024-09-13T16:48:08.166109+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:08.166109+0000","last_undegraded":"2024-09-13T16:48:08.166109+0000","last_fullsized":"2024-09-13T16:48:08.166109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.178540+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T16:48:09.178540+0000","last_peered":"2024-09-13T16:48:09.178540+0000","last_clean":"2024-09-13T16:48:09.178540+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T16:48:09.178540+0000","last_undegraded":"2024-09-13T16:48:09.178540+0000","last_fullsized":"2024-09-13T16:48:09.178540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143140+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:08.143140+0000","last_peered":"2024-09-13T16:48:08.143140+0000","last_clean":"2024-09-13T16:48:08.143140+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:08.143140+0000","last_undegraded":"2024-09-13T16:48:08.143140+0000","last_fullsized":"2024-09-13T16:48:08.143140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.160931+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T16:48:09.160931+0000","last_peered":"2024-09-13T16:48:09.160931+0000","last_clean":"2024-09-13T16:48:09.160931+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T16:48:09.160931+0000","last_undegraded":"2024-09-13T16:48:09.160931+0000","last_fullsized":"2024-09-13T16:48:09.160931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167027+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:08.167027+0000","last_peered":"2024-09-13T16:48:08.167027+0000","last_clean":"2024-09-13T16:48:08.167027+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:08.167027+0000","last_undegraded":"2024-09-13T16:48:08.167027+0000","last_fullsized":"2024-09-13T16:48:08.167027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173532+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T16:48:09.173532+0000","last_peered":"2024-09-13T16:48:09.173532+0000","last_clean":"2024-09-13T16:48:09.173532+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T16:48:09.173532+0000","last_undegraded":"2024-09-13T16:48:09.173532+0000","last_fullsized":"2024-09-13T16:48:09.173532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148753+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:08.148753+0000","last_peered":"2024-09-13T16:48:08.148753+0000","last_clean":"2024-09-13T16:48:08.148753+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:08.148753+0000","last_undegraded":"2024-09-13T16:48:08.148753+0000","last_fullsized":"2024-09-13T16:48:08.148753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161898+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T16:48:09.161898+0000","last_peered":"2024-09-13T16:48:09.161898+0000","last_clean":"2024-09-13T16:48:09.161898+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T16:48:09.161898+0000","last_undegraded":"2024-09-13T16:48:09.161898+0000","last_fullsized":"2024-09-13T16:48:09.161898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160351+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:08.160351+0000","last_peered":"2024-09-13T16:48:08.160351+0000","last_clean":"2024-09-13T16:48:08.160351+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:08.160351+0000","last_undegraded":"2024-09-13T16:48:08.160351+0000","last_fullsized":"2024-09-13T16:48:08.160351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176220+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T16:48:09.176220+0000","last_peered":"2024-09-13T16:48:09.176220+0000","last_clean":"2024-09-13T16:48:09.176220+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T16:48:09.176220+0000","last_undegraded":"2024-09-13T16:48:09.176220+0000","last_fullsized":"2024-09-13T16:48:09.176220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151613+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:08.151613+0000","last_peered":"2024-09-13T16:48:08.151613+0000","last_clean":"2024-09-13T16:48:08.151613+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:08.151613+0000","last_undegraded":"2024-09-13T16:48:08.151613+0000","last_fullsized":"2024-09-13T16:48:08.151613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158220+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T16:48:09.158220+0000","last_peered":"2024-09-13T16:48:09.158220+0000","last_clean":"2024-09-13T16:48:09.158220+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T16:48:09.158220+0000","last_undegraded":"2024-09-13T16:48:09.158220+0000","last_fullsized":"2024-09-13T16:48:09.158220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.162959+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:08.162959+0000","last_peered":"2024-09-13T16:48:08.162959+0000","last_clean":"2024-09-13T16:48:08.162959+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:08.162959+0000","last_undegraded":"2024-09-13T16:48:08.162959+0000","last_fullsized":"2024-09-13T16:48:08.162959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173056+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T16:48:09.173056+0000","last_peered":"2024-09-13T16:48:09.173056+0000","last_clean":"2024-09-13T16:48:09.173056+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T16:48:09.173056+0000","last_undegraded":"2024-09-13T16:48:09.173056+0000","last_fullsized":"2024-09-13T16:48:09.173056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.150218+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:08.150218+0000","last_peered":"2024-09-13T16:48:08.150218+0000","last_clean":"2024-09-13T16:48:08.150218+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:08.150218+0000","last_undegraded":"2024-09-13T16:48:08.150218+0000","last_fullsized":"2024-09-13T16:48:08.150218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173906+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T16:48:09.173906+0000","last_peered":"2024-09-13T16:48:09.173906+0000","last_clean":"2024-09-13T16:48:09.173906+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T16:48:09.173906+0000","last_undegraded":"2024-09-13T16:48:09.173906+0000","last_fullsized":"2024-09-13T16:48:09.173906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166070+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:08.166070+0000","last_peered":"2024-09-13T16:48:08.166070+0000","last_clean":"2024-09-13T16:48:08.166070+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:08.166070+0000","last_undegraded":"2024-09-13T16:48:08.166070+0000","last_fullsized":"2024-09-13T16:48:08.166070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":690265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":101405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":99455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":282624,"data_stored":102375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":258048,"data_stored":99295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":100971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":690231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:13.969 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T16:48:14.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.124+0000 7f908e16b640 1 Processor -- start 2024-09-13T16:48:14.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.124+0000 7f908e16b640 1 -- start start 2024-09-13T16:48:14.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f908e16b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f90880aabc0 0x7f90880aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f908e16b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f90880c40b0 0x7f908815c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f908e16b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9088152a80 con 0x7f908806c870 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f908e16b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90880c30f0 con 0x7f90880aabc0 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f908e16b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9088151e00 con 0x7f90880c40b0 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f90877fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f90880aabc0 0x7f90880aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f90877fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f90880aabc0 0x7f90880aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34282/0 (socket says 172.21.15.39:34282) 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f90877fe640 1 -- 172.21.15.39:0/2176140565 learned_addr learned my addr 172.21.15.39:0/2176140565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f9087fff640 1 --2- 172.21.15.39:0/2176140565 >> v2:172.21.15.84:3300/0 conn(0x7f90880c40b0 0x7f908815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f90877fe640 1 -- 172.21.15.39:0/2176140565 >> v2:172.21.15.84:3300/0 conn(0x7f90880c40b0 msgr2=0x7f908815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f90877fe640 1 --2- 172.21.15.39:0/2176140565 >> v2:172.21.15.84:3300/0 conn(0x7f90880c40b0 0x7f908815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f90877fe640 1 -- 172.21.15.39:0/2176140565 >> v1:172.21.15.84:6789/0 conn(0x7f908806c870 legacy=0x7f90880c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f90877fe640 1 -- 172.21.15.39:0/2176140565 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90881520e0 con 0x7f90880aabc0 2024-09-13T16:48:14.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f90877fe640 1 --2- 172.21.15.39:0/2176140565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880aabc0 0x7f90880aafa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f908806f220 tx=0x7f907801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1e53affb98f6ff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:14.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.128+0000 7f90867fc640 1 -- 172.21.15.39:0/2176140565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780ab620 con 0x7f90880aabc0 2024-09-13T16:48:14.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f90867fc640 1 -- 172.21.15.39:0/2176140565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90780ab780 con 0x7f90880aabc0 2024-09-13T16:48:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f90867fc640 1 -- 172.21.15.39:0/2176140565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780abb10 con 0x7f90880aabc0 2024-09-13T16:48:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- 172.21.15.39:0/2176140565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880aabc0 msgr2=0x7f90880aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 --2- 172.21.15.39:0/2176140565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880aabc0 0x7f90880aafa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f908806f220 tx=0x7f907801c420 comp rx=0 tx=0).stop 2024-09-13T16:48:14.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- 172.21.15.39:0/2176140565 shutdown_connections 2024-09-13T16:48:14.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 --2- 172.21.15.39:0/2176140565 >> v2:172.21.15.84:3300/0 conn(0x7f90880c40b0 0x7f908815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 --2- 172.21.15.39:0/2176140565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880aabc0 0x7f90880aafa0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- 172.21.15.39:0/2176140565 >> 172.21.15.39:0/2176140565 conn(0x7f9088069e10 msgr2=0x7f908806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:14.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- 172.21.15.39:0/2176140565 shutdown_connections 2024-09-13T16:48:14.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- 172.21.15.39:0/2176140565 wait complete. 2024-09-13T16:48:14.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 Processor -- start 2024-09-13T16:48:14.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- start start 2024-09-13T16:48:14.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908806c870 0x7f9088148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f90880aabc0 0x7f9088072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880c40b0 0x7f90880a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9088071270 con 0x7f90880c40b0 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90881555c0 con 0x7f908806c870 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90880a7c40 con 0x7f90880aabc0 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9087fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880c40b0 0x7f90880a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9087fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880c40b0 0x7f90880a6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34296/0 (socket says 172.21.15.39:34296) 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9087fff640 1 -- 172.21.15.39:0/939106101 learned_addr learned my addr 172.21.15.39:0/939106101 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f90877fe640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908806c870 0x7f9088148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9086ffd640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f90880aabc0 0x7f9088072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9087fff640 1 -- 172.21.15.39:0/939106101 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f90880aabc0 msgr2=0x7f9088072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9087fff640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f90880aabc0 0x7f9088072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9087fff640 1 -- 172.21.15.39:0/939106101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908806c870 msgr2=0x7f9088148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9087fff640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908806c870 0x7f9088148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9087fff640 1 -- 172.21.15.39:0/939106101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90780094d0 con 0x7f90880c40b0 2024-09-13T16:48:14.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f9087fff640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880c40b0 0x7f90880a6e80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9080007e60 tx=0x7f908000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:14.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f906ffff640 1 -- 172.21.15.39:0/939106101 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080014420 con 0x7f90880c40b0 2024-09-13T16:48:14.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f906ffff640 1 -- 172.21.15.39:0/939106101 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90800145a0 con 0x7f90880c40b0 2024-09-13T16:48:14.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90880a7f20 con 0x7f90880c40b0 2024-09-13T16:48:14.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f906ffff640 1 -- 172.21.15.39:0/939106101 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080014950 con 0x7f90880c40b0 2024-09-13T16:48:14.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.132+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90880c3db0 con 0x7f90880c40b0 2024-09-13T16:48:14.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.136+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9088071e10 con 0x7f90880c40b0 2024-09-13T16:48:14.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.136+0000 7f906ffff640 1 -- 172.21.15.39:0/939106101 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9080014ab0 con 0x7f90880c40b0 2024-09-13T16:48:14.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.136+0000 7f906ffff640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f90740666c0 0x7f9074068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.136+0000 7f906ffff640 1 -- 172.21.15.39:0/939106101 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f9080084dd0 con 0x7f90880c40b0 2024-09-13T16:48:14.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.136+0000 7f90877fe640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f90740666c0 0x7f9074068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.136+0000 7f90877fe640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f90740666c0 0x7f9074068b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f90780af000 tx=0x7f90780a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:14.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.140+0000 7f906ffff640 1 -- 172.21.15.39:0/939106101 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9080051e10 con 0x7f90880c40b0 2024-09-13T16:48:14.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.356+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f90880aafa0 con 0x7f90880c40b0 2024-09-13T16:48:14.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.356+0000 7f906ffff640 1 -- 172.21.15.39:0/939106101 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1114 (secure 0 0 0) 0x7f9080086470 con 0x7f90880c40b0 2024-09-13T16:48:14.361 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:14.361 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":39,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":24,"num_osds":8,"num_up_osds":8,"osd_up_since":1726246066,"num_in_osds":8,"osd_in_since":1726246057,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":193}],"num_pgs":193,"num_pools":5,"num_objects":45,"data_bytes":595092,"bytes_used":241475584,"bytes_avail":772852637696,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":14,"id":1,"up":1,"in":1,"max":1,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4260},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4264}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-13T16:47:34.826912+0000","services":{}},"progress_events":{}} 2024-09-13T16:48:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f90740666c0 msgr2=0x7f9074068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f90740666c0 0x7f9074068b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f90780af000 tx=0x7f90780a7000 comp rx=0 tx=0).stop 2024-09-13T16:48:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880c40b0 msgr2=0x7f90880a6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880c40b0 0x7f90880a6e80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9080007e60 tx=0x7f908000e4b0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 shutdown_connections 2024-09-13T16:48:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f90740666c0 0x7f9074068b80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90880c40b0 0x7f90880a6e80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f90880aabc0 0x7f9088072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 --2- 172.21.15.39:0/939106101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f908806c870 0x7f9088148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 >> 172.21.15.39:0/939106101 conn(0x7f9088069e10 msgr2=0x7f9088150fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:14.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 shutdown_connections 2024-09-13T16:48:14.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.360+0000 7f908e16b640 1 -- 172.21.15.39:0/939106101 wait complete. 2024-09-13T16:48:14.379 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-09-13T16:48:14.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:14.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b70a18640 1 Processor -- start 2024-09-13T16:48:14.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b70a18640 1 -- start start 2024-09-13T16:48:14.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b70a18640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1b6c0aafa0 0x7f1b6c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b70a18640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c152a80 con 0x7f1b6c0aabc0 2024-09-13T16:48:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b70a18640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c0ab410 con 0x7f1b6c06c870 2024-09-13T16:48:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b70a18640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c0c30f0 con 0x7f1b6c0aafa0 2024-09-13T16:48:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b6a59c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1b6c0aabc0 0x7f1b6c0c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33126/0 (socket says 172.21.15.39:33126) 2024-09-13T16:48:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b6a59c640 1 -- 172.21.15.39:0/1915910728 learned_addr learned my addr 172.21.15.39:0/1915910728 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636363618 0 0) 0x7f1b5c001120 con 0x7f1b6c0aabc0 2024-09-13T16:48:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b6ad9d640 1 --2- 172.21.15.39:0/1915910728 >> v2:172.21.15.84:3300/0 conn(0x7f1b6c0aafa0 0x7f1b6c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b58003530 con 0x7f1b6c0aabc0 2024-09-13T16:48:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 789048445 0 0) 0x7f1b58003530 con 0x7f1b6c0aabc0 2024-09-13T16:48:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 >> v2:172.21.15.84:3300/0 conn(0x7f1b6c0aafa0 msgr2=0x7f1b6c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 --2- 172.21.15.39:0/1915910728 >> v2:172.21.15.84:3300/0 conn(0x7f1b6c0aafa0 0x7f1b6c15d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 >> v1:172.21.15.84:6789/0 conn(0x7f1b6c06c870 legacy=0x7f1b6c0c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c151e00 con 0x7f1b6c0aabc0 2024-09-13T16:48:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1b5c001810 con 0x7f1b6c0aabc0 2024-09-13T16:48:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 >> v1:172.21.15.39:6789/0 conn(0x7f1b6c0aabc0 legacy=0x7f1b6c0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b58003980 0x7f1b58003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b580052b0 0x7f1b5800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0c30f0 con 0x7f1b580047d0 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5800c180 con 0x7f1b580052b0 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5800c340 con 0x7f1b58003980 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b61d9b640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b6a59c640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b58003980 0x7f1b58003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b61d9b640 1 -- 172.21.15.39:0/1915910728 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b58003980 msgr2=0x7f1b58003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b61d9b640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b58003980 0x7f1b58003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b61d9b640 1 -- 172.21.15.39:0/1915910728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b580052b0 msgr2=0x7f1b5800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b61d9b640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b580052b0 0x7f1b5800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b61d9b640 1 -- 172.21.15.39:0/1915910728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b58003530 con 0x7f1b580047d0 2024-09-13T16:48:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.492+0000 7f1b6ad9d640 1 -- 172.21.15.39:0/1915910728 reap_dead start 2024-09-13T16:48:14.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b61d9b640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b54001750 tx=0x7f1b5401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5505f61ea1396b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:14.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b540052d0 con 0x7f1b580047d0 2024-09-13T16:48:14.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b540a7050 con 0x7f1b580047d0 2024-09-13T16:48:14.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b69d9b640 1 -- 172.21.15.39:0/1915910728 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54003600 con 0x7f1b580047d0 2024-09-13T16:48:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 -- 172.21.15.39:0/1915910728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b580047d0 msgr2=0x7f1b58004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1b54001750 tx=0x7f1b5401c390 comp rx=0 tx=0).stop 2024-09-13T16:48:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 -- 172.21.15.39:0/1915910728 shutdown_connections 2024-09-13T16:48:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 --2- 172.21.15.39:0/1915910728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 -- 172.21.15.39:0/1915910728 >> 172.21.15.39:0/1915910728 conn(0x7f1b6c069e10 msgr2=0x7f1b6c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 -- 172.21.15.39:0/1915910728 shutdown_connections 2024-09-13T16:48:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 -- 172.21.15.39:0/1915910728 wait complete. 2024-09-13T16:48:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 Processor -- start 2024-09-13T16:48:14.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 -- start start 2024-09-13T16:48:14.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b580047d0 0x7f1b6c0a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b6c071430 0x7f1b6c149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c06f340 0x7f1b6c14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0700f0 con 0x7f1b6c06f340 2024-09-13T16:48:14.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c14a710 con 0x7f1b580047d0 2024-09-13T16:48:14.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b70a18640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c149760 con 0x7f1b6c071430 2024-09-13T16:48:14.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c06f340 0x7f1b6c14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c06f340 0x7f1b6c14a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34314/0 (socket says 172.21.15.39:34314) 2024-09-13T16:48:14.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6ad9d640 1 -- 172.21.15.39:0/3041159025 learned_addr learned my addr 172.21.15.39:0/3041159025 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:14.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6a59c640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b580047d0 0x7f1b6c0a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b61d9b640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b6c071430 0x7f1b6c149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6ad9d640 1 -- 172.21.15.39:0/3041159025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b6c071430 msgr2=0x7f1b6c149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6ad9d640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b6c071430 0x7f1b6c149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6ad9d640 1 -- 172.21.15.39:0/3041159025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b580047d0 msgr2=0x7f1b6c0a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6ad9d640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b580047d0 0x7f1b6c0a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6ad9d640 1 -- 172.21.15.39:0/3041159025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c06fa30 con 0x7f1b6c06f340 2024-09-13T16:48:14.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.496+0000 7f1b6ad9d640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c06f340 0x7f1b6c14a1d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0ab380 tx=0x7f1b6400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:14.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b637fe640 1 -- 172.21.15.39:0/3041159025 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64010040 con 0x7f1b6c06f340 2024-09-13T16:48:14.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b637fe640 1 -- 172.21.15.39:0/3041159025 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6400bb90 con 0x7f1b6c06f340 2024-09-13T16:48:14.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b637fe640 1 -- 172.21.15.39:0/3041159025 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64014400 con 0x7f1b6c06f340 2024-09-13T16:48:14.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c06fd10 con 0x7f1b6c06f340 2024-09-13T16:48:14.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c0a7510 con 0x7f1b6c06f340 2024-09-13T16:48:14.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c06f870 con 0x7f1b6c06f340 2024-09-13T16:48:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b637fe640 1 -- 172.21.15.39:0/3041159025 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b64014560 con 0x7f1b6c06f340 2024-09-13T16:48:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b637fe640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b4c0666c0 0x7f1b4c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b637fe640 1 -- 172.21.15.39:0/3041159025 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f1b64084b90 con 0x7f1b6c06f340 2024-09-13T16:48:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b6a59c640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b4c0666c0 0x7f1b4c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.500+0000 7f1b6a59c640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b4c0666c0 0x7f1b4c068b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c001e20 tx=0x7f1b5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:14.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.504+0000 7f1b637fe640 1 -- 172.21.15.39:0/3041159025 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b64051c40 con 0x7f1b6c06f340 2024-09-13T16:48:14.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.656+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b6c151af0 con 0x7f1b4c0666c0 2024-09-13T16:48:14.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.688+0000 7f1b637fe640 1 -- 172.21.15.39:0/3041159025 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+503124 (secure 0 0 0) 0x7f1b5c001180 con 0x7f1b4c0666c0 2024-09-13T16:48:14.698 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:14.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b4c0666c0 msgr2=0x7f1b4c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b4c0666c0 0x7f1b4c068b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c001e20 tx=0x7f1b5c009040 comp rx=0 tx=0).stop 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c06f340 msgr2=0x7f1b6c14a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c06f340 0x7f1b6c14a1d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0ab380 tx=0x7f1b6400ef50 comp rx=0 tx=0).stop 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 shutdown_connections 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b4c0666c0 0x7f1b4c068b80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b6c06f340 0x7f1b6c14a1d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b6c071430 0x7f1b6c149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 --2- 172.21.15.39:0/3041159025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b580047d0 0x7f1b6c0a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 >> 172.21.15.39:0/3041159025 conn(0x7f1b6c069e10 msgr2=0x7f1b6c0c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 shutdown_connections 2024-09-13T16:48:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:14.696+0000 7f1b70a18640 1 -- 172.21.15.39:0/3041159025 wait complete. 2024-09-13T16:48:14.715 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-13T16:48:13.561783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595092,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":248,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194,"ondisk_log_size":194,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":375,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235816,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738904,"statfs":{"total":773094113280,"available":772852637696,"internally_reserved":0,"allocated":3350528,"data_stored":1983984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001967"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174677+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T16:48:09.174677+0000","last_peered":"2024-09-13T16:48:09.174677+0000","last_clean":"2024-09-13T16:48:09.174677+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T16:48:09.174677+0000","last_undegraded":"2024-09-13T16:48:09.174677+0000","last_fullsized":"2024-09-13T16:48:09.174677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163864+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:08.163864+0000","last_peered":"2024-09-13T16:48:08.163864+0000","last_clean":"2024-09-13T16:48:08.163864+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:08.163864+0000","last_undegraded":"2024-09-13T16:48:08.163864+0000","last_fullsized":"2024-09-13T16:48:08.163864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146105+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:08.146105+0000","last_peered":"2024-09-13T16:48:08.146105+0000","last_clean":"2024-09-13T16:48:08.146105+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:08.146105+0000","last_undegraded":"2024-09-13T16:48:08.146105+0000","last_fullsized":"2024-09-13T16:48:08.146105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143176+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T16:48:08.143176+0000","last_peered":"2024-09-13T16:48:08.143176+0000","last_clean":"2024-09-13T16:48:08.143176+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T16:48:08.143176+0000","last_undegraded":"2024-09-13T16:48:08.143176+0000","last_fullsized":"2024-09-13T16:48:08.143176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173274+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T16:48:09.173274+0000","last_peered":"2024-09-13T16:48:09.173274+0000","last_clean":"2024-09-13T16:48:09.173274+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T16:48:09.173274+0000","last_undegraded":"2024-09-13T16:48:09.173274+0000","last_fullsized":"2024-09-13T16:48:09.173274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146246+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:08.146246+0000","last_peered":"2024-09-13T16:48:08.146246+0000","last_clean":"2024-09-13T16:48:08.146246+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:08.146246+0000","last_undegraded":"2024-09-13T16:48:08.146246+0000","last_fullsized":"2024-09-13T16:48:08.146246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163480+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:08.163480+0000","last_peered":"2024-09-13T16:48:08.163480+0000","last_clean":"2024-09-13T16:48:08.163480+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:08.163480+0000","last_undegraded":"2024-09-13T16:48:08.163480+0000","last_fullsized":"2024-09-13T16:48:08.163480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184157+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T16:48:08.184157+0000","last_peered":"2024-09-13T16:48:08.184157+0000","last_clean":"2024-09-13T16:48:08.184157+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T16:48:08.184157+0000","last_undegraded":"2024-09-13T16:48:08.184157+0000","last_fullsized":"2024-09-13T16:48:08.184157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173949+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T16:48:09.173949+0000","last_peered":"2024-09-13T16:48:09.173949+0000","last_clean":"2024-09-13T16:48:09.173949+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T16:48:09.173949+0000","last_undegraded":"2024-09-13T16:48:09.173949+0000","last_fullsized":"2024-09-13T16:48:09.173949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178923+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:08.178923+0000","last_peered":"2024-09-13T16:48:08.178923+0000","last_clean":"2024-09-13T16:48:08.178923+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:08.178923+0000","last_undegraded":"2024-09-13T16:48:08.178923+0000","last_fullsized":"2024-09-13T16:48:08.178923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151847+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:08.151847+0000","last_peered":"2024-09-13T16:48:08.151847+0000","last_clean":"2024-09-13T16:48:08.151847+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:08.151847+0000","last_undegraded":"2024-09-13T16:48:08.151847+0000","last_fullsized":"2024-09-13T16:48:08.151847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208404+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T16:48:08.208404+0000","last_peered":"2024-09-13T16:48:08.208404+0000","last_clean":"2024-09-13T16:48:08.208404+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T16:48:08.208404+0000","last_undegraded":"2024-09-13T16:48:08.208404+0000","last_fullsized":"2024-09-13T16:48:08.208404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159591+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T16:48:09.159591+0000","last_peered":"2024-09-13T16:48:09.159591+0000","last_clean":"2024-09-13T16:48:09.159591+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T16:48:09.159591+0000","last_undegraded":"2024-09-13T16:48:09.159591+0000","last_fullsized":"2024-09-13T16:48:09.159591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165397+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:08.165397+0000","last_peered":"2024-09-13T16:48:08.165397+0000","last_clean":"2024-09-13T16:48:08.165397+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:08.165397+0000","last_undegraded":"2024-09-13T16:48:08.165397+0000","last_fullsized":"2024-09-13T16:48:08.165397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153028+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:08.153028+0000","last_peered":"2024-09-13T16:48:08.153028+0000","last_clean":"2024-09-13T16:48:08.153028+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:08.153028+0000","last_undegraded":"2024-09-13T16:48:08.153028+0000","last_fullsized":"2024-09-13T16:48:08.153028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.203658+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T16:48:08.203658+0000","last_peered":"2024-09-13T16:48:08.203658+0000","last_clean":"2024-09-13T16:48:08.203658+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T16:48:08.203658+0000","last_undegraded":"2024-09-13T16:48:08.203658+0000","last_fullsized":"2024-09-13T16:48:08.203658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173632+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T16:48:09.173632+0000","last_peered":"2024-09-13T16:48:09.173632+0000","last_clean":"2024-09-13T16:48:09.173632+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T16:48:09.173632+0000","last_undegraded":"2024-09-13T16:48:09.173632+0000","last_fullsized":"2024-09-13T16:48:09.173632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146884+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:08.146884+0000","last_peered":"2024-09-13T16:48:08.146884+0000","last_clean":"2024-09-13T16:48:08.146884+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:08.146884+0000","last_undegraded":"2024-09-13T16:48:08.146884+0000","last_fullsized":"2024-09-13T16:48:08.146884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160788+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:08.160788+0000","last_peered":"2024-09-13T16:48:08.160788+0000","last_clean":"2024-09-13T16:48:08.160788+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:08.160788+0000","last_undegraded":"2024-09-13T16:48:08.160788+0000","last_fullsized":"2024-09-13T16:48:08.160788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.206623+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T16:48:08.206623+0000","last_peered":"2024-09-13T16:48:08.206623+0000","last_clean":"2024-09-13T16:48:08.206623+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T16:48:08.206623+0000","last_undegraded":"2024-09-13T16:48:08.206623+0000","last_fullsized":"2024-09-13T16:48:08.206623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159080+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T16:48:09.159080+0000","last_peered":"2024-09-13T16:48:09.159080+0000","last_clean":"2024-09-13T16:48:09.159080+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T16:48:09.159080+0000","last_undegraded":"2024-09-13T16:48:09.159080+0000","last_fullsized":"2024-09-13T16:48:09.159080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144345+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:08.144345+0000","last_peered":"2024-09-13T16:48:08.144345+0000","last_clean":"2024-09-13T16:48:08.144345+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:08.144345+0000","last_undegraded":"2024-09-13T16:48:08.144345+0000","last_fullsized":"2024-09-13T16:48:08.144345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147467+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:08.147467+0000","last_peered":"2024-09-13T16:48:08.147467+0000","last_clean":"2024-09-13T16:48:08.147467+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:08.147467+0000","last_undegraded":"2024-09-13T16:48:08.147467+0000","last_fullsized":"2024-09-13T16:48:08.147467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184081+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T16:48:08.184081+0000","last_peered":"2024-09-13T16:48:08.184081+0000","last_clean":"2024-09-13T16:48:08.184081+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T16:48:08.184081+0000","last_undegraded":"2024-09-13T16:48:08.184081+0000","last_fullsized":"2024-09-13T16:48:08.184081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143476+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:08.143476+0000","last_peered":"2024-09-13T16:48:08.143476+0000","last_clean":"2024-09-13T16:48:08.143476+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:08.143476+0000","last_undegraded":"2024-09-13T16:48:08.143476+0000","last_fullsized":"2024-09-13T16:48:08.143476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147565+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:08.147565+0000","last_peered":"2024-09-13T16:48:08.147565+0000","last_clean":"2024-09-13T16:48:08.147565+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:08.147565+0000","last_undegraded":"2024-09-13T16:48:08.147565+0000","last_fullsized":"2024-09-13T16:48:08.147565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174007+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T16:48:08.174007+0000","last_peered":"2024-09-13T16:48:08.174007+0000","last_clean":"2024-09-13T16:48:08.174007+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T16:48:08.174007+0000","last_undegraded":"2024-09-13T16:48:08.174007+0000","last_fullsized":"2024-09-13T16:48:08.174007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162482+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T16:48:09.162482+0000","last_peered":"2024-09-13T16:48:09.162482+0000","last_clean":"2024-09-13T16:48:09.162482+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T16:48:09.162482+0000","last_undegraded":"2024-09-13T16:48:09.162482+0000","last_fullsized":"2024-09-13T16:48:09.162482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186454+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:08.186454+0000","last_peered":"2024-09-13T16:48:08.186454+0000","last_clean":"2024-09-13T16:48:08.186454+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:08.186454+0000","last_undegraded":"2024-09-13T16:48:08.186454+0000","last_fullsized":"2024-09-13T16:48:08.186454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160985+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:08.160985+0000","last_peered":"2024-09-13T16:48:08.160985+0000","last_clean":"2024-09-13T16:48:08.160985+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:08.160985+0000","last_undegraded":"2024-09-13T16:48:08.160985+0000","last_fullsized":"2024-09-13T16:48:08.160985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208189+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T16:48:08.208189+0000","last_peered":"2024-09-13T16:48:08.208189+0000","last_clean":"2024-09-13T16:48:08.208189+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T16:48:08.208189+0000","last_undegraded":"2024-09-13T16:48:08.208189+0000","last_fullsized":"2024-09-13T16:48:08.208189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176059+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T16:48:09.176059+0000","last_peered":"2024-09-13T16:48:09.176059+0000","last_clean":"2024-09-13T16:48:09.176059+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T16:48:09.176059+0000","last_undegraded":"2024-09-13T16:48:09.176059+0000","last_fullsized":"2024-09-13T16:48:09.176059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163768+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:08.163768+0000","last_peered":"2024-09-13T16:48:08.163768+0000","last_clean":"2024-09-13T16:48:08.163768+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:08.163768+0000","last_undegraded":"2024-09-13T16:48:08.163768+0000","last_fullsized":"2024-09-13T16:48:08.163768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148721+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:08.148721+0000","last_peered":"2024-09-13T16:48:08.148721+0000","last_clean":"2024-09-13T16:48:08.148721+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:08.148721+0000","last_undegraded":"2024-09-13T16:48:08.148721+0000","last_fullsized":"2024-09-13T16:48:08.148721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185371+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T16:48:08.185371+0000","last_peered":"2024-09-13T16:48:08.185371+0000","last_clean":"2024-09-13T16:48:08.185371+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T16:48:08.185371+0000","last_undegraded":"2024-09-13T16:48:08.185371+0000","last_fullsized":"2024-09-13T16:48:08.185371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174416+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T16:48:09.174416+0000","last_peered":"2024-09-13T16:48:09.174416+0000","last_clean":"2024-09-13T16:48:09.174416+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T16:48:09.174416+0000","last_undegraded":"2024-09-13T16:48:09.174416+0000","last_fullsized":"2024-09-13T16:48:09.174416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145183+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:08.145183+0000","last_peered":"2024-09-13T16:48:08.145183+0000","last_clean":"2024-09-13T16:48:08.145183+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:08.145183+0000","last_undegraded":"2024-09-13T16:48:08.145183+0000","last_fullsized":"2024-09-13T16:48:08.145183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152363+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:08.152363+0000","last_peered":"2024-09-13T16:48:08.152363+0000","last_clean":"2024-09-13T16:48:08.152363+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:08.152363+0000","last_undegraded":"2024-09-13T16:48:08.152363+0000","last_fullsized":"2024-09-13T16:48:08.152363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143237+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T16:48:08.143237+0000","last_peered":"2024-09-13T16:48:08.143237+0000","last_clean":"2024-09-13T16:48:08.143237+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T16:48:08.143237+0000","last_undegraded":"2024-09-13T16:48:08.143237+0000","last_fullsized":"2024-09-13T16:48:08.143237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158858+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T16:48:09.158858+0000","last_peered":"2024-09-13T16:48:09.158858+0000","last_clean":"2024-09-13T16:48:09.158858+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T16:48:09.158858+0000","last_undegraded":"2024-09-13T16:48:09.158858+0000","last_fullsized":"2024-09-13T16:48:09.158858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148446+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:08.148446+0000","last_peered":"2024-09-13T16:48:08.148446+0000","last_clean":"2024-09-13T16:48:08.148446+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:08.148446+0000","last_undegraded":"2024-09-13T16:48:08.148446+0000","last_fullsized":"2024-09-13T16:48:08.148446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179426+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:08.179426+0000","last_peered":"2024-09-13T16:48:08.179426+0000","last_clean":"2024-09-13T16:48:08.179426+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:08.179426+0000","last_undegraded":"2024-09-13T16:48:08.179426+0000","last_fullsized":"2024-09-13T16:48:08.179426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161207+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T16:48:08.161207+0000","last_peered":"2024-09-13T16:48:08.161207+0000","last_clean":"2024-09-13T16:48:08.161207+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T16:48:08.161207+0000","last_undegraded":"2024-09-13T16:48:08.161207+0000","last_fullsized":"2024-09-13T16:48:08.161207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173006+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T16:48:09.173006+0000","last_peered":"2024-09-13T16:48:09.173006+0000","last_clean":"2024-09-13T16:48:09.173006+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T16:48:09.173006+0000","last_undegraded":"2024-09-13T16:48:09.173006+0000","last_fullsized":"2024-09-13T16:48:09.173006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153222+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:08.153222+0000","last_peered":"2024-09-13T16:48:08.153222+0000","last_clean":"2024-09-13T16:48:08.153222+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:08.153222+0000","last_undegraded":"2024-09-13T16:48:08.153222+0000","last_fullsized":"2024-09-13T16:48:08.153222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148522+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:08.148522+0000","last_peered":"2024-09-13T16:48:08.148522+0000","last_clean":"2024-09-13T16:48:08.148522+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:08.148522+0000","last_undegraded":"2024-09-13T16:48:08.148522+0000","last_fullsized":"2024-09-13T16:48:08.148522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146684+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T16:48:08.146684+0000","last_peered":"2024-09-13T16:48:08.146684+0000","last_clean":"2024-09-13T16:48:08.146684+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T16:48:08.146684+0000","last_undegraded":"2024-09-13T16:48:08.146684+0000","last_fullsized":"2024-09-13T16:48:08.146684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174867+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T16:48:09.174867+0000","last_peered":"2024-09-13T16:48:09.174867+0000","last_clean":"2024-09-13T16:48:09.174867+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T16:48:09.174867+0000","last_undegraded":"2024-09-13T16:48:09.174867+0000","last_fullsized":"2024-09-13T16:48:09.174867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185855+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:08.185855+0000","last_peered":"2024-09-13T16:48:08.185855+0000","last_clean":"2024-09-13T16:48:08.185855+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:08.185855+0000","last_undegraded":"2024-09-13T16:48:08.185855+0000","last_fullsized":"2024-09-13T16:48:08.185855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149466+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:08.149466+0000","last_peered":"2024-09-13T16:48:08.149466+0000","last_clean":"2024-09-13T16:48:08.149466+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:08.149466+0000","last_undegraded":"2024-09-13T16:48:08.149466+0000","last_fullsized":"2024-09-13T16:48:08.149466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148372+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T16:48:08.148372+0000","last_peered":"2024-09-13T16:48:08.148372+0000","last_clean":"2024-09-13T16:48:08.148372+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T16:48:08.148372+0000","last_undegraded":"2024-09-13T16:48:08.148372+0000","last_fullsized":"2024-09-13T16:48:08.148372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159186+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T16:48:09.159186+0000","last_peered":"2024-09-13T16:48:09.159186+0000","last_clean":"2024-09-13T16:48:09.159186+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T16:48:09.159186+0000","last_undegraded":"2024-09-13T16:48:09.159186+0000","last_fullsized":"2024-09-13T16:48:09.159186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146860+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:08.146860+0000","last_peered":"2024-09-13T16:48:08.146860+0000","last_clean":"2024-09-13T16:48:08.146860+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:08.146860+0000","last_undegraded":"2024-09-13T16:48:08.146860+0000","last_fullsized":"2024-09-13T16:48:08.146860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154111+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:08.154111+0000","last_peered":"2024-09-13T16:48:08.154111+0000","last_clean":"2024-09-13T16:48:08.154111+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:08.154111+0000","last_undegraded":"2024-09-13T16:48:08.154111+0000","last_fullsized":"2024-09-13T16:48:08.154111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175388+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T16:48:08.175388+0000","last_peered":"2024-09-13T16:48:08.175388+0000","last_clean":"2024-09-13T16:48:08.175388+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T16:48:08.175388+0000","last_undegraded":"2024-09-13T16:48:08.175388+0000","last_fullsized":"2024-09-13T16:48:08.175388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162556+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T16:48:09.162556+0000","last_peered":"2024-09-13T16:48:09.162556+0000","last_clean":"2024-09-13T16:48:09.162556+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T16:48:09.162556+0000","last_undegraded":"2024-09-13T16:48:09.162556+0000","last_fullsized":"2024-09-13T16:48:09.162556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160990+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:08.160990+0000","last_peered":"2024-09-13T16:48:08.160990+0000","last_clean":"2024-09-13T16:48:08.160990+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:08.160990+0000","last_undegraded":"2024-09-13T16:48:08.160990+0000","last_fullsized":"2024-09-13T16:48:08.160990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161254+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:08.161254+0000","last_peered":"2024-09-13T16:48:08.161254+0000","last_clean":"2024-09-13T16:48:08.161254+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:08.161254+0000","last_undegraded":"2024-09-13T16:48:08.161254+0000","last_fullsized":"2024-09-13T16:48:08.161254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163476+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T16:48:08.163476+0000","last_peered":"2024-09-13T16:48:08.163476+0000","last_clean":"2024-09-13T16:48:08.163476+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T16:48:08.163476+0000","last_undegraded":"2024-09-13T16:48:08.163476+0000","last_fullsized":"2024-09-13T16:48:08.163476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175463+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T16:48:09.175463+0000","last_peered":"2024-09-13T16:48:09.175463+0000","last_clean":"2024-09-13T16:48:09.175463+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T16:48:09.175463+0000","last_undegraded":"2024-09-13T16:48:09.175463+0000","last_fullsized":"2024-09-13T16:48:09.175463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143792+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:08.143792+0000","last_peered":"2024-09-13T16:48:08.143792+0000","last_clean":"2024-09-13T16:48:08.143792+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:08.143792+0000","last_undegraded":"2024-09-13T16:48:08.143792+0000","last_fullsized":"2024-09-13T16:48:08.143792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148284+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:08.148284+0000","last_peered":"2024-09-13T16:48:08.148284+0000","last_clean":"2024-09-13T16:48:08.148284+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:08.148284+0000","last_undegraded":"2024-09-13T16:48:08.148284+0000","last_fullsized":"2024-09-13T16:48:08.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161478+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:08.161478+0000","last_peered":"2024-09-13T16:48:08.161478+0000","last_clean":"2024-09-13T16:48:08.161478+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:08.161478+0000","last_undegraded":"2024-09-13T16:48:08.161478+0000","last_fullsized":"2024-09-13T16:48:08.161478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148905+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T16:48:08.148905+0000","last_peered":"2024-09-13T16:48:08.148905+0000","last_clean":"2024-09-13T16:48:08.148905+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T16:48:08.148905+0000","last_undegraded":"2024-09-13T16:48:08.148905+0000","last_fullsized":"2024-09-13T16:48:08.148905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158934+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T16:48:09.158934+0000","last_peered":"2024-09-13T16:48:09.158934+0000","last_clean":"2024-09-13T16:48:09.158934+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T16:48:09.158934+0000","last_undegraded":"2024-09-13T16:48:09.158934+0000","last_fullsized":"2024-09-13T16:48:09.158934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178617+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:08.178617+0000","last_peered":"2024-09-13T16:48:08.178617+0000","last_clean":"2024-09-13T16:48:08.178617+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:08.178617+0000","last_undegraded":"2024-09-13T16:48:08.178617+0000","last_fullsized":"2024-09-13T16:48:08.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155913+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:08.155913+0000","last_peered":"2024-09-13T16:48:08.155913+0000","last_clean":"2024-09-13T16:48:08.155913+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:08.155913+0000","last_undegraded":"2024-09-13T16:48:08.155913+0000","last_fullsized":"2024-09-13T16:48:08.155913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146319+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T16:48:08.146319+0000","last_peered":"2024-09-13T16:48:08.146319+0000","last_clean":"2024-09-13T16:48:08.146319+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T16:48:08.146319+0000","last_undegraded":"2024-09-13T16:48:08.146319+0000","last_fullsized":"2024-09-13T16:48:08.146319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158609+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T16:48:09.158609+0000","last_peered":"2024-09-13T16:48:09.158609+0000","last_clean":"2024-09-13T16:48:09.158609+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T16:48:09.158609+0000","last_undegraded":"2024-09-13T16:48:09.158609+0000","last_fullsized":"2024-09-13T16:48:09.158609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152057+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:08.152057+0000","last_peered":"2024-09-13T16:48:08.152057+0000","last_clean":"2024-09-13T16:48:08.152057+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:08.152057+0000","last_undegraded":"2024-09-13T16:48:08.152057+0000","last_fullsized":"2024-09-13T16:48:08.152057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149487+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:08.149487+0000","last_peered":"2024-09-13T16:48:08.149487+0000","last_clean":"2024-09-13T16:48:08.149487+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:08.149487+0000","last_undegraded":"2024-09-13T16:48:08.149487+0000","last_fullsized":"2024-09-13T16:48:08.149487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175317+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T16:48:08.175317+0000","last_peered":"2024-09-13T16:48:08.175317+0000","last_clean":"2024-09-13T16:48:08.175317+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T16:48:08.175317+0000","last_undegraded":"2024-09-13T16:48:08.175317+0000","last_fullsized":"2024-09-13T16:48:08.175317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173209+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T16:48:09.173209+0000","last_peered":"2024-09-13T16:48:09.173209+0000","last_clean":"2024-09-13T16:48:09.173209+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T16:48:09.173209+0000","last_undegraded":"2024-09-13T16:48:09.173209+0000","last_fullsized":"2024-09-13T16:48:09.173209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.177954+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:08.177954+0000","last_peered":"2024-09-13T16:48:08.177954+0000","last_clean":"2024-09-13T16:48:08.177954+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:08.177954+0000","last_undegraded":"2024-09-13T16:48:08.177954+0000","last_fullsized":"2024-09-13T16:48:08.177954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155936+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:08.155936+0000","last_peered":"2024-09-13T16:48:08.155936+0000","last_clean":"2024-09-13T16:48:08.155936+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:08.155936+0000","last_undegraded":"2024-09-13T16:48:08.155936+0000","last_fullsized":"2024-09-13T16:48:08.155936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158692+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T16:48:08.158692+0000","last_peered":"2024-09-13T16:48:08.158692+0000","last_clean":"2024-09-13T16:48:08.158692+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T16:48:08.158692+0000","last_undegraded":"2024-09-13T16:48:08.158692+0000","last_fullsized":"2024-09-13T16:48:08.158692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158034+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T16:48:09.158034+0000","last_peered":"2024-09-13T16:48:09.158034+0000","last_clean":"2024-09-13T16:48:09.158034+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T16:48:09.158034+0000","last_undegraded":"2024-09-13T16:48:09.158034+0000","last_fullsized":"2024-09-13T16:48:09.158034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160811+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:08.160811+0000","last_peered":"2024-09-13T16:48:08.160811+0000","last_clean":"2024-09-13T16:48:08.160811+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:08.160811+0000","last_undegraded":"2024-09-13T16:48:08.160811+0000","last_fullsized":"2024-09-13T16:48:08.160811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.192267+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:08.192267+0000","last_peered":"2024-09-13T16:48:08.192267+0000","last_clean":"2024-09-13T16:48:08.192267+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:08.192267+0000","last_undegraded":"2024-09-13T16:48:08.192267+0000","last_fullsized":"2024-09-13T16:48:08.192267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185117+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T16:48:08.185117+0000","last_peered":"2024-09-13T16:48:08.185117+0000","last_clean":"2024-09-13T16:48:08.185117+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T16:48:08.185117+0000","last_undegraded":"2024-09-13T16:48:08.185117+0000","last_fullsized":"2024-09-13T16:48:08.185117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177888+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T16:48:09.177888+0000","last_peered":"2024-09-13T16:48:09.177888+0000","last_clean":"2024-09-13T16:48:09.177888+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T16:48:09.177888+0000","last_undegraded":"2024-09-13T16:48:09.177888+0000","last_fullsized":"2024-09-13T16:48:09.177888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167237+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:08.167237+0000","last_peered":"2024-09-13T16:48:08.167237+0000","last_clean":"2024-09-13T16:48:08.167237+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:08.167237+0000","last_undegraded":"2024-09-13T16:48:08.167237+0000","last_fullsized":"2024-09-13T16:48:08.167237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156043+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:08.156043+0000","last_peered":"2024-09-13T16:48:08.156043+0000","last_clean":"2024-09-13T16:48:08.156043+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:08.156043+0000","last_undegraded":"2024-09-13T16:48:08.156043+0000","last_fullsized":"2024-09-13T16:48:08.156043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163690+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T16:48:08.163690+0000","last_peered":"2024-09-13T16:48:08.163690+0000","last_clean":"2024-09-13T16:48:08.163690+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T16:48:08.163690+0000","last_undegraded":"2024-09-13T16:48:08.163690+0000","last_fullsized":"2024-09-13T16:48:08.163690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162212+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T16:48:09.162212+0000","last_peered":"2024-09-13T16:48:09.162212+0000","last_clean":"2024-09-13T16:48:09.162212+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T16:48:09.162212+0000","last_undegraded":"2024-09-13T16:48:09.162212+0000","last_fullsized":"2024-09-13T16:48:09.162212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144923+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:08.144923+0000","last_peered":"2024-09-13T16:48:08.144923+0000","last_clean":"2024-09-13T16:48:08.144923+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:08.144923+0000","last_undegraded":"2024-09-13T16:48:08.144923+0000","last_fullsized":"2024-09-13T16:48:08.144923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145599+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:08.145599+0000","last_peered":"2024-09-13T16:48:08.145599+0000","last_clean":"2024-09-13T16:48:08.145599+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:08.145599+0000","last_undegraded":"2024-09-13T16:48:08.145599+0000","last_fullsized":"2024-09-13T16:48:08.145599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145022+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T16:48:08.145022+0000","last_peered":"2024-09-13T16:48:08.145022+0000","last_clean":"2024-09-13T16:48:08.145022+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T16:48:08.145022+0000","last_undegraded":"2024-09-13T16:48:08.145022+0000","last_fullsized":"2024-09-13T16:48:08.145022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161391+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T16:48:09.161391+0000","last_peered":"2024-09-13T16:48:09.161391+0000","last_clean":"2024-09-13T16:48:09.161391+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T16:48:09.161391+0000","last_undegraded":"2024-09-13T16:48:09.161391+0000","last_fullsized":"2024-09-13T16:48:09.161391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160556+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:08.160556+0000","last_peered":"2024-09-13T16:48:08.160556+0000","last_clean":"2024-09-13T16:48:08.160556+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:08.160556+0000","last_undegraded":"2024-09-13T16:48:08.160556+0000","last_fullsized":"2024-09-13T16:48:08.160556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149235+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:08.149235+0000","last_peered":"2024-09-13T16:48:08.149235+0000","last_clean":"2024-09-13T16:48:08.149235+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:08.149235+0000","last_undegraded":"2024-09-13T16:48:08.149235+0000","last_fullsized":"2024-09-13T16:48:08.149235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.209853+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T16:48:08.209853+0000","last_peered":"2024-09-13T16:48:08.209853+0000","last_clean":"2024-09-13T16:48:08.209853+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T16:48:08.209853+0000","last_undegraded":"2024-09-13T16:48:08.209853+0000","last_fullsized":"2024-09-13T16:48:08.209853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161110+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:08.161110+0000","last_peered":"2024-09-13T16:48:08.161110+0000","last_clean":"2024-09-13T16:48:08.161110+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:08.161110+0000","last_undegraded":"2024-09-13T16:48:08.161110+0000","last_fullsized":"2024-09-13T16:48:08.161110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161421+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:08.161421+0000","last_peered":"2024-09-13T16:48:08.161421+0000","last_clean":"2024-09-13T16:48:08.161421+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:08.161421+0000","last_undegraded":"2024-09-13T16:48:08.161421+0000","last_fullsized":"2024-09-13T16:48:08.161421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.188902+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T16:48:08.188902+0000","last_peered":"2024-09-13T16:48:08.188902+0000","last_clean":"2024-09-13T16:48:08.188902+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T16:48:08.188902+0000","last_undegraded":"2024-09-13T16:48:08.188902+0000","last_fullsized":"2024-09-13T16:48:08.188902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165383+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:08.165383+0000","last_peered":"2024-09-13T16:48:08.165383+0000","last_clean":"2024-09-13T16:48:08.165383+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:08.165383+0000","last_undegraded":"2024-09-13T16:48:08.165383+0000","last_fullsized":"2024-09-13T16:48:08.165383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148543+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:08.148543+0000","last_peered":"2024-09-13T16:48:08.148543+0000","last_clean":"2024-09-13T16:48:08.148543+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:08.148543+0000","last_undegraded":"2024-09-13T16:48:08.148543+0000","last_fullsized":"2024-09-13T16:48:08.148543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146582+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T16:48:08.146582+0000","last_peered":"2024-09-13T16:48:08.146582+0000","last_clean":"2024-09-13T16:48:08.146582+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T16:48:08.146582+0000","last_undegraded":"2024-09-13T16:48:08.146582+0000","last_fullsized":"2024-09-13T16:48:08.146582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173477+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T16:48:09.173477+0000","last_peered":"2024-09-13T16:48:09.173477+0000","last_clean":"2024-09-13T16:48:09.173477+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T16:48:09.173477+0000","last_undegraded":"2024-09-13T16:48:09.173477+0000","last_fullsized":"2024-09-13T16:48:09.173477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.190852+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:08.190852+0000","last_peered":"2024-09-13T16:48:08.190852+0000","last_clean":"2024-09-13T16:48:08.190852+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:08.190852+0000","last_undegraded":"2024-09-13T16:48:08.190852+0000","last_fullsized":"2024-09-13T16:48:08.190852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149403+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:08.149403+0000","last_peered":"2024-09-13T16:48:08.149403+0000","last_clean":"2024-09-13T16:48:08.149403+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:08.149403+0000","last_undegraded":"2024-09-13T16:48:08.149403+0000","last_fullsized":"2024-09-13T16:48:08.149403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208558+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T16:48:08.208558+0000","last_peered":"2024-09-13T16:48:08.208558+0000","last_clean":"2024-09-13T16:48:08.208558+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T16:48:08.208558+0000","last_undegraded":"2024-09-13T16:48:08.208558+0000","last_fullsized":"2024-09-13T16:48:08.208558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158601+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T16:48:09.158601+0000","last_peered":"2024-09-13T16:48:09.158601+0000","last_clean":"2024-09-13T16:48:09.158601+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T16:48:09.158601+0000","last_undegraded":"2024-09-13T16:48:09.158601+0000","last_fullsized":"2024-09-13T16:48:09.158601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163596+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:08.163596+0000","last_peered":"2024-09-13T16:48:08.163596+0000","last_clean":"2024-09-13T16:48:08.163596+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:08.163596+0000","last_undegraded":"2024-09-13T16:48:08.163596+0000","last_fullsized":"2024-09-13T16:48:08.163596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156189+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:08.156189+0000","last_peered":"2024-09-13T16:48:08.156189+0000","last_clean":"2024-09-13T16:48:08.156189+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:08.156189+0000","last_undegraded":"2024-09-13T16:48:08.156189+0000","last_fullsized":"2024-09-13T16:48:08.156189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174258+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T16:48:08.174258+0000","last_peered":"2024-09-13T16:48:08.174258+0000","last_clean":"2024-09-13T16:48:08.174258+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T16:48:08.174258+0000","last_undegraded":"2024-09-13T16:48:08.174258+0000","last_fullsized":"2024-09-13T16:48:08.174258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173305+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T16:48:09.173305+0000","last_peered":"2024-09-13T16:48:09.173305+0000","last_clean":"2024-09-13T16:48:09.173305+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T16:48:09.173305+0000","last_undegraded":"2024-09-13T16:48:09.173305+0000","last_fullsized":"2024-09-13T16:48:09.173305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155963+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:08.155963+0000","last_peered":"2024-09-13T16:48:08.155963+0000","last_clean":"2024-09-13T16:48:08.155963+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:08.155963+0000","last_undegraded":"2024-09-13T16:48:08.155963+0000","last_fullsized":"2024-09-13T16:48:08.155963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148500+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:08.148500+0000","last_peered":"2024-09-13T16:48:08.148500+0000","last_clean":"2024-09-13T16:48:08.148500+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:08.148500+0000","last_undegraded":"2024-09-13T16:48:08.148500+0000","last_fullsized":"2024-09-13T16:48:08.148500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143144+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T16:48:08.143144+0000","last_peered":"2024-09-13T16:48:08.143144+0000","last_clean":"2024-09-13T16:48:08.143144+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T16:48:08.143144+0000","last_undegraded":"2024-09-13T16:48:08.143144+0000","last_fullsized":"2024-09-13T16:48:08.143144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161066+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T16:48:09.161066+0000","last_peered":"2024-09-13T16:48:09.161066+0000","last_clean":"2024-09-13T16:48:09.161066+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T16:48:09.161066+0000","last_undegraded":"2024-09-13T16:48:09.161066+0000","last_fullsized":"2024-09-13T16:48:09.161066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186757+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T16:48:08.186757+0000","last_peered":"2024-09-13T16:48:08.186757+0000","last_clean":"2024-09-13T16:48:08.186757+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T16:48:08.186757+0000","last_undegraded":"2024-09-13T16:48:08.186757+0000","last_fullsized":"2024-09-13T16:48:08.186757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147291+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:08.147291+0000","last_peered":"2024-09-13T16:48:08.147291+0000","last_clean":"2024-09-13T16:48:08.147291+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:08.147291+0000","last_undegraded":"2024-09-13T16:48:08.147291+0000","last_fullsized":"2024-09-13T16:48:08.147291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158941+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:08.158941+0000","last_peered":"2024-09-13T16:48:08.158941+0000","last_clean":"2024-09-13T16:48:08.158941+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:08.158941+0000","last_undegraded":"2024-09-13T16:48:08.158941+0000","last_fullsized":"2024-09-13T16:48:08.158941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175068+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T16:48:09.175068+0000","last_peered":"2024-09-13T16:48:09.175068+0000","last_clean":"2024-09-13T16:48:09.175068+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T16:48:09.175068+0000","last_undegraded":"2024-09-13T16:48:09.175068+0000","last_fullsized":"2024-09-13T16:48:09.175068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.182997+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T16:48:08.182997+0000","last_peered":"2024-09-13T16:48:08.182997+0000","last_clean":"2024-09-13T16:48:08.182997+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T16:48:08.182997+0000","last_undegraded":"2024-09-13T16:48:08.182997+0000","last_fullsized":"2024-09-13T16:48:08.182997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143381+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:08.143381+0000","last_peered":"2024-09-13T16:48:08.143381+0000","last_clean":"2024-09-13T16:48:08.143381+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:08.143381+0000","last_undegraded":"2024-09-13T16:48:08.143381+0000","last_fullsized":"2024-09-13T16:48:08.143381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143921+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:08.143921+0000","last_peered":"2024-09-13T16:48:08.143921+0000","last_clean":"2024-09-13T16:48:08.143921+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:08.143921+0000","last_undegraded":"2024-09-13T16:48:08.143921+0000","last_fullsized":"2024-09-13T16:48:08.143921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174477+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T16:48:09.174477+0000","last_peered":"2024-09-13T16:48:09.174477+0000","last_clean":"2024-09-13T16:48:09.174477+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T16:48:09.174477+0000","last_undegraded":"2024-09-13T16:48:09.174477+0000","last_fullsized":"2024-09-13T16:48:09.174477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147114+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:08.147114+0000","last_peered":"2024-09-13T16:48:08.147114+0000","last_clean":"2024-09-13T16:48:08.147114+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:08.147114+0000","last_undegraded":"2024-09-13T16:48:08.147114+0000","last_fullsized":"2024-09-13T16:48:08.147114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161410+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:08.161410+0000","last_peered":"2024-09-13T16:48:08.161410+0000","last_clean":"2024-09-13T16:48:08.161410+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:08.161410+0000","last_undegraded":"2024-09-13T16:48:08.161410+0000","last_fullsized":"2024-09-13T16:48:08.161410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170387+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T16:48:08.170387+0000","last_peered":"2024-09-13T16:48:08.170387+0000","last_clean":"2024-09-13T16:48:08.170387+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T16:48:08.170387+0000","last_undegraded":"2024-09-13T16:48:08.170387+0000","last_fullsized":"2024-09-13T16:48:08.170387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175603+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T16:48:09.175603+0000","last_peered":"2024-09-13T16:48:09.175603+0000","last_clean":"2024-09-13T16:48:09.175603+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T16:48:09.175603+0000","last_undegraded":"2024-09-13T16:48:09.175603+0000","last_fullsized":"2024-09-13T16:48:09.175603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144443+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:08.144443+0000","last_peered":"2024-09-13T16:48:08.144443+0000","last_clean":"2024-09-13T16:48:08.144443+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:08.144443+0000","last_undegraded":"2024-09-13T16:48:08.144443+0000","last_fullsized":"2024-09-13T16:48:08.144443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185002+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:08.185002+0000","last_peered":"2024-09-13T16:48:08.185002+0000","last_clean":"2024-09-13T16:48:08.185002+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:08.185002+0000","last_undegraded":"2024-09-13T16:48:08.185002+0000","last_fullsized":"2024-09-13T16:48:08.185002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170142+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T16:48:08.170142+0000","last_peered":"2024-09-13T16:48:08.170142+0000","last_clean":"2024-09-13T16:48:08.170142+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T16:48:08.170142+0000","last_undegraded":"2024-09-13T16:48:08.170142+0000","last_fullsized":"2024-09-13T16:48:08.170142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177615+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T16:48:09.177615+0000","last_peered":"2024-09-13T16:48:09.177615+0000","last_clean":"2024-09-13T16:48:09.177615+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T16:48:09.177615+0000","last_undegraded":"2024-09-13T16:48:09.177615+0000","last_fullsized":"2024-09-13T16:48:09.177615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144121+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:08.144121+0000","last_peered":"2024-09-13T16:48:08.144121+0000","last_clean":"2024-09-13T16:48:08.144121+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:08.144121+0000","last_undegraded":"2024-09-13T16:48:08.144121+0000","last_fullsized":"2024-09-13T16:48:08.144121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174330+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T16:48:09.174330+0000","last_peered":"2024-09-13T16:48:09.174330+0000","last_clean":"2024-09-13T16:48:09.174330+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T16:48:09.174330+0000","last_undegraded":"2024-09-13T16:48:09.174330+0000","last_fullsized":"2024-09-13T16:48:09.174330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153996+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:08.153996+0000","last_peered":"2024-09-13T16:48:08.153996+0000","last_clean":"2024-09-13T16:48:08.153996+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:08.153996+0000","last_undegraded":"2024-09-13T16:48:08.153996+0000","last_fullsized":"2024-09-13T16:48:08.153996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161029+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T16:48:09.161029+0000","last_peered":"2024-09-13T16:48:09.161029+0000","last_clean":"2024-09-13T16:48:09.161029+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T16:48:09.161029+0000","last_undegraded":"2024-09-13T16:48:09.161029+0000","last_fullsized":"2024-09-13T16:48:09.161029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154547+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:08.154547+0000","last_peered":"2024-09-13T16:48:08.154547+0000","last_clean":"2024-09-13T16:48:08.154547+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:08.154547+0000","last_undegraded":"2024-09-13T16:48:08.154547+0000","last_fullsized":"2024-09-13T16:48:08.154547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174304+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T16:48:09.174304+0000","last_peered":"2024-09-13T16:48:09.174304+0000","last_clean":"2024-09-13T16:48:09.174304+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T16:48:09.174304+0000","last_undegraded":"2024-09-13T16:48:09.174304+0000","last_fullsized":"2024-09-13T16:48:09.174304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161339+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:08.161339+0000","last_peered":"2024-09-13T16:48:08.161339+0000","last_clean":"2024-09-13T16:48:08.161339+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:08.161339+0000","last_undegraded":"2024-09-13T16:48:08.161339+0000","last_fullsized":"2024-09-13T16:48:08.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173686+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T16:48:09.173686+0000","last_peered":"2024-09-13T16:48:09.173686+0000","last_clean":"2024-09-13T16:48:09.173686+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T16:48:09.173686+0000","last_undegraded":"2024-09-13T16:48:09.173686+0000","last_fullsized":"2024-09-13T16:48:09.173686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145104+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:08.145104+0000","last_peered":"2024-09-13T16:48:08.145104+0000","last_clean":"2024-09-13T16:48:08.145104+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:08.145104+0000","last_undegraded":"2024-09-13T16:48:08.145104+0000","last_fullsized":"2024-09-13T16:48:08.145104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175359+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T16:48:09.175359+0000","last_peered":"2024-09-13T16:48:09.175359+0000","last_clean":"2024-09-13T16:48:09.175359+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T16:48:09.175359+0000","last_undegraded":"2024-09-13T16:48:09.175359+0000","last_fullsized":"2024-09-13T16:48:09.175359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149320+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:08.149320+0000","last_peered":"2024-09-13T16:48:08.149320+0000","last_clean":"2024-09-13T16:48:08.149320+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:08.149320+0000","last_undegraded":"2024-09-13T16:48:08.149320+0000","last_fullsized":"2024-09-13T16:48:08.149320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173205+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T16:48:09.173205+0000","last_peered":"2024-09-13T16:48:09.173205+0000","last_clean":"2024-09-13T16:48:09.173205+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T16:48:09.173205+0000","last_undegraded":"2024-09-13T16:48:09.173205+0000","last_fullsized":"2024-09-13T16:48:09.173205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149001+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:08.149001+0000","last_peered":"2024-09-13T16:48:08.149001+0000","last_clean":"2024-09-13T16:48:08.149001+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:08.149001+0000","last_undegraded":"2024-09-13T16:48:08.149001+0000","last_fullsized":"2024-09-13T16:48:08.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158758+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T16:48:09.158758+0000","last_peered":"2024-09-13T16:48:09.158758+0000","last_clean":"2024-09-13T16:48:09.158758+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T16:48:09.158758+0000","last_undegraded":"2024-09-13T16:48:09.158758+0000","last_fullsized":"2024-09-13T16:48:09.158758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145105+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:08.145105+0000","last_peered":"2024-09-13T16:48:08.145105+0000","last_clean":"2024-09-13T16:48:08.145105+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:08.145105+0000","last_undegraded":"2024-09-13T16:48:08.145105+0000","last_fullsized":"2024-09-13T16:48:08.145105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177538+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T16:48:09.177538+0000","last_peered":"2024-09-13T16:48:09.177538+0000","last_clean":"2024-09-13T16:48:09.177538+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T16:48:09.177538+0000","last_undegraded":"2024-09-13T16:48:09.177538+0000","last_fullsized":"2024-09-13T16:48:09.177538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158985+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:08.158985+0000","last_peered":"2024-09-13T16:48:08.158985+0000","last_clean":"2024-09-13T16:48:08.158985+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:08.158985+0000","last_undegraded":"2024-09-13T16:48:08.158985+0000","last_fullsized":"2024-09-13T16:48:08.158985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177461+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T16:48:09.177461+0000","last_peered":"2024-09-13T16:48:09.177461+0000","last_clean":"2024-09-13T16:48:09.177461+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T16:48:09.177461+0000","last_undegraded":"2024-09-13T16:48:09.177461+0000","last_fullsized":"2024-09-13T16:48:09.177461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143902+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:08.143902+0000","last_peered":"2024-09-13T16:48:08.143902+0000","last_clean":"2024-09-13T16:48:08.143902+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:08.143902+0000","last_undegraded":"2024-09-13T16:48:08.143902+0000","last_fullsized":"2024-09-13T16:48:08.143902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159127+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T16:48:09.159127+0000","last_peered":"2024-09-13T16:48:09.159127+0000","last_clean":"2024-09-13T16:48:09.159127+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T16:48:09.159127+0000","last_undegraded":"2024-09-13T16:48:09.159127+0000","last_fullsized":"2024-09-13T16:48:09.159127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148988+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:08.148988+0000","last_peered":"2024-09-13T16:48:08.148988+0000","last_clean":"2024-09-13T16:48:08.148988+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:08.148988+0000","last_undegraded":"2024-09-13T16:48:08.148988+0000","last_fullsized":"2024-09-13T16:48:08.148988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177190+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T16:48:09.177190+0000","last_peered":"2024-09-13T16:48:09.177190+0000","last_clean":"2024-09-13T16:48:09.177190+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T16:48:09.177190+0000","last_undegraded":"2024-09-13T16:48:09.177190+0000","last_fullsized":"2024-09-13T16:48:09.177190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161327+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:08.161327+0000","last_peered":"2024-09-13T16:48:08.161327+0000","last_clean":"2024-09-13T16:48:08.161327+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:08.161327+0000","last_undegraded":"2024-09-13T16:48:08.161327+0000","last_fullsized":"2024-09-13T16:48:08.161327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148225+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:08.148225+0000","last_peered":"2024-09-13T16:48:08.148225+0000","last_clean":"2024-09-13T16:48:08.148225+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:08.148225+0000","last_undegraded":"2024-09-13T16:48:08.148225+0000","last_fullsized":"2024-09-13T16:48:08.148225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175099+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T16:48:09.175099+0000","last_peered":"2024-09-13T16:48:09.175099+0000","last_clean":"2024-09-13T16:48:09.175099+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T16:48:09.175099+0000","last_undegraded":"2024-09-13T16:48:09.175099+0000","last_fullsized":"2024-09-13T16:48:09.175099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179166+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:08.179166+0000","last_peered":"2024-09-13T16:48:08.179166+0000","last_clean":"2024-09-13T16:48:08.179166+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:08.179166+0000","last_undegraded":"2024-09-13T16:48:08.179166+0000","last_fullsized":"2024-09-13T16:48:08.179166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158364+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T16:48:09.158364+0000","last_peered":"2024-09-13T16:48:09.158364+0000","last_clean":"2024-09-13T16:48:09.158364+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T16:48:09.158364+0000","last_undegraded":"2024-09-13T16:48:09.158364+0000","last_fullsized":"2024-09-13T16:48:09.158364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155807+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:08.155807+0000","last_peered":"2024-09-13T16:48:08.155807+0000","last_clean":"2024-09-13T16:48:08.155807+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:08.155807+0000","last_undegraded":"2024-09-13T16:48:08.155807+0000","last_fullsized":"2024-09-13T16:48:08.155807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.172410+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T16:48:09.172410+0000","last_peered":"2024-09-13T16:48:09.172410+0000","last_clean":"2024-09-13T16:48:09.172410+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T16:48:09.172410+0000","last_undegraded":"2024-09-13T16:48:09.172410+0000","last_fullsized":"2024-09-13T16:48:09.172410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.157093+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:08.157093+0000","last_peered":"2024-09-13T16:48:08.157093+0000","last_clean":"2024-09-13T16:48:08.157093+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:08.157093+0000","last_undegraded":"2024-09-13T16:48:08.157093+0000","last_fullsized":"2024-09-13T16:48:08.157093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173535+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T16:48:09.173535+0000","last_peered":"2024-09-13T16:48:09.173535+0000","last_clean":"2024-09-13T16:48:09.173535+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T16:48:09.173535+0000","last_undegraded":"2024-09-13T16:48:09.173535+0000","last_fullsized":"2024-09-13T16:48:09.173535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158715+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:08.158715+0000","last_peered":"2024-09-13T16:48:08.158715+0000","last_clean":"2024-09-13T16:48:08.158715+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:08.158715+0000","last_undegraded":"2024-09-13T16:48:08.158715+0000","last_fullsized":"2024-09-13T16:48:08.158715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175097+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T16:48:09.175097+0000","last_peered":"2024-09-13T16:48:09.175097+0000","last_clean":"2024-09-13T16:48:09.175097+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T16:48:09.175097+0000","last_undegraded":"2024-09-13T16:48:09.175097+0000","last_fullsized":"2024-09-13T16:48:09.175097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147419+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:08.147419+0000","last_peered":"2024-09-13T16:48:08.147419+0000","last_clean":"2024-09-13T16:48:08.147419+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:08.147419+0000","last_undegraded":"2024-09-13T16:48:08.147419+0000","last_fullsized":"2024-09-13T16:48:08.147419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159429+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T16:48:09.159429+0000","last_peered":"2024-09-13T16:48:09.159429+0000","last_clean":"2024-09-13T16:48:09.159429+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T16:48:09.159429+0000","last_undegraded":"2024-09-13T16:48:09.159429+0000","last_fullsized":"2024-09-13T16:48:09.159429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161028+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:08.161028+0000","last_peered":"2024-09-13T16:48:08.161028+0000","last_clean":"2024-09-13T16:48:08.161028+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:08.161028+0000","last_undegraded":"2024-09-13T16:48:08.161028+0000","last_fullsized":"2024-09-13T16:48:08.161028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177095+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T16:48:09.177095+0000","last_peered":"2024-09-13T16:48:09.177095+0000","last_clean":"2024-09-13T16:48:09.177095+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T16:48:09.177095+0000","last_undegraded":"2024-09-13T16:48:09.177095+0000","last_fullsized":"2024-09-13T16:48:09.177095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143741+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:08.143741+0000","last_peered":"2024-09-13T16:48:08.143741+0000","last_clean":"2024-09-13T16:48:08.143741+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:08.143741+0000","last_undegraded":"2024-09-13T16:48:08.143741+0000","last_fullsized":"2024-09-13T16:48:08.143741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176397+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T16:48:09.176397+0000","last_peered":"2024-09-13T16:48:09.176397+0000","last_clean":"2024-09-13T16:48:09.176397+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T16:48:09.176397+0000","last_undegraded":"2024-09-13T16:48:09.176397+0000","last_fullsized":"2024-09-13T16:48:09.176397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186343+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:08.186343+0000","last_peered":"2024-09-13T16:48:08.186343+0000","last_clean":"2024-09-13T16:48:08.186343+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:08.186343+0000","last_undegraded":"2024-09-13T16:48:08.186343+0000","last_fullsized":"2024-09-13T16:48:08.186343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158832+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T16:48:09.158832+0000","last_peered":"2024-09-13T16:48:09.158832+0000","last_clean":"2024-09-13T16:48:09.158832+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T16:48:09.158832+0000","last_undegraded":"2024-09-13T16:48:09.158832+0000","last_fullsized":"2024-09-13T16:48:09.158832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149361+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:08.149361+0000","last_peered":"2024-09-13T16:48:08.149361+0000","last_clean":"2024-09-13T16:48:08.149361+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:08.149361+0000","last_undegraded":"2024-09-13T16:48:08.149361+0000","last_fullsized":"2024-09-13T16:48:08.149361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173784+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T16:48:09.173784+0000","last_peered":"2024-09-13T16:48:09.173784+0000","last_clean":"2024-09-13T16:48:09.173784+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T16:48:09.173784+0000","last_undegraded":"2024-09-13T16:48:09.173784+0000","last_fullsized":"2024-09-13T16:48:09.173784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148791+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:08.148791+0000","last_peered":"2024-09-13T16:48:08.148791+0000","last_clean":"2024-09-13T16:48:08.148791+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:08.148791+0000","last_undegraded":"2024-09-13T16:48:08.148791+0000","last_fullsized":"2024-09-13T16:48:08.148791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159787+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T16:48:09.159787+0000","last_peered":"2024-09-13T16:48:09.159787+0000","last_clean":"2024-09-13T16:48:09.159787+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T16:48:09.159787+0000","last_undegraded":"2024-09-13T16:48:09.159787+0000","last_fullsized":"2024-09-13T16:48:09.159787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166109+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:08.166109+0000","last_peered":"2024-09-13T16:48:08.166109+0000","last_clean":"2024-09-13T16:48:08.166109+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:08.166109+0000","last_undegraded":"2024-09-13T16:48:08.166109+0000","last_fullsized":"2024-09-13T16:48:08.166109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.178540+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T16:48:09.178540+0000","last_peered":"2024-09-13T16:48:09.178540+0000","last_clean":"2024-09-13T16:48:09.178540+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T16:48:09.178540+0000","last_undegraded":"2024-09-13T16:48:09.178540+0000","last_fullsized":"2024-09-13T16:48:09.178540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143140+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:08.143140+0000","last_peered":"2024-09-13T16:48:08.143140+0000","last_clean":"2024-09-13T16:48:08.143140+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:08.143140+0000","last_undegraded":"2024-09-13T16:48:08.143140+0000","last_fullsized":"2024-09-13T16:48:08.143140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.160931+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T16:48:09.160931+0000","last_peered":"2024-09-13T16:48:09.160931+0000","last_clean":"2024-09-13T16:48:09.160931+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T16:48:09.160931+0000","last_undegraded":"2024-09-13T16:48:09.160931+0000","last_fullsized":"2024-09-13T16:48:09.160931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167027+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:08.167027+0000","last_peered":"2024-09-13T16:48:08.167027+0000","last_clean":"2024-09-13T16:48:08.167027+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:08.167027+0000","last_undegraded":"2024-09-13T16:48:08.167027+0000","last_fullsized":"2024-09-13T16:48:08.167027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173532+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T16:48:09.173532+0000","last_peered":"2024-09-13T16:48:09.173532+0000","last_clean":"2024-09-13T16:48:09.173532+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T16:48:09.173532+0000","last_undegraded":"2024-09-13T16:48:09.173532+0000","last_fullsized":"2024-09-13T16:48:09.173532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148753+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:08.148753+0000","last_peered":"2024-09-13T16:48:08.148753+0000","last_clean":"2024-09-13T16:48:08.148753+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:08.148753+0000","last_undegraded":"2024-09-13T16:48:08.148753+0000","last_fullsized":"2024-09-13T16:48:08.148753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161898+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T16:48:09.161898+0000","last_peered":"2024-09-13T16:48:09.161898+0000","last_clean":"2024-09-13T16:48:09.161898+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T16:48:09.161898+0000","last_undegraded":"2024-09-13T16:48:09.161898+0000","last_fullsized":"2024-09-13T16:48:09.161898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160351+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:08.160351+0000","last_peered":"2024-09-13T16:48:08.160351+0000","last_clean":"2024-09-13T16:48:08.160351+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:08.160351+0000","last_undegraded":"2024-09-13T16:48:08.160351+0000","last_fullsized":"2024-09-13T16:48:08.160351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176220+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T16:48:09.176220+0000","last_peered":"2024-09-13T16:48:09.176220+0000","last_clean":"2024-09-13T16:48:09.176220+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T16:48:09.176220+0000","last_undegraded":"2024-09-13T16:48:09.176220+0000","last_fullsized":"2024-09-13T16:48:09.176220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151613+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:08.151613+0000","last_peered":"2024-09-13T16:48:08.151613+0000","last_clean":"2024-09-13T16:48:08.151613+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:08.151613+0000","last_undegraded":"2024-09-13T16:48:08.151613+0000","last_fullsized":"2024-09-13T16:48:08.151613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158220+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T16:48:09.158220+0000","last_peered":"2024-09-13T16:48:09.158220+0000","last_clean":"2024-09-13T16:48:09.158220+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T16:48:09.158220+0000","last_undegraded":"2024-09-13T16:48:09.158220+0000","last_fullsized":"2024-09-13T16:48:09.158220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.162959+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:08.162959+0000","last_peered":"2024-09-13T16:48:08.162959+0000","last_clean":"2024-09-13T16:48:08.162959+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:08.162959+0000","last_undegraded":"2024-09-13T16:48:08.162959+0000","last_fullsized":"2024-09-13T16:48:08.162959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173056+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T16:48:09.173056+0000","last_peered":"2024-09-13T16:48:09.173056+0000","last_clean":"2024-09-13T16:48:09.173056+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T16:48:09.173056+0000","last_undegraded":"2024-09-13T16:48:09.173056+0000","last_fullsized":"2024-09-13T16:48:09.173056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.150218+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:08.150218+0000","last_peered":"2024-09-13T16:48:08.150218+0000","last_clean":"2024-09-13T16:48:08.150218+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:08.150218+0000","last_undegraded":"2024-09-13T16:48:08.150218+0000","last_fullsized":"2024-09-13T16:48:08.150218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173906+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T16:48:09.173906+0000","last_peered":"2024-09-13T16:48:09.173906+0000","last_clean":"2024-09-13T16:48:09.173906+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T16:48:09.173906+0000","last_undegraded":"2024-09-13T16:48:09.173906+0000","last_fullsized":"2024-09-13T16:48:09.173906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166070+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:08.166070+0000","last_peered":"2024-09-13T16:48:08.166070+0000","last_clean":"2024-09-13T16:48:08.166070+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:08.166070+0000","last_undegraded":"2024-09-13T16:48:08.166070+0000","last_fullsized":"2024-09-13T16:48:08.166070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":690265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":101405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":99455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":282624,"data_stored":102375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":258048,"data_stored":99295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":100971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":690231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:17.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:17.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef1b1d9640 1 Processor -- start 2024-09-13T16:48:17.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef1b1d9640 1 -- start start 2024-09-13T16:48:17.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef1b1d9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fef1406e5e0 0x7fef14151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef1b1d9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1414f590 con 0x7fef1414f1b0 2024-09-13T16:48:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef1b1d9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1406eae0 con 0x7fef140c7470 2024-09-13T16:48:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef1b1d9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1406f660 con 0x7fef1406e5e0 2024-09-13T16:48:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef18f75640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fef1414f1b0 0x7fef140714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33130/0 (socket says 172.21.15.39:33130) 2024-09-13T16:48:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef18f75640 1 -- 172.21.15.39:0/73764866 learned_addr learned my addr 172.21.15.39:0/73764866 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef13fff640 1 --2- 172.21.15.39:0/73764866 >> v2:172.21.15.84:3300/0 conn(0x7fef1406e5e0 0x7fef14151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812770584 0 0) 0x7fef1414f590 con 0x7fef1414f1b0 2024-09-13T16:48:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feefc003530 con 0x7fef1414f1b0 2024-09-13T16:48:17.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805238838 0 0) 0x7feefc003530 con 0x7fef1414f1b0 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 >> v2:172.21.15.84:3300/0 conn(0x7fef1406e5e0 msgr2=0x7fef14151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 --2- 172.21.15.39:0/73764866 >> v2:172.21.15.84:3300/0 conn(0x7fef1406e5e0 0x7fef14151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 >> v1:172.21.15.84:6789/0 conn(0x7fef140c7470 legacy=0x7fef140c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1406edc0 con 0x7fef1414f1b0 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fef04001a00 con 0x7fef1414f1b0 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 >> v1:172.21.15.39:6789/0 conn(0x7fef1414f1b0 legacy=0x7fef140714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7feefc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feefc0047d0 0x7feefc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc005190 0x7feefc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1406f660 con 0x7feefc003980 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc00c030 con 0x7feefc005190 2024-09-13T16:48:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc00c1f0 con 0x7feefc0047d0 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef18f75640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7feefc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef13fff640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feefc0047d0 0x7feefc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef18f75640 1 -- 172.21.15.39:0/73764866 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feefc0047d0 msgr2=0x7feefc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef18f75640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feefc0047d0 0x7feefc004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef19776640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc005190 0x7feefc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef18f75640 1 -- 172.21.15.39:0/73764866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc005190 msgr2=0x7feefc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef18f75640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc005190 0x7feefc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef18f75640 1 -- 172.21.15.39:0/73764866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc003530 con 0x7feefc003980 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef19776640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc005190 0x7feefc00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.864+0000 7fef19776640 1 -- 172.21.15.39:0/73764866 reap_dead start 2024-09-13T16:48:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef18f75640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7feefc003da0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fef0400a830 tx=0x7fef0401d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d472001dc2be31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040070d0 con 0x7feefc003980 2024-09-13T16:48:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef040a8070 con 0x7feefc003980 2024-09-13T16:48:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef137fe640 1 -- 172.21.15.39:0/73764866 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040072f0 con 0x7feefc003980 2024-09-13T16:48:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef1b1d9640 1 -- 172.21.15.39:0/73764866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 msgr2=0x7feefc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef1b1d9640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7feefc003da0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fef0400a830 tx=0x7fef0401d330 comp rx=0 tx=0).stop 2024-09-13T16:48:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef1b1d9640 1 -- 172.21.15.39:0/73764866 shutdown_connections 2024-09-13T16:48:17.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef1b1d9640 1 --2- 172.21.15.39:0/73764866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7feefc003da0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef1b1d9640 1 -- 172.21.15.39:0/73764866 >> 172.21.15.39:0/73764866 conn(0x7fef140a9720 msgr2=0x7fef140a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef1b1d9640 1 -- 172.21.15.39:0/73764866 shutdown_connections 2024-09-13T16:48:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef1b1d9640 1 -- 172.21.15.39:0/73764866 wait complete. 2024-09-13T16:48:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef1b1d9640 1 Processor -- start 2024-09-13T16:48:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.868+0000 7fef1b1d9640 1 -- start start 2024-09-13T16:48:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef1b1d9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7fef141dff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef1b1d9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef1414bf80 0x7fef141dd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef1b1d9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef141e04d0 0x7fef141ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef1b1d9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1414bd30 con 0x7feefc003980 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef1b1d9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141e1020 con 0x7fef1414bf80 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef1b1d9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef141eb300 con 0x7fef141e04d0 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef18f75640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7fef141dff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef18f75640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7fef141dff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34342/0 (socket says 172.21.15.39:34342) 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef18f75640 1 -- 172.21.15.39:0/3832292451 learned_addr learned my addr 172.21.15.39:0/3832292451 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef13fff640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef1414bf80 0x7fef141dd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef19776640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef141e04d0 0x7fef141ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef18f75640 1 -- 172.21.15.39:0/3832292451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef141e04d0 msgr2=0x7fef141ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef18f75640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef141e04d0 0x7fef141ddd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef18f75640 1 -- 172.21.15.39:0/3832292451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef1414bf80 msgr2=0x7fef141dd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef18f75640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef1414bf80 0x7fef141dd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef18f75640 1 -- 172.21.15.39:0/3832292451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef141eb5e0 con 0x7feefc003980 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef18f75640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7fef141dff90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef0400a830 tx=0x7feefc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef117fa640 1 -- 172.21.15.39:0/3832292451 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04001a00 con 0x7feefc003980 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef117fa640 1 -- 172.21.15.39:0/3832292451 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef040a5ec0 con 0x7feefc003980 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef117fa640 1 -- 172.21.15.39:0/3832292451 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04007310 con 0x7feefc003980 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef140a8fa0 con 0x7feefc003980 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef14155d90 con 0x7feefc003980 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef141eb7d0 con 0x7feefc003980 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef117fa640 1 -- 172.21.15.39:0/3832292451 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef040aa030 con 0x7feefc003980 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef117fa640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef000666c0 0x7fef00068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef117fa640 1 -- 172.21.15.39:0/3832292451 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fef0411e520 con 0x7feefc003980 2024-09-13T16:48:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef13fff640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef000666c0 0x7fef00068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.872+0000 7fef13fff640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef000666c0 0x7fef00068b80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fef141564d0 tx=0x7fef08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:17.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:17.876+0000 7fef117fa640 1 -- 172.21.15.39:0/3832292451 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fef040eb5f0 con 0x7feefc003980 2024-09-13T16:48:18.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.036+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef140a8e20 con 0x7fef000666c0 2024-09-13T16:48:18.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.068+0000 7fef117fa640 1 -- 172.21.15.39:0/3832292451 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+503124 (secure 0 0 0) 0x7fef080128c0 con 0x7fef000666c0 2024-09-13T16:48:18.079 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:18.083 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:18.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef000666c0 msgr2=0x7fef00068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef000666c0 0x7fef00068b80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fef141564d0 tx=0x7fef08008040 comp rx=0 tx=0).stop 2024-09-13T16:48:18.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 msgr2=0x7fef141dff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7fef141dff90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef0400a830 tx=0x7feefc005800 comp rx=0 tx=0).stop 2024-09-13T16:48:18.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 shutdown_connections 2024-09-13T16:48:18.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef000666c0 0x7fef00068b80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef1414bf80 0x7fef141dd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef141e04d0 0x7fef141ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 --2- 172.21.15.39:0/3832292451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc003980 0x7fef141dff90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 >> 172.21.15.39:0/3832292451 conn(0x7fef140a9720 msgr2=0x7fef140ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:18.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 shutdown_connections 2024-09-13T16:48:18.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.076+0000 7fef1b1d9640 1 -- 172.21.15.39:0/3832292451 wait complete. 2024-09-13T16:48:18.098 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-13T16:48:17.563162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595092,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":248,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194,"ondisk_log_size":194,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":375,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235816,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738904,"statfs":{"total":773094113280,"available":772852637696,"internally_reserved":0,"allocated":3350528,"data_stored":1983984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.422068"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174677+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T16:48:09.174677+0000","last_peered":"2024-09-13T16:48:09.174677+0000","last_clean":"2024-09-13T16:48:09.174677+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T16:48:09.174677+0000","last_undegraded":"2024-09-13T16:48:09.174677+0000","last_fullsized":"2024-09-13T16:48:09.174677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163864+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:08.163864+0000","last_peered":"2024-09-13T16:48:08.163864+0000","last_clean":"2024-09-13T16:48:08.163864+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:08.163864+0000","last_undegraded":"2024-09-13T16:48:08.163864+0000","last_fullsized":"2024-09-13T16:48:08.163864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146105+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:08.146105+0000","last_peered":"2024-09-13T16:48:08.146105+0000","last_clean":"2024-09-13T16:48:08.146105+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:08.146105+0000","last_undegraded":"2024-09-13T16:48:08.146105+0000","last_fullsized":"2024-09-13T16:48:08.146105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143176+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T16:48:08.143176+0000","last_peered":"2024-09-13T16:48:08.143176+0000","last_clean":"2024-09-13T16:48:08.143176+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T16:48:08.143176+0000","last_undegraded":"2024-09-13T16:48:08.143176+0000","last_fullsized":"2024-09-13T16:48:08.143176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173274+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T16:48:09.173274+0000","last_peered":"2024-09-13T16:48:09.173274+0000","last_clean":"2024-09-13T16:48:09.173274+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T16:48:09.173274+0000","last_undegraded":"2024-09-13T16:48:09.173274+0000","last_fullsized":"2024-09-13T16:48:09.173274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146246+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:08.146246+0000","last_peered":"2024-09-13T16:48:08.146246+0000","last_clean":"2024-09-13T16:48:08.146246+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:08.146246+0000","last_undegraded":"2024-09-13T16:48:08.146246+0000","last_fullsized":"2024-09-13T16:48:08.146246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163480+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:08.163480+0000","last_peered":"2024-09-13T16:48:08.163480+0000","last_clean":"2024-09-13T16:48:08.163480+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:08.163480+0000","last_undegraded":"2024-09-13T16:48:08.163480+0000","last_fullsized":"2024-09-13T16:48:08.163480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184157+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T16:48:08.184157+0000","last_peered":"2024-09-13T16:48:08.184157+0000","last_clean":"2024-09-13T16:48:08.184157+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T16:48:08.184157+0000","last_undegraded":"2024-09-13T16:48:08.184157+0000","last_fullsized":"2024-09-13T16:48:08.184157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173949+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T16:48:09.173949+0000","last_peered":"2024-09-13T16:48:09.173949+0000","last_clean":"2024-09-13T16:48:09.173949+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T16:48:09.173949+0000","last_undegraded":"2024-09-13T16:48:09.173949+0000","last_fullsized":"2024-09-13T16:48:09.173949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178923+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:08.178923+0000","last_peered":"2024-09-13T16:48:08.178923+0000","last_clean":"2024-09-13T16:48:08.178923+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:08.178923+0000","last_undegraded":"2024-09-13T16:48:08.178923+0000","last_fullsized":"2024-09-13T16:48:08.178923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151847+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:08.151847+0000","last_peered":"2024-09-13T16:48:08.151847+0000","last_clean":"2024-09-13T16:48:08.151847+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:08.151847+0000","last_undegraded":"2024-09-13T16:48:08.151847+0000","last_fullsized":"2024-09-13T16:48:08.151847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208404+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T16:48:08.208404+0000","last_peered":"2024-09-13T16:48:08.208404+0000","last_clean":"2024-09-13T16:48:08.208404+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T16:48:08.208404+0000","last_undegraded":"2024-09-13T16:48:08.208404+0000","last_fullsized":"2024-09-13T16:48:08.208404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159591+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T16:48:09.159591+0000","last_peered":"2024-09-13T16:48:09.159591+0000","last_clean":"2024-09-13T16:48:09.159591+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T16:48:09.159591+0000","last_undegraded":"2024-09-13T16:48:09.159591+0000","last_fullsized":"2024-09-13T16:48:09.159591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165397+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:08.165397+0000","last_peered":"2024-09-13T16:48:08.165397+0000","last_clean":"2024-09-13T16:48:08.165397+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:08.165397+0000","last_undegraded":"2024-09-13T16:48:08.165397+0000","last_fullsized":"2024-09-13T16:48:08.165397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153028+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:08.153028+0000","last_peered":"2024-09-13T16:48:08.153028+0000","last_clean":"2024-09-13T16:48:08.153028+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:08.153028+0000","last_undegraded":"2024-09-13T16:48:08.153028+0000","last_fullsized":"2024-09-13T16:48:08.153028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.203658+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T16:48:08.203658+0000","last_peered":"2024-09-13T16:48:08.203658+0000","last_clean":"2024-09-13T16:48:08.203658+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T16:48:08.203658+0000","last_undegraded":"2024-09-13T16:48:08.203658+0000","last_fullsized":"2024-09-13T16:48:08.203658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173632+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T16:48:09.173632+0000","last_peered":"2024-09-13T16:48:09.173632+0000","last_clean":"2024-09-13T16:48:09.173632+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T16:48:09.173632+0000","last_undegraded":"2024-09-13T16:48:09.173632+0000","last_fullsized":"2024-09-13T16:48:09.173632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146884+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:08.146884+0000","last_peered":"2024-09-13T16:48:08.146884+0000","last_clean":"2024-09-13T16:48:08.146884+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:08.146884+0000","last_undegraded":"2024-09-13T16:48:08.146884+0000","last_fullsized":"2024-09-13T16:48:08.146884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160788+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:08.160788+0000","last_peered":"2024-09-13T16:48:08.160788+0000","last_clean":"2024-09-13T16:48:08.160788+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:08.160788+0000","last_undegraded":"2024-09-13T16:48:08.160788+0000","last_fullsized":"2024-09-13T16:48:08.160788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.206623+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T16:48:08.206623+0000","last_peered":"2024-09-13T16:48:08.206623+0000","last_clean":"2024-09-13T16:48:08.206623+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T16:48:08.206623+0000","last_undegraded":"2024-09-13T16:48:08.206623+0000","last_fullsized":"2024-09-13T16:48:08.206623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159080+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T16:48:09.159080+0000","last_peered":"2024-09-13T16:48:09.159080+0000","last_clean":"2024-09-13T16:48:09.159080+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T16:48:09.159080+0000","last_undegraded":"2024-09-13T16:48:09.159080+0000","last_fullsized":"2024-09-13T16:48:09.159080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144345+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:08.144345+0000","last_peered":"2024-09-13T16:48:08.144345+0000","last_clean":"2024-09-13T16:48:08.144345+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:08.144345+0000","last_undegraded":"2024-09-13T16:48:08.144345+0000","last_fullsized":"2024-09-13T16:48:08.144345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147467+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:08.147467+0000","last_peered":"2024-09-13T16:48:08.147467+0000","last_clean":"2024-09-13T16:48:08.147467+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:08.147467+0000","last_undegraded":"2024-09-13T16:48:08.147467+0000","last_fullsized":"2024-09-13T16:48:08.147467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184081+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T16:48:08.184081+0000","last_peered":"2024-09-13T16:48:08.184081+0000","last_clean":"2024-09-13T16:48:08.184081+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T16:48:08.184081+0000","last_undegraded":"2024-09-13T16:48:08.184081+0000","last_fullsized":"2024-09-13T16:48:08.184081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143476+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:08.143476+0000","last_peered":"2024-09-13T16:48:08.143476+0000","last_clean":"2024-09-13T16:48:08.143476+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:08.143476+0000","last_undegraded":"2024-09-13T16:48:08.143476+0000","last_fullsized":"2024-09-13T16:48:08.143476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147565+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:08.147565+0000","last_peered":"2024-09-13T16:48:08.147565+0000","last_clean":"2024-09-13T16:48:08.147565+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:08.147565+0000","last_undegraded":"2024-09-13T16:48:08.147565+0000","last_fullsized":"2024-09-13T16:48:08.147565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174007+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T16:48:08.174007+0000","last_peered":"2024-09-13T16:48:08.174007+0000","last_clean":"2024-09-13T16:48:08.174007+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T16:48:08.174007+0000","last_undegraded":"2024-09-13T16:48:08.174007+0000","last_fullsized":"2024-09-13T16:48:08.174007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162482+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T16:48:09.162482+0000","last_peered":"2024-09-13T16:48:09.162482+0000","last_clean":"2024-09-13T16:48:09.162482+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T16:48:09.162482+0000","last_undegraded":"2024-09-13T16:48:09.162482+0000","last_fullsized":"2024-09-13T16:48:09.162482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186454+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:08.186454+0000","last_peered":"2024-09-13T16:48:08.186454+0000","last_clean":"2024-09-13T16:48:08.186454+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:08.186454+0000","last_undegraded":"2024-09-13T16:48:08.186454+0000","last_fullsized":"2024-09-13T16:48:08.186454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160985+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:08.160985+0000","last_peered":"2024-09-13T16:48:08.160985+0000","last_clean":"2024-09-13T16:48:08.160985+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:08.160985+0000","last_undegraded":"2024-09-13T16:48:08.160985+0000","last_fullsized":"2024-09-13T16:48:08.160985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208189+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T16:48:08.208189+0000","last_peered":"2024-09-13T16:48:08.208189+0000","last_clean":"2024-09-13T16:48:08.208189+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T16:48:08.208189+0000","last_undegraded":"2024-09-13T16:48:08.208189+0000","last_fullsized":"2024-09-13T16:48:08.208189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176059+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T16:48:09.176059+0000","last_peered":"2024-09-13T16:48:09.176059+0000","last_clean":"2024-09-13T16:48:09.176059+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T16:48:09.176059+0000","last_undegraded":"2024-09-13T16:48:09.176059+0000","last_fullsized":"2024-09-13T16:48:09.176059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163768+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:08.163768+0000","last_peered":"2024-09-13T16:48:08.163768+0000","last_clean":"2024-09-13T16:48:08.163768+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:08.163768+0000","last_undegraded":"2024-09-13T16:48:08.163768+0000","last_fullsized":"2024-09-13T16:48:08.163768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148721+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:08.148721+0000","last_peered":"2024-09-13T16:48:08.148721+0000","last_clean":"2024-09-13T16:48:08.148721+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:08.148721+0000","last_undegraded":"2024-09-13T16:48:08.148721+0000","last_fullsized":"2024-09-13T16:48:08.148721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185371+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T16:48:08.185371+0000","last_peered":"2024-09-13T16:48:08.185371+0000","last_clean":"2024-09-13T16:48:08.185371+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T16:48:08.185371+0000","last_undegraded":"2024-09-13T16:48:08.185371+0000","last_fullsized":"2024-09-13T16:48:08.185371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174416+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T16:48:09.174416+0000","last_peered":"2024-09-13T16:48:09.174416+0000","last_clean":"2024-09-13T16:48:09.174416+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T16:48:09.174416+0000","last_undegraded":"2024-09-13T16:48:09.174416+0000","last_fullsized":"2024-09-13T16:48:09.174416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145183+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:08.145183+0000","last_peered":"2024-09-13T16:48:08.145183+0000","last_clean":"2024-09-13T16:48:08.145183+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:08.145183+0000","last_undegraded":"2024-09-13T16:48:08.145183+0000","last_fullsized":"2024-09-13T16:48:08.145183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152363+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:08.152363+0000","last_peered":"2024-09-13T16:48:08.152363+0000","last_clean":"2024-09-13T16:48:08.152363+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:08.152363+0000","last_undegraded":"2024-09-13T16:48:08.152363+0000","last_fullsized":"2024-09-13T16:48:08.152363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143237+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T16:48:08.143237+0000","last_peered":"2024-09-13T16:48:08.143237+0000","last_clean":"2024-09-13T16:48:08.143237+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T16:48:08.143237+0000","last_undegraded":"2024-09-13T16:48:08.143237+0000","last_fullsized":"2024-09-13T16:48:08.143237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158858+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T16:48:09.158858+0000","last_peered":"2024-09-13T16:48:09.158858+0000","last_clean":"2024-09-13T16:48:09.158858+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T16:48:09.158858+0000","last_undegraded":"2024-09-13T16:48:09.158858+0000","last_fullsized":"2024-09-13T16:48:09.158858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148446+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:08.148446+0000","last_peered":"2024-09-13T16:48:08.148446+0000","last_clean":"2024-09-13T16:48:08.148446+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:08.148446+0000","last_undegraded":"2024-09-13T16:48:08.148446+0000","last_fullsized":"2024-09-13T16:48:08.148446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179426+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:08.179426+0000","last_peered":"2024-09-13T16:48:08.179426+0000","last_clean":"2024-09-13T16:48:08.179426+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:08.179426+0000","last_undegraded":"2024-09-13T16:48:08.179426+0000","last_fullsized":"2024-09-13T16:48:08.179426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161207+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T16:48:08.161207+0000","last_peered":"2024-09-13T16:48:08.161207+0000","last_clean":"2024-09-13T16:48:08.161207+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T16:48:08.161207+0000","last_undegraded":"2024-09-13T16:48:08.161207+0000","last_fullsized":"2024-09-13T16:48:08.161207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173006+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T16:48:09.173006+0000","last_peered":"2024-09-13T16:48:09.173006+0000","last_clean":"2024-09-13T16:48:09.173006+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T16:48:09.173006+0000","last_undegraded":"2024-09-13T16:48:09.173006+0000","last_fullsized":"2024-09-13T16:48:09.173006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153222+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:08.153222+0000","last_peered":"2024-09-13T16:48:08.153222+0000","last_clean":"2024-09-13T16:48:08.153222+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:08.153222+0000","last_undegraded":"2024-09-13T16:48:08.153222+0000","last_fullsized":"2024-09-13T16:48:08.153222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148522+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:08.148522+0000","last_peered":"2024-09-13T16:48:08.148522+0000","last_clean":"2024-09-13T16:48:08.148522+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:08.148522+0000","last_undegraded":"2024-09-13T16:48:08.148522+0000","last_fullsized":"2024-09-13T16:48:08.148522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146684+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T16:48:08.146684+0000","last_peered":"2024-09-13T16:48:08.146684+0000","last_clean":"2024-09-13T16:48:08.146684+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T16:48:08.146684+0000","last_undegraded":"2024-09-13T16:48:08.146684+0000","last_fullsized":"2024-09-13T16:48:08.146684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174867+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T16:48:09.174867+0000","last_peered":"2024-09-13T16:48:09.174867+0000","last_clean":"2024-09-13T16:48:09.174867+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T16:48:09.174867+0000","last_undegraded":"2024-09-13T16:48:09.174867+0000","last_fullsized":"2024-09-13T16:48:09.174867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185855+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:08.185855+0000","last_peered":"2024-09-13T16:48:08.185855+0000","last_clean":"2024-09-13T16:48:08.185855+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:08.185855+0000","last_undegraded":"2024-09-13T16:48:08.185855+0000","last_fullsized":"2024-09-13T16:48:08.185855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149466+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:08.149466+0000","last_peered":"2024-09-13T16:48:08.149466+0000","last_clean":"2024-09-13T16:48:08.149466+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:08.149466+0000","last_undegraded":"2024-09-13T16:48:08.149466+0000","last_fullsized":"2024-09-13T16:48:08.149466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148372+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T16:48:08.148372+0000","last_peered":"2024-09-13T16:48:08.148372+0000","last_clean":"2024-09-13T16:48:08.148372+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T16:48:08.148372+0000","last_undegraded":"2024-09-13T16:48:08.148372+0000","last_fullsized":"2024-09-13T16:48:08.148372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159186+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T16:48:09.159186+0000","last_peered":"2024-09-13T16:48:09.159186+0000","last_clean":"2024-09-13T16:48:09.159186+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T16:48:09.159186+0000","last_undegraded":"2024-09-13T16:48:09.159186+0000","last_fullsized":"2024-09-13T16:48:09.159186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146860+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:08.146860+0000","last_peered":"2024-09-13T16:48:08.146860+0000","last_clean":"2024-09-13T16:48:08.146860+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:08.146860+0000","last_undegraded":"2024-09-13T16:48:08.146860+0000","last_fullsized":"2024-09-13T16:48:08.146860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154111+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:08.154111+0000","last_peered":"2024-09-13T16:48:08.154111+0000","last_clean":"2024-09-13T16:48:08.154111+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:08.154111+0000","last_undegraded":"2024-09-13T16:48:08.154111+0000","last_fullsized":"2024-09-13T16:48:08.154111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175388+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T16:48:08.175388+0000","last_peered":"2024-09-13T16:48:08.175388+0000","last_clean":"2024-09-13T16:48:08.175388+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T16:48:08.175388+0000","last_undegraded":"2024-09-13T16:48:08.175388+0000","last_fullsized":"2024-09-13T16:48:08.175388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162556+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T16:48:09.162556+0000","last_peered":"2024-09-13T16:48:09.162556+0000","last_clean":"2024-09-13T16:48:09.162556+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T16:48:09.162556+0000","last_undegraded":"2024-09-13T16:48:09.162556+0000","last_fullsized":"2024-09-13T16:48:09.162556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160990+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:08.160990+0000","last_peered":"2024-09-13T16:48:08.160990+0000","last_clean":"2024-09-13T16:48:08.160990+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:08.160990+0000","last_undegraded":"2024-09-13T16:48:08.160990+0000","last_fullsized":"2024-09-13T16:48:08.160990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161254+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:08.161254+0000","last_peered":"2024-09-13T16:48:08.161254+0000","last_clean":"2024-09-13T16:48:08.161254+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:08.161254+0000","last_undegraded":"2024-09-13T16:48:08.161254+0000","last_fullsized":"2024-09-13T16:48:08.161254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163476+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T16:48:08.163476+0000","last_peered":"2024-09-13T16:48:08.163476+0000","last_clean":"2024-09-13T16:48:08.163476+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T16:48:08.163476+0000","last_undegraded":"2024-09-13T16:48:08.163476+0000","last_fullsized":"2024-09-13T16:48:08.163476+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175463+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T16:48:09.175463+0000","last_peered":"2024-09-13T16:48:09.175463+0000","last_clean":"2024-09-13T16:48:09.175463+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T16:48:09.175463+0000","last_undegraded":"2024-09-13T16:48:09.175463+0000","last_fullsized":"2024-09-13T16:48:09.175463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143792+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:08.143792+0000","last_peered":"2024-09-13T16:48:08.143792+0000","last_clean":"2024-09-13T16:48:08.143792+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:08.143792+0000","last_undegraded":"2024-09-13T16:48:08.143792+0000","last_fullsized":"2024-09-13T16:48:08.143792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148284+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:08.148284+0000","last_peered":"2024-09-13T16:48:08.148284+0000","last_clean":"2024-09-13T16:48:08.148284+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:08.148284+0000","last_undegraded":"2024-09-13T16:48:08.148284+0000","last_fullsized":"2024-09-13T16:48:08.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161478+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:08.161478+0000","last_peered":"2024-09-13T16:48:08.161478+0000","last_clean":"2024-09-13T16:48:08.161478+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:08.161478+0000","last_undegraded":"2024-09-13T16:48:08.161478+0000","last_fullsized":"2024-09-13T16:48:08.161478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148905+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T16:48:08.148905+0000","last_peered":"2024-09-13T16:48:08.148905+0000","last_clean":"2024-09-13T16:48:08.148905+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T16:48:08.148905+0000","last_undegraded":"2024-09-13T16:48:08.148905+0000","last_fullsized":"2024-09-13T16:48:08.148905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158934+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T16:48:09.158934+0000","last_peered":"2024-09-13T16:48:09.158934+0000","last_clean":"2024-09-13T16:48:09.158934+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T16:48:09.158934+0000","last_undegraded":"2024-09-13T16:48:09.158934+0000","last_fullsized":"2024-09-13T16:48:09.158934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178617+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:08.178617+0000","last_peered":"2024-09-13T16:48:08.178617+0000","last_clean":"2024-09-13T16:48:08.178617+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:08.178617+0000","last_undegraded":"2024-09-13T16:48:08.178617+0000","last_fullsized":"2024-09-13T16:48:08.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155913+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:08.155913+0000","last_peered":"2024-09-13T16:48:08.155913+0000","last_clean":"2024-09-13T16:48:08.155913+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:08.155913+0000","last_undegraded":"2024-09-13T16:48:08.155913+0000","last_fullsized":"2024-09-13T16:48:08.155913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146319+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T16:48:08.146319+0000","last_peered":"2024-09-13T16:48:08.146319+0000","last_clean":"2024-09-13T16:48:08.146319+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T16:48:08.146319+0000","last_undegraded":"2024-09-13T16:48:08.146319+0000","last_fullsized":"2024-09-13T16:48:08.146319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158609+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T16:48:09.158609+0000","last_peered":"2024-09-13T16:48:09.158609+0000","last_clean":"2024-09-13T16:48:09.158609+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T16:48:09.158609+0000","last_undegraded":"2024-09-13T16:48:09.158609+0000","last_fullsized":"2024-09-13T16:48:09.158609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152057+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:08.152057+0000","last_peered":"2024-09-13T16:48:08.152057+0000","last_clean":"2024-09-13T16:48:08.152057+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:08.152057+0000","last_undegraded":"2024-09-13T16:48:08.152057+0000","last_fullsized":"2024-09-13T16:48:08.152057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149487+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:08.149487+0000","last_peered":"2024-09-13T16:48:08.149487+0000","last_clean":"2024-09-13T16:48:08.149487+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:08.149487+0000","last_undegraded":"2024-09-13T16:48:08.149487+0000","last_fullsized":"2024-09-13T16:48:08.149487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175317+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T16:48:08.175317+0000","last_peered":"2024-09-13T16:48:08.175317+0000","last_clean":"2024-09-13T16:48:08.175317+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T16:48:08.175317+0000","last_undegraded":"2024-09-13T16:48:08.175317+0000","last_fullsized":"2024-09-13T16:48:08.175317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173209+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T16:48:09.173209+0000","last_peered":"2024-09-13T16:48:09.173209+0000","last_clean":"2024-09-13T16:48:09.173209+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T16:48:09.173209+0000","last_undegraded":"2024-09-13T16:48:09.173209+0000","last_fullsized":"2024-09-13T16:48:09.173209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.177954+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:08.177954+0000","last_peered":"2024-09-13T16:48:08.177954+0000","last_clean":"2024-09-13T16:48:08.177954+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:08.177954+0000","last_undegraded":"2024-09-13T16:48:08.177954+0000","last_fullsized":"2024-09-13T16:48:08.177954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155936+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:08.155936+0000","last_peered":"2024-09-13T16:48:08.155936+0000","last_clean":"2024-09-13T16:48:08.155936+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:08.155936+0000","last_undegraded":"2024-09-13T16:48:08.155936+0000","last_fullsized":"2024-09-13T16:48:08.155936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158692+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T16:48:08.158692+0000","last_peered":"2024-09-13T16:48:08.158692+0000","last_clean":"2024-09-13T16:48:08.158692+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T16:48:08.158692+0000","last_undegraded":"2024-09-13T16:48:08.158692+0000","last_fullsized":"2024-09-13T16:48:08.158692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158034+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T16:48:09.158034+0000","last_peered":"2024-09-13T16:48:09.158034+0000","last_clean":"2024-09-13T16:48:09.158034+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T16:48:09.158034+0000","last_undegraded":"2024-09-13T16:48:09.158034+0000","last_fullsized":"2024-09-13T16:48:09.158034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160811+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:08.160811+0000","last_peered":"2024-09-13T16:48:08.160811+0000","last_clean":"2024-09-13T16:48:08.160811+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:08.160811+0000","last_undegraded":"2024-09-13T16:48:08.160811+0000","last_fullsized":"2024-09-13T16:48:08.160811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.192267+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:08.192267+0000","last_peered":"2024-09-13T16:48:08.192267+0000","last_clean":"2024-09-13T16:48:08.192267+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:08.192267+0000","last_undegraded":"2024-09-13T16:48:08.192267+0000","last_fullsized":"2024-09-13T16:48:08.192267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185117+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T16:48:08.185117+0000","last_peered":"2024-09-13T16:48:08.185117+0000","last_clean":"2024-09-13T16:48:08.185117+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T16:48:08.185117+0000","last_undegraded":"2024-09-13T16:48:08.185117+0000","last_fullsized":"2024-09-13T16:48:08.185117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177888+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T16:48:09.177888+0000","last_peered":"2024-09-13T16:48:09.177888+0000","last_clean":"2024-09-13T16:48:09.177888+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T16:48:09.177888+0000","last_undegraded":"2024-09-13T16:48:09.177888+0000","last_fullsized":"2024-09-13T16:48:09.177888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167237+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:08.167237+0000","last_peered":"2024-09-13T16:48:08.167237+0000","last_clean":"2024-09-13T16:48:08.167237+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:08.167237+0000","last_undegraded":"2024-09-13T16:48:08.167237+0000","last_fullsized":"2024-09-13T16:48:08.167237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156043+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:08.156043+0000","last_peered":"2024-09-13T16:48:08.156043+0000","last_clean":"2024-09-13T16:48:08.156043+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:08.156043+0000","last_undegraded":"2024-09-13T16:48:08.156043+0000","last_fullsized":"2024-09-13T16:48:08.156043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163690+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T16:48:08.163690+0000","last_peered":"2024-09-13T16:48:08.163690+0000","last_clean":"2024-09-13T16:48:08.163690+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T16:48:08.163690+0000","last_undegraded":"2024-09-13T16:48:08.163690+0000","last_fullsized":"2024-09-13T16:48:08.163690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162212+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T16:48:09.162212+0000","last_peered":"2024-09-13T16:48:09.162212+0000","last_clean":"2024-09-13T16:48:09.162212+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T16:48:09.162212+0000","last_undegraded":"2024-09-13T16:48:09.162212+0000","last_fullsized":"2024-09-13T16:48:09.162212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144923+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:08.144923+0000","last_peered":"2024-09-13T16:48:08.144923+0000","last_clean":"2024-09-13T16:48:08.144923+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:08.144923+0000","last_undegraded":"2024-09-13T16:48:08.144923+0000","last_fullsized":"2024-09-13T16:48:08.144923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145599+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:08.145599+0000","last_peered":"2024-09-13T16:48:08.145599+0000","last_clean":"2024-09-13T16:48:08.145599+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:08.145599+0000","last_undegraded":"2024-09-13T16:48:08.145599+0000","last_fullsized":"2024-09-13T16:48:08.145599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145022+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T16:48:08.145022+0000","last_peered":"2024-09-13T16:48:08.145022+0000","last_clean":"2024-09-13T16:48:08.145022+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T16:48:08.145022+0000","last_undegraded":"2024-09-13T16:48:08.145022+0000","last_fullsized":"2024-09-13T16:48:08.145022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161391+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T16:48:09.161391+0000","last_peered":"2024-09-13T16:48:09.161391+0000","last_clean":"2024-09-13T16:48:09.161391+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T16:48:09.161391+0000","last_undegraded":"2024-09-13T16:48:09.161391+0000","last_fullsized":"2024-09-13T16:48:09.161391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160556+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:08.160556+0000","last_peered":"2024-09-13T16:48:08.160556+0000","last_clean":"2024-09-13T16:48:08.160556+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:08.160556+0000","last_undegraded":"2024-09-13T16:48:08.160556+0000","last_fullsized":"2024-09-13T16:48:08.160556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149235+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:08.149235+0000","last_peered":"2024-09-13T16:48:08.149235+0000","last_clean":"2024-09-13T16:48:08.149235+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:08.149235+0000","last_undegraded":"2024-09-13T16:48:08.149235+0000","last_fullsized":"2024-09-13T16:48:08.149235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.209853+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T16:48:08.209853+0000","last_peered":"2024-09-13T16:48:08.209853+0000","last_clean":"2024-09-13T16:48:08.209853+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T16:48:08.209853+0000","last_undegraded":"2024-09-13T16:48:08.209853+0000","last_fullsized":"2024-09-13T16:48:08.209853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161110+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:08.161110+0000","last_peered":"2024-09-13T16:48:08.161110+0000","last_clean":"2024-09-13T16:48:08.161110+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:08.161110+0000","last_undegraded":"2024-09-13T16:48:08.161110+0000","last_fullsized":"2024-09-13T16:48:08.161110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161421+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:08.161421+0000","last_peered":"2024-09-13T16:48:08.161421+0000","last_clean":"2024-09-13T16:48:08.161421+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:08.161421+0000","last_undegraded":"2024-09-13T16:48:08.161421+0000","last_fullsized":"2024-09-13T16:48:08.161421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.188902+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T16:48:08.188902+0000","last_peered":"2024-09-13T16:48:08.188902+0000","last_clean":"2024-09-13T16:48:08.188902+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T16:48:08.188902+0000","last_undegraded":"2024-09-13T16:48:08.188902+0000","last_fullsized":"2024-09-13T16:48:08.188902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165383+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:08.165383+0000","last_peered":"2024-09-13T16:48:08.165383+0000","last_clean":"2024-09-13T16:48:08.165383+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:08.165383+0000","last_undegraded":"2024-09-13T16:48:08.165383+0000","last_fullsized":"2024-09-13T16:48:08.165383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148543+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:08.148543+0000","last_peered":"2024-09-13T16:48:08.148543+0000","last_clean":"2024-09-13T16:48:08.148543+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:08.148543+0000","last_undegraded":"2024-09-13T16:48:08.148543+0000","last_fullsized":"2024-09-13T16:48:08.148543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146582+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T16:48:08.146582+0000","last_peered":"2024-09-13T16:48:08.146582+0000","last_clean":"2024-09-13T16:48:08.146582+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T16:48:08.146582+0000","last_undegraded":"2024-09-13T16:48:08.146582+0000","last_fullsized":"2024-09-13T16:48:08.146582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173477+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T16:48:09.173477+0000","last_peered":"2024-09-13T16:48:09.173477+0000","last_clean":"2024-09-13T16:48:09.173477+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T16:48:09.173477+0000","last_undegraded":"2024-09-13T16:48:09.173477+0000","last_fullsized":"2024-09-13T16:48:09.173477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.190852+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:08.190852+0000","last_peered":"2024-09-13T16:48:08.190852+0000","last_clean":"2024-09-13T16:48:08.190852+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:08.190852+0000","last_undegraded":"2024-09-13T16:48:08.190852+0000","last_fullsized":"2024-09-13T16:48:08.190852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149403+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:08.149403+0000","last_peered":"2024-09-13T16:48:08.149403+0000","last_clean":"2024-09-13T16:48:08.149403+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:08.149403+0000","last_undegraded":"2024-09-13T16:48:08.149403+0000","last_fullsized":"2024-09-13T16:48:08.149403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208558+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T16:48:08.208558+0000","last_peered":"2024-09-13T16:48:08.208558+0000","last_clean":"2024-09-13T16:48:08.208558+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T16:48:08.208558+0000","last_undegraded":"2024-09-13T16:48:08.208558+0000","last_fullsized":"2024-09-13T16:48:08.208558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158601+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T16:48:09.158601+0000","last_peered":"2024-09-13T16:48:09.158601+0000","last_clean":"2024-09-13T16:48:09.158601+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T16:48:09.158601+0000","last_undegraded":"2024-09-13T16:48:09.158601+0000","last_fullsized":"2024-09-13T16:48:09.158601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163596+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:08.163596+0000","last_peered":"2024-09-13T16:48:08.163596+0000","last_clean":"2024-09-13T16:48:08.163596+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:08.163596+0000","last_undegraded":"2024-09-13T16:48:08.163596+0000","last_fullsized":"2024-09-13T16:48:08.163596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156189+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:08.156189+0000","last_peered":"2024-09-13T16:48:08.156189+0000","last_clean":"2024-09-13T16:48:08.156189+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:08.156189+0000","last_undegraded":"2024-09-13T16:48:08.156189+0000","last_fullsized":"2024-09-13T16:48:08.156189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174258+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T16:48:08.174258+0000","last_peered":"2024-09-13T16:48:08.174258+0000","last_clean":"2024-09-13T16:48:08.174258+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T16:48:08.174258+0000","last_undegraded":"2024-09-13T16:48:08.174258+0000","last_fullsized":"2024-09-13T16:48:08.174258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173305+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T16:48:09.173305+0000","last_peered":"2024-09-13T16:48:09.173305+0000","last_clean":"2024-09-13T16:48:09.173305+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T16:48:09.173305+0000","last_undegraded":"2024-09-13T16:48:09.173305+0000","last_fullsized":"2024-09-13T16:48:09.173305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155963+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:08.155963+0000","last_peered":"2024-09-13T16:48:08.155963+0000","last_clean":"2024-09-13T16:48:08.155963+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:08.155963+0000","last_undegraded":"2024-09-13T16:48:08.155963+0000","last_fullsized":"2024-09-13T16:48:08.155963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148500+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:08.148500+0000","last_peered":"2024-09-13T16:48:08.148500+0000","last_clean":"2024-09-13T16:48:08.148500+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:08.148500+0000","last_undegraded":"2024-09-13T16:48:08.148500+0000","last_fullsized":"2024-09-13T16:48:08.148500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143144+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T16:48:08.143144+0000","last_peered":"2024-09-13T16:48:08.143144+0000","last_clean":"2024-09-13T16:48:08.143144+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T16:48:08.143144+0000","last_undegraded":"2024-09-13T16:48:08.143144+0000","last_fullsized":"2024-09-13T16:48:08.143144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161066+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T16:48:09.161066+0000","last_peered":"2024-09-13T16:48:09.161066+0000","last_clean":"2024-09-13T16:48:09.161066+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T16:48:09.161066+0000","last_undegraded":"2024-09-13T16:48:09.161066+0000","last_fullsized":"2024-09-13T16:48:09.161066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186757+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T16:48:08.186757+0000","last_peered":"2024-09-13T16:48:08.186757+0000","last_clean":"2024-09-13T16:48:08.186757+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T16:48:08.186757+0000","last_undegraded":"2024-09-13T16:48:08.186757+0000","last_fullsized":"2024-09-13T16:48:08.186757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147291+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:08.147291+0000","last_peered":"2024-09-13T16:48:08.147291+0000","last_clean":"2024-09-13T16:48:08.147291+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:08.147291+0000","last_undegraded":"2024-09-13T16:48:08.147291+0000","last_fullsized":"2024-09-13T16:48:08.147291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158941+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:08.158941+0000","last_peered":"2024-09-13T16:48:08.158941+0000","last_clean":"2024-09-13T16:48:08.158941+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:08.158941+0000","last_undegraded":"2024-09-13T16:48:08.158941+0000","last_fullsized":"2024-09-13T16:48:08.158941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175068+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T16:48:09.175068+0000","last_peered":"2024-09-13T16:48:09.175068+0000","last_clean":"2024-09-13T16:48:09.175068+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T16:48:09.175068+0000","last_undegraded":"2024-09-13T16:48:09.175068+0000","last_fullsized":"2024-09-13T16:48:09.175068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.182997+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T16:48:08.182997+0000","last_peered":"2024-09-13T16:48:08.182997+0000","last_clean":"2024-09-13T16:48:08.182997+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T16:48:08.182997+0000","last_undegraded":"2024-09-13T16:48:08.182997+0000","last_fullsized":"2024-09-13T16:48:08.182997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143381+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:08.143381+0000","last_peered":"2024-09-13T16:48:08.143381+0000","last_clean":"2024-09-13T16:48:08.143381+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:08.143381+0000","last_undegraded":"2024-09-13T16:48:08.143381+0000","last_fullsized":"2024-09-13T16:48:08.143381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143921+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:08.143921+0000","last_peered":"2024-09-13T16:48:08.143921+0000","last_clean":"2024-09-13T16:48:08.143921+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:08.143921+0000","last_undegraded":"2024-09-13T16:48:08.143921+0000","last_fullsized":"2024-09-13T16:48:08.143921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174477+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T16:48:09.174477+0000","last_peered":"2024-09-13T16:48:09.174477+0000","last_clean":"2024-09-13T16:48:09.174477+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T16:48:09.174477+0000","last_undegraded":"2024-09-13T16:48:09.174477+0000","last_fullsized":"2024-09-13T16:48:09.174477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147114+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:08.147114+0000","last_peered":"2024-09-13T16:48:08.147114+0000","last_clean":"2024-09-13T16:48:08.147114+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:08.147114+0000","last_undegraded":"2024-09-13T16:48:08.147114+0000","last_fullsized":"2024-09-13T16:48:08.147114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161410+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:08.161410+0000","last_peered":"2024-09-13T16:48:08.161410+0000","last_clean":"2024-09-13T16:48:08.161410+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:08.161410+0000","last_undegraded":"2024-09-13T16:48:08.161410+0000","last_fullsized":"2024-09-13T16:48:08.161410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170387+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T16:48:08.170387+0000","last_peered":"2024-09-13T16:48:08.170387+0000","last_clean":"2024-09-13T16:48:08.170387+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T16:48:08.170387+0000","last_undegraded":"2024-09-13T16:48:08.170387+0000","last_fullsized":"2024-09-13T16:48:08.170387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175603+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T16:48:09.175603+0000","last_peered":"2024-09-13T16:48:09.175603+0000","last_clean":"2024-09-13T16:48:09.175603+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T16:48:09.175603+0000","last_undegraded":"2024-09-13T16:48:09.175603+0000","last_fullsized":"2024-09-13T16:48:09.175603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144443+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:08.144443+0000","last_peered":"2024-09-13T16:48:08.144443+0000","last_clean":"2024-09-13T16:48:08.144443+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:08.144443+0000","last_undegraded":"2024-09-13T16:48:08.144443+0000","last_fullsized":"2024-09-13T16:48:08.144443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185002+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:08.185002+0000","last_peered":"2024-09-13T16:48:08.185002+0000","last_clean":"2024-09-13T16:48:08.185002+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:08.185002+0000","last_undegraded":"2024-09-13T16:48:08.185002+0000","last_fullsized":"2024-09-13T16:48:08.185002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170142+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T16:48:08.170142+0000","last_peered":"2024-09-13T16:48:08.170142+0000","last_clean":"2024-09-13T16:48:08.170142+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T16:48:08.170142+0000","last_undegraded":"2024-09-13T16:48:08.170142+0000","last_fullsized":"2024-09-13T16:48:08.170142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177615+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T16:48:09.177615+0000","last_peered":"2024-09-13T16:48:09.177615+0000","last_clean":"2024-09-13T16:48:09.177615+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T16:48:09.177615+0000","last_undegraded":"2024-09-13T16:48:09.177615+0000","last_fullsized":"2024-09-13T16:48:09.177615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144121+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:08.144121+0000","last_peered":"2024-09-13T16:48:08.144121+0000","last_clean":"2024-09-13T16:48:08.144121+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:08.144121+0000","last_undegraded":"2024-09-13T16:48:08.144121+0000","last_fullsized":"2024-09-13T16:48:08.144121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174330+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T16:48:09.174330+0000","last_peered":"2024-09-13T16:48:09.174330+0000","last_clean":"2024-09-13T16:48:09.174330+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T16:48:09.174330+0000","last_undegraded":"2024-09-13T16:48:09.174330+0000","last_fullsized":"2024-09-13T16:48:09.174330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153996+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:08.153996+0000","last_peered":"2024-09-13T16:48:08.153996+0000","last_clean":"2024-09-13T16:48:08.153996+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:08.153996+0000","last_undegraded":"2024-09-13T16:48:08.153996+0000","last_fullsized":"2024-09-13T16:48:08.153996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161029+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T16:48:09.161029+0000","last_peered":"2024-09-13T16:48:09.161029+0000","last_clean":"2024-09-13T16:48:09.161029+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T16:48:09.161029+0000","last_undegraded":"2024-09-13T16:48:09.161029+0000","last_fullsized":"2024-09-13T16:48:09.161029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154547+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:08.154547+0000","last_peered":"2024-09-13T16:48:08.154547+0000","last_clean":"2024-09-13T16:48:08.154547+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:08.154547+0000","last_undegraded":"2024-09-13T16:48:08.154547+0000","last_fullsized":"2024-09-13T16:48:08.154547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174304+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T16:48:09.174304+0000","last_peered":"2024-09-13T16:48:09.174304+0000","last_clean":"2024-09-13T16:48:09.174304+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T16:48:09.174304+0000","last_undegraded":"2024-09-13T16:48:09.174304+0000","last_fullsized":"2024-09-13T16:48:09.174304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161339+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:08.161339+0000","last_peered":"2024-09-13T16:48:08.161339+0000","last_clean":"2024-09-13T16:48:08.161339+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:08.161339+0000","last_undegraded":"2024-09-13T16:48:08.161339+0000","last_fullsized":"2024-09-13T16:48:08.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173686+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T16:48:09.173686+0000","last_peered":"2024-09-13T16:48:09.173686+0000","last_clean":"2024-09-13T16:48:09.173686+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T16:48:09.173686+0000","last_undegraded":"2024-09-13T16:48:09.173686+0000","last_fullsized":"2024-09-13T16:48:09.173686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145104+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:08.145104+0000","last_peered":"2024-09-13T16:48:08.145104+0000","last_clean":"2024-09-13T16:48:08.145104+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:08.145104+0000","last_undegraded":"2024-09-13T16:48:08.145104+0000","last_fullsized":"2024-09-13T16:48:08.145104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175359+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T16:48:09.175359+0000","last_peered":"2024-09-13T16:48:09.175359+0000","last_clean":"2024-09-13T16:48:09.175359+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T16:48:09.175359+0000","last_undegraded":"2024-09-13T16:48:09.175359+0000","last_fullsized":"2024-09-13T16:48:09.175359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149320+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:08.149320+0000","last_peered":"2024-09-13T16:48:08.149320+0000","last_clean":"2024-09-13T16:48:08.149320+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:08.149320+0000","last_undegraded":"2024-09-13T16:48:08.149320+0000","last_fullsized":"2024-09-13T16:48:08.149320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173205+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T16:48:09.173205+0000","last_peered":"2024-09-13T16:48:09.173205+0000","last_clean":"2024-09-13T16:48:09.173205+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T16:48:09.173205+0000","last_undegraded":"2024-09-13T16:48:09.173205+0000","last_fullsized":"2024-09-13T16:48:09.173205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149001+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:08.149001+0000","last_peered":"2024-09-13T16:48:08.149001+0000","last_clean":"2024-09-13T16:48:08.149001+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:08.149001+0000","last_undegraded":"2024-09-13T16:48:08.149001+0000","last_fullsized":"2024-09-13T16:48:08.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158758+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T16:48:09.158758+0000","last_peered":"2024-09-13T16:48:09.158758+0000","last_clean":"2024-09-13T16:48:09.158758+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T16:48:09.158758+0000","last_undegraded":"2024-09-13T16:48:09.158758+0000","last_fullsized":"2024-09-13T16:48:09.158758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145105+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:08.145105+0000","last_peered":"2024-09-13T16:48:08.145105+0000","last_clean":"2024-09-13T16:48:08.145105+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:08.145105+0000","last_undegraded":"2024-09-13T16:48:08.145105+0000","last_fullsized":"2024-09-13T16:48:08.145105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177538+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T16:48:09.177538+0000","last_peered":"2024-09-13T16:48:09.177538+0000","last_clean":"2024-09-13T16:48:09.177538+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T16:48:09.177538+0000","last_undegraded":"2024-09-13T16:48:09.177538+0000","last_fullsized":"2024-09-13T16:48:09.177538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158985+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:08.158985+0000","last_peered":"2024-09-13T16:48:08.158985+0000","last_clean":"2024-09-13T16:48:08.158985+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:08.158985+0000","last_undegraded":"2024-09-13T16:48:08.158985+0000","last_fullsized":"2024-09-13T16:48:08.158985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177461+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T16:48:09.177461+0000","last_peered":"2024-09-13T16:48:09.177461+0000","last_clean":"2024-09-13T16:48:09.177461+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T16:48:09.177461+0000","last_undegraded":"2024-09-13T16:48:09.177461+0000","last_fullsized":"2024-09-13T16:48:09.177461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143902+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:08.143902+0000","last_peered":"2024-09-13T16:48:08.143902+0000","last_clean":"2024-09-13T16:48:08.143902+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:08.143902+0000","last_undegraded":"2024-09-13T16:48:08.143902+0000","last_fullsized":"2024-09-13T16:48:08.143902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159127+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T16:48:09.159127+0000","last_peered":"2024-09-13T16:48:09.159127+0000","last_clean":"2024-09-13T16:48:09.159127+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T16:48:09.159127+0000","last_undegraded":"2024-09-13T16:48:09.159127+0000","last_fullsized":"2024-09-13T16:48:09.159127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148988+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:08.148988+0000","last_peered":"2024-09-13T16:48:08.148988+0000","last_clean":"2024-09-13T16:48:08.148988+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:08.148988+0000","last_undegraded":"2024-09-13T16:48:08.148988+0000","last_fullsized":"2024-09-13T16:48:08.148988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177190+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T16:48:09.177190+0000","last_peered":"2024-09-13T16:48:09.177190+0000","last_clean":"2024-09-13T16:48:09.177190+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T16:48:09.177190+0000","last_undegraded":"2024-09-13T16:48:09.177190+0000","last_fullsized":"2024-09-13T16:48:09.177190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161327+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:08.161327+0000","last_peered":"2024-09-13T16:48:08.161327+0000","last_clean":"2024-09-13T16:48:08.161327+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:08.161327+0000","last_undegraded":"2024-09-13T16:48:08.161327+0000","last_fullsized":"2024-09-13T16:48:08.161327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148225+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:08.148225+0000","last_peered":"2024-09-13T16:48:08.148225+0000","last_clean":"2024-09-13T16:48:08.148225+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:08.148225+0000","last_undegraded":"2024-09-13T16:48:08.148225+0000","last_fullsized":"2024-09-13T16:48:08.148225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175099+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T16:48:09.175099+0000","last_peered":"2024-09-13T16:48:09.175099+0000","last_clean":"2024-09-13T16:48:09.175099+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T16:48:09.175099+0000","last_undegraded":"2024-09-13T16:48:09.175099+0000","last_fullsized":"2024-09-13T16:48:09.175099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179166+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:08.179166+0000","last_peered":"2024-09-13T16:48:08.179166+0000","last_clean":"2024-09-13T16:48:08.179166+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:08.179166+0000","last_undegraded":"2024-09-13T16:48:08.179166+0000","last_fullsized":"2024-09-13T16:48:08.179166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158364+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T16:48:09.158364+0000","last_peered":"2024-09-13T16:48:09.158364+0000","last_clean":"2024-09-13T16:48:09.158364+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T16:48:09.158364+0000","last_undegraded":"2024-09-13T16:48:09.158364+0000","last_fullsized":"2024-09-13T16:48:09.158364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155807+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:08.155807+0000","last_peered":"2024-09-13T16:48:08.155807+0000","last_clean":"2024-09-13T16:48:08.155807+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:08.155807+0000","last_undegraded":"2024-09-13T16:48:08.155807+0000","last_fullsized":"2024-09-13T16:48:08.155807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.172410+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T16:48:09.172410+0000","last_peered":"2024-09-13T16:48:09.172410+0000","last_clean":"2024-09-13T16:48:09.172410+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T16:48:09.172410+0000","last_undegraded":"2024-09-13T16:48:09.172410+0000","last_fullsized":"2024-09-13T16:48:09.172410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.157093+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:08.157093+0000","last_peered":"2024-09-13T16:48:08.157093+0000","last_clean":"2024-09-13T16:48:08.157093+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:08.157093+0000","last_undegraded":"2024-09-13T16:48:08.157093+0000","last_fullsized":"2024-09-13T16:48:08.157093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173535+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T16:48:09.173535+0000","last_peered":"2024-09-13T16:48:09.173535+0000","last_clean":"2024-09-13T16:48:09.173535+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T16:48:09.173535+0000","last_undegraded":"2024-09-13T16:48:09.173535+0000","last_fullsized":"2024-09-13T16:48:09.173535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158715+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:08.158715+0000","last_peered":"2024-09-13T16:48:08.158715+0000","last_clean":"2024-09-13T16:48:08.158715+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:08.158715+0000","last_undegraded":"2024-09-13T16:48:08.158715+0000","last_fullsized":"2024-09-13T16:48:08.158715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175097+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T16:48:09.175097+0000","last_peered":"2024-09-13T16:48:09.175097+0000","last_clean":"2024-09-13T16:48:09.175097+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T16:48:09.175097+0000","last_undegraded":"2024-09-13T16:48:09.175097+0000","last_fullsized":"2024-09-13T16:48:09.175097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147419+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:08.147419+0000","last_peered":"2024-09-13T16:48:08.147419+0000","last_clean":"2024-09-13T16:48:08.147419+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:08.147419+0000","last_undegraded":"2024-09-13T16:48:08.147419+0000","last_fullsized":"2024-09-13T16:48:08.147419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159429+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T16:48:09.159429+0000","last_peered":"2024-09-13T16:48:09.159429+0000","last_clean":"2024-09-13T16:48:09.159429+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T16:48:09.159429+0000","last_undegraded":"2024-09-13T16:48:09.159429+0000","last_fullsized":"2024-09-13T16:48:09.159429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161028+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:08.161028+0000","last_peered":"2024-09-13T16:48:08.161028+0000","last_clean":"2024-09-13T16:48:08.161028+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:08.161028+0000","last_undegraded":"2024-09-13T16:48:08.161028+0000","last_fullsized":"2024-09-13T16:48:08.161028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177095+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T16:48:09.177095+0000","last_peered":"2024-09-13T16:48:09.177095+0000","last_clean":"2024-09-13T16:48:09.177095+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T16:48:09.177095+0000","last_undegraded":"2024-09-13T16:48:09.177095+0000","last_fullsized":"2024-09-13T16:48:09.177095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143741+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:08.143741+0000","last_peered":"2024-09-13T16:48:08.143741+0000","last_clean":"2024-09-13T16:48:08.143741+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:08.143741+0000","last_undegraded":"2024-09-13T16:48:08.143741+0000","last_fullsized":"2024-09-13T16:48:08.143741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176397+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T16:48:09.176397+0000","last_peered":"2024-09-13T16:48:09.176397+0000","last_clean":"2024-09-13T16:48:09.176397+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T16:48:09.176397+0000","last_undegraded":"2024-09-13T16:48:09.176397+0000","last_fullsized":"2024-09-13T16:48:09.176397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186343+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:08.186343+0000","last_peered":"2024-09-13T16:48:08.186343+0000","last_clean":"2024-09-13T16:48:08.186343+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:08.186343+0000","last_undegraded":"2024-09-13T16:48:08.186343+0000","last_fullsized":"2024-09-13T16:48:08.186343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158832+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T16:48:09.158832+0000","last_peered":"2024-09-13T16:48:09.158832+0000","last_clean":"2024-09-13T16:48:09.158832+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T16:48:09.158832+0000","last_undegraded":"2024-09-13T16:48:09.158832+0000","last_fullsized":"2024-09-13T16:48:09.158832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149361+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:08.149361+0000","last_peered":"2024-09-13T16:48:08.149361+0000","last_clean":"2024-09-13T16:48:08.149361+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:08.149361+0000","last_undegraded":"2024-09-13T16:48:08.149361+0000","last_fullsized":"2024-09-13T16:48:08.149361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173784+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T16:48:09.173784+0000","last_peered":"2024-09-13T16:48:09.173784+0000","last_clean":"2024-09-13T16:48:09.173784+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T16:48:09.173784+0000","last_undegraded":"2024-09-13T16:48:09.173784+0000","last_fullsized":"2024-09-13T16:48:09.173784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148791+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:08.148791+0000","last_peered":"2024-09-13T16:48:08.148791+0000","last_clean":"2024-09-13T16:48:08.148791+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:08.148791+0000","last_undegraded":"2024-09-13T16:48:08.148791+0000","last_fullsized":"2024-09-13T16:48:08.148791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159787+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T16:48:09.159787+0000","last_peered":"2024-09-13T16:48:09.159787+0000","last_clean":"2024-09-13T16:48:09.159787+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T16:48:09.159787+0000","last_undegraded":"2024-09-13T16:48:09.159787+0000","last_fullsized":"2024-09-13T16:48:09.159787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166109+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:08.166109+0000","last_peered":"2024-09-13T16:48:08.166109+0000","last_clean":"2024-09-13T16:48:08.166109+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:08.166109+0000","last_undegraded":"2024-09-13T16:48:08.166109+0000","last_fullsized":"2024-09-13T16:48:08.166109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.178540+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T16:48:09.178540+0000","last_peered":"2024-09-13T16:48:09.178540+0000","last_clean":"2024-09-13T16:48:09.178540+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T16:48:09.178540+0000","last_undegraded":"2024-09-13T16:48:09.178540+0000","last_fullsized":"2024-09-13T16:48:09.178540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143140+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:08.143140+0000","last_peered":"2024-09-13T16:48:08.143140+0000","last_clean":"2024-09-13T16:48:08.143140+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:08.143140+0000","last_undegraded":"2024-09-13T16:48:08.143140+0000","last_fullsized":"2024-09-13T16:48:08.143140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.160931+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T16:48:09.160931+0000","last_peered":"2024-09-13T16:48:09.160931+0000","last_clean":"2024-09-13T16:48:09.160931+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T16:48:09.160931+0000","last_undegraded":"2024-09-13T16:48:09.160931+0000","last_fullsized":"2024-09-13T16:48:09.160931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167027+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:08.167027+0000","last_peered":"2024-09-13T16:48:08.167027+0000","last_clean":"2024-09-13T16:48:08.167027+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:08.167027+0000","last_undegraded":"2024-09-13T16:48:08.167027+0000","last_fullsized":"2024-09-13T16:48:08.167027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173532+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T16:48:09.173532+0000","last_peered":"2024-09-13T16:48:09.173532+0000","last_clean":"2024-09-13T16:48:09.173532+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T16:48:09.173532+0000","last_undegraded":"2024-09-13T16:48:09.173532+0000","last_fullsized":"2024-09-13T16:48:09.173532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148753+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:08.148753+0000","last_peered":"2024-09-13T16:48:08.148753+0000","last_clean":"2024-09-13T16:48:08.148753+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:08.148753+0000","last_undegraded":"2024-09-13T16:48:08.148753+0000","last_fullsized":"2024-09-13T16:48:08.148753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161898+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T16:48:09.161898+0000","last_peered":"2024-09-13T16:48:09.161898+0000","last_clean":"2024-09-13T16:48:09.161898+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T16:48:09.161898+0000","last_undegraded":"2024-09-13T16:48:09.161898+0000","last_fullsized":"2024-09-13T16:48:09.161898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160351+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:08.160351+0000","last_peered":"2024-09-13T16:48:08.160351+0000","last_clean":"2024-09-13T16:48:08.160351+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:08.160351+0000","last_undegraded":"2024-09-13T16:48:08.160351+0000","last_fullsized":"2024-09-13T16:48:08.160351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176220+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T16:48:09.176220+0000","last_peered":"2024-09-13T16:48:09.176220+0000","last_clean":"2024-09-13T16:48:09.176220+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T16:48:09.176220+0000","last_undegraded":"2024-09-13T16:48:09.176220+0000","last_fullsized":"2024-09-13T16:48:09.176220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151613+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:08.151613+0000","last_peered":"2024-09-13T16:48:08.151613+0000","last_clean":"2024-09-13T16:48:08.151613+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:08.151613+0000","last_undegraded":"2024-09-13T16:48:08.151613+0000","last_fullsized":"2024-09-13T16:48:08.151613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158220+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T16:48:09.158220+0000","last_peered":"2024-09-13T16:48:09.158220+0000","last_clean":"2024-09-13T16:48:09.158220+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T16:48:09.158220+0000","last_undegraded":"2024-09-13T16:48:09.158220+0000","last_fullsized":"2024-09-13T16:48:09.158220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.162959+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:08.162959+0000","last_peered":"2024-09-13T16:48:08.162959+0000","last_clean":"2024-09-13T16:48:08.162959+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:08.162959+0000","last_undegraded":"2024-09-13T16:48:08.162959+0000","last_fullsized":"2024-09-13T16:48:08.162959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173056+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T16:48:09.173056+0000","last_peered":"2024-09-13T16:48:09.173056+0000","last_clean":"2024-09-13T16:48:09.173056+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T16:48:09.173056+0000","last_undegraded":"2024-09-13T16:48:09.173056+0000","last_fullsized":"2024-09-13T16:48:09.173056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.150218+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:08.150218+0000","last_peered":"2024-09-13T16:48:08.150218+0000","last_clean":"2024-09-13T16:48:08.150218+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:08.150218+0000","last_undegraded":"2024-09-13T16:48:08.150218+0000","last_fullsized":"2024-09-13T16:48:08.150218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173906+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T16:48:09.173906+0000","last_peered":"2024-09-13T16:48:09.173906+0000","last_clean":"2024-09-13T16:48:09.173906+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T16:48:09.173906+0000","last_undegraded":"2024-09-13T16:48:09.173906+0000","last_fullsized":"2024-09-13T16:48:09.173906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166070+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:08.166070+0000","last_peered":"2024-09-13T16:48:08.166070+0000","last_clean":"2024-09-13T16:48:08.166070+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:08.166070+0000","last_undegraded":"2024-09-13T16:48:08.166070+0000","last_fullsized":"2024-09-13T16:48:08.166070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":690265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":101405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":99455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":282624,"data_stored":102375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":258048,"data_stored":99295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":100971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":690231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:18.105 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T16:48:18.106 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T16:48:18.106 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T16:48:18.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T16:48:18.106 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T16:48:18.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.260+0000 7fe10377f640 1 Processor -- start 2024-09-13T16:48:18.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.260+0000 7fe10377f640 1 -- start start 2024-09-13T16:48:18.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10377f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe0fc0aabc0 0x7fe0fc0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10377f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 0x7fe0fc0c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10377f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe0fc0c3ee0 0x7fe0fc1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10377f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc152a80 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10377f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc0c30f0 con 0x7fe0fc0c3ee0 2024-09-13T16:48:18.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10377f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc151e00 con 0x7fe0fc06c870 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10151b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe0fc0aabc0 0x7fe0fc0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10151b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe0fc0aabc0 0x7fe0fc0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34354/0 (socket says 172.21.15.39:34354) 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10151b640 1 -- 172.21.15.39:0/902161371 learned_addr learned my addr 172.21.15.39:0/902161371 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe100d1a640 1 --2- 172.21.15.39:0/902161371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 0x7fe0fc0c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe101d1c640 1 --2- 172.21.15.39:0/902161371 >> v2:172.21.15.84:3300/0 conn(0x7fe0fc0c3ee0 0x7fe0fc1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10151b640 1 -- 172.21.15.39:0/902161371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 msgr2=0x7fe0fc0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10151b640 1 --2- 172.21.15.39:0/902161371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 0x7fe0fc0c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10151b640 1 -- 172.21.15.39:0/902161371 >> v2:172.21.15.84:3300/0 conn(0x7fe0fc0c3ee0 msgr2=0x7fe0fc1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10151b640 1 --2- 172.21.15.39:0/902161371 >> v2:172.21.15.84:3300/0 conn(0x7fe0fc0c3ee0 0x7fe0fc1550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10151b640 1 -- 172.21.15.39:0/902161371 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc1520e0 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe10151b640 1 --2- 172.21.15.39:0/902161371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 0x7fe0fc0aafa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc06c5f0 tx=0x7fe0f401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbafa26da10b3a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:18.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe0fbfff640 1 -- 172.21.15.39:0/902161371 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40a5750 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe0fbfff640 1 -- 172.21.15.39:0/902161371 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f40a58b0 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.264+0000 7fe0fbfff640 1 -- 172.21.15.39:0/902161371 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40a5c40 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- 172.21.15.39:0/902161371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 msgr2=0x7fe0fc0aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 --2- 172.21.15.39:0/902161371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 0x7fe0fc0aafa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc06c5f0 tx=0x7fe0f401c9c0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- 172.21.15.39:0/902161371 shutdown_connections 2024-09-13T16:48:18.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 --2- 172.21.15.39:0/902161371 >> v2:172.21.15.84:3300/0 conn(0x7fe0fc0c3ee0 0x7fe0fc1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 --2- 172.21.15.39:0/902161371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 0x7fe0fc0c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 --2- 172.21.15.39:0/902161371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 0x7fe0fc0aafa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- 172.21.15.39:0/902161371 >> 172.21.15.39:0/902161371 conn(0x7fe0fc069e10 msgr2=0x7fe0fc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:18.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- 172.21.15.39:0/902161371 shutdown_connections 2024-09-13T16:48:18.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- 172.21.15.39:0/902161371 wait complete. 2024-09-13T16:48:18.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 Processor -- start 2024-09-13T16:48:18.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- start start 2024-09-13T16:48:18.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 0x7fe0fc1f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 0x7fe0fc149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0fc0c3ee0 0x7fe0fc1edff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1ed460 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1f00f0 con 0x7fe0fc0c3ee0 2024-09-13T16:48:18.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1f1080 con 0x7fe0fc06c870 2024-09-13T16:48:18.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe100d1a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 0x7fe0fc149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe100d1a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 0x7fe0fc149a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34356/0 (socket says 172.21.15.39:34356) 2024-09-13T16:48:18.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe100d1a640 1 -- 172.21.15.39:0/1301203310 learned_addr learned my addr 172.21.15.39:0/1301203310 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:18.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10151b640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 0x7fe0fc1f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe101d1c640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0fc0c3ee0 0x7fe0fc1edff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe100d1a640 1 -- 172.21.15.39:0/1301203310 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 msgr2=0x7fe0fc1f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe100d1a640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 0x7fe0fc1f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe100d1a640 1 -- 172.21.15.39:0/1301203310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0fc0c3ee0 msgr2=0x7fe0fc1edff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe100d1a640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0fc0c3ee0 0x7fe0fc1edff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe100d1a640 1 -- 172.21.15.39:0/1301203310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f4009b30 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe100d1a640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 0x7fe0fc149a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000d7d0 tx=0x7fe0f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:18.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe0f9ffb640 1 -- 172.21.15.39:0/1301203310 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0004020 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe0f9ffb640 1 -- 172.21.15.39:0/1301203310 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f0004180 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc1ed6a0 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe0f9ffb640 1 -- 172.21.15.39:0/1301203310 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0004330 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.268+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc1f0340 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.272+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc06c650 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.272+0000 7fe0f9ffb640 1 -- 172.21.15.39:0/1301203310 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe0f0009640 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.272+0000 7fe0f9ffb640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0e8066690 0x7fe0e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.272+0000 7fe0f9ffb640 1 -- 172.21.15.39:0/1301203310 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fe0f0084680 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.272+0000 7fe10151b640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0e8066690 0x7fe0e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.272+0000 7fe10151b640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0e8066690 0x7fe0e8068b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40ae000 tx=0x7fe0f40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:18.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.276+0000 7fe0f9ffb640 1 -- 172.21.15.39:0/1301203310 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe0f0051750 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.472+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe0fc0bfee0 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.480+0000 7fe0f9ffb640 1 -- 172.21.15.39:0/1301203310 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164899 (secure 0 0 0) 0x7fe0f0054d70 con 0x7fe0fc0aabc0 2024-09-13T16:48:18.488 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:18.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0e8066690 msgr2=0x7fe0e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0e8066690 0x7fe0e8068b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40ae000 tx=0x7fe0f40a7070 comp rx=0 tx=0).stop 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 msgr2=0x7fe0fc149a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 0x7fe0fc149a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000d7d0 tx=0x7fe0f000dc90 comp rx=0 tx=0).stop 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 shutdown_connections 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe0e8066690 0x7fe0e8068b50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0fc0c3ee0 0x7fe0fc1edff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0aabc0 0x7fe0fc149a30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 --2- 172.21.15.39:0/1301203310 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0fc06c870 0x7fe0fc1f0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 >> 172.21.15.39:0/1301203310 conn(0x7fe0fc069e10 msgr2=0x7fe0fc15bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 shutdown_connections 2024-09-13T16:48:18.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.484+0000 7fe10377f640 1 -- 172.21.15.39:0/1301203310 wait complete. 2024-09-13T16:48:18.507 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1271482023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1271482023}]},"active_addr":"172.21.15.39:6833/1271482023","active_change":"2024-09-13T16:47:39.545242+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":106766488}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3818920016}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3548809210}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":2359699636}]}]} 2024-09-13T16:48:18.511 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T16:48:18.511 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T16:48:18.511 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:18.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f71a6367640 1 Processor -- start 2024-09-13T16:48:18.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f71a6367640 1 -- start start 2024-09-13T16:48:18.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f71a6367640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f71a00c7800 0x7f71a0154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f71a6367640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00c5a30 con 0x7f71a0071dd0 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f71a6367640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a01576a0 con 0x7f71a0151500 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f71a6367640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a0157980 con 0x7f71a00c7800 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f71a0071dd0 0x7f71a0154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33132/0 (socket says 172.21.15.39:33132) 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719f7fe640 1 -- 172.21.15.39:0/3690753324 learned_addr learned my addr 172.21.15.39:0/3690753324 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861778097 0 0) 0x7f71a00c5a30 con 0x7f71a0071dd0 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7188003530 con 0x7f71a0071dd0 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f71a4904640 1 --2- 172.21.15.39:0/3690753324 >> v2:172.21.15.84:3300/0 conn(0x7f71a00c7800 0x7f71a0154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3500120308 0 0) 0x7f7188003530 con 0x7f71a0071dd0 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 >> v2:172.21.15.84:3300/0 conn(0x7f71a00c7800 msgr2=0x7f71a0154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 --2- 172.21.15.39:0/3690753324 >> v2:172.21.15.84:3300/0 conn(0x7f71a00c7800 0x7f71a0154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 >> v1:172.21.15.84:6789/0 conn(0x7f71a0151500 legacy=0x7f71a00c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0157c60 con 0x7f71a0071dd0 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f71940016a0 con 0x7f71a0071dd0 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 >> v1:172.21.15.39:6789/0 conn(0x7f71a0071dd0 legacy=0x7f71a0154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f7188003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71880047d0 0x7f7188004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.636+0000 7f719effd640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7188005190 0x7f718800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0157980 con 0x7f7188003980 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718800c130 con 0x7f7188005190 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718800c2f0 con 0x7f71880047d0 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719ffff640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f7188003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719f7fe640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71880047d0 0x7f7188004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719ffff640 1 -- 172.21.15.39:0/3690753324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71880047d0 msgr2=0x7f7188004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:18.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719ffff640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71880047d0 0x7f7188004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719ffff640 1 -- 172.21.15.39:0/3690753324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7188005190 msgr2=0x7f718800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:18.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719ffff640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7188005190 0x7f718800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719ffff640 1 -- 172.21.15.39:0/3690753324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188003530 con 0x7f7188003980 2024-09-13T16:48:18.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f71a4904640 1 -- 172.21.15.39:0/3690753324 reap_dead start 2024-09-13T16:48:18.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719ffff640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f7188003da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7190001780 tx=0x7f719001ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2758f5a7e5a2807 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:18.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71900050d0 con 0x7f7188003980 2024-09-13T16:48:18.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71900a7050 con 0x7f7188003980 2024-09-13T16:48:18.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f719effd640 1 -- 172.21.15.39:0/3690753324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7190005490 con 0x7f7188003980 2024-09-13T16:48:18.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f71a6367640 1 -- 172.21.15.39:0/3690753324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 msgr2=0x7f7188003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f71a6367640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f7188003da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7190001780 tx=0x7f719001ce50 comp rx=0 tx=0).stop 2024-09-13T16:48:18.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f71a6367640 1 -- 172.21.15.39:0/3690753324 shutdown_connections 2024-09-13T16:48:18.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f71a6367640 1 --2- 172.21.15.39:0/3690753324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f7188003da0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f71a6367640 1 -- 172.21.15.39:0/3690753324 >> 172.21.15.39:0/3690753324 conn(0x7f71a00ab530 msgr2=0x7f71a00ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:18.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f71a6367640 1 -- 172.21.15.39:0/3690753324 shutdown_connections 2024-09-13T16:48:18.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.640+0000 7f71a6367640 1 -- 172.21.15.39:0/3690753324 wait complete. 2024-09-13T16:48:18.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 Processor -- start 2024-09-13T16:48:18.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 -- start start 2024-09-13T16:48:18.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f71a00c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a00c34b0 0x7f71a01ef770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a01efcb0 0x7f71a01554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0152a30 con 0x7f7188003980 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0153240 con 0x7f71a00c34b0 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00afb10 con 0x7f71a01efcb0 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f71a00c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f71a00c2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34362/0 (socket says 172.21.15.39:34362) 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719ffff640 1 -- 172.21.15.39:0/1027715425 learned_addr learned my addr 172.21.15.39:0/1027715425 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719f7fe640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a00c34b0 0x7f71a01ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a4904640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a01efcb0 0x7f71a01554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719ffff640 1 -- 172.21.15.39:0/1027715425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a01efcb0 msgr2=0x7f71a01554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719ffff640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a01efcb0 0x7f71a01554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719ffff640 1 -- 172.21.15.39:0/1027715425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a00c34b0 msgr2=0x7f71a01ef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719ffff640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a00c34b0 0x7f71a01ef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719ffff640 1 -- 172.21.15.39:0/1027715425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00c38f0 con 0x7f7188003980 2024-09-13T16:48:18.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719ffff640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f71a00c2f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f719001c220 tx=0x7f7188005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:18.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719cff9640 1 -- 172.21.15.39:0/1027715425 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71900072f0 con 0x7f7188003980 2024-09-13T16:48:18.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719cff9640 1 -- 172.21.15.39:0/1027715425 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7190007450 con 0x7f7188003980 2024-09-13T16:48:18.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a01539f0 con 0x7f7188003980 2024-09-13T16:48:18.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719cff9640 1 -- 172.21.15.39:0/1027715425 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71900067e0 con 0x7f7188003980 2024-09-13T16:48:18.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a0152f60 con 0x7f7188003980 2024-09-13T16:48:18.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a0154580 con 0x7f7188003980 2024-09-13T16:48:18.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.644+0000 7f719cff9640 1 -- 172.21.15.39:0/1027715425 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7190006940 con 0x7f7188003980 2024-09-13T16:48:18.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.648+0000 7f719cff9640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666c0 0x7f718c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.648+0000 7f719cff9640 1 -- 172.21.15.39:0/1027715425 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f719011e2c0 con 0x7f7188003980 2024-09-13T16:48:18.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.648+0000 7f719f7fe640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666c0 0x7f718c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.648+0000 7f719f7fe640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666c0 0x7f718c068b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7194001060 tx=0x7f7194007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:18.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.652+0000 7f719cff9640 1 -- 172.21.15.39:0/1027715425 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f71900eb440 con 0x7f7188003980 2024-09-13T16:48:18.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.808+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71a0071bb0 con 0x7f7188003980 2024-09-13T16:48:18.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.812+0000 7f719cff9640 1 -- 172.21.15.39:0/1027715425 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7f71900eea60 con 0x7f7188003980 2024-09-13T16:48:18.818 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:18.818 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:18.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666c0 msgr2=0x7f718c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666c0 0x7f718c068b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7194001060 tx=0x7f7194007040 comp rx=0 tx=0).stop 2024-09-13T16:48:18.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 msgr2=0x7f71a00c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f71a00c2f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f719001c220 tx=0x7f7188005800 comp rx=0 tx=0).stop 2024-09-13T16:48:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 shutdown_connections 2024-09-13T16:48:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666c0 0x7f718c068b80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a00c34b0 0x7f71a01ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a01efcb0 0x7f71a01554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 --2- 172.21.15.39:0/1027715425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188003980 0x7f71a00c2f70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 >> 172.21.15.39:0/1027715425 conn(0x7f71a00ab530 msgr2=0x7f71a00ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 shutdown_connections 2024-09-13T16:48:18.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.816+0000 7f71a6367640 1 -- 172.21.15.39:0/1027715425 wait complete. 2024-09-13T16:48:18.838 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T16:48:18.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:18.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec87c5a640 1 Processor -- start 2024-09-13T16:48:18.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec87c5a640 1 -- start start 2024-09-13T16:48:18.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec87c5a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec800c21d0 0x7fec800aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec87c5a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec8014b850 0x7fec8014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec87c5a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec800c3ae0 con 0x7fec8006aac0 2024-09-13T16:48:18.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec87c5a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec800ab340 con 0x7fec800c21d0 2024-09-13T16:48:18.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec87c5a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec800abb30 con 0x7fec8014b850 2024-09-13T16:48:18.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec859f6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec800c21d0 0x7fec800aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec861f7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec8014b850 0x7fec8014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec859f6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec800c21d0 0x7fec800aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54984/0 (socket says 172.21.15.39:54984) 2024-09-13T16:48:18.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec859f6640 1 -- 172.21.15.39:0/2433295211 learned_addr learned my addr 172.21.15.39:0/2433295211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:18.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec859f6640 1 -- 172.21.15.39:0/2433295211 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec8014b850 msgr2=0x7fec8014bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:18.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec859f6640 1 --2- 172.21.15.39:0/2433295211 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec8014b850 0x7fec8014bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec859f6640 1 -- 172.21.15.39:0/2433295211 >> v1:172.21.15.84:6789/0 conn(0x7fec8006aac0 legacy=0x7fec80151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.952+0000 7fec859f6640 1 -- 172.21.15.39:0/2433295211 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec800abe10 con 0x7fec800c21d0 2024-09-13T16:48:18.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec859f6640 1 --2- 172.21.15.39:0/2433295211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec800c21d0 0x7fec800aae00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009870 tx=0x7fec7c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=615fdb4b32ee70ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:18.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec849f4640 1 -- 172.21.15.39:0/2433295211 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0a6040 con 0x7fec800c21d0 2024-09-13T16:48:18.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec849f4640 1 -- 172.21.15.39:0/2433295211 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c0aa600 con 0x7fec800c21d0 2024-09-13T16:48:18.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec849f4640 1 -- 172.21.15.39:0/2433295211 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0aa990 con 0x7fec800c21d0 2024-09-13T16:48:18.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 -- 172.21.15.39:0/2433295211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec800c21d0 msgr2=0x7fec800aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 --2- 172.21.15.39:0/2433295211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec800c21d0 0x7fec800aae00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009870 tx=0x7fec7c01c4a0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 -- 172.21.15.39:0/2433295211 shutdown_connections 2024-09-13T16:48:18.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 --2- 172.21.15.39:0/2433295211 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec8014b850 0x7fec8014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 --2- 172.21.15.39:0/2433295211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec800c21d0 0x7fec800aae00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 -- 172.21.15.39:0/2433295211 >> 172.21.15.39:0/2433295211 conn(0x7fec80069e10 msgr2=0x7fec8006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:18.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 -- 172.21.15.39:0/2433295211 shutdown_connections 2024-09-13T16:48:18.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 -- 172.21.15.39:0/2433295211 wait complete. 2024-09-13T16:48:18.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 Processor -- start 2024-09-13T16:48:18.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 -- start start 2024-09-13T16:48:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec8006aac0 0x7fec80143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec800c21d0 0x7fec801517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8014b850 0x7fec801ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec801ee660 con 0x7fec8006aac0 2024-09-13T16:48:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec801ea3a0 con 0x7fec8014b850 2024-09-13T16:48:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec87c5a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec80155ce0 con 0x7fec800c21d0 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec859f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec8006aac0 0x7fec80143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec859f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec8006aac0 0x7fec80143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34366/0 (socket says 172.21.15.39:34366) 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.956+0000 7fec859f6640 1 -- 172.21.15.39:0/2166230592 learned_addr learned my addr 172.21.15.39:0/2166230592 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec861f7640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8014b850 0x7fec801ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec851f5640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec800c21d0 0x7fec801517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec859f6640 1 -- 172.21.15.39:0/2166230592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec800c21d0 msgr2=0x7fec801517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec859f6640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec800c21d0 0x7fec801517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec859f6640 1 -- 172.21.15.39:0/2166230592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8014b850 msgr2=0x7fec801ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec859f6640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8014b850 0x7fec801ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec859f6640 1 -- 172.21.15.39:0/2166230592 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0094d0 con 0x7fec8006aac0 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec859f6640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec8006aac0 0x7fec80143480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fec7c01cd40 tx=0x7fec7c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec767fc640 1 -- 172.21.15.39:0/2166230592 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b3600 con 0x7fec8006aac0 2024-09-13T16:48:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec767fc640 1 -- 172.21.15.39:0/2166230592 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c0b3780 con 0x7fec8006aac0 2024-09-13T16:48:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec800c1bb0 con 0x7fec8006aac0 2024-09-13T16:48:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec767fc640 1 -- 172.21.15.39:0/2166230592 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b3b30 con 0x7fec8006aac0 2024-09-13T16:48:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8014dde0 con 0x7fec8006aac0 2024-09-13T16:48:18.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec801ea610 con 0x7fec8006aac0 2024-09-13T16:48:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec767fc640 1 -- 172.21.15.39:0/2166230592 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec7c0b3c90 con 0x7fec8006aac0 2024-09-13T16:48:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec767fc640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec64066690 0x7fec64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec767fc640 1 -- 172.21.15.39:0/2166230592 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fec7c11ec80 con 0x7fec8006aac0 2024-09-13T16:48:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec851f5640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec64066690 0x7fec64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:18.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.960+0000 7fec851f5640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec64066690 0x7fec64068b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fec800c4df0 tx=0x7fec70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:18.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:18.964+0000 7fec767fc640 1 -- 172.21.15.39:0/2166230592 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fec7c0ebcc0 con 0x7fec8006aac0 2024-09-13T16:48:19.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.124+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec800c25b0 con 0x7fec8006aac0 2024-09-13T16:48:19.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.128+0000 7fec767fc640 1 -- 172.21.15.39:0/2166230592 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7fec7c0ef2e0 con 0x7fec8006aac0 2024-09-13T16:48:19.133 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:19.134 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:19.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec64066690 msgr2=0x7fec64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec64066690 0x7fec64068b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fec800c4df0 tx=0x7fec70009040 comp rx=0 tx=0).stop 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec8006aac0 msgr2=0x7fec80143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec8006aac0 0x7fec80143480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fec7c01cd40 tx=0x7fec7c01ace0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 shutdown_connections 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec64066690 0x7fec64068b50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8014b850 0x7fec801ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec800c21d0 0x7fec801517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 --2- 172.21.15.39:0/2166230592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec8006aac0 0x7fec80143480 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 >> 172.21.15.39:0/2166230592 conn(0x7fec80069e10 msgr2=0x7fec800c60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 shutdown_connections 2024-09-13T16:48:19.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.132+0000 7fec87c5a640 1 -- 172.21.15.39:0/2166230592 wait complete. 2024-09-13T16:48:19.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T16:48:19.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T16:48:19.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T16:48:19.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T16:48:19.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T16:48:19.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T16:48:19.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T16:48:19.155 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T16:48:19.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3e2efe640 1 Processor -- start 2024-09-13T16:48:19.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3e2efe640 1 -- start start 2024-09-13T16:48:19.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3e2efe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3dc0c18b0 0x7ff3dc14fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3e2efe640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc153390 con 0x7ff3dc14d700 2024-09-13T16:48:19.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3e2efe640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc153670 con 0x7ff3dc1502b0 2024-09-13T16:48:19.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3e2efe640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc154010 con 0x7ff3dc0c18b0 2024-09-13T16:48:19.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3e0c9a640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff3dc14d700 0x7ff3dc06c160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33134/0 (socket says 172.21.15.39:33134) 2024-09-13T16:48:19.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3dbfff640 1 --2- 172.21.15.39:0/4272193934 >> v2:172.21.15.84:3300/0 conn(0x7ff3dc0c18b0 0x7ff3dc14fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3e0c9a640 1 -- 172.21.15.39:0/4272193934 learned_addr learned my addr 172.21.15.39:0/4272193934 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:19.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3dbfff640 1 -- 172.21.15.39:0/4272193934 >> v1:172.21.15.84:6789/0 conn(0x7ff3dc1502b0 legacy=0x7ff3dc15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3dbfff640 1 -- 172.21.15.39:0/4272193934 >> v1:172.21.15.39:6789/0 conn(0x7ff3dc14d700 legacy=0x7ff3dc06c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.320+0000 7ff3dbfff640 1 -- 172.21.15.39:0/4272193934 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc153950 con 0x7ff3dc0c18b0 2024-09-13T16:48:19.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.324+0000 7fad61587640 1 Processor -- start 2024-09-13T16:48:19.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.324+0000 7fad61587640 1 -- start start 2024-09-13T16:48:19.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.324+0000 7ff3dbfff640 1 --2- 172.21.15.39:0/4272193934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3dc0c18b0 0x7ff3dc14fd70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009d90 tx=0x7ff3d401eb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb899516fd13a163 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad61587640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fad5404cee0 0x7fad540404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad61587640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad5400c960 0x7fad540d4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad61587640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fad540409e0 0x7fad540d4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad61587640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5404d2c0 con 0x7fad540409e0 2024-09-13T16:48:19.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad61587640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad540e22a0 con 0x7fad5404cee0 2024-09-13T16:48:19.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad61587640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad54048330 con 0x7fad5400c960 2024-09-13T16:48:19.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.324+0000 7ff3db7fe640 1 -- 172.21.15.39:0/4272193934 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40a79f0 con 0x7ff3dc0c18b0 2024-09-13T16:48:19.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.324+0000 7ff3db7fe640 1 -- 172.21.15.39:0/4272193934 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d40a7b50 con 0x7ff3dc0c18b0 2024-09-13T16:48:19.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.324+0000 7ff3db7fe640 1 -- 172.21.15.39:0/4272193934 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40af440 con 0x7ff3dc0c18b0 2024-09-13T16:48:19.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4272193934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3dc0c18b0 msgr2=0x7ff3dc14fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7ff3e2efe640 1 --2- 172.21.15.39:0/4272193934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3dc0c18b0 0x7ff3dc14fd70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009d90 tx=0x7ff3d401eb00 comp rx=0 tx=0).stop 2024-09-13T16:48:19.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad60d86640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fad540409e0 0x7fad540d4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad60d86640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fad540409e0 0x7fad540d4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34376/0 (socket says 172.21.15.39:34376) 2024-09-13T16:48:19.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad60d86640 1 -- 172.21.15.39:0/3124465330 learned_addr learned my addr 172.21.15.39:0/3124465330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4272193934 shutdown_connections 2024-09-13T16:48:19.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7ff3e2efe640 1 --2- 172.21.15.39:0/4272193934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3dc0c18b0 0x7ff3dc14fd70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4272193934 >> 172.21.15.39:0/4272193934 conn(0x7ff3dc066480 msgr2=0x7ff3dc0c5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad60d86640 1 -- 172.21.15.39:0/3124465330 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad5400c960 msgr2=0x7fad540d4300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T16:48:19.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad60d86640 1 --2- 172.21.15.39:0/3124465330 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad5400c960 0x7fad540d4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad60d86640 1 -- 172.21.15.39:0/3124465330 >> v2:172.21.15.84:3300/0 conn(0x7fad5404cee0 msgr2=0x7fad540404a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T16:48:19.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad60d86640 1 --2- 172.21.15.39:0/3124465330 >> v2:172.21.15.84:3300/0 conn(0x7fad5404cee0 0x7fad540404a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad60d86640 1 -- 172.21.15.39:0/3124465330 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad540e1620 con 0x7fad540409e0 2024-09-13T16:48:19.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4272193934 shutdown_connections 2024-09-13T16:48:19.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad60d86640 1 --2- 172.21.15.39:0/3124465330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 0x7fad540d4840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad50009870 tx=0x7fad5001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85cd730ccf8180f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7fad5affd640 1 -- 172.21.15.39:0/3124465330 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500ac630 con 0x7fad540409e0 2024-09-13T16:48:19.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad5affd640 1 -- 172.21.15.39:0/3124465330 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad500ac790 con 0x7fad540409e0 2024-09-13T16:48:19.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad5affd640 1 -- 172.21.15.39:0/3124465330 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500acb20 con 0x7fad540409e0 2024-09-13T16:48:19.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3df4a59640 1 Processor -- start 2024-09-13T16:48:19.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 -- 172.21.15.39:0/3124465330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 msgr2=0x7fad540d4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 --2- 172.21.15.39:0/3124465330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 0x7fad540d4840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fad50009870 tx=0x7fad5001c410 comp rx=0 tx=0).stop 2024-09-13T16:48:19.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.328+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4272193934 wait complete. 2024-09-13T16:48:19.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e2efe640 1 Processor -- start 2024-09-13T16:48:19.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 -- 172.21.15.39:0/3124465330 shutdown_connections 2024-09-13T16:48:19.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 --2- 172.21.15.39:0/3124465330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 0x7fad540d4840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 --2- 172.21.15.39:0/3124465330 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad5400c960 0x7fad540d4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 --2- 172.21.15.39:0/3124465330 >> v2:172.21.15.84:3300/0 conn(0x7fad5404cee0 0x7fad540404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 -- 172.21.15.39:0/3124465330 >> 172.21.15.39:0/3124465330 conn(0x7fad5404d4d0 msgr2=0x7fad5404d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3df4a59640 1 -- start start 2024-09-13T16:48:19.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 -- 172.21.15.39:0/3124465330 shutdown_connections 2024-09-13T16:48:19.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3df4a59640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3de80d5030 0x7f3de80d8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3df4a59640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de80d8850 con 0x7f3de800cda0 2024-09-13T16:48:19.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3df4a59640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de800c5e0 con 0x7f3de800d9e0 2024-09-13T16:48:19.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3df4a59640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de8049550 con 0x7f3de80d5030 2024-09-13T16:48:19.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3deed9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f3de800cda0 0x7f3de80e2210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33140/0 (socket says 172.21.15.39:33140) 2024-09-13T16:48:19.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3deed9d640 1 -- 172.21.15.39:0/756507229 learned_addr learned my addr 172.21.15.39:0/756507229 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:19.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e2efe640 1 -- start start 2024-09-13T16:48:19.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e2efe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3dc0c18b0 0x7ff3dc1f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e2efe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3dc14d700 0x7ff3dc141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e2efe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3dc1502b0 0x7ff3dc151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e2efe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1f5cb0 con 0x7ff3dc14d700 2024-09-13T16:48:19.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e2efe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc151690 con 0x7ff3dc1502b0 2024-09-13T16:48:19.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e2efe640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c4fc0 con 0x7ff3dc0c18b0 2024-09-13T16:48:19.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3dee59c640 1 --2- 172.21.15.39:0/756507229 >> v2:172.21.15.39:3300/0 conn(0x7f3de80d5030 0x7f3de80d8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3dbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3dc14d700 0x7ff3dc141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3dbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3dc14d700 0x7ff3dc141eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34386/0 (socket says 172.21.15.39:34386) 2024-09-13T16:48:19.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3dbfff640 1 -- 172.21.15.39:0/4203302459 learned_addr learned my addr 172.21.15.39:0/4203302459 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e0c9a640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3dc0c18b0 0x7ff3dc1f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e149b640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3dc1502b0 0x7ff3dc151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411370168 0 0) 0x7f3de80d8850 con 0x7f3de800cda0 2024-09-13T16:48:19.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd8003530 con 0x7f3de800cda0 2024-09-13T16:48:19.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 -- 172.21.15.39:0/3124465330 wait complete. 2024-09-13T16:48:19.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 Processor -- start 2024-09-13T16:48:19.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2451017571 0 0) 0x7f3dd8003530 con 0x7f3de800cda0 2024-09-13T16:48:19.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 >> v2:172.21.15.39:3300/0 conn(0x7f3de80d5030 msgr2=0x7f3de80d8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 --2- 172.21.15.39:0/756507229 >> v2:172.21.15.39:3300/0 conn(0x7f3de80d5030 0x7f3de80d8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 >> v1:172.21.15.84:6789/0 conn(0x7f3de800d9e0 legacy=0x7f3de802ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8047f60 con 0x7f3de800cda0 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f3df0068160 con 0x7f3de800cda0 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 >> v1:172.21.15.39:6789/0 conn(0x7f3de800cda0 legacy=0x7f3de80e2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd8003980 0x7f3dd8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd8004300 0x7f3dd80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd8005070 0x7f3dd800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd800bf80 con 0x7f3dd8005070 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd800c0f0 con 0x7f3dd8004300 2024-09-13T16:48:19.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd800c2b0 con 0x7f3dd8003980 2024-09-13T16:48:19.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e149b640 1 -- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3dc0c18b0 msgr2=0x7ff3dc1f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e149b640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3dc0c18b0 0x7ff3dc1f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e149b640 1 -- 172.21.15.39:0/4203302459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3dc14d700 msgr2=0x7ff3dc141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e149b640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3dc14d700 0x7ff3dc141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3e149b640 1 -- 172.21.15.39:0/4203302459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4009a70 con 0x7ff3dc1502b0 2024-09-13T16:48:19.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7ff3dbfff640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3dc14d700 0x7ff3dc141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:19.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3deed9d640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd8005070 0x7f3dd800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7f3dee59c640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd8003980 0x7f3dd8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.332+0000 7fad61587640 1 -- start start 2024-09-13T16:48:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad61587640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad5400c960 0x7fad540418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad61587640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 0x7fad540434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad61587640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad5404cee0 0x7fad540421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad61587640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54041000 con 0x7fad540409e0 2024-09-13T16:48:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad61587640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54041550 con 0x7fad5400c960 2024-09-13T16:48:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad61587640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad54180460 con 0x7fad5404cee0 2024-09-13T16:48:19.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 0x7fad540434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad5400c960 0x7fad540418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 0x7fad540434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34398/0 (socket says 172.21.15.39:34398) 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5b7fe640 1 -- 172.21.15.39:0/1848972270 learned_addr learned my addr 172.21.15.39:0/1848972270 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad60d86640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad5404cee0 0x7fad540421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5b7fe640 1 -- 172.21.15.39:0/1848972270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad5404cee0 msgr2=0x7fad540421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5b7fe640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad5404cee0 0x7fad540421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5b7fe640 1 -- 172.21.15.39:0/1848972270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad5400c960 msgr2=0x7fad540418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5b7fe640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad5400c960 0x7fad540418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5b7fe640 1 -- 172.21.15.39:0/1848972270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad500094d0 con 0x7fad540409e0 2024-09-13T16:48:19.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad5b7fe640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 0x7fad540434b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fad4800b3e0 tx=0x7fad4800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9ea5426640 1 Processor -- start 2024-09-13T16:48:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9ea5426640 1 -- start start 2024-09-13T16:48:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fe13965e640 1 Processor -- start 2024-09-13T16:48:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fe13965e640 1 -- start start 2024-09-13T16:48:19.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f3deed9d640 1 -- 172.21.15.39:0/756507229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd8003980 msgr2=0x7f3dd8003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f3deed9d640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd8003980 0x7f3dd8003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f3deed9d640 1 -- 172.21.15.39:0/756507229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd8004300 msgr2=0x7f3dd80047d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T16:48:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f3deed9d640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd8004300 0x7f3dd80047d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f3deed9d640 1 -- 172.21.15.39:0/756507229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8003530 con 0x7f3dd8005070 2024-09-13T16:48:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a7765640 1 Processor -- start 2024-09-13T16:48:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a7765640 1 -- start start 2024-09-13T16:48:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a7765640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 0x7f09a00c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a7765640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f09a00c45e0 0x7f09a0153400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a7765640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09a00c5180 0x7f09a0153940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a7765640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a00c4d30 con 0x7f09a00c45e0 2024-09-13T16:48:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a7765640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a00c0190 con 0x7f09a00c5180 2024-09-13T16:48:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a7765640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a006d2a0 con 0x7f09a014d570 2024-09-13T16:48:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a5501640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 0x7f09a00c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a5501640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 0x7f09a00c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45374/0 (socket says 172.21.15.39:45374) 2024-09-13T16:48:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a5501640 1 -- 172.21.15.39:0/3474770524 learned_addr learned my addr 172.21.15.39:0/3474770524 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a5d02640 1 --2- 172.21.15.39:0/3474770524 >> v2:172.21.15.84:3300/0 conn(0x7f09a00c5180 0x7f09a0153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3e149b640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3dc1502b0 0x7ff3dc151b20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00c390 tx=0x7ff3cc00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3d97fa640 1 -- 172.21.15.39:0/4203302459 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc0052d0 con 0x7ff3dc1502b0 2024-09-13T16:48:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3dc1429e0 con 0x7ff3dc1502b0 2024-09-13T16:48:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3dc0c3d30 con 0x7ff3dc1502b0 2024-09-13T16:48:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3d97fa640 1 -- 172.21.15.39:0/4203302459 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc005430 con 0x7ff3dc1502b0 2024-09-13T16:48:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3d97fa640 1 -- 172.21.15.39:0/4203302459 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc003840 con 0x7ff3dc1502b0 2024-09-13T16:48:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3bc000fd0 con 0x7ff3dc1502b0 2024-09-13T16:48:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad58ff9640 1 -- 172.21.15.39:0/1848972270 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48011830 con 0x7fad540409e0 2024-09-13T16:48:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad58ff9640 1 -- 172.21.15.39:0/1848972270 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad48011990 con 0x7fad540409e0 2024-09-13T16:48:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad58ff9640 1 -- 172.21.15.39:0/1848972270 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48011d20 con 0x7fad540409e0 2024-09-13T16:48:19.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad54180740 con 0x7fad540409e0 2024-09-13T16:48:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad54042700 con 0x7fad540409e0 2024-09-13T16:48:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f3deed9d640 1 -- 172.21.15.39:0/756507229 reap_dead start 2024-09-13T16:48:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad3c000fd0 con 0x7fad540409e0 2024-09-13T16:48:19.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9ea5426640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9ea014db70 0x7f9ea00c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9ea5426640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea0157950 0x7f9ea015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9ea5426640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea006a930 con 0x7f9ea00c76c0 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9ea5426640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00c5480 con 0x7f9ea014db70 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9ea5426640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0154690 con 0x7f9ea0157950 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9e9e7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9ea00c76c0 0x7f9ea0157240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:44308/0 (socket says 172.21.15.39:44308) 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9e9e7fc640 1 -- 172.21.15.39:0/615793667 learned_addr learned my addr 172.21.15.39:0/615793667 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9e9f7fe640 1 --2- 172.21.15.39:0/615793667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea0157950 0x7f9ea015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9e9effd640 1 --2- 172.21.15.39:0/615793667 >> v2:172.21.15.84:3300/0 conn(0x7f9ea014db70 0x7f9ea00c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a5d02640 1 -- 172.21.15.39:0/3474770524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 msgr2=0x7f09a00c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a5d02640 1 --2- 172.21.15.39:0/3474770524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 0x7f09a00c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3d97fa640 1 -- 172.21.15.39:0/4203302459 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3cc0039a0 con 0x7ff3dc1502b0 2024-09-13T16:48:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a5d02640 1 -- 172.21.15.39:0/3474770524 >> v2:172.21.15.39:3300/0 conn(0x7f09a00c45e0 msgr2=0x7f09a0153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a4d00640 1 --2- 172.21.15.39:0/3474770524 >> v2:172.21.15.39:3300/0 conn(0x7f09a00c45e0 0x7f09a0153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f3deed9d640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd8005070 0x7f3dd800b990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3df0074390 tx=0x7f3df0099840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b96438cde1861c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fe13965e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe13415b900 0x7fe13415bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fe13965e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1340c8980 con 0x7fe1340bcdb0 2024-09-13T16:48:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fe13965e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1340c8c60 con 0x7fe134155a40 2024-09-13T16:48:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fe13965e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1340c8f40 con 0x7fe13415b900 2024-09-13T16:48:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fe1337fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe1340bcdb0 0x7fe134069e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33156/0 (socket says 172.21.15.39:33156) 2024-09-13T16:48:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7fe1337fe640 1 -- 172.21.15.39:0/3599852275 learned_addr learned my addr 172.21.15.39:0/3599852275 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:19.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3d97fa640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3c80666c0 0x7ff3c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a5d02640 1 --2- 172.21.15.39:0/3474770524 >> v2:172.21.15.39:3300/0 conn(0x7f09a00c45e0 0x7f09a0153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a5d02640 1 -- 172.21.15.39:0/3474770524 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a00c03e0 con 0x7f09a00c5180 2024-09-13T16:48:19.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3d97fa640 1 -- 172.21.15.39:0/4203302459 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7ff3cc083d00 con 0x7ff3dc1502b0 2024-09-13T16:48:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f09a4d00640 1 --2- 172.21.15.39:0/3474770524 >> v2:172.21.15.39:3300/0 conn(0x7f09a00c45e0 0x7f09a0153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3d97fa640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7ff3c8070ee0 0x7ff3c8073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3d97fa640 1 -- 172.21.15.39:0/4203302459 --> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3c80739b0 con 0x7ff3c8070ee0 2024-09-13T16:48:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df012a680 con 0x7f3dd8005070 2024-09-13T16:48:19.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df012a800 con 0x7f3dd8005070 2024-09-13T16:48:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7ff3d97fa640 1 -- 172.21.15.39:0/4203302459 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3cc084070 con 0x7ff3dc1502b0 2024-09-13T16:48:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3ded59a640 1 -- 172.21.15.39:0/756507229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df012abb0 con 0x7f3dd8005070 2024-09-13T16:48:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7ff3dbfff640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7ff3c8070ee0 0x7ff3c8073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fe138e5d640 1 --2- 172.21.15.39:0/3599852275 >> v2:172.21.15.84:3300/0 conn(0x7fe13415b900 0x7fe13415bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3df4a59640 1 -- 172.21.15.39:0/756507229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd8005070 msgr2=0x7f3dd800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3df4a59640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd8005070 0x7f3dd800b990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3df0074390 tx=0x7f3df0099840 comp rx=0 tx=0).stop 2024-09-13T16:48:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3df4a59640 1 -- 172.21.15.39:0/756507229 shutdown_connections 2024-09-13T16:48:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3df4a59640 1 --2- 172.21.15.39:0/756507229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd8005070 0x7f3dd800b990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3df4a59640 1 -- 172.21.15.39:0/756507229 >> 172.21.15.39:0/756507229 conn(0x7f3de804d460 msgr2=0x7f3de804d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3df4a59640 1 -- 172.21.15.39:0/756507229 shutdown_connections 2024-09-13T16:48:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3df4a59640 1 -- 172.21.15.39:0/756507229 wait complete. 2024-09-13T16:48:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad58ff9640 1 -- 172.21.15.39:0/1848972270 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad4801a400 con 0x7fad540409e0 2024-09-13T16:48:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad58ff9640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad400666e0 0x7fad40068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad58ff9640 1 -- 172.21.15.39:0/1848972270 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fad48084a70 con 0x7fad540409e0 2024-09-13T16:48:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad58ff9640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fad40070f80 0x7fad400733a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad58ff9640 1 -- 172.21.15.39:0/1848972270 --> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad40073a50 con 0x7fad40070f80 2024-09-13T16:48:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad58ff9640 1 -- 172.21.15.39:0/1848972270 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad48084f10 con 0x7fad540409e0 2024-09-13T16:48:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad60d86640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fad40070f80 0x7fad400733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad5bfff640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad400666e0 0x7fad40068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f09a5d02640 1 --2- 172.21.15.39:0/3474770524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c5180 0x7f09a0153940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f09a00c2c40 tx=0x7f099801c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0accd0adee35d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9e9effd640 1 -- 172.21.15.39:0/615793667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea0157950 msgr2=0x7f9ea015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9e9effd640 1 --2- 172.21.15.39:0/615793667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea0157950 0x7f9ea015e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9e9effd640 1 -- 172.21.15.39:0/615793667 >> v1:172.21.15.84:6789/0 conn(0x7f9ea00c76c0 legacy=0x7f9ea0157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.336+0000 7f9e9effd640 1 -- 172.21.15.39:0/615793667 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea0153d60 con 0x7f9ea014db70 2024-09-13T16:48:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad60d86640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fad40070f80 0x7fad400733a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7ff3e0c9a640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3c80666c0 0x7ff3c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f0997fff640 1 -- 172.21.15.39:0/3474770524 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980a5910 con 0x7f09a00c5180 2024-09-13T16:48:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f0997fff640 1 -- 172.21.15.39:0/3474770524 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09980a5a70 con 0x7f09a00c5180 2024-09-13T16:48:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f0997fff640 1 -- 172.21.15.39:0/3474770524 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980a5e00 con 0x7f09a00c5180 2024-09-13T16:48:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad5bfff640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad400666e0 0x7fad40068ba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad5c04fb60 tx=0x7fad5c08af90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3df4a59640 1 Processor -- start 2024-09-13T16:48:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f2c4c974640 1 Processor -- start 2024-09-13T16:48:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fad58ff9640 1 -- 172.21.15.39:0/1848972270 <== osd.0 v2:172.21.15.39:6800/3633417446 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fad40073a50 con 0x7fad40070f80 2024-09-13T16:48:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f9e9effd640 1 --2- 172.21.15.39:0/615793667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea014db70 0x7f9ea00c4140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009d90 tx=0x7f9e9801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad27a4a0d6b6b858 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f9e9dffb640 1 -- 172.21.15.39:0/615793667 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980a5970 con 0x7f9ea014db70 2024-09-13T16:48:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f9e9dffb640 1 -- 172.21.15.39:0/615793667 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e980a5ad0 con 0x7f9ea014db70 2024-09-13T16:48:19.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f9e9dffb640 1 -- 172.21.15.39:0/615793667 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980a5e60 con 0x7f9ea014db70 2024-09-13T16:48:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fe132ffd640 1 -- 172.21.15.39:0/3599852275 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023517989 0 0) 0x7fe1340c8980 con 0x7fe1340bcdb0 2024-09-13T16:48:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fe132ffd640 1 -- 172.21.15.39:0/3599852275 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe11c003530 con 0x7fe1340bcdb0 2024-09-13T16:48:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fe132ffd640 1 -- 172.21.15.39:0/3599852275 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879066217 0 0) 0x7fe1340c8c60 con 0x7fe134155a40 2024-09-13T16:48:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7fe132ffd640 1 -- 172.21.15.39:0/3599852275 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1340c8980 con 0x7fe134155a40 2024-09-13T16:48:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f3df4a59640 1 -- start start 2024-09-13T16:48:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3df4a59640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd8005070 0x7f3de8030a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3df4a59640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de8043b10 0x7f3de8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3df4a59640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3de8032710 0x7f3de81763a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3df4a59640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de80da070 con 0x7f3de8043b10 2024-09-13T16:48:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3df4a59640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8030fb0 con 0x7f3de8032710 2024-09-13T16:48:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3df4a59640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8033060 con 0x7f3dd8005070 2024-09-13T16:48:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f09a7765640 1 -- 172.21.15.39:0/3474770524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c5180 msgr2=0x7f09a0153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f09a7765640 1 --2- 172.21.15.39:0/3474770524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c5180 0x7f09a0153940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f09a00c2c40 tx=0x7f099801c870 comp rx=0 tx=0).stop 2024-09-13T16:48:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f2c4c974640 1 -- start start 2024-09-13T16:48:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f2c4c974640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 0x7f2c40045bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f2c4c974640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c40006a40 0x7f2c40006ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f2c4c974640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4000aa10 con 0x7f2c400d3f70 2024-09-13T16:48:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f2c4c974640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4000b5b0 con 0x7f2c40006a40 2024-09-13T16:48:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7f2c4c974640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4004aed0 con 0x7f2c400d68c0 2024-09-13T16:48:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.340+0000 7ff3dbfff640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7ff3c8070ee0 0x7ff3c8073300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7ff3e0c9a640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3c80666c0 0x7ff3c8068b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0005890 tx=0x7ff3d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f09a7765640 1 -- 172.21.15.39:0/3474770524 shutdown_connections 2024-09-13T16:48:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f09a7765640 1 --2- 172.21.15.39:0/3474770524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c5180 0x7f09a0153940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f09a7765640 1 --2- 172.21.15.39:0/3474770524 >> v2:172.21.15.39:3300/0 conn(0x7f09a00c45e0 0x7f09a0153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f09a7765640 1 --2- 172.21.15.39:0/3474770524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 0x7f09a00c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f09a7765640 1 -- 172.21.15.39:0/3474770524 >> 172.21.15.39:0/3474770524 conn(0x7f09a00c5790 msgr2=0x7f09a00c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7fe138e5d640 1 -- 172.21.15.39:0/3599852275 >> v1:172.21.15.84:6789/0 conn(0x7fe134155a40 legacy=0x7fe134158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7fe138e5d640 1 -- 172.21.15.39:0/3599852275 >> v1:172.21.15.39:6789/0 conn(0x7fe1340bcdb0 legacy=0x7fe134069e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c4659c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 0x7f2c40045bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c4759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c40006a40 0x7f2c40006ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 -- 172.21.15.39:0/615793667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea014db70 msgr2=0x7f9ea00c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 --2- 172.21.15.39:0/615793667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea014db70 0x7f9ea00c4140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009d90 tx=0x7f9e9801c8b0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c4659c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 0x7f2c40045bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45392/0 (socket says 172.21.15.39:45392) 2024-09-13T16:48:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c4659c640 1 -- 172.21.15.39:0/1406115747 learned_addr learned my addr 172.21.15.39:0/1406115747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c46d9d640 1 --1- 172.21.15.39:0/1406115747 >> v1:172.21.15.84:6789/0 conn(0x7f2c400d3f70 0x7f2c4000aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:44322/0 (socket says 172.21.15.39:44322) 2024-09-13T16:48:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dee59c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd8005070 0x7f3de8030a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3deed9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3de8032710 0x7f3de81763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dee59c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd8005070 0x7f3de8030a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45394/0 (socket says 172.21.15.39:45394) 2024-09-13T16:48:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dee59c640 1 -- 172.21.15.39:0/3626128361 learned_addr learned my addr 172.21.15.39:0/3626128361 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dedd9b640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de8043b10 0x7f3de8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 -- 172.21.15.39:0/615793667 shutdown_connections 2024-09-13T16:48:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 --2- 172.21.15.39:0/615793667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea0157950 0x7f9ea015e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 --2- 172.21.15.39:0/615793667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea014db70 0x7f9ea00c4140 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 -- 172.21.15.39:0/615793667 >> 172.21.15.39:0/615793667 conn(0x7f9ea0070600 msgr2=0x7f9ea0070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7fe138e5d640 1 -- 172.21.15.39:0/3599852275 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1340c9220 con 0x7fe13415b900 2024-09-13T16:48:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 -- 172.21.15.39:0/615793667 shutdown_connections 2024-09-13T16:48:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dedd9b640 1 -- 172.21.15.39:0/3626128361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd8005070 msgr2=0x7f3de8030a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dedd9b640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd8005070 0x7f3de8030a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dedd9b640 1 -- 172.21.15.39:0/3626128361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3de8032710 msgr2=0x7f3de81763a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dedd9b640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3de8032710 0x7f3de81763a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dedd9b640 1 -- 172.21.15.39:0/3626128361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8033300 con 0x7f3de8043b10 2024-09-13T16:48:19.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c4759e640 1 -- 172.21.15.39:0/1406115747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 msgr2=0x7f2c40045bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c4759e640 1 --2- 172.21.15.39:0/1406115747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 0x7f2c40045bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c4759e640 1 -- 172.21.15.39:0/1406115747 >> v1:172.21.15.84:6789/0 conn(0x7f2c400d3f70 legacy=0x7f2c4000aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c4759e640 1 -- 172.21.15.39:0/1406115747 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40009e00 con 0x7f2c40006a40 2024-09-13T16:48:19.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3deed9d640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3de8032710 0x7f3de81763a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:19.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3dedd9b640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de8043b10 0x7f3de8042080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3de000b410 tx=0x7f3de000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3ddeffd640 1 -- 172.21.15.39:0/3626128361 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0011830 con 0x7f3de8043b10 2024-09-13T16:48:19.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3ddeffd640 1 -- 172.21.15.39:0/3626128361 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de0011990 con 0x7f3de8043b10 2024-09-13T16:48:19.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de8173950 con 0x7f3de8043b10 2024-09-13T16:48:19.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de80413f0 con 0x7f3de8043b10 2024-09-13T16:48:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 -- 172.21.15.39:0/615793667 wait complete. 2024-09-13T16:48:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 Processor -- start 2024-09-13T16:48:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f9ea5426640 1 -- start start 2024-09-13T16:48:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3de800c880 con 0x7f3de8043b10 2024-09-13T16:48:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7ff3d97fa640 1 -- 172.21.15.39:0/4203302459 <== osd.7 v2:172.21.15.84:6824/413977365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff3c80739b0 con 0x7ff3c8070ee0 2024-09-13T16:48:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c4759e640 1 --2- 172.21.15.39:0/1406115747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 0x7f2c40006ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2c380097c0 tx=0x7f2c3801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9b5e8ee44cb32b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.344+0000 7f2c45d9b640 1 -- 172.21.15.39:0/1406115747 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c380ab600 con 0x7f2c40006a40 2024-09-13T16:48:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9ea5426640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea00c76c0 0x7f9ea01ef840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f2c45d9b640 1 -- 172.21.15.39:0/1406115747 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c380a7050 con 0x7f2c40006a40 2024-09-13T16:48:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f2c45d9b640 1 -- 172.21.15.39:0/1406115747 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c380ab980 con 0x7f2c40006a40 2024-09-13T16:48:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9ea5426640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea014db70 0x7f9ea00ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9ea5426640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ea0157950 0x7f9ea00ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9e7fc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea014db70 0x7f9ea00ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ea0157950 0x7f9ea00ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ea0157950 0x7f9ea00ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34430/0 (socket says 172.21.15.39:34430) 2024-09-13T16:48:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea00c76c0 0x7f9ea01ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea00c76c0 0x7f9ea01ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55072/0 (socket says 172.21.15.39:55072) 2024-09-13T16:48:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9e7fc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea014db70 0x7f9ea00ad370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45410/0 (socket says 172.21.15.39:45410) 2024-09-13T16:48:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9f7fe640 1 -- 172.21.15.39:0/2263723961 learned_addr learned my addr 172.21.15.39:0/2263723961 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9ea5426640 1 -- 172.21.15.39:0/2263723961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00addf0 con 0x7f9ea0157950 2024-09-13T16:48:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9ea5426640 1 -- 172.21.15.39:0/2263723961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00ab520 con 0x7f9ea00c76c0 2024-09-13T16:48:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9ea5426640 1 -- 172.21.15.39:0/2263723961 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01f0340 con 0x7f9ea014db70 2024-09-13T16:48:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f2c4c974640 1 -- 172.21.15.39:0/1406115747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 msgr2=0x7f2c40006ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f2c4c974640 1 --2- 172.21.15.39:0/1406115747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 0x7f2c40006ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2c380097c0 tx=0x7f2c3801c450 comp rx=0 tx=0).stop 2024-09-13T16:48:19.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe138e5d640 1 --2- 172.21.15.39:0/3599852275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe13415b900 0x7fe13415bd90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe1240097c0 tx=0x7fe12401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e91f2a08584c07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9f7fe640 1 -- 172.21.15.39:0/2263723961 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea014db70 msgr2=0x7f9ea00ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9f7fe640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9ea014db70 0x7f9ea00ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9f7fe640 1 -- 172.21.15.39:0/2263723961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea00c76c0 msgr2=0x7f9ea01ef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9f7fe640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ea00c76c0 0x7f9ea01ef840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9f7fe640 1 -- 172.21.15.39:0/2263723961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e98009a70 con 0x7f9ea0157950 2024-09-13T16:48:19.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e9f7fe640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ea0157950 0x7f9ea00ad8b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000e7f0 tx=0x7f9e9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f3ddeffd640 1 -- 172.21.15.39:0/3626128361 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0011d40 con 0x7f3de8043b10 2024-09-13T16:48:19.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f3ddeffd640 1 -- 172.21.15.39:0/3626128361 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3de001a400 con 0x7f3de8043b10 2024-09-13T16:48:19.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe132ffd640 1 -- 172.21.15.39:0/3599852275 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1240ac600 con 0x7fe13415b900 2024-09-13T16:48:19.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe132ffd640 1 -- 172.21.15.39:0/3599852275 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1240a8050 con 0x7fe13415b900 2024-09-13T16:48:19.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f2c4c974640 1 -- 172.21.15.39:0/1406115747 shutdown_connections 2024-09-13T16:48:19.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f2c4c974640 1 --2- 172.21.15.39:0/1406115747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 0x7f2c40006ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f2c4c974640 1 --2- 172.21.15.39:0/1406115747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 0x7f2c40045bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f09a7765640 1 -- 172.21.15.39:0/3474770524 shutdown_connections 2024-09-13T16:48:19.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f2c4c974640 1 -- 172.21.15.39:0/1406115747 >> 172.21.15.39:0/1406115747 conn(0x7f2c40047cf0 msgr2=0x7f2c40048100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe132ffd640 1 -- 172.21.15.39:0/3599852275 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1240ac980 con 0x7fe13415b900 2024-09-13T16:48:19.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e877fe640 1 -- 172.21.15.39:0/2263723961 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e90009530 con 0x7f9ea0157950 2024-09-13T16:48:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e877fe640 1 -- 172.21.15.39:0/2263723961 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9000eea0 con 0x7f9ea0157950 2024-09-13T16:48:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e877fe640 1 -- 172.21.15.39:0/2263723961 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9000cb60 con 0x7f9ea0157950 2024-09-13T16:48:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9ea5426640 1 -- 172.21.15.39:0/2263723961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea01f0620 con 0x7f9ea0157950 2024-09-13T16:48:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9ea5426640 1 -- 172.21.15.39:0/2263723961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea00abac0 con 0x7f9ea0157950 2024-09-13T16:48:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f9e857fa640 1 -- 172.21.15.39:0/2263723961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e78000fd0 con 0x7f9ea0157950 2024-09-13T16:48:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe13965e640 1 -- 172.21.15.39:0/3599852275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe13415b900 msgr2=0x7fe13415bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe13965e640 1 --2- 172.21.15.39:0/3599852275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe13415b900 0x7fe13415bd90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe1240097c0 tx=0x7fe12401c380 comp rx=0 tx=0).stop 2024-09-13T16:48:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f44d8d82640 1 Processor -- start 2024-09-13T16:48:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7f44d8d82640 1 -- start start 2024-09-13T16:48:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe13965e640 1 -- 172.21.15.39:0/3599852275 shutdown_connections 2024-09-13T16:48:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe13965e640 1 --2- 172.21.15.39:0/3599852275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe13415b900 0x7fe13415bd90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe13965e640 1 -- 172.21.15.39:0/3599852275 >> 172.21.15.39:0/3599852275 conn(0x7fe1340c9720 msgr2=0x7fe1340c9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe13965e640 1 -- 172.21.15.39:0/3599852275 shutdown_connections 2024-09-13T16:48:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 -- 172.21.15.39:0/1406115747 shutdown_connections 2024-09-13T16:48:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 -- 172.21.15.39:0/1406115747 wait complete. 2024-09-13T16:48:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e877fe640 1 -- 172.21.15.39:0/2263723961 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e9000ccc0 con 0x7f9ea0157950 2024-09-13T16:48:19.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 Processor -- start 2024-09-13T16:48:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a7765640 1 -- 172.21.15.39:0/3474770524 wait complete. 2024-09-13T16:48:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a7765640 1 Processor -- start 2024-09-13T16:48:19.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 -- start start 2024-09-13T16:48:19.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.348+0000 7fe13965e640 1 -- 172.21.15.39:0/3599852275 wait complete. 2024-09-13T16:48:19.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe13965e640 1 Processor -- start 2024-09-13T16:48:19.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e877fe640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e8c066690 0x7f9e8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e877fe640 1 -- 172.21.15.39:0/2263723961 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f9e900845b0 con 0x7f9ea0157950 2024-09-13T16:48:19.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e877fe640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f9e8c070eb0 0x7f9e8c0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e877fe640 1 -- 172.21.15.39:0/2263723961 --> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e8c073980 con 0x7f9e8c070eb0 2024-09-13T16:48:19.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e877fe640 1 -- 172.21.15.39:0/2263723961 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e90086b90 con 0x7f9ea0157950 2024-09-13T16:48:19.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 0x7f2c40171100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c400d3f70 0x7f2c400d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 0x7f2c400c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c400c9440 con 0x7f2c400d3f70 2024-09-13T16:48:19.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c400cdf80 con 0x7f2c40006a40 2024-09-13T16:48:19.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4c974640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4002c660 con 0x7f2c400d68c0 2024-09-13T16:48:19.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f3ddeffd640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3dd0066810 0x7f3dd0068cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e9effd640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e8c066690 0x7f9e8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e9e7fc640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f9e8c070eb0 0x7f9e8c0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a7765640 1 -- start start 2024-09-13T16:48:19.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a7765640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c45e0 0x7f09a01f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a7765640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a00c5180 0x7f09a00be300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a7765640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 0x7f09a01fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a7765640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a01fa8c0 con 0x7f09a00c5180 2024-09-13T16:48:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a7765640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a01fcfe0 con 0x7f09a00c45e0 2024-09-13T16:48:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a7765640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a0153130 con 0x7f09a014d570 2024-09-13T16:48:19.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5501640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c45e0 0x7f09a01f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c46d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 0x7f2c40171100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c46d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 0x7f2c40171100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55084/0 (socket says 172.21.15.39:55084) 2024-09-13T16:48:19.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c46d9d640 1 -- 172.21.15.39:0/3219918960 learned_addr learned my addr 172.21.15.39:0/3219918960 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c4759e640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 0x7f2c400c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe13965e640 1 -- start start 2024-09-13T16:48:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe13965e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340bcdb0 0x7fe1341fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe13965e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe134155a40 0x7fe1341faff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5501640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c45e0 0x7f09a01f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55086/0 (socket says 172.21.15.39:55086) 2024-09-13T16:48:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5501640 1 -- 172.21.15.39:0/2161967589 learned_addr learned my addr 172.21.15.39:0/2161967589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe13965e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe13415b900 0x7fe1341fb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe13965e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1341fdc30 con 0x7fe134155a40 2024-09-13T16:48:19.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe13965e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340c1380 con 0x7fe1340bcdb0 2024-09-13T16:48:19.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe13965e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340c1660 con 0x7fe13415b900 2024-09-13T16:48:19.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5d02640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 0x7f09a01fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe133fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340bcdb0 0x7fe1341fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe133fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340bcdb0 0x7fe1341fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55090/0 (socket says 172.21.15.39:55090) 2024-09-13T16:48:19.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe133fff640 1 -- 172.21.15.39:0/3949399374 learned_addr learned my addr 172.21.15.39:0/3949399374 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f3dee59c640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3dd0066810 0x7f3dd0068cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e9e7fc640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f9e8c070eb0 0x7f9e8c0732d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7fe138e5d640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe13415b900 0x7fe1341fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f9e9effd640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e8c066690 0x7f9e8c068b50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e980af000 tx=0x7f9e980a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5501640 1 -- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 msgr2=0x7f09a01fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5501640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a014d570 0x7f09a01fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5501640 1 -- 172.21.15.39:0/2161967589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a00c5180 msgr2=0x7f09a00be300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5501640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a00c5180 0x7f09a00be300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5501640 1 -- 172.21.15.39:0/2161967589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0998009a70 con 0x7f09a00c45e0 2024-09-13T16:48:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c46d9d640 1 -- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 msgr2=0x7f2c400c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c46d9d640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c400d68c0 0x7f2c400c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c46d9d640 1 -- 172.21.15.39:0/3219918960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c400d3f70 msgr2=0x7f2c400d59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c46d9d640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c400d3f70 0x7f2c400d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f2c46d9d640 1 -- 172.21.15.39:0/3219918960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c38009420 con 0x7f2c40006a40 2024-09-13T16:48:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f44d8d82640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 0x7f44d4154620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f09a5501640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c45e0 0x7f09a01f0660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f099c006f50 tx=0x7f099c00eb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f44d8d82640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d4157170 con 0x7f44d4157bc0 2024-09-13T16:48:19.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f44d8d82640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d4154b60 con 0x7f44d40c5640 2024-09-13T16:48:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.352+0000 7f44d8d82640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40c1700 con 0x7f44d40c48a0 2024-09-13T16:48:19.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f2c46d9d640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 0x7f2c40171100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c4c0 tx=0x7f2c3c00c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f44d1d9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f44d4157bc0 0x7f44d406e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33172/0 (socket says 172.21.15.39:33172) 2024-09-13T16:48:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f44d1d9b640 1 -- 172.21.15.39:0/1026970910 learned_addr learned my addr 172.21.15.39:0/1026970910 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f0995ffb640 1 -- 172.21.15.39:0/2161967589 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c00c890 con 0x7f09a00c45e0 2024-09-13T16:48:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f9e877fe640 1 -- 172.21.15.39:0/2263723961 <== osd.1 v2:172.21.15.39:6808/2114478470 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9e8c073980 con 0x7f9e8c070eb0 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe133fff640 1 -- 172.21.15.39:0/3949399374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe13415b900 msgr2=0x7fe1341fb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe133fff640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe13415b900 0x7fe1341fb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe133fff640 1 -- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe134155a40 msgr2=0x7fe1341faff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe133fff640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe134155a40 0x7fe1341faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe133fff640 1 -- 172.21.15.39:0/3949399374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe124009420 con 0x7fe1340bcdb0 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe133fff640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340bcdb0 0x7fe1341fd350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c730 tx=0x7fe12c00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe130ff9640 1 -- 172.21.15.39:0/3949399374 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c003b80 con 0x7fe1340bcdb0 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1340c1940 con 0x7fe1340bcdb0 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1341fd890 con 0x7fe1340bcdb0 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1340c1200 con 0x7fe1340bcdb0 2024-09-13T16:48:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe130ff9640 1 -- 172.21.15.39:0/3949399374 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c003ce0 con 0x7fe1340bcdb0 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7fe130ff9640 1 -- 172.21.15.39:0/3949399374 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00aa70 con 0x7fe1340bcdb0 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe130ff9640 1 -- 172.21.15.39:0/3949399374 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe12c00abd0 con 0x7fe1340bcdb0 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe130ff9640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 0x7fe120068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe130ff9640 1 -- 172.21.15.39:0/3949399374 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fe12c084d00 con 0x7fe1340bcdb0 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe1337fe640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 0x7fe120068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe130ff9640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7fe120070eb0 0x7fe1200732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe130ff9640 1 -- 172.21.15.39:0/3949399374 --> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe120073980 con 0x7fe120070eb0 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe130ff9640 1 -- 172.21.15.39:0/3949399374 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe12c085070 con 0x7fe1340bcdb0 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe138e5d640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7fe120070eb0 0x7fe1200732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe1337fe640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 0x7fe120068b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe128007fd0 tx=0x7fe128007940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f3ddeffd640 1 -- 172.21.15.39:0/3626128361 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f3de00835e0 con 0x7f3de8043b10 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f3ddeffd640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7f3dd00710b0 0x7f3dd00734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f3ddeffd640 1 -- 172.21.15.39:0/3626128361 --> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3dd0073b80 con 0x7f3dd00710b0 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f3ddeffd640 1 -- 172.21.15.39:0/3626128361 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de0083860 con 0x7f3de8043b10 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f3deed9d640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7f3dd00710b0 0x7f3dd00734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f3deed9d640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7f3dd00710b0 0x7f3dd00734d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f3dee59c640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3dd0066810 0x7f3dd0068cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009c80 tx=0x7f3de4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f3ddeffd640 1 -- 172.21.15.39:0/3626128361 <== osd.3 v2:172.21.15.39:6824/1984006749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3dd0073b80 con 0x7f3dd00710b0 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f44d259c640 1 --2- 172.21.15.39:0/1026970910 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 0x7f44d4154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f44d259c640 1 -- 172.21.15.39:0/1026970910 >> v1:172.21.15.84:6789/0 conn(0x7f44d40c5640 legacy=0x7f44d415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f44d259c640 1 -- 172.21.15.39:0/1026970910 >> v1:172.21.15.39:6789/0 conn(0x7f44d4157bc0 legacy=0x7f44d406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f44d259c640 1 -- 172.21.15.39:0/1026970910 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d40c19e0 con 0x7f44d40c48a0 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f44d259c640 1 --2- 172.21.15.39:0/1026970910 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 0x7f44d4154620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f44c40097f0 tx=0x7f44c401c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba155d40b8ea946 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f44d159a640 1 -- 172.21.15.39:0/1026970910 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c40ab600 con 0x7f44d40c48a0 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f44d159a640 1 -- 172.21.15.39:0/1026970910 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c40a7050 con 0x7f44d40c48a0 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f44d159a640 1 -- 172.21.15.39:0/1026970910 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c40ab9c0 con 0x7f44d40c48a0 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f44d8d82640 1 -- 172.21.15.39:0/1026970910 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 msgr2=0x7f44d4154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f44d8d82640 1 --2- 172.21.15.39:0/1026970910 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 0x7f44d4154620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f44c40097f0 tx=0x7f44c401c430 comp rx=0 tx=0).stop 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a0152b70 con 0x7f09a00c45e0 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a01f0ba0 con 0x7f09a00c45e0 2024-09-13T16:48:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f0995ffb640 1 -- 172.21.15.39:0/2161967589 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c010040 con 0x7f09a00c45e0 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f0995ffb640 1 -- 172.21.15.39:0/2161967589 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c00cbf0 con 0x7f09a00c45e0 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0980000fd0 con 0x7f09a00c45e0 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f0995ffb640 1 -- 172.21.15.39:0/2161967589 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f099c00cd50 con 0x7f09a00c45e0 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f0995ffb640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09840666c0 0x7f0984068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f0995ffb640 1 -- 172.21.15.39:0/2161967589 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f099c084e00 con 0x7f09a00c45e0 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f0995ffb640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7f0984070f60 0x7f0984073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f09a4d00640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09840666c0 0x7f0984068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f0995ffb640 1 -- 172.21.15.39:0/2161967589 --> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0984073a30 con 0x7f0984070f60 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f0995ffb640 1 -- 172.21.15.39:0/2161967589 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f099c0851b0 con 0x7f09a00c45e0 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f09a5d02640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7f0984070f60 0x7f0984073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f09a4d00640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09840666c0 0x7f0984068b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09a01f7710 tx=0x7f0990008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f09a5d02640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7f0984070f60 0x7f0984073380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f0995ffb640 1 -- 172.21.15.39:0/2161967589 <== osd.5 v2:172.21.15.84:6808/2432365689 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0984073a30 con 0x7f0984070f60 2024-09-13T16:48:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f2c2f7fe640 1 -- 172.21.15.39:0/3219918960 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c007480 con 0x7f2c40006a40 2024-09-13T16:48:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f2c2f7fe640 1 -- 172.21.15.39:0/3219918960 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c0075e0 con 0x7f2c40006a40 2024-09-13T16:48:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c400cdbb0 con 0x7f2c40006a40 2024-09-13T16:48:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c40171640 con 0x7f2c40006a40 2024-09-13T16:48:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f2c2f7fe640 1 -- 172.21.15.39:0/3219918960 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c004a10 con 0x7f2c40006a40 2024-09-13T16:48:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.356+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c400cda30 con 0x7f2c40006a40 2024-09-13T16:48:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c2f7fe640 1 -- 172.21.15.39:0/3219918960 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c3c004b70 con 0x7f2c40006a40 2024-09-13T16:48:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c2f7fe640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c34066690 0x7f2c34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c2f7fe640 1 -- 172.21.15.39:0/3219918960 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f2c3c083d60 con 0x7f2c40006a40 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c4659c640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c34066690 0x7f2c34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c2f7fe640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f2c34070e30 0x7f2c34073250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c2f7fe640 1 -- 172.21.15.39:0/3219918960 --> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c34073900 con 0x7f2c34070e30 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c2f7fe640 1 -- 172.21.15.39:0/3219918960 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c3c0841b0 con 0x7f2c40006a40 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c4759e640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f2c34070e30 0x7f2c34073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c4659c640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c34066690 0x7f2c34068b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2c48069930 tx=0x7f2c4808af90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7f2c4759e640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f2c34070e30 0x7f2c34073250 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.364+0000 7f2c2f7fe640 1 -- 172.21.15.39:0/3219918960 <== osd.6 v2:172.21.15.84:6809/2893476104 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2c34073900 con 0x7f2c34070e30 2024-09-13T16:48:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.360+0000 7fe138e5d640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7fe120070eb0 0x7fe1200732d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.364+0000 7fe130ff9640 1 -- 172.21.15.39:0/3949399374 <== osd.2 v2:172.21.15.39:6816/3103155192 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe120073980 con 0x7fe120070eb0 2024-09-13T16:48:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7f44d8d82640 1 -- 172.21.15.39:0/1026970910 shutdown_connections 2024-09-13T16:48:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7f44d8d82640 1 --2- 172.21.15.39:0/1026970910 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 0x7f44d4154620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7f44d8d82640 1 -- 172.21.15.39:0/1026970910 >> 172.21.15.39:0/1026970910 conn(0x7f44d40c7910 msgr2=0x7f44d40c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7f44d8d82640 1 -- 172.21.15.39:0/1026970910 shutdown_connections 2024-09-13T16:48:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7f44d8d82640 1 -- 172.21.15.39:0/1026970910 wait complete. 2024-09-13T16:48:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 --> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff3bc002cc0 con 0x7ff3c8070ee0 2024-09-13T16:48:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7f44d8d82640 1 Processor -- start 2024-09-13T16:48:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7f44d8d82640 1 -- start start 2024-09-13T16:48:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7ff3d97fa640 1 -- 172.21.15.39:0/4203302459 <== osd.7 v2:172.21.15.84:6824/413977365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff3bc002cc0 con 0x7ff3c8070ee0 2024-09-13T16:48:19.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7f44d8d82640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 0x7f44d414cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.368+0000 7f44d8d82640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40c5640 0x7f44d41efef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d8d82640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44d4157bc0 0x7f44d41f0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40c5640 0x7f44d41efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40c5640 0x7f44d41efef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34450/0 (socket says 172.21.15.39:34450) 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d259c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 0x7f44d414cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d259c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 0x7f44d414cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45438/0 (socket says 172.21.15.39:45438) 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d1d9b640 1 -- 172.21.15.39:0/1100815766 learned_addr learned my addr 172.21.15.39:0/1100815766 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d406a040 con 0x7f44d40c5640 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d414d310 con 0x7f44d4157bc0 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d406cdb0 con 0x7f44d40c48a0 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d2d9d640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44d4157bc0 0x7f44d41f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d1d9b640 1 -- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 msgr2=0x7f44d414cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d1d9b640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f44d40c48a0 0x7f44d414cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d1d9b640 1 -- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44d4157bc0 msgr2=0x7f44d41f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d1d9b640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44d4157bc0 0x7f44d41f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d1d9b640 1 -- 172.21.15.39:0/1100815766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c4009420 con 0x7f44d40c5640 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d1d9b640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40c5640 0x7f44d41efef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f44c800c4b0 tx=0x7f44c800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44c2ffd640 1 -- 172.21.15.39:0/1100815766 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c8005cf0 con 0x7f44d40c5640 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44c2ffd640 1 -- 172.21.15.39:0/1100815766 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c8005e50 con 0x7f44d40c5640 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d40c3800 con 0x7f44d40c5640 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.372+0000 7f44c2ffd640 1 -- 172.21.15.39:0/1100815766 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c80072b0 con 0x7f44d40c5640 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d40c3dc0 con 0x7f44d40c5640 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44c2ffd640 1 -- 172.21.15.39:0/1100815766 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44c8007410 con 0x7f44d40c5640 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44c2ffd640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44b4066690 0x7f44b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f44d40c67c0 con 0x7f44d40c5640 2024-09-13T16:48:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44c2ffd640 1 -- 172.21.15.39:0/1100815766 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f44c8054480 con 0x7f44d40c5640 2024-09-13T16:48:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44d259c640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44b4066690 0x7f44b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44c2ffd640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7f44b4070db0 0x7f44b40731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44c2ffd640 1 -- 172.21.15.39:0/1100815766 --> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44b4073880 con 0x7f44b4070db0 2024-09-13T16:48:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 -- 172.21.15.39:0/2263723961 --> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e78002cc0 con 0x7f9e8c070eb0 2024-09-13T16:48:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44c2ffd640 1 -- 172.21.15.39:0/1100815766 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44c80507c0 con 0x7f44d40c5640 2024-09-13T16:48:19.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44d2d9d640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7f44b4070db0 0x7f44b40731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44d259c640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44b4066690 0x7f44b4068b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44c40ad000 tx=0x7f44c40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e877fe640 1 -- 172.21.15.39:0/2263723961 <== osd.1 v2:172.21.15.39:6808/2114478470 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9e78002cc0 con 0x7f9e8c070eb0 2024-09-13T16:48:19.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f44d2d9d640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7f44b4070db0 0x7f44b40731d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 -- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f9e8c070eb0 msgr2=0x7f9e8c0732d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f9e8c070eb0 0x7f9e8c0732d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 -- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e8c066690 msgr2=0x7f9e8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e8c066690 0x7f9e8c068b50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e980af000 tx=0x7f9e980a7070 comp rx=0 tx=0).stop 2024-09-13T16:48:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 -- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ea0157950 msgr2=0x7f9ea00ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 --2- 172.21.15.39:0/2263723961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ea0157950 0x7f9ea00ad8b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000e7f0 tx=0x7f9e9000ecb0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e9f7fe640 1 -- 172.21.15.39:0/2263723961 reap_dead start 2024-09-13T16:48:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 -- 172.21.15.39:0/2263723961 shutdown_connections 2024-09-13T16:48:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 -- 172.21.15.39:0/2263723961 >> 172.21.15.39:0/2263723961 conn(0x7f9ea0070600 msgr2=0x7f9ea0153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 -- 172.21.15.39:0/2263723961 shutdown_connections 2024-09-13T16:48:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.376+0000 7f9e857fa640 1 -- 172.21.15.39:0/2263723961 wait complete. 2024-09-13T16:48:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.380+0000 7f44c2ffd640 1 -- 172.21.15.39:0/1100815766 <== osd.4 v2:172.21.15.84:6800/1588544227 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f44b4073880 con 0x7f44b4070db0 2024-09-13T16:48:19.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 --> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c4002e9d0 con 0x7f2c34070e30 2024-09-13T16:48:19.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c2f7fe640 1 -- 172.21.15.39:0/3219918960 <== osd.6 v2:172.21.15.84:6809/2893476104 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2c380ab680 con 0x7f2c34070e30 2024-09-13T16:48:19.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f2c34070e30 msgr2=0x7f2c34073250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f2c34070e30 0x7f2c34073250 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c34066690 msgr2=0x7f2c34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c34066690 0x7f2c34068b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2c48069930 tx=0x7f2c4808af90 comp rx=0 tx=0).stop 2024-09-13T16:48:19.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 msgr2=0x7f2c40171100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 --2- 172.21.15.39:0/3219918960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c40006a40 0x7f2c40171100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c4c0 tx=0x7f2c3c00c980 comp rx=0 tx=0).stop 2024-09-13T16:48:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4759e640 1 -- 172.21.15.39:0/3219918960 reap_dead start 2024-09-13T16:48:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 shutdown_connections 2024-09-13T16:48:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 >> 172.21.15.39:0/3219918960 conn(0x7f2c40047cf0 msgr2=0x7f2c400cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 shutdown_connections 2024-09-13T16:48:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.384+0000 7f2c4c974640 1 -- 172.21.15.39:0/3219918960 wait complete. 2024-09-13T16:48:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 --> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0980002cf0 con 0x7f0984070f60 2024-09-13T16:48:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 --> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3de8047de0 con 0x7f3dd00710b0 2024-09-13T16:48:19.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f3ddeffd640 1 -- 172.21.15.39:0/3626128361 <== osd.3 v2:172.21.15.39:6824/1984006749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3df004f1b0 con 0x7f3dd00710b0 2024-09-13T16:48:19.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f0995ffb640 1 -- 172.21.15.39:0/2161967589 <== osd.5 v2:172.21.15.84:6808/2432365689 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0980002cf0 con 0x7f0984070f60 2024-09-13T16:48:19.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7f0984070f60 msgr2=0x7f0984073380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7f0984070f60 0x7f0984073380 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09840666c0 msgr2=0x7f0984068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09840666c0 0x7f0984068b80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09a01f7710 tx=0x7f0990008040 comp rx=0 tx=0).stop 2024-09-13T16:48:19.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c45e0 msgr2=0x7f09a01f0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 --2- 172.21.15.39:0/2161967589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a00c45e0 0x7f09a01f0660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f099c006f50 tx=0x7f099c00eb30 comp rx=0 tx=0).stop 2024-09-13T16:48:19.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a5d02640 1 -- 172.21.15.39:0/2161967589 reap_dead start 2024-09-13T16:48:19.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 shutdown_connections 2024-09-13T16:48:19.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 >> 172.21.15.39:0/2161967589 conn(0x7f09a00c5790 msgr2=0x7f09a00bf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7ff3c8070ee0 msgr2=0x7ff3c8073300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7ff3c8070ee0 0x7ff3c8073300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 shutdown_connections 2024-09-13T16:48:19.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7f09a7765640 1 -- 172.21.15.39:0/2161967589 wait complete. 2024-09-13T16:48:19.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3c80666c0 msgr2=0x7ff3c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3c80666c0 0x7ff3c8068b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0005890 tx=0x7ff3d0009040 comp rx=0 tx=0).stop 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3dc1502b0 msgr2=0x7ff3dc151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 --2- 172.21.15.39:0/4203302459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3dc1502b0 0x7ff3dc151b20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00c390 tx=0x7ff3cc00c850 comp rx=0 tx=0).stop 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e149b640 1 -- 172.21.15.39:0/4203302459 reap_dead start 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 shutdown_connections 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 >> 172.21.15.39:0/4203302459 conn(0x7ff3dc066480 msgr2=0x7ff3dc14bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 shutdown_connections 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.388+0000 7ff3e2efe640 1 -- 172.21.15.39:0/4203302459 wait complete. 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.392+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7f3dd00710b0 msgr2=0x7f3dd00734d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.392+0000 7f3df4a59640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7f3dd00710b0 0x7f3dd00734d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.392+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 --> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe13415bd90 con 0x7fe120070eb0 2024-09-13T16:48:19.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.396+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3dd0066810 msgr2=0x7f3dd0068cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.396+0000 7f3df4a59640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3dd0066810 0x7f3dd0068cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009c80 tx=0x7f3de4009190 comp rx=0 tx=0).stop 2024-09-13T16:48:19.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.396+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de8043b10 msgr2=0x7f3de8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.396+0000 7f3df4a59640 1 --2- 172.21.15.39:0/3626128361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3de8043b10 0x7f3de8042080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3de000b410 tx=0x7f3de000b810 comp rx=0 tx=0).stop 2024-09-13T16:48:19.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.396+0000 7f3deed9d640 1 -- 172.21.15.39:0/3626128361 reap_dead start 2024-09-13T16:48:19.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.396+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 shutdown_connections 2024-09-13T16:48:19.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.396+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 >> 172.21.15.39:0/3626128361 conn(0x7f3de804d460 msgr2=0x7f3de802f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.434 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:19.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T16:48:19.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.396+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 --> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad3c002cf0 con 0x7fad40070f80 2024-09-13T16:48:19.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 shutdown_connections 2024-09-13T16:48:19.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7f3df4a59640 1 -- 172.21.15.39:0/3626128361 wait complete. 2024-09-13T16:48:19.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad58ff9640 1 -- 172.21.15.39:0/1848972270 <== osd.0 v2:172.21.15.39:6800/3633417446 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad3c002cf0 con 0x7fad40070f80 2024-09-13T16:48:19.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fad40070f80 msgr2=0x7fad400733a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fad40070f80 0x7fad400733a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad400666e0 msgr2=0x7fad40068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad400666e0 0x7fad40068ba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad5c04fb60 tx=0x7fad5c08af90 comp rx=0 tx=0).stop 2024-09-13T16:48:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fe130ff9640 1 -- 172.21.15.39:0/3949399374 <== osd.2 v2:172.21.15.39:6816/3103155192 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe13415bd90 con 0x7fe120070eb0 2024-09-13T16:48:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 msgr2=0x7fad540434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 --2- 172.21.15.39:0/1848972270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad540409e0 0x7fad540434b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fad4800b3e0 tx=0x7fad4800b8a0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad60d86640 1 -- 172.21.15.39:0/1848972270 reap_dead start 2024-09-13T16:48:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 shutdown_connections 2024-09-13T16:48:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 >> 172.21.15.39:0/1848972270 conn(0x7fad5404d4d0 msgr2=0x7fad540e0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 shutdown_connections 2024-09-13T16:48:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.400+0000 7fad61587640 1 -- 172.21.15.39:0/1848972270 wait complete. 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7fe120070eb0 msgr2=0x7fe1200732d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7fe120070eb0 0x7fe1200732d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 msgr2=0x7fe120068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 0x7fe120068b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe128007fd0 tx=0x7fe128007940 comp rx=0 tx=0).stop 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340bcdb0 msgr2=0x7fe1341fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 --2- 172.21.15.39:0/3949399374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340bcdb0 0x7fe1341fd350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c730 tx=0x7fe12c00cbf0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe138e5d640 1 -- 172.21.15.39:0/3949399374 reap_dead start 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 shutdown_connections 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 >> 172.21.15.39:0/3949399374 conn(0x7fe1340c9720 msgr2=0x7fe1340c6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 shutdown_connections 2024-09-13T16:48:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7fe13965e640 1 -- 172.21.15.39:0/3949399374 wait complete. 2024-09-13T16:48:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 --> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f44d414ee20 con 0x7f44b4070db0 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.404+0000 7f44c2ffd640 1 -- 172.21.15.39:0/1100815766 <== osd.4 v2:172.21.15.84:6800/1588544227 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f44d414ee20 con 0x7f44b4070db0 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7f44b4070db0 msgr2=0x7f44b40731d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7f44b4070db0 0x7f44b40731d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44b4066690 msgr2=0x7f44b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44b4066690 0x7f44b4068b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44c40ad000 tx=0x7f44c40a6000 comp rx=0 tx=0).stop 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40c5640 msgr2=0x7f44d41efef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 --2- 172.21.15.39:0/1100815766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44d40c5640 0x7f44d41efef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f44c800c4b0 tx=0x7f44c800c970 comp rx=0 tx=0).stop 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d2d9d640 1 -- 172.21.15.39:0/1100815766 reap_dead start 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 shutdown_connections 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 >> 172.21.15.39:0/1100815766 conn(0x7f44d40c7910 msgr2=0x7f44d40609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 shutdown_connections 2024-09-13T16:48:19.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.408+0000 7f44d8d82640 1 -- 172.21.15.39:0/1100815766 wait complete. 2024-09-13T16:48:19.442 INFO:teuthology.orchestra.run.smithi039.stdout:55834574856 2024-09-13T16:48:19.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T16:48:19.443 INFO:teuthology.orchestra.run.smithi039.stdout:55834574859 2024-09-13T16:48:19.443 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T16:48:19.444 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:19.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T16:48:19.445 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:19.445 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T16:48:19.446 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:19.446 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T16:48:19.447 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:19.447 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T16:48:19.448 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:19.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T16:48:19.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7efff0997640 1 Processor -- start 2024-09-13T16:48:19.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7efff0997640 1 -- start start 2024-09-13T16:48:19.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7fa3647fb640 1 Processor -- start 2024-09-13T16:48:19.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7fa3647fb640 1 -- start start 2024-09-13T16:48:19.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7f16dd990640 1 Processor -- start 2024-09-13T16:48:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7efff0997640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec06cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7efff0997640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7effec06b180 0x7effec14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7efff0997640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec153c60 con 0x7effec14d710 2024-09-13T16:48:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7efff0997640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effec06d000 con 0x7effec06b180 2024-09-13T16:48:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7efff0997640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec06bcb0 con 0x7effec0c6030 2024-09-13T16:48:19.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f71a5692640 1 Processor -- start 2024-09-13T16:48:19.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.656+0000 7f16dd990640 1 -- start start 2024-09-13T16:48:19.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc342456640 1 Processor -- start 2024-09-13T16:48:19.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dd990640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f16d806c820 0x7f16d806cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc342456640 1 -- start start 2024-09-13T16:48:19.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dd990640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f16d80c3ef0 0x7f16d8158a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dd990640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d8158fb0 con 0x7f16d80c4680 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dd990640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d80c4c40 con 0x7f16d806c820 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dd990640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d80c4f20 con 0x7f16d80c3ef0 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dc98e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f16d806c820 0x7f16d806cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dc98e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f16d806c820 0x7f16d806cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34474/0 (socket says 172.21.15.39:34474) 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dc98e640 1 -- 172.21.15.39:0/3665703474 learned_addr learned my addr 172.21.15.39:0/3665703474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dd18f640 1 --2- 172.21.15.39:0/3665703474 >> v2:172.21.15.84:3300/0 conn(0x7f16d80c3ef0 0x7f16d8158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16d77fe640 1 -- 172.21.15.39:0/3665703474 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033119095 0 0) 0x7f16cc001120 con 0x7f16d80c4680 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16d77fe640 1 -- 172.21.15.39:0/3665703474 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c0003530 con 0x7f16d80c4680 2024-09-13T16:48:19.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f71a5692640 1 -- start start 2024-09-13T16:48:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7effebfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7effec14d710 0x7effec0c1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:44344/0 (socket says 172.21.15.39:44344) 2024-09-13T16:48:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7effebfff640 1 -- 172.21.15.39:0/1287675059 learned_addr learned my addr 172.21.15.39:0/1287675059 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7effeb7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec06cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa3647fb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c155b10 0x7fa35c0c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa3647fb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa35c0c3690 0x7fa35c0c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa3647fb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0c4080 con 0x7fa35c06b1c0 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa3647fb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c06bf30 con 0x7fa35c0c3690 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa3647fb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c155ef0 con 0x7fa35c155b10 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa361d96640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa35c06b1c0 0x7fa35c06ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33178/0 (socket says 172.21.15.39:33178) 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa361d96640 1 -- 172.21.15.39:0/1266053738 learned_addr learned my addr 172.21.15.39:0/1266053738 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa362597640 1 --2- 172.21.15.39:0/1266053738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c155b10 0x7fa35c0c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa362d98640 1 --2- 172.21.15.39:0/1266053738 >> v2:172.21.15.84:3300/0 conn(0x7fa35c0c3690 0x7fa35c0c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa362d98640 1 -- 172.21.15.39:0/1266053738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c155b10 msgr2=0x7fa35c0c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa362d98640 1 --2- 172.21.15.39:0/1266053738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c155b10 0x7fa35c0c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa362d98640 1 -- 172.21.15.39:0/1266053738 >> v1:172.21.15.39:6789/0 conn(0x7fa35c06b1c0 legacy=0x7fa35c06ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fa362d98640 1 -- 172.21.15.39:0/1266053738 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c158c60 con 0x7fa35c0c3690 2024-09-13T16:48:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dc98e640 1 -- 172.21.15.39:0/3665703474 >> v2:172.21.15.84:3300/0 conn(0x7f16d80c3ef0 msgr2=0x7f16d8158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dc98e640 1 --2- 172.21.15.39:0/3665703474 >> v2:172.21.15.84:3300/0 conn(0x7f16d80c3ef0 0x7f16d8158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dc98e640 1 -- 172.21.15.39:0/3665703474 >> v1:172.21.15.39:6789/0 conn(0x7f16d80c4680 legacy=0x7f16d80c17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f16dc98e640 1 -- 172.21.15.39:0/3665703474 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80c5200 con 0x7f16d806c820 2024-09-13T16:48:19.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc342456640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc33c155b10 0x7fc33c06d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc342456640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c159a70 con 0x7fc33c0c0450 2024-09-13T16:48:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc342456640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c158f30 con 0x7fc33c159640 2024-09-13T16:48:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc342456640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33c155ef0 con 0x7fc33c155b10 2024-09-13T16:48:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7effeaffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7effec06b180 0x7effec14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc33bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc33c155b10 0x7fc33c06d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc33bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc33c155b10 0x7fc33c06d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55130/0 (socket says 172.21.15.39:55130) 2024-09-13T16:48:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc33bfff640 1 -- 172.21.15.39:0/1801939250 learned_addr learned my addr 172.21.15.39:0/1801939250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc33b7fe640 1 --1- 172.21.15.39:0/1801939250 >> v1:172.21.15.39:6789/0 conn(0x7fc33c0c0450 0x7fc33c0c1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33208/0 (socket says 172.21.15.39:33208) 2024-09-13T16:48:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7effeaffd640 1 -- 172.21.15.39:0/1287675059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 msgr2=0x7effec06cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7effeaffd640 1 --2- 172.21.15.39:0/1287675059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec06cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7effeaffd640 1 -- 172.21.15.39:0/1287675059 >> v1:172.21.15.84:6789/0 conn(0x7effec14d710 legacy=0x7effec0c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7effeaffd640 1 -- 172.21.15.39:0/1287675059 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec0bfa40 con 0x7effec06b180 2024-09-13T16:48:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fe955f8c640 1 Processor -- start 2024-09-13T16:48:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fe955f8c640 1 -- start start 2024-09-13T16:48:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc33affd640 1 -- 172.21.15.39:0/1801939250 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000770739 0 0) 0x7fc33c159a70 con 0x7fc33c0c0450 2024-09-13T16:48:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc33affd640 1 -- 172.21.15.39:0/1801939250 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc324003530 con 0x7fc33c0c0450 2024-09-13T16:48:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f71a5692640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71a014df20 0x7f71a00c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f71a5692640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a0070a80 0x7f71a006ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f719ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71a014df20 0x7f71a00c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f71a5692640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a006cf80 con 0x7f71a006b360 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f71a5692640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a0150760 con 0x7f71a014df20 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f71a5692640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00c2a00 con 0x7f71a0070a80 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f719f7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a0070a80 0x7f71a006ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f719f7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a0070a80 0x7f71a006ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45468/0 (socket says 172.21.15.39:45468) 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f719f7fe640 1 -- 172.21.15.39:0/3023235187 learned_addr learned my addr 172.21.15.39:0/3023235187 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f719ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71a014df20 0x7f71a00c1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34476/0 (socket says 172.21.15.39:34476) 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f719f7fe640 1 -- 172.21.15.39:0/3023235187 >> v2:172.21.15.39:3300/0 conn(0x7f71a014df20 msgr2=0x7f71a00c1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f719f7fe640 1 --2- 172.21.15.39:0/3023235187 >> v2:172.21.15.39:3300/0 conn(0x7f71a014df20 0x7f71a00c1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f719f7fe640 1 -- 172.21.15.39:0/3023235187 >> v1:172.21.15.84:6789/0 conn(0x7f71a006b360 legacy=0x7f71a0150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7f719f7fe640 1 -- 172.21.15.39:0/3023235187 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00c2ce0 con 0x7f71a0070a80 2024-09-13T16:48:19.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc33bfff640 1 -- 172.21.15.39:0/1801939250 >> v1:172.21.15.84:6789/0 conn(0x7fc33c159640 legacy=0x7fc33c06b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc33bfff640 1 -- 172.21.15.39:0/1801939250 >> v1:172.21.15.39:6789/0 conn(0x7fc33c0c0450 legacy=0x7fc33c0c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.660+0000 7fc33bfff640 1 -- 172.21.15.39:0/1801939250 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c0c1010 con 0x7fc33c155b10 2024-09-13T16:48:19.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe955f8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9500c1890 0x7fe9500c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefc1a6f640 1 Processor -- start 2024-09-13T16:48:19.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe955f8c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe950153320 0x7fe9501537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe955f8c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe950150a70 con 0x7fe950149f50 2024-09-13T16:48:19.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe955f8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95014fc70 con 0x7fe9500c1890 2024-09-13T16:48:19.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe955f8c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95014fe60 con 0x7fe950153320 2024-09-13T16:48:19.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefc1a6f640 1 -- start start 2024-09-13T16:48:19.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe950153320 0x7fe9501537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9500c1890 0x7fe9500c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fa362d98640 1 --2- 172.21.15.39:0/1266053738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0c3690 0x7fa35c0c3b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa354009d90 tx=0x7fa35401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fcb3e20d18c9375 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94ffff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe950153320 0x7fe9501537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55134/0 (socket says 172.21.15.39:55134) 2024-09-13T16:48:19.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94ffff640 1 -- 172.21.15.39:0/3031022453 learned_addr learned my addr 172.21.15.39:0/3031022453 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9500c1890 0x7fe9500c1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34482/0 (socket says 172.21.15.39:34482) 2024-09-13T16:48:19.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16dc98e640 1 --2- 172.21.15.39:0/3665703474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d806c820 0x7f16d806cc00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16d8066db0 tx=0x7f16d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae6dd1ce9e21ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fa361595640 1 -- 172.21.15.39:0/1266053738 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3540a5a00 con 0x7fa35c0c3690 2024-09-13T16:48:19.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fa361595640 1 -- 172.21.15.39:0/1266053738 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3540a5b60 con 0x7fa35c0c3690 2024-09-13T16:48:19.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fa361595640 1 -- 172.21.15.39:0/1266053738 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3540ad440 con 0x7fa35c0c3690 2024-09-13T16:48:19.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16d77fe640 1 -- 172.21.15.39:0/3665703474 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00a57a0 con 0x7f16d806c820 2024-09-13T16:48:19.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16d77fe640 1 -- 172.21.15.39:0/3665703474 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d00a5900 con 0x7f16d806c820 2024-09-13T16:48:19.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16d77fe640 1 -- 172.21.15.39:0/3665703474 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00a5c90 con 0x7f16d806c820 2024-09-13T16:48:19.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefc1a6f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fefb404a2d0 0x7fefb40d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefc1a6f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4008000 0x7fefb4008470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefc1a6f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb40d6c70 con 0x7fefb40484c0 2024-09-13T16:48:19.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefc1a6f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb40d0ec0 con 0x7fefb404a2d0 2024-09-13T16:48:19.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefc1a6f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb403fc00 con 0x7fefb4008000 2024-09-13T16:48:19.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7effeaffd640 1 --2- 172.21.15.39:0/1287675059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 0x7effec14d1d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7effdc0097c0 tx=0x7effdc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b2b885f81739e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239824730 0 0) 0x7fe950150a70 con 0x7fe950149f50 2024-09-13T16:48:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe938003530 con 0x7fe950149f50 2024-09-13T16:48:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3426275094 0 0) 0x7fe938003530 con 0x7fe950149f50 2024-09-13T16:48:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 >> v2:172.21.15.84:3300/0 conn(0x7fe950153320 msgr2=0x7fe9501537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 --2- 172.21.15.39:0/3031022453 >> v2:172.21.15.84:3300/0 conn(0x7fe950153320 0x7fe9501537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 >> v2:172.21.15.39:3300/0 conn(0x7fe9500c1890 msgr2=0x7fe9500c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 --2- 172.21.15.39:0/3031022453 >> v2:172.21.15.39:3300/0 conn(0x7fe9500c1890 0x7fe9500c1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950150140 con 0x7fe950149f50 2024-09-13T16:48:19.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fe944001680 con 0x7fe950149f50 2024-09-13T16:48:19.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 >> v1:172.21.15.84:6789/0 conn(0x7fe950149f50 legacy=0x7fe95006c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9380039b0 0x7fe938003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9380047d0 0x7fe938004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7effea7fc640 1 -- 172.21.15.39:0/1287675059 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc0ab600 con 0x7effec06b180 2024-09-13T16:48:19.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7effea7fc640 1 -- 172.21.15.39:0/1287675059 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc0a7050 con 0x7effec06b180 2024-09-13T16:48:19.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7effea7fc640 1 -- 172.21.15.39:0/1287675059 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc0ab980 con 0x7effec06b180 2024-09-13T16:48:19.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fa3647fb640 1 -- 172.21.15.39:0/1266053738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0c3690 msgr2=0x7fa35c0c3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fa3647fb640 1 --2- 172.21.15.39:0/1266053738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0c3690 0x7fa35c0c3b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa354009d90 tx=0x7fa35401c940 comp rx=0 tx=0).stop 2024-09-13T16:48:19.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe938005190 0x7fe93800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95014fe60 con 0x7fe938005190 2024-09-13T16:48:19.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93800c010 con 0x7fe9380039b0 2024-09-13T16:48:19.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93800c1d0 con 0x7fe9380047d0 2024-09-13T16:48:19.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94f7fe640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9380039b0 0x7fe938003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefbaffd640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fefb40484c0 0x7fefb4046d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:44382/0 (socket says 172.21.15.39:44382) 2024-09-13T16:48:19.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefbaffd640 1 -- 172.21.15.39:0/3778593300 learned_addr learned my addr 172.21.15.39:0/3778593300 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:19.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16dd990640 1 -- 172.21.15.39:0/3665703474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d806c820 msgr2=0x7f16d806cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16dd990640 1 --2- 172.21.15.39:0/3665703474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d806c820 0x7f16d806cc00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16d8066db0 tx=0x7f16d001c9f0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fc33bfff640 1 --2- 172.21.15.39:0/1801939250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc33c155b10 0x7fc33c06d3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009870 tx=0x7fc32c01c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=403c5b9ef455dfcd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fc33affd640 1 -- 172.21.15.39:0/1801939250 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0ac640 con 0x7fc33c155b10 2024-09-13T16:48:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f719f7fe640 1 --2- 172.21.15.39:0/3023235187 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a0070a80 0x7f71a006ae20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7194009870 tx=0x7f719401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28773589b2db830d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94ffff640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe938005190 0x7fe93800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16dd990640 1 -- 172.21.15.39:0/3665703474 shutdown_connections 2024-09-13T16:48:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16dd990640 1 --2- 172.21.15.39:0/3665703474 >> v2:172.21.15.84:3300/0 conn(0x7f16d80c3ef0 0x7f16d8158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16dd990640 1 --2- 172.21.15.39:0/3665703474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d806c820 0x7f16d806cc00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16dd990640 1 -- 172.21.15.39:0/3665703474 >> 172.21.15.39:0/3665703474 conn(0x7f16d80c5510 msgr2=0x7f16d80c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16dd990640 1 -- 172.21.15.39:0/3665703474 shutdown_connections 2024-09-13T16:48:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f719effd640 1 -- 172.21.15.39:0/3023235187 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940ab600 con 0x7f71a0070a80 2024-09-13T16:48:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597835802 0 0) 0x7fefb40d6c70 con 0x7fefb40484c0 2024-09-13T16:48:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefac003530 con 0x7fefb40484c0 2024-09-13T16:48:19.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f719effd640 1 -- 172.21.15.39:0/3023235187 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71940a7050 con 0x7f71a0070a80 2024-09-13T16:48:19.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f719effd640 1 -- 172.21.15.39:0/3023235187 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940ab980 con 0x7f71a0070a80 2024-09-13T16:48:19.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefbbfff640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4008000 0x7fefb4008470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefbb7fe640 1 --2- 172.21.15.39:0/3778593300 >> v2:172.21.15.39:3300/0 conn(0x7fefb404a2d0 0x7fefb40d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 182651643 0 0) 0x7fefac003530 con 0x7fefb40484c0 2024-09-13T16:48:19.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4008000 msgr2=0x7fefb4008470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4008000 0x7fefb4008470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 >> v2:172.21.15.39:3300/0 conn(0x7fefb404a2d0 msgr2=0x7fefb40d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 --2- 172.21.15.39:0/3778593300 >> v2:172.21.15.39:3300/0 conn(0x7fefb404a2d0 0x7fefb40d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4007800 con 0x7fefb40484c0 2024-09-13T16:48:19.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fefb0001660 con 0x7fefb40484c0 2024-09-13T16:48:19.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 >> v1:172.21.15.84:6789/0 conn(0x7fefb40484c0 legacy=0x7fefb4046d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefac003980 0x7fefac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac004390 0x7fefac004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7efff0997640 1 -- 172.21.15.39:0/1287675059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 msgr2=0x7effec14d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7efff0997640 1 --2- 172.21.15.39:0/1287675059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 0x7effec14d1d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7effdc0097c0 tx=0x7effdc01c3c0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7efff0997640 1 -- 172.21.15.39:0/1287675059 shutdown_connections 2024-09-13T16:48:19.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7efff0997640 1 --2- 172.21.15.39:0/1287675059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 0x7effec14d1d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7efff0997640 1 --2- 172.21.15.39:0/1287675059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec06cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7efff0997640 1 -- 172.21.15.39:0/1287675059 >> 172.21.15.39:0/1287675059 conn(0x7effec0c5510 msgr2=0x7effec0c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4008000 0x7fefac005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40d0ec0 con 0x7fefac003980 2024-09-13T16:48:19.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb403fc00 con 0x7fefac004390 2024-09-13T16:48:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac00baf0 con 0x7fefb4008000 2024-09-13T16:48:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7efff0997640 1 -- 172.21.15.39:0/1287675059 shutdown_connections 2024-09-13T16:48:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fc33affd640 1 -- 172.21.15.39:0/1801939250 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c0ac7a0 con 0x7fc33c155b10 2024-09-13T16:48:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fc33affd640 1 -- 172.21.15.39:0/1801939250 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0acb10 con 0x7fc33c155b10 2024-09-13T16:48:19.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fcf129ac640 1 Processor -- start 2024-09-13T16:48:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f71a5692640 1 -- 172.21.15.39:0/3023235187 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a0070a80 msgr2=0x7f71a006ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f71a5692640 1 --2- 172.21.15.39:0/3023235187 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a0070a80 0x7f71a006ae20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7194009870 tx=0x7f719401c440 comp rx=0 tx=0).stop 2024-09-13T16:48:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f71a5692640 1 -- 172.21.15.39:0/3023235187 shutdown_connections 2024-09-13T16:48:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f71a5692640 1 --2- 172.21.15.39:0/3023235187 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a0070a80 0x7f71a006ae20 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7194009870 tx=0x7f719401c440 comp rx=0 tx=0).stop 2024-09-13T16:48:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f71a5692640 1 --2- 172.21.15.39:0/3023235187 >> v2:172.21.15.39:3300/0 conn(0x7f71a014df20 0x7f71a00c1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f71a5692640 1 -- 172.21.15.39:0/3023235187 >> 172.21.15.39:0/3023235187 conn(0x7f71a00c30c0 msgr2=0x7f71a00c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7efff0997640 1 -- 172.21.15.39:0/1287675059 wait complete. 2024-09-13T16:48:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 -- 172.21.15.39:0/3023235187 shutdown_connections 2024-09-13T16:48:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 -- 172.21.15.39:0/3023235187 wait complete. 2024-09-13T16:48:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fc342456640 1 -- 172.21.15.39:0/1801939250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc33c155b10 msgr2=0x7fc33c06d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fc342456640 1 --2- 172.21.15.39:0/1801939250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc33c155b10 0x7fc33c06d3a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009870 tx=0x7fc32c01c4f0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7f16dd990640 1 -- 172.21.15.39:0/3665703474 wait complete. 2024-09-13T16:48:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd990640 1 Processor -- start 2024-09-13T16:48:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fc342456640 1 -- 172.21.15.39:0/1801939250 shutdown_connections 2024-09-13T16:48:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fc342456640 1 --2- 172.21.15.39:0/1801939250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc33c155b10 0x7fc33c06d3a0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009870 tx=0x7fc32c01c4f0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fc342456640 1 -- 172.21.15.39:0/1801939250 >> 172.21.15.39:0/1801939250 conn(0x7fc33c0a7800 msgr2=0x7fc33c0a7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf129ac640 1 -- start start 2024-09-13T16:48:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf129ac640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcf0402bab0 0x7fcf040e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf129ac640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf04049ad0 con 0x7fcf04048310 2024-09-13T16:48:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf129ac640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf040473e0 con 0x7fcf0403f5c0 2024-09-13T16:48:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf129ac640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf040471d0 con 0x7fcf0402bab0 2024-09-13T16:48:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7efff0997640 1 Processor -- start 2024-09-13T16:48:19.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd990640 1 -- start start 2024-09-13T16:48:19.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd990640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16d806c820 0x7f16d8154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd990640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d80c3ef0 0x7f16d81518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd990640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d80c4680 0x7f16d8157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf0b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcf0402bab0 0x7fcf040e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf0b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcf0402bab0 0x7fcf040e1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55158/0 (socket says 172.21.15.39:55158) 2024-09-13T16:48:19.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf0b7fe640 1 -- 172.21.15.39:0/3993504197 learned_addr learned my addr 172.21.15.39:0/3993504197 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf0bfff640 1 --1- 172.21.15.39:0/3993504197 >> v1:172.21.15.84:6789/0 conn(0x7fcf0403f5c0 0x7fcf040de8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:44390/0 (socket says 172.21.15.39:44390) 2024-09-13T16:48:19.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fefbaffd640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac004390 0x7fefac004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fefbbfff640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4008000 0x7fefac005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 Processor -- start 2024-09-13T16:48:19.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 -- start start 2024-09-13T16:48:19.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd990640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d81f20c0 con 0x7f16d80c4680 2024-09-13T16:48:19.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd990640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d81f1a40 con 0x7f16d80c3ef0 2024-09-13T16:48:19.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd990640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8154b50 con 0x7f16d806c820 2024-09-13T16:48:19.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fefbaffd640 1 -- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4008000 msgr2=0x7fefac005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fefbaffd640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4008000 0x7fefac005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fefbaffd640 1 -- 172.21.15.39:0/3778593300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefac003980 msgr2=0x7fefac003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fefbaffd640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefac003980 0x7fefac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94f7fe640 1 -- 172.21.15.39:0/3031022453 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9380047d0 msgr2=0x7fe938004c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T16:48:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94f7fe640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9380047d0 0x7fe938004c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94f7fe640 1 -- 172.21.15.39:0/3031022453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe938005190 msgr2=0x7fe93800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94f7fe640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe938005190 0x7fe93800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fefbaffd640 1 -- 172.21.15.39:0/3778593300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4007800 con 0x7fefac004390 2024-09-13T16:48:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.664+0000 7fe94f7fe640 1 -- 172.21.15.39:0/3031022453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe938003530 con 0x7fe9380039b0 2024-09-13T16:48:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fe94ffff640 1 -- 172.21.15.39:0/3031022453 reap_dead start 2024-09-13T16:48:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16d7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d80c3ef0 0x7f16d81518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd18f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d80c4680 0x7f16d8157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd18f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d80c4680 0x7f16d8157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34534/0 (socket says 172.21.15.39:34534) 2024-09-13T16:48:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd18f640 1 -- 172.21.15.39:0/3265895971 learned_addr learned my addr 172.21.15.39:0/3265895971 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fa3647fb640 1 -- 172.21.15.39:0/1266053738 shutdown_connections 2024-09-13T16:48:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fa3647fb640 1 --2- 172.21.15.39:0/1266053738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c0c3690 0x7fa35c0c3b40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fa3647fb640 1 --2- 172.21.15.39:0/1266053738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c155b10 0x7fa35c0c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fa3647fb640 1 -- 172.21.15.39:0/1266053738 >> 172.21.15.39:0/1266053738 conn(0x7fa35c0c5790 msgr2=0x7fa35c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fc342456640 1 -- 172.21.15.39:0/1801939250 shutdown_connections 2024-09-13T16:48:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a006b360 0x7f71a0152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71a014df20 0x7f71a00aa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a0153920 0x7f71a01f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00a9e70 con 0x7f71a014df20 2024-09-13T16:48:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fa3647fb640 1 -- 172.21.15.39:0/1266053738 shutdown_connections 2024-09-13T16:48:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a014b4c0 con 0x7f71a0153920 2024-09-13T16:48:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a5692640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00aa080 con 0x7f71a006b360 2024-09-13T16:48:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7efff0997640 1 -- start start 2024-09-13T16:48:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7efff0997640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 0x7effec149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7efff0997640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec139db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fe94f7fe640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9380039b0 0x7fe938003dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c110 tx=0x7fe944023700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b9507ca3c134b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7efff0997640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec14d710 0x7effec146ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7efff0997640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec13c680 con 0x7effec06b180 2024-09-13T16:48:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7efff0997640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec148d40 con 0x7effec14d710 2024-09-13T16:48:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7efff0997640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec13a700 con 0x7effec0c6030 2024-09-13T16:48:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeaffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeaffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec139db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45522/0 (socket says 172.21.15.39:45522) 2024-09-13T16:48:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeaffd640 1 -- 172.21.15.39:0/1082597171 learned_addr learned my addr 172.21.15.39:0/1082597171 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effebfff640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec14d710 0x7effec146ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd18f640 1 -- 172.21.15.39:0/3265895971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16d806c820 msgr2=0x7f16d8154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd18f640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16d806c820 0x7f16d8154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd18f640 1 -- 172.21.15.39:0/3265895971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d80c3ef0 msgr2=0x7f16d81518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd18f640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d80c3ef0 0x7f16d81518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd18f640 1 -- 172.21.15.39:0/3265895971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d0009b30 con 0x7f16d80c4680 2024-09-13T16:48:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fc342456640 1 -- 172.21.15.39:0/1801939250 wait complete. 2024-09-13T16:48:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a4e91640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a0153920 0x7f71a01f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a4e91640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a0153920 0x7f71a01f3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55172/0 (socket says 172.21.15.39:55172) 2024-09-13T16:48:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f71a4e91640 1 -- 172.21.15.39:0/451866424 learned_addr learned my addr 172.21.15.39:0/451866424 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f71a4e91640 1 -- 172.21.15.39:0/451866424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a006b360 msgr2=0x7f71a0152f90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T16:48:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f71a4e91640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a006b360 0x7f71a0152f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f71a4e91640 1 -- 172.21.15.39:0/451866424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71a014df20 msgr2=0x7f71a00aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f71a4e91640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71a014df20 0x7f71a00aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f71a4e91640 1 -- 172.21.15.39:0/451866424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71940094d0 con 0x7f71a0153920 2024-09-13T16:48:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeb7fe640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 0x7effec149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeaffd640 1 -- 172.21.15.39:0/1082597171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec14d710 msgr2=0x7effec146ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeaffd640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec14d710 0x7effec146ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeaffd640 1 -- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 msgr2=0x7effec149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeaffd640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 0x7effec149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeaffd640 1 -- 172.21.15.39:0/1082597171 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc009420 con 0x7effec0c6030 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effeb7fe640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 0x7effec149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7effebfff640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec14d710 0x7effec146ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7effeaffd640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec139db0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7effdc019ff0 tx=0x7effdc019160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7effd3fff640 1 -- 172.21.15.39:0/1082597171 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc0b4810 con 0x7effec0c6030 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9440b3600 con 0x7fe9380039b0 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9440af050 con 0x7fe9380039b0 2024-09-13T16:48:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fe94e7fc640 1 -- 172.21.15.39:0/3031022453 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9440b39e0 con 0x7fe9380039b0 2024-09-13T16:48:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fe955f8c640 1 -- 172.21.15.39:0/3031022453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9380039b0 msgr2=0x7fe938003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fe955f8c640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9380039b0 0x7fe938003dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c110 tx=0x7fe944023700 comp rx=0 tx=0).stop 2024-09-13T16:48:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fe955f8c640 1 -- 172.21.15.39:0/3031022453 shutdown_connections 2024-09-13T16:48:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fe955f8c640 1 --2- 172.21.15.39:0/3031022453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9380039b0 0x7fe938003dd0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c110 tx=0x7fe944023700 comp rx=0 tx=0).stop 2024-09-13T16:48:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fe955f8c640 1 -- 172.21.15.39:0/3031022453 >> 172.21.15.39:0/3031022453 conn(0x7fe9500c5540 msgr2=0x7fe9500c5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7f16dd18f640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d80c4680 0x7f16d8157030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f16c800d800 tx=0x7f16c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f16d57fa640 1 -- 172.21.15.39:0/3265895971 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c800b6c0 con 0x7f16d80c4680 2024-09-13T16:48:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f16d57fa640 1 -- 172.21.15.39:0/3265895971 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c800b820 con 0x7f16d80c4680 2024-09-13T16:48:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f16d57fa640 1 -- 172.21.15.39:0/3265895971 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c800bbd0 con 0x7f16d80c4680 2024-09-13T16:48:19.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f71a4e91640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a0153920 0x7f71a01f3b60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f719800c770 tx=0x7f719800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d8154e30 con 0x7f16d80c4680 2024-09-13T16:48:19.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d8151e20 con 0x7f16d80c4680 2024-09-13T16:48:19.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc342456640 1 Processor -- start 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf0b7fe640 1 -- 172.21.15.39:0/3993504197 >> v1:172.21.15.84:6789/0 conn(0x7fcf0403f5c0 legacy=0x7fcf040de8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf0b7fe640 1 -- 172.21.15.39:0/3993504197 >> v1:172.21.15.39:6789/0 conn(0x7fcf04048310 legacy=0x7fcf040e8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fcf0b7fe640 1 -- 172.21.15.39:0/3993504197 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04047df0 con 0x7fcf0402bab0 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fcf0b7fe640 1 --2- 172.21.15.39:0/3993504197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf0402bab0 0x7fcf040e1770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf04047310 tx=0x7fcef801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d99ec0e724a59c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fcf0affd640 1 -- 172.21.15.39:0/3993504197 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80ab600 con 0x7fcf0402bab0 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fcf0affd640 1 -- 172.21.15.39:0/3993504197 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef80a7050 con 0x7fcf0402bab0 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fcf0affd640 1 -- 172.21.15.39:0/3993504197 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80ab980 con 0x7fcf0402bab0 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fcf129ac640 1 -- 172.21.15.39:0/3993504197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf0402bab0 msgr2=0x7fcf040e1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fcf129ac640 1 --2- 172.21.15.39:0/3993504197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf0402bab0 0x7fcf040e1770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf04047310 tx=0x7fcef801c3a0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f719cff9640 1 -- 172.21.15.39:0/451866424 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198003b80 con 0x7f71a0153920 2024-09-13T16:48:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f719cff9640 1 -- 172.21.15.39:0/451866424 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198003ce0 con 0x7f71a0153920 2024-09-13T16:48:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f719cff9640 1 -- 172.21.15.39:0/451866424 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719800aa70 con 0x7f71a0153920 2024-09-13T16:48:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f71a5692640 1 -- 172.21.15.39:0/451866424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a00aa360 con 0x7f71a0153920 2024-09-13T16:48:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f71a5692640 1 -- 172.21.15.39:0/451866424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a01547e0 con 0x7f71a0153920 2024-09-13T16:48:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec148f90 con 0x7effec0c6030 2024-09-13T16:48:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec149270 con 0x7effec0c6030 2024-09-13T16:48:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec0aab30 con 0x7effec0c6030 2024-09-13T16:48:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.668+0000 7fa3647fb640 1 -- 172.21.15.39:0/1266053738 wait complete. 2024-09-13T16:48:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fa3647fb640 1 Processor -- start 2024-09-13T16:48:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fe955f8c640 1 -- 172.21.15.39:0/3031022453 shutdown_connections 2024-09-13T16:48:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fe955f8c640 1 -- 172.21.15.39:0/3031022453 wait complete. 2024-09-13T16:48:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fe955f8c640 1 Processor -- start 2024-09-13T16:48:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fefbaffd640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac004390 0x7fefac004870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefb0001ec0 tx=0x7fefb001cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dece470af5fe9f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb00a9030 con 0x7fefac004390 2024-09-13T16:48:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb00073d0 con 0x7fefac004390 2024-09-13T16:48:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fefba7fc640 1 -- 172.21.15.39:0/3778593300 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0007780 con 0x7fefac004390 2024-09-13T16:48:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d81f2200 con 0x7f16d80c4680 2024-09-13T16:48:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc342456640 1 -- start start 2024-09-13T16:48:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc342456640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc33c0c0450 0x7fc33c1f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc342456640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc33c159640 0x7fc33c0a8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc342456640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33c0ab7d0 0x7fc33c14b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc342456640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c1f09a0 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc342456640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c1f1950 con 0x7fc33c159640 2024-09-13T16:48:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc342456640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c1f0b90 con 0x7fc33c0c0450 2024-09-13T16:48:19.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc3409f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33c0ab7d0 0x7fc33c14b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc3409f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33c0ab7d0 0x7fc33c14b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34564/0 (socket says 172.21.15.39:34564) 2024-09-13T16:48:19.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc3409f3640 1 -- 172.21.15.39:0/130815863 learned_addr learned my addr 172.21.15.39:0/130815863 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc33b7fe640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc33c159640 0x7fc33c0a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fcf129ac640 1 -- 172.21.15.39:0/3993504197 shutdown_connections 2024-09-13T16:48:19.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fcf129ac640 1 --2- 172.21.15.39:0/3993504197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf0402bab0 0x7fcf040e1770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fcf129ac640 1 -- 172.21.15.39:0/3993504197 >> 172.21.15.39:0/3993504197 conn(0x7fcf04048e10 msgr2=0x7fcf04049220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f71a5692640 1 -- 172.21.15.39:0/451866424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a01549d0 con 0x7f71a0153920 2024-09-13T16:48:19.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7effd3fff640 1 -- 172.21.15.39:0/1082597171 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc0b4970 con 0x7effec0c6030 2024-09-13T16:48:19.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7effd3fff640 1 -- 172.21.15.39:0/1082597171 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc0b4d20 con 0x7effec0c6030 2024-09-13T16:48:19.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fefc1a6f640 1 -- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac004390 msgr2=0x7fefac004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fefc1a6f640 1 --2- 172.21.15.39:0/3778593300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefac004390 0x7fefac004870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefb0001ec0 tx=0x7fefb001cce0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fefbbfff640 1 -- 172.21.15.39:0/3778593300 reap_dead start 2024-09-13T16:48:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f16d57fa640 1 -- 172.21.15.39:0/3265895971 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f16c800bd30 con 0x7f16d80c4680 2024-09-13T16:48:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f16d57fa640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16c4066690 0x7f16c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7f16d57fa640 1 -- 172.21.15.39:0/3265895971 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f16c8084890 con 0x7f16d80c4680 2024-09-13T16:48:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc3409f3640 1 -- 172.21.15.39:0/130815863 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc33c0c0450 msgr2=0x7fc33c1f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc3409f3640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc33c0c0450 0x7fc33c1f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc3409f3640 1 -- 172.21.15.39:0/130815863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc33c159640 msgr2=0x7fc33c0a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc3409f3640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc33c159640 0x7fc33c0a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc3409f3640 1 -- 172.21.15.39:0/130815863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0094d0 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fc3409f3640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33c0ab7d0 0x7fc33c14b2a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc33400c760 tx=0x7fc33400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fe955f8c640 1 -- start start 2024-09-13T16:48:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe955f8c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9500c4510 0x7fe950141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe955f8c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9501f6880 0x7fe950148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe955f8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe950141b80 0x7fe9501489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe955f8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501f3810 con 0x7fe950141b80 2024-09-13T16:48:19.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe955f8c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501f5b20 con 0x7fe9501f6880 2024-09-13T16:48:19.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe955f8c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501f9230 con 0x7fe9500c4510 2024-09-13T16:48:19.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe950141b80 0x7fe9501489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9501f6880 0x7fe950148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe950141b80 0x7fe9501489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34576/0 (socket says 172.21.15.39:34576) 2024-09-13T16:48:19.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94ffff640 1 -- 172.21.15.39:0/3924650844 learned_addr learned my addr 172.21.15.39:0/3924650844 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.672+0000 7fa3647fb640 1 -- start start 2024-09-13T16:48:19.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa3647fb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c06b1c0 0x7fa35c1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7f719cff9640 1 -- 172.21.15.39:0/451866424 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f719800abd0 con 0x7f71a0153920 2024-09-13T16:48:19.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94f7fe640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9500c4510 0x7fe950141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7f719cff9640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666e0 0x7f718c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94effd640 1 -- 172.21.15.39:0/3924650844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9500c4510 msgr2=0x7fe950141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94effd640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9500c4510 0x7fe950141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94effd640 1 -- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe950141b80 msgr2=0x7fe9501489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94effd640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe950141b80 0x7fe9501489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94effd640 1 -- 172.21.15.39:0/3924650844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9501f5da0 con 0x7fe9501f6880 2024-09-13T16:48:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7f719cff9640 1 -- 172.21.15.39:0/451866424 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f7198084d00 con 0x7f71a0153920 2024-09-13T16:48:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94ffff640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe950141b80 0x7fe9501489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fe94effd640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9501f6880 0x7fe950148480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe94000d4d0 tx=0x7fe94000d990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa3647fb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c0c3690 0x7fa35c15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa3647fb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35c155b10 0x7fa35c070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa3647fb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c159520 con 0x7fa35c155b10 2024-09-13T16:48:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa3647fb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1f95d0 con 0x7fa35c06b1c0 2024-09-13T16:48:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa3647fb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1f98b0 con 0x7fa35c0c3690 2024-09-13T16:48:19.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362597640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c06b1c0 0x7fa35c1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362d98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35c155b10 0x7fa35c070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362597640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c06b1c0 0x7fa35c1544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55190/0 (socket says 172.21.15.39:55190) 2024-09-13T16:48:19.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362597640 1 -- 172.21.15.39:0/4030489776 learned_addr learned my addr 172.21.15.39:0/4030489776 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa361d96640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c0c3690 0x7fa35c15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fefc1a6f640 1 -- 172.21.15.39:0/3778593300 shutdown_connections 2024-09-13T16:48:19.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fefc1a6f640 1 -- 172.21.15.39:0/3778593300 >> 172.21.15.39:0/3778593300 conn(0x7fefb4048e50 msgr2=0x7fefb4049260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:19.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7f16dc98e640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16c4066690 0x7f16c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362d98640 1 -- 172.21.15.39:0/4030489776 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c0c3690 msgr2=0x7fa35c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362d98640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c0c3690 0x7fa35c15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362d98640 1 -- 172.21.15.39:0/4030489776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c06b1c0 msgr2=0x7fa35c1544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362d98640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c06b1c0 0x7fa35c1544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362d98640 1 -- 172.21.15.39:0/4030489776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354009a70 con 0x7fa35c155b10 2024-09-13T16:48:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fefc1a6f640 1 -- 172.21.15.39:0/3778593300 shutdown_connections 2024-09-13T16:48:19.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fefc1a6f640 1 -- 172.21.15.39:0/3778593300 wait complete. 2024-09-13T16:48:19.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fa362d98640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35c155b10 0x7fa35c070b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3540ac000 tx=0x7fa3540196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7f16dc98e640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16c4066690 0x7f16c4068b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16d00ae000 tx=0x7f16d0004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7effd3fff640 1 -- 172.21.15.39:0/1082597171 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7effdc0b2040 con 0x7effec0c6030 2024-09-13T16:48:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fcf129ac640 1 -- 172.21.15.39:0/3993504197 shutdown_connections 2024-09-13T16:48:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fcf129ac640 1 -- 172.21.15.39:0/3993504197 wait complete. 2024-09-13T16:48:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fefc1a6f640 1 Processor -- start 2024-09-13T16:48:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fefc1a6f640 1 -- start start 2024-09-13T16:48:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7effd3fff640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd8066690 0x7effd8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7effd3fff640 1 -- 172.21.15.39:0/1082597171 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7effdc11f0e0 con 0x7effec0c6030 2024-09-13T16:48:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.676+0000 7fc338ff9640 1 -- 172.21.15.39:0/130815863 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334003b80 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33c1f0e70 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7effeb7fe640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd8066690 0x7effd8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7f719ffff640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666e0 0x7f718c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefc1a6f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4179fa0 0x7fefb417a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefc1a6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefb40d1000 0x7fefb40d1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7effeb7fe640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd8066690 0x7effd8068b50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7effe0009bb0 tx=0x7effe0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefc1a6f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4042980 0x7fefb4040f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefc1a6f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb417ac80 con 0x7fefb4042980 2024-09-13T16:48:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefc1a6f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb4041b30 con 0x7fefb40d1000 2024-09-13T16:48:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefc1a6f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb417a9a0 con 0x7fefb4179fa0 2024-09-13T16:48:19.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7f719ffff640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666e0 0x7f718c068ba0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f71900099f0 tx=0x7f7190009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4042980 0x7fefb4040f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4042980 0x7fefb4040f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34590/0 (socket says 172.21.15.39:34590) 2024-09-13T16:48:19.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbbfff640 1 -- 172.21.15.39:0/1816022272 learned_addr learned my addr 172.21.15.39:0/1816022272 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33c1f1430 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33c1f1640 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf129ac640 1 Processor -- start 2024-09-13T16:48:19.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf129ac640 1 -- start start 2024-09-13T16:48:19.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf129ac640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf0402bab0 0x7fcf040d7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf129ac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf0403f5c0 0x7fcf040308a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf129ac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf04048310 0x7fcf04030de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf129ac640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf041770a0 con 0x7fcf04048310 2024-09-13T16:48:19.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf129ac640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0402f1e0 con 0x7fcf0403f5c0 2024-09-13T16:48:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf129ac640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf04177ec0 con 0x7fcf0402bab0 2024-09-13T16:48:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf10f49640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf04048310 0x7fcf04030de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf10f49640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf04048310 0x7fcf04030de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34596/0 (socket says 172.21.15.39:34596) 2024-09-13T16:48:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf10f49640 1 -- 172.21.15.39:0/2469056229 learned_addr learned my addr 172.21.15.39:0/2469056229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf0bfff640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf0402bab0 0x7fcf040d7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf0b7fe640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf0403f5c0 0x7fcf040308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbb7fe640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4179fa0 0x7fefb417a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbbfff640 1 -- 172.21.15.39:0/1816022272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4179fa0 msgr2=0x7fefb417a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbbfff640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4179fa0 0x7fefb417a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbbfff640 1 -- 172.21.15.39:0/1816022272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefb40d1000 msgr2=0x7fefb40d1920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T16:48:19.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbbfff640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefb40d1000 0x7fefb40d1920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbbfff640 1 -- 172.21.15.39:0/1816022272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4040a40 con 0x7fefb4042980 2024-09-13T16:48:19.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefbbfff640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4042980 0x7fefb4040f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefbc079d30 tx=0x7fefbc068670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf10f49640 1 -- 172.21.15.39:0/2469056229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf0402bab0 msgr2=0x7fcf040d7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf10f49640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf0402bab0 0x7fcf040d7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf10f49640 1 -- 172.21.15.39:0/2469056229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf0403f5c0 msgr2=0x7fcf040308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:19.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf10f49640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf0403f5c0 0x7fcf040308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:19.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf10f49640 1 -- 172.21.15.39:0/2469056229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef80094d0 con 0x7fcf04048310 2024-09-13T16:48:19.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fcf10f49640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf04048310 0x7fcf04030de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c067df0 tx=0x7fcf0c068670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fa34effd640 1 -- 172.21.15.39:0/4030489776 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3540addb0 con 0x7fa35c155b10 2024-09-13T16:48:19.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fc338ff9640 1 -- 172.21.15.39:0/130815863 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33400ce70 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fc338ff9640 1 -- 172.21.15.39:0/130815863 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334006b60 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.680+0000 7fefa7fff640 1 -- 172.21.15.39:0/1816022272 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc08f070 con 0x7fefb4042980 2024-09-13T16:48:19.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fa3647fb640 1 -- 172.21.15.39:0/4030489776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c15a250 con 0x7fa35c155b10 2024-09-13T16:48:19.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7f16d57fa640 1 -- 172.21.15.39:0/3265895971 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f16d81f2200 con 0x7f16d80c4680 2024-09-13T16:48:19.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fa3647fb640 1 -- 172.21.15.39:0/4030489776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c15a810 con 0x7fa35c155b10 2024-09-13T16:48:19.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fcf08ff9640 1 -- 172.21.15.39:0/2469056229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c072470 con 0x7fcf04048310 2024-09-13T16:48:19.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf041772b0 con 0x7fcf04048310 2024-09-13T16:48:19.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf04177870 con 0x7fcf04048310 2024-09-13T16:48:19.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb4042670 con 0x7fefb4042980 2024-09-13T16:48:19.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb4041670 con 0x7fefb4042980 2024-09-13T16:48:19.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb40d0980 con 0x7fefb4042980 2024-09-13T16:48:19.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fa3647fb640 1 -- 172.21.15.39:0/4030489776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c0674c0 con 0x7fa35c155b10 2024-09-13T16:48:19.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fa34effd640 1 -- 172.21.15.39:0/4030489776 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa354009450 con 0x7fa35c155b10 2024-09-13T16:48:19.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fa34effd640 1 -- 172.21.15.39:0/4030489776 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3540cb6a0 con 0x7fa35c155b10 2024-09-13T16:48:19.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf040e1770 con 0x7fcf04048310 2024-09-13T16:48:19.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fcf08ff9640 1 -- 172.21.15.39:0/2469056229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c0725d0 con 0x7fcf04048310 2024-09-13T16:48:19.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fcf08ff9640 1 -- 172.21.15.39:0/2469056229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c060ba0 con 0x7fcf04048310 2024-09-13T16:48:19.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fefa7fff640 1 -- 172.21.15.39:0/1816022272 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc04f1b0 con 0x7fefb4042980 2024-09-13T16:48:19.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fefa7fff640 1 -- 172.21.15.39:0/1816022272 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc060ba0 con 0x7fefb4042980 2024-09-13T16:48:19.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fc338ff9640 1 -- 172.21.15.39:0/130815863 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc33400a7f0 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fc338ff9640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc328066690 0x7fc328068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fc338ff9640 1 -- 172.21.15.39:0/130815863 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fc334084aa0 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.684+0000 7fc33bfff640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc328066690 0x7fc328068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fc33bfff640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc328066690 0x7fc328068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0ae000 tx=0x7fc32c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fefa7fff640 1 -- 172.21.15.39:0/1816022272 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fefbc060d00 con 0x7fefb4042980 2024-09-13T16:48:19.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fefa7fff640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fefa0066690 0x7fefa0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fefbb7fe640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fefa0066690 0x7fefa0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fa34effd640 1 -- 172.21.15.39:0/4030489776 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3540b4050 con 0x7fa35c155b10 2024-09-13T16:48:19.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fa34effd640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa340066690 0x7fa340068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fcf08ff9640 1 -- 172.21.15.39:0/2469056229 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf0c06f690 con 0x7fcf04048310 2024-09-13T16:48:19.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fcf08ff9640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcef00666c0 0x7fcef0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fcf0bfff640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcef00666c0 0x7fcef0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fefbb7fe640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fefa0066690 0x7fefa0068b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefa80047c0 tx=0x7fefa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fcf0bfff640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcef00666c0 0x7fcef0068b80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf00000f70 tx=0x7fcf00007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fa362597640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa340066690 0x7fa340068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fa362597640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa340066690 0x7fa340068b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa350009b80 tx=0x7fa350009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fe937fff640 1 -- 172.21.15.39:0/3924650844 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940014620 con 0x7fe9501f6880 2024-09-13T16:48:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7f719cff9640 1 -- 172.21.15.39:0/451866424 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7198051eb0 con 0x7f71a0153920 2024-09-13T16:48:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9501f6080 con 0x7fe9501f6880 2024-09-13T16:48:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9500c48f0 con 0x7fe9501f6880 2024-09-13T16:48:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fe937fff640 1 -- 172.21.15.39:0/3924650844 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9400147a0 con 0x7fe9501f6880 2024-09-13T16:48:19.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.688+0000 7fe937fff640 1 -- 172.21.15.39:0/3924650844 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940014b50 con 0x7fe9501f6880 2024-09-13T16:48:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7effd3fff640 1 -- 172.21.15.39:0/1082597171 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7effdc0ec110 con 0x7effec0c6030 2024-09-13T16:48:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe930005260 con 0x7fe9501f6880 2024-09-13T16:48:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fe937fff640 1 -- 172.21.15.39:0/3924650844 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe940014cb0 con 0x7fe9501f6880 2024-09-13T16:48:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fe937fff640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe93c066690 0x7fe93c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fe937fff640 1 -- 172.21.15.39:0/3924650844 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fe940084e80 con 0x7fe9501f6880 2024-09-13T16:48:19.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fe94f7fe640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe93c066690 0x7fe93c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:19.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fe94f7fe640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe93c066690 0x7fe93c068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9440b5000 tx=0x7fe944005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:19.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fa34effd640 1 -- 172.21.15.39:0/4030489776 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fa3540f0700 con 0x7fa35c155b10 2024-09-13T16:48:19.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fc338ff9640 1 -- 172.21.15.39:0/130815863 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc334051a40 con 0x7fc33c0ab7d0 2024-09-13T16:48:19.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fefa7fff640 1 -- 172.21.15.39:0/1816022272 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fefbc0cf4e0 con 0x7fefb4042980 2024-09-13T16:48:19.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.692+0000 7fcf08ff9640 1 -- 172.21.15.39:0/2469056229 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fcf0c0cf830 con 0x7fcf04048310 2024-09-13T16:48:19.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.704+0000 7fa34effd640 1 -- 172.21.15.39:0/4030489776 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa3540eca40 con 0x7fa35c155b10 2024-09-13T16:48:19.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.712+0000 7fe937fff640 1 -- 172.21.15.39:0/3924650844 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe940051f30 con 0x7fe9501f6880 2024-09-13T16:48:19.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.712+0000 7fefa7fff640 1 -- 172.21.15.39:0/1816022272 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fefbc0cb120 con 0x7fefb4042980 2024-09-13T16:48:19.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:19.720+0000 7fcf08ff9640 1 -- 172.21.15.39:0/2469056229 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf0c0cb470 con 0x7fcf04048310 2024-09-13T16:48:20.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.004+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc33c06d3a0 con 0x7fc33c0ab7d0 2024-09-13T16:48:20.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.004+0000 7fc338ff9640 1 -- 172.21.15.39:0/130815863 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc334055060 con 0x7fc33c0ab7d0 2024-09-13T16:48:20.011 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.004+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f16d81f2200 con 0x7f16d80c4680 2024-09-13T16:48:20.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.004+0000 7f16d57fa640 1 -- 172.21.15.39:0/3265895971 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16d81f2200 con 0x7f16d80c4680 2024-09-13T16:48:20.011 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc328066690 msgr2=0x7fc328068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc328066690 0x7fc328068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0ae000 tx=0x7fc32c004740 comp rx=0 tx=0).stop 2024-09-13T16:48:20.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33c0ab7d0 msgr2=0x7fc33c14b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33c0ab7d0 0x7fc33c14b2a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc33400c760 tx=0x7fc33400cc20 comp rx=0 tx=0).stop 2024-09-13T16:48:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 shutdown_connections 2024-09-13T16:48:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc328066690 0x7fc328068b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc33c159640 0x7fc33c0a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33c0ab7d0 0x7fc33c14b2a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 --2- 172.21.15.39:0/130815863 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc33c0c0450 0x7fc33c1f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 >> 172.21.15.39:0/130815863 conn(0x7fc33c0a7800 msgr2=0x7fc33c06f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 shutdown_connections 2024-09-13T16:48:20.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.008+0000 7fc342456640 1 -- 172.21.15.39:0/130815863 wait complete. 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16c4066690 msgr2=0x7f16c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16c4066690 0x7f16c4068b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f16d00ae000 tx=0x7f16d0004ac0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d80c4680 msgr2=0x7f16d8157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d80c4680 0x7f16d8157030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f16c800d800 tx=0x7f16c800dcc0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 shutdown_connections 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16c4066690 0x7f16c4068b50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16d80c3ef0 0x7f16d81518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d80c4680 0x7f16d8157030 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 --2- 172.21.15.39:0/3265895971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16d806c820 0x7f16d8154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 >> 172.21.15.39:0/3265895971 conn(0x7f16d80c5510 msgr2=0x7f16d80c58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 shutdown_connections 2024-09-13T16:48:20.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.012+0000 7f16dd990640 1 -- 172.21.15.39:0/3265895971 wait complete. 2024-09-13T16:48:20.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.016+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7effec13c7f0 con 0x7effec0c6030 2024-09-13T16:48:20.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.016+0000 7effd3fff640 1 -- 172.21.15.39:0/1082597171 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7effec13c7f0 con 0x7effec0c6030 2024-09-13T16:48:20.021 INFO:teuthology.orchestra.run.smithi039.stdout:55834574859 2024-09-13T16:48:20.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.016+0000 7fa3647fb640 1 -- 172.21.15.39:0/4030489776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa35c0708a0 con 0x7fa35c155b10 2024-09-13T16:48:20.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.016+0000 7fa34effd640 1 -- 172.21.15.39:0/4030489776 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa35c0708a0 con 0x7fa35c155b10 2024-09-13T16:48:20.022 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:20.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7f71a5692640 1 -- 172.21.15.39:0/451866424 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f71a006ae20 con 0x7f71a0153920 2024-09-13T16:48:20.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd8066690 msgr2=0x7effd8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd8066690 0x7effd8068b50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7effe0009bb0 tx=0x7effe0008040 comp rx=0 tx=0).stop 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 msgr2=0x7effec139db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec139db0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7effdc019ff0 tx=0x7effdc019160 comp rx=0 tx=0).stop 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 shutdown_connections 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd8066690 0x7effd8068b50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec14d710 0x7effec146ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c6030 0x7effec139db0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 --2- 172.21.15.39:0/1082597171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec06b180 0x7effec149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 >> 172.21.15.39:0/1082597171 conn(0x7effec0c5510 msgr2=0x7effec0c3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 shutdown_connections 2024-09-13T16:48:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7efff0997640 1 -- 172.21.15.39:0/1082597171 wait complete. 2024-09-13T16:48:20.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.020+0000 7f719cff9640 1 -- 172.21.15.39:0/451866424 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71980554d0 con 0x7f71a0153920 2024-09-13T16:48:20.027 INFO:teuthology.orchestra.run.smithi039.stdout:55834574856 2024-09-13T16:48:20.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 -- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa340066690 msgr2=0x7fa340068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa340066690 0x7fa340068b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa350009b80 tx=0x7fa350009190 comp rx=0 tx=0).stop 2024-09-13T16:48:20.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 -- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35c155b10 msgr2=0x7fa35c070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35c155b10 0x7fa35c070b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa3540ac000 tx=0x7fa3540196e0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 -- 172.21.15.39:0/451866424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666e0 msgr2=0x7f718c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666e0 0x7f718c068ba0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f71900099f0 tx=0x7f7190009190 comp rx=0 tx=0).stop 2024-09-13T16:48:20.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 -- 172.21.15.39:0/451866424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a0153920 msgr2=0x7f71a01f3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a0153920 0x7f71a01f3b60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f719800c770 tx=0x7f719800cc30 comp rx=0 tx=0).stop 2024-09-13T16:48:20.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 -- 172.21.15.39:0/4030489776 shutdown_connections 2024-09-13T16:48:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa340066690 0x7fa340068b50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa35c155b10 0x7fa35c070b80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa35c0c3690 0x7fa35c15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 --2- 172.21.15.39:0/4030489776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa35c06b1c0 0x7fa35c1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 -- 172.21.15.39:0/4030489776 >> 172.21.15.39:0/4030489776 conn(0x7fa35c0c5790 msgr2=0x7fa35c0c18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 -- 172.21.15.39:0/451866424 shutdown_connections 2024-09-13T16:48:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f718c0666e0 0x7f718c068ba0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71a0153920 0x7f71a01f3b60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71a014df20 0x7f71a00aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 --2- 172.21.15.39:0/451866424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71a006b360 0x7f71a0152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 -- 172.21.15.39:0/451866424 >> 172.21.15.39:0/451866424 conn(0x7f71a00c30c0 msgr2=0x7f71a0150bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 -- 172.21.15.39:0/4030489776 shutdown_connections 2024-09-13T16:48:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7fa34cff9640 1 -- 172.21.15.39:0/4030489776 wait complete. 2024-09-13T16:48:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 -- 172.21.15.39:0/451866424 shutdown_connections 2024-09-13T16:48:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.024+0000 7f71867fc640 1 -- 172.21.15.39:0/451866424 wait complete. 2024-09-13T16:48:20.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-09-13T16:48:20.038 DEBUG:teuthology.parallel:result is None 2024-09-13T16:48:20.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2024-09-13T16:48:20.049 DEBUG:teuthology.parallel:result is None 2024-09-13T16:48:20.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.044+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fefb404af60 con 0x7fefb4042980 2024-09-13T16:48:20.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.044+0000 7fefa7fff640 1 -- 172.21.15.39:0/1816022272 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fefbc099090 con 0x7fefb4042980 2024-09-13T16:48:20.050 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:20.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe9300057e0 con 0x7fe9501f6880 2024-09-13T16:48:20.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fefa0066690 msgr2=0x7fefa0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fefa0066690 0x7fefa0068b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefa80047c0 tx=0x7fefa8008040 comp rx=0 tx=0).stop 2024-09-13T16:48:20.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4042980 msgr2=0x7fefb4040f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4042980 0x7fefb4040f60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefbc079d30 tx=0x7fefbc068670 comp rx=0 tx=0).stop 2024-09-13T16:48:20.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fe937fff640 1 -- 172.21.15.39:0/3924650844 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe940018020 con 0x7fe9501f6880 2024-09-13T16:48:20.053 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 shutdown_connections 2024-09-13T16:48:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fefa0066690 0x7fefa0068b50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefb40d1000 0x7fefb40d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4042980 0x7fefb4040f60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 --2- 172.21.15.39:0/1816022272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fefb4179fa0 0x7fefb417a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 >> 172.21.15.39:0/1816022272 conn(0x7fefb4048e50 msgr2=0x7fefb40d52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 shutdown_connections 2024-09-13T16:48:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fefc1a6f640 1 -- 172.21.15.39:0/1816022272 wait complete. 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe93c066690 msgr2=0x7fe93c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fe955f8c640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe93c066690 0x7fe93c068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9440b5000 tx=0x7fe944005040 comp rx=0 tx=0).stop 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9501f6880 msgr2=0x7fe950148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.048+0000 7fe955f8c640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9501f6880 0x7fe950148480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe94000d4d0 tx=0x7fe94000d990 comp rx=0 tx=0).stop 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 shutdown_connections 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fe955f8c640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe93c066690 0x7fe93c068b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fe955f8c640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9501f6880 0x7fe950148480 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fe955f8c640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe950141b80 0x7fe9501489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fe955f8c640 1 --2- 172.21.15.39:0/3924650844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9500c4510 0x7fe950141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 >> 172.21.15.39:0/3924650844 conn(0x7fe9500c5540 msgr2=0x7fe9500a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 shutdown_connections 2024-09-13T16:48:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fe955f8c640 1 -- 172.21.15.39:0/3924650844 wait complete. 2024-09-13T16:48:20.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-09-13T16:48:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcf040e9fa0 con 0x7fcf04048310 2024-09-13T16:48:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.052+0000 7fcf08ff9640 1 -- 172.21.15.39:0/2469056229 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcf0c08e070 con 0x7fcf04048310 2024-09-13T16:48:20.059 DEBUG:teuthology.parallel:result is None 2024-09-13T16:48:20.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-09-13T16:48:20.060 INFO:teuthology.orchestra.run.smithi039.stdout:55834574857 2024-09-13T16:48:20.060 DEBUG:teuthology.parallel:result is None 2024-09-13T16:48:20.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-09-13T16:48:20.062 DEBUG:teuthology.parallel:result is None 2024-09-13T16:48:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcef00666c0 msgr2=0x7fcef0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcef00666c0 0x7fcef0068b80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf00000f70 tx=0x7fcf00007cb0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf04048310 msgr2=0x7fcf04030de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf04048310 0x7fcf04030de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c067df0 tx=0x7fcf0c068670 comp rx=0 tx=0).stop 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 shutdown_connections 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcef00666c0 0x7fcef0068b80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf0403f5c0 0x7fcf040308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf04048310 0x7fcf04030de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 --2- 172.21.15.39:0/2469056229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf0402bab0 0x7fcf040d7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 >> 172.21.15.39:0/2469056229 conn(0x7fcf04048e10 msgr2=0x7fcf0404ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 shutdown_connections 2024-09-13T16:48:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.056+0000 7fcf129ac640 1 -- 172.21.15.39:0/2469056229 wait complete. 2024-09-13T16:48:20.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-09-13T16:48:20.074 DEBUG:teuthology.parallel:result is None 2024-09-13T16:48:20.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-09-13T16:48:20.075 DEBUG:teuthology.parallel:result is None 2024-09-13T16:48:20.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-09-13T16:48:20.080 DEBUG:teuthology.parallel:result is None 2024-09-13T16:48:20.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T16:48:20.081 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:20.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc4253c1640 1 Processor -- start 2024-09-13T16:48:20.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc4253c1640 1 -- start start 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc4253c1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc4200c3be0 0x7fc420150b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc4253c1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc4200c5380 0x7fc4201510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc4253c1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4200c16f0 con 0x7fc4200ab560 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc4253c1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4200c1ab0 con 0x7fc4200c5380 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc4253c1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4200a4cc0 con 0x7fc4200c3be0 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41e7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc4200c5380 0x7fc4201510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41e7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc4200c5380 0x7fc4201510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58078/0 (socket says 172.21.15.39:58078) 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41e7fc640 1 -- 172.21.15.39:0/3134463883 learned_addr learned my addr 172.21.15.39:0/3134463883 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41effd640 1 --2- 172.21.15.39:0/3134463883 >> v2:172.21.15.84:3300/0 conn(0x7fc4200c3be0 0x7fc420150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41e7fc640 1 -- 172.21.15.39:0/3134463883 >> v2:172.21.15.84:3300/0 conn(0x7fc4200c3be0 msgr2=0x7fc420150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41e7fc640 1 --2- 172.21.15.39:0/3134463883 >> v2:172.21.15.84:3300/0 conn(0x7fc4200c3be0 0x7fc420150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41e7fc640 1 -- 172.21.15.39:0/3134463883 >> v1:172.21.15.84:6789/0 conn(0x7fc4200ab560 legacy=0x7fc42006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41e7fc640 1 -- 172.21.15.39:0/3134463883 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4200c45b0 con 0x7fc4200c5380 2024-09-13T16:48:20.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41e7fc640 1 --2- 172.21.15.39:0/3134463883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 0x7fc4201510a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc42006ace0 tx=0x7fc41401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90871ef3ebe82e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:20.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.228+0000 7fc41dffb640 1 -- 172.21.15.39:0/3134463883 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140ab600 con 0x7fc4200c5380 2024-09-13T16:48:20.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41dffb640 1 -- 172.21.15.39:0/3134463883 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4140a7050 con 0x7fc4200c5380 2024-09-13T16:48:20.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41dffb640 1 -- 172.21.15.39:0/3134463883 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140ab980 con 0x7fc4200c5380 2024-09-13T16:48:20.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- 172.21.15.39:0/3134463883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 msgr2=0x7fc4201510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 --2- 172.21.15.39:0/3134463883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 0x7fc4201510a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc42006ace0 tx=0x7fc41401c420 comp rx=0 tx=0).stop 2024-09-13T16:48:20.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- 172.21.15.39:0/3134463883 shutdown_connections 2024-09-13T16:48:20.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 --2- 172.21.15.39:0/3134463883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 0x7fc4201510a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 --2- 172.21.15.39:0/3134463883 >> v2:172.21.15.84:3300/0 conn(0x7fc4200c3be0 0x7fc420150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- 172.21.15.39:0/3134463883 >> 172.21.15.39:0/3134463883 conn(0x7fc420069e10 msgr2=0x7fc42006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- 172.21.15.39:0/3134463883 shutdown_connections 2024-09-13T16:48:20.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- 172.21.15.39:0/3134463883 wait complete. 2024-09-13T16:48:20.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 Processor -- start 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- start start 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4200ab560 0x7fc420159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc4200c3be0 0x7fc4201ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 0x7fc420148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42014f560 con 0x7fc4200c5380 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42015ce10 con 0x7fc4200ab560 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42015d3d0 con 0x7fc4200c3be0 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 0x7fc420148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 0x7fc420148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58086/0 (socket says 172.21.15.39:58086) 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41f7fe640 1 -- 172.21.15.39:0/4056231762 learned_addr learned my addr 172.21.15.39:0/4056231762 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:20.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41e7fc640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc4200c3be0 0x7fc4201ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41f7fe640 1 -- 172.21.15.39:0/4056231762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc4200c3be0 msgr2=0x7fc4201ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41effd640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4200ab560 0x7fc420159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41f7fe640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc4200c3be0 0x7fc4201ef8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41f7fe640 1 -- 172.21.15.39:0/4056231762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4200ab560 msgr2=0x7fc420159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41f7fe640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4200ab560 0x7fc420159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41f7fe640 1 -- 172.21.15.39:0/4056231762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4140094d0 con 0x7fc4200c5380 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc41f7fe640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 0x7fc420148150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc41800b970 tx=0x7fc41800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4077fe640 1 -- 172.21.15.39:0/4056231762 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41800cae0 con 0x7fc4200c5380 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4077fe640 1 -- 172.21.15.39:0/4056231762 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41800cc40 con 0x7fc4200c5380 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc420148f10 con 0x7fc4200c5380 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4077fe640 1 -- 172.21.15.39:0/4056231762 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418014680 con 0x7fc4200c5380 2024-09-13T16:48:20.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.232+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc420149330 con 0x7fc4200c5380 2024-09-13T16:48:20.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.236+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4200c3410 con 0x7fc4200c5380 2024-09-13T16:48:20.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.236+0000 7fc4077fe640 1 -- 172.21.15.39:0/4056231762 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc418003980 con 0x7fc4200c5380 2024-09-13T16:48:20.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.236+0000 7fc4077fe640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc40c066690 0x7fc40c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.236+0000 7fc4077fe640 1 -- 172.21.15.39:0/4056231762 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fc418084240 con 0x7fc4200c5380 2024-09-13T16:48:20.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.236+0000 7fc41effd640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc40c066690 0x7fc40c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.236+0000 7fc41effd640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc40c066690 0x7fc40c068b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc410009790 tx=0x7fc410008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:20.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.240+0000 7fc4077fe640 1 -- 172.21.15.39:0/4056231762 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc4180512b0 con 0x7fc4200c5380 2024-09-13T16:48:20.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.388+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4200c11d0 con 0x7fc40c066690 2024-09-13T16:48:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.424+0000 7fc4077fe640 1 -- 172.21.15.39:0/4056231762 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+502938 (secure 0 0 0) 0x7fc410009d70 con 0x7fc40c066690 2024-09-13T16:48:20.436 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc40c066690 msgr2=0x7fc40c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc40c066690 0x7fc40c068b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc410009790 tx=0x7fc410008040 comp rx=0 tx=0).stop 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 msgr2=0x7fc420148150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 0x7fc420148150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc41800b970 tx=0x7fc41800be30 comp rx=0 tx=0).stop 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 shutdown_connections 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc40c066690 0x7fc40c068b50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4200c5380 0x7fc420148150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc4200c3be0 0x7fc4201ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 --2- 172.21.15.39:0/4056231762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc4200ab560 0x7fc420159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 >> 172.21.15.39:0/4056231762 conn(0x7fc420069e10 msgr2=0x7fc42006bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 shutdown_connections 2024-09-13T16:48:20.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.432+0000 7fc4253c1640 1 -- 172.21.15.39:0/4056231762 wait complete. 2024-09-13T16:48:20.453 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-13T16:48:19.563787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595092,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235816,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738904,"statfs":{"total":773094113280,"available":772852637696,"internally_reserved":0,"allocated":3350528,"data_stored":1983984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.408382"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174677+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T16:48:09.174677+0000","last_peered":"2024-09-13T16:48:09.174677+0000","last_clean":"2024-09-13T16:48:09.174677+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T16:48:09.174677+0000","last_undegraded":"2024-09-13T16:48:09.174677+0000","last_fullsized":"2024-09-13T16:48:09.174677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163864+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:08.163864+0000","last_peered":"2024-09-13T16:48:08.163864+0000","last_clean":"2024-09-13T16:48:08.163864+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:08.163864+0000","last_undegraded":"2024-09-13T16:48:08.163864+0000","last_fullsized":"2024-09-13T16:48:08.163864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146105+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:08.146105+0000","last_peered":"2024-09-13T16:48:08.146105+0000","last_clean":"2024-09-13T16:48:08.146105+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:08.146105+0000","last_undegraded":"2024-09-13T16:48:08.146105+0000","last_fullsized":"2024-09-13T16:48:08.146105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143176+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T16:48:08.143176+0000","last_peered":"2024-09-13T16:48:08.143176+0000","last_clean":"2024-09-13T16:48:08.143176+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T16:48:08.143176+0000","last_undegraded":"2024-09-13T16:48:08.143176+0000","last_fullsized":"2024-09-13T16:48:08.143176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173274+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T16:48:09.173274+0000","last_peered":"2024-09-13T16:48:09.173274+0000","last_clean":"2024-09-13T16:48:09.173274+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T16:48:09.173274+0000","last_undegraded":"2024-09-13T16:48:09.173274+0000","last_fullsized":"2024-09-13T16:48:09.173274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146246+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:08.146246+0000","last_peered":"2024-09-13T16:48:08.146246+0000","last_clean":"2024-09-13T16:48:08.146246+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:08.146246+0000","last_undegraded":"2024-09-13T16:48:08.146246+0000","last_fullsized":"2024-09-13T16:48:08.146246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163480+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:08.163480+0000","last_peered":"2024-09-13T16:48:08.163480+0000","last_clean":"2024-09-13T16:48:08.163480+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:08.163480+0000","last_undegraded":"2024-09-13T16:48:08.163480+0000","last_fullsized":"2024-09-13T16:48:08.163480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184157+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T16:48:08.184157+0000","last_peered":"2024-09-13T16:48:08.184157+0000","last_clean":"2024-09-13T16:48:08.184157+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T16:48:08.184157+0000","last_undegraded":"2024-09-13T16:48:08.184157+0000","last_fullsized":"2024-09-13T16:48:08.184157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173949+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T16:48:09.173949+0000","last_peered":"2024-09-13T16:48:09.173949+0000","last_clean":"2024-09-13T16:48:09.173949+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T16:48:09.173949+0000","last_undegraded":"2024-09-13T16:48:09.173949+0000","last_fullsized":"2024-09-13T16:48:09.173949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178923+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:08.178923+0000","last_peered":"2024-09-13T16:48:08.178923+0000","last_clean":"2024-09-13T16:48:08.178923+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:08.178923+0000","last_undegraded":"2024-09-13T16:48:08.178923+0000","last_fullsized":"2024-09-13T16:48:08.178923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151847+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:08.151847+0000","last_peered":"2024-09-13T16:48:08.151847+0000","last_clean":"2024-09-13T16:48:08.151847+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:08.151847+0000","last_undegraded":"2024-09-13T16:48:08.151847+0000","last_fullsized":"2024-09-13T16:48:08.151847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208404+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T16:48:08.208404+0000","last_peered":"2024-09-13T16:48:08.208404+0000","last_clean":"2024-09-13T16:48:08.208404+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T16:48:08.208404+0000","last_undegraded":"2024-09-13T16:48:08.208404+0000","last_fullsized":"2024-09-13T16:48:08.208404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159591+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T16:48:09.159591+0000","last_peered":"2024-09-13T16:48:09.159591+0000","last_clean":"2024-09-13T16:48:09.159591+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T16:48:09.159591+0000","last_undegraded":"2024-09-13T16:48:09.159591+0000","last_fullsized":"2024-09-13T16:48:09.159591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165397+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:08.165397+0000","last_peered":"2024-09-13T16:48:08.165397+0000","last_clean":"2024-09-13T16:48:08.165397+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:08.165397+0000","last_undegraded":"2024-09-13T16:48:08.165397+0000","last_fullsized":"2024-09-13T16:48:08.165397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153028+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:08.153028+0000","last_peered":"2024-09-13T16:48:08.153028+0000","last_clean":"2024-09-13T16:48:08.153028+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:08.153028+0000","last_undegraded":"2024-09-13T16:48:08.153028+0000","last_fullsized":"2024-09-13T16:48:08.153028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.203658+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T16:48:08.203658+0000","last_peered":"2024-09-13T16:48:08.203658+0000","last_clean":"2024-09-13T16:48:08.203658+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T16:48:08.203658+0000","last_undegraded":"2024-09-13T16:48:08.203658+0000","last_fullsized":"2024-09-13T16:48:08.203658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173632+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T16:48:09.173632+0000","last_peered":"2024-09-13T16:48:09.173632+0000","last_clean":"2024-09-13T16:48:09.173632+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T16:48:09.173632+0000","last_undegraded":"2024-09-13T16:48:09.173632+0000","last_fullsized":"2024-09-13T16:48:09.173632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146884+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:08.146884+0000","last_peered":"2024-09-13T16:48:08.146884+0000","last_clean":"2024-09-13T16:48:08.146884+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:08.146884+0000","last_undegraded":"2024-09-13T16:48:08.146884+0000","last_fullsized":"2024-09-13T16:48:08.146884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160788+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:08.160788+0000","last_peered":"2024-09-13T16:48:08.160788+0000","last_clean":"2024-09-13T16:48:08.160788+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:08.160788+0000","last_undegraded":"2024-09-13T16:48:08.160788+0000","last_fullsized":"2024-09-13T16:48:08.160788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.206623+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T16:48:08.206623+0000","last_peered":"2024-09-13T16:48:08.206623+0000","last_clean":"2024-09-13T16:48:08.206623+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T16:48:08.206623+0000","last_undegraded":"2024-09-13T16:48:08.206623+0000","last_fullsized":"2024-09-13T16:48:08.206623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159080+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T16:48:09.159080+0000","last_peered":"2024-09-13T16:48:09.159080+0000","last_clean":"2024-09-13T16:48:09.159080+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T16:48:09.159080+0000","last_undegraded":"2024-09-13T16:48:09.159080+0000","last_fullsized":"2024-09-13T16:48:09.159080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144345+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:08.144345+0000","last_peered":"2024-09-13T16:48:08.144345+0000","last_clean":"2024-09-13T16:48:08.144345+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:08.144345+0000","last_undegraded":"2024-09-13T16:48:08.144345+0000","last_fullsized":"2024-09-13T16:48:08.144345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147467+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:08.147467+0000","last_peered":"2024-09-13T16:48:08.147467+0000","last_clean":"2024-09-13T16:48:08.147467+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:08.147467+0000","last_undegraded":"2024-09-13T16:48:08.147467+0000","last_fullsized":"2024-09-13T16:48:08.147467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184081+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T16:48:08.184081+0000","last_peered":"2024-09-13T16:48:08.184081+0000","last_clean":"2024-09-13T16:48:08.184081+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T16:48:08.184081+0000","last_undegraded":"2024-09-13T16:48:08.184081+0000","last_fullsized":"2024-09-13T16:48:08.184081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143476+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:08.143476+0000","last_peered":"2024-09-13T16:48:08.143476+0000","last_clean":"2024-09-13T16:48:08.143476+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:08.143476+0000","last_undegraded":"2024-09-13T16:48:08.143476+0000","last_fullsized":"2024-09-13T16:48:08.143476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147565+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:08.147565+0000","last_peered":"2024-09-13T16:48:08.147565+0000","last_clean":"2024-09-13T16:48:08.147565+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:08.147565+0000","last_undegraded":"2024-09-13T16:48:08.147565+0000","last_fullsized":"2024-09-13T16:48:08.147565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174007+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T16:48:08.174007+0000","last_peered":"2024-09-13T16:48:08.174007+0000","last_clean":"2024-09-13T16:48:08.174007+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T16:48:08.174007+0000","last_undegraded":"2024-09-13T16:48:08.174007+0000","last_fullsized":"2024-09-13T16:48:08.174007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162482+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T16:48:09.162482+0000","last_peered":"2024-09-13T16:48:09.162482+0000","last_clean":"2024-09-13T16:48:09.162482+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T16:48:09.162482+0000","last_undegraded":"2024-09-13T16:48:09.162482+0000","last_fullsized":"2024-09-13T16:48:09.162482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186454+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:08.186454+0000","last_peered":"2024-09-13T16:48:08.186454+0000","last_clean":"2024-09-13T16:48:08.186454+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:08.186454+0000","last_undegraded":"2024-09-13T16:48:08.186454+0000","last_fullsized":"2024-09-13T16:48:08.186454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160985+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:08.160985+0000","last_peered":"2024-09-13T16:48:08.160985+0000","last_clean":"2024-09-13T16:48:08.160985+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:08.160985+0000","last_undegraded":"2024-09-13T16:48:08.160985+0000","last_fullsized":"2024-09-13T16:48:08.160985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208189+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T16:48:08.208189+0000","last_peered":"2024-09-13T16:48:08.208189+0000","last_clean":"2024-09-13T16:48:08.208189+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T16:48:08.208189+0000","last_undegraded":"2024-09-13T16:48:08.208189+0000","last_fullsized":"2024-09-13T16:48:08.208189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176059+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T16:48:09.176059+0000","last_peered":"2024-09-13T16:48:09.176059+0000","last_clean":"2024-09-13T16:48:09.176059+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T16:48:09.176059+0000","last_undegraded":"2024-09-13T16:48:09.176059+0000","last_fullsized":"2024-09-13T16:48:09.176059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163768+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:08.163768+0000","last_peered":"2024-09-13T16:48:08.163768+0000","last_clean":"2024-09-13T16:48:08.163768+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:08.163768+0000","last_undegraded":"2024-09-13T16:48:08.163768+0000","last_fullsized":"2024-09-13T16:48:08.163768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148721+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:08.148721+0000","last_peered":"2024-09-13T16:48:08.148721+0000","last_clean":"2024-09-13T16:48:08.148721+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:08.148721+0000","last_undegraded":"2024-09-13T16:48:08.148721+0000","last_fullsized":"2024-09-13T16:48:08.148721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185371+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T16:48:08.185371+0000","last_peered":"2024-09-13T16:48:08.185371+0000","last_clean":"2024-09-13T16:48:08.185371+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T16:48:08.185371+0000","last_undegraded":"2024-09-13T16:48:08.185371+0000","last_fullsized":"2024-09-13T16:48:08.185371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174416+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T16:48:09.174416+0000","last_peered":"2024-09-13T16:48:09.174416+0000","last_clean":"2024-09-13T16:48:09.174416+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T16:48:09.174416+0000","last_undegraded":"2024-09-13T16:48:09.174416+0000","last_fullsized":"2024-09-13T16:48:09.174416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145183+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:08.145183+0000","last_peered":"2024-09-13T16:48:08.145183+0000","last_clean":"2024-09-13T16:48:08.145183+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:08.145183+0000","last_undegraded":"2024-09-13T16:48:08.145183+0000","last_fullsized":"2024-09-13T16:48:08.145183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152363+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:08.152363+0000","last_peered":"2024-09-13T16:48:08.152363+0000","last_clean":"2024-09-13T16:48:08.152363+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:08.152363+0000","last_undegraded":"2024-09-13T16:48:08.152363+0000","last_fullsized":"2024-09-13T16:48:08.152363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143237+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T16:48:08.143237+0000","last_peered":"2024-09-13T16:48:08.143237+0000","last_clean":"2024-09-13T16:48:08.143237+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T16:48:08.143237+0000","last_undegraded":"2024-09-13T16:48:08.143237+0000","last_fullsized":"2024-09-13T16:48:08.143237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158858+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T16:48:09.158858+0000","last_peered":"2024-09-13T16:48:09.158858+0000","last_clean":"2024-09-13T16:48:09.158858+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T16:48:09.158858+0000","last_undegraded":"2024-09-13T16:48:09.158858+0000","last_fullsized":"2024-09-13T16:48:09.158858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148446+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:08.148446+0000","last_peered":"2024-09-13T16:48:08.148446+0000","last_clean":"2024-09-13T16:48:08.148446+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:08.148446+0000","last_undegraded":"2024-09-13T16:48:08.148446+0000","last_fullsized":"2024-09-13T16:48:08.148446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179426+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:08.179426+0000","last_peered":"2024-09-13T16:48:08.179426+0000","last_clean":"2024-09-13T16:48:08.179426+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:08.179426+0000","last_undegraded":"2024-09-13T16:48:08.179426+0000","last_fullsized":"2024-09-13T16:48:08.179426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161207+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T16:48:08.161207+0000","last_peered":"2024-09-13T16:48:08.161207+0000","last_clean":"2024-09-13T16:48:08.161207+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T16:48:08.161207+0000","last_undegraded":"2024-09-13T16:48:08.161207+0000","last_fullsized":"2024-09-13T16:48:08.161207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173006+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T16:48:09.173006+0000","last_peered":"2024-09-13T16:48:09.173006+0000","last_clean":"2024-09-13T16:48:09.173006+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T16:48:09.173006+0000","last_undegraded":"2024-09-13T16:48:09.173006+0000","last_fullsized":"2024-09-13T16:48:09.173006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153222+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:08.153222+0000","last_peered":"2024-09-13T16:48:08.153222+0000","last_clean":"2024-09-13T16:48:08.153222+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:08.153222+0000","last_undegraded":"2024-09-13T16:48:08.153222+0000","last_fullsized":"2024-09-13T16:48:08.153222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148522+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:08.148522+0000","last_peered":"2024-09-13T16:48:08.148522+0000","last_clean":"2024-09-13T16:48:08.148522+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:08.148522+0000","last_undegraded":"2024-09-13T16:48:08.148522+0000","last_fullsized":"2024-09-13T16:48:08.148522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146684+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T16:48:08.146684+0000","last_peered":"2024-09-13T16:48:08.146684+0000","last_clean":"2024-09-13T16:48:08.146684+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T16:48:08.146684+0000","last_undegraded":"2024-09-13T16:48:08.146684+0000","last_fullsized":"2024-09-13T16:48:08.146684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174867+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T16:48:09.174867+0000","last_peered":"2024-09-13T16:48:09.174867+0000","last_clean":"2024-09-13T16:48:09.174867+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T16:48:09.174867+0000","last_undegraded":"2024-09-13T16:48:09.174867+0000","last_fullsized":"2024-09-13T16:48:09.174867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185855+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:08.185855+0000","last_peered":"2024-09-13T16:48:08.185855+0000","last_clean":"2024-09-13T16:48:08.185855+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:08.185855+0000","last_undegraded":"2024-09-13T16:48:08.185855+0000","last_fullsized":"2024-09-13T16:48:08.185855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149466+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:08.149466+0000","last_peered":"2024-09-13T16:48:08.149466+0000","last_clean":"2024-09-13T16:48:08.149466+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:08.149466+0000","last_undegraded":"2024-09-13T16:48:08.149466+0000","last_fullsized":"2024-09-13T16:48:08.149466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148372+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T16:48:08.148372+0000","last_peered":"2024-09-13T16:48:08.148372+0000","last_clean":"2024-09-13T16:48:08.148372+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T16:48:08.148372+0000","last_undegraded":"2024-09-13T16:48:08.148372+0000","last_fullsized":"2024-09-13T16:48:08.148372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159186+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T16:48:09.159186+0000","last_peered":"2024-09-13T16:48:09.159186+0000","last_clean":"2024-09-13T16:48:09.159186+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T16:48:09.159186+0000","last_undegraded":"2024-09-13T16:48:09.159186+0000","last_fullsized":"2024-09-13T16:48:09.159186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146860+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:08.146860+0000","last_peered":"2024-09-13T16:48:08.146860+0000","last_clean":"2024-09-13T16:48:08.146860+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:08.146860+0000","last_undegraded":"2024-09-13T16:48:08.146860+0000","last_fullsized":"2024-09-13T16:48:08.146860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154111+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:08.154111+0000","last_peered":"2024-09-13T16:48:08.154111+0000","last_clean":"2024-09-13T16:48:08.154111+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:08.154111+0000","last_undegraded":"2024-09-13T16:48:08.154111+0000","last_fullsized":"2024-09-13T16:48:08.154111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175388+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T16:48:08.175388+0000","last_peered":"2024-09-13T16:48:08.175388+0000","last_clean":"2024-09-13T16:48:08.175388+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T16:48:08.175388+0000","last_undegraded":"2024-09-13T16:48:08.175388+0000","last_fullsized":"2024-09-13T16:48:08.175388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162556+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T16:48:09.162556+0000","last_peered":"2024-09-13T16:48:09.162556+0000","last_clean":"2024-09-13T16:48:09.162556+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T16:48:09.162556+0000","last_undegraded":"2024-09-13T16:48:09.162556+0000","last_fullsized":"2024-09-13T16:48:09.162556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160990+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:08.160990+0000","last_peered":"2024-09-13T16:48:08.160990+0000","last_clean":"2024-09-13T16:48:08.160990+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:08.160990+0000","last_undegraded":"2024-09-13T16:48:08.160990+0000","last_fullsized":"2024-09-13T16:48:08.160990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161254+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:08.161254+0000","last_peered":"2024-09-13T16:48:08.161254+0000","last_clean":"2024-09-13T16:48:08.161254+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:08.161254+0000","last_undegraded":"2024-09-13T16:48:08.161254+0000","last_fullsized":"2024-09-13T16:48:08.161254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175218+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T16:48:09.175218+0000","last_peered":"2024-09-13T16:48:09.175218+0000","last_clean":"2024-09-13T16:48:09.175218+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T16:48:09.175218+0000","last_undegraded":"2024-09-13T16:48:09.175218+0000","last_fullsized":"2024-09-13T16:48:09.175218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175463+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T16:48:09.175463+0000","last_peered":"2024-09-13T16:48:09.175463+0000","last_clean":"2024-09-13T16:48:09.175463+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T16:48:09.175463+0000","last_undegraded":"2024-09-13T16:48:09.175463+0000","last_fullsized":"2024-09-13T16:48:09.175463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143792+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:08.143792+0000","last_peered":"2024-09-13T16:48:08.143792+0000","last_clean":"2024-09-13T16:48:08.143792+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:08.143792+0000","last_undegraded":"2024-09-13T16:48:08.143792+0000","last_fullsized":"2024-09-13T16:48:08.143792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148284+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:08.148284+0000","last_peered":"2024-09-13T16:48:08.148284+0000","last_clean":"2024-09-13T16:48:08.148284+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:08.148284+0000","last_undegraded":"2024-09-13T16:48:08.148284+0000","last_fullsized":"2024-09-13T16:48:08.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161478+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:08.161478+0000","last_peered":"2024-09-13T16:48:08.161478+0000","last_clean":"2024-09-13T16:48:08.161478+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:08.161478+0000","last_undegraded":"2024-09-13T16:48:08.161478+0000","last_fullsized":"2024-09-13T16:48:08.161478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148905+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T16:48:08.148905+0000","last_peered":"2024-09-13T16:48:08.148905+0000","last_clean":"2024-09-13T16:48:08.148905+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T16:48:08.148905+0000","last_undegraded":"2024-09-13T16:48:08.148905+0000","last_fullsized":"2024-09-13T16:48:08.148905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158934+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T16:48:09.158934+0000","last_peered":"2024-09-13T16:48:09.158934+0000","last_clean":"2024-09-13T16:48:09.158934+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T16:48:09.158934+0000","last_undegraded":"2024-09-13T16:48:09.158934+0000","last_fullsized":"2024-09-13T16:48:09.158934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178617+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:08.178617+0000","last_peered":"2024-09-13T16:48:08.178617+0000","last_clean":"2024-09-13T16:48:08.178617+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:08.178617+0000","last_undegraded":"2024-09-13T16:48:08.178617+0000","last_fullsized":"2024-09-13T16:48:08.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155913+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:08.155913+0000","last_peered":"2024-09-13T16:48:08.155913+0000","last_clean":"2024-09-13T16:48:08.155913+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:08.155913+0000","last_undegraded":"2024-09-13T16:48:08.155913+0000","last_fullsized":"2024-09-13T16:48:08.155913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146319+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T16:48:08.146319+0000","last_peered":"2024-09-13T16:48:08.146319+0000","last_clean":"2024-09-13T16:48:08.146319+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T16:48:08.146319+0000","last_undegraded":"2024-09-13T16:48:08.146319+0000","last_fullsized":"2024-09-13T16:48:08.146319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158609+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T16:48:09.158609+0000","last_peered":"2024-09-13T16:48:09.158609+0000","last_clean":"2024-09-13T16:48:09.158609+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T16:48:09.158609+0000","last_undegraded":"2024-09-13T16:48:09.158609+0000","last_fullsized":"2024-09-13T16:48:09.158609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152057+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:08.152057+0000","last_peered":"2024-09-13T16:48:08.152057+0000","last_clean":"2024-09-13T16:48:08.152057+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:08.152057+0000","last_undegraded":"2024-09-13T16:48:08.152057+0000","last_fullsized":"2024-09-13T16:48:08.152057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149487+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:08.149487+0000","last_peered":"2024-09-13T16:48:08.149487+0000","last_clean":"2024-09-13T16:48:08.149487+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:08.149487+0000","last_undegraded":"2024-09-13T16:48:08.149487+0000","last_fullsized":"2024-09-13T16:48:08.149487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175317+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T16:48:08.175317+0000","last_peered":"2024-09-13T16:48:08.175317+0000","last_clean":"2024-09-13T16:48:08.175317+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T16:48:08.175317+0000","last_undegraded":"2024-09-13T16:48:08.175317+0000","last_fullsized":"2024-09-13T16:48:08.175317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173209+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T16:48:09.173209+0000","last_peered":"2024-09-13T16:48:09.173209+0000","last_clean":"2024-09-13T16:48:09.173209+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T16:48:09.173209+0000","last_undegraded":"2024-09-13T16:48:09.173209+0000","last_fullsized":"2024-09-13T16:48:09.173209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.177954+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:08.177954+0000","last_peered":"2024-09-13T16:48:08.177954+0000","last_clean":"2024-09-13T16:48:08.177954+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:08.177954+0000","last_undegraded":"2024-09-13T16:48:08.177954+0000","last_fullsized":"2024-09-13T16:48:08.177954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155936+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:08.155936+0000","last_peered":"2024-09-13T16:48:08.155936+0000","last_clean":"2024-09-13T16:48:08.155936+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:08.155936+0000","last_undegraded":"2024-09-13T16:48:08.155936+0000","last_fullsized":"2024-09-13T16:48:08.155936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158692+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T16:48:08.158692+0000","last_peered":"2024-09-13T16:48:08.158692+0000","last_clean":"2024-09-13T16:48:08.158692+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T16:48:08.158692+0000","last_undegraded":"2024-09-13T16:48:08.158692+0000","last_fullsized":"2024-09-13T16:48:08.158692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158034+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T16:48:09.158034+0000","last_peered":"2024-09-13T16:48:09.158034+0000","last_clean":"2024-09-13T16:48:09.158034+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T16:48:09.158034+0000","last_undegraded":"2024-09-13T16:48:09.158034+0000","last_fullsized":"2024-09-13T16:48:09.158034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160811+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:08.160811+0000","last_peered":"2024-09-13T16:48:08.160811+0000","last_clean":"2024-09-13T16:48:08.160811+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:08.160811+0000","last_undegraded":"2024-09-13T16:48:08.160811+0000","last_fullsized":"2024-09-13T16:48:08.160811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.192267+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:08.192267+0000","last_peered":"2024-09-13T16:48:08.192267+0000","last_clean":"2024-09-13T16:48:08.192267+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:08.192267+0000","last_undegraded":"2024-09-13T16:48:08.192267+0000","last_fullsized":"2024-09-13T16:48:08.192267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185117+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T16:48:08.185117+0000","last_peered":"2024-09-13T16:48:08.185117+0000","last_clean":"2024-09-13T16:48:08.185117+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T16:48:08.185117+0000","last_undegraded":"2024-09-13T16:48:08.185117+0000","last_fullsized":"2024-09-13T16:48:08.185117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177888+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T16:48:09.177888+0000","last_peered":"2024-09-13T16:48:09.177888+0000","last_clean":"2024-09-13T16:48:09.177888+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T16:48:09.177888+0000","last_undegraded":"2024-09-13T16:48:09.177888+0000","last_fullsized":"2024-09-13T16:48:09.177888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167237+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:08.167237+0000","last_peered":"2024-09-13T16:48:08.167237+0000","last_clean":"2024-09-13T16:48:08.167237+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:08.167237+0000","last_undegraded":"2024-09-13T16:48:08.167237+0000","last_fullsized":"2024-09-13T16:48:08.167237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156043+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:08.156043+0000","last_peered":"2024-09-13T16:48:08.156043+0000","last_clean":"2024-09-13T16:48:08.156043+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:08.156043+0000","last_undegraded":"2024-09-13T16:48:08.156043+0000","last_fullsized":"2024-09-13T16:48:08.156043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163690+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T16:48:08.163690+0000","last_peered":"2024-09-13T16:48:08.163690+0000","last_clean":"2024-09-13T16:48:08.163690+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T16:48:08.163690+0000","last_undegraded":"2024-09-13T16:48:08.163690+0000","last_fullsized":"2024-09-13T16:48:08.163690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162212+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T16:48:09.162212+0000","last_peered":"2024-09-13T16:48:09.162212+0000","last_clean":"2024-09-13T16:48:09.162212+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T16:48:09.162212+0000","last_undegraded":"2024-09-13T16:48:09.162212+0000","last_fullsized":"2024-09-13T16:48:09.162212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144923+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:08.144923+0000","last_peered":"2024-09-13T16:48:08.144923+0000","last_clean":"2024-09-13T16:48:08.144923+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:08.144923+0000","last_undegraded":"2024-09-13T16:48:08.144923+0000","last_fullsized":"2024-09-13T16:48:08.144923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145599+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:08.145599+0000","last_peered":"2024-09-13T16:48:08.145599+0000","last_clean":"2024-09-13T16:48:08.145599+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:08.145599+0000","last_undegraded":"2024-09-13T16:48:08.145599+0000","last_fullsized":"2024-09-13T16:48:08.145599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145022+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T16:48:08.145022+0000","last_peered":"2024-09-13T16:48:08.145022+0000","last_clean":"2024-09-13T16:48:08.145022+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T16:48:08.145022+0000","last_undegraded":"2024-09-13T16:48:08.145022+0000","last_fullsized":"2024-09-13T16:48:08.145022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161391+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T16:48:09.161391+0000","last_peered":"2024-09-13T16:48:09.161391+0000","last_clean":"2024-09-13T16:48:09.161391+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T16:48:09.161391+0000","last_undegraded":"2024-09-13T16:48:09.161391+0000","last_fullsized":"2024-09-13T16:48:09.161391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160556+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:08.160556+0000","last_peered":"2024-09-13T16:48:08.160556+0000","last_clean":"2024-09-13T16:48:08.160556+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:08.160556+0000","last_undegraded":"2024-09-13T16:48:08.160556+0000","last_fullsized":"2024-09-13T16:48:08.160556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149235+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:08.149235+0000","last_peered":"2024-09-13T16:48:08.149235+0000","last_clean":"2024-09-13T16:48:08.149235+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:08.149235+0000","last_undegraded":"2024-09-13T16:48:08.149235+0000","last_fullsized":"2024-09-13T16:48:08.149235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.209853+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T16:48:08.209853+0000","last_peered":"2024-09-13T16:48:08.209853+0000","last_clean":"2024-09-13T16:48:08.209853+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T16:48:08.209853+0000","last_undegraded":"2024-09-13T16:48:08.209853+0000","last_fullsized":"2024-09-13T16:48:08.209853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161110+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:08.161110+0000","last_peered":"2024-09-13T16:48:08.161110+0000","last_clean":"2024-09-13T16:48:08.161110+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:08.161110+0000","last_undegraded":"2024-09-13T16:48:08.161110+0000","last_fullsized":"2024-09-13T16:48:08.161110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161421+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:08.161421+0000","last_peered":"2024-09-13T16:48:08.161421+0000","last_clean":"2024-09-13T16:48:08.161421+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:08.161421+0000","last_undegraded":"2024-09-13T16:48:08.161421+0000","last_fullsized":"2024-09-13T16:48:08.161421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.188902+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T16:48:08.188902+0000","last_peered":"2024-09-13T16:48:08.188902+0000","last_clean":"2024-09-13T16:48:08.188902+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T16:48:08.188902+0000","last_undegraded":"2024-09-13T16:48:08.188902+0000","last_fullsized":"2024-09-13T16:48:08.188902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165383+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:08.165383+0000","last_peered":"2024-09-13T16:48:08.165383+0000","last_clean":"2024-09-13T16:48:08.165383+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:08.165383+0000","last_undegraded":"2024-09-13T16:48:08.165383+0000","last_fullsized":"2024-09-13T16:48:08.165383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148543+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:08.148543+0000","last_peered":"2024-09-13T16:48:08.148543+0000","last_clean":"2024-09-13T16:48:08.148543+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:08.148543+0000","last_undegraded":"2024-09-13T16:48:08.148543+0000","last_fullsized":"2024-09-13T16:48:08.148543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146582+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T16:48:08.146582+0000","last_peered":"2024-09-13T16:48:08.146582+0000","last_clean":"2024-09-13T16:48:08.146582+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T16:48:08.146582+0000","last_undegraded":"2024-09-13T16:48:08.146582+0000","last_fullsized":"2024-09-13T16:48:08.146582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173477+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T16:48:09.173477+0000","last_peered":"2024-09-13T16:48:09.173477+0000","last_clean":"2024-09-13T16:48:09.173477+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T16:48:09.173477+0000","last_undegraded":"2024-09-13T16:48:09.173477+0000","last_fullsized":"2024-09-13T16:48:09.173477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.190852+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:08.190852+0000","last_peered":"2024-09-13T16:48:08.190852+0000","last_clean":"2024-09-13T16:48:08.190852+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:08.190852+0000","last_undegraded":"2024-09-13T16:48:08.190852+0000","last_fullsized":"2024-09-13T16:48:08.190852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149403+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:08.149403+0000","last_peered":"2024-09-13T16:48:08.149403+0000","last_clean":"2024-09-13T16:48:08.149403+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:08.149403+0000","last_undegraded":"2024-09-13T16:48:08.149403+0000","last_fullsized":"2024-09-13T16:48:08.149403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208558+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T16:48:08.208558+0000","last_peered":"2024-09-13T16:48:08.208558+0000","last_clean":"2024-09-13T16:48:08.208558+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T16:48:08.208558+0000","last_undegraded":"2024-09-13T16:48:08.208558+0000","last_fullsized":"2024-09-13T16:48:08.208558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158601+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T16:48:09.158601+0000","last_peered":"2024-09-13T16:48:09.158601+0000","last_clean":"2024-09-13T16:48:09.158601+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T16:48:09.158601+0000","last_undegraded":"2024-09-13T16:48:09.158601+0000","last_fullsized":"2024-09-13T16:48:09.158601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163596+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:08.163596+0000","last_peered":"2024-09-13T16:48:08.163596+0000","last_clean":"2024-09-13T16:48:08.163596+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:08.163596+0000","last_undegraded":"2024-09-13T16:48:08.163596+0000","last_fullsized":"2024-09-13T16:48:08.163596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156189+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:08.156189+0000","last_peered":"2024-09-13T16:48:08.156189+0000","last_clean":"2024-09-13T16:48:08.156189+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:08.156189+0000","last_undegraded":"2024-09-13T16:48:08.156189+0000","last_fullsized":"2024-09-13T16:48:08.156189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174258+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T16:48:08.174258+0000","last_peered":"2024-09-13T16:48:08.174258+0000","last_clean":"2024-09-13T16:48:08.174258+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T16:48:08.174258+0000","last_undegraded":"2024-09-13T16:48:08.174258+0000","last_fullsized":"2024-09-13T16:48:08.174258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173305+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T16:48:09.173305+0000","last_peered":"2024-09-13T16:48:09.173305+0000","last_clean":"2024-09-13T16:48:09.173305+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T16:48:09.173305+0000","last_undegraded":"2024-09-13T16:48:09.173305+0000","last_fullsized":"2024-09-13T16:48:09.173305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155963+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:08.155963+0000","last_peered":"2024-09-13T16:48:08.155963+0000","last_clean":"2024-09-13T16:48:08.155963+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:08.155963+0000","last_undegraded":"2024-09-13T16:48:08.155963+0000","last_fullsized":"2024-09-13T16:48:08.155963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148500+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:08.148500+0000","last_peered":"2024-09-13T16:48:08.148500+0000","last_clean":"2024-09-13T16:48:08.148500+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:08.148500+0000","last_undegraded":"2024-09-13T16:48:08.148500+0000","last_fullsized":"2024-09-13T16:48:08.148500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143144+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T16:48:08.143144+0000","last_peered":"2024-09-13T16:48:08.143144+0000","last_clean":"2024-09-13T16:48:08.143144+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T16:48:08.143144+0000","last_undegraded":"2024-09-13T16:48:08.143144+0000","last_fullsized":"2024-09-13T16:48:08.143144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161066+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T16:48:09.161066+0000","last_peered":"2024-09-13T16:48:09.161066+0000","last_clean":"2024-09-13T16:48:09.161066+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T16:48:09.161066+0000","last_undegraded":"2024-09-13T16:48:09.161066+0000","last_fullsized":"2024-09-13T16:48:09.161066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186757+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T16:48:08.186757+0000","last_peered":"2024-09-13T16:48:08.186757+0000","last_clean":"2024-09-13T16:48:08.186757+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T16:48:08.186757+0000","last_undegraded":"2024-09-13T16:48:08.186757+0000","last_fullsized":"2024-09-13T16:48:08.186757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147291+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:08.147291+0000","last_peered":"2024-09-13T16:48:08.147291+0000","last_clean":"2024-09-13T16:48:08.147291+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:08.147291+0000","last_undegraded":"2024-09-13T16:48:08.147291+0000","last_fullsized":"2024-09-13T16:48:08.147291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158941+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:08.158941+0000","last_peered":"2024-09-13T16:48:08.158941+0000","last_clean":"2024-09-13T16:48:08.158941+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:08.158941+0000","last_undegraded":"2024-09-13T16:48:08.158941+0000","last_fullsized":"2024-09-13T16:48:08.158941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175068+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T16:48:09.175068+0000","last_peered":"2024-09-13T16:48:09.175068+0000","last_clean":"2024-09-13T16:48:09.175068+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T16:48:09.175068+0000","last_undegraded":"2024-09-13T16:48:09.175068+0000","last_fullsized":"2024-09-13T16:48:09.175068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.182997+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T16:48:08.182997+0000","last_peered":"2024-09-13T16:48:08.182997+0000","last_clean":"2024-09-13T16:48:08.182997+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T16:48:08.182997+0000","last_undegraded":"2024-09-13T16:48:08.182997+0000","last_fullsized":"2024-09-13T16:48:08.182997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143381+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:08.143381+0000","last_peered":"2024-09-13T16:48:08.143381+0000","last_clean":"2024-09-13T16:48:08.143381+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:08.143381+0000","last_undegraded":"2024-09-13T16:48:08.143381+0000","last_fullsized":"2024-09-13T16:48:08.143381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143921+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:08.143921+0000","last_peered":"2024-09-13T16:48:08.143921+0000","last_clean":"2024-09-13T16:48:08.143921+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:08.143921+0000","last_undegraded":"2024-09-13T16:48:08.143921+0000","last_fullsized":"2024-09-13T16:48:08.143921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174477+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T16:48:09.174477+0000","last_peered":"2024-09-13T16:48:09.174477+0000","last_clean":"2024-09-13T16:48:09.174477+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T16:48:09.174477+0000","last_undegraded":"2024-09-13T16:48:09.174477+0000","last_fullsized":"2024-09-13T16:48:09.174477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147114+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:08.147114+0000","last_peered":"2024-09-13T16:48:08.147114+0000","last_clean":"2024-09-13T16:48:08.147114+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:08.147114+0000","last_undegraded":"2024-09-13T16:48:08.147114+0000","last_fullsized":"2024-09-13T16:48:08.147114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161410+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:08.161410+0000","last_peered":"2024-09-13T16:48:08.161410+0000","last_clean":"2024-09-13T16:48:08.161410+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:08.161410+0000","last_undegraded":"2024-09-13T16:48:08.161410+0000","last_fullsized":"2024-09-13T16:48:08.161410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170387+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T16:48:08.170387+0000","last_peered":"2024-09-13T16:48:08.170387+0000","last_clean":"2024-09-13T16:48:08.170387+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T16:48:08.170387+0000","last_undegraded":"2024-09-13T16:48:08.170387+0000","last_fullsized":"2024-09-13T16:48:08.170387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175603+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T16:48:09.175603+0000","last_peered":"2024-09-13T16:48:09.175603+0000","last_clean":"2024-09-13T16:48:09.175603+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T16:48:09.175603+0000","last_undegraded":"2024-09-13T16:48:09.175603+0000","last_fullsized":"2024-09-13T16:48:09.175603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144443+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:08.144443+0000","last_peered":"2024-09-13T16:48:08.144443+0000","last_clean":"2024-09-13T16:48:08.144443+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:08.144443+0000","last_undegraded":"2024-09-13T16:48:08.144443+0000","last_fullsized":"2024-09-13T16:48:08.144443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185002+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:08.185002+0000","last_peered":"2024-09-13T16:48:08.185002+0000","last_clean":"2024-09-13T16:48:08.185002+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:08.185002+0000","last_undegraded":"2024-09-13T16:48:08.185002+0000","last_fullsized":"2024-09-13T16:48:08.185002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170142+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T16:48:08.170142+0000","last_peered":"2024-09-13T16:48:08.170142+0000","last_clean":"2024-09-13T16:48:08.170142+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T16:48:08.170142+0000","last_undegraded":"2024-09-13T16:48:08.170142+0000","last_fullsized":"2024-09-13T16:48:08.170142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177615+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T16:48:09.177615+0000","last_peered":"2024-09-13T16:48:09.177615+0000","last_clean":"2024-09-13T16:48:09.177615+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T16:48:09.177615+0000","last_undegraded":"2024-09-13T16:48:09.177615+0000","last_fullsized":"2024-09-13T16:48:09.177615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144121+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:08.144121+0000","last_peered":"2024-09-13T16:48:08.144121+0000","last_clean":"2024-09-13T16:48:08.144121+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:08.144121+0000","last_undegraded":"2024-09-13T16:48:08.144121+0000","last_fullsized":"2024-09-13T16:48:08.144121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174330+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T16:48:09.174330+0000","last_peered":"2024-09-13T16:48:09.174330+0000","last_clean":"2024-09-13T16:48:09.174330+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T16:48:09.174330+0000","last_undegraded":"2024-09-13T16:48:09.174330+0000","last_fullsized":"2024-09-13T16:48:09.174330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153996+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:08.153996+0000","last_peered":"2024-09-13T16:48:08.153996+0000","last_clean":"2024-09-13T16:48:08.153996+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:08.153996+0000","last_undegraded":"2024-09-13T16:48:08.153996+0000","last_fullsized":"2024-09-13T16:48:08.153996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161029+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T16:48:09.161029+0000","last_peered":"2024-09-13T16:48:09.161029+0000","last_clean":"2024-09-13T16:48:09.161029+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T16:48:09.161029+0000","last_undegraded":"2024-09-13T16:48:09.161029+0000","last_fullsized":"2024-09-13T16:48:09.161029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154547+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:08.154547+0000","last_peered":"2024-09-13T16:48:08.154547+0000","last_clean":"2024-09-13T16:48:08.154547+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:08.154547+0000","last_undegraded":"2024-09-13T16:48:08.154547+0000","last_fullsized":"2024-09-13T16:48:08.154547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174304+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T16:48:09.174304+0000","last_peered":"2024-09-13T16:48:09.174304+0000","last_clean":"2024-09-13T16:48:09.174304+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T16:48:09.174304+0000","last_undegraded":"2024-09-13T16:48:09.174304+0000","last_fullsized":"2024-09-13T16:48:09.174304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161339+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:08.161339+0000","last_peered":"2024-09-13T16:48:08.161339+0000","last_clean":"2024-09-13T16:48:08.161339+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:08.161339+0000","last_undegraded":"2024-09-13T16:48:08.161339+0000","last_fullsized":"2024-09-13T16:48:08.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173686+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T16:48:09.173686+0000","last_peered":"2024-09-13T16:48:09.173686+0000","last_clean":"2024-09-13T16:48:09.173686+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T16:48:09.173686+0000","last_undegraded":"2024-09-13T16:48:09.173686+0000","last_fullsized":"2024-09-13T16:48:09.173686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145104+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:08.145104+0000","last_peered":"2024-09-13T16:48:08.145104+0000","last_clean":"2024-09-13T16:48:08.145104+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:08.145104+0000","last_undegraded":"2024-09-13T16:48:08.145104+0000","last_fullsized":"2024-09-13T16:48:08.145104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175359+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T16:48:09.175359+0000","last_peered":"2024-09-13T16:48:09.175359+0000","last_clean":"2024-09-13T16:48:09.175359+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T16:48:09.175359+0000","last_undegraded":"2024-09-13T16:48:09.175359+0000","last_fullsized":"2024-09-13T16:48:09.175359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149320+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:08.149320+0000","last_peered":"2024-09-13T16:48:08.149320+0000","last_clean":"2024-09-13T16:48:08.149320+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:08.149320+0000","last_undegraded":"2024-09-13T16:48:08.149320+0000","last_fullsized":"2024-09-13T16:48:08.149320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173205+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T16:48:09.173205+0000","last_peered":"2024-09-13T16:48:09.173205+0000","last_clean":"2024-09-13T16:48:09.173205+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T16:48:09.173205+0000","last_undegraded":"2024-09-13T16:48:09.173205+0000","last_fullsized":"2024-09-13T16:48:09.173205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149001+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:08.149001+0000","last_peered":"2024-09-13T16:48:08.149001+0000","last_clean":"2024-09-13T16:48:08.149001+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:08.149001+0000","last_undegraded":"2024-09-13T16:48:08.149001+0000","last_fullsized":"2024-09-13T16:48:08.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158758+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T16:48:09.158758+0000","last_peered":"2024-09-13T16:48:09.158758+0000","last_clean":"2024-09-13T16:48:09.158758+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T16:48:09.158758+0000","last_undegraded":"2024-09-13T16:48:09.158758+0000","last_fullsized":"2024-09-13T16:48:09.158758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145105+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:08.145105+0000","last_peered":"2024-09-13T16:48:08.145105+0000","last_clean":"2024-09-13T16:48:08.145105+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:08.145105+0000","last_undegraded":"2024-09-13T16:48:08.145105+0000","last_fullsized":"2024-09-13T16:48:08.145105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177538+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T16:48:09.177538+0000","last_peered":"2024-09-13T16:48:09.177538+0000","last_clean":"2024-09-13T16:48:09.177538+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T16:48:09.177538+0000","last_undegraded":"2024-09-13T16:48:09.177538+0000","last_fullsized":"2024-09-13T16:48:09.177538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158985+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:08.158985+0000","last_peered":"2024-09-13T16:48:08.158985+0000","last_clean":"2024-09-13T16:48:08.158985+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:08.158985+0000","last_undegraded":"2024-09-13T16:48:08.158985+0000","last_fullsized":"2024-09-13T16:48:08.158985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177461+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T16:48:09.177461+0000","last_peered":"2024-09-13T16:48:09.177461+0000","last_clean":"2024-09-13T16:48:09.177461+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T16:48:09.177461+0000","last_undegraded":"2024-09-13T16:48:09.177461+0000","last_fullsized":"2024-09-13T16:48:09.177461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143902+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:08.143902+0000","last_peered":"2024-09-13T16:48:08.143902+0000","last_clean":"2024-09-13T16:48:08.143902+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:08.143902+0000","last_undegraded":"2024-09-13T16:48:08.143902+0000","last_fullsized":"2024-09-13T16:48:08.143902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159127+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T16:48:09.159127+0000","last_peered":"2024-09-13T16:48:09.159127+0000","last_clean":"2024-09-13T16:48:09.159127+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T16:48:09.159127+0000","last_undegraded":"2024-09-13T16:48:09.159127+0000","last_fullsized":"2024-09-13T16:48:09.159127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148988+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:08.148988+0000","last_peered":"2024-09-13T16:48:08.148988+0000","last_clean":"2024-09-13T16:48:08.148988+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:08.148988+0000","last_undegraded":"2024-09-13T16:48:08.148988+0000","last_fullsized":"2024-09-13T16:48:08.148988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177190+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T16:48:09.177190+0000","last_peered":"2024-09-13T16:48:09.177190+0000","last_clean":"2024-09-13T16:48:09.177190+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T16:48:09.177190+0000","last_undegraded":"2024-09-13T16:48:09.177190+0000","last_fullsized":"2024-09-13T16:48:09.177190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161327+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:08.161327+0000","last_peered":"2024-09-13T16:48:08.161327+0000","last_clean":"2024-09-13T16:48:08.161327+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:08.161327+0000","last_undegraded":"2024-09-13T16:48:08.161327+0000","last_fullsized":"2024-09-13T16:48:08.161327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148225+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:08.148225+0000","last_peered":"2024-09-13T16:48:08.148225+0000","last_clean":"2024-09-13T16:48:08.148225+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:08.148225+0000","last_undegraded":"2024-09-13T16:48:08.148225+0000","last_fullsized":"2024-09-13T16:48:08.148225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175099+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T16:48:09.175099+0000","last_peered":"2024-09-13T16:48:09.175099+0000","last_clean":"2024-09-13T16:48:09.175099+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T16:48:09.175099+0000","last_undegraded":"2024-09-13T16:48:09.175099+0000","last_fullsized":"2024-09-13T16:48:09.175099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179166+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:08.179166+0000","last_peered":"2024-09-13T16:48:08.179166+0000","last_clean":"2024-09-13T16:48:08.179166+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:08.179166+0000","last_undegraded":"2024-09-13T16:48:08.179166+0000","last_fullsized":"2024-09-13T16:48:08.179166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158364+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T16:48:09.158364+0000","last_peered":"2024-09-13T16:48:09.158364+0000","last_clean":"2024-09-13T16:48:09.158364+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T16:48:09.158364+0000","last_undegraded":"2024-09-13T16:48:09.158364+0000","last_fullsized":"2024-09-13T16:48:09.158364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155807+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:08.155807+0000","last_peered":"2024-09-13T16:48:08.155807+0000","last_clean":"2024-09-13T16:48:08.155807+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:08.155807+0000","last_undegraded":"2024-09-13T16:48:08.155807+0000","last_fullsized":"2024-09-13T16:48:08.155807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.172410+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T16:48:09.172410+0000","last_peered":"2024-09-13T16:48:09.172410+0000","last_clean":"2024-09-13T16:48:09.172410+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T16:48:09.172410+0000","last_undegraded":"2024-09-13T16:48:09.172410+0000","last_fullsized":"2024-09-13T16:48:09.172410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.157093+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:08.157093+0000","last_peered":"2024-09-13T16:48:08.157093+0000","last_clean":"2024-09-13T16:48:08.157093+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:08.157093+0000","last_undegraded":"2024-09-13T16:48:08.157093+0000","last_fullsized":"2024-09-13T16:48:08.157093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173535+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T16:48:09.173535+0000","last_peered":"2024-09-13T16:48:09.173535+0000","last_clean":"2024-09-13T16:48:09.173535+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T16:48:09.173535+0000","last_undegraded":"2024-09-13T16:48:09.173535+0000","last_fullsized":"2024-09-13T16:48:09.173535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158715+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:08.158715+0000","last_peered":"2024-09-13T16:48:08.158715+0000","last_clean":"2024-09-13T16:48:08.158715+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:08.158715+0000","last_undegraded":"2024-09-13T16:48:08.158715+0000","last_fullsized":"2024-09-13T16:48:08.158715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175097+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T16:48:09.175097+0000","last_peered":"2024-09-13T16:48:09.175097+0000","last_clean":"2024-09-13T16:48:09.175097+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T16:48:09.175097+0000","last_undegraded":"2024-09-13T16:48:09.175097+0000","last_fullsized":"2024-09-13T16:48:09.175097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147419+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:08.147419+0000","last_peered":"2024-09-13T16:48:08.147419+0000","last_clean":"2024-09-13T16:48:08.147419+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:08.147419+0000","last_undegraded":"2024-09-13T16:48:08.147419+0000","last_fullsized":"2024-09-13T16:48:08.147419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159429+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T16:48:09.159429+0000","last_peered":"2024-09-13T16:48:09.159429+0000","last_clean":"2024-09-13T16:48:09.159429+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T16:48:09.159429+0000","last_undegraded":"2024-09-13T16:48:09.159429+0000","last_fullsized":"2024-09-13T16:48:09.159429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161028+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:08.161028+0000","last_peered":"2024-09-13T16:48:08.161028+0000","last_clean":"2024-09-13T16:48:08.161028+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:08.161028+0000","last_undegraded":"2024-09-13T16:48:08.161028+0000","last_fullsized":"2024-09-13T16:48:08.161028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177095+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T16:48:09.177095+0000","last_peered":"2024-09-13T16:48:09.177095+0000","last_clean":"2024-09-13T16:48:09.177095+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T16:48:09.177095+0000","last_undegraded":"2024-09-13T16:48:09.177095+0000","last_fullsized":"2024-09-13T16:48:09.177095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143741+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:08.143741+0000","last_peered":"2024-09-13T16:48:08.143741+0000","last_clean":"2024-09-13T16:48:08.143741+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:08.143741+0000","last_undegraded":"2024-09-13T16:48:08.143741+0000","last_fullsized":"2024-09-13T16:48:08.143741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176397+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T16:48:09.176397+0000","last_peered":"2024-09-13T16:48:09.176397+0000","last_clean":"2024-09-13T16:48:09.176397+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T16:48:09.176397+0000","last_undegraded":"2024-09-13T16:48:09.176397+0000","last_fullsized":"2024-09-13T16:48:09.176397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186343+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:08.186343+0000","last_peered":"2024-09-13T16:48:08.186343+0000","last_clean":"2024-09-13T16:48:08.186343+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:08.186343+0000","last_undegraded":"2024-09-13T16:48:08.186343+0000","last_fullsized":"2024-09-13T16:48:08.186343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158832+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T16:48:09.158832+0000","last_peered":"2024-09-13T16:48:09.158832+0000","last_clean":"2024-09-13T16:48:09.158832+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T16:48:09.158832+0000","last_undegraded":"2024-09-13T16:48:09.158832+0000","last_fullsized":"2024-09-13T16:48:09.158832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149361+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:08.149361+0000","last_peered":"2024-09-13T16:48:08.149361+0000","last_clean":"2024-09-13T16:48:08.149361+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:08.149361+0000","last_undegraded":"2024-09-13T16:48:08.149361+0000","last_fullsized":"2024-09-13T16:48:08.149361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173784+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T16:48:09.173784+0000","last_peered":"2024-09-13T16:48:09.173784+0000","last_clean":"2024-09-13T16:48:09.173784+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T16:48:09.173784+0000","last_undegraded":"2024-09-13T16:48:09.173784+0000","last_fullsized":"2024-09-13T16:48:09.173784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148791+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:08.148791+0000","last_peered":"2024-09-13T16:48:08.148791+0000","last_clean":"2024-09-13T16:48:08.148791+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:08.148791+0000","last_undegraded":"2024-09-13T16:48:08.148791+0000","last_fullsized":"2024-09-13T16:48:08.148791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159787+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T16:48:09.159787+0000","last_peered":"2024-09-13T16:48:09.159787+0000","last_clean":"2024-09-13T16:48:09.159787+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T16:48:09.159787+0000","last_undegraded":"2024-09-13T16:48:09.159787+0000","last_fullsized":"2024-09-13T16:48:09.159787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166109+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:08.166109+0000","last_peered":"2024-09-13T16:48:08.166109+0000","last_clean":"2024-09-13T16:48:08.166109+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:08.166109+0000","last_undegraded":"2024-09-13T16:48:08.166109+0000","last_fullsized":"2024-09-13T16:48:08.166109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.178540+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T16:48:09.178540+0000","last_peered":"2024-09-13T16:48:09.178540+0000","last_clean":"2024-09-13T16:48:09.178540+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T16:48:09.178540+0000","last_undegraded":"2024-09-13T16:48:09.178540+0000","last_fullsized":"2024-09-13T16:48:09.178540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143140+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:08.143140+0000","last_peered":"2024-09-13T16:48:08.143140+0000","last_clean":"2024-09-13T16:48:08.143140+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:08.143140+0000","last_undegraded":"2024-09-13T16:48:08.143140+0000","last_fullsized":"2024-09-13T16:48:08.143140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.160931+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T16:48:09.160931+0000","last_peered":"2024-09-13T16:48:09.160931+0000","last_clean":"2024-09-13T16:48:09.160931+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T16:48:09.160931+0000","last_undegraded":"2024-09-13T16:48:09.160931+0000","last_fullsized":"2024-09-13T16:48:09.160931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167027+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:08.167027+0000","last_peered":"2024-09-13T16:48:08.167027+0000","last_clean":"2024-09-13T16:48:08.167027+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:08.167027+0000","last_undegraded":"2024-09-13T16:48:08.167027+0000","last_fullsized":"2024-09-13T16:48:08.167027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173532+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T16:48:09.173532+0000","last_peered":"2024-09-13T16:48:09.173532+0000","last_clean":"2024-09-13T16:48:09.173532+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T16:48:09.173532+0000","last_undegraded":"2024-09-13T16:48:09.173532+0000","last_fullsized":"2024-09-13T16:48:09.173532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148753+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:08.148753+0000","last_peered":"2024-09-13T16:48:08.148753+0000","last_clean":"2024-09-13T16:48:08.148753+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:08.148753+0000","last_undegraded":"2024-09-13T16:48:08.148753+0000","last_fullsized":"2024-09-13T16:48:08.148753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161898+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T16:48:09.161898+0000","last_peered":"2024-09-13T16:48:09.161898+0000","last_clean":"2024-09-13T16:48:09.161898+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T16:48:09.161898+0000","last_undegraded":"2024-09-13T16:48:09.161898+0000","last_fullsized":"2024-09-13T16:48:09.161898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160351+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:08.160351+0000","last_peered":"2024-09-13T16:48:08.160351+0000","last_clean":"2024-09-13T16:48:08.160351+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:08.160351+0000","last_undegraded":"2024-09-13T16:48:08.160351+0000","last_fullsized":"2024-09-13T16:48:08.160351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176220+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T16:48:09.176220+0000","last_peered":"2024-09-13T16:48:09.176220+0000","last_clean":"2024-09-13T16:48:09.176220+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T16:48:09.176220+0000","last_undegraded":"2024-09-13T16:48:09.176220+0000","last_fullsized":"2024-09-13T16:48:09.176220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151613+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:08.151613+0000","last_peered":"2024-09-13T16:48:08.151613+0000","last_clean":"2024-09-13T16:48:08.151613+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:08.151613+0000","last_undegraded":"2024-09-13T16:48:08.151613+0000","last_fullsized":"2024-09-13T16:48:08.151613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158220+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T16:48:09.158220+0000","last_peered":"2024-09-13T16:48:09.158220+0000","last_clean":"2024-09-13T16:48:09.158220+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T16:48:09.158220+0000","last_undegraded":"2024-09-13T16:48:09.158220+0000","last_fullsized":"2024-09-13T16:48:09.158220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.162959+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:08.162959+0000","last_peered":"2024-09-13T16:48:08.162959+0000","last_clean":"2024-09-13T16:48:08.162959+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:08.162959+0000","last_undegraded":"2024-09-13T16:48:08.162959+0000","last_fullsized":"2024-09-13T16:48:08.162959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173056+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T16:48:09.173056+0000","last_peered":"2024-09-13T16:48:09.173056+0000","last_clean":"2024-09-13T16:48:09.173056+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T16:48:09.173056+0000","last_undegraded":"2024-09-13T16:48:09.173056+0000","last_fullsized":"2024-09-13T16:48:09.173056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.150218+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:08.150218+0000","last_peered":"2024-09-13T16:48:08.150218+0000","last_clean":"2024-09-13T16:48:08.150218+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:08.150218+0000","last_undegraded":"2024-09-13T16:48:08.150218+0000","last_fullsized":"2024-09-13T16:48:08.150218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173906+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T16:48:09.173906+0000","last_peered":"2024-09-13T16:48:09.173906+0000","last_clean":"2024-09-13T16:48:09.173906+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T16:48:09.173906+0000","last_undegraded":"2024-09-13T16:48:09.173906+0000","last_fullsized":"2024-09-13T16:48:09.173906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166070+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:08.166070+0000","last_peered":"2024-09-13T16:48:08.166070+0000","last_clean":"2024-09-13T16:48:08.166070+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:08.166070+0000","last_undegraded":"2024-09-13T16:48:08.166070+0000","last_fullsized":"2024-09-13T16:48:08.166070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":690265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":101405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":99455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.367}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":282624,"data_stored":102375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":258048,"data_stored":99295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":100971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":690231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:20.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:20.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.608+0000 7feac84ec640 1 Processor -- start 2024-09-13T16:48:20.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.608+0000 7feac84ec640 1 -- start start 2024-09-13T16:48:20.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac84ec640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feac00c4260 0x7feac00c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac84ec640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feac0070b40 0x7feac015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac84ec640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac0157360 con 0x7feac00bcfb0 2024-09-13T16:48:20.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac84ec640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac00c80b0 con 0x7feac00c4260 2024-09-13T16:48:20.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac84ec640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac01575b0 con 0x7feac0070b40 2024-09-13T16:48:20.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5a87640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7feac00bcfb0 0x7feac0070430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57374/0 (socket says 172.21.15.39:57374) 2024-09-13T16:48:20.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5a87640 1 -- 172.21.15.39:0/467564836 learned_addr learned my addr 172.21.15.39:0/467564836 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:20.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac6288640 1 --2- 172.21.15.39:0/467564836 >> v2:172.21.15.84:3300/0 conn(0x7feac00c4260 0x7feac00c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac6a89640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feac0070b40 0x7feac015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099396750 0 0) 0x7feac0157360 con 0x7feac00bcfb0 2024-09-13T16:48:20.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa8003530 con 0x7feac00bcfb0 2024-09-13T16:48:20.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3315041331 0 0) 0x7feaa8003530 con 0x7feac00bcfb0 2024-09-13T16:48:20.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feac0070b40 msgr2=0x7feac015e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feac0070b40 0x7feac015e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 >> v2:172.21.15.84:3300/0 conn(0x7feac00c4260 msgr2=0x7feac00c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 --2- 172.21.15.39:0/467564836 >> v2:172.21.15.84:3300/0 conn(0x7feac00c4260 0x7feac00c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0157890 con 0x7feac00bcfb0 2024-09-13T16:48:20.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7feab4001680 con 0x7feac00bcfb0 2024-09-13T16:48:20.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 >> v1:172.21.15.39:6789/0 conn(0x7feac00bcfb0 legacy=0x7feac0070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feaa8003980 0x7feaa8003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa80042e0 0x7feaa80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa8004ec0 0x7feaa800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00c80b0 con 0x7feaa8004ec0 2024-09-13T16:48:20.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac01575b0 con 0x7feaa80042e0 2024-09-13T16:48:20.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa800bed0 con 0x7feaa8003980 2024-09-13T16:48:20.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac6a89640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa8004ec0 0x7feaa800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac5a87640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa80042e0 0x7feaa80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac6288640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feaa8003980 0x7feaa8003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac6a89640 1 -- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feaa8003980 msgr2=0x7feaa8003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac6a89640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feaa8003980 0x7feaa8003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac6a89640 1 -- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa80042e0 msgr2=0x7feaa80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac6a89640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feaa80042e0 0x7feaa80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.612+0000 7feac6a89640 1 -- 172.21.15.39:0/467564836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa8003530 con 0x7feaa8004ec0 2024-09-13T16:48:20.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6a89640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa8004ec0 0x7feaa800b990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feac00ca9b0 tx=0x7feab801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cf8de2399d55c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:20.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00c7330 con 0x7feaa8004ec0 2024-09-13T16:48:20.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab80a64e0 con 0x7feaa8004ec0 2024-09-13T16:48:20.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac5286640 1 -- 172.21.15.39:0/467564836 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab80a6890 con 0x7feaa8004ec0 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- 172.21.15.39:0/467564836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa8004ec0 msgr2=0x7feaa800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 --2- 172.21.15.39:0/467564836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa8004ec0 0x7feaa800b990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feac00ca9b0 tx=0x7feab801c9f0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6a89640 1 -- 172.21.15.39:0/467564836 reap_dead start 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- 172.21.15.39:0/467564836 shutdown_connections 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- 172.21.15.39:0/467564836 >> 172.21.15.39:0/467564836 conn(0x7feac0069e10 msgr2=0x7feac006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- 172.21.15.39:0/467564836 shutdown_connections 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- 172.21.15.39:0/467564836 wait complete. 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 Processor -- start 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- start start 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac014fa00 0x7feac014fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feac01ef6d0 0x7feac01584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feac01589f0 0x7feac0147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0154c10 con 0x7feac014fa00 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac0153920 con 0x7feac01ef6d0 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac0153b70 con 0x7feac01589f0 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6288640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac014fa00 0x7feac014fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6288640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac014fa00 0x7feac014fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58110/0 (socket says 172.21.15.39:58110) 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6288640 1 -- 172.21.15.39:0/41265925 learned_addr learned my addr 172.21.15.39:0/41265925 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6a89640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feac01589f0 0x7feac0147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac5a87640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feac01ef6d0 0x7feac01584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6288640 1 -- 172.21.15.39:0/41265925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feac01589f0 msgr2=0x7feac0147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6288640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feac01589f0 0x7feac0147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6288640 1 -- 172.21.15.39:0/41265925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feac01ef6d0 msgr2=0x7feac01584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6288640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feac01ef6d0 0x7feac01584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6288640 1 -- 172.21.15.39:0/41265925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0153e50 con 0x7feac014fa00 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac6288640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac014fa00 0x7feac014fde0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feab0007f60 tx=0x7feab0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feabeffd640 1 -- 172.21.15.39:0/41265925 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab000e030 con 0x7feac014fa00 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feabeffd640 1 -- 172.21.15.39:0/41265925 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab0016470 con 0x7feac014fa00 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac0154130 con 0x7feac014fa00 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feabeffd640 1 -- 172.21.15.39:0/41265925 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0016820 con 0x7feac014fa00 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.616+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac01546f0 con 0x7feac014fa00 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.620+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac01548b0 con 0x7feac014fa00 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.620+0000 7feabeffd640 1 -- 172.21.15.39:0/41265925 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feab0016b90 con 0x7feac014fa00 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.620+0000 7feabeffd640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaac066690 0x7feaac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.620+0000 7feabeffd640 1 -- 172.21.15.39:0/41265925 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7feab000f070 con 0x7feac014fa00 2024-09-13T16:48:20.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.620+0000 7feac5a87640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaac066690 0x7feaac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.620+0000 7feac5a87640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaac066690 0x7feaac068b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feab4001ec0 tx=0x7feab4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:20.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.624+0000 7feabeffd640 1 -- 172.21.15.39:0/41265925 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feab00526e0 con 0x7feac014fa00 2024-09-13T16:48:20.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.776+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feac00c4640 con 0x7feaac066690 2024-09-13T16:48:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.788+0000 7feabeffd640 1 -- 172.21.15.39:0/41265925 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+502938 (secure 0 0 0) 0x7feac00c4640 con 0x7feaac066690 2024-09-13T16:48:20.795 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:20.801 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaac066690 msgr2=0x7feaac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaac066690 0x7feaac068b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feab4001ec0 tx=0x7feab4009040 comp rx=0 tx=0).stop 2024-09-13T16:48:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac014fa00 msgr2=0x7feac014fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac014fa00 0x7feac014fde0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feab0007f60 tx=0x7feab0007f90 comp rx=0 tx=0).stop 2024-09-13T16:48:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 shutdown_connections 2024-09-13T16:48:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaac066690 0x7feaac068b50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feac01ef6d0 0x7feac01584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feac01589f0 0x7feac0147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 --2- 172.21.15.39:0/41265925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac014fa00 0x7feac014fde0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 >> 172.21.15.39:0/41265925 conn(0x7feac0069e10 msgr2=0x7feac014dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 shutdown_connections 2024-09-13T16:48:20.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.796+0000 7feac84ec640 1 -- 172.21.15.39:0/41265925 wait complete. 2024-09-13T16:48:20.815 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-13T16:48:19.563787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595092,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235816,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738904,"statfs":{"total":773094113280,"available":772852637696,"internally_reserved":0,"allocated":3350528,"data_stored":1983984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.408382"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174677+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T16:48:09.174677+0000","last_peered":"2024-09-13T16:48:09.174677+0000","last_clean":"2024-09-13T16:48:09.174677+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T16:48:09.174677+0000","last_undegraded":"2024-09-13T16:48:09.174677+0000","last_fullsized":"2024-09-13T16:48:09.174677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163864+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:08.163864+0000","last_peered":"2024-09-13T16:48:08.163864+0000","last_clean":"2024-09-13T16:48:08.163864+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:08.163864+0000","last_undegraded":"2024-09-13T16:48:08.163864+0000","last_fullsized":"2024-09-13T16:48:08.163864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146105+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:08.146105+0000","last_peered":"2024-09-13T16:48:08.146105+0000","last_clean":"2024-09-13T16:48:08.146105+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:08.146105+0000","last_undegraded":"2024-09-13T16:48:08.146105+0000","last_fullsized":"2024-09-13T16:48:08.146105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143176+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T16:48:08.143176+0000","last_peered":"2024-09-13T16:48:08.143176+0000","last_clean":"2024-09-13T16:48:08.143176+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T16:48:08.143176+0000","last_undegraded":"2024-09-13T16:48:08.143176+0000","last_fullsized":"2024-09-13T16:48:08.143176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173274+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T16:48:09.173274+0000","last_peered":"2024-09-13T16:48:09.173274+0000","last_clean":"2024-09-13T16:48:09.173274+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T16:48:09.173274+0000","last_undegraded":"2024-09-13T16:48:09.173274+0000","last_fullsized":"2024-09-13T16:48:09.173274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146246+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:08.146246+0000","last_peered":"2024-09-13T16:48:08.146246+0000","last_clean":"2024-09-13T16:48:08.146246+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:08.146246+0000","last_undegraded":"2024-09-13T16:48:08.146246+0000","last_fullsized":"2024-09-13T16:48:08.146246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163480+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:08.163480+0000","last_peered":"2024-09-13T16:48:08.163480+0000","last_clean":"2024-09-13T16:48:08.163480+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:08.163480+0000","last_undegraded":"2024-09-13T16:48:08.163480+0000","last_fullsized":"2024-09-13T16:48:08.163480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184157+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T16:48:08.184157+0000","last_peered":"2024-09-13T16:48:08.184157+0000","last_clean":"2024-09-13T16:48:08.184157+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T16:48:08.184157+0000","last_undegraded":"2024-09-13T16:48:08.184157+0000","last_fullsized":"2024-09-13T16:48:08.184157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173949+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T16:48:09.173949+0000","last_peered":"2024-09-13T16:48:09.173949+0000","last_clean":"2024-09-13T16:48:09.173949+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T16:48:09.173949+0000","last_undegraded":"2024-09-13T16:48:09.173949+0000","last_fullsized":"2024-09-13T16:48:09.173949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178923+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:08.178923+0000","last_peered":"2024-09-13T16:48:08.178923+0000","last_clean":"2024-09-13T16:48:08.178923+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:08.178923+0000","last_undegraded":"2024-09-13T16:48:08.178923+0000","last_fullsized":"2024-09-13T16:48:08.178923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151847+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:08.151847+0000","last_peered":"2024-09-13T16:48:08.151847+0000","last_clean":"2024-09-13T16:48:08.151847+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:08.151847+0000","last_undegraded":"2024-09-13T16:48:08.151847+0000","last_fullsized":"2024-09-13T16:48:08.151847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208404+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T16:48:08.208404+0000","last_peered":"2024-09-13T16:48:08.208404+0000","last_clean":"2024-09-13T16:48:08.208404+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T16:48:08.208404+0000","last_undegraded":"2024-09-13T16:48:08.208404+0000","last_fullsized":"2024-09-13T16:48:08.208404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159591+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T16:48:09.159591+0000","last_peered":"2024-09-13T16:48:09.159591+0000","last_clean":"2024-09-13T16:48:09.159591+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T16:48:09.159591+0000","last_undegraded":"2024-09-13T16:48:09.159591+0000","last_fullsized":"2024-09-13T16:48:09.159591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165397+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:08.165397+0000","last_peered":"2024-09-13T16:48:08.165397+0000","last_clean":"2024-09-13T16:48:08.165397+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:08.165397+0000","last_undegraded":"2024-09-13T16:48:08.165397+0000","last_fullsized":"2024-09-13T16:48:08.165397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153028+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:08.153028+0000","last_peered":"2024-09-13T16:48:08.153028+0000","last_clean":"2024-09-13T16:48:08.153028+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:08.153028+0000","last_undegraded":"2024-09-13T16:48:08.153028+0000","last_fullsized":"2024-09-13T16:48:08.153028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.203658+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T16:48:08.203658+0000","last_peered":"2024-09-13T16:48:08.203658+0000","last_clean":"2024-09-13T16:48:08.203658+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T16:48:08.203658+0000","last_undegraded":"2024-09-13T16:48:08.203658+0000","last_fullsized":"2024-09-13T16:48:08.203658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173632+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T16:48:09.173632+0000","last_peered":"2024-09-13T16:48:09.173632+0000","last_clean":"2024-09-13T16:48:09.173632+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T16:48:09.173632+0000","last_undegraded":"2024-09-13T16:48:09.173632+0000","last_fullsized":"2024-09-13T16:48:09.173632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146884+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:08.146884+0000","last_peered":"2024-09-13T16:48:08.146884+0000","last_clean":"2024-09-13T16:48:08.146884+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:08.146884+0000","last_undegraded":"2024-09-13T16:48:08.146884+0000","last_fullsized":"2024-09-13T16:48:08.146884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160788+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:08.160788+0000","last_peered":"2024-09-13T16:48:08.160788+0000","last_clean":"2024-09-13T16:48:08.160788+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:08.160788+0000","last_undegraded":"2024-09-13T16:48:08.160788+0000","last_fullsized":"2024-09-13T16:48:08.160788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.206623+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T16:48:08.206623+0000","last_peered":"2024-09-13T16:48:08.206623+0000","last_clean":"2024-09-13T16:48:08.206623+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T16:48:08.206623+0000","last_undegraded":"2024-09-13T16:48:08.206623+0000","last_fullsized":"2024-09-13T16:48:08.206623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159080+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T16:48:09.159080+0000","last_peered":"2024-09-13T16:48:09.159080+0000","last_clean":"2024-09-13T16:48:09.159080+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T16:48:09.159080+0000","last_undegraded":"2024-09-13T16:48:09.159080+0000","last_fullsized":"2024-09-13T16:48:09.159080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144345+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:08.144345+0000","last_peered":"2024-09-13T16:48:08.144345+0000","last_clean":"2024-09-13T16:48:08.144345+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:08.144345+0000","last_undegraded":"2024-09-13T16:48:08.144345+0000","last_fullsized":"2024-09-13T16:48:08.144345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147467+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:08.147467+0000","last_peered":"2024-09-13T16:48:08.147467+0000","last_clean":"2024-09-13T16:48:08.147467+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:08.147467+0000","last_undegraded":"2024-09-13T16:48:08.147467+0000","last_fullsized":"2024-09-13T16:48:08.147467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184081+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T16:48:08.184081+0000","last_peered":"2024-09-13T16:48:08.184081+0000","last_clean":"2024-09-13T16:48:08.184081+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T16:48:08.184081+0000","last_undegraded":"2024-09-13T16:48:08.184081+0000","last_fullsized":"2024-09-13T16:48:08.184081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143476+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:08.143476+0000","last_peered":"2024-09-13T16:48:08.143476+0000","last_clean":"2024-09-13T16:48:08.143476+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:08.143476+0000","last_undegraded":"2024-09-13T16:48:08.143476+0000","last_fullsized":"2024-09-13T16:48:08.143476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147565+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:08.147565+0000","last_peered":"2024-09-13T16:48:08.147565+0000","last_clean":"2024-09-13T16:48:08.147565+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:08.147565+0000","last_undegraded":"2024-09-13T16:48:08.147565+0000","last_fullsized":"2024-09-13T16:48:08.147565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174007+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T16:48:08.174007+0000","last_peered":"2024-09-13T16:48:08.174007+0000","last_clean":"2024-09-13T16:48:08.174007+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T16:48:08.174007+0000","last_undegraded":"2024-09-13T16:48:08.174007+0000","last_fullsized":"2024-09-13T16:48:08.174007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162482+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T16:48:09.162482+0000","last_peered":"2024-09-13T16:48:09.162482+0000","last_clean":"2024-09-13T16:48:09.162482+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T16:48:09.162482+0000","last_undegraded":"2024-09-13T16:48:09.162482+0000","last_fullsized":"2024-09-13T16:48:09.162482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186454+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:08.186454+0000","last_peered":"2024-09-13T16:48:08.186454+0000","last_clean":"2024-09-13T16:48:08.186454+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:08.186454+0000","last_undegraded":"2024-09-13T16:48:08.186454+0000","last_fullsized":"2024-09-13T16:48:08.186454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160985+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:08.160985+0000","last_peered":"2024-09-13T16:48:08.160985+0000","last_clean":"2024-09-13T16:48:08.160985+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:08.160985+0000","last_undegraded":"2024-09-13T16:48:08.160985+0000","last_fullsized":"2024-09-13T16:48:08.160985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208189+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T16:48:08.208189+0000","last_peered":"2024-09-13T16:48:08.208189+0000","last_clean":"2024-09-13T16:48:08.208189+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T16:48:08.208189+0000","last_undegraded":"2024-09-13T16:48:08.208189+0000","last_fullsized":"2024-09-13T16:48:08.208189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176059+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T16:48:09.176059+0000","last_peered":"2024-09-13T16:48:09.176059+0000","last_clean":"2024-09-13T16:48:09.176059+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T16:48:09.176059+0000","last_undegraded":"2024-09-13T16:48:09.176059+0000","last_fullsized":"2024-09-13T16:48:09.176059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163768+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:08.163768+0000","last_peered":"2024-09-13T16:48:08.163768+0000","last_clean":"2024-09-13T16:48:08.163768+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:08.163768+0000","last_undegraded":"2024-09-13T16:48:08.163768+0000","last_fullsized":"2024-09-13T16:48:08.163768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148721+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:08.148721+0000","last_peered":"2024-09-13T16:48:08.148721+0000","last_clean":"2024-09-13T16:48:08.148721+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:08.148721+0000","last_undegraded":"2024-09-13T16:48:08.148721+0000","last_fullsized":"2024-09-13T16:48:08.148721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185371+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T16:48:08.185371+0000","last_peered":"2024-09-13T16:48:08.185371+0000","last_clean":"2024-09-13T16:48:08.185371+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T16:48:08.185371+0000","last_undegraded":"2024-09-13T16:48:08.185371+0000","last_fullsized":"2024-09-13T16:48:08.185371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174416+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T16:48:09.174416+0000","last_peered":"2024-09-13T16:48:09.174416+0000","last_clean":"2024-09-13T16:48:09.174416+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T16:48:09.174416+0000","last_undegraded":"2024-09-13T16:48:09.174416+0000","last_fullsized":"2024-09-13T16:48:09.174416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145183+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:08.145183+0000","last_peered":"2024-09-13T16:48:08.145183+0000","last_clean":"2024-09-13T16:48:08.145183+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:08.145183+0000","last_undegraded":"2024-09-13T16:48:08.145183+0000","last_fullsized":"2024-09-13T16:48:08.145183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152363+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:08.152363+0000","last_peered":"2024-09-13T16:48:08.152363+0000","last_clean":"2024-09-13T16:48:08.152363+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:08.152363+0000","last_undegraded":"2024-09-13T16:48:08.152363+0000","last_fullsized":"2024-09-13T16:48:08.152363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143237+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T16:48:08.143237+0000","last_peered":"2024-09-13T16:48:08.143237+0000","last_clean":"2024-09-13T16:48:08.143237+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T16:48:08.143237+0000","last_undegraded":"2024-09-13T16:48:08.143237+0000","last_fullsized":"2024-09-13T16:48:08.143237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158858+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T16:48:09.158858+0000","last_peered":"2024-09-13T16:48:09.158858+0000","last_clean":"2024-09-13T16:48:09.158858+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T16:48:09.158858+0000","last_undegraded":"2024-09-13T16:48:09.158858+0000","last_fullsized":"2024-09-13T16:48:09.158858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148446+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:08.148446+0000","last_peered":"2024-09-13T16:48:08.148446+0000","last_clean":"2024-09-13T16:48:08.148446+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:08.148446+0000","last_undegraded":"2024-09-13T16:48:08.148446+0000","last_fullsized":"2024-09-13T16:48:08.148446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179426+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:08.179426+0000","last_peered":"2024-09-13T16:48:08.179426+0000","last_clean":"2024-09-13T16:48:08.179426+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:08.179426+0000","last_undegraded":"2024-09-13T16:48:08.179426+0000","last_fullsized":"2024-09-13T16:48:08.179426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161207+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T16:48:08.161207+0000","last_peered":"2024-09-13T16:48:08.161207+0000","last_clean":"2024-09-13T16:48:08.161207+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T16:48:08.161207+0000","last_undegraded":"2024-09-13T16:48:08.161207+0000","last_fullsized":"2024-09-13T16:48:08.161207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173006+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T16:48:09.173006+0000","last_peered":"2024-09-13T16:48:09.173006+0000","last_clean":"2024-09-13T16:48:09.173006+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T16:48:09.173006+0000","last_undegraded":"2024-09-13T16:48:09.173006+0000","last_fullsized":"2024-09-13T16:48:09.173006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153222+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:08.153222+0000","last_peered":"2024-09-13T16:48:08.153222+0000","last_clean":"2024-09-13T16:48:08.153222+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:08.153222+0000","last_undegraded":"2024-09-13T16:48:08.153222+0000","last_fullsized":"2024-09-13T16:48:08.153222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148522+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:08.148522+0000","last_peered":"2024-09-13T16:48:08.148522+0000","last_clean":"2024-09-13T16:48:08.148522+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:08.148522+0000","last_undegraded":"2024-09-13T16:48:08.148522+0000","last_fullsized":"2024-09-13T16:48:08.148522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146684+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T16:48:08.146684+0000","last_peered":"2024-09-13T16:48:08.146684+0000","last_clean":"2024-09-13T16:48:08.146684+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T16:48:08.146684+0000","last_undegraded":"2024-09-13T16:48:08.146684+0000","last_fullsized":"2024-09-13T16:48:08.146684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174867+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T16:48:09.174867+0000","last_peered":"2024-09-13T16:48:09.174867+0000","last_clean":"2024-09-13T16:48:09.174867+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T16:48:09.174867+0000","last_undegraded":"2024-09-13T16:48:09.174867+0000","last_fullsized":"2024-09-13T16:48:09.174867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185855+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:08.185855+0000","last_peered":"2024-09-13T16:48:08.185855+0000","last_clean":"2024-09-13T16:48:08.185855+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:08.185855+0000","last_undegraded":"2024-09-13T16:48:08.185855+0000","last_fullsized":"2024-09-13T16:48:08.185855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149466+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:08.149466+0000","last_peered":"2024-09-13T16:48:08.149466+0000","last_clean":"2024-09-13T16:48:08.149466+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:08.149466+0000","last_undegraded":"2024-09-13T16:48:08.149466+0000","last_fullsized":"2024-09-13T16:48:08.149466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148372+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T16:48:08.148372+0000","last_peered":"2024-09-13T16:48:08.148372+0000","last_clean":"2024-09-13T16:48:08.148372+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T16:48:08.148372+0000","last_undegraded":"2024-09-13T16:48:08.148372+0000","last_fullsized":"2024-09-13T16:48:08.148372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159186+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T16:48:09.159186+0000","last_peered":"2024-09-13T16:48:09.159186+0000","last_clean":"2024-09-13T16:48:09.159186+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T16:48:09.159186+0000","last_undegraded":"2024-09-13T16:48:09.159186+0000","last_fullsized":"2024-09-13T16:48:09.159186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146860+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:08.146860+0000","last_peered":"2024-09-13T16:48:08.146860+0000","last_clean":"2024-09-13T16:48:08.146860+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:08.146860+0000","last_undegraded":"2024-09-13T16:48:08.146860+0000","last_fullsized":"2024-09-13T16:48:08.146860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154111+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:08.154111+0000","last_peered":"2024-09-13T16:48:08.154111+0000","last_clean":"2024-09-13T16:48:08.154111+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:08.154111+0000","last_undegraded":"2024-09-13T16:48:08.154111+0000","last_fullsized":"2024-09-13T16:48:08.154111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175388+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T16:48:08.175388+0000","last_peered":"2024-09-13T16:48:08.175388+0000","last_clean":"2024-09-13T16:48:08.175388+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T16:48:08.175388+0000","last_undegraded":"2024-09-13T16:48:08.175388+0000","last_fullsized":"2024-09-13T16:48:08.175388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162556+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T16:48:09.162556+0000","last_peered":"2024-09-13T16:48:09.162556+0000","last_clean":"2024-09-13T16:48:09.162556+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T16:48:09.162556+0000","last_undegraded":"2024-09-13T16:48:09.162556+0000","last_fullsized":"2024-09-13T16:48:09.162556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160990+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:08.160990+0000","last_peered":"2024-09-13T16:48:08.160990+0000","last_clean":"2024-09-13T16:48:08.160990+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:08.160990+0000","last_undegraded":"2024-09-13T16:48:08.160990+0000","last_fullsized":"2024-09-13T16:48:08.160990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161254+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:08.161254+0000","last_peered":"2024-09-13T16:48:08.161254+0000","last_clean":"2024-09-13T16:48:08.161254+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:08.161254+0000","last_undegraded":"2024-09-13T16:48:08.161254+0000","last_fullsized":"2024-09-13T16:48:08.161254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175218+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T16:48:09.175218+0000","last_peered":"2024-09-13T16:48:09.175218+0000","last_clean":"2024-09-13T16:48:09.175218+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T16:48:09.175218+0000","last_undegraded":"2024-09-13T16:48:09.175218+0000","last_fullsized":"2024-09-13T16:48:09.175218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175463+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T16:48:09.175463+0000","last_peered":"2024-09-13T16:48:09.175463+0000","last_clean":"2024-09-13T16:48:09.175463+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T16:48:09.175463+0000","last_undegraded":"2024-09-13T16:48:09.175463+0000","last_fullsized":"2024-09-13T16:48:09.175463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143792+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:08.143792+0000","last_peered":"2024-09-13T16:48:08.143792+0000","last_clean":"2024-09-13T16:48:08.143792+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:08.143792+0000","last_undegraded":"2024-09-13T16:48:08.143792+0000","last_fullsized":"2024-09-13T16:48:08.143792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148284+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:08.148284+0000","last_peered":"2024-09-13T16:48:08.148284+0000","last_clean":"2024-09-13T16:48:08.148284+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:08.148284+0000","last_undegraded":"2024-09-13T16:48:08.148284+0000","last_fullsized":"2024-09-13T16:48:08.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161478+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:08.161478+0000","last_peered":"2024-09-13T16:48:08.161478+0000","last_clean":"2024-09-13T16:48:08.161478+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:08.161478+0000","last_undegraded":"2024-09-13T16:48:08.161478+0000","last_fullsized":"2024-09-13T16:48:08.161478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148905+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T16:48:08.148905+0000","last_peered":"2024-09-13T16:48:08.148905+0000","last_clean":"2024-09-13T16:48:08.148905+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T16:48:08.148905+0000","last_undegraded":"2024-09-13T16:48:08.148905+0000","last_fullsized":"2024-09-13T16:48:08.148905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158934+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T16:48:09.158934+0000","last_peered":"2024-09-13T16:48:09.158934+0000","last_clean":"2024-09-13T16:48:09.158934+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T16:48:09.158934+0000","last_undegraded":"2024-09-13T16:48:09.158934+0000","last_fullsized":"2024-09-13T16:48:09.158934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178617+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:08.178617+0000","last_peered":"2024-09-13T16:48:08.178617+0000","last_clean":"2024-09-13T16:48:08.178617+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:08.178617+0000","last_undegraded":"2024-09-13T16:48:08.178617+0000","last_fullsized":"2024-09-13T16:48:08.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155913+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:08.155913+0000","last_peered":"2024-09-13T16:48:08.155913+0000","last_clean":"2024-09-13T16:48:08.155913+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:08.155913+0000","last_undegraded":"2024-09-13T16:48:08.155913+0000","last_fullsized":"2024-09-13T16:48:08.155913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146319+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T16:48:08.146319+0000","last_peered":"2024-09-13T16:48:08.146319+0000","last_clean":"2024-09-13T16:48:08.146319+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T16:48:08.146319+0000","last_undegraded":"2024-09-13T16:48:08.146319+0000","last_fullsized":"2024-09-13T16:48:08.146319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158609+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T16:48:09.158609+0000","last_peered":"2024-09-13T16:48:09.158609+0000","last_clean":"2024-09-13T16:48:09.158609+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T16:48:09.158609+0000","last_undegraded":"2024-09-13T16:48:09.158609+0000","last_fullsized":"2024-09-13T16:48:09.158609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152057+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:08.152057+0000","last_peered":"2024-09-13T16:48:08.152057+0000","last_clean":"2024-09-13T16:48:08.152057+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:08.152057+0000","last_undegraded":"2024-09-13T16:48:08.152057+0000","last_fullsized":"2024-09-13T16:48:08.152057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149487+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:08.149487+0000","last_peered":"2024-09-13T16:48:08.149487+0000","last_clean":"2024-09-13T16:48:08.149487+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:08.149487+0000","last_undegraded":"2024-09-13T16:48:08.149487+0000","last_fullsized":"2024-09-13T16:48:08.149487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175317+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T16:48:08.175317+0000","last_peered":"2024-09-13T16:48:08.175317+0000","last_clean":"2024-09-13T16:48:08.175317+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T16:48:08.175317+0000","last_undegraded":"2024-09-13T16:48:08.175317+0000","last_fullsized":"2024-09-13T16:48:08.175317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173209+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T16:48:09.173209+0000","last_peered":"2024-09-13T16:48:09.173209+0000","last_clean":"2024-09-13T16:48:09.173209+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T16:48:09.173209+0000","last_undegraded":"2024-09-13T16:48:09.173209+0000","last_fullsized":"2024-09-13T16:48:09.173209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.177954+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:08.177954+0000","last_peered":"2024-09-13T16:48:08.177954+0000","last_clean":"2024-09-13T16:48:08.177954+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:08.177954+0000","last_undegraded":"2024-09-13T16:48:08.177954+0000","last_fullsized":"2024-09-13T16:48:08.177954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155936+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:08.155936+0000","last_peered":"2024-09-13T16:48:08.155936+0000","last_clean":"2024-09-13T16:48:08.155936+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:08.155936+0000","last_undegraded":"2024-09-13T16:48:08.155936+0000","last_fullsized":"2024-09-13T16:48:08.155936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158692+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T16:48:08.158692+0000","last_peered":"2024-09-13T16:48:08.158692+0000","last_clean":"2024-09-13T16:48:08.158692+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T16:48:08.158692+0000","last_undegraded":"2024-09-13T16:48:08.158692+0000","last_fullsized":"2024-09-13T16:48:08.158692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158034+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T16:48:09.158034+0000","last_peered":"2024-09-13T16:48:09.158034+0000","last_clean":"2024-09-13T16:48:09.158034+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T16:48:09.158034+0000","last_undegraded":"2024-09-13T16:48:09.158034+0000","last_fullsized":"2024-09-13T16:48:09.158034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160811+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:08.160811+0000","last_peered":"2024-09-13T16:48:08.160811+0000","last_clean":"2024-09-13T16:48:08.160811+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:08.160811+0000","last_undegraded":"2024-09-13T16:48:08.160811+0000","last_fullsized":"2024-09-13T16:48:08.160811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.192267+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:08.192267+0000","last_peered":"2024-09-13T16:48:08.192267+0000","last_clean":"2024-09-13T16:48:08.192267+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:08.192267+0000","last_undegraded":"2024-09-13T16:48:08.192267+0000","last_fullsized":"2024-09-13T16:48:08.192267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185117+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T16:48:08.185117+0000","last_peered":"2024-09-13T16:48:08.185117+0000","last_clean":"2024-09-13T16:48:08.185117+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T16:48:08.185117+0000","last_undegraded":"2024-09-13T16:48:08.185117+0000","last_fullsized":"2024-09-13T16:48:08.185117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177888+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T16:48:09.177888+0000","last_peered":"2024-09-13T16:48:09.177888+0000","last_clean":"2024-09-13T16:48:09.177888+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T16:48:09.177888+0000","last_undegraded":"2024-09-13T16:48:09.177888+0000","last_fullsized":"2024-09-13T16:48:09.177888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167237+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:08.167237+0000","last_peered":"2024-09-13T16:48:08.167237+0000","last_clean":"2024-09-13T16:48:08.167237+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:08.167237+0000","last_undegraded":"2024-09-13T16:48:08.167237+0000","last_fullsized":"2024-09-13T16:48:08.167237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156043+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:08.156043+0000","last_peered":"2024-09-13T16:48:08.156043+0000","last_clean":"2024-09-13T16:48:08.156043+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:08.156043+0000","last_undegraded":"2024-09-13T16:48:08.156043+0000","last_fullsized":"2024-09-13T16:48:08.156043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163690+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T16:48:08.163690+0000","last_peered":"2024-09-13T16:48:08.163690+0000","last_clean":"2024-09-13T16:48:08.163690+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T16:48:08.163690+0000","last_undegraded":"2024-09-13T16:48:08.163690+0000","last_fullsized":"2024-09-13T16:48:08.163690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162212+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T16:48:09.162212+0000","last_peered":"2024-09-13T16:48:09.162212+0000","last_clean":"2024-09-13T16:48:09.162212+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T16:48:09.162212+0000","last_undegraded":"2024-09-13T16:48:09.162212+0000","last_fullsized":"2024-09-13T16:48:09.162212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144923+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:08.144923+0000","last_peered":"2024-09-13T16:48:08.144923+0000","last_clean":"2024-09-13T16:48:08.144923+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:08.144923+0000","last_undegraded":"2024-09-13T16:48:08.144923+0000","last_fullsized":"2024-09-13T16:48:08.144923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145599+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:08.145599+0000","last_peered":"2024-09-13T16:48:08.145599+0000","last_clean":"2024-09-13T16:48:08.145599+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:08.145599+0000","last_undegraded":"2024-09-13T16:48:08.145599+0000","last_fullsized":"2024-09-13T16:48:08.145599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145022+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T16:48:08.145022+0000","last_peered":"2024-09-13T16:48:08.145022+0000","last_clean":"2024-09-13T16:48:08.145022+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T16:48:08.145022+0000","last_undegraded":"2024-09-13T16:48:08.145022+0000","last_fullsized":"2024-09-13T16:48:08.145022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161391+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T16:48:09.161391+0000","last_peered":"2024-09-13T16:48:09.161391+0000","last_clean":"2024-09-13T16:48:09.161391+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T16:48:09.161391+0000","last_undegraded":"2024-09-13T16:48:09.161391+0000","last_fullsized":"2024-09-13T16:48:09.161391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160556+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:08.160556+0000","last_peered":"2024-09-13T16:48:08.160556+0000","last_clean":"2024-09-13T16:48:08.160556+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:08.160556+0000","last_undegraded":"2024-09-13T16:48:08.160556+0000","last_fullsized":"2024-09-13T16:48:08.160556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149235+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:08.149235+0000","last_peered":"2024-09-13T16:48:08.149235+0000","last_clean":"2024-09-13T16:48:08.149235+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:08.149235+0000","last_undegraded":"2024-09-13T16:48:08.149235+0000","last_fullsized":"2024-09-13T16:48:08.149235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"23'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.209853+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T16:48:08.209853+0000","last_peered":"2024-09-13T16:48:08.209853+0000","last_clean":"2024-09-13T16:48:08.209853+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T16:48:08.209853+0000","last_undegraded":"2024-09-13T16:48:08.209853+0000","last_fullsized":"2024-09-13T16:48:08.209853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161110+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:08.161110+0000","last_peered":"2024-09-13T16:48:08.161110+0000","last_clean":"2024-09-13T16:48:08.161110+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:08.161110+0000","last_undegraded":"2024-09-13T16:48:08.161110+0000","last_fullsized":"2024-09-13T16:48:08.161110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161421+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:08.161421+0000","last_peered":"2024-09-13T16:48:08.161421+0000","last_clean":"2024-09-13T16:48:08.161421+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:08.161421+0000","last_undegraded":"2024-09-13T16:48:08.161421+0000","last_fullsized":"2024-09-13T16:48:08.161421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.188902+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T16:48:08.188902+0000","last_peered":"2024-09-13T16:48:08.188902+0000","last_clean":"2024-09-13T16:48:08.188902+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T16:48:08.188902+0000","last_undegraded":"2024-09-13T16:48:08.188902+0000","last_fullsized":"2024-09-13T16:48:08.188902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165383+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:08.165383+0000","last_peered":"2024-09-13T16:48:08.165383+0000","last_clean":"2024-09-13T16:48:08.165383+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:08.165383+0000","last_undegraded":"2024-09-13T16:48:08.165383+0000","last_fullsized":"2024-09-13T16:48:08.165383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148543+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:08.148543+0000","last_peered":"2024-09-13T16:48:08.148543+0000","last_clean":"2024-09-13T16:48:08.148543+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:08.148543+0000","last_undegraded":"2024-09-13T16:48:08.148543+0000","last_fullsized":"2024-09-13T16:48:08.148543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146582+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T16:48:08.146582+0000","last_peered":"2024-09-13T16:48:08.146582+0000","last_clean":"2024-09-13T16:48:08.146582+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T16:48:08.146582+0000","last_undegraded":"2024-09-13T16:48:08.146582+0000","last_fullsized":"2024-09-13T16:48:08.146582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173477+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T16:48:09.173477+0000","last_peered":"2024-09-13T16:48:09.173477+0000","last_clean":"2024-09-13T16:48:09.173477+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T16:48:09.173477+0000","last_undegraded":"2024-09-13T16:48:09.173477+0000","last_fullsized":"2024-09-13T16:48:09.173477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.190852+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:08.190852+0000","last_peered":"2024-09-13T16:48:08.190852+0000","last_clean":"2024-09-13T16:48:08.190852+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:08.190852+0000","last_undegraded":"2024-09-13T16:48:08.190852+0000","last_fullsized":"2024-09-13T16:48:08.190852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149403+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:08.149403+0000","last_peered":"2024-09-13T16:48:08.149403+0000","last_clean":"2024-09-13T16:48:08.149403+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:08.149403+0000","last_undegraded":"2024-09-13T16:48:08.149403+0000","last_fullsized":"2024-09-13T16:48:08.149403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208558+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T16:48:08.208558+0000","last_peered":"2024-09-13T16:48:08.208558+0000","last_clean":"2024-09-13T16:48:08.208558+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T16:48:08.208558+0000","last_undegraded":"2024-09-13T16:48:08.208558+0000","last_fullsized":"2024-09-13T16:48:08.208558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158601+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T16:48:09.158601+0000","last_peered":"2024-09-13T16:48:09.158601+0000","last_clean":"2024-09-13T16:48:09.158601+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T16:48:09.158601+0000","last_undegraded":"2024-09-13T16:48:09.158601+0000","last_fullsized":"2024-09-13T16:48:09.158601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163596+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:08.163596+0000","last_peered":"2024-09-13T16:48:08.163596+0000","last_clean":"2024-09-13T16:48:08.163596+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:08.163596+0000","last_undegraded":"2024-09-13T16:48:08.163596+0000","last_fullsized":"2024-09-13T16:48:08.163596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156189+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:08.156189+0000","last_peered":"2024-09-13T16:48:08.156189+0000","last_clean":"2024-09-13T16:48:08.156189+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:08.156189+0000","last_undegraded":"2024-09-13T16:48:08.156189+0000","last_fullsized":"2024-09-13T16:48:08.156189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174258+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T16:48:08.174258+0000","last_peered":"2024-09-13T16:48:08.174258+0000","last_clean":"2024-09-13T16:48:08.174258+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T16:48:08.174258+0000","last_undegraded":"2024-09-13T16:48:08.174258+0000","last_fullsized":"2024-09-13T16:48:08.174258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173305+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T16:48:09.173305+0000","last_peered":"2024-09-13T16:48:09.173305+0000","last_clean":"2024-09-13T16:48:09.173305+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T16:48:09.173305+0000","last_undegraded":"2024-09-13T16:48:09.173305+0000","last_fullsized":"2024-09-13T16:48:09.173305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155963+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:08.155963+0000","last_peered":"2024-09-13T16:48:08.155963+0000","last_clean":"2024-09-13T16:48:08.155963+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:08.155963+0000","last_undegraded":"2024-09-13T16:48:08.155963+0000","last_fullsized":"2024-09-13T16:48:08.155963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148500+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:08.148500+0000","last_peered":"2024-09-13T16:48:08.148500+0000","last_clean":"2024-09-13T16:48:08.148500+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:08.148500+0000","last_undegraded":"2024-09-13T16:48:08.148500+0000","last_fullsized":"2024-09-13T16:48:08.148500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143144+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T16:48:08.143144+0000","last_peered":"2024-09-13T16:48:08.143144+0000","last_clean":"2024-09-13T16:48:08.143144+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T16:48:08.143144+0000","last_undegraded":"2024-09-13T16:48:08.143144+0000","last_fullsized":"2024-09-13T16:48:08.143144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161066+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T16:48:09.161066+0000","last_peered":"2024-09-13T16:48:09.161066+0000","last_clean":"2024-09-13T16:48:09.161066+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T16:48:09.161066+0000","last_undegraded":"2024-09-13T16:48:09.161066+0000","last_fullsized":"2024-09-13T16:48:09.161066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186757+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T16:48:08.186757+0000","last_peered":"2024-09-13T16:48:08.186757+0000","last_clean":"2024-09-13T16:48:08.186757+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T16:48:08.186757+0000","last_undegraded":"2024-09-13T16:48:08.186757+0000","last_fullsized":"2024-09-13T16:48:08.186757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147291+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:08.147291+0000","last_peered":"2024-09-13T16:48:08.147291+0000","last_clean":"2024-09-13T16:48:08.147291+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:08.147291+0000","last_undegraded":"2024-09-13T16:48:08.147291+0000","last_fullsized":"2024-09-13T16:48:08.147291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158941+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:08.158941+0000","last_peered":"2024-09-13T16:48:08.158941+0000","last_clean":"2024-09-13T16:48:08.158941+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:08.158941+0000","last_undegraded":"2024-09-13T16:48:08.158941+0000","last_fullsized":"2024-09-13T16:48:08.158941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175068+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T16:48:09.175068+0000","last_peered":"2024-09-13T16:48:09.175068+0000","last_clean":"2024-09-13T16:48:09.175068+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T16:48:09.175068+0000","last_undegraded":"2024-09-13T16:48:09.175068+0000","last_fullsized":"2024-09-13T16:48:09.175068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.182997+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T16:48:08.182997+0000","last_peered":"2024-09-13T16:48:08.182997+0000","last_clean":"2024-09-13T16:48:08.182997+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T16:48:08.182997+0000","last_undegraded":"2024-09-13T16:48:08.182997+0000","last_fullsized":"2024-09-13T16:48:08.182997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143381+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:08.143381+0000","last_peered":"2024-09-13T16:48:08.143381+0000","last_clean":"2024-09-13T16:48:08.143381+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:08.143381+0000","last_undegraded":"2024-09-13T16:48:08.143381+0000","last_fullsized":"2024-09-13T16:48:08.143381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143921+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:08.143921+0000","last_peered":"2024-09-13T16:48:08.143921+0000","last_clean":"2024-09-13T16:48:08.143921+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:08.143921+0000","last_undegraded":"2024-09-13T16:48:08.143921+0000","last_fullsized":"2024-09-13T16:48:08.143921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174477+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T16:48:09.174477+0000","last_peered":"2024-09-13T16:48:09.174477+0000","last_clean":"2024-09-13T16:48:09.174477+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T16:48:09.174477+0000","last_undegraded":"2024-09-13T16:48:09.174477+0000","last_fullsized":"2024-09-13T16:48:09.174477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147114+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:08.147114+0000","last_peered":"2024-09-13T16:48:08.147114+0000","last_clean":"2024-09-13T16:48:08.147114+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:08.147114+0000","last_undegraded":"2024-09-13T16:48:08.147114+0000","last_fullsized":"2024-09-13T16:48:08.147114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161410+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:08.161410+0000","last_peered":"2024-09-13T16:48:08.161410+0000","last_clean":"2024-09-13T16:48:08.161410+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:08.161410+0000","last_undegraded":"2024-09-13T16:48:08.161410+0000","last_fullsized":"2024-09-13T16:48:08.161410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170387+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T16:48:08.170387+0000","last_peered":"2024-09-13T16:48:08.170387+0000","last_clean":"2024-09-13T16:48:08.170387+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T16:48:08.170387+0000","last_undegraded":"2024-09-13T16:48:08.170387+0000","last_fullsized":"2024-09-13T16:48:08.170387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175603+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T16:48:09.175603+0000","last_peered":"2024-09-13T16:48:09.175603+0000","last_clean":"2024-09-13T16:48:09.175603+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T16:48:09.175603+0000","last_undegraded":"2024-09-13T16:48:09.175603+0000","last_fullsized":"2024-09-13T16:48:09.175603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144443+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:08.144443+0000","last_peered":"2024-09-13T16:48:08.144443+0000","last_clean":"2024-09-13T16:48:08.144443+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:08.144443+0000","last_undegraded":"2024-09-13T16:48:08.144443+0000","last_fullsized":"2024-09-13T16:48:08.144443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185002+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:08.185002+0000","last_peered":"2024-09-13T16:48:08.185002+0000","last_clean":"2024-09-13T16:48:08.185002+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:08.185002+0000","last_undegraded":"2024-09-13T16:48:08.185002+0000","last_fullsized":"2024-09-13T16:48:08.185002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170142+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T16:48:08.170142+0000","last_peered":"2024-09-13T16:48:08.170142+0000","last_clean":"2024-09-13T16:48:08.170142+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T16:48:08.170142+0000","last_undegraded":"2024-09-13T16:48:08.170142+0000","last_fullsized":"2024-09-13T16:48:08.170142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177615+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T16:48:09.177615+0000","last_peered":"2024-09-13T16:48:09.177615+0000","last_clean":"2024-09-13T16:48:09.177615+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T16:48:09.177615+0000","last_undegraded":"2024-09-13T16:48:09.177615+0000","last_fullsized":"2024-09-13T16:48:09.177615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144121+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:08.144121+0000","last_peered":"2024-09-13T16:48:08.144121+0000","last_clean":"2024-09-13T16:48:08.144121+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:08.144121+0000","last_undegraded":"2024-09-13T16:48:08.144121+0000","last_fullsized":"2024-09-13T16:48:08.144121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174330+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T16:48:09.174330+0000","last_peered":"2024-09-13T16:48:09.174330+0000","last_clean":"2024-09-13T16:48:09.174330+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T16:48:09.174330+0000","last_undegraded":"2024-09-13T16:48:09.174330+0000","last_fullsized":"2024-09-13T16:48:09.174330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153996+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:08.153996+0000","last_peered":"2024-09-13T16:48:08.153996+0000","last_clean":"2024-09-13T16:48:08.153996+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:08.153996+0000","last_undegraded":"2024-09-13T16:48:08.153996+0000","last_fullsized":"2024-09-13T16:48:08.153996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161029+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T16:48:09.161029+0000","last_peered":"2024-09-13T16:48:09.161029+0000","last_clean":"2024-09-13T16:48:09.161029+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T16:48:09.161029+0000","last_undegraded":"2024-09-13T16:48:09.161029+0000","last_fullsized":"2024-09-13T16:48:09.161029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154547+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:08.154547+0000","last_peered":"2024-09-13T16:48:08.154547+0000","last_clean":"2024-09-13T16:48:08.154547+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:08.154547+0000","last_undegraded":"2024-09-13T16:48:08.154547+0000","last_fullsized":"2024-09-13T16:48:08.154547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174304+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T16:48:09.174304+0000","last_peered":"2024-09-13T16:48:09.174304+0000","last_clean":"2024-09-13T16:48:09.174304+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T16:48:09.174304+0000","last_undegraded":"2024-09-13T16:48:09.174304+0000","last_fullsized":"2024-09-13T16:48:09.174304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161339+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:08.161339+0000","last_peered":"2024-09-13T16:48:08.161339+0000","last_clean":"2024-09-13T16:48:08.161339+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:08.161339+0000","last_undegraded":"2024-09-13T16:48:08.161339+0000","last_fullsized":"2024-09-13T16:48:08.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173686+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T16:48:09.173686+0000","last_peered":"2024-09-13T16:48:09.173686+0000","last_clean":"2024-09-13T16:48:09.173686+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T16:48:09.173686+0000","last_undegraded":"2024-09-13T16:48:09.173686+0000","last_fullsized":"2024-09-13T16:48:09.173686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145104+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:08.145104+0000","last_peered":"2024-09-13T16:48:08.145104+0000","last_clean":"2024-09-13T16:48:08.145104+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:08.145104+0000","last_undegraded":"2024-09-13T16:48:08.145104+0000","last_fullsized":"2024-09-13T16:48:08.145104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175359+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T16:48:09.175359+0000","last_peered":"2024-09-13T16:48:09.175359+0000","last_clean":"2024-09-13T16:48:09.175359+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T16:48:09.175359+0000","last_undegraded":"2024-09-13T16:48:09.175359+0000","last_fullsized":"2024-09-13T16:48:09.175359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149320+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:08.149320+0000","last_peered":"2024-09-13T16:48:08.149320+0000","last_clean":"2024-09-13T16:48:08.149320+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:08.149320+0000","last_undegraded":"2024-09-13T16:48:08.149320+0000","last_fullsized":"2024-09-13T16:48:08.149320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173205+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T16:48:09.173205+0000","last_peered":"2024-09-13T16:48:09.173205+0000","last_clean":"2024-09-13T16:48:09.173205+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T16:48:09.173205+0000","last_undegraded":"2024-09-13T16:48:09.173205+0000","last_fullsized":"2024-09-13T16:48:09.173205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149001+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:08.149001+0000","last_peered":"2024-09-13T16:48:08.149001+0000","last_clean":"2024-09-13T16:48:08.149001+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:08.149001+0000","last_undegraded":"2024-09-13T16:48:08.149001+0000","last_fullsized":"2024-09-13T16:48:08.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158758+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T16:48:09.158758+0000","last_peered":"2024-09-13T16:48:09.158758+0000","last_clean":"2024-09-13T16:48:09.158758+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T16:48:09.158758+0000","last_undegraded":"2024-09-13T16:48:09.158758+0000","last_fullsized":"2024-09-13T16:48:09.158758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145105+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:08.145105+0000","last_peered":"2024-09-13T16:48:08.145105+0000","last_clean":"2024-09-13T16:48:08.145105+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:08.145105+0000","last_undegraded":"2024-09-13T16:48:08.145105+0000","last_fullsized":"2024-09-13T16:48:08.145105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177538+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T16:48:09.177538+0000","last_peered":"2024-09-13T16:48:09.177538+0000","last_clean":"2024-09-13T16:48:09.177538+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T16:48:09.177538+0000","last_undegraded":"2024-09-13T16:48:09.177538+0000","last_fullsized":"2024-09-13T16:48:09.177538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158985+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:08.158985+0000","last_peered":"2024-09-13T16:48:08.158985+0000","last_clean":"2024-09-13T16:48:08.158985+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:08.158985+0000","last_undegraded":"2024-09-13T16:48:08.158985+0000","last_fullsized":"2024-09-13T16:48:08.158985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177461+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T16:48:09.177461+0000","last_peered":"2024-09-13T16:48:09.177461+0000","last_clean":"2024-09-13T16:48:09.177461+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T16:48:09.177461+0000","last_undegraded":"2024-09-13T16:48:09.177461+0000","last_fullsized":"2024-09-13T16:48:09.177461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143902+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:08.143902+0000","last_peered":"2024-09-13T16:48:08.143902+0000","last_clean":"2024-09-13T16:48:08.143902+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:08.143902+0000","last_undegraded":"2024-09-13T16:48:08.143902+0000","last_fullsized":"2024-09-13T16:48:08.143902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159127+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T16:48:09.159127+0000","last_peered":"2024-09-13T16:48:09.159127+0000","last_clean":"2024-09-13T16:48:09.159127+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T16:48:09.159127+0000","last_undegraded":"2024-09-13T16:48:09.159127+0000","last_fullsized":"2024-09-13T16:48:09.159127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148988+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:08.148988+0000","last_peered":"2024-09-13T16:48:08.148988+0000","last_clean":"2024-09-13T16:48:08.148988+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:08.148988+0000","last_undegraded":"2024-09-13T16:48:08.148988+0000","last_fullsized":"2024-09-13T16:48:08.148988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177190+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T16:48:09.177190+0000","last_peered":"2024-09-13T16:48:09.177190+0000","last_clean":"2024-09-13T16:48:09.177190+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T16:48:09.177190+0000","last_undegraded":"2024-09-13T16:48:09.177190+0000","last_fullsized":"2024-09-13T16:48:09.177190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161327+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:08.161327+0000","last_peered":"2024-09-13T16:48:08.161327+0000","last_clean":"2024-09-13T16:48:08.161327+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:08.161327+0000","last_undegraded":"2024-09-13T16:48:08.161327+0000","last_fullsized":"2024-09-13T16:48:08.161327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148225+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:08.148225+0000","last_peered":"2024-09-13T16:48:08.148225+0000","last_clean":"2024-09-13T16:48:08.148225+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:08.148225+0000","last_undegraded":"2024-09-13T16:48:08.148225+0000","last_fullsized":"2024-09-13T16:48:08.148225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175099+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T16:48:09.175099+0000","last_peered":"2024-09-13T16:48:09.175099+0000","last_clean":"2024-09-13T16:48:09.175099+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T16:48:09.175099+0000","last_undegraded":"2024-09-13T16:48:09.175099+0000","last_fullsized":"2024-09-13T16:48:09.175099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179166+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:08.179166+0000","last_peered":"2024-09-13T16:48:08.179166+0000","last_clean":"2024-09-13T16:48:08.179166+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:08.179166+0000","last_undegraded":"2024-09-13T16:48:08.179166+0000","last_fullsized":"2024-09-13T16:48:08.179166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158364+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T16:48:09.158364+0000","last_peered":"2024-09-13T16:48:09.158364+0000","last_clean":"2024-09-13T16:48:09.158364+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T16:48:09.158364+0000","last_undegraded":"2024-09-13T16:48:09.158364+0000","last_fullsized":"2024-09-13T16:48:09.158364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155807+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:08.155807+0000","last_peered":"2024-09-13T16:48:08.155807+0000","last_clean":"2024-09-13T16:48:08.155807+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:08.155807+0000","last_undegraded":"2024-09-13T16:48:08.155807+0000","last_fullsized":"2024-09-13T16:48:08.155807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.172410+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T16:48:09.172410+0000","last_peered":"2024-09-13T16:48:09.172410+0000","last_clean":"2024-09-13T16:48:09.172410+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T16:48:09.172410+0000","last_undegraded":"2024-09-13T16:48:09.172410+0000","last_fullsized":"2024-09-13T16:48:09.172410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.157093+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:08.157093+0000","last_peered":"2024-09-13T16:48:08.157093+0000","last_clean":"2024-09-13T16:48:08.157093+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:08.157093+0000","last_undegraded":"2024-09-13T16:48:08.157093+0000","last_fullsized":"2024-09-13T16:48:08.157093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173535+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T16:48:09.173535+0000","last_peered":"2024-09-13T16:48:09.173535+0000","last_clean":"2024-09-13T16:48:09.173535+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T16:48:09.173535+0000","last_undegraded":"2024-09-13T16:48:09.173535+0000","last_fullsized":"2024-09-13T16:48:09.173535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158715+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:08.158715+0000","last_peered":"2024-09-13T16:48:08.158715+0000","last_clean":"2024-09-13T16:48:08.158715+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:08.158715+0000","last_undegraded":"2024-09-13T16:48:08.158715+0000","last_fullsized":"2024-09-13T16:48:08.158715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175097+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T16:48:09.175097+0000","last_peered":"2024-09-13T16:48:09.175097+0000","last_clean":"2024-09-13T16:48:09.175097+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T16:48:09.175097+0000","last_undegraded":"2024-09-13T16:48:09.175097+0000","last_fullsized":"2024-09-13T16:48:09.175097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147419+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:08.147419+0000","last_peered":"2024-09-13T16:48:08.147419+0000","last_clean":"2024-09-13T16:48:08.147419+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:08.147419+0000","last_undegraded":"2024-09-13T16:48:08.147419+0000","last_fullsized":"2024-09-13T16:48:08.147419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159429+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T16:48:09.159429+0000","last_peered":"2024-09-13T16:48:09.159429+0000","last_clean":"2024-09-13T16:48:09.159429+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T16:48:09.159429+0000","last_undegraded":"2024-09-13T16:48:09.159429+0000","last_fullsized":"2024-09-13T16:48:09.159429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161028+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:08.161028+0000","last_peered":"2024-09-13T16:48:08.161028+0000","last_clean":"2024-09-13T16:48:08.161028+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:08.161028+0000","last_undegraded":"2024-09-13T16:48:08.161028+0000","last_fullsized":"2024-09-13T16:48:08.161028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177095+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T16:48:09.177095+0000","last_peered":"2024-09-13T16:48:09.177095+0000","last_clean":"2024-09-13T16:48:09.177095+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T16:48:09.177095+0000","last_undegraded":"2024-09-13T16:48:09.177095+0000","last_fullsized":"2024-09-13T16:48:09.177095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143741+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:08.143741+0000","last_peered":"2024-09-13T16:48:08.143741+0000","last_clean":"2024-09-13T16:48:08.143741+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:08.143741+0000","last_undegraded":"2024-09-13T16:48:08.143741+0000","last_fullsized":"2024-09-13T16:48:08.143741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176397+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T16:48:09.176397+0000","last_peered":"2024-09-13T16:48:09.176397+0000","last_clean":"2024-09-13T16:48:09.176397+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T16:48:09.176397+0000","last_undegraded":"2024-09-13T16:48:09.176397+0000","last_fullsized":"2024-09-13T16:48:09.176397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186343+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:08.186343+0000","last_peered":"2024-09-13T16:48:08.186343+0000","last_clean":"2024-09-13T16:48:08.186343+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:08.186343+0000","last_undegraded":"2024-09-13T16:48:08.186343+0000","last_fullsized":"2024-09-13T16:48:08.186343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158832+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T16:48:09.158832+0000","last_peered":"2024-09-13T16:48:09.158832+0000","last_clean":"2024-09-13T16:48:09.158832+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T16:48:09.158832+0000","last_undegraded":"2024-09-13T16:48:09.158832+0000","last_fullsized":"2024-09-13T16:48:09.158832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149361+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:08.149361+0000","last_peered":"2024-09-13T16:48:08.149361+0000","last_clean":"2024-09-13T16:48:08.149361+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:08.149361+0000","last_undegraded":"2024-09-13T16:48:08.149361+0000","last_fullsized":"2024-09-13T16:48:08.149361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173784+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T16:48:09.173784+0000","last_peered":"2024-09-13T16:48:09.173784+0000","last_clean":"2024-09-13T16:48:09.173784+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T16:48:09.173784+0000","last_undegraded":"2024-09-13T16:48:09.173784+0000","last_fullsized":"2024-09-13T16:48:09.173784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148791+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:08.148791+0000","last_peered":"2024-09-13T16:48:08.148791+0000","last_clean":"2024-09-13T16:48:08.148791+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:08.148791+0000","last_undegraded":"2024-09-13T16:48:08.148791+0000","last_fullsized":"2024-09-13T16:48:08.148791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159787+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T16:48:09.159787+0000","last_peered":"2024-09-13T16:48:09.159787+0000","last_clean":"2024-09-13T16:48:09.159787+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T16:48:09.159787+0000","last_undegraded":"2024-09-13T16:48:09.159787+0000","last_fullsized":"2024-09-13T16:48:09.159787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166109+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:08.166109+0000","last_peered":"2024-09-13T16:48:08.166109+0000","last_clean":"2024-09-13T16:48:08.166109+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:08.166109+0000","last_undegraded":"2024-09-13T16:48:08.166109+0000","last_fullsized":"2024-09-13T16:48:08.166109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.178540+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T16:48:09.178540+0000","last_peered":"2024-09-13T16:48:09.178540+0000","last_clean":"2024-09-13T16:48:09.178540+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T16:48:09.178540+0000","last_undegraded":"2024-09-13T16:48:09.178540+0000","last_fullsized":"2024-09-13T16:48:09.178540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143140+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:08.143140+0000","last_peered":"2024-09-13T16:48:08.143140+0000","last_clean":"2024-09-13T16:48:08.143140+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:08.143140+0000","last_undegraded":"2024-09-13T16:48:08.143140+0000","last_fullsized":"2024-09-13T16:48:08.143140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.160931+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T16:48:09.160931+0000","last_peered":"2024-09-13T16:48:09.160931+0000","last_clean":"2024-09-13T16:48:09.160931+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T16:48:09.160931+0000","last_undegraded":"2024-09-13T16:48:09.160931+0000","last_fullsized":"2024-09-13T16:48:09.160931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167027+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:08.167027+0000","last_peered":"2024-09-13T16:48:08.167027+0000","last_clean":"2024-09-13T16:48:08.167027+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:08.167027+0000","last_undegraded":"2024-09-13T16:48:08.167027+0000","last_fullsized":"2024-09-13T16:48:08.167027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173532+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T16:48:09.173532+0000","last_peered":"2024-09-13T16:48:09.173532+0000","last_clean":"2024-09-13T16:48:09.173532+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T16:48:09.173532+0000","last_undegraded":"2024-09-13T16:48:09.173532+0000","last_fullsized":"2024-09-13T16:48:09.173532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148753+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:08.148753+0000","last_peered":"2024-09-13T16:48:08.148753+0000","last_clean":"2024-09-13T16:48:08.148753+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:08.148753+0000","last_undegraded":"2024-09-13T16:48:08.148753+0000","last_fullsized":"2024-09-13T16:48:08.148753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161898+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T16:48:09.161898+0000","last_peered":"2024-09-13T16:48:09.161898+0000","last_clean":"2024-09-13T16:48:09.161898+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T16:48:09.161898+0000","last_undegraded":"2024-09-13T16:48:09.161898+0000","last_fullsized":"2024-09-13T16:48:09.161898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160351+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:08.160351+0000","last_peered":"2024-09-13T16:48:08.160351+0000","last_clean":"2024-09-13T16:48:08.160351+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:08.160351+0000","last_undegraded":"2024-09-13T16:48:08.160351+0000","last_fullsized":"2024-09-13T16:48:08.160351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176220+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T16:48:09.176220+0000","last_peered":"2024-09-13T16:48:09.176220+0000","last_clean":"2024-09-13T16:48:09.176220+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T16:48:09.176220+0000","last_undegraded":"2024-09-13T16:48:09.176220+0000","last_fullsized":"2024-09-13T16:48:09.176220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151613+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:08.151613+0000","last_peered":"2024-09-13T16:48:08.151613+0000","last_clean":"2024-09-13T16:48:08.151613+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:08.151613+0000","last_undegraded":"2024-09-13T16:48:08.151613+0000","last_fullsized":"2024-09-13T16:48:08.151613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158220+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T16:48:09.158220+0000","last_peered":"2024-09-13T16:48:09.158220+0000","last_clean":"2024-09-13T16:48:09.158220+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T16:48:09.158220+0000","last_undegraded":"2024-09-13T16:48:09.158220+0000","last_fullsized":"2024-09-13T16:48:09.158220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.162959+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:08.162959+0000","last_peered":"2024-09-13T16:48:08.162959+0000","last_clean":"2024-09-13T16:48:08.162959+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:08.162959+0000","last_undegraded":"2024-09-13T16:48:08.162959+0000","last_fullsized":"2024-09-13T16:48:08.162959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173056+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T16:48:09.173056+0000","last_peered":"2024-09-13T16:48:09.173056+0000","last_clean":"2024-09-13T16:48:09.173056+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T16:48:09.173056+0000","last_undegraded":"2024-09-13T16:48:09.173056+0000","last_fullsized":"2024-09-13T16:48:09.173056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.150218+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:08.150218+0000","last_peered":"2024-09-13T16:48:08.150218+0000","last_clean":"2024-09-13T16:48:08.150218+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:08.150218+0000","last_undegraded":"2024-09-13T16:48:08.150218+0000","last_fullsized":"2024-09-13T16:48:08.150218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173906+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T16:48:09.173906+0000","last_peered":"2024-09-13T16:48:09.173906+0000","last_clean":"2024-09-13T16:48:09.173906+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T16:48:09.173906+0000","last_undegraded":"2024-09-13T16:48:09.173906+0000","last_fullsized":"2024-09-13T16:48:09.173906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166070+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:08.166070+0000","last_peered":"2024-09-13T16:48:08.166070+0000","last_clean":"2024-09-13T16:48:08.166070+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:08.166070+0000","last_undegraded":"2024-09-13T16:48:08.166070+0000","last_fullsized":"2024-09-13T16:48:08.166070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":690265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":101405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.093}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":99455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.367}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":282624,"data_stored":102375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":258048,"data_stored":99295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":278528,"data_stored":100971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":690231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:20.822 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T16:48:20.822 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T16:48:20.823 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T16:48:20.823 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T16:48:20.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f58432c8640 1 Processor -- start 2024-09-13T16:48:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f58432c8640 1 -- start start 2024-09-13T16:48:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f58432c8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f583c06c940 0x7f583c0c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f58432c8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c1555f0 0x7f583c155a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f58432c8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c14f790 con 0x7f583c1526b0 2024-09-13T16:48:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f58432c8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c152a90 con 0x7f583c06c940 2024-09-13T16:48:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f58432c8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c06ce40 con 0x7f583c1555f0 2024-09-13T16:48:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5841064640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f583c1526b0 0x7f583c0aabc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57378/0 (socket says 172.21.15.39:57378) 2024-09-13T16:48:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5841064640 1 -- 172.21.15.39:0/836191597 learned_addr learned my addr 172.21.15.39:0/836191597 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5840863640 1 --2- 172.21.15.39:0/836191597 >> v2:172.21.15.84:3300/0 conn(0x7f583c06c940 0x7f583c0c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5841865640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c1555f0 0x7f583c155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303085143 0 0) 0x7f583c14f790 con 0x7f583c1526b0 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5828003530 con 0x7f583c1526b0 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243479230 0 0) 0x7f5828003530 con 0x7f583c1526b0 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c1555f0 msgr2=0x7f583c155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c1555f0 0x7f583c155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 >> v2:172.21.15.84:3300/0 conn(0x7f583c06c940 msgr2=0x7f583c0c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 --2- 172.21.15.39:0/836191597 >> v2:172.21.15.84:3300/0 conn(0x7f583c06c940 0x7f583c0c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c06d120 con 0x7f583c1526b0 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f58380016a0 con 0x7f583c1526b0 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 >> v1:172.21.15.39:6789/0 conn(0x7f583c1526b0 legacy=0x7f583c0aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c1555f0 0x7f5828003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f58280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5828004d10 0x7f5828005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c152a90 con 0x7f5828003f00 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c06ce40 con 0x7f5828004d10 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582800b990 con 0x7f583c1555f0 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.968+0000 7f5840863640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f58280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5841064640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c1555f0 0x7f5828003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5840863640 1 -- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c1555f0 msgr2=0x7f5828003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5840863640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c1555f0 0x7f5828003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5840863640 1 -- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5828004d10 msgr2=0x7f5828005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:20.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5840863640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5828004d10 0x7f5828005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5840863640 1 -- 172.21.15.39:0/836191597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5828003530 con 0x7f5828003f00 2024-09-13T16:48:20.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5840863640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f58280047d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f582c009ab0 tx=0x7f582c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3a727a7b7d8c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0a5570 con 0x7f5828003f00 2024-09-13T16:48:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c0a56d0 con 0x7f5828003f00 2024-09-13T16:48:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5833fff640 1 -- 172.21.15.39:0/836191597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0a5a80 con 0x7f5828003f00 2024-09-13T16:48:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f58432c8640 1 -- 172.21.15.39:0/836191597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 msgr2=0x7f58280047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f58432c8640 1 --2- 172.21.15.39:0/836191597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f58280047d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f582c009ab0 tx=0x7f582c01ca30 comp rx=0 tx=0).stop 2024-09-13T16:48:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f5841865640 1 -- 172.21.15.39:0/836191597 reap_dead start 2024-09-13T16:48:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f58432c8640 1 -- 172.21.15.39:0/836191597 shutdown_connections 2024-09-13T16:48:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f58432c8640 1 -- 172.21.15.39:0/836191597 >> 172.21.15.39:0/836191597 conn(0x7f583c069e10 msgr2=0x7f583c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f58432c8640 1 -- 172.21.15.39:0/836191597 shutdown_connections 2024-09-13T16:48:20.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f58432c8640 1 -- 172.21.15.39:0/836191597 wait complete. 2024-09-13T16:48:20.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f58432c8640 1 Processor -- start 2024-09-13T16:48:20.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.972+0000 7f58432c8640 1 -- start start 2024-09-13T16:48:20.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f58432c8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f583c0a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f58432c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f583c06fe20 0x7f583c0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f58432c8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c14c7b0 0x7f583c1e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f58432c8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c070d20 con 0x7f5828003f00 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f58432c8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1e7c50 con 0x7f583c06fe20 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f58432c8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0a56e0 con 0x7f583c14c7b0 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841064640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f583c0a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841064640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f583c0a7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58140/0 (socket says 172.21.15.39:58140) 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841064640 1 -- 172.21.15.39:0/1099320854 learned_addr learned my addr 172.21.15.39:0/1099320854 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5840863640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f583c06fe20 0x7f583c0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841865640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c14c7b0 0x7f583c1e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841064640 1 -- 172.21.15.39:0/1099320854 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c14c7b0 msgr2=0x7f583c1e6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841064640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c14c7b0 0x7f583c1e6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841064640 1 -- 172.21.15.39:0/1099320854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f583c06fe20 msgr2=0x7f583c0707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841064640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f583c06fe20 0x7f583c0707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841064640 1 -- 172.21.15.39:0/1099320854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c0a59c0 con 0x7f5828003f00 2024-09-13T16:48:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5840863640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f583c06fe20 0x7f583c0707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5841064640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f583c0a7fb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5838011950 tx=0x7f5838011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5831ffb640 1 -- 172.21.15.39:0/1099320854 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583801a030 con 0x7f5828003f00 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5831ffb640 1 -- 172.21.15.39:0/1099320854 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583800fb80 con 0x7f5828003f00 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c0a5ca0 con 0x7f5828003f00 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5831ffb640 1 -- 172.21.15.39:0/1099320854 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838015410 con 0x7f5828003f00 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c1dd640 con 0x7f5828003f00 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583c0a5160 con 0x7f5828003f00 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5831ffb640 1 -- 172.21.15.39:0/1099320854 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f583801d070 con 0x7f5828003f00 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5831ffb640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58200666c0 0x7f5820068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5831ffb640 1 -- 172.21.15.39:0/1099320854 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f583801b070 con 0x7f5828003f00 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5840863640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58200666c0 0x7f5820068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.976+0000 7f5840863640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58200666c0 0x7f5820068b80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f582c0af000 tx=0x7f582c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:20.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:20.980+0000 7f5831ffb640 1 -- 172.21.15.39:0/1099320854 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5838054530 con 0x7f5828003f00 2024-09-13T16:48:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.172+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f583c0a68f0 con 0x7f5828003f00 2024-09-13T16:48:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.172+0000 7f5831ffb640 1 -- 172.21.15.39:0/1099320854 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5838057b50 con 0x7f5828003f00 2024-09-13T16:48:21.178 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:21.178 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58200666c0 msgr2=0x7f5820068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58200666c0 0x7f5820068b80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f582c0af000 tx=0x7f582c0046d0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 msgr2=0x7f583c0a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f583c0a7fb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5838011950 tx=0x7f5838011e10 comp rx=0 tx=0).stop 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 shutdown_connections 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58200666c0 0x7f5820068b80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f583c06fe20 0x7f583c0707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f583c14c7b0 0x7f583c1e6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 --2- 172.21.15.39:0/1099320854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5828003f00 0x7f583c0a7fb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 >> 172.21.15.39:0/1099320854 conn(0x7f583c069e10 msgr2=0x7f583c150970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 shutdown_connections 2024-09-13T16:48:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.176+0000 7f58432c8640 1 -- 172.21.15.39:0/1099320854 wait complete. 2024-09-13T16:48:21.195 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T16:48:21.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:21.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.300+0000 7fe058714640 1 Processor -- start 2024-09-13T16:48:21.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.300+0000 7fe058714640 1 -- start start 2024-09-13T16:48:21.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe058714640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe0500aabc0 0x7fe0500aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe058714640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05006c870 0x7fe0500c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe058714640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe050152a80 con 0x7fe0500c3ee0 2024-09-13T16:48:21.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe058714640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0500c30f0 con 0x7fe0500aabc0 2024-09-13T16:48:21.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe058714640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe050151e00 con 0x7fe05006c870 2024-09-13T16:48:21.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe056cb1640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe0500c3ee0 0x7fe0501550e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57394/0 (socket says 172.21.15.39:57394) 2024-09-13T16:48:21.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe056cb1640 1 -- 172.21.15.39:0/3690539992 learned_addr learned my addr 172.21.15.39:0/3690539992 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe055caf640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05006c870 0x7fe0500c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0564b0640 1 --2- 172.21.15.39:0/3690539992 >> v2:172.21.15.84:3300/0 conn(0x7fe0500aabc0 0x7fe0500aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222182589 0 0) 0x7fe048001120 con 0x7fe0500c3ee0 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe03c003530 con 0x7fe0500c3ee0 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2031711996 0 0) 0x7fe03c003530 con 0x7fe0500c3ee0 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05006c870 msgr2=0x7fe0500c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05006c870 0x7fe0500c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 >> v2:172.21.15.84:3300/0 conn(0x7fe0500aabc0 msgr2=0x7fe0500aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 --2- 172.21.15.39:0/3690539992 >> v2:172.21.15.84:3300/0 conn(0x7fe0500aabc0 0x7fe0500aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0501520e0 con 0x7fe0500c3ee0 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fe048001810 con 0x7fe0500c3ee0 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 >> v1:172.21.15.39:6789/0 conn(0x7fe0500c3ee0 legacy=0x7fe0501550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe03c003980 0x7fe03c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05006c870 0x7fe03c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe03c004d10 0x7fe03c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0500c30f0 con 0x7fe03c004d10 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe050151e00 con 0x7fe03c003980 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c00b990 con 0x7fe05006c870 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe056cb1640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe03c004d10 0x7fe03c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe0564b0640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe03c003980 0x7fe03c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe055caf640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05006c870 0x7fe03c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe056cb1640 1 -- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05006c870 msgr2=0x7fe03c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe056cb1640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05006c870 0x7fe03c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe056cb1640 1 -- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe03c003980 msgr2=0x7fe03c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe056cb1640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe03c003980 0x7fe03c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.304+0000 7fe056cb1640 1 -- 172.21.15.39:0/3690539992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c003530 con 0x7fe03c004d10 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe03c004d10 0x7fe03c005190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe04800c240 tx=0x7fe04801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca961ccce0249a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0480a5940 con 0x7fe03c004d10 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0480a5aa0 con 0x7fe03c004d10 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe0554ae640 1 -- 172.21.15.39:0/3690539992 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0480a5e50 con 0x7fe03c004d10 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- 172.21.15.39:0/3690539992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe03c004d10 msgr2=0x7fe03c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 --2- 172.21.15.39:0/3690539992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe03c004d10 0x7fe03c005190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe04800c240 tx=0x7fe04801cc40 comp rx=0 tx=0).stop 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 -- 172.21.15.39:0/3690539992 reap_dead start 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- 172.21.15.39:0/3690539992 shutdown_connections 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- 172.21.15.39:0/3690539992 >> 172.21.15.39:0/3690539992 conn(0x7fe050069e10 msgr2=0x7fe05006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- 172.21.15.39:0/3690539992 shutdown_connections 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- 172.21.15.39:0/3690539992 wait complete. 2024-09-13T16:48:21.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 Processor -- start 2024-09-13T16:48:21.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- start start 2024-09-13T16:48:21.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe03c004d10 0x7fe0500a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05014e5a0 0x7fe05014ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05014b4d0 0x7fe05014b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05014ac90 con 0x7fe05014b4d0 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0500a63f0 con 0x7fe03c004d10 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0501eaed0 con 0x7fe05014e5a0 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05014b4d0 0x7fe05014b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05014b4d0 0x7fe05014b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58160/0 (socket says 172.21.15.39:58160) 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 -- 172.21.15.39:0/1372521652 learned_addr learned my addr 172.21.15.39:0/1372521652 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe0564b0640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe03c004d10 0x7fe0500a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe055caf640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05014e5a0 0x7fe05014ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 -- 172.21.15.39:0/1372521652 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05014e5a0 msgr2=0x7fe05014ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05014e5a0 0x7fe05014ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 -- 172.21.15.39:0/1372521652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe03c004d10 msgr2=0x7fe0500a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe03c004d10 0x7fe0500a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 -- 172.21.15.39:0/1372521652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05014be90 con 0x7fe05014b4d0 2024-09-13T16:48:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe056cb1640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05014b4d0 0x7fe05014b950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0480ab040 tx=0x7fe03c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe042ffd640 1 -- 172.21.15.39:0/1372521652 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0480a5940 con 0x7fe05014b4d0 2024-09-13T16:48:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe042ffd640 1 -- 172.21.15.39:0/1372521652 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0480072d0 con 0x7fe05014b4d0 2024-09-13T16:48:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05014a2f0 con 0x7fe05014b4d0 2024-09-13T16:48:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe042ffd640 1 -- 172.21.15.39:0/1372521652 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048000e90 con 0x7fe05014b4d0 2024-09-13T16:48:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.308+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05014a8b0 con 0x7fe05014b4d0 2024-09-13T16:48:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.312+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05014add0 con 0x7fe05014b4d0 2024-09-13T16:48:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.312+0000 7fe042ffd640 1 -- 172.21.15.39:0/1372521652 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe0480b5030 con 0x7fe05014b4d0 2024-09-13T16:48:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.312+0000 7fe042ffd640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe034066690 0x7fe034068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.312+0000 7fe042ffd640 1 -- 172.21.15.39:0/1372521652 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fe04811dfb0 con 0x7fe05014b4d0 2024-09-13T16:48:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.312+0000 7fe0564b0640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe034066690 0x7fe034068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.312+0000 7fe0564b0640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe034066690 0x7fe034068b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe044009ba0 tx=0x7fe044008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:21.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.316+0000 7fe042ffd640 1 -- 172.21.15.39:0/1372521652 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe05014add0 con 0x7fe05014b4d0 2024-09-13T16:48:21.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.468+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe050152a80 con 0x7fe05014b4d0 2024-09-13T16:48:21.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.472+0000 7fe042ffd640 1 -- 172.21.15.39:0/1372521652 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7fe050152a80 con 0x7fe05014b4d0 2024-09-13T16:48:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:21.478 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:21.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe034066690 msgr2=0x7fe034068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe034066690 0x7fe034068b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe044009ba0 tx=0x7fe044008040 comp rx=0 tx=0).stop 2024-09-13T16:48:21.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05014b4d0 msgr2=0x7fe05014b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05014b4d0 0x7fe05014b950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0480ab040 tx=0x7fe03c005800 comp rx=0 tx=0).stop 2024-09-13T16:48:21.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 shutdown_connections 2024-09-13T16:48:21.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe034066690 0x7fe034068b50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe05014b4d0 0x7fe05014b950 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe05014e5a0 0x7fe05014ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 --2- 172.21.15.39:0/1372521652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe03c004d10 0x7fe0500a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 >> 172.21.15.39:0/1372521652 conn(0x7fe050069e10 msgr2=0x7fe0501536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:21.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 shutdown_connections 2024-09-13T16:48:21.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.476+0000 7fe058714640 1 -- 172.21.15.39:0/1372521652 wait complete. 2024-09-13T16:48:21.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:48:21.497 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:48:21.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.612+0000 7fc7fdc33640 1 Processor -- start 2024-09-13T16:48:21.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.612+0000 7fc7fdc33640 1 -- start start 2024-09-13T16:48:21.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7f80aa2d0 0x7fc7f80aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f8155d20 con 0x7fc7f80a4d60 2024-09-13T16:48:21.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f80c4750 con 0x7fc7f8151cf0 2024-09-13T16:48:21.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f8155b30 con 0x7fc7f80aa2d0 2024-09-13T16:48:21.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7f80aa2d0 0x7fc7f80aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f6ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc7f80a4d60 0x7fc7f814ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57404/0 (socket says 172.21.15.39:57404) 2024-09-13T16:48:21.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f6ffd640 1 -- 172.21.15.39:0/3685981928 learned_addr learned my addr 172.21.15.39:0/3685981928 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:21.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f77fe640 1 -- 172.21.15.39:0/3685981928 >> v1:172.21.15.84:6789/0 conn(0x7fc7f8151cf0 legacy=0x7fc7f8152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:21.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f77fe640 1 -- 172.21.15.39:0/3685981928 >> v1:172.21.15.39:6789/0 conn(0x7fc7f80a4d60 legacy=0x7fc7f814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f77fe640 1 -- 172.21.15.39:0/3685981928 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f8155fc0 con 0x7fc7f80aa2d0 2024-09-13T16:48:21.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f67fc640 1 -- 172.21.15.39:0/3685981928 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922872956 0 0) 0x7fc7f8155d20 con 0x7fc7f80a4d60 2024-09-13T16:48:21.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f77fe640 1 --2- 172.21.15.39:0/3685981928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80aa2d0 0x7fc7f80aa6b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00097c0 tx=0x7fc7e001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fd28b691057e43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:21.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f67fc640 1 -- 172.21.15.39:0/3685981928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00ab600 con 0x7fc7f80aa2d0 2024-09-13T16:48:21.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f67fc640 1 -- 172.21.15.39:0/3685981928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e00a7050 con 0x7fc7f80aa2d0 2024-09-13T16:48:21.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7f67fc640 1 -- 172.21.15.39:0/3685981928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00ab980 con 0x7fc7f80aa2d0 2024-09-13T16:48:21.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 -- 172.21.15.39:0/3685981928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80aa2d0 msgr2=0x7fc7f80aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 --2- 172.21.15.39:0/3685981928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80aa2d0 0x7fc7f80aa6b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00097c0 tx=0x7fc7e001c3a0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 -- 172.21.15.39:0/3685981928 shutdown_connections 2024-09-13T16:48:21.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 --2- 172.21.15.39:0/3685981928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80aa2d0 0x7fc7f80aa6b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 -- 172.21.15.39:0/3685981928 >> 172.21.15.39:0/3685981928 conn(0x7fc7f80c5790 msgr2=0x7fc7f80c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 -- 172.21.15.39:0/3685981928 shutdown_connections 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.616+0000 7fc7fdc33640 1 -- 172.21.15.39:0/3685981928 wait complete. 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 Processor -- start 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 -- start start 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80a4d60 0x7fc7f80bee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7f80aa2d0 0x7fc7f80c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7f8151cf0 0x7fc7f80c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f81458b0 con 0x7fc7f80a4d60 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f814e7c0 con 0x7fc7f8151cf0 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8147d90 con 0x7fc7f80aa2d0 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80a4d60 0x7fc7f80bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80a4d60 0x7fc7f80bee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58178/0 (socket says 172.21.15.39:58178) 2024-09-13T16:48:21.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f77fe640 1 -- 172.21.15.39:0/4246498449 learned_addr learned my addr 172.21.15.39:0/4246498449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f77fe640 1 -- 172.21.15.39:0/4246498449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7f80aa2d0 msgr2=0x7fc7f80c17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f77fe640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7f80aa2d0 0x7fc7f80c17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f77fe640 1 -- 172.21.15.39:0/4246498449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7f8151cf0 msgr2=0x7fc7f80c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f7fff640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7f8151cf0 0x7fc7f80c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f77fe640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7f8151cf0 0x7fc7f80c1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f77fe640 1 -- 172.21.15.39:0/4246498449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e0009420 con 0x7fc7f80a4d60 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f7fff640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7f8151cf0 0x7fc7f80c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7f77fe640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80a4d60 0x7fc7f80bee20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0019fd0 tx=0x7fc7e001abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7effff640 1 -- 172.21.15.39:0/4246498449 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0004230 con 0x7fc7f80a4d60 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7effff640 1 -- 172.21.15.39:0/4246498449 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e0004390 con 0x7fc7f80a4d60 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f806b7b0 con 0x7fc7f80a4d60 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7effff640 1 -- 172.21.15.39:0/4246498449 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0003820 con 0x7fc7f80a4d60 2024-09-13T16:48:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f8147ab0 con 0x7fc7f80a4d60 2024-09-13T16:48:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f80aa6b0 con 0x7fc7f80a4d60 2024-09-13T16:48:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.620+0000 7fc7effff640 1 -- 172.21.15.39:0/4246498449 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7e0003980 con 0x7fc7f80a4d60 2024-09-13T16:48:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.624+0000 7fc7effff640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7d80666c0 0x7fc7d8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.624+0000 7fc7effff640 1 -- 172.21.15.39:0/4246498449 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fc7e011eda0 con 0x7fc7f80a4d60 2024-09-13T16:48:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.624+0000 7fc7f6ffd640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7d80666c0 0x7fc7d8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.624+0000 7fc7f6ffd640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7d80666c0 0x7fc7d8068b80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8000c10 tx=0x7fc7e80079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:21.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.628+0000 7fc7effff640 1 -- 172.21.15.39:0/4246498449 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc7e00ebd50 con 0x7fc7f80a4d60 2024-09-13T16:48:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.788+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7f80bf360 con 0x7fc7f80a4d60 2024-09-13T16:48:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.788+0000 7fc7effff640 1 -- 172.21.15.39:0/4246498449 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7e00b0020 con 0x7fc7f80a4d60 2024-09-13T16:48:21.793 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:48:21.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7d80666c0 msgr2=0x7fc7d8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7d80666c0 0x7fc7d8068b80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8000c10 tx=0x7fc7e80079d0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80a4d60 msgr2=0x7fc7f80bee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80a4d60 0x7fc7f80bee20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0019fd0 tx=0x7fc7e001abe0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 shutdown_connections 2024-09-13T16:48:21.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7d80666c0 0x7fc7d8068b80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7f8151cf0 0x7fc7f80c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7f80aa2d0 0x7fc7f80c17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 --2- 172.21.15.39:0/4246498449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7f80a4d60 0x7fc7f80bee20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 >> 172.21.15.39:0/4246498449 conn(0x7fc7f80c5790 msgr2=0x7fc7f814ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:21.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 shutdown_connections 2024-09-13T16:48:21.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.792+0000 7fc7fdc33640 1 -- 172.21.15.39:0/4246498449 wait complete. 2024-09-13T16:48:21.812 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T16:48:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef56e76640 1 Processor -- start 2024-09-13T16:48:21.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef56e76640 1 -- start start 2024-09-13T16:48:21.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef56e76640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fef5006cb90 0x7fef500c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef56e76640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fef5014f7a0 0x7fef500c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef56e76640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5006d1d0 con 0x7fef500c6270 2024-09-13T16:48:21.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef56e76640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef50155e60 con 0x7fef5014f7a0 2024-09-13T16:48:21.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef56e76640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef500c2920 con 0x7fef5006cb90 2024-09-13T16:48:21.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef55413640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fef5014f7a0 0x7fef500c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef55413640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fef5014f7a0 0x7fef500c3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58188/0 (socket says 172.21.15.39:58188) 2024-09-13T16:48:21.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef55413640 1 -- 172.21.15.39:0/4246003490 learned_addr learned my addr 172.21.15.39:0/4246003490 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:21.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef4ffff640 1 --2- 172.21.15.39:0/4246003490 >> v2:172.21.15.84:3300/0 conn(0x7fef5006cb90 0x7fef500c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef55413640 1 -- 172.21.15.39:0/4246003490 >> v2:172.21.15.84:3300/0 conn(0x7fef5006cb90 msgr2=0x7fef500c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:21.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef55413640 1 --2- 172.21.15.39:0/4246003490 >> v2:172.21.15.84:3300/0 conn(0x7fef5006cb90 0x7fef500c2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef55413640 1 -- 172.21.15.39:0/4246003490 >> v1:172.21.15.84:6789/0 conn(0x7fef500c6270 legacy=0x7fef5014f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:21.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.936+0000 7fef55413640 1 -- 172.21.15.39:0/4246003490 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef50156140 con 0x7fef5014f7a0 2024-09-13T16:48:21.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef55413640 1 --2- 172.21.15.39:0/4246003490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5014f7a0 0x7fef500c3380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fef500a9220 tx=0x7fef4001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ca631ba5396d35 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:21.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef4f7fe640 1 -- 172.21.15.39:0/4246003490 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400ab600 con 0x7fef5014f7a0 2024-09-13T16:48:21.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef4f7fe640 1 -- 172.21.15.39:0/4246003490 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef400a7050 con 0x7fef5014f7a0 2024-09-13T16:48:21.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef4f7fe640 1 -- 172.21.15.39:0/4246003490 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400ab980 con 0x7fef5014f7a0 2024-09-13T16:48:21.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 -- 172.21.15.39:0/4246003490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5014f7a0 msgr2=0x7fef500c3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 --2- 172.21.15.39:0/4246003490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5014f7a0 0x7fef500c3380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fef500a9220 tx=0x7fef4001c3c0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 -- 172.21.15.39:0/4246003490 shutdown_connections 2024-09-13T16:48:21.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 --2- 172.21.15.39:0/4246003490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5014f7a0 0x7fef500c3380 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 --2- 172.21.15.39:0/4246003490 >> v2:172.21.15.84:3300/0 conn(0x7fef5006cb90 0x7fef500c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 -- 172.21.15.39:0/4246003490 >> 172.21.15.39:0/4246003490 conn(0x7fef500a8860 msgr2=0x7fef5006bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:21.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 -- 172.21.15.39:0/4246003490 shutdown_connections 2024-09-13T16:48:21.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 -- 172.21.15.39:0/4246003490 wait complete. 2024-09-13T16:48:21.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 Processor -- start 2024-09-13T16:48:21.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.940+0000 7fef56e76640 1 -- start start 2024-09-13T16:48:21.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef56e76640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5006cb90 0x7fef500c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef56e76640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef500c6270 0x7fef5014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef56e76640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5014f7a0 0x7fef501500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef56e76640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef50154590 con 0x7fef5006cb90 2024-09-13T16:48:21.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef56e76640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef50153970 con 0x7fef5014f7a0 2024-09-13T16:48:21.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef56e76640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef50153040 con 0x7fef500c6270 2024-09-13T16:48:21.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef54c12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5006cb90 0x7fef500c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef54c12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5006cb90 0x7fef500c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58200/0 (socket says 172.21.15.39:58200) 2024-09-13T16:48:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef54c12640 1 -- 172.21.15.39:0/1742527011 learned_addr learned my addr 172.21.15.39:0/1742527011 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef4ffff640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef500c6270 0x7fef5014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef55413640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5014f7a0 0x7fef501500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef54c12640 1 -- 172.21.15.39:0/1742527011 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef500c6270 msgr2=0x7fef5014fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef54c12640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef500c6270 0x7fef5014fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef54c12640 1 -- 172.21.15.39:0/1742527011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5014f7a0 msgr2=0x7fef501500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:21.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef54c12640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5014f7a0 0x7fef501500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:21.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef54c12640 1 -- 172.21.15.39:0/1742527011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400094d0 con 0x7fef5006cb90 2024-09-13T16:48:21.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef55413640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5014f7a0 0x7fef501500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:21.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef54c12640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5006cb90 0x7fef500c4f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef4800ba50 tx=0x7fef4800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef4d7fa640 1 -- 172.21.15.39:0/1742527011 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800cb80 con 0x7fef5006cb90 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef4d7fa640 1 -- 172.21.15.39:0/1742527011 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4800cd20 con 0x7fef5006cb90 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef50153320 con 0x7fef5006cb90 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef4d7fa640 1 -- 172.21.15.39:0/1742527011 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48014680 con 0x7fef5006cb90 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef500c54a0 con 0x7fef5006cb90 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5014f090 con 0x7fef5006cb90 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef4d7fa640 1 -- 172.21.15.39:0/1742527011 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef480147e0 con 0x7fef5006cb90 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef4d7fa640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef3c066690 0x7fef3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef4ffff640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef3c066690 0x7fef3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef4d7fa640 1 -- 172.21.15.39:0/1742527011 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fef48084260 con 0x7fef5006cb90 2024-09-13T16:48:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.944+0000 7fef4ffff640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef3c066690 0x7fef3c068b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fef500685e0 tx=0x7fef44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:21.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:21.948+0000 7fef4d7fa640 1 -- 172.21.15.39:0/1742527011 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fef480513e0 con 0x7fef5006cb90 2024-09-13T16:48:22.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.112+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef500c2000 con 0x7fef5006cb90 2024-09-13T16:48:22.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.112+0000 7fef4d7fa640 1 -- 172.21.15.39:0/1742527011 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7fef48054a00 con 0x7fef5006cb90 2024-09-13T16:48:22.119 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef3c066690 msgr2=0x7fef3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef3c066690 0x7fef3c068b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fef500685e0 tx=0x7fef44008040 comp rx=0 tx=0).stop 2024-09-13T16:48:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5006cb90 msgr2=0x7fef500c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5006cb90 0x7fef500c4f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef4800ba50 tx=0x7fef4800bf10 comp rx=0 tx=0).stop 2024-09-13T16:48:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 shutdown_connections 2024-09-13T16:48:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef3c066690 0x7fef3c068b50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef5014f7a0 0x7fef501500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef500c6270 0x7fef5014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 --2- 172.21.15.39:0/1742527011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5006cb90 0x7fef500c4f60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 >> 172.21.15.39:0/1742527011 conn(0x7fef500a8860 msgr2=0x7fef5014e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 shutdown_connections 2024-09-13T16:48:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.116+0000 7fef56e76640 1 -- 172.21.15.39:0/1742527011 wait complete. 2024-09-13T16:48:22.138 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T16:48:22.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c18a8e640 1 Processor -- start 2024-09-13T16:48:22.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c18a8e640 1 -- start start 2024-09-13T16:48:22.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c18a8e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3c141514d0 0x7f3c140c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c18a8e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3c14157370 0x7f3c14154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c18a8e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c14155270 con 0x7f3c140bcf90 2024-09-13T16:48:22.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c18a8e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c14153d80 con 0x7f3c14157370 2024-09-13T16:48:22.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c18a8e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c14157870 con 0x7f3c141514d0 2024-09-13T16:48:22.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c12d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3c14157370 0x7f3c14154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c12d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3c14157370 0x7f3c14154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58204/0 (socket says 172.21.15.39:58204) 2024-09-13T16:48:22.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c12d9d640 1 -- 172.21.15.39:0/2594090231 learned_addr learned my addr 172.21.15.39:0/2594090231 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:22.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c1259c640 1 --2- 172.21.15.39:0/2594090231 >> v2:172.21.15.84:3300/0 conn(0x7f3c141514d0 0x7f3c140c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c12d9d640 1 -- 172.21.15.39:0/2594090231 >> v2:172.21.15.84:3300/0 conn(0x7f3c141514d0 msgr2=0x7f3c140c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:22.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c12d9d640 1 --2- 172.21.15.39:0/2594090231 >> v2:172.21.15.84:3300/0 conn(0x7f3c141514d0 0x7f3c140c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c12d9d640 1 -- 172.21.15.39:0/2594090231 >> v1:172.21.15.84:6789/0 conn(0x7f3c140bcf90 legacy=0x7f3c14154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:22.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.248+0000 7f3c12d9d640 1 -- 172.21.15.39:0/2594090231 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c14157b50 con 0x7f3c14157370 2024-09-13T16:48:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c12d9d640 1 --2- 172.21.15.39:0/2594090231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c14157370 0x7f3c14154950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c14070f70 tx=0x7f3c0401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57acd08746aafe33 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c1159a640 1 -- 172.21.15.39:0/2594090231 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c040ab600 con 0x7f3c14157370 2024-09-13T16:48:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c1159a640 1 -- 172.21.15.39:0/2594090231 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c040a7050 con 0x7f3c14157370 2024-09-13T16:48:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c1159a640 1 -- 172.21.15.39:0/2594090231 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c040ab980 con 0x7f3c14157370 2024-09-13T16:48:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c18a8e640 1 -- 172.21.15.39:0/2594090231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c14157370 msgr2=0x7f3c14154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c18a8e640 1 --2- 172.21.15.39:0/2594090231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c14157370 0x7f3c14154950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c14070f70 tx=0x7f3c0401c450 comp rx=0 tx=0).stop 2024-09-13T16:48:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c18a8e640 1 -- 172.21.15.39:0/2594090231 shutdown_connections 2024-09-13T16:48:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c18a8e640 1 --2- 172.21.15.39:0/2594090231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c14157370 0x7f3c14154950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c18a8e640 1 --2- 172.21.15.39:0/2594090231 >> v2:172.21.15.84:3300/0 conn(0x7f3c141514d0 0x7f3c140c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c18a8e640 1 -- 172.21.15.39:0/2594090231 >> 172.21.15.39:0/2594090231 conn(0x7f3c1406a210 msgr2=0x7f3c1406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c18a8e640 1 -- 172.21.15.39:0/2594090231 shutdown_connections 2024-09-13T16:48:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.252+0000 7f3c18a8e640 1 -- 172.21.15.39:0/2594090231 wait complete. 2024-09-13T16:48:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 Processor -- start 2024-09-13T16:48:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 -- start start 2024-09-13T16:48:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c140bcf90 0x7f3c141445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c141514d0 0x7f3c14144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c14157370 0x7f3c14145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c14148be0 con 0x7f3c140bcf90 2024-09-13T16:48:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140aa460 con 0x7f3c14157370 2024-09-13T16:48:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140aa700 con 0x7f3c141514d0 2024-09-13T16:48:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c1259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c140bcf90 0x7f3c141445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c1259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c140bcf90 0x7f3c141445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58206/0 (socket says 172.21.15.39:58206) 2024-09-13T16:48:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c1259c640 1 -- 172.21.15.39:0/3287641141 learned_addr learned my addr 172.21.15.39:0/3287641141 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c12d9d640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c14157370 0x7f3c14145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c11d9b640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c141514d0 0x7f3c14144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c1259c640 1 -- 172.21.15.39:0/3287641141 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c141514d0 msgr2=0x7f3c14144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c1259c640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c141514d0 0x7f3c14144b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c1259c640 1 -- 172.21.15.39:0/3287641141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c14157370 msgr2=0x7f3c14145060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c1259c640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c14157370 0x7f3c14145060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c1259c640 1 -- 172.21.15.39:0/3287641141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c040094d0 con 0x7f3c140bcf90 2024-09-13T16:48:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c12d9d640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c14157370 0x7f3c14145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c1259c640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c140bcf90 0x7f3c141445e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c007ef0 tx=0x7f3c0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c02ffd640 1 -- 172.21.15.39:0/3287641141 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c014620 con 0x7f3c140bcf90 2024-09-13T16:48:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c02ffd640 1 -- 172.21.15.39:0/3287641141 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c014780 con 0x7f3c140bcf90 2024-09-13T16:48:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c02ffd640 1 -- 172.21.15.39:0/3287641141 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c014b30 con 0x7f3c140bcf90 2024-09-13T16:48:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c140aa9e0 con 0x7f3c140bcf90 2024-09-13T16:48:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c141efb50 con 0x7f3c140bcf90 2024-09-13T16:48:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c140c41f0 con 0x7f3c140bcf90 2024-09-13T16:48:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c02ffd640 1 -- 172.21.15.39:0/3287641141 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c0c014c90 con 0x7f3c140bcf90 2024-09-13T16:48:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c02ffd640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bf4066690 0x7f3bf4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c02ffd640 1 -- 172.21.15.39:0/3287641141 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f3c0c084e20 con 0x7f3c140bcf90 2024-09-13T16:48:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.256+0000 7f3c11d9b640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bf4066690 0x7f3bf4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.260+0000 7f3c11d9b640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bf4066690 0x7f3bf4068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c1406a8c0 tx=0x7f3c08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.260+0000 7f3c02ffd640 1 -- 172.21.15.39:0/3287641141 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c0c051ef0 con 0x7f3c140bcf90 2024-09-13T16:48:22.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.420+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f3c14061350 con 0x7f3c140bcf90 2024-09-13T16:48:22.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.420+0000 7f3c02ffd640 1 -- 172.21.15.39:0/3287641141 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7f3c0c055510 con 0x7f3c140bcf90 2024-09-13T16:48:22.425 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:22.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.420+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bf4066690 msgr2=0x7f3bf4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.420+0000 7f3c18a8e640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bf4066690 0x7f3bf4068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3c1406a8c0 tx=0x7f3c08008040 comp rx=0 tx=0).stop 2024-09-13T16:48:22.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c140bcf90 msgr2=0x7f3c141445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c140bcf90 0x7f3c141445e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c007ef0 tx=0x7f3c0c00e4b0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 shutdown_connections 2024-09-13T16:48:22.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bf4066690 0x7f3bf4068b50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c14157370 0x7f3c14145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c141514d0 0x7f3c14144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 --2- 172.21.15.39:0/3287641141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c140bcf90 0x7f3c141445e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 >> 172.21.15.39:0/3287641141 conn(0x7f3c1406a210 msgr2=0x7f3c14155620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:22.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 shutdown_connections 2024-09-13T16:48:22.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.424+0000 7f3c18a8e640 1 -- 172.21.15.39:0/3287641141 wait complete. 2024-09-13T16:48:22.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T16:48:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387eb99640 1 Processor -- start 2024-09-13T16:48:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387eb99640 1 -- start start 2024-09-13T16:48:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387eb99640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f387814e530 0x7f38780c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387eb99640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f38781540e0 0x7f38780c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387eb99640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3878150b70 con 0x7f38780c4ce0 2024-09-13T16:48:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387eb99640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3878154570 con 0x7f38781540e0 2024-09-13T16:48:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387eb99640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3878154850 con 0x7f387814e530 2024-09-13T16:48:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387d136640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f38780c4ce0 0x7f3878151070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57412/0 (socket says 172.21.15.39:57412) 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387d136640 1 -- 172.21.15.39:0/1941539108 learned_addr learned my addr 172.21.15.39:0/1941539108 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f3877fff640 1 --2- 172.21.15.39:0/1941539108 >> v2:172.21.15.84:3300/0 conn(0x7f38781540e0 0x7f38780c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387c935640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f387814e530 0x7f38780c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167108132 0 0) 0x7f3878150b70 con 0x7f38780c4ce0 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3864005740 con 0x7f38780c4ce0 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 508085785 0 0) 0x7f3864005740 con 0x7f38780c4ce0 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f387814e530 msgr2=0x7f38780c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f387814e530 0x7f38780c1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 >> v2:172.21.15.84:3300/0 conn(0x7f38781540e0 msgr2=0x7f38780c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 --2- 172.21.15.39:0/1941539108 >> v2:172.21.15.84:3300/0 conn(0x7f38781540e0 0x7f38780c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3878154b30 con 0x7f38780c4ce0 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f3870001a00 con 0x7f38780c4ce0 2024-09-13T16:48:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 >> v1:172.21.15.39:6789/0 conn(0x7f38780c4ce0 legacy=0x7f3878151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864005b90 0x7f3864005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f3864006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f387814e530 0x7f38640073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3878154570 con 0x7f38640069e0 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3878154850 con 0x7f3864005b90 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386400dba0 con 0x7f387814e530 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f3877fff640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f3864006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387c935640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864005b90 0x7f3864005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387d136640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f387814e530 0x7f38640073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f3877fff640 1 -- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f387814e530 msgr2=0x7f38640073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f3877fff640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f387814e530 0x7f38640073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f3877fff640 1 -- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864005b90 msgr2=0x7f3864005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f3877fff640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3864005b90 0x7f3864005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f3877fff640 1 -- 172.21.15.39:0/1941539108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3864005740 con 0x7f38640069e0 2024-09-13T16:48:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.568+0000 7f387d136640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f387814e530 0x7f38640073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f3877fff640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f3864006e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3860009ae0 tx=0x7f386001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a973d409c0fee240 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600a8420 con 0x7f38640069e0 2024-09-13T16:48:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38600a8580 con 0x7f38640069e0 2024-09-13T16:48:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f38777fe640 1 -- 172.21.15.39:0/1941539108 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600a8930 con 0x7f38640069e0 2024-09-13T16:48:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 -- 172.21.15.39:0/1941539108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 msgr2=0x7f3864006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 --2- 172.21.15.39:0/1941539108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f3864006e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3860009ae0 tx=0x7f386001ca40 comp rx=0 tx=0).stop 2024-09-13T16:48:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387d136640 1 -- 172.21.15.39:0/1941539108 reap_dead start 2024-09-13T16:48:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 -- 172.21.15.39:0/1941539108 shutdown_connections 2024-09-13T16:48:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 -- 172.21.15.39:0/1941539108 >> 172.21.15.39:0/1941539108 conn(0x7f38780c5790 msgr2=0x7f38780c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 -- 172.21.15.39:0/1941539108 shutdown_connections 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 -- 172.21.15.39:0/1941539108 wait complete. 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 Processor -- start 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 -- start start 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f387814d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38781499a0 0x7f387813eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38781ebd10 0x7f3878149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387814c360 con 0x7f38640069e0 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3878150100 con 0x7f38781499a0 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387eb99640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3878150350 con 0x7f38781ebd10 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387c935640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f387814d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387c935640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f387814d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58230/0 (socket says 172.21.15.39:58230) 2024-09-13T16:48:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387c935640 1 -- 172.21.15.39:0/1548261581 learned_addr learned my addr 172.21.15.39:0/1548261581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f3877fff640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38781499a0 0x7f387813eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387d136640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38781ebd10 0x7f3878149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387c935640 1 -- 172.21.15.39:0/1548261581 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38781ebd10 msgr2=0x7f3878149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387c935640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38781ebd10 0x7f3878149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387c935640 1 -- 172.21.15.39:0/1548261581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38781499a0 msgr2=0x7f387813eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387c935640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38781499a0 0x7f387813eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f387c935640 1 -- 172.21.15.39:0/1548261581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3878150630 con 0x7f38640069e0 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.572+0000 7f3877fff640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38781499a0 0x7f387813eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f387c935640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f387814d9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3868011610 tx=0x7f3868011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f38757fa640 1 -- 172.21.15.39:0/1548261581 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386800f8f0 con 0x7f38640069e0 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f38757fa640 1 -- 172.21.15.39:0/1548261581 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386800fa50 con 0x7f38640069e0 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3878150910 con 0x7f38640069e0 2024-09-13T16:48:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f38757fa640 1 -- 172.21.15.39:0/1548261581 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386800fdc0 con 0x7f38640069e0 2024-09-13T16:48:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387814d160 con 0x7f38640069e0 2024-09-13T16:48:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3878151050 con 0x7f38640069e0 2024-09-13T16:48:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f38757fa640 1 -- 172.21.15.39:0/1548261581 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3868015460 con 0x7f38640069e0 2024-09-13T16:48:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f38757fa640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f3877fff640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f38757fa640 1 -- 172.21.15.39:0/1548261581 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f3868084c80 con 0x7f38640069e0 2024-09-13T16:48:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.576+0000 7f3877fff640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38600af000 tx=0x7f38600b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.580+0000 7f38757fa640 1 -- 172.21.15.39:0/1548261581 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3868051cc0 con 0x7f38640069e0 2024-09-13T16:48:22.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.744+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38781ded40 con 0x7f38640069e0 2024-09-13T16:48:22.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.744+0000 7f38757fa640 1 -- 172.21.15.39:0/1548261581 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7f38680552e0 con 0x7f38640069e0 2024-09-13T16:48:22.750 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:22.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 msgr2=0x7f3858068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f38600af000 tx=0x7f38600b2040 comp rx=0 tx=0).stop 2024-09-13T16:48:22.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 msgr2=0x7f387814d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f387814d9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3868011610 tx=0x7f3868011ad0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 shutdown_connections 2024-09-13T16:48:22.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38781499a0 0x7f387813eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38781ebd10 0x7f3878149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 --2- 172.21.15.39:0/1548261581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38640069e0 0x7f387814d9f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 >> 172.21.15.39:0/1548261581 conn(0x7f38780c5790 msgr2=0x7f38780c3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:22.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 shutdown_connections 2024-09-13T16:48:22.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.748+0000 7f387eb99640 1 -- 172.21.15.39:0/1548261581 wait complete. 2024-09-13T16:48:22.769 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T16:48:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c704d4640 1 Processor -- start 2024-09-13T16:48:22.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c704d4640 1 -- start start 2024-09-13T16:48:22.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c704d4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c68151070 0x7f2c68151480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c704d4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c680c1d70 con 0x7f2c6814e530 2024-09-13T16:48:22.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c704d4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c68150b70 con 0x7f2c681540e0 2024-09-13T16:48:22.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c704d4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c68154570 con 0x7f2c68151070 2024-09-13T16:48:22.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6e270640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2c6814e530 0x7f2c680c1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57416/0 (socket says 172.21.15.39:57416) 2024-09-13T16:48:22.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6e270640 1 -- 172.21.15.39:0/3354379037 learned_addr learned my addr 172.21.15.39:0/3354379037 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:22.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6ea71640 1 --2- 172.21.15.39:0/3354379037 >> v2:172.21.15.84:3300/0 conn(0x7f2c68151070 0x7f2c68151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368173863 0 0) 0x7f2c680c1d70 con 0x7f2c6814e530 2024-09-13T16:48:22.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c54003530 con 0x7f2c6814e530 2024-09-13T16:48:22.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988667315 0 0) 0x7f2c54003530 con 0x7f2c6814e530 2024-09-13T16:48:22.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 >> v2:172.21.15.84:3300/0 conn(0x7f2c68151070 msgr2=0x7f2c68151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:22.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 --2- 172.21.15.39:0/3354379037 >> v2:172.21.15.84:3300/0 conn(0x7f2c68151070 0x7f2c68151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 >> v1:172.21.15.84:6789/0 conn(0x7f2c681540e0 legacy=0x7f2c680c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68154850 con 0x7f2c6814e530 2024-09-13T16:48:22.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f2c64001680 con 0x7f2c6814e530 2024-09-13T16:48:22.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 >> v1:172.21.15.39:6789/0 conn(0x7f2c6814e530 legacy=0x7f2c680c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c540039d0 0x7f2c54003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c540047d0 0x7f2c54004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c540051e0 0x7f2c5400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68154570 con 0x7f2c540047d0 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5400c130 con 0x7f2c540051e0 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5400c2f0 con 0x7f2c540039d0 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6da6f640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c540047d0 0x7f2c54004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6e270640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c540039d0 0x7f2c54003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6ea71640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c540051e0 0x7f2c5400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6da6f640 1 -- 172.21.15.39:0/3354379037 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c540039d0 msgr2=0x7f2c54003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6da6f640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c540039d0 0x7f2c54003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6da6f640 1 -- 172.21.15.39:0/3354379037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c540051e0 msgr2=0x7f2c5400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6da6f640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c540051e0 0x7f2c5400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6da6f640 1 -- 172.21.15.39:0/3354379037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c54003530 con 0x7f2c540047d0 2024-09-13T16:48:22.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.896+0000 7f2c6ea71640 1 -- 172.21.15.39:0/3354379037 reap_dead start 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c6da6f640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c540047d0 0x7f2c54004c50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2c58001750 tx=0x7f2c5801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e7c08a3684e0af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580052d0 con 0x7f2c540047d0 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c580a7050 con 0x7f2c540047d0 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c6d26e640 1 -- 172.21.15.39:0/3354379037 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58003600 con 0x7f2c540047d0 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 -- 172.21.15.39:0/3354379037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c540047d0 msgr2=0x7f2c54004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c540047d0 0x7f2c54004c50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2c58001750 tx=0x7f2c5801c350 comp rx=0 tx=0).stop 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 -- 172.21.15.39:0/3354379037 shutdown_connections 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 --2- 172.21.15.39:0/3354379037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c540047d0 0x7f2c54004c50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 -- 172.21.15.39:0/3354379037 >> 172.21.15.39:0/3354379037 conn(0x7f2c680c5790 msgr2=0x7f2c680c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 -- 172.21.15.39:0/3354379037 shutdown_connections 2024-09-13T16:48:22.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 -- 172.21.15.39:0/3354379037 wait complete. 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 Processor -- start 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 -- start start 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c540047d0 0x7f2c681ec210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c681ec750 0x7f2c680c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c680c51b0 0x7f2c680c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68153cd0 con 0x7f2c680c51b0 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680c3540 con 0x7f2c681ec750 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.900+0000 7f2c704d4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68153710 con 0x7f2c540047d0 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6ea71640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c680c51b0 0x7f2c680c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6ea71640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c680c51b0 0x7f2c680c3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58260/0 (socket says 172.21.15.39:58260) 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6ea71640 1 -- 172.21.15.39:0/1465808889 learned_addr learned my addr 172.21.15.39:0/1465808889 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6da6f640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c681ec750 0x7f2c680c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6e270640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c540047d0 0x7f2c681ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6ea71640 1 -- 172.21.15.39:0/1465808889 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c540047d0 msgr2=0x7f2c681ec210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6ea71640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c540047d0 0x7f2c681ec210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6ea71640 1 -- 172.21.15.39:0/1465808889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c681ec750 msgr2=0x7f2c680c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6ea71640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c681ec750 0x7f2c680c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6ea71640 1 -- 172.21.15.39:0/1465808889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c681539f0 con 0x7f2c680c51b0 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6da6f640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c681ec750 0x7f2c680c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6ea71640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c680c51b0 0x7f2c680c3f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c6806c7b0 tx=0x7f2c60007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c5effd640 1 -- 172.21.15.39:0/1465808889 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60013640 con 0x7f2c680c51b0 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c5effd640 1 -- 172.21.15.39:0/1465808889 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c600137c0 con 0x7f2c680c51b0 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c5effd640 1 -- 172.21.15.39:0/1465808889 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60013b70 con 0x7f2c680c51b0 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c681ecc10 con 0x7f2c680c51b0 2024-09-13T16:48:22.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6814dc60 con 0x7f2c680c51b0 2024-09-13T16:48:22.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c680c36d0 con 0x7f2c680c51b0 2024-09-13T16:48:22.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c5effd640 1 -- 172.21.15.39:0/1465808889 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c60013cd0 con 0x7f2c680c51b0 2024-09-13T16:48:22.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c5effd640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c4c066690 0x7f2c4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:22.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c5effd640 1 -- 172.21.15.39:0/1465808889 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f2c600876e0 con 0x7f2c680c51b0 2024-09-13T16:48:22.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6e270640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c4c066690 0x7f2c4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:22.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.904+0000 7f2c6e270640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c4c066690 0x7f2c4c068b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c64001ec0 tx=0x7f2c64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:22.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:22.908+0000 7f2c5effd640 1 -- 172.21.15.39:0/1465808889 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2c60054890 con 0x7f2c680c51b0 2024-09-13T16:48:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.068+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7f2c680c2590 con 0x7f2c680c51b0 2024-09-13T16:48:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.068+0000 7f2c5effd640 1 -- 172.21.15.39:0/1465808889 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7f2c60022020 con 0x7f2c680c51b0 2024-09-13T16:48:23.075 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c4c066690 msgr2=0x7f2c4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c4c066690 0x7f2c4c068b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2c64001ec0 tx=0x7f2c64009040 comp rx=0 tx=0).stop 2024-09-13T16:48:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c680c51b0 msgr2=0x7f2c680c3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c680c51b0 0x7f2c680c3f10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c6806c7b0 tx=0x7f2c60007700 comp rx=0 tx=0).stop 2024-09-13T16:48:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 shutdown_connections 2024-09-13T16:48:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c4c066690 0x7f2c4c068b50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c681ec750 0x7f2c680c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c680c51b0 0x7f2c680c3f10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 --2- 172.21.15.39:0/1465808889 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c540047d0 0x7f2c681ec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 >> 172.21.15.39:0/1465808889 conn(0x7f2c680c5790 msgr2=0x7f2c681474f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:23.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 shutdown_connections 2024-09-13T16:48:23.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.072+0000 7f2c704d4640 1 -- 172.21.15.39:0/1465808889 wait complete. 2024-09-13T16:48:23.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:23.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efebb5b5640 1 Processor -- start 2024-09-13T16:48:23.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efebb5b5640 1 -- start start 2024-09-13T16:48:23.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efebb5b5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efeb4151550 0x7efeb406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efebb5b5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb41519e0 con 0x7efeb414acf0 2024-09-13T16:48:23.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efebb5b5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb40c4e20 con 0x7efeb406b360 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efebb5b5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb4154ab0 con 0x7efeb4151550 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb9351640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7efeb414acf0 0x7efeb40c4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57430/0 (socket says 172.21.15.39:57430) 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb9351640 1 -- 172.21.15.39:0/1224224538 learned_addr learned my addr 172.21.15.39:0/1224224538 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb8b50640 1 --2- 172.21.15.39:0/1224224538 >> v2:172.21.15.84:3300/0 conn(0x7efeb4151550 0x7efeb406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053879778 0 0) 0x7efeb41519e0 con 0x7efeb414acf0 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe9c003530 con 0x7efeb414acf0 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1336373477 0 0) 0x7efe9c003530 con 0x7efeb414acf0 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 >> v2:172.21.15.84:3300/0 conn(0x7efeb4151550 msgr2=0x7efeb406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 --2- 172.21.15.39:0/1224224538 >> v2:172.21.15.84:3300/0 conn(0x7efeb4151550 0x7efeb406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 >> v1:172.21.15.84:6789/0 conn(0x7efeb406b360 legacy=0x7efeb4153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4154d90 con 0x7efeb414acf0 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7efea4001680 con 0x7efeb414acf0 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 >> v1:172.21.15.39:6789/0 conn(0x7efeb414acf0 legacy=0x7efeb40c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe9c0039b0 0x7efe9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe9c0047d0 0x7efe9c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe9c0051e0 0x7efe9c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb4154ab0 con 0x7efe9c0047d0 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c00c110 con 0x7efe9c0051e0 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c00c2d0 con 0x7efe9c0039b0 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb8b50640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe9c0047d0 0x7efe9c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb9b52640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe9c0051e0 0x7efe9c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb9351640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe9c0039b0 0x7efe9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb8b50640 1 -- 172.21.15.39:0/1224224538 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe9c0039b0 msgr2=0x7efe9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:23.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb8b50640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe9c0039b0 0x7efe9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb8b50640 1 -- 172.21.15.39:0/1224224538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe9c0051e0 msgr2=0x7efe9c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:23.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb8b50640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe9c0051e0 0x7efe9c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb8b50640 1 -- 172.21.15.39:0/1224224538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c003530 con 0x7efe9c0047d0 2024-09-13T16:48:23.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.208+0000 7efeb9b52640 1 -- 172.21.15.39:0/1224224538 reap_dead start 2024-09-13T16:48:23.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efeb8b50640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe9c0047d0 0x7efe9c004c50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efea8009fc0 tx=0x7efea801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8763742e073575b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:23.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80a5b50 con 0x7efe9c0047d0 2024-09-13T16:48:23.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea800f040 con 0x7efe9c0047d0 2024-09-13T16:48:23.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efeb3fff640 1 -- 172.21.15.39:0/1224224538 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80ad440 con 0x7efe9c0047d0 2024-09-13T16:48:23.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efebb5b5640 1 -- 172.21.15.39:0/1224224538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe9c0047d0 msgr2=0x7efe9c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efebb5b5640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe9c0047d0 0x7efe9c004c50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efea8009fc0 tx=0x7efea801cc50 comp rx=0 tx=0).stop 2024-09-13T16:48:23.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efebb5b5640 1 -- 172.21.15.39:0/1224224538 shutdown_connections 2024-09-13T16:48:23.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efebb5b5640 1 --2- 172.21.15.39:0/1224224538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe9c0047d0 0x7efe9c004c50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efebb5b5640 1 -- 172.21.15.39:0/1224224538 >> 172.21.15.39:0/1224224538 conn(0x7efeb40c32a0 msgr2=0x7efeb40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:23.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efebb5b5640 1 -- 172.21.15.39:0/1224224538 shutdown_connections 2024-09-13T16:48:23.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efebb5b5640 1 -- 172.21.15.39:0/1224224538 wait complete. 2024-09-13T16:48:23.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efebb5b5640 1 Processor -- start 2024-09-13T16:48:23.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.212+0000 7efebb5b5640 1 -- start start 2024-09-13T16:48:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efebb5b5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe9c0047d0 0x7efeb41f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efebb5b5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efeb40707e0 0x7efeb4149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efebb5b5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efeb414e8b0 0x7efeb414ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efebb5b5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb41f1380 con 0x7efeb40707e0 2024-09-13T16:48:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efebb5b5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb4149e40 con 0x7efe9c0047d0 2024-09-13T16:48:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efebb5b5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40a8010 con 0x7efeb414e8b0 2024-09-13T16:48:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb8b50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efeb40707e0 0x7efeb4149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb8b50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efeb40707e0 0x7efeb4149530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58290/0 (socket says 172.21.15.39:58290) 2024-09-13T16:48:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb8b50640 1 -- 172.21.15.39:0/1924319699 learned_addr learned my addr 172.21.15.39:0/1924319699 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb9351640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe9c0047d0 0x7efeb41f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb9b52640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efeb414e8b0 0x7efeb414ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb8b50640 1 -- 172.21.15.39:0/1924319699 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efeb414e8b0 msgr2=0x7efeb414ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb8b50640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efeb414e8b0 0x7efeb414ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb8b50640 1 -- 172.21.15.39:0/1924319699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe9c0047d0 msgr2=0x7efeb41f0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb8b50640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe9c0047d0 0x7efeb41f0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb8b50640 1 -- 172.21.15.39:0/1924319699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb414a090 con 0x7efeb40707e0 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb8b50640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efeb40707e0 0x7efeb4149530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efea80b0040 tx=0x7efea8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb1ffb640 1 -- 172.21.15.39:0/1924319699 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8003ba0 con 0x7efeb40707e0 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb1ffb640 1 -- 172.21.15.39:0/1924319699 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea8003d00 con 0x7efeb40707e0 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb40a6c30 con 0x7efeb40707e0 2024-09-13T16:48:23.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb1ffb640 1 -- 172.21.15.39:0/1924319699 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8004240 con 0x7efeb40707e0 2024-09-13T16:48:23.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb414a370 con 0x7efeb40707e0 2024-09-13T16:48:23.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb4150a20 con 0x7efeb40707e0 2024-09-13T16:48:23.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb1ffb640 1 -- 172.21.15.39:0/1924319699 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efea80b4040 con 0x7efeb40707e0 2024-09-13T16:48:23.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb1ffb640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efea0066690 0x7efea0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb9351640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efea0066690 0x7efea0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb1ffb640 1 -- 172.21.15.39:0/1924319699 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7efea811ec80 con 0x7efeb40707e0 2024-09-13T16:48:23.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.216+0000 7efeb9351640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efea0066690 0x7efea0068b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efea4000c10 tx=0x7efea4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:23.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.220+0000 7efeb1ffb640 1 -- 172.21.15.39:0/1924319699 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efeb4150a20 con 0x7efeb40707e0 2024-09-13T16:48:23.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efeb4150a20 con 0x7efeb40707e0 2024-09-13T16:48:23.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efeb1ffb640 1 -- 172.21.15.39:0/1924319699 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7efeb4150a20 con 0x7efeb40707e0 2024-09-13T16:48:23.421 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:23.421 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:48:23.421 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:48:23.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efea0066690 msgr2=0x7efea0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efea0066690 0x7efea0068b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efea4000c10 tx=0x7efea4009040 comp rx=0 tx=0).stop 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efeb40707e0 msgr2=0x7efeb4149530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efeb40707e0 0x7efeb4149530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efea80b0040 tx=0x7efea8009750 comp rx=0 tx=0).stop 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 shutdown_connections 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efea0066690 0x7efea0068b50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efeb414e8b0 0x7efeb414ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efeb40707e0 0x7efeb4149530 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 --2- 172.21.15.39:0/1924319699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe9c0047d0 0x7efeb41f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 >> 172.21.15.39:0/1924319699 conn(0x7efeb40c32a0 msgr2=0x7efeb41dfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.416+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 shutdown_connections 2024-09-13T16:48:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.420+0000 7efebb5b5640 1 -- 172.21.15.39:0/1924319699 wait complete. 2024-09-13T16:48:23.439 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:23.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.556+0000 7fcfb83ef640 1 Processor -- start 2024-09-13T16:48:23.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb83ef640 1 -- start start 2024-09-13T16:48:23.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb83ef640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcfb00a4e70 0x7fcfb006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb83ef640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb00c2780 con 0x7fcfb006d300 2024-09-13T16:48:23.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb83ef640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb00c2a60 con 0x7fcfb014e0c0 2024-09-13T16:48:23.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb83ef640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb00c2d40 con 0x7fcfb00a4e70 2024-09-13T16:48:23.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb598a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcfb00a4e70 0x7fcfb006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb598a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcfb00a4e70 0x7fcfb006cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58306/0 (socket says 172.21.15.39:58306) 2024-09-13T16:48:23.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb698c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcfb006d300 0x7fcfb00c4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57444/0 (socket says 172.21.15.39:57444) 2024-09-13T16:48:23.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb598a640 1 -- 172.21.15.39:0/3934495316 learned_addr learned my addr 172.21.15.39:0/3934495316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:23.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb598a640 1 -- 172.21.15.39:0/3934495316 >> v1:172.21.15.84:6789/0 conn(0x7fcfb014e0c0 legacy=0x7fcfb00c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb598a640 1 -- 172.21.15.39:0/3934495316 >> v1:172.21.15.39:6789/0 conn(0x7fcfb006d300 legacy=0x7fcfb00c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb5189640 1 -- 172.21.15.39:0/3934495316 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532226408 0 0) 0x7fcfb00c2780 con 0x7fcfb006d300 2024-09-13T16:48:23.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb598a640 1 -- 172.21.15.39:0/3934495316 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb00c3020 con 0x7fcfb00a4e70 2024-09-13T16:48:23.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb598a640 1 --2- 172.21.15.39:0/3934495316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb00a4e70 0x7fcfb006cdc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009870 tx=0x7fcfac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f600fab81c986fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:23.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb5189640 1 -- 172.21.15.39:0/3934495316 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0ab600 con 0x7fcfb00a4e70 2024-09-13T16:48:23.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb5189640 1 -- 172.21.15.39:0/3934495316 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac0a7050 con 0x7fcfb00a4e70 2024-09-13T16:48:23.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.560+0000 7fcfb5189640 1 -- 172.21.15.39:0/3934495316 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0ab980 con 0x7fcfb00a4e70 2024-09-13T16:48:23.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 -- 172.21.15.39:0/3934495316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb00a4e70 msgr2=0x7fcfb006cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 --2- 172.21.15.39:0/3934495316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb00a4e70 0x7fcfb006cdc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcfac009870 tx=0x7fcfac01c4d0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 -- 172.21.15.39:0/3934495316 shutdown_connections 2024-09-13T16:48:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 --2- 172.21.15.39:0/3934495316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb00a4e70 0x7fcfb006cdc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 -- 172.21.15.39:0/3934495316 >> 172.21.15.39:0/3934495316 conn(0x7fcfb0069e10 msgr2=0x7fcfb006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 -- 172.21.15.39:0/3934495316 shutdown_connections 2024-09-13T16:48:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 -- 172.21.15.39:0/3934495316 wait complete. 2024-09-13T16:48:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 Processor -- start 2024-09-13T16:48:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 -- start start 2024-09-13T16:48:23.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb006d300 0x7fcfb00a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfb00a4e70 0x7fcfb013fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb014e0c0 0x7fcfb01402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb0144960 con 0x7fcfb006d300 2024-09-13T16:48:23.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb0154840 con 0x7fcfb014e0c0 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb83ef640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb01ee9b0 con 0x7fcfb00a4e70 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb618b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb006d300 0x7fcfb00a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb618b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb006d300 0x7fcfb00a67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58316/0 (socket says 172.21.15.39:58316) 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb618b640 1 -- 172.21.15.39:0/1785297478 learned_addr learned my addr 172.21.15.39:0/1785297478 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb598a640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfb00a4e70 0x7fcfb013fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb698c640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb014e0c0 0x7fcfb01402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb618b640 1 -- 172.21.15.39:0/1785297478 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfb00a4e70 msgr2=0x7fcfb013fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb618b640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfb00a4e70 0x7fcfb013fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb618b640 1 -- 172.21.15.39:0/1785297478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb014e0c0 msgr2=0x7fcfb01402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb618b640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb014e0c0 0x7fcfb01402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb618b640 1 -- 172.21.15.39:0/1785297478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0094d0 con 0x7fcfb006d300 2024-09-13T16:48:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb598a640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfb00a4e70 0x7fcfb013fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.564+0000 7fcfb618b640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb006d300 0x7fcfb00a67c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c3c0 tx=0x7fcfa000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfa6ffd640 1 -- 172.21.15.39:0/1785297478 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00052d0 con 0x7fcfb006d300 2024-09-13T16:48:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfa6ffd640 1 -- 172.21.15.39:0/1785297478 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa0005430 con 0x7fcfb006d300 2024-09-13T16:48:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfa6ffd640 1 -- 172.21.15.39:0/1785297478 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0003840 con 0x7fcfb006d300 2024-09-13T16:48:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb01eec70 con 0x7fcfb006d300 2024-09-13T16:48:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb0154d70 con 0x7fcfb006d300 2024-09-13T16:48:23.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb01eee00 con 0x7fcfb006d300 2024-09-13T16:48:23.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfa6ffd640 1 -- 172.21.15.39:0/1785297478 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcfa00039a0 con 0x7fcfb006d300 2024-09-13T16:48:23.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfa6ffd640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf94066690 0x7fcf94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfb598a640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf94066690 0x7fcf94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfa6ffd640 1 -- 172.21.15.39:0/1785297478 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fcfa0083940 con 0x7fcfb006d300 2024-09-13T16:48:23.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.568+0000 7fcfb598a640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf94066690 0x7fcf94068b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0ad000 tx=0x7fcfac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:23.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.572+0000 7fcfa6ffd640 1 -- 172.21.15.39:0/1785297478 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcfa0050a10 con 0x7fcfb006d300 2024-09-13T16:48:23.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.736+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfb0141400 con 0x7fcfb006d300 2024-09-13T16:48:23.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfa6ffd640 1 -- 172.21.15.39:0/1785297478 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7fcfa0054030 con 0x7fcfb006d300 2024-09-13T16:48:23.743 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:23.744 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:23.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf94066690 msgr2=0x7fcf94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf94066690 0x7fcf94068b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0ad000 tx=0x7fcfac008040 comp rx=0 tx=0).stop 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb006d300 msgr2=0x7fcfb00a67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb006d300 0x7fcfb00a67c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c3c0 tx=0x7fcfa000c880 comp rx=0 tx=0).stop 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 shutdown_connections 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf94066690 0x7fcf94068b50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb014e0c0 0x7fcfb01402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfb00a4e70 0x7fcfb013fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 --2- 172.21.15.39:0/1785297478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfb006d300 0x7fcfb00a67c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 >> 172.21.15.39:0/1785297478 conn(0x7fcfb0069e10 msgr2=0x7fcfb014c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 shutdown_connections 2024-09-13T16:48:23.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.740+0000 7fcfb83ef640 1 -- 172.21.15.39:0/1785297478 wait complete. 2024-09-13T16:48:23.762 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:48:23.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:48:23.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508bcd2640 1 Processor -- start 2024-09-13T16:48:23.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508bcd2640 1 -- start start 2024-09-13T16:48:23.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508bcd2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50840bf5e0 0x7f5084154a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508bcd2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840bff20 0x7f50840c0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508bcd2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508406c600 con 0x7f508406ca90 2024-09-13T16:48:23.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508bcd2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508406bcd0 con 0x7f50840bf5e0 2024-09-13T16:48:23.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508bcd2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508406bfb0 con 0x7f50840bff20 2024-09-13T16:48:23.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508a26f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f508406ca90 0x7f5084151260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57458/0 (socket says 172.21.15.39:57458) 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508a26f640 1 -- 172.21.15.39:0/1540355192 learned_addr learned my addr 172.21.15.39:0/1540355192 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5089a6e640 1 --2- 172.21.15.39:0/1540355192 >> v2:172.21.15.84:3300/0 conn(0x7f50840bf5e0 0x7f5084154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508926d640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840bff20 0x7f50840c0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290535663 0 0) 0x7f508406c600 con 0x7f508406ca90 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f506c003530 con 0x7f508406ca90 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3134334206 0 0) 0x7f506c003530 con 0x7f508406ca90 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840bff20 msgr2=0x7f50840c0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840bff20 0x7f50840c0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 >> v2:172.21.15.84:3300/0 conn(0x7f50840bf5e0 msgr2=0x7f5084154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 --2- 172.21.15.39:0/1540355192 >> v2:172.21.15.84:3300/0 conn(0x7f50840bf5e0 0x7f5084154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508406c290 con 0x7f508406ca90 2024-09-13T16:48:23.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f5074001680 con 0x7f508406ca90 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 >> v1:172.21.15.39:6789/0 conn(0x7f508406ca90 legacy=0x7f5084151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f506c003980 0x7f506c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840bff20 0x7f506c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c004d10 0x7f506c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508406bcd0 con 0x7f506c004d10 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508406bfb0 con 0x7f506c003980 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c00b9c0 con 0x7f50840bff20 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508a26f640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c004d10 0x7f506c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f5089a6e640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f506c003980 0x7f506c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508926d640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840bff20 0x7f506c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508a26f640 1 -- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840bff20 msgr2=0x7f506c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:23.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508a26f640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840bff20 0x7f506c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508a26f640 1 -- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f506c003980 msgr2=0x7f506c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:23.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508a26f640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f506c003980 0x7f506c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.884+0000 7f508a26f640 1 -- 172.21.15.39:0/1540355192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c003530 con 0x7f506c004d10 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508a26f640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c004d10 0x7f506c005190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f507400c290 tx=0x7f507401f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8321469e6ec395b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074005ae0 con 0x7f506c004d10 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50740ab070 con 0x7f506c004d10 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f5088a6c640 1 -- 172.21.15.39:0/1540355192 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074005e80 con 0x7f506c004d10 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508bcd2640 1 -- 172.21.15.39:0/1540355192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c004d10 msgr2=0x7f506c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508bcd2640 1 --2- 172.21.15.39:0/1540355192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c004d10 0x7f506c005190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f507400c290 tx=0x7f507401f450 comp rx=0 tx=0).stop 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508a26f640 1 -- 172.21.15.39:0/1540355192 reap_dead start 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508bcd2640 1 -- 172.21.15.39:0/1540355192 shutdown_connections 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508bcd2640 1 -- 172.21.15.39:0/1540355192 >> 172.21.15.39:0/1540355192 conn(0x7f5084069e10 msgr2=0x7f508406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:23.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508bcd2640 1 -- 172.21.15.39:0/1540355192 shutdown_connections 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508bcd2640 1 -- 172.21.15.39:0/1540355192 wait complete. 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508bcd2640 1 Processor -- start 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.888+0000 7f508bcd2640 1 -- start start 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508bcd2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508414ffa0 0x7f50841e9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508bcd2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5084157eb0 0x7f50841e9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508bcd2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840c3970 0x7f50841ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508bcd2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50841e94a0 con 0x7f508414ffa0 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508bcd2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840aa000 con 0x7f5084157eb0 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508bcd2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50840aa290 con 0x7f50840c3970 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f5089a6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508414ffa0 0x7f50841e9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f5089a6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508414ffa0 0x7f50841e9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58334/0 (socket says 172.21.15.39:58334) 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f5089a6e640 1 -- 172.21.15.39:0/3939115787 learned_addr learned my addr 172.21.15.39:0/3939115787 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508926d640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5084157eb0 0x7f50841e9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508a26f640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840c3970 0x7f50841ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f5089a6e640 1 -- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840c3970 msgr2=0x7f50841ec7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f5089a6e640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840c3970 0x7f50841ec7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f5089a6e640 1 -- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5084157eb0 msgr2=0x7f50841e9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f5089a6e640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5084157eb0 0x7f50841e9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f5089a6e640 1 -- 172.21.15.39:0/3939115787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50840aa570 con 0x7f508414ffa0 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508926d640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5084157eb0 0x7f50841e9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508a26f640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840c3970 0x7f50841ec7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f5089a6e640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508414ffa0 0x7f50841e9930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f507c009fb0 tx=0x7f507c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f50827fc640 1 -- 172.21.15.39:0/3939115787 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c00e030 con 0x7f508414ffa0 2024-09-13T16:48:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f50827fc640 1 -- 172.21.15.39:0/3939115787 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c016470 con 0x7f508414ffa0 2024-09-13T16:48:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508414ba30 con 0x7f508414ffa0 2024-09-13T16:48:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f50827fc640 1 -- 172.21.15.39:0/3939115787 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c016820 con 0x7f508414ffa0 2024-09-13T16:48:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50841508f0 con 0x7f508414ffa0 2024-09-13T16:48:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50841e9270 con 0x7f508414ffa0 2024-09-13T16:48:23.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f50827fc640 1 -- 172.21.15.39:0/3939115787 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f507c016980 con 0x7f508414ffa0 2024-09-13T16:48:23.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f50827fc640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5070066690 0x7f5070068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:23.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f50827fc640 1 -- 172.21.15.39:0/3939115787 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f507c00f070 con 0x7f508414ffa0 2024-09-13T16:48:23.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508926d640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5070066690 0x7f5070068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:23.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.892+0000 7f508926d640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5070066690 0x7f5070068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5084144100 tx=0x7f5078008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:23.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:23.896+0000 7f50827fc640 1 -- 172.21.15.39:0/3939115787 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f50841e9270 con 0x7f508414ffa0 2024-09-13T16:48:24.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50841e9270 con 0x7f508414ffa0 2024-09-13T16:48:24.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f50827fc640 1 -- 172.21.15.39:0/3939115787 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50841e9270 con 0x7f508414ffa0 2024-09-13T16:48:24.068 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:48:24.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5070066690 msgr2=0x7f5070068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5070066690 0x7f5070068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5084144100 tx=0x7f5078008040 comp rx=0 tx=0).stop 2024-09-13T16:48:24.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508414ffa0 msgr2=0x7f50841e9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508414ffa0 0x7f50841e9930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f507c009fb0 tx=0x7f507c0115a0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 shutdown_connections 2024-09-13T16:48:24.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5070066690 0x7f5070068b50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5084157eb0 0x7f50841e9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50840c3970 0x7f50841ec7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 --2- 172.21.15.39:0/3939115787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508414ffa0 0x7f50841e9930 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 >> 172.21.15.39:0/3939115787 conn(0x7f5084069e10 msgr2=0x7f508406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:24.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 shutdown_connections 2024-09-13T16:48:24.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.064+0000 7f508bcd2640 1 -- 172.21.15.39:0/3939115787 wait complete. 2024-09-13T16:48:24.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T16:48:24.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f46980c0640 1 Processor -- start 2024-09-13T16:48:24.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f46980c0640 1 -- start start 2024-09-13T16:48:24.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f46980c0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46900aa2d0 0x7f46900aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f46980c0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4690151cf0 0x7f4690152180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f46980c0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4690155d20 con 0x7f46900a4d60 2024-09-13T16:48:24.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f46980c0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4690155b30 con 0x7f4690151cf0 2024-09-13T16:48:24.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f46980c0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46900c4750 con 0x7f46900aa2d0 2024-09-13T16:48:24.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f469565b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f46900a4d60 0x7f469014ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57464/0 (socket says 172.21.15.39:57464) 2024-09-13T16:48:24.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f469565b640 1 -- 172.21.15.39:0/93299588 learned_addr learned my addr 172.21.15.39:0/93299588 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f469665d640 1 --2- 172.21.15.39:0/93299588 >> v2:172.21.15.84:3300/0 conn(0x7f4690151cf0 0x7f4690152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908696428 0 0) 0x7f4690155d20 con 0x7f46900a4d60 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4695e5c640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46900aa2d0 0x7f46900aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f467c003530 con 0x7f46900a4d60 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 657234339 0 0) 0x7f467c003530 con 0x7f46900a4d60 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46900aa2d0 msgr2=0x7f46900aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46900aa2d0 0x7f46900aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 >> v2:172.21.15.84:3300/0 conn(0x7f4690151cf0 msgr2=0x7f4690152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 --2- 172.21.15.39:0/93299588 >> v2:172.21.15.84:3300/0 conn(0x7f4690151cf0 0x7f4690152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4690155fc0 con 0x7f46900a4d60 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4680001680 con 0x7f46900a4d60 2024-09-13T16:48:24.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 >> v1:172.21.15.39:6789/0 conn(0x7f46900a4d60 legacy=0x7f469014ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f467c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c0047d0 0x7f467c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46900aa2d0 0x7f467c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46900c4750 con 0x7f467c003980 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c00b990 con 0x7f467c0047d0 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c00bad0 con 0x7f46900aa2d0 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4695e5c640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f467c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f469665d640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46900aa2d0 0x7f467c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4695e5c640 1 -- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46900aa2d0 msgr2=0x7f467c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f469565b640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c0047d0 0x7f467c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4695e5c640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46900aa2d0 0x7f467c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4695e5c640 1 -- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c0047d0 msgr2=0x7f467c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:24.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4695e5c640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f467c0047d0 0x7f467c004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f4695e5c640 1 -- 172.21.15.39:0/93299588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c003530 con 0x7f467c003980 2024-09-13T16:48:24.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.208+0000 7f469665d640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46900aa2d0 0x7f467c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:24.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f4695e5c640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f467c003da0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f468c009b10 tx=0x7f468c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94565c849250ef80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:24.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c0a8430 con 0x7f467c003980 2024-09-13T16:48:24.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c0a8590 con 0x7f467c003980 2024-09-13T16:48:24.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f4694e5a640 1 -- 172.21.15.39:0/93299588 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c0a8940 con 0x7f467c003980 2024-09-13T16:48:24.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f46980c0640 1 -- 172.21.15.39:0/93299588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 msgr2=0x7f467c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f46980c0640 1 --2- 172.21.15.39:0/93299588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f467c003da0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f468c009b10 tx=0x7f468c01ca40 comp rx=0 tx=0).stop 2024-09-13T16:48:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f469665d640 1 -- 172.21.15.39:0/93299588 reap_dead start 2024-09-13T16:48:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f46980c0640 1 -- 172.21.15.39:0/93299588 shutdown_connections 2024-09-13T16:48:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f46980c0640 1 -- 172.21.15.39:0/93299588 >> 172.21.15.39:0/93299588 conn(0x7f46900c5790 msgr2=0x7f46900c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f46980c0640 1 -- 172.21.15.39:0/93299588 shutdown_connections 2024-09-13T16:48:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f46980c0640 1 -- 172.21.15.39:0/93299588 wait complete. 2024-09-13T16:48:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f46980c0640 1 Processor -- start 2024-09-13T16:48:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.212+0000 7f46980c0640 1 -- start start 2024-09-13T16:48:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46980c0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f46900c0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46980c0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46901584e0 0x7f469015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46980c0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46901595c0 0x7f46901edd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46980c0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4690142090 con 0x7f467c003980 2024-09-13T16:48:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46980c0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46901eea90 con 0x7f46901595c0 2024-09-13T16:48:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46980c0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4690141970 con 0x7f46901584e0 2024-09-13T16:48:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f4695e5c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f46900c0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f4695e5c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f46900c0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58362/0 (socket says 172.21.15.39:58362) 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f4695e5c640 1 -- 172.21.15.39:0/2308988589 learned_addr learned my addr 172.21.15.39:0/2308988589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f469565b640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46901584e0 0x7f469015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f469665d640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46901595c0 0x7f46901edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f4695e5c640 1 -- 172.21.15.39:0/2308988589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46901584e0 msgr2=0x7f469015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f4695e5c640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46901584e0 0x7f469015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f4695e5c640 1 -- 172.21.15.39:0/2308988589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46901595c0 msgr2=0x7f46901edd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f4695e5c640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46901595c0 0x7f46901edd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f4695e5c640 1 -- 172.21.15.39:0/2308988589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46901ee720 con 0x7f467c003980 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f469665d640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46901595c0 0x7f46901edd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f4695e5c640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f46900c0690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f468c01c870 tx=0x7f468c0197e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46867fc640 1 -- 172.21.15.39:0/2308988589 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00f000 con 0x7f467c003980 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46867fc640 1 -- 172.21.15.39:0/2308988589 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c0a8d80 con 0x7f467c003980 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46867fc640 1 -- 172.21.15.39:0/2308988589 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c004220 con 0x7f467c003980 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469014e8b0 con 0x7f467c003980 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469015ae20 con 0x7f467c003980 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46901ee8b0 con 0x7f467c003980 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46867fc640 1 -- 172.21.15.39:0/2308988589 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f468c004380 con 0x7f467c003980 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46867fc640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4674066690 0x7f4674068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f46867fc640 1 -- 172.21.15.39:0/2308988589 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f468c11e6c0 con 0x7f467c003980 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f469565b640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4674066690 0x7f4674068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.216+0000 7f469565b640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4674066690 0x7f4674068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46900c6060 tx=0x7f4680009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:24.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.220+0000 7f46867fc640 1 -- 172.21.15.39:0/2308988589 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f468c0eb770 con 0x7f467c003980 2024-09-13T16:48:24.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.380+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4690152160 con 0x7f467c003980 2024-09-13T16:48:24.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.380+0000 7f46867fc640 1 -- 172.21.15.39:0/2308988589 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7f468c0eed90 con 0x7f467c003980 2024-09-13T16:48:24.386 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:24.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4674066690 msgr2=0x7f4674068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4674066690 0x7f4674068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46900c6060 tx=0x7f4680009040 comp rx=0 tx=0).stop 2024-09-13T16:48:24.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 msgr2=0x7f46900c0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f46900c0690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f468c01c870 tx=0x7f468c0197e0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 shutdown_connections 2024-09-13T16:48:24.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4674066690 0x7f4674068b50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46901595c0 0x7f46901edd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46901584e0 0x7f469015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 --2- 172.21.15.39:0/2308988589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f467c003980 0x7f46900c0690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 >> 172.21.15.39:0/2308988589 conn(0x7f46900c5790 msgr2=0x7f469014ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:24.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 shutdown_connections 2024-09-13T16:48:24.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.384+0000 7f46980c0640 1 -- 172.21.15.39:0/2308988589 wait complete. 2024-09-13T16:48:24.405 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T16:48:24.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f9862b6e640 1 Processor -- start 2024-09-13T16:48:24.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f9862b6e640 1 -- start start 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f9862b6e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f985c151cf0 0x7f985c152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f9862b6e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c155d20 con 0x7f985c0a4d60 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f9862b6e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c0c4750 con 0x7f985c0aa2d0 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f9862b6e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c155b30 con 0x7f985c151cf0 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f985c0a4d60 0x7f985c14ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57472/0 (socket says 172.21.15.39:57472) 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985bfff640 1 -- 172.21.15.39:0/4041464205 learned_addr learned my addr 172.21.15.39:0/4041464205 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f986110b640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f985c151cf0 0x7f985c152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690353736 0 0) 0x7f985c155d20 con 0x7f985c0a4d60 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9848003530 con 0x7f985c0a4d60 2024-09-13T16:48:24.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2619968474 0 0) 0x7f9848003530 con 0x7f985c0a4d60 2024-09-13T16:48:24.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f985c151cf0 msgr2=0x7f985c152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:24.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f985c151cf0 0x7f985c152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 >> v1:172.21.15.84:6789/0 conn(0x7f985c0aa2d0 legacy=0x7f985c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c155fc0 con 0x7f985c0a4d60 2024-09-13T16:48:24.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f9844001680 con 0x7f985c0a4d60 2024-09-13T16:48:24.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 >> v1:172.21.15.39:6789/0 conn(0x7f985c0a4d60 legacy=0x7f985c14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f985c151cf0 0x7f98480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9848003f30 0x7f98480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9848004d10 0x7f9848005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984800bb20 con 0x7f9848003f30 2024-09-13T16:48:24.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984800bc60 con 0x7f9848004d10 2024-09-13T16:48:24.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984800be20 con 0x7f985c151cf0 2024-09-13T16:48:24.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.520+0000 7f985bfff640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9848003f30 0x7f98480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f986110b640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9848004d10 0x7f9848005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f986090a640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f985c151cf0 0x7f98480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f985bfff640 1 -- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f985c151cf0 msgr2=0x7f98480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:24.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f985bfff640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f985c151cf0 0x7f98480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f985bfff640 1 -- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9848004d10 msgr2=0x7f9848005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:24.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f985bfff640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9848004d10 0x7f9848005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f985bfff640 1 -- 172.21.15.39:0/4041464205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9848003530 con 0x7f9848003f30 2024-09-13T16:48:24.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f985bfff640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9848003f30 0x7f98480047d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f984400c1c0 tx=0x7f984401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c98e18e9f9dcfbc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844005ae0 con 0x7f9848003f30 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9844005c40 con 0x7f9848003f30 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f985b7fe640 1 -- 172.21.15.39:0/4041464205 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844007070 con 0x7f9848003f30 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f9862b6e640 1 -- 172.21.15.39:0/4041464205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9848003f30 msgr2=0x7f98480047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f9862b6e640 1 --2- 172.21.15.39:0/4041464205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9848003f30 0x7f98480047d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f984400c1c0 tx=0x7f984401c980 comp rx=0 tx=0).stop 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f986110b640 1 -- 172.21.15.39:0/4041464205 reap_dead start 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f9862b6e640 1 -- 172.21.15.39:0/4041464205 shutdown_connections 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f9862b6e640 1 -- 172.21.15.39:0/4041464205 >> 172.21.15.39:0/4041464205 conn(0x7f985c0c5790 msgr2=0x7f985c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f9862b6e640 1 -- 172.21.15.39:0/4041464205 shutdown_connections 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.524+0000 7f9862b6e640 1 -- 172.21.15.39:0/4041464205 wait complete. 2024-09-13T16:48:24.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 Processor -- start 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 -- start start 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9848003f30 0x7f985c1ecd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f985c06cfe0 0x7f985c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c154b50 0x7f985c1ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c151a70 con 0x7f985c154b50 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0a8690 con 0x7f985c06cfe0 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c1543a0 con 0x7f9848003f30 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986110b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c154b50 0x7f985c1ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986110b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c154b50 0x7f985c1ef6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58388/0 (socket says 172.21.15.39:58388) 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986110b640 1 -- 172.21.15.39:0/3107538275 learned_addr learned my addr 172.21.15.39:0/3107538275 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:24.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986090a640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9848003f30 0x7f985c1ecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f985bfff640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f985c06cfe0 0x7f985c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986110b640 1 -- 172.21.15.39:0/3107538275 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9848003f30 msgr2=0x7f985c1ecd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986110b640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9848003f30 0x7f985c1ecd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986110b640 1 -- 172.21.15.39:0/3107538275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f985c06cfe0 msgr2=0x7f985c06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986110b640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f985c06cfe0 0x7f985c06c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986110b640 1 -- 172.21.15.39:0/3107538275 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c1ec260 con 0x7f985c154b50 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986110b640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c154b50 0x7f985c1ef6e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f985c152d10 tx=0x7f9854011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f98597fa640 1 -- 172.21.15.39:0/3107538275 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9854004500 con 0x7f985c154b50 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f98597fa640 1 -- 172.21.15.39:0/3107538275 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9854003620 con 0x7f985c154b50 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c1ec540 con 0x7f985c154b50 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f98597fa640 1 -- 172.21.15.39:0/3107538275 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540039d0 con 0x7f985c154b50 2024-09-13T16:48:24.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c06ca20 con 0x7f985c154b50 2024-09-13T16:48:24.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c1eca60 con 0x7f985c154b50 2024-09-13T16:48:24.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f98597fa640 1 -- 172.21.15.39:0/3107538275 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9854003b30 con 0x7f985c154b50 2024-09-13T16:48:24.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f98597fa640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f983c066690 0x7f983c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f98597fa640 1 -- 172.21.15.39:0/3107538275 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f9854084960 con 0x7f985c154b50 2024-09-13T16:48:24.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.528+0000 7f986090a640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f983c066690 0x7f983c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.532+0000 7f986090a640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f983c066690 0x7f983c068b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f984c005210 tx=0x7f984c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:24.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.536+0000 7f98597fa640 1 -- 172.21.15.39:0/3107538275 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9854051a30 con 0x7f985c154b50 2024-09-13T16:48:24.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.692+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f985c14ef70 con 0x7f985c154b50 2024-09-13T16:48:24.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.692+0000 7f98597fa640 1 -- 172.21.15.39:0/3107538275 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7f9854055050 con 0x7f985c154b50 2024-09-13T16:48:24.698 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f983c066690 msgr2=0x7f983c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f983c066690 0x7f983c068b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f984c005210 tx=0x7f984c009040 comp rx=0 tx=0).stop 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c154b50 msgr2=0x7f985c1ef6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c154b50 0x7f985c1ef6e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f985c152d10 tx=0x7f9854011920 comp rx=0 tx=0).stop 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 shutdown_connections 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f983c066690 0x7f983c068b50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f985c06cfe0 0x7f985c06c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c154b50 0x7f985c1ef6e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 --2- 172.21.15.39:0/3107538275 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9848003f30 0x7f985c1ecd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 >> 172.21.15.39:0/3107538275 conn(0x7f985c0c5790 msgr2=0x7f985c15a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 shutdown_connections 2024-09-13T16:48:24.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.696+0000 7f9862b6e640 1 -- 172.21.15.39:0/3107538275 wait complete. 2024-09-13T16:48:24.717 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T16:48:24.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.828+0000 7f06645d0640 1 Processor -- start 2024-09-13T16:48:24.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.828+0000 7f06645d0640 1 -- start start 2024-09-13T16:48:24.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.828+0000 7f06645d0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f065c1517b0 0x7f065c0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.828+0000 7f06645d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c0c6f70 0x7f065c0c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.828+0000 7f06645d0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f065c0c7f60 0x7f065c070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.828+0000 7f06645d0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c0c5330 con 0x7f065c1517b0 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.828+0000 7f06645d0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c0c6b50 con 0x7f065c0c7f60 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0c6240 con 0x7f065c0c6f70 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066236c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f065c1517b0 0x7f065c0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066236c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f065c1517b0 0x7f065c0c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58400/0 (socket says 172.21.15.39:58400) 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066236c640 1 -- 172.21.15.39:0/1736100089 learned_addr learned my addr 172.21.15.39:0/1736100089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f0662b6d640 1 --2- 172.21.15.39:0/1736100089 >> v2:172.21.15.84:3300/0 conn(0x7f065c0c7f60 0x7f065c070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066236c640 1 -- 172.21.15.39:0/1736100089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c0c6f70 msgr2=0x7f065c0c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f0661b6b640 1 --2- 172.21.15.39:0/1736100089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c0c6f70 0x7f065c0c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066236c640 1 --2- 172.21.15.39:0/1736100089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c0c6f70 0x7f065c0c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066236c640 1 -- 172.21.15.39:0/1736100089 >> v2:172.21.15.84:3300/0 conn(0x7f065c0c7f60 msgr2=0x7f065c070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066236c640 1 --2- 172.21.15.39:0/1736100089 >> v2:172.21.15.84:3300/0 conn(0x7f065c0c7f60 0x7f065c070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066236c640 1 -- 172.21.15.39:0/1736100089 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c154bd0 con 0x7f065c1517b0 2024-09-13T16:48:24.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f0662b6d640 1 --2- 172.21.15.39:0/1736100089 >> v2:172.21.15.84:3300/0 conn(0x7f065c0c7f60 0x7f065c070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:24.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f0661b6b640 1 --2- 172.21.15.39:0/1736100089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c0c6f70 0x7f065c0c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:24.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066236c640 1 --2- 172.21.15.39:0/1736100089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c1517b0 0x7f065c0c7950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f065c08baf0 tx=0x7f065801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de032ccdccdf68e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066136a640 1 -- 172.21.15.39:0/1736100089 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06580ad640 con 0x7f065c1517b0 2024-09-13T16:48:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066136a640 1 -- 172.21.15.39:0/1736100089 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06580ad7a0 con 0x7f065c1517b0 2024-09-13T16:48:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f066136a640 1 -- 172.21.15.39:0/1736100089 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06580adb30 con 0x7f065c1517b0 2024-09-13T16:48:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 -- 172.21.15.39:0/1736100089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c1517b0 msgr2=0x7f065c0c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 --2- 172.21.15.39:0/1736100089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c1517b0 0x7f065c0c7950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f065c08baf0 tx=0x7f065801c3d0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 -- 172.21.15.39:0/1736100089 shutdown_connections 2024-09-13T16:48:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 --2- 172.21.15.39:0/1736100089 >> v2:172.21.15.84:3300/0 conn(0x7f065c0c7f60 0x7f065c070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 --2- 172.21.15.39:0/1736100089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c0c6f70 0x7f065c0c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 --2- 172.21.15.39:0/1736100089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c1517b0 0x7f065c0c7950 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 -- 172.21.15.39:0/1736100089 >> 172.21.15.39:0/1736100089 conn(0x7f065c0ab3e0 msgr2=0x7f065c0ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 -- 172.21.15.39:0/1736100089 shutdown_connections 2024-09-13T16:48:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.832+0000 7f06645d0640 1 -- 172.21.15.39:0/1736100089 wait complete. 2024-09-13T16:48:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 Processor -- start 2024-09-13T16:48:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 -- start start 2024-09-13T16:48:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c0c6f70 0x7f065c0c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c7f60 0x7f065c14a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c1517b0 0x7f065c14ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0bf620 con 0x7f065c0c7f60 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c14b4a0 con 0x7f065c0c6f70 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c1ed870 con 0x7f065c1517b0 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0661b6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c7f60 0x7f065c14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0661b6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c7f60 0x7f065c14a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58404/0 (socket says 172.21.15.39:58404) 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0661b6b640 1 -- 172.21.15.39:0/3292514104 learned_addr learned my addr 172.21.15.39:0/3292514104 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0662b6d640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c1517b0 0x7f065c14ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f066236c640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c0c6f70 0x7f065c0c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0661b6b640 1 -- 172.21.15.39:0/3292514104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c1517b0 msgr2=0x7f065c14ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0661b6b640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c1517b0 0x7f065c14ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0661b6b640 1 -- 172.21.15.39:0/3292514104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c0c6f70 msgr2=0x7f065c0c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0661b6b640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c0c6f70 0x7f065c0c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0661b6b640 1 -- 172.21.15.39:0/3292514104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06580094d0 con 0x7f065c0c7f60 2024-09-13T16:48:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0662b6d640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c1517b0 0x7f065c14ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0661b6b640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c7f60 0x7f065c14a830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c7f0 tx=0x7f064c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0652ffd640 1 -- 172.21.15.39:0/3292514104 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c004220 con 0x7f065c0c7f60 2024-09-13T16:48:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0652ffd640 1 -- 172.21.15.39:0/3292514104 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c004380 con 0x7f065c0c7f60 2024-09-13T16:48:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c0bf870 con 0x7f065c0c7f60 2024-09-13T16:48:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0652ffd640 1 -- 172.21.15.39:0/3292514104 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c005bc0 con 0x7f065c0c7f60 2024-09-13T16:48:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c0c8b50 con 0x7f065c0c7f60 2024-09-13T16:48:24.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c14cbb0 con 0x7f065c0c7f60 2024-09-13T16:48:24.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.836+0000 7f0652ffd640 1 -- 172.21.15.39:0/3292514104 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f064c006030 con 0x7f065c0c7f60 2024-09-13T16:48:24.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.840+0000 7f0652ffd640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0640066690 0x7f0640068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:24.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.840+0000 7f0652ffd640 1 -- 172.21.15.39:0/3292514104 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f064c084960 con 0x7f065c0c7f60 2024-09-13T16:48:24.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.840+0000 7f066236c640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0640066690 0x7f0640068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.840+0000 7f066236c640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0640066690 0x7f0640068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06580af000 tx=0x7f0658008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:24.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.844+0000 7f0652ffd640 1 -- 172.21.15.39:0/3292514104 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f064c0519a0 con 0x7f065c0c7f60 2024-09-13T16:48:25.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.996+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f065c066970 con 0x7f065c0c7f60 2024-09-13T16:48:25.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:24.996+0000 7f0652ffd640 1 -- 172.21.15.39:0/3292514104 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7f064c054fc0 con 0x7f065c0c7f60 2024-09-13T16:48:25.003 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:25.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0640066690 msgr2=0x7f0640068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0640066690 0x7f0640068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06580af000 tx=0x7f0658008040 comp rx=0 tx=0).stop 2024-09-13T16:48:25.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c7f60 msgr2=0x7f065c14a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c7f60 0x7f065c14a830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c7f0 tx=0x7f064c00ccb0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 shutdown_connections 2024-09-13T16:48:25.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0640066690 0x7f0640068b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c1517b0 0x7f065c14ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c7f60 0x7f065c14a830 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 --2- 172.21.15.39:0/3292514104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c0c6f70 0x7f065c0c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 >> 172.21.15.39:0/3292514104 conn(0x7f065c0ab3e0 msgr2=0x7f065c0ab7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:25.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 shutdown_connections 2024-09-13T16:48:25.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.000+0000 7f06645d0640 1 -- 172.21.15.39:0/3292514104 wait complete. 2024-09-13T16:48:25.020 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T16:48:25.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28fbf15640 1 Processor -- start 2024-09-13T16:48:25.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28fbf15640 1 -- start start 2024-09-13T16:48:25.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28fbf15640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f28f40aa2d0 0x7f28f40aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28fbf15640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f4155d20 con 0x7f28f4151cf0 2024-09-13T16:48:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28fbf15640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f40c4750 con 0x7f28f40a4d60 2024-09-13T16:48:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28fbf15640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f4155b30 con 0x7f28f40aa2d0 2024-09-13T16:48:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28f9cb1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f28f40aa2d0 0x7f28f40aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28f9cb1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f28f40aa2d0 0x7f28f40aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58412/0 (socket says 172.21.15.39:58412) 2024-09-13T16:48:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28f9cb1640 1 -- 172.21.15.39:0/1284737830 learned_addr learned my addr 172.21.15.39:0/1284737830 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:25.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28f9cb1640 1 -- 172.21.15.39:0/1284737830 >> v1:172.21.15.84:6789/0 conn(0x7f28f40a4d60 legacy=0x7f28f414ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28f9cb1640 1 -- 172.21.15.39:0/1284737830 >> v1:172.21.15.39:6789/0 conn(0x7f28f4151cf0 legacy=0x7f28f4152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.132+0000 7f28f9cb1640 1 -- 172.21.15.39:0/1284737830 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4155fc0 con 0x7f28f40aa2d0 2024-09-13T16:48:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f9cb1640 1 --2- 172.21.15.39:0/1284737830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 0x7f28f40aa6b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009870 tx=0x7f28e401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48af1bc89dd6f29c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f8caf640 1 -- 172.21.15.39:0/1284737830 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e40ab660 con 0x7f28f40aa2d0 2024-09-13T16:48:25.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f8caf640 1 -- 172.21.15.39:0/1284737830 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e40ab7c0 con 0x7f28f40aa2d0 2024-09-13T16:48:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f8caf640 1 -- 172.21.15.39:0/1284737830 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e40abb50 con 0x7f28f40aa2d0 2024-09-13T16:48:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 -- 172.21.15.39:0/1284737830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 msgr2=0x7f28f40aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 --2- 172.21.15.39:0/1284737830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 0x7f28f40aa6b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009870 tx=0x7f28e401c460 comp rx=0 tx=0).stop 2024-09-13T16:48:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 -- 172.21.15.39:0/1284737830 shutdown_connections 2024-09-13T16:48:25.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 --2- 172.21.15.39:0/1284737830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 0x7f28f40aa6b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 -- 172.21.15.39:0/1284737830 >> 172.21.15.39:0/1284737830 conn(0x7f28f40c5790 msgr2=0x7f28f40c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 -- 172.21.15.39:0/1284737830 shutdown_connections 2024-09-13T16:48:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 -- 172.21.15.39:0/1284737830 wait complete. 2024-09-13T16:48:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 Processor -- start 2024-09-13T16:48:25.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 -- start start 2024-09-13T16:48:25.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28f40a4d60 0x7f28f406f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 0x7f28f40c1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28f4151cf0 0x7f28f40701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41eaa40 con 0x7f28f40aa2d0 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41ea4c0 con 0x7f28f40a4d60 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fbf15640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41454e0 con 0x7f28f4151cf0 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f94b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 0x7f28f40c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f94b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 0x7f28f40c1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58418/0 (socket says 172.21.15.39:58418) 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f94b0640 1 -- 172.21.15.39:0/2995360272 learned_addr learned my addr 172.21.15.39:0/2995360272 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f9cb1640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28f40a4d60 0x7f28f406f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28fa4b2640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28f4151cf0 0x7f28f40701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f94b0640 1 -- 172.21.15.39:0/2995360272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28f4151cf0 msgr2=0x7f28f40701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f94b0640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28f4151cf0 0x7f28f40701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f94b0640 1 -- 172.21.15.39:0/2995360272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28f40a4d60 msgr2=0x7f28f406f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f94b0640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28f40a4d60 0x7f28f406f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f94b0640 1 -- 172.21.15.39:0/2995360272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e40094d0 con 0x7f28f40aa2d0 2024-09-13T16:48:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f9cb1640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28f40a4d60 0x7f28f406f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28f94b0640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 0x7f28f40c1680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f28f000b410 tx=0x7f28f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.136+0000 7f28ea7fc640 1 -- 172.21.15.39:0/2995360272 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0004020 con 0x7f28f40aa2d0 2024-09-13T16:48:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28ea7fc640 1 -- 172.21.15.39:0/2995360272 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f0004180 con 0x7f28f40aa2d0 2024-09-13T16:48:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f41ea760 con 0x7f28f40aa2d0 2024-09-13T16:48:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28ea7fc640 1 -- 172.21.15.39:0/2995360272 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0011630 con 0x7f28f40aa2d0 2024-09-13T16:48:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f40c1bc0 con 0x7f28f40aa2d0 2024-09-13T16:48:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f40aa6b0 con 0x7f28f40aa2d0 2024-09-13T16:48:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28ea7fc640 1 -- 172.21.15.39:0/2995360272 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f28f0011790 con 0x7f28f40aa2d0 2024-09-13T16:48:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28ea7fc640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28d8066690 0x7f28d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28ea7fc640 1 -- 172.21.15.39:0/2995360272 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f28f0084200 con 0x7f28f40aa2d0 2024-09-13T16:48:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28f9cb1640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28d8066690 0x7f28d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.140+0000 7f28f9cb1640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28d8066690 0x7f28d8068b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28e40af000 tx=0x7f28e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:25.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.144+0000 7f28ea7fc640 1 -- 172.21.15.39:0/2995360272 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f28f00512d0 con 0x7f28f40aa2d0 2024-09-13T16:48:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.300+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7f28f406f3b0 con 0x7f28f40aa2d0 2024-09-13T16:48:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.300+0000 7f28ea7fc640 1 -- 172.21.15.39:0/2995360272 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7f28f00548f0 con 0x7f28f40aa2d0 2024-09-13T16:48:25.306 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:25.307 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:48:25.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28d8066690 msgr2=0x7f28d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28d8066690 0x7f28d8068b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28e40af000 tx=0x7f28e4008040 comp rx=0 tx=0).stop 2024-09-13T16:48:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 msgr2=0x7f28f40c1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 0x7f28f40c1680 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f28f000b410 tx=0x7f28f000b8d0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 shutdown_connections 2024-09-13T16:48:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28d8066690 0x7f28d8068b50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28f4151cf0 0x7f28f40701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28f40aa2d0 0x7f28f40c1680 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 --2- 172.21.15.39:0/2995360272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28f40a4d60 0x7f28f406f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 >> 172.21.15.39:0/2995360272 conn(0x7f28f40c5790 msgr2=0x7f28f414d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 shutdown_connections 2024-09-13T16:48:25.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.304+0000 7f28fbf15640 1 -- 172.21.15.39:0/2995360272 wait complete. 2024-09-13T16:48:25.310 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:48:25.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:25.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ecbf99640 1 Processor -- start 2024-09-13T16:48:25.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ecbf99640 1 -- start start 2024-09-13T16:48:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ecbf99640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ec40c1210 0x7f1ec406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ecbf99640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec41546c0 con 0x7f1ec40c5330 2024-09-13T16:48:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ecbf99640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec40c49f0 con 0x7f1ec4150f80 2024-09-13T16:48:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ecbf99640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec40c19a0 con 0x7f1ec40c1210 2024-09-13T16:48:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec9534640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1ec40c5330 0x7f1ec406c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57492/0 (socket says 172.21.15.39:57492) 2024-09-13T16:48:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec9534640 1 -- 172.21.15.39:0/3177913894 learned_addr learned my addr 172.21.15.39:0/3177913894 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1eca536640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ec40c1210 0x7f1ec406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280004457 0 0) 0x7f1ec41546c0 con 0x7f1ec40c5330 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb0003530 con 0x7f1ec40c5330 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3828544137 0 0) 0x7f1eb0003530 con 0x7f1ec40c5330 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ec40c1210 msgr2=0x7f1ec406c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ec40c1210 0x7f1ec406c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 >> v1:172.21.15.84:6789/0 conn(0x7f1ec4150f80 legacy=0x7f1ec4153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec41543e0 con 0x7f1ec40c5330 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1eb4001680 con 0x7f1ec40c5330 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 >> v1:172.21.15.39:6789/0 conn(0x7f1ec40c5330 legacy=0x7f1ec406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ec40c1210 0x7f1eb00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb0003f30 0x7f1eb00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1eb0004d10 0x7f1eb0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec40c19a0 con 0x7f1eb0004d10 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb000cc30 con 0x7f1eb0003f30 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb000cdf0 con 0x7f1ec40c1210 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1eca536640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1eb0004d10 0x7f1eb0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1ec9534640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb0003f30 0x7f1eb00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1eca536640 1 -- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ec40c1210 msgr2=0x7f1eb00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1eca536640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ec40c1210 0x7f1eb00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1eca536640 1 -- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb0003f30 msgr2=0x7f1eb00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1eca536640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1eb0003f30 0x7f1eb00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.496+0000 7f1eca536640 1 -- 172.21.15.39:0/3177913894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0003530 con 0x7f1eb0004d10 2024-09-13T16:48:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1eca536640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1eb0004d10 0x7f1eb0005190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009fc0 tx=0x7f1ebc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8f7143fe6f46d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0a5bc0 con 0x7f1eb0004d10 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc00f040 con 0x7f1eb0004d10 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ec8d33640 1 -- 172.21.15.39:0/3177913894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0ad440 con 0x7f1eb0004d10 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3177913894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1eb0004d10 msgr2=0x7f1eb0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 --2- 172.21.15.39:0/3177913894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1eb0004d10 0x7f1eb0005190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009fc0 tx=0x7f1ebc01ccc0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1eca536640 1 -- 172.21.15.39:0/3177913894 reap_dead start 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3177913894 shutdown_connections 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3177913894 >> 172.21.15.39:0/3177913894 conn(0x7f1ec4069e10 msgr2=0x7f1ec406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3177913894 shutdown_connections 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3177913894 wait complete. 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 Processor -- start 2024-09-13T16:48:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 -- start start 2024-09-13T16:48:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1eb0004d10 0x7f1ec41e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ec40a8430 0x7f1ec41ec710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec406cbf0 0x7f1ec41df080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec40a68b0 con 0x7f1ec406cbf0 2024-09-13T16:48:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec41e9180 con 0x7f1ec40a8430 2024-09-13T16:48:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1ecbf99640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec41ea0d0 con 0x7f1eb0004d10 2024-09-13T16:48:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1eca536640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec406cbf0 0x7f1ec41df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1eca536640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec406cbf0 0x7f1ec41df080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58438/0 (socket says 172.21.15.39:58438) 2024-09-13T16:48:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.500+0000 7f1eca536640 1 -- 172.21.15.39:0/3113556740 learned_addr learned my addr 172.21.15.39:0/3113556740 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1ec9534640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ec40a8430 0x7f1ec41ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1ec9d35640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1eb0004d10 0x7f1ec41e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eca536640 1 -- 172.21.15.39:0/3113556740 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1eb0004d10 msgr2=0x7f1ec41e9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eca536640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1eb0004d10 0x7f1ec41e9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eca536640 1 -- 172.21.15.39:0/3113556740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ec40a8430 msgr2=0x7f1ec41ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eca536640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ec40a8430 0x7f1ec41ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eca536640 1 -- 172.21.15.39:0/3113556740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec41ecc50 con 0x7f1ec406cbf0 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eca536640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec406cbf0 0x7f1ec41df080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc002240 tx=0x7f1ebc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eba7fc640 1 -- 172.21.15.39:0/3113556740 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0039a0 con 0x7f1ec406cbf0 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eba7fc640 1 -- 172.21.15.39:0/3113556740 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc003b00 con 0x7f1ec406cbf0 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec40a5f10 con 0x7f1ec406cbf0 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eba7fc640 1 -- 172.21.15.39:0/3113556740 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003e90 con 0x7f1ec406cbf0 2024-09-13T16:48:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec41e9390 con 0x7f1ec406cbf0 2024-09-13T16:48:25.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec40a6430 con 0x7f1ec406cbf0 2024-09-13T16:48:25.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eba7fc640 1 -- 172.21.15.39:0/3113556740 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ebc0b4040 con 0x7f1ec406cbf0 2024-09-13T16:48:25.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eba7fc640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ea80666c0 0x7f1ea8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1eba7fc640 1 -- 172.21.15.39:0/3113556740 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f1ebc11e520 con 0x7f1ec406cbf0 2024-09-13T16:48:25.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1ec9d35640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ea80666c0 0x7f1ea8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.504+0000 7f1ec9d35640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ea80666c0 0x7f1ea8068b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0001650 tx=0x7f1ec0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:25.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.508+0000 7f1eba7fc640 1 -- 172.21.15.39:0/3113556740 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1ec40a6430 con 0x7f1ec406cbf0 2024-09-13T16:48:25.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.700+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ec40a6430 con 0x7f1ec406cbf0 2024-09-13T16:48:25.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.700+0000 7f1eba7fc640 1 -- 172.21.15.39:0/3113556740 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7f1ec40a6430 con 0x7f1ec406cbf0 2024-09-13T16:48:25.705 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:25.705 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:48:25.706 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:48:25.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ea80666c0 msgr2=0x7f1ea8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ea80666c0 0x7f1ea8068b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0001650 tx=0x7f1ec0009040 comp rx=0 tx=0).stop 2024-09-13T16:48:25.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec406cbf0 msgr2=0x7f1ec41df080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec406cbf0 0x7f1ec41df080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc002240 tx=0x7f1ebc009750 comp rx=0 tx=0).stop 2024-09-13T16:48:25.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 shutdown_connections 2024-09-13T16:48:25.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ea80666c0 0x7f1ea8068b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ec40a8430 0x7f1ec41ec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ec406cbf0 0x7f1ec41df080 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 --2- 172.21.15.39:0/3113556740 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1eb0004d10 0x7f1ec41e9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 >> 172.21.15.39:0/3113556740 conn(0x7f1ec4069e10 msgr2=0x7f1ec414f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:25.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 shutdown_connections 2024-09-13T16:48:25.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.704+0000 7f1ecbf99640 1 -- 172.21.15.39:0/3113556740 wait complete. 2024-09-13T16:48:25.723 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:25.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.832+0000 7f46ce857640 1 Processor -- start 2024-09-13T16:48:25.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.832+0000 7f46ce857640 1 -- start start 2024-09-13T16:48:25.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46ce857640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f46c8071dd0 0x7f46c8154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46ce857640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c80c7800 0x7f46c81545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46ce857640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80c5a30 con 0x7f46c8151500 2024-09-13T16:48:25.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46ce857640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c81576a0 con 0x7f46c8071dd0 2024-09-13T16:48:25.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46ce857640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8157980 con 0x7f46c80c7800 2024-09-13T16:48:25.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c77fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f46c8071dd0 0x7f46c8154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46ccdf4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c80c7800 0x7f46c81545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46ccdf4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c80c7800 0x7f46c81545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45178/0 (socket says 172.21.15.39:45178) 2024-09-13T16:48:25.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46ccdf4640 1 -- 172.21.15.39:0/2430901350 learned_addr learned my addr 172.21.15.39:0/2430901350 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:25.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f46c8151500 0x7f46c80c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:43680/0 (socket says 172.21.15.39:43680) 2024-09-13T16:48:25.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c77fe640 1 -- 172.21.15.39:0/2430901350 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c80c7800 msgr2=0x7f46c81545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:25.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c77fe640 1 --2- 172.21.15.39:0/2430901350 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c80c7800 0x7f46c81545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c77fe640 1 -- 172.21.15.39:0/2430901350 >> v1:172.21.15.84:6789/0 conn(0x7f46c8151500 legacy=0x7f46c80c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c77fe640 1 -- 172.21.15.39:0/2430901350 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c8157c60 con 0x7f46c8071dd0 2024-09-13T16:48:25.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46ccdf4640 1 --2- 172.21.15.39:0/2430901350 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c80c7800 0x7f46c81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:25.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c77fe640 1 --2- 172.21.15.39:0/2430901350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46c8071dd0 0x7f46c8154070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0097f0 tx=0x7f46bc01ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8c9cbe40dd450d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:25.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c6ffd640 1 -- 172.21.15.39:0/2430901350 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0ab600 con 0x7f46c8071dd0 2024-09-13T16:48:25.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c6ffd640 1 -- 172.21.15.39:0/2430901350 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc0ab760 con 0x7f46c8071dd0 2024-09-13T16:48:25.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.836+0000 7f46c6ffd640 1 -- 172.21.15.39:0/2430901350 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0abaf0 con 0x7f46c8071dd0 2024-09-13T16:48:25.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- 172.21.15.39:0/2430901350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46c8071dd0 msgr2=0x7f46c8154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 --2- 172.21.15.39:0/2430901350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46c8071dd0 0x7f46c8154070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0097f0 tx=0x7f46bc01ce40 comp rx=0 tx=0).stop 2024-09-13T16:48:25.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- 172.21.15.39:0/2430901350 shutdown_connections 2024-09-13T16:48:25.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 --2- 172.21.15.39:0/2430901350 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c80c7800 0x7f46c81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 --2- 172.21.15.39:0/2430901350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46c8071dd0 0x7f46c8154070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- 172.21.15.39:0/2430901350 >> 172.21.15.39:0/2430901350 conn(0x7f46c80ab530 msgr2=0x7f46c80ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:25.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- 172.21.15.39:0/2430901350 shutdown_connections 2024-09-13T16:48:25.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- 172.21.15.39:0/2430901350 wait complete. 2024-09-13T16:48:25.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 Processor -- start 2024-09-13T16:48:25.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- start start 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c8071dd0 0x7f46c81479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46c80c7800 0x7f46c80aa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8151500 0x7f46c8145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8147ee0 con 0x7f46c8151500 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80ab360 con 0x7f46c80c7800 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8146270 con 0x7f46c8071dd0 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ccdf4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8151500 0x7f46c8145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ccdf4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8151500 0x7f46c8145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58440/0 (socket says 172.21.15.39:58440) 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ccdf4640 1 -- 172.21.15.39:0/1029573879 learned_addr learned my addr 172.21.15.39:0/1029573879 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46c77fe640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46c80c7800 0x7f46c80aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46c7fff640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c8071dd0 0x7f46c81479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ccdf4640 1 -- 172.21.15.39:0/1029573879 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c8071dd0 msgr2=0x7f46c81479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ccdf4640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c8071dd0 0x7f46c81479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ccdf4640 1 -- 172.21.15.39:0/1029573879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46c80c7800 msgr2=0x7f46c80aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ccdf4640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46c80c7800 0x7f46c80aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ccdf4640 1 -- 172.21.15.39:0/1029573879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009b30 con 0x7f46c8151500 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46c7fff640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c8071dd0 0x7f46c81479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ccdf4640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8151500 0x7f46c8145480 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46c000efc0 tx=0x7f46c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46c4ff9640 1 -- 172.21.15.39:0/1029573879 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000cdc0 con 0x7f46c8151500 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46c4ff9640 1 -- 172.21.15.39:0/1029573879 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c000f040 con 0x7f46c8151500 2024-09-13T16:48:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc009420 con 0x7f46c8151500 2024-09-13T16:48:25.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46c4ff9640 1 -- 172.21.15.39:0/1029573879 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c00146c0 con 0x7f46c8151500 2024-09-13T16:48:25.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.840+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c80aa310 con 0x7f46c8151500 2024-09-13T16:48:25.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.844+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c80c6290 con 0x7f46c8151500 2024-09-13T16:48:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.844+0000 7f46c4ff9640 1 -- 172.21.15.39:0/1029573879 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f46c0014820 con 0x7f46c8151500 2024-09-13T16:48:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.844+0000 7f46c4ff9640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46b4066690 0x7f46b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.844+0000 7f46c7fff640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46b4066690 0x7f46b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.844+0000 7f46c4ff9640 1 -- 172.21.15.39:0/1029573879 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f46c0084de0 con 0x7f46c8151500 2024-09-13T16:48:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.844+0000 7f46c7fff640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46b4066690 0x7f46b4068b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46b8000f70 tx=0x7f46b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:25.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:25.848+0000 7f46c4ff9640 1 -- 172.21.15.39:0/1029573879 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f46c0051e10 con 0x7f46c8151500 2024-09-13T16:48:26.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.004+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46c80c4280 con 0x7f46c8151500 2024-09-13T16:48:26.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46c4ff9640 1 -- 172.21.15.39:0/1029573879 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7f46c0055430 con 0x7f46c8151500 2024-09-13T16:48:26.012 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:26.012 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46b4066690 msgr2=0x7f46b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46b4066690 0x7f46b4068b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46b8000f70 tx=0x7f46b8009040 comp rx=0 tx=0).stop 2024-09-13T16:48:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8151500 msgr2=0x7f46c8145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8151500 0x7f46c8145480 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46c000efc0 tx=0x7f46c000c490 comp rx=0 tx=0).stop 2024-09-13T16:48:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 shutdown_connections 2024-09-13T16:48:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46b4066690 0x7f46b4068b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46c80c7800 0x7f46c80aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46c8151500 0x7f46c8145480 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 --2- 172.21.15.39:0/1029573879 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46c8071dd0 0x7f46c81479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 >> 172.21.15.39:0/1029573879 conn(0x7f46c80ab530 msgr2=0x7f46c80c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:26.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 shutdown_connections 2024-09-13T16:48:26.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.008+0000 7f46ce857640 1 -- 172.21.15.39:0/1029573879 wait complete. 2024-09-13T16:48:26.030 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:26.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.144+0000 7f668317e640 1 Processor -- start 2024-09-13T16:48:26.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.144+0000 7f668317e640 1 -- start start 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f668317e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c0a5cd0 0x7f667c0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f668317e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667c0c11a0 con 0x7f667c072030 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f668317e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667c0c6430 con 0x7f667c155300 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f668317e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667c0c13f0 con 0x7f667c0a5cd0 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f6680f1a640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f667c072030 0x7f667c06ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57500/0 (socket says 172.21.15.39:57500) 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f6680f1a640 1 -- 172.21.15.39:0/1984360262 learned_addr learned my addr 172.21.15.39:0/1984360262 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667bfff640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c0a5cd0 0x7f667c0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384678868 0 0) 0x7f667c0c11a0 con 0x7f667c072030 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6664003530 con 0x7f667c072030 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2443774823 0 0) 0x7f6664003530 con 0x7f667c072030 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c0a5cd0 msgr2=0x7f667c0c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c0a5cd0 0x7f667c0c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 >> v1:172.21.15.84:6789/0 conn(0x7f667c155300 legacy=0x7f667c155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c0c16d0 con 0x7f667c072030 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f6674001a00 con 0x7f667c072030 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 >> v1:172.21.15.39:6789/0 conn(0x7f667c072030 legacy=0x7f667c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640039b0 0x7f6664003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66640047d0 0x7f6664004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c0a5cd0 0x7f6664005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c0c13f0 con 0x7f66640047d0 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666400bb50 con 0x7f66640039b0 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666400bd10 con 0x7f667c0a5cd0 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667bfff640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66640047d0 0x7f6664004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f6680f1a640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640039b0 0x7f6664003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f668171b640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c0a5cd0 0x7f6664005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667bfff640 1 -- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c0a5cd0 msgr2=0x7f6664005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667bfff640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c0a5cd0 0x7f6664005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667bfff640 1 -- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640039b0 msgr2=0x7f6664003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667bfff640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640039b0 0x7f6664003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.148+0000 7f667bfff640 1 -- 172.21.15.39:0/1984360262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664003530 con 0x7f66640047d0 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66640047d0 0x7f6664004c50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6670009ae0 tx=0x7f667001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a62031627ea7941 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66700a5520 con 0x7f66640047d0 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66700a5680 con 0x7f66640047d0 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667b7fe640 1 -- 172.21.15.39:0/1984360262 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66700a5a30 con 0x7f66640047d0 2024-09-13T16:48:26.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- 172.21.15.39:0/1984360262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66640047d0 msgr2=0x7f6664004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 --2- 172.21.15.39:0/1984360262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66640047d0 0x7f6664004c50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6670009ae0 tx=0x7f667001ca40 comp rx=0 tx=0).stop 2024-09-13T16:48:26.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668171b640 1 -- 172.21.15.39:0/1984360262 reap_dead start 2024-09-13T16:48:26.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- 172.21.15.39:0/1984360262 shutdown_connections 2024-09-13T16:48:26.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- 172.21.15.39:0/1984360262 >> 172.21.15.39:0/1984360262 conn(0x7f667c0c32a0 msgr2=0x7f667c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:26.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- 172.21.15.39:0/1984360262 shutdown_connections 2024-09-13T16:48:26.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- 172.21.15.39:0/1984360262 wait complete. 2024-09-13T16:48:26.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 Processor -- start 2024-09-13T16:48:26.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- start start 2024-09-13T16:48:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640047d0 0x7f667c14d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667c15bce0 0x7f667c1eda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c1edf40 0x7f667c149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c153190 con 0x7f667c15bce0 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c143af0 con 0x7f66640047d0 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667c154960 con 0x7f667c1edf40 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667c15bce0 0x7f667c1eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667c15bce0 0x7f667c1eda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58460/0 (socket says 172.21.15.39:58460) 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 -- 172.21.15.39:0/3405691352 learned_addr learned my addr 172.21.15.39:0/3405691352 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f6680f1a640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640047d0 0x7f667c14d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668171b640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c1edf40 0x7f667c149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 -- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c1edf40 msgr2=0x7f667c149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c1edf40 0x7f667c149ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 -- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640047d0 msgr2=0x7f667c14d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640047d0 0x7f667c14d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 -- 172.21.15.39:0/3405691352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c0c2050 con 0x7f667c15bce0 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668171b640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c1edf40 0x7f667c149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f6680f1a640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640047d0 0x7f667c14d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f667bfff640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667c15bce0 0x7f667c1eda00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6670009ae0 tx=0x7f6664005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f66797fa640 1 -- 172.21.15.39:0/3405691352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66700be070 con 0x7f667c15bce0 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f66797fa640 1 -- 172.21.15.39:0/3405691352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66700a5e70 con 0x7f667c15bce0 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f66797fa640 1 -- 172.21.15.39:0/3405691352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670004020 con 0x7f667c15bce0 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c15c1e0 con 0x7f667c15bce0 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.152+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c1545f0 con 0x7f667c15bce0 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.156+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667c06ae20 con 0x7f667c15bce0 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.156+0000 7f66797fa640 1 -- 172.21.15.39:0/3405691352 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f66700b4040 con 0x7f667c15bce0 2024-09-13T16:48:26.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.156+0000 7f66797fa640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6668066690 0x7f6668068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.156+0000 7f66797fa640 1 -- 172.21.15.39:0/3405691352 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f667011dd50 con 0x7f667c15bce0 2024-09-13T16:48:26.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.156+0000 7f6680f1a640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6668066690 0x7f6668068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.156+0000 7f6680f1a640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6668066690 0x7f6668068b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6674001470 tx=0x7f6674009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.160+0000 7f66797fa640 1 -- 172.21.15.39:0/3405691352 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6670121030 con 0x7f667c15bce0 2024-09-13T16:48:26.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.356+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f667c0c4700 con 0x7f667c15bce0 2024-09-13T16:48:26.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f66797fa640 1 -- 172.21.15.39:0/3405691352 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7f66700eaf40 con 0x7f667c15bce0 2024-09-13T16:48:26.363 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:26.363 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:48:26.364 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:48:26.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6668066690 msgr2=0x7f6668068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6668066690 0x7f6668068b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6674001470 tx=0x7f6674009040 comp rx=0 tx=0).stop 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667c15bce0 msgr2=0x7f667c1eda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667c15bce0 0x7f667c1eda00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6670009ae0 tx=0x7f6664005800 comp rx=0 tx=0).stop 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 shutdown_connections 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6668066690 0x7f6668068b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f667c1edf40 0x7f667c149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667c15bce0 0x7f667c1eda00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 --2- 172.21.15.39:0/3405691352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f66640047d0 0x7f667c14d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 >> 172.21.15.39:0/3405691352 conn(0x7f667c0c32a0 msgr2=0x7f667c157970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 shutdown_connections 2024-09-13T16:48:26.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.360+0000 7f668317e640 1 -- 172.21.15.39:0/3405691352 wait complete. 2024-09-13T16:48:26.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:47:59.023080+0000', 'modified': '2024-09-13T16:48:02.679109+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.84:6833/1392588044', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6832', 'nonce': 1392588044}, {'type': 'v1', 'addr': '172.21.15.84:6833', 'nonce': 1392588044}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'a', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:48:26.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T16:48:26.787 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T16:48:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb9327b7640 1 Processor -- start 2024-09-13T16:48:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb9327b7640 1 -- start start 2024-09-13T16:48:26.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb9327b7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c14f4b0 0x7fb92c065c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb9327b7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb92c0c0c20 0x7fb92c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb9327b7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92c0c4d70 con 0x7fb92c0661c0 2024-09-13T16:48:26.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb9327b7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92c0c47b0 con 0x7fb92c0c0c20 2024-09-13T16:48:26.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb9327b7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0c4a90 con 0x7fb92c14f4b0 2024-09-13T16:48:26.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c14f4b0 0x7fb92c065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb930d54640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb92c0c0c20 0x7fb92c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb92b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb92c0661c0 0x7fb92c1552b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:43688/0 (socket says 172.21.15.39:43688) 2024-09-13T16:48:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c14f4b0 0x7fb92c065c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45218/0 (socket says 172.21.15.39:45218) 2024-09-13T16:48:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb92b7fe640 1 -- 172.21.15.39:0/543840304 learned_addr learned my addr 172.21.15.39:0/543840304 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb930d54640 1 -- 172.21.15.39:0/543840304 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c14f4b0 msgr2=0x7fb92c065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb930d54640 1 --2- 172.21.15.39:0/543840304 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c14f4b0 0x7fb92c065c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb930d54640 1 -- 172.21.15.39:0/543840304 >> v1:172.21.15.84:6789/0 conn(0x7fb92c0661c0 legacy=0x7fb92c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb930d54640 1 -- 172.21.15.39:0/543840304 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c06df90 con 0x7fb92c0c0c20 2024-09-13T16:48:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.920+0000 7fb92affd640 1 -- 172.21.15.39:0/543840304 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40607215 0 0) 0x7fb92c0c4d70 con 0x7fb92c0661c0 2024-09-13T16:48:26.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb930d54640 1 --2- 172.21.15.39:0/543840304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0c0c20 0x7fb92c1559c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb924009d90 tx=0x7fb92401c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c59fccb1e604898 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb92affd640 1 -- 172.21.15.39:0/543840304 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240a58a0 con 0x7fb92c0c0c20 2024-09-13T16:48:26.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb92affd640 1 -- 172.21.15.39:0/543840304 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9240a5a00 con 0x7fb92c0c0c20 2024-09-13T16:48:26.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb92affd640 1 -- 172.21.15.39:0/543840304 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240a5d90 con 0x7fb92c0c0c20 2024-09-13T16:48:26.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb9327b7640 1 -- 172.21.15.39:0/543840304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0c0c20 msgr2=0x7fb92c1559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb9327b7640 1 --2- 172.21.15.39:0/543840304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0c0c20 0x7fb92c1559c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb924009d90 tx=0x7fb92401c820 comp rx=0 tx=0).stop 2024-09-13T16:48:26.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb9327b7640 1 -- 172.21.15.39:0/543840304 shutdown_connections 2024-09-13T16:48:26.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb9327b7640 1 --2- 172.21.15.39:0/543840304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0c0c20 0x7fb92c1559c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb9327b7640 1 --2- 172.21.15.39:0/543840304 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c14f4b0 0x7fb92c065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb9327b7640 1 -- 172.21.15.39:0/543840304 >> 172.21.15.39:0/543840304 conn(0x7fb92c0c3310 msgr2=0x7fb92c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:26.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb9327b7640 1 -- 172.21.15.39:0/543840304 shutdown_connections 2024-09-13T16:48:26.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.924+0000 7fb9327b7640 1 -- 172.21.15.39:0/543840304 wait complete. 2024-09-13T16:48:26.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 Processor -- start 2024-09-13T16:48:26.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 -- start start 2024-09-13T16:48:26.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0661c0 0x7fb92c0aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c0c0c20 0x7fb92c1ea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb92c14f4b0 0x7fb92c149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c149540 con 0x7fb92c14f4b0 2024-09-13T16:48:26.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c152ab0 con 0x7fb92c0661c0 2024-09-13T16:48:26.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c1527d0 con 0x7fb92c0c0c20 2024-09-13T16:48:26.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb930d54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb92c14f4b0 0x7fb92c149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb930d54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb92c14f4b0 0x7fb92c149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58466/0 (socket says 172.21.15.39:58466) 2024-09-13T16:48:26.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb930d54640 1 -- 172.21.15.39:0/796629453 learned_addr learned my addr 172.21.15.39:0/796629453 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb92bfff640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0661c0 0x7fb92c0aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb92b7fe640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c0c0c20 0x7fb92c1ea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb930d54640 1 -- 172.21.15.39:0/796629453 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c0c0c20 msgr2=0x7fb92c1ea1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb930d54640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c0c0c20 0x7fb92c1ea1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb930d54640 1 -- 172.21.15.39:0/796629453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0661c0 msgr2=0x7fb92c0aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb930d54640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0661c0 0x7fb92c0aa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb930d54640 1 -- 172.21.15.39:0/796629453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb924009a70 con 0x7fb92c14f4b0 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb92bfff640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0661c0 0x7fb92c0aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb930d54640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb92c14f4b0 0x7fb92c149000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb92401a450 tx=0x7fb9240195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb928ff9640 1 -- 172.21.15.39:0/796629453 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924003ba0 con 0x7fb92c14f4b0 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb928ff9640 1 -- 172.21.15.39:0/796629453 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb924003d00 con 0x7fb92c14f4b0 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 -- 172.21.15.39:0/796629453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92c151f60 con 0x7fb92c14f4b0 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb928ff9640 1 -- 172.21.15.39:0/796629453 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924004230 con 0x7fb92c14f4b0 2024-09-13T16:48:26.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.928+0000 7fb9327b7640 1 -- 172.21.15.39:0/796629453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92c152520 con 0x7fb92c14f4b0 2024-09-13T16:48:26.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb928ff9640 1 -- 172.21.15.39:0/796629453 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb9240b4070 con 0x7fb92c14f4b0 2024-09-13T16:48:26.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb928ff9640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb918066690 0x7fb918068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb928ff9640 1 -- 172.21.15.39:0/796629453 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb92411f0c0 con 0x7fb92c14f4b0 2024-09-13T16:48:26.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb92bfff640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb918066690 0x7fb918068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb92bfff640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb918066690 0x7fb918068b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb92c155cc0 tx=0x7fb91c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb935900000 1 Processor -- start 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb935900000 1 -- start start 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb935900000 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 0x55c0a169e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb935900000 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a169eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb935900000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a169f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb935900000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a1568e80 con 0x55c0a169f050 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb935900000 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a14f6af0 con 0x55c0a16dec10 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb935900000 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0a14f40c0 con 0x55c0a169e730 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb930d54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a169f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb930d54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a169f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58478/0 (socket says 172.21.15.39:58478) 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb930d54640 1 -- 172.21.15.39:0/2155168938 learned_addr learned my addr 172.21.15.39:0/2155168938 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb92bfff640 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 0x55c0a169e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb92b7fe640 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a169eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb930d54640 1 -- 172.21.15.39:0/2155168938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 msgr2=0x55c0a169eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb930d54640 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a169eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb930d54640 1 -- 172.21.15.39:0/2155168938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 msgr2=0x55c0a169e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb930d54640 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 0x55c0a169e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb930d54640 1 -- 172.21.15.39:0/2155168938 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0a11eb890 con 0x55c0a169f050 2024-09-13T16:48:26.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb92b7fe640 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a169eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:26.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb930d54640 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a169f430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb9240f7c70 tx=0x7fb924003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896b5183a3be3303 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb911ffb640 1 -- 172.21.15.39:0/2155168938 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240efdb0 con 0x55c0a169f050 2024-09-13T16:48:26.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb911ffb640 1 -- 172.21.15.39:0/2155168938 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9240f3be0 con 0x55c0a169f050 2024-09-13T16:48:26.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.932+0000 7fb911ffb640 1 -- 172.21.15.39:0/2155168938 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240f81a0 con 0x55c0a169f050 2024-09-13T16:48:26.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- 172.21.15.39:0/2155168938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 msgr2=0x55c0a169f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a169f430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb9240f7c70 tx=0x7fb924003fb0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- 172.21.15.39:0/2155168938 shutdown_connections 2024-09-13T16:48:26.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a169f430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a169eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 --2- 172.21.15.39:0/2155168938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 0x55c0a169e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- 172.21.15.39:0/2155168938 >> 172.21.15.39:0/2155168938 conn(0x55c0a16dc110 msgr2=0x55c0a16de500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- 172.21.15.39:0/2155168938 shutdown_connections 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- 172.21.15.39:0/2155168938 wait complete. 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- start start 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 Processor -- start 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a16a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a16a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 0x55c0a16a5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a14ef9b0 con 0x55c0a169f050 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a148ec00 con 0x55c0a16dec10 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0a14ad070 con 0x55c0a169e730 2024-09-13T16:48:26.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 10 client.0 _ll_register_callbacks cb 0x55c0a168cae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a16a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a16a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58482/0 (socket says 172.21.15.39:58482) 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92b7fe640 1 -- 172.21.15.39:0/1106816569 learned_addr learned my addr 172.21.15.39:0/1106816569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92bfff640 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a16a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb930d54640 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 0x55c0a16a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92b7fe640 1 -- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 msgr2=0x55c0a16a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92b7fe640 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a16a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92b7fe640 1 -- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 msgr2=0x55c0a16a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92b7fe640 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 0x55c0a16a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92b7fe640 1 -- 172.21.15.39:0/1106816569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92c152520 con 0x55c0a169f050 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92b7fe640 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a16a4c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb920000f70 tx=0x7fb9200014c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfb37dbb8341a00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb90affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-13T16:48:26.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb90affd640 1 -- 172.21.15.39:0/1106816569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92000d070 con 0x55c0a169f050 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb90affd640 1 -- 172.21.15.39:0/1106816569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb9200143f0 con 0x55c0a169f050 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- 172.21.15.39:0/1106816569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0a14ac5f0 con 0x55c0a169f050 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb90affd640 1 -- 172.21.15.39:0/1106816569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9200145b0 con 0x55c0a169f050 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 10 client.4749 fetch_fsmap learned FSMap version 14 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- 172.21.15.39:0/1106816569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55c0a14b4ab0 con 0x55c0a169f050 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb90affd640 1 -- 172.21.15.39:0/1106816569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c0a14ac5f0 con 0x55c0a169f050 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb90affd640 1 -- 172.21.15.39:0/1106816569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 14) v1 ==== 3668+0+0 (secure 0 0 0) 0x7fb9200047f0 con 0x55c0a169f050 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 10 client.4749 fetch_fsmap finished waiting for FSMap version 14 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 10 client.4749 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 20 client.4749 populate_metadata read hostname 'smithi039' 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] conn(0x55c0a16a5b80 0x55c0a16f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 4 client.4749 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:48:26.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb935900000 1 -- 172.21.15.39:0/1106816569 --> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0a14ec790 con 0x55c0a16a5b80 2024-09-13T16:48:26.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.936+0000 7fb92bfff640 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] conn(0x55c0a16a5b80 0x55c0a16f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb90affd640 10 client.4749 ms_handle_connect on v2:172.21.15.84:6832/1392588044 2024-09-13T16:48:26.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb92bfff640 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] conn(0x55c0a16a5b80 0x55c0a16f34e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf465161c5fb7db8 server_cookie=dcb0fa2cdb30cadf in_seq=0 out_seq=0 2024-09-13T16:48:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb90affd640 1 -- 172.21.15.39:0/1106816569 <== mds.0 v2:172.21.15.84:6832/1392588044 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0a14ec790 con 0x55c0a16a5b80 2024-09-13T16:48:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb90affd640 10 client.4749 handle_command_reply: tid=0 2024-09-13T16:48:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 client.4749 shutdown 2024-09-13T16:48:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 -- 172.21.15.39:0/1106816569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 msgr2=0x55c0a16a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a16a4c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb920000f70 tx=0x7fb9200014c0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 -- 172.21.15.39:0/1106816569 shutdown_connections 2024-09-13T16:48:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] conn(0x55c0a16a5b80 0x55c0a16f34e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dec10 0x55c0a16a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a16a4c30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 --2- 172.21.15.39:0/1106816569 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a16a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb90affd640 0 client.4749 ms_handle_reset on v2:172.21.15.84:6832/1392588044 2024-09-13T16:48:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 -- 172.21.15.39:0/1106816569 >> 172.21.15.39:0/1106816569 conn(0x55c0a16dc110 msgr2=0x55c0a16de500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 -- 172.21.15.39:0/1106816569 shutdown_connections 2024-09-13T16:48:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 1 -- 172.21.15.39:0/1106816569 wait complete. 2024-09-13T16:48:26.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.940+0000 7fb935900000 20 client.4749 trim_cache size 0 max 16384 2024-09-13T16:48:26.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb935900000 1 Processor -- start 2024-09-13T16:48:26.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb935900000 1 -- start start 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb935900000 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dc110 0x55c0a173f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb935900000 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a173fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb935900000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a1740190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb935900000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a11ac940 con 0x55c0a169f050 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb935900000 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a1568e80 con 0x55c0a16dc110 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb935900000 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0a14f39b0 con 0x55c0a169e730 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb930d54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a1740190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb930d54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a1740190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58492/0 (socket says 172.21.15.39:58492) 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb930d54640 1 -- 172.21.15.39:0/3021889677 learned_addr learned my addr 172.21.15.39:0/3021889677 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb92bfff640 1 --2- 172.21.15.39:0/3021889677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dc110 0x55c0a173f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb930d54640 1 -- 172.21.15.39:0/3021889677 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 msgr2=0x55c0a173fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb930d54640 1 --2- 172.21.15.39:0/3021889677 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a173fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb930d54640 1 -- 172.21.15.39:0/3021889677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dc110 msgr2=0x55c0a173f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb930d54640 1 --2- 172.21.15.39:0/3021889677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dc110 0x55c0a173f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb930d54640 1 -- 172.21.15.39:0/3021889677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0a169e500 con 0x55c0a169f050 2024-09-13T16:48:26.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.956+0000 7fb930d54640 1 --2- 172.21.15.39:0/3021889677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a1740190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb9240efab0 tx=0x7fb924104000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365a272fd821361 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb9127fc640 1 -- 172.21.15.39:0/3021889677 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240efb80 con 0x55c0a169f050 2024-09-13T16:48:26.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb9127fc640 1 -- 172.21.15.39:0/3021889677 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9240f3be0 con 0x55c0a169f050 2024-09-13T16:48:26.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb9127fc640 1 -- 172.21.15.39:0/3021889677 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240efdb0 con 0x55c0a169f050 2024-09-13T16:48:26.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- 172.21.15.39:0/3021889677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 msgr2=0x55c0a1740190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 --2- 172.21.15.39:0/3021889677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a1740190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb9240efab0 tx=0x7fb924104000 comp rx=0 tx=0).stop 2024-09-13T16:48:26.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- 172.21.15.39:0/3021889677 shutdown_connections 2024-09-13T16:48:26.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 --2- 172.21.15.39:0/3021889677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a169f050 0x55c0a1740190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 --2- 172.21.15.39:0/3021889677 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169e730 0x55c0a173fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 --2- 172.21.15.39:0/3021889677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a16dc110 0x55c0a173f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- 172.21.15.39:0/3021889677 >> 172.21.15.39:0/3021889677 conn(0x55c0a16da910 msgr2=0x55c0a16a6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- 172.21.15.39:0/3021889677 shutdown_connections 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- 172.21.15.39:0/3021889677 wait complete. 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- start start 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 Processor -- start 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a169e730 0x55c0a16f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169f050 0x55c0a16f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a16dc110 0x55c0a16f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a14f7d80 con 0x55c0a16dc110 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a14ed1c0 con 0x55c0a169e730 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0a14f6ad0 con 0x55c0a169f050 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 10 client.0 _ll_register_callbacks cb 0x55c0a1568dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:48:26.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb930d54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a16dc110 0x55c0a16f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb930d54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a16dc110 0x55c0a16f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58506/0 (socket says 172.21.15.39:58506) 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb930d54640 1 -- 172.21.15.39:0/1108572399 learned_addr learned my addr 172.21.15.39:0/1108572399 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb930d54640 1 -- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169f050 msgr2=0x55c0a16f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb930d54640 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169f050 0x55c0a16f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb930d54640 1 -- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a169e730 msgr2=0x55c0a16f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb92bfff640 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a169e730 0x55c0a16f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb930d54640 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a169e730 0x55c0a16f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb930d54640 1 -- 172.21.15.39:0/1108572399 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0a11eb890 con 0x55c0a16dc110 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb930d54640 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a16dc110 0x55c0a16f5880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb924009d90 tx=0x7fb92401c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922d1b5256077ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb90affd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb90affd640 1 -- 172.21.15.39:0/1108572399 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240f0210 con 0x55c0a16dc110 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb90affd640 1 -- 172.21.15.39:0/1108572399 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb9240f81a0 con 0x55c0a16dc110 2024-09-13T16:48:26.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- 172.21.15.39:0/1108572399 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0a14bf460 con 0x55c0a16dc110 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb90affd640 1 -- 172.21.15.39:0/1108572399 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9240efb80 con 0x55c0a16dc110 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 10 client.4755 fetch_fsmap learned FSMap version 14 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- 172.21.15.39:0/1108572399 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55c0a149ac00 con 0x55c0a16dc110 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb90affd640 1 -- 172.21.15.39:0/1108572399 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c0a14bf460 con 0x55c0a16dc110 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb90affd640 1 -- 172.21.15.39:0/1108572399 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 14) v1 ==== 3668+0+0 (secure 0 0 0) 0x7fb924115070 con 0x55c0a16dc110 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 10 client.4755 fetch_fsmap finished waiting for FSMap version 14 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 10 client.4755 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 20 client.4755 populate_metadata read hostname 'smithi039' 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] conn(0x55c0a16dec10 0x55c0a17093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 4 client.4755 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb935900000 1 -- 172.21.15.39:0/1108572399 --> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c0a14b89c0 con 0x55c0a16dec10 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.960+0000 7fb92bfff640 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] conn(0x55c0a16dec10 0x55c0a17093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:26.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb92bfff640 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] conn(0x55c0a16dec10 0x55c0a17093b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21bcbd55d572211e server_cookie=b5de886ef22704d4 in_seq=0 out_seq=0 2024-09-13T16:48:26.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb90affd640 10 client.4755 ms_handle_connect on v2:172.21.15.84:6832/1392588044 2024-09-13T16:48:26.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb90affd640 1 -- 172.21.15.39:0/1108572399 <== mds.0 v2:172.21.15.84:6832/1392588044 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55c0a14b89c0 con 0x55c0a16dec10 2024-09-13T16:48:26.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb90affd640 10 client.4755 handle_command_reply: tid=0 2024-09-13T16:48:26.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 client.4755 shutdown 2024-09-13T16:48:26.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 -- 172.21.15.39:0/1108572399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a16dc110 msgr2=0x55c0a16f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a16dc110 0x55c0a16f5880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb924009d90 tx=0x7fb92401c820 comp rx=0 tx=0).stop 2024-09-13T16:48:26.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 -- 172.21.15.39:0/1108572399 shutdown_connections 2024-09-13T16:48:26.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:6832/1392588044,v1:172.21.15.84:6833/1392588044] conn(0x55c0a16dec10 0x55c0a17093b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0a16dc110 0x55c0a16f5880 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x55c0a169f050 0x55c0a16f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb90affd640 0 client.4755 ms_handle_reset on v2:172.21.15.84:6832/1392588044 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 --2- 172.21.15.39:0/1108572399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55c0a169e730 0x55c0a16f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 -- 172.21.15.39:0/1108572399 >> 172.21.15.39:0/1108572399 conn(0x55c0a16da910 msgr2=0x55c0a1742940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 -- 172.21.15.39:0/1108572399 shutdown_connections 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 1 -- 172.21.15.39:0/1108572399 wait complete. 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb935900000 20 client.4755 trim_cache size 0 max 16384 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T16:48:26.973 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "b9195ac8-73ed-4db1-a594-cde2ff5ccb0f", 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 4264, 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "a", 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 27.940353581, 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 8, 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 24, 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 20, 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 33.756854369999999 2024-09-13T16:48:26.974 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T16:48:26.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 -- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb918066690 msgr2=0x7fb918068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb918066690 0x7fb918068b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb92c155cc0 tx=0x7fb91c008040 comp rx=0 tx=0).stop 2024-09-13T16:48:26.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 -- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb92c14f4b0 msgr2=0x7fb92c149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:26.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb92c14f4b0 0x7fb92c149000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb92401a450 tx=0x7fb9240195c0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 -- 172.21.15.39:0/796629453 shutdown_connections 2024-09-13T16:48:26.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb918066690 0x7fb918068b50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb92c14f4b0 0x7fb92c149000 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb92c0c0c20 0x7fb92c1ea1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 --2- 172.21.15.39:0/796629453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb92c0661c0 0x7fb92c0aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:26.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 -- 172.21.15.39:0/796629453 >> 172.21.15.39:0/796629453 conn(0x7fb92c0c3310 msgr2=0x7fb92c14a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:26.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 -- 172.21.15.39:0/796629453 shutdown_connections 2024-09-13T16:48:26.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:26.964+0000 7fb9327b7640 1 -- 172.21.15.39:0/796629453 wait complete. 2024-09-13T16:48:26.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:27.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.092+0000 7fb01508a640 1 Processor -- start 2024-09-13T16:48:27.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.092+0000 7fb01508a640 1 -- start start 2024-09-13T16:48:27.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.092+0000 7fb01508a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb01014e0c0 0x7fb0100a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.092+0000 7fb01508a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0100c2780 con 0x7fb0100c1f50 2024-09-13T16:48:27.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.092+0000 7fb01508a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0100c2a60 con 0x7fb0100c4740 2024-09-13T16:48:27.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.092+0000 7fb01508a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0100c2d40 con 0x7fb01014e0c0 2024-09-13T16:48:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00e59c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb0100c1f50 0x7fb01006cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57514/0 (socket says 172.21.15.39:57514) 2024-09-13T16:48:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00e59c640 1 -- 172.21.15.39:0/3636983171 learned_addr learned my addr 172.21.15.39:0/3636983171 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00ed9d640 1 --2- 172.21.15.39:0/3636983171 >> v2:172.21.15.84:3300/0 conn(0x7fb01014e0c0 0x7fb0100a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242897148 0 0) 0x7fb0100c2780 con 0x7fb0100c1f50 2024-09-13T16:48:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff8003530 con 0x7fb0100c1f50 2024-09-13T16:48:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3932117544 0 0) 0x7faff8003530 con 0x7fb0100c1f50 2024-09-13T16:48:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 >> v2:172.21.15.84:3300/0 conn(0x7fb01014e0c0 msgr2=0x7fb0100a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:27.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 --2- 172.21.15.39:0/3636983171 >> v2:172.21.15.84:3300/0 conn(0x7fb01014e0c0 0x7fb0100a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 >> v1:172.21.15.84:6789/0 conn(0x7fb0100c4740 legacy=0x7fb0100c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100c3020 con 0x7fb0100c1f50 2024-09-13T16:48:27.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fb004001680 con 0x7fb0100c1f50 2024-09-13T16:48:27.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 >> v1:172.21.15.39:6789/0 conn(0x7fb0100c1f50 legacy=0x7fb01006cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff8003980 0x7faff8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faff80047d0 0x7faff8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faff8005190 0x7faff800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c2d40 con 0x7faff80047d0 2024-09-13T16:48:27.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff800c130 con 0x7faff8005190 2024-09-13T16:48:27.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff800c2f0 con 0x7faff8003980 2024-09-13T16:48:27.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00e59c640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faff80047d0 0x7faff8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00ed9d640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff8003980 0x7faff8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00f59e640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faff8005190 0x7faff800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00e59c640 1 -- 172.21.15.39:0/3636983171 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff8003980 msgr2=0x7faff8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00e59c640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff8003980 0x7faff8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00e59c640 1 -- 172.21.15.39:0/3636983171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faff8005190 msgr2=0x7faff800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00e59c640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faff8005190 0x7faff800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00f59e640 1 -- 172.21.15.39:0/3636983171 reap_dead start 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00e59c640 1 -- 172.21.15.39:0/3636983171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff8003530 con 0x7faff80047d0 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00e59c640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faff80047d0 0x7faff8004c50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb004001060 tx=0x7fb00401cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9110055aa1613c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0040058e0 con 0x7faff80047d0 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb004005a40 con 0x7faff80047d0 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.096+0000 7fb00dd9b640 1 -- 172.21.15.39:0/3636983171 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004005c70 con 0x7faff80047d0 2024-09-13T16:48:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- 172.21.15.39:0/3636983171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faff80047d0 msgr2=0x7faff8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faff80047d0 0x7faff8004c50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb004001060 tx=0x7fb00401cbd0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- 172.21.15.39:0/3636983171 shutdown_connections 2024-09-13T16:48:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 --2- 172.21.15.39:0/3636983171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faff80047d0 0x7faff8004c50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- 172.21.15.39:0/3636983171 >> 172.21.15.39:0/3636983171 conn(0x7fb010069e10 msgr2=0x7fb01006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- 172.21.15.39:0/3636983171 shutdown_connections 2024-09-13T16:48:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- 172.21.15.39:0/3636983171 wait complete. 2024-09-13T16:48:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 Processor -- start 2024-09-13T16:48:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- start start 2024-09-13T16:48:27.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff80047d0 0x7fb01006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0101ee6a0 0x7fb010153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0100bfa80 0x7fb0101ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0101ee280 con 0x7fb0101ee6a0 2024-09-13T16:48:27.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01006c690 con 0x7fb0100bfa80 2024-09-13T16:48:27.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0101edc00 con 0x7faff80047d0 2024-09-13T16:48:27.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0101ee6a0 0x7fb010153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0101ee6a0 0x7fb010153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58528/0 (socket says 172.21.15.39:58528) 2024-09-13T16:48:27.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00e59c640 1 -- 172.21.15.39:0/494260506 learned_addr learned my addr 172.21.15.39:0/494260506 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00f59e640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0100bfa80 0x7fb0101ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00ed9d640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff80047d0 0x7fb01006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00e59c640 1 -- 172.21.15.39:0/494260506 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff80047d0 msgr2=0x7fb01006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00e59c640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff80047d0 0x7fb01006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00e59c640 1 -- 172.21.15.39:0/494260506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0100bfa80 msgr2=0x7fb0101ea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00e59c640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0100bfa80 0x7fb0101ea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00e59c640 1 -- 172.21.15.39:0/494260506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010154d60 con 0x7fb0101ee6a0 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00ed9d640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff80047d0 0x7fb01006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb00e59c640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0101ee6a0 0x7fb010153790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb0040ab040 tx=0x7faff8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7faff77fe640 1 -- 172.21.15.39:0/494260506 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004006f00 con 0x7fb0101ee6a0 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7faff77fe640 1 -- 172.21.15.39:0/494260506 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0040047f0 con 0x7fb0101ee6a0 2024-09-13T16:48:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7faff77fe640 1 -- 172.21.15.39:0/494260506 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004004b60 con 0x7fb0101ee6a0 2024-09-13T16:48:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01006c3b0 con 0x7fb0101ee6a0 2024-09-13T16:48:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.100+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb010153cd0 con 0x7fb0101ee6a0 2024-09-13T16:48:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.104+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0101eda60 con 0x7fb0101ee6a0 2024-09-13T16:48:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.104+0000 7faff77fe640 1 -- 172.21.15.39:0/494260506 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb004004cc0 con 0x7fb0101ee6a0 2024-09-13T16:48:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.104+0000 7faff77fe640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faffc0666c0 0x7faffc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.104+0000 7fb00ed9d640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faffc0666c0 0x7faffc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.104+0000 7faff77fe640 1 -- 172.21.15.39:0/494260506 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb00411dcc0 con 0x7fb0101ee6a0 2024-09-13T16:48:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.104+0000 7fb00ed9d640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faffc0666c0 0x7faffc068b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb000009aa0 tx=0x7fb000008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:27.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.108+0000 7faff77fe640 1 -- 172.21.15.39:0/494260506 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb0101eda60 con 0x7fb0101ee6a0 2024-09-13T16:48:27.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.268+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0101eda60 con 0x7fb0101ee6a0 2024-09-13T16:48:27.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.268+0000 7faff77fe640 1 -- 172.21.15.39:0/494260506 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7fb0101eda60 con 0x7fb0101ee6a0 2024-09-13T16:48:27.273 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:27.273 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faffc0666c0 msgr2=0x7faffc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faffc0666c0 0x7faffc068b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb000009aa0 tx=0x7fb000008040 comp rx=0 tx=0).stop 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0101ee6a0 msgr2=0x7fb010153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0101ee6a0 0x7fb010153790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb0040ab040 tx=0x7faff8005800 comp rx=0 tx=0).stop 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 shutdown_connections 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faffc0666c0 0x7faffc068b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb0100bfa80 0x7fb0101ea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0101ee6a0 0x7fb010153790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 --2- 172.21.15.39:0/494260506 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faff80047d0 0x7fb01006bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 >> 172.21.15.39:0/494260506 conn(0x7fb010069e10 msgr2=0x7fb01014f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:27.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 shutdown_connections 2024-09-13T16:48:27.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.272+0000 7fb01508a640 1 -- 172.21.15.39:0/494260506 wait complete. 2024-09-13T16:48:27.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:48:27.292 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:48:27.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.400+0000 7f59fc979640 1 Processor -- start 2024-09-13T16:48:27.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59fc979640 1 -- start start 2024-09-13T16:48:27.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59fc979640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c0ba0 0x7f59f80c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59fc979640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59f8151200 0x7f59f80c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59fc979640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f80a5280 con 0x7f59f80c1810 2024-09-13T16:48:27.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59fc979640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f80c20e0 con 0x7f59f8151200 2024-09-13T16:48:27.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59fc979640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80a4cc0 con 0x7f59f80c0ba0 2024-09-13T16:48:27.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f6d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59f8151200 0x7f59f80c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f6d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59f8151200 0x7f59f80c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58544/0 (socket says 172.21.15.39:58544) 2024-09-13T16:48:27.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f6d9d640 1 -- 172.21.15.39:0/2849685779 learned_addr learned my addr 172.21.15.39:0/2849685779 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:27.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f5d9b640 1 --2- 172.21.15.39:0/2849685779 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c0ba0 0x7f59f80c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f6d9d640 1 -- 172.21.15.39:0/2849685779 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c0ba0 msgr2=0x7f59f80c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:27.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f6d9d640 1 --2- 172.21.15.39:0/2849685779 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c0ba0 0x7f59f80c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f6d9d640 1 -- 172.21.15.39:0/2849685779 >> v1:172.21.15.84:6789/0 conn(0x7f59f80c1810 legacy=0x7f59f8150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f6d9d640 1 -- 172.21.15.39:0/2849685779 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f80a4fa0 con 0x7f59f8151200 2024-09-13T16:48:27.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f6d9d640 1 --2- 172.21.15.39:0/2849685779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f8151200 0x7f59f80c3cd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009e50 tx=0x7f59f001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11e4c4033c98d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:27.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f559a640 1 -- 172.21.15.39:0/2849685779 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f00a58d0 con 0x7f59f8151200 2024-09-13T16:48:27.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f559a640 1 -- 172.21.15.39:0/2849685779 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f00a5a30 con 0x7f59f8151200 2024-09-13T16:48:27.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.404+0000 7f59f559a640 1 -- 172.21.15.39:0/2849685779 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f00a5dc0 con 0x7f59f8151200 2024-09-13T16:48:27.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- 172.21.15.39:0/2849685779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f8151200 msgr2=0x7f59f80c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 --2- 172.21.15.39:0/2849685779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f8151200 0x7f59f80c3cd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009e50 tx=0x7f59f001ca00 comp rx=0 tx=0).stop 2024-09-13T16:48:27.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- 172.21.15.39:0/2849685779 shutdown_connections 2024-09-13T16:48:27.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 --2- 172.21.15.39:0/2849685779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f8151200 0x7f59f80c3cd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 --2- 172.21.15.39:0/2849685779 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c0ba0 0x7f59f80c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- 172.21.15.39:0/2849685779 >> 172.21.15.39:0/2849685779 conn(0x7f59f80c5790 msgr2=0x7f59f80c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:27.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- 172.21.15.39:0/2849685779 shutdown_connections 2024-09-13T16:48:27.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- 172.21.15.39:0/2849685779 wait complete. 2024-09-13T16:48:27.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 Processor -- start 2024-09-13T16:48:27.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- start start 2024-09-13T16:48:27.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f80c0ba0 0x7f59f80becf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c1810 0x7f59f80a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59f8151200 0x7f59f80a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f81eaf20 con 0x7f59f80c0ba0 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80ba7b0 con 0x7f59f8151200 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80baa90 con 0x7f59f80c1810 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f80c0ba0 0x7f59f80becf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f80c0ba0 0x7f59f80becf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58556/0 (socket says 172.21.15.39:58556) 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f659c640 1 -- 172.21.15.39:0/3473393198 learned_addr learned my addr 172.21.15.39:0/3473393198 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f5d9b640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c1810 0x7f59f80a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f6d9d640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59f8151200 0x7f59f80a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f659c640 1 -- 172.21.15.39:0/3473393198 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c1810 msgr2=0x7f59f80a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f659c640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c1810 0x7f59f80a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f659c640 1 -- 172.21.15.39:0/3473393198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59f8151200 msgr2=0x7f59f80a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f659c640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59f8151200 0x7f59f80a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f659c640 1 -- 172.21.15.39:0/3473393198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f0009b30 con 0x7f59f80c0ba0 2024-09-13T16:48:27.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59f659c640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f80c0ba0 0x7f59f80becf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00e710 tx=0x7f59ec00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:27.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59e6ffd640 1 -- 172.21.15.39:0/3473393198 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec0052d0 con 0x7f59f80c0ba0 2024-09-13T16:48:27.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59e6ffd640 1 -- 172.21.15.39:0/3473393198 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec005430 con 0x7f59f80c0ba0 2024-09-13T16:48:27.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f80bf6f0 con 0x7f59f80c0ba0 2024-09-13T16:48:27.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59e6ffd640 1 -- 172.21.15.39:0/3473393198 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec00cca0 con 0x7f59f80c0ba0 2024-09-13T16:48:27.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.408+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f80a99c0 con 0x7f59f80c0ba0 2024-09-13T16:48:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.412+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f8150af0 con 0x7f59f80c0ba0 2024-09-13T16:48:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.412+0000 7f59e6ffd640 1 -- 172.21.15.39:0/3473393198 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59ec003600 con 0x7f59f80c0ba0 2024-09-13T16:48:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.412+0000 7f59e6ffd640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59d80666c0 0x7f59d8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.412+0000 7f59e6ffd640 1 -- 172.21.15.39:0/3473393198 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f59ec086640 con 0x7f59f80c0ba0 2024-09-13T16:48:27.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.412+0000 7f59f5d9b640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59d80666c0 0x7f59d8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.412+0000 7f59f5d9b640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59d80666c0 0x7f59d8068b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59f80c5fc0 tx=0x7f59e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:27.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.416+0000 7f59e6ffd640 1 -- 172.21.15.39:0/3473393198 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59ec053710 con 0x7f59f80c0ba0 2024-09-13T16:48:27.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59f80a8360 con 0x7f59f80c0ba0 2024-09-13T16:48:27.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59e6ffd640 1 -- 172.21.15.39:0/3473393198 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59ec056d30 con 0x7f59f80c0ba0 2024-09-13T16:48:27.572 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:48:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59d80666c0 msgr2=0x7f59d8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59d80666c0 0x7f59d8068b80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59f80c5fc0 tx=0x7f59e8008040 comp rx=0 tx=0).stop 2024-09-13T16:48:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f80c0ba0 msgr2=0x7f59f80becf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f80c0ba0 0x7f59f80becf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00e710 tx=0x7f59ec00ebd0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 shutdown_connections 2024-09-13T16:48:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59d80666c0 0x7f59d8068b80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59f8151200 0x7f59f80a8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59f80c1810 0x7f59f80a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 --2- 172.21.15.39:0/3473393198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59f80c0ba0 0x7f59f80becf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 >> 172.21.15.39:0/3473393198 conn(0x7f59f80c5790 msgr2=0x7f59f814f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:27.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 shutdown_connections 2024-09-13T16:48:27.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.568+0000 7f59fc979640 1 -- 172.21.15.39:0/3473393198 wait complete. 2024-09-13T16:48:27.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T16:48:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.712+0000 7f201c403640 1 Processor -- start 2024-09-13T16:48:27.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.712+0000 7f201c403640 1 -- start start 2024-09-13T16:48:27.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.712+0000 7f201c403640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f201414e530 0x7f20140c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.712+0000 7f201c403640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2014151070 0x7f20141514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.712+0000 7f201c403640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2014150b70 con 0x7f20141540e0 2024-09-13T16:48:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.712+0000 7f201c403640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2014154570 con 0x7f2014151070 2024-09-13T16:48:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.712+0000 7f201c403640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2014154850 con 0x7f201414e530 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201a19f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f201414e530 0x7f20140c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201a9a0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2014151070 0x7f20141514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201999e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f20141540e0 0x7f20140c47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:43708/0 (socket says 172.21.15.39:43708) 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201999e640 1 -- 172.21.15.39:0/1517249831 learned_addr learned my addr 172.21.15.39:0/1517249831 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201a19f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f201414e530 0x7f20140c1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45308/0 (socket says 172.21.15.39:45308) 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201919d640 1 -- 172.21.15.39:0/1517249831 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202860677 0 0) 0x7f2014150b70 con 0x7f20141540e0 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201919d640 1 -- 172.21.15.39:0/1517249831 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2000003530 con 0x7f20141540e0 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201a19f640 1 -- 172.21.15.39:0/1517249831 >> v2:172.21.15.84:3300/0 conn(0x7f2014151070 msgr2=0x7f20141514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201a19f640 1 --2- 172.21.15.39:0/1517249831 >> v2:172.21.15.84:3300/0 conn(0x7f2014151070 0x7f20141514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201a19f640 1 -- 172.21.15.39:0/1517249831 >> v1:172.21.15.84:6789/0 conn(0x7f20141540e0 legacy=0x7f20140c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201a19f640 1 -- 172.21.15.39:0/1517249831 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014154b30 con 0x7f201414e530 2024-09-13T16:48:27.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201a19f640 1 --2- 172.21.15.39:0/1517249831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f201414e530 0x7f20140c1660 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f20100097c0 tx=0x7f201001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35bf241958ba364d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:27.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201919d640 1 -- 172.21.15.39:0/1517249831 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20100ac620 con 0x7f201414e530 2024-09-13T16:48:27.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201919d640 1 -- 172.21.15.39:0/1517249831 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20100ac780 con 0x7f201414e530 2024-09-13T16:48:27.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.716+0000 7f201919d640 1 -- 172.21.15.39:0/1517249831 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20100acb10 con 0x7f201414e530 2024-09-13T16:48:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- 172.21.15.39:0/1517249831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f201414e530 msgr2=0x7f20140c1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 --2- 172.21.15.39:0/1517249831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f201414e530 0x7f20140c1660 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f20100097c0 tx=0x7f201001c3e0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- 172.21.15.39:0/1517249831 shutdown_connections 2024-09-13T16:48:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 --2- 172.21.15.39:0/1517249831 >> v2:172.21.15.84:3300/0 conn(0x7f2014151070 0x7f20141514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 --2- 172.21.15.39:0/1517249831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f201414e530 0x7f20140c1660 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- 172.21.15.39:0/1517249831 >> 172.21.15.39:0/1517249831 conn(0x7f20140c5790 msgr2=0x7f20140c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- 172.21.15.39:0/1517249831 shutdown_connections 2024-09-13T16:48:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- 172.21.15.39:0/1517249831 wait complete. 2024-09-13T16:48:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 Processor -- start 2024-09-13T16:48:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- start start 2024-09-13T16:48:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201414e530 0x7f2014140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2014151070 0x7f2014142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20141438b0 0x7f20141432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20141ea350 con 0x7f20141438b0 2024-09-13T16:48:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2014153650 con 0x7f201414e530 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20141ea540 con 0x7f2014151070 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a9a0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20141438b0 0x7f20141432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a9a0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20141438b0 0x7f20141432c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58566/0 (socket says 172.21.15.39:58566) 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a9a0640 1 -- 172.21.15.39:0/1770901351 learned_addr learned my addr 172.21.15.39:0/1770901351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a19f640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201414e530 0x7f2014140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201999e640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2014151070 0x7f2014142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a9a0640 1 -- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2014151070 msgr2=0x7f2014142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a9a0640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2014151070 0x7f2014142d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a9a0640 1 -- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201414e530 msgr2=0x7f2014140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a9a0640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201414e530 0x7f2014140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a9a0640 1 -- 172.21.15.39:0/1770901351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010009420 con 0x7f20141438b0 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201999e640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2014151070 0x7f2014142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a19f640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201414e530 0x7f2014140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201a9a0640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20141438b0 0x7f20141432c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f200c00efc0 tx=0x7f200c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f200affd640 1 -- 172.21.15.39:0/1770901351 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c009ba0 con 0x7f20141438b0 2024-09-13T16:48:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f200affd640 1 -- 172.21.15.39:0/1770901351 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200c00f040 con 0x7f20141438b0 2024-09-13T16:48:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20141ea820 con 0x7f20141438b0 2024-09-13T16:48:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f200affd640 1 -- 172.21.15.39:0/1770901351 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200c0146c0 con 0x7f20141438b0 2024-09-13T16:48:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.720+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2014146980 con 0x7f20141438b0 2024-09-13T16:48:27.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.724+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20141ead40 con 0x7f20141438b0 2024-09-13T16:48:27.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.724+0000 7f200affd640 1 -- 172.21.15.39:0/1770901351 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f200c014820 con 0x7f20141438b0 2024-09-13T16:48:27.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.724+0000 7f200affd640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ff8066690 0x7f1ff8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:27.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.724+0000 7f200affd640 1 -- 172.21.15.39:0/1770901351 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f200c084f80 con 0x7f20141438b0 2024-09-13T16:48:27.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.724+0000 7f201a19f640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ff8066690 0x7f1ff8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:27.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.724+0000 7f201a19f640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ff8066690 0x7f1ff8068b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f20100ae000 tx=0x7f2010008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:27.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.728+0000 7f200affd640 1 -- 172.21.15.39:0/1770901351 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f200c0b6db0 con 0x7f20141438b0 2024-09-13T16:48:27.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.888+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20140c1660 con 0x7f20141438b0 2024-09-13T16:48:27.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.888+0000 7f200affd640 1 -- 172.21.15.39:0/1770901351 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7f200c0180a0 con 0x7f20141438b0 2024-09-13T16:48:27.894 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ff8066690 msgr2=0x7f1ff8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ff8066690 0x7f1ff8068b50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f20100ae000 tx=0x7f2010008040 comp rx=0 tx=0).stop 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20141438b0 msgr2=0x7f20141432c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20141438b0 0x7f20141432c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f200c00efc0 tx=0x7f200c00c490 comp rx=0 tx=0).stop 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 shutdown_connections 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ff8066690 0x7f1ff8068b50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20141438b0 0x7f20141432c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2014151070 0x7f2014142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 --2- 172.21.15.39:0/1770901351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f201414e530 0x7f2014140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 >> 172.21.15.39:0/1770901351 conn(0x7f20140c5790 msgr2=0x7f201406bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:27.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 shutdown_connections 2024-09-13T16:48:27.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:27.892+0000 7f201c403640 1 -- 172.21.15.39:0/1770901351 wait complete. 2024-09-13T16:48:27.912 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T16:48:28.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.024+0000 7fbf4279e640 1 Processor -- start 2024-09-13T16:48:28.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.024+0000 7fbf4279e640 1 -- start start 2024-09-13T16:48:28.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf4279e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c0c7190 0x7fbf3c0c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf4279e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c06de90 con 0x7fbf3c071f00 2024-09-13T16:48:28.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf4279e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c06f2f0 con 0x7fbf3c06edf0 2024-09-13T16:48:28.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf4279e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0c6170 con 0x7fbf3c0c7190 2024-09-13T16:48:28.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf3bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbf3c071f00 0x7fbf3c0c4090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57520/0 (socket says 172.21.15.39:57520) 2024-09-13T16:48:28.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf3bfff640 1 -- 172.21.15.39:0/320760113 learned_addr learned my addr 172.21.15.39:0/320760113 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:28.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf40d3b640 1 --2- 172.21.15.39:0/320760113 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c0c7190 0x7fbf3c0c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf3b7fe640 1 -- 172.21.15.39:0/320760113 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848870307 0 0) 0x7fbf3c06de90 con 0x7fbf3c071f00 2024-09-13T16:48:28.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf3b7fe640 1 -- 172.21.15.39:0/320760113 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf28003530 con 0x7fbf3c071f00 2024-09-13T16:48:28.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf3b7fe640 1 -- 172.21.15.39:0/320760113 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442681848 0 0) 0x7fbf3c06f2f0 con 0x7fbf3c06edf0 2024-09-13T16:48:28.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf3b7fe640 1 -- 172.21.15.39:0/320760113 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf3c06de90 con 0x7fbf3c06edf0 2024-09-13T16:48:28.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf40d3b640 1 -- 172.21.15.39:0/320760113 >> v1:172.21.15.84:6789/0 conn(0x7fbf3c06edf0 legacy=0x7fbf3c0c2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf40d3b640 1 -- 172.21.15.39:0/320760113 >> v1:172.21.15.39:6789/0 conn(0x7fbf3c071f00 legacy=0x7fbf3c0c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf40d3b640 1 -- 172.21.15.39:0/320760113 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0c5e20 con 0x7fbf3c0c7190 2024-09-13T16:48:28.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf40d3b640 1 --2- 172.21.15.39:0/320760113 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c0c7190 0x7fbf3c0c2b80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009dc0 tx=0x7fbf3401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d880afb86f22ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf3b7fe640 1 -- 172.21.15.39:0/320760113 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf340a58e0 con 0x7fbf3c0c7190 2024-09-13T16:48:28.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf3b7fe640 1 -- 172.21.15.39:0/320760113 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf340a5a40 con 0x7fbf3c0c7190 2024-09-13T16:48:28.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.028+0000 7fbf3b7fe640 1 -- 172.21.15.39:0/320760113 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf340a5db0 con 0x7fbf3c0c7190 2024-09-13T16:48:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- 172.21.15.39:0/320760113 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c0c7190 msgr2=0x7fbf3c0c2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 --2- 172.21.15.39:0/320760113 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c0c7190 0x7fbf3c0c2b80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009dc0 tx=0x7fbf3401c900 comp rx=0 tx=0).stop 2024-09-13T16:48:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- 172.21.15.39:0/320760113 shutdown_connections 2024-09-13T16:48:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 --2- 172.21.15.39:0/320760113 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c0c7190 0x7fbf3c0c2b80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- 172.21.15.39:0/320760113 >> 172.21.15.39:0/320760113 conn(0x7fbf3c06c020 msgr2=0x7fbf3c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- 172.21.15.39:0/320760113 shutdown_connections 2024-09-13T16:48:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- 172.21.15.39:0/320760113 wait complete. 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 Processor -- start 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- start start 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c06edf0 0x7fbf3c0a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c071f00 0x7fbf3c14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf3c0c7190 0x7fbf3c14dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0aacb0 con 0x7fbf3c071f00 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c1edcd0 con 0x7fbf3c0c7190 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c1edf70 con 0x7fbf3c06edf0 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf33fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c071f00 0x7fbf3c14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf33fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c071f00 0x7fbf3c14d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58572/0 (socket says 172.21.15.39:58572) 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf33fff640 1 -- 172.21.15.39:0/1664460770 learned_addr learned my addr 172.21.15.39:0/1664460770 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf3bfff640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c06edf0 0x7fbf3c0a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf40d3b640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf3c0c7190 0x7fbf3c14dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf33fff640 1 -- 172.21.15.39:0/1664460770 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c06edf0 msgr2=0x7fbf3c0a9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf33fff640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c06edf0 0x7fbf3c0a9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf33fff640 1 -- 172.21.15.39:0/1664460770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf3c0c7190 msgr2=0x7fbf3c14dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf33fff640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf3c0c7190 0x7fbf3c14dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf33fff640 1 -- 172.21.15.39:0/1664460770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf34009a70 con 0x7fbf3c071f00 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf33fff640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c071f00 0x7fbf3c14d810 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400c430 tx=0x7fbf2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf397fa640 1 -- 172.21.15.39:0/1664460770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf240052d0 con 0x7fbf3c071f00 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf397fa640 1 -- 172.21.15.39:0/1664460770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf24005450 con 0x7fbf3c071f00 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c1ee250 con 0x7fbf3c071f00 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf397fa640 1 -- 172.21.15.39:0/1664460770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24003840 con 0x7fbf3c071f00 2024-09-13T16:48:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c1ee810 con 0x7fbf3c071f00 2024-09-13T16:48:28.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.032+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c0ace40 con 0x7fbf3c071f00 2024-09-13T16:48:28.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.036+0000 7fbf397fa640 1 -- 172.21.15.39:0/1664460770 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf240039a0 con 0x7fbf3c071f00 2024-09-13T16:48:28.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.036+0000 7fbf397fa640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf1c0666c0 0x7fbf1c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.036+0000 7fbf3bfff640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf1c0666c0 0x7fbf1c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.036+0000 7fbf397fa640 1 -- 172.21.15.39:0/1664460770 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fbf24083b10 con 0x7fbf3c071f00 2024-09-13T16:48:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.036+0000 7fbf3bfff640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf1c0666c0 0x7fbf1c068b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0011f0 tx=0x7fbf2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.040+0000 7fbf397fa640 1 -- 172.21.15.39:0/1664460770 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf24050b50 con 0x7fbf3c071f00 2024-09-13T16:48:28.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.196+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7fbf3c1eea00 con 0x7fbf3c071f00 2024-09-13T16:48:28.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf397fa640 1 -- 172.21.15.39:0/1664460770 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7fbf240170a0 con 0x7fbf3c071f00 2024-09-13T16:48:28.203 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:28.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf1c0666c0 msgr2=0x7fbf1c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf1c0666c0 0x7fbf1c068b80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0011f0 tx=0x7fbf2c009040 comp rx=0 tx=0).stop 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c071f00 msgr2=0x7fbf3c14d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c071f00 0x7fbf3c14d810 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400c430 tx=0x7fbf2400c8f0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 shutdown_connections 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf1c0666c0 0x7fbf1c068b80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf3c0c7190 0x7fbf3c14dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c071f00 0x7fbf3c14d810 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 --2- 172.21.15.39:0/1664460770 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf3c06edf0 0x7fbf3c0a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 >> 172.21.15.39:0/1664460770 conn(0x7fbf3c06c020 msgr2=0x7fbf3c06c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 shutdown_connections 2024-09-13T16:48:28.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.200+0000 7fbf4279e640 1 -- 172.21.15.39:0/1664460770 wait complete. 2024-09-13T16:48:28.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T16:48:28.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.344+0000 7feadffde640 1 Processor -- start 2024-09-13T16:48:28.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.344+0000 7feadffde640 1 -- start start 2024-09-13T16:48:28.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadffde640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 0x7fead80c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadffde640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fead81540e0 0x7fead80c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadffde640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead8150b70 con 0x7fead80c4ce0 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadffde640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead8154570 con 0x7fead81540e0 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadffde640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead8154850 con 0x7fead814e530 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadd579640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fead81540e0 0x7fead80c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadd579640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fead81540e0 0x7fead80c47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58574/0 (socket says 172.21.15.39:58574) 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadd579640 1 -- 172.21.15.39:0/3424646512 learned_addr learned my addr 172.21.15.39:0/3424646512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feaddd7a640 1 --2- 172.21.15.39:0/3424646512 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 0x7fead80c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadd579640 1 -- 172.21.15.39:0/3424646512 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 msgr2=0x7fead80c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadd579640 1 --2- 172.21.15.39:0/3424646512 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 0x7fead80c1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadd579640 1 -- 172.21.15.39:0/3424646512 >> v1:172.21.15.84:6789/0 conn(0x7fead80c4ce0 legacy=0x7fead8151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadd579640 1 -- 172.21.15.39:0/3424646512 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8154b30 con 0x7fead81540e0 2024-09-13T16:48:28.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadd579640 1 --2- 172.21.15.39:0/3424646512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81540e0 0x7fead80c47a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fead4009870 tx=0x7fead401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3d6667332961c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.348+0000 7feadcd78640 1 -- 172.21.15.39:0/3424646512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead40ac410 con 0x7fead81540e0 2024-09-13T16:48:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadcd78640 1 -- 172.21.15.39:0/3424646512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead40ac570 con 0x7fead81540e0 2024-09-13T16:48:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadcd78640 1 -- 172.21.15.39:0/3424646512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead40ac900 con 0x7fead81540e0 2024-09-13T16:48:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- 172.21.15.39:0/3424646512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81540e0 msgr2=0x7fead80c47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 --2- 172.21.15.39:0/3424646512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81540e0 0x7fead80c47a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fead4009870 tx=0x7fead401c4a0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- 172.21.15.39:0/3424646512 shutdown_connections 2024-09-13T16:48:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 --2- 172.21.15.39:0/3424646512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81540e0 0x7fead80c47a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 --2- 172.21.15.39:0/3424646512 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 0x7fead80c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- 172.21.15.39:0/3424646512 >> 172.21.15.39:0/3424646512 conn(0x7fead80c5790 msgr2=0x7fead80c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- 172.21.15.39:0/3424646512 shutdown_connections 2024-09-13T16:48:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- 172.21.15.39:0/3424646512 wait complete. 2024-09-13T16:48:28.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 Processor -- start 2024-09-13T16:48:28.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- start start 2024-09-13T16:48:28.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead80c4ce0 0x7fead81ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 0x7fead8144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81ec460 0x7fead81452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead8145940 con 0x7fead81ec460 2024-09-13T16:48:28.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead80a67d0 con 0x7fead80c4ce0 2024-09-13T16:48:28.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead8145e70 con 0x7fead814e530 2024-09-13T16:48:28.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feade57b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81ec460 0x7fead81452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feade57b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81ec460 0x7fead81452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58582/0 (socket says 172.21.15.39:58582) 2024-09-13T16:48:28.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feade57b640 1 -- 172.21.15.39:0/2177916869 learned_addr learned my addr 172.21.15.39:0/2177916869 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:28.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feaddd7a640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead80c4ce0 0x7fead81ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadd579640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 0x7fead8144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feade57b640 1 -- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 msgr2=0x7fead8144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feade57b640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 0x7fead8144d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feade57b640 1 -- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead80c4ce0 msgr2=0x7fead81ebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feade57b640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead80c4ce0 0x7fead81ebf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feade57b640 1 -- 172.21.15.39:0/2177916869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead40094d0 con 0x7fead81ec460 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadd579640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 0x7fead8144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feaddd7a640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead80c4ce0 0x7fead81ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feade57b640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81ec460 0x7fead81452c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fead000c770 tx=0x7fead000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feaca7fc640 1 -- 172.21.15.39:0/2177916869 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0003b80 con 0x7fead81ec460 2024-09-13T16:48:28.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feaca7fc640 1 -- 172.21.15.39:0/2177916869 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead0003ce0 con 0x7fead81ec460 2024-09-13T16:48:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feaca7fc640 1 -- 172.21.15.39:0/2177916869 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead000aa30 con 0x7fead81ec460 2024-09-13T16:48:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead81ec9e0 con 0x7fead81ec460 2024-09-13T16:48:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.352+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead80c2b30 con 0x7fead81ec460 2024-09-13T16:48:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.356+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead80a6980 con 0x7fead81ec460 2024-09-13T16:48:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.356+0000 7feaca7fc640 1 -- 172.21.15.39:0/2177916869 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fead000ab90 con 0x7fead81ec460 2024-09-13T16:48:28.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.356+0000 7feaca7fc640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feabc066690 0x7feabc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.356+0000 7feaca7fc640 1 -- 172.21.15.39:0/2177916869 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fead0084310 con 0x7fead81ec460 2024-09-13T16:48:28.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.356+0000 7feaddd7a640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feabc066690 0x7feabc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.356+0000 7feaddd7a640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feabc066690 0x7feabc068b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feacc007a70 tx=0x7feacc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.360+0000 7feaca7fc640 1 -- 172.21.15.39:0/2177916869 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fead80a6980 con 0x7fead81ec460 2024-09-13T16:48:28.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.520+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fead80a6980 con 0x7fead81ec460 2024-09-13T16:48:28.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.520+0000 7feaca7fc640 1 -- 172.21.15.39:0/2177916869 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7fead80a6980 con 0x7fead81ec460 2024-09-13T16:48:28.525 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:28.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.520+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feabc066690 msgr2=0x7feabc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.520+0000 7feadffde640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feabc066690 0x7feabc068b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feacc007a70 tx=0x7feacc008040 comp rx=0 tx=0).stop 2024-09-13T16:48:28.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.520+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81ec460 msgr2=0x7fead81452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.520+0000 7feadffde640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81ec460 0x7fead81452c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fead000c770 tx=0x7fead000cc30 comp rx=0 tx=0).stop 2024-09-13T16:48:28.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.524+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 shutdown_connections 2024-09-13T16:48:28.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.524+0000 7feadffde640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feabc066690 0x7feabc068b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.524+0000 7feadffde640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead81ec460 0x7fead81452c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.524+0000 7feadffde640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fead814e530 0x7fead8144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.524+0000 7feadffde640 1 --2- 172.21.15.39:0/2177916869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fead80c4ce0 0x7fead81ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.524+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 >> 172.21.15.39:0/2177916869 conn(0x7fead80c5790 msgr2=0x7fead80c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:28.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.524+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 shutdown_connections 2024-09-13T16:48:28.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.524+0000 7feadffde640 1 -- 172.21.15.39:0/2177916869 wait complete. 2024-09-13T16:48:28.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T16:48:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158acdf640 1 Processor -- start 2024-09-13T16:48:28.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158acdf640 1 -- start start 2024-09-13T16:48:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158acdf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1584151200 0x7f15840c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158acdf640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840a5280 con 0x7f15840c0ba0 2024-09-13T16:48:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158acdf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840a4cc0 con 0x7f15840c1810 2024-09-13T16:48:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158acdf640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15840c20e0 con 0x7f1584151200 2024-09-13T16:48:28.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158927c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1584151200 0x7f15840c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f1583fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f15840c0ba0 0x7f15840c3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57536/0 (socket says 172.21.15.39:57536) 2024-09-13T16:48:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f1583fff640 1 -- 172.21.15.39:0/3246015436 learned_addr learned my addr 172.21.15.39:0/3246015436 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158927c640 1 -- 172.21.15.39:0/3246015436 >> v1:172.21.15.84:6789/0 conn(0x7f15840c1810 legacy=0x7f1584150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158927c640 1 -- 172.21.15.39:0/3246015436 >> v1:172.21.15.39:6789/0 conn(0x7f15840c0ba0 legacy=0x7f15840c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f158927c640 1 -- 172.21.15.39:0/3246015436 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840a4fa0 con 0x7f1584151200 2024-09-13T16:48:28.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.672+0000 7f15837fe640 1 -- 172.21.15.39:0/3246015436 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422835609 0 0) 0x7f15840a5280 con 0x7f15840c0ba0 2024-09-13T16:48:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158927c640 1 --2- 172.21.15.39:0/3246015436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 0x7f15840c3ea0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f157c00a960 tx=0x7f157c01d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef667e12a64eab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f15837fe640 1 -- 172.21.15.39:0/3246015436 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0a6870 con 0x7f1584151200 2024-09-13T16:48:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f15837fe640 1 -- 172.21.15.39:0/3246015436 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c0a69d0 con 0x7f1584151200 2024-09-13T16:48:28.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f15837fe640 1 -- 172.21.15.39:0/3246015436 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0a6d60 con 0x7f1584151200 2024-09-13T16:48:28.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 -- 172.21.15.39:0/3246015436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 msgr2=0x7f15840c3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 --2- 172.21.15.39:0/3246015436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 0x7f15840c3ea0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f157c00a960 tx=0x7f157c01d840 comp rx=0 tx=0).stop 2024-09-13T16:48:28.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 -- 172.21.15.39:0/3246015436 shutdown_connections 2024-09-13T16:48:28.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 --2- 172.21.15.39:0/3246015436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 0x7f15840c3ea0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 -- 172.21.15.39:0/3246015436 >> 172.21.15.39:0/3246015436 conn(0x7f15840c5790 msgr2=0x7f15840c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:28.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 -- 172.21.15.39:0/3246015436 shutdown_connections 2024-09-13T16:48:28.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 -- 172.21.15.39:0/3246015436 wait complete. 2024-09-13T16:48:28.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 Processor -- start 2024-09-13T16:48:28.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 -- start start 2024-09-13T16:48:28.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15840c0ba0 0x7f15840bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15840c1810 0x7f15841ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 0x7f15841ed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1584072310 con 0x7f1584151200 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1584071de0 con 0x7f15840c1810 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158acdf640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158414bf20 con 0x7f15840c0ba0 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158927c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 0x7f15841ed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158927c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 0x7f15841ed4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58608/0 (socket says 172.21.15.39:58608) 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.676+0000 7f158927c640 1 -- 172.21.15.39:0/1841726258 learned_addr learned my addr 172.21.15.39:0/1841726258 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f1583fff640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15840c1810 0x7f15841ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f1588a7b640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15840c0ba0 0x7f15840bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f158927c640 1 -- 172.21.15.39:0/1841726258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15840c0ba0 msgr2=0x7f15840bebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f158927c640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15840c0ba0 0x7f15840bebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f158927c640 1 -- 172.21.15.39:0/1841726258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15840c1810 msgr2=0x7f15841ecf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f158927c640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15840c1810 0x7f15841ecf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f158927c640 1 -- 172.21.15.39:0/1841726258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c00a5c0 con 0x7f1584151200 2024-09-13T16:48:28.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f1588a7b640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15840c0ba0 0x7f15840bebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:28.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f158927c640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 0x7f15841ed4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f157c00a960 tx=0x7f157c01a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f15817fa640 1 -- 172.21.15.39:0/1841726258 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0ae850 con 0x7f1584151200 2024-09-13T16:48:28.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f15817fa640 1 -- 172.21.15.39:0/1841726258 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c0ae9b0 con 0x7f1584151200 2024-09-13T16:48:28.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f15817fa640 1 -- 172.21.15.39:0/1841726258 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0aed40 con 0x7f1584151200 2024-09-13T16:48:28.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15841ecaa0 con 0x7f1584151200 2024-09-13T16:48:28.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15841f0ab0 con 0x7f1584151200 2024-09-13T16:48:28.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15841f0910 con 0x7f1584151200 2024-09-13T16:48:28.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f15817fa640 1 -- 172.21.15.39:0/1841726258 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f157c0b5040 con 0x7f1584151200 2024-09-13T16:48:28.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f15817fa640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1570066690 0x7f1570068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f1588a7b640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1570066690 0x7f1570068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f15817fa640 1 -- 172.21.15.39:0/1841726258 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f157c11ea90 con 0x7f1584151200 2024-09-13T16:48:28.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.680+0000 7f1588a7b640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1570066690 0x7f1570068b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f15740012f0 tx=0x7f1574009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.684+0000 7f15817fa640 1 -- 172.21.15.39:0/1841726258 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15841f0910 con 0x7f1584151200 2024-09-13T16:48:28.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.844+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7f15841f0910 con 0x7f1584151200 2024-09-13T16:48:28.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.844+0000 7f15817fa640 1 -- 172.21.15.39:0/1841726258 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7f15841f0910 con 0x7f1584151200 2024-09-13T16:48:28.849 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.844+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1570066690 msgr2=0x7f1570068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.844+0000 7f158acdf640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1570066690 0x7f1570068b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f15740012f0 tx=0x7f1574009040 comp rx=0 tx=0).stop 2024-09-13T16:48:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.844+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 msgr2=0x7f15841ed4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.848+0000 7f158acdf640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 0x7f15841ed4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f157c00a960 tx=0x7f157c01a5e0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.848+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 shutdown_connections 2024-09-13T16:48:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.848+0000 7f158acdf640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1570066690 0x7f1570068b50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.848+0000 7f158acdf640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15840c1810 0x7f15841ecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.848+0000 7f158acdf640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1584151200 0x7f15841ed4c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.848+0000 7f158acdf640 1 --2- 172.21.15.39:0/1841726258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15840c0ba0 0x7f15840bebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.848+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 >> 172.21.15.39:0/1841726258 conn(0x7f15840c5790 msgr2=0x7f1584153570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.848+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 shutdown_connections 2024-09-13T16:48:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.848+0000 7f158acdf640 1 -- 172.21.15.39:0/1841726258 wait complete. 2024-09-13T16:48:28.867 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:28.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.972+0000 7efdd80d7640 1 Processor -- start 2024-09-13T16:48:28.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd80d7640 1 -- start start 2024-09-13T16:48:28.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd80d7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efdd00c4740 0x7efdd00c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd80d7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd00c2780 con 0x7efdd014e0c0 2024-09-13T16:48:28.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd80d7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd00c2a60 con 0x7efdd00a4e70 2024-09-13T16:48:28.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd80d7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd00c2d40 con 0x7efdd00c4740 2024-09-13T16:48:28.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd6674640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efdd00c4740 0x7efdd00c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd6674640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efdd00c4740 0x7efdd00c4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58622/0 (socket says 172.21.15.39:58622) 2024-09-13T16:48:28.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd6674640 1 -- 172.21.15.39:0/1655511456 learned_addr learned my addr 172.21.15.39:0/1655511456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:28.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd6674640 1 -- 172.21.15.39:0/1655511456 >> v1:172.21.15.84:6789/0 conn(0x7efdd00a4e70 legacy=0x7efdd006cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:28.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd6674640 1 -- 172.21.15.39:0/1655511456 >> v1:172.21.15.39:6789/0 conn(0x7efdd014e0c0 legacy=0x7efdd00c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.976+0000 7efdd6674640 1 -- 172.21.15.39:0/1655511456 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00c3020 con 0x7efdd00c4740 2024-09-13T16:48:28.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd6674640 1 --2- 172.21.15.39:0/1655511456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 0x7efdd00c4b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009870 tx=0x7efdcc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d9846601a23589 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd4e71640 1 -- 172.21.15.39:0/1655511456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0ab600 con 0x7efdd00c4740 2024-09-13T16:48:28.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd4e71640 1 -- 172.21.15.39:0/1655511456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc0a7050 con 0x7efdd00c4740 2024-09-13T16:48:28.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd4e71640 1 -- 172.21.15.39:0/1655511456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0ab980 con 0x7efdd00c4740 2024-09-13T16:48:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- 172.21.15.39:0/1655511456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 msgr2=0x7efdd00c4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 --2- 172.21.15.39:0/1655511456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 0x7efdd00c4b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009870 tx=0x7efdcc01c490 comp rx=0 tx=0).stop 2024-09-13T16:48:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- 172.21.15.39:0/1655511456 shutdown_connections 2024-09-13T16:48:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 --2- 172.21.15.39:0/1655511456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 0x7efdd00c4b50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- 172.21.15.39:0/1655511456 >> 172.21.15.39:0/1655511456 conn(0x7efdd0069e10 msgr2=0x7efdd006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- 172.21.15.39:0/1655511456 shutdown_connections 2024-09-13T16:48:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- 172.21.15.39:0/1655511456 wait complete. 2024-09-13T16:48:28.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 Processor -- start 2024-09-13T16:48:28.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- start start 2024-09-13T16:48:28.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd00a4e70 0x7efdd00a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 0x7efdd0070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd014e0c0 0x7efdd00713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd01ee910 con 0x7efdd00c4740 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd01eee40 con 0x7efdd014e0c0 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd006f630 con 0x7efdd00a4e70 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5672640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 0x7efdd0070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5672640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 0x7efdd0070e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58624/0 (socket says 172.21.15.39:58624) 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5672640 1 -- 172.21.15.39:0/1138110644 learned_addr learned my addr 172.21.15.39:0/1138110644 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd6674640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd014e0c0 0x7efdd00713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5e73640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd00a4e70 0x7efdd00a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5672640 1 -- 172.21.15.39:0/1138110644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd00a4e70 msgr2=0x7efdd00a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5672640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd00a4e70 0x7efdd00a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5672640 1 -- 172.21.15.39:0/1138110644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd014e0c0 msgr2=0x7efdd00713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5672640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd014e0c0 0x7efdd00713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:28.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5672640 1 -- 172.21.15.39:0/1138110644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc0094d0 con 0x7efdd00c4740 2024-09-13T16:48:28.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd5672640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 0x7efdd0070e70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efdc800c480 tx=0x7efdc800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdc27fc640 1 -- 172.21.15.39:0/1138110644 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8012830 con 0x7efdd00c4740 2024-09-13T16:48:28.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdc27fc640 1 -- 172.21.15.39:0/1138110644 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc80129b0 con 0x7efdd00c4740 2024-09-13T16:48:28.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdc27fc640 1 -- 172.21.15.39:0/1138110644 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8012d60 con 0x7efdd00c4740 2024-09-13T16:48:28.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd0070a50 con 0x7efdd00c4740 2024-09-13T16:48:28.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.980+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd01eeb60 con 0x7efdd00c4740 2024-09-13T16:48:28.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.984+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd006f7c0 con 0x7efdd00c4740 2024-09-13T16:48:28.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.984+0000 7efdc27fc640 1 -- 172.21.15.39:0/1138110644 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efdc801c460 con 0x7efdd00c4740 2024-09-13T16:48:28.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.984+0000 7efdc27fc640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdb4066690 0x7efdb4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:28.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.984+0000 7efdc27fc640 1 -- 172.21.15.39:0/1138110644 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7efdc8084320 con 0x7efdd00c4740 2024-09-13T16:48:28.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.984+0000 7efdd5e73640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdb4066690 0x7efdb4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:28.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.984+0000 7efdd5e73640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdb4066690 0x7efdb4068b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efdc40012f0 tx=0x7efdc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:28.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:28.988+0000 7efdc27fc640 1 -- 172.21.15.39:0/1138110644 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efdc80513f0 con 0x7efdd00c4740 2024-09-13T16:48:29.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.172+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdd006ccc0 con 0x7efdd00c4740 2024-09-13T16:48:29.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdc27fc640 1 -- 172.21.15.39:0/1138110644 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7efdc8054a10 con 0x7efdd00c4740 2024-09-13T16:48:29.179 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:29.180 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:48:29.180 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdb4066690 msgr2=0x7efdb4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdb4066690 0x7efdb4068b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efdc40012f0 tx=0x7efdc4009040 comp rx=0 tx=0).stop 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 msgr2=0x7efdd0070e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 0x7efdd0070e70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efdc800c480 tx=0x7efdc800c940 comp rx=0 tx=0).stop 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 shutdown_connections 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdb4066690 0x7efdb4068b50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd014e0c0 0x7efdd00713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00c4740 0x7efdd0070e70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 --2- 172.21.15.39:0/1138110644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd00a4e70 0x7efdd00a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 >> 172.21.15.39:0/1138110644 conn(0x7efdd0069e10 msgr2=0x7efdd006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 shutdown_connections 2024-09-13T16:48:29.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.176+0000 7efdd80d7640 1 -- 172.21.15.39:0/1138110644 wait complete. 2024-09-13T16:48:29.197 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:29.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfb6d87640 1 Processor -- start 2024-09-13T16:48:29.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfb6d87640 1 -- start start 2024-09-13T16:48:29.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfb6d87640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfb6d87640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdfb00c7800 0x7fdfb01545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfb6d87640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb00c5a30 con 0x7fdfb0151500 2024-09-13T16:48:29.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfb6d87640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb01576a0 con 0x7fdfb00c7800 2024-09-13T16:48:29.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfb6d87640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0157980 con 0x7fdfb0071dd0 2024-09-13T16:48:29.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfaffff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfb5324640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdfb00c7800 0x7fdfb01545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfaffff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:45368/0 (socket says 172.21.15.39:45368) 2024-09-13T16:48:29.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.300+0000 7fdfaffff640 1 -- 172.21.15.39:0/3289587748 learned_addr learned my addr 172.21.15.39:0/3289587748 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:29.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb5324640 1 -- 172.21.15.39:0/3289587748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 msgr2=0x7fdfb0154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:29.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb5324640 1 --2- 172.21.15.39:0/3289587748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb5324640 1 -- 172.21.15.39:0/3289587748 >> v1:172.21.15.84:6789/0 conn(0x7fdfb0151500 legacy=0x7fdfb00c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb5324640 1 -- 172.21.15.39:0/3289587748 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0157c60 con 0x7fdfb00c7800 2024-09-13T16:48:29.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfaffff640 1 --2- 172.21.15.39:0/3289587748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:29.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb5324640 1 --2- 172.21.15.39:0/3289587748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfb00c7800 0x7fdfb01545b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00097c0 tx=0x7fdfa001ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c4099be3995d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:29.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfaf7fe640 1 -- 172.21.15.39:0/3289587748 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00ab600 con 0x7fdfb00c7800 2024-09-13T16:48:29.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfaf7fe640 1 -- 172.21.15.39:0/3289587748 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa00ab760 con 0x7fdfb00c7800 2024-09-13T16:48:29.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfaf7fe640 1 -- 172.21.15.39:0/3289587748 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00abaf0 con 0x7fdfb00c7800 2024-09-13T16:48:29.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb6d87640 1 -- 172.21.15.39:0/3289587748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfb00c7800 msgr2=0x7fdfb01545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb6d87640 1 --2- 172.21.15.39:0/3289587748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfb00c7800 0x7fdfb01545b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00097c0 tx=0x7fdfa001ce40 comp rx=0 tx=0).stop 2024-09-13T16:48:29.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb6d87640 1 -- 172.21.15.39:0/3289587748 shutdown_connections 2024-09-13T16:48:29.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb6d87640 1 --2- 172.21.15.39:0/3289587748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfb00c7800 0x7fdfb01545b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb6d87640 1 --2- 172.21.15.39:0/3289587748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb6d87640 1 -- 172.21.15.39:0/3289587748 >> 172.21.15.39:0/3289587748 conn(0x7fdfb00ab530 msgr2=0x7fdfb00ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:29.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb6d87640 1 -- 172.21.15.39:0/3289587748 shutdown_connections 2024-09-13T16:48:29.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.304+0000 7fdfb6d87640 1 -- 172.21.15.39:0/3289587748 wait complete. 2024-09-13T16:48:29.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 Processor -- start 2024-09-13T16:48:29.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 -- start start 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00c7800 0x7fdfb01f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfb0151500 0x7fdfb014e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0153730 con 0x7fdfb00c7800 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb01effb0 con 0x7fdfb0151500 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb014c080 con 0x7fdfb0071dd0 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00c7800 0x7fdfb01f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00c7800 0x7fdfb01f0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58632/0 (socket says 172.21.15.39:58632) 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfaffff640 1 -- 172.21.15.39:0/1466176937 learned_addr learned my addr 172.21.15.39:0/1466176937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb5324640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfb0151500 0x7fdfb014e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb4b23640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfaffff640 1 -- 172.21.15.39:0/1466176937 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 msgr2=0x7fdfb0155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfaffff640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfaffff640 1 -- 172.21.15.39:0/1466176937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfb0151500 msgr2=0x7fdfb014e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfaffff640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfb0151500 0x7fdfb014e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfaffff640 1 -- 172.21.15.39:0/1466176937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf980094d0 con 0x7fdfb00c7800 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb4b23640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfaffff640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00c7800 0x7fdfb01f0440 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800e9d0 tx=0x7fdf9800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfad7fa640 1 -- 172.21.15.39:0/1466176937 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800ca60 con 0x7fdfb00c7800 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfad7fa640 1 -- 172.21.15.39:0/1466176937 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9800cbc0 con 0x7fdfb00c7800 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa0009420 con 0x7fdfb00c7800 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfad7fa640 1 -- 172.21.15.39:0/1466176937 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98013470 con 0x7fdfb00c7800 2024-09-13T16:48:29.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb0153980 con 0x7fdfb00c7800 2024-09-13T16:48:29.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb0153ba0 con 0x7fdfb00c7800 2024-09-13T16:48:29.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.308+0000 7fdfad7fa640 1 -- 172.21.15.39:0/1466176937 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf980135d0 con 0x7fdfb00c7800 2024-09-13T16:48:29.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.312+0000 7fdfad7fa640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf900666c0 0x7fdf90068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.312+0000 7fdfad7fa640 1 -- 172.21.15.39:0/1466176937 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fdf98084720 con 0x7fdfb00c7800 2024-09-13T16:48:29.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.312+0000 7fdfb4b23640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf900666c0 0x7fdf90068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.312+0000 7fdfb4b23640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf900666c0 0x7fdf90068b80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8007fd0 tx=0x7fdfa8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:29.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.316+0000 7fdfad7fa640 1 -- 172.21.15.39:0/1466176937 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf980517f0 con 0x7fdfb00c7800 2024-09-13T16:48:29.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.476+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfb00c4280 con 0x7fdfb00c7800 2024-09-13T16:48:29.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfad7fa640 1 -- 172.21.15.39:0/1466176937 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7fdf98054e10 con 0x7fdfb00c7800 2024-09-13T16:48:29.483 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:29.484 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:29.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf900666c0 msgr2=0x7fdf90068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf900666c0 0x7fdf90068b80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8007fd0 tx=0x7fdfa8009040 comp rx=0 tx=0).stop 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00c7800 msgr2=0x7fdfb01f0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00c7800 0x7fdfb01f0440 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800e9d0 tx=0x7fdf9800ee90 comp rx=0 tx=0).stop 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 shutdown_connections 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf900666c0 0x7fdf90068b80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfb0151500 0x7fdfb014e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00c7800 0x7fdfb01f0440 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 --2- 172.21.15.39:0/1466176937 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfb0071dd0 0x7fdfb0155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 >> 172.21.15.39:0/1466176937 conn(0x7fdfb00ab530 msgr2=0x7fdfb00ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 shutdown_connections 2024-09-13T16:48:29.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.480+0000 7fdfb6d87640 1 -- 172.21.15.39:0/1466176937 wait complete. 2024-09-13T16:48:29.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:29.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa387242640 1 Processor -- start 2024-09-13T16:48:29.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa387242640 1 -- start start 2024-09-13T16:48:29.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa387242640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa380070e30 0x7fa3800611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa387242640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3801548a0 0x7fa3800c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa387242640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa380154d30 con 0x7fa380151650 2024-09-13T16:48:29.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa387242640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa380157670 con 0x7fa3801548a0 2024-09-13T16:48:29.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa387242640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa380061880 con 0x7fa380070e30 2024-09-13T16:48:29.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa384fde640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa380151650 0x7fa380157a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57558/0 (socket says 172.21.15.39:57558) 2024-09-13T16:48:29.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa384fde640 1 -- 172.21.15.39:0/1336248964 learned_addr learned my addr 172.21.15.39:0/1336248964 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:29.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa3857df640 1 --2- 172.21.15.39:0/1336248964 >> v2:172.21.15.84:3300/0 conn(0x7fa3801548a0 0x7fa3800c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa37ffff640 1 --2- 172.21.15.39:0/1336248964 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa380070e30 0x7fa3800611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa37f7fe640 1 -- 172.21.15.39:0/1336248964 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484965471 0 0) 0x7fa380154d30 con 0x7fa380151650 2024-09-13T16:48:29.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa37f7fe640 1 -- 172.21.15.39:0/1336248964 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa368003530 con 0x7fa380151650 2024-09-13T16:48:29.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa37ffff640 1 -- 172.21.15.39:0/1336248964 >> v2:172.21.15.84:3300/0 conn(0x7fa3801548a0 msgr2=0x7fa3800c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:29.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa37ffff640 1 --2- 172.21.15.39:0/1336248964 >> v2:172.21.15.84:3300/0 conn(0x7fa3801548a0 0x7fa3800c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa37ffff640 1 -- 172.21.15.39:0/1336248964 >> v1:172.21.15.39:6789/0 conn(0x7fa380151650 legacy=0x7fa380157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa37f7fe640 1 -- 172.21.15.39:0/1336248964 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3318375445 0 0) 0x7fa368003530 con 0x7fa380151650 2024-09-13T16:48:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.620+0000 7fa37ffff640 1 -- 172.21.15.39:0/1336248964 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800abc00 con 0x7fa380070e30 2024-09-13T16:48:29.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa37ffff640 1 --2- 172.21.15.39:0/1336248964 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa380070e30 0x7fa3800611c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa3740097c0 tx=0x7fa37401c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56538704b23df598 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:29.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa37f7fe640 1 -- 172.21.15.39:0/1336248964 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378001680 con 0x7fa380070e30 2024-09-13T16:48:29.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa37f7fe640 1 -- 172.21.15.39:0/1336248964 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3740a7050 con 0x7fa380070e30 2024-09-13T16:48:29.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa37f7fe640 1 -- 172.21.15.39:0/1336248964 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740ab620 con 0x7fa380070e30 2024-09-13T16:48:29.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 -- 172.21.15.39:0/1336248964 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa380070e30 msgr2=0x7fa3800611c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 --2- 172.21.15.39:0/1336248964 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa380070e30 0x7fa3800611c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa3740097c0 tx=0x7fa37401c330 comp rx=0 tx=0).stop 2024-09-13T16:48:29.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 -- 172.21.15.39:0/1336248964 shutdown_connections 2024-09-13T16:48:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 --2- 172.21.15.39:0/1336248964 >> v2:172.21.15.84:3300/0 conn(0x7fa3801548a0 0x7fa3800c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 --2- 172.21.15.39:0/1336248964 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa380070e30 0x7fa3800611c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 -- 172.21.15.39:0/1336248964 >> 172.21.15.39:0/1336248964 conn(0x7fa3800c2a50 msgr2=0x7fa3800c2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 -- 172.21.15.39:0/1336248964 shutdown_connections 2024-09-13T16:48:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 -- 172.21.15.39:0/1336248964 wait complete. 2024-09-13T16:48:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 Processor -- start 2024-09-13T16:48:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.624+0000 7fa387242640 1 -- start start 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa387242640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380070e30 0x7fa3800c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa387242640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa380151650 0x7fa38014b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa387242640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3801548a0 0x7fa38014bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa387242640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800a89a0 con 0x7fa380151650 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa387242640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800a8c80 con 0x7fa380070e30 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa387242640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801ed640 con 0x7fa3801548a0 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa380151650 0x7fa38014b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa380151650 0x7fa38014b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58644/0 (socket says 172.21.15.39:58644) 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37ffff640 1 -- 172.21.15.39:0/3406793487 learned_addr learned my addr 172.21.15.39:0/3406793487 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa384fde640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380070e30 0x7fa3800c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa3857df640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3801548a0 0x7fa38014bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37ffff640 1 -- 172.21.15.39:0/3406793487 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3801548a0 msgr2=0x7fa38014bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37ffff640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3801548a0 0x7fa38014bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37ffff640 1 -- 172.21.15.39:0/3406793487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380070e30 msgr2=0x7fa3800c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37ffff640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380070e30 0x7fa3800c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37ffff640 1 -- 172.21.15.39:0/3406793487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374009420 con 0x7fa380151650 2024-09-13T16:48:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37ffff640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa380151650 0x7fa38014b630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3740097c0 tx=0x7fa37401ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37d7fa640 1 -- 172.21.15.39:0/3406793487 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740b7450 con 0x7fa380151650 2024-09-13T16:48:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37d7fa640 1 -- 172.21.15.39:0/3406793487 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3740b75d0 con 0x7fa380151650 2024-09-13T16:48:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3801ed0e0 con 0x7fa380151650 2024-09-13T16:48:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37d7fa640 1 -- 172.21.15.39:0/3406793487 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3740b7980 con 0x7fa380151650 2024-09-13T16:48:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3801ed380 con 0x7fa380151650 2024-09-13T16:48:29.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3800a8460 con 0x7fa380151650 2024-09-13T16:48:29.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37d7fa640 1 -- 172.21.15.39:0/3406793487 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3740b7ae0 con 0x7fa380151650 2024-09-13T16:48:29.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.628+0000 7fa37d7fa640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa36c066690 0x7fa36c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:29.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.632+0000 7fa37d7fa640 1 -- 172.21.15.39:0/3406793487 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fa37411eb70 con 0x7fa380151650 2024-09-13T16:48:29.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.632+0000 7fa384fde640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa36c066690 0x7fa36c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:29.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.632+0000 7fa384fde640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa36c066690 0x7fa36c068b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa378001b00 tx=0x7fa378009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.636+0000 7fa37d7fa640 1 -- 172.21.15.39:0/3406793487 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa3800a8460 con 0x7fa380151650 2024-09-13T16:48:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.832+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3800a8ae0 con 0x7fa380151650 2024-09-13T16:48:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.832+0000 7fa37d7fa640 1 -- 172.21.15.39:0/3406793487 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7fa3800a8ae0 con 0x7fa380151650 2024-09-13T16:48:29.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:29.837 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:48:29.837 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:48:29.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa36c066690 msgr2=0x7fa36c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa36c066690 0x7fa36c068b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa378001b00 tx=0x7fa378009040 comp rx=0 tx=0).stop 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa380151650 msgr2=0x7fa38014b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa380151650 0x7fa38014b630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3740097c0 tx=0x7fa37401ab70 comp rx=0 tx=0).stop 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 shutdown_connections 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa36c066690 0x7fa36c068b50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3801548a0 0x7fa38014bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa380151650 0x7fa38014b630 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 --2- 172.21.15.39:0/3406793487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa380070e30 0x7fa3800c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 >> 172.21.15.39:0/3406793487 conn(0x7fa3800c2a50 msgr2=0x7fa3800bf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 shutdown_connections 2024-09-13T16:48:29.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:29.836+0000 7fa387242640 1 -- 172.21.15.39:0/3406793487 wait complete. 2024-09-13T16:48:30.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T16:48:08.135456+0000', 'modified': '2024-09-13T16:48:12.162981+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4260': {'gid': 4260, 'name': 'a', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/3163580917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 3163580917}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3163580917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'b', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T16:48:30.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T16:48:30.259 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T16:48:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004fd22640 1 Processor -- start 2024-09-13T16:48:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004fd22640 1 -- start start 2024-09-13T16:48:30.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004fd22640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048071dd0 0x7f0048154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004fd22640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00480c7800 0x7f00481545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004fd22640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00480c5a30 con 0x7f0048151500 2024-09-13T16:48:30.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004fd22640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00481576a0 con 0x7f00480c7800 2024-09-13T16:48:30.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004fd22640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0048157980 con 0x7f0048071dd0 2024-09-13T16:48:30.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004e2bf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00480c7800 0x7f00481545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004dabe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0048151500 0x7f00480c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53738/0 (socket says 172.21.15.39:53738) 2024-09-13T16:48:30.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004dabe640 1 -- 172.21.15.39:0/2321558525 learned_addr learned my addr 172.21.15.39:0/2321558525 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:30.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.388+0000 7f004d2bd640 1 --2- 172.21.15.39:0/2321558525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048071dd0 0x7f0048154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004e2bf640 1 -- 172.21.15.39:0/2321558525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048071dd0 msgr2=0x7f0048154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004e2bf640 1 --2- 172.21.15.39:0/2321558525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048071dd0 0x7f0048154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004e2bf640 1 -- 172.21.15.39:0/2321558525 >> v1:172.21.15.39:6789/0 conn(0x7f0048151500 legacy=0x7f00480c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004cabc640 1 -- 172.21.15.39:0/2321558525 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478056773 0 0) 0x7f00480c5a30 con 0x7f0048151500 2024-09-13T16:48:30.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004e2bf640 1 -- 172.21.15.39:0/2321558525 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0048157c60 con 0x7f00480c7800 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004e2bf640 1 --2- 172.21.15.39:0/2321558525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00480c7800 0x7f00481545b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0040009d90 tx=0x7f004001d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b42736ae24788bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004cabc640 1 -- 172.21.15.39:0/2321558525 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00400a5990 con 0x7f00480c7800 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004cabc640 1 -- 172.21.15.39:0/2321558525 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00400a5af0 con 0x7f00480c7800 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004cabc640 1 -- 172.21.15.39:0/2321558525 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00400a5e80 con 0x7f00480c7800 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004fd22640 1 -- 172.21.15.39:0/2321558525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00480c7800 msgr2=0x7f00481545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004fd22640 1 --2- 172.21.15.39:0/2321558525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00480c7800 0x7f00481545b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0040009d90 tx=0x7f004001d310 comp rx=0 tx=0).stop 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004fd22640 1 -- 172.21.15.39:0/2321558525 shutdown_connections 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004fd22640 1 --2- 172.21.15.39:0/2321558525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00480c7800 0x7f00481545b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004fd22640 1 --2- 172.21.15.39:0/2321558525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048071dd0 0x7f0048154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.392+0000 7f004fd22640 1 -- 172.21.15.39:0/2321558525 >> 172.21.15.39:0/2321558525 conn(0x7f00480ab530 msgr2=0x7f00480ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:30.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 -- 172.21.15.39:0/2321558525 shutdown_connections 2024-09-13T16:48:30.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 -- 172.21.15.39:0/2321558525 wait complete. 2024-09-13T16:48:30.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 Processor -- start 2024-09-13T16:48:30.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 -- start start 2024-09-13T16:48:30.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0048071dd0 0x7f004814ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00480c7800 0x7f00481476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048151500 0x7f0048155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004814bf70 con 0x7f0048071dd0 2024-09-13T16:48:30.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0048146f30 con 0x7f00480c7800 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004814ad90 con 0x7f0048151500 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004dabe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0048071dd0 0x7f004814ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004dabe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0048071dd0 0x7f004814ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36416/0 (socket says 172.21.15.39:36416) 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004dabe640 1 -- 172.21.15.39:0/4221077885 learned_addr learned my addr 172.21.15.39:0/4221077885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004e2bf640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048151500 0x7f0048155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004d2bd640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00480c7800 0x7f00481476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004dabe640 1 -- 172.21.15.39:0/4221077885 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048151500 msgr2=0x7f0048155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004dabe640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048151500 0x7f0048155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004dabe640 1 -- 172.21.15.39:0/4221077885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00480c7800 msgr2=0x7f00481476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004dabe640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00480c7800 0x7f00481476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004dabe640 1 -- 172.21.15.39:0/4221077885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0040009a70 con 0x7f0048071dd0 2024-09-13T16:48:30.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004e2bf640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048151500 0x7f0048155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:30.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004dabe640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0048071dd0 0x7f004814ba30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0038007fd0 tx=0x7f003800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f003e7fc640 1 -- 172.21.15.39:0/4221077885 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038005cf0 con 0x7f0048071dd0 2024-09-13T16:48:30.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f003e7fc640 1 -- 172.21.15.39:0/4221077885 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0038005e70 con 0x7f0048071dd0 2024-09-13T16:48:30.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 -- 172.21.15.39:0/4221077885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004814a480 con 0x7f0048071dd0 2024-09-13T16:48:30.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f003e7fc640 1 -- 172.21.15.39:0/4221077885 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380072f0 con 0x7f0048071dd0 2024-09-13T16:48:30.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.396+0000 7f004fd22640 1 -- 172.21.15.39:0/4221077885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0048147c20 con 0x7f0048071dd0 2024-09-13T16:48:30.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f003e7fc640 1 -- 172.21.15.39:0/4221077885 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0038007450 con 0x7f0048071dd0 2024-09-13T16:48:30.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f003e7fc640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f002c0666c0 0x7f002c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f003e7fc640 1 -- 172.21.15.39:0/4221077885 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f0038083a00 con 0x7f0048071dd0 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004d2bd640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f002c0666c0 0x7f002c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004d2bd640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f002c0666c0 0x7f002c068b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f00480c6bb0 tx=0x7f0044008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 Processor -- start 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 -- start start 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03edc10 0x559af03ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af03adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ae050 0x559af03ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af0277e80 con 0x559af03edc10 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af0205af0 con 0x559af03ae050 2024-09-13T16:48:30.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x559af02030c0 con 0x559af03ad730 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004dabe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03edc10 0x559af03ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004dabe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03edc10 0x559af03ad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36432/0 (socket says 172.21.15.39:36432) 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004dabe640 1 -- 172.21.15.39:0/1605394492 learned_addr learned my addr 172.21.15.39:0/1605394492 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004d2bd640 1 --2- 172.21.15.39:0/1605394492 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af03adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004e2bf640 1 --2- 172.21.15.39:0/1605394492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ae050 0x559af03ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004dabe640 1 -- 172.21.15.39:0/1605394492 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 msgr2=0x559af03adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004dabe640 1 --2- 172.21.15.39:0/1605394492 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af03adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004dabe640 1 -- 172.21.15.39:0/1605394492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ae050 msgr2=0x559af03ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004dabe640 1 --2- 172.21.15.39:0/1605394492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ae050 0x559af03ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004dabe640 1 -- 172.21.15.39:0/1605394492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559aefefa890 con 0x559af03edc10 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f004dabe640 1 --2- 172.21.15.39:0/1605394492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03edc10 0x559af03ad1f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0038081cb0 tx=0x7f0038054fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee23821701f2894 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f002b7fe640 1 -- 172.21.15.39:0/1605394492 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380528d0 con 0x559af03edc10 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f002b7fe640 1 -- 172.21.15.39:0/1605394492 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0038052b30 con 0x559af03edc10 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f002b7fe640 1 -- 172.21.15.39:0/1605394492 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003805c6f0 con 0x559af03edc10 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 -- 172.21.15.39:0/1605394492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03edc10 msgr2=0x559af03ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1605394492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03edc10 0x559af03ad1f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0038081cb0 tx=0x7f0038054fd0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 -- 172.21.15.39:0/1605394492 shutdown_connections 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1605394492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ae050 0x559af03ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1605394492 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af03adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1605394492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03edc10 0x559af03ad1f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 -- 172.21.15.39:0/1605394492 >> 172.21.15.39:0/1605394492 conn(0x559af03eb110 msgr2=0x559af03ed500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 -- 172.21.15.39:0/1605394492 shutdown_connections 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.400+0000 7f0052e6b000 1 -- 172.21.15.39:0/1605394492 wait complete. 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- start start 2024-09-13T16:48:30.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 Processor -- start 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af03b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af03b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03edc10 0x559af03b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af01fe9b0 con 0x559af03ae050 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af019dc00 con 0x559af03edc10 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x559af01bc070 con 0x559af03ad730 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 10 client.0 _ll_register_callbacks cb 0x559af039bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004d2bd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af03b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004d2bd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af03b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36434/0 (socket says 172.21.15.39:36434) 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004d2bd640 1 -- 172.21.15.39:0/1286585791 learned_addr learned my addr 172.21.15.39:0/1286585791 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:30.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004e2bf640 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03edc10 0x559af03b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004dabe640 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af03b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004d2bd640 1 -- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 msgr2=0x559af03b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004d2bd640 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af03b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004d2bd640 1 -- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03edc10 msgr2=0x559af03b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004d2bd640 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03edc10 0x559af03b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004d2bd640 1 -- 172.21.15.39:0/1286585791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0048147c20 con 0x559af03ae050 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004e2bf640 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03edc10 0x559af03b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004dabe640 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af03b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004d2bd640 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af03b3cd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f00440157c0 tx=0x7f0044004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5997975478b80d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 1 -- 172.21.15.39:0/1286585791 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044016ba0 con 0x559af03ae050 2024-09-13T16:48:30.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 1 -- 172.21.15.39:0/1286585791 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f0044016d00 con 0x559af03ae050 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- 172.21.15.39:0/1286585791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559af01bb5f0 con 0x559af03ae050 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 1 -- 172.21.15.39:0/1286585791 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0044022420 con 0x559af03ae050 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 10 client.4824 fetch_fsmap learned FSMap version 14 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- 172.21.15.39:0/1286585791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x559af01c3ab0 con 0x559af03ae050 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 1 -- 172.21.15.39:0/1286585791 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559af01bb5f0 con 0x559af03ae050 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 1 -- 172.21.15.39:0/1286585791 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 14) v1 ==== 3668+0+0 (secure 0 0 0) 0x7f004402a350 con 0x559af03ae050 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 10 client.4824 fetch_fsmap finished waiting for FSMap version 14 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 10 client.4824 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 20 client.4824 populate_metadata read hostname 'smithi039' 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] conn(0x559af03b4c20 0x559af04024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 4 client.4824 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- 172.21.15.39:0/1286585791 --> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559af01fb790 con 0x559af03b4c20 2024-09-13T16:48:30.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004dabe640 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] conn(0x559af03b4c20 0x559af04024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f004dabe640 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] conn(0x559af03b4c20 0x559af04024e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd15835cca73b117 server_cookie=18c5bade57c83ae1 in_seq=0 out_seq=0 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 10 client.4824 ms_handle_connect on v2:172.21.15.39:6834/3163580917 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 1 -- 172.21.15.39:0/1286585791 <== mds.0 v2:172.21.15.39:6834/3163580917 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559af01fb790 con 0x559af03b4c20 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 10 client.4824 handle_command_reply: tid=0 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 client.4824 shutdown 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- 172.21.15.39:0/1286585791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 msgr2=0x559af03b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af03b3cd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f00440157c0 tx=0x7f0044004740 comp rx=0 tx=0).stop 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- 172.21.15.39:0/1286585791 shutdown_connections 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] conn(0x559af03b4c20 0x559af04024e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03edc10 0x559af03b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af03b3cd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0028ff9640 0 client.4824 ms_handle_reset on v2:172.21.15.39:6834/3163580917 2024-09-13T16:48:30.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1286585791 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af03b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.404+0000 7f0052e6b000 1 -- 172.21.15.39:0/1286585791 >> 172.21.15.39:0/1286585791 conn(0x559af03eb110 msgr2=0x559af03ed500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:30.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.408+0000 7f0052e6b000 1 -- 172.21.15.39:0/1286585791 shutdown_connections 2024-09-13T16:48:30.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.408+0000 7f0052e6b000 1 -- 172.21.15.39:0/1286585791 wait complete. 2024-09-13T16:48:30.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.408+0000 7f0052e6b000 20 client.4824 trim_cache size 0 max 16384 2024-09-13T16:48:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.420+0000 7f0052e6b000 1 Processor -- start 2024-09-13T16:48:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.420+0000 7f0052e6b000 1 -- start start 2024-09-13T16:48:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03eb110 0x559af03ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af04034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af0403a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aefebb940 con 0x559af03ae050 2024-09-13T16:48:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af0206d80 con 0x559af03eb110 2024-09-13T16:48:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x559af01c5c60 con 0x559af03ad730 2024-09-13T16:48:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af0403a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af0403a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36448/0 (socket says 172.21.15.39:36448) 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 -- 172.21.15.39:0/1730929482 learned_addr learned my addr 172.21.15.39:0/1730929482 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004dabe640 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03eb110 0x559af03ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004d2bd640 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af04034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 -- 172.21.15.39:0/1730929482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 msgr2=0x559af04034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af04034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 -- 172.21.15.39:0/1730929482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03eb110 msgr2=0x559af03ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03eb110 0x559af03ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 -- 172.21.15.39:0/1730929482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559af01a9c00 con 0x559af03ae050 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004dabe640 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03eb110 0x559af03ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af0403a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f00400b0040 tx=0x7f0040004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecc210d3bed92c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f002bfff640 1 -- 172.21.15.39:0/1730929482 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00400b7640 con 0x559af03ae050 2024-09-13T16:48:30.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f002bfff640 1 -- 172.21.15.39:0/1730929482 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00400b77c0 con 0x559af03ae050 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f002bfff640 1 -- 172.21.15.39:0/1730929482 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00400b7b70 con 0x559af03ae050 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- 172.21.15.39:0/1730929482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 msgr2=0x559af0403a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af0403a30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f00400b0040 tx=0x7f0040004a50 comp rx=0 tx=0).stop 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- 172.21.15.39:0/1730929482 shutdown_connections 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03ae050 0x559af0403a30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ad730 0x559af04034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1730929482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03eb110 0x559af03ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- 172.21.15.39:0/1730929482 >> 172.21.15.39:0/1730929482 conn(0x559af03e9910 msgr2=0x559af03ab9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- 172.21.15.39:0/1730929482 shutdown_connections 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- 172.21.15.39:0/1730929482 wait complete. 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- start start 2024-09-13T16:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 Processor -- start 2024-09-13T16:48:30.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ad730 0x559af0451ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ae050 0x559af04523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03eb110 0x559af0415f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af03ad5f0 con 0x559af03eb110 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x559af01fc1c0 con 0x559af03ad730 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x559af01b7590 con 0x559af03ae050 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0052e6b000 10 client.0 _ll_register_callbacks cb 0x559af0277dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03eb110 0x559af0415f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03eb110 0x559af0415f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36464/0 (socket says 172.21.15.39:36464) 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 -- 172.21.15.39:0/1405001300 learned_addr learned my addr 172.21.15.39:0/1405001300 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004dabe640 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ad730 0x559af0451ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004d2bd640 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ae050 0x559af04523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 -- 172.21.15.39:0/1405001300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ae050 msgr2=0x559af04523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ae050 0x559af04523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 -- 172.21.15.39:0/1405001300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ad730 msgr2=0x559af0451ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ad730 0x559af0451ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 -- 172.21.15.39:0/1405001300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0048157c60 con 0x559af03eb110 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004d2bd640 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ae050 0x559af04523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f004e2bf640 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03eb110 0x559af0415f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f004001a4e0 tx=0x7f004001b0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7492ebd2e15e899 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.424+0000 7f0028ff9640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0028ff9640 1 -- 172.21.15.39:0/1405001300 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040003da0 con 0x559af03eb110 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0028ff9640 1 -- 172.21.15.39:0/1405001300 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f00400b7930 con 0x559af03eb110 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 -- 172.21.15.39:0/1405001300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559af01ce460 con 0x559af03eb110 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0028ff9640 1 -- 172.21.15.39:0/1405001300 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00400b7af0 con 0x559af03eb110 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 10 client.4830 fetch_fsmap learned FSMap version 14 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 -- 172.21.15.39:0/1405001300 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x559aefefacc0 con 0x559af03eb110 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0028ff9640 1 -- 172.21.15.39:0/1405001300 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559af01ce460 con 0x559af03eb110 2024-09-13T16:48:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0028ff9640 1 -- 172.21.15.39:0/1405001300 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 14) v1 ==== 3668+0+0 (secure 0 0 0) 0x7f00400b7c50 con 0x559af03eb110 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 10 client.4830 fetch_fsmap finished waiting for FSMap version 14 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 10 client.4830 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 20 client.4830 populate_metadata read hostname 'smithi039' 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] conn(0x559af03edc10 0x559af0416440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 4 client.4830 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 -- 172.21.15.39:0/1405001300 --> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559af01c79c0 con 0x559af03edc10 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004dabe640 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] conn(0x559af03edc10 0x559af0416440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004dabe640 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] conn(0x559af03edc10 0x559af0416440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec4000f70b0b6a9 server_cookie=214a1352074d3126 in_seq=0 out_seq=0 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0028ff9640 10 client.4830 ms_handle_connect on v2:172.21.15.39:6834/3163580917 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0028ff9640 1 -- 172.21.15.39:0/1405001300 <== mds.0 v2:172.21.15.39:6834/3163580917 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x559af01c79c0 con 0x559af03edc10 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0028ff9640 10 client.4830 handle_command_reply: tid=0 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 client.4830 shutdown 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 -- 172.21.15.39:0/1405001300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03eb110 msgr2=0x559af0415f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03eb110 0x559af0415f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f004001a4e0 tx=0x7f004001b0f0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 -- 172.21.15.39:0/1405001300 shutdown_connections 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.39:6834/3163580917,v1:172.21.15.39:6835/3163580917] conn(0x559af03edc10 0x559af0416440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559af03eb110 0x559af0415f00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x559af03ae050 0x559af04523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 --2- 172.21.15.39:0/1405001300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559af03ad730 0x559af0451ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 -- 172.21.15.39:0/1405001300 >> 172.21.15.39:0/1405001300 conn(0x559af03e9910 msgr2=0x559af03ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0028ff9640 0 client.4830 ms_handle_reset on v2:172.21.15.39:6834/3163580917 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 -- 172.21.15.39:0/1405001300 shutdown_connections 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 1 -- 172.21.15.39:0/1405001300 wait complete. 2024-09-13T16:48:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f0052e6b000 20 client.4830 trim_cache size 0 max 16384 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "b9195ac8-73ed-4db1-a594-cde2ff5ccb0f", 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 4260, 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "b", 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 22.293639847000001, 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 14, 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 24, 2024-09-13T16:48:30.440 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 24, 2024-09-13T16:48:30.441 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 37.326738044000003 2024-09-13T16:48:30.441 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T16:48:30.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 -- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f002c0666c0 msgr2=0x7f002c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f002c0666c0 0x7f002c068b80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f00480c6bb0 tx=0x7f0044008040 comp rx=0 tx=0).stop 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 -- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0048071dd0 msgr2=0x7f004814ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0048071dd0 0x7f004814ba30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0038007fd0 tx=0x7f003800c8e0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 -- 172.21.15.39:0/4221077885 shutdown_connections 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f002c0666c0 0x7f002c068b80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00480c7800 0x7f00481476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0048151500 0x7f0048155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 --2- 172.21.15.39:0/4221077885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0048071dd0 0x7f004814ba30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 -- 172.21.15.39:0/4221077885 >> 172.21.15.39:0/4221077885 conn(0x7f00480ab530 msgr2=0x7f0048070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 -- 172.21.15.39:0/4221077885 shutdown_connections 2024-09-13T16:48:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.428+0000 7f004fd22640 1 -- 172.21.15.39:0/4221077885 wait complete. 2024-09-13T16:48:30.449 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T16:48:30.459 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T16:48:30.459 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T16:48:30.460 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T16:48:30.460 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T16:48:30.495 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T16:48:30.495 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-09-13T16:48:30.496 INFO:tasks.ceph_fuse:client.0 config is {'cephfs_name': 'a'} 2024-09-13T16:48:30.496 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T16:48:30.496 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:30.496 INFO:tasks.ceph_fuse:client.1 config is {'cephfs_name': 'b'} 2024-09-13T16:48:30.496 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T16:48:30.496 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:30.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:30.496 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-13T16:48:30.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:30.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-09-13T16:48:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2024-09-13T16:48:30.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:48:30.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:30.561 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-09-13T16:48:30.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:30.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2024-09-13T16:48:30.628 INFO:teuthology.orchestra.run.smithi084.stderr:Cannot find device "ceph-brx" 2024-09-13T16:48:30.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T16:48:30.629 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T16:48:30.630 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T16:48:30.630 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T16:48:30.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5e1711640 1 Processor -- start 2024-09-13T16:48:30.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5e1711640 1 -- start start 2024-09-13T16:48:30.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5e1711640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe5dc06c870 0x7fe5dc0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5e1711640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc152a80 con 0x7fe5dc0c40b0 2024-09-13T16:48:30.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5e1711640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc0c30f0 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5e1711640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc151e00 con 0x7fe5dc06c870 2024-09-13T16:48:30.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5da7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe5dc06c870 0x7fe5dc0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5db7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe5dc0c40b0 0x7fe5dc15c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53742/0 (socket says 172.21.15.39:53742) 2024-09-13T16:48:30.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5db7fe640 1 -- 172.21.15.39:0/1817497569 learned_addr learned my addr 172.21.15.39:0/1817497569 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:30.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5da7fc640 1 -- 172.21.15.39:0/1817497569 >> v1:172.21.15.84:6789/0 conn(0x7fe5dc0aabc0 legacy=0x7fe5dc0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:30.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5da7fc640 1 -- 172.21.15.39:0/1817497569 >> v1:172.21.15.39:6789/0 conn(0x7fe5dc0c40b0 legacy=0x7fe5dc15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.760+0000 7fe5da7fc640 1 -- 172.21.15.39:0/1817497569 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc1520e0 con 0x7fe5dc06c870 2024-09-13T16:48:30.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 --2- 172.21.15.39:0/1817497569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc06c870 0x7fe5dc0aafa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00097c0 tx=0x7fe5d001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146f8b047c61d36a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5d9ffb640 1 -- 172.21.15.39:0/1817497569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00ac600 con 0x7fe5dc06c870 2024-09-13T16:48:30.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5d9ffb640 1 -- 172.21.15.39:0/1817497569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d00a8050 con 0x7fe5dc06c870 2024-09-13T16:48:30.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5d9ffb640 1 -- 172.21.15.39:0/1817497569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00ac980 con 0x7fe5dc06c870 2024-09-13T16:48:30.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 -- 172.21.15.39:0/1817497569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc06c870 msgr2=0x7fe5dc0aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 --2- 172.21.15.39:0/1817497569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc06c870 0x7fe5dc0aafa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00097c0 tx=0x7fe5d001c3a0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 -- 172.21.15.39:0/1817497569 shutdown_connections 2024-09-13T16:48:30.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 --2- 172.21.15.39:0/1817497569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc06c870 0x7fe5dc0aafa0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 -- 172.21.15.39:0/1817497569 >> 172.21.15.39:0/1817497569 conn(0x7fe5dc069e10 msgr2=0x7fe5dc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:30.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 -- 172.21.15.39:0/1817497569 shutdown_connections 2024-09-13T16:48:30.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 -- 172.21.15.39:0/1817497569 wait complete. 2024-09-13T16:48:30.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 Processor -- start 2024-09-13T16:48:30.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 -- start start 2024-09-13T16:48:30.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5dc06c870 0x7fe5dc14ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc0aabc0 0x7fe5dc1edba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5dc0c40b0 0x7fe5dc149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc1ee630 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc149d90 con 0x7fe5dc0c40b0 2024-09-13T16:48:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5e1711640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc14a070 con 0x7fe5dc06c870 2024-09-13T16:48:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc0aabc0 0x7fe5dc1edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc0aabc0 0x7fe5dc1edba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36484/0 (socket says 172.21.15.39:36484) 2024-09-13T16:48:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 -- 172.21.15.39:0/4249921246 learned_addr learned my addr 172.21.15.39:0/4249921246 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5db7fe640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5dc0c40b0 0x7fe5dc149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5daffd640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5dc06c870 0x7fe5dc14ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 -- 172.21.15.39:0/4249921246 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5dc06c870 msgr2=0x7fe5dc14ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5dc06c870 0x7fe5dc14ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 -- 172.21.15.39:0/4249921246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5dc0c40b0 msgr2=0x7fe5dc149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5dc0c40b0 0x7fe5dc149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 -- 172.21.15.39:0/4249921246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0009420 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5daffd640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5dc06c870 0x7fe5dc14ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.764+0000 7fe5da7fc640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc0aabc0 0x7fe5dc1edba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0019fd0 tx=0x7fe5d001abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5c37fe640 1 -- 172.21.15.39:0/4249921246 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00b6890 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5c37fe640 1 -- 172.21.15.39:0/4249921246 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d00ba070 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc151660 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5c37fe640 1 -- 172.21.15.39:0/4249921246 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00b6c50 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc14b9f0 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5dc0aafa0 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5c37fe640 1 -- 172.21.15.39:0/4249921246 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe5d00c9440 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5c37fe640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5c8066690 0x7fe5c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5daffd640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5c8066690 0x7fe5c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5c37fe640 1 -- 172.21.15.39:0/4249921246 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fe5d01223d0 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.768+0000 7fe5daffd640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5c8066690 0x7fe5c8068b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00a850 tx=0x7fe5cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:30.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.772+0000 7fe5c37fe640 1 -- 172.21.15.39:0/4249921246 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe5d00ef4a0 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.972+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe5dc06c650 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.972+0000 7fe5c37fe640 1 -- 172.21.15.39:0/4249921246 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+116 (secure 0 0 0) 0x7fe5d00b6db0 con 0x7fe5dc0aabc0 2024-09-13T16:48:30.977 INFO:teuthology.orchestra.run.smithi039.stdout:name: a, metadata pool: a_metadata, data pools: [a_data ] 2024-09-13T16:48:30.978 INFO:teuthology.orchestra.run.smithi039.stdout:name: b, metadata pool: b_metadata, data pools: [b_data ] 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5c8066690 msgr2=0x7fe5c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5c8066690 0x7fe5c8068b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00a850 tx=0x7fe5cc008040 comp rx=0 tx=0).stop 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc0aabc0 msgr2=0x7fe5dc1edba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc0aabc0 0x7fe5dc1edba0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0019fd0 tx=0x7fe5d001abe0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 shutdown_connections 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe5c8066690 0x7fe5c8068b50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5dc0c40b0 0x7fe5dc149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5dc0aabc0 0x7fe5dc1edba0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 --2- 172.21.15.39:0/4249921246 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5dc06c870 0x7fe5dc14ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 >> 172.21.15.39:0/4249921246 conn(0x7fe5dc069e10 msgr2=0x7fe5dc150f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:30.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 shutdown_connections 2024-09-13T16:48:30.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:30.976+0000 7fe5e1711640 1 -- 172.21.15.39:0/4249921246 wait complete. 2024-09-13T16:48:30.995 INFO:tasks.cephfs.mount:Mounting Ceph FS a; just confirmed its presence on cluster 2024-09-13T16:48:30.996 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:48:30.996 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-13T16:48:30.996 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T16:48:30.996 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:30.996 INFO:tasks.cephfs.mount:self.cephfs_name = a 2024-09-13T16:48:30.997 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T16:48:30.997 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:48:30.997 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:48:30.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:30.997 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-13T16:48:31.003 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:48:31.003 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:48:31.003 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:48:31.004 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:48:31.004 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-13T16:48:31.004 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:48:31.004 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:48:31.004 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:31.004 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-13T16:48:31.004 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:48:31.004 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:31.005 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-13T16:48:31.005 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:48:31.005 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:31.005 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-13T16:48:31.005 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T16:48:31.005 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42449sec preferred_lft 42449sec 2024-09-13T16:48:31.006 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-13T16:48:31.006 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:48:31.007 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:48:31.007 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:31.007 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-13T16:48:31.007 INFO:teuthology.orchestra.run.smithi039.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T16:48:31.007 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 02:42:8b:46:41:84 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:31.007 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T16:48:31.007 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:48:31.007 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T16:48:31.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.008 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-09-13T16:48:31.008 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-13T16:48:31.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2024-09-13T16:48:31.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2024-09-13T16:48:31.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2024-09-13T16:48:31.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T16:48:31.008 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-13T16:48:31.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.080 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:48:31.131 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-13T16:48:31.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.132 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-13T16:48:31.207 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-13T16:48:31.207 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:48:31.208 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T16:48:31.208 INFO:teuthology.orchestra.run.smithi039.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T16:48:31.208 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T16:48:31.208 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:48:31.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.208 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-09-13T16:48:31.208 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-13T16:48:31.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T16:48:31.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T16:48:31.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T16:48:31.209 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-13T16:48:31.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.280 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-13T16:48:31.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.331 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-09-13T16:48:31.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.380 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-09-13T16:48:31.380 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-13T16:48:31.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T16:48:31.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T16:48:31.381 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-13T16:48:31.448 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T16:48:31.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.448 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-09-13T16:48:31.448 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-13T16:48:31.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T16:48:31.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T16:48:31.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T16:48:31.449 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T16:48:31.449 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T16:48:31.449 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-13T16:48:31.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.553 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2024-09-13T16:48:31.553 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-13T16:48:31.553 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-13T16:48:31.553 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T16:48:31.553 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-13T16:48:31.572 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'cephfs_name': 'a'} 2024-09-13T16:48:31.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:48:31.572 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:31.623 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T16:48:31.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:48:31.623 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:31.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-09-13T16:48:31.726 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:48:31.726 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:48:31.779 INFO:teuthology.orchestra.run.smithi039.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:48:31.780 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:48:31.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.781 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-09-13T16:48:31.828 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T16:48:31.828 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_fs=a) 2024-09-13T16:48:31.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-09-13T16:48:31.886 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:48:31.886 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:48:31.945 INFO:teuthology.orchestra.run.smithi039.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:48:31.946 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:48:31.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:31.947 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-09-13T16:48:31.956 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:2024-09-13T16:48:31.952+0000 7f3e51ef7580 -1 init, newargv = 0x55f135e266d0 newargc=15 2024-09-13T16:48:31.956 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:ceph-fuse[36878]: starting ceph client 2024-09-13T16:48:31.964 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:ceph-fuse[36878]: starting fuse 2024-09-13T16:48:33.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-09-13T16:48:33.078 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:48:33.078 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:48:33.131 INFO:teuthology.orchestra.run.smithi039.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:48:33.133 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:48:33.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:33.134 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-09-13T16:48:33.180 INFO:teuthology.orchestra.run.smithi039.stdout:45 2024-09-13T16:48:33.181 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-13T16:48:33.181 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:48:33.181 DEBUG:teuthology.orchestra.run.smithi039:> import glob 2024-09-13T16:48:33.181 DEBUG:teuthology.orchestra.run.smithi039:> import re 2024-09-13T16:48:33.181 DEBUG:teuthology.orchestra.run.smithi039:> import os 2024-09-13T16:48:33.181 DEBUG:teuthology.orchestra.run.smithi039:> import subprocess 2024-09-13T16:48:33.181 DEBUG:teuthology.orchestra.run.smithi039:> 2024-09-13T16:48:33.181 DEBUG:teuthology.orchestra.run.smithi039:> def _find_admin_socket(client_name): 2024-09-13T16:48:33.181 DEBUG:teuthology.orchestra.run.smithi039:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> files = glob.glob(asok_path) 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> # Given a non-glob path, it better be there 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> if "*" not in asok_path: 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> assert(len(files) == 1) 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> return files[0] 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> for f in files: 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> contents = proc_f.read() 2024-09-13T16:48:33.182 DEBUG:teuthology.orchestra.run.smithi039:> if mountpoint in contents: 2024-09-13T16:48:33.183 DEBUG:teuthology.orchestra.run.smithi039:> return f 2024-09-13T16:48:33.183 DEBUG:teuthology.orchestra.run.smithi039:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T16:48:33.183 DEBUG:teuthology.orchestra.run.smithi039:> 2024-09-13T16:48:33.183 DEBUG:teuthology.orchestra.run.smithi039:> print(_find_admin_socket("client.0")) 2024-09-13T16:48:33.183 DEBUG:teuthology.orchestra.run.smithi039:> ' 2024-09-13T16:48:33.305 INFO:teuthology.orchestra.run.smithi039.stdout:/var/run/ceph/ceph-client.0.36878.asok 2024-09-13T16:48:33.486 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36878.asok 2024-09-13T16:48:33.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:33.487 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36878.asok status 2024-09-13T16:48:33.654 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T16:48:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "metadata": { 2024-09-13T16:48:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_sha1": "50bbe7f395997dcd871813949c49addbbf630361", 2024-09-13T16:48:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)", 2024-09-13T16:48:33.654 INFO:teuthology.orchestra.run.smithi039.stdout: "entity_id": "0", 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": "smithi039", 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "pid": "36878", 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "root": "/" 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_count": 0, 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_pinned_count": 0, 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 4845, 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "inst": { 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "name": { 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "client", 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: "num": 4845 2024-09-13T16:48:33.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1249971345 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1249971345 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-13T16:48:33.656 INFO:teuthology.orchestra.run.smithi039.stdout: "inst_str": "client.4845 192.168.0.1:0/1249971345", 2024-09-13T16:48:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "addr_str": "192.168.0.1:0/1249971345", 2024-09-13T16:48:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "inode_count": 1, 2024-09-13T16:48:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_epoch": 8, 2024-09-13T16:48:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch": 24, 2024-09-13T16:48:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch_barrier": 0, 2024-09-13T16:48:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklisted": false, 2024-09-13T16:48:33.657 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "a" 2024-09-13T16:48:33.657 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T16:48:33.668 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T16:48:33.668 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T16:48:33.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.791+0000 7f62d61d2640 1 Processor -- start 2024-09-13T16:48:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.791+0000 7f62d61d2640 1 -- start start 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.791+0000 7f62d61d2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d006b080 0x7f62d006e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.791+0000 7f62d61d2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f62d00c1810 0x7f62d0151390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.791+0000 7f62d61d2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d00c15d0 con 0x7f62d01518d0 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.791+0000 7f62d61d2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d006d0b0 con 0x7f62d00c1810 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.791+0000 7f62d61d2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00c1f00 con 0x7f62d006b080 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f62d01518d0 0x7f62d01576e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53752/0 (socket says 172.21.15.39:53752) 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cffff640 1 -- 172.21.15.39:0/4090401206 learned_addr learned my addr 172.21.15.39:0/4090401206 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62c6ffd640 1 --2- 172.21.15.39:0/4090401206 >> v2:172.21.15.84:3300/0 conn(0x7f62d00c1810 0x7f62d0151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cf7fe640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d006b080 0x7f62d006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626210670 0 0) 0x7f62d00c15d0 con 0x7f62d01518d0 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62bc003530 con 0x7f62d01518d0 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 91313469 0 0) 0x7f62bc003530 con 0x7f62d01518d0 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d006b080 msgr2=0x7f62d006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d006b080 0x7f62d006e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 >> v2:172.21.15.84:3300/0 conn(0x7f62d00c1810 msgr2=0x7f62d0151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 --2- 172.21.15.39:0/4090401206 >> v2:172.21.15.84:3300/0 conn(0x7f62d00c1810 0x7f62d0151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00c21a0 con 0x7f62d01518d0 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f62c8001a20 con 0x7f62d01518d0 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 >> v1:172.21.15.39:6789/0 conn(0x7f62d01518d0 legacy=0x7f62d01576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62bc003980 0x7f62bc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc0047d0 0x7f62bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d006b080 0x7f62bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00c1f00 con 0x7f62bc003980 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc00b990 con 0x7f62bc0047d0 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc00bad0 con 0x7f62d006b080 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cf7fe640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62bc003980 0x7f62bc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cffff640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d006b080 0x7f62bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62c6ffd640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc0047d0 0x7f62bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cf7fe640 1 -- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d006b080 msgr2=0x7f62bc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cf7fe640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d006b080 0x7f62bc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cf7fe640 1 -- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc0047d0 msgr2=0x7f62bc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cf7fe640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc0047d0 0x7f62bc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cf7fe640 1 -- 172.21.15.39:0/4090401206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc003530 con 0x7f62bc003980 2024-09-13T16:48:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62cf7fe640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62bc003980 0x7f62bc003da0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009ae0 tx=0x7f62c001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc53f0f604be764 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00a6450 con 0x7f62bc003980 2024-09-13T16:48:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c00a65b0 con 0x7f62bc003980 2024-09-13T16:48:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.795+0000 7f62ceffd640 1 -- 172.21.15.39:0/4090401206 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00a6960 con 0x7f62bc003980 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- 172.21.15.39:0/4090401206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62bc003980 msgr2=0x7f62bc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 --2- 172.21.15.39:0/4090401206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62bc003980 0x7f62bc003da0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009ae0 tx=0x7f62c001ca40 comp rx=0 tx=0).stop 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 -- 172.21.15.39:0/4090401206 reap_dead start 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- 172.21.15.39:0/4090401206 shutdown_connections 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- 172.21.15.39:0/4090401206 >> 172.21.15.39:0/4090401206 conn(0x7f62d00c32a0 msgr2=0x7f62d00c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- 172.21.15.39:0/4090401206 shutdown_connections 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- 172.21.15.39:0/4090401206 wait complete. 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 Processor -- start 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- start start 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc003980 0x7f62d014dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d014cad0 0x7f62d0071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62d014cf50 0x7f62d0071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01ed4e0 con 0x7f62d014cf50 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01ec8d0 con 0x7f62bc003980 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01ecbb0 con 0x7f62d014cad0 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62d014cf50 0x7f62d0071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62d014cf50 0x7f62d0071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36504/0 (socket says 172.21.15.39:36504) 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 -- 172.21.15.39:0/4031568172 learned_addr learned my addr 172.21.15.39:0/4031568172 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62c6ffd640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d014cad0 0x7f62d0071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cf7fe640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc003980 0x7f62d014dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 -- 172.21.15.39:0/4031568172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d014cad0 msgr2=0x7f62d0071a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d014cad0 0x7f62d0071a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 -- 172.21.15.39:0/4031568172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc003980 msgr2=0x7f62d014dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc003980 0x7f62d014dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 -- 172.21.15.39:0/4031568172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d01ece90 con 0x7f62d014cf50 2024-09-13T16:48:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62cffff640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62d014cf50 0x7f62d0071f60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f62c8002240 tx=0x7f62c800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62ccff9640 1 -- 172.21.15.39:0/4031568172 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8007ae0 con 0x7f62d014cf50 2024-09-13T16:48:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d01ed170 con 0x7f62d014cf50 2024-09-13T16:48:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62ccff9640 1 -- 172.21.15.39:0/4031568172 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c800ee20 con 0x7f62d014cf50 2024-09-13T16:48:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62ccff9640 1 -- 172.21.15.39:0/4031568172 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8013640 con 0x7f62d014cf50 2024-09-13T16:48:33.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.799+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d0150ce0 con 0x7f62d014cf50 2024-09-13T16:48:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.803+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d00c3bd0 con 0x7f62d014cf50 2024-09-13T16:48:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.803+0000 7f62ccff9640 1 -- 172.21.15.39:0/4031568172 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f62c80137a0 con 0x7f62d014cf50 2024-09-13T16:48:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.803+0000 7f62ccff9640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62b0066690 0x7f62b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.803+0000 7f62ccff9640 1 -- 172.21.15.39:0/4031568172 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f62c8084780 con 0x7f62d014cf50 2024-09-13T16:48:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.803+0000 7f62cf7fe640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62b0066690 0x7f62b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.803+0000 7f62cf7fe640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62b0066690 0x7f62b0068b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ad000 tx=0x7f62c00b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:33.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:33.807+0000 7f62ccff9640 1 -- 172.21.15.39:0/4031568172 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f62c8051900 con 0x7f62d014cf50 2024-09-13T16:48:34.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.003+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62d0151390 con 0x7f62d014cf50 2024-09-13T16:48:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.003+0000 7f62ccff9640 1 -- 172.21.15.39:0/4031568172 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+116 (secure 0 0 0) 0x7f62c8054f20 con 0x7f62d014cf50 2024-09-13T16:48:34.010 INFO:teuthology.orchestra.run.smithi039.stdout:name: a, metadata pool: a_metadata, data pools: [a_data ] 2024-09-13T16:48:34.010 INFO:teuthology.orchestra.run.smithi039.stdout:name: b, metadata pool: b_metadata, data pools: [b_data ] 2024-09-13T16:48:34.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62b0066690 msgr2=0x7f62b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62b0066690 0x7f62b0068b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ad000 tx=0x7f62c00b3040 comp rx=0 tx=0).stop 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62d014cf50 msgr2=0x7f62d0071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62d014cf50 0x7f62d0071f60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f62c8002240 tx=0x7f62c800e530 comp rx=0 tx=0).stop 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 shutdown_connections 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62b0066690 0x7f62b0068b50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62d014cf50 0x7f62d0071f60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62d014cad0 0x7f62d0071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 --2- 172.21.15.39:0/4031568172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62bc003980 0x7f62d014dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 >> 172.21.15.39:0/4031568172 conn(0x7f62d00c32a0 msgr2=0x7f62d00bd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 shutdown_connections 2024-09-13T16:48:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:34.007+0000 7f62d61d2640 1 -- 172.21.15.39:0/4031568172 wait complete. 2024-09-13T16:48:34.030 INFO:tasks.cephfs.mount:Mounting Ceph FS b; just confirmed its presence on cluster 2024-09-13T16:48:34.030 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T16:48:34.030 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2024-09-13T16:48:34.030 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T16:48:34.030 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:34.030 INFO:tasks.cephfs.mount:self.cephfs_name = b 2024-09-13T16:48:34.030 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T16:48:34.030 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T16:48:34.030 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T16:48:34.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.031 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2024-09-13T16:48:34.037 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2024-09-13T16:48:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42409sec preferred_lft 42409sec 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::ec4:7aff:fe8f:d751/64 scope link 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 02:42:85:bc:f9:81 brd ff:ff:ff:ff:ff:ff 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T16:48:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-09-13T16:48:34.039 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T16:48:34.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.040 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-09-13T16:48:34.040 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-09-13T16:48:34.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add name ceph-brx type bridge 2024-09-13T16:48:34.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr flush dev ceph-brx 2024-09-13T16:48:34.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set ceph-brx up 2024-09-13T16:48:34.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T16:48:34.040 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-09-13T16:48:34.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.122 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T16:48:34.172 INFO:teuthology.orchestra.run.smithi084.stdout:1 2024-09-13T16:48:34.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.174 DEBUG:teuthology.orchestra.run.smithi084:> route 2024-09-13T16:48:34.315 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2024-09-13T16:48:34.315 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T16:48:34.315 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T16:48:34.316 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T16:48:34.316 INFO:teuthology.orchestra.run.smithi084.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T16:48:34.316 INFO:teuthology.orchestra.run.smithi084.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T16:48:34.316 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T16:48:34.316 INFO:teuthology.orchestra.run.smithi084.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T16:48:34.316 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T16:48:34.316 INFO:teuthology.orchestra.run.smithi084.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T16:48:34.316 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T16:48:34.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.317 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-09-13T16:48:34.317 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-09-13T16:48:34.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T16:48:34.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T16:48:34.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T16:48:34.317 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-09-13T16:48:34.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.353 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-09-13T16:48:34.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.401 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2024-09-13T16:48:34.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.449 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-09-13T16:48:34.449 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-09-13T16:48:34.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T16:48:34.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-13T16:48:34.449 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-09-13T16:48:34.517 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-13T16:48:34.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.517 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-09-13T16:48:34.518 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-09-13T16:48:34.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-13T16:48:34.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T16:48:34.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-13T16:48:34.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-13T16:48:34.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-13T16:48:34.518 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-09-13T16:48:34.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.633 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2024-09-13T16:48:34.633 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-09-13T16:48:34.633 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2024-09-13T16:48:34.633 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T16:48:34.633 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-09-13T16:48:34.653 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {'cephfs_name': 'b'} 2024-09-13T16:48:34.653 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:48:34.653 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:34.699 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T16:48:34.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T16:48:34.699 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:34.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2024-09-13T16:48:34.804 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:48:34.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:48:34.858 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:48:34.859 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:48:34.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:34.859 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2024-09-13T16:48:34.908 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T16:48:34.908 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1 --client_fs=b) 2024-09-13T16:48:34.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2024-09-13T16:48:34.969 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:48:34.969 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:48:35.023 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:48:35.024 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:48:35.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:35.025 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2024-09-13T16:48:35.031 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi084.stderr:2024-09-13T16:48:35.030+0000 7fca8a130580 -1 init, newargv = 0x55e087e12500 newargc=15 2024-09-13T16:48:35.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi084.stderr:ceph-fuse[34829]: starting ceph client 2024-09-13T16:48:35.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi084.stderr:ceph-fuse[34829]: starting fuse 2024-09-13T16:48:36.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2024-09-13T16:48:36.157 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T16:48:36.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T16:48:36.212 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T16:48:36.213 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T16:48:36.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:36.214 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2024-09-13T16:48:36.260 INFO:teuthology.orchestra.run.smithi084.stdout:45 2024-09-13T16:48:36.260 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-13T16:48:36.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T16:48:36.260 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2024-09-13T16:48:36.260 DEBUG:teuthology.orchestra.run.smithi084:> import re 2024-09-13T16:48:36.260 DEBUG:teuthology.orchestra.run.smithi084:> import os 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2024-09-13T16:48:36.261 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T16:48:36.262 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T16:48:36.262 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T16:48:36.262 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2024-09-13T16:48:36.262 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2024-09-13T16:48:36.262 DEBUG:teuthology.orchestra.run.smithi084:> return f 2024-09-13T16:48:36.262 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T16:48:36.262 DEBUG:teuthology.orchestra.run.smithi084:> 2024-09-13T16:48:36.262 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.1")) 2024-09-13T16:48:36.262 DEBUG:teuthology.orchestra.run.smithi084:> ' 2024-09-13T16:48:36.381 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.1.34829.asok 2024-09-13T16:48:36.563 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34829.asok 2024-09-13T16:48:36.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:36.564 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34829.asok status 2024-09-13T16:48:36.724 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-13T16:48:36.724 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "50bbe7f395997dcd871813949c49addbbf630361", 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)", 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "1", 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "34829", 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 4789, 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2024-09-13T16:48:36.725 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 4789 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1485881945 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1485881945 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-13T16:48:36.726 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.4789 192.168.0.1:0/1485881945", 2024-09-13T16:48:36.727 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/1485881945", 2024-09-13T16:48:36.727 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2024-09-13T16:48:36.727 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 14, 2024-09-13T16:48:36.727 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 24, 2024-09-13T16:48:36.727 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2024-09-13T16:48:36.727 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2024-09-13T16:48:36.727 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "b" 2024-09-13T16:48:36.727 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-13T16:48:36.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:36.739 DEBUG:teuthology.orchestra.run.smithi039:> stat --file-system '--printf=%T 2024-09-13T16:48:36.739 DEBUG:teuthology.orchestra.run.smithi039:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:36.747 INFO:teuthology.orchestra.run.smithi039.stdout:fuseblk 2024-09-13T16:48:36.747 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:36.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:36.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:36.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:36.807 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2024-09-13T16:48:36.807 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:36.814 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2024-09-13T16:48:36.814 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:36.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T16:48:36.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:36.869 INFO:teuthology.run_tasks:Running task mds_thrash... 2024-09-13T16:48:36.884 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:37.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.011+0000 7f098888d640 1 Processor -- start 2024-09-13T16:48:37.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f098888d640 1 -- start start 2024-09-13T16:48:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f098888d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0984070fb0 0x7f0984071420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f098888d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984153d20 0x7f0984154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f098888d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09840c9710 con 0x7f09840ca080 2024-09-13T16:48:37.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f098888d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0984157fc0 con 0x7f0984070fb0 2024-09-13T16:48:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f098888d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0984071960 con 0x7f0984153d20 2024-09-13T16:48:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f0981d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0984070fb0 0x7f0984071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f0981d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0984070fb0 0x7f0984071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36510/0 (socket says 172.21.15.39:36510) 2024-09-13T16:48:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f0981d9b640 1 -- 172.21.15.39:0/3675892641 learned_addr learned my addr 172.21.15.39:0/3675892641 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f0982d9d640 1 --2- 172.21.15.39:0/3675892641 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984153d20 0x7f0984154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f0981d9b640 1 -- 172.21.15.39:0/3675892641 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984153d20 msgr2=0x7f0984154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f0981d9b640 1 --2- 172.21.15.39:0/3675892641 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984153d20 0x7f0984154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f0981d9b640 1 -- 172.21.15.39:0/3675892641 >> v1:172.21.15.84:6789/0 conn(0x7f09840ca080 legacy=0x7f0984151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f0981d9b640 1 -- 172.21.15.39:0/3675892641 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0984157340 con 0x7f0984070fb0 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.015+0000 7f0981d9b640 1 --2- 172.21.15.39:0/3675892641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0984070fb0 0x7f0984071420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0974009870 tx=0x7f097401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc1ce7852b222f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098159a640 1 -- 172.21.15.39:0/3675892641 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09740ab600 con 0x7f0984070fb0 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098159a640 1 -- 172.21.15.39:0/3675892641 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09740a7050 con 0x7f0984070fb0 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098159a640 1 -- 172.21.15.39:0/3675892641 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09740ab980 con 0x7f0984070fb0 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 -- 172.21.15.39:0/3675892641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0984070fb0 msgr2=0x7f0984071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 --2- 172.21.15.39:0/3675892641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0984070fb0 0x7f0984071420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0974009870 tx=0x7f097401c440 comp rx=0 tx=0).stop 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 -- 172.21.15.39:0/3675892641 shutdown_connections 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 --2- 172.21.15.39:0/3675892641 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984153d20 0x7f0984154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 --2- 172.21.15.39:0/3675892641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0984070fb0 0x7f0984071420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 -- 172.21.15.39:0/3675892641 >> 172.21.15.39:0/3675892641 conn(0x7f098406a210 msgr2=0x7f098406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:37.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 -- 172.21.15.39:0/3675892641 shutdown_connections 2024-09-13T16:48:37.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 -- 172.21.15.39:0/3675892641 wait complete. 2024-09-13T16:48:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 Processor -- start 2024-09-13T16:48:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 -- start start 2024-09-13T16:48:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984070fb0 0x7f0984152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09840ca080 0x7f09841ec310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0984153d20 0x7f09841ec850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09841f0800 con 0x7f09840ca080 2024-09-13T16:48:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09840abc50 con 0x7f0984153d20 2024-09-13T16:48:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098888d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09840ac440 con 0x7f0984070fb0 2024-09-13T16:48:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0981d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09840ca080 0x7f09841ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0981d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09840ca080 0x7f09841ec310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36512/0 (socket says 172.21.15.39:36512) 2024-09-13T16:48:37.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0981d9b640 1 -- 172.21.15.39:0/2866322283 learned_addr learned my addr 172.21.15.39:0/2866322283 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:37.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0982d9d640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0984153d20 0x7f09841ec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f098259c640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984070fb0 0x7f0984152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0981d9b640 1 -- 172.21.15.39:0/2866322283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984070fb0 msgr2=0x7f0984152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0981d9b640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984070fb0 0x7f0984152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0981d9b640 1 -- 172.21.15.39:0/2866322283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0984153d20 msgr2=0x7f09841ec850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0981d9b640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0984153d20 0x7f09841ec850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0981d9b640 1 -- 172.21.15.39:0/2866322283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09740094d0 con 0x7f09840ca080 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0981d9b640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09840ca080 0x7f09841ec310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f09740ae040 tx=0x7f09740191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.019+0000 7f0972ffd640 1 -- 172.21.15.39:0/2866322283 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09740b4440 con 0x7f09840ca080 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f0972ffd640 1 -- 172.21.15.39:0/2866322283 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09740b45a0 con 0x7f09840ca080 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09841ecd90 con 0x7f09840ca080 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f0972ffd640 1 -- 172.21.15.39:0/2866322283 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09740b4950 con 0x7f09840ca080 2024-09-13T16:48:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09840ac160 con 0x7f09840ca080 2024-09-13T16:48:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0984071420 con 0x7f09840ca080 2024-09-13T16:48:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f0972ffd640 1 -- 172.21.15.39:0/2866322283 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09740b2070 con 0x7f09840ca080 2024-09-13T16:48:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f0972ffd640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0964066690 0x7f0964068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f0972ffd640 1 -- 172.21.15.39:0/2866322283 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f097411e1f0 con 0x7f09840ca080 2024-09-13T16:48:37.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f098259c640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0964066690 0x7f0964068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.023+0000 7f098259c640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0964066690 0x7f0964068b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0978007fd0 tx=0x7f0978009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.027+0000 7f0972ffd640 1 -- 172.21.15.39:0/2866322283 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09740eb2e0 con 0x7f09840ca080 2024-09-13T16:48:37.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.191+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0984151ee0 con 0x7f09840ca080 2024-09-13T16:48:37.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.191+0000 7f0972ffd640 1 -- 172.21.15.39:0/2866322283 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7f09740ee900 con 0x7f09840ca080 2024-09-13T16:48:37.199 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:37.199 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:37.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0964066690 msgr2=0x7f0964068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0964066690 0x7f0964068b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0978007fd0 tx=0x7f0978009040 comp rx=0 tx=0).stop 2024-09-13T16:48:37.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09840ca080 msgr2=0x7f09841ec310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09840ca080 0x7f09841ec310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f09740ae040 tx=0x7f09740191e0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 shutdown_connections 2024-09-13T16:48:37.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0964066690 0x7f0964068b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0984153d20 0x7f09841ec850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09840ca080 0x7f09841ec310 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 --2- 172.21.15.39:0/2866322283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0984070fb0 0x7f0984152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 >> 172.21.15.39:0/2866322283 conn(0x7f098406a210 msgr2=0x7f09840c3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:37.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 shutdown_connections 2024-09-13T16:48:37.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.195+0000 7f098888d640 1 -- 172.21.15.39:0/2866322283 wait complete. 2024-09-13T16:48:37.218 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:48:37.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:48:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb8b7b640 1 Processor -- start 2024-09-13T16:48:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb8b7b640 1 -- start start 2024-09-13T16:48:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb8b7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbcb414e530 0x7fbcb40c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb8b7b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb41540e0 0x7fbcb40c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb8b7b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbcb40c4ce0 0x7fbcb4151070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb8b7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb4150b70 con 0x7fbcb414e530 2024-09-13T16:48:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb8b7b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb4154570 con 0x7fbcb40c4ce0 2024-09-13T16:48:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb8b7b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb4154850 con 0x7fbcb41540e0 2024-09-13T16:48:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbcb414e530 0x7fbcb40c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbcb414e530 0x7fbcb40c1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36528/0 (socket says 172.21.15.39:36528) 2024-09-13T16:48:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb259c640 1 -- 172.21.15.39:0/968123660 learned_addr learned my addr 172.21.15.39:0/968123660 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:37.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb2d9d640 1 --2- 172.21.15.39:0/968123660 >> v2:172.21.15.84:3300/0 conn(0x7fbcb40c4ce0 0x7fbcb4151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb1d9b640 1 --2- 172.21.15.39:0/968123660 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb41540e0 0x7fbcb40c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb259c640 1 -- 172.21.15.39:0/968123660 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb41540e0 msgr2=0x7fbcb40c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:37.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb259c640 1 --2- 172.21.15.39:0/968123660 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb41540e0 0x7fbcb40c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb259c640 1 -- 172.21.15.39:0/968123660 >> v2:172.21.15.84:3300/0 conn(0x7fbcb40c4ce0 msgr2=0x7fbcb4151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:37.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb259c640 1 --2- 172.21.15.39:0/968123660 >> v2:172.21.15.84:3300/0 conn(0x7fbcb40c4ce0 0x7fbcb4151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.335+0000 7fbcb259c640 1 -- 172.21.15.39:0/968123660 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb4154b30 con 0x7fbcb414e530 2024-09-13T16:48:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb259c640 1 --2- 172.21.15.39:0/968123660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb414e530 0x7fbcb40c1660 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40a85e0 tx=0x7fbca401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17713b5fb853bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb159a640 1 -- 172.21.15.39:0/968123660 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40ab600 con 0x7fbcb414e530 2024-09-13T16:48:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb159a640 1 -- 172.21.15.39:0/968123660 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca40ab760 con 0x7fbcb414e530 2024-09-13T16:48:37.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb159a640 1 -- 172.21.15.39:0/968123660 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca40abaf0 con 0x7fbcb414e530 2024-09-13T16:48:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/968123660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb414e530 msgr2=0x7fbcb40c1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/968123660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb414e530 0x7fbcb40c1660 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40a85e0 tx=0x7fbca401c460 comp rx=0 tx=0).stop 2024-09-13T16:48:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/968123660 shutdown_connections 2024-09-13T16:48:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/968123660 >> v2:172.21.15.84:3300/0 conn(0x7fbcb40c4ce0 0x7fbcb4151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/968123660 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb41540e0 0x7fbcb40c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/968123660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb414e530 0x7fbcb40c1660 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/968123660 >> 172.21.15.39:0/968123660 conn(0x7fbcb40c5790 msgr2=0x7fbcb40c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:37.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/968123660 shutdown_connections 2024-09-13T16:48:37.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/968123660 wait complete. 2024-09-13T16:48:37.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 Processor -- start 2024-09-13T16:48:37.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 -- start start 2024-09-13T16:48:37.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb40c4ce0 0x7fbcb4143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb414e530 0x7fbcb4149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb4142b30 0x7fbcb40bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb414ccb0 con 0x7fbcb4142b30 2024-09-13T16:48:37.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb4147bf0 con 0x7fbcb40c4ce0 2024-09-13T16:48:37.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb8b7b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb414dba0 con 0x7fbcb414e530 2024-09-13T16:48:37.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb4142b30 0x7fbcb40bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb4142b30 0x7fbcb40bebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36534/0 (socket says 172.21.15.39:36534) 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb2d9d640 1 -- 172.21.15.39:0/2755929073 learned_addr learned my addr 172.21.15.39:0/2755929073 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb259c640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb40c4ce0 0x7fbcb4143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb1d9b640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb414e530 0x7fbcb4149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb2d9d640 1 -- 172.21.15.39:0/2755929073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb414e530 msgr2=0x7fbcb4149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb2d9d640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb414e530 0x7fbcb4149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb2d9d640 1 -- 172.21.15.39:0/2755929073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb40c4ce0 msgr2=0x7fbcb4143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb2d9d640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb40c4ce0 0x7fbcb4143010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.339+0000 7fbcb2d9d640 1 -- 172.21.15.39:0/2755929073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca40094d0 con 0x7fbcb4142b30 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbcb2d9d640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb4142b30 0x7fbcb40bebc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbcac007e60 tx=0x7fbcac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbca2ffd640 1 -- 172.21.15.39:0/2755929073 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac014620 con 0x7fbcb4142b30 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbca2ffd640 1 -- 172.21.15.39:0/2755929073 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac00ee90 con 0x7fbcb4142b30 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbca2ffd640 1 -- 172.21.15.39:0/2755929073 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0149e0 con 0x7fbcb4142b30 2024-09-13T16:48:37.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb4147880 con 0x7fbcb4142b30 2024-09-13T16:48:37.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb414a180 con 0x7fbcb4142b30 2024-09-13T16:48:37.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb40c2dc0 con 0x7fbcb4142b30 2024-09-13T16:48:37.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbca2ffd640 1 -- 172.21.15.39:0/2755929073 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbcac014b40 con 0x7fbcb4142b30 2024-09-13T16:48:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbca2ffd640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc94066690 0x7fbc94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbca2ffd640 1 -- 172.21.15.39:0/2755929073 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fbcac084ca0 con 0x7fbcb4142b30 2024-09-13T16:48:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbcb259c640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc94066690 0x7fbc94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.343+0000 7fbcb259c640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc94066690 0x7fbc94068b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbca40af000 tx=0x7fbca40b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.347+0000 7fbca2ffd640 1 -- 172.21.15.39:0/2755929073 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbcac051d90 con 0x7fbcb4142b30 2024-09-13T16:48:37.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.507+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbcb414bd60 con 0x7fbcb4142b30 2024-09-13T16:48:37.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.507+0000 7fbca2ffd640 1 -- 172.21.15.39:0/2755929073 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbcac0553b0 con 0x7fbcb4142b30 2024-09-13T16:48:37.514 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:48:37.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc94066690 msgr2=0x7fbc94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc94066690 0x7fbc94068b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbca40af000 tx=0x7fbca40b6040 comp rx=0 tx=0).stop 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb4142b30 msgr2=0x7fbcb40bebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb4142b30 0x7fbcb40bebc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbcac007e60 tx=0x7fbcac00e4b0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 shutdown_connections 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc94066690 0x7fbc94068b50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb4142b30 0x7fbcb40bebc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb414e530 0x7fbcb4149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 --2- 172.21.15.39:0/2755929073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb40c4ce0 0x7fbcb4143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 >> 172.21.15.39:0/2755929073 conn(0x7fbcb40c5790 msgr2=0x7fbcb414e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 shutdown_connections 2024-09-13T16:48:37.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.511+0000 7fbcb8b7b640 1 -- 172.21.15.39:0/2755929073 wait complete. 2024-09-13T16:48:37.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T16:48:37.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0610d90640 1 Processor -- start 2024-09-13T16:48:37.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0610d90640 1 -- start start 2024-09-13T16:48:37.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0610d90640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0aa2d0 0x7f060c0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0610d90640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c155d20 con 0x7f060c151cf0 2024-09-13T16:48:37.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0610d90640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c0c4750 con 0x7f060c0a4d60 2024-09-13T16:48:37.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0610d90640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c155b30 con 0x7f060c0aa2d0 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060ad9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f060c151cf0 0x7f060c152180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53766/0 (socket says 172.21.15.39:53766) 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060ad9d640 1 -- 172.21.15.39:0/2854329624 learned_addr learned my addr 172.21.15.39:0/2854329624 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060a59c640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0aa2d0 0x7f060c0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939276997 0 0) 0x7f060c155d20 con 0x7f060c151cf0 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f4003530 con 0x7f060c151cf0 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836752505 0 0) 0x7f060c0c4750 con 0x7f060c0a4d60 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f060c155d20 con 0x7f060c0a4d60 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2481097815 0 0) 0x7f05f4003530 con 0x7f060c151cf0 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0aa2d0 msgr2=0x7f060c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0aa2d0 0x7f060c0aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 >> v1:172.21.15.84:6789/0 conn(0x7f060c0a4d60 legacy=0x7f060c14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c155fc0 con 0x7f060c151cf0 2024-09-13T16:48:37.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f0604001a00 con 0x7f060c151cf0 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 >> v1:172.21.15.39:6789/0 conn(0x7f060c151cf0 legacy=0x7f060c152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05f4003b40 0x7f05f4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f40047d0 0x7f05f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0aa2d0 0x7f05f40051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f400bb20 con 0x7f05f40047d0 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f400bc60 con 0x7f05f4003b40 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f400bdf0 con 0x7f060c0aa2d0 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0609d9b640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f40047d0 0x7f05f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060a59c640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05f4003b40 0x7f05f4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0609d9b640 1 -- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0aa2d0 msgr2=0x7f05f40051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060ad9d640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0aa2d0 0x7f05f40051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0609d9b640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0aa2d0 0x7f05f40051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0609d9b640 1 -- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05f4003b40 msgr2=0x7f05f4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:37.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0609d9b640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05f4003b40 0x7f05f4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f0609d9b640 1 -- 172.21.15.39:0/2854329624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f4003530 con 0x7f05f40047d0 2024-09-13T16:48:37.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.647+0000 7f060a59c640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05f4003b40 0x7f05f4003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:37.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0609d9b640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f40047d0 0x7f05f4004c50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0018d0 tx=0x7f05fc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2722bdc818ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0052d0 con 0x7f05f40047d0 2024-09-13T16:48:37.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc0a7050 con 0x7f05f40047d0 2024-09-13T16:48:37.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060959a640 1 -- 172.21.15.39:0/2854329624 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc003600 con 0x7f05f40047d0 2024-09-13T16:48:37.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 -- 172.21.15.39:0/2854329624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f40047d0 msgr2=0x7f05f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 --2- 172.21.15.39:0/2854329624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f40047d0 0x7f05f4004c50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0018d0 tx=0x7f05fc01c510 comp rx=0 tx=0).stop 2024-09-13T16:48:37.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 -- 172.21.15.39:0/2854329624 reap_dead start 2024-09-13T16:48:37.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 -- 172.21.15.39:0/2854329624 shutdown_connections 2024-09-13T16:48:37.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 -- 172.21.15.39:0/2854329624 >> 172.21.15.39:0/2854329624 conn(0x7f060c0c5790 msgr2=0x7f060c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 -- 172.21.15.39:0/2854329624 shutdown_connections 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 -- 172.21.15.39:0/2854329624 wait complete. 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 Processor -- start 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 -- start start 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0c24c0 0x7f060c072120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f060c0719a0 0x7f060c1ea710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060c1ed2f0 0x7f060c141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0a6830 con 0x7f060c1ed2f0 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c1e9d00 con 0x7f060c0719a0 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0610d90640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c1eac50 con 0x7f060c0c24c0 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060c1ed2f0 0x7f060c141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060c1ed2f0 0x7f060c141900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36548/0 (socket says 172.21.15.39:36548) 2024-09-13T16:48:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 -- 172.21.15.39:0/3516063872 learned_addr learned my addr 172.21.15.39:0/3516063872 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060a59c640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0c24c0 0x7f060c072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 -- 172.21.15.39:0/3516063872 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0c24c0 msgr2=0x7f060c072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f0609d9b640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f060c0719a0 0x7f060c1ea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0c24c0 0x7f060c072120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 -- 172.21.15.39:0/3516063872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f060c0719a0 msgr2=0x7f060c1ea710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f060c0719a0 0x7f060c1ea710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 -- 172.21.15.39:0/3516063872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c152a10 con 0x7f060c1ed2f0 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.651+0000 7f060ad9d640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060c1ed2f0 0x7f060c141900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f060c1520d0 tx=0x7f060400de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f05faffd640 1 -- 172.21.15.39:0/3516063872 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060400baf0 con 0x7f060c1ed2f0 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f05faffd640 1 -- 172.21.15.39:0/3516063872 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060400bc70 con 0x7f060c1ed2f0 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c0a83c0 con 0x7f060c1ed2f0 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f05faffd640 1 -- 172.21.15.39:0/3516063872 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06040123d0 con 0x7f060c1ed2f0 2024-09-13T16:48:37.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c1e9f10 con 0x7f060c1ed2f0 2024-09-13T16:48:37.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c0a88e0 con 0x7f060c1ed2f0 2024-09-13T16:48:37.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f05faffd640 1 -- 172.21.15.39:0/3516063872 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0604012530 con 0x7f060c1ed2f0 2024-09-13T16:48:37.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f05faffd640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05ec066690 0x7f05ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f05faffd640 1 -- 172.21.15.39:0/3516063872 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f0604084440 con 0x7f060c1ed2f0 2024-09-13T16:48:37.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f060a59c640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05ec066690 0x7f05ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.655+0000 7f060a59c640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05ec066690 0x7f05ec068b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f06000098f0 tx=0x7f0600008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.659+0000 7f05faffd640 1 -- 172.21.15.39:0/3516063872 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f06040515f0 con 0x7f060c1ed2f0 2024-09-13T16:48:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.815+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f060c14ef70 con 0x7f060c1ed2f0 2024-09-13T16:48:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.815+0000 7f05faffd640 1 -- 172.21.15.39:0/3516063872 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7f0604054c10 con 0x7f060c1ed2f0 2024-09-13T16:48:37.822 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05ec066690 msgr2=0x7f05ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05ec066690 0x7f05ec068b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f06000098f0 tx=0x7f0600008040 comp rx=0 tx=0).stop 2024-09-13T16:48:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060c1ed2f0 msgr2=0x7f060c141900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060c1ed2f0 0x7f060c141900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f060c1520d0 tx=0x7f060400de30 comp rx=0 tx=0).stop 2024-09-13T16:48:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 shutdown_connections 2024-09-13T16:48:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05ec066690 0x7f05ec068b50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f060c0719a0 0x7f060c1ea710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060c1ed2f0 0x7f060c141900 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 --2- 172.21.15.39:0/3516063872 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060c0c24c0 0x7f060c072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 >> 172.21.15.39:0/3516063872 conn(0x7f060c0c5790 msgr2=0x7f060c152180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:37.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 shutdown_connections 2024-09-13T16:48:37.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.819+0000 7f0610d90640 1 -- 172.21.15.39:0/3516063872 wait complete. 2024-09-13T16:48:37.839 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T16:48:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf461c640 1 Processor -- start 2024-09-13T16:48:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf461c640 1 -- start start 2024-09-13T16:48:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf461c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1cec14e530 0x7f1cec0c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf461c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1cec151070 0x7f1cec151480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf461c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cec150b70 con 0x7f1cec1540e0 2024-09-13T16:48:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf461c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cec154570 con 0x7f1cec14e530 2024-09-13T16:48:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf461c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cec154850 con 0x7f1cec151070 2024-09-13T16:48:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf23b8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1cec14e530 0x7f1cec0c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf23b8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1cec14e530 0x7f1cec0c1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36560/0 (socket says 172.21.15.39:36560) 2024-09-13T16:48:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf1bb7640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1cec1540e0 0x7f1cec0c47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53772/0 (socket says 172.21.15.39:53772) 2024-09-13T16:48:37.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.951+0000 7f1cf23b8640 1 -- 172.21.15.39:0/2437156717 learned_addr learned my addr 172.21.15.39:0/2437156717 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:37.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf2bb9640 1 --2- 172.21.15.39:0/2437156717 >> v2:172.21.15.84:3300/0 conn(0x7f1cec151070 0x7f1cec151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf13b6640 1 -- 172.21.15.39:0/2437156717 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343845119 0 0) 0x7f1cec150b70 con 0x7f1cec1540e0 2024-09-13T16:48:37.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf13b6640 1 -- 172.21.15.39:0/2437156717 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cd8003530 con 0x7f1cec1540e0 2024-09-13T16:48:37.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf23b8640 1 -- 172.21.15.39:0/2437156717 >> v2:172.21.15.84:3300/0 conn(0x7f1cec151070 msgr2=0x7f1cec151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:37.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf23b8640 1 --2- 172.21.15.39:0/2437156717 >> v2:172.21.15.84:3300/0 conn(0x7f1cec151070 0x7f1cec151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf23b8640 1 -- 172.21.15.39:0/2437156717 >> v1:172.21.15.39:6789/0 conn(0x7f1cec1540e0 legacy=0x7f1cec0c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf23b8640 1 -- 172.21.15.39:0/2437156717 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec154b30 con 0x7f1cec14e530 2024-09-13T16:48:37.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf23b8640 1 --2- 172.21.15.39:0/2437156717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14e530 0x7f1cec0c1660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0c02d0 tx=0x7f1ce801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3e00278e03a4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf13b6640 1 -- 172.21.15.39:0/2437156717 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80ab640 con 0x7f1cec14e530 2024-09-13T16:48:37.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf13b6640 1 -- 172.21.15.39:0/2437156717 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce80a7070 con 0x7f1cec14e530 2024-09-13T16:48:37.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf13b6640 1 -- 172.21.15.39:0/2437156717 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80ab9a0 con 0x7f1cec14e530 2024-09-13T16:48:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf461c640 1 -- 172.21.15.39:0/2437156717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14e530 msgr2=0x7f1cec0c1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf461c640 1 --2- 172.21.15.39:0/2437156717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14e530 0x7f1cec0c1660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1cec0c02d0 tx=0x7f1ce801c460 comp rx=0 tx=0).stop 2024-09-13T16:48:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf461c640 1 -- 172.21.15.39:0/2437156717 shutdown_connections 2024-09-13T16:48:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf461c640 1 --2- 172.21.15.39:0/2437156717 >> v2:172.21.15.84:3300/0 conn(0x7f1cec151070 0x7f1cec151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf461c640 1 --2- 172.21.15.39:0/2437156717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14e530 0x7f1cec0c1660 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf461c640 1 -- 172.21.15.39:0/2437156717 >> 172.21.15.39:0/2437156717 conn(0x7f1cec0c5790 msgr2=0x7f1cec0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf461c640 1 -- 172.21.15.39:0/2437156717 shutdown_connections 2024-09-13T16:48:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.955+0000 7f1cf461c640 1 -- 172.21.15.39:0/2437156717 wait complete. 2024-09-13T16:48:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 Processor -- start 2024-09-13T16:48:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 -- start start 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1cec14e530 0x7f1cec152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cec151070 0x7f1cec1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14c9f0 0x7f1cec14ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec153a30 con 0x7f1cec14c9f0 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec14c560 con 0x7f1cec151070 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec14f680 con 0x7f1cec14e530 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf2bb9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14c9f0 0x7f1cec14ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf2bb9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14c9f0 0x7f1cec14ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36576/0 (socket says 172.21.15.39:36576) 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf2bb9640 1 -- 172.21.15.39:0/3935309308 learned_addr learned my addr 172.21.15.39:0/3935309308 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf1bb7640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cec151070 0x7f1cec1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf2bb9640 1 -- 172.21.15.39:0/3935309308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1cec14e530 msgr2=0x7f1cec152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf23b8640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1cec14e530 0x7f1cec152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf2bb9640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1cec14e530 0x7f1cec152bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf2bb9640 1 -- 172.21.15.39:0/3935309308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cec151070 msgr2=0x7f1cec1503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf2bb9640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cec151070 0x7f1cec1503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf2bb9640 1 -- 172.21.15.39:0/3935309308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce80094d0 con 0x7f1cec14c9f0 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf1bb7640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cec151070 0x7f1cec1503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf2bb9640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14c9f0 0x7f1cec14ce80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4007e60 tx=0x7f1ce400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1ce2ffd640 1 -- 172.21.15.39:0/3935309308 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4014420 con 0x7f1cec14c9f0 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1ce2ffd640 1 -- 172.21.15.39:0/3935309308 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce40145a0 con 0x7f1cec14c9f0 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cec151b40 con 0x7f1cec14c9f0 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1ce2ffd640 1 -- 172.21.15.39:0/3935309308 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4014950 con 0x7f1cec14c9f0 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cec14fc40 con 0x7f1cec14c9f0 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cec0c1660 con 0x7f1cec14c9f0 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1ce2ffd640 1 -- 172.21.15.39:0/3935309308 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ce4014ab0 con 0x7f1cec14c9f0 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1ce2ffd640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cd0066690 0x7f1cd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1ce2ffd640 1 -- 172.21.15.39:0/3935309308 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f1ce4084dc0 con 0x7f1cec14c9f0 2024-09-13T16:48:37.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.959+0000 7f1cf23b8640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cd0066690 0x7f1cd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:37.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.963+0000 7f1cf23b8640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cd0066690 0x7f1cd0068b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80af000 tx=0x7f1ce8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:37.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:37.963+0000 7f1ce2ffd640 1 -- 172.21.15.39:0/3935309308 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1ce4051e00 con 0x7f1cec14c9f0 2024-09-13T16:48:38.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.123+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f1cec14c6a0 con 0x7f1cec14c9f0 2024-09-13T16:48:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.123+0000 7f1ce2ffd640 1 -- 172.21.15.39:0/3935309308 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7f1ce40860a0 con 0x7f1cec14c9f0 2024-09-13T16:48:38.130 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cd0066690 msgr2=0x7f1cd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cd0066690 0x7f1cd0068b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80af000 tx=0x7f1ce8008040 comp rx=0 tx=0).stop 2024-09-13T16:48:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14c9f0 msgr2=0x7f1cec14ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14c9f0 0x7f1cec14ce80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4007e60 tx=0x7f1ce400e4b0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 shutdown_connections 2024-09-13T16:48:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cd0066690 0x7f1cd0068b50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cec151070 0x7f1cec1503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cec14c9f0 0x7f1cec14ce80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 --2- 172.21.15.39:0/3935309308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1cec14e530 0x7f1cec152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 >> 172.21.15.39:0/3935309308 conn(0x7f1cec0c5790 msgr2=0x7f1cec0c0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:38.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 shutdown_connections 2024-09-13T16:48:38.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.127+0000 7f1cf461c640 1 -- 172.21.15.39:0/3935309308 wait complete. 2024-09-13T16:48:38.149 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T16:48:38.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.259+0000 7fb8df291640 1 Processor -- start 2024-09-13T16:48:38.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.259+0000 7fb8df291640 1 -- start start 2024-09-13T16:48:38.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8df291640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 0x7fb8d80c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8df291640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb8d8151990 0x7fb8d81540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8df291640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d8158070 con 0x7fb8d81515b0 2024-09-13T16:48:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8df291640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d80c17b0 con 0x7fb8d8151990 2024-09-13T16:48:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8df291640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d8158350 con 0x7fb8d80c0b80 2024-09-13T16:48:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dd82e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb8d8151990 0x7fb8d81540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dd02d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb8d81515b0 0x7fb8d80c21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53778/0 (socket says 172.21.15.39:53778) 2024-09-13T16:48:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dd02d640 1 -- 172.21.15.39:0/1809650224 learned_addr learned my addr 172.21.15.39:0/1809650224 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dc82c640 1 --2- 172.21.15.39:0/1809650224 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 0x7fb8d80c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dd82e640 1 -- 172.21.15.39:0/1809650224 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 msgr2=0x7fb8d80c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dd82e640 1 --2- 172.21.15.39:0/1809650224 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 0x7fb8d80c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dd82e640 1 -- 172.21.15.39:0/1809650224 >> v1:172.21.15.39:6789/0 conn(0x7fb8d81515b0 legacy=0x7fb8d80c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dd82e640 1 -- 172.21.15.39:0/1809650224 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d80c3690 con 0x7fb8d8151990 2024-09-13T16:48:38.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8cffff640 1 -- 172.21.15.39:0/1809650224 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809126744 0 0) 0x7fb8d8158070 con 0x7fb8d81515b0 2024-09-13T16:48:38.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dc82c640 1 --2- 172.21.15.39:0/1809650224 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 0x7fb8d80c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:38.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8dd82e640 1 --2- 172.21.15.39:0/1809650224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 0x7fb8d81540e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009d90 tx=0x7fb8d001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87daf1184ee502c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:38.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8cffff640 1 -- 172.21.15.39:0/1809650224 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8154e30 con 0x7fb8d8151990 2024-09-13T16:48:38.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8cffff640 1 -- 172.21.15.39:0/1809650224 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d00a58a0 con 0x7fb8d8151990 2024-09-13T16:48:38.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.263+0000 7fb8cffff640 1 -- 172.21.15.39:0/1809650224 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00a5c30 con 0x7fb8d8151990 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- 172.21.15.39:0/1809650224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 msgr2=0x7fb8d81540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 --2- 172.21.15.39:0/1809650224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 0x7fb8d81540e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009d90 tx=0x7fb8d001c820 comp rx=0 tx=0).stop 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- 172.21.15.39:0/1809650224 shutdown_connections 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 --2- 172.21.15.39:0/1809650224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 0x7fb8d81540e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 --2- 172.21.15.39:0/1809650224 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 0x7fb8d80c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- 172.21.15.39:0/1809650224 >> 172.21.15.39:0/1809650224 conn(0x7fb8d80c5790 msgr2=0x7fb8d80c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- 172.21.15.39:0/1809650224 shutdown_connections 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- 172.21.15.39:0/1809650224 wait complete. 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 Processor -- start 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- start start 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 0x7fb8d81ec920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8d81515b0 0x7fb8d80c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 0x7fb8d81ed2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d81ebe60 con 0x7fb8d8151990 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d81efe80 con 0x7fb8d81515b0 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d81f0160 con 0x7fb8d80c0b80 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd82e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 0x7fb8d81ed2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd82e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 0x7fb8d81ed2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36590/0 (socket says 172.21.15.39:36590) 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd82e640 1 -- 172.21.15.39:0/2645172052 learned_addr learned my addr 172.21.15.39:0/2645172052 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dc82c640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8d81515b0 0x7fb8d80c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd02d640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 0x7fb8d81ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd82e640 1 -- 172.21.15.39:0/2645172052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 msgr2=0x7fb8d81ec920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd82e640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 0x7fb8d81ec920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd82e640 1 -- 172.21.15.39:0/2645172052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8d81515b0 msgr2=0x7fb8d80c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd82e640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8d81515b0 0x7fb8d80c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd82e640 1 -- 172.21.15.39:0/2645172052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0009a70 con 0x7fb8d8151990 2024-09-13T16:48:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dc82c640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8d81515b0 0x7fb8d80c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8dd82e640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 0x7fb8d81ed2d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb8d001aed0 tx=0x7fb8d00195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8cdffb640 1 -- 172.21.15.39:0/2645172052 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00b9860 con 0x7fb8d8151990 2024-09-13T16:48:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8cdffb640 1 -- 172.21.15.39:0/2645172052 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d00b99c0 con 0x7fb8d8151990 2024-09-13T16:48:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d81ec120 con 0x7fb8d8151990 2024-09-13T16:48:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8cdffb640 1 -- 172.21.15.39:0/2645172052 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00b9d30 con 0x7fb8d8151990 2024-09-13T16:48:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.267+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d80c4670 con 0x7fb8d8151990 2024-09-13T16:48:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.271+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d80c51b0 con 0x7fb8d8151990 2024-09-13T16:48:38.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.271+0000 7fb8cdffb640 1 -- 172.21.15.39:0/2645172052 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb8d00ad440 con 0x7fb8d8151990 2024-09-13T16:48:38.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.271+0000 7fb8cdffb640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8bc066690 0x7fb8bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.271+0000 7fb8cdffb640 1 -- 172.21.15.39:0/2645172052 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb8d011f910 con 0x7fb8d8151990 2024-09-13T16:48:38.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.271+0000 7fb8dd02d640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8bc066690 0x7fb8bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.271+0000 7fb8dd02d640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8bc066690 0x7fb8bc068b50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4000f70 tx=0x7fb8d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:38.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.275+0000 7fb8cdffb640 1 -- 172.21.15.39:0/2645172052 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb8d00ec9e0 con 0x7fb8d8151990 2024-09-13T16:48:38.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8d81ec5d0 con 0x7fb8d8151990 2024-09-13T16:48:38.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8cdffb640 1 -- 172.21.15.39:0/2645172052 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7fb8d00f0000 con 0x7fb8d8151990 2024-09-13T16:48:38.440 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:38.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8bc066690 msgr2=0x7fb8bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8bc066690 0x7fb8bc068b50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4000f70 tx=0x7fb8d4009040 comp rx=0 tx=0).stop 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 msgr2=0x7fb8d81ed2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 0x7fb8d81ed2d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb8d001aed0 tx=0x7fb8d00195c0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 shutdown_connections 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8bc066690 0x7fb8bc068b50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8d81515b0 0x7fb8d80c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8d8151990 0x7fb8d81ed2d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 --2- 172.21.15.39:0/2645172052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8d80c0b80 0x7fb8d81ec920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 >> 172.21.15.39:0/2645172052 conn(0x7fb8d80c5790 msgr2=0x7fb8d815e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 shutdown_connections 2024-09-13T16:48:38.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.435+0000 7fb8df291640 1 -- 172.21.15.39:0/2645172052 wait complete. 2024-09-13T16:48:38.460 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T16:48:38.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.579+0000 7fc7306fb640 1 Processor -- start 2024-09-13T16:48:38.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.579+0000 7fc7306fb640 1 -- start start 2024-09-13T16:48:38.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7280a4d60 0x7fc72814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc728155d20 con 0x7fc72814f4b0 2024-09-13T16:48:38.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7280c4750 con 0x7fc7280aa2d0 2024-09-13T16:48:38.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc728155b30 con 0x7fc7280a4d60 2024-09-13T16:48:38.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72dc96640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7280a4d60 0x7fc72814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72dc96640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7280a4d60 0x7fc72814ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36604/0 (socket says 172.21.15.39:36604) 2024-09-13T16:48:38.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72dc96640 1 -- 172.21.15.39:0/3620513462 learned_addr learned my addr 172.21.15.39:0/3620513462 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:38.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72ec98640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc72814f4b0 0x7fc728151cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53790/0 (socket says 172.21.15.39:53790) 2024-09-13T16:48:38.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72dc96640 1 -- 172.21.15.39:0/3620513462 >> v1:172.21.15.84:6789/0 conn(0x7fc7280aa2d0 legacy=0x7fc7280aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:38.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72dc96640 1 -- 172.21.15.39:0/3620513462 >> v1:172.21.15.39:6789/0 conn(0x7fc72814f4b0 legacy=0x7fc728151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72dc96640 1 -- 172.21.15.39:0/3620513462 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728155fc0 con 0x7fc7280a4d60 2024-09-13T16:48:38.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72d495640 1 -- 172.21.15.39:0/3620513462 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405271537 0 0) 0x7fc728155d20 con 0x7fc72814f4b0 2024-09-13T16:48:38.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72dc96640 1 --2- 172.21.15.39:0/3620513462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280a4d60 0x7fc72814ef70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc718009870 tx=0x7fc71801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d683957358eb70 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:38.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72d495640 1 -- 172.21.15.39:0/3620513462 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180ab600 con 0x7fc7280a4d60 2024-09-13T16:48:38.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72d495640 1 -- 172.21.15.39:0/3620513462 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7180a7050 con 0x7fc7280a4d60 2024-09-13T16:48:38.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc72d495640 1 -- 172.21.15.39:0/3620513462 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180ab980 con 0x7fc7280a4d60 2024-09-13T16:48:38.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 -- 172.21.15.39:0/3620513462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280a4d60 msgr2=0x7fc72814ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 --2- 172.21.15.39:0/3620513462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280a4d60 0x7fc72814ef70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc718009870 tx=0x7fc71801c490 comp rx=0 tx=0).stop 2024-09-13T16:48:38.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 -- 172.21.15.39:0/3620513462 shutdown_connections 2024-09-13T16:48:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 --2- 172.21.15.39:0/3620513462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280a4d60 0x7fc72814ef70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 -- 172.21.15.39:0/3620513462 >> 172.21.15.39:0/3620513462 conn(0x7fc7280c5790 msgr2=0x7fc7280c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 -- 172.21.15.39:0/3620513462 shutdown_connections 2024-09-13T16:48:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.583+0000 7fc7306fb640 1 -- 172.21.15.39:0/3620513462 wait complete. 2024-09-13T16:48:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 Processor -- start 2024-09-13T16:48:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 -- start start 2024-09-13T16:48:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7280a4d60 0x7fc7280bee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7280aa2d0 0x7fc72806c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72814f4b0 0x7fc72806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280a5950 con 0x7fc72814f4b0 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280c2c40 con 0x7fc7280aa2d0 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7281425c0 con 0x7fc7280a4d60 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72ec98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72814f4b0 0x7fc72806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72ec98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72814f4b0 0x7fc72806c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36608/0 (socket says 172.21.15.39:36608) 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72ec98640 1 -- 172.21.15.39:0/1929946403 learned_addr learned my addr 172.21.15.39:0/1929946403 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72dc96640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7280aa2d0 0x7fc72806c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72e497640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7280a4d60 0x7fc7280bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72ec98640 1 -- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7280a4d60 msgr2=0x7fc7280bee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72ec98640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7280a4d60 0x7fc7280bee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72ec98640 1 -- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7280aa2d0 msgr2=0x7fc72806c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72ec98640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7280aa2d0 0x7fc72806c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72ec98640 1 -- 172.21.15.39:0/1929946403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7180094d0 con 0x7fc72814f4b0 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72e497640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7280a4d60 0x7fc7280bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72dc96640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7280aa2d0 0x7fc72806c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc72ec98640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72814f4b0 0x7fc72806c9b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc720001570 tx=0x7fc72000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc71effd640 1 -- 172.21.15.39:0/1929946403 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000a640 con 0x7fc72814f4b0 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc71effd640 1 -- 172.21.15.39:0/1929946403 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72000a7c0 con 0x7fc72814f4b0 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7281f05e0 con 0x7fc72814f4b0 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc71effd640 1 -- 172.21.15.39:0/1929946403 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000ab70 con 0x7fc72814f4b0 2024-09-13T16:48:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7280a53b0 con 0x7fc72814f4b0 2024-09-13T16:48:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72814ef70 con 0x7fc72814f4b0 2024-09-13T16:48:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.587+0000 7fc71effd640 1 -- 172.21.15.39:0/1929946403 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc72000acd0 con 0x7fc72814f4b0 2024-09-13T16:48:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.591+0000 7fc71effd640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc70c066690 0x7fc70c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.591+0000 7fc71effd640 1 -- 172.21.15.39:0/1929946403 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fc720083a00 con 0x7fc72814f4b0 2024-09-13T16:48:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.591+0000 7fc72e497640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc70c066690 0x7fc70c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.591+0000 7fc72e497640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc70c066690 0x7fc70c068b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc724003e40 tx=0x7fc724009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:38.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.595+0000 7fc71effd640 1 -- 172.21.15.39:0/1929946403 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc720050ad0 con 0x7fc72814f4b0 2024-09-13T16:48:38.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.751+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fc7280c5b70 con 0x7fc72814f4b0 2024-09-13T16:48:38.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.751+0000 7fc71effd640 1 -- 172.21.15.39:0/1929946403 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v24) v1 ==== 95+0+11 (secure 0 0 0) 0x7fc720018070 con 0x7fc72814f4b0 2024-09-13T16:48:38.758 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:38.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc70c066690 msgr2=0x7fc70c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc70c066690 0x7fc70c068b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc724003e40 tx=0x7fc724009040 comp rx=0 tx=0).stop 2024-09-13T16:48:38.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72814f4b0 msgr2=0x7fc72806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72814f4b0 0x7fc72806c9b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc720001570 tx=0x7fc72000cca0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 shutdown_connections 2024-09-13T16:48:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc70c066690 0x7fc70c068b50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7280aa2d0 0x7fc72806c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72814f4b0 0x7fc72806c9b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 --2- 172.21.15.39:0/1929946403 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7280a4d60 0x7fc7280bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 >> 172.21.15.39:0/1929946403 conn(0x7fc7280c5790 msgr2=0x7fc72815cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 shutdown_connections 2024-09-13T16:48:38.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.755+0000 7fc7306fb640 1 -- 172.21.15.39:0/1929946403 wait complete. 2024-09-13T16:48:38.776 INFO:tasks.mds_thrash:mds thrasher using random seed: 1726246118 2024-09-13T16:48:38.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1fcc1a640 1 Processor -- start 2024-09-13T16:48:38.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1fcc1a640 1 -- start start 2024-09-13T16:48:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1fcc1a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb1f806d020 0x7fb1f80a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1fcc1a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1f80c4530 0x7fb1f80c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1fcc1a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f81518f0 con 0x7fb1f8151460 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1fcc1a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f80a8c50 con 0x7fb1f80c4530 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1fcc1a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f8154100 con 0x7fb1f806d020 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1f6d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1f80c4530 0x7fb1f80c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1f6d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1f80c4530 0x7fb1f80c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36618/0 (socket says 172.21.15.39:36618) 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1f6d9d640 1 -- 172.21.15.39:0/2021893355 learned_addr learned my addr 172.21.15.39:0/2021893355 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1f659c640 1 --2- 172.21.15.39:0/2021893355 >> v2:172.21.15.84:3300/0 conn(0x7fb1f806d020 0x7fb1f80a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1f6d9d640 1 -- 172.21.15.39:0/2021893355 >> v2:172.21.15.84:3300/0 conn(0x7fb1f806d020 msgr2=0x7fb1f80a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1f6d9d640 1 --2- 172.21.15.39:0/2021893355 >> v2:172.21.15.84:3300/0 conn(0x7fb1f806d020 0x7fb1f80a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1f6d9d640 1 -- 172.21.15.39:0/2021893355 >> v1:172.21.15.84:6789/0 conn(0x7fb1f8151460 legacy=0x7fb1f814db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:38.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.883+0000 7fb1f6d9d640 1 -- 172.21.15.39:0/2021893355 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f81543e0 con 0x7fb1f80c4530 2024-09-13T16:48:38.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 --2- 172.21.15.39:0/2021893355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f80c4530 0x7fb1f80c4940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb1f806d820 tx=0x7fb1e801eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3d7d147ec90db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:38.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f559a640 1 -- 172.21.15.39:0/2021893355 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e80a79d0 con 0x7fb1f80c4530 2024-09-13T16:48:38.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f559a640 1 -- 172.21.15.39:0/2021893355 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e80a7b30 con 0x7fb1f80c4530 2024-09-13T16:48:38.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f559a640 1 -- 172.21.15.39:0/2021893355 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e80af440 con 0x7fb1f80c4530 2024-09-13T16:48:38.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2021893355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f80c4530 msgr2=0x7fb1f80c4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 --2- 172.21.15.39:0/2021893355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f80c4530 0x7fb1f80c4940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb1f806d820 tx=0x7fb1e801eaf0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2021893355 shutdown_connections 2024-09-13T16:48:38.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 --2- 172.21.15.39:0/2021893355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f80c4530 0x7fb1f80c4940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 --2- 172.21.15.39:0/2021893355 >> v2:172.21.15.84:3300/0 conn(0x7fb1f806d020 0x7fb1f80a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2021893355 >> 172.21.15.39:0/2021893355 conn(0x7fb1f8069e10 msgr2=0x7fb1f806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:38.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2021893355 shutdown_connections 2024-09-13T16:48:38.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2021893355 wait complete. 2024-09-13T16:48:38.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 Processor -- start 2024-09-13T16:48:38.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- start start 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f806d020 0x7fb1f80a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f80c4530 0x7fb1f80bee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f8151460 0x7fb1f81efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f8150da0 con 0x7fb1f8151460 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80a5960 con 0x7fb1f80c4530 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f8070920 con 0x7fb1f806d020 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f8151460 0x7fb1f81efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f8151460 0x7fb1f81efcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36634/0 (socket says 172.21.15.39:36634) 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 -- 172.21.15.39:0/2562305279 learned_addr learned my addr 172.21.15.39:0/2562305279 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:38.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f659c640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f806d020 0x7fb1f80a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 -- 172.21.15.39:0/2562305279 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f806d020 msgr2=0x7fb1f80a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f5d9b640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f80c4530 0x7fb1f80bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f806d020 0x7fb1f80a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 -- 172.21.15.39:0/2562305279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f80c4530 msgr2=0x7fb1f80bee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:38.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f80c4530 0x7fb1f80bee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:38.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 -- 172.21.15.39:0/2562305279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e800b770 con 0x7fb1f8151460 2024-09-13T16:48:38.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1f6d9d640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f8151460 0x7fb1f81efcd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800b2c0 tx=0x7fb1e801b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:38.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1eeffd640 1 -- 172.21.15.39:0/2562305279 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e80afe20 con 0x7fb1f8151460 2024-09-13T16:48:38.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1eeffd640 1 -- 172.21.15.39:0/2562305279 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e80ba440 con 0x7fb1f8151460 2024-09-13T16:48:38.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f8070c00 con 0x7fb1f8151460 2024-09-13T16:48:38.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1eeffd640 1 -- 172.21.15.39:0/2562305279 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e80ba7d0 con 0x7fb1f8151460 2024-09-13T16:48:38.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.887+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f8070620 con 0x7fb1f8151460 2024-09-13T16:48:38.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.891+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f80a4e70 con 0x7fb1f8151460 2024-09-13T16:48:38.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.891+0000 7fb1eeffd640 1 -- 172.21.15.39:0/2562305279 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb1e80b93e0 con 0x7fb1f8151460 2024-09-13T16:48:38.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.891+0000 7fb1eeffd640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1d8066690 0x7fb1d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:38.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.891+0000 7fb1eeffd640 1 -- 172.21.15.39:0/2562305279 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fb1e811ef60 con 0x7fb1f8151460 2024-09-13T16:48:38.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.891+0000 7fb1f659c640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1d8066690 0x7fb1d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:38.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.891+0000 7fb1f659c640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1d8066690 0x7fb1d8068b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0003fd0 tx=0x7fb1f00039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:38.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:38.895+0000 7fb1eeffd640 1 -- 172.21.15.39:0/2562305279 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb1e80ec030 con 0x7fb1f8151460 2024-09-13T16:48:39.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.051+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1f80a92b0 con 0x7fb1f8151460 2024-09-13T16:48:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1eeffd640 1 -- 172.21.15.39:0/2562305279 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+20129 (secure 0 0 0) 0x7fb1e80bf090 con 0x7fb1f8151460 2024-09-13T16:48:39.060 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:39.060 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:09.138452+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1d8066690 msgr2=0x7fb1d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1d8066690 0x7fb1d8068b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0003fd0 tx=0x7fb1f00039a0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f8151460 msgr2=0x7fb1f81efcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f8151460 0x7fb1f81efcd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800b2c0 tx=0x7fb1e801b890 comp rx=0 tx=0).stop 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 shutdown_connections 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1d8066690 0x7fb1d8068b50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f80c4530 0x7fb1f80bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f8151460 0x7fb1f81efcd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 --2- 172.21.15.39:0/2562305279 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f806d020 0x7fb1f80a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 >> 172.21.15.39:0/2562305279 conn(0x7fb1f8069e10 msgr2=0x7fb1f806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 shutdown_connections 2024-09-13T16:48:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.055+0000 7fb1fcc1a640 1 -- 172.21.15.39:0/2562305279 wait complete. 2024-09-13T16:48:39.078 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:48:39.078 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:48:39.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.195+0000 7fd945dc3640 1 Processor -- start 2024-09-13T16:48:39.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd945dc3640 1 -- start start 2024-09-13T16:48:39.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd945dc3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9400a4d60 0x7fd94014ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd945dc3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd94014f4b0 0x7fd940151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd945dc3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd940155d20 con 0x7fd9400aa2d0 2024-09-13T16:48:39.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd945dc3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd940155b30 con 0x7fd94014f4b0 2024-09-13T16:48:39.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd945dc3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400aadc0 con 0x7fd9400a4d60 2024-09-13T16:48:39.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd9400aa2d0 0x7fd9400aa6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53798/0 (socket says 172.21.15.39:53798) 2024-09-13T16:48:39.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93f7fe640 1 -- 172.21.15.39:0/4178607833 learned_addr learned my addr 172.21.15.39:0/4178607833 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:39.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93effd640 1 --2- 172.21.15.39:0/4178607833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9400a4d60 0x7fd94014ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93ffff640 1 --2- 172.21.15.39:0/4178607833 >> v2:172.21.15.84:3300/0 conn(0x7fd94014f4b0 0x7fd940151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93e7fc640 1 -- 172.21.15.39:0/4178607833 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633575787 0 0) 0x7fd940155d20 con 0x7fd9400aa2d0 2024-09-13T16:48:39.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93e7fc640 1 -- 172.21.15.39:0/4178607833 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd928003530 con 0x7fd9400aa2d0 2024-09-13T16:48:39.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93effd640 1 -- 172.21.15.39:0/4178607833 >> v2:172.21.15.84:3300/0 conn(0x7fd94014f4b0 msgr2=0x7fd940151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:39.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93e7fc640 1 -- 172.21.15.39:0/4178607833 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1073310855 0 0) 0x7fd928003530 con 0x7fd9400aa2d0 2024-09-13T16:48:39.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93effd640 1 --2- 172.21.15.39:0/4178607833 >> v2:172.21.15.84:3300/0 conn(0x7fd94014f4b0 0x7fd940151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93effd640 1 -- 172.21.15.39:0/4178607833 >> v1:172.21.15.39:6789/0 conn(0x7fd9400aa2d0 legacy=0x7fd9400aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93effd640 1 -- 172.21.15.39:0/4178607833 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9400c4750 con 0x7fd9400a4d60 2024-09-13T16:48:39.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.199+0000 7fd93ffff640 1 --2- 172.21.15.39:0/4178607833 >> v2:172.21.15.84:3300/0 conn(0x7fd94014f4b0 0x7fd940151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:48:39.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 --2- 172.21.15.39:0/4178607833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9400a4d60 0x7fd94014ef70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd9340097c0 tx=0x7fd93401c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a276a001aa2db8c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:39.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93e7fc640 1 -- 172.21.15.39:0/4178607833 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940152a40 con 0x7fd9400a4d60 2024-09-13T16:48:39.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93e7fc640 1 -- 172.21.15.39:0/4178607833 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9340a7050 con 0x7fd9400a4d60 2024-09-13T16:48:39.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93e7fc640 1 -- 172.21.15.39:0/4178607833 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930001680 con 0x7fd9400a4d60 2024-09-13T16:48:39.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 -- 172.21.15.39:0/4178607833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9400a4d60 msgr2=0x7fd94014ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 --2- 172.21.15.39:0/4178607833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9400a4d60 0x7fd94014ef70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd9340097c0 tx=0x7fd93401c300 comp rx=0 tx=0).stop 2024-09-13T16:48:39.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 -- 172.21.15.39:0/4178607833 shutdown_connections 2024-09-13T16:48:39.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 --2- 172.21.15.39:0/4178607833 >> v2:172.21.15.84:3300/0 conn(0x7fd94014f4b0 0x7fd940151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 --2- 172.21.15.39:0/4178607833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9400a4d60 0x7fd94014ef70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 -- 172.21.15.39:0/4178607833 >> 172.21.15.39:0/4178607833 conn(0x7fd9400c5790 msgr2=0x7fd9400c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:39.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 -- 172.21.15.39:0/4178607833 shutdown_connections 2024-09-13T16:48:39.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 -- 172.21.15.39:0/4178607833 wait complete. 2024-09-13T16:48:39.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 Processor -- start 2024-09-13T16:48:39.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 -- start start 2024-09-13T16:48:39.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9400a4d60 0x7fd940150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9400aa2d0 0x7fd940141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd94014f4b0 0x7fd940141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400a5210 con 0x7fd9400aa2d0 2024-09-13T16:48:39.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd940142380 con 0x7fd9400a4d60 2024-09-13T16:48:39.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd945dc3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400a5460 con 0x7fd94014f4b0 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9400aa2d0 0x7fd940141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9400aa2d0 0x7fd940141900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36648/0 (socket says 172.21.15.39:36648) 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 -- 172.21.15.39:0/1436658088 learned_addr learned my addr 172.21.15.39:0/1436658088 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93f7fe640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9400a4d60 0x7fd940150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93ffff640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd94014f4b0 0x7fd940141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 -- 172.21.15.39:0/1436658088 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd94014f4b0 msgr2=0x7fd940141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd94014f4b0 0x7fd940141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 -- 172.21.15.39:0/1436658088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9400a4d60 msgr2=0x7fd940150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9400a4d60 0x7fd940150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 -- 172.21.15.39:0/1436658088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd938009a70 con 0x7fd9400aa2d0 2024-09-13T16:48:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93f7fe640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9400a4d60 0x7fd940150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.203+0000 7fd93effd640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9400aa2d0 0x7fd940141900 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd93401a9b0 tx=0x7fd9340190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd927fff640 1 -- 172.21.15.39:0/1436658088 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934003ab0 con 0x7fd9400aa2d0 2024-09-13T16:48:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd927fff640 1 -- 172.21.15.39:0/1436658088 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd934003c10 con 0x7fd9400aa2d0 2024-09-13T16:48:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd934009420 con 0x7fd9400aa2d0 2024-09-13T16:48:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd927fff640 1 -- 172.21.15.39:0/1436658088 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9340b5410 con 0x7fd9400aa2d0 2024-09-13T16:48:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9400a5a20 con 0x7fd9400aa2d0 2024-09-13T16:48:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9401e9b30 con 0x7fd9400aa2d0 2024-09-13T16:48:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd927fff640 1 -- 172.21.15.39:0/1436658088 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9340b2050 con 0x7fd9400aa2d0 2024-09-13T16:48:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd927fff640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd92c066690 0x7fd92c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd927fff640 1 -- 172.21.15.39:0/1436658088 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7fd93411e7f0 con 0x7fd9400aa2d0 2024-09-13T16:48:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd93f7fe640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd92c066690 0x7fd92c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.207+0000 7fd93f7fe640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd92c066690 0x7fd92c068b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9300079c0 tx=0x7fd930009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:39.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.211+0000 7fd927fff640 1 -- 172.21.15.39:0/1436658088 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd9401e9b30 con 0x7fd9400aa2d0 2024-09-13T16:48:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9401e9b30 con 0x7fd9400aa2d0 2024-09-13T16:48:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd927fff640 1 -- 172.21.15.39:0/1436658088 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9401e9b30 con 0x7fd9400aa2d0 2024-09-13T16:48:39.376 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:48:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd92c066690 msgr2=0x7fd92c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd92c066690 0x7fd92c068b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9300079c0 tx=0x7fd930009040 comp rx=0 tx=0).stop 2024-09-13T16:48:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9400aa2d0 msgr2=0x7fd940141900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9400aa2d0 0x7fd940141900 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd93401a9b0 tx=0x7fd9340190a0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 shutdown_connections 2024-09-13T16:48:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd92c066690 0x7fd92c068b50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd94014f4b0 0x7fd940141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9400aa2d0 0x7fd940141900 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 --2- 172.21.15.39:0/1436658088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9400a4d60 0x7fd940150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 >> 172.21.15.39:0/1436658088 conn(0x7fd9400c5790 msgr2=0x7fd9400aa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:39.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 shutdown_connections 2024-09-13T16:48:39.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.371+0000 7fd945dc3640 1 -- 172.21.15.39:0/1436658088 wait complete. 2024-09-13T16:48:39.396 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T16:48:39.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.519+0000 7f20362f5640 1 Processor -- start 2024-09-13T16:48:39.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.519+0000 7f20362f5640 1 -- start start 2024-09-13T16:48:39.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f20362f5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f20300c2850 0x7f2030153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f20362f5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20300c6500 con 0x7f20300c2470 2024-09-13T16:48:39.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f20362f5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20300c4d60 con 0x7f20300c0c90 2024-09-13T16:48:39.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f20362f5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20300c3dc0 con 0x7f20300c2850 2024-09-13T16:48:39.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f202f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f20300c2850 0x7f2030153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f202f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f20300c2850 0x7f2030153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36650/0 (socket says 172.21.15.39:36650) 2024-09-13T16:48:39.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f202ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f20300c2470 0x7f2030065c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53810/0 (socket says 172.21.15.39:53810) 2024-09-13T16:48:39.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f202f7fe640 1 -- 172.21.15.39:0/1193004719 learned_addr learned my addr 172.21.15.39:0/1193004719 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:39.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f202f7fe640 1 -- 172.21.15.39:0/1193004719 >> v1:172.21.15.84:6789/0 conn(0x7f20300c0c90 legacy=0x7f2030154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:39.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f202f7fe640 1 -- 172.21.15.39:0/1193004719 >> v1:172.21.15.39:6789/0 conn(0x7f20300c2470 legacy=0x7f2030065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f202effd640 1 -- 172.21.15.39:0/1193004719 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094321775 0 0) 0x7f20300c6500 con 0x7f20300c2470 2024-09-13T16:48:39.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.523+0000 7f202f7fe640 1 -- 172.21.15.39:0/1193004719 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20300c40a0 con 0x7f20300c2850 2024-09-13T16:48:39.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f202f7fe640 1 --2- 172.21.15.39:0/1193004719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20300c2850 0x7f2030153dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2018009870 tx=0x7f201801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb50fa3d159e0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:39.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f202effd640 1 -- 172.21.15.39:0/1193004719 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20180ab660 con 0x7f20300c2850 2024-09-13T16:48:39.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f202effd640 1 -- 172.21.15.39:0/1193004719 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20180ab7c0 con 0x7f20300c2850 2024-09-13T16:48:39.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f202effd640 1 -- 172.21.15.39:0/1193004719 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20180abb50 con 0x7f20300c2850 2024-09-13T16:48:39.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 -- 172.21.15.39:0/1193004719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20300c2850 msgr2=0x7f2030153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 --2- 172.21.15.39:0/1193004719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20300c2850 0x7f2030153dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2018009870 tx=0x7f201801c460 comp rx=0 tx=0).stop 2024-09-13T16:48:39.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 -- 172.21.15.39:0/1193004719 shutdown_connections 2024-09-13T16:48:39.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 --2- 172.21.15.39:0/1193004719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20300c2850 0x7f2030153dd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 -- 172.21.15.39:0/1193004719 >> 172.21.15.39:0/1193004719 conn(0x7f20300c32a0 msgr2=0x7f20300c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:39.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 -- 172.21.15.39:0/1193004719 shutdown_connections 2024-09-13T16:48:39.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 -- 172.21.15.39:0/1193004719 wait complete. 2024-09-13T16:48:39.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 Processor -- start 2024-09-13T16:48:39.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 -- start start 2024-09-13T16:48:39.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20300c0c90 0x7f20301496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20300c2470 0x7f2030070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20301f0080 0x7f20300713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20301f05d0 con 0x7f20301f0080 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20301ec100 con 0x7f20300c2470 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f20362f5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20301ed0d0 con 0x7f20300c0c90 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f2034892640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20301f0080 0x7f20300713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f2034892640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20301f0080 0x7f20300713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36658/0 (socket says 172.21.15.39:36658) 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.527+0000 7f2034892640 1 -- 172.21.15.39:0/3926768324 learned_addr learned my addr 172.21.15.39:0/3926768324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202ffff640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20300c0c90 0x7f20301496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202f7fe640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20300c2470 0x7f2030070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f2034892640 1 -- 172.21.15.39:0/3926768324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20300c0c90 msgr2=0x7f20301496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f2034892640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20300c0c90 0x7f20301496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f2034892640 1 -- 172.21.15.39:0/3926768324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20300c2470 msgr2=0x7f2030070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f2034892640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20300c2470 0x7f2030070e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f2034892640 1 -- 172.21.15.39:0/3926768324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180094d0 con 0x7f20301f0080 2024-09-13T16:48:39.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f2034892640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20301f0080 0x7f20300713b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f202800c3c0 tx=0x7f202800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:39.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202cff9640 1 -- 172.21.15.39:0/3926768324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028011790 con 0x7f20301f0080 2024-09-13T16:48:39.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202cff9640 1 -- 172.21.15.39:0/3926768324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20280118f0 con 0x7f20301f0080 2024-09-13T16:48:39.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202cff9640 1 -- 172.21.15.39:0/3926768324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028011ca0 con 0x7f20301f0080 2024-09-13T16:48:39.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20301ec310 con 0x7f20301f0080 2024-09-13T16:48:39.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20301ec8d0 con 0x7f20301f0080 2024-09-13T16:48:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2030065c80 con 0x7f20301f0080 2024-09-13T16:48:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202cff9640 1 -- 172.21.15.39:0/3926768324 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f202801c460 con 0x7f20301f0080 2024-09-13T16:48:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202cff9640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2010066690 0x7f2010068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202ffff640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2010066690 0x7f2010068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202cff9640 1 -- 172.21.15.39:0/3926768324 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6521+0+0 (secure 0 0 0) 0x7f20280842f0 con 0x7f20301f0080 2024-09-13T16:48:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.531+0000 7f202ffff640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2010066690 0x7f2010068b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2020007fd0 tx=0x7f2020007f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.535+0000 7f202cff9640 1 -- 172.21.15.39:0/3926768324 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f20280513a0 con 0x7f20301f0080 2024-09-13T16:48:39.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.695+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20300718f0 con 0x7f20301f0080 2024-09-13T16:48:39.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f202cff9640 1 -- 172.21.15.39:0/3926768324 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v24) v1 ==== 99+0+11 (secure 0 0 0) 0x7f20280549c0 con 0x7f20301f0080 2024-09-13T16:48:39.703 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:39.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2010066690 msgr2=0x7f2010068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2010066690 0x7f2010068b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2020007fd0 tx=0x7f2020007f00 comp rx=0 tx=0).stop 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20301f0080 msgr2=0x7f20300713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20301f0080 0x7f20300713b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f202800c3c0 tx=0x7f202800c880 comp rx=0 tx=0).stop 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 shutdown_connections 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2010066690 0x7f2010068b50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20300c2470 0x7f2030070e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20301f0080 0x7f20300713b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 --2- 172.21.15.39:0/3926768324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20300c0c90 0x7f20301496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 >> 172.21.15.39:0/3926768324 conn(0x7f20300c32a0 msgr2=0x7f20300c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 shutdown_connections 2024-09-13T16:48:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.699+0000 7f20362f5640 1 -- 172.21.15.39:0/3926768324 wait complete. 2024-09-13T16:48:39.723 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T16:48:39.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f493e31b640 1 Processor -- start 2024-09-13T16:48:39.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f493e31b640 1 -- start start 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f493e31b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f49381540e0 0x7f49380c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f493e31b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 0x7f4938151070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f493e31b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49380c1d70 con 0x7f493814e530 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f493e31b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4938150b70 con 0x7f49381540e0 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f493e31b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4938154570 con 0x7f49380c4ce0 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f49377fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f49381540e0 0x7f49380c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f4937fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f493814e530 0x7f49380c1660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53812/0 (socket says 172.21.15.39:53812) 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f4937fff640 1 -- 172.21.15.39:0/2959952687 learned_addr learned my addr 172.21.15.39:0/2959952687 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f493c8b8640 1 --2- 172.21.15.39:0/2959952687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 0x7f4938151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f49377fe640 1 -- 172.21.15.39:0/2959952687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 msgr2=0x7f4938151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f49377fe640 1 --2- 172.21.15.39:0/2959952687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 0x7f4938151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f49377fe640 1 -- 172.21.15.39:0/2959952687 >> v1:172.21.15.39:6789/0 conn(0x7f493814e530 legacy=0x7f49380c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.851+0000 7f49377fe640 1 -- 172.21.15.39:0/2959952687 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938154850 con 0x7f49381540e0 2024-09-13T16:48:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f49377fe640 1 --2- 172.21.15.39:0/2959952687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381540e0 0x7f49380c47a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f492c00ba60 tx=0x7f492c01eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6525e501a159f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f4936ffd640 1 -- 172.21.15.39:0/2959952687 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c0a7a80 con 0x7f49381540e0 2024-09-13T16:48:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f4936ffd640 1 -- 172.21.15.39:0/2959952687 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c0a7be0 con 0x7f49381540e0 2024-09-13T16:48:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f4936ffd640 1 -- 172.21.15.39:0/2959952687 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c0af440 con 0x7f49381540e0 2024-09-13T16:48:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- 172.21.15.39:0/2959952687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381540e0 msgr2=0x7f49380c47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 --2- 172.21.15.39:0/2959952687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381540e0 0x7f49380c47a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f492c00ba60 tx=0x7f492c01eab0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- 172.21.15.39:0/2959952687 shutdown_connections 2024-09-13T16:48:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 --2- 172.21.15.39:0/2959952687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 0x7f4938151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 --2- 172.21.15.39:0/2959952687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381540e0 0x7f49380c47a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- 172.21.15.39:0/2959952687 >> 172.21.15.39:0/2959952687 conn(0x7f49380c5790 msgr2=0x7f49380c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- 172.21.15.39:0/2959952687 shutdown_connections 2024-09-13T16:48:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- 172.21.15.39:0/2959952687 wait complete. 2024-09-13T16:48:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 Processor -- start 2024-09-13T16:48:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- start start 2024-09-13T16:48:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 0x7f49381ed220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f493814e530 0x7f49381efd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4938071e00 0x7f493806bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4938152690 con 0x7f4938071e00 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4938153740 con 0x7f493814e530 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49381ec810 con 0x7f49380c4ce0 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493c8b8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4938071e00 0x7f493806bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493c8b8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4938071e00 0x7f493806bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36676/0 (socket says 172.21.15.39:36676) 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493c8b8640 1 -- 172.21.15.39:0/2596431013 learned_addr learned my addr 172.21.15.39:0/2596431013 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f4937fff640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 0x7f49381ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f49377fe640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f493814e530 0x7f49381efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493c8b8640 1 -- 172.21.15.39:0/2596431013 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 msgr2=0x7f49381ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493c8b8640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 0x7f49381ed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493c8b8640 1 -- 172.21.15.39:0/2596431013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f493814e530 msgr2=0x7f49381efd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493c8b8640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f493814e530 0x7f49381efd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493c8b8640 1 -- 172.21.15.39:0/2596431013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c00b6c0 con 0x7f4938071e00 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f4937fff640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 0x7f49381ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493c8b8640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4938071e00 0x7f493806bac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4930007e40 tx=0x7f493000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f4934ff9640 1 -- 172.21.15.39:0/2596431013 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930014620 con 0x7f4938071e00 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f4934ff9640 1 -- 172.21.15.39:0/2596431013 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493000ee90 con 0x7f4938071e00 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4938151b40 con 0x7f4938071e00 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f4934ff9640 1 -- 172.21.15.39:0/2596431013 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49300149e0 con 0x7f4938071e00 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.855+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49381f02c0 con 0x7f4938071e00 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.859+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49380c42e0 con 0x7f4938071e00 2024-09-13T16:48:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.859+0000 7f4934ff9640 1 -- 172.21.15.39:0/2596431013 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4930014b40 con 0x7f4938071e00 2024-09-13T16:48:39.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.859+0000 7f4934ff9640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4924066690 0x7f4924068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:39.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.859+0000 7f4934ff9640 1 -- 172.21.15.39:0/2596431013 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f4930084e90 con 0x7f4938071e00 2024-09-13T16:48:39.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.859+0000 7f4937fff640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4924066690 0x7f4924068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:39.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.859+0000 7f4937fff640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4924066690 0x7f4924068b50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4928005890 tx=0x7f4928009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:39.863+0000 7f4934ff9640 1 -- 172.21.15.39:0/2596431013 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4930051ef0 con 0x7f4938071e00 2024-09-13T16:48:40.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.015+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f49380bfb30 con 0x7f4938071e00 2024-09-13T16:48:40.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.015+0000 7f4934ff9640 1 -- 172.21.15.39:0/2596431013 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v25) v1 ==== 95+0+11 (secure 0 0 0) 0x7f4930055510 con 0x7f4938071e00 2024-09-13T16:48:40.022 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:40.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4924066690 msgr2=0x7f4924068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4924066690 0x7f4924068b50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4928005890 tx=0x7f4928009040 comp rx=0 tx=0).stop 2024-09-13T16:48:40.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4938071e00 msgr2=0x7f493806bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4938071e00 0x7f493806bac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4930007e40 tx=0x7f493000e4b0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 shutdown_connections 2024-09-13T16:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4924066690 0x7f4924068b50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f493814e530 0x7f49381efd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4938071e00 0x7f493806bac0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 --2- 172.21.15.39:0/2596431013 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49380c4ce0 0x7f49381ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 >> 172.21.15.39:0/2596431013 conn(0x7f49380c5790 msgr2=0x7f4938147e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 shutdown_connections 2024-09-13T16:48:40.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.019+0000 7f493e31b640 1 -- 172.21.15.39:0/2596431013 wait complete. 2024-09-13T16:48:40.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T16:48:40.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc174855640 1 Processor -- start 2024-09-13T16:48:40.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc174855640 1 -- start start 2024-09-13T16:48:40.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc174855640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc17014e200 0x7fc17006ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc174855640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc17006b360 0x7fc170154920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc174855640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17014e5e0 con 0x7fc1700c46a0 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc174855640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc170154e60 con 0x7fc17006b360 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc174855640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701583d0 con 0x7fc17014e200 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16dd9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc1700c46a0 0x7fc170154210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34882/0 (socket says 172.21.15.39:34882) 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16dd9b640 1 -- 172.21.15.39:0/2568621429 learned_addr learned my addr 172.21.15.39:0/2568621429 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16e59c640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc17014e200 0x7fc17006ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16ed9d640 1 --2- 172.21.15.39:0/2568621429 >> v2:172.21.15.84:3300/0 conn(0x7fc17006b360 0x7fc170154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943236627 0 0) 0x7fc17014e5e0 con 0x7fc1700c46a0 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc15c003530 con 0x7fc1700c46a0 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792983080 0 0) 0x7fc15c003530 con 0x7fc1700c46a0 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc17014e200 msgr2=0x7fc17006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc17014e200 0x7fc17006ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 >> v2:172.21.15.84:3300/0 conn(0x7fc17006b360 msgr2=0x7fc170154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 --2- 172.21.15.39:0/2568621429 >> v2:172.21.15.84:3300/0 conn(0x7fc17006b360 0x7fc170154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc170157850 con 0x7fc1700c46a0 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fc160001680 con 0x7fc1700c46a0 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 >> v1:172.21.15.39:6789/0 conn(0x7fc1700c46a0 legacy=0x7fc170154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc15c003980 0x7fc15c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc17014e200 0x7fc15c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc15c004d10 0x7fc15c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170154e60 con 0x7fc15c003980 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701583d0 con 0x7fc15c004d10 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c00b990 con 0x7fc17014e200 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16e59c640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc15c003980 0x7fc15c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16ed9d640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc15c004d10 0x7fc15c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16e59c640 1 -- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc17014e200 msgr2=0x7fc15c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16e59c640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc17014e200 0x7fc15c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16e59c640 1 -- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc15c004d10 msgr2=0x7fc15c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:40.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16e59c640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc15c004d10 0x7fc15c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.159+0000 7fc16e59c640 1 -- 172.21.15.39:0/2568621429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c003530 con 0x7fc15c003980 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc16e59c640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc15c003980 0x7fc15c003da0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc158009ae0 tx=0x7fc15801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b231e7880bea4949 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580a5650 con 0x7fc15c003980 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1580a57b0 con 0x7fc15c003980 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc16d59a640 1 -- 172.21.15.39:0/2568621429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1580a5b60 con 0x7fc15c003980 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc174855640 1 -- 172.21.15.39:0/2568621429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc15c003980 msgr2=0x7fc15c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc174855640 1 --2- 172.21.15.39:0/2568621429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc15c003980 0x7fc15c003da0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc158009ae0 tx=0x7fc15801ca80 comp rx=0 tx=0).stop 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc16ed9d640 1 -- 172.21.15.39:0/2568621429 reap_dead start 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc174855640 1 -- 172.21.15.39:0/2568621429 shutdown_connections 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc174855640 1 -- 172.21.15.39:0/2568621429 >> 172.21.15.39:0/2568621429 conn(0x7fc1700c32a0 msgr2=0x7fc1700c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:40.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc174855640 1 -- 172.21.15.39:0/2568621429 shutdown_connections 2024-09-13T16:48:40.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.163+0000 7fc174855640 1 -- 172.21.15.39:0/2568621429 wait complete. 2024-09-13T16:48:40.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 Processor -- start 2024-09-13T16:48:40.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 -- start start 2024-09-13T16:48:40.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc15c003980 0x7fc17015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17015d460 0x7fc1700a7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1700a7a80 0x7fc170070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700720a0 con 0x7fc1700a7a80 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17014d480 con 0x7fc17015d460 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701486b0 con 0x7fc15c003980 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1700a7a80 0x7fc170070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1700a7a80 0x7fc170070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57096/0 (socket says 172.21.15.39:57096) 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16ed9d640 1 -- 172.21.15.39:0/488095216 learned_addr learned my addr 172.21.15.39:0/488095216 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16dd9b640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17015d460 0x7fc1700a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16e59c640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc15c003980 0x7fc17015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16ed9d640 1 -- 172.21.15.39:0/488095216 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc15c003980 msgr2=0x7fc17015cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16ed9d640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc15c003980 0x7fc17015cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16ed9d640 1 -- 172.21.15.39:0/488095216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17015d460 msgr2=0x7fc1700a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16ed9d640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17015d460 0x7fc1700a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16ed9d640 1 -- 172.21.15.39:0/488095216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17014c800 con 0x7fc1700a7a80 2024-09-13T16:48:40.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc16ed9d640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1700a7a80 0x7fc170070440 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc1700a9610 tx=0x7fc1680115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:40.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc166ffd640 1 -- 172.21.15.39:0/488095216 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16800e070 con 0x7fc1700a7a80 2024-09-13T16:48:40.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc166ffd640 1 -- 172.21.15.39:0/488095216 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc168011e10 con 0x7fc1700a7a80 2024-09-13T16:48:40.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17014cae0 con 0x7fc1700a7a80 2024-09-13T16:48:40.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc166ffd640 1 -- 172.21.15.39:0/488095216 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1680039a0 con 0x7fc1700a7a80 2024-09-13T16:48:40.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17014d0a0 con 0x7fc1700a7a80 2024-09-13T16:48:40.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17014d260 con 0x7fc1700a7a80 2024-09-13T16:48:40.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.167+0000 7fc166ffd640 1 -- 172.21.15.39:0/488095216 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc168003b00 con 0x7fc1700a7a80 2024-09-13T16:48:40.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.171+0000 7fc166ffd640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc1500666c0 0x7fc150068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.171+0000 7fc166ffd640 1 -- 172.21.15.39:0/488095216 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fc168084a10 con 0x7fc1700a7a80 2024-09-13T16:48:40.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.171+0000 7fc16e59c640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc1500666c0 0x7fc150068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.171+0000 7fc16e59c640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc1500666c0 0x7fc150068b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc1580af000 tx=0x7fc1580046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:40.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.175+0000 7fc166ffd640 1 -- 172.21.15.39:0/488095216 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc168051a70 con 0x7fc1700a7a80 2024-09-13T16:48:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.331+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc170154210 con 0x7fc1700a7a80 2024-09-13T16:48:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.331+0000 7fc166ffd640 1 -- 172.21.15.39:0/488095216 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v25) v1 ==== 99+0+11 (secure 0 0 0) 0x7fc168003db0 con 0x7fc1700a7a80 2024-09-13T16:48:40.339 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc1500666c0 msgr2=0x7fc150068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc1500666c0 0x7fc150068b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc1580af000 tx=0x7fc1580046d0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1700a7a80 msgr2=0x7fc170070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1700a7a80 0x7fc170070440 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc1700a9610 tx=0x7fc1680115a0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 shutdown_connections 2024-09-13T16:48:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc1500666c0 0x7fc150068b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc17015d460 0x7fc1700a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1700a7a80 0x7fc170070440 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 --2- 172.21.15.39:0/488095216 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc15c003980 0x7fc17015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 >> 172.21.15.39:0/488095216 conn(0x7fc1700c32a0 msgr2=0x7fc170065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 shutdown_connections 2024-09-13T16:48:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.335+0000 7fc174855640 1 -- 172.21.15.39:0/488095216 wait complete. 2024-09-13T16:48:40.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T16:48:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f676b66c640 1 Processor -- start 2024-09-13T16:48:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f676b66c640 1 -- start start 2024-09-13T16:48:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f676b66c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f67640c0ba0 0x7f67640c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f676b66c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6764151200 0x7f67640c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f676b66c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67640a5280 con 0x7f67640c1810 2024-09-13T16:48:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f676b66c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67640c20e0 con 0x7f67640c0ba0 2024-09-13T16:48:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f676b66c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67640a4cc0 con 0x7f6764151200 2024-09-13T16:48:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f6768c07640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f67640c0ba0 0x7f67640c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f6768c07640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f67640c0ba0 0x7f67640c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57108/0 (socket says 172.21.15.39:57108) 2024-09-13T16:48:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.467+0000 7f6768c07640 1 -- 172.21.15.39:0/3682264494 learned_addr learned my addr 172.21.15.39:0/3682264494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:40.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f6769c09640 1 --2- 172.21.15.39:0/3682264494 >> v2:172.21.15.84:3300/0 conn(0x7f6764151200 0x7f67640c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f6768c07640 1 -- 172.21.15.39:0/3682264494 >> v2:172.21.15.84:3300/0 conn(0x7f6764151200 msgr2=0x7f67640c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:40.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f6768c07640 1 --2- 172.21.15.39:0/3682264494 >> v2:172.21.15.84:3300/0 conn(0x7f6764151200 0x7f67640c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f6768c07640 1 -- 172.21.15.39:0/3682264494 >> v1:172.21.15.84:6789/0 conn(0x7f67640c1810 legacy=0x7f6764150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:40.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f6768c07640 1 -- 172.21.15.39:0/3682264494 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67640a4fa0 con 0x7f67640c0ba0 2024-09-13T16:48:40.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f6769c09640 1 --2- 172.21.15.39:0/3682264494 >> v2:172.21.15.84:3300/0 conn(0x7f6764151200 0x7f67640c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:40.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f6768c07640 1 --2- 172.21.15.39:0/3682264494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67640c0ba0 0x7f67640c3790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f67640c54b0 tx=0x7f675401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac0be0a5cf9bf69 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:40.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f675bfff640 1 -- 172.21.15.39:0/3682264494 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540aa610 con 0x7f67640c0ba0 2024-09-13T16:48:40.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f675bfff640 1 -- 172.21.15.39:0/3682264494 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67540a6040 con 0x7f67640c0ba0 2024-09-13T16:48:40.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f675bfff640 1 -- 172.21.15.39:0/3682264494 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540aa990 con 0x7f67640c0ba0 2024-09-13T16:48:40.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f676b66c640 1 -- 172.21.15.39:0/3682264494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67640c0ba0 msgr2=0x7f67640c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f676b66c640 1 --2- 172.21.15.39:0/3682264494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67640c0ba0 0x7f67640c3790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f67640c54b0 tx=0x7f675401c4b0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f676b66c640 1 -- 172.21.15.39:0/3682264494 shutdown_connections 2024-09-13T16:48:40.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f676b66c640 1 --2- 172.21.15.39:0/3682264494 >> v2:172.21.15.84:3300/0 conn(0x7f6764151200 0x7f67640c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f676b66c640 1 --2- 172.21.15.39:0/3682264494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67640c0ba0 0x7f67640c3790 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f676b66c640 1 -- 172.21.15.39:0/3682264494 >> 172.21.15.39:0/3682264494 conn(0x7f67640c5790 msgr2=0x7f67640c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:40.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f676b66c640 1 -- 172.21.15.39:0/3682264494 shutdown_connections 2024-09-13T16:48:40.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.471+0000 7f676b66c640 1 -- 172.21.15.39:0/3682264494 wait complete. 2024-09-13T16:48:40.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 Processor -- start 2024-09-13T16:48:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 -- start start 2024-09-13T16:48:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640c0ba0 0x7f676406f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f67640c1810 0x7f67640bee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6764151200 0x7f67641ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676406fd80 con 0x7f6764151200 2024-09-13T16:48:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6764070060 con 0x7f67640c0ba0 2024-09-13T16:48:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67640717b0 con 0x7f67640c1810 2024-09-13T16:48:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769c09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6764151200 0x7f67641ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769c09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6764151200 0x7f67641ecc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57112/0 (socket says 172.21.15.39:57112) 2024-09-13T16:48:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769c09640 1 -- 172.21.15.39:0/802685810 learned_addr learned my addr 172.21.15.39:0/802685810 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6768c07640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f67640c1810 0x7f67640bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769c09640 1 -- 172.21.15.39:0/802685810 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f67640c1810 msgr2=0x7f67640bee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769c09640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f67640c1810 0x7f67640bee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769c09640 1 -- 172.21.15.39:0/802685810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640c0ba0 msgr2=0x7f676406f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769c09640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640c0ba0 0x7f676406f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769c09640 1 -- 172.21.15.39:0/802685810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67540094d0 con 0x7f6764151200 2024-09-13T16:48:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769c09640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6764151200 0x7f67641ecc90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c7f0 tx=0x7f675c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6759ffb640 1 -- 172.21.15.39:0/802685810 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c004230 con 0x7f6764151200 2024-09-13T16:48:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6759ffb640 1 -- 172.21.15.39:0/802685810 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c0043b0 con 0x7f6764151200 2024-09-13T16:48:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6764071a90 con 0x7f6764151200 2024-09-13T16:48:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6759ffb640 1 -- 172.21.15.39:0/802685810 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c003860 con 0x7f6764151200 2024-09-13T16:48:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676406fa70 con 0x7f6764151200 2024-09-13T16:48:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6764150af0 con 0x7f6764151200 2024-09-13T16:48:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6759ffb640 1 -- 172.21.15.39:0/802685810 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f675c003bd0 con 0x7f6764151200 2024-09-13T16:48:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6759ffb640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6748066690 0x7f6748068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6759ffb640 1 -- 172.21.15.39:0/802685810 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f675c0849a0 con 0x7f6764151200 2024-09-13T16:48:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.475+0000 7f6769408640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6748066690 0x7f6748068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.479+0000 7f6769408640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6748066690 0x7f6748068b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6760007fd0 tx=0x7f6760007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:40.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.479+0000 7f6759ffb640 1 -- 172.21.15.39:0/802685810 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f675c051a00 con 0x7f6764151200 2024-09-13T16:48:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.639+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7f67640c2e10 con 0x7f6764151200 2024-09-13T16:48:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.639+0000 7f6759ffb640 1 -- 172.21.15.39:0/802685810 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v25) v1 ==== 95+0+11 (secure 0 0 0) 0x7f675c055020 con 0x7f6764151200 2024-09-13T16:48:40.645 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6748066690 msgr2=0x7f6748068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6748066690 0x7f6748068b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6760007fd0 tx=0x7f6760007d40 comp rx=0 tx=0).stop 2024-09-13T16:48:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6764151200 msgr2=0x7f67641ecc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6764151200 0x7f67641ecc90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c7f0 tx=0x7f675c00ccb0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 shutdown_connections 2024-09-13T16:48:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6748066690 0x7f6748068b50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6764151200 0x7f67641ecc90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f67640c1810 0x7f67640bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 --2- 172.21.15.39:0/802685810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640c0ba0 0x7f676406f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 >> 172.21.15.39:0/802685810 conn(0x7f67640c5790 msgr2=0x7f67640a6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 shutdown_connections 2024-09-13T16:48:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.643+0000 7f676b66c640 1 -- 172.21.15.39:0/802685810 wait complete. 2024-09-13T16:48:40.664 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2024-09-13T16:48:40.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:40.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2f55bd640 1 Processor -- start 2024-09-13T16:48:40.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2f55bd640 1 -- start start 2024-09-13T16:48:40.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2f55bd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa2f00c4470 0x7fa2f00c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2f55bd640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f00c2d30 con 0x7fa2f0071bb0 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2f55bd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f006fe70 con 0x7fa2f00a7120 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2f55bd640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f006fb90 con 0x7fa2f00c4470 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2eeffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa2f00c4470 0x7fa2f00c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2ef7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa2f0071bb0 0x7fa2f0155a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34892/0 (socket says 172.21.15.39:34892) 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2eeffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa2f00c4470 0x7fa2f00c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57116/0 (socket says 172.21.15.39:57116) 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2ef7fe640 1 -- 172.21.15.39:0/2711472154 learned_addr learned my addr 172.21.15.39:0/2711472154 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2eeffd640 1 -- 172.21.15.39:0/2711472154 >> v1:172.21.15.84:6789/0 conn(0x7fa2f00a7120 legacy=0x7fa2f00714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2eeffd640 1 -- 172.21.15.39:0/2711472154 >> v1:172.21.15.39:6789/0 conn(0x7fa2f0071bb0 legacy=0x7fa2f0155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2eeffd640 1 -- 172.21.15.39:0/2711472154 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f00c26e0 con 0x7fa2f00c4470 2024-09-13T16:48:40.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2edffb640 1 -- 172.21.15.39:0/2711472154 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775735959 0 0) 0x7fa2f00c2d30 con 0x7fa2f0071bb0 2024-09-13T16:48:40.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.783+0000 7fa2eeffd640 1 --2- 172.21.15.39:0/2711472154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 0x7fa2f00c6280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009d90 tx=0x7fa2e001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3c1ce14029876a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:40.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2edffb640 1 -- 172.21.15.39:0/2711472154 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00a5820 con 0x7fa2f00c4470 2024-09-13T16:48:40.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2edffb640 1 -- 172.21.15.39:0/2711472154 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e00a5980 con 0x7fa2f00c4470 2024-09-13T16:48:40.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2edffb640 1 -- 172.21.15.39:0/2711472154 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00a5d10 con 0x7fa2f00c4470 2024-09-13T16:48:40.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2711472154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 msgr2=0x7fa2f00c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 --2- 172.21.15.39:0/2711472154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 0x7fa2f00c6280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009d90 tx=0x7fa2e001c930 comp rx=0 tx=0).stop 2024-09-13T16:48:40.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2711472154 shutdown_connections 2024-09-13T16:48:40.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 --2- 172.21.15.39:0/2711472154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 0x7fa2f00c6280 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2711472154 >> 172.21.15.39:0/2711472154 conn(0x7fa2f006c020 msgr2=0x7fa2f006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:40.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2711472154 shutdown_connections 2024-09-13T16:48:40.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2711472154 wait complete. 2024-09-13T16:48:40.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 Processor -- start 2024-09-13T16:48:40.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- start start 2024-09-13T16:48:40.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2f0071bb0 0x7fa2f01efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2f00a7120 0x7fa2f006a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 0x7fa2f006b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0151070 con 0x7fa2f00c4470 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f006b220 con 0x7fa2f00a7120 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f0151e00 con 0x7fa2f0071bb0 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ef7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 0x7fa2f006b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ef7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 0x7fa2f006b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57120/0 (socket says 172.21.15.39:57120) 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ef7fe640 1 -- 172.21.15.39:0/2602644792 learned_addr learned my addr 172.21.15.39:0/2602644792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2eeffd640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2f0071bb0 0x7fa2f01efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ee7fc640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2f00a7120 0x7fa2f006a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ef7fe640 1 -- 172.21.15.39:0/2602644792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2f0071bb0 msgr2=0x7fa2f01efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ef7fe640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2f0071bb0 0x7fa2f01efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ef7fe640 1 -- 172.21.15.39:0/2602644792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2f00a7120 msgr2=0x7fa2f006a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ef7fe640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2f00a7120 0x7fa2f006a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ef7fe640 1 -- 172.21.15.39:0/2602644792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e0009a70 con 0x7fa2f00c4470 2024-09-13T16:48:40.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ee7fc640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2f00a7120 0x7fa2f006a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:40.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2ef7fe640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 0x7fa2f006b700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800c450 tx=0x7fa2e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:40.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2e77fe640 1 -- 172.21.15.39:0/2602644792 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e80074e0 con 0x7fa2f00c4470 2024-09-13T16:48:40.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2e77fe640 1 -- 172.21.15.39:0/2602644792 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e8007660 con 0x7fa2f00c4470 2024-09-13T16:48:40.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2e77fe640 1 -- 172.21.15.39:0/2602644792 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8004a30 con 0x7fa2f00c4470 2024-09-13T16:48:40.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f01512c0 con 0x7fa2f00c4470 2024-09-13T16:48:40.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.787+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f01515a0 con 0x7fa2f00c4470 2024-09-13T16:48:40.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.791+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f00c20e0 con 0x7fa2f00c4470 2024-09-13T16:48:40.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.791+0000 7fa2e77fe640 1 -- 172.21.15.39:0/2602644792 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa2e8004b90 con 0x7fa2f00c4470 2024-09-13T16:48:40.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.791+0000 7fa2e77fe640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2d00666c0 0x7fa2d0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:40.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.791+0000 7fa2e77fe640 1 -- 172.21.15.39:0/2602644792 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fa2e8084240 con 0x7fa2f00c4470 2024-09-13T16:48:40.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.791+0000 7fa2eeffd640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2d00666c0 0x7fa2d0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:40.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.791+0000 7fa2eeffd640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2d00666c0 0x7fa2d0068b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00af000 tx=0x7fa2e00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:40.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.795+0000 7fa2e77fe640 1 -- 172.21.15.39:0/2602644792 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa2e80512a0 con 0x7fa2f00c4470 2024-09-13T16:48:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2f006b3e0 con 0x7fa2f00c4470 2024-09-13T16:48:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2e77fe640 1 -- 172.21.15.39:0/2602644792 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7fa2e80548c0 con 0x7fa2f00c4470 2024-09-13T16:48:40.988 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:40.988 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:48:40.988 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2d00666c0 msgr2=0x7fa2d0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2d00666c0 0x7fa2d0068b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00af000 tx=0x7fa2e00a7070 comp rx=0 tx=0).stop 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 msgr2=0x7fa2f006b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 0x7fa2f006b700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800c450 tx=0x7fa2e800c910 comp rx=0 tx=0).stop 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 shutdown_connections 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2d00666c0 0x7fa2d0068b80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2f00a7120 0x7fa2f006a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2f00c4470 0x7fa2f006b700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 --2- 172.21.15.39:0/2602644792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2f0071bb0 0x7fa2f01efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 >> 172.21.15.39:0/2602644792 conn(0x7fa2f006c020 msgr2=0x7fa2f015d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 shutdown_connections 2024-09-13T16:48:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:40.983+0000 7fa2f55bd640 1 -- 172.21.15.39:0/2602644792 wait complete. 2024-09-13T16:48:41.005 INFO:tasks.mds_thrash:Ready to start thrashing 2024-09-13T16:48:41.005 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2024-09-13T16:48:41.006 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:41.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd78adb8640 1 Processor -- start 2024-09-13T16:48:41.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd78adb8640 1 -- start start 2024-09-13T16:48:41.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd78adb8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd78414e0c0 0x7fd7840a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd78adb8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840c4740 0x7fd7840c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd78adb8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7840c2780 con 0x7fd7840c1f50 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd78adb8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7840c2a60 con 0x7fd78414e0c0 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd78adb8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840c2d40 con 0x7fd7840c4740 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd783fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd7840c1f50 0x7fd78406cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34898/0 (socket says 172.21.15.39:34898) 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd783fff640 1 -- 172.21.15.39:0/2852607481 learned_addr learned my addr 172.21.15.39:0/2852607481 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd788b54640 1 --2- 172.21.15.39:0/2852607481 >> v2:172.21.15.84:3300/0 conn(0x7fd78414e0c0 0x7fd7840a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd789355640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840c4740 0x7fd7840c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734153949 0 0) 0x7fd7840c2780 con 0x7fd7840c1f50 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd774003530 con 0x7fd7840c1f50 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 742181630 0 0) 0x7fd774003530 con 0x7fd7840c1f50 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840c4740 msgr2=0x7fd7840c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840c4740 0x7fd7840c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 >> v2:172.21.15.84:3300/0 conn(0x7fd78414e0c0 msgr2=0x7fd7840a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 --2- 172.21.15.39:0/2852607481 >> v2:172.21.15.84:3300/0 conn(0x7fd78414e0c0 0x7fd7840a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7840c3020 con 0x7fd7840c1f50 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fd770001680 con 0x7fd7840c1f50 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 >> v1:172.21.15.39:6789/0 conn(0x7fd7840c1f50 legacy=0x7fd78406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 0x7fd774003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7740047d0 0x7fd774004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840c4740 0x7fd774005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840c2a60 con 0x7fd774003980 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840c2d40 con 0x7fd7740047d0 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77400b990 con 0x7fd7840c4740 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd788b54640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 0x7fd774003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd788b54640 1 -- 172.21.15.39:0/2852607481 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840c4740 msgr2=0x7fd774005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd788b54640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840c4740 0x7fd774005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd788b54640 1 -- 172.21.15.39:0/2852607481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7740047d0 msgr2=0x7fd774004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:41.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd788b54640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7740047d0 0x7fd774004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.123+0000 7fd788b54640 1 -- 172.21.15.39:0/2852607481 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774003530 con 0x7fd774003980 2024-09-13T16:48:41.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd788b54640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 0x7fd774003da0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd778009b10 tx=0x7fd77801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1815ac0a8d4d74e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:41.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7780a64b0 con 0x7fd774003980 2024-09-13T16:48:41.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7780a6610 con 0x7fd774003980 2024-09-13T16:48:41.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd7837fe640 1 -- 172.21.15.39:0/2852607481 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7780a69c0 con 0x7fd774003980 2024-09-13T16:48:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd78adb8640 1 -- 172.21.15.39:0/2852607481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 msgr2=0x7fd774003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd78adb8640 1 --2- 172.21.15.39:0/2852607481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 0x7fd774003da0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd778009b10 tx=0x7fd77801ca80 comp rx=0 tx=0).stop 2024-09-13T16:48:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd789355640 1 -- 172.21.15.39:0/2852607481 reap_dead start 2024-09-13T16:48:41.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd78adb8640 1 -- 172.21.15.39:0/2852607481 shutdown_connections 2024-09-13T16:48:41.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd78adb8640 1 -- 172.21.15.39:0/2852607481 >> 172.21.15.39:0/2852607481 conn(0x7fd784069e10 msgr2=0x7fd78406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:41.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd78adb8640 1 -- 172.21.15.39:0/2852607481 shutdown_connections 2024-09-13T16:48:41.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd78adb8640 1 -- 172.21.15.39:0/2852607481 wait complete. 2024-09-13T16:48:41.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd78adb8640 1 Processor -- start 2024-09-13T16:48:41.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.127+0000 7fd78adb8640 1 -- start start 2024-09-13T16:48:41.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd78adb8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840aab00 0x7fd78415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd78adb8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7841f0b50 0x7fd7840aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd78adb8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78415de50 0x7fd7841ec960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd78adb8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7841ed240 con 0x7fd78415de50 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd78adb8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78415b950 con 0x7fd7841f0b50 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd78adb8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7841f0070 con 0x7fd7840aab00 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd789355640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78415de50 0x7fd7841ec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd789355640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78415de50 0x7fd7841ec960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57138/0 (socket says 172.21.15.39:57138) 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd789355640 1 -- 172.21.15.39:0/522558347 learned_addr learned my addr 172.21.15.39:0/522558347 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd788b54640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840aab00 0x7fd78415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd783fff640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7841f0b50 0x7fd7840aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd789355640 1 -- 172.21.15.39:0/522558347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840aab00 msgr2=0x7fd78415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd789355640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840aab00 0x7fd78415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd789355640 1 -- 172.21.15.39:0/522558347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7841f0b50 msgr2=0x7fd7840aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd789355640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7841f0b50 0x7fd7840aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd789355640 1 -- 172.21.15.39:0/522558347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7841f0350 con 0x7fd78415de50 2024-09-13T16:48:41.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd783fff640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7841f0b50 0x7fd7840aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:41.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd789355640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78415de50 0x7fd7841ec960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd7840a7560 tx=0x7fd77c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:41.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd7817fa640 1 -- 172.21.15.39:0/522558347 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c010040 con 0x7fd78415de50 2024-09-13T16:48:41.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd7817fa640 1 -- 172.21.15.39:0/522558347 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c00bae0 con 0x7fd78415de50 2024-09-13T16:48:41.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7841f0630 con 0x7fd78415de50 2024-09-13T16:48:41.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd7817fa640 1 -- 172.21.15.39:0/522558347 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00be70 con 0x7fd78415de50 2024-09-13T16:48:41.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7841ecea0 con 0x7fd78415de50 2024-09-13T16:48:41.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7841efed0 con 0x7fd78415de50 2024-09-13T16:48:41.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd7817fa640 1 -- 172.21.15.39:0/522558347 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd77c01e410 con 0x7fd78415de50 2024-09-13T16:48:41.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd7817fa640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7680666c0 0x7fd768068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd7817fa640 1 -- 172.21.15.39:0/522558347 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fd77c084790 con 0x7fd78415de50 2024-09-13T16:48:41.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd788b54640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7680666c0 0x7fd768068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.131+0000 7fd788b54640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7680666c0 0x7fd768068b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7780af000 tx=0x7fd778008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:41.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.135+0000 7fd7817fa640 1 -- 172.21.15.39:0/522558347 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd7841efed0 con 0x7fd78415de50 2024-09-13T16:48:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.295+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd78406bcb0 con 0x7fd7680666c0 2024-09-13T16:48:41.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.331+0000 7fd7817fa640 1 -- 172.21.15.39:0/522558347 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+502912 (secure 0 0 0) 0x7fd77800f080 con 0x7fd7680666c0 2024-09-13T16:48:41.344 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:41.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7680666c0 msgr2=0x7fd768068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7680666c0 0x7fd768068b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7780af000 tx=0x7fd778008040 comp rx=0 tx=0).stop 2024-09-13T16:48:41.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78415de50 msgr2=0x7fd7841ec960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78415de50 0x7fd7841ec960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd7840a7560 tx=0x7fd77c00b480 comp rx=0 tx=0).stop 2024-09-13T16:48:41.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 shutdown_connections 2024-09-13T16:48:41.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7680666c0 0x7fd768068b80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7841f0b50 0x7fd7840aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78415de50 0x7fd7841ec960 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 --2- 172.21.15.39:0/522558347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7840aab00 0x7fd78415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 >> 172.21.15.39:0/522558347 conn(0x7fd784069e10 msgr2=0x7fd78414c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:41.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 shutdown_connections 2024-09-13T16:48:41.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.335+0000 7fd78adb8640 1 -- 172.21.15.39:0/522558347 wait complete. 2024-09-13T16:48:41.364 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-09-13T16:48:40.757539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595976,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":253,"num_write_kb":2448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235804,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738916,"statfs":{"total":773094113280,"available":772852649984,"internally_reserved":0,"allocated":3350528,"data_stored":1985752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":884,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.189661"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174677+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T16:48:09.174677+0000","last_peered":"2024-09-13T16:48:09.174677+0000","last_clean":"2024-09-13T16:48:09.174677+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T16:48:09.174677+0000","last_undegraded":"2024-09-13T16:48:09.174677+0000","last_fullsized":"2024-09-13T16:48:09.174677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163864+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:08.163864+0000","last_peered":"2024-09-13T16:48:08.163864+0000","last_clean":"2024-09-13T16:48:08.163864+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:08.163864+0000","last_undegraded":"2024-09-13T16:48:08.163864+0000","last_fullsized":"2024-09-13T16:48:08.163864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146105+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:08.146105+0000","last_peered":"2024-09-13T16:48:08.146105+0000","last_clean":"2024-09-13T16:48:08.146105+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:08.146105+0000","last_undegraded":"2024-09-13T16:48:08.146105+0000","last_fullsized":"2024-09-13T16:48:08.146105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143176+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T16:48:08.143176+0000","last_peered":"2024-09-13T16:48:08.143176+0000","last_clean":"2024-09-13T16:48:08.143176+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T16:48:08.143176+0000","last_undegraded":"2024-09-13T16:48:08.143176+0000","last_fullsized":"2024-09-13T16:48:08.143176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173274+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T16:48:09.173274+0000","last_peered":"2024-09-13T16:48:09.173274+0000","last_clean":"2024-09-13T16:48:09.173274+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T16:48:09.173274+0000","last_undegraded":"2024-09-13T16:48:09.173274+0000","last_fullsized":"2024-09-13T16:48:09.173274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146246+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:08.146246+0000","last_peered":"2024-09-13T16:48:08.146246+0000","last_clean":"2024-09-13T16:48:08.146246+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:08.146246+0000","last_undegraded":"2024-09-13T16:48:08.146246+0000","last_fullsized":"2024-09-13T16:48:08.146246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163480+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:08.163480+0000","last_peered":"2024-09-13T16:48:08.163480+0000","last_clean":"2024-09-13T16:48:08.163480+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:08.163480+0000","last_undegraded":"2024-09-13T16:48:08.163480+0000","last_fullsized":"2024-09-13T16:48:08.163480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184157+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T16:48:08.184157+0000","last_peered":"2024-09-13T16:48:08.184157+0000","last_clean":"2024-09-13T16:48:08.184157+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T16:48:08.184157+0000","last_undegraded":"2024-09-13T16:48:08.184157+0000","last_fullsized":"2024-09-13T16:48:08.184157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173949+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T16:48:09.173949+0000","last_peered":"2024-09-13T16:48:09.173949+0000","last_clean":"2024-09-13T16:48:09.173949+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T16:48:09.173949+0000","last_undegraded":"2024-09-13T16:48:09.173949+0000","last_fullsized":"2024-09-13T16:48:09.173949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178923+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:08.178923+0000","last_peered":"2024-09-13T16:48:08.178923+0000","last_clean":"2024-09-13T16:48:08.178923+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:08.178923+0000","last_undegraded":"2024-09-13T16:48:08.178923+0000","last_fullsized":"2024-09-13T16:48:08.178923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151847+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:08.151847+0000","last_peered":"2024-09-13T16:48:08.151847+0000","last_clean":"2024-09-13T16:48:08.151847+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:08.151847+0000","last_undegraded":"2024-09-13T16:48:08.151847+0000","last_fullsized":"2024-09-13T16:48:08.151847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208404+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T16:48:08.208404+0000","last_peered":"2024-09-13T16:48:08.208404+0000","last_clean":"2024-09-13T16:48:08.208404+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T16:48:08.208404+0000","last_undegraded":"2024-09-13T16:48:08.208404+0000","last_fullsized":"2024-09-13T16:48:08.208404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159591+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T16:48:09.159591+0000","last_peered":"2024-09-13T16:48:09.159591+0000","last_clean":"2024-09-13T16:48:09.159591+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T16:48:09.159591+0000","last_undegraded":"2024-09-13T16:48:09.159591+0000","last_fullsized":"2024-09-13T16:48:09.159591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165397+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:08.165397+0000","last_peered":"2024-09-13T16:48:08.165397+0000","last_clean":"2024-09-13T16:48:08.165397+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:08.165397+0000","last_undegraded":"2024-09-13T16:48:08.165397+0000","last_fullsized":"2024-09-13T16:48:08.165397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153028+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:08.153028+0000","last_peered":"2024-09-13T16:48:08.153028+0000","last_clean":"2024-09-13T16:48:08.153028+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:08.153028+0000","last_undegraded":"2024-09-13T16:48:08.153028+0000","last_fullsized":"2024-09-13T16:48:08.153028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.203658+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T16:48:08.203658+0000","last_peered":"2024-09-13T16:48:08.203658+0000","last_clean":"2024-09-13T16:48:08.203658+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T16:48:08.203658+0000","last_undegraded":"2024-09-13T16:48:08.203658+0000","last_fullsized":"2024-09-13T16:48:08.203658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173632+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T16:48:09.173632+0000","last_peered":"2024-09-13T16:48:09.173632+0000","last_clean":"2024-09-13T16:48:09.173632+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T16:48:09.173632+0000","last_undegraded":"2024-09-13T16:48:09.173632+0000","last_fullsized":"2024-09-13T16:48:09.173632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146884+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:08.146884+0000","last_peered":"2024-09-13T16:48:08.146884+0000","last_clean":"2024-09-13T16:48:08.146884+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:08.146884+0000","last_undegraded":"2024-09-13T16:48:08.146884+0000","last_fullsized":"2024-09-13T16:48:08.146884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160788+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:08.160788+0000","last_peered":"2024-09-13T16:48:08.160788+0000","last_clean":"2024-09-13T16:48:08.160788+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:08.160788+0000","last_undegraded":"2024-09-13T16:48:08.160788+0000","last_fullsized":"2024-09-13T16:48:08.160788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.206623+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T16:48:08.206623+0000","last_peered":"2024-09-13T16:48:08.206623+0000","last_clean":"2024-09-13T16:48:08.206623+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T16:48:08.206623+0000","last_undegraded":"2024-09-13T16:48:08.206623+0000","last_fullsized":"2024-09-13T16:48:08.206623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159080+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T16:48:09.159080+0000","last_peered":"2024-09-13T16:48:09.159080+0000","last_clean":"2024-09-13T16:48:09.159080+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T16:48:09.159080+0000","last_undegraded":"2024-09-13T16:48:09.159080+0000","last_fullsized":"2024-09-13T16:48:09.159080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144345+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:08.144345+0000","last_peered":"2024-09-13T16:48:08.144345+0000","last_clean":"2024-09-13T16:48:08.144345+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:08.144345+0000","last_undegraded":"2024-09-13T16:48:08.144345+0000","last_fullsized":"2024-09-13T16:48:08.144345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147467+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:08.147467+0000","last_peered":"2024-09-13T16:48:08.147467+0000","last_clean":"2024-09-13T16:48:08.147467+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:08.147467+0000","last_undegraded":"2024-09-13T16:48:08.147467+0000","last_fullsized":"2024-09-13T16:48:08.147467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184081+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T16:48:08.184081+0000","last_peered":"2024-09-13T16:48:08.184081+0000","last_clean":"2024-09-13T16:48:08.184081+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T16:48:08.184081+0000","last_undegraded":"2024-09-13T16:48:08.184081+0000","last_fullsized":"2024-09-13T16:48:08.184081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143476+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:08.143476+0000","last_peered":"2024-09-13T16:48:08.143476+0000","last_clean":"2024-09-13T16:48:08.143476+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:08.143476+0000","last_undegraded":"2024-09-13T16:48:08.143476+0000","last_fullsized":"2024-09-13T16:48:08.143476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147565+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:08.147565+0000","last_peered":"2024-09-13T16:48:08.147565+0000","last_clean":"2024-09-13T16:48:08.147565+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:08.147565+0000","last_undegraded":"2024-09-13T16:48:08.147565+0000","last_fullsized":"2024-09-13T16:48:08.147565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174007+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T16:48:08.174007+0000","last_peered":"2024-09-13T16:48:08.174007+0000","last_clean":"2024-09-13T16:48:08.174007+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T16:48:08.174007+0000","last_undegraded":"2024-09-13T16:48:08.174007+0000","last_fullsized":"2024-09-13T16:48:08.174007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162482+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T16:48:09.162482+0000","last_peered":"2024-09-13T16:48:09.162482+0000","last_clean":"2024-09-13T16:48:09.162482+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T16:48:09.162482+0000","last_undegraded":"2024-09-13T16:48:09.162482+0000","last_fullsized":"2024-09-13T16:48:09.162482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186454+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:08.186454+0000","last_peered":"2024-09-13T16:48:08.186454+0000","last_clean":"2024-09-13T16:48:08.186454+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:08.186454+0000","last_undegraded":"2024-09-13T16:48:08.186454+0000","last_fullsized":"2024-09-13T16:48:08.186454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160985+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:08.160985+0000","last_peered":"2024-09-13T16:48:08.160985+0000","last_clean":"2024-09-13T16:48:08.160985+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:08.160985+0000","last_undegraded":"2024-09-13T16:48:08.160985+0000","last_fullsized":"2024-09-13T16:48:08.160985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208189+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T16:48:08.208189+0000","last_peered":"2024-09-13T16:48:08.208189+0000","last_clean":"2024-09-13T16:48:08.208189+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T16:48:08.208189+0000","last_undegraded":"2024-09-13T16:48:08.208189+0000","last_fullsized":"2024-09-13T16:48:08.208189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176059+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T16:48:09.176059+0000","last_peered":"2024-09-13T16:48:09.176059+0000","last_clean":"2024-09-13T16:48:09.176059+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T16:48:09.176059+0000","last_undegraded":"2024-09-13T16:48:09.176059+0000","last_fullsized":"2024-09-13T16:48:09.176059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163768+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:08.163768+0000","last_peered":"2024-09-13T16:48:08.163768+0000","last_clean":"2024-09-13T16:48:08.163768+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:08.163768+0000","last_undegraded":"2024-09-13T16:48:08.163768+0000","last_fullsized":"2024-09-13T16:48:08.163768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148721+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:08.148721+0000","last_peered":"2024-09-13T16:48:08.148721+0000","last_clean":"2024-09-13T16:48:08.148721+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:08.148721+0000","last_undegraded":"2024-09-13T16:48:08.148721+0000","last_fullsized":"2024-09-13T16:48:08.148721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185371+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T16:48:08.185371+0000","last_peered":"2024-09-13T16:48:08.185371+0000","last_clean":"2024-09-13T16:48:08.185371+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T16:48:08.185371+0000","last_undegraded":"2024-09-13T16:48:08.185371+0000","last_fullsized":"2024-09-13T16:48:08.185371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174416+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T16:48:09.174416+0000","last_peered":"2024-09-13T16:48:09.174416+0000","last_clean":"2024-09-13T16:48:09.174416+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T16:48:09.174416+0000","last_undegraded":"2024-09-13T16:48:09.174416+0000","last_fullsized":"2024-09-13T16:48:09.174416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145183+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:08.145183+0000","last_peered":"2024-09-13T16:48:08.145183+0000","last_clean":"2024-09-13T16:48:08.145183+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:08.145183+0000","last_undegraded":"2024-09-13T16:48:08.145183+0000","last_fullsized":"2024-09-13T16:48:08.145183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152363+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:08.152363+0000","last_peered":"2024-09-13T16:48:08.152363+0000","last_clean":"2024-09-13T16:48:08.152363+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:08.152363+0000","last_undegraded":"2024-09-13T16:48:08.152363+0000","last_fullsized":"2024-09-13T16:48:08.152363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143237+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T16:48:08.143237+0000","last_peered":"2024-09-13T16:48:08.143237+0000","last_clean":"2024-09-13T16:48:08.143237+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T16:48:08.143237+0000","last_undegraded":"2024-09-13T16:48:08.143237+0000","last_fullsized":"2024-09-13T16:48:08.143237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158858+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T16:48:09.158858+0000","last_peered":"2024-09-13T16:48:09.158858+0000","last_clean":"2024-09-13T16:48:09.158858+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T16:48:09.158858+0000","last_undegraded":"2024-09-13T16:48:09.158858+0000","last_fullsized":"2024-09-13T16:48:09.158858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148446+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:08.148446+0000","last_peered":"2024-09-13T16:48:08.148446+0000","last_clean":"2024-09-13T16:48:08.148446+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:08.148446+0000","last_undegraded":"2024-09-13T16:48:08.148446+0000","last_fullsized":"2024-09-13T16:48:08.148446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179426+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:08.179426+0000","last_peered":"2024-09-13T16:48:08.179426+0000","last_clean":"2024-09-13T16:48:08.179426+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:08.179426+0000","last_undegraded":"2024-09-13T16:48:08.179426+0000","last_fullsized":"2024-09-13T16:48:08.179426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161207+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T16:48:08.161207+0000","last_peered":"2024-09-13T16:48:08.161207+0000","last_clean":"2024-09-13T16:48:08.161207+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T16:48:08.161207+0000","last_undegraded":"2024-09-13T16:48:08.161207+0000","last_fullsized":"2024-09-13T16:48:08.161207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173006+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T16:48:09.173006+0000","last_peered":"2024-09-13T16:48:09.173006+0000","last_clean":"2024-09-13T16:48:09.173006+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T16:48:09.173006+0000","last_undegraded":"2024-09-13T16:48:09.173006+0000","last_fullsized":"2024-09-13T16:48:09.173006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153222+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:08.153222+0000","last_peered":"2024-09-13T16:48:08.153222+0000","last_clean":"2024-09-13T16:48:08.153222+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:08.153222+0000","last_undegraded":"2024-09-13T16:48:08.153222+0000","last_fullsized":"2024-09-13T16:48:08.153222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148522+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:08.148522+0000","last_peered":"2024-09-13T16:48:08.148522+0000","last_clean":"2024-09-13T16:48:08.148522+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:08.148522+0000","last_undegraded":"2024-09-13T16:48:08.148522+0000","last_fullsized":"2024-09-13T16:48:08.148522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146684+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T16:48:08.146684+0000","last_peered":"2024-09-13T16:48:08.146684+0000","last_clean":"2024-09-13T16:48:08.146684+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T16:48:08.146684+0000","last_undegraded":"2024-09-13T16:48:08.146684+0000","last_fullsized":"2024-09-13T16:48:08.146684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174867+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T16:48:09.174867+0000","last_peered":"2024-09-13T16:48:09.174867+0000","last_clean":"2024-09-13T16:48:09.174867+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T16:48:09.174867+0000","last_undegraded":"2024-09-13T16:48:09.174867+0000","last_fullsized":"2024-09-13T16:48:09.174867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185855+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:08.185855+0000","last_peered":"2024-09-13T16:48:08.185855+0000","last_clean":"2024-09-13T16:48:08.185855+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:08.185855+0000","last_undegraded":"2024-09-13T16:48:08.185855+0000","last_fullsized":"2024-09-13T16:48:08.185855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149466+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:08.149466+0000","last_peered":"2024-09-13T16:48:08.149466+0000","last_clean":"2024-09-13T16:48:08.149466+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:08.149466+0000","last_undegraded":"2024-09-13T16:48:08.149466+0000","last_fullsized":"2024-09-13T16:48:08.149466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148372+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T16:48:08.148372+0000","last_peered":"2024-09-13T16:48:08.148372+0000","last_clean":"2024-09-13T16:48:08.148372+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T16:48:08.148372+0000","last_undegraded":"2024-09-13T16:48:08.148372+0000","last_fullsized":"2024-09-13T16:48:08.148372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159186+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T16:48:09.159186+0000","last_peered":"2024-09-13T16:48:09.159186+0000","last_clean":"2024-09-13T16:48:09.159186+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T16:48:09.159186+0000","last_undegraded":"2024-09-13T16:48:09.159186+0000","last_fullsized":"2024-09-13T16:48:09.159186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146860+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:08.146860+0000","last_peered":"2024-09-13T16:48:08.146860+0000","last_clean":"2024-09-13T16:48:08.146860+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:08.146860+0000","last_undegraded":"2024-09-13T16:48:08.146860+0000","last_fullsized":"2024-09-13T16:48:08.146860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154111+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:08.154111+0000","last_peered":"2024-09-13T16:48:08.154111+0000","last_clean":"2024-09-13T16:48:08.154111+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:08.154111+0000","last_undegraded":"2024-09-13T16:48:08.154111+0000","last_fullsized":"2024-09-13T16:48:08.154111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175388+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T16:48:08.175388+0000","last_peered":"2024-09-13T16:48:08.175388+0000","last_clean":"2024-09-13T16:48:08.175388+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T16:48:08.175388+0000","last_undegraded":"2024-09-13T16:48:08.175388+0000","last_fullsized":"2024-09-13T16:48:08.175388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162556+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T16:48:09.162556+0000","last_peered":"2024-09-13T16:48:09.162556+0000","last_clean":"2024-09-13T16:48:09.162556+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T16:48:09.162556+0000","last_undegraded":"2024-09-13T16:48:09.162556+0000","last_fullsized":"2024-09-13T16:48:09.162556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160990+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:08.160990+0000","last_peered":"2024-09-13T16:48:08.160990+0000","last_clean":"2024-09-13T16:48:08.160990+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:08.160990+0000","last_undegraded":"2024-09-13T16:48:08.160990+0000","last_fullsized":"2024-09-13T16:48:08.160990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161254+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:08.161254+0000","last_peered":"2024-09-13T16:48:08.161254+0000","last_clean":"2024-09-13T16:48:08.161254+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:08.161254+0000","last_undegraded":"2024-09-13T16:48:08.161254+0000","last_fullsized":"2024-09-13T16:48:08.161254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175218+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T16:48:09.175218+0000","last_peered":"2024-09-13T16:48:09.175218+0000","last_clean":"2024-09-13T16:48:09.175218+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T16:48:09.175218+0000","last_undegraded":"2024-09-13T16:48:09.175218+0000","last_fullsized":"2024-09-13T16:48:09.175218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175463+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T16:48:09.175463+0000","last_peered":"2024-09-13T16:48:09.175463+0000","last_clean":"2024-09-13T16:48:09.175463+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T16:48:09.175463+0000","last_undegraded":"2024-09-13T16:48:09.175463+0000","last_fullsized":"2024-09-13T16:48:09.175463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143792+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:08.143792+0000","last_peered":"2024-09-13T16:48:08.143792+0000","last_clean":"2024-09-13T16:48:08.143792+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:08.143792+0000","last_undegraded":"2024-09-13T16:48:08.143792+0000","last_fullsized":"2024-09-13T16:48:08.143792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148284+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:08.148284+0000","last_peered":"2024-09-13T16:48:08.148284+0000","last_clean":"2024-09-13T16:48:08.148284+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:08.148284+0000","last_undegraded":"2024-09-13T16:48:08.148284+0000","last_fullsized":"2024-09-13T16:48:08.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161478+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:08.161478+0000","last_peered":"2024-09-13T16:48:08.161478+0000","last_clean":"2024-09-13T16:48:08.161478+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:08.161478+0000","last_undegraded":"2024-09-13T16:48:08.161478+0000","last_fullsized":"2024-09-13T16:48:08.161478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148905+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T16:48:08.148905+0000","last_peered":"2024-09-13T16:48:08.148905+0000","last_clean":"2024-09-13T16:48:08.148905+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T16:48:08.148905+0000","last_undegraded":"2024-09-13T16:48:08.148905+0000","last_fullsized":"2024-09-13T16:48:08.148905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158934+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T16:48:09.158934+0000","last_peered":"2024-09-13T16:48:09.158934+0000","last_clean":"2024-09-13T16:48:09.158934+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T16:48:09.158934+0000","last_undegraded":"2024-09-13T16:48:09.158934+0000","last_fullsized":"2024-09-13T16:48:09.158934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178617+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:08.178617+0000","last_peered":"2024-09-13T16:48:08.178617+0000","last_clean":"2024-09-13T16:48:08.178617+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:08.178617+0000","last_undegraded":"2024-09-13T16:48:08.178617+0000","last_fullsized":"2024-09-13T16:48:08.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155913+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:08.155913+0000","last_peered":"2024-09-13T16:48:08.155913+0000","last_clean":"2024-09-13T16:48:08.155913+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:08.155913+0000","last_undegraded":"2024-09-13T16:48:08.155913+0000","last_fullsized":"2024-09-13T16:48:08.155913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146319+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T16:48:08.146319+0000","last_peered":"2024-09-13T16:48:08.146319+0000","last_clean":"2024-09-13T16:48:08.146319+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T16:48:08.146319+0000","last_undegraded":"2024-09-13T16:48:08.146319+0000","last_fullsized":"2024-09-13T16:48:08.146319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158609+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T16:48:09.158609+0000","last_peered":"2024-09-13T16:48:09.158609+0000","last_clean":"2024-09-13T16:48:09.158609+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T16:48:09.158609+0000","last_undegraded":"2024-09-13T16:48:09.158609+0000","last_fullsized":"2024-09-13T16:48:09.158609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152057+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:08.152057+0000","last_peered":"2024-09-13T16:48:08.152057+0000","last_clean":"2024-09-13T16:48:08.152057+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:08.152057+0000","last_undegraded":"2024-09-13T16:48:08.152057+0000","last_fullsized":"2024-09-13T16:48:08.152057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149487+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:08.149487+0000","last_peered":"2024-09-13T16:48:08.149487+0000","last_clean":"2024-09-13T16:48:08.149487+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:08.149487+0000","last_undegraded":"2024-09-13T16:48:08.149487+0000","last_fullsized":"2024-09-13T16:48:08.149487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175317+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T16:48:08.175317+0000","last_peered":"2024-09-13T16:48:08.175317+0000","last_clean":"2024-09-13T16:48:08.175317+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T16:48:08.175317+0000","last_undegraded":"2024-09-13T16:48:08.175317+0000","last_fullsized":"2024-09-13T16:48:08.175317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173209+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T16:48:09.173209+0000","last_peered":"2024-09-13T16:48:09.173209+0000","last_clean":"2024-09-13T16:48:09.173209+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T16:48:09.173209+0000","last_undegraded":"2024-09-13T16:48:09.173209+0000","last_fullsized":"2024-09-13T16:48:09.173209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.177954+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:08.177954+0000","last_peered":"2024-09-13T16:48:08.177954+0000","last_clean":"2024-09-13T16:48:08.177954+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:08.177954+0000","last_undegraded":"2024-09-13T16:48:08.177954+0000","last_fullsized":"2024-09-13T16:48:08.177954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155936+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:08.155936+0000","last_peered":"2024-09-13T16:48:08.155936+0000","last_clean":"2024-09-13T16:48:08.155936+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:08.155936+0000","last_undegraded":"2024-09-13T16:48:08.155936+0000","last_fullsized":"2024-09-13T16:48:08.155936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158692+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T16:48:08.158692+0000","last_peered":"2024-09-13T16:48:08.158692+0000","last_clean":"2024-09-13T16:48:08.158692+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T16:48:08.158692+0000","last_undegraded":"2024-09-13T16:48:08.158692+0000","last_fullsized":"2024-09-13T16:48:08.158692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158034+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T16:48:09.158034+0000","last_peered":"2024-09-13T16:48:09.158034+0000","last_clean":"2024-09-13T16:48:09.158034+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T16:48:09.158034+0000","last_undegraded":"2024-09-13T16:48:09.158034+0000","last_fullsized":"2024-09-13T16:48:09.158034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160811+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:08.160811+0000","last_peered":"2024-09-13T16:48:08.160811+0000","last_clean":"2024-09-13T16:48:08.160811+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:08.160811+0000","last_undegraded":"2024-09-13T16:48:08.160811+0000","last_fullsized":"2024-09-13T16:48:08.160811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.192267+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:08.192267+0000","last_peered":"2024-09-13T16:48:08.192267+0000","last_clean":"2024-09-13T16:48:08.192267+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:08.192267+0000","last_undegraded":"2024-09-13T16:48:08.192267+0000","last_fullsized":"2024-09-13T16:48:08.192267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185117+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T16:48:08.185117+0000","last_peered":"2024-09-13T16:48:08.185117+0000","last_clean":"2024-09-13T16:48:08.185117+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T16:48:08.185117+0000","last_undegraded":"2024-09-13T16:48:08.185117+0000","last_fullsized":"2024-09-13T16:48:08.185117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177888+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T16:48:09.177888+0000","last_peered":"2024-09-13T16:48:09.177888+0000","last_clean":"2024-09-13T16:48:09.177888+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T16:48:09.177888+0000","last_undegraded":"2024-09-13T16:48:09.177888+0000","last_fullsized":"2024-09-13T16:48:09.177888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167237+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:08.167237+0000","last_peered":"2024-09-13T16:48:08.167237+0000","last_clean":"2024-09-13T16:48:08.167237+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:08.167237+0000","last_undegraded":"2024-09-13T16:48:08.167237+0000","last_fullsized":"2024-09-13T16:48:08.167237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156043+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:08.156043+0000","last_peered":"2024-09-13T16:48:08.156043+0000","last_clean":"2024-09-13T16:48:08.156043+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:08.156043+0000","last_undegraded":"2024-09-13T16:48:08.156043+0000","last_fullsized":"2024-09-13T16:48:08.156043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163690+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T16:48:08.163690+0000","last_peered":"2024-09-13T16:48:08.163690+0000","last_clean":"2024-09-13T16:48:08.163690+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T16:48:08.163690+0000","last_undegraded":"2024-09-13T16:48:08.163690+0000","last_fullsized":"2024-09-13T16:48:08.163690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162212+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T16:48:09.162212+0000","last_peered":"2024-09-13T16:48:09.162212+0000","last_clean":"2024-09-13T16:48:09.162212+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T16:48:09.162212+0000","last_undegraded":"2024-09-13T16:48:09.162212+0000","last_fullsized":"2024-09-13T16:48:09.162212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144923+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:08.144923+0000","last_peered":"2024-09-13T16:48:08.144923+0000","last_clean":"2024-09-13T16:48:08.144923+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:08.144923+0000","last_undegraded":"2024-09-13T16:48:08.144923+0000","last_fullsized":"2024-09-13T16:48:08.144923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145599+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:08.145599+0000","last_peered":"2024-09-13T16:48:08.145599+0000","last_clean":"2024-09-13T16:48:08.145599+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:08.145599+0000","last_undegraded":"2024-09-13T16:48:08.145599+0000","last_fullsized":"2024-09-13T16:48:08.145599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145022+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T16:48:08.145022+0000","last_peered":"2024-09-13T16:48:08.145022+0000","last_clean":"2024-09-13T16:48:08.145022+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T16:48:08.145022+0000","last_undegraded":"2024-09-13T16:48:08.145022+0000","last_fullsized":"2024-09-13T16:48:08.145022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161391+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T16:48:09.161391+0000","last_peered":"2024-09-13T16:48:09.161391+0000","last_clean":"2024-09-13T16:48:09.161391+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T16:48:09.161391+0000","last_undegraded":"2024-09-13T16:48:09.161391+0000","last_fullsized":"2024-09-13T16:48:09.161391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160556+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:08.160556+0000","last_peered":"2024-09-13T16:48:08.160556+0000","last_clean":"2024-09-13T16:48:08.160556+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:08.160556+0000","last_undegraded":"2024-09-13T16:48:08.160556+0000","last_fullsized":"2024-09-13T16:48:08.160556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149235+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:08.149235+0000","last_peered":"2024-09-13T16:48:08.149235+0000","last_clean":"2024-09-13T16:48:08.149235+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:08.149235+0000","last_undegraded":"2024-09-13T16:48:08.149235+0000","last_fullsized":"2024-09-13T16:48:08.149235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"24'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:35.040239+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T16:48:35.040239+0000","last_peered":"2024-09-13T16:48:35.040239+0000","last_clean":"2024-09-13T16:48:35.040239+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T16:48:35.040239+0000","last_undegraded":"2024-09-13T16:48:35.040239+0000","last_fullsized":"2024-09-13T16:48:35.040239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2006,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161110+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:08.161110+0000","last_peered":"2024-09-13T16:48:08.161110+0000","last_clean":"2024-09-13T16:48:08.161110+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:08.161110+0000","last_undegraded":"2024-09-13T16:48:08.161110+0000","last_fullsized":"2024-09-13T16:48:08.161110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161421+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:08.161421+0000","last_peered":"2024-09-13T16:48:08.161421+0000","last_clean":"2024-09-13T16:48:08.161421+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:08.161421+0000","last_undegraded":"2024-09-13T16:48:08.161421+0000","last_fullsized":"2024-09-13T16:48:08.161421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.188902+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T16:48:08.188902+0000","last_peered":"2024-09-13T16:48:08.188902+0000","last_clean":"2024-09-13T16:48:08.188902+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T16:48:08.188902+0000","last_undegraded":"2024-09-13T16:48:08.188902+0000","last_fullsized":"2024-09-13T16:48:08.188902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165383+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:08.165383+0000","last_peered":"2024-09-13T16:48:08.165383+0000","last_clean":"2024-09-13T16:48:08.165383+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:08.165383+0000","last_undegraded":"2024-09-13T16:48:08.165383+0000","last_fullsized":"2024-09-13T16:48:08.165383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:31.963081+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:31.963081+0000","last_peered":"2024-09-13T16:48:31.963081+0000","last_clean":"2024-09-13T16:48:31.963081+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:31.963081+0000","last_undegraded":"2024-09-13T16:48:31.963081+0000","last_fullsized":"2024-09-13T16:48:31.963081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2006,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146582+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T16:48:08.146582+0000","last_peered":"2024-09-13T16:48:08.146582+0000","last_clean":"2024-09-13T16:48:08.146582+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T16:48:08.146582+0000","last_undegraded":"2024-09-13T16:48:08.146582+0000","last_fullsized":"2024-09-13T16:48:08.146582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173477+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T16:48:09.173477+0000","last_peered":"2024-09-13T16:48:09.173477+0000","last_clean":"2024-09-13T16:48:09.173477+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T16:48:09.173477+0000","last_undegraded":"2024-09-13T16:48:09.173477+0000","last_fullsized":"2024-09-13T16:48:09.173477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.190852+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:08.190852+0000","last_peered":"2024-09-13T16:48:08.190852+0000","last_clean":"2024-09-13T16:48:08.190852+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:08.190852+0000","last_undegraded":"2024-09-13T16:48:08.190852+0000","last_fullsized":"2024-09-13T16:48:08.190852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149403+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:08.149403+0000","last_peered":"2024-09-13T16:48:08.149403+0000","last_clean":"2024-09-13T16:48:08.149403+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:08.149403+0000","last_undegraded":"2024-09-13T16:48:08.149403+0000","last_fullsized":"2024-09-13T16:48:08.149403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208558+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T16:48:08.208558+0000","last_peered":"2024-09-13T16:48:08.208558+0000","last_clean":"2024-09-13T16:48:08.208558+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T16:48:08.208558+0000","last_undegraded":"2024-09-13T16:48:08.208558+0000","last_fullsized":"2024-09-13T16:48:08.208558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158601+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T16:48:09.158601+0000","last_peered":"2024-09-13T16:48:09.158601+0000","last_clean":"2024-09-13T16:48:09.158601+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T16:48:09.158601+0000","last_undegraded":"2024-09-13T16:48:09.158601+0000","last_fullsized":"2024-09-13T16:48:09.158601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163596+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:08.163596+0000","last_peered":"2024-09-13T16:48:08.163596+0000","last_clean":"2024-09-13T16:48:08.163596+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:08.163596+0000","last_undegraded":"2024-09-13T16:48:08.163596+0000","last_fullsized":"2024-09-13T16:48:08.163596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156189+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:08.156189+0000","last_peered":"2024-09-13T16:48:08.156189+0000","last_clean":"2024-09-13T16:48:08.156189+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:08.156189+0000","last_undegraded":"2024-09-13T16:48:08.156189+0000","last_fullsized":"2024-09-13T16:48:08.156189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174258+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T16:48:08.174258+0000","last_peered":"2024-09-13T16:48:08.174258+0000","last_clean":"2024-09-13T16:48:08.174258+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T16:48:08.174258+0000","last_undegraded":"2024-09-13T16:48:08.174258+0000","last_fullsized":"2024-09-13T16:48:08.174258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173305+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T16:48:09.173305+0000","last_peered":"2024-09-13T16:48:09.173305+0000","last_clean":"2024-09-13T16:48:09.173305+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T16:48:09.173305+0000","last_undegraded":"2024-09-13T16:48:09.173305+0000","last_fullsized":"2024-09-13T16:48:09.173305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155963+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:08.155963+0000","last_peered":"2024-09-13T16:48:08.155963+0000","last_clean":"2024-09-13T16:48:08.155963+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:08.155963+0000","last_undegraded":"2024-09-13T16:48:08.155963+0000","last_fullsized":"2024-09-13T16:48:08.155963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148500+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:08.148500+0000","last_peered":"2024-09-13T16:48:08.148500+0000","last_clean":"2024-09-13T16:48:08.148500+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:08.148500+0000","last_undegraded":"2024-09-13T16:48:08.148500+0000","last_fullsized":"2024-09-13T16:48:08.148500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143144+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T16:48:08.143144+0000","last_peered":"2024-09-13T16:48:08.143144+0000","last_clean":"2024-09-13T16:48:08.143144+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T16:48:08.143144+0000","last_undegraded":"2024-09-13T16:48:08.143144+0000","last_fullsized":"2024-09-13T16:48:08.143144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161066+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T16:48:09.161066+0000","last_peered":"2024-09-13T16:48:09.161066+0000","last_clean":"2024-09-13T16:48:09.161066+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T16:48:09.161066+0000","last_undegraded":"2024-09-13T16:48:09.161066+0000","last_fullsized":"2024-09-13T16:48:09.161066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186757+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T16:48:08.186757+0000","last_peered":"2024-09-13T16:48:08.186757+0000","last_clean":"2024-09-13T16:48:08.186757+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T16:48:08.186757+0000","last_undegraded":"2024-09-13T16:48:08.186757+0000","last_fullsized":"2024-09-13T16:48:08.186757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147291+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:08.147291+0000","last_peered":"2024-09-13T16:48:08.147291+0000","last_clean":"2024-09-13T16:48:08.147291+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:08.147291+0000","last_undegraded":"2024-09-13T16:48:08.147291+0000","last_fullsized":"2024-09-13T16:48:08.147291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158941+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:08.158941+0000","last_peered":"2024-09-13T16:48:08.158941+0000","last_clean":"2024-09-13T16:48:08.158941+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:08.158941+0000","last_undegraded":"2024-09-13T16:48:08.158941+0000","last_fullsized":"2024-09-13T16:48:08.158941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175068+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T16:48:09.175068+0000","last_peered":"2024-09-13T16:48:09.175068+0000","last_clean":"2024-09-13T16:48:09.175068+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T16:48:09.175068+0000","last_undegraded":"2024-09-13T16:48:09.175068+0000","last_fullsized":"2024-09-13T16:48:09.175068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.182997+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T16:48:08.182997+0000","last_peered":"2024-09-13T16:48:08.182997+0000","last_clean":"2024-09-13T16:48:08.182997+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T16:48:08.182997+0000","last_undegraded":"2024-09-13T16:48:08.182997+0000","last_fullsized":"2024-09-13T16:48:08.182997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143381+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:08.143381+0000","last_peered":"2024-09-13T16:48:08.143381+0000","last_clean":"2024-09-13T16:48:08.143381+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:08.143381+0000","last_undegraded":"2024-09-13T16:48:08.143381+0000","last_fullsized":"2024-09-13T16:48:08.143381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143921+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:08.143921+0000","last_peered":"2024-09-13T16:48:08.143921+0000","last_clean":"2024-09-13T16:48:08.143921+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:08.143921+0000","last_undegraded":"2024-09-13T16:48:08.143921+0000","last_fullsized":"2024-09-13T16:48:08.143921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174477+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T16:48:09.174477+0000","last_peered":"2024-09-13T16:48:09.174477+0000","last_clean":"2024-09-13T16:48:09.174477+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T16:48:09.174477+0000","last_undegraded":"2024-09-13T16:48:09.174477+0000","last_fullsized":"2024-09-13T16:48:09.174477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147114+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:08.147114+0000","last_peered":"2024-09-13T16:48:08.147114+0000","last_clean":"2024-09-13T16:48:08.147114+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:08.147114+0000","last_undegraded":"2024-09-13T16:48:08.147114+0000","last_fullsized":"2024-09-13T16:48:08.147114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161410+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:08.161410+0000","last_peered":"2024-09-13T16:48:08.161410+0000","last_clean":"2024-09-13T16:48:08.161410+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:08.161410+0000","last_undegraded":"2024-09-13T16:48:08.161410+0000","last_fullsized":"2024-09-13T16:48:08.161410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170387+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T16:48:08.170387+0000","last_peered":"2024-09-13T16:48:08.170387+0000","last_clean":"2024-09-13T16:48:08.170387+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T16:48:08.170387+0000","last_undegraded":"2024-09-13T16:48:08.170387+0000","last_fullsized":"2024-09-13T16:48:08.170387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175603+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T16:48:09.175603+0000","last_peered":"2024-09-13T16:48:09.175603+0000","last_clean":"2024-09-13T16:48:09.175603+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T16:48:09.175603+0000","last_undegraded":"2024-09-13T16:48:09.175603+0000","last_fullsized":"2024-09-13T16:48:09.175603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144443+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:08.144443+0000","last_peered":"2024-09-13T16:48:08.144443+0000","last_clean":"2024-09-13T16:48:08.144443+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:08.144443+0000","last_undegraded":"2024-09-13T16:48:08.144443+0000","last_fullsized":"2024-09-13T16:48:08.144443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185002+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:08.185002+0000","last_peered":"2024-09-13T16:48:08.185002+0000","last_clean":"2024-09-13T16:48:08.185002+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:08.185002+0000","last_undegraded":"2024-09-13T16:48:08.185002+0000","last_fullsized":"2024-09-13T16:48:08.185002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170142+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T16:48:08.170142+0000","last_peered":"2024-09-13T16:48:08.170142+0000","last_clean":"2024-09-13T16:48:08.170142+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T16:48:08.170142+0000","last_undegraded":"2024-09-13T16:48:08.170142+0000","last_fullsized":"2024-09-13T16:48:08.170142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177615+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T16:48:09.177615+0000","last_peered":"2024-09-13T16:48:09.177615+0000","last_clean":"2024-09-13T16:48:09.177615+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T16:48:09.177615+0000","last_undegraded":"2024-09-13T16:48:09.177615+0000","last_fullsized":"2024-09-13T16:48:09.177615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144121+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:08.144121+0000","last_peered":"2024-09-13T16:48:08.144121+0000","last_clean":"2024-09-13T16:48:08.144121+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:08.144121+0000","last_undegraded":"2024-09-13T16:48:08.144121+0000","last_fullsized":"2024-09-13T16:48:08.144121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174330+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T16:48:09.174330+0000","last_peered":"2024-09-13T16:48:09.174330+0000","last_clean":"2024-09-13T16:48:09.174330+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T16:48:09.174330+0000","last_undegraded":"2024-09-13T16:48:09.174330+0000","last_fullsized":"2024-09-13T16:48:09.174330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153996+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:08.153996+0000","last_peered":"2024-09-13T16:48:08.153996+0000","last_clean":"2024-09-13T16:48:08.153996+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:08.153996+0000","last_undegraded":"2024-09-13T16:48:08.153996+0000","last_fullsized":"2024-09-13T16:48:08.153996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161029+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T16:48:09.161029+0000","last_peered":"2024-09-13T16:48:09.161029+0000","last_clean":"2024-09-13T16:48:09.161029+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T16:48:09.161029+0000","last_undegraded":"2024-09-13T16:48:09.161029+0000","last_fullsized":"2024-09-13T16:48:09.161029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154547+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:08.154547+0000","last_peered":"2024-09-13T16:48:08.154547+0000","last_clean":"2024-09-13T16:48:08.154547+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:08.154547+0000","last_undegraded":"2024-09-13T16:48:08.154547+0000","last_fullsized":"2024-09-13T16:48:08.154547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174304+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T16:48:09.174304+0000","last_peered":"2024-09-13T16:48:09.174304+0000","last_clean":"2024-09-13T16:48:09.174304+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T16:48:09.174304+0000","last_undegraded":"2024-09-13T16:48:09.174304+0000","last_fullsized":"2024-09-13T16:48:09.174304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161339+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:08.161339+0000","last_peered":"2024-09-13T16:48:08.161339+0000","last_clean":"2024-09-13T16:48:08.161339+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:08.161339+0000","last_undegraded":"2024-09-13T16:48:08.161339+0000","last_fullsized":"2024-09-13T16:48:08.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173686+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T16:48:09.173686+0000","last_peered":"2024-09-13T16:48:09.173686+0000","last_clean":"2024-09-13T16:48:09.173686+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T16:48:09.173686+0000","last_undegraded":"2024-09-13T16:48:09.173686+0000","last_fullsized":"2024-09-13T16:48:09.173686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145104+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:08.145104+0000","last_peered":"2024-09-13T16:48:08.145104+0000","last_clean":"2024-09-13T16:48:08.145104+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:08.145104+0000","last_undegraded":"2024-09-13T16:48:08.145104+0000","last_fullsized":"2024-09-13T16:48:08.145104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175359+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T16:48:09.175359+0000","last_peered":"2024-09-13T16:48:09.175359+0000","last_clean":"2024-09-13T16:48:09.175359+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T16:48:09.175359+0000","last_undegraded":"2024-09-13T16:48:09.175359+0000","last_fullsized":"2024-09-13T16:48:09.175359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149320+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:08.149320+0000","last_peered":"2024-09-13T16:48:08.149320+0000","last_clean":"2024-09-13T16:48:08.149320+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:08.149320+0000","last_undegraded":"2024-09-13T16:48:08.149320+0000","last_fullsized":"2024-09-13T16:48:08.149320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173205+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T16:48:09.173205+0000","last_peered":"2024-09-13T16:48:09.173205+0000","last_clean":"2024-09-13T16:48:09.173205+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T16:48:09.173205+0000","last_undegraded":"2024-09-13T16:48:09.173205+0000","last_fullsized":"2024-09-13T16:48:09.173205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149001+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:08.149001+0000","last_peered":"2024-09-13T16:48:08.149001+0000","last_clean":"2024-09-13T16:48:08.149001+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:08.149001+0000","last_undegraded":"2024-09-13T16:48:08.149001+0000","last_fullsized":"2024-09-13T16:48:08.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158758+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T16:48:09.158758+0000","last_peered":"2024-09-13T16:48:09.158758+0000","last_clean":"2024-09-13T16:48:09.158758+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T16:48:09.158758+0000","last_undegraded":"2024-09-13T16:48:09.158758+0000","last_fullsized":"2024-09-13T16:48:09.158758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145105+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:08.145105+0000","last_peered":"2024-09-13T16:48:08.145105+0000","last_clean":"2024-09-13T16:48:08.145105+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:08.145105+0000","last_undegraded":"2024-09-13T16:48:08.145105+0000","last_fullsized":"2024-09-13T16:48:08.145105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177538+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T16:48:09.177538+0000","last_peered":"2024-09-13T16:48:09.177538+0000","last_clean":"2024-09-13T16:48:09.177538+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T16:48:09.177538+0000","last_undegraded":"2024-09-13T16:48:09.177538+0000","last_fullsized":"2024-09-13T16:48:09.177538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158985+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:08.158985+0000","last_peered":"2024-09-13T16:48:08.158985+0000","last_clean":"2024-09-13T16:48:08.158985+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:08.158985+0000","last_undegraded":"2024-09-13T16:48:08.158985+0000","last_fullsized":"2024-09-13T16:48:08.158985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177461+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T16:48:09.177461+0000","last_peered":"2024-09-13T16:48:09.177461+0000","last_clean":"2024-09-13T16:48:09.177461+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T16:48:09.177461+0000","last_undegraded":"2024-09-13T16:48:09.177461+0000","last_fullsized":"2024-09-13T16:48:09.177461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143902+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:08.143902+0000","last_peered":"2024-09-13T16:48:08.143902+0000","last_clean":"2024-09-13T16:48:08.143902+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:08.143902+0000","last_undegraded":"2024-09-13T16:48:08.143902+0000","last_fullsized":"2024-09-13T16:48:08.143902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159127+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T16:48:09.159127+0000","last_peered":"2024-09-13T16:48:09.159127+0000","last_clean":"2024-09-13T16:48:09.159127+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T16:48:09.159127+0000","last_undegraded":"2024-09-13T16:48:09.159127+0000","last_fullsized":"2024-09-13T16:48:09.159127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148988+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:08.148988+0000","last_peered":"2024-09-13T16:48:08.148988+0000","last_clean":"2024-09-13T16:48:08.148988+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:08.148988+0000","last_undegraded":"2024-09-13T16:48:08.148988+0000","last_fullsized":"2024-09-13T16:48:08.148988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177190+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T16:48:09.177190+0000","last_peered":"2024-09-13T16:48:09.177190+0000","last_clean":"2024-09-13T16:48:09.177190+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T16:48:09.177190+0000","last_undegraded":"2024-09-13T16:48:09.177190+0000","last_fullsized":"2024-09-13T16:48:09.177190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161327+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:08.161327+0000","last_peered":"2024-09-13T16:48:08.161327+0000","last_clean":"2024-09-13T16:48:08.161327+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:08.161327+0000","last_undegraded":"2024-09-13T16:48:08.161327+0000","last_fullsized":"2024-09-13T16:48:08.161327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148225+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:08.148225+0000","last_peered":"2024-09-13T16:48:08.148225+0000","last_clean":"2024-09-13T16:48:08.148225+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:08.148225+0000","last_undegraded":"2024-09-13T16:48:08.148225+0000","last_fullsized":"2024-09-13T16:48:08.148225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175099+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T16:48:09.175099+0000","last_peered":"2024-09-13T16:48:09.175099+0000","last_clean":"2024-09-13T16:48:09.175099+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T16:48:09.175099+0000","last_undegraded":"2024-09-13T16:48:09.175099+0000","last_fullsized":"2024-09-13T16:48:09.175099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179166+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:08.179166+0000","last_peered":"2024-09-13T16:48:08.179166+0000","last_clean":"2024-09-13T16:48:08.179166+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:08.179166+0000","last_undegraded":"2024-09-13T16:48:08.179166+0000","last_fullsized":"2024-09-13T16:48:08.179166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158364+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T16:48:09.158364+0000","last_peered":"2024-09-13T16:48:09.158364+0000","last_clean":"2024-09-13T16:48:09.158364+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T16:48:09.158364+0000","last_undegraded":"2024-09-13T16:48:09.158364+0000","last_fullsized":"2024-09-13T16:48:09.158364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155807+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:08.155807+0000","last_peered":"2024-09-13T16:48:08.155807+0000","last_clean":"2024-09-13T16:48:08.155807+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:08.155807+0000","last_undegraded":"2024-09-13T16:48:08.155807+0000","last_fullsized":"2024-09-13T16:48:08.155807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.172410+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T16:48:09.172410+0000","last_peered":"2024-09-13T16:48:09.172410+0000","last_clean":"2024-09-13T16:48:09.172410+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T16:48:09.172410+0000","last_undegraded":"2024-09-13T16:48:09.172410+0000","last_fullsized":"2024-09-13T16:48:09.172410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.157093+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:08.157093+0000","last_peered":"2024-09-13T16:48:08.157093+0000","last_clean":"2024-09-13T16:48:08.157093+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:08.157093+0000","last_undegraded":"2024-09-13T16:48:08.157093+0000","last_fullsized":"2024-09-13T16:48:08.157093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173535+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T16:48:09.173535+0000","last_peered":"2024-09-13T16:48:09.173535+0000","last_clean":"2024-09-13T16:48:09.173535+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T16:48:09.173535+0000","last_undegraded":"2024-09-13T16:48:09.173535+0000","last_fullsized":"2024-09-13T16:48:09.173535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158715+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:08.158715+0000","last_peered":"2024-09-13T16:48:08.158715+0000","last_clean":"2024-09-13T16:48:08.158715+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:08.158715+0000","last_undegraded":"2024-09-13T16:48:08.158715+0000","last_fullsized":"2024-09-13T16:48:08.158715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175097+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T16:48:09.175097+0000","last_peered":"2024-09-13T16:48:09.175097+0000","last_clean":"2024-09-13T16:48:09.175097+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T16:48:09.175097+0000","last_undegraded":"2024-09-13T16:48:09.175097+0000","last_fullsized":"2024-09-13T16:48:09.175097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147419+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:08.147419+0000","last_peered":"2024-09-13T16:48:08.147419+0000","last_clean":"2024-09-13T16:48:08.147419+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:08.147419+0000","last_undegraded":"2024-09-13T16:48:08.147419+0000","last_fullsized":"2024-09-13T16:48:08.147419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159429+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T16:48:09.159429+0000","last_peered":"2024-09-13T16:48:09.159429+0000","last_clean":"2024-09-13T16:48:09.159429+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T16:48:09.159429+0000","last_undegraded":"2024-09-13T16:48:09.159429+0000","last_fullsized":"2024-09-13T16:48:09.159429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161028+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:08.161028+0000","last_peered":"2024-09-13T16:48:08.161028+0000","last_clean":"2024-09-13T16:48:08.161028+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:08.161028+0000","last_undegraded":"2024-09-13T16:48:08.161028+0000","last_fullsized":"2024-09-13T16:48:08.161028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177095+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T16:48:09.177095+0000","last_peered":"2024-09-13T16:48:09.177095+0000","last_clean":"2024-09-13T16:48:09.177095+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T16:48:09.177095+0000","last_undegraded":"2024-09-13T16:48:09.177095+0000","last_fullsized":"2024-09-13T16:48:09.177095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143741+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:08.143741+0000","last_peered":"2024-09-13T16:48:08.143741+0000","last_clean":"2024-09-13T16:48:08.143741+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:08.143741+0000","last_undegraded":"2024-09-13T16:48:08.143741+0000","last_fullsized":"2024-09-13T16:48:08.143741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176397+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T16:48:09.176397+0000","last_peered":"2024-09-13T16:48:09.176397+0000","last_clean":"2024-09-13T16:48:09.176397+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T16:48:09.176397+0000","last_undegraded":"2024-09-13T16:48:09.176397+0000","last_fullsized":"2024-09-13T16:48:09.176397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186343+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:08.186343+0000","last_peered":"2024-09-13T16:48:08.186343+0000","last_clean":"2024-09-13T16:48:08.186343+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:08.186343+0000","last_undegraded":"2024-09-13T16:48:08.186343+0000","last_fullsized":"2024-09-13T16:48:08.186343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158832+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T16:48:09.158832+0000","last_peered":"2024-09-13T16:48:09.158832+0000","last_clean":"2024-09-13T16:48:09.158832+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T16:48:09.158832+0000","last_undegraded":"2024-09-13T16:48:09.158832+0000","last_fullsized":"2024-09-13T16:48:09.158832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149361+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:08.149361+0000","last_peered":"2024-09-13T16:48:08.149361+0000","last_clean":"2024-09-13T16:48:08.149361+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:08.149361+0000","last_undegraded":"2024-09-13T16:48:08.149361+0000","last_fullsized":"2024-09-13T16:48:08.149361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173784+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T16:48:09.173784+0000","last_peered":"2024-09-13T16:48:09.173784+0000","last_clean":"2024-09-13T16:48:09.173784+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T16:48:09.173784+0000","last_undegraded":"2024-09-13T16:48:09.173784+0000","last_fullsized":"2024-09-13T16:48:09.173784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148791+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:08.148791+0000","last_peered":"2024-09-13T16:48:08.148791+0000","last_clean":"2024-09-13T16:48:08.148791+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:08.148791+0000","last_undegraded":"2024-09-13T16:48:08.148791+0000","last_fullsized":"2024-09-13T16:48:08.148791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159787+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T16:48:09.159787+0000","last_peered":"2024-09-13T16:48:09.159787+0000","last_clean":"2024-09-13T16:48:09.159787+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T16:48:09.159787+0000","last_undegraded":"2024-09-13T16:48:09.159787+0000","last_fullsized":"2024-09-13T16:48:09.159787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166109+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:08.166109+0000","last_peered":"2024-09-13T16:48:08.166109+0000","last_clean":"2024-09-13T16:48:08.166109+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:08.166109+0000","last_undegraded":"2024-09-13T16:48:08.166109+0000","last_fullsized":"2024-09-13T16:48:08.166109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.178540+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T16:48:09.178540+0000","last_peered":"2024-09-13T16:48:09.178540+0000","last_clean":"2024-09-13T16:48:09.178540+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T16:48:09.178540+0000","last_undegraded":"2024-09-13T16:48:09.178540+0000","last_fullsized":"2024-09-13T16:48:09.178540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143140+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:08.143140+0000","last_peered":"2024-09-13T16:48:08.143140+0000","last_clean":"2024-09-13T16:48:08.143140+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:08.143140+0000","last_undegraded":"2024-09-13T16:48:08.143140+0000","last_fullsized":"2024-09-13T16:48:08.143140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.160931+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T16:48:09.160931+0000","last_peered":"2024-09-13T16:48:09.160931+0000","last_clean":"2024-09-13T16:48:09.160931+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T16:48:09.160931+0000","last_undegraded":"2024-09-13T16:48:09.160931+0000","last_fullsized":"2024-09-13T16:48:09.160931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167027+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:08.167027+0000","last_peered":"2024-09-13T16:48:08.167027+0000","last_clean":"2024-09-13T16:48:08.167027+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:08.167027+0000","last_undegraded":"2024-09-13T16:48:08.167027+0000","last_fullsized":"2024-09-13T16:48:08.167027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173532+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T16:48:09.173532+0000","last_peered":"2024-09-13T16:48:09.173532+0000","last_clean":"2024-09-13T16:48:09.173532+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T16:48:09.173532+0000","last_undegraded":"2024-09-13T16:48:09.173532+0000","last_fullsized":"2024-09-13T16:48:09.173532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148753+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:08.148753+0000","last_peered":"2024-09-13T16:48:08.148753+0000","last_clean":"2024-09-13T16:48:08.148753+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:08.148753+0000","last_undegraded":"2024-09-13T16:48:08.148753+0000","last_fullsized":"2024-09-13T16:48:08.148753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161898+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T16:48:09.161898+0000","last_peered":"2024-09-13T16:48:09.161898+0000","last_clean":"2024-09-13T16:48:09.161898+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T16:48:09.161898+0000","last_undegraded":"2024-09-13T16:48:09.161898+0000","last_fullsized":"2024-09-13T16:48:09.161898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160351+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:08.160351+0000","last_peered":"2024-09-13T16:48:08.160351+0000","last_clean":"2024-09-13T16:48:08.160351+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:08.160351+0000","last_undegraded":"2024-09-13T16:48:08.160351+0000","last_fullsized":"2024-09-13T16:48:08.160351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176220+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T16:48:09.176220+0000","last_peered":"2024-09-13T16:48:09.176220+0000","last_clean":"2024-09-13T16:48:09.176220+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T16:48:09.176220+0000","last_undegraded":"2024-09-13T16:48:09.176220+0000","last_fullsized":"2024-09-13T16:48:09.176220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151613+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:08.151613+0000","last_peered":"2024-09-13T16:48:08.151613+0000","last_clean":"2024-09-13T16:48:08.151613+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:08.151613+0000","last_undegraded":"2024-09-13T16:48:08.151613+0000","last_fullsized":"2024-09-13T16:48:08.151613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158220+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T16:48:09.158220+0000","last_peered":"2024-09-13T16:48:09.158220+0000","last_clean":"2024-09-13T16:48:09.158220+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T16:48:09.158220+0000","last_undegraded":"2024-09-13T16:48:09.158220+0000","last_fullsized":"2024-09-13T16:48:09.158220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.162959+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:08.162959+0000","last_peered":"2024-09-13T16:48:08.162959+0000","last_clean":"2024-09-13T16:48:08.162959+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:08.162959+0000","last_undegraded":"2024-09-13T16:48:08.162959+0000","last_fullsized":"2024-09-13T16:48:08.162959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173056+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T16:48:09.173056+0000","last_peered":"2024-09-13T16:48:09.173056+0000","last_clean":"2024-09-13T16:48:09.173056+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T16:48:09.173056+0000","last_undegraded":"2024-09-13T16:48:09.173056+0000","last_fullsized":"2024-09-13T16:48:09.173056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.150218+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:08.150218+0000","last_peered":"2024-09-13T16:48:08.150218+0000","last_clean":"2024-09-13T16:48:08.150218+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:08.150218+0000","last_undegraded":"2024-09-13T16:48:08.150218+0000","last_fullsized":"2024-09-13T16:48:08.150218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173906+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T16:48:09.173906+0000","last_peered":"2024-09-13T16:48:09.173906+0000","last_clean":"2024-09-13T16:48:09.173906+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T16:48:09.173906+0000","last_undegraded":"2024-09-13T16:48:09.173906+0000","last_fullsized":"2024-09-13T16:48:09.173906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166070+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:08.166070+0000","last_peered":"2024-09-13T16:48:08.166070+0000","last_clean":"2024-09-13T16:48:08.166070+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:08.166070+0000","last_undegraded":"2024-09-13T16:48:08.166070+0000","last_fullsized":"2024-09-13T16:48:08.166070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2804,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2804,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":690265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":101847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":99455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":282624,"data_stored":103259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":258048,"data_stored":99295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":101413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":690231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:41.373 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b43f66640 1 Processor -- start 2024-09-13T16:48:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b43f66640 1 -- start start 2024-09-13T16:48:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b43f66640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1b3c0a4e70 0x7f1b3c151460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b43f66640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c0c4530 0x7f1b3c0c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b43f66640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b3c154100 con 0x7f1b3c06d020 2024-09-13T16:48:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b43f66640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b3c0a8c50 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b43f66640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c1519a0 con 0x7f1b3c0c4530 2024-09-13T16:48:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b41501640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1b3c0a4e70 0x7f1b3c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b41d02640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1b3c06d020 0x7f1b3c14db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:51436/0 (socket says 172.21.15.39:51436) 2024-09-13T16:48:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b42503640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c0c4530 0x7f1b3c0c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b41d02640 1 -- 172.21.15.39:0/1263768783 learned_addr learned my addr 172.21.15.39:0/1263768783 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b41501640 1 -- 172.21.15.39:0/1263768783 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c0c4530 msgr2=0x7f1b3c0c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b41501640 1 --2- 172.21.15.39:0/1263768783 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c0c4530 0x7f1b3c0c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b41501640 1 -- 172.21.15.39:0/1263768783 >> v1:172.21.15.84:6789/0 conn(0x7f1b3c06d020 legacy=0x7f1b3c14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.531+0000 7f1b41501640 1 -- 172.21.15.39:0/1263768783 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c1543e0 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b41501640 1 --2- 172.21.15.39:0/1263768783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 0x7f1b3c151460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0097c0 tx=0x7f1b2c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a11c80542a6098 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:41.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b40d00640 1 -- 172.21.15.39:0/1263768783 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0ab600 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b40d00640 1 -- 172.21.15.39:0/1263768783 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c0a7050 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b40d00640 1 -- 172.21.15.39:0/1263768783 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0ab980 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b43f66640 1 -- 172.21.15.39:0/1263768783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 msgr2=0x7f1b3c151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b43f66640 1 --2- 172.21.15.39:0/1263768783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 0x7f1b3c151460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0097c0 tx=0x7f1b2c01c3e0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b43f66640 1 -- 172.21.15.39:0/1263768783 shutdown_connections 2024-09-13T16:48:41.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b43f66640 1 --2- 172.21.15.39:0/1263768783 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c0c4530 0x7f1b3c0c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b43f66640 1 --2- 172.21.15.39:0/1263768783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 0x7f1b3c151460 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b43f66640 1 -- 172.21.15.39:0/1263768783 >> 172.21.15.39:0/1263768783 conn(0x7f1b3c069e10 msgr2=0x7f1b3c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b43f66640 1 -- 172.21.15.39:0/1263768783 shutdown_connections 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.535+0000 7f1b43f66640 1 -- 172.21.15.39:0/1263768783 wait complete. 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 Processor -- start 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 -- start start 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c06d020 0x7f1b3c148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 0x7f1b3c14c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b3c0c4530 0x7f1b3c0a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c14c010 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c14c2b0 con 0x7f1b3c0c4530 2024-09-13T16:48:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c0a65d0 con 0x7f1b3c06d020 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41501640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 0x7f1b3c14c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41501640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 0x7f1b3c14c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57158/0 (socket says 172.21.15.39:57158) 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41501640 1 -- 172.21.15.39:0/1938505174 learned_addr learned my addr 172.21.15.39:0/1938505174 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b42503640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b3c0c4530 0x7f1b3c0a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41d02640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c06d020 0x7f1b3c148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41501640 1 -- 172.21.15.39:0/1938505174 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c06d020 msgr2=0x7f1b3c148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41501640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c06d020 0x7f1b3c148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41501640 1 -- 172.21.15.39:0/1938505174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b3c0c4530 msgr2=0x7f1b3c0a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41501640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b3c0c4530 0x7f1b3c0a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41501640 1 -- 172.21.15.39:0/1938505174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c009420 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b41501640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 0x7f1b3c14c4f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0ae040 tx=0x7f1b2c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:41.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b327fc640 1 -- 172.21.15.39:0/1938505174 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0b4430 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b327fc640 1 -- 172.21.15.39:0/1938505174 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c0b4590 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3c0a68b0 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b327fc640 1 -- 172.21.15.39:0/1938505174 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0b4940 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3c0a74a0 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3c151460 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.539+0000 7f1b327fc640 1 -- 172.21.15.39:0/1938505174 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b2c0b2040 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.543+0000 7f1b327fc640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b20066690 0x7f1b20068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.543+0000 7f1b327fc640 1 -- 172.21.15.39:0/1938505174 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f1b2c11e2a0 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.543+0000 7f1b41d02640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b20066690 0x7f1b20068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.543+0000 7f1b41d02640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b20066690 0x7f1b20068b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1b38000f70 tx=0x7f1b38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:41.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.547+0000 7f1b327fc640 1 -- 172.21.15.39:0/1938505174 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b2c0eb3b0 con 0x7f1b3c0a4e70 2024-09-13T16:48:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.703+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b3c06d400 con 0x7f1b20066690 2024-09-13T16:48:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.739+0000 7f1b327fc640 1 -- 172.21.15.39:0/1938505174 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+502912 (secure 0 0 0) 0x7f1b38001470 con 0x7f1b20066690 2024-09-13T16:48:41.752 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:41.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b20066690 msgr2=0x7f1b20068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b20066690 0x7f1b20068b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1b38000f70 tx=0x7f1b38009040 comp rx=0 tx=0).stop 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 msgr2=0x7f1b3c14c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 0x7f1b3c14c4f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0ae040 tx=0x7f1b2c019180 comp rx=0 tx=0).stop 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 shutdown_connections 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b20066690 0x7f1b20068b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b3c0c4530 0x7f1b3c0a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3c0a4e70 0x7f1b3c14c4f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 --2- 172.21.15.39:0/1938505174 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b3c06d020 0x7f1b3c148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 >> 172.21.15.39:0/1938505174 conn(0x7f1b3c069e10 msgr2=0x7f1b3c152f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 shutdown_connections 2024-09-13T16:48:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.747+0000 7f1b43f66640 1 -- 172.21.15.39:0/1938505174 wait complete. 2024-09-13T16:48:41.772 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2024-09-13T16:48:41.571289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595976,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":253,"num_write_kb":2448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235804,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738916,"statfs":{"total":773094113280,"available":772852649984,"internally_reserved":0,"allocated":3350528,"data_stored":1985752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":442,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003048"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174677+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T16:48:09.174677+0000","last_peered":"2024-09-13T16:48:09.174677+0000","last_clean":"2024-09-13T16:48:09.174677+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T16:48:09.174677+0000","last_undegraded":"2024-09-13T16:48:09.174677+0000","last_fullsized":"2024-09-13T16:48:09.174677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163864+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:08.163864+0000","last_peered":"2024-09-13T16:48:08.163864+0000","last_clean":"2024-09-13T16:48:08.163864+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:08.163864+0000","last_undegraded":"2024-09-13T16:48:08.163864+0000","last_fullsized":"2024-09-13T16:48:08.163864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146105+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:08.146105+0000","last_peered":"2024-09-13T16:48:08.146105+0000","last_clean":"2024-09-13T16:48:08.146105+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:08.146105+0000","last_undegraded":"2024-09-13T16:48:08.146105+0000","last_fullsized":"2024-09-13T16:48:08.146105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143176+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T16:48:08.143176+0000","last_peered":"2024-09-13T16:48:08.143176+0000","last_clean":"2024-09-13T16:48:08.143176+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T16:48:08.143176+0000","last_undegraded":"2024-09-13T16:48:08.143176+0000","last_fullsized":"2024-09-13T16:48:08.143176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173274+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T16:48:09.173274+0000","last_peered":"2024-09-13T16:48:09.173274+0000","last_clean":"2024-09-13T16:48:09.173274+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T16:48:09.173274+0000","last_undegraded":"2024-09-13T16:48:09.173274+0000","last_fullsized":"2024-09-13T16:48:09.173274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146246+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:08.146246+0000","last_peered":"2024-09-13T16:48:08.146246+0000","last_clean":"2024-09-13T16:48:08.146246+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:08.146246+0000","last_undegraded":"2024-09-13T16:48:08.146246+0000","last_fullsized":"2024-09-13T16:48:08.146246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163480+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:08.163480+0000","last_peered":"2024-09-13T16:48:08.163480+0000","last_clean":"2024-09-13T16:48:08.163480+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:08.163480+0000","last_undegraded":"2024-09-13T16:48:08.163480+0000","last_fullsized":"2024-09-13T16:48:08.163480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184157+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T16:48:08.184157+0000","last_peered":"2024-09-13T16:48:08.184157+0000","last_clean":"2024-09-13T16:48:08.184157+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T16:48:08.184157+0000","last_undegraded":"2024-09-13T16:48:08.184157+0000","last_fullsized":"2024-09-13T16:48:08.184157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173949+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T16:48:09.173949+0000","last_peered":"2024-09-13T16:48:09.173949+0000","last_clean":"2024-09-13T16:48:09.173949+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T16:48:09.173949+0000","last_undegraded":"2024-09-13T16:48:09.173949+0000","last_fullsized":"2024-09-13T16:48:09.173949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178923+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:08.178923+0000","last_peered":"2024-09-13T16:48:08.178923+0000","last_clean":"2024-09-13T16:48:08.178923+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:08.178923+0000","last_undegraded":"2024-09-13T16:48:08.178923+0000","last_fullsized":"2024-09-13T16:48:08.178923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151847+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:08.151847+0000","last_peered":"2024-09-13T16:48:08.151847+0000","last_clean":"2024-09-13T16:48:08.151847+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:08.151847+0000","last_undegraded":"2024-09-13T16:48:08.151847+0000","last_fullsized":"2024-09-13T16:48:08.151847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208404+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T16:48:08.208404+0000","last_peered":"2024-09-13T16:48:08.208404+0000","last_clean":"2024-09-13T16:48:08.208404+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T16:48:08.208404+0000","last_undegraded":"2024-09-13T16:48:08.208404+0000","last_fullsized":"2024-09-13T16:48:08.208404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159591+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T16:48:09.159591+0000","last_peered":"2024-09-13T16:48:09.159591+0000","last_clean":"2024-09-13T16:48:09.159591+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T16:48:09.159591+0000","last_undegraded":"2024-09-13T16:48:09.159591+0000","last_fullsized":"2024-09-13T16:48:09.159591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165397+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:08.165397+0000","last_peered":"2024-09-13T16:48:08.165397+0000","last_clean":"2024-09-13T16:48:08.165397+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:08.165397+0000","last_undegraded":"2024-09-13T16:48:08.165397+0000","last_fullsized":"2024-09-13T16:48:08.165397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153028+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:08.153028+0000","last_peered":"2024-09-13T16:48:08.153028+0000","last_clean":"2024-09-13T16:48:08.153028+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:08.153028+0000","last_undegraded":"2024-09-13T16:48:08.153028+0000","last_fullsized":"2024-09-13T16:48:08.153028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.203658+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T16:48:08.203658+0000","last_peered":"2024-09-13T16:48:08.203658+0000","last_clean":"2024-09-13T16:48:08.203658+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T16:48:08.203658+0000","last_undegraded":"2024-09-13T16:48:08.203658+0000","last_fullsized":"2024-09-13T16:48:08.203658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173632+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T16:48:09.173632+0000","last_peered":"2024-09-13T16:48:09.173632+0000","last_clean":"2024-09-13T16:48:09.173632+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T16:48:09.173632+0000","last_undegraded":"2024-09-13T16:48:09.173632+0000","last_fullsized":"2024-09-13T16:48:09.173632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146884+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:08.146884+0000","last_peered":"2024-09-13T16:48:08.146884+0000","last_clean":"2024-09-13T16:48:08.146884+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:08.146884+0000","last_undegraded":"2024-09-13T16:48:08.146884+0000","last_fullsized":"2024-09-13T16:48:08.146884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160788+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:08.160788+0000","last_peered":"2024-09-13T16:48:08.160788+0000","last_clean":"2024-09-13T16:48:08.160788+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:08.160788+0000","last_undegraded":"2024-09-13T16:48:08.160788+0000","last_fullsized":"2024-09-13T16:48:08.160788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.206623+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T16:48:08.206623+0000","last_peered":"2024-09-13T16:48:08.206623+0000","last_clean":"2024-09-13T16:48:08.206623+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T16:48:08.206623+0000","last_undegraded":"2024-09-13T16:48:08.206623+0000","last_fullsized":"2024-09-13T16:48:08.206623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159080+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T16:48:09.159080+0000","last_peered":"2024-09-13T16:48:09.159080+0000","last_clean":"2024-09-13T16:48:09.159080+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T16:48:09.159080+0000","last_undegraded":"2024-09-13T16:48:09.159080+0000","last_fullsized":"2024-09-13T16:48:09.159080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144345+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:08.144345+0000","last_peered":"2024-09-13T16:48:08.144345+0000","last_clean":"2024-09-13T16:48:08.144345+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:08.144345+0000","last_undegraded":"2024-09-13T16:48:08.144345+0000","last_fullsized":"2024-09-13T16:48:08.144345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147467+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:08.147467+0000","last_peered":"2024-09-13T16:48:08.147467+0000","last_clean":"2024-09-13T16:48:08.147467+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:08.147467+0000","last_undegraded":"2024-09-13T16:48:08.147467+0000","last_fullsized":"2024-09-13T16:48:08.147467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184081+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T16:48:08.184081+0000","last_peered":"2024-09-13T16:48:08.184081+0000","last_clean":"2024-09-13T16:48:08.184081+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T16:48:08.184081+0000","last_undegraded":"2024-09-13T16:48:08.184081+0000","last_fullsized":"2024-09-13T16:48:08.184081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143476+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:08.143476+0000","last_peered":"2024-09-13T16:48:08.143476+0000","last_clean":"2024-09-13T16:48:08.143476+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:08.143476+0000","last_undegraded":"2024-09-13T16:48:08.143476+0000","last_fullsized":"2024-09-13T16:48:08.143476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147565+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:08.147565+0000","last_peered":"2024-09-13T16:48:08.147565+0000","last_clean":"2024-09-13T16:48:08.147565+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:08.147565+0000","last_undegraded":"2024-09-13T16:48:08.147565+0000","last_fullsized":"2024-09-13T16:48:08.147565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174007+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T16:48:08.174007+0000","last_peered":"2024-09-13T16:48:08.174007+0000","last_clean":"2024-09-13T16:48:08.174007+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T16:48:08.174007+0000","last_undegraded":"2024-09-13T16:48:08.174007+0000","last_fullsized":"2024-09-13T16:48:08.174007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162482+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T16:48:09.162482+0000","last_peered":"2024-09-13T16:48:09.162482+0000","last_clean":"2024-09-13T16:48:09.162482+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T16:48:09.162482+0000","last_undegraded":"2024-09-13T16:48:09.162482+0000","last_fullsized":"2024-09-13T16:48:09.162482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186454+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:08.186454+0000","last_peered":"2024-09-13T16:48:08.186454+0000","last_clean":"2024-09-13T16:48:08.186454+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:08.186454+0000","last_undegraded":"2024-09-13T16:48:08.186454+0000","last_fullsized":"2024-09-13T16:48:08.186454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160985+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:08.160985+0000","last_peered":"2024-09-13T16:48:08.160985+0000","last_clean":"2024-09-13T16:48:08.160985+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:08.160985+0000","last_undegraded":"2024-09-13T16:48:08.160985+0000","last_fullsized":"2024-09-13T16:48:08.160985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208189+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T16:48:08.208189+0000","last_peered":"2024-09-13T16:48:08.208189+0000","last_clean":"2024-09-13T16:48:08.208189+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T16:48:08.208189+0000","last_undegraded":"2024-09-13T16:48:08.208189+0000","last_fullsized":"2024-09-13T16:48:08.208189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176059+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T16:48:09.176059+0000","last_peered":"2024-09-13T16:48:09.176059+0000","last_clean":"2024-09-13T16:48:09.176059+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T16:48:09.176059+0000","last_undegraded":"2024-09-13T16:48:09.176059+0000","last_fullsized":"2024-09-13T16:48:09.176059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163768+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:08.163768+0000","last_peered":"2024-09-13T16:48:08.163768+0000","last_clean":"2024-09-13T16:48:08.163768+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:08.163768+0000","last_undegraded":"2024-09-13T16:48:08.163768+0000","last_fullsized":"2024-09-13T16:48:08.163768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148721+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:08.148721+0000","last_peered":"2024-09-13T16:48:08.148721+0000","last_clean":"2024-09-13T16:48:08.148721+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:08.148721+0000","last_undegraded":"2024-09-13T16:48:08.148721+0000","last_fullsized":"2024-09-13T16:48:08.148721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185371+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T16:48:08.185371+0000","last_peered":"2024-09-13T16:48:08.185371+0000","last_clean":"2024-09-13T16:48:08.185371+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T16:48:08.185371+0000","last_undegraded":"2024-09-13T16:48:08.185371+0000","last_fullsized":"2024-09-13T16:48:08.185371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174416+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T16:48:09.174416+0000","last_peered":"2024-09-13T16:48:09.174416+0000","last_clean":"2024-09-13T16:48:09.174416+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T16:48:09.174416+0000","last_undegraded":"2024-09-13T16:48:09.174416+0000","last_fullsized":"2024-09-13T16:48:09.174416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145183+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:08.145183+0000","last_peered":"2024-09-13T16:48:08.145183+0000","last_clean":"2024-09-13T16:48:08.145183+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:08.145183+0000","last_undegraded":"2024-09-13T16:48:08.145183+0000","last_fullsized":"2024-09-13T16:48:08.145183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152363+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:08.152363+0000","last_peered":"2024-09-13T16:48:08.152363+0000","last_clean":"2024-09-13T16:48:08.152363+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:08.152363+0000","last_undegraded":"2024-09-13T16:48:08.152363+0000","last_fullsized":"2024-09-13T16:48:08.152363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143237+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T16:48:08.143237+0000","last_peered":"2024-09-13T16:48:08.143237+0000","last_clean":"2024-09-13T16:48:08.143237+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T16:48:08.143237+0000","last_undegraded":"2024-09-13T16:48:08.143237+0000","last_fullsized":"2024-09-13T16:48:08.143237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158858+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T16:48:09.158858+0000","last_peered":"2024-09-13T16:48:09.158858+0000","last_clean":"2024-09-13T16:48:09.158858+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T16:48:09.158858+0000","last_undegraded":"2024-09-13T16:48:09.158858+0000","last_fullsized":"2024-09-13T16:48:09.158858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148446+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:08.148446+0000","last_peered":"2024-09-13T16:48:08.148446+0000","last_clean":"2024-09-13T16:48:08.148446+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:08.148446+0000","last_undegraded":"2024-09-13T16:48:08.148446+0000","last_fullsized":"2024-09-13T16:48:08.148446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179426+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:08.179426+0000","last_peered":"2024-09-13T16:48:08.179426+0000","last_clean":"2024-09-13T16:48:08.179426+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:08.179426+0000","last_undegraded":"2024-09-13T16:48:08.179426+0000","last_fullsized":"2024-09-13T16:48:08.179426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161207+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T16:48:08.161207+0000","last_peered":"2024-09-13T16:48:08.161207+0000","last_clean":"2024-09-13T16:48:08.161207+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T16:48:08.161207+0000","last_undegraded":"2024-09-13T16:48:08.161207+0000","last_fullsized":"2024-09-13T16:48:08.161207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173006+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T16:48:09.173006+0000","last_peered":"2024-09-13T16:48:09.173006+0000","last_clean":"2024-09-13T16:48:09.173006+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T16:48:09.173006+0000","last_undegraded":"2024-09-13T16:48:09.173006+0000","last_fullsized":"2024-09-13T16:48:09.173006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153222+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:08.153222+0000","last_peered":"2024-09-13T16:48:08.153222+0000","last_clean":"2024-09-13T16:48:08.153222+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:08.153222+0000","last_undegraded":"2024-09-13T16:48:08.153222+0000","last_fullsized":"2024-09-13T16:48:08.153222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148522+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:08.148522+0000","last_peered":"2024-09-13T16:48:08.148522+0000","last_clean":"2024-09-13T16:48:08.148522+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:08.148522+0000","last_undegraded":"2024-09-13T16:48:08.148522+0000","last_fullsized":"2024-09-13T16:48:08.148522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146684+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T16:48:08.146684+0000","last_peered":"2024-09-13T16:48:08.146684+0000","last_clean":"2024-09-13T16:48:08.146684+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T16:48:08.146684+0000","last_undegraded":"2024-09-13T16:48:08.146684+0000","last_fullsized":"2024-09-13T16:48:08.146684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174867+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T16:48:09.174867+0000","last_peered":"2024-09-13T16:48:09.174867+0000","last_clean":"2024-09-13T16:48:09.174867+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T16:48:09.174867+0000","last_undegraded":"2024-09-13T16:48:09.174867+0000","last_fullsized":"2024-09-13T16:48:09.174867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185855+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:08.185855+0000","last_peered":"2024-09-13T16:48:08.185855+0000","last_clean":"2024-09-13T16:48:08.185855+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:08.185855+0000","last_undegraded":"2024-09-13T16:48:08.185855+0000","last_fullsized":"2024-09-13T16:48:08.185855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149466+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:08.149466+0000","last_peered":"2024-09-13T16:48:08.149466+0000","last_clean":"2024-09-13T16:48:08.149466+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:08.149466+0000","last_undegraded":"2024-09-13T16:48:08.149466+0000","last_fullsized":"2024-09-13T16:48:08.149466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148372+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T16:48:08.148372+0000","last_peered":"2024-09-13T16:48:08.148372+0000","last_clean":"2024-09-13T16:48:08.148372+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T16:48:08.148372+0000","last_undegraded":"2024-09-13T16:48:08.148372+0000","last_fullsized":"2024-09-13T16:48:08.148372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159186+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T16:48:09.159186+0000","last_peered":"2024-09-13T16:48:09.159186+0000","last_clean":"2024-09-13T16:48:09.159186+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T16:48:09.159186+0000","last_undegraded":"2024-09-13T16:48:09.159186+0000","last_fullsized":"2024-09-13T16:48:09.159186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146860+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:08.146860+0000","last_peered":"2024-09-13T16:48:08.146860+0000","last_clean":"2024-09-13T16:48:08.146860+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:08.146860+0000","last_undegraded":"2024-09-13T16:48:08.146860+0000","last_fullsized":"2024-09-13T16:48:08.146860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154111+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:08.154111+0000","last_peered":"2024-09-13T16:48:08.154111+0000","last_clean":"2024-09-13T16:48:08.154111+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:08.154111+0000","last_undegraded":"2024-09-13T16:48:08.154111+0000","last_fullsized":"2024-09-13T16:48:08.154111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175388+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T16:48:08.175388+0000","last_peered":"2024-09-13T16:48:08.175388+0000","last_clean":"2024-09-13T16:48:08.175388+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T16:48:08.175388+0000","last_undegraded":"2024-09-13T16:48:08.175388+0000","last_fullsized":"2024-09-13T16:48:08.175388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162556+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T16:48:09.162556+0000","last_peered":"2024-09-13T16:48:09.162556+0000","last_clean":"2024-09-13T16:48:09.162556+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T16:48:09.162556+0000","last_undegraded":"2024-09-13T16:48:09.162556+0000","last_fullsized":"2024-09-13T16:48:09.162556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160990+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:08.160990+0000","last_peered":"2024-09-13T16:48:08.160990+0000","last_clean":"2024-09-13T16:48:08.160990+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:08.160990+0000","last_undegraded":"2024-09-13T16:48:08.160990+0000","last_fullsized":"2024-09-13T16:48:08.160990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161254+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:08.161254+0000","last_peered":"2024-09-13T16:48:08.161254+0000","last_clean":"2024-09-13T16:48:08.161254+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:08.161254+0000","last_undegraded":"2024-09-13T16:48:08.161254+0000","last_fullsized":"2024-09-13T16:48:08.161254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175218+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T16:48:09.175218+0000","last_peered":"2024-09-13T16:48:09.175218+0000","last_clean":"2024-09-13T16:48:09.175218+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T16:48:09.175218+0000","last_undegraded":"2024-09-13T16:48:09.175218+0000","last_fullsized":"2024-09-13T16:48:09.175218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175463+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T16:48:09.175463+0000","last_peered":"2024-09-13T16:48:09.175463+0000","last_clean":"2024-09-13T16:48:09.175463+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T16:48:09.175463+0000","last_undegraded":"2024-09-13T16:48:09.175463+0000","last_fullsized":"2024-09-13T16:48:09.175463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143792+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:08.143792+0000","last_peered":"2024-09-13T16:48:08.143792+0000","last_clean":"2024-09-13T16:48:08.143792+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:08.143792+0000","last_undegraded":"2024-09-13T16:48:08.143792+0000","last_fullsized":"2024-09-13T16:48:08.143792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148284+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:08.148284+0000","last_peered":"2024-09-13T16:48:08.148284+0000","last_clean":"2024-09-13T16:48:08.148284+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:08.148284+0000","last_undegraded":"2024-09-13T16:48:08.148284+0000","last_fullsized":"2024-09-13T16:48:08.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161478+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:08.161478+0000","last_peered":"2024-09-13T16:48:08.161478+0000","last_clean":"2024-09-13T16:48:08.161478+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:08.161478+0000","last_undegraded":"2024-09-13T16:48:08.161478+0000","last_fullsized":"2024-09-13T16:48:08.161478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148905+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T16:48:08.148905+0000","last_peered":"2024-09-13T16:48:08.148905+0000","last_clean":"2024-09-13T16:48:08.148905+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T16:48:08.148905+0000","last_undegraded":"2024-09-13T16:48:08.148905+0000","last_fullsized":"2024-09-13T16:48:08.148905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158934+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T16:48:09.158934+0000","last_peered":"2024-09-13T16:48:09.158934+0000","last_clean":"2024-09-13T16:48:09.158934+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T16:48:09.158934+0000","last_undegraded":"2024-09-13T16:48:09.158934+0000","last_fullsized":"2024-09-13T16:48:09.158934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178617+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:08.178617+0000","last_peered":"2024-09-13T16:48:08.178617+0000","last_clean":"2024-09-13T16:48:08.178617+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:08.178617+0000","last_undegraded":"2024-09-13T16:48:08.178617+0000","last_fullsized":"2024-09-13T16:48:08.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155913+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:08.155913+0000","last_peered":"2024-09-13T16:48:08.155913+0000","last_clean":"2024-09-13T16:48:08.155913+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:08.155913+0000","last_undegraded":"2024-09-13T16:48:08.155913+0000","last_fullsized":"2024-09-13T16:48:08.155913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146319+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T16:48:08.146319+0000","last_peered":"2024-09-13T16:48:08.146319+0000","last_clean":"2024-09-13T16:48:08.146319+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T16:48:08.146319+0000","last_undegraded":"2024-09-13T16:48:08.146319+0000","last_fullsized":"2024-09-13T16:48:08.146319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158609+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T16:48:09.158609+0000","last_peered":"2024-09-13T16:48:09.158609+0000","last_clean":"2024-09-13T16:48:09.158609+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T16:48:09.158609+0000","last_undegraded":"2024-09-13T16:48:09.158609+0000","last_fullsized":"2024-09-13T16:48:09.158609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152057+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:08.152057+0000","last_peered":"2024-09-13T16:48:08.152057+0000","last_clean":"2024-09-13T16:48:08.152057+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:08.152057+0000","last_undegraded":"2024-09-13T16:48:08.152057+0000","last_fullsized":"2024-09-13T16:48:08.152057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149487+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:08.149487+0000","last_peered":"2024-09-13T16:48:08.149487+0000","last_clean":"2024-09-13T16:48:08.149487+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:08.149487+0000","last_undegraded":"2024-09-13T16:48:08.149487+0000","last_fullsized":"2024-09-13T16:48:08.149487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175317+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T16:48:08.175317+0000","last_peered":"2024-09-13T16:48:08.175317+0000","last_clean":"2024-09-13T16:48:08.175317+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T16:48:08.175317+0000","last_undegraded":"2024-09-13T16:48:08.175317+0000","last_fullsized":"2024-09-13T16:48:08.175317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173209+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T16:48:09.173209+0000","last_peered":"2024-09-13T16:48:09.173209+0000","last_clean":"2024-09-13T16:48:09.173209+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T16:48:09.173209+0000","last_undegraded":"2024-09-13T16:48:09.173209+0000","last_fullsized":"2024-09-13T16:48:09.173209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.177954+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:08.177954+0000","last_peered":"2024-09-13T16:48:08.177954+0000","last_clean":"2024-09-13T16:48:08.177954+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:08.177954+0000","last_undegraded":"2024-09-13T16:48:08.177954+0000","last_fullsized":"2024-09-13T16:48:08.177954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155936+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:08.155936+0000","last_peered":"2024-09-13T16:48:08.155936+0000","last_clean":"2024-09-13T16:48:08.155936+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:08.155936+0000","last_undegraded":"2024-09-13T16:48:08.155936+0000","last_fullsized":"2024-09-13T16:48:08.155936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158692+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T16:48:08.158692+0000","last_peered":"2024-09-13T16:48:08.158692+0000","last_clean":"2024-09-13T16:48:08.158692+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T16:48:08.158692+0000","last_undegraded":"2024-09-13T16:48:08.158692+0000","last_fullsized":"2024-09-13T16:48:08.158692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158034+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T16:48:09.158034+0000","last_peered":"2024-09-13T16:48:09.158034+0000","last_clean":"2024-09-13T16:48:09.158034+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T16:48:09.158034+0000","last_undegraded":"2024-09-13T16:48:09.158034+0000","last_fullsized":"2024-09-13T16:48:09.158034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160811+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:08.160811+0000","last_peered":"2024-09-13T16:48:08.160811+0000","last_clean":"2024-09-13T16:48:08.160811+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:08.160811+0000","last_undegraded":"2024-09-13T16:48:08.160811+0000","last_fullsized":"2024-09-13T16:48:08.160811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.192267+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:08.192267+0000","last_peered":"2024-09-13T16:48:08.192267+0000","last_clean":"2024-09-13T16:48:08.192267+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:08.192267+0000","last_undegraded":"2024-09-13T16:48:08.192267+0000","last_fullsized":"2024-09-13T16:48:08.192267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185117+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T16:48:08.185117+0000","last_peered":"2024-09-13T16:48:08.185117+0000","last_clean":"2024-09-13T16:48:08.185117+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T16:48:08.185117+0000","last_undegraded":"2024-09-13T16:48:08.185117+0000","last_fullsized":"2024-09-13T16:48:08.185117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177888+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T16:48:09.177888+0000","last_peered":"2024-09-13T16:48:09.177888+0000","last_clean":"2024-09-13T16:48:09.177888+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T16:48:09.177888+0000","last_undegraded":"2024-09-13T16:48:09.177888+0000","last_fullsized":"2024-09-13T16:48:09.177888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167237+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:08.167237+0000","last_peered":"2024-09-13T16:48:08.167237+0000","last_clean":"2024-09-13T16:48:08.167237+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:08.167237+0000","last_undegraded":"2024-09-13T16:48:08.167237+0000","last_fullsized":"2024-09-13T16:48:08.167237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156043+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:08.156043+0000","last_peered":"2024-09-13T16:48:08.156043+0000","last_clean":"2024-09-13T16:48:08.156043+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:08.156043+0000","last_undegraded":"2024-09-13T16:48:08.156043+0000","last_fullsized":"2024-09-13T16:48:08.156043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163690+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T16:48:08.163690+0000","last_peered":"2024-09-13T16:48:08.163690+0000","last_clean":"2024-09-13T16:48:08.163690+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T16:48:08.163690+0000","last_undegraded":"2024-09-13T16:48:08.163690+0000","last_fullsized":"2024-09-13T16:48:08.163690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162212+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T16:48:09.162212+0000","last_peered":"2024-09-13T16:48:09.162212+0000","last_clean":"2024-09-13T16:48:09.162212+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T16:48:09.162212+0000","last_undegraded":"2024-09-13T16:48:09.162212+0000","last_fullsized":"2024-09-13T16:48:09.162212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144923+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:08.144923+0000","last_peered":"2024-09-13T16:48:08.144923+0000","last_clean":"2024-09-13T16:48:08.144923+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:08.144923+0000","last_undegraded":"2024-09-13T16:48:08.144923+0000","last_fullsized":"2024-09-13T16:48:08.144923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145599+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:08.145599+0000","last_peered":"2024-09-13T16:48:08.145599+0000","last_clean":"2024-09-13T16:48:08.145599+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:08.145599+0000","last_undegraded":"2024-09-13T16:48:08.145599+0000","last_fullsized":"2024-09-13T16:48:08.145599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145022+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T16:48:08.145022+0000","last_peered":"2024-09-13T16:48:08.145022+0000","last_clean":"2024-09-13T16:48:08.145022+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T16:48:08.145022+0000","last_undegraded":"2024-09-13T16:48:08.145022+0000","last_fullsized":"2024-09-13T16:48:08.145022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161391+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T16:48:09.161391+0000","last_peered":"2024-09-13T16:48:09.161391+0000","last_clean":"2024-09-13T16:48:09.161391+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T16:48:09.161391+0000","last_undegraded":"2024-09-13T16:48:09.161391+0000","last_fullsized":"2024-09-13T16:48:09.161391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160556+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:08.160556+0000","last_peered":"2024-09-13T16:48:08.160556+0000","last_clean":"2024-09-13T16:48:08.160556+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:08.160556+0000","last_undegraded":"2024-09-13T16:48:08.160556+0000","last_fullsized":"2024-09-13T16:48:08.160556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149235+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:08.149235+0000","last_peered":"2024-09-13T16:48:08.149235+0000","last_clean":"2024-09-13T16:48:08.149235+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:08.149235+0000","last_undegraded":"2024-09-13T16:48:08.149235+0000","last_fullsized":"2024-09-13T16:48:08.149235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"24'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:35.040239+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T16:48:35.040239+0000","last_peered":"2024-09-13T16:48:35.040239+0000","last_clean":"2024-09-13T16:48:35.040239+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T16:48:35.040239+0000","last_undegraded":"2024-09-13T16:48:35.040239+0000","last_fullsized":"2024-09-13T16:48:35.040239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2006,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161110+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:08.161110+0000","last_peered":"2024-09-13T16:48:08.161110+0000","last_clean":"2024-09-13T16:48:08.161110+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:08.161110+0000","last_undegraded":"2024-09-13T16:48:08.161110+0000","last_fullsized":"2024-09-13T16:48:08.161110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161421+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:08.161421+0000","last_peered":"2024-09-13T16:48:08.161421+0000","last_clean":"2024-09-13T16:48:08.161421+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:08.161421+0000","last_undegraded":"2024-09-13T16:48:08.161421+0000","last_fullsized":"2024-09-13T16:48:08.161421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.188902+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T16:48:08.188902+0000","last_peered":"2024-09-13T16:48:08.188902+0000","last_clean":"2024-09-13T16:48:08.188902+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T16:48:08.188902+0000","last_undegraded":"2024-09-13T16:48:08.188902+0000","last_fullsized":"2024-09-13T16:48:08.188902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165383+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:08.165383+0000","last_peered":"2024-09-13T16:48:08.165383+0000","last_clean":"2024-09-13T16:48:08.165383+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:08.165383+0000","last_undegraded":"2024-09-13T16:48:08.165383+0000","last_fullsized":"2024-09-13T16:48:08.165383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:31.963081+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:31.963081+0000","last_peered":"2024-09-13T16:48:31.963081+0000","last_clean":"2024-09-13T16:48:31.963081+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:31.963081+0000","last_undegraded":"2024-09-13T16:48:31.963081+0000","last_fullsized":"2024-09-13T16:48:31.963081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2006,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146582+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T16:48:08.146582+0000","last_peered":"2024-09-13T16:48:08.146582+0000","last_clean":"2024-09-13T16:48:08.146582+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T16:48:08.146582+0000","last_undegraded":"2024-09-13T16:48:08.146582+0000","last_fullsized":"2024-09-13T16:48:08.146582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173477+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T16:48:09.173477+0000","last_peered":"2024-09-13T16:48:09.173477+0000","last_clean":"2024-09-13T16:48:09.173477+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T16:48:09.173477+0000","last_undegraded":"2024-09-13T16:48:09.173477+0000","last_fullsized":"2024-09-13T16:48:09.173477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.190852+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:08.190852+0000","last_peered":"2024-09-13T16:48:08.190852+0000","last_clean":"2024-09-13T16:48:08.190852+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:08.190852+0000","last_undegraded":"2024-09-13T16:48:08.190852+0000","last_fullsized":"2024-09-13T16:48:08.190852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149403+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:08.149403+0000","last_peered":"2024-09-13T16:48:08.149403+0000","last_clean":"2024-09-13T16:48:08.149403+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:08.149403+0000","last_undegraded":"2024-09-13T16:48:08.149403+0000","last_fullsized":"2024-09-13T16:48:08.149403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208558+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T16:48:08.208558+0000","last_peered":"2024-09-13T16:48:08.208558+0000","last_clean":"2024-09-13T16:48:08.208558+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T16:48:08.208558+0000","last_undegraded":"2024-09-13T16:48:08.208558+0000","last_fullsized":"2024-09-13T16:48:08.208558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158601+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T16:48:09.158601+0000","last_peered":"2024-09-13T16:48:09.158601+0000","last_clean":"2024-09-13T16:48:09.158601+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T16:48:09.158601+0000","last_undegraded":"2024-09-13T16:48:09.158601+0000","last_fullsized":"2024-09-13T16:48:09.158601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163596+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:08.163596+0000","last_peered":"2024-09-13T16:48:08.163596+0000","last_clean":"2024-09-13T16:48:08.163596+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:08.163596+0000","last_undegraded":"2024-09-13T16:48:08.163596+0000","last_fullsized":"2024-09-13T16:48:08.163596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156189+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:08.156189+0000","last_peered":"2024-09-13T16:48:08.156189+0000","last_clean":"2024-09-13T16:48:08.156189+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:08.156189+0000","last_undegraded":"2024-09-13T16:48:08.156189+0000","last_fullsized":"2024-09-13T16:48:08.156189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174258+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T16:48:08.174258+0000","last_peered":"2024-09-13T16:48:08.174258+0000","last_clean":"2024-09-13T16:48:08.174258+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T16:48:08.174258+0000","last_undegraded":"2024-09-13T16:48:08.174258+0000","last_fullsized":"2024-09-13T16:48:08.174258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173305+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T16:48:09.173305+0000","last_peered":"2024-09-13T16:48:09.173305+0000","last_clean":"2024-09-13T16:48:09.173305+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T16:48:09.173305+0000","last_undegraded":"2024-09-13T16:48:09.173305+0000","last_fullsized":"2024-09-13T16:48:09.173305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155963+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:08.155963+0000","last_peered":"2024-09-13T16:48:08.155963+0000","last_clean":"2024-09-13T16:48:08.155963+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:08.155963+0000","last_undegraded":"2024-09-13T16:48:08.155963+0000","last_fullsized":"2024-09-13T16:48:08.155963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148500+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:08.148500+0000","last_peered":"2024-09-13T16:48:08.148500+0000","last_clean":"2024-09-13T16:48:08.148500+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:08.148500+0000","last_undegraded":"2024-09-13T16:48:08.148500+0000","last_fullsized":"2024-09-13T16:48:08.148500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143144+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T16:48:08.143144+0000","last_peered":"2024-09-13T16:48:08.143144+0000","last_clean":"2024-09-13T16:48:08.143144+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T16:48:08.143144+0000","last_undegraded":"2024-09-13T16:48:08.143144+0000","last_fullsized":"2024-09-13T16:48:08.143144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161066+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T16:48:09.161066+0000","last_peered":"2024-09-13T16:48:09.161066+0000","last_clean":"2024-09-13T16:48:09.161066+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T16:48:09.161066+0000","last_undegraded":"2024-09-13T16:48:09.161066+0000","last_fullsized":"2024-09-13T16:48:09.161066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186757+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T16:48:08.186757+0000","last_peered":"2024-09-13T16:48:08.186757+0000","last_clean":"2024-09-13T16:48:08.186757+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T16:48:08.186757+0000","last_undegraded":"2024-09-13T16:48:08.186757+0000","last_fullsized":"2024-09-13T16:48:08.186757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147291+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:08.147291+0000","last_peered":"2024-09-13T16:48:08.147291+0000","last_clean":"2024-09-13T16:48:08.147291+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:08.147291+0000","last_undegraded":"2024-09-13T16:48:08.147291+0000","last_fullsized":"2024-09-13T16:48:08.147291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158941+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:08.158941+0000","last_peered":"2024-09-13T16:48:08.158941+0000","last_clean":"2024-09-13T16:48:08.158941+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:08.158941+0000","last_undegraded":"2024-09-13T16:48:08.158941+0000","last_fullsized":"2024-09-13T16:48:08.158941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175068+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T16:48:09.175068+0000","last_peered":"2024-09-13T16:48:09.175068+0000","last_clean":"2024-09-13T16:48:09.175068+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T16:48:09.175068+0000","last_undegraded":"2024-09-13T16:48:09.175068+0000","last_fullsized":"2024-09-13T16:48:09.175068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.182997+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T16:48:08.182997+0000","last_peered":"2024-09-13T16:48:08.182997+0000","last_clean":"2024-09-13T16:48:08.182997+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T16:48:08.182997+0000","last_undegraded":"2024-09-13T16:48:08.182997+0000","last_fullsized":"2024-09-13T16:48:08.182997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143381+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:08.143381+0000","last_peered":"2024-09-13T16:48:08.143381+0000","last_clean":"2024-09-13T16:48:08.143381+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:08.143381+0000","last_undegraded":"2024-09-13T16:48:08.143381+0000","last_fullsized":"2024-09-13T16:48:08.143381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143921+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:08.143921+0000","last_peered":"2024-09-13T16:48:08.143921+0000","last_clean":"2024-09-13T16:48:08.143921+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:08.143921+0000","last_undegraded":"2024-09-13T16:48:08.143921+0000","last_fullsized":"2024-09-13T16:48:08.143921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174477+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T16:48:09.174477+0000","last_peered":"2024-09-13T16:48:09.174477+0000","last_clean":"2024-09-13T16:48:09.174477+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T16:48:09.174477+0000","last_undegraded":"2024-09-13T16:48:09.174477+0000","last_fullsized":"2024-09-13T16:48:09.174477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147114+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:08.147114+0000","last_peered":"2024-09-13T16:48:08.147114+0000","last_clean":"2024-09-13T16:48:08.147114+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:08.147114+0000","last_undegraded":"2024-09-13T16:48:08.147114+0000","last_fullsized":"2024-09-13T16:48:08.147114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161410+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:08.161410+0000","last_peered":"2024-09-13T16:48:08.161410+0000","last_clean":"2024-09-13T16:48:08.161410+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:08.161410+0000","last_undegraded":"2024-09-13T16:48:08.161410+0000","last_fullsized":"2024-09-13T16:48:08.161410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170387+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T16:48:08.170387+0000","last_peered":"2024-09-13T16:48:08.170387+0000","last_clean":"2024-09-13T16:48:08.170387+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T16:48:08.170387+0000","last_undegraded":"2024-09-13T16:48:08.170387+0000","last_fullsized":"2024-09-13T16:48:08.170387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175603+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T16:48:09.175603+0000","last_peered":"2024-09-13T16:48:09.175603+0000","last_clean":"2024-09-13T16:48:09.175603+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T16:48:09.175603+0000","last_undegraded":"2024-09-13T16:48:09.175603+0000","last_fullsized":"2024-09-13T16:48:09.175603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144443+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:08.144443+0000","last_peered":"2024-09-13T16:48:08.144443+0000","last_clean":"2024-09-13T16:48:08.144443+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:08.144443+0000","last_undegraded":"2024-09-13T16:48:08.144443+0000","last_fullsized":"2024-09-13T16:48:08.144443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185002+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:08.185002+0000","last_peered":"2024-09-13T16:48:08.185002+0000","last_clean":"2024-09-13T16:48:08.185002+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:08.185002+0000","last_undegraded":"2024-09-13T16:48:08.185002+0000","last_fullsized":"2024-09-13T16:48:08.185002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170142+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T16:48:08.170142+0000","last_peered":"2024-09-13T16:48:08.170142+0000","last_clean":"2024-09-13T16:48:08.170142+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T16:48:08.170142+0000","last_undegraded":"2024-09-13T16:48:08.170142+0000","last_fullsized":"2024-09-13T16:48:08.170142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177615+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T16:48:09.177615+0000","last_peered":"2024-09-13T16:48:09.177615+0000","last_clean":"2024-09-13T16:48:09.177615+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T16:48:09.177615+0000","last_undegraded":"2024-09-13T16:48:09.177615+0000","last_fullsized":"2024-09-13T16:48:09.177615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144121+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:08.144121+0000","last_peered":"2024-09-13T16:48:08.144121+0000","last_clean":"2024-09-13T16:48:08.144121+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:08.144121+0000","last_undegraded":"2024-09-13T16:48:08.144121+0000","last_fullsized":"2024-09-13T16:48:08.144121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174330+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T16:48:09.174330+0000","last_peered":"2024-09-13T16:48:09.174330+0000","last_clean":"2024-09-13T16:48:09.174330+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T16:48:09.174330+0000","last_undegraded":"2024-09-13T16:48:09.174330+0000","last_fullsized":"2024-09-13T16:48:09.174330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153996+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:08.153996+0000","last_peered":"2024-09-13T16:48:08.153996+0000","last_clean":"2024-09-13T16:48:08.153996+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:08.153996+0000","last_undegraded":"2024-09-13T16:48:08.153996+0000","last_fullsized":"2024-09-13T16:48:08.153996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161029+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T16:48:09.161029+0000","last_peered":"2024-09-13T16:48:09.161029+0000","last_clean":"2024-09-13T16:48:09.161029+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T16:48:09.161029+0000","last_undegraded":"2024-09-13T16:48:09.161029+0000","last_fullsized":"2024-09-13T16:48:09.161029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154547+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:08.154547+0000","last_peered":"2024-09-13T16:48:08.154547+0000","last_clean":"2024-09-13T16:48:08.154547+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:08.154547+0000","last_undegraded":"2024-09-13T16:48:08.154547+0000","last_fullsized":"2024-09-13T16:48:08.154547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174304+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T16:48:09.174304+0000","last_peered":"2024-09-13T16:48:09.174304+0000","last_clean":"2024-09-13T16:48:09.174304+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T16:48:09.174304+0000","last_undegraded":"2024-09-13T16:48:09.174304+0000","last_fullsized":"2024-09-13T16:48:09.174304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161339+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:08.161339+0000","last_peered":"2024-09-13T16:48:08.161339+0000","last_clean":"2024-09-13T16:48:08.161339+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:08.161339+0000","last_undegraded":"2024-09-13T16:48:08.161339+0000","last_fullsized":"2024-09-13T16:48:08.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173686+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T16:48:09.173686+0000","last_peered":"2024-09-13T16:48:09.173686+0000","last_clean":"2024-09-13T16:48:09.173686+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T16:48:09.173686+0000","last_undegraded":"2024-09-13T16:48:09.173686+0000","last_fullsized":"2024-09-13T16:48:09.173686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145104+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:08.145104+0000","last_peered":"2024-09-13T16:48:08.145104+0000","last_clean":"2024-09-13T16:48:08.145104+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:08.145104+0000","last_undegraded":"2024-09-13T16:48:08.145104+0000","last_fullsized":"2024-09-13T16:48:08.145104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175359+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T16:48:09.175359+0000","last_peered":"2024-09-13T16:48:09.175359+0000","last_clean":"2024-09-13T16:48:09.175359+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T16:48:09.175359+0000","last_undegraded":"2024-09-13T16:48:09.175359+0000","last_fullsized":"2024-09-13T16:48:09.175359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149320+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:08.149320+0000","last_peered":"2024-09-13T16:48:08.149320+0000","last_clean":"2024-09-13T16:48:08.149320+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:08.149320+0000","last_undegraded":"2024-09-13T16:48:08.149320+0000","last_fullsized":"2024-09-13T16:48:08.149320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173205+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T16:48:09.173205+0000","last_peered":"2024-09-13T16:48:09.173205+0000","last_clean":"2024-09-13T16:48:09.173205+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T16:48:09.173205+0000","last_undegraded":"2024-09-13T16:48:09.173205+0000","last_fullsized":"2024-09-13T16:48:09.173205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149001+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:08.149001+0000","last_peered":"2024-09-13T16:48:08.149001+0000","last_clean":"2024-09-13T16:48:08.149001+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:08.149001+0000","last_undegraded":"2024-09-13T16:48:08.149001+0000","last_fullsized":"2024-09-13T16:48:08.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158758+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T16:48:09.158758+0000","last_peered":"2024-09-13T16:48:09.158758+0000","last_clean":"2024-09-13T16:48:09.158758+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T16:48:09.158758+0000","last_undegraded":"2024-09-13T16:48:09.158758+0000","last_fullsized":"2024-09-13T16:48:09.158758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145105+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:08.145105+0000","last_peered":"2024-09-13T16:48:08.145105+0000","last_clean":"2024-09-13T16:48:08.145105+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:08.145105+0000","last_undegraded":"2024-09-13T16:48:08.145105+0000","last_fullsized":"2024-09-13T16:48:08.145105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177538+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T16:48:09.177538+0000","last_peered":"2024-09-13T16:48:09.177538+0000","last_clean":"2024-09-13T16:48:09.177538+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T16:48:09.177538+0000","last_undegraded":"2024-09-13T16:48:09.177538+0000","last_fullsized":"2024-09-13T16:48:09.177538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158985+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:08.158985+0000","last_peered":"2024-09-13T16:48:08.158985+0000","last_clean":"2024-09-13T16:48:08.158985+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:08.158985+0000","last_undegraded":"2024-09-13T16:48:08.158985+0000","last_fullsized":"2024-09-13T16:48:08.158985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177461+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T16:48:09.177461+0000","last_peered":"2024-09-13T16:48:09.177461+0000","last_clean":"2024-09-13T16:48:09.177461+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T16:48:09.177461+0000","last_undegraded":"2024-09-13T16:48:09.177461+0000","last_fullsized":"2024-09-13T16:48:09.177461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143902+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:08.143902+0000","last_peered":"2024-09-13T16:48:08.143902+0000","last_clean":"2024-09-13T16:48:08.143902+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:08.143902+0000","last_undegraded":"2024-09-13T16:48:08.143902+0000","last_fullsized":"2024-09-13T16:48:08.143902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159127+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T16:48:09.159127+0000","last_peered":"2024-09-13T16:48:09.159127+0000","last_clean":"2024-09-13T16:48:09.159127+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T16:48:09.159127+0000","last_undegraded":"2024-09-13T16:48:09.159127+0000","last_fullsized":"2024-09-13T16:48:09.159127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148988+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:08.148988+0000","last_peered":"2024-09-13T16:48:08.148988+0000","last_clean":"2024-09-13T16:48:08.148988+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:08.148988+0000","last_undegraded":"2024-09-13T16:48:08.148988+0000","last_fullsized":"2024-09-13T16:48:08.148988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177190+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T16:48:09.177190+0000","last_peered":"2024-09-13T16:48:09.177190+0000","last_clean":"2024-09-13T16:48:09.177190+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T16:48:09.177190+0000","last_undegraded":"2024-09-13T16:48:09.177190+0000","last_fullsized":"2024-09-13T16:48:09.177190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161327+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:08.161327+0000","last_peered":"2024-09-13T16:48:08.161327+0000","last_clean":"2024-09-13T16:48:08.161327+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:08.161327+0000","last_undegraded":"2024-09-13T16:48:08.161327+0000","last_fullsized":"2024-09-13T16:48:08.161327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148225+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:08.148225+0000","last_peered":"2024-09-13T16:48:08.148225+0000","last_clean":"2024-09-13T16:48:08.148225+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:08.148225+0000","last_undegraded":"2024-09-13T16:48:08.148225+0000","last_fullsized":"2024-09-13T16:48:08.148225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175099+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T16:48:09.175099+0000","last_peered":"2024-09-13T16:48:09.175099+0000","last_clean":"2024-09-13T16:48:09.175099+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T16:48:09.175099+0000","last_undegraded":"2024-09-13T16:48:09.175099+0000","last_fullsized":"2024-09-13T16:48:09.175099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179166+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:08.179166+0000","last_peered":"2024-09-13T16:48:08.179166+0000","last_clean":"2024-09-13T16:48:08.179166+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:08.179166+0000","last_undegraded":"2024-09-13T16:48:08.179166+0000","last_fullsized":"2024-09-13T16:48:08.179166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158364+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T16:48:09.158364+0000","last_peered":"2024-09-13T16:48:09.158364+0000","last_clean":"2024-09-13T16:48:09.158364+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T16:48:09.158364+0000","last_undegraded":"2024-09-13T16:48:09.158364+0000","last_fullsized":"2024-09-13T16:48:09.158364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155807+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:08.155807+0000","last_peered":"2024-09-13T16:48:08.155807+0000","last_clean":"2024-09-13T16:48:08.155807+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:08.155807+0000","last_undegraded":"2024-09-13T16:48:08.155807+0000","last_fullsized":"2024-09-13T16:48:08.155807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.172410+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T16:48:09.172410+0000","last_peered":"2024-09-13T16:48:09.172410+0000","last_clean":"2024-09-13T16:48:09.172410+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T16:48:09.172410+0000","last_undegraded":"2024-09-13T16:48:09.172410+0000","last_fullsized":"2024-09-13T16:48:09.172410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.157093+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:08.157093+0000","last_peered":"2024-09-13T16:48:08.157093+0000","last_clean":"2024-09-13T16:48:08.157093+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:08.157093+0000","last_undegraded":"2024-09-13T16:48:08.157093+0000","last_fullsized":"2024-09-13T16:48:08.157093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173535+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T16:48:09.173535+0000","last_peered":"2024-09-13T16:48:09.173535+0000","last_clean":"2024-09-13T16:48:09.173535+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T16:48:09.173535+0000","last_undegraded":"2024-09-13T16:48:09.173535+0000","last_fullsized":"2024-09-13T16:48:09.173535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158715+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:08.158715+0000","last_peered":"2024-09-13T16:48:08.158715+0000","last_clean":"2024-09-13T16:48:08.158715+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:08.158715+0000","last_undegraded":"2024-09-13T16:48:08.158715+0000","last_fullsized":"2024-09-13T16:48:08.158715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175097+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T16:48:09.175097+0000","last_peered":"2024-09-13T16:48:09.175097+0000","last_clean":"2024-09-13T16:48:09.175097+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T16:48:09.175097+0000","last_undegraded":"2024-09-13T16:48:09.175097+0000","last_fullsized":"2024-09-13T16:48:09.175097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147419+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:08.147419+0000","last_peered":"2024-09-13T16:48:08.147419+0000","last_clean":"2024-09-13T16:48:08.147419+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:08.147419+0000","last_undegraded":"2024-09-13T16:48:08.147419+0000","last_fullsized":"2024-09-13T16:48:08.147419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159429+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T16:48:09.159429+0000","last_peered":"2024-09-13T16:48:09.159429+0000","last_clean":"2024-09-13T16:48:09.159429+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T16:48:09.159429+0000","last_undegraded":"2024-09-13T16:48:09.159429+0000","last_fullsized":"2024-09-13T16:48:09.159429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161028+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:08.161028+0000","last_peered":"2024-09-13T16:48:08.161028+0000","last_clean":"2024-09-13T16:48:08.161028+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:08.161028+0000","last_undegraded":"2024-09-13T16:48:08.161028+0000","last_fullsized":"2024-09-13T16:48:08.161028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177095+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T16:48:09.177095+0000","last_peered":"2024-09-13T16:48:09.177095+0000","last_clean":"2024-09-13T16:48:09.177095+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T16:48:09.177095+0000","last_undegraded":"2024-09-13T16:48:09.177095+0000","last_fullsized":"2024-09-13T16:48:09.177095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143741+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:08.143741+0000","last_peered":"2024-09-13T16:48:08.143741+0000","last_clean":"2024-09-13T16:48:08.143741+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:08.143741+0000","last_undegraded":"2024-09-13T16:48:08.143741+0000","last_fullsized":"2024-09-13T16:48:08.143741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176397+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T16:48:09.176397+0000","last_peered":"2024-09-13T16:48:09.176397+0000","last_clean":"2024-09-13T16:48:09.176397+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T16:48:09.176397+0000","last_undegraded":"2024-09-13T16:48:09.176397+0000","last_fullsized":"2024-09-13T16:48:09.176397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186343+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:08.186343+0000","last_peered":"2024-09-13T16:48:08.186343+0000","last_clean":"2024-09-13T16:48:08.186343+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:08.186343+0000","last_undegraded":"2024-09-13T16:48:08.186343+0000","last_fullsized":"2024-09-13T16:48:08.186343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158832+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T16:48:09.158832+0000","last_peered":"2024-09-13T16:48:09.158832+0000","last_clean":"2024-09-13T16:48:09.158832+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T16:48:09.158832+0000","last_undegraded":"2024-09-13T16:48:09.158832+0000","last_fullsized":"2024-09-13T16:48:09.158832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149361+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:08.149361+0000","last_peered":"2024-09-13T16:48:08.149361+0000","last_clean":"2024-09-13T16:48:08.149361+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:08.149361+0000","last_undegraded":"2024-09-13T16:48:08.149361+0000","last_fullsized":"2024-09-13T16:48:08.149361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173784+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T16:48:09.173784+0000","last_peered":"2024-09-13T16:48:09.173784+0000","last_clean":"2024-09-13T16:48:09.173784+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T16:48:09.173784+0000","last_undegraded":"2024-09-13T16:48:09.173784+0000","last_fullsized":"2024-09-13T16:48:09.173784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148791+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:08.148791+0000","last_peered":"2024-09-13T16:48:08.148791+0000","last_clean":"2024-09-13T16:48:08.148791+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:08.148791+0000","last_undegraded":"2024-09-13T16:48:08.148791+0000","last_fullsized":"2024-09-13T16:48:08.148791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159787+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T16:48:09.159787+0000","last_peered":"2024-09-13T16:48:09.159787+0000","last_clean":"2024-09-13T16:48:09.159787+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T16:48:09.159787+0000","last_undegraded":"2024-09-13T16:48:09.159787+0000","last_fullsized":"2024-09-13T16:48:09.159787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166109+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:08.166109+0000","last_peered":"2024-09-13T16:48:08.166109+0000","last_clean":"2024-09-13T16:48:08.166109+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:08.166109+0000","last_undegraded":"2024-09-13T16:48:08.166109+0000","last_fullsized":"2024-09-13T16:48:08.166109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.178540+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T16:48:09.178540+0000","last_peered":"2024-09-13T16:48:09.178540+0000","last_clean":"2024-09-13T16:48:09.178540+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T16:48:09.178540+0000","last_undegraded":"2024-09-13T16:48:09.178540+0000","last_fullsized":"2024-09-13T16:48:09.178540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143140+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:08.143140+0000","last_peered":"2024-09-13T16:48:08.143140+0000","last_clean":"2024-09-13T16:48:08.143140+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:08.143140+0000","last_undegraded":"2024-09-13T16:48:08.143140+0000","last_fullsized":"2024-09-13T16:48:08.143140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.160931+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T16:48:09.160931+0000","last_peered":"2024-09-13T16:48:09.160931+0000","last_clean":"2024-09-13T16:48:09.160931+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T16:48:09.160931+0000","last_undegraded":"2024-09-13T16:48:09.160931+0000","last_fullsized":"2024-09-13T16:48:09.160931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167027+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:08.167027+0000","last_peered":"2024-09-13T16:48:08.167027+0000","last_clean":"2024-09-13T16:48:08.167027+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:08.167027+0000","last_undegraded":"2024-09-13T16:48:08.167027+0000","last_fullsized":"2024-09-13T16:48:08.167027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173532+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T16:48:09.173532+0000","last_peered":"2024-09-13T16:48:09.173532+0000","last_clean":"2024-09-13T16:48:09.173532+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T16:48:09.173532+0000","last_undegraded":"2024-09-13T16:48:09.173532+0000","last_fullsized":"2024-09-13T16:48:09.173532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148753+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:08.148753+0000","last_peered":"2024-09-13T16:48:08.148753+0000","last_clean":"2024-09-13T16:48:08.148753+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:08.148753+0000","last_undegraded":"2024-09-13T16:48:08.148753+0000","last_fullsized":"2024-09-13T16:48:08.148753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161898+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T16:48:09.161898+0000","last_peered":"2024-09-13T16:48:09.161898+0000","last_clean":"2024-09-13T16:48:09.161898+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T16:48:09.161898+0000","last_undegraded":"2024-09-13T16:48:09.161898+0000","last_fullsized":"2024-09-13T16:48:09.161898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160351+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:08.160351+0000","last_peered":"2024-09-13T16:48:08.160351+0000","last_clean":"2024-09-13T16:48:08.160351+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:08.160351+0000","last_undegraded":"2024-09-13T16:48:08.160351+0000","last_fullsized":"2024-09-13T16:48:08.160351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176220+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T16:48:09.176220+0000","last_peered":"2024-09-13T16:48:09.176220+0000","last_clean":"2024-09-13T16:48:09.176220+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T16:48:09.176220+0000","last_undegraded":"2024-09-13T16:48:09.176220+0000","last_fullsized":"2024-09-13T16:48:09.176220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151613+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:08.151613+0000","last_peered":"2024-09-13T16:48:08.151613+0000","last_clean":"2024-09-13T16:48:08.151613+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:08.151613+0000","last_undegraded":"2024-09-13T16:48:08.151613+0000","last_fullsized":"2024-09-13T16:48:08.151613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158220+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T16:48:09.158220+0000","last_peered":"2024-09-13T16:48:09.158220+0000","last_clean":"2024-09-13T16:48:09.158220+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T16:48:09.158220+0000","last_undegraded":"2024-09-13T16:48:09.158220+0000","last_fullsized":"2024-09-13T16:48:09.158220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.162959+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:08.162959+0000","last_peered":"2024-09-13T16:48:08.162959+0000","last_clean":"2024-09-13T16:48:08.162959+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:08.162959+0000","last_undegraded":"2024-09-13T16:48:08.162959+0000","last_fullsized":"2024-09-13T16:48:08.162959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173056+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T16:48:09.173056+0000","last_peered":"2024-09-13T16:48:09.173056+0000","last_clean":"2024-09-13T16:48:09.173056+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T16:48:09.173056+0000","last_undegraded":"2024-09-13T16:48:09.173056+0000","last_fullsized":"2024-09-13T16:48:09.173056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.150218+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:08.150218+0000","last_peered":"2024-09-13T16:48:08.150218+0000","last_clean":"2024-09-13T16:48:08.150218+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:08.150218+0000","last_undegraded":"2024-09-13T16:48:08.150218+0000","last_fullsized":"2024-09-13T16:48:08.150218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173906+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T16:48:09.173906+0000","last_peered":"2024-09-13T16:48:09.173906+0000","last_clean":"2024-09-13T16:48:09.173906+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T16:48:09.173906+0000","last_undegraded":"2024-09-13T16:48:09.173906+0000","last_fullsized":"2024-09-13T16:48:09.173906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166070+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:08.166070+0000","last_peered":"2024-09-13T16:48:08.166070+0000","last_clean":"2024-09-13T16:48:08.166070+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:08.166070+0000","last_undegraded":"2024-09-13T16:48:08.166070+0000","last_fullsized":"2024-09-13T16:48:08.166070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2804,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2804,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":690265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":101847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":99455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":282624,"data_stored":103259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":258048,"data_stored":99295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":101413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":690231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:41.779 INFO:tasks.mds_thrash.ceph_manager:clean! 2024-09-13T16:48:41.779 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T16:48:41.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.939+0000 7f50fceb1640 1 Processor -- start 2024-09-13T16:48:41.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.939+0000 7f50fceb1640 1 -- start start 2024-09-13T16:48:41.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50fceb1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80aabc0 0x7f50f80aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50fceb1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f50f80c40b0 0x7f50f815c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50fceb1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f8152a80 con 0x7f50f806c870 2024-09-13T16:48:41.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50fceb1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f80c30f0 con 0x7f50f80c40b0 2024-09-13T16:48:41.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50fceb1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f8151e00 con 0x7f50f80aabc0 2024-09-13T16:48:41.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f6d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f50f80c40b0 0x7f50f815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50edd9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f50f806c870 0x7f50f80c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34902/0 (socket says 172.21.15.39:34902) 2024-09-13T16:48:41.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50edd9b640 1 -- 172.21.15.39:0/286894298 learned_addr learned my addr 172.21.15.39:0/286894298 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:41.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f659c640 1 --2- 172.21.15.39:0/286894298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80aabc0 0x7f50f80aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f6d9d640 1 -- 172.21.15.39:0/286894298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80aabc0 msgr2=0x7f50f80aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:41.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f6d9d640 1 --2- 172.21.15.39:0/286894298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80aabc0 0x7f50f80aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f6d9d640 1 -- 172.21.15.39:0/286894298 >> v1:172.21.15.39:6789/0 conn(0x7f50f806c870 legacy=0x7f50f80c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f6d9d640 1 -- 172.21.15.39:0/286894298 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f81520e0 con 0x7f50f80c40b0 2024-09-13T16:48:41.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f6d9d640 1 --2- 172.21.15.39:0/286894298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80c40b0 0x7f50f815c470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009d90 tx=0x7f50f001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbddec17ddfa5b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:41.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f5d9b640 1 -- 172.21.15.39:0/286894298 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00a5a00 con 0x7f50f80c40b0 2024-09-13T16:48:41.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f5d9b640 1 -- 172.21.15.39:0/286894298 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f00a5b60 con 0x7f50f80c40b0 2024-09-13T16:48:41.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.943+0000 7f50f5d9b640 1 -- 172.21.15.39:0/286894298 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00ad440 con 0x7f50f80c40b0 2024-09-13T16:48:41.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- 172.21.15.39:0/286894298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80c40b0 msgr2=0x7f50f815c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 --2- 172.21.15.39:0/286894298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80c40b0 0x7f50f815c470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009d90 tx=0x7f50f001c940 comp rx=0 tx=0).stop 2024-09-13T16:48:41.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- 172.21.15.39:0/286894298 shutdown_connections 2024-09-13T16:48:41.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 --2- 172.21.15.39:0/286894298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80c40b0 0x7f50f815c470 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 --2- 172.21.15.39:0/286894298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80aabc0 0x7f50f80aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- 172.21.15.39:0/286894298 >> 172.21.15.39:0/286894298 conn(0x7f50f8069e10 msgr2=0x7f50f806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:41.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- 172.21.15.39:0/286894298 shutdown_connections 2024-09-13T16:48:41.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- 172.21.15.39:0/286894298 wait complete. 2024-09-13T16:48:41.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 Processor -- start 2024-09-13T16:48:41.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- start start 2024-09-13T16:48:41.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f806c870 0x7f50f814ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f80aabc0 0x7f50f814d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80c40b0 0x7f50f80a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f814e030 con 0x7f50f806c870 2024-09-13T16:48:41.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f814d0f0 con 0x7f50f80aabc0 2024-09-13T16:48:41.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f81f09e0 con 0x7f50f80c40b0 2024-09-13T16:48:41.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f806c870 0x7f50f814ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f806c870 0x7f50f814ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57180/0 (socket says 172.21.15.39:57180) 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f659c640 1 -- 172.21.15.39:0/3550517800 learned_addr learned my addr 172.21.15.39:0/3550517800 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f6d9d640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80c40b0 0x7f50f80a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50edd9b640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f80aabc0 0x7f50f814d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f659c640 1 -- 172.21.15.39:0/3550517800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80c40b0 msgr2=0x7f50f80a9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f659c640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80c40b0 0x7f50f80a9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f659c640 1 -- 172.21.15.39:0/3550517800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f80aabc0 msgr2=0x7f50f814d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f659c640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f80aabc0 0x7f50f814d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f659c640 1 -- 172.21.15.39:0/3550517800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0009a70 con 0x7f50f806c870 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50edd9b640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f80aabc0 0x7f50f814d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50f659c640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f806c870 0x7f50f814ca90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f50e800d800 tx=0x7f50e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50ef7fe640 1 -- 172.21.15.39:0/3550517800 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800b6c0 con 0x7f50f806c870 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50ef7fe640 1 -- 172.21.15.39:0/3550517800 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e800b820 con 0x7f50f806c870 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f81efc20 con 0x7f50f806c870 2024-09-13T16:48:41.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50ef7fe640 1 -- 172.21.15.39:0/3550517800 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800bb90 con 0x7f50f806c870 2024-09-13T16:48:41.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.947+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f81f01e0 con 0x7f50f806c870 2024-09-13T16:48:41.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.951+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f8151ab0 con 0x7f50f806c870 2024-09-13T16:48:41.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.951+0000 7f50ef7fe640 1 -- 172.21.15.39:0/3550517800 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f50e800bcf0 con 0x7f50f806c870 2024-09-13T16:48:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.951+0000 7f50ef7fe640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50d8066690 0x7f50d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.951+0000 7f50ef7fe640 1 -- 172.21.15.39:0/3550517800 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f50e80849f0 con 0x7f50f806c870 2024-09-13T16:48:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.951+0000 7f50edd9b640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50d8066690 0x7f50d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.951+0000 7f50edd9b640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50d8066690 0x7f50d8068b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f50f81ea160 tx=0x7f50e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:41.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:41.955+0000 7f50ef7fe640 1 -- 172.21.15.39:0/3550517800 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f50e8051a50 con 0x7f50f806c870 2024-09-13T16:48:42.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.111+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50f80a4fe0 con 0x7f50d8066690 2024-09-13T16:48:42.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.131+0000 7f50ef7fe640 1 -- 172.21.15.39:0/3550517800 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+502912 (secure 0 0 0) 0x7f50e00013e0 con 0x7f50d8066690 2024-09-13T16:48:42.140 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:42.144 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T16:48:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50d8066690 msgr2=0x7f50d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50d8066690 0x7f50d8068b50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f50f81ea160 tx=0x7f50e0009040 comp rx=0 tx=0).stop 2024-09-13T16:48:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f806c870 msgr2=0x7f50f814ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f806c870 0x7f50f814ca90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f50e800d800 tx=0x7f50e800dcc0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 shutdown_connections 2024-09-13T16:48:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50d8066690 0x7f50d8068b50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50f80aabc0 0x7f50f814d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50f80c40b0 0x7f50f80a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 --2- 172.21.15.39:0/3550517800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f806c870 0x7f50f814ca90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 >> 172.21.15.39:0/3550517800 conn(0x7f50f8069e10 msgr2=0x7f50f806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:42.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 shutdown_connections 2024-09-13T16:48:42.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.135+0000 7f50fceb1640 1 -- 172.21.15.39:0/3550517800 wait complete. 2024-09-13T16:48:42.161 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2024-09-13T16:48:41.571289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595976,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":253,"num_write_kb":2448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235804,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738916,"statfs":{"total":773094113280,"available":772852649984,"internally_reserved":0,"allocated":3350528,"data_stored":1985752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238014011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":442,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003048"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174677+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T16:48:09.174677+0000","last_peered":"2024-09-13T16:48:09.174677+0000","last_clean":"2024-09-13T16:48:09.174677+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T16:48:09.174677+0000","last_undegraded":"2024-09-13T16:48:09.174677+0000","last_fullsized":"2024-09-13T16:48:09.174677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163864+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T16:48:08.163864+0000","last_peered":"2024-09-13T16:48:08.163864+0000","last_clean":"2024-09-13T16:48:08.163864+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T16:48:08.163864+0000","last_undegraded":"2024-09-13T16:48:08.163864+0000","last_fullsized":"2024-09-13T16:48:08.163864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146105+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T16:48:08.146105+0000","last_peered":"2024-09-13T16:48:08.146105+0000","last_clean":"2024-09-13T16:48:08.146105+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T16:48:08.146105+0000","last_undegraded":"2024-09-13T16:48:08.146105+0000","last_fullsized":"2024-09-13T16:48:08.146105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143176+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T16:48:08.143176+0000","last_peered":"2024-09-13T16:48:08.143176+0000","last_clean":"2024-09-13T16:48:08.143176+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T16:48:08.143176+0000","last_undegraded":"2024-09-13T16:48:08.143176+0000","last_fullsized":"2024-09-13T16:48:08.143176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173274+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T16:48:09.173274+0000","last_peered":"2024-09-13T16:48:09.173274+0000","last_clean":"2024-09-13T16:48:09.173274+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T16:48:09.173274+0000","last_undegraded":"2024-09-13T16:48:09.173274+0000","last_fullsized":"2024-09-13T16:48:09.173274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146246+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T16:48:08.146246+0000","last_peered":"2024-09-13T16:48:08.146246+0000","last_clean":"2024-09-13T16:48:08.146246+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T16:48:08.146246+0000","last_undegraded":"2024-09-13T16:48:08.146246+0000","last_fullsized":"2024-09-13T16:48:08.146246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163480+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T16:48:08.163480+0000","last_peered":"2024-09-13T16:48:08.163480+0000","last_clean":"2024-09-13T16:48:08.163480+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T16:48:08.163480+0000","last_undegraded":"2024-09-13T16:48:08.163480+0000","last_fullsized":"2024-09-13T16:48:08.163480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184157+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T16:48:08.184157+0000","last_peered":"2024-09-13T16:48:08.184157+0000","last_clean":"2024-09-13T16:48:08.184157+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T16:48:08.184157+0000","last_undegraded":"2024-09-13T16:48:08.184157+0000","last_fullsized":"2024-09-13T16:48:08.184157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173949+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T16:48:09.173949+0000","last_peered":"2024-09-13T16:48:09.173949+0000","last_clean":"2024-09-13T16:48:09.173949+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T16:48:09.173949+0000","last_undegraded":"2024-09-13T16:48:09.173949+0000","last_fullsized":"2024-09-13T16:48:09.173949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178923+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T16:48:08.178923+0000","last_peered":"2024-09-13T16:48:08.178923+0000","last_clean":"2024-09-13T16:48:08.178923+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T16:48:08.178923+0000","last_undegraded":"2024-09-13T16:48:08.178923+0000","last_fullsized":"2024-09-13T16:48:08.178923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151847+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T16:48:08.151847+0000","last_peered":"2024-09-13T16:48:08.151847+0000","last_clean":"2024-09-13T16:48:08.151847+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T16:48:08.151847+0000","last_undegraded":"2024-09-13T16:48:08.151847+0000","last_fullsized":"2024-09-13T16:48:08.151847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208404+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T16:48:08.208404+0000","last_peered":"2024-09-13T16:48:08.208404+0000","last_clean":"2024-09-13T16:48:08.208404+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T16:48:08.208404+0000","last_undegraded":"2024-09-13T16:48:08.208404+0000","last_fullsized":"2024-09-13T16:48:08.208404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159591+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T16:48:09.159591+0000","last_peered":"2024-09-13T16:48:09.159591+0000","last_clean":"2024-09-13T16:48:09.159591+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T16:48:09.159591+0000","last_undegraded":"2024-09-13T16:48:09.159591+0000","last_fullsized":"2024-09-13T16:48:09.159591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165397+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T16:48:08.165397+0000","last_peered":"2024-09-13T16:48:08.165397+0000","last_clean":"2024-09-13T16:48:08.165397+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T16:48:08.165397+0000","last_undegraded":"2024-09-13T16:48:08.165397+0000","last_fullsized":"2024-09-13T16:48:08.165397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153028+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T16:48:08.153028+0000","last_peered":"2024-09-13T16:48:08.153028+0000","last_clean":"2024-09-13T16:48:08.153028+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T16:48:08.153028+0000","last_undegraded":"2024-09-13T16:48:08.153028+0000","last_fullsized":"2024-09-13T16:48:08.153028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.203658+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T16:48:08.203658+0000","last_peered":"2024-09-13T16:48:08.203658+0000","last_clean":"2024-09-13T16:48:08.203658+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T16:48:08.203658+0000","last_undegraded":"2024-09-13T16:48:08.203658+0000","last_fullsized":"2024-09-13T16:48:08.203658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173632+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T16:48:09.173632+0000","last_peered":"2024-09-13T16:48:09.173632+0000","last_clean":"2024-09-13T16:48:09.173632+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T16:48:09.173632+0000","last_undegraded":"2024-09-13T16:48:09.173632+0000","last_fullsized":"2024-09-13T16:48:09.173632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146884+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T16:48:08.146884+0000","last_peered":"2024-09-13T16:48:08.146884+0000","last_clean":"2024-09-13T16:48:08.146884+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T16:48:08.146884+0000","last_undegraded":"2024-09-13T16:48:08.146884+0000","last_fullsized":"2024-09-13T16:48:08.146884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160788+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T16:48:08.160788+0000","last_peered":"2024-09-13T16:48:08.160788+0000","last_clean":"2024-09-13T16:48:08.160788+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T16:48:08.160788+0000","last_undegraded":"2024-09-13T16:48:08.160788+0000","last_fullsized":"2024-09-13T16:48:08.160788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.206623+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T16:48:08.206623+0000","last_peered":"2024-09-13T16:48:08.206623+0000","last_clean":"2024-09-13T16:48:08.206623+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T16:48:08.206623+0000","last_undegraded":"2024-09-13T16:48:08.206623+0000","last_fullsized":"2024-09-13T16:48:08.206623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159080+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T16:48:09.159080+0000","last_peered":"2024-09-13T16:48:09.159080+0000","last_clean":"2024-09-13T16:48:09.159080+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T16:48:09.159080+0000","last_undegraded":"2024-09-13T16:48:09.159080+0000","last_fullsized":"2024-09-13T16:48:09.159080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144345+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T16:48:08.144345+0000","last_peered":"2024-09-13T16:48:08.144345+0000","last_clean":"2024-09-13T16:48:08.144345+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T16:48:08.144345+0000","last_undegraded":"2024-09-13T16:48:08.144345+0000","last_fullsized":"2024-09-13T16:48:08.144345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147467+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T16:48:08.147467+0000","last_peered":"2024-09-13T16:48:08.147467+0000","last_clean":"2024-09-13T16:48:08.147467+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T16:48:08.147467+0000","last_undegraded":"2024-09-13T16:48:08.147467+0000","last_fullsized":"2024-09-13T16:48:08.147467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.184081+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T16:48:08.184081+0000","last_peered":"2024-09-13T16:48:08.184081+0000","last_clean":"2024-09-13T16:48:08.184081+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T16:48:08.184081+0000","last_undegraded":"2024-09-13T16:48:08.184081+0000","last_fullsized":"2024-09-13T16:48:08.184081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143476+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T16:48:08.143476+0000","last_peered":"2024-09-13T16:48:08.143476+0000","last_clean":"2024-09-13T16:48:08.143476+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T16:48:08.143476+0000","last_undegraded":"2024-09-13T16:48:08.143476+0000","last_fullsized":"2024-09-13T16:48:08.143476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147565+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T16:48:08.147565+0000","last_peered":"2024-09-13T16:48:08.147565+0000","last_clean":"2024-09-13T16:48:08.147565+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T16:48:08.147565+0000","last_undegraded":"2024-09-13T16:48:08.147565+0000","last_fullsized":"2024-09-13T16:48:08.147565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174007+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T16:48:08.174007+0000","last_peered":"2024-09-13T16:48:08.174007+0000","last_clean":"2024-09-13T16:48:08.174007+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T16:48:08.174007+0000","last_undegraded":"2024-09-13T16:48:08.174007+0000","last_fullsized":"2024-09-13T16:48:08.174007+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162482+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T16:48:09.162482+0000","last_peered":"2024-09-13T16:48:09.162482+0000","last_clean":"2024-09-13T16:48:09.162482+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T16:48:09.162482+0000","last_undegraded":"2024-09-13T16:48:09.162482+0000","last_fullsized":"2024-09-13T16:48:09.162482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186454+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T16:48:08.186454+0000","last_peered":"2024-09-13T16:48:08.186454+0000","last_clean":"2024-09-13T16:48:08.186454+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T16:48:08.186454+0000","last_undegraded":"2024-09-13T16:48:08.186454+0000","last_fullsized":"2024-09-13T16:48:08.186454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160985+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T16:48:08.160985+0000","last_peered":"2024-09-13T16:48:08.160985+0000","last_clean":"2024-09-13T16:48:08.160985+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T16:48:08.160985+0000","last_undegraded":"2024-09-13T16:48:08.160985+0000","last_fullsized":"2024-09-13T16:48:08.160985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208189+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T16:48:08.208189+0000","last_peered":"2024-09-13T16:48:08.208189+0000","last_clean":"2024-09-13T16:48:08.208189+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T16:48:08.208189+0000","last_undegraded":"2024-09-13T16:48:08.208189+0000","last_fullsized":"2024-09-13T16:48:08.208189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176059+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T16:48:09.176059+0000","last_peered":"2024-09-13T16:48:09.176059+0000","last_clean":"2024-09-13T16:48:09.176059+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T16:48:09.176059+0000","last_undegraded":"2024-09-13T16:48:09.176059+0000","last_fullsized":"2024-09-13T16:48:09.176059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163768+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T16:48:08.163768+0000","last_peered":"2024-09-13T16:48:08.163768+0000","last_clean":"2024-09-13T16:48:08.163768+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T16:48:08.163768+0000","last_undegraded":"2024-09-13T16:48:08.163768+0000","last_fullsized":"2024-09-13T16:48:08.163768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148721+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T16:48:08.148721+0000","last_peered":"2024-09-13T16:48:08.148721+0000","last_clean":"2024-09-13T16:48:08.148721+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T16:48:08.148721+0000","last_undegraded":"2024-09-13T16:48:08.148721+0000","last_fullsized":"2024-09-13T16:48:08.148721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185371+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T16:48:08.185371+0000","last_peered":"2024-09-13T16:48:08.185371+0000","last_clean":"2024-09-13T16:48:08.185371+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T16:48:08.185371+0000","last_undegraded":"2024-09-13T16:48:08.185371+0000","last_fullsized":"2024-09-13T16:48:08.185371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174416+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T16:48:09.174416+0000","last_peered":"2024-09-13T16:48:09.174416+0000","last_clean":"2024-09-13T16:48:09.174416+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T16:48:09.174416+0000","last_undegraded":"2024-09-13T16:48:09.174416+0000","last_fullsized":"2024-09-13T16:48:09.174416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145183+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T16:48:08.145183+0000","last_peered":"2024-09-13T16:48:08.145183+0000","last_clean":"2024-09-13T16:48:08.145183+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T16:48:08.145183+0000","last_undegraded":"2024-09-13T16:48:08.145183+0000","last_fullsized":"2024-09-13T16:48:08.145183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152363+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T16:48:08.152363+0000","last_peered":"2024-09-13T16:48:08.152363+0000","last_clean":"2024-09-13T16:48:08.152363+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T16:48:08.152363+0000","last_undegraded":"2024-09-13T16:48:08.152363+0000","last_fullsized":"2024-09-13T16:48:08.152363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143237+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T16:48:08.143237+0000","last_peered":"2024-09-13T16:48:08.143237+0000","last_clean":"2024-09-13T16:48:08.143237+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T16:48:08.143237+0000","last_undegraded":"2024-09-13T16:48:08.143237+0000","last_fullsized":"2024-09-13T16:48:08.143237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158858+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T16:48:09.158858+0000","last_peered":"2024-09-13T16:48:09.158858+0000","last_clean":"2024-09-13T16:48:09.158858+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T16:48:09.158858+0000","last_undegraded":"2024-09-13T16:48:09.158858+0000","last_fullsized":"2024-09-13T16:48:09.158858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148446+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T16:48:08.148446+0000","last_peered":"2024-09-13T16:48:08.148446+0000","last_clean":"2024-09-13T16:48:08.148446+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T16:48:08.148446+0000","last_undegraded":"2024-09-13T16:48:08.148446+0000","last_fullsized":"2024-09-13T16:48:08.148446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179426+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T16:48:08.179426+0000","last_peered":"2024-09-13T16:48:08.179426+0000","last_clean":"2024-09-13T16:48:08.179426+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T16:48:08.179426+0000","last_undegraded":"2024-09-13T16:48:08.179426+0000","last_fullsized":"2024-09-13T16:48:08.179426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161207+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T16:48:08.161207+0000","last_peered":"2024-09-13T16:48:08.161207+0000","last_clean":"2024-09-13T16:48:08.161207+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T16:48:08.161207+0000","last_undegraded":"2024-09-13T16:48:08.161207+0000","last_fullsized":"2024-09-13T16:48:08.161207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173006+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T16:48:09.173006+0000","last_peered":"2024-09-13T16:48:09.173006+0000","last_clean":"2024-09-13T16:48:09.173006+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T16:48:09.173006+0000","last_undegraded":"2024-09-13T16:48:09.173006+0000","last_fullsized":"2024-09-13T16:48:09.173006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153222+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T16:48:08.153222+0000","last_peered":"2024-09-13T16:48:08.153222+0000","last_clean":"2024-09-13T16:48:08.153222+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T16:48:08.153222+0000","last_undegraded":"2024-09-13T16:48:08.153222+0000","last_fullsized":"2024-09-13T16:48:08.153222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148522+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T16:48:08.148522+0000","last_peered":"2024-09-13T16:48:08.148522+0000","last_clean":"2024-09-13T16:48:08.148522+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T16:48:08.148522+0000","last_undegraded":"2024-09-13T16:48:08.148522+0000","last_fullsized":"2024-09-13T16:48:08.148522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146684+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T16:48:08.146684+0000","last_peered":"2024-09-13T16:48:08.146684+0000","last_clean":"2024-09-13T16:48:08.146684+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T16:48:08.146684+0000","last_undegraded":"2024-09-13T16:48:08.146684+0000","last_fullsized":"2024-09-13T16:48:08.146684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174867+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T16:48:09.174867+0000","last_peered":"2024-09-13T16:48:09.174867+0000","last_clean":"2024-09-13T16:48:09.174867+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T16:48:09.174867+0000","last_undegraded":"2024-09-13T16:48:09.174867+0000","last_fullsized":"2024-09-13T16:48:09.174867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185855+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T16:48:08.185855+0000","last_peered":"2024-09-13T16:48:08.185855+0000","last_clean":"2024-09-13T16:48:08.185855+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T16:48:08.185855+0000","last_undegraded":"2024-09-13T16:48:08.185855+0000","last_fullsized":"2024-09-13T16:48:08.185855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149466+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T16:48:08.149466+0000","last_peered":"2024-09-13T16:48:08.149466+0000","last_clean":"2024-09-13T16:48:08.149466+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T16:48:08.149466+0000","last_undegraded":"2024-09-13T16:48:08.149466+0000","last_fullsized":"2024-09-13T16:48:08.149466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148372+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T16:48:08.148372+0000","last_peered":"2024-09-13T16:48:08.148372+0000","last_clean":"2024-09-13T16:48:08.148372+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T16:48:08.148372+0000","last_undegraded":"2024-09-13T16:48:08.148372+0000","last_fullsized":"2024-09-13T16:48:08.148372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159186+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T16:48:09.159186+0000","last_peered":"2024-09-13T16:48:09.159186+0000","last_clean":"2024-09-13T16:48:09.159186+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T16:48:09.159186+0000","last_undegraded":"2024-09-13T16:48:09.159186+0000","last_fullsized":"2024-09-13T16:48:09.159186+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146860+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T16:48:08.146860+0000","last_peered":"2024-09-13T16:48:08.146860+0000","last_clean":"2024-09-13T16:48:08.146860+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T16:48:08.146860+0000","last_undegraded":"2024-09-13T16:48:08.146860+0000","last_fullsized":"2024-09-13T16:48:08.146860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154111+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T16:48:08.154111+0000","last_peered":"2024-09-13T16:48:08.154111+0000","last_clean":"2024-09-13T16:48:08.154111+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T16:48:08.154111+0000","last_undegraded":"2024-09-13T16:48:08.154111+0000","last_fullsized":"2024-09-13T16:48:08.154111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175388+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T16:48:08.175388+0000","last_peered":"2024-09-13T16:48:08.175388+0000","last_clean":"2024-09-13T16:48:08.175388+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T16:48:08.175388+0000","last_undegraded":"2024-09-13T16:48:08.175388+0000","last_fullsized":"2024-09-13T16:48:08.175388+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162556+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T16:48:09.162556+0000","last_peered":"2024-09-13T16:48:09.162556+0000","last_clean":"2024-09-13T16:48:09.162556+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T16:48:09.162556+0000","last_undegraded":"2024-09-13T16:48:09.162556+0000","last_fullsized":"2024-09-13T16:48:09.162556+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160990+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T16:48:08.160990+0000","last_peered":"2024-09-13T16:48:08.160990+0000","last_clean":"2024-09-13T16:48:08.160990+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T16:48:08.160990+0000","last_undegraded":"2024-09-13T16:48:08.160990+0000","last_fullsized":"2024-09-13T16:48:08.160990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161254+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T16:48:08.161254+0000","last_peered":"2024-09-13T16:48:08.161254+0000","last_clean":"2024-09-13T16:48:08.161254+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T16:48:08.161254+0000","last_undegraded":"2024-09-13T16:48:08.161254+0000","last_fullsized":"2024-09-13T16:48:08.161254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175218+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T16:48:09.175218+0000","last_peered":"2024-09-13T16:48:09.175218+0000","last_clean":"2024-09-13T16:48:09.175218+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T16:48:09.175218+0000","last_undegraded":"2024-09-13T16:48:09.175218+0000","last_fullsized":"2024-09-13T16:48:09.175218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175463+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T16:48:09.175463+0000","last_peered":"2024-09-13T16:48:09.175463+0000","last_clean":"2024-09-13T16:48:09.175463+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T16:48:09.175463+0000","last_undegraded":"2024-09-13T16:48:09.175463+0000","last_fullsized":"2024-09-13T16:48:09.175463+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143792+0000","last_change":"2024-09-13T16:47:59.055438+0000","last_active":"2024-09-13T16:48:08.143792+0000","last_peered":"2024-09-13T16:48:08.143792+0000","last_clean":"2024-09-13T16:48:08.143792+0000","last_became_active":"2024-09-13T16:47:59.055179+0000","last_became_peered":"2024-09-13T16:47:59.055179+0000","last_unstale":"2024-09-13T16:48:08.143792+0000","last_undegraded":"2024-09-13T16:48:08.143792+0000","last_fullsized":"2024-09-13T16:48:08.143792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148284+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T16:48:08.148284+0000","last_peered":"2024-09-13T16:48:08.148284+0000","last_clean":"2024-09-13T16:48:08.148284+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T16:48:08.148284+0000","last_undegraded":"2024-09-13T16:48:08.148284+0000","last_fullsized":"2024-09-13T16:48:08.148284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161478+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T16:48:08.161478+0000","last_peered":"2024-09-13T16:48:08.161478+0000","last_clean":"2024-09-13T16:48:08.161478+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T16:48:08.161478+0000","last_undegraded":"2024-09-13T16:48:08.161478+0000","last_fullsized":"2024-09-13T16:48:08.161478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148905+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T16:48:08.148905+0000","last_peered":"2024-09-13T16:48:08.148905+0000","last_clean":"2024-09-13T16:48:08.148905+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T16:48:08.148905+0000","last_undegraded":"2024-09-13T16:48:08.148905+0000","last_fullsized":"2024-09-13T16:48:08.148905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158934+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T16:48:09.158934+0000","last_peered":"2024-09-13T16:48:09.158934+0000","last_clean":"2024-09-13T16:48:09.158934+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T16:48:09.158934+0000","last_undegraded":"2024-09-13T16:48:09.158934+0000","last_fullsized":"2024-09-13T16:48:09.158934+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.178617+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T16:48:08.178617+0000","last_peered":"2024-09-13T16:48:08.178617+0000","last_clean":"2024-09-13T16:48:08.178617+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T16:48:08.178617+0000","last_undegraded":"2024-09-13T16:48:08.178617+0000","last_fullsized":"2024-09-13T16:48:08.178617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155913+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T16:48:08.155913+0000","last_peered":"2024-09-13T16:48:08.155913+0000","last_clean":"2024-09-13T16:48:08.155913+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T16:48:08.155913+0000","last_undegraded":"2024-09-13T16:48:08.155913+0000","last_fullsized":"2024-09-13T16:48:08.155913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146319+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T16:48:08.146319+0000","last_peered":"2024-09-13T16:48:08.146319+0000","last_clean":"2024-09-13T16:48:08.146319+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T16:48:08.146319+0000","last_undegraded":"2024-09-13T16:48:08.146319+0000","last_fullsized":"2024-09-13T16:48:08.146319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158609+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T16:48:09.158609+0000","last_peered":"2024-09-13T16:48:09.158609+0000","last_clean":"2024-09-13T16:48:09.158609+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T16:48:09.158609+0000","last_undegraded":"2024-09-13T16:48:09.158609+0000","last_fullsized":"2024-09-13T16:48:09.158609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.152057+0000","last_change":"2024-09-13T16:47:59.060995+0000","last_active":"2024-09-13T16:48:08.152057+0000","last_peered":"2024-09-13T16:48:08.152057+0000","last_clean":"2024-09-13T16:48:08.152057+0000","last_became_active":"2024-09-13T16:47:59.060628+0000","last_became_peered":"2024-09-13T16:47:59.060628+0000","last_unstale":"2024-09-13T16:48:08.152057+0000","last_undegraded":"2024-09-13T16:48:08.152057+0000","last_fullsized":"2024-09-13T16:48:08.152057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149487+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T16:48:08.149487+0000","last_peered":"2024-09-13T16:48:08.149487+0000","last_clean":"2024-09-13T16:48:08.149487+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T16:48:08.149487+0000","last_undegraded":"2024-09-13T16:48:08.149487+0000","last_fullsized":"2024-09-13T16:48:08.149487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.175317+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T16:48:08.175317+0000","last_peered":"2024-09-13T16:48:08.175317+0000","last_clean":"2024-09-13T16:48:08.175317+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T16:48:08.175317+0000","last_undegraded":"2024-09-13T16:48:08.175317+0000","last_fullsized":"2024-09-13T16:48:08.175317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173209+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T16:48:09.173209+0000","last_peered":"2024-09-13T16:48:09.173209+0000","last_clean":"2024-09-13T16:48:09.173209+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T16:48:09.173209+0000","last_undegraded":"2024-09-13T16:48:09.173209+0000","last_fullsized":"2024-09-13T16:48:09.173209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.177954+0000","last_change":"2024-09-13T16:47:59.058395+0000","last_active":"2024-09-13T16:48:08.177954+0000","last_peered":"2024-09-13T16:48:08.177954+0000","last_clean":"2024-09-13T16:48:08.177954+0000","last_became_active":"2024-09-13T16:47:59.058146+0000","last_became_peered":"2024-09-13T16:47:59.058146+0000","last_unstale":"2024-09-13T16:48:08.177954+0000","last_undegraded":"2024-09-13T16:48:08.177954+0000","last_fullsized":"2024-09-13T16:48:08.177954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155936+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T16:48:08.155936+0000","last_peered":"2024-09-13T16:48:08.155936+0000","last_clean":"2024-09-13T16:48:08.155936+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T16:48:08.155936+0000","last_undegraded":"2024-09-13T16:48:08.155936+0000","last_fullsized":"2024-09-13T16:48:08.155936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158692+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T16:48:08.158692+0000","last_peered":"2024-09-13T16:48:08.158692+0000","last_clean":"2024-09-13T16:48:08.158692+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T16:48:08.158692+0000","last_undegraded":"2024-09-13T16:48:08.158692+0000","last_fullsized":"2024-09-13T16:48:08.158692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158034+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T16:48:09.158034+0000","last_peered":"2024-09-13T16:48:09.158034+0000","last_clean":"2024-09-13T16:48:09.158034+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T16:48:09.158034+0000","last_undegraded":"2024-09-13T16:48:09.158034+0000","last_fullsized":"2024-09-13T16:48:09.158034+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160811+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T16:48:08.160811+0000","last_peered":"2024-09-13T16:48:08.160811+0000","last_clean":"2024-09-13T16:48:08.160811+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T16:48:08.160811+0000","last_undegraded":"2024-09-13T16:48:08.160811+0000","last_fullsized":"2024-09-13T16:48:08.160811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.192267+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T16:48:08.192267+0000","last_peered":"2024-09-13T16:48:08.192267+0000","last_clean":"2024-09-13T16:48:08.192267+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T16:48:08.192267+0000","last_undegraded":"2024-09-13T16:48:08.192267+0000","last_fullsized":"2024-09-13T16:48:08.192267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185117+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T16:48:08.185117+0000","last_peered":"2024-09-13T16:48:08.185117+0000","last_clean":"2024-09-13T16:48:08.185117+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T16:48:08.185117+0000","last_undegraded":"2024-09-13T16:48:08.185117+0000","last_fullsized":"2024-09-13T16:48:08.185117+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177888+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T16:48:09.177888+0000","last_peered":"2024-09-13T16:48:09.177888+0000","last_clean":"2024-09-13T16:48:09.177888+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T16:48:09.177888+0000","last_undegraded":"2024-09-13T16:48:09.177888+0000","last_fullsized":"2024-09-13T16:48:09.177888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167237+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T16:48:08.167237+0000","last_peered":"2024-09-13T16:48:08.167237+0000","last_clean":"2024-09-13T16:48:08.167237+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T16:48:08.167237+0000","last_undegraded":"2024-09-13T16:48:08.167237+0000","last_fullsized":"2024-09-13T16:48:08.167237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156043+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T16:48:08.156043+0000","last_peered":"2024-09-13T16:48:08.156043+0000","last_clean":"2024-09-13T16:48:08.156043+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T16:48:08.156043+0000","last_undegraded":"2024-09-13T16:48:08.156043+0000","last_fullsized":"2024-09-13T16:48:08.156043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163690+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T16:48:08.163690+0000","last_peered":"2024-09-13T16:48:08.163690+0000","last_clean":"2024-09-13T16:48:08.163690+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T16:48:08.163690+0000","last_undegraded":"2024-09-13T16:48:08.163690+0000","last_fullsized":"2024-09-13T16:48:08.163690+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.162212+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T16:48:09.162212+0000","last_peered":"2024-09-13T16:48:09.162212+0000","last_clean":"2024-09-13T16:48:09.162212+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T16:48:09.162212+0000","last_undegraded":"2024-09-13T16:48:09.162212+0000","last_fullsized":"2024-09-13T16:48:09.162212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144923+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T16:48:08.144923+0000","last_peered":"2024-09-13T16:48:08.144923+0000","last_clean":"2024-09-13T16:48:08.144923+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T16:48:08.144923+0000","last_undegraded":"2024-09-13T16:48:08.144923+0000","last_fullsized":"2024-09-13T16:48:08.144923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145599+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T16:48:08.145599+0000","last_peered":"2024-09-13T16:48:08.145599+0000","last_clean":"2024-09-13T16:48:08.145599+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T16:48:08.145599+0000","last_undegraded":"2024-09-13T16:48:08.145599+0000","last_fullsized":"2024-09-13T16:48:08.145599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145022+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T16:48:08.145022+0000","last_peered":"2024-09-13T16:48:08.145022+0000","last_clean":"2024-09-13T16:48:08.145022+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T16:48:08.145022+0000","last_undegraded":"2024-09-13T16:48:08.145022+0000","last_fullsized":"2024-09-13T16:48:08.145022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161391+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T16:48:09.161391+0000","last_peered":"2024-09-13T16:48:09.161391+0000","last_clean":"2024-09-13T16:48:09.161391+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T16:48:09.161391+0000","last_undegraded":"2024-09-13T16:48:09.161391+0000","last_fullsized":"2024-09-13T16:48:09.161391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160556+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T16:48:08.160556+0000","last_peered":"2024-09-13T16:48:08.160556+0000","last_clean":"2024-09-13T16:48:08.160556+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T16:48:08.160556+0000","last_undegraded":"2024-09-13T16:48:08.160556+0000","last_fullsized":"2024-09-13T16:48:08.160556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149235+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T16:48:08.149235+0000","last_peered":"2024-09-13T16:48:08.149235+0000","last_clean":"2024-09-13T16:48:08.149235+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T16:48:08.149235+0000","last_undegraded":"2024-09-13T16:48:08.149235+0000","last_fullsized":"2024-09-13T16:48:08.149235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"24'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:35.040239+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T16:48:35.040239+0000","last_peered":"2024-09-13T16:48:35.040239+0000","last_clean":"2024-09-13T16:48:35.040239+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T16:48:35.040239+0000","last_undegraded":"2024-09-13T16:48:35.040239+0000","last_fullsized":"2024-09-13T16:48:35.040239+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2006,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161110+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T16:48:08.161110+0000","last_peered":"2024-09-13T16:48:08.161110+0000","last_clean":"2024-09-13T16:48:08.161110+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T16:48:08.161110+0000","last_undegraded":"2024-09-13T16:48:08.161110+0000","last_fullsized":"2024-09-13T16:48:08.161110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161421+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T16:48:08.161421+0000","last_peered":"2024-09-13T16:48:08.161421+0000","last_clean":"2024-09-13T16:48:08.161421+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T16:48:08.161421+0000","last_undegraded":"2024-09-13T16:48:08.161421+0000","last_fullsized":"2024-09-13T16:48:08.161421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.188902+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T16:48:08.188902+0000","last_peered":"2024-09-13T16:48:08.188902+0000","last_clean":"2024-09-13T16:48:08.188902+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T16:48:08.188902+0000","last_undegraded":"2024-09-13T16:48:08.188902+0000","last_fullsized":"2024-09-13T16:48:08.188902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177284+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T16:48:09.177284+0000","last_peered":"2024-09-13T16:48:09.177284+0000","last_clean":"2024-09-13T16:48:09.177284+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T16:48:09.177284+0000","last_undegraded":"2024-09-13T16:48:09.177284+0000","last_fullsized":"2024-09-13T16:48:09.177284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.165383+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T16:48:08.165383+0000","last_peered":"2024-09-13T16:48:08.165383+0000","last_clean":"2024-09-13T16:48:08.165383+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T16:48:08.165383+0000","last_undegraded":"2024-09-13T16:48:08.165383+0000","last_fullsized":"2024-09-13T16:48:08.165383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:31.963081+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T16:48:31.963081+0000","last_peered":"2024-09-13T16:48:31.963081+0000","last_clean":"2024-09-13T16:48:31.963081+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T16:48:31.963081+0000","last_undegraded":"2024-09-13T16:48:31.963081+0000","last_fullsized":"2024-09-13T16:48:31.963081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2006,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.146582+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T16:48:08.146582+0000","last_peered":"2024-09-13T16:48:08.146582+0000","last_clean":"2024-09-13T16:48:08.146582+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T16:48:08.146582+0000","last_undegraded":"2024-09-13T16:48:08.146582+0000","last_fullsized":"2024-09-13T16:48:08.146582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173477+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T16:48:09.173477+0000","last_peered":"2024-09-13T16:48:09.173477+0000","last_clean":"2024-09-13T16:48:09.173477+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T16:48:09.173477+0000","last_undegraded":"2024-09-13T16:48:09.173477+0000","last_fullsized":"2024-09-13T16:48:09.173477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.190852+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T16:48:08.190852+0000","last_peered":"2024-09-13T16:48:08.190852+0000","last_clean":"2024-09-13T16:48:08.190852+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T16:48:08.190852+0000","last_undegraded":"2024-09-13T16:48:08.190852+0000","last_fullsized":"2024-09-13T16:48:08.190852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149403+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T16:48:08.149403+0000","last_peered":"2024-09-13T16:48:08.149403+0000","last_clean":"2024-09-13T16:48:08.149403+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T16:48:08.149403+0000","last_undegraded":"2024-09-13T16:48:08.149403+0000","last_fullsized":"2024-09-13T16:48:08.149403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.208558+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T16:48:08.208558+0000","last_peered":"2024-09-13T16:48:08.208558+0000","last_clean":"2024-09-13T16:48:08.208558+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T16:48:08.208558+0000","last_undegraded":"2024-09-13T16:48:08.208558+0000","last_fullsized":"2024-09-13T16:48:08.208558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158601+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T16:48:09.158601+0000","last_peered":"2024-09-13T16:48:09.158601+0000","last_clean":"2024-09-13T16:48:09.158601+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T16:48:09.158601+0000","last_undegraded":"2024-09-13T16:48:09.158601+0000","last_fullsized":"2024-09-13T16:48:09.158601+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.163596+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T16:48:08.163596+0000","last_peered":"2024-09-13T16:48:08.163596+0000","last_clean":"2024-09-13T16:48:08.163596+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T16:48:08.163596+0000","last_undegraded":"2024-09-13T16:48:08.163596+0000","last_fullsized":"2024-09-13T16:48:08.163596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.156189+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T16:48:08.156189+0000","last_peered":"2024-09-13T16:48:08.156189+0000","last_clean":"2024-09-13T16:48:08.156189+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T16:48:08.156189+0000","last_undegraded":"2024-09-13T16:48:08.156189+0000","last_fullsized":"2024-09-13T16:48:08.156189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.174258+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T16:48:08.174258+0000","last_peered":"2024-09-13T16:48:08.174258+0000","last_clean":"2024-09-13T16:48:08.174258+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T16:48:08.174258+0000","last_undegraded":"2024-09-13T16:48:08.174258+0000","last_fullsized":"2024-09-13T16:48:08.174258+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173305+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T16:48:09.173305+0000","last_peered":"2024-09-13T16:48:09.173305+0000","last_clean":"2024-09-13T16:48:09.173305+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T16:48:09.173305+0000","last_undegraded":"2024-09-13T16:48:09.173305+0000","last_fullsized":"2024-09-13T16:48:09.173305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155963+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T16:48:08.155963+0000","last_peered":"2024-09-13T16:48:08.155963+0000","last_clean":"2024-09-13T16:48:08.155963+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T16:48:08.155963+0000","last_undegraded":"2024-09-13T16:48:08.155963+0000","last_fullsized":"2024-09-13T16:48:08.155963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148500+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T16:48:08.148500+0000","last_peered":"2024-09-13T16:48:08.148500+0000","last_clean":"2024-09-13T16:48:08.148500+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T16:48:08.148500+0000","last_undegraded":"2024-09-13T16:48:08.148500+0000","last_fullsized":"2024-09-13T16:48:08.148500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143144+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T16:48:08.143144+0000","last_peered":"2024-09-13T16:48:08.143144+0000","last_clean":"2024-09-13T16:48:08.143144+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T16:48:08.143144+0000","last_undegraded":"2024-09-13T16:48:08.143144+0000","last_fullsized":"2024-09-13T16:48:08.143144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161066+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T16:48:09.161066+0000","last_peered":"2024-09-13T16:48:09.161066+0000","last_clean":"2024-09-13T16:48:09.161066+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T16:48:09.161066+0000","last_undegraded":"2024-09-13T16:48:09.161066+0000","last_fullsized":"2024-09-13T16:48:09.161066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186757+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T16:48:08.186757+0000","last_peered":"2024-09-13T16:48:08.186757+0000","last_clean":"2024-09-13T16:48:08.186757+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T16:48:08.186757+0000","last_undegraded":"2024-09-13T16:48:08.186757+0000","last_fullsized":"2024-09-13T16:48:08.186757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147291+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T16:48:08.147291+0000","last_peered":"2024-09-13T16:48:08.147291+0000","last_clean":"2024-09-13T16:48:08.147291+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T16:48:08.147291+0000","last_undegraded":"2024-09-13T16:48:08.147291+0000","last_fullsized":"2024-09-13T16:48:08.147291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158941+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T16:48:08.158941+0000","last_peered":"2024-09-13T16:48:08.158941+0000","last_clean":"2024-09-13T16:48:08.158941+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T16:48:08.158941+0000","last_undegraded":"2024-09-13T16:48:08.158941+0000","last_fullsized":"2024-09-13T16:48:08.158941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175068+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T16:48:09.175068+0000","last_peered":"2024-09-13T16:48:09.175068+0000","last_clean":"2024-09-13T16:48:09.175068+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T16:48:09.175068+0000","last_undegraded":"2024-09-13T16:48:09.175068+0000","last_fullsized":"2024-09-13T16:48:09.175068+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"23'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.182997+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T16:48:08.182997+0000","last_peered":"2024-09-13T16:48:08.182997+0000","last_clean":"2024-09-13T16:48:08.182997+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T16:48:08.182997+0000","last_undegraded":"2024-09-13T16:48:08.182997+0000","last_fullsized":"2024-09-13T16:48:08.182997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143381+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T16:48:08.143381+0000","last_peered":"2024-09-13T16:48:08.143381+0000","last_clean":"2024-09-13T16:48:08.143381+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T16:48:08.143381+0000","last_undegraded":"2024-09-13T16:48:08.143381+0000","last_fullsized":"2024-09-13T16:48:08.143381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143921+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T16:48:08.143921+0000","last_peered":"2024-09-13T16:48:08.143921+0000","last_clean":"2024-09-13T16:48:08.143921+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T16:48:08.143921+0000","last_undegraded":"2024-09-13T16:48:08.143921+0000","last_fullsized":"2024-09-13T16:48:08.143921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174477+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T16:48:09.174477+0000","last_peered":"2024-09-13T16:48:09.174477+0000","last_clean":"2024-09-13T16:48:09.174477+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T16:48:09.174477+0000","last_undegraded":"2024-09-13T16:48:09.174477+0000","last_fullsized":"2024-09-13T16:48:09.174477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147114+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T16:48:08.147114+0000","last_peered":"2024-09-13T16:48:08.147114+0000","last_clean":"2024-09-13T16:48:08.147114+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T16:48:08.147114+0000","last_undegraded":"2024-09-13T16:48:08.147114+0000","last_fullsized":"2024-09-13T16:48:08.147114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161410+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T16:48:08.161410+0000","last_peered":"2024-09-13T16:48:08.161410+0000","last_clean":"2024-09-13T16:48:08.161410+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T16:48:08.161410+0000","last_undegraded":"2024-09-13T16:48:08.161410+0000","last_fullsized":"2024-09-13T16:48:08.161410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170387+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T16:48:08.170387+0000","last_peered":"2024-09-13T16:48:08.170387+0000","last_clean":"2024-09-13T16:48:08.170387+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T16:48:08.170387+0000","last_undegraded":"2024-09-13T16:48:08.170387+0000","last_fullsized":"2024-09-13T16:48:08.170387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175603+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T16:48:09.175603+0000","last_peered":"2024-09-13T16:48:09.175603+0000","last_clean":"2024-09-13T16:48:09.175603+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T16:48:09.175603+0000","last_undegraded":"2024-09-13T16:48:09.175603+0000","last_fullsized":"2024-09-13T16:48:09.175603+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144443+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T16:48:08.144443+0000","last_peered":"2024-09-13T16:48:08.144443+0000","last_clean":"2024-09-13T16:48:08.144443+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T16:48:08.144443+0000","last_undegraded":"2024-09-13T16:48:08.144443+0000","last_fullsized":"2024-09-13T16:48:08.144443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.185002+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T16:48:08.185002+0000","last_peered":"2024-09-13T16:48:08.185002+0000","last_clean":"2024-09-13T16:48:08.185002+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T16:48:08.185002+0000","last_undegraded":"2024-09-13T16:48:08.185002+0000","last_fullsized":"2024-09-13T16:48:08.185002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.170142+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T16:48:08.170142+0000","last_peered":"2024-09-13T16:48:08.170142+0000","last_clean":"2024-09-13T16:48:08.170142+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T16:48:08.170142+0000","last_undegraded":"2024-09-13T16:48:08.170142+0000","last_fullsized":"2024-09-13T16:48:08.170142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177615+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T16:48:09.177615+0000","last_peered":"2024-09-13T16:48:09.177615+0000","last_clean":"2024-09-13T16:48:09.177615+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T16:48:09.177615+0000","last_undegraded":"2024-09-13T16:48:09.177615+0000","last_fullsized":"2024-09-13T16:48:09.177615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.144121+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T16:48:08.144121+0000","last_peered":"2024-09-13T16:48:08.144121+0000","last_clean":"2024-09-13T16:48:08.144121+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T16:48:08.144121+0000","last_undegraded":"2024-09-13T16:48:08.144121+0000","last_fullsized":"2024-09-13T16:48:08.144121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174330+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T16:48:09.174330+0000","last_peered":"2024-09-13T16:48:09.174330+0000","last_clean":"2024-09-13T16:48:09.174330+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T16:48:09.174330+0000","last_undegraded":"2024-09-13T16:48:09.174330+0000","last_fullsized":"2024-09-13T16:48:09.174330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.153996+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T16:48:08.153996+0000","last_peered":"2024-09-13T16:48:08.153996+0000","last_clean":"2024-09-13T16:48:08.153996+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T16:48:08.153996+0000","last_undegraded":"2024-09-13T16:48:08.153996+0000","last_fullsized":"2024-09-13T16:48:08.153996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161029+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T16:48:09.161029+0000","last_peered":"2024-09-13T16:48:09.161029+0000","last_clean":"2024-09-13T16:48:09.161029+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T16:48:09.161029+0000","last_undegraded":"2024-09-13T16:48:09.161029+0000","last_fullsized":"2024-09-13T16:48:09.161029+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.154547+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T16:48:08.154547+0000","last_peered":"2024-09-13T16:48:08.154547+0000","last_clean":"2024-09-13T16:48:08.154547+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T16:48:08.154547+0000","last_undegraded":"2024-09-13T16:48:08.154547+0000","last_fullsized":"2024-09-13T16:48:08.154547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174304+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T16:48:09.174304+0000","last_peered":"2024-09-13T16:48:09.174304+0000","last_clean":"2024-09-13T16:48:09.174304+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T16:48:09.174304+0000","last_undegraded":"2024-09-13T16:48:09.174304+0000","last_fullsized":"2024-09-13T16:48:09.174304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161339+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T16:48:08.161339+0000","last_peered":"2024-09-13T16:48:08.161339+0000","last_clean":"2024-09-13T16:48:08.161339+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T16:48:08.161339+0000","last_undegraded":"2024-09-13T16:48:08.161339+0000","last_fullsized":"2024-09-13T16:48:08.161339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173686+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T16:48:09.173686+0000","last_peered":"2024-09-13T16:48:09.173686+0000","last_clean":"2024-09-13T16:48:09.173686+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T16:48:09.173686+0000","last_undegraded":"2024-09-13T16:48:09.173686+0000","last_fullsized":"2024-09-13T16:48:09.173686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145104+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T16:48:08.145104+0000","last_peered":"2024-09-13T16:48:08.145104+0000","last_clean":"2024-09-13T16:48:08.145104+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T16:48:08.145104+0000","last_undegraded":"2024-09-13T16:48:08.145104+0000","last_fullsized":"2024-09-13T16:48:08.145104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175359+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T16:48:09.175359+0000","last_peered":"2024-09-13T16:48:09.175359+0000","last_clean":"2024-09-13T16:48:09.175359+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T16:48:09.175359+0000","last_undegraded":"2024-09-13T16:48:09.175359+0000","last_fullsized":"2024-09-13T16:48:09.175359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149320+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T16:48:08.149320+0000","last_peered":"2024-09-13T16:48:08.149320+0000","last_clean":"2024-09-13T16:48:08.149320+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T16:48:08.149320+0000","last_undegraded":"2024-09-13T16:48:08.149320+0000","last_fullsized":"2024-09-13T16:48:08.149320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173205+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T16:48:09.173205+0000","last_peered":"2024-09-13T16:48:09.173205+0000","last_clean":"2024-09-13T16:48:09.173205+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T16:48:09.173205+0000","last_undegraded":"2024-09-13T16:48:09.173205+0000","last_fullsized":"2024-09-13T16:48:09.173205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149001+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T16:48:08.149001+0000","last_peered":"2024-09-13T16:48:08.149001+0000","last_clean":"2024-09-13T16:48:08.149001+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T16:48:08.149001+0000","last_undegraded":"2024-09-13T16:48:08.149001+0000","last_fullsized":"2024-09-13T16:48:08.149001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158758+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T16:48:09.158758+0000","last_peered":"2024-09-13T16:48:09.158758+0000","last_clean":"2024-09-13T16:48:09.158758+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T16:48:09.158758+0000","last_undegraded":"2024-09-13T16:48:09.158758+0000","last_fullsized":"2024-09-13T16:48:09.158758+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.145105+0000","last_change":"2024-09-13T16:47:59.052165+0000","last_active":"2024-09-13T16:48:08.145105+0000","last_peered":"2024-09-13T16:48:08.145105+0000","last_clean":"2024-09-13T16:48:08.145105+0000","last_became_active":"2024-09-13T16:47:59.051889+0000","last_became_peered":"2024-09-13T16:47:59.051889+0000","last_unstale":"2024-09-13T16:48:08.145105+0000","last_undegraded":"2024-09-13T16:48:08.145105+0000","last_fullsized":"2024-09-13T16:48:08.145105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177538+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T16:48:09.177538+0000","last_peered":"2024-09-13T16:48:09.177538+0000","last_clean":"2024-09-13T16:48:09.177538+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T16:48:09.177538+0000","last_undegraded":"2024-09-13T16:48:09.177538+0000","last_fullsized":"2024-09-13T16:48:09.177538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158985+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T16:48:08.158985+0000","last_peered":"2024-09-13T16:48:08.158985+0000","last_clean":"2024-09-13T16:48:08.158985+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T16:48:08.158985+0000","last_undegraded":"2024-09-13T16:48:08.158985+0000","last_fullsized":"2024-09-13T16:48:08.158985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177461+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T16:48:09.177461+0000","last_peered":"2024-09-13T16:48:09.177461+0000","last_clean":"2024-09-13T16:48:09.177461+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T16:48:09.177461+0000","last_undegraded":"2024-09-13T16:48:09.177461+0000","last_fullsized":"2024-09-13T16:48:09.177461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143902+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T16:48:08.143902+0000","last_peered":"2024-09-13T16:48:08.143902+0000","last_clean":"2024-09-13T16:48:08.143902+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T16:48:08.143902+0000","last_undegraded":"2024-09-13T16:48:08.143902+0000","last_fullsized":"2024-09-13T16:48:08.143902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159127+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T16:48:09.159127+0000","last_peered":"2024-09-13T16:48:09.159127+0000","last_clean":"2024-09-13T16:48:09.159127+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T16:48:09.159127+0000","last_undegraded":"2024-09-13T16:48:09.159127+0000","last_fullsized":"2024-09-13T16:48:09.159127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148988+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T16:48:08.148988+0000","last_peered":"2024-09-13T16:48:08.148988+0000","last_clean":"2024-09-13T16:48:08.148988+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T16:48:08.148988+0000","last_undegraded":"2024-09-13T16:48:08.148988+0000","last_fullsized":"2024-09-13T16:48:08.148988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177190+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T16:48:09.177190+0000","last_peered":"2024-09-13T16:48:09.177190+0000","last_clean":"2024-09-13T16:48:09.177190+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T16:48:09.177190+0000","last_undegraded":"2024-09-13T16:48:09.177190+0000","last_fullsized":"2024-09-13T16:48:09.177190+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161327+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T16:48:08.161327+0000","last_peered":"2024-09-13T16:48:08.161327+0000","last_clean":"2024-09-13T16:48:08.161327+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T16:48:08.161327+0000","last_undegraded":"2024-09-13T16:48:08.161327+0000","last_fullsized":"2024-09-13T16:48:08.161327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.174523+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T16:48:09.174523+0000","last_peered":"2024-09-13T16:48:09.174523+0000","last_clean":"2024-09-13T16:48:09.174523+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T16:48:09.174523+0000","last_undegraded":"2024-09-13T16:48:09.174523+0000","last_fullsized":"2024-09-13T16:48:09.174523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148225+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T16:48:08.148225+0000","last_peered":"2024-09-13T16:48:08.148225+0000","last_clean":"2024-09-13T16:48:08.148225+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T16:48:08.148225+0000","last_undegraded":"2024-09-13T16:48:08.148225+0000","last_fullsized":"2024-09-13T16:48:08.148225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175099+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T16:48:09.175099+0000","last_peered":"2024-09-13T16:48:09.175099+0000","last_clean":"2024-09-13T16:48:09.175099+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T16:48:09.175099+0000","last_undegraded":"2024-09-13T16:48:09.175099+0000","last_fullsized":"2024-09-13T16:48:09.175099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.179166+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T16:48:08.179166+0000","last_peered":"2024-09-13T16:48:08.179166+0000","last_clean":"2024-09-13T16:48:08.179166+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T16:48:08.179166+0000","last_undegraded":"2024-09-13T16:48:08.179166+0000","last_fullsized":"2024-09-13T16:48:08.179166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158364+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T16:48:09.158364+0000","last_peered":"2024-09-13T16:48:09.158364+0000","last_clean":"2024-09-13T16:48:09.158364+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T16:48:09.158364+0000","last_undegraded":"2024-09-13T16:48:09.158364+0000","last_fullsized":"2024-09-13T16:48:09.158364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.155807+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T16:48:08.155807+0000","last_peered":"2024-09-13T16:48:08.155807+0000","last_clean":"2024-09-13T16:48:08.155807+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T16:48:08.155807+0000","last_undegraded":"2024-09-13T16:48:08.155807+0000","last_fullsized":"2024-09-13T16:48:08.155807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.172410+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T16:48:09.172410+0000","last_peered":"2024-09-13T16:48:09.172410+0000","last_clean":"2024-09-13T16:48:09.172410+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T16:48:09.172410+0000","last_undegraded":"2024-09-13T16:48:09.172410+0000","last_fullsized":"2024-09-13T16:48:09.172410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.157093+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T16:48:08.157093+0000","last_peered":"2024-09-13T16:48:08.157093+0000","last_clean":"2024-09-13T16:48:08.157093+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T16:48:08.157093+0000","last_undegraded":"2024-09-13T16:48:08.157093+0000","last_fullsized":"2024-09-13T16:48:08.157093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173535+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T16:48:09.173535+0000","last_peered":"2024-09-13T16:48:09.173535+0000","last_clean":"2024-09-13T16:48:09.173535+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T16:48:09.173535+0000","last_undegraded":"2024-09-13T16:48:09.173535+0000","last_fullsized":"2024-09-13T16:48:09.173535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.158715+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T16:48:08.158715+0000","last_peered":"2024-09-13T16:48:08.158715+0000","last_clean":"2024-09-13T16:48:08.158715+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T16:48:08.158715+0000","last_undegraded":"2024-09-13T16:48:08.158715+0000","last_fullsized":"2024-09-13T16:48:08.158715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.175097+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T16:48:09.175097+0000","last_peered":"2024-09-13T16:48:09.175097+0000","last_clean":"2024-09-13T16:48:09.175097+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T16:48:09.175097+0000","last_undegraded":"2024-09-13T16:48:09.175097+0000","last_fullsized":"2024-09-13T16:48:09.175097+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.147419+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T16:48:08.147419+0000","last_peered":"2024-09-13T16:48:08.147419+0000","last_clean":"2024-09-13T16:48:08.147419+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T16:48:08.147419+0000","last_undegraded":"2024-09-13T16:48:08.147419+0000","last_fullsized":"2024-09-13T16:48:08.147419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159429+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T16:48:09.159429+0000","last_peered":"2024-09-13T16:48:09.159429+0000","last_clean":"2024-09-13T16:48:09.159429+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T16:48:09.159429+0000","last_undegraded":"2024-09-13T16:48:09.159429+0000","last_fullsized":"2024-09-13T16:48:09.159429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.161028+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T16:48:08.161028+0000","last_peered":"2024-09-13T16:48:08.161028+0000","last_clean":"2024-09-13T16:48:08.161028+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T16:48:08.161028+0000","last_undegraded":"2024-09-13T16:48:08.161028+0000","last_fullsized":"2024-09-13T16:48:08.161028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.177095+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T16:48:09.177095+0000","last_peered":"2024-09-13T16:48:09.177095+0000","last_clean":"2024-09-13T16:48:09.177095+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T16:48:09.177095+0000","last_undegraded":"2024-09-13T16:48:09.177095+0000","last_fullsized":"2024-09-13T16:48:09.177095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143741+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T16:48:08.143741+0000","last_peered":"2024-09-13T16:48:08.143741+0000","last_clean":"2024-09-13T16:48:08.143741+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T16:48:08.143741+0000","last_undegraded":"2024-09-13T16:48:08.143741+0000","last_fullsized":"2024-09-13T16:48:08.143741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176397+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T16:48:09.176397+0000","last_peered":"2024-09-13T16:48:09.176397+0000","last_clean":"2024-09-13T16:48:09.176397+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T16:48:09.176397+0000","last_undegraded":"2024-09-13T16:48:09.176397+0000","last_fullsized":"2024-09-13T16:48:09.176397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.186343+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T16:48:08.186343+0000","last_peered":"2024-09-13T16:48:08.186343+0000","last_clean":"2024-09-13T16:48:08.186343+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T16:48:08.186343+0000","last_undegraded":"2024-09-13T16:48:08.186343+0000","last_fullsized":"2024-09-13T16:48:08.186343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158832+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T16:48:09.158832+0000","last_peered":"2024-09-13T16:48:09.158832+0000","last_clean":"2024-09-13T16:48:09.158832+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T16:48:09.158832+0000","last_undegraded":"2024-09-13T16:48:09.158832+0000","last_fullsized":"2024-09-13T16:48:09.158832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.149361+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T16:48:08.149361+0000","last_peered":"2024-09-13T16:48:08.149361+0000","last_clean":"2024-09-13T16:48:08.149361+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T16:48:08.149361+0000","last_undegraded":"2024-09-13T16:48:08.149361+0000","last_fullsized":"2024-09-13T16:48:08.149361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173784+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T16:48:09.173784+0000","last_peered":"2024-09-13T16:48:09.173784+0000","last_clean":"2024-09-13T16:48:09.173784+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T16:48:09.173784+0000","last_undegraded":"2024-09-13T16:48:09.173784+0000","last_fullsized":"2024-09-13T16:48:09.173784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148791+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T16:48:08.148791+0000","last_peered":"2024-09-13T16:48:08.148791+0000","last_clean":"2024-09-13T16:48:08.148791+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T16:48:08.148791+0000","last_undegraded":"2024-09-13T16:48:08.148791+0000","last_fullsized":"2024-09-13T16:48:08.148791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.159787+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T16:48:09.159787+0000","last_peered":"2024-09-13T16:48:09.159787+0000","last_clean":"2024-09-13T16:48:09.159787+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T16:48:09.159787+0000","last_undegraded":"2024-09-13T16:48:09.159787+0000","last_fullsized":"2024-09-13T16:48:09.159787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166109+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T16:48:08.166109+0000","last_peered":"2024-09-13T16:48:08.166109+0000","last_clean":"2024-09-13T16:48:08.166109+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T16:48:08.166109+0000","last_undegraded":"2024-09-13T16:48:08.166109+0000","last_fullsized":"2024-09-13T16:48:08.166109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.178540+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T16:48:09.178540+0000","last_peered":"2024-09-13T16:48:09.178540+0000","last_clean":"2024-09-13T16:48:09.178540+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T16:48:09.178540+0000","last_undegraded":"2024-09-13T16:48:09.178540+0000","last_fullsized":"2024-09-13T16:48:09.178540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.143140+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T16:48:08.143140+0000","last_peered":"2024-09-13T16:48:08.143140+0000","last_clean":"2024-09-13T16:48:08.143140+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T16:48:08.143140+0000","last_undegraded":"2024-09-13T16:48:08.143140+0000","last_fullsized":"2024-09-13T16:48:08.143140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.160931+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T16:48:09.160931+0000","last_peered":"2024-09-13T16:48:09.160931+0000","last_clean":"2024-09-13T16:48:09.160931+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T16:48:09.160931+0000","last_undegraded":"2024-09-13T16:48:09.160931+0000","last_fullsized":"2024-09-13T16:48:09.160931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.167027+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T16:48:08.167027+0000","last_peered":"2024-09-13T16:48:08.167027+0000","last_clean":"2024-09-13T16:48:08.167027+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T16:48:08.167027+0000","last_undegraded":"2024-09-13T16:48:08.167027+0000","last_fullsized":"2024-09-13T16:48:08.167027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173532+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T16:48:09.173532+0000","last_peered":"2024-09-13T16:48:09.173532+0000","last_clean":"2024-09-13T16:48:09.173532+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T16:48:09.173532+0000","last_undegraded":"2024-09-13T16:48:09.173532+0000","last_fullsized":"2024-09-13T16:48:09.173532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.148753+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T16:48:08.148753+0000","last_peered":"2024-09-13T16:48:08.148753+0000","last_clean":"2024-09-13T16:48:08.148753+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T16:48:08.148753+0000","last_undegraded":"2024-09-13T16:48:08.148753+0000","last_fullsized":"2024-09-13T16:48:08.148753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.161898+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T16:48:09.161898+0000","last_peered":"2024-09-13T16:48:09.161898+0000","last_clean":"2024-09-13T16:48:09.161898+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T16:48:09.161898+0000","last_undegraded":"2024-09-13T16:48:09.161898+0000","last_fullsized":"2024-09-13T16:48:09.161898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.160351+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T16:48:08.160351+0000","last_peered":"2024-09-13T16:48:08.160351+0000","last_clean":"2024-09-13T16:48:08.160351+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T16:48:08.160351+0000","last_undegraded":"2024-09-13T16:48:08.160351+0000","last_fullsized":"2024-09-13T16:48:08.160351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.176220+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T16:48:09.176220+0000","last_peered":"2024-09-13T16:48:09.176220+0000","last_clean":"2024-09-13T16:48:09.176220+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T16:48:09.176220+0000","last_undegraded":"2024-09-13T16:48:09.176220+0000","last_fullsized":"2024-09-13T16:48:09.176220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.151613+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T16:48:08.151613+0000","last_peered":"2024-09-13T16:48:08.151613+0000","last_clean":"2024-09-13T16:48:08.151613+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T16:48:08.151613+0000","last_undegraded":"2024-09-13T16:48:08.151613+0000","last_fullsized":"2024-09-13T16:48:08.151613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.158220+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T16:48:09.158220+0000","last_peered":"2024-09-13T16:48:09.158220+0000","last_clean":"2024-09-13T16:48:09.158220+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T16:48:09.158220+0000","last_undegraded":"2024-09-13T16:48:09.158220+0000","last_fullsized":"2024-09-13T16:48:09.158220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.162959+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T16:48:08.162959+0000","last_peered":"2024-09-13T16:48:08.162959+0000","last_clean":"2024-09-13T16:48:08.162959+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T16:48:08.162959+0000","last_undegraded":"2024-09-13T16:48:08.162959+0000","last_fullsized":"2024-09-13T16:48:08.162959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173056+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T16:48:09.173056+0000","last_peered":"2024-09-13T16:48:09.173056+0000","last_clean":"2024-09-13T16:48:09.173056+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T16:48:09.173056+0000","last_undegraded":"2024-09-13T16:48:09.173056+0000","last_fullsized":"2024-09-13T16:48:09.173056+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.150218+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T16:48:08.150218+0000","last_peered":"2024-09-13T16:48:08.150218+0000","last_clean":"2024-09-13T16:48:08.150218+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T16:48:08.150218+0000","last_undegraded":"2024-09-13T16:48:08.150218+0000","last_fullsized":"2024-09-13T16:48:08.150218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T16:48:09.173906+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T16:48:09.173906+0000","last_peered":"2024-09-13T16:48:09.173906+0000","last_clean":"2024-09-13T16:48:09.173906+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T16:48:09.173906+0000","last_undegraded":"2024-09-13T16:48:09.173906+0000","last_fullsized":"2024-09-13T16:48:09.173906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T16:48:08.166070+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T16:48:08.166070+0000","last_peered":"2024-09-13T16:48:08.166070+0000","last_clean":"2024-09-13T16:48:08.166070+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T16:48:08.166070+0000","last_undegraded":"2024-09-13T16:48:08.166070+0000","last_fullsized":"2024-09-13T16:48:08.166070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2804,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2804,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":856064,"data_stored":690265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":101847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":99455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":282624,"data_stored":103259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":258048,"data_stored":99295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":101413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":860160,"data_stored":690231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T16:48:42.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:42.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.343+0000 7efc2e1f3640 1 Processor -- start 2024-09-13T16:48:42.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.343+0000 7efc2e1f3640 1 -- start start 2024-09-13T16:48:42.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc2e1f3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc281526b0 0x7efc280c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc2e1f3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc2806c940 0x7efc280aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc2e1f3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2814f790 con 0x7efc280ab100 2024-09-13T16:48:42.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc2e1f3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc28152a90 con 0x7efc2806c940 2024-09-13T16:48:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc2e1f3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2806ce40 con 0x7efc281526b0 2024-09-13T16:48:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc26ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc2806c940 0x7efc280aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc26ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc2806c940 0x7efc280aabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57194/0 (socket says 172.21.15.39:57194) 2024-09-13T16:48:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc26ffd640 1 -- 172.21.15.39:0/1791668424 learned_addr learned my addr 172.21.15.39:0/1791668424 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc277fe640 1 --2- 172.21.15.39:0/1791668424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc281526b0 0x7efc280c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc26ffd640 1 -- 172.21.15.39:0/1791668424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc281526b0 msgr2=0x7efc280c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc26ffd640 1 --2- 172.21.15.39:0/1791668424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc281526b0 0x7efc280c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc26ffd640 1 -- 172.21.15.39:0/1791668424 >> v1:172.21.15.84:6789/0 conn(0x7efc280ab100 legacy=0x7efc281555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc26ffd640 1 -- 172.21.15.39:0/1791668424 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2806d120 con 0x7efc2806c940 2024-09-13T16:48:42.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc26ffd640 1 --2- 172.21.15.39:0/1791668424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2806c940 0x7efc280aabc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009870 tx=0x7efc1c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80aea0a50eb12e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:42.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc267fc640 1 -- 172.21.15.39:0/1791668424 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0ab600 con 0x7efc2806c940 2024-09-13T16:48:42.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc267fc640 1 -- 172.21.15.39:0/1791668424 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c0a7050 con 0x7efc2806c940 2024-09-13T16:48:42.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.347+0000 7efc267fc640 1 -- 172.21.15.39:0/1791668424 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0ab980 con 0x7efc2806c940 2024-09-13T16:48:42.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- 172.21.15.39:0/1791668424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2806c940 msgr2=0x7efc280aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 --2- 172.21.15.39:0/1791668424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2806c940 0x7efc280aabc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009870 tx=0x7efc1c01c520 comp rx=0 tx=0).stop 2024-09-13T16:48:42.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- 172.21.15.39:0/1791668424 shutdown_connections 2024-09-13T16:48:42.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 --2- 172.21.15.39:0/1791668424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2806c940 0x7efc280aabc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 --2- 172.21.15.39:0/1791668424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc281526b0 0x7efc280c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- 172.21.15.39:0/1791668424 >> 172.21.15.39:0/1791668424 conn(0x7efc28069e10 msgr2=0x7efc2806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:42.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- 172.21.15.39:0/1791668424 shutdown_connections 2024-09-13T16:48:42.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- 172.21.15.39:0/1791668424 wait complete. 2024-09-13T16:48:42.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 Processor -- start 2024-09-13T16:48:42.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- start start 2024-09-13T16:48:42.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2806c940 0x7efc28150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc280ab100 0x7efc2814b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc281526b0 0x7efc281f0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc28149ee0 con 0x7efc280ab100 2024-09-13T16:48:42.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2814b580 con 0x7efc281526b0 2024-09-13T16:48:42.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2814ee90 con 0x7efc2806c940 2024-09-13T16:48:42.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc26ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc280ab100 0x7efc2814b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc26ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc280ab100 0x7efc2814b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57204/0 (socket says 172.21.15.39:57204) 2024-09-13T16:48:42.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc26ffd640 1 -- 172.21.15.39:0/2179252138 learned_addr learned my addr 172.21.15.39:0/2179252138 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:42.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc277fe640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2806c940 0x7efc28150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc27fff640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc281526b0 0x7efc281f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc26ffd640 1 -- 172.21.15.39:0/2179252138 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2806c940 msgr2=0x7efc28150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc26ffd640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2806c940 0x7efc28150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc26ffd640 1 -- 172.21.15.39:0/2179252138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc281526b0 msgr2=0x7efc281f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc26ffd640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc281526b0 0x7efc281f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc26ffd640 1 -- 172.21.15.39:0/2179252138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0094d0 con 0x7efc280ab100 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc26ffd640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc280ab100 0x7efc2814b9a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0ae040 tx=0x7efc1c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc0ffff640 1 -- 172.21.15.39:0/2179252138 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0b4420 con 0x7efc280ab100 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc0ffff640 1 -- 172.21.15.39:0/2179252138 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c0b4580 con 0x7efc280ab100 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc281ed150 con 0x7efc280ab100 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc0ffff640 1 -- 172.21.15.39:0/2179252138 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0b4930 con 0x7efc280ab100 2024-09-13T16:48:42.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.351+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc28150690 con 0x7efc280ab100 2024-09-13T16:48:42.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.355+0000 7efc0ffff640 1 -- 172.21.15.39:0/2179252138 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc1c0b2070 con 0x7efc280ab100 2024-09-13T16:48:42.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.355+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2814a050 con 0x7efc280ab100 2024-09-13T16:48:42.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.355+0000 7efc0ffff640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc14066690 0x7efc14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.355+0000 7efc0ffff640 1 -- 172.21.15.39:0/2179252138 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7efc1c11d900 con 0x7efc280ab100 2024-09-13T16:48:42.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.355+0000 7efc277fe640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc14066690 0x7efc14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.355+0000 7efc277fe640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc14066690 0x7efc14068b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efc280c51f0 tx=0x7efc18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:42.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.359+0000 7efc0ffff640 1 -- 172.21.15.39:0/2179252138 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efc1c0ea8c0 con 0x7efc280ab100 2024-09-13T16:48:42.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.523+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc2806c670 con 0x7efc280ab100 2024-09-13T16:48:42.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc0ffff640 1 -- 172.21.15.39:0/2179252138 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20376 (secure 0 0 0) 0x7efc1c0edee0 con 0x7efc280ab100 2024-09-13T16:48:42.531 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:42.531 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:40.750805+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:42.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc14066690 msgr2=0x7efc14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc14066690 0x7efc14068b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efc280c51f0 tx=0x7efc18008040 comp rx=0 tx=0).stop 2024-09-13T16:48:42.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc280ab100 msgr2=0x7efc2814b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc280ab100 0x7efc2814b9a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0ae040 tx=0x7efc1c0192c0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 shutdown_connections 2024-09-13T16:48:42.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc14066690 0x7efc14068b50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc281526b0 0x7efc281f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc280ab100 0x7efc2814b9a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 --2- 172.21.15.39:0/2179252138 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2806c940 0x7efc28150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 >> 172.21.15.39:0/2179252138 conn(0x7efc28069e10 msgr2=0x7efc2806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:42.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 shutdown_connections 2024-09-13T16:48:42.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.527+0000 7efc2e1f3640 1 -- 172.21.15.39:0/2179252138 wait complete. 2024-09-13T16:48:42.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:48:42.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:48:42.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb614be1640 1 Processor -- start 2024-09-13T16:48:42.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb614be1640 1 -- start start 2024-09-13T16:48:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb614be1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6101514d0 0x7fb6100c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb614be1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb6100bcf90 0x7fb610154240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb614be1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb610155270 con 0x7fb610157370 2024-09-13T16:48:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb614be1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb610153d80 con 0x7fb6100bcf90 2024-09-13T16:48:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb614be1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb610157870 con 0x7fb6101514d0 2024-09-13T16:48:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb60e59c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6101514d0 0x7fb6100c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb60dd9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb6100bcf90 0x7fb610154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb60e59c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6101514d0 0x7fb6100c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:48668/0 (socket says 172.21.15.39:48668) 2024-09-13T16:48:42.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb60e59c640 1 -- 172.21.15.39:0/3708916430 learned_addr learned my addr 172.21.15.39:0/3708916430 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:42.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb60e59c640 1 -- 172.21.15.39:0/3708916430 >> v2:172.21.15.84:3300/0 conn(0x7fb6100bcf90 msgr2=0x7fb610154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:42.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb60e59c640 1 --2- 172.21.15.39:0/3708916430 >> v2:172.21.15.84:3300/0 conn(0x7fb6100bcf90 0x7fb610154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb60e59c640 1 -- 172.21.15.39:0/3708916430 >> v1:172.21.15.84:6789/0 conn(0x7fb610157370 legacy=0x7fb610154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.667+0000 7fb60e59c640 1 -- 172.21.15.39:0/3708916430 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610157b50 con 0x7fb6101514d0 2024-09-13T16:48:42.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb60e59c640 1 --2- 172.21.15.39:0/3708916430 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6101514d0 0x7fb6100c41f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb600009870 tx=0x7fb60001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f6450e1c897eac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:42.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb60d59a640 1 -- 172.21.15.39:0/3708916430 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6000ac620 con 0x7fb6101514d0 2024-09-13T16:48:42.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb60d59a640 1 -- 172.21.15.39:0/3708916430 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6000ac780 con 0x7fb6101514d0 2024-09-13T16:48:42.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb60d59a640 1 -- 172.21.15.39:0/3708916430 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6000acb10 con 0x7fb6101514d0 2024-09-13T16:48:42.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 -- 172.21.15.39:0/3708916430 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6101514d0 msgr2=0x7fb6100c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 --2- 172.21.15.39:0/3708916430 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6101514d0 0x7fb6100c41f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb600009870 tx=0x7fb60001c4d0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 -- 172.21.15.39:0/3708916430 shutdown_connections 2024-09-13T16:48:42.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 --2- 172.21.15.39:0/3708916430 >> v2:172.21.15.84:3300/0 conn(0x7fb6100bcf90 0x7fb610154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 --2- 172.21.15.39:0/3708916430 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6101514d0 0x7fb6100c41f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 -- 172.21.15.39:0/3708916430 >> 172.21.15.39:0/3708916430 conn(0x7fb61006a210 msgr2=0x7fb61006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:42.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 -- 172.21.15.39:0/3708916430 shutdown_connections 2024-09-13T16:48:42.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 -- 172.21.15.39:0/3708916430 wait complete. 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 Processor -- start 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 -- start start 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6100bcf90 0x7fb6100c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6101514d0 0x7fb6100aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb610157370 0x7fb610144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100c7440 con 0x7fb6101514d0 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100a9f30 con 0x7fb6100bcf90 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.671+0000 7fb614be1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100c7690 con 0x7fb610157370 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6101514d0 0x7fb6100aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6101514d0 0x7fb6100aaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57208/0 (socket says 172.21.15.39:57208) 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60dd9b640 1 -- 172.21.15.39:0/3759218599 learned_addr learned my addr 172.21.15.39:0/3759218599 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:42.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60e59c640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6100bcf90 0x7fb6100c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60ed9d640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb610157370 0x7fb610144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60dd9b640 1 -- 172.21.15.39:0/3759218599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb610157370 msgr2=0x7fb610144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60dd9b640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb610157370 0x7fb610144a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60dd9b640 1 -- 172.21.15.39:0/3759218599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6100bcf90 msgr2=0x7fb6100c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60dd9b640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6100bcf90 0x7fb6100c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60dd9b640 1 -- 172.21.15.39:0/3759218599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000094d0 con 0x7fb6101514d0 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60dd9b640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6101514d0 0x7fb6100aaac0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb60400e840 tx=0x7fb60400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb5feffd640 1 -- 172.21.15.39:0/3759218599 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604009510 con 0x7fb6101514d0 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb5feffd640 1 -- 172.21.15.39:0/3759218599 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60400ee90 con 0x7fb6101514d0 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6100aa180 con 0x7fb6101514d0 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb5feffd640 1 -- 172.21.15.39:0/3759218599 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60400cb80 con 0x7fb6101514d0 2024-09-13T16:48:42.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6100ab000 con 0x7fb6101514d0 2024-09-13T16:48:42.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6100aa6a0 con 0x7fb6101514d0 2024-09-13T16:48:42.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb5feffd640 1 -- 172.21.15.39:0/3759218599 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb60400cce0 con 0x7fb6101514d0 2024-09-13T16:48:42.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb5feffd640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f00666c0 0x7fb5f0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb5feffd640 1 -- 172.21.15.39:0/3759218599 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fb604084470 con 0x7fb6101514d0 2024-09-13T16:48:42.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60e59c640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f00666c0 0x7fb5f0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.675+0000 7fb60e59c640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f00666c0 0x7fb5f0068b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6000ae000 tx=0x7fb600009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:42.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.679+0000 7fb5feffd640 1 -- 172.21.15.39:0/3759218599 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb604051580 con 0x7fb6101514d0 2024-09-13T16:48:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.835+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb610145bd0 con 0x7fb6101514d0 2024-09-13T16:48:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.835+0000 7fb5feffd640 1 -- 172.21.15.39:0/3759218599 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb604054ba0 con 0x7fb6101514d0 2024-09-13T16:48:42.841 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:48:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f00666c0 msgr2=0x7fb5f0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f00666c0 0x7fb5f0068b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6000ae000 tx=0x7fb600009040 comp rx=0 tx=0).stop 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6101514d0 msgr2=0x7fb6100aaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6101514d0 0x7fb6100aaac0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb60400e840 tx=0x7fb60400ec40 comp rx=0 tx=0).stop 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 shutdown_connections 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f00666c0 0x7fb5f0068b80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb610157370 0x7fb610144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6101514d0 0x7fb6100aaac0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 --2- 172.21.15.39:0/3759218599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6100bcf90 0x7fb6100c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 >> 172.21.15.39:0/3759218599 conn(0x7fb61006a210 msgr2=0x7fb610155490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 shutdown_connections 2024-09-13T16:48:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.839+0000 7fb614be1640 1 -- 172.21.15.39:0/3759218599 wait complete. 2024-09-13T16:48:42.859 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T16:48:42.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.967+0000 7fa43f9e3640 1 Processor -- start 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.967+0000 7fa43f9e3640 1 -- start start 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43f9e3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa438151200 0x7fa4380c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43f9e3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4380a5280 con 0x7fa4380c1810 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43f9e3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4380a4cc0 con 0x7fa4380c0ba0 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43f9e3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4380c20e0 con 0x7fa438151200 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43d77f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa4380c1810 0x7fa438150af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34904/0 (socket says 172.21.15.39:34904) 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43d77f640 1 -- 172.21.15.39:0/3805872063 learned_addr learned my addr 172.21.15.39:0/3805872063 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43df80640 1 --2- 172.21.15.39:0/3805872063 >> v2:172.21.15.39:3300/0 conn(0x7fa438151200 0x7fa4380c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437238011 0 0) 0x7fa4380a5280 con 0x7fa4380c1810 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa424003530 con 0x7fa4380c1810 2024-09-13T16:48:42.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3547511701 0 0) 0x7fa424003530 con 0x7fa4380c1810 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 >> v2:172.21.15.39:3300/0 conn(0x7fa438151200 msgr2=0x7fa4380c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 --2- 172.21.15.39:0/3805872063 >> v2:172.21.15.39:3300/0 conn(0x7fa438151200 0x7fa4380c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 >> v1:172.21.15.84:6789/0 conn(0x7fa4380c0ba0 legacy=0x7fa4380c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380a4fa0 con 0x7fa4380c1810 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fa434001680 con 0x7fa4380c1810 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 >> v1:172.21.15.39:6789/0 conn(0x7fa4380c1810 legacy=0x7fa438150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa424003980 0x7fa424003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4240047d0 0x7fa424004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa424005190 0x7fa42400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380c20e0 con 0x7fa424003980 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42400c030 con 0x7fa4240047d0 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42400c1f0 con 0x7fa424005190 2024-09-13T16:48:42.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43d77f640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa424003980 0x7fa424003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43cf7e640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4240047d0 0x7fa424004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43df80640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa424005190 0x7fa42400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43d77f640 1 -- 172.21.15.39:0/3805872063 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa424005190 msgr2=0x7fa42400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43d77f640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa424005190 0x7fa42400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43d77f640 1 -- 172.21.15.39:0/3805872063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4240047d0 msgr2=0x7fa424004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43d77f640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4240047d0 0x7fa424004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43d77f640 1 -- 172.21.15.39:0/3805872063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa424003530 con 0x7fa424003980 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43df80640 1 -- 172.21.15.39:0/3805872063 reap_dead start 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa43d77f640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa424003980 0x7fa424003da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa434007cb0 tx=0x7fa43401eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db5a8466ba00e30 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4340a7c60 con 0x7fa424003980 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4340a7dc0 con 0x7fa424003980 2024-09-13T16:48:42.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.971+0000 7fa42ffff640 1 -- 172.21.15.39:0/3805872063 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434005b20 con 0x7fa424003980 2024-09-13T16:48:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- 172.21.15.39:0/3805872063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa424003980 msgr2=0x7fa424003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa424003980 0x7fa424003da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa434007cb0 tx=0x7fa43401eda0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- 172.21.15.39:0/3805872063 shutdown_connections 2024-09-13T16:48:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 --2- 172.21.15.39:0/3805872063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa424003980 0x7fa424003da0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- 172.21.15.39:0/3805872063 >> 172.21.15.39:0/3805872063 conn(0x7fa4380c5790 msgr2=0x7fa4380c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- 172.21.15.39:0/3805872063 shutdown_connections 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- 172.21.15.39:0/3805872063 wait complete. 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 Processor -- start 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- start start 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa424003980 0x7fa4380a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa43806f750 0x7fa4380bf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa43806fbd0 0x7fa4380bf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380bfb50 con 0x7fa43806fbd0 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380aa4c0 con 0x7fa424003980 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380a9f00 con 0x7fa43806f750 2024-09-13T16:48:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43df80640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa43806fbd0 0x7fa4380bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43df80640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa43806fbd0 0x7fa4380bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57228/0 (socket says 172.21.15.39:57228) 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43df80640 1 -- 172.21.15.39:0/2377164643 learned_addr learned my addr 172.21.15.39:0/2377164643 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43cf7e640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa43806f750 0x7fa4380bf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43d77f640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa424003980 0x7fa4380a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43df80640 1 -- 172.21.15.39:0/2377164643 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa43806f750 msgr2=0x7fa4380bf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43df80640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa43806f750 0x7fa4380bf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43df80640 1 -- 172.21.15.39:0/2377164643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa424003980 msgr2=0x7fa4380a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43df80640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa424003980 0x7fa4380a8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43df80640 1 -- 172.21.15.39:0/2377164643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380aa1e0 con 0x7fa43806fbd0 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43d77f640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa424003980 0x7fa4380a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43df80640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa43806fbd0 0x7fa4380bf610 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa43806cd40 tx=0x7fa43000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa42dffb640 1 -- 172.21.15.39:0/2377164643 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430019070 con 0x7fa43806fbd0 2024-09-13T16:48:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa42dffb640 1 -- 172.21.15.39:0/2377164643 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa430010040 con 0x7fa43806fbd0 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4380a8060 con 0x7fa43806fbd0 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa42dffb640 1 -- 172.21.15.39:0/2377164643 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43000fd50 con 0x7fa43806fbd0 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4380a8620 con 0x7fa43806fbd0 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.975+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4380a8810 con 0x7fa43806fbd0 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.979+0000 7fa42dffb640 1 -- 172.21.15.39:0/2377164643 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa430015480 con 0x7fa43806fbd0 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.979+0000 7fa42dffb640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa41c066690 0x7fa41c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.979+0000 7fa42dffb640 1 -- 172.21.15.39:0/2377164643 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fa430084500 con 0x7fa43806fbd0 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.979+0000 7fa43d77f640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa41c066690 0x7fa41c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.979+0000 7fa43d77f640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa41c066690 0x7fa41c068b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4340ad000 tx=0x7fa434006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:42.983+0000 7fa42dffb640 1 -- 172.21.15.39:0/2377164643 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa430051560 con 0x7fa43806fbd0 2024-09-13T16:48:43.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.135+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4380c3790 con 0x7fa43806fbd0 2024-09-13T16:48:43.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.135+0000 7fa42dffb640 1 -- 172.21.15.39:0/2377164643 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v26) v1 ==== 99+0+11 (secure 0 0 0) 0x7fa43000a2e0 con 0x7fa43806fbd0 2024-09-13T16:48:43.142 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa41c066690 msgr2=0x7fa41c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa41c066690 0x7fa41c068b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4340ad000 tx=0x7fa434006040 comp rx=0 tx=0).stop 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa43806fbd0 msgr2=0x7fa4380bf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa43806fbd0 0x7fa4380bf610 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa43806cd40 tx=0x7fa43000f510 comp rx=0 tx=0).stop 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 shutdown_connections 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa41c066690 0x7fa41c068b50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa43806fbd0 0x7fa4380bf610 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa43806f750 0x7fa4380bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 --2- 172.21.15.39:0/2377164643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa424003980 0x7fa4380a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 >> 172.21.15.39:0/2377164643 conn(0x7fa4380c5790 msgr2=0x7fa4381538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 shutdown_connections 2024-09-13T16:48:43.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.139+0000 7fa43f9e3640 1 -- 172.21.15.39:0/2377164643 wait complete. 2024-09-13T16:48:43.160 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T16:48:43.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63bbdb1640 1 Processor -- start 2024-09-13T16:48:43.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63bbdb1640 1 -- start start 2024-09-13T16:48:43.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63bbdb1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63b4151200 0x7f63b40c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63bbdb1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b40a5280 con 0x7f63b40c0ba0 2024-09-13T16:48:43.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63bbdb1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b40a4cc0 con 0x7f63b40c1810 2024-09-13T16:48:43.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63bbdb1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b40c20e0 con 0x7f63b4151200 2024-09-13T16:48:43.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63b934c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f63b40c0ba0 0x7f63b40c3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34912/0 (socket says 172.21.15.39:34912) 2024-09-13T16:48:43.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63b934c640 1 -- 172.21.15.39:0/1045211998 learned_addr learned my addr 172.21.15.39:0/1045211998 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:43.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63ba34e640 1 --2- 172.21.15.39:0/1045211998 >> v2:172.21.15.39:3300/0 conn(0x7f63b4151200 0x7f63b40c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63ba34e640 1 -- 172.21.15.39:0/1045211998 >> v1:172.21.15.84:6789/0 conn(0x7f63b40c1810 legacy=0x7f63b4150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63ba34e640 1 -- 172.21.15.39:0/1045211998 >> v1:172.21.15.39:6789/0 conn(0x7f63b40c0ba0 legacy=0x7f63b40c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63ba34e640 1 -- 172.21.15.39:0/1045211998 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b40a4fa0 con 0x7f63b4151200 2024-09-13T16:48:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.275+0000 7f63b8b4b640 1 -- 172.21.15.39:0/1045211998 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763053933 0 0) 0x7f63b40a5280 con 0x7f63b40c0ba0 2024-09-13T16:48:43.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63ba34e640 1 --2- 172.21.15.39:0/1045211998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b4151200 0x7f63b40c3ea0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009d90 tx=0x7f63ac01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65264f35d7d97b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:43.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b8b4b640 1 -- 172.21.15.39:0/1045211998 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0a5a00 con 0x7f63b4151200 2024-09-13T16:48:43.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b8b4b640 1 -- 172.21.15.39:0/1045211998 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac0a5b60 con 0x7f63b4151200 2024-09-13T16:48:43.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b8b4b640 1 -- 172.21.15.39:0/1045211998 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0ad440 con 0x7f63b4151200 2024-09-13T16:48:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- 172.21.15.39:0/1045211998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b4151200 msgr2=0x7f63b40c3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 --2- 172.21.15.39:0/1045211998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b4151200 0x7f63b40c3ea0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009d90 tx=0x7f63ac01c940 comp rx=0 tx=0).stop 2024-09-13T16:48:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- 172.21.15.39:0/1045211998 shutdown_connections 2024-09-13T16:48:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 --2- 172.21.15.39:0/1045211998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b4151200 0x7f63b40c3ea0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- 172.21.15.39:0/1045211998 >> 172.21.15.39:0/1045211998 conn(0x7f63b40c5790 msgr2=0x7f63b40c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- 172.21.15.39:0/1045211998 shutdown_connections 2024-09-13T16:48:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- 172.21.15.39:0/1045211998 wait complete. 2024-09-13T16:48:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 Processor -- start 2024-09-13T16:48:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- start start 2024-09-13T16:48:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63b40c0ba0 0x7f63b40a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b40c1810 0x7f63b41ea750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63b4151200 0x7f63b41eac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41eb1d0 con 0x7f63b40c1810 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40a82f0 con 0x7f63b40c0ba0 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b40a8010 con 0x7f63b4151200 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b934c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b40c1810 0x7f63b41ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b934c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b40c1810 0x7f63b41ea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57256/0 (socket says 172.21.15.39:57256) 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b934c640 1 -- 172.21.15.39:0/4198011007 learned_addr learned my addr 172.21.15.39:0/4198011007 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b9b4d640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63b40c0ba0 0x7f63b40a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63ba34e640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63b4151200 0x7f63b41eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b934c640 1 -- 172.21.15.39:0/4198011007 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63b4151200 msgr2=0x7f63b41eac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b934c640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63b4151200 0x7f63b41eac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b934c640 1 -- 172.21.15.39:0/4198011007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63b40c0ba0 msgr2=0x7f63b40a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b934c640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63b40c0ba0 0x7f63b40a97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b934c640 1 -- 172.21.15.39:0/4198011007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac009a70 con 0x7f63b40c1810 2024-09-13T16:48:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63b934c640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b40c1810 0x7f63b41ea750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f63a400c4a0 tx=0x7f63a400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63aa7fc640 1 -- 172.21.15.39:0/4198011007 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40074e0 con 0x7f63b40c1810 2024-09-13T16:48:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63aa7fc640 1 -- 172.21.15.39:0/4198011007 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a4007640 con 0x7f63b40c1810 2024-09-13T16:48:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b40aa360 con 0x7f63b40c1810 2024-09-13T16:48:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63aa7fc640 1 -- 172.21.15.39:0/4198011007 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4004a30 con 0x7f63b40c1810 2024-09-13T16:48:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.279+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b40aa9b0 con 0x7f63b40c1810 2024-09-13T16:48:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.283+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b40a9510 con 0x7f63b40c1810 2024-09-13T16:48:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.283+0000 7f63aa7fc640 1 -- 172.21.15.39:0/4198011007 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63a4004b90 con 0x7f63b40c1810 2024-09-13T16:48:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.283+0000 7f63aa7fc640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6398066690 0x7f6398068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.283+0000 7f63b9b4d640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6398066690 0x7f6398068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.283+0000 7f63aa7fc640 1 -- 172.21.15.39:0/4198011007 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f63a4083710 con 0x7f63b40c1810 2024-09-13T16:48:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.283+0000 7f63b9b4d640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6398066690 0x7f6398068b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f63b40bfb80 tx=0x7f63b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.287+0000 7f63aa7fc640 1 -- 172.21.15.39:0/4198011007 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f63b40a9510 con 0x7f63b40c1810 2024-09-13T16:48:43.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.439+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f63b40a9510 con 0x7f63b40c1810 2024-09-13T16:48:43.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.439+0000 7f63aa7fc640 1 -- 172.21.15.39:0/4198011007 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v26) v1 ==== 95+0+11 (secure 0 0 0) 0x7f63b40a9510 con 0x7f63b40c1810 2024-09-13T16:48:43.446 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6398066690 msgr2=0x7f6398068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6398066690 0x7f6398068b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f63b40bfb80 tx=0x7f63b0009040 comp rx=0 tx=0).stop 2024-09-13T16:48:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b40c1810 msgr2=0x7f63b41ea750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b40c1810 0x7f63b41ea750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f63a400c4a0 tx=0x7f63a400c960 comp rx=0 tx=0).stop 2024-09-13T16:48:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 shutdown_connections 2024-09-13T16:48:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6398066690 0x7f6398068b50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63b4151200 0x7f63b41eac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63b40c1810 0x7f63b41ea750 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 --2- 172.21.15.39:0/4198011007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63b40c0ba0 0x7f63b40a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 >> 172.21.15.39:0/4198011007 conn(0x7f63b40c5790 msgr2=0x7f63b414fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:43.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 shutdown_connections 2024-09-13T16:48:43.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.443+0000 7f63bbdb1640 1 -- 172.21.15.39:0/4198011007 wait complete. 2024-09-13T16:48:43.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T16:48:43.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f16027e8640 1 Processor -- start 2024-09-13T16:48:43.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f16027e8640 1 -- start start 2024-09-13T16:48:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f16027e8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f15fc0bfc10 0x7f15fc0a8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f16027e8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15fc155190 0x7f15fc1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f16027e8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc0c47c0 con 0x7f15fc14f320 2024-09-13T16:48:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f16027e8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc155b20 con 0x7f15fc0bfc10 2024-09-13T16:48:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f16027e8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc151cb0 con 0x7f15fc155190 2024-09-13T16:48:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f15fbfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f15fc14f320 0x7f15fc0c2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34924/0 (socket says 172.21.15.39:34924) 2024-09-13T16:48:43.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f15fbfff640 1 -- 172.21.15.39:0/4236026534 learned_addr learned my addr 172.21.15.39:0/4236026534 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f15fb7fe640 1 --2- 172.21.15.39:0/4236026534 >> v2:172.21.15.39:3300/0 conn(0x7f15fc0bfc10 0x7f15fc0a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f1600d85640 1 --2- 172.21.15.39:0/4236026534 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15fc155190 0x7f15fc1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f15faffd640 1 -- 172.21.15.39:0/4236026534 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459523003 0 0) 0x7f15fc0c47c0 con 0x7f15fc14f320 2024-09-13T16:48:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f15faffd640 1 -- 172.21.15.39:0/4236026534 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e4003530 con 0x7f15fc14f320 2024-09-13T16:48:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f15fb7fe640 1 -- 172.21.15.39:0/4236026534 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15fc155190 msgr2=0x7f15fc1555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f15fb7fe640 1 --2- 172.21.15.39:0/4236026534 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15fc155190 0x7f15fc1555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f15fb7fe640 1 -- 172.21.15.39:0/4236026534 >> v1:172.21.15.39:6789/0 conn(0x7f15fc14f320 legacy=0x7f15fc0c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.575+0000 7f15fb7fe640 1 -- 172.21.15.39:0/4236026534 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc151f90 con 0x7f15fc0bfc10 2024-09-13T16:48:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fb7fe640 1 --2- 172.21.15.39:0/4236026534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 0x7f15fc0a8840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0097c0 tx=0x7f15ec01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e626b4d0593a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15faffd640 1 -- 172.21.15.39:0/4236026534 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0ac640 con 0x7f15fc0bfc10 2024-09-13T16:48:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15faffd640 1 -- 172.21.15.39:0/4236026534 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec0ac7a0 con 0x7f15fc0bfc10 2024-09-13T16:48:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15faffd640 1 -- 172.21.15.39:0/4236026534 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0acb10 con 0x7f15fc0bfc10 2024-09-13T16:48:43.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- 172.21.15.39:0/4236026534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 msgr2=0x7f15fc0a8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 --2- 172.21.15.39:0/4236026534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 0x7f15fc0a8840 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0097c0 tx=0x7f15ec01c3e0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- 172.21.15.39:0/4236026534 shutdown_connections 2024-09-13T16:48:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 --2- 172.21.15.39:0/4236026534 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15fc155190 0x7f15fc1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 --2- 172.21.15.39:0/4236026534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 0x7f15fc0a8840 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- 172.21.15.39:0/4236026534 >> 172.21.15.39:0/4236026534 conn(0x7f15fc0c5790 msgr2=0x7f15fc0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- 172.21.15.39:0/4236026534 shutdown_connections 2024-09-13T16:48:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- 172.21.15.39:0/4236026534 wait complete. 2024-09-13T16:48:43.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 Processor -- start 2024-09-13T16:48:43.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- start start 2024-09-13T16:48:43.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 0x7f15fc0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc14f320 0x7f15fc14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15fc155190 0x7f15fc0c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc0c0350 con 0x7f15fc0bfc10 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc14de60 con 0x7f15fc14f320 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc0a6680 con 0x7f15fc155190 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 0x7f15fc0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 0x7f15fc0a6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57272/0 (socket says 172.21.15.39:57272) 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fbfff640 1 -- 172.21.15.39:0/3732587139 learned_addr learned my addr 172.21.15.39:0/3732587139 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fb7fe640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc14f320 0x7f15fc14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fbfff640 1 -- 172.21.15.39:0/3732587139 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15fc155190 msgr2=0x7f15fc0c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fbfff640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15fc155190 0x7f15fc0c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fbfff640 1 -- 172.21.15.39:0/3732587139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc14f320 msgr2=0x7f15fc14ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fbfff640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc14f320 0x7f15fc14ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fbfff640 1 -- 172.21.15.39:0/3732587139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec009420 con 0x7f15fc0bfc10 2024-09-13T16:48:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15fbfff640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 0x7f15fc0a6140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c3c0 tx=0x7f15f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15f8ff9640 1 -- 172.21.15.39:0/3732587139 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0005cf0 con 0x7f15fc0bfc10 2024-09-13T16:48:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15f8ff9640 1 -- 172.21.15.39:0/3732587139 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f0005e50 con 0x7f15fc0bfc10 2024-09-13T16:48:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc0c0070 con 0x7f15fc0bfc10 2024-09-13T16:48:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f15f8ff9640 1 -- 172.21.15.39:0/3732587139 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00072f0 con 0x7f15fc0bfc10 2024-09-13T16:48:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.579+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc14db80 con 0x7f15fc0bfc10 2024-09-13T16:48:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.583+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc145cb0 con 0x7f15fc0bfc10 2024-09-13T16:48:43.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.583+0000 7f15f8ff9640 1 -- 172.21.15.39:0/3732587139 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f15f0007450 con 0x7f15fc0bfc10 2024-09-13T16:48:43.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.583+0000 7f15f8ff9640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15e8066690 0x7f15e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.583+0000 7f15fb7fe640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15e8066690 0x7f15e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.583+0000 7f15f8ff9640 1 -- 172.21.15.39:0/3732587139 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f15f0083ab0 con 0x7f15fc0bfc10 2024-09-13T16:48:43.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.583+0000 7f15fb7fe640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15e8066690 0x7f15e8068b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0ae000 tx=0x7f15ec0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:43.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.587+0000 7f15f8ff9640 1 -- 172.21.15.39:0/3732587139 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15fc145cb0 con 0x7f15fc0bfc10 2024-09-13T16:48:43.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.739+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15fc14d9e0 con 0x7f15fc0bfc10 2024-09-13T16:48:43.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.739+0000 7f15f8ff9640 1 -- 172.21.15.39:0/3732587139 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v26) v1 ==== 99+0+11 (secure 0 0 0) 0x7f15fc14d9e0 con 0x7f15fc0bfc10 2024-09-13T16:48:43.745 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15e8066690 msgr2=0x7f15e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15e8066690 0x7f15e8068b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0ae000 tx=0x7f15ec0a7000 comp rx=0 tx=0).stop 2024-09-13T16:48:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 msgr2=0x7f15fc0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 0x7f15fc0a6140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f15f000c3c0 tx=0x7f15f000c880 comp rx=0 tx=0).stop 2024-09-13T16:48:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 shutdown_connections 2024-09-13T16:48:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15e8066690 0x7f15e8068b50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15fc14f320 0x7f15fc14ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15fc155190 0x7f15fc0c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 --2- 172.21.15.39:0/3732587139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15fc0bfc10 0x7f15fc0a6140 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 >> 172.21.15.39:0/3732587139 conn(0x7f15fc0c5790 msgr2=0x7f15fc154340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 shutdown_connections 2024-09-13T16:48:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.743+0000 7f16027e8640 1 -- 172.21.15.39:0/3732587139 wait complete. 2024-09-13T16:48:43.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T16:48:43.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f749983e640 1 Processor -- start 2024-09-13T16:48:43.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f749983e640 1 -- start start 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f749983e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74941514d0 0x7f74940c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f749983e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f74940bcf90 0x7f7494154240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f749983e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7494155270 con 0x7f7494157370 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f749983e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7494153d80 con 0x7f74940bcf90 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f749983e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7494157870 con 0x7f74941514d0 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74937fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7494157370 0x7f7494154780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34934/0 (socket says 172.21.15.39:34934) 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74937fe640 1 -- 172.21.15.39:0/2039674365 learned_addr learned my addr 172.21.15.39:0/2039674365 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7492ffd640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74941514d0 0x7f74940c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74927fc640 1 --2- 172.21.15.39:0/2039674365 >> v2:172.21.15.84:3300/0 conn(0x7f74940bcf90 0x7f7494154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737855058 0 0) 0x7f7494155270 con 0x7f7494157370 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f747c003530 con 0x7f7494157370 2024-09-13T16:48:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3762605218 0 0) 0x7f747c003530 con 0x7f7494157370 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74941514d0 msgr2=0x7f74940c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74941514d0 0x7f74940c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 >> v2:172.21.15.84:3300/0 conn(0x7f74940bcf90 msgr2=0x7f7494154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 --2- 172.21.15.39:0/2039674365 >> v2:172.21.15.84:3300/0 conn(0x7f74940bcf90 0x7f7494154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494157b50 con 0x7f7494157370 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f748c001a00 con 0x7f7494157370 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 >> v1:172.21.15.39:6789/0 conn(0x7f7494157370 legacy=0x7f7494154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74941514d0 0x7f747c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f747c003f00 0x7f747c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f747c004d10 0x7f747c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494153d80 con 0x7f747c004d10 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494157870 con 0x7f747c003f00 2024-09-13T16:48:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c00b9e0 con 0x7f74941514d0 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74937fe640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f747c004d10 0x7f747c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7492ffd640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74941514d0 0x7f747c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74927fc640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f747c003f00 0x7f747c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74937fe640 1 -- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74941514d0 msgr2=0x7f747c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74937fe640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74941514d0 0x7f747c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74937fe640 1 -- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f747c003f00 msgr2=0x7f747c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74937fe640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f747c003f00 0x7f747c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74937fe640 1 -- 172.21.15.39:0/2039674365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c003530 con 0x7f747c004d10 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f7492ffd640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74941514d0 0x7f747c003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:43.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.875+0000 7f74927fc640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f747c003f00 0x7f747c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:43.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f747c004d10 0x7f747c005190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f748c00a8d0 tx=0x7f748c01fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa865dcd2d0feca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:43.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0aa6c0 con 0x7f747c004d10 2024-09-13T16:48:43.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c0aa820 con 0x7f747c004d10 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f7491ffb640 1 -- 172.21.15.39:0/2039674365 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0aabd0 con 0x7f747c004d10 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 -- 172.21.15.39:0/2039674365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f747c004d10 msgr2=0x7f747c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 --2- 172.21.15.39:0/2039674365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f747c004d10 0x7f747c005190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f748c00a8d0 tx=0x7f748c01fb20 comp rx=0 tx=0).stop 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 -- 172.21.15.39:0/2039674365 reap_dead start 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 -- 172.21.15.39:0/2039674365 shutdown_connections 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 -- 172.21.15.39:0/2039674365 >> 172.21.15.39:0/2039674365 conn(0x7f749406a210 msgr2=0x7f749406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 -- 172.21.15.39:0/2039674365 shutdown_connections 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 -- 172.21.15.39:0/2039674365 wait complete. 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 Processor -- start 2024-09-13T16:48:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 -- start start 2024-09-13T16:48:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f747c004d10 0x7f7494150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74940c86d0 0x7f74940aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74940aaf20 0x7f7494149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74940be600 con 0x7f74940aaf20 2024-09-13T16:48:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74940bdfb0 con 0x7f74940c86d0 2024-09-13T16:48:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f749983e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74940be290 con 0x7f747c004d10 2024-09-13T16:48:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74940aaf20 0x7f7494149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74940aaf20 0x7f7494149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57282/0 (socket says 172.21.15.39:57282) 2024-09-13T16:48:43.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 -- 172.21.15.39:0/2582841242 learned_addr learned my addr 172.21.15.39:0/2582841242 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74927fc640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74940c86d0 0x7f74940aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f7492ffd640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f747c004d10 0x7f7494150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 -- 172.21.15.39:0/2582841242 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f747c004d10 msgr2=0x7f7494150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f747c004d10 0x7f7494150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 -- 172.21.15.39:0/2582841242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74940c86d0 msgr2=0x7f74940aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74940c86d0 0x7f74940aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.879+0000 7f74937fe640 1 -- 172.21.15.39:0/2582841242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940aef70 con 0x7f74940aaf20 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f74937fe640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74940aaf20 0x7f7494149cb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f748c0203c0 tx=0x7f747c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f747bfff640 1 -- 172.21.15.39:0/2582841242 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0aa6c0 con 0x7f74940aaf20 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f747bfff640 1 -- 172.21.15.39:0/2582841242 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c0aae80 con 0x7f74940aaf20 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74940aed00 con 0x7f74940aaf20 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f747bfff640 1 -- 172.21.15.39:0/2582841242 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0019c0 con 0x7f74940aaf20 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74940af1b0 con 0x7f74940aaf20 2024-09-13T16:48:43.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74940be0f0 con 0x7f74940aaf20 2024-09-13T16:48:43.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f747bfff640 1 -- 172.21.15.39:0/2582841242 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f748c004b70 con 0x7f74940aaf20 2024-09-13T16:48:43.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f747bfff640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7480066690 0x7f7480068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:43.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f7492ffd640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7480066690 0x7f7480068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:43.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f747bfff640 1 -- 172.21.15.39:0/2582841242 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f748c0b9070 con 0x7f74940aaf20 2024-09-13T16:48:43.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.883+0000 7f7492ffd640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7480066690 0x7f7480068b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7488009ac0 tx=0x7f7488008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:43.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:43.887+0000 7f747bfff640 1 -- 172.21.15.39:0/2582841242 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74940be0f0 con 0x7f74940aaf20 2024-09-13T16:48:44.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7f74940be0f0 con 0x7f74940aaf20 2024-09-13T16:48:44.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f747bfff640 1 -- 172.21.15.39:0/2582841242 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v26) v1 ==== 95+0+11 (secure 0 0 0) 0x7f74940be0f0 con 0x7f74940aaf20 2024-09-13T16:48:44.049 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7480066690 msgr2=0x7f7480068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7480066690 0x7f7480068b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7488009ac0 tx=0x7f7488008040 comp rx=0 tx=0).stop 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74940aaf20 msgr2=0x7f7494149cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74940aaf20 0x7f7494149cb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f748c0203c0 tx=0x7f747c005800 comp rx=0 tx=0).stop 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 shutdown_connections 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7480066690 0x7f7480068b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74940c86d0 0x7f74940aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74940aaf20 0x7f7494149cb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 --2- 172.21.15.39:0/2582841242 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f747c004d10 0x7f7494150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.043+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 >> 172.21.15.39:0/2582841242 conn(0x7f749406a210 msgr2=0x7f74940c9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:44.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.047+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 shutdown_connections 2024-09-13T16:48:44.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.047+0000 7f749983e640 1 -- 172.21.15.39:0/2582841242 wait complete. 2024-09-13T16:48:44.066 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:44.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.171+0000 7f8139e9f640 1 Processor -- start 2024-09-13T16:48:44.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.171+0000 7f8139e9f640 1 -- start start 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81340a4e70 0x7f8134151460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340c4530 0x7f81340c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8134154100 con 0x7f813406d020 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81340a8c50 con 0x7f81340a4e70 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81341519a0 con 0x7f81340c4530 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8132ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81340a4e70 0x7f8134151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f81337fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f813406d020 0x7f813414db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34946/0 (socket says 172.21.15.39:34946) 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f81337fe640 1 -- 172.21.15.39:0/2988552736 learned_addr learned my addr 172.21.15.39:0/2988552736 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8133fff640 1 --2- 172.21.15.39:0/2988552736 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340c4530 0x7f81340c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8132ffd640 1 -- 172.21.15.39:0/2988552736 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340c4530 msgr2=0x7f81340c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:44.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8132ffd640 1 --2- 172.21.15.39:0/2988552736 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340c4530 0x7f81340c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8132ffd640 1 -- 172.21.15.39:0/2988552736 >> v1:172.21.15.39:6789/0 conn(0x7f813406d020 legacy=0x7f813414db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8132ffd640 1 -- 172.21.15.39:0/2988552736 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81341543e0 con 0x7f81340a4e70 2024-09-13T16:48:44.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8132ffd640 1 --2- 172.21.15.39:0/2988552736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81340a4e70 0x7f8134151460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f81280097c0 tx=0x7f812801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b05826161546ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:44.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f81327fc640 1 -- 172.21.15.39:0/2988552736 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280ab600 con 0x7f81340a4e70 2024-09-13T16:48:44.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f81327fc640 1 -- 172.21.15.39:0/2988552736 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81280a7050 con 0x7f81340a4e70 2024-09-13T16:48:44.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f81327fc640 1 -- 172.21.15.39:0/2988552736 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280ab980 con 0x7f81340a4e70 2024-09-13T16:48:44.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 -- 172.21.15.39:0/2988552736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81340a4e70 msgr2=0x7f8134151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 --2- 172.21.15.39:0/2988552736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81340a4e70 0x7f8134151460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f81280097c0 tx=0x7f812801c3e0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 -- 172.21.15.39:0/2988552736 shutdown_connections 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 --2- 172.21.15.39:0/2988552736 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340c4530 0x7f81340c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.175+0000 7f8139e9f640 1 --2- 172.21.15.39:0/2988552736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81340a4e70 0x7f8134151460 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- 172.21.15.39:0/2988552736 >> 172.21.15.39:0/2988552736 conn(0x7f8134069e10 msgr2=0x7f813406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- 172.21.15.39:0/2988552736 shutdown_connections 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- 172.21.15.39:0/2988552736 wait complete. 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 Processor -- start 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- start start 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f813406d020 0x7f81340a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340a4e70 0x7f813406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81340c4530 0x7f813406ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813406f5a0 con 0x7f813406d020 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8134071f10 con 0x7f81340c4530 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81340721f0 con 0x7f81340a4e70 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f81337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f813406d020 0x7f81340a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f81337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f813406d020 0x7f81340a7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57304/0 (socket says 172.21.15.39:57304) 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f81337fe640 1 -- 172.21.15.39:0/3635455362 learned_addr learned my addr 172.21.15.39:0/3635455362 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8133fff640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81340c4530 0x7f813406ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8132ffd640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340a4e70 0x7f813406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f81337fe640 1 -- 172.21.15.39:0/3635455362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340a4e70 msgr2=0x7f813406fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f81337fe640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340a4e70 0x7f813406fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f81337fe640 1 -- 172.21.15.39:0/3635455362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81340c4530 msgr2=0x7f813406ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f81337fe640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81340c4530 0x7f813406ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f81337fe640 1 -- 172.21.15.39:0/3635455362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8128009420 con 0x7f813406d020 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8133fff640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81340c4530 0x7f813406ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f81337fe640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f813406d020 0x7f81340a7c70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f812400c420 tx=0x7f812400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f811bfff640 1 -- 172.21.15.39:0/3635455362 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124005cf0 con 0x7f813406d020 2024-09-13T16:48:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f811bfff640 1 -- 172.21.15.39:0/3635455362 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124005e50 con 0x7f813406d020 2024-09-13T16:48:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f811bfff640 1 -- 172.21.15.39:0/3635455362 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81240072f0 con 0x7f813406d020 2024-09-13T16:48:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81340724d0 con 0x7f813406d020 2024-09-13T16:48:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81340a81b0 con 0x7f813406d020 2024-09-13T16:48:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.179+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8134151460 con 0x7f813406d020 2024-09-13T16:48:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.183+0000 7f811bfff640 1 -- 172.21.15.39:0/3635455362 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8124007450 con 0x7f813406d020 2024-09-13T16:48:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.183+0000 7f811bfff640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8120066690 0x7f8120068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.183+0000 7f811bfff640 1 -- 172.21.15.39:0/3635455362 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f8124083dd0 con 0x7f813406d020 2024-09-13T16:48:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.183+0000 7f8132ffd640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8120066690 0x7f8120068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.183+0000 7f8132ffd640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8120066690 0x7f8120068b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f81280ad000 tx=0x7f81280a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.187+0000 7f811bfff640 1 -- 172.21.15.39:0/3635455362 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8124050f10 con 0x7f813406d020 2024-09-13T16:48:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.375+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f813415a390 con 0x7f813406d020 2024-09-13T16:48:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.375+0000 7f811bfff640 1 -- 172.21.15.39:0/3635455362 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7f8124017020 con 0x7f813406d020 2024-09-13T16:48:44.381 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:44.381 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:48:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:48:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8120066690 msgr2=0x7f8120068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8120066690 0x7f8120068b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f81280ad000 tx=0x7f81280a6000 comp rx=0 tx=0).stop 2024-09-13T16:48:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f813406d020 msgr2=0x7f81340a7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f813406d020 0x7f81340a7c70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f812400c420 tx=0x7f812400c8e0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 shutdown_connections 2024-09-13T16:48:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8120066690 0x7f8120068b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81340c4530 0x7f813406ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81340a4e70 0x7f813406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 --2- 172.21.15.39:0/3635455362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f813406d020 0x7f81340a7c70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 >> 172.21.15.39:0/3635455362 conn(0x7f8134069e10 msgr2=0x7f813406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:44.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 shutdown_connections 2024-09-13T16:48:44.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.379+0000 7f8139e9f640 1 -- 172.21.15.39:0/3635455362 wait complete. 2024-09-13T16:48:44.398 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d70851640 1 Processor -- start 2024-09-13T16:48:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d70851640 1 -- start start 2024-09-13T16:48:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d70851640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d6c06c870 0x7f5d6c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d70851640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d6c0c40b0 0x7f5d6c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d70851640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6c152a80 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d70851640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c0c30f0 con 0x7f5d6c06c870 2024-09-13T16:48:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d70851640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c151e00 con 0x7f5d6c0c40b0 2024-09-13T16:48:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d69d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d6c06c870 0x7f5d6c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d69d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d6c06c870 0x7f5d6c0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57316/0 (socket says 172.21.15.39:57316) 2024-09-13T16:48:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d69d9b640 1 -- 172.21.15.39:0/4062147711 learned_addr learned my addr 172.21.15.39:0/4062147711 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d6ad9d640 1 --2- 172.21.15.39:0/4062147711 >> v2:172.21.15.84:3300/0 conn(0x7f5d6c0c40b0 0x7f5d6c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d69d9b640 1 -- 172.21.15.39:0/4062147711 >> v2:172.21.15.84:3300/0 conn(0x7f5d6c0c40b0 msgr2=0x7f5d6c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:44.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d69d9b640 1 --2- 172.21.15.39:0/4062147711 >> v2:172.21.15.84:3300/0 conn(0x7f5d6c0c40b0 0x7f5d6c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d69d9b640 1 -- 172.21.15.39:0/4062147711 >> v1:172.21.15.84:6789/0 conn(0x7f5d6c0aabc0 legacy=0x7f5d6c0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.519+0000 7f5d69d9b640 1 -- 172.21.15.39:0/4062147711 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c1520e0 con 0x7f5d6c06c870 2024-09-13T16:48:44.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d69d9b640 1 --2- 172.21.15.39:0/4062147711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c06c870 0x7f5d6c0aafa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c06f220 tx=0x7f5d6001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64aefbde257917b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:44.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d6959a640 1 -- 172.21.15.39:0/4062147711 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d600ab660 con 0x7f5d6c06c870 2024-09-13T16:48:44.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d6959a640 1 -- 172.21.15.39:0/4062147711 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d600ab7c0 con 0x7f5d6c06c870 2024-09-13T16:48:44.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d6959a640 1 -- 172.21.15.39:0/4062147711 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d600abb50 con 0x7f5d6c06c870 2024-09-13T16:48:44.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d70851640 1 -- 172.21.15.39:0/4062147711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c06c870 msgr2=0x7f5d6c0aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d70851640 1 --2- 172.21.15.39:0/4062147711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c06c870 0x7f5d6c0aafa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c06f220 tx=0x7f5d6001c460 comp rx=0 tx=0).stop 2024-09-13T16:48:44.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d70851640 1 -- 172.21.15.39:0/4062147711 shutdown_connections 2024-09-13T16:48:44.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d70851640 1 --2- 172.21.15.39:0/4062147711 >> v2:172.21.15.84:3300/0 conn(0x7f5d6c0c40b0 0x7f5d6c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d70851640 1 --2- 172.21.15.39:0/4062147711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c06c870 0x7f5d6c0aafa0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d70851640 1 -- 172.21.15.39:0/4062147711 >> 172.21.15.39:0/4062147711 conn(0x7f5d6c069e10 msgr2=0x7f5d6c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:44.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d70851640 1 -- 172.21.15.39:0/4062147711 shutdown_connections 2024-09-13T16:48:44.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.523+0000 7f5d70851640 1 -- 172.21.15.39:0/4062147711 wait complete. 2024-09-13T16:48:44.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 Processor -- start 2024-09-13T16:48:44.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 -- start start 2024-09-13T16:48:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d6c06c870 0x7f5d6c14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c0aabc0 0x7f5d6c150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d6c0c40b0 0x7f5d6c141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c14aa90 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c14a720 con 0x7f5d6c0c40b0 2024-09-13T16:48:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c1edc90 con 0x7f5d6c06c870 2024-09-13T16:48:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c0aabc0 0x7f5d6c150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d69d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c0aabc0 0x7f5d6c150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57326/0 (socket says 172.21.15.39:57326) 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d69d9b640 1 -- 172.21.15.39:0/1947140302 learned_addr learned my addr 172.21.15.39:0/1947140302 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d6a59c640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d6c06c870 0x7f5d6c14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d6ad9d640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d6c0c40b0 0x7f5d6c141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d69d9b640 1 -- 172.21.15.39:0/1947140302 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d6c06c870 msgr2=0x7f5d6c14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d69d9b640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d6c06c870 0x7f5d6c14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d69d9b640 1 -- 172.21.15.39:0/1947140302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d6c0c40b0 msgr2=0x7f5d6c141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d69d9b640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d6c0c40b0 0x7f5d6c141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d69d9b640 1 -- 172.21.15.39:0/1947140302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d600094d0 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d6ad9d640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d6c0c40b0 0x7f5d6c141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d69d9b640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c0aabc0 0x7f5d6c150c10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d60019070 tx=0x7f5d6001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d5affd640 1 -- 172.21.15.39:0/1947140302 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d600b4800 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d5affd640 1 -- 172.21.15.39:0/1947140302 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d600b4960 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c14b760 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d5affd640 1 -- 172.21.15.39:0/1947140302 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d600b4cd0 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c1edea0 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6c0c3e60 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d5affd640 1 -- 172.21.15.39:0/1947140302 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d600a9070 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.527+0000 7f5d5affd640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d4c066690 0x7f5d4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.531+0000 7f5d5affd640 1 -- 172.21.15.39:0/1947140302 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f5d6011ef50 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.531+0000 7f5d6a59c640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d4c066690 0x7f5d4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.531+0000 7f5d6a59c640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d4c066690 0x7f5d4c068b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c148da0 tx=0x7f5d5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:44.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.535+0000 7f5d5affd640 1 -- 172.21.15.39:0/1947140302 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d600ec0d0 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.691+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d6c06c650 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d5affd640 1 -- 172.21.15.39:0/1947140302 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20376 (secure 0 0 0) 0x7f5d600ef6f0 con 0x7f5d6c0aabc0 2024-09-13T16:48:44.699 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:44.700 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:40.750805+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d4c066690 msgr2=0x7f5d4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d4c066690 0x7f5d4c068b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c148da0 tx=0x7f5d5c009040 comp rx=0 tx=0).stop 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c0aabc0 msgr2=0x7f5d6c150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c0aabc0 0x7f5d6c150c10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d60019070 tx=0x7f5d6001aca0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 shutdown_connections 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d4c066690 0x7f5d4c068b50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d6c0c40b0 0x7f5d6c141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d6c0aabc0 0x7f5d6c150c10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 --2- 172.21.15.39:0/1947140302 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d6c06c870 0x7f5d6c14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 >> 172.21.15.39:0/1947140302 conn(0x7f5d6c069e10 msgr2=0x7f5d6c14ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 shutdown_connections 2024-09-13T16:48:44.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.695+0000 7f5d70851640 1 -- 172.21.15.39:0/1947140302 wait complete. 2024-09-13T16:48:44.718 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:44.719 INFO:tasks.mds_thrash.fs.[a]:starting mds_do_thrash for fs a 2024-09-13T16:48:44.719 INFO:tasks.mds_thrash.fs.[a]:waiting for 109 secs before thrashing 2024-09-13T16:48:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.835+0000 7f3dda6fb640 1 Processor -- start 2024-09-13T16:48:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.835+0000 7f3dda6fb640 1 -- start start 2024-09-13T16:48:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dda6fb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3dd40c5330 0x7f3dd4150f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dda6fb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd4153930 0x7f3dd4153de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dda6fb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd41546c0 con 0x7f3dd40c1310 2024-09-13T16:48:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dda6fb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd40c19a0 con 0x7f3dd40c5330 2024-09-13T16:48:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dda6fb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd41514c0 con 0x7f3dd4153930 2024-09-13T16:48:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dd37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3dd40c5330 0x7f3dd4150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dd8c98640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd4153930 0x7f3dd4153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dd37fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3dd40c5330 0x7f3dd4150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54892/0 (socket says 172.21.15.39:54892) 2024-09-13T16:48:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dd37fe640 1 -- 172.21.15.39:0/1682647781 learned_addr learned my addr 172.21.15.39:0/1682647781 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dd37fe640 1 -- 172.21.15.39:0/1682647781 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd4153930 msgr2=0x7f3dd4153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dd37fe640 1 --2- 172.21.15.39:0/1682647781 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd4153930 0x7f3dd4153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dd37fe640 1 -- 172.21.15.39:0/1682647781 >> v1:172.21.15.84:6789/0 conn(0x7f3dd40c1310 legacy=0x7f3dd40c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.839+0000 7f3dd37fe640 1 -- 172.21.15.39:0/1682647781 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40c49f0 con 0x7f3dd40c5330 2024-09-13T16:48:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dd37fe640 1 --2- 172.21.15.39:0/1682647781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c5330 0x7f3dd4150f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0098a0 tx=0x7f3dbc01c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cf3d731659c5139 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dd2ffd640 1 -- 172.21.15.39:0/1682647781 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0ac600 con 0x7f3dd40c5330 2024-09-13T16:48:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dd2ffd640 1 -- 172.21.15.39:0/1682647781 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc0a8050 con 0x7f3dd40c5330 2024-09-13T16:48:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dd2ffd640 1 -- 172.21.15.39:0/1682647781 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0ac980 con 0x7f3dd40c5330 2024-09-13T16:48:44.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 -- 172.21.15.39:0/1682647781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c5330 msgr2=0x7f3dd4150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 --2- 172.21.15.39:0/1682647781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c5330 0x7f3dd4150f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0098a0 tx=0x7f3dbc01c4b0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 -- 172.21.15.39:0/1682647781 shutdown_connections 2024-09-13T16:48:44.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 --2- 172.21.15.39:0/1682647781 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd4153930 0x7f3dd4153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 --2- 172.21.15.39:0/1682647781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd40c5330 0x7f3dd4150f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 -- 172.21.15.39:0/1682647781 >> 172.21.15.39:0/1682647781 conn(0x7f3dd4069e10 msgr2=0x7f3dd406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 -- 172.21.15.39:0/1682647781 shutdown_connections 2024-09-13T16:48:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 -- 172.21.15.39:0/1682647781 wait complete. 2024-09-13T16:48:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 Processor -- start 2024-09-13T16:48:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 -- start start 2024-09-13T16:48:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd40c1310 0x7f3dd40aa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd40c5330 0x7f3dd40bcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd4153930 0x7f3dd414ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40bb3c0 con 0x7f3dd40c5330 2024-09-13T16:48:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40bbc60 con 0x7f3dd4153930 2024-09-13T16:48:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dda6fb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40bf930 con 0x7f3dd40c1310 2024-09-13T16:48:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dd37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd40c5330 0x7f3dd40bcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dd37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd40c5330 0x7f3dd40bcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57338/0 (socket says 172.21.15.39:57338) 2024-09-13T16:48:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.843+0000 7f3dd37fe640 1 -- 172.21.15.39:0/2463613624 learned_addr learned my addr 172.21.15.39:0/2463613624 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd3fff640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd40c1310 0x7f3dd40aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd8c98640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd4153930 0x7f3dd414ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd37fe640 1 -- 172.21.15.39:0/2463613624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd40c1310 msgr2=0x7f3dd40aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd37fe640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd40c1310 0x7f3dd40aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd37fe640 1 -- 172.21.15.39:0/2463613624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd4153930 msgr2=0x7f3dd414ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd37fe640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd4153930 0x7f3dd414ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd37fe640 1 -- 172.21.15.39:0/2463613624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc0094d0 con 0x7f3dd40c5330 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd37fe640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd40c5330 0x7f3dd40bcf50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc01ab60 tx=0x7f3dbc01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd0ff9640 1 -- 172.21.15.39:0/2463613624 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0b7420 con 0x7f3dd40c5330 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd0ff9640 1 -- 172.21.15.39:0/2463613624 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc0b7580 con 0x7f3dd40c5330 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd0ff9640 1 -- 172.21.15.39:0/2463613624 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0b78f0 con 0x7f3dd40c5330 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd40bf090 con 0x7f3dd40c5330 2024-09-13T16:48:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd40bf650 con 0x7f3dd40c5330 2024-09-13T16:48:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd41eb0e0 con 0x7f3dd40c5330 2024-09-13T16:48:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd0ff9640 1 -- 172.21.15.39:0/2463613624 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3dbc0b3040 con 0x7f3dd40c5330 2024-09-13T16:48:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd0ff9640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db40666c0 0x7f3db4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd0ff9640 1 -- 172.21.15.39:0/2463613624 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f3dbc11e7f0 con 0x7f3dd40c5330 2024-09-13T16:48:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd3fff640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db40666c0 0x7f3db4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.847+0000 7f3dd3fff640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db40666c0 0x7f3db4068b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc000f70 tx=0x7f3dcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:44.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:44.851+0000 7f3dd0ff9640 1 -- 172.21.15.39:0/2463613624 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3dd41eb0e0 con 0x7f3dd40c5330 2024-09-13T16:48:45.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.011+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dd41eb0e0 con 0x7f3dd40c5330 2024-09-13T16:48:45.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.015+0000 7f3dd0ff9640 1 -- 172.21.15.39:0/2463613624 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20376 (secure 0 0 0) 0x7f3dd41eb0e0 con 0x7f3dd40c5330 2024-09-13T16:48:45.021 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:45.021 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:40.750805+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:45.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db40666c0 msgr2=0x7f3db4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db40666c0 0x7f3db4068b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc000f70 tx=0x7f3dcc009040 comp rx=0 tx=0).stop 2024-09-13T16:48:45.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd40c5330 msgr2=0x7f3dd40bcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd40c5330 0x7f3dd40bcf50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc01ab60 tx=0x7f3dbc01acf0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 shutdown_connections 2024-09-13T16:48:45.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db40666c0 0x7f3db4068b80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd4153930 0x7f3dd414ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd40c5330 0x7f3dd40bcf50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 --2- 172.21.15.39:0/2463613624 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd40c1310 0x7f3dd40aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 >> 172.21.15.39:0/2463613624 conn(0x7f3dd4069e10 msgr2=0x7f3dd406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:45.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 shutdown_connections 2024-09-13T16:48:45.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.019+0000 7f3dda6fb640 1 -- 172.21.15.39:0/2463613624 wait complete. 2024-09-13T16:48:45.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T16:48:45.041 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T16:48:45.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53f4b7640 1 Processor -- start 2024-09-13T16:48:45.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53f4b7640 1 -- start start 2024-09-13T16:48:45.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53f4b7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe53814f3d0 0x7fe538155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53f4b7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe5380c7400 0x7fe5380714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53f4b7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53814f7b0 con 0x7fe538071a20 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53f4b7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe538152a90 con 0x7fe5380c7400 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53f4b7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe538155700 con 0x7fe53814f3d0 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53ca52640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe5380c7400 0x7fe5380714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53ca52640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe5380c7400 0x7fe5380714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57350/0 (socket says 172.21.15.39:57350) 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53ca52640 1 -- 172.21.15.39:0/3874688183 learned_addr learned my addr 172.21.15.39:0/3874688183 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53d253640 1 --2- 172.21.15.39:0/3874688183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe53814f3d0 0x7fe538155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53ca52640 1 -- 172.21.15.39:0/3874688183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe53814f3d0 msgr2=0x7fe538155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53ca52640 1 --2- 172.21.15.39:0/3874688183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe53814f3d0 0x7fe538155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53ca52640 1 -- 172.21.15.39:0/3874688183 >> v1:172.21.15.84:6789/0 conn(0x7fe538071a20 legacy=0x7fe5380c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.155+0000 7fe53ca52640 1 -- 172.21.15.39:0/3874688183 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5381559e0 con 0x7fe5380c7400 2024-09-13T16:48:45.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53ca52640 1 --2- 172.21.15.39:0/3874688183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5380c7400 0x7fe5380714e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe528009870 tx=0x7fe52801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145b39b110e496b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:45.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe52ffff640 1 -- 172.21.15.39:0/3874688183 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280ac620 con 0x7fe5380c7400 2024-09-13T16:48:45.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe52ffff640 1 -- 172.21.15.39:0/3874688183 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5280ac780 con 0x7fe5380c7400 2024-09-13T16:48:45.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe52ffff640 1 -- 172.21.15.39:0/3874688183 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280acb10 con 0x7fe5380c7400 2024-09-13T16:48:45.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- 172.21.15.39:0/3874688183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5380c7400 msgr2=0x7fe5380714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 --2- 172.21.15.39:0/3874688183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5380c7400 0x7fe5380714e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe528009870 tx=0x7fe52801c4d0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- 172.21.15.39:0/3874688183 shutdown_connections 2024-09-13T16:48:45.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 --2- 172.21.15.39:0/3874688183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5380c7400 0x7fe5380714e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 --2- 172.21.15.39:0/3874688183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe53814f3d0 0x7fe538155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- 172.21.15.39:0/3874688183 >> 172.21.15.39:0/3874688183 conn(0x7fe5380a9720 msgr2=0x7fe5380a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:45.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- 172.21.15.39:0/3874688183 shutdown_connections 2024-09-13T16:48:45.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- 172.21.15.39:0/3874688183 wait complete. 2024-09-13T16:48:45.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 Processor -- start 2024-09-13T16:48:45.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- start start 2024-09-13T16:48:45.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe538071a20 0x7fe5381f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5380c7400 0x7fe5380c1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe53814f3d0 0x7fe5381eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe538141570 con 0x7fe538071a20 2024-09-13T16:48:45.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe538151450 con 0x7fe53814f3d0 2024-09-13T16:48:45.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe538141830 con 0x7fe5380c7400 2024-09-13T16:48:45.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53d253640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe538071a20 0x7fe5381f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53d253640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe538071a20 0x7fe5381f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57364/0 (socket says 172.21.15.39:57364) 2024-09-13T16:48:45.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53d253640 1 -- 172.21.15.39:0/1356706334 learned_addr learned my addr 172.21.15.39:0/1356706334 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53da54640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe53814f3d0 0x7fe5381eccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53ca52640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5380c7400 0x7fe5380c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53d253640 1 -- 172.21.15.39:0/1356706334 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5380c7400 msgr2=0x7fe5380c1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53d253640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5380c7400 0x7fe5380c1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53d253640 1 -- 172.21.15.39:0/1356706334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe53814f3d0 msgr2=0x7fe5381eccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53d253640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe53814f3d0 0x7fe5381eccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53d253640 1 -- 172.21.15.39:0/1356706334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280094d0 con 0x7fe538071a20 2024-09-13T16:48:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53d253640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe538071a20 0x7fe5381f0c00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe53400d800 tx=0x7fe53400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe52dffb640 1 -- 172.21.15.39:0/1356706334 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400b6c0 con 0x7fe538071a20 2024-09-13T16:48:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe52dffb640 1 -- 172.21.15.39:0/1356706334 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53400b820 con 0x7fe538071a20 2024-09-13T16:48:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe52dffb640 1 -- 172.21.15.39:0/1356706334 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400bb90 con 0x7fe538071a20 2024-09-13T16:48:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5381f1140 con 0x7fe538071a20 2024-09-13T16:48:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.159+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5381f0400 con 0x7fe538071a20 2024-09-13T16:48:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.163+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5381439e0 con 0x7fe538071a20 2024-09-13T16:48:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.163+0000 7fe52dffb640 1 -- 172.21.15.39:0/1356706334 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe53400bcf0 con 0x7fe538071a20 2024-09-13T16:48:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.163+0000 7fe52dffb640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe51c066690 0x7fe51c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.163+0000 7fe52dffb640 1 -- 172.21.15.39:0/1356706334 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fe534085220 con 0x7fe538071a20 2024-09-13T16:48:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.163+0000 7fe53ca52640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe51c066690 0x7fe51c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.163+0000 7fe53ca52640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe51c066690 0x7fe51c068b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe5280ae000 tx=0x7fe528008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:45.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.167+0000 7fe52dffb640 1 -- 172.21.15.39:0/1356706334 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe534052360 con 0x7fe538071a20 2024-09-13T16:48:45.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5381f0260 con 0x7fe538071a20 2024-09-13T16:48:45.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe52dffb640 1 -- 172.21.15.39:0/1356706334 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5381f0260 con 0x7fe538071a20 2024-09-13T16:48:45.325 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe51c066690 msgr2=0x7fe51c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe51c066690 0x7fe51c068b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe5280ae000 tx=0x7fe528008040 comp rx=0 tx=0).stop 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe538071a20 msgr2=0x7fe5381f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe538071a20 0x7fe5381f0c00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe53400d800 tx=0x7fe53400dcc0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 shutdown_connections 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe51c066690 0x7fe51c068b50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe53814f3d0 0x7fe5381eccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe5380c7400 0x7fe5380c1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 --2- 172.21.15.39:0/1356706334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe538071a20 0x7fe5381f0c00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 >> 172.21.15.39:0/1356706334 conn(0x7fe5380a9720 msgr2=0x7fe53814cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.319+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 shutdown_connections 2024-09-13T16:48:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.323+0000 7fe53f4b7640 1 -- 172.21.15.39:0/1356706334 wait complete. 2024-09-13T16:48:45.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T16:48:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4edb80e640 1 Processor -- start 2024-09-13T16:48:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4edb80e640 1 -- start start 2024-09-13T16:48:45.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4edb80e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 0x7f4ed40c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4edb80e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4ed40bcf90 0x7f4ed4154240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4edb80e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed4155270 con 0x7f4ed4157370 2024-09-13T16:48:45.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4edb80e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed4153d80 con 0x7f4ed40bcf90 2024-09-13T16:48:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4edb80e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed4157870 con 0x7f4ed41514d0 2024-09-13T16:48:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4ed8da9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4ed40bcf90 0x7f4ed4154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4ed8da9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4ed40bcf90 0x7f4ed4154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57376/0 (socket says 172.21.15.39:57376) 2024-09-13T16:48:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4ed8da9640 1 -- 172.21.15.39:0/4286582847 learned_addr learned my addr 172.21.15.39:0/4286582847 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4ed95aa640 1 --2- 172.21.15.39:0/4286582847 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 0x7f4ed40c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4ed8da9640 1 -- 172.21.15.39:0/4286582847 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 msgr2=0x7f4ed40c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4ed8da9640 1 --2- 172.21.15.39:0/4286582847 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 0x7f4ed40c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4ed8da9640 1 -- 172.21.15.39:0/4286582847 >> v1:172.21.15.84:6789/0 conn(0x7f4ed4157370 legacy=0x7f4ed4154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.463+0000 7f4ed8da9640 1 -- 172.21.15.39:0/4286582847 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4157b50 con 0x7f4ed40bcf90 2024-09-13T16:48:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4ed8da9640 1 --2- 172.21.15.39:0/4286582847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 0x7f4ed4154240 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009870 tx=0x7f4ec401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f1acee815dc985 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4ecbfff640 1 -- 172.21.15.39:0/4286582847 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40ab600 con 0x7f4ed40bcf90 2024-09-13T16:48:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4ecbfff640 1 -- 172.21.15.39:0/4286582847 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec40a7050 con 0x7f4ed40bcf90 2024-09-13T16:48:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4ecbfff640 1 -- 172.21.15.39:0/4286582847 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40ab980 con 0x7f4ed40bcf90 2024-09-13T16:48:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4edb80e640 1 -- 172.21.15.39:0/4286582847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 msgr2=0x7f4ed4154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4edb80e640 1 --2- 172.21.15.39:0/4286582847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 0x7f4ed4154240 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009870 tx=0x7f4ec401c490 comp rx=0 tx=0).stop 2024-09-13T16:48:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4edb80e640 1 -- 172.21.15.39:0/4286582847 shutdown_connections 2024-09-13T16:48:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4edb80e640 1 --2- 172.21.15.39:0/4286582847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 0x7f4ed4154240 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4edb80e640 1 --2- 172.21.15.39:0/4286582847 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 0x7f4ed40c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4edb80e640 1 -- 172.21.15.39:0/4286582847 >> 172.21.15.39:0/4286582847 conn(0x7f4ed406a210 msgr2=0x7f4ed406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4edb80e640 1 -- 172.21.15.39:0/4286582847 shutdown_connections 2024-09-13T16:48:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.467+0000 7f4edb80e640 1 -- 172.21.15.39:0/4286582847 wait complete. 2024-09-13T16:48:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 Processor -- start 2024-09-13T16:48:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 -- start start 2024-09-13T16:48:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 0x7f4ed40a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 0x7f4ed4148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4157370 0x7f4ed4149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed414aa60 con 0x7f4ed40bcf90 2024-09-13T16:48:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed414a4a0 con 0x7f4ed4157370 2024-09-13T16:48:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed414a780 con 0x7f4ed41514d0 2024-09-13T16:48:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed95aa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 0x7f4ed40a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed95aa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 0x7f4ed40a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57390/0 (socket says 172.21.15.39:57390) 2024-09-13T16:48:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed95aa640 1 -- 172.21.15.39:0/2684533277 learned_addr learned my addr 172.21.15.39:0/2684533277 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed9dab640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4157370 0x7f4ed4149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed8da9640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 0x7f4ed4148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed95aa640 1 -- 172.21.15.39:0/2684533277 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 msgr2=0x7f4ed4148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed95aa640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 0x7f4ed4148ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed95aa640 1 -- 172.21.15.39:0/2684533277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4157370 msgr2=0x7f4ed4149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed95aa640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4157370 0x7f4ed4149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed95aa640 1 -- 172.21.15.39:0/2684533277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec40094d0 con 0x7f4ed40bcf90 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed9dab640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4157370 0x7f4ed4149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ed95aa640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 0x7f4ed40a9490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000d5f0 tx=0x7f4ed000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ec9ffb640 1 -- 172.21.15.39:0/2684533277 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00125f0 con 0x7f4ed40bcf90 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ec9ffb640 1 -- 172.21.15.39:0/2684533277 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed0012750 con 0x7f4ed40bcf90 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed414eea0 con 0x7f4ed40bcf90 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ec9ffb640 1 -- 172.21.15.39:0/2684533277 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0012ae0 con 0x7f4ed40bcf90 2024-09-13T16:48:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed40a99d0 con 0x7f4ed40bcf90 2024-09-13T16:48:45.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed414ed00 con 0x7f4ed40bcf90 2024-09-13T16:48:45.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.471+0000 7f4ec9ffb640 1 -- 172.21.15.39:0/2684533277 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ed0012c40 con 0x7f4ed40bcf90 2024-09-13T16:48:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.475+0000 7f4ec9ffb640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4eb8066690 0x7f4eb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.475+0000 7f4ec9ffb640 1 -- 172.21.15.39:0/2684533277 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f4ed0084a10 con 0x7f4ed40bcf90 2024-09-13T16:48:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.475+0000 7f4ed8da9640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4eb8066690 0x7f4eb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.475+0000 7f4ed8da9640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4eb8066690 0x7f4eb8068b50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40c6180 tx=0x7f4ec4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.479+0000 7f4ec9ffb640 1 -- 172.21.15.39:0/2684533277 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4ed414ed00 con 0x7f4ed40bcf90 2024-09-13T16:48:45.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ed414ed00 con 0x7f4ed40bcf90 2024-09-13T16:48:45.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4ec9ffb640 1 -- 172.21.15.39:0/2684533277 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v26) v1 ==== 99+0+11 (secure 0 0 0) 0x7f4ed414ed00 con 0x7f4ed40bcf90 2024-09-13T16:48:45.644 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4eb8066690 msgr2=0x7f4eb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4eb8066690 0x7f4eb8068b50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40c6180 tx=0x7f4ec4008040 comp rx=0 tx=0).stop 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 msgr2=0x7f4ed40a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 0x7f4ed40a9490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000d5f0 tx=0x7f4ed000dab0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 shutdown_connections 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4eb8066690 0x7f4eb8068b50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ed4157370 0x7f4ed4149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ed41514d0 0x7f4ed4148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 --2- 172.21.15.39:0/2684533277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ed40bcf90 0x7f4ed40a9490 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 >> 172.21.15.39:0/2684533277 conn(0x7f4ed406a210 msgr2=0x7f4ed4154240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 shutdown_connections 2024-09-13T16:48:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.639+0000 7f4edb80e640 1 -- 172.21.15.39:0/2684533277 wait complete. 2024-09-13T16:48:45.663 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T16:48:45.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f790def2640 1 Processor -- start 2024-09-13T16:48:45.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f790def2640 1 -- start start 2024-09-13T16:48:45.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f790def2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f79080aa2d0 0x7f79080aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f790def2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7908151cf0 0x7f7908152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f790def2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7908155d20 con 0x7f79080a4d60 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f790def2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7908155b30 con 0x7f79080aa2d0 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f790def2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79080c4750 con 0x7f7908151cf0 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f79077fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f79080aa2d0 0x7f79080aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f79077fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f79080aa2d0 0x7f79080aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57400/0 (socket says 172.21.15.39:57400) 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f79077fe640 1 -- 172.21.15.39:0/96884229 learned_addr learned my addr 172.21.15.39:0/96884229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f7907fff640 1 --2- 172.21.15.39:0/96884229 >> v2:172.21.15.84:3300/0 conn(0x7f7908151cf0 0x7f7908152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f79077fe640 1 -- 172.21.15.39:0/96884229 >> v2:172.21.15.84:3300/0 conn(0x7f7908151cf0 msgr2=0x7f7908152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f79077fe640 1 --2- 172.21.15.39:0/96884229 >> v2:172.21.15.84:3300/0 conn(0x7f7908151cf0 0x7f7908152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f79077fe640 1 -- 172.21.15.39:0/96884229 >> v1:172.21.15.84:6789/0 conn(0x7f79080a4d60 legacy=0x7f790814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f79077fe640 1 -- 172.21.15.39:0/96884229 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7908155fc0 con 0x7f79080aa2d0 2024-09-13T16:48:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.783+0000 7f7907fff640 1 --2- 172.21.15.39:0/96884229 >> v2:172.21.15.84:3300/0 conn(0x7f7908151cf0 0x7f7908152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:45.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f79077fe640 1 --2- 172.21.15.39:0/96884229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79080aa2d0 0x7f79080aa6b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f79080c6640 tx=0x7f790001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870a692bdcd5ff22 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:45.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f79067fc640 1 -- 172.21.15.39:0/96884229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79000a56c0 con 0x7f79080aa2d0 2024-09-13T16:48:45.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f79067fc640 1 -- 172.21.15.39:0/96884229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79000a5820 con 0x7f79080aa2d0 2024-09-13T16:48:45.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f79067fc640 1 -- 172.21.15.39:0/96884229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79000a5bb0 con 0x7f79080aa2d0 2024-09-13T16:48:45.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 -- 172.21.15.39:0/96884229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79080aa2d0 msgr2=0x7f79080aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 --2- 172.21.15.39:0/96884229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79080aa2d0 0x7f79080aa6b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f79080c6640 tx=0x7f790001c8f0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 -- 172.21.15.39:0/96884229 shutdown_connections 2024-09-13T16:48:45.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 --2- 172.21.15.39:0/96884229 >> v2:172.21.15.84:3300/0 conn(0x7f7908151cf0 0x7f7908152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 --2- 172.21.15.39:0/96884229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79080aa2d0 0x7f79080aa6b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 -- 172.21.15.39:0/96884229 >> 172.21.15.39:0/96884229 conn(0x7f79080c5790 msgr2=0x7f79080c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:45.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 -- 172.21.15.39:0/96884229 shutdown_connections 2024-09-13T16:48:45.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 -- 172.21.15.39:0/96884229 wait complete. 2024-09-13T16:48:45.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 Processor -- start 2024-09-13T16:48:45.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 -- start start 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79080a4d60 0x7f79080710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79080aa2d0 0x7f790806f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7908151cf0 0x7f79081edf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79081ee960 con 0x7f7908151cf0 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7908071f20 con 0x7f79080aa2d0 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f790def2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7908144bd0 con 0x7f79080a4d60 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f7907fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7908151cf0 0x7f79081edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f7907fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7908151cf0 0x7f79081edf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57406/0 (socket says 172.21.15.39:57406) 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.787+0000 7f7907fff640 1 -- 172.21.15.39:0/577059962 learned_addr learned my addr 172.21.15.39:0/577059962 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:45.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f79077fe640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79080a4d60 0x7f79080710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f7906ffd640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79080aa2d0 0x7f790806f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f7907fff640 1 -- 172.21.15.39:0/577059962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79080a4d60 msgr2=0x7f79080710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f7907fff640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79080a4d60 0x7f79080710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f7907fff640 1 -- 172.21.15.39:0/577059962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79080aa2d0 msgr2=0x7f790806f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f7907fff640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79080aa2d0 0x7f790806f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f7907fff640 1 -- 172.21.15.39:0/577059962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7900009b30 con 0x7f7908151cf0 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f7906ffd640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79080aa2d0 0x7f790806f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f7907fff640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7908151cf0 0x7f79081edf10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f78f800c820 tx=0x7f78f800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f78fffff640 1 -- 172.21.15.39:0/577059962 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8007a30 con 0x7f7908151cf0 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f78fffff640 1 -- 172.21.15.39:0/577059962 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f8007bd0 con 0x7f7908151cf0 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f78fffff640 1 -- 172.21.15.39:0/577059962 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f800ce10 con 0x7f7908151cf0 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79080721a0 con 0x7f7908151cf0 2024-09-13T16:48:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790806fd30 con 0x7f7908151cf0 2024-09-13T16:48:45.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79081eeb10 con 0x7f7908151cf0 2024-09-13T16:48:45.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f78fffff640 1 -- 172.21.15.39:0/577059962 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f78f80113f0 con 0x7f7908151cf0 2024-09-13T16:48:45.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f78fffff640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78ec066690 0x7f78ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:45.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f78fffff640 1 -- 172.21.15.39:0/577059962 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f78f80846f0 con 0x7f7908151cf0 2024-09-13T16:48:45.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f79077fe640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78ec066690 0x7f78ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:45.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.791+0000 7f79077fe640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78ec066690 0x7f78ec068b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79000ae000 tx=0x7f79000a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:45.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.799+0000 7f78fffff640 1 -- 172.21.15.39:0/577059962 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f79081eeb10 con 0x7f7908151cf0 2024-09-13T16:48:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f790814e9b0 con 0x7f7908151cf0 2024-09-13T16:48:45.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f78fffff640 1 -- 172.21.15.39:0/577059962 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v26) v1 ==== 95+0+11 (secure 0 0 0) 0x7f78f80516b0 con 0x7f7908151cf0 2024-09-13T16:48:45.964 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:45.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78ec066690 msgr2=0x7f78ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78ec066690 0x7f78ec068b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f79000ae000 tx=0x7f79000a7070 comp rx=0 tx=0).stop 2024-09-13T16:48:45.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7908151cf0 msgr2=0x7f79081edf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:45.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7908151cf0 0x7f79081edf10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f78f800c820 tx=0x7f78f800cce0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 shutdown_connections 2024-09-13T16:48:45.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78ec066690 0x7f78ec068b50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79080aa2d0 0x7f790806f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7908151cf0 0x7f79081edf10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 --2- 172.21.15.39:0/577059962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f79080a4d60 0x7f79080710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:45.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 >> 172.21.15.39:0/577059962 conn(0x7f79080c5790 msgr2=0x7f790814d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:45.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 shutdown_connections 2024-09-13T16:48:45.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:45.959+0000 7f790def2640 1 -- 172.21.15.39:0/577059962 wait complete. 2024-09-13T16:48:45.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T16:48:46.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.103+0000 7fab10f23640 1 Processor -- start 2024-09-13T16:48:46.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.103+0000 7fab10f23640 1 -- start start 2024-09-13T16:48:46.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab10f23640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c0a4d60 0x7fab0c14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab10f23640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0c155d20 con 0x7fab0c14f4b0 2024-09-13T16:48:46.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab10f23640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0c0c4750 con 0x7fab0c0aa2d0 2024-09-13T16:48:46.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab10f23640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c155b30 con 0x7fab0c0a4d60 2024-09-13T16:48:46.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0ad9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fab0c14f4b0 0x7fab0c151cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34962/0 (socket says 172.21.15.39:34962) 2024-09-13T16:48:46.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0ad9d640 1 -- 172.21.15.39:0/1001977969 learned_addr learned my addr 172.21.15.39:0/1001977969 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab09d9b640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c0a4d60 0x7fab0c14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115091557 0 0) 0x7fab0c155d20 con 0x7fab0c14f4b0 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaf4003530 con 0x7fab0c14f4b0 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2390722696 0 0) 0x7faaf4003530 con 0x7fab0c14f4b0 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c0a4d60 msgr2=0x7fab0c14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c0a4d60 0x7fab0c14ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 >> v1:172.21.15.84:6789/0 conn(0x7fab0c0aa2d0 legacy=0x7fab0c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c155fc0 con 0x7fab0c14f4b0 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fab04001680 con 0x7fab0c14f4b0 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 >> v1:172.21.15.39:6789/0 conn(0x7fab0c14f4b0 legacy=0x7fab0c151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaf40039b0 0x7faaf4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaf40047d0 0x7faaf4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c0a4d60 0x7faaf40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf400ba90 con 0x7faaf40039b0 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf400bc00 con 0x7faaf40047d0 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf400bdc0 con 0x7fab0c0a4d60 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0a59c640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaf40039b0 0x7faaf4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab09d9b640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaf40047d0 0x7faaf4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0ad9d640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c0a4d60 0x7faaf40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0a59c640 1 -- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c0a4d60 msgr2=0x7faaf40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0a59c640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c0a4d60 0x7faaf40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0a59c640 1 -- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaf40047d0 msgr2=0x7faaf4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0a59c640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faaf40047d0 0x7faaf4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.107+0000 7fab0a59c640 1 -- 172.21.15.39:0/1001977969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf4003530 con 0x7faaf40039b0 2024-09-13T16:48:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab0a59c640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaf40039b0 0x7faaf4003dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faafc001e70 tx=0x7faafc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff4e4d28de38705 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc0a6040 con 0x7faaf40039b0 2024-09-13T16:48:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc003b80 con 0x7faaf40039b0 2024-09-13T16:48:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab0959a640 1 -- 172.21.15.39:0/1001977969 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc0050d0 con 0x7faaf40039b0 2024-09-13T16:48:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 -- 172.21.15.39:0/1001977969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaf40039b0 msgr2=0x7faaf4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 --2- 172.21.15.39:0/1001977969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaf40039b0 0x7faaf4003dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faafc001e70 tx=0x7faafc01c780 comp rx=0 tx=0).stop 2024-09-13T16:48:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab0ad9d640 1 -- 172.21.15.39:0/1001977969 reap_dead start 2024-09-13T16:48:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 -- 172.21.15.39:0/1001977969 shutdown_connections 2024-09-13T16:48:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 -- 172.21.15.39:0/1001977969 >> 172.21.15.39:0/1001977969 conn(0x7fab0c0c5790 msgr2=0x7fab0c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 -- 172.21.15.39:0/1001977969 shutdown_connections 2024-09-13T16:48:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 -- 172.21.15.39:0/1001977969 wait complete. 2024-09-13T16:48:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 Processor -- start 2024-09-13T16:48:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 -- start start 2024-09-13T16:48:46.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab0c14a960 0x7fab0c1ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0a5b80 0x7fab0c06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c06c290 0x7fab0c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c1ed820 con 0x7fab0c0a5b80 2024-09-13T16:48:46.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c06c740 con 0x7fab0c14a960 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab10f23640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c06ca50 con 0x7fab0c06c290 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab09d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0a5b80 0x7fab0c06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab09d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0a5b80 0x7fab0c06bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57436/0 (socket says 172.21.15.39:57436) 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab09d9b640 1 -- 172.21.15.39:0/1165045944 learned_addr learned my addr 172.21.15.39:0/1165045944 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab0a59c640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab0c14a960 0x7fab0c1ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab0ad9d640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c06c290 0x7fab0c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab09d9b640 1 -- 172.21.15.39:0/1165045944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c06c290 msgr2=0x7fab0c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab09d9b640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c06c290 0x7fab0c15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab09d9b640 1 -- 172.21.15.39:0/1165045944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab0c14a960 msgr2=0x7fab0c1ed2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab09d9b640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab0c14a960 0x7fab0c1ed2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab09d9b640 1 -- 172.21.15.39:0/1165045944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c06cd30 con 0x7fab0c0a5b80 2024-09-13T16:48:46.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7fab09d9b640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0a5b80 0x7fab0c06bd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab000115e0 tx=0x7fab00011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.111+0000 7faafaffd640 1 -- 172.21.15.39:0/1165045944 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0000f8f0 con 0x7fab0c0a5b80 2024-09-13T16:48:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7faafaffd640 1 -- 172.21.15.39:0/1165045944 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0000fa50 con 0x7fab0c0a5b80 2024-09-13T16:48:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7faafaffd640 1 -- 172.21.15.39:0/1165045944 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0000fdc0 con 0x7fab0c0a5b80 2024-09-13T16:48:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0c06d010 con 0x7fab0c0a5b80 2024-09-13T16:48:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c0bec50 con 0x7fab0c0a5b80 2024-09-13T16:48:46.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c06d200 con 0x7fab0c0a5b80 2024-09-13T16:48:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7faafaffd640 1 -- 172.21.15.39:0/1165045944 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab00015460 con 0x7fab0c0a5b80 2024-09-13T16:48:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7faafaffd640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faaec0666c0 0x7faaec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7fab0a59c640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faaec0666c0 0x7faaec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7faafaffd640 1 -- 172.21.15.39:0/1165045944 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fab00084910 con 0x7fab0c0a5b80 2024-09-13T16:48:46.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.115+0000 7fab0a59c640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faaec0666c0 0x7faaec068b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faafc0ac000 tx=0x7faafc001520 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:46.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.119+0000 7faafaffd640 1 -- 172.21.15.39:0/1165045944 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fab00051850 con 0x7fab0c0a5b80 2024-09-13T16:48:46.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab0c0aa6b0 con 0x7fab0c0a5b80 2024-09-13T16:48:46.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7faafaffd640 1 -- 172.21.15.39:0/1165045944 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v26) v1 ==== 99+0+11 (secure 0 0 0) 0x7fab000220a0 con 0x7fab0c0a5b80 2024-09-13T16:48:46.284 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T16:48:46.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faaec0666c0 msgr2=0x7faaec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faaec0666c0 0x7faaec068b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faafc0ac000 tx=0x7faafc001520 comp rx=0 tx=0).stop 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0a5b80 msgr2=0x7fab0c06bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0a5b80 0x7fab0c06bd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab000115e0 tx=0x7fab00011aa0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 shutdown_connections 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faaec0666c0 0x7faaec068b80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab0c06c290 0x7fab0c15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0a5b80 0x7fab0c06bd50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 --2- 172.21.15.39:0/1165045944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab0c14a960 0x7fab0c1ed2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 >> 172.21.15.39:0/1165045944 conn(0x7fab0c0c5790 msgr2=0x7fab0c1562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 shutdown_connections 2024-09-13T16:48:46.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.279+0000 7fab10f23640 1 -- 172.21.15.39:0/1165045944 wait complete. 2024-09-13T16:48:46.303 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T16:48:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.415+0000 7fb98a7d8640 1 Processor -- start 2024-09-13T16:48:46.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.415+0000 7fb98a7d8640 1 -- start start 2024-09-13T16:48:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.415+0000 7fb98a7d8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9841515b0 0x7fb9840c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.415+0000 7fb98a7d8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb984151990 0x7fb9841540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.415+0000 7fb98a7d8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb984158070 con 0x7fb9840c0b80 2024-09-13T16:48:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb98a7d8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9840c17b0 con 0x7fb984151990 2024-09-13T16:48:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb98a7d8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb984158350 con 0x7fb9841515b0 2024-09-13T16:48:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb9837fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb9840c0b80 0x7fb9840c21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34964/0 (socket says 172.21.15.39:34964) 2024-09-13T16:48:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb9837fe640 1 -- 172.21.15.39:0/1252033570 learned_addr learned my addr 172.21.15.39:0/1252033570 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:46.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb988d75640 1 --2- 172.21.15.39:0/1252033570 >> v2:172.21.15.39:3300/0 conn(0x7fb984151990 0x7fb9841540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb983fff640 1 --2- 172.21.15.39:0/1252033570 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9841515b0 0x7fb9840c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb982ffd640 1 -- 172.21.15.39:0/1252033570 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698991133 0 0) 0x7fb984158070 con 0x7fb9840c0b80 2024-09-13T16:48:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb982ffd640 1 -- 172.21.15.39:0/1252033570 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb96c003530 con 0x7fb9840c0b80 2024-09-13T16:48:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb988d75640 1 -- 172.21.15.39:0/1252033570 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9841515b0 msgr2=0x7fb9840c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb988d75640 1 --2- 172.21.15.39:0/1252033570 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9841515b0 0x7fb9840c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb982ffd640 1 -- 172.21.15.39:0/1252033570 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 431059773 0 0) 0x7fb96c003530 con 0x7fb9840c0b80 2024-09-13T16:48:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb988d75640 1 -- 172.21.15.39:0/1252033570 >> v1:172.21.15.39:6789/0 conn(0x7fb9840c0b80 legacy=0x7fb9840c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.419+0000 7fb988d75640 1 -- 172.21.15.39:0/1252033570 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9840c3690 con 0x7fb984151990 2024-09-13T16:48:46.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 --2- 172.21.15.39:0/1252033570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 0x7fb9841540e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009d90 tx=0x7fb97c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fa12ff8805a74e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:46.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb982ffd640 1 -- 172.21.15.39:0/1252033570 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984154e30 con 0x7fb984151990 2024-09-13T16:48:46.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb982ffd640 1 -- 172.21.15.39:0/1252033570 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c0a6510 con 0x7fb984151990 2024-09-13T16:48:46.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb982ffd640 1 -- 172.21.15.39:0/1252033570 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978001680 con 0x7fb984151990 2024-09-13T16:48:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1252033570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 msgr2=0x7fb9841540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 --2- 172.21.15.39:0/1252033570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 0x7fb9841540e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009d90 tx=0x7fb97c01c740 comp rx=0 tx=0).stop 2024-09-13T16:48:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1252033570 shutdown_connections 2024-09-13T16:48:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 --2- 172.21.15.39:0/1252033570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 0x7fb9841540e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 --2- 172.21.15.39:0/1252033570 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9841515b0 0x7fb9840c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1252033570 >> 172.21.15.39:0/1252033570 conn(0x7fb9840c5790 msgr2=0x7fb9840c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:46.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1252033570 shutdown_connections 2024-09-13T16:48:46.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1252033570 wait complete. 2024-09-13T16:48:46.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 Processor -- start 2024-09-13T16:48:46.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 -- start start 2024-09-13T16:48:46.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9840c0b80 0x7fb9841500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9841515b0 0x7fb98406f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 0x7fb98406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb984070660 con 0x7fb984151990 2024-09-13T16:48:46.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb984150fb0 con 0x7fb9841515b0 2024-09-13T16:48:46.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb98a7d8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb984150a10 con 0x7fb9840c0b80 2024-09-13T16:48:46.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 0x7fb98406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 0x7fb98406fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57460/0 (socket says 172.21.15.39:57460) 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 -- 172.21.15.39:0/1669570764 learned_addr learned my addr 172.21.15.39:0/1669570764 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb983fff640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9840c0b80 0x7fb9841500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb9837fe640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9841515b0 0x7fb98406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 -- 172.21.15.39:0/1669570764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9840c0b80 msgr2=0x7fb9841500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9840c0b80 0x7fb9841500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 -- 172.21.15.39:0/1669570764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9841515b0 msgr2=0x7fb98406f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9841515b0 0x7fb98406f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 -- 172.21.15.39:0/1669570764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c009a70 con 0x7fb984151990 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb9837fe640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9841515b0 0x7fb98406f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.423+0000 7fb988d75640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 0x7fb98406fc40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb97c01cfe0 tx=0x7fb97c0194e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb980ff9640 1 -- 172.21.15.39:0/1669570764 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c007000 con 0x7fb984151990 2024-09-13T16:48:46.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb980ff9640 1 -- 172.21.15.39:0/1669570764 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c0ba880 con 0x7fb984151990 2024-09-13T16:48:46.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9841ec5f0 con 0x7fb984151990 2024-09-13T16:48:46.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb980ff9640 1 -- 172.21.15.39:0/1669570764 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0babf0 con 0x7fb984151990 2024-09-13T16:48:46.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb984150cd0 con 0x7fb984151990 2024-09-13T16:48:46.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9840c51b0 con 0x7fb984151990 2024-09-13T16:48:46.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb980ff9640 1 -- 172.21.15.39:0/1669570764 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb97c0bad50 con 0x7fb984151990 2024-09-13T16:48:46.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb980ff9640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9700666c0 0x7fb970068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb980ff9640 1 -- 172.21.15.39:0/1669570764 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fb97c0aa070 con 0x7fb984151990 2024-09-13T16:48:46.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb983fff640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9700666c0 0x7fb970068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.427+0000 7fb983fff640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9700666c0 0x7fb970068b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb984157fd0 tx=0x7fb974008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:46.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.431+0000 7fb980ff9640 1 -- 172.21.15.39:0/1669570764 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb97c0f0460 con 0x7fb984151990 2024-09-13T16:48:46.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.583+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fb9840614e0 con 0x7fb984151990 2024-09-13T16:48:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.583+0000 7fb980ff9640 1 -- 172.21.15.39:0/1669570764 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v26) v1 ==== 95+0+11 (secure 0 0 0) 0x7fb97c009450 con 0x7fb984151990 2024-09-13T16:48:46.590 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T16:48:46.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9700666c0 msgr2=0x7fb970068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9700666c0 0x7fb970068b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb984157fd0 tx=0x7fb974008040 comp rx=0 tx=0).stop 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 msgr2=0x7fb98406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 0x7fb98406fc40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb97c01cfe0 tx=0x7fb97c0194e0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 shutdown_connections 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9700666c0 0x7fb970068b80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9841515b0 0x7fb98406f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb984151990 0x7fb98406fc40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 --2- 172.21.15.39:0/1669570764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb9840c0b80 0x7fb9841500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 >> 172.21.15.39:0/1669570764 conn(0x7fb9840c5790 msgr2=0x7fb98415e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 shutdown_connections 2024-09-13T16:48:46.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.587+0000 7fb98a7d8640 1 -- 172.21.15.39:0/1669570764 wait complete. 2024-09-13T16:48:46.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:48:46.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.711+0000 7f4f4a26a640 1 Processor -- start 2024-09-13T16:48:46.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.711+0000 7f4f4a26a640 1 -- start start 2024-09-13T16:48:46.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f4a26a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f4414f540 0x7f4f440c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f4a26a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4f440c1ab0 0x7f4f4406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f4a26a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4414f920 con 0x7f4f4406b360 2024-09-13T16:48:46.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f4a26a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f440709c0 con 0x7f4f440c1ab0 2024-09-13T16:48:46.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f4a26a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f440c1780 con 0x7f4f4414f540 2024-09-13T16:48:46.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f437fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4f440c1ab0 0x7f4f4406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f437fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4f440c1ab0 0x7f4f4406ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57472/0 (socket says 172.21.15.39:57472) 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f48807640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4f4406b360 0x7f4f44155200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34974/0 (socket says 172.21.15.39:34974) 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f437fe640 1 -- 172.21.15.39:0/3179090686 learned_addr learned my addr 172.21.15.39:0/3179090686 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f437fe640 1 -- 172.21.15.39:0/3179090686 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f4414f540 msgr2=0x7f4f440c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f43fff640 1 --2- 172.21.15.39:0/3179090686 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f4414f540 0x7f4f440c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f437fe640 1 --2- 172.21.15.39:0/3179090686 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f4414f540 0x7f4f440c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f437fe640 1 -- 172.21.15.39:0/3179090686 >> v1:172.21.15.39:6789/0 conn(0x7f4f4406b360 legacy=0x7f4f44155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f437fe640 1 -- 172.21.15.39:0/3179090686 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44071580 con 0x7f4f440c1ab0 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f42ffd640 1 -- 172.21.15.39:0/3179090686 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150819761 0 0) 0x7f4f4414f920 con 0x7f4f4406b360 2024-09-13T16:48:46.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f43fff640 1 --2- 172.21.15.39:0/3179090686 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f4414f540 0x7f4f440c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:46.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f437fe640 1 --2- 172.21.15.39:0/3179090686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f440c1ab0 0x7f4f4406ae20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009870 tx=0x7f4f3801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ce1c899a2f58c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:46.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f42ffd640 1 -- 172.21.15.39:0/3179090686 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f380a95f0 con 0x7f4f440c1ab0 2024-09-13T16:48:46.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f42ffd640 1 -- 172.21.15.39:0/3179090686 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f380a9750 con 0x7f4f440c1ab0 2024-09-13T16:48:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.715+0000 7f4f42ffd640 1 -- 172.21.15.39:0/3179090686 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f380a9ae0 con 0x7f4f440c1ab0 2024-09-13T16:48:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- 172.21.15.39:0/3179090686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f440c1ab0 msgr2=0x7f4f4406ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 --2- 172.21.15.39:0/3179090686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f440c1ab0 0x7f4f4406ae20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009870 tx=0x7f4f3801c530 comp rx=0 tx=0).stop 2024-09-13T16:48:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- 172.21.15.39:0/3179090686 shutdown_connections 2024-09-13T16:48:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 --2- 172.21.15.39:0/3179090686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f440c1ab0 0x7f4f4406ae20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 --2- 172.21.15.39:0/3179090686 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f4414f540 0x7f4f440c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- 172.21.15.39:0/3179090686 >> 172.21.15.39:0/3179090686 conn(0x7f4f440c32a0 msgr2=0x7f4f440c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- 172.21.15.39:0/3179090686 shutdown_connections 2024-09-13T16:48:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- 172.21.15.39:0/3179090686 wait complete. 2024-09-13T16:48:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 Processor -- start 2024-09-13T16:48:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- start start 2024-09-13T16:48:46.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f4406b360 0x7f4f441487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f440c1ab0 0x7f4f4414bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4414f540 0x7f4f44150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4414a200 con 0x7f4f4406b360 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4414b060 con 0x7f4f4414f540 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4414c720 con 0x7f4f440c1ab0 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f43fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f4406b360 0x7f4f441487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f43fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f4406b360 0x7f4f441487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57482/0 (socket says 172.21.15.39:57482) 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f43fff640 1 -- 172.21.15.39:0/1130018760 learned_addr learned my addr 172.21.15.39:0/1130018760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f437fe640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f440c1ab0 0x7f4f4414bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f48807640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4414f540 0x7f4f44150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f43fff640 1 -- 172.21.15.39:0/1130018760 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f440c1ab0 msgr2=0x7f4f4414bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f43fff640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f440c1ab0 0x7f4f4414bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f43fff640 1 -- 172.21.15.39:0/1130018760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4414f540 msgr2=0x7f4f44150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f43fff640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4414f540 0x7f4f44150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f43fff640 1 -- 172.21.15.39:0/1130018760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f380094d0 con 0x7f4f4406b360 2024-09-13T16:48:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f48807640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4414f540 0x7f4f44150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:48:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f43fff640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f4406b360 0x7f4f441487c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c7c0 tx=0x7f4f3400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f40ff9640 1 -- 172.21.15.39:0/1130018760 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34007a30 con 0x7f4f4406b360 2024-09-13T16:48:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f40ff9640 1 -- 172.21.15.39:0/1130018760 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f34007b90 con 0x7f4f4406b360 2024-09-13T16:48:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4414a450 con 0x7f4f4406b360 2024-09-13T16:48:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f40ff9640 1 -- 172.21.15.39:0/1130018760 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3400ce70 con 0x7f4f4406b360 2024-09-13T16:48:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.719+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4414aa10 con 0x7f4f4406b360 2024-09-13T16:48:46.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.723+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f440619c0 con 0x7f4f4406b360 2024-09-13T16:48:46.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.723+0000 7f4f40ff9640 1 -- 172.21.15.39:0/1130018760 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f340113f0 con 0x7f4f4406b360 2024-09-13T16:48:46.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.723+0000 7f4f40ff9640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4f300666c0 0x7f4f30068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:46.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.723+0000 7f4f437fe640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4f300666c0 0x7f4f30068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:46.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.723+0000 7f4f40ff9640 1 -- 172.21.15.39:0/1130018760 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f4f340849a0 con 0x7f4f4406b360 2024-09-13T16:48:46.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.723+0000 7f4f437fe640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4f300666c0 0x7f4f30068b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f380ae000 tx=0x7f4f380b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:46.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.727+0000 7f4f40ff9640 1 -- 172.21.15.39:0/1130018760 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4f34051960 con 0x7f4f4406b360 2024-09-13T16:48:46.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f4406ae20 con 0x7f4f4406b360 2024-09-13T16:48:46.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f40ff9640 1 -- 172.21.15.39:0/1130018760 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7f4f3401e090 con 0x7f4f4406b360 2024-09-13T16:48:46.920 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:46.921 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:48:46.921 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:48:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4f300666c0 msgr2=0x7f4f30068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4f300666c0 0x7f4f30068b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4f380ae000 tx=0x7f4f380b3040 comp rx=0 tx=0).stop 2024-09-13T16:48:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f4406b360 msgr2=0x7f4f441487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f4406b360 0x7f4f441487c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c7c0 tx=0x7f4f3400cc80 comp rx=0 tx=0).stop 2024-09-13T16:48:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 shutdown_connections 2024-09-13T16:48:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4f300666c0 0x7f4f30068b80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4414f540 0x7f4f44150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4f440c1ab0 0x7f4f4414bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 --2- 172.21.15.39:0/1130018760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f4406b360 0x7f4f441487c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 >> 172.21.15.39:0/1130018760 conn(0x7f4f440c32a0 msgr2=0x7f4f440bd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 shutdown_connections 2024-09-13T16:48:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:46.915+0000 7f4f4a26a640 1 -- 172.21.15.39:0/1130018760 wait complete. 2024-09-13T16:48:46.938 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T16:48:47.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f068e09b640 1 Processor -- start 2024-09-13T16:48:47.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f068e09b640 1 -- start start 2024-09-13T16:48:47.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f068e09b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0688070b40 0x7f06880c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:47.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f068e09b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0688157360 con 0x7f06880c4260 2024-09-13T16:48:47.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f068e09b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06880c80b0 con 0x7f06880bcfb0 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f068e09b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06881575b0 con 0x7f0688070b40 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06877fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f06880c4260 0x7f0688070430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34990/0 (socket says 172.21.15.39:34990) 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06877fe640 1 -- 172.21.15.39:0/3903465288 learned_addr learned my addr 172.21.15.39:0/3903465288 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636883107 0 0) 0x7f0688157360 con 0x7f06880c4260 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0674003530 con 0x7f06880c4260 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f0686ffd640 1 --2- 172.21.15.39:0/3903465288 >> v2:172.21.15.84:3300/0 conn(0x7f0688070b40 0x7f06880c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606686118 0 0) 0x7f0674003530 con 0x7f06880c4260 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 >> v2:172.21.15.84:3300/0 conn(0x7f0688070b40 msgr2=0x7f06880c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 --2- 172.21.15.39:0/3903465288 >> v2:172.21.15.84:3300/0 conn(0x7f0688070b40 0x7f06880c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 >> v1:172.21.15.84:6789/0 conn(0x7f06880bcfb0 legacy=0x7f068815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0688157890 con 0x7f06880c4260 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f0678001680 con 0x7f06880c4260 2024-09-13T16:48:47.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 >> v1:172.21.15.39:6789/0 conn(0x7f06880c4260 legacy=0x7f0688070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0674003980 0x7f0674003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740047d0 0x7f0674004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0674005260 0x7f067400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06881575b0 con 0x7f0674005260 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067400c1b0 con 0x7f06740047d0 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.043+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067400c370 con 0x7f0674003980 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0687fff640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0674005260 0x7f067400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f06877fe640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0674003980 0x7f0674003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0686ffd640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740047d0 0x7f0674004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0687fff640 1 -- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0674003980 msgr2=0x7f0674003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0687fff640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0674003980 0x7f0674003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0687fff640 1 -- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740047d0 msgr2=0x7f0674004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:48:47.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0687fff640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740047d0 0x7f0674004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0687fff640 1 -- 172.21.15.39:0/3903465288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0674003530 con 0x7f0674005260 2024-09-13T16:48:47.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f06877fe640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0674003980 0x7f0674003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:47.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0686ffd640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740047d0 0x7f0674004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:48:47.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0687fff640 1 -- 172.21.15.39:0/3903465288 reap_dead start 2024-09-13T16:48:47.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f0687fff640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0674005260 0x7f067400b990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f068808b1c0 tx=0x7f068001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340b5781d90f8a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:47.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880c7330 con 0x7f0674005260 2024-09-13T16:48:47.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06800a7050 con 0x7f0674005260 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f06867fc640 1 -- 172.21.15.39:0/3903465288 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680003600 con 0x7f0674005260 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f068e09b640 1 -- 172.21.15.39:0/3903465288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0674005260 msgr2=0x7f067400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f068e09b640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0674005260 0x7f067400b990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f068808b1c0 tx=0x7f068001c400 comp rx=0 tx=0).stop 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f068e09b640 1 -- 172.21.15.39:0/3903465288 shutdown_connections 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f068e09b640 1 --2- 172.21.15.39:0/3903465288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0674005260 0x7f067400b990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f068e09b640 1 -- 172.21.15.39:0/3903465288 >> 172.21.15.39:0/3903465288 conn(0x7f0688069e10 msgr2=0x7f068806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f068e09b640 1 -- 172.21.15.39:0/3903465288 shutdown_connections 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.047+0000 7f068e09b640 1 -- 172.21.15.39:0/3903465288 wait complete. 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 Processor -- start 2024-09-13T16:48:47.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 -- start start 2024-09-13T16:48:47.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0674005260 0x7f0688149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:47.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688148130 0x7f06881568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:47.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f06881f02b0 0x7f06881f0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:47.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06881efe90 con 0x7f0688148130 2024-09-13T16:48:47.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0688149d10 con 0x7f0674005260 2024-09-13T16:48:47.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0688149ff0 con 0x7f06881f02b0 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0686ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688148130 0x7f06881568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0686ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688148130 0x7f06881568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57496/0 (socket says 172.21.15.39:57496) 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0686ffd640 1 -- 172.21.15.39:0/702508295 learned_addr learned my addr 172.21.15.39:0/702508295 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f06877fe640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0674005260 0x7f0688149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0687fff640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f06881f02b0 0x7f06881f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0686ffd640 1 -- 172.21.15.39:0/702508295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f06881f02b0 msgr2=0x7f06881f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0686ffd640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f06881f02b0 0x7f06881f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0686ffd640 1 -- 172.21.15.39:0/702508295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0674005260 msgr2=0x7f0688149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0686ffd640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0674005260 0x7f0688149730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0686ffd640 1 -- 172.21.15.39:0/702508295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068814a2d0 con 0x7f0688148130 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f0686ffd640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688148130 0x7f06881568d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0670009b90 tx=0x7f0670009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f067ffff640 1 -- 172.21.15.39:0/702508295 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670014410 con 0x7f0688148130 2024-09-13T16:48:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f067ffff640 1 -- 172.21.15.39:0/702508295 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067000f050 con 0x7f0688148130 2024-09-13T16:48:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068814a5b0 con 0x7f0688148130 2024-09-13T16:48:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f067ffff640 1 -- 172.21.15.39:0/702508295 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06700147b0 con 0x7f0688148130 2024-09-13T16:48:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0688156e10 con 0x7f0688148130 2024-09-13T16:48:47.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06881effd0 con 0x7f0688148130 2024-09-13T16:48:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.051+0000 7f067ffff640 1 -- 172.21.15.39:0/702508295 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0670014910 con 0x7f0688148130 2024-09-13T16:48:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.055+0000 7f067ffff640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f06680666c0 0x7f0668068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:48:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.055+0000 7f067ffff640 1 -- 172.21.15.39:0/702508295 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f0670084d20 con 0x7f0688148130 2024-09-13T16:48:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.055+0000 7f06877fe640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f06680666c0 0x7f0668068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:48:47.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.055+0000 7f06877fe640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f06680666c0 0x7f0668068b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0678001130 tx=0x7f0678009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:48:47.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.059+0000 7f067ffff640 1 -- 172.21.15.39:0/702508295 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f06881effd0 con 0x7f0688148130 2024-09-13T16:48:47.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.215+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06881effd0 con 0x7f0688148130 2024-09-13T16:48:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.219+0000 7f067ffff640 1 -- 172.21.15.39:0/702508295 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+20376 (secure 0 0 0) 0x7f06881effd0 con 0x7f0688148130 2024-09-13T16:48:47.225 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:48:47.226 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T16:48:40.750805+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T16:48:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f06680666c0 msgr2=0x7f0668068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f06680666c0 0x7f0668068b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0678001130 tx=0x7f0678009040 comp rx=0 tx=0).stop 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688148130 msgr2=0x7f06881568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688148130 0x7f06881568d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0670009b90 tx=0x7f0670009f90 comp rx=0 tx=0).stop 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 shutdown_connections 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f06680666c0 0x7f0668068b80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f06881f02b0 0x7f06881f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0688148130 0x7f06881568d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 --2- 172.21.15.39:0/702508295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0674005260 0x7f0688149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 >> 172.21.15.39:0/702508295 conn(0x7f0688069e10 msgr2=0x7f06880aacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 shutdown_connections 2024-09-13T16:48:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:48:47.223+0000 7f068e09b640 1 -- 172.21.15.39:0/702508295 wait complete. 2024-09-13T16:48:47.245 DEBUG:tasks.mds_thrash:Yielding 2024-09-13T16:48:47.245 INFO:teuthology.run_tasks:Running task workunit... 2024-09-13T16:48:47.258 INFO:tasks.workunit:Pulling workunits from ref 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:48:47.259 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-13T16:48:47.259 INFO:tasks.workunit:timeout=3h 2024-09-13T16:48:47.259 INFO:tasks.workunit:cleanup=True 2024-09-13T16:48:47.259 DEBUG:teuthology.orchestra.run.smithi039:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:47.260 INFO:tasks.mds_thrash.fs.[b]:starting mds_do_thrash for fs b 2024-09-13T16:48:47.260 INFO:tasks.mds_thrash.fs.[b]:waiting for 44 secs before thrashing 2024-09-13T16:48:47.269 INFO:teuthology.orchestra.run.smithi039.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:47.270 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-13T16:48:47.270 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 2dh/45d Inode: 1 Links: 2 2024-09-13T16:48:47.270 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:48:47.270 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-13T16:48:47.270 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-13 16:47:59.029255059 +0000 2024-09-13T16:48:47.270 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-13 16:48:36.804605451 +0000 2024-09-13T16:48:47.270 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2024-09-13T16:48:47.270 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-13T16:48:47.270 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-13T16:48:47.331 DEBUG:teuthology.orchestra.run.smithi084:> stat -- /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:47.339 INFO:teuthology.orchestra.run.smithi084.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:47.339 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-13T16:48:47.339 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 2dh/45d Inode: 1 Links: 2 2024-09-13T16:48:47.339 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T16:48:47.339 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-13T16:48:47.339 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-13 16:48:08.139770362 +0000 2024-09-13T16:48:47.339 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-13 16:48:36.866618336 +0000 2024-09-13T16:48:47.339 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2024-09-13T16:48:47.340 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-09-13T16:48:47.340 DEBUG:teuthology.orchestra.run.smithi084:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-09-13T16:48:47.404 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:48:47.405 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T16:48:47.413 INFO:tasks.workunit.client.0.smithi039.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-13T16:48:47.453 INFO:tasks.workunit.client.1.smithi084.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-09-13T16:48:55.428 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:48:55.431 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:49:25.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:49:25.481 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:49:31.260 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:31.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.414+0000 7f5494778640 1 Processor -- start 2024-09-13T16:49:31.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.414+0000 7f5494778640 1 -- start start 2024-09-13T16:49:31.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.414+0000 7f5494778640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f548c0c5870 0x7f548c0c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:31.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.414+0000 7f5494778640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c153e30 con 0x7f548c0c2370 2024-09-13T16:49:31.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.414+0000 7f5494778640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c0a7000 con 0x7f548c154bb0 2024-09-13T16:49:31.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.414+0000 7f5494778640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548c0720f0 con 0x7f548c0c5870 2024-09-13T16:49:31.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.414+0000 7f5492514640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f548c0c2370 0x7f548c0c0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52180/0 (socket says 172.21.15.39:52180) 2024-09-13T16:49:31.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.414+0000 7f5492514640 1 -- 172.21.15.39:0/1002535317 learned_addr learned my addr 172.21.15.39:0/1002535317 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:49:31.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.414+0000 7f5492d15640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f548c0c5870 0x7f548c0c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:31.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.418+0000 7f5492d15640 1 -- 172.21.15.39:0/1002535317 >> v1:172.21.15.84:6789/0 conn(0x7f548c154bb0 legacy=0x7f548c15c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:31.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.418+0000 7f5492d15640 1 -- 172.21.15.39:0/1002535317 >> v1:172.21.15.39:6789/0 conn(0x7f548c0c2370 legacy=0x7f548c0c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:31.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.418+0000 7f5491512640 1 -- 172.21.15.39:0/1002535317 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052342553 0 0) 0x7f548c153e30 con 0x7f548c0c2370 2024-09-13T16:49:31.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.418+0000 7f5492d15640 1 -- 172.21.15.39:0/1002535317 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c1540d0 con 0x7f548c0c5870 2024-09-13T16:49:31.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.418+0000 7f5492d15640 1 --2- 172.21.15.39:0/1002535317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c0c5870 0x7f548c0c2750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5484009d90 tx=0x7f548401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faa1140edfe0464 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:31.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.418+0000 7f5491512640 1 -- 172.21.15.39:0/1002535317 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840a57a0 con 0x7f548c0c5870 2024-09-13T16:49:31.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.418+0000 7f5491512640 1 -- 172.21.15.39:0/1002535317 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54840a5900 con 0x7f548c0c5870 2024-09-13T16:49:31.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.418+0000 7f5491512640 1 -- 172.21.15.39:0/1002535317 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840a5c90 con 0x7f548c0c5870 2024-09-13T16:49:31.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 -- 172.21.15.39:0/1002535317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c0c5870 msgr2=0x7f548c0c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:31.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 --2- 172.21.15.39:0/1002535317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c0c5870 0x7f548c0c2750 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5484009d90 tx=0x7f548401c8e0 comp rx=0 tx=0).stop 2024-09-13T16:49:31.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 -- 172.21.15.39:0/1002535317 shutdown_connections 2024-09-13T16:49:31.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 --2- 172.21.15.39:0/1002535317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c0c5870 0x7f548c0c2750 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:31.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 -- 172.21.15.39:0/1002535317 >> 172.21.15.39:0/1002535317 conn(0x7f548c0a9720 msgr2=0x7f548c0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:31.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 -- 172.21.15.39:0/1002535317 shutdown_connections 2024-09-13T16:49:31.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 -- 172.21.15.39:0/1002535317 wait complete. 2024-09-13T16:49:31.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 Processor -- start 2024-09-13T16:49:31.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 -- start start 2024-09-13T16:49:31.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f548c0c2370 0x7f548c142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:31.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0c5870 0x7f548c152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:31.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c154bb0 0x7f548c1ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:31.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c14d170 con 0x7f548c154bb0 2024-09-13T16:49:31.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c1f0570 con 0x7f548c0c5870 2024-09-13T16:49:31.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5494778640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c1efc80 con 0x7f548c0c2370 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5492d15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c154bb0 0x7f548c1ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5492d15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c154bb0 0x7f548c1ec7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49130/0 (socket says 172.21.15.39:49130) 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5492d15640 1 -- 172.21.15.39:0/1364103656 learned_addr learned my addr 172.21.15.39:0/1364103656 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5492514640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f548c0c2370 0x7f548c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5491d13640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0c5870 0x7f548c152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5492d15640 1 -- 172.21.15.39:0/1364103656 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f548c0c2370 msgr2=0x7f548c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.422+0000 7f5492d15640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f548c0c2370 0x7f548c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5492d15640 1 -- 172.21.15.39:0/1364103656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0c5870 msgr2=0x7f548c152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5492d15640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0c5870 0x7f548c152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5492d15640 1 -- 172.21.15.39:0/1364103656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484009a70 con 0x7f548c154bb0 2024-09-13T16:49:31.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5492d15640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c154bb0 0x7f548c1ec7d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f548401a510 tx=0x7f5484019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:31.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5482ffd640 1 -- 172.21.15.39:0/1364103656 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840ad850 con 0x7f548c154bb0 2024-09-13T16:49:31.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5482ffd640 1 -- 172.21.15.39:0/1364103656 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54840ad9b0 con 0x7f548c154bb0 2024-09-13T16:49:31.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c153a80 con 0x7f548c154bb0 2024-09-13T16:49:31.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c1f0200 con 0x7f548c154bb0 2024-09-13T16:49:31.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c1f06e0 con 0x7f548c154bb0 2024-09-13T16:49:31.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5482ffd640 1 -- 172.21.15.39:0/1364103656 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840add40 con 0x7f548c154bb0 2024-09-13T16:49:31.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5482ffd640 1 -- 172.21.15.39:0/1364103656 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54840b4050 con 0x7f548c154bb0 2024-09-13T16:49:31.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5482ffd640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5470066710 0x7f5470068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:31.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5492514640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5470066710 0x7f5470068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:31.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.426+0000 7f5482ffd640 1 -- 172.21.15.39:0/1364103656 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f548411f3a0 con 0x7f548c154bb0 2024-09-13T16:49:31.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.430+0000 7f5492514640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5470066710 0x7f5470068bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5488007fd0 tx=0x7f5488007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:31.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.438+0000 7f5482ffd640 1 -- 172.21.15.39:0/1364103656 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f54840ec550 con 0x7f548c154bb0 2024-09-13T16:49:31.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.818+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f548c0c2750 con 0x7f548c154bb0 2024-09-13T16:49:31.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5482ffd640 1 -- 172.21.15.39:0/1364103656 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7f54840efb70 con 0x7f548c154bb0 2024-09-13T16:49:31.824 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:31.824 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:49:31.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5470066710 msgr2=0x7f5470068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:31.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5470066710 0x7f5470068bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5488007fd0 tx=0x7f5488007d60 comp rx=0 tx=0).stop 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c154bb0 msgr2=0x7f548c1ec7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c154bb0 0x7f548c1ec7d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f548401a510 tx=0x7f5484019680 comp rx=0 tx=0).stop 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 shutdown_connections 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5470066710 0x7f5470068bd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f548c0c5870 0x7f548c152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548c154bb0 0x7f548c1ec7d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 --2- 172.21.15.39:0/1364103656 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f548c0c2370 0x7f548c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 >> 172.21.15.39:0/1364103656 conn(0x7f548c0a9720 msgr2=0x7f548c149950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 shutdown_connections 2024-09-13T16:49:31.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:31.822+0000 7f5494778640 1 -- 172.21.15.39:0/1364103656 wait complete. 2024-09-13T16:49:31.847 INFO:tasks.mds_thrash.fs.[b]:kill mds.a (rank=0) 2024-09-13T16:49:31.847 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:49:31.848 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:49:31.860 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:49:31.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbed0385640 1 Processor -- start 2024-09-13T16:49:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbed0385640 1 -- start start 2024-09-13T16:49:32.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbed0385640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbec8065ee0 0x7fbec814f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:32.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbed0385640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbec80c2ef0 0x7fbec80c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:32.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbed0385640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec806ddf0 con 0x7fbec80c0a90 2024-09-13T16:49:32.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbed0385640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec80c53d0 con 0x7fbec8065ee0 2024-09-13T16:49:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbed0385640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec80c5950 con 0x7fbec80c2ef0 2024-09-13T16:49:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbece121640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbec8065ee0 0x7fbec814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbece922640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbec80c2ef0 0x7fbec80c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbece121640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbec8065ee0 0x7fbec814f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49138/0 (socket says 172.21.15.39:49138) 2024-09-13T16:49:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbece121640 1 -- 172.21.15.39:0/3427005301 learned_addr learned my addr 172.21.15.39:0/3427005301 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbecd920640 1 --1- 172.21.15.39:0/3427005301 >> v1:172.21.15.84:6789/0 conn(0x7fbec80c0a90 0x7fbec80c1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:52506/0 (socket says 172.21.15.39:52506) 2024-09-13T16:49:32.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbece121640 1 -- 172.21.15.39:0/3427005301 >> v2:172.21.15.84:3300/0 conn(0x7fbec80c2ef0 msgr2=0x7fbec80c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:32.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbece121640 1 --2- 172.21.15.39:0/3427005301 >> v2:172.21.15.84:3300/0 conn(0x7fbec80c2ef0 0x7fbec80c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:32.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbece121640 1 -- 172.21.15.39:0/3427005301 >> v1:172.21.15.84:6789/0 conn(0x7fbec80c0a90 legacy=0x7fbec80c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:32.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.058+0000 7fbece121640 1 -- 172.21.15.39:0/3427005301 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec806e040 con 0x7fbec8065ee0 2024-09-13T16:49:32.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbece121640 1 --2- 172.21.15.39:0/3427005301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 0x7fbec814f230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbec8066cc0 tx=0x7fbec401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f8f5dbd5c9be7b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:32.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbecd11f640 1 -- 172.21.15.39:0/3427005301 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec40aa650 con 0x7fbec8065ee0 2024-09-13T16:49:32.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbecd11f640 1 -- 172.21.15.39:0/3427005301 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec40aa7b0 con 0x7fbec8065ee0 2024-09-13T16:49:32.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbecd11f640 1 -- 172.21.15.39:0/3427005301 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec40aab40 con 0x7fbec8065ee0 2024-09-13T16:49:32.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbed0385640 1 -- 172.21.15.39:0/3427005301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 msgr2=0x7fbec814f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:32.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbed0385640 1 --2- 172.21.15.39:0/3427005301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 0x7fbec814f230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbec8066cc0 tx=0x7fbec401c4f0 comp rx=0 tx=0).stop 2024-09-13T16:49:32.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbed0385640 1 -- 172.21.15.39:0/3427005301 shutdown_connections 2024-09-13T16:49:32.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbed0385640 1 --2- 172.21.15.39:0/3427005301 >> v2:172.21.15.84:3300/0 conn(0x7fbec80c2ef0 0x7fbec80c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbed0385640 1 --2- 172.21.15.39:0/3427005301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 0x7fbec814f230 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbed0385640 1 -- 172.21.15.39:0/3427005301 >> 172.21.15.39:0/3427005301 conn(0x7fbec80c3310 msgr2=0x7fbec80c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbed0385640 1 -- 172.21.15.39:0/3427005301 shutdown_connections 2024-09-13T16:49:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.062+0000 7fbed0385640 1 -- 172.21.15.39:0/3427005301 wait complete. 2024-09-13T16:49:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 Processor -- start 2024-09-13T16:49:32.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 -- start start 2024-09-13T16:49:32.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 0x7fbec8156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec80c0a90 0x7fbec80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece121640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 0x7fbec8156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece121640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 0x7fbec8156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49142/0 (socket says 172.21.15.39:49142) 2024-09-13T16:49:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece121640 1 -- 172.21.15.39:0/174501908 learned_addr learned my addr 172.21.15.39:0/174501908 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c2ef0 0x7fbec80bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8070910 con 0x7fbec8065ee0 2024-09-13T16:49:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec814aad0 con 0x7fbec80c0a90 2024-09-13T16:49:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8071de0 con 0x7fbec80c2ef0 2024-09-13T16:49:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbecd920640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec80c0a90 0x7fbec80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece922640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c2ef0 0x7fbec80bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece121640 1 -- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c2ef0 msgr2=0x7fbec80bfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece121640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c2ef0 0x7fbec80bfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece121640 1 -- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec80c0a90 msgr2=0x7fbec80703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece121640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec80c0a90 0x7fbec80703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece121640 1 -- 172.21.15.39:0/174501908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec40094d0 con 0x7fbec8065ee0 2024-09-13T16:49:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece922640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c2ef0 0x7fbec80bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:49:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbece121640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 0x7fbec8156e70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbec40ac040 tx=0x7fbec401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbebeffd640 1 -- 172.21.15.39:0/174501908 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec40b3800 con 0x7fbec8065ee0 2024-09-13T16:49:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbebeffd640 1 -- 172.21.15.39:0/174501908 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec40b3980 con 0x7fbec8065ee0 2024-09-13T16:49:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec80720c0 con 0x7fbec8065ee0 2024-09-13T16:49:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbebeffd640 1 -- 172.21.15.39:0/174501908 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec40b3d30 con 0x7fbec8065ee0 2024-09-13T16:49:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.066+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec81efa40 con 0x7fbec8065ee0 2024-09-13T16:49:32.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.070+0000 7fbebeffd640 1 -- 172.21.15.39:0/174501908 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbec40b2380 con 0x7fbec8065ee0 2024-09-13T16:49:32.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.070+0000 7fbebeffd640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbeac066690 0x7fbeac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:32.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.070+0000 7fbebeffd640 1 -- 172.21.15.39:0/174501908 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fbec411e7b0 con 0x7fbec8065ee0 2024-09-13T16:49:32.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.070+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb0005260 con 0x7fbec8065ee0 2024-09-13T16:49:32.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.070+0000 7fbecd920640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbeac066690 0x7fbeac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:32.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.070+0000 7fbecd920640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbeac066690 0x7fbeac068b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbec81ed4d0 tx=0x7fbeb8007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:32.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.078+0000 7fbebeffd640 1 -- 172.21.15.39:0/174501908 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbec40eb810 con 0x7fbec8065ee0 2024-09-13T16:49:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.458+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbeb0005510 con 0x7fbec8065ee0 2024-09-13T16:49:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.458+0000 7fbebeffd640 1 -- 172.21.15.39:0/174501908 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7fbec40eee30 con 0x7fbec8065ee0 2024-09-13T16:49:32.462 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:32.462 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:32.463 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:49:32.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbeac066690 msgr2=0x7fbeac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbeac066690 0x7fbeac068b50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbec81ed4d0 tx=0x7fbeb8007c90 comp rx=0 tx=0).stop 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 msgr2=0x7fbec8156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 0x7fbec8156e70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbec40ac040 tx=0x7fbec401ad30 comp rx=0 tx=0).stop 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 shutdown_connections 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbeac066690 0x7fbeac068b50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec80c0a90 0x7fbec80703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c2ef0 0x7fbec80bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 --2- 172.21.15.39:0/174501908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8065ee0 0x7fbec8156e70 secure :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbec40ac040 tx=0x7fbec401ad30 comp rx=0 tx=0).stop 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 >> 172.21.15.39:0/174501908 conn(0x7fbec80c3310 msgr2=0x7fbec815bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 shutdown_connections 2024-09-13T16:49:32.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:32.462+0000 7fbed0385640 1 -- 172.21.15.39:0/174501908 wait complete. 2024-09-13T16:49:32.488 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:49:34.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.642+0000 7fc77ba5d640 1 Processor -- start 2024-09-13T16:49:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.642+0000 7fc77ba5d640 1 -- start start 2024-09-13T16:49:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc77ba5d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc77414acf0 0x7fc774151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc77ba5d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc774154ab0 con 0x7fc7740c4b70 2024-09-13T16:49:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc77ba5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc774154d90 con 0x7fc7740c4710 2024-09-13T16:49:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc77ba5d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc774150d50 con 0x7fc77414acf0 2024-09-13T16:49:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc779ffa640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc7740c4b70 0x7fc774153ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52188/0 (socket says 172.21.15.39:52188) 2024-09-13T16:49:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc779ffa640 1 -- 172.21.15.39:0/3259319771 learned_addr learned my addr 172.21.15.39:0/3259319771 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:49:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc7797f9640 1 --2- 172.21.15.39:0/3259319771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc77414acf0 0x7fc774151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc76bfff640 1 -- 172.21.15.39:0/3259319771 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048778116 0 0) 0x7fc774154ab0 con 0x7fc7740c4b70 2024-09-13T16:49:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc76bfff640 1 -- 172.21.15.39:0/3259319771 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc760003530 con 0x7fc7740c4b70 2024-09-13T16:49:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc7797f9640 1 -- 172.21.15.39:0/3259319771 >> v1:172.21.15.84:6789/0 conn(0x7fc7740c4710 legacy=0x7fc77406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc7797f9640 1 -- 172.21.15.39:0/3259319771 >> v1:172.21.15.39:6789/0 conn(0x7fc7740c4b70 legacy=0x7fc774153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:34.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.646+0000 7fc7797f9640 1 -- 172.21.15.39:0/3259319771 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc774151030 con 0x7fc77414acf0 2024-09-13T16:49:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc7797f9640 1 --2- 172.21.15.39:0/3259319771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc77414acf0 0x7fc774151550 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc770009820 tx=0x7fc77001c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfdcef9a847c65cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc76bfff640 1 -- 172.21.15.39:0/3259319771 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700ab5f0 con 0x7fc77414acf0 2024-09-13T16:49:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc76bfff640 1 -- 172.21.15.39:0/3259319771 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7700ab750 con 0x7fc77414acf0 2024-09-13T16:49:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc76bfff640 1 -- 172.21.15.39:0/3259319771 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700abae0 con 0x7fc77414acf0 2024-09-13T16:49:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc77ba5d640 1 -- 172.21.15.39:0/3259319771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc77414acf0 msgr2=0x7fc774151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc77ba5d640 1 --2- 172.21.15.39:0/3259319771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc77414acf0 0x7fc774151550 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc770009820 tx=0x7fc77001c3d0 comp rx=0 tx=0).stop 2024-09-13T16:49:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc77ba5d640 1 -- 172.21.15.39:0/3259319771 shutdown_connections 2024-09-13T16:49:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc77ba5d640 1 --2- 172.21.15.39:0/3259319771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc77414acf0 0x7fc774151550 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc77ba5d640 1 -- 172.21.15.39:0/3259319771 >> 172.21.15.39:0/3259319771 conn(0x7fc7740c32a0 msgr2=0x7fc7740c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc77ba5d640 1 -- 172.21.15.39:0/3259319771 shutdown_connections 2024-09-13T16:49:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc77ba5d640 1 -- 172.21.15.39:0/3259319771 wait complete. 2024-09-13T16:49:34.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc77ba5d640 1 Processor -- start 2024-09-13T16:49:34.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.650+0000 7fc77ba5d640 1 -- start start 2024-09-13T16:49:34.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc77ba5d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7740c4710 0x7fc7740a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:34.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc77ba5d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7740c4b70 0x7fc7740aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:34.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc7797f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7740c4710 0x7fc7740a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:34.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc7797f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7740c4710 0x7fc7740a99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49154/0 (socket says 172.21.15.39:49154) 2024-09-13T16:49:34.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc7797f9640 1 -- 172.21.15.39:0/792458160 learned_addr learned my addr 172.21.15.39:0/792458160 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc77ba5d640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc77414acf0 0x7fc77414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc774148500 con 0x7fc7740c4710 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7740aad00 con 0x7fc77414acf0 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7740ab5a0 con 0x7fc7740c4b70 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc778ff8640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7740c4b70 0x7fc7740aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc779ffa640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc77414acf0 0x7fc77414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc778ff8640 1 -- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc77414acf0 msgr2=0x7fc77414b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc778ff8640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc77414acf0 0x7fc77414b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc778ff8640 1 -- 172.21.15.39:0/792458160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7740c4710 msgr2=0x7fc7740a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc778ff8640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7740c4710 0x7fc7740a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc778ff8640 1 -- 172.21.15.39:0/792458160 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc770009420 con 0x7fc7740c4b70 2024-09-13T16:49:34.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc778ff8640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7740c4b70 0x7fc7740aba30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc764000f70 tx=0x7fc76400c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:34.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc769ffb640 1 -- 172.21.15.39:0/792458160 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640052d0 con 0x7fc7740c4b70 2024-09-13T16:49:34.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc769ffb640 1 -- 172.21.15.39:0/792458160 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc764005430 con 0x7fc7740c4b70 2024-09-13T16:49:34.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7740abf70 con 0x7fc7740c4b70 2024-09-13T16:49:34.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7740aaf50 con 0x7fc7740c4b70 2024-09-13T16:49:34.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc769ffb640 1 -- 172.21.15.39:0/792458160 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764003840 con 0x7fc7740c4b70 2024-09-13T16:49:34.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.654+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc754005260 con 0x7fc7740c4b70 2024-09-13T16:49:34.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.658+0000 7fc769ffb640 1 -- 172.21.15.39:0/792458160 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7640039a0 con 0x7fc7740c4b70 2024-09-13T16:49:34.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.658+0000 7fc769ffb640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc758066690 0x7fc758068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:34.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.658+0000 7fc7797f9640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc758066690 0x7fc758068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:34.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.658+0000 7fc769ffb640 1 -- 172.21.15.39:0/792458160 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fc764083900 con 0x7fc7740c4b70 2024-09-13T16:49:34.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.658+0000 7fc7797f9640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc758066690 0x7fc758068b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc770009740 tx=0x7fc7700b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:34.666+0000 7fc769ffb640 1 -- 172.21.15.39:0/792458160 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc764050970 con 0x7fc7740c4b70 2024-09-13T16:49:35.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.046+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc754005a90 con 0x7fc7740c4b70 2024-09-13T16:49:35.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc769ffb640 1 -- 172.21.15.39:0/792458160 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7fc764053f90 con 0x7fc7740c4b70 2024-09-13T16:49:35.052 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:35.052 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:35.053 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:49:35.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc758066690 msgr2=0x7fc758068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:35.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc758066690 0x7fc758068b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc770009740 tx=0x7fc7700b6040 comp rx=0 tx=0).stop 2024-09-13T16:49:35.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7740c4b70 msgr2=0x7fc7740aba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:35.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7740c4b70 0x7fc7740aba30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc764000f70 tx=0x7fc76400c820 comp rx=0 tx=0).stop 2024-09-13T16:49:35.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 shutdown_connections 2024-09-13T16:49:35.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc758066690 0x7fc758068b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:35.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc77414acf0 0x7fc77414b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:35.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7740c4b70 0x7fc7740aba30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:35.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 --2- 172.21.15.39:0/792458160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7740c4710 0x7fc7740a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:35.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 >> 172.21.15.39:0/792458160 conn(0x7fc7740c32a0 msgr2=0x7fc77406ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:35.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 shutdown_connections 2024-09-13T16:49:35.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:35.050+0000 7fc77ba5d640 1 -- 172.21.15.39:0/792458160 wait complete. 2024-09-13T16:49:35.077 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:49:37.078 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:37.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274fcb7640 1 Processor -- start 2024-09-13T16:49:37.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274fcb7640 1 -- start start 2024-09-13T16:49:37.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274fcb7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f27480a5ef0 0x7f2748152800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274fcb7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2748151bf0 con 0x7f2748155160 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274fcb7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2748155660 con 0x7f274814bf30 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274fcb7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2748155940 con 0x7f27480a5ef0 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274d252640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2748155160 0x7f27481520f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52200/0 (socket says 172.21.15.39:52200) 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274d252640 1 -- 172.21.15.39:0/3695712458 learned_addr learned my addr 172.21.15.39:0/3695712458 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274e254640 1 --2- 172.21.15.39:0/3695712458 >> v2:172.21.15.84:3300/0 conn(0x7f27480a5ef0 0x7f2748152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274ca51640 1 -- 172.21.15.39:0/3695712458 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765502949 0 0) 0x7f2748151bf0 con 0x7f2748155160 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274ca51640 1 -- 172.21.15.39:0/3695712458 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2734003530 con 0x7f2748155160 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274e254640 1 -- 172.21.15.39:0/3695712458 >> v1:172.21.15.84:6789/0 conn(0x7f274814bf30 legacy=0x7f274806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274e254640 1 -- 172.21.15.39:0/3695712458 >> v1:172.21.15.39:6789/0 conn(0x7f2748155160 legacy=0x7f27481520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:37.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.234+0000 7f274e254640 1 -- 172.21.15.39:0/3695712458 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748155c20 con 0x7f27480a5ef0 2024-09-13T16:49:37.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274e254640 1 --2- 172.21.15.39:0/3695712458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 0x7f2748152800 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2740009d90 tx=0x7f274001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e2c0759a07e4327 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:37.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274ca51640 1 -- 172.21.15.39:0/3695712458 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27400a59e0 con 0x7f27480a5ef0 2024-09-13T16:49:37.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274ca51640 1 -- 172.21.15.39:0/3695712458 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27400a5b40 con 0x7f27480a5ef0 2024-09-13T16:49:37.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274ca51640 1 -- 172.21.15.39:0/3695712458 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27400ad440 con 0x7f27480a5ef0 2024-09-13T16:49:37.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274fcb7640 1 -- 172.21.15.39:0/3695712458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 msgr2=0x7f2748152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:37.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274fcb7640 1 --2- 172.21.15.39:0/3695712458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 0x7f2748152800 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2740009d90 tx=0x7f274001c940 comp rx=0 tx=0).stop 2024-09-13T16:49:37.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274fcb7640 1 -- 172.21.15.39:0/3695712458 shutdown_connections 2024-09-13T16:49:37.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274fcb7640 1 --2- 172.21.15.39:0/3695712458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 0x7f2748152800 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:37.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274fcb7640 1 -- 172.21.15.39:0/3695712458 >> 172.21.15.39:0/3695712458 conn(0x7f27480c32a0 msgr2=0x7f27480c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:37.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274fcb7640 1 -- 172.21.15.39:0/3695712458 shutdown_connections 2024-09-13T16:49:37.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.238+0000 7f274fcb7640 1 -- 172.21.15.39:0/3695712458 wait complete. 2024-09-13T16:49:37.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 Processor -- start 2024-09-13T16:49:37.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 -- start start 2024-09-13T16:49:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 0x7f27480c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f274814bf30 0x7f274814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2748155160 0x7f274814edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274da53640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 0x7f27480c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274d252640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f274814bf30 0x7f274814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274e254640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2748155160 0x7f274814edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274da53640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 0x7f27480c23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:49426/0 (socket says 172.21.15.39:49426) 2024-09-13T16:49:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274d252640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f274814bf30 0x7f274814d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:56904/0 (socket says 172.21.15.39:56904) 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274e254640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2748155160 0x7f274814edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49162/0 (socket says 172.21.15.39:49162) 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274da53640 1 -- 172.21.15.39:0/1170092792 learned_addr learned my addr 172.21.15.39:0/1170092792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 -- 172.21.15.39:0/1170092792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2748154c90 con 0x7f2748155160 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 -- 172.21.15.39:0/1170092792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274814c560 con 0x7f27480a5ef0 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 -- 172.21.15.39:0/1170092792 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27480c2dd0 con 0x7f274814bf30 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274da53640 1 -- 172.21.15.39:0/1170092792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f274814bf30 msgr2=0x7f274814d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274da53640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f274814bf30 0x7f274814d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274da53640 1 -- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2748155160 msgr2=0x7f274814edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274da53640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2748155160 0x7f274814edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274da53640 1 -- 172.21.15.39:0/1170092792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2740009a70 con 0x7f27480a5ef0 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274d252640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f274814bf30 0x7f274814d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274e254640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2748155160 0x7f274814edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:49:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274da53640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 0x7f27480c23e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f274400c3c0 tx=0x7f274400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f273e7fc640 1 -- 172.21.15.39:0/1170092792 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27440052d0 con 0x7f27480a5ef0 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f273e7fc640 1 -- 172.21.15.39:0/1170092792 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2744005430 con 0x7f27480a5ef0 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f273e7fc640 1 -- 172.21.15.39:0/1170092792 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744003840 con 0x7f27480a5ef0 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 -- 172.21.15.39:0/1170092792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27481546d0 con 0x7f27480a5ef0 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.242+0000 7f274fcb7640 1 -- 172.21.15.39:0/1170092792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2748072260 con 0x7f27480a5ef0 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.246+0000 7f272bfff640 1 -- 172.21.15.39:0/1170092792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2724005260 con 0x7f27480a5ef0 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.246+0000 7f273e7fc640 1 -- 172.21.15.39:0/1170092792 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27440039a0 con 0x7f27480a5ef0 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.246+0000 7f273e7fc640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f272c0666c0 0x7f272c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.246+0000 7f274d252640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f272c0666c0 0x7f272c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.246+0000 7f273e7fc640 1 -- 172.21.15.39:0/1170092792 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f2744083da0 con 0x7f27480a5ef0 2024-09-13T16:49:37.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.250+0000 7f274d252640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f272c0666c0 0x7f272c068b80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f27480a7bd0 tx=0x7f2738007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:37.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.254+0000 7f273e7fc640 1 -- 172.21.15.39:0/1170092792 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2744050e90 con 0x7f27480a5ef0 2024-09-13T16:49:37.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.622+0000 7f272bfff640 1 -- 172.21.15.39:0/1170092792 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27240057e0 con 0x7f27480a5ef0 2024-09-13T16:49:37.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.622+0000 7f273e7fc640 1 -- 172.21.15.39:0/1170092792 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7f2744017090 con 0x7f27480a5ef0 2024-09-13T16:49:37.625 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:37.625 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:37.625 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:49:37.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 -- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f272c0666c0 msgr2=0x7f272c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:37.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f272c0666c0 0x7f272c068b80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f27480a7bd0 tx=0x7f2738007d40 comp rx=0 tx=0).stop 2024-09-13T16:49:37.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 -- 172.21.15.39:0/1170092792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 msgr2=0x7f27480c23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:37.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 0x7f27480c23e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f274400c3c0 tx=0x7f274400c880 comp rx=0 tx=0).stop 2024-09-13T16:49:37.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 -- 172.21.15.39:0/1170092792 shutdown_connections 2024-09-13T16:49:37.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f272c0666c0 0x7f272c068b80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:37.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2748155160 0x7f274814edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:37.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f274814bf30 0x7f274814d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:37.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 --2- 172.21.15.39:0/1170092792 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27480a5ef0 0x7f27480c23e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:37.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 -- 172.21.15.39:0/1170092792 >> 172.21.15.39:0/1170092792 conn(0x7f27480c32a0 msgr2=0x7f27480c4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:37.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 -- 172.21.15.39:0/1170092792 shutdown_connections 2024-09-13T16:49:37.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:37.626+0000 7f272bfff640 1 -- 172.21.15.39:0/1170092792 wait complete. 2024-09-13T16:49:37.648 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:49:39.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.802+0000 7fee5ce71640 1 Processor -- start 2024-09-13T16:49:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee5ce71640 1 -- start start 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee5ce71640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fee5806e5e0 0x7fee58151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee5ce71640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee5ce71640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5814f590 con 0x7fee5814f1b0 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee5ce71640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5806f660 con 0x7fee5806e5e0 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee5ce71640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5806eae0 con 0x7fee580c7470 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee4ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fee5806e5e0 0x7fee58151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee4ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fee5806e5e0 0x7fee58151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49174/0 (socket says 172.21.15.39:49174) 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee4ffff640 1 -- 172.21.15.39:0/102206524 learned_addr learned my addr 172.21.15.39:0/102206524 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee56d9d640 1 --2- 172.21.15.39:0/102206524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee4ffff640 1 -- 172.21.15.39:0/102206524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 msgr2=0x7fee580c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee4ffff640 1 --2- 172.21.15.39:0/102206524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee4ffff640 1 -- 172.21.15.39:0/102206524 >> v1:172.21.15.84:6789/0 conn(0x7fee5814f1b0 legacy=0x7fee580714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:39.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.806+0000 7fee4ffff640 1 -- 172.21.15.39:0/102206524 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5806edc0 con 0x7fee5806e5e0 2024-09-13T16:49:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee4ffff640 1 --2- 172.21.15.39:0/102206524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee5806e5e0 0x7fee58151bb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fee40009870 tx=0x7fee40020940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd6bd38cee9fd68 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee55d9b640 1 -- 172.21.15.39:0/102206524 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee580c69a0 con 0x7fee5806e5e0 2024-09-13T16:49:39.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee55d9b640 1 -- 172.21.15.39:0/102206524 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee400a9700 con 0x7fee5806e5e0 2024-09-13T16:49:39.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee55d9b640 1 -- 172.21.15.39:0/102206524 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee400a9a90 con 0x7fee5806e5e0 2024-09-13T16:49:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee5ce71640 1 -- 172.21.15.39:0/102206524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee5806e5e0 msgr2=0x7fee58151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee5ce71640 1 --2- 172.21.15.39:0/102206524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee5806e5e0 0x7fee58151bb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fee40009870 tx=0x7fee40020940 comp rx=0 tx=0).stop 2024-09-13T16:49:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee5ce71640 1 -- 172.21.15.39:0/102206524 shutdown_connections 2024-09-13T16:49:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee5ce71640 1 --2- 172.21.15.39:0/102206524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee5ce71640 1 --2- 172.21.15.39:0/102206524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee5806e5e0 0x7fee58151bb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee5ce71640 1 -- 172.21.15.39:0/102206524 >> 172.21.15.39:0/102206524 conn(0x7fee580a9720 msgr2=0x7fee580a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee5ce71640 1 -- 172.21.15.39:0/102206524 shutdown_connections 2024-09-13T16:49:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.810+0000 7fee5ce71640 1 -- 172.21.15.39:0/102206524 wait complete. 2024-09-13T16:49:39.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 Processor -- start 2024-09-13T16:49:39.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 -- start start 2024-09-13T16:49:39.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5806e5e0 0x7fee58147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:39.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee5814f1b0 0x7fee580c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee580aab40 con 0x7fee5814f1b0 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee580aae20 con 0x7fee5806e5e0 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee4ffff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5806e5e0 0x7fee58147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee4ffff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:56922/0 (socket says 172.21.15.39:56922) 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5806e5e0 0x7fee58147ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:49432/0 (socket says 172.21.15.39:49432) 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee4ffff640 1 -- 172.21.15.39:0/1448660712 learned_addr learned my addr 172.21.15.39:0/1448660712 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee56d9d640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee5814f1b0 0x7fee580c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee580ab3c0 con 0x7fee580c7470 2024-09-13T16:49:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5659c640 1 -- 172.21.15.39:0/1448660712 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 msgr2=0x7fee580c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5659c640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5659c640 1 -- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee5814f1b0 msgr2=0x7fee580c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5659c640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee5814f1b0 0x7fee580c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5659c640 1 -- 172.21.15.39:0/1448660712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee400094d0 con 0x7fee5806e5e0 2024-09-13T16:49:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee4ffff640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:49:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5659c640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5806e5e0 0x7fee58147ca0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee48000f70 tx=0x7fee4800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee4effd640 1 -- 172.21.15.39:0/1448660712 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee480052d0 con 0x7fee5806e5e0 2024-09-13T16:49:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee4effd640 1 -- 172.21.15.39:0/1448660712 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee48005430 con 0x7fee5806e5e0 2024-09-13T16:49:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee580ab6a0 con 0x7fee5806e5e0 2024-09-13T16:49:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee4effd640 1 -- 172.21.15.39:0/1448660712 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48003840 con 0x7fee5806e5e0 2024-09-13T16:49:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.814+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee58149960 con 0x7fee5806e5e0 2024-09-13T16:49:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.818+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee580ab220 con 0x7fee5806e5e0 2024-09-13T16:49:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.818+0000 7fee4effd640 1 -- 172.21.15.39:0/1448660712 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee480039a0 con 0x7fee5806e5e0 2024-09-13T16:49:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.818+0000 7fee4effd640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee38066690 0x7fee38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.818+0000 7fee4effd640 1 -- 172.21.15.39:0/1448660712 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fee48083a40 con 0x7fee5806e5e0 2024-09-13T16:49:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.818+0000 7fee4ffff640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee38066690 0x7fee38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.818+0000 7fee4ffff640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee38066690 0x7fee38068b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee40020980 tx=0x7fee40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:39.826+0000 7fee4effd640 1 -- 172.21.15.39:0/1448660712 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fee580ab220 con 0x7fee5806e5e0 2024-09-13T16:49:40.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.206+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee580ab220 con 0x7fee5806e5e0 2024-09-13T16:49:40.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.206+0000 7fee4effd640 1 -- 172.21.15.39:0/1448660712 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7fee580ab220 con 0x7fee5806e5e0 2024-09-13T16:49:40.209 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:40.209 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:40.209 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee38066690 msgr2=0x7fee38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee38066690 0x7fee38068b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee40020980 tx=0x7fee40009040 comp rx=0 tx=0).stop 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5806e5e0 msgr2=0x7fee58147ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5806e5e0 0x7fee58147ca0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fee48000f70 tx=0x7fee4800c820 comp rx=0 tx=0).stop 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 shutdown_connections 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee38066690 0x7fee38068b50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee5814f1b0 0x7fee580c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee580c7470 0x7fee580c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 --2- 172.21.15.39:0/1448660712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5806e5e0 0x7fee58147ca0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 >> 172.21.15.39:0/1448660712 conn(0x7fee580a9720 msgr2=0x7fee580a9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:40.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 shutdown_connections 2024-09-13T16:49:40.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:40.210+0000 7fee5ce71640 1 -- 172.21.15.39:0/1448660712 wait complete. 2024-09-13T16:49:40.233 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:49:42.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:42.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c0b59f640 1 Processor -- start 2024-09-13T16:49:42.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c0b59f640 1 -- start start 2024-09-13T16:49:42.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c0b59f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04151500 0x7f9c040c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:42.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c0b59f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9c04071dd0 0x7f9c04154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:42.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c0b59f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c040c5a30 con 0x7f9c040c47c0 2024-09-13T16:49:42.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c0b59f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c041576a0 con 0x7f9c04071dd0 2024-09-13T16:49:42.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c0b59f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04157980 con 0x7f9c04151500 2024-09-13T16:49:42.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c08b3a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9c04071dd0 0x7f9c04154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:42.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c09b3c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9c040c47c0 0x7f9c041545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37118/0 (socket says 172.21.15.39:37118) 2024-09-13T16:49:42.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c09b3c640 1 -- 172.21.15.39:0/2318654785 learned_addr learned my addr 172.21.15.39:0/2318654785 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:49:42.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c0933b640 1 --2- 172.21.15.39:0/2318654785 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04151500 0x7f9c040c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:42.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c08b3a640 1 -- 172.21.15.39:0/2318654785 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04151500 msgr2=0x7f9c040c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:42.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c08b3a640 1 --2- 172.21.15.39:0/2318654785 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04151500 0x7f9c040c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c08b3a640 1 -- 172.21.15.39:0/2318654785 >> v1:172.21.15.39:6789/0 conn(0x7f9c040c47c0 legacy=0x7f9c041545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:42.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9c08b3a640 1 -- 172.21.15.39:0/2318654785 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c04157c60 con 0x7f9c04071dd0 2024-09-13T16:49:42.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.390+0000 7f9bfbfff640 1 -- 172.21.15.39:0/2318654785 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836071610 0 0) 0x7f9c040c5a30 con 0x7f9c040c47c0 2024-09-13T16:49:42.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9c08b3a640 1 --2- 172.21.15.39:0/2318654785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c04071dd0 0x7f9c04154070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40097c0 tx=0x7f9bf401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7331a615b6047f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:42.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9bfbfff640 1 -- 172.21.15.39:0/2318654785 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40ab600 con 0x7f9c04071dd0 2024-09-13T16:49:42.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9bfbfff640 1 -- 172.21.15.39:0/2318654785 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf40ab760 con 0x7f9c04071dd0 2024-09-13T16:49:42.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9bfbfff640 1 -- 172.21.15.39:0/2318654785 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40abaf0 con 0x7f9c04071dd0 2024-09-13T16:49:42.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9c0b59f640 1 -- 172.21.15.39:0/2318654785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c04071dd0 msgr2=0x7f9c04154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:42.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9c0b59f640 1 --2- 172.21.15.39:0/2318654785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c04071dd0 0x7f9c04154070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40097c0 tx=0x7f9bf401c3e0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9c0b59f640 1 -- 172.21.15.39:0/2318654785 shutdown_connections 2024-09-13T16:49:42.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9c0b59f640 1 --2- 172.21.15.39:0/2318654785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c04071dd0 0x7f9c04154070 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9c0b59f640 1 --2- 172.21.15.39:0/2318654785 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04151500 0x7f9c040c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9c0b59f640 1 -- 172.21.15.39:0/2318654785 >> 172.21.15.39:0/2318654785 conn(0x7f9c040ab530 msgr2=0x7f9c040ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:42.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9c0b59f640 1 -- 172.21.15.39:0/2318654785 shutdown_connections 2024-09-13T16:49:42.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.394+0000 7f9c0b59f640 1 -- 172.21.15.39:0/2318654785 wait complete. 2024-09-13T16:49:42.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 Processor -- start 2024-09-13T16:49:42.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 -- start start 2024-09-13T16:49:42.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04071dd0 0x7f9c0415dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:42.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c040c47c0 0x7f9c041ec780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:42.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c04151500 0x7f9c041eccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04149d00 con 0x7f9c040c47c0 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04153150 con 0x7f9c04151500 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04153430 con 0x7f9c04071dd0 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c08b3a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c040c47c0 0x7f9c041ec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c08b3a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c040c47c0 0x7f9c041ec780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36254/0 (socket says 172.21.15.39:36254) 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c08b3a640 1 -- 172.21.15.39:0/693481049 learned_addr learned my addr 172.21.15.39:0/693481049 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c09b3c640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c04151500 0x7f9c041eccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0933b640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04071dd0 0x7f9c0415dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c08b3a640 1 -- 172.21.15.39:0/693481049 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04071dd0 msgr2=0x7f9c0415dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c08b3a640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04071dd0 0x7f9c0415dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c08b3a640 1 -- 172.21.15.39:0/693481049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c04151500 msgr2=0x7f9c041eccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c08b3a640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c04151500 0x7f9c041eccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c08b3a640 1 -- 172.21.15.39:0/693481049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf4009420 con 0x7f9c040c47c0 2024-09-13T16:49:42.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c08b3a640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c040c47c0 0x7f9c041ec780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9bf401a010 tx=0x7f9bf401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:42.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9bf9ffb640 1 -- 172.21.15.39:0/693481049 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40bf610 con 0x7f9c040c47c0 2024-09-13T16:49:42.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9bf9ffb640 1 -- 172.21.15.39:0/693481049 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf40bf790 con 0x7f9c040c47c0 2024-09-13T16:49:42.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c04153710 con 0x7f9c040c47c0 2024-09-13T16:49:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9bf9ffb640 1 -- 172.21.15.39:0/693481049 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40bfb40 con 0x7f9c040c47c0 2024-09-13T16:49:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.398+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0415e260 con 0x7f9c040c47c0 2024-09-13T16:49:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.402+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be4005260 con 0x7f9c040c47c0 2024-09-13T16:49:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.402+0000 7f9bf9ffb640 1 -- 172.21.15.39:0/693481049 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9bf40bfca0 con 0x7f9c040c47c0 2024-09-13T16:49:42.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.402+0000 7f9bf9ffb640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9be8066690 0x7f9be8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:42.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.402+0000 7f9bf9ffb640 1 -- 172.21.15.39:0/693481049 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f9bf411f0f0 con 0x7f9c040c47c0 2024-09-13T16:49:42.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.402+0000 7f9c0933b640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9be8066690 0x7f9be8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:42.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.402+0000 7f9c0933b640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9be8066690 0x7f9be8068b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009ac0 tx=0x7f9c00009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:42.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.410+0000 7f9bf9ffb640 1 -- 172.21.15.39:0/693481049 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9bf40ec160 con 0x7f9c040c47c0 2024-09-13T16:49:42.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.794+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9be4005a90 con 0x7f9c040c47c0 2024-09-13T16:49:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.794+0000 7f9bf9ffb640 1 -- 172.21.15.39:0/693481049 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5925 (secure 0 0 0) 0x7f9bf40ef780 con 0x7f9c040c47c0 2024-09-13T16:49:42.797 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:42.797 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:48:12.162981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":9,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/3163580917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3163580917},{"type":"v1","addr":"172.21.15.39:6835","nonce":3163580917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 14 2024-09-13T16:49:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9be8066690 msgr2=0x7f9be8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9be8066690 0x7f9be8068b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009ac0 tx=0x7f9c00009190 comp rx=0 tx=0).stop 2024-09-13T16:49:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c040c47c0 msgr2=0x7f9c041ec780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c040c47c0 0x7f9c041ec780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9bf401a010 tx=0x7f9bf401ac20 comp rx=0 tx=0).stop 2024-09-13T16:49:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 shutdown_connections 2024-09-13T16:49:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9be8066690 0x7f9be8068b50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c04151500 0x7f9c041eccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c040c47c0 0x7f9c041ec780 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 --2- 172.21.15.39:0/693481049 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9c04071dd0 0x7f9c0415dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 >> 172.21.15.39:0/693481049 conn(0x7f9c040ab530 msgr2=0x7f9c040ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 shutdown_connections 2024-09-13T16:49:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:42.798+0000 7f9c0b59f640 1 -- 172.21.15.39:0/693481049 wait complete. 2024-09-13T16:49:42.821 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:49:44.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca3abc640 1 Processor -- start 2024-09-13T16:49:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca3abc640 1 -- start start 2024-09-13T16:49:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca3abc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c9c0c5f40 0x7f6c9c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca3abc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6c9c071ed0 0x7f6c9c1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca3abc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9c0c2e60 con 0x7f6c9c0c5790 2024-09-13T16:49:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca3abc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c0c7470 con 0x7f6c9c071ed0 2024-09-13T16:49:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca3abc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c0c2570 con 0x7f6c9c0c5f40 2024-09-13T16:49:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca1057640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6c9c071ed0 0x7f6c9c1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca1057640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6c9c071ed0 0x7f6c9c1577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36264/0 (socket says 172.21.15.39:36264) 2024-09-13T16:49:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca1057640 1 -- 172.21.15.39:0/1777665412 learned_addr learned my addr 172.21.15.39:0/1777665412 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca1858640 1 --2- 172.21.15.39:0/1777665412 >> v2:172.21.15.84:3300/0 conn(0x7f6c9c0c5f40 0x7f6c9c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca1057640 1 -- 172.21.15.39:0/1777665412 >> v2:172.21.15.84:3300/0 conn(0x7f6c9c0c5f40 msgr2=0x7f6c9c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca1057640 1 --2- 172.21.15.39:0/1777665412 >> v2:172.21.15.84:3300/0 conn(0x7f6c9c0c5f40 0x7f6c9c156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca1057640 1 -- 172.21.15.39:0/1777665412 >> v1:172.21.15.84:6789/0 conn(0x7f6c9c0c5790 legacy=0x7f6c9c157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca1057640 1 -- 172.21.15.39:0/1777665412 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c072360 con 0x7f6c9c071ed0 2024-09-13T16:49:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.958+0000 7f6ca1057640 1 --2- 172.21.15.39:0/1777665412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c071ed0 0x7f6c9c1577d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0c3c90 tx=0x7f6c8c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60109e38c5c6509d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca0856640 1 -- 172.21.15.39:0/1777665412 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0ab5f0 con 0x7f6c9c071ed0 2024-09-13T16:49:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca0856640 1 -- 172.21.15.39:0/1777665412 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0ab750 con 0x7f6c9c071ed0 2024-09-13T16:49:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca0856640 1 -- 172.21.15.39:0/1777665412 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0abae0 con 0x7f6c9c071ed0 2024-09-13T16:49:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1777665412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c071ed0 msgr2=0x7f6c9c1577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 --2- 172.21.15.39:0/1777665412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c071ed0 0x7f6c9c1577d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0c3c90 tx=0x7f6c8c01c4e0 comp rx=0 tx=0).stop 2024-09-13T16:49:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1777665412 shutdown_connections 2024-09-13T16:49:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 --2- 172.21.15.39:0/1777665412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c071ed0 0x7f6c9c1577d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 --2- 172.21.15.39:0/1777665412 >> v2:172.21.15.84:3300/0 conn(0x7f6c9c0c5f40 0x7f6c9c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1777665412 >> 172.21.15.39:0/1777665412 conn(0x7f6c9c06c020 msgr2=0x7f6c9c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1777665412 shutdown_connections 2024-09-13T16:49:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1777665412 wait complete. 2024-09-13T16:49:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 Processor -- start 2024-09-13T16:49:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- start start 2024-09-13T16:49:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c9c0c5790 0x7f6c9c150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c9c0c5f40 0x7f6c9c1ec660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c14e410 0x7f6c9c1ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0bdd70 con 0x7f6c9c14e410 2024-09-13T16:49:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0be200 con 0x7f6c9c0c5790 2024-09-13T16:49:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c06f300 con 0x7f6c9c0c5f40 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca2059640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c14e410 0x7f6c9c1ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca2059640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c14e410 0x7f6c9c1ecba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36276/0 (socket says 172.21.15.39:36276) 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca2059640 1 -- 172.21.15.39:0/1294711567 learned_addr learned my addr 172.21.15.39:0/1294711567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca1858640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c9c0c5790 0x7f6c9c150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca1057640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c9c0c5f40 0x7f6c9c1ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca2059640 1 -- 172.21.15.39:0/1294711567 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c9c0c5f40 msgr2=0x7f6c9c1ec660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca2059640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c9c0c5f40 0x7f6c9c1ec660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca2059640 1 -- 172.21.15.39:0/1294711567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c9c0c5790 msgr2=0x7f6c9c150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca2059640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c9c0c5790 0x7f6c9c150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca2059640 1 -- 172.21.15.39:0/1294711567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c0094d0 con 0x7f6c9c14e410 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca2059640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c14e410 0x7f6c9c1ecba0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c390 tx=0x7f6c9400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6c927fc640 1 -- 172.21.15.39:0/1294711567 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94011790 con 0x7f6c9c14e410 2024-09-13T16:49:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6c927fc640 1 -- 172.21.15.39:0/1294711567 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c940118f0 con 0x7f6c9c14e410 2024-09-13T16:49:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c0a9310 con 0x7f6c9c14e410 2024-09-13T16:49:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6c927fc640 1 -- 172.21.15.39:0/1294711567 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94011ca0 con 0x7f6c9c14e410 2024-09-13T16:49:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.962+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c0bda00 con 0x7f6c9c14e410 2024-09-13T16:49:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.966+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c0bdee0 con 0x7f6c9c14e410 2024-09-13T16:49:44.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.966+0000 7f6c927fc640 1 -- 172.21.15.39:0/1294711567 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c9401c460 con 0x7f6c9c14e410 2024-09-13T16:49:44.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.966+0000 7f6c927fc640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c800666c0 0x7f6c80068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:44.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.966+0000 7f6c927fc640 1 -- 172.21.15.39:0/1294711567 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f6c940840a0 con 0x7f6c9c14e410 2024-09-13T16:49:44.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.966+0000 7f6ca1858640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c800666c0 0x7f6c80068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:44.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.966+0000 7f6ca1858640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c800666c0 0x7f6c80068b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c98003ab0 tx=0x7f6c98009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:44.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:44.970+0000 7f6c927fc640 1 -- 172.21.15.39:0/1294711567 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6c940510a0 con 0x7f6c9c14e410 2024-09-13T16:49:45.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.166+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c9c156cf0 con 0x7f6c9c14e410 2024-09-13T16:49:45.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6c927fc640 1 -- 172.21.15.39:0/1294711567 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+5171 (secure 0 0 0) 0x7f6c940546c0 con 0x7f6c9c14e410 2024-09-13T16:49:45.172 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:45.172 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:44.549928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:45.172 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 15 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c800666c0 msgr2=0x7f6c80068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c800666c0 0x7f6c80068b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c98003ab0 tx=0x7f6c98009830 comp rx=0 tx=0).stop 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c14e410 msgr2=0x7f6c9c1ecba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c14e410 0x7f6c9c1ecba0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400c390 tx=0x7f6c9400c850 comp rx=0 tx=0).stop 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 shutdown_connections 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c800666c0 0x7f6c80068b80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9c14e410 0x7f6c9c1ecba0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c9c0c5f40 0x7f6c9c1ec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 --2- 172.21.15.39:0/1294711567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c9c0c5790 0x7f6c9c150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 >> 172.21.15.39:0/1294711567 conn(0x7f6c9c06c020 msgr2=0x7f6c9c0a82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 shutdown_connections 2024-09-13T16:49:45.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.170+0000 7f6ca3abc640 1 -- 172.21.15.39:0/1294711567 wait complete. 2024-09-13T16:49:45.191 INFO:tasks.mds_thrash.fs.[b]:mds.a down, removed from mdsmap 2024-09-13T16:49:45.191 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T16:49:45.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:45.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f5211dfd640 1 Processor -- start 2024-09-13T16:49:45.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f5211dfd640 1 -- start start 2024-09-13T16:49:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f5211dfd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f520c14aa40 0x7f520c0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f5211dfd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c4710 0x7f520c15b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f5211dfd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520c0c76a0 con 0x7f520c0c1750 2024-09-13T16:49:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f5211dfd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520c151b30 con 0x7f520c14aa40 2024-09-13T16:49:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f5211dfd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c151570 con 0x7f520c0c4710 2024-09-13T16:49:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f520b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f520c14aa40 0x7f520c0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f520bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c4710 0x7f520c15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f520bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c4710 0x7f520c15b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:51104/0 (socket says 172.21.15.39:51104) 2024-09-13T16:49:45.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f520b7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f520c14aa40 0x7f520c0c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:57040/0 (socket says 172.21.15.39:57040) 2024-09-13T16:49:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f520bfff640 1 -- 172.21.15.39:0/2093340115 learned_addr learned my addr 172.21.15.39:0/2093340115 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f520bfff640 1 -- 172.21.15.39:0/2093340115 >> v2:172.21.15.84:3300/0 conn(0x7f520c14aa40 msgr2=0x7f520c0c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f520bfff640 1 --2- 172.21.15.39:0/2093340115 >> v2:172.21.15.84:3300/0 conn(0x7f520c14aa40 0x7f520c0c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f520bfff640 1 -- 172.21.15.39:0/2093340115 >> v1:172.21.15.84:6789/0 conn(0x7f520c0c1750 legacy=0x7f520c0c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.306+0000 7f520bfff640 1 -- 172.21.15.39:0/2093340115 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c151850 con 0x7f520c0c4710 2024-09-13T16:49:45.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f520bfff640 1 --2- 172.21.15.39:0/2093340115 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c4710 0x7f520c15b470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f520400d150 tx=0x7f520401eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5db785dd808e7141 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:45.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f520a7fc640 1 -- 172.21.15.39:0/2093340115 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520400b2d0 con 0x7f520c0c4710 2024-09-13T16:49:45.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f520a7fc640 1 -- 172.21.15.39:0/2093340115 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520400b430 con 0x7f520c0c4710 2024-09-13T16:49:45.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f520a7fc640 1 -- 172.21.15.39:0/2093340115 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52040a7b60 con 0x7f520c0c4710 2024-09-13T16:49:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 -- 172.21.15.39:0/2093340115 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c4710 msgr2=0x7f520c15b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 --2- 172.21.15.39:0/2093340115 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c4710 0x7f520c15b470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f520400d150 tx=0x7f520401eb70 comp rx=0 tx=0).stop 2024-09-13T16:49:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 -- 172.21.15.39:0/2093340115 shutdown_connections 2024-09-13T16:49:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 --2- 172.21.15.39:0/2093340115 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c4710 0x7f520c15b470 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 --2- 172.21.15.39:0/2093340115 >> v2:172.21.15.84:3300/0 conn(0x7f520c14aa40 0x7f520c0c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 -- 172.21.15.39:0/2093340115 >> 172.21.15.39:0/2093340115 conn(0x7f520c0c3310 msgr2=0x7f520c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 -- 172.21.15.39:0/2093340115 shutdown_connections 2024-09-13T16:49:45.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 -- 172.21.15.39:0/2093340115 wait complete. 2024-09-13T16:49:45.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 Processor -- start 2024-09-13T16:49:45.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 -- start start 2024-09-13T16:49:45.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c1750 0x7f520c0bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:45.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f520c0c4710 0x7f520c071510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:45.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f520c14aa40 0x7f520c071a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:45.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c148230 con 0x7f520c14aa40 2024-09-13T16:49:45.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520c148a90 con 0x7f520c0c4710 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f5211dfd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520c14a430 con 0x7f520c0c1750 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f520bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f520c14aa40 0x7f520c071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f520bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f520c14aa40 0x7f520c071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36284/0 (socket says 172.21.15.39:36284) 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f520bfff640 1 -- 172.21.15.39:0/1795337558 learned_addr learned my addr 172.21.15.39:0/1795337558 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f520affd640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f520c0c4710 0x7f520c071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.310+0000 7f520b7fe640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c1750 0x7f520c0bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f520bfff640 1 -- 172.21.15.39:0/1795337558 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c1750 msgr2=0x7f520c0bfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f520bfff640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c1750 0x7f520c0bfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f520bfff640 1 -- 172.21.15.39:0/1795337558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f520c0c4710 msgr2=0x7f520c071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f520bfff640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f520c0c4710 0x7f520c071510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f520bfff640 1 -- 172.21.15.39:0/1795337558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520400bd40 con 0x7f520c14aa40 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f520bfff640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f520c14aa40 0x7f520c071a50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f52040b1040 tx=0x7f520401b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f51f3fff640 1 -- 172.21.15.39:0/1795337558 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204007000 con 0x7f520c14aa40 2024-09-13T16:49:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f51f3fff640 1 -- 172.21.15.39:0/1795337558 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5204009840 con 0x7f520c14aa40 2024-09-13T16:49:45.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520c148440 con 0x7f520c14aa40 2024-09-13T16:49:45.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f51f3fff640 1 -- 172.21.15.39:0/1795337558 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204009bf0 con 0x7f520c14aa40 2024-09-13T16:49:45.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520c0c03e0 con 0x7f520c14aa40 2024-09-13T16:49:45.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520c148c40 con 0x7f520c14aa40 2024-09-13T16:49:45.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f51f3fff640 1 -- 172.21.15.39:0/1795337558 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5204009d50 con 0x7f520c14aa40 2024-09-13T16:49:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f51f3fff640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51f8066690 0x7f51f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f520b7fe640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51f8066690 0x7f51f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f51f3fff640 1 -- 172.21.15.39:0/1795337558 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f520411fa90 con 0x7f520c14aa40 2024-09-13T16:49:45.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.314+0000 7f520b7fe640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51f8066690 0x7f51f8068b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009a50 tx=0x7f51fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:45.323 INFO:tasks.workunit.client.1.smithi084.stderr:Note: switching to '50bbe7f395997dcd871813949c49addbbf630361'. 2024-09-13T16:49:45.323 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-09-13T16:49:45.323 INFO:tasks.workunit.client.1.smithi084.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-13T16:49:45.323 INFO:tasks.workunit.client.1.smithi084.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr:state without impacting any branches by switching back to a branch. 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr: git switch -c 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr:Or undo this operation with: 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr: git switch - 2024-09-13T16:49:45.324 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-09-13T16:49:45.325 INFO:tasks.workunit.client.1.smithi084.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-13T16:49:45.325 INFO:tasks.workunit.client.1.smithi084.stderr: 2024-09-13T16:49:45.325 INFO:tasks.workunit.client.1.smithi084.stderr:HEAD is now at 50bbe7f3959 Merge PR #58789 into wip-rishabh-testing-20240910.095512-reef 2024-09-13T16:49:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.318+0000 7f51f3fff640 1 -- 172.21.15.39:0/1795337558 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f520c148c40 con 0x7f520c14aa40 2024-09-13T16:49:45.331 DEBUG:teuthology.orchestra.run.smithi084:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-09-13T16:49:45.338 INFO:tasks.workunit.client.1.smithi084.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-13T16:49:45.340 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-13T16:49:45.340 INFO:tasks.workunit.client.1.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-13T16:49:45.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.510+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f520c1488f0 con 0x7f520c14aa40 2024-09-13T16:49:45.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.510+0000 7f51f3fff640 1 -- 172.21.15.39:0/1795337558 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+5171 (secure 0 0 0) 0x7f520c1488f0 con 0x7f520c14aa40 2024-09-13T16:49:45.514 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:45.514 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:44.549928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:45.514 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 15 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51f8066690 msgr2=0x7f51f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51f8066690 0x7f51f8068b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009a50 tx=0x7f51fc008040 comp rx=0 tx=0).stop 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f520c14aa40 msgr2=0x7f520c071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f520c14aa40 0x7f520c071a50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f52040b1040 tx=0x7f520401b910 comp rx=0 tx=0).stop 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 shutdown_connections 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51f8066690 0x7f51f8068b50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f520c0c4710 0x7f520c071510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f520c14aa40 0x7f520c071a50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 --2- 172.21.15.39:0/1795337558 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f520c0c1750 0x7f520c0bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:45.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 >> 172.21.15.39:0/1795337558 conn(0x7f520c0c3310 msgr2=0x7f520c0c0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:45.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 shutdown_connections 2024-09-13T16:49:45.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:45.514+0000 7f5211dfd640 1 -- 172.21.15.39:0/1795337558 wait complete. 2024-09-13T16:49:45.532 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 15, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 8, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_4264": { "addr": "172.21.15.84:6833/1392588044", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 1392588044, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 1392588044, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4264, "incarnation": 4, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:48:02.679109+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4264 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 15, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_4232": { "addr": "172.21.15.84:6835/3540656099", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 3540656099, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 3540656099, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4232, "incarnation": 15, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:49:44.549928+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4232 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:49:45.922 INFO:tasks.workunit.client.0.smithi039.stderr:Updating files: 95% (12108/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-13T16:49:46.190 INFO:tasks.workunit.client.1.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-13T16:49:46.272 INFO:tasks.workunit.client.1.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-13T16:49:46.323 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-13T16:49:46.324 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-13T16:49:46.324 INFO:tasks.workunit.client.1.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-13T16:49:46.373 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-13T16:49:46.377 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T16:49:46.377 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-09-13T16:49:46.426 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.1... 2024-09-13T16:49:46.428 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-13T16:49:46.428 DEBUG:teuthology.orchestra.run.smithi084:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=50bbe7f395997dcd871813949c49addbbf630361 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/ffsb.sh 2024-09-13T16:49:46.987 INFO:tasks.workunit.client.0.smithi039.stderr:Note: switching to '50bbe7f395997dcd871813949c49addbbf630361'. 2024-09-13T16:49:46.987 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-13T16:49:46.987 INFO:tasks.workunit.client.0.smithi039.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-13T16:49:46.987 INFO:tasks.workunit.client.0.smithi039.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-13T16:49:46.987 INFO:tasks.workunit.client.0.smithi039.stderr:state without impacting any branches by switching back to a branch. 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr: git switch -c 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr:Or undo this operation with: 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr: git switch - 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-09-13T16:49:46.988 INFO:tasks.workunit.client.0.smithi039.stderr:HEAD is now at 50bbe7f3959 Merge PR #58789 into wip-rishabh-testing-20240910.095512-reef 2024-09-13T16:49:46.996 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-13T16:49:47.001 INFO:tasks.workunit.client.0.smithi039.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-13T16:49:47.004 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T16:49:47.004 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-13T16:49:47.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:47.619 INFO:tasks.workunit.client.1.smithi084.stderr:Cloning into 'ffsb'... 2024-09-13T16:49:47.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.670+0000 7ff6d0a07640 1 Processor -- start 2024-09-13T16:49:47.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.670+0000 7ff6d0a07640 1 -- start start 2024-09-13T16:49:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc14d9b0 0x7ff6cc071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6cc150ce0 0x7ff6cc06c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc0708c0 con 0x7ff6cc0c0a50 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc151190 con 0x7ff6cc150ce0 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc06dcb0 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6c9d9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff6cc0c0a50 0x7ff6cc06fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37128/0 (socket says 172.21.15.39:37128) 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6c9d9b640 1 -- 172.21.15.39:0/338247717 learned_addr learned my addr 172.21.15.39:0/338247717 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6ca59c640 1 --2- 172.21.15.39:0/338247717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc14d9b0 0x7ff6cc071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6cad9d640 1 --2- 172.21.15.39:0/338247717 >> v2:172.21.15.84:3300/0 conn(0x7ff6cc150ce0 0x7ff6cc06c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6c959a640 1 -- 172.21.15.39:0/338247717 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153856624 0 0) 0x7ff6c0001120 con 0x7ff6cc0c0a50 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6c959a640 1 -- 172.21.15.39:0/338247717 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6b4003530 con 0x7ff6cc0c0a50 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6cad9d640 1 -- 172.21.15.39:0/338247717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc14d9b0 msgr2=0x7ff6cc071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6cad9d640 1 --2- 172.21.15.39:0/338247717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc14d9b0 0x7ff6cc071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6cad9d640 1 -- 172.21.15.39:0/338247717 >> v1:172.21.15.39:6789/0 conn(0x7ff6cc0c0a50 legacy=0x7ff6cc06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6cad9d640 1 -- 172.21.15.39:0/338247717 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc06d6f0 con 0x7ff6cc150ce0 2024-09-13T16:49:47.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6ca59c640 1 --2- 172.21.15.39:0/338247717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc14d9b0 0x7ff6cc071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:49:47.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6cad9d640 1 --2- 172.21.15.39:0/338247717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6cc150ce0 0x7ff6cc06c380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009d90 tx=0x7ff6c401c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=335529d86b089a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:47.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6c959a640 1 -- 172.21.15.39:0/338247717 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c40a57e0 con 0x7ff6cc150ce0 2024-09-13T16:49:47.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6c959a640 1 -- 172.21.15.39:0/338247717 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c40a5940 con 0x7ff6cc150ce0 2024-09-13T16:49:47.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6c959a640 1 -- 172.21.15.39:0/338247717 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c40a5cd0 con 0x7ff6cc150ce0 2024-09-13T16:49:47.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 -- 172.21.15.39:0/338247717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6cc150ce0 msgr2=0x7ff6cc06c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 --2- 172.21.15.39:0/338247717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6cc150ce0 0x7ff6cc06c380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009d90 tx=0x7ff6c401c8b0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 -- 172.21.15.39:0/338247717 shutdown_connections 2024-09-13T16:49:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 --2- 172.21.15.39:0/338247717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6cc150ce0 0x7ff6cc06c380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 --2- 172.21.15.39:0/338247717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc14d9b0 0x7ff6cc071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.674+0000 7ff6d0a07640 1 -- 172.21.15.39:0/338247717 >> 172.21.15.39:0/338247717 conn(0x7ff6cc0c6050 msgr2=0x7ff6cc0c6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 -- 172.21.15.39:0/338247717 shutdown_connections 2024-09-13T16:49:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 -- 172.21.15.39:0/338247717 wait complete. 2024-09-13T16:49:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 Processor -- start 2024-09-13T16:49:47.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 -- start start 2024-09-13T16:49:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc0c0a50 0x7ff6cc0ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6cc14d9b0 0x7ff6cc14fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6cc150ce0 0x7ff6cc1f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc15d8e0 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc15de60 con 0x7ff6cc150ce0 2024-09-13T16:49:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc15e2f0 con 0x7ff6cc0c0a50 2024-09-13T16:49:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6c9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6cc14d9b0 0x7ff6cc14fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6c9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6cc14d9b0 0x7ff6cc14fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36294/0 (socket says 172.21.15.39:36294) 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6c9d9b640 1 -- 172.21.15.39:0/4072234946 learned_addr learned my addr 172.21.15.39:0/4072234946 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6ca59c640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc0c0a50 0x7ff6cc0ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6cad9d640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6cc150ce0 0x7ff6cc1f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6c9d9b640 1 -- 172.21.15.39:0/4072234946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc0c0a50 msgr2=0x7ff6cc0ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6c9d9b640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc0c0a50 0x7ff6cc0ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6c9d9b640 1 -- 172.21.15.39:0/4072234946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6cc150ce0 msgr2=0x7ff6cc1f00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6c9d9b640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6cc150ce0 0x7ff6cc1f00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6c9d9b640 1 -- 172.21.15.39:0/4072234946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc009420 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6c9d9b640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6cc14d9b0 0x7ff6cc14fa50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000c400 tx=0x7ff6c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6baffd640 1 -- 172.21.15.39:0/4072234946 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0005cf0 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6baffd640 1 -- 172.21.15.39:0/4072234946 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c0005e50 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c4009a70 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6baffd640 1 -- 172.21.15.39:0/4072234946 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00072f0 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc0aa870 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.678+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc061930 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.682+0000 7ff6baffd640 1 -- 172.21.15.39:0/4072234946 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff6c0007450 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.682+0000 7ff6baffd640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff6ac066690 0x7ff6ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.682+0000 7ff6baffd640 1 -- 172.21.15.39:0/4072234946 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7ff6c00839a0 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.682+0000 7ff6ca59c640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff6ac066690 0x7ff6ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.682+0000 7ff6ca59c640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff6ac066690 0x7ff6ac068b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009cf0 tx=0x7ff6bc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.686+0000 7ff6baffd640 1 -- 172.21.15.39:0/4072234946 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff6c00509c0 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.757 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-13T16:49:47.846 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-13T16:49:47.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6cc0708c0 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6baffd640 1 -- 172.21.15.39:0/4072234946 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5172 (secure 0 0 0) 0x7ff6cc0708c0 con 0x7ff6cc14d9b0 2024-09-13T16:49:47.888 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:47.888 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:47.888 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 18 2024-09-13T16:49:47.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff6ac066690 msgr2=0x7ff6ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:47.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff6ac066690 0x7ff6ac068b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009cf0 tx=0x7ff6bc009190 comp rx=0 tx=0).stop 2024-09-13T16:49:47.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6cc14d9b0 msgr2=0x7ff6cc14fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:47.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6cc14d9b0 0x7ff6cc14fa50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000c400 tx=0x7ff6c000c8c0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 shutdown_connections 2024-09-13T16:49:47.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff6ac066690 0x7ff6ac068b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6cc150ce0 0x7ff6cc1f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6cc14d9b0 0x7ff6cc14fa50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 --2- 172.21.15.39:0/4072234946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff6cc0c0a50 0x7ff6cc0ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:47.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 >> 172.21.15.39:0/4072234946 conn(0x7ff6cc0c6050 msgr2=0x7ff6cc14bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:47.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 shutdown_connections 2024-09-13T16:49:47.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:47.886+0000 7ff6d0a07640 1 -- 172.21.15.39:0/4072234946 wait complete. 2024-09-13T16:49:47.895 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T16:49:47.897 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T16:49:47.897 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-13T16:49:47.908 INFO:tasks.mds_thrash.fs.[b]:mds.d has gained rank=0, replacing gid=4260 2024-09-13T16:49:47.908 INFO:tasks.mds_thrash.fs.[b]:waiting for 4 secs before reviving mds.a 2024-09-13T16:49:47.942 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T16:49:47.945 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T16:49:47.945 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-13T16:49:47.991 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-13T16:49:47.993 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-13T16:49:47.993 DEBUG:teuthology.orchestra.run.smithi039:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=50bbe7f395997dcd871813949c49addbbf630361 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-13T16:49:48.058 INFO:tasks.workunit.client.0.smithi039.stderr:Cloning into 'ffsb'... 2024-09-13T16:49:49.151 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T16:49:49.144+0000 7fce129ab640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:49:49.748 INFO:tasks.workunit.client.1.smithi084.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-13T16:49:50.214 INFO:tasks.workunit.client.0.smithi039.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-13T16:49:50.764 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether build environment is sane... yes 2024-09-13T16:49:50.786 INFO:tasks.workunit.client.1.smithi084.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-13T16:49:50.787 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gawk... gawk 2024-09-13T16:49:50.816 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether make sets $(MAKE)... yes 2024-09-13T16:49:50.869 INFO:tasks.workunit.client.1.smithi084.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-13T16:49:50.869 INFO:tasks.workunit.client.1.smithi084.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-13T16:49:50.891 INFO:tasks.workunit.client.1.smithi084.stdout:checking for style of include used by make... GNU 2024-09-13T16:49:50.896 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gcc... gcc 2024-09-13T16:49:50.993 INFO:tasks.workunit.client.1.smithi084.stdout:checking for C compiler default output file name... a.out 2024-09-13T16:49:50.996 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether the C compiler works... yes 2024-09-13T16:49:50.999 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether we are cross compiling... no 2024-09-13T16:49:51.059 INFO:tasks.workunit.client.1.smithi084.stdout:checking for suffix of executables... 2024-09-13T16:49:51.104 INFO:tasks.workunit.client.1.smithi084.stdout:checking for suffix of object files... o 2024-09-13T16:49:51.144 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether we are using the GNU C compiler... yes 2024-09-13T16:49:51.184 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether gcc accepts -g... yes 2024-09-13T16:49:51.229 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether build environment is sane... yes 2024-09-13T16:49:51.252 INFO:tasks.workunit.client.0.smithi039.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-13T16:49:51.253 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gawk... gawk 2024-09-13T16:49:51.282 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether make sets $(MAKE)... yes 2024-09-13T16:49:51.288 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-13T16:49:51.338 INFO:tasks.workunit.client.0.smithi039.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-13T16:49:51.339 INFO:tasks.workunit.client.0.smithi039.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-13T16:49:51.361 INFO:tasks.workunit.client.0.smithi039.stdout:checking for style of include used by make... GNU 2024-09-13T16:49:51.367 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gcc... gcc 2024-09-13T16:49:51.441 INFO:tasks.workunit.client.1.smithi084.stdout:checking dependency style of gcc... gcc3 2024-09-13T16:49:51.467 INFO:tasks.workunit.client.0.smithi039.stdout:checking for C compiler default output file name... a.out 2024-09-13T16:49:51.470 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether the C compiler works... yes 2024-09-13T16:49:51.474 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether we are cross compiling... no 2024-09-13T16:49:51.509 INFO:tasks.workunit.client.1.smithi084.stdout:checking how to run the C preprocessor... gcc -E 2024-09-13T16:49:51.535 INFO:tasks.workunit.client.0.smithi039.stdout:checking for suffix of executables... 2024-09-13T16:49:51.564 INFO:tasks.workunit.client.1.smithi084.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-13T16:49:51.567 INFO:tasks.workunit.client.1.smithi084.stdout:checking for egrep... /usr/bin/grep -E 2024-09-13T16:49:51.583 INFO:tasks.workunit.client.0.smithi039.stdout:checking for suffix of object files... o 2024-09-13T16:49:51.583 INFO:tasks.workunit.client.1.smithi084.stdout:checking for AIX... no 2024-09-13T16:49:51.612 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gcc... (cached) gcc 2024-09-13T16:49:51.624 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether we are using the GNU C compiler... yes 2024-09-13T16:49:51.625 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-13T16:49:51.626 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-13T16:49:51.627 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-13T16:49:51.627 INFO:tasks.workunit.client.1.smithi084.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-13T16:49:51.666 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether gcc accepts -g... yes 2024-09-13T16:49:51.709 INFO:tasks.workunit.client.1.smithi084.stdout:checking for main in -lm... yes 2024-09-13T16:49:51.756 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-13T16:49:51.783 INFO:tasks.workunit.client.1.smithi084.stdout:checking for main in -lpthread... yes 2024-09-13T16:49:51.909 INFO:tasks.mds_thrash.fs.[b]:reviving mds.a 2024-09-13T16:49:51.909 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:49:51.909 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:49:51.911 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:49:51.912 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:51.926 INFO:tasks.workunit.client.0.smithi039.stdout:checking dependency style of gcc... gcc3 2024-09-13T16:49:51.982 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T16:49:51.984 INFO:tasks.workunit.client.1.smithi084.stdout:checking for ANSI C header files... yes 2024-09-13T16:49:52.019 INFO:tasks.workunit.client.0.smithi039.stdout:checking how to run the C preprocessor... gcc -E 2024-09-13T16:49:52.037 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-13T16:49:52.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb862de4640 1 Processor -- start 2024-09-13T16:49:52.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb862de4640 1 -- start start 2024-09-13T16:49:52.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb862de4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb85c0c7a90 0x7fb85c157840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:52.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb862de4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb85c157d80 0x7fb85c15e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:52.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb862de4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb85c154ed0 con 0x7fb85c151960 2024-09-13T16:49:52.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb862de4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c0c51f0 con 0x7fb85c0c7a90 2024-09-13T16:49:52.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb862de4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c0c7330 con 0x7fb85c157d80 2024-09-13T16:49:52.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb85bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb85c0c7a90 0x7fb85c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:52.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb85bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb85c0c7a90 0x7fb85c157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55404/0 (socket says 172.21.15.39:55404) 2024-09-13T16:49:52.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb85bfff640 1 -- 172.21.15.39:0/2520636099 learned_addr learned my addr 172.21.15.39:0/2520636099 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:52.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb861381640 1 --2- 172.21.15.39:0/2520636099 >> v2:172.21.15.84:3300/0 conn(0x7fb85c157d80 0x7fb85c15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:52.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb85bfff640 1 -- 172.21.15.39:0/2520636099 >> v2:172.21.15.84:3300/0 conn(0x7fb85c157d80 msgr2=0x7fb85c15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:52.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb85bfff640 1 --2- 172.21.15.39:0/2520636099 >> v2:172.21.15.84:3300/0 conn(0x7fb85c157d80 0x7fb85c15e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:52.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb85bfff640 1 -- 172.21.15.39:0/2520636099 >> v1:172.21.15.84:6789/0 conn(0x7fb85c151960 legacy=0x7fb85c0c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:52.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.037+0000 7fb85bfff640 1 -- 172.21.15.39:0/2520636099 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c154350 con 0x7fb85c0c7a90 2024-09-13T16:49:52.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb85bfff640 1 --2- 172.21.15.39:0/2520636099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c0c7a90 0x7fb85c157840 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0c8090 tx=0x7fb84801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c2ec23b72f11aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:52.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb85b7fe640 1 -- 172.21.15.39:0/2520636099 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480ad680 con 0x7fb85c0c7a90 2024-09-13T16:49:52.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb85b7fe640 1 -- 172.21.15.39:0/2520636099 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8480ad7e0 con 0x7fb85c0c7a90 2024-09-13T16:49:52.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb85b7fe640 1 -- 172.21.15.39:0/2520636099 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480adb70 con 0x7fb85c0c7a90 2024-09-13T16:49:52.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb862de4640 1 -- 172.21.15.39:0/2520636099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c0c7a90 msgr2=0x7fb85c157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:52.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb862de4640 1 --2- 172.21.15.39:0/2520636099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c0c7a90 0x7fb85c157840 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0c8090 tx=0x7fb84801c490 comp rx=0 tx=0).stop 2024-09-13T16:49:52.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb862de4640 1 -- 172.21.15.39:0/2520636099 shutdown_connections 2024-09-13T16:49:52.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb862de4640 1 --2- 172.21.15.39:0/2520636099 >> v2:172.21.15.84:3300/0 conn(0x7fb85c157d80 0x7fb85c15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:52.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb862de4640 1 --2- 172.21.15.39:0/2520636099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c0c7a90 0x7fb85c157840 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:52.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb862de4640 1 -- 172.21.15.39:0/2520636099 >> 172.21.15.39:0/2520636099 conn(0x7fb85c0ab500 msgr2=0x7fb85c0ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:52.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb862de4640 1 -- 172.21.15.39:0/2520636099 shutdown_connections 2024-09-13T16:49:52.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.041+0000 7fb862de4640 1 -- 172.21.15.39:0/2520636099 wait complete. 2024-09-13T16:49:52.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 Processor -- start 2024-09-13T16:49:52.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 -- start start 2024-09-13T16:49:52.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb85c0c7a90 0x7fb85c145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:52.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb85c151960 0x7fb85c1ec740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:52.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c157d80 0x7fb85c1ecc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:52.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1f0bd0 con 0x7fb85c157d80 2024-09-13T16:49:52.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1486f0 con 0x7fb85c151960 2024-09-13T16:49:52.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1f0740 con 0x7fb85c0c7a90 2024-09-13T16:49:52.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb861381640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c157d80 0x7fb85c1ecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb85bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb85c151960 0x7fb85c1ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb861381640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c157d80 0x7fb85c1ecc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55406/0 (socket says 172.21.15.39:55406) 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb860b80640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb85c0c7a90 0x7fb85c145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb861381640 1 -- 172.21.15.39:0/3841877579 learned_addr learned my addr 172.21.15.39:0/3841877579 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb861381640 1 -- 172.21.15.39:0/3841877579 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb85c0c7a90 msgr2=0x7fb85c145050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb861381640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb85c0c7a90 0x7fb85c145050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb861381640 1 -- 172.21.15.39:0/3841877579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb85c151960 msgr2=0x7fb85c1ec740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb861381640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb85c151960 0x7fb85c1ec740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb861381640 1 -- 172.21.15.39:0/3841877579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8480094d0 con 0x7fb85c157d80 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb861381640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c157d80 0x7fb85c1ecc80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb854009d30 tx=0x7fb85400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb8597fa640 1 -- 172.21.15.39:0/3841877579 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb854009ba0 con 0x7fb85c157d80 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb8597fa640 1 -- 172.21.15.39:0/3841877579 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85400f040 con 0x7fb85c157d80 2024-09-13T16:49:52.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c1efe50 con 0x7fb85c157d80 2024-09-13T16:49:52.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb8597fa640 1 -- 172.21.15.39:0/3841877579 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8540146c0 con 0x7fb85c157d80 2024-09-13T16:49:52.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c1f03d0 con 0x7fb85c157d80 2024-09-13T16:49:52.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c1f08d0 con 0x7fb85c157d80 2024-09-13T16:49:52.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.045+0000 7fb8597fa640 1 -- 172.21.15.39:0/3841877579 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb854014820 con 0x7fb85c157d80 2024-09-13T16:49:52.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.049+0000 7fb8597fa640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb840066690 0x7fb840068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:52.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.049+0000 7fb8597fa640 1 -- 172.21.15.39:0/3841877579 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fb854085640 con 0x7fb85c157d80 2024-09-13T16:49:52.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.049+0000 7fb860b80640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb840066690 0x7fb840068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:52.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.049+0000 7fb860b80640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb840066690 0x7fb840068b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb850007fd0 tx=0x7fb850009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:52.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.053+0000 7fb8597fa640 1 -- 172.21.15.39:0/3841877579 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb8540527d0 con 0x7fb85c157d80 2024-09-13T16:49:52.080 INFO:tasks.workunit.client.0.smithi039.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-13T16:49:52.082 INFO:tasks.workunit.client.0.smithi039.stdout:checking for egrep... /usr/bin/grep -E 2024-09-13T16:49:52.093 INFO:tasks.workunit.client.0.smithi039.stdout:checking for AIX... no 2024-09-13T16:49:52.101 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/types.h... yes 2024-09-13T16:49:52.119 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gcc... (cached) gcc 2024-09-13T16:49:52.133 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-13T16:49:52.134 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-13T16:49:52.135 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-13T16:49:52.135 INFO:tasks.workunit.client.0.smithi039.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-13T16:49:52.163 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/stat.h... yes 2024-09-13T16:49:52.221 INFO:tasks.workunit.client.0.smithi039.stdout:checking for main in -lm... yes 2024-09-13T16:49:52.224 INFO:tasks.workunit.client.1.smithi084.stdout:checking for stdlib.h... yes 2024-09-13T16:49:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb85c153f00 con 0x7fb85c157d80 2024-09-13T16:49:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb8597fa640 1 -- 172.21.15.39:0/3841877579 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5172 (secure 0 0 0) 0x7fb854055df0 con 0x7fb85c157d80 2024-09-13T16:49:52.273 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:52.273 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 18 2024-09-13T16:49:52.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb840066690 msgr2=0x7fb840068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:52.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb840066690 0x7fb840068b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb850007fd0 tx=0x7fb850009040 comp rx=0 tx=0).stop 2024-09-13T16:49:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c157d80 msgr2=0x7fb85c1ecc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c157d80 0x7fb85c1ecc80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb854009d30 tx=0x7fb85400c490 comp rx=0 tx=0).stop 2024-09-13T16:49:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 shutdown_connections 2024-09-13T16:49:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb840066690 0x7fb840068b50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb85c151960 0x7fb85c1ec740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb85c157d80 0x7fb85c1ecc80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:52.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 --2- 172.21.15.39:0/3841877579 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb85c0c7a90 0x7fb85c145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:52.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 >> 172.21.15.39:0/3841877579 conn(0x7fb85c0ab500 msgr2=0x7fb85c0ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:52.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 shutdown_connections 2024-09-13T16:49:52.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:52.269+0000 7fb862de4640 1 -- 172.21.15.39:0/3841877579 wait complete. 2024-09-13T16:49:52.284 INFO:tasks.workunit.client.1.smithi084.stdout:checking for string.h... yes 2024-09-13T16:49:52.292 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T16:49:52.295 INFO:tasks.workunit.client.0.smithi039.stdout:checking for main in -lpthread... yes 2024-09-13T16:49:52.348 INFO:tasks.workunit.client.1.smithi084.stdout:checking for memory.h... yes 2024-09-13T16:49:52.417 INFO:tasks.workunit.client.1.smithi084.stdout:checking for strings.h... yes 2024-09-13T16:49:52.481 INFO:tasks.workunit.client.1.smithi084.stdout:checking for inttypes.h... yes 2024-09-13T16:49:52.511 INFO:tasks.workunit.client.0.smithi039.stdout:checking for ANSI C header files... yes 2024-09-13T16:49:52.545 INFO:tasks.workunit.client.1.smithi084.stdout:checking for stdint.h... yes 2024-09-13T16:49:52.565 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-13T16:49:52.619 INFO:tasks.workunit.client.1.smithi084.stdout:checking for unistd.h... yes 2024-09-13T16:49:52.624 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/types.h... yes 2024-09-13T16:49:52.686 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/stat.h... yes 2024-09-13T16:49:52.701 INFO:tasks.workunit.client.1.smithi084.stdout:checking pthread.h usability... yes 2024-09-13T16:49:52.729 INFO:tasks.workunit.client.1.smithi084.stdout:checking pthread.h presence... yes 2024-09-13T16:49:52.730 INFO:tasks.workunit.client.1.smithi084.stdout:checking for pthread.h... yes 2024-09-13T16:49:52.746 INFO:tasks.workunit.client.0.smithi039.stdout:checking for stdlib.h... yes 2024-09-13T16:49:52.804 INFO:tasks.workunit.client.1.smithi084.stdout:checking fcntl.h usability... yes 2024-09-13T16:49:52.814 INFO:tasks.workunit.client.0.smithi039.stdout:checking for string.h... yes 2024-09-13T16:49:52.831 INFO:tasks.workunit.client.1.smithi084.stdout:checking fcntl.h presence... yes 2024-09-13T16:49:52.832 INFO:tasks.workunit.client.1.smithi084.stdout:checking for fcntl.h... yes 2024-09-13T16:49:52.877 INFO:tasks.workunit.client.0.smithi039.stdout:checking for memory.h... yes 2024-09-13T16:49:52.904 INFO:tasks.workunit.client.1.smithi084.stdout:checking limits.h usability... yes 2024-09-13T16:49:52.928 INFO:tasks.workunit.client.1.smithi084.stdout:checking limits.h presence... yes 2024-09-13T16:49:52.928 INFO:tasks.workunit.client.1.smithi084.stdout:checking for limits.h... yes 2024-09-13T16:49:52.942 INFO:tasks.workunit.client.0.smithi039.stdout:checking for strings.h... yes 2024-09-13T16:49:52.953 INFO:tasks.workunit.client.1.smithi084.stdout:checking for stdint.h... (cached) yes 2024-09-13T16:49:53.005 INFO:tasks.workunit.client.0.smithi039.stdout:checking for inttypes.h... yes 2024-09-13T16:49:53.020 INFO:tasks.workunit.client.1.smithi084.stdout:checking sys/time.h usability... yes 2024-09-13T16:49:53.046 INFO:tasks.workunit.client.1.smithi084.stdout:checking sys/time.h presence... yes 2024-09-13T16:49:53.046 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/time.h... yes 2024-09-13T16:49:53.070 INFO:tasks.workunit.client.0.smithi039.stdout:checking for stdint.h... yes 2024-09-13T16:49:53.075 INFO:tasks.workunit.client.1.smithi084.stdout:checking for unistd.h... (cached) yes 2024-09-13T16:49:53.142 INFO:tasks.workunit.client.1.smithi084.stdout:checking sys/vfs.h usability... yes 2024-09-13T16:49:53.150 INFO:tasks.workunit.client.0.smithi039.stdout:checking for unistd.h... yes 2024-09-13T16:49:53.169 INFO:tasks.workunit.client.1.smithi084.stdout:checking sys/vfs.h presence... yes 2024-09-13T16:49:53.169 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/vfs.h... yes 2024-09-13T16:49:53.247 INFO:tasks.workunit.client.0.smithi039.stdout:checking pthread.h usability... yes 2024-09-13T16:49:53.254 INFO:tasks.workunit.client.1.smithi084.stdout:checking sys/limits.h usability... no 2024-09-13T16:49:53.275 INFO:tasks.workunit.client.0.smithi039.stdout:checking pthread.h presence... yes 2024-09-13T16:49:53.276 INFO:tasks.workunit.client.0.smithi039.stdout:checking for pthread.h... yes 2024-09-13T16:49:53.281 INFO:tasks.workunit.client.1.smithi084.stdout:checking sys/limits.h presence... no 2024-09-13T16:49:53.282 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/limits.h... no 2024-09-13T16:49:53.322 INFO:tasks.workunit.client.1.smithi084.stdout:checking for an ANSI C-conforming const... yes 2024-09-13T16:49:53.344 INFO:tasks.workunit.client.0.smithi039.stdout:checking fcntl.h usability... yes 2024-09-13T16:49:53.363 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-13T16:49:53.365 INFO:tasks.workunit.client.1.smithi084.stdout:checking for function prototypes... yes 2024-09-13T16:49:53.370 INFO:tasks.workunit.client.0.smithi039.stdout:checking fcntl.h presence... yes 2024-09-13T16:49:53.371 INFO:tasks.workunit.client.0.smithi039.stdout:checking for fcntl.h... yes 2024-09-13T16:49:53.408 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether setvbuf arguments are reversed... no 2024-09-13T16:49:53.441 INFO:tasks.workunit.client.0.smithi039.stdout:checking limits.h usability... yes 2024-09-13T16:49:53.465 INFO:tasks.workunit.client.0.smithi039.stdout:checking limits.h presence... yes 2024-09-13T16:49:53.466 INFO:tasks.workunit.client.0.smithi039.stdout:checking for limits.h... yes 2024-09-13T16:49:53.486 INFO:tasks.workunit.client.0.smithi039.stdout:checking for stdint.h... (cached) yes 2024-09-13T16:49:53.509 INFO:tasks.workunit.client.1.smithi084.stdout:checking for system... yes 2024-09-13T16:49:53.545 INFO:tasks.workunit.client.0.smithi039.stdout:checking sys/time.h usability... yes 2024-09-13T16:49:53.569 INFO:tasks.workunit.client.0.smithi039.stdout:checking sys/time.h presence... yes 2024-09-13T16:49:53.570 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/time.h... yes 2024-09-13T16:49:53.599 INFO:tasks.workunit.client.0.smithi039.stdout:checking for unistd.h... (cached) yes 2024-09-13T16:49:53.608 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gettimeofday... yes 2024-09-13T16:49:53.665 INFO:tasks.workunit.client.0.smithi039.stdout:checking sys/vfs.h usability... yes 2024-09-13T16:49:53.690 INFO:tasks.workunit.client.0.smithi039.stdout:checking sys/vfs.h presence... yes 2024-09-13T16:49:53.690 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/vfs.h... yes 2024-09-13T16:49:53.695 INFO:tasks.workunit.client.1.smithi084.stdout:checking for mkdir... yes 2024-09-13T16:49:53.767 INFO:tasks.workunit.client.0.smithi039.stdout:checking sys/limits.h usability... no 2024-09-13T16:49:53.783 INFO:tasks.workunit.client.1.smithi084.stdout:checking for strerror... yes 2024-09-13T16:49:53.795 INFO:tasks.workunit.client.0.smithi039.stdout:checking sys/limits.h presence... no 2024-09-13T16:49:53.796 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/limits.h... no 2024-09-13T16:49:53.837 INFO:tasks.workunit.client.0.smithi039.stdout:checking for an ANSI C-conforming const... yes 2024-09-13T16:49:53.866 INFO:tasks.workunit.client.1.smithi084.stdout:checking for open64... yes 2024-09-13T16:49:53.878 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-13T16:49:53.881 INFO:tasks.workunit.client.0.smithi039.stdout:checking for function prototypes... yes 2024-09-13T16:49:53.934 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether setvbuf arguments are reversed... no 2024-09-13T16:49:53.954 INFO:tasks.workunit.client.1.smithi084.stdout:checking for stat64... yes 2024-09-13T16:49:54.014 INFO:tasks.workunit.client.0.smithi039.stdout:checking for system... yes 2024-09-13T16:49:54.042 INFO:tasks.workunit.client.1.smithi084.stdout:checking for fseeko64... yes 2024-09-13T16:49:54.104 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gettimeofday... yes 2024-09-13T16:49:54.132 INFO:tasks.workunit.client.1.smithi084.stdout:checking for lrand48_r... yes 2024-09-13T16:49:54.192 INFO:tasks.workunit.client.0.smithi039.stdout:checking for mkdir... yes 2024-09-13T16:49:54.219 INFO:tasks.workunit.client.1.smithi084.stdout:checking for srand48_r... yes 2024-09-13T16:49:54.251 INFO:tasks.workunit.client.1.smithi084.stdout:configure: creating ./config.status 2024-09-13T16:49:54.280 INFO:tasks.workunit.client.0.smithi039.stdout:checking for strerror... yes 2024-09-13T16:49:54.292 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:54.372 INFO:tasks.workunit.client.0.smithi039.stdout:checking for open64... yes 2024-09-13T16:49:54.383 INFO:tasks.workunit.client.1.smithi084.stdout:config.status: creating Makefile 2024-09-13T16:49:54.402 INFO:tasks.workunit.client.1.smithi084.stdout:config.status: creating config.h 2024-09-13T16:49:54.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5224174640 1 Processor -- start 2024-09-13T16:49:54.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5224174640 1 -- start start 2024-09-13T16:49:54.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5224174640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f521c14acf0 0x7f521c151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5224174640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c0c4ba0 0x7f521c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5224174640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521c154ab0 con 0x7f521c0c4710 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5224174640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c154d90 con 0x7f521c14acf0 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5224174640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c150d50 con 0x7f521c0c4ba0 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5221f10640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f521c14acf0 0x7f521c151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f522170f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f521c0c4710 0x7f521c06ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37190/0 (socket says 172.21.15.39:37190) 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5221f10640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f521c14acf0 0x7f521c151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55408/0 (socket says 172.21.15.39:55408) 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f522170f640 1 -- 172.21.15.39:0/346750427 learned_addr learned my addr 172.21.15.39:0/346750427 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5222711640 1 --2- 172.21.15.39:0/346750427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c0c4ba0 0x7f521c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5221f10640 1 -- 172.21.15.39:0/346750427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c0c4ba0 msgr2=0x7f521c153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5221f10640 1 --2- 172.21.15.39:0/346750427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c0c4ba0 0x7f521c153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5221f10640 1 -- 172.21.15.39:0/346750427 >> v1:172.21.15.39:6789/0 conn(0x7f521c0c4710 legacy=0x7f521c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:54.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5220f0e640 1 -- 172.21.15.39:0/346750427 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258991280 0 0) 0x7f521c154ab0 con 0x7f521c0c4710 2024-09-13T16:49:54.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.405+0000 7f5221f10640 1 -- 172.21.15.39:0/346750427 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c151030 con 0x7f521c14acf0 2024-09-13T16:49:54.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5221f10640 1 --2- 172.21.15.39:0/346750427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c14acf0 0x7f521c151550 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f521800ba90 tx=0x7f521801ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743c19615a5dfa28 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:54.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5220f0e640 1 -- 172.21.15.39:0/346750427 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52180a7a70 con 0x7f521c14acf0 2024-09-13T16:49:54.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5220f0e640 1 -- 172.21.15.39:0/346750427 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52180a7bd0 con 0x7f521c14acf0 2024-09-13T16:49:54.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5220f0e640 1 -- 172.21.15.39:0/346750427 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52180af440 con 0x7f521c14acf0 2024-09-13T16:49:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 -- 172.21.15.39:0/346750427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c14acf0 msgr2=0x7f521c151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 --2- 172.21.15.39:0/346750427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c14acf0 0x7f521c151550 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f521800ba90 tx=0x7f521801ea80 comp rx=0 tx=0).stop 2024-09-13T16:49:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 -- 172.21.15.39:0/346750427 shutdown_connections 2024-09-13T16:49:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 --2- 172.21.15.39:0/346750427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c0c4ba0 0x7f521c153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 --2- 172.21.15.39:0/346750427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c14acf0 0x7f521c151550 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 -- 172.21.15.39:0/346750427 >> 172.21.15.39:0/346750427 conn(0x7f521c0c32a0 msgr2=0x7f521c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 -- 172.21.15.39:0/346750427 shutdown_connections 2024-09-13T16:49:54.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 -- 172.21.15.39:0/346750427 wait complete. 2024-09-13T16:49:54.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 Processor -- start 2024-09-13T16:49:54.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 -- start start 2024-09-13T16:49:54.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f521c0c4710 0x7f521c1efee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:54.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c0c4ba0 0x7f521c0a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c14acf0 0x7f521c1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0abf80 con 0x7f521c0c4ba0 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0ac490 con 0x7f521c0c4710 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5224174640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0ac1b0 con 0x7f521c14acf0 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f522170f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c0c4ba0 0x7f521c0a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f522170f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c0c4ba0 0x7f521c0a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55410/0 (socket says 172.21.15.39:55410) 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f522170f640 1 -- 172.21.15.39:0/1247620174 learned_addr learned my addr 172.21.15.39:0/1247620174 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5221f10640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f521c0c4710 0x7f521c1efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f5222711640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c14acf0 0x7f521c1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f522170f640 1 -- 172.21.15.39:0/1247620174 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c14acf0 msgr2=0x7f521c1459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f522170f640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c14acf0 0x7f521c1459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f522170f640 1 -- 172.21.15.39:0/1247620174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f521c0c4710 msgr2=0x7f521c1efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:54.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f522170f640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f521c0c4710 0x7f521c1efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f522170f640 1 -- 172.21.15.39:0/1247620174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521800b6c0 con 0x7f521c0c4ba0 2024-09-13T16:49:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.409+0000 7f522170f640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c0c4ba0 0x7f521c0a9e50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f520c0011f0 tx=0x7f520c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f52127fc640 1 -- 172.21.15.39:0/1247620174 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c0074e0 con 0x7f521c0c4ba0 2024-09-13T16:49:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f52127fc640 1 -- 172.21.15.39:0/1247620174 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c007640 con 0x7f521c0c4ba0 2024-09-13T16:49:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c1f07c0 con 0x7f521c0c4ba0 2024-09-13T16:49:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f52127fc640 1 -- 172.21.15.39:0/1247620174 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c004a10 con 0x7f521c0c4ba0 2024-09-13T16:49:54.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c0abaf0 con 0x7f521c0c4ba0 2024-09-13T16:49:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c154ab0 con 0x7f521c0c4ba0 2024-09-13T16:49:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f52127fc640 1 -- 172.21.15.39:0/1247620174 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f520c004b70 con 0x7f521c0c4ba0 2024-09-13T16:49:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f52127fc640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5200066690 0x7f5200068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f52127fc640 1 -- 172.21.15.39:0/1247620174 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f520c0839c0 con 0x7f521c0c4ba0 2024-09-13T16:49:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f5221f10640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5200066690 0x7f5200068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.413+0000 7f5221f10640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5200066690 0x7f5200068b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f52180b1000 tx=0x7f5218008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.417+0000 7f52127fc640 1 -- 172.21.15.39:0/1247620174 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f520c050a70 con 0x7f521c0c4ba0 2024-09-13T16:49:54.426 INFO:tasks.workunit.client.1.smithi084.stdout:config.status: executing depfiles commands 2024-09-13T16:49:54.464 INFO:tasks.workunit.client.0.smithi039.stdout:checking for stat64... yes 2024-09-13T16:49:54.498 INFO:tasks.workunit.client.1.smithi084.stdout:config.status: executing default commands 2024-09-13T16:49:54.555 INFO:tasks.workunit.client.0.smithi039.stdout:checking for fseeko64... yes 2024-09-13T16:49:54.602 INFO:tasks.workunit.client.1.smithi084.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run aclocal-1.10 2024-09-13T16:49:54.611 INFO:tasks.workunit.client.1.smithi084.stderr:/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-13T16:49:54.612 INFO:tasks.workunit.client.1.smithi084.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-13T16:49:54.612 INFO:tasks.workunit.client.1.smithi084.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-13T16:49:54.612 INFO:tasks.workunit.client.1.smithi084.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-13T16:49:54.612 INFO:tasks.workunit.client.1.smithi084.stderr: any GNU archive site. 2024-09-13T16:49:54.615 INFO:tasks.workunit.client.1.smithi084.stdout: cd . && /bin/bash /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-13T16:49:54.619 INFO:tasks.workunit.client.1.smithi084.stderr:/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-13T16:49:54.620 INFO:tasks.workunit.client.1.smithi084.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-13T16:49:54.620 INFO:tasks.workunit.client.1.smithi084.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-13T16:49:54.621 INFO:tasks.workunit.client.1.smithi084.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-13T16:49:54.621 INFO:tasks.workunit.client.1.smithi084.stderr: Grab them from any GNU archive site. 2024-09-13T16:49:54.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.637+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f521c0a5ac0 con 0x7f521c0c4ba0 2024-09-13T16:49:54.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.637+0000 7f52127fc640 1 -- 172.21.15.39:0/1247620174 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f520c018090 con 0x7f521c0c4ba0 2024-09-13T16:49:54.642 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:54.642 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:54.643 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 19 2024-09-13T16:49:54.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5200066690 msgr2=0x7f5200068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:54.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5200066690 0x7f5200068b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f52180b1000 tx=0x7f5218008040 comp rx=0 tx=0).stop 2024-09-13T16:49:54.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c0c4ba0 msgr2=0x7f521c0a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:54.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c0c4ba0 0x7f521c0a9e50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f520c0011f0 tx=0x7f520c00c9b0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 shutdown_connections 2024-09-13T16:49:54.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5200066690 0x7f5200068b50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f521c14acf0 0x7f521c1459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f521c0c4ba0 0x7f521c0a9e50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 --2- 172.21.15.39:0/1247620174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f521c0c4710 0x7f521c1efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:54.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 >> 172.21.15.39:0/1247620174 conn(0x7f521c0c32a0 msgr2=0x7f521c0c05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:54.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 shutdown_connections 2024-09-13T16:49:54.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:54.641+0000 7f5224174640 1 -- 172.21.15.39:0/1247620174 wait complete. 2024-09-13T16:49:54.648 INFO:tasks.workunit.client.0.smithi039.stdout:checking for lrand48_r... yes 2024-09-13T16:49:54.661 INFO:tasks.mds_thrash.fs.[b]:mds.a reported in up:standby state 2024-09-13T16:49:54.662 INFO:tasks.mds_thrash.fs.[b]:waiting for 5 secs before thrashing 2024-09-13T16:49:54.730 INFO:tasks.workunit.client.0.smithi039.stdout:checking for srand48_r... yes 2024-09-13T16:49:54.765 INFO:tasks.workunit.client.0.smithi039.stdout:configure: creating ./config.status 2024-09-13T16:49:54.787 INFO:tasks.workunit.client.1.smithi084.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run autoconf 2024-09-13T16:49:54.888 INFO:tasks.workunit.client.0.smithi039.stdout:config.status: creating Makefile 2024-09-13T16:49:54.906 INFO:tasks.workunit.client.0.smithi039.stdout:config.status: creating config.h 2024-09-13T16:49:54.930 INFO:tasks.workunit.client.0.smithi039.stdout:config.status: executing depfiles commands 2024-09-13T16:49:54.934 INFO:tasks.workunit.client.1.smithi084.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-13T16:49:54.934 INFO:tasks.workunit.client.1.smithi084.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-13T16:49:54.934 INFO:tasks.workunit.client.1.smithi084.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-13T16:49:54.934 INFO:tasks.workunit.client.1.smithi084.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-13T16:49:55.022 INFO:tasks.workunit.client.0.smithi039.stdout:config.status: executing default commands 2024-09-13T16:49:55.056 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:4: warning: The macro `AC_FOREACH' is obsolete. 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:4: You should run autoupdate. 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:./lib/autoconf/general.m4:191: AC_FOREACH is expanded from... 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:aclocal.m4:684: _AM_SET_OPTIONS is expanded from... 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:aclocal.m4:410: AM_INIT_AUTOMAKE is expanded from... 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:4: the top level 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:6: warning: The macro `AM_CONFIG_HEADER' is obsolete. 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:6: You should run autoupdate. 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:aclocal.m4:383: AM_CONFIG_HEADER is expanded from... 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:6: the top level 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:10: warning: The macro `AC_AIX' is obsolete. 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:10: You should run autoupdate. 2024-09-13T16:49:55.057 INFO:tasks.workunit.client.1.smithi084.stderr:./lib/autoconf/specific.m4:540: AC_AIX is expanded from... 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:10: the top level 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:36: warning: The macro `AC_HEADER_STDC' is obsolete. 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:36: You should run autoupdate. 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:./lib/autoconf/headers.m4:704: AC_HEADER_STDC is expanded from... 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:36: the top level 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:42: warning: The macro `AC_HEADER_TIME' is obsolete. 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:42: You should run autoupdate. 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:./lib/autoconf/headers.m4:743: AC_HEADER_TIME is expanded from... 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:42: the top level 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:45: warning: The macro `AC_FUNC_SETVBUF_REVERSED' is obsolete. Remove it and all references to SETVBUF_REVERSED. 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:./lib/autoconf/functions.m4:1766: AC_FUNC_SETVBUF_REVERSED is expanded from... 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:45: the top level 2024-09-13T16:49:55.058 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:50: warning: AC_OUTPUT should be used without arguments. 2024-09-13T16:49:55.059 INFO:tasks.workunit.client.1.smithi084.stderr:configure.in:50: You should run autoupdate. 2024-09-13T16:49:55.127 INFO:tasks.workunit.client.0.smithi039.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-13T16:49:55.135 INFO:tasks.workunit.client.0.smithi039.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-13T16:49:55.137 INFO:tasks.workunit.client.0.smithi039.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-13T16:49:55.137 INFO:tasks.workunit.client.0.smithi039.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-13T16:49:55.137 INFO:tasks.workunit.client.0.smithi039.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-13T16:49:55.137 INFO:tasks.workunit.client.0.smithi039.stderr: any GNU archive site. 2024-09-13T16:49:55.148 INFO:tasks.workunit.client.0.smithi039.stdout: cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-13T16:49:55.152 INFO:tasks.workunit.client.0.smithi039.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-13T16:49:55.153 INFO:tasks.workunit.client.0.smithi039.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-13T16:49:55.153 INFO:tasks.workunit.client.0.smithi039.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-13T16:49:55.153 INFO:tasks.workunit.client.0.smithi039.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-13T16:49:55.154 INFO:tasks.workunit.client.0.smithi039.stderr: Grab them from any GNU archive site. 2024-09-13T16:49:55.156 INFO:tasks.workunit.client.1.smithi084.stdout:/bin/bash ./config.status --recheck 2024-09-13T16:49:55.218 INFO:tasks.workunit.client.1.smithi084.stdout:running CONFIG_SHELL=/bin/bash /bin/bash ./configure --no-create --no-recursion 2024-09-13T16:49:55.332 INFO:tasks.workunit.client.0.smithi039.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-13T16:49:55.356 INFO:tasks.workunit.client.1.smithi084.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-13T16:49:55.487 INFO:tasks.workunit.client.0.smithi039.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-13T16:49:55.487 INFO:tasks.workunit.client.0.smithi039.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-13T16:49:55.488 INFO:tasks.workunit.client.0.smithi039.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-13T16:49:55.488 INFO:tasks.workunit.client.0.smithi039.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-13T16:49:55.585 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:49:55.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:49:55.607 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:4: warning: The macro `AC_FOREACH' is obsolete. 2024-09-13T16:49:55.608 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:4: You should run autoupdate. 2024-09-13T16:49:55.608 INFO:tasks.workunit.client.0.smithi039.stderr:./lib/autoconf/general.m4:191: AC_FOREACH is expanded from... 2024-09-13T16:49:55.608 INFO:tasks.workunit.client.0.smithi039.stderr:aclocal.m4:684: _AM_SET_OPTIONS is expanded from... 2024-09-13T16:49:55.608 INFO:tasks.workunit.client.0.smithi039.stderr:aclocal.m4:410: AM_INIT_AUTOMAKE is expanded from... 2024-09-13T16:49:55.608 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:4: the top level 2024-09-13T16:49:55.608 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:6: warning: The macro `AM_CONFIG_HEADER' is obsolete. 2024-09-13T16:49:55.608 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:6: You should run autoupdate. 2024-09-13T16:49:55.608 INFO:tasks.workunit.client.0.smithi039.stderr:aclocal.m4:383: AM_CONFIG_HEADER is expanded from... 2024-09-13T16:49:55.608 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:6: the top level 2024-09-13T16:49:55.609 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:10: warning: The macro `AC_AIX' is obsolete. 2024-09-13T16:49:55.609 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:10: You should run autoupdate. 2024-09-13T16:49:55.609 INFO:tasks.workunit.client.0.smithi039.stderr:./lib/autoconf/specific.m4:540: AC_AIX is expanded from... 2024-09-13T16:49:55.609 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:10: the top level 2024-09-13T16:49:55.609 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:36: warning: The macro `AC_HEADER_STDC' is obsolete. 2024-09-13T16:49:55.609 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:36: You should run autoupdate. 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:./lib/autoconf/headers.m4:704: AC_HEADER_STDC is expanded from... 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:36: the top level 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:42: warning: The macro `AC_HEADER_TIME' is obsolete. 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:42: You should run autoupdate. 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:./lib/autoconf/headers.m4:743: AC_HEADER_TIME is expanded from... 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:42: the top level 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:45: warning: The macro `AC_FUNC_SETVBUF_REVERSED' is obsolete. Remove it and all references to SETVBUF_REVERSED. 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:./lib/autoconf/functions.m4:1766: AC_FUNC_SETVBUF_REVERSED is expanded from... 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:45: the top level 2024-09-13T16:49:55.610 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:50: warning: AC_OUTPUT should be used without arguments. 2024-09-13T16:49:55.611 INFO:tasks.workunit.client.0.smithi039.stderr:configure.in:50: You should run autoupdate. 2024-09-13T16:49:55.687 INFO:tasks.workunit.client.0.smithi039.stdout:/bin/bash ./config.status --recheck 2024-09-13T16:49:55.771 INFO:tasks.workunit.client.0.smithi039.stdout:running CONFIG_SHELL=/bin/bash /bin/bash ./configure --no-create --no-recursion 2024-09-13T16:49:55.967 INFO:tasks.workunit.client.0.smithi039.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-13T16:49:56.370 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether build environment is sane... yes 2024-09-13T16:49:56.384 INFO:tasks.workunit.client.1.smithi084.stdout:checking for a race-free mkdir -p... /usr/bin/mkdir -p 2024-09-13T16:49:56.386 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gawk... gawk 2024-09-13T16:49:56.417 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether make sets $(MAKE)... yes 2024-09-13T16:49:56.467 INFO:tasks.workunit.client.1.smithi084.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-13T16:49:56.467 INFO:tasks.workunit.client.1.smithi084.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-13T16:49:56.493 INFO:tasks.workunit.client.1.smithi084.stdout:checking for style of include used by make... GNU 2024-09-13T16:49:56.496 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gcc... gcc 2024-09-13T16:49:56.641 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether the C compiler works... yes 2024-09-13T16:49:56.642 INFO:tasks.workunit.client.1.smithi084.stdout:checking for C compiler default output file name... a.out 2024-09-13T16:49:56.699 INFO:tasks.workunit.client.1.smithi084.stdout:checking for suffix of executables... 2024-09-13T16:49:56.759 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether we are cross compiling... no 2024-09-13T16:49:56.809 INFO:tasks.workunit.client.1.smithi084.stdout:checking for suffix of object files... o 2024-09-13T16:49:56.840 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether the compiler supports GNU C... yes 2024-09-13T16:49:56.870 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether gcc accepts -g... yes 2024-09-13T16:49:56.941 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gcc option to enable C11 features... none needed 2024-09-13T16:49:56.980 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether build environment is sane... yes 2024-09-13T16:49:56.992 INFO:tasks.workunit.client.0.smithi039.stdout:checking for a race-free mkdir -p... /usr/bin/mkdir -p 2024-09-13T16:49:56.993 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gawk... gawk 2024-09-13T16:49:57.020 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether make sets $(MAKE)... yes 2024-09-13T16:49:57.073 INFO:tasks.workunit.client.0.smithi039.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-13T16:49:57.073 INFO:tasks.workunit.client.0.smithi039.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-13T16:49:57.092 INFO:tasks.workunit.client.1.smithi084.stdout:checking dependency style of gcc... gcc3 2024-09-13T16:49:57.095 INFO:tasks.workunit.client.0.smithi039.stdout:checking for style of include used by make... GNU 2024-09-13T16:49:57.100 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gcc... gcc 2024-09-13T16:49:57.125 INFO:tasks.workunit.client.1.smithi084.stdout:checking for stdio.h... yes 2024-09-13T16:49:57.171 INFO:tasks.workunit.client.1.smithi084.stdout:checking for stdlib.h... yes 2024-09-13T16:49:57.212 INFO:tasks.workunit.client.1.smithi084.stdout:checking for string.h... yes 2024-09-13T16:49:57.248 INFO:tasks.workunit.client.1.smithi084.stdout:checking for inttypes.h... yes 2024-09-13T16:49:57.259 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether the C compiler works... yes 2024-09-13T16:49:57.259 INFO:tasks.workunit.client.0.smithi039.stdout:checking for C compiler default output file name... a.out 2024-09-13T16:49:57.287 INFO:tasks.workunit.client.1.smithi084.stdout:checking for stdint.h... yes 2024-09-13T16:49:57.324 INFO:tasks.workunit.client.0.smithi039.stdout:checking for suffix of executables... 2024-09-13T16:49:57.324 INFO:tasks.workunit.client.1.smithi084.stdout:checking for strings.h... yes 2024-09-13T16:49:57.362 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/stat.h... yes 2024-09-13T16:49:57.385 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether we are cross compiling... no 2024-09-13T16:49:57.402 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/types.h... yes 2024-09-13T16:49:57.433 INFO:tasks.workunit.client.0.smithi039.stdout:checking for suffix of object files... o 2024-09-13T16:49:57.446 INFO:tasks.workunit.client.1.smithi084.stdout:checking for unistd.h... yes 2024-09-13T16:49:57.466 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether the compiler supports GNU C... yes 2024-09-13T16:49:57.497 INFO:tasks.workunit.client.1.smithi084.stdout:checking for wchar.h... yes 2024-09-13T16:49:57.498 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether gcc accepts -g... yes 2024-09-13T16:49:57.551 INFO:tasks.workunit.client.1.smithi084.stdout:checking for minix/config.h... no 2024-09-13T16:49:57.567 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gcc option to enable C11 features... none needed 2024-09-13T16:49:57.595 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/time.h... yes 2024-09-13T16:49:57.648 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-13T16:49:57.685 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether _XOPEN_SOURCE should be defined... no 2024-09-13T16:49:57.703 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gcc... (cached) gcc 2024-09-13T16:49:57.726 INFO:tasks.workunit.client.0.smithi039.stdout:checking dependency style of gcc... gcc3 2024-09-13T16:49:57.760 INFO:tasks.workunit.client.0.smithi039.stdout:checking for stdio.h... yes 2024-09-13T16:49:57.766 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether the compiler supports GNU C... (cached) yes 2024-09-13T16:49:57.767 INFO:tasks.workunit.client.1.smithi084.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-13T16:49:57.768 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gcc option to enable C11 features... (cached) none needed 2024-09-13T16:49:57.768 INFO:tasks.workunit.client.1.smithi084.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-13T16:49:57.803 INFO:tasks.workunit.client.0.smithi039.stdout:checking for stdlib.h... yes 2024-09-13T16:49:57.826 INFO:tasks.workunit.client.1.smithi084.stdout:checking for main in -lm... yes 2024-09-13T16:49:57.844 INFO:tasks.workunit.client.0.smithi039.stdout:checking for string.h... yes 2024-09-13T16:49:57.885 INFO:tasks.workunit.client.0.smithi039.stdout:checking for inttypes.h... yes 2024-09-13T16:49:57.892 INFO:tasks.workunit.client.1.smithi084.stdout:checking for main in -lpthread... yes 2024-09-13T16:49:57.899 INFO:tasks.workunit.client.1.smithi084.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-13T16:49:57.900 INFO:tasks.workunit.client.1.smithi084.stdout:checking for egrep... /usr/bin/grep -E 2024-09-13T16:49:57.923 INFO:tasks.workunit.client.0.smithi039.stdout:checking for stdint.h... yes 2024-09-13T16:49:57.942 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-13T16:49:57.960 INFO:tasks.workunit.client.0.smithi039.stdout:checking for strings.h... yes 2024-09-13T16:49:57.990 INFO:tasks.workunit.client.1.smithi084.stdout:checking for pthread.h... yes 2024-09-13T16:49:58.000 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/stat.h... yes 2024-09-13T16:49:58.034 INFO:tasks.workunit.client.1.smithi084.stdout:checking for fcntl.h... yes 2024-09-13T16:49:58.040 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/types.h... yes 2024-09-13T16:49:58.085 INFO:tasks.workunit.client.0.smithi039.stdout:checking for unistd.h... yes 2024-09-13T16:49:58.086 INFO:tasks.workunit.client.1.smithi084.stdout:checking for limits.h... yes 2024-09-13T16:49:58.088 INFO:tasks.workunit.client.1.smithi084.stdout:checking for stdint.h... (cached) yes 2024-09-13T16:49:58.089 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/time.h... (cached) yes 2024-09-13T16:49:58.094 INFO:tasks.workunit.client.1.smithi084.stdout:checking for unistd.h... (cached) yes 2024-09-13T16:49:58.137 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/vfs.h... yes 2024-09-13T16:49:58.138 INFO:tasks.workunit.client.0.smithi039.stdout:checking for wchar.h... yes 2024-09-13T16:49:58.192 INFO:tasks.workunit.client.0.smithi039.stdout:checking for minix/config.h... no 2024-09-13T16:49:58.193 INFO:tasks.workunit.client.1.smithi084.stdout:checking for sys/limits.h... no 2024-09-13T16:49:58.226 INFO:tasks.workunit.client.1.smithi084.stdout:checking for an ANSI C-conforming const... yes 2024-09-13T16:49:58.239 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/time.h... yes 2024-09-13T16:49:58.291 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-13T16:49:58.296 INFO:tasks.workunit.client.1.smithi084.stdout:checking for system... yes 2024-09-13T16:49:58.326 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether _XOPEN_SOURCE should be defined... no 2024-09-13T16:49:58.344 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gcc... (cached) gcc 2024-09-13T16:49:58.364 INFO:tasks.workunit.client.1.smithi084.stdout:checking for gettimeofday... yes 2024-09-13T16:49:58.405 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether the compiler supports GNU C... (cached) yes 2024-09-13T16:49:58.406 INFO:tasks.workunit.client.0.smithi039.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-13T16:49:58.406 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gcc option to enable C11 features... (cached) none needed 2024-09-13T16:49:58.407 INFO:tasks.workunit.client.0.smithi039.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-13T16:49:58.430 INFO:tasks.workunit.client.1.smithi084.stdout:checking for mkdir... yes 2024-09-13T16:49:58.467 INFO:tasks.workunit.client.0.smithi039.stdout:checking for main in -lm... yes 2024-09-13T16:49:58.496 INFO:tasks.workunit.client.1.smithi084.stdout:checking for strerror... yes 2024-09-13T16:49:58.527 INFO:tasks.workunit.client.0.smithi039.stdout:checking for main in -lpthread... yes 2024-09-13T16:49:58.533 INFO:tasks.workunit.client.0.smithi039.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-13T16:49:58.534 INFO:tasks.workunit.client.0.smithi039.stdout:checking for egrep... /usr/bin/grep -E 2024-09-13T16:49:58.560 INFO:tasks.workunit.client.1.smithi084.stdout:checking for open64... yes 2024-09-13T16:49:58.576 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-13T16:49:58.627 INFO:tasks.workunit.client.1.smithi084.stdout:checking for stat64... yes 2024-09-13T16:49:58.627 INFO:tasks.workunit.client.0.smithi039.stdout:checking for pthread.h... yes 2024-09-13T16:49:58.676 INFO:tasks.workunit.client.0.smithi039.stdout:checking for fcntl.h... yes 2024-09-13T16:49:58.697 INFO:tasks.workunit.client.1.smithi084.stdout:checking for fseeko64... yes 2024-09-13T16:49:58.724 INFO:tasks.workunit.client.0.smithi039.stdout:checking for limits.h... yes 2024-09-13T16:49:58.726 INFO:tasks.workunit.client.0.smithi039.stdout:checking for stdint.h... (cached) yes 2024-09-13T16:49:58.727 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/time.h... (cached) yes 2024-09-13T16:49:58.731 INFO:tasks.workunit.client.0.smithi039.stdout:checking for unistd.h... (cached) yes 2024-09-13T16:49:58.760 INFO:tasks.workunit.client.1.smithi084.stdout:checking for lrand48_r... yes 2024-09-13T16:49:58.776 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/vfs.h... yes 2024-09-13T16:49:58.828 INFO:tasks.workunit.client.1.smithi084.stdout:checking for srand48_r... yes 2024-09-13T16:49:58.831 INFO:tasks.workunit.client.0.smithi039.stdout:checking for sys/limits.h... no 2024-09-13T16:49:58.847 INFO:tasks.workunit.client.1.smithi084.stdout:configure: creating ./config.status 2024-09-13T16:49:58.863 INFO:tasks.workunit.client.0.smithi039.stdout:checking for an ANSI C-conforming const... yes 2024-09-13T16:49:58.933 INFO:tasks.workunit.client.0.smithi039.stdout:checking for system... yes 2024-09-13T16:49:58.958 INFO:tasks.workunit.client.1.smithi084.stdout: /bin/bash ./config.status 2024-09-13T16:49:58.998 INFO:tasks.workunit.client.0.smithi039.stdout:checking for gettimeofday... yes 2024-09-13T16:49:59.016 INFO:tasks.workunit.client.1.smithi084.stdout:config.status: creating Makefile 2024-09-13T16:49:59.032 INFO:tasks.workunit.client.1.smithi084.stdout:config.status: creating config.h 2024-09-13T16:49:59.060 INFO:tasks.workunit.client.1.smithi084.stdout:config.status: executing depfiles commands 2024-09-13T16:49:59.064 INFO:tasks.workunit.client.0.smithi039.stdout:checking for mkdir... yes 2024-09-13T16:49:59.082 INFO:tasks.workunit.client.1.smithi084.stdout:config.status: executing default commands 2024-09-13T16:49:59.131 INFO:tasks.workunit.client.0.smithi039.stdout:checking for strerror... yes 2024-09-13T16:49:59.135 INFO:tasks.workunit.client.1.smithi084.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb/missing --run autoheader 2024-09-13T16:49:59.191 INFO:tasks.workunit.client.1.smithi084.stderr:autoheader: warning: autoconf input should be named 'configure.ac', not 'configure.in' 2024-09-13T16:49:59.199 INFO:tasks.workunit.client.0.smithi039.stdout:checking for open64... yes 2024-09-13T16:49:59.262 INFO:tasks.workunit.client.0.smithi039.stdout:checking for stat64... yes 2024-09-13T16:49:59.267 INFO:tasks.workunit.client.1.smithi084.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-13T16:49:59.267 INFO:tasks.workunit.client.1.smithi084.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-13T16:49:59.268 INFO:tasks.workunit.client.1.smithi084.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-13T16:49:59.268 INFO:tasks.workunit.client.1.smithi084.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-13T16:49:59.331 INFO:tasks.workunit.client.0.smithi039.stdout:checking for fseeko64... yes 2024-09-13T16:49:59.396 INFO:tasks.workunit.client.1.smithi084.stdout:rm -f stamp-h1 2024-09-13T16:49:59.400 INFO:tasks.workunit.client.0.smithi039.stdout:checking for lrand48_r... yes 2024-09-13T16:49:59.401 INFO:tasks.workunit.client.1.smithi084.stdout:touch config.h.in 2024-09-13T16:49:59.408 INFO:tasks.workunit.client.1.smithi084.stdout:cd . && /bin/bash ./config.status config.h 2024-09-13T16:49:59.458 INFO:tasks.workunit.client.1.smithi084.stdout:config.status: creating config.h 2024-09-13T16:49:59.463 INFO:tasks.workunit.client.0.smithi039.stdout:checking for srand48_r... yes 2024-09-13T16:49:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:configure: creating ./config.status 2024-09-13T16:49:59.497 INFO:tasks.workunit.client.1.smithi084.stdout:make all-am 2024-09-13T16:49:59.528 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb' 2024-09-13T16:49:59.528 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-13T16:49:59.586 INFO:tasks.workunit.client.0.smithi039.stdout: /bin/bash ./config.status 2024-09-13T16:49:59.651 INFO:tasks.workunit.client.0.smithi039.stdout:config.status: creating Makefile 2024-09-13T16:49:59.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:49:59.666 INFO:tasks.workunit.client.0.smithi039.stdout:config.status: creating config.h 2024-09-13T16:49:59.702 INFO:tasks.workunit.client.0.smithi039.stdout:config.status: executing depfiles commands 2024-09-13T16:49:59.720 INFO:tasks.workunit.client.0.smithi039.stdout:config.status: executing default commands 2024-09-13T16:49:59.743 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-13T16:49:59.752 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-13T16:49:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f9352825640 1 Processor -- start 2024-09-13T16:49:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f9352825640 1 -- start start 2024-09-13T16:49:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f9352825640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c065ee0 0x7f934c0662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f9352825640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f934c0c2ee0 0x7f934c0c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f9352825640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934c06ddf0 con 0x7f934c0c6270 2024-09-13T16:49:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f9352825640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934c0c4710 con 0x7f934c0c2ee0 2024-09-13T16:49:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f9352825640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c0c49d0 con 0x7f934c065ee0 2024-09-13T16:49:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f934bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c065ee0 0x7f934c0662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f934bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c065ee0 0x7f934c0662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:57064/0 (socket says 172.21.15.39:57064) 2024-09-13T16:49:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f934bfff640 1 -- 172.21.15.39:0/2528538791 learned_addr learned my addr 172.21.15.39:0/2528538791 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:59.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f934b7fe640 1 --2- 172.21.15.39:0/2528538791 >> v2:172.21.15.84:3300/0 conn(0x7f934c0c2ee0 0x7f934c0c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:59.782 INFO:tasks.workunit.client.0.smithi039.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-13T16:49:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f934bfff640 1 -- 172.21.15.39:0/2528538791 >> v2:172.21.15.84:3300/0 conn(0x7f934c0c2ee0 msgr2=0x7f934c0c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:49:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f934bfff640 1 --2- 172.21.15.39:0/2528538791 >> v2:172.21.15.84:3300/0 conn(0x7f934c0c2ee0 0x7f934c0c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f934bfff640 1 -- 172.21.15.39:0/2528538791 >> v1:172.21.15.84:6789/0 conn(0x7f934c0c6270 legacy=0x7f934c0bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f934bfff640 1 -- 172.21.15.39:0/2528538791 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c154880 con 0x7f934c065ee0 2024-09-13T16:49:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.777+0000 7f934b7fe640 1 --2- 172.21.15.39:0/2528538791 >> v2:172.21.15.84:3300/0 conn(0x7f934c0c2ee0 0x7f934c0c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:49:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934bfff640 1 --2- 172.21.15.39:0/2528538791 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c065ee0 0x7f934c0662c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f933c009870 tx=0x7f933c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e072c48e9bc5750e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934affd640 1 -- 172.21.15.39:0/2528538791 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c0a7070 con 0x7f934c065ee0 2024-09-13T16:49:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934affd640 1 -- 172.21.15.39:0/2528538791 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c0ab430 con 0x7f934c065ee0 2024-09-13T16:49:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934affd640 1 -- 172.21.15.39:0/2528538791 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c0ab7c0 con 0x7f934c065ee0 2024-09-13T16:49:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 -- 172.21.15.39:0/2528538791 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c065ee0 msgr2=0x7f934c0662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 --2- 172.21.15.39:0/2528538791 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c065ee0 0x7f934c0662c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f933c009870 tx=0x7f933c01c4d0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 -- 172.21.15.39:0/2528538791 shutdown_connections 2024-09-13T16:49:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 --2- 172.21.15.39:0/2528538791 >> v2:172.21.15.84:3300/0 conn(0x7f934c0c2ee0 0x7f934c0c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 --2- 172.21.15.39:0/2528538791 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c065ee0 0x7f934c0662c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 -- 172.21.15.39:0/2528538791 >> 172.21.15.39:0/2528538791 conn(0x7f934c0c3310 msgr2=0x7f934c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 -- 172.21.15.39:0/2528538791 shutdown_connections 2024-09-13T16:49:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 -- 172.21.15.39:0/2528538791 wait complete. 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 Processor -- start 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 -- start start 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f934c065ee0 0x7f934c0aa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0c2ee0 0x7f934c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c0c6270 0x7f934c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1ed2e0 con 0x7f934c065ee0 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1ed5c0 con 0x7f934c0c2ee0 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9352825640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c156c80 con 0x7f934c0c6270 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f934c065ee0 0x7f934c0aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f934c065ee0 0x7f934c0aa400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55416/0 (socket says 172.21.15.39:55416) 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934bfff640 1 -- 172.21.15.39:0/1265681376 learned_addr learned my addr 172.21.15.39:0/1265681376 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:49:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934b7fe640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0c2ee0 0x7f934c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f9350dc2640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c0c6270 0x7f934c157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934bfff640 1 -- 172.21.15.39:0/1265681376 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c0c6270 msgr2=0x7f934c157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934bfff640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c0c6270 0x7f934c157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934bfff640 1 -- 172.21.15.39:0/1265681376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0c2ee0 msgr2=0x7f934c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.781+0000 7f934bfff640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0c2ee0 0x7f934c155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f934bfff640 1 -- 172.21.15.39:0/1265681376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340009420 con 0x7f934c065ee0 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f934b7fe640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0c2ee0 0x7f934c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f934bfff640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f934c065ee0 0x7f934c0aa400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f933c0ad040 tx=0x7f933c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f9348ff9640 1 -- 172.21.15.39:0/1265681376 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c0ba070 con 0x7f934c065ee0 2024-09-13T16:49:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f9348ff9640 1 -- 172.21.15.39:0/1265681376 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c0b4420 con 0x7f934c065ee0 2024-09-13T16:49:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f9348ff9640 1 -- 172.21.15.39:0/1265681376 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c0b47d0 con 0x7f934c065ee0 2024-09-13T16:49:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c0094d0 con 0x7f934c065ee0 2024-09-13T16:49:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c157240 con 0x7f934c065ee0 2024-09-13T16:49:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c06b540 con 0x7f934c065ee0 2024-09-13T16:49:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f9348ff9640 1 -- 172.21.15.39:0/1265681376 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f933c0b4930 con 0x7f934c065ee0 2024-09-13T16:49:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f9348ff9640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93380666e0 0x7f9338068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:49:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f9348ff9640 1 -- 172.21.15.39:0/1265681376 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f933c11dc30 con 0x7f934c065ee0 2024-09-13T16:49:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f934b7fe640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93380666e0 0x7f9338068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:49:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.785+0000 7f934b7fe640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93380666e0 0x7f9338068ba0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f934c152d00 tx=0x7f9340009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:49:59.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.789+0000 7f9348ff9640 1 -- 172.21.15.39:0/1265681376 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f933c0eaee0 con 0x7f934c065ee0 2024-09-13T16:49:59.843 INFO:tasks.workunit.client.0.smithi039.stderr:autoheader: warning: autoconf input should be named 'configure.ac', not 'configure.in' 2024-09-13T16:49:59.858 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-13T16:49:59.865 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-13T16:49:59.920 INFO:tasks.workunit.client.0.smithi039.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-13T16:49:59.920 INFO:tasks.workunit.client.0.smithi039.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-13T16:49:59.920 INFO:tasks.workunit.client.0.smithi039.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-13T16:49:59.920 INFO:tasks.workunit.client.0.smithi039.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-13T16:49:59.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.981+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f934c0662c0 con 0x7f934c065ee0 2024-09-13T16:49:59.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.981+0000 7f9348ff9640 1 -- 172.21.15.39:0/1265681376 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f933c0ee500 con 0x7f934c065ee0 2024-09-13T16:49:59.985 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:49:59.986 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:49:59.986 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 19 2024-09-13T16:49:59.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93380666e0 msgr2=0x7f9338068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93380666e0 0x7f9338068ba0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f934c152d00 tx=0x7f9340009190 comp rx=0 tx=0).stop 2024-09-13T16:49:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f934c065ee0 msgr2=0x7f934c0aa400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:49:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f934c065ee0 0x7f934c0aa400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f933c0ad040 tx=0x7f933c019270 comp rx=0 tx=0).stop 2024-09-13T16:49:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 shutdown_connections 2024-09-13T16:49:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93380666e0 0x7f9338068ba0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f934c0c2ee0 0x7f934c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f934c0c6270 0x7f934c157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 --2- 172.21.15.39:0/1265681376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f934c065ee0 0x7f934c0aa400 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:49:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 >> 172.21.15.39:0/1265681376 conn(0x7f934c0c3310 msgr2=0x7f934c153710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:49:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 shutdown_connections 2024-09-13T16:49:59.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:49:59.985+0000 7f9352825640 1 -- 172.21.15.39:0/1265681376 wait complete. 2024-09-13T16:49:59.991 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-13T16:50:00.001 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-13T16:50:00.003 INFO:tasks.mds_thrash.fs.[b]:kill mds.d (rank=0) 2024-09-13T16:50:00.003 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:50:00.004 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:50:00.021 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:50:00.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:00.046 INFO:tasks.workunit.client.0.smithi039.stdout:rm -f stamp-h1 2024-09-13T16:50:00.050 INFO:tasks.workunit.client.0.smithi039.stdout:touch config.h.in 2024-09-13T16:50:00.057 INFO:tasks.workunit.client.0.smithi039.stdout:cd . && /bin/bash ./config.status config.h 2024-09-13T16:50:00.114 INFO:tasks.workunit.client.0.smithi039.stdout:config.status: creating config.h 2024-09-13T16:50:00.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c294d640 1 Processor -- start 2024-09-13T16:50:00.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c294d640 1 -- start start 2024-09-13T16:50:00.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c294d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 0x7fd0bc06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:00.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c294d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd0bc1576e0 0x7fd0bc157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:00.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c294d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0bc0c15d0 con 0x7fd0bc0c1810 2024-09-13T16:50:00.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c294d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0bc06d0b0 con 0x7fd0bc1576e0 2024-09-13T16:50:00.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c294d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0c1f00 con 0x7fd0bc06b080 2024-09-13T16:50:00.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c0eea640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd0bc1576e0 0x7fd0bc157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:00.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c0eea640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd0bc1576e0 0x7fd0bc157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:34220/0 (socket says 172.21.15.39:34220) 2024-09-13T16:50:00.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c0eea640 1 -- 172.21.15.39:0/1879400119 learned_addr learned my addr 172.21.15.39:0/1879400119 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:00.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0b3fff640 1 --1- 172.21.15.39:0/1879400119 >> v1:172.21.15.84:6789/0 conn(0x7fd0bc0c1810 0x7fd0bc151390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:60174/0 (socket says 172.21.15.39:60174) 2024-09-13T16:50:00.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0bbfff640 1 --2- 172.21.15.39:0/1879400119 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 0x7fd0bc06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:00.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c0eea640 1 -- 172.21.15.39:0/1879400119 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 msgr2=0x7fd0bc06e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c0eea640 1 --2- 172.21.15.39:0/1879400119 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 0x7fd0bc06e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c0eea640 1 -- 172.21.15.39:0/1879400119 >> v1:172.21.15.84:6789/0 conn(0x7fd0bc0c1810 legacy=0x7fd0bc151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c0eea640 1 -- 172.21.15.39:0/1879400119 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc0c21a0 con 0x7fd0bc1576e0 2024-09-13T16:50:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0bb7fe640 1 -- 172.21.15.39:0/1879400119 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650943766 0 0) 0x7fd0bc0c15d0 con 0x7fd0bc0c1810 2024-09-13T16:50:00.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.153+0000 7fd0c0eea640 1 --2- 172.21.15.39:0/1879400119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0bc1576e0 0x7fd0bc157af0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009e50 tx=0x7fd0b401ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bf5612ee5a9e563 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bb7fe640 1 -- 172.21.15.39:0/1879400119 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40a57e0 con 0x7fd0bc1576e0 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bb7fe640 1 -- 172.21.15.39:0/1879400119 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b40a5940 con 0x7fd0bc1576e0 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bb7fe640 1 -- 172.21.15.39:0/1879400119 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40a5cd0 con 0x7fd0bc1576e0 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- 172.21.15.39:0/1879400119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0bc1576e0 msgr2=0x7fd0bc157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 --2- 172.21.15.39:0/1879400119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0bc1576e0 0x7fd0bc157af0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009e50 tx=0x7fd0b401ca00 comp rx=0 tx=0).stop 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- 172.21.15.39:0/1879400119 shutdown_connections 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 --2- 172.21.15.39:0/1879400119 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0bc1576e0 0x7fd0bc157af0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 --2- 172.21.15.39:0/1879400119 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 0x7fd0bc06e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- 172.21.15.39:0/1879400119 >> 172.21.15.39:0/1879400119 conn(0x7fd0bc0c32a0 msgr2=0x7fd0bc0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- 172.21.15.39:0/1879400119 shutdown_connections 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- 172.21.15.39:0/1879400119 wait complete. 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 Processor -- start 2024-09-13T16:50:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- start start 2024-09-13T16:50:00.164 INFO:tasks.workunit.client.0.smithi039.stdout:make all-am 2024-09-13T16:50:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 0x7fd0bc1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0bc0c1810 0x7fd0bc152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0bc1576e0 0x7fd0bc153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc1573e0 con 0x7fd0bc0c1810 2024-09-13T16:50:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc156ed0 con 0x7fd0bc1576e0 2024-09-13T16:50:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc157100 con 0x7fd0bc06b080 2024-09-13T16:50:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0bc0c1810 0x7fd0bc152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0bc0c1810 0x7fd0bc152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54208/0 (socket says 172.21.15.39:54208) 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0b3fff640 1 -- 172.21.15.39:0/1920549544 learned_addr learned my addr 172.21.15.39:0/1920549544 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bbfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 0x7fd0bc1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c0eea640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0bc1576e0 0x7fd0bc153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bbfff640 1 -- 172.21.15.39:0/1920549544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0bc1576e0 msgr2=0x7fd0bc153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bbfff640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0bc1576e0 0x7fd0bc153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bbfff640 1 -- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0bc0c1810 msgr2=0x7fd0bc152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bbfff640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0bc0c1810 0x7fd0bc152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bbfff640 1 -- 172.21.15.39:0/1920549544 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4009b30 con 0x7fd0bc06b080 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0b3fff640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0bc0c1810 0x7fd0bc152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0bbfff640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 0x7fd0bc1550d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00e8a0 tx=0x7fd0ac00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0b97fa640 1 -- 172.21.15.39:0/1920549544 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00c940 con 0x7fd0bc06b080 2024-09-13T16:50:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0b97fa640 1 -- 172.21.15.39:0/1920549544 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac00caa0 con 0x7fd0bc06b080 2024-09-13T16:50:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc1ecdb0 con 0x7fd0bc06b080 2024-09-13T16:50:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.157+0000 7fd0b97fa640 1 -- 172.21.15.39:0/1920549544 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00ce10 con 0x7fd0bc06b080 2024-09-13T16:50:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.161+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc0a5fb0 con 0x7fd0bc06b080 2024-09-13T16:50:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.161+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc0619c0 con 0x7fd0bc06b080 2024-09-13T16:50:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.161+0000 7fd0b97fa640 1 -- 172.21.15.39:0/1920549544 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0ac013470 con 0x7fd0bc06b080 2024-09-13T16:50:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.161+0000 7fd0b97fa640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd09c066690 0x7fd09c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.161+0000 7fd0b97fa640 1 -- 172.21.15.39:0/1920549544 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fd0ac084640 con 0x7fd0bc06b080 2024-09-13T16:50:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.161+0000 7fd0b3fff640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd09c066690 0x7fd09c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.161+0000 7fd0b3fff640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd09c066690 0x7fd09c068b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc066b90 tx=0x7fd0a4007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.165+0000 7fd0b97fa640 1 -- 172.21.15.39:0/1920549544 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd0ac0516d0 con 0x7fd0bc06b080 2024-09-13T16:50:00.195 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-13T16:50:00.195 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-13T16:50:00.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-13T16:50:00.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0bc1ecc10 con 0x7fd0bc06b080 2024-09-13T16:50:00.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0b97fa640 1 -- 172.21.15.39:0/1920549544 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5929 (secure 0 0 0) 0x7fd0bc1ecc10 con 0x7fd0bc06b080 2024-09-13T16:50:00.369 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:00.369 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:00.370 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 19 2024-09-13T16:50:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd09c066690 msgr2=0x7fd09c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd09c066690 0x7fd09c068b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc066b90 tx=0x7fd0a4007df0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 msgr2=0x7fd0bc1550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 0x7fd0bc1550d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00e8a0 tx=0x7fd0ac00ed60 comp rx=0 tx=0).stop 2024-09-13T16:50:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 shutdown_connections 2024-09-13T16:50:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd09c066690 0x7fd09c068b50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0bc1576e0 0x7fd0bc153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0bc0c1810 0x7fd0bc152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 --2- 172.21.15.39:0/1920549544 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd0bc06b080 0x7fd0bc1550d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:00.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 >> 172.21.15.39:0/1920549544 conn(0x7fd0bc0c32a0 msgr2=0x7fd0bc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:00.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 shutdown_connections 2024-09-13T16:50:00.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:00.365+0000 7fd0c294d640 1 -- 172.21.15.39:0/1920549544 wait complete. 2024-09-13T16:50:00.387 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:50:00.394 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-13T16:50:00.404 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-13T16:50:00.531 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-13T16:50:00.542 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-13T16:50:00.708 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-13T16:50:00.721 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-13T16:50:00.885 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-13T16:50:00.901 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-13T16:50:01.116 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-13T16:50:01.134 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-13T16:50:01.236 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-13T16:50:01.247 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-13T16:50:01.318 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-13T16:50:01.328 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-13T16:50:01.414 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-13T16:50:01.424 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-13T16:50:01.679 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-13T16:50:01.691 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-13T16:50:01.871 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-13T16:50:01.883 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-13T16:50:02.116 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-13T16:50:02.131 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-13T16:50:02.268 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-13T16:50:02.282 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-13T16:50:02.355 INFO:tasks.workunit.client.0.smithi039.stderr:ffsb_op.c: In function 'print_op_results': 2024-09-13T16:50:02.355 INFO:tasks.workunit.client.0.smithi039.stderr:ffsb_op.c:83:37: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-13T16:50:02.355 INFO:tasks.workunit.client.0.smithi039.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-13T16:50:02.356 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~ 2024-09-13T16:50:02.356 INFO:tasks.workunit.client.0.smithi039.stderr:ffsb_op.c:86:33: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-13T16:50:02.356 INFO:tasks.workunit.client.0.smithi039.stderr: 86 | sprintf(buf, "NA\0"); 2024-09-13T16:50:02.356 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~ 2024-09-13T16:50:02.371 INFO:tasks.workunit.client.0.smithi039.stderr:ffsb_op.c: In function 'print_results': 2024-09-13T16:50:02.371 INFO:tasks.workunit.client.0.smithi039.stderr:ffsb_op.c:83:33: warning: '/sec' directive writing 4 bytes into a region of size between 1 and 256 [-Wformat-overflow=] 2024-09-13T16:50:02.371 INFO:tasks.workunit.client.0.smithi039.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-13T16:50:02.371 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~ 2024-09-13T16:50:02.372 INFO:tasks.workunit.client.0.smithi039.stderr:In file included from /usr/include/stdio.h:894, 2024-09-13T16:50:02.372 INFO:tasks.workunit.client.0.smithi039.stderr: from ffsb_op.c:20: 2024-09-13T16:50:02.372 INFO:tasks.workunit.client.0.smithi039.stderr:/usr/include/x86_64-linux-gnu/bits/stdio2.h:38:10: note: '__builtin___sprintf_chk' output between 5 and 260 bytes into a destination of size 256 2024-09-13T16:50:02.372 INFO:tasks.workunit.client.0.smithi039.stderr: 38 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, 2024-09-13T16:50:02.372 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.372 INFO:tasks.workunit.client.0.smithi039.stderr: 39 | __glibc_objsize (__s), __fmt, 2024-09-13T16:50:02.372 INFO:tasks.workunit.client.0.smithi039.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.372 INFO:tasks.workunit.client.0.smithi039.stderr: 40 | __va_arg_pack ()); 2024-09-13T16:50:02.372 INFO:tasks.workunit.client.0.smithi039.stderr: | ~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.389 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:02.421 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-13T16:50:02.437 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-13T16:50:02.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d2e4d640 1 Processor -- start 2024-09-13T16:50:02.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d2e4d640 1 -- start start 2024-09-13T16:50:02.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d2e4d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc14f1b0 0x7f46cc151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d2e4d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc14f590 con 0x7f46cc0c7470 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d2e4d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc06eae0 con 0x7f46cc06e5e0 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d2e4d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc06f660 con 0x7f46cc14f1b0 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d13ea640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f46cc0c7470 0x7f46cc0c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34720/0 (socket says 172.21.15.39:34720) 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d13ea640 1 -- 172.21.15.39:0/1339119489 learned_addr learned my addr 172.21.15.39:0/1339119489 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d0be9640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc14f1b0 0x7f46cc151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896723815 0 0) 0x7f46cc14f590 con 0x7f46cc0c7470 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46b8003530 con 0x7f46cc0c7470 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1521160572 0 0) 0x7f46b8003530 con 0x7f46cc0c7470 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc14f1b0 msgr2=0x7f46cc151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc14f1b0 0x7f46cc151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 >> v1:172.21.15.84:6789/0 conn(0x7f46cc06e5e0 legacy=0x7f46cc0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc06edc0 con 0x7f46cc0c7470 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f46bc001a00 con 0x7f46cc0c7470 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 >> v1:172.21.15.39:6789/0 conn(0x7f46cc0c7470 legacy=0x7f46cc0c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc14f1b0 0x7f46b80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b8003f30 0x7f46b80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46b8004d10 0x7f46b8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc06f660 con 0x7f46b8004d10 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b800ba70 con 0x7f46b8003f30 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b800bc30 con 0x7f46cc14f1b0 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d13ea640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46b8004d10 0x7f46b8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cbfff640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b8003f30 0x7f46b80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:02.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d0be9640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc14f1b0 0x7f46b80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:02.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d13ea640 1 -- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc14f1b0 msgr2=0x7f46b80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:02.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d13ea640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc14f1b0 0x7f46b80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:02.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d13ea640 1 -- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b8003f30 msgr2=0x7f46b80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:02.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d13ea640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b8003f30 0x7f46b80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:02.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d13ea640 1 -- 172.21.15.39:0/1339119489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8003530 con 0x7f46b8004d10 2024-09-13T16:50:02.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46d0be9640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc14f1b0 0x7f46b80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:02.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.525+0000 7f46cbfff640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46b8003f30 0x7f46b80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:02.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46d13ea640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46b8004d10 0x7f46b8005190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00a830 tx=0x7f46bc01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d603a6a7bd4671 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:02.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0072d0 con 0x7f46b8004d10 2024-09-13T16:50:02.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc0a8070 con 0x7f46b8004d10 2024-09-13T16:50:02.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46cb7fe640 1 -- 172.21.15.39:0/1339119489 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0076f0 con 0x7f46b8004d10 2024-09-13T16:50:02.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46d2e4d640 1 -- 172.21.15.39:0/1339119489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46b8004d10 msgr2=0x7f46b8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:02.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46d2e4d640 1 --2- 172.21.15.39:0/1339119489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46b8004d10 0x7f46b8005190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00a830 tx=0x7f46bc01d360 comp rx=0 tx=0).stop 2024-09-13T16:50:02.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46d13ea640 1 -- 172.21.15.39:0/1339119489 reap_dead start 2024-09-13T16:50:02.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46d2e4d640 1 -- 172.21.15.39:0/1339119489 shutdown_connections 2024-09-13T16:50:02.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46d2e4d640 1 -- 172.21.15.39:0/1339119489 >> 172.21.15.39:0/1339119489 conn(0x7f46cc0a9720 msgr2=0x7f46cc0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:02.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46d2e4d640 1 -- 172.21.15.39:0/1339119489 shutdown_connections 2024-09-13T16:50:02.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.529+0000 7f46d2e4d640 1 -- 172.21.15.39:0/1339119489 wait complete. 2024-09-13T16:50:02.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 Processor -- start 2024-09-13T16:50:02.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 -- start start 2024-09-13T16:50:02.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc0c32c0 0x7f46cc0c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:02.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc147910 0x7f46cc1e87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:02.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46cc1e79f0 0x7f46cc1e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:02.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc0aaf20 con 0x7f46cc147910 2024-09-13T16:50:02.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1e9130 con 0x7f46cc1e79f0 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc1e72c0 con 0x7f46cc0c32c0 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc147910 0x7f46cc1e87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc147910 0x7f46cc1e87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54222/0 (socket says 172.21.15.39:54222) 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46cbfff640 1 -- 172.21.15.39:0/2858847364 learned_addr learned my addr 172.21.15.39:0/2858847364 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d0be9640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc0c32c0 0x7f46cc0c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d13ea640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46cc1e79f0 0x7f46cc1e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46cbfff640 1 -- 172.21.15.39:0/2858847364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc0c32c0 msgr2=0x7f46cc0c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46cbfff640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc0c32c0 0x7f46cc0c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46cbfff640 1 -- 172.21.15.39:0/2858847364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46cc1e79f0 msgr2=0x7f46cc1e7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46cbfff640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46cc1e79f0 0x7f46cc1e7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46cbfff640 1 -- 172.21.15.39:0/2858847364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc1e7580 con 0x7f46cc147910 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46cbfff640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc147910 0x7f46cc1e87b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f46b400e480 tx=0x7f46b400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46c97fa640 1 -- 172.21.15.39:0/2858847364 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40052e0 con 0x7f46cc147910 2024-09-13T16:50:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46c97fa640 1 -- 172.21.15.39:0/2858847364 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b4005440 con 0x7f46cc147910 2024-09-13T16:50:02.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc1460f0 con 0x7f46cc147910 2024-09-13T16:50:02.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46c97fa640 1 -- 172.21.15.39:0/2858847364 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4003800 con 0x7f46cc147910 2024-09-13T16:50:02.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc1466b0 con 0x7f46cc147910 2024-09-13T16:50:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc14e9e0 con 0x7f46cc147910 2024-09-13T16:50:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.533+0000 7f46c97fa640 1 -- 172.21.15.39:0/2858847364 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f46b4003960 con 0x7f46cc147910 2024-09-13T16:50:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.537+0000 7f46c97fa640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46ac0666c0 0x7f46ac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.537+0000 7f46c97fa640 1 -- 172.21.15.39:0/2858847364 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f46b4084240 con 0x7f46cc147910 2024-09-13T16:50:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.537+0000 7f46d0be9640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46ac0666c0 0x7f46ac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.537+0000 7f46d0be9640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46ac0666c0 0x7f46ac068b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009ab0 tx=0x7f46c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:02.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.541+0000 7f46c97fa640 1 -- 172.21.15.39:0/2858847364 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f46b4054cd0 con 0x7f46cc147910 2024-09-13T16:50:02.617 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-13T16:50:02.632 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-13T16:50:02.691 INFO:tasks.workunit.client.0.smithi039.stderr:In file included from parser.h:22, 2024-09-13T16:50:02.691 INFO:tasks.workunit.client.0.smithi039.stderr: from parser.c:25: 2024-09-13T16:50:02.692 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c: In function 'init_filesys': 2024-09-13T16:50:02.692 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:02.692 INFO:tasks.workunit.client.0.smithi039.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:02.692 INFO:tasks.workunit.client.0.smithi039.stderr: | ^ 2024-09-13T16:50:02.692 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:02.692 INFO:tasks.workunit.client.0.smithi039.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:02.692 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~ 2024-09-13T16:50:02.692 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:02.692 INFO:tasks.workunit.client.0.smithi039.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:02.693 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:02.693 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:02.693 INFO:tasks.workunit.client.0.smithi039.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T16:50:02.693 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:02.693 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:934:17: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:02.693 INFO:tasks.workunit.client.0.smithi039.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-13T16:50:02.693 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.694 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:02.694 INFO:tasks.workunit.client.0.smithi039.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:02.694 INFO:tasks.workunit.client.0.smithi039.stderr: | ^ 2024-09-13T16:50:02.694 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:02.694 INFO:tasks.workunit.client.0.smithi039.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~ 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:934:17: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-13T16:50:02.695 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr: | ^ 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~ 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T16:50:02.696 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:941:17: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr: | ^ 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~ 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:02.697 INFO:tasks.workunit.client.0.smithi039.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:941:17: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c: In function 'init_tg_stats': 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr: | ^ 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~ 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:02.698 INFO:tasks.workunit.client.0.smithi039.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:971:41: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr: | ^ 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~ 2024-09-13T16:50:02.699 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:971:41: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: | ^ 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:02.700 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~ 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:980:41: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr: | ^ 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:02.701 INFO:tasks.workunit.client.0.smithi039.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~ 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:980:41: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-13T16:50:02.702 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.750 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c: In function 'get_optstr': 2024-09-13T16:50:02.750 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:206:23: warning: 'strnlen' specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-13T16:50:02.750 INFO:tasks.workunit.client.0.smithi039.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-13T16:50:02.750 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.750 INFO:tasks.workunit.client.0.smithi039.stderr:parser.c:200:14: note: source object declared here 2024-09-13T16:50:02.750 INFO:tasks.workunit.client.0.smithi039.stderr: 200 | char temp[BUFSIZE]; 2024-09-13T16:50:02.750 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~ 2024-09-13T16:50:02.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.749+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46cc0c2050 con 0x7f46cc147910 2024-09-13T16:50:02.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.749+0000 7f46c97fa640 1 -- 172.21.15.39:0/2858847364 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f46b4054860 con 0x7f46cc147910 2024-09-13T16:50:02.755 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:02.755 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:02.755 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 19 2024-09-13T16:50:02.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46ac0666c0 msgr2=0x7f46ac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:02.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46ac0666c0 0x7f46ac068b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009ab0 tx=0x7f46c4008040 comp rx=0 tx=0).stop 2024-09-13T16:50:02.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc147910 msgr2=0x7f46cc1e87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:02.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc147910 0x7f46cc1e87b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f46b400e480 tx=0x7f46b400e940 comp rx=0 tx=0).stop 2024-09-13T16:50:02.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 shutdown_connections 2024-09-13T16:50:02.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f46ac0666c0 0x7f46ac068b80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:02.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46cc1e79f0 0x7f46cc1e7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:02.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc147910 0x7f46cc1e87b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:02.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 --2- 172.21.15.39:0/2858847364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f46cc0c32c0 0x7f46cc0c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:02.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 >> 172.21.15.39:0/2858847364 conn(0x7f46cc0a9720 msgr2=0x7f46cc150280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:02.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 shutdown_connections 2024-09-13T16:50:02.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:02.753+0000 7f46d2e4d640 1 -- 172.21.15.39:0/2858847364 wait complete. 2024-09-13T16:50:02.768 INFO:tasks.workunit.client.0.smithi039.stderr:In file included from /usr/include/string.h:535, 2024-09-13T16:50:02.768 INFO:tasks.workunit.client.0.smithi039.stderr: from parser.c:19: 2024-09-13T16:50:02.768 INFO:tasks.workunit.client.0.smithi039.stderr:In function 'memcpy', 2024-09-13T16:50:02.768 INFO:tasks.workunit.client.0.smithi039.stderr: inlined from 'size64_convert' at parser.c:119:2: 2024-09-13T16:50:02.768 INFO:tasks.workunit.client.0.smithi039.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10: warning: '__builtin_memcpy' reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-13T16:50:02.769 INFO:tasks.workunit.client.0.smithi039.stderr: 29 | return __builtin___memcpy_chk (__dest, __src, __len, 2024-09-13T16:50:02.769 INFO:tasks.workunit.client.0.smithi039.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.769 INFO:tasks.workunit.client.0.smithi039.stderr: 30 | __glibc_objsize0 (__dest)); 2024-09-13T16:50:02.769 INFO:tasks.workunit.client.0.smithi039.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:02.775 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:50:03.115 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-13T16:50:03.131 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-13T16:50:03.311 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-13T16:50:03.326 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-13T16:50:03.501 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-13T16:50:03.513 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-13T16:50:03.572 INFO:tasks.workunit.client.0.smithi039.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-13T16:50:03.586 INFO:tasks.workunit.client.0.smithi039.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-13T16:50:03.746 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-13T16:50:03.756 INFO:tasks.workunit.client.0.smithi039.stdout:FFSB version 6.0-RC2 started 2024-09-13T16:50:03.756 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T16:50:03.760 INFO:tasks.workunit.client.0.smithi039.stdout:benchmark time = 300 2024-09-13T16:50:03.760 INFO:tasks.workunit.client.0.smithi039.stdout:ThreadGroup 0 2024-09-13T16:50:03.760 INFO:tasks.workunit.client.0.smithi039.stdout:================ 2024-09-13T16:50:03.760 INFO:tasks.workunit.client.0.smithi039.stdout: num_threads = 32 2024-09-13T16:50:03.760 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: read_random = off 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: read_size = 0 (0B) 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: read_blocksize = 0 (0B) 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: read_skip = off 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: read_skipsize = 0 (0B) 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: write_random = on 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: write_size = 5242880 (5MB) 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: fsync_file = 0 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: write_blocksize = 4096 (4KB) 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: wait time = 0 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T16:50:03.761 INFO:tasks.workunit.client.0.smithi039.stdout: op weights 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: read = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: readall = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: write = 1 (100.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: create = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: append = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: delete = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: metaop = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: createdir = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: stat = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: writeall = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: writeall_fsync = 0 (0.00%) 2024-09-13T16:50:03.762 INFO:tasks.workunit.client.0.smithi039.stdout: open_close = 0 (0.00%) 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: write_fsync = 0 (0.00%) 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: create_fsync = 0 (0.00%) 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: append_fsync = 0 (0.00%) 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout:FileSystem . 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout:========== 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: num_dirs = 0 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: starting files = 128 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: min file size = 104857600 (100MB) 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: max file size = 104857600 (100MB) 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: directio = off 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: alignedio = on 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: bufferedio = off 2024-09-13T16:50:03.763 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T16:50:03.764 INFO:tasks.workunit.client.0.smithi039.stdout: aging is off 2024-09-13T16:50:03.764 INFO:tasks.workunit.client.0.smithi039.stdout: current utilization = 0.00% 2024-09-13T16:50:03.764 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T16:50:03.764 INFO:tasks.workunit.client.0.smithi039.stdout:checking existing fs: . 2024-09-13T16:50:03.764 INFO:tasks.workunit.client.0.smithi039.stdout:recreating new fileset 2024-09-13T16:50:03.765 INFO:tasks.workunit.client.0.smithi039.stderr:opendir: No such file or directory 2024-09-13T16:50:04.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.925+0000 7f9547986640 1 Processor -- start 2024-09-13T16:50:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.925+0000 7f9547986640 1 -- start start 2024-09-13T16:50:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9547986640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540071dd0 0x7f9540154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9547986640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f95400c7800 0x7f95401545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9547986640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95400c5a30 con 0x7f9540151500 2024-09-13T16:50:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9547986640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95401576a0 con 0x7f95400c7800 2024-09-13T16:50:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9547986640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9540157980 con 0x7f9540071dd0 2024-09-13T16:50:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545722640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9540151500 0x7f95400c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34722/0 (socket says 172.21.15.39:34722) 2024-09-13T16:50:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545722640 1 -- 172.21.15.39:0/3134324212 learned_addr learned my addr 172.21.15.39:0/3134324212 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9544f21640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540071dd0 0x7f9540154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545f23640 1 --2- 172.21.15.39:0/3134324212 >> v2:172.21.15.84:3300/0 conn(0x7f95400c7800 0x7f95401545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271472832 0 0) 0x7f95400c5a30 con 0x7f9540151500 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f952c003530 con 0x7f9540151500 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1800742181 0 0) 0x7f952c003530 con 0x7f9540151500 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540071dd0 msgr2=0x7f9540154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540071dd0 0x7f9540154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 >> v2:172.21.15.84:3300/0 conn(0x7f95400c7800 msgr2=0x7f95401545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 --2- 172.21.15.39:0/3134324212 >> v2:172.21.15.84:3300/0 conn(0x7f95400c7800 0x7f95401545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540157c60 con 0x7f9540151500 2024-09-13T16:50:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f953c0016a0 con 0x7f9540151500 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 >> v1:172.21.15.39:6789/0 conn(0x7f9540151500 legacy=0x7f95400c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540071dd0 0x7f952c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f952c003f00 0x7f952c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f952c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95401576a0 con 0x7f952c004d10 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9540157980 con 0x7f952c003f00 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c00b9e0 con 0x7f9540071dd0 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545f23640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f952c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9544f21640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f952c003f00 0x7f952c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545722640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540071dd0 0x7f952c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545f23640 1 -- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540071dd0 msgr2=0x7f952c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545f23640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540071dd0 0x7f952c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545f23640 1 -- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f952c003f00 msgr2=0x7f952c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:04.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545f23640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f952c003f00 0x7f952c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:04.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.929+0000 7f9545f23640 1 -- 172.21.15.39:0/3134324212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c003530 con 0x7f952c004d10 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545f23640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f952c005190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f953800e170 tx=0x7f953801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620ac81fdb8ab84 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380a5ba0 con 0x7f952c004d10 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953800f040 con 0x7f952c004d10 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9537fff640 1 -- 172.21.15.39:0/3134324212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380ad440 con 0x7f952c004d10 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 -- 172.21.15.39:0/3134324212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 msgr2=0x7f952c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 --2- 172.21.15.39:0/3134324212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f952c005190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f953800e170 tx=0x7f953801cca0 comp rx=0 tx=0).stop 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545f23640 1 -- 172.21.15.39:0/3134324212 reap_dead start 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 -- 172.21.15.39:0/3134324212 shutdown_connections 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 -- 172.21.15.39:0/3134324212 >> 172.21.15.39:0/3134324212 conn(0x7f95400ab530 msgr2=0x7f95400ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 -- 172.21.15.39:0/3134324212 shutdown_connections 2024-09-13T16:50:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 -- 172.21.15.39:0/3134324212 wait complete. 2024-09-13T16:50:04.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 Processor -- start 2024-09-13T16:50:04.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 -- start start 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f95401504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9540151ee0 0x7f9540155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540150a30 0x7f9540155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95401e9820 con 0x7f952c004d10 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95401e9d10 con 0x7f9540151ee0 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9547986640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9540155490 con 0x7f9540150a30 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545722640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f95401504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545722640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f95401504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54242/0 (socket says 172.21.15.39:54242) 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545722640 1 -- 172.21.15.39:0/1680870630 learned_addr learned my addr 172.21.15.39:0/1680870630 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:04.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545f23640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540150a30 0x7f9540155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9544f21640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9540151ee0 0x7f9540155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545722640 1 -- 172.21.15.39:0/1680870630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540150a30 msgr2=0x7f9540155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545722640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540150a30 0x7f9540155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545722640 1 -- 172.21.15.39:0/1680870630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9540151ee0 msgr2=0x7f9540155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545722640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9540151ee0 0x7f9540155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545722640 1 -- 172.21.15.39:0/1680870630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540153070 con 0x7f952c004d10 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.933+0000 7f9545f23640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540150a30 0x7f9540155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9545722640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f95401504f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f953c011980 tx=0x7f953c011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9535ffb640 1 -- 172.21.15.39:0/1680870630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00fbc0 con 0x7f952c004d10 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9535ffb640 1 -- 172.21.15.39:0/1680870630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c00fd20 con 0x7f952c004d10 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95401e9a30 con 0x7f952c004d10 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9535ffb640 1 -- 172.21.15.39:0/1680870630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c015650 con 0x7f952c004d10 2024-09-13T16:50:04.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95400a9450 con 0x7f952c004d10 2024-09-13T16:50:04.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95400c4280 con 0x7f952c004d10 2024-09-13T16:50:04.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9535ffb640 1 -- 172.21.15.39:0/1680870630 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f953c0157b0 con 0x7f952c004d10 2024-09-13T16:50:04.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9535ffb640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9524066690 0x7f9524068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:04.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9535ffb640 1 -- 172.21.15.39:0/1680870630 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f953c087900 con 0x7f952c004d10 2024-09-13T16:50:04.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9544f21640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9524066690 0x7f9524068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:04.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.937+0000 7f9544f21640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9524066690 0x7f9524068b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95400c3f20 tx=0x7f9530008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:04.945+0000 7f9535ffb640 1 -- 172.21.15.39:0/1680870630 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f953c054900 con 0x7f952c004d10 2024-09-13T16:50:05.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.189+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9540071bb0 con 0x7f952c004d10 2024-09-13T16:50:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.189+0000 7f9535ffb640 1 -- 172.21.15.39:0/1680870630 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f953c057f20 con 0x7f952c004d10 2024-09-13T16:50:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:05.195 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 19 2024-09-13T16:50:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9524066690 msgr2=0x7f9524068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:05.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9524066690 0x7f9524068b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95400c3f20 tx=0x7f9530008040 comp rx=0 tx=0).stop 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 msgr2=0x7f95401504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f95401504f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f953c011980 tx=0x7f953c011e40 comp rx=0 tx=0).stop 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 shutdown_connections 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9524066690 0x7f9524068b50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9540151ee0 0x7f9540155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9540150a30 0x7f9540155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 --2- 172.21.15.39:0/1680870630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c004d10 0x7f95401504f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 >> 172.21.15.39:0/1680870630 conn(0x7f95400ab530 msgr2=0x7f95400711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 shutdown_connections 2024-09-13T16:50:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:05.193+0000 7f9547986640 1 -- 172.21.15.39:0/1680870630 wait complete. 2024-09-13T16:50:05.216 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:50:06.837 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-13T16:50:07.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:07.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f6002805640 1 Processor -- start 2024-09-13T16:50:07.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f6002805640 1 -- start start 2024-09-13T16:50:07.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f6002805640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5ffc071dd0 0x7f5ffc154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f6002805640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc0c7800 0x7f5ffc1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f6002805640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ffc0c5a30 con 0x7f5ffc151500 2024-09-13T16:50:07.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f6002805640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ffc1576a0 con 0x7f5ffc071dd0 2024-09-13T16:50:07.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f6002805640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc157980 con 0x7f5ffc0c7800 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f5ff3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5ffc071dd0 0x7f5ffc154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f6000da2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc0c7800 0x7f5ffc1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f5ffbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5ffc151500 0x7f5ffc0c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:60190/0 (socket says 172.21.15.39:60190) 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f5ff3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5ffc071dd0 0x7f5ffc154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:34262/0 (socket says 172.21.15.39:34262) 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f6000da2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc0c7800 0x7f5ffc1545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:36004/0 (socket says 172.21.15.39:36004) 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.349+0000 7f5ffbfff640 1 -- 172.21.15.39:0/2991316343 learned_addr learned my addr 172.21.15.39:0/2991316343 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f6000da2640 1 -- 172.21.15.39:0/2991316343 >> v2:172.21.15.84:3300/0 conn(0x7f5ffc071dd0 msgr2=0x7f5ffc154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f6000da2640 1 --2- 172.21.15.39:0/2991316343 >> v2:172.21.15.84:3300/0 conn(0x7f5ffc071dd0 0x7f5ffc154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f6000da2640 1 -- 172.21.15.39:0/2991316343 >> v1:172.21.15.84:6789/0 conn(0x7f5ffc151500 legacy=0x7f5ffc0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f6000da2640 1 -- 172.21.15.39:0/2991316343 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc157c60 con 0x7f5ffc0c7800 2024-09-13T16:50:07.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f5ff3fff640 1 --2- 172.21.15.39:0/2991316343 >> v2:172.21.15.84:3300/0 conn(0x7f5ffc071dd0 0x7f5ffc154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:07.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f6000da2640 1 --2- 172.21.15.39:0/2991316343 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc0c7800 0x7f5ffc1545b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009dc0 tx=0x7f5ff401d2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f637f1f7d0bff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:07.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f5ffb7fe640 1 -- 172.21.15.39:0/2991316343 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40a5970 con 0x7f5ffc0c7800 2024-09-13T16:50:07.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f5ffb7fe640 1 -- 172.21.15.39:0/2991316343 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff40a5ad0 con 0x7f5ffc0c7800 2024-09-13T16:50:07.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f5ffb7fe640 1 -- 172.21.15.39:0/2991316343 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40a5e60 con 0x7f5ffc0c7800 2024-09-13T16:50:07.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f6002805640 1 -- 172.21.15.39:0/2991316343 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc0c7800 msgr2=0x7f5ffc1545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f6002805640 1 --2- 172.21.15.39:0/2991316343 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc0c7800 0x7f5ffc1545b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009dc0 tx=0x7f5ff401d2f0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.353+0000 7f6002805640 1 -- 172.21.15.39:0/2991316343 shutdown_connections 2024-09-13T16:50:07.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 --2- 172.21.15.39:0/2991316343 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc0c7800 0x7f5ffc1545b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 --2- 172.21.15.39:0/2991316343 >> v2:172.21.15.84:3300/0 conn(0x7f5ffc071dd0 0x7f5ffc154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 -- 172.21.15.39:0/2991316343 >> 172.21.15.39:0/2991316343 conn(0x7f5ffc0ab530 msgr2=0x7f5ffc0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:07.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 -- 172.21.15.39:0/2991316343 shutdown_connections 2024-09-13T16:50:07.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 -- 172.21.15.39:0/2991316343 wait complete. 2024-09-13T16:50:07.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 Processor -- start 2024-09-13T16:50:07.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 -- start start 2024-09-13T16:50:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc071dd0 0x7f5ffc14b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ffc0c7800 0x7f5ffc151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc151500 0x7f5ffc143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc14c130 con 0x7f5ffc071dd0 2024-09-13T16:50:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc148790 con 0x7f5ffc0c7800 2024-09-13T16:50:07.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc14bb00 con 0x7f5ffc151500 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc071dd0 0x7f5ffc14b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ffbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc071dd0 0x7f5ffc14b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54258/0 (socket says 172.21.15.39:54258) 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ffbfff640 1 -- 172.21.15.39:0/2198690223 learned_addr learned my addr 172.21.15.39:0/2198690223 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ff3fff640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ffc0c7800 0x7f5ffc151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6000da2640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc151500 0x7f5ffc143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ffbfff640 1 -- 172.21.15.39:0/2198690223 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc151500 msgr2=0x7f5ffc143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ffbfff640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc151500 0x7f5ffc143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ffbfff640 1 -- 172.21.15.39:0/2198690223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ffc0c7800 msgr2=0x7f5ffc151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ffbfff640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ffc0c7800 0x7f5ffc151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ffbfff640 1 -- 172.21.15.39:0/2198690223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe4009420 con 0x7f5ffc071dd0 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ff3fff640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ffc0c7800 0x7f5ffc151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ffbfff640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc071dd0 0x7f5ffc14b140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5fec007fd0 tx=0x7f5fec00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:07.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ff97fa640 1 -- 172.21.15.39:0/2198690223 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec005cf0 con 0x7f5ffc071dd0 2024-09-13T16:50:07.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ff97fa640 1 -- 172.21.15.39:0/2198690223 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec005e70 con 0x7f5ffc071dd0 2024-09-13T16:50:07.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f5ff97fa640 1 -- 172.21.15.39:0/2198690223 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0072f0 con 0x7f5ffc071dd0 2024-09-13T16:50:07.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff4009a70 con 0x7f5ffc071dd0 2024-09-13T16:50:07.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.357+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ffc1484b0 con 0x7f5ffc071dd0 2024-09-13T16:50:07.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.361+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ffc0c4280 con 0x7f5ffc071dd0 2024-09-13T16:50:07.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.361+0000 7f5ff97fa640 1 -- 172.21.15.39:0/2198690223 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5fec007450 con 0x7f5ffc071dd0 2024-09-13T16:50:07.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.361+0000 7f5ff97fa640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fdc066690 0x7f5fdc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:07.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.361+0000 7f5ff97fa640 1 -- 172.21.15.39:0/2198690223 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f5fec083ab0 con 0x7f5ffc071dd0 2024-09-13T16:50:07.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.361+0000 7f5ff3fff640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fdc066690 0x7f5fdc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:07.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.361+0000 7f5ff3fff640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fdc066690 0x7f5fdc068b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc14a770 tx=0x7f5fe4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:07.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.365+0000 7f5ff97fa640 1 -- 172.21.15.39:0/2198690223 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5fec050ab0 con 0x7f5ffc071dd0 2024-09-13T16:50:07.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.609+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ffc0afa30 con 0x7f5ffc071dd0 2024-09-13T16:50:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.609+0000 7f5ff97fa640 1 -- 172.21.15.39:0/2198690223 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f5fec0540d0 con 0x7f5ffc071dd0 2024-09-13T16:50:07.613 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:07.613 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:07.613 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 19 2024-09-13T16:50:07.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.609+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fdc066690 msgr2=0x7f5fdc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.609+0000 7f6002805640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fdc066690 0x7f5fdc068b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc14a770 tx=0x7f5fe4008040 comp rx=0 tx=0).stop 2024-09-13T16:50:07.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.609+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc071dd0 msgr2=0x7f5ffc14b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.609+0000 7f6002805640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc071dd0 0x7f5ffc14b140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5fec007fd0 tx=0x7f5fec00c890 comp rx=0 tx=0).stop 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.613+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 shutdown_connections 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.613+0000 7f6002805640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fdc066690 0x7f5fdc068b50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.613+0000 7f6002805640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ffc0c7800 0x7f5ffc151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.613+0000 7f6002805640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ffc151500 0x7f5ffc143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.613+0000 7f6002805640 1 --2- 172.21.15.39:0/2198690223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc071dd0 0x7f5ffc14b140 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.613+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 >> 172.21.15.39:0/2198690223 conn(0x7f5ffc0ab530 msgr2=0x7f5ffc070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.613+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 shutdown_connections 2024-09-13T16:50:07.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:07.613+0000 7f6002805640 1 -- 172.21.15.39:0/2198690223 wait complete. 2024-09-13T16:50:07.633 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:50:09.634 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:09.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2383fff640 1 Processor -- start 2024-09-13T16:50:09.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2383fff640 1 -- start start 2024-09-13T16:50:09.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2383fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2384151330 0x7f23840c0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2383fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f23840c31b0 0x7f23841545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2383fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23841581e0 con 0x7f2384157c90 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2383fff640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23840c6780 con 0x7f2384151330 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2383fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23840c6a60 con 0x7f23840c31b0 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2382ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2384151330 0x7f23840c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2382ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2384151330 0x7f23840c0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54268/0 (socket says 172.21.15.39:54268) 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2382ffd640 1 -- 172.21.15.39:0/13002928 learned_addr learned my addr 172.21.15.39:0/13002928 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f23827fc640 1 --2- 172.21.15.39:0/13002928 >> v2:172.21.15.84:3300/0 conn(0x7f23840c31b0 0x7f23841545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2382ffd640 1 -- 172.21.15.39:0/13002928 >> v2:172.21.15.84:3300/0 conn(0x7f23840c31b0 msgr2=0x7f23841545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2382ffd640 1 --2- 172.21.15.39:0/13002928 >> v2:172.21.15.84:3300/0 conn(0x7f23840c31b0 0x7f23841545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2382ffd640 1 -- 172.21.15.39:0/13002928 >> v1:172.21.15.84:6789/0 conn(0x7f2384157c90 legacy=0x7f23840c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:09.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.765+0000 7f2382ffd640 1 -- 172.21.15.39:0/13002928 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23841540b0 con 0x7f2384151330 2024-09-13T16:50:09.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2382ffd640 1 --2- 172.21.15.39:0/13002928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 0x7f23840c0960 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f23840aaac0 tx=0x7f237401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec08a6eca7ef0a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2381ffb640 1 -- 172.21.15.39:0/13002928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740ad680 con 0x7f2384151330 2024-09-13T16:50:09.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2381ffb640 1 -- 172.21.15.39:0/13002928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23740ad7e0 con 0x7f2384151330 2024-09-13T16:50:09.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2381ffb640 1 -- 172.21.15.39:0/13002928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740adb70 con 0x7f2384151330 2024-09-13T16:50:09.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2383fff640 1 -- 172.21.15.39:0/13002928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 msgr2=0x7f23840c0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2383fff640 1 --2- 172.21.15.39:0/13002928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 0x7f23840c0960 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f23840aaac0 tx=0x7f237401c450 comp rx=0 tx=0).stop 2024-09-13T16:50:09.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2383fff640 1 -- 172.21.15.39:0/13002928 shutdown_connections 2024-09-13T16:50:09.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2383fff640 1 --2- 172.21.15.39:0/13002928 >> v2:172.21.15.84:3300/0 conn(0x7f23840c31b0 0x7f23841545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2383fff640 1 --2- 172.21.15.39:0/13002928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 0x7f23840c0960 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2383fff640 1 -- 172.21.15.39:0/13002928 >> 172.21.15.39:0/13002928 conn(0x7f23840c4030 msgr2=0x7f23840c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:09.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2383fff640 1 -- 172.21.15.39:0/13002928 shutdown_connections 2024-09-13T16:50:09.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.769+0000 7f2383fff640 1 -- 172.21.15.39:0/13002928 wait complete. 2024-09-13T16:50:09.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 Processor -- start 2024-09-13T16:50:09.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 -- start start 2024-09-13T16:50:09.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23840c31b0 0x7f23841ed270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 0x7f238414fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2384157c90 0x7f23840ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841ec860 con 0x7f2384151330 2024-09-13T16:50:09.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23840ab1d0 con 0x7f2384157c90 2024-09-13T16:50:09.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23841ed7b0 con 0x7f23840c31b0 2024-09-13T16:50:09.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23827fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 0x7f238414fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23827fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 0x7f238414fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54276/0 (socket says 172.21.15.39:54276) 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23827fc640 1 -- 172.21.15.39:0/3818332517 learned_addr learned my addr 172.21.15.39:0/3818332517 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23837fe640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2384157c90 0x7f23840ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2382ffd640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23840c31b0 0x7f23841ed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23827fc640 1 -- 172.21.15.39:0/3818332517 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23840c31b0 msgr2=0x7f23841ed270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23827fc640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23840c31b0 0x7f23841ed270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23827fc640 1 -- 172.21.15.39:0/3818332517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2384157c90 msgr2=0x7f23840ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23827fc640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2384157c90 0x7f23840ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23827fc640 1 -- 172.21.15.39:0/3818332517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740094d0 con 0x7f2384151330 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f23827fc640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 0x7f238414fd20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f237800d800 tx=0x7f237800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f236b7fe640 1 -- 172.21.15.39:0/3818332517 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237800b6c0 con 0x7f2384151330 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f236b7fe640 1 -- 172.21.15.39:0/3818332517 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237800b820 con 0x7f2384151330 2024-09-13T16:50:09.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2384150260 con 0x7f2384151330 2024-09-13T16:50:09.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f236b7fe640 1 -- 172.21.15.39:0/3818332517 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237800bb90 con 0x7f2384151330 2024-09-13T16:50:09.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23841eca70 con 0x7f2384151330 2024-09-13T16:50:09.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.773+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23840c4f90 con 0x7f2384151330 2024-09-13T16:50:09.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.777+0000 7f236b7fe640 1 -- 172.21.15.39:0/3818332517 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f237800bcf0 con 0x7f2384151330 2024-09-13T16:50:09.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.777+0000 7f236b7fe640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23700666c0 0x7f2370068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:09.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.777+0000 7f236b7fe640 1 -- 172.21.15.39:0/3818332517 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f2378084a50 con 0x7f2384151330 2024-09-13T16:50:09.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.777+0000 7f2382ffd640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23700666c0 0x7f2370068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:09.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.777+0000 7f2382ffd640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23700666c0 0x7f2370068b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23740af000 tx=0x7f23740a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:09.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:09.781+0000 7f236b7fe640 1 -- 172.21.15.39:0/3818332517 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f23780519c0 con 0x7f2384151330 2024-09-13T16:50:10.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.025+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23840ab6b0 con 0x7f2384151330 2024-09-13T16:50:10.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.025+0000 7f236b7fe640 1 -- 172.21.15.39:0/3818332517 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f2378054fe0 con 0x7f2384151330 2024-09-13T16:50:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:10.029 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:10.030 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 19 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23700666c0 msgr2=0x7f2370068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23700666c0 0x7f2370068b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23740af000 tx=0x7f23740a7000 comp rx=0 tx=0).stop 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 msgr2=0x7f238414fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 0x7f238414fd20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f237800d800 tx=0x7f237800dcc0 comp rx=0 tx=0).stop 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 shutdown_connections 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23700666c0 0x7f2370068b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2384157c90 0x7f23840ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2384151330 0x7f238414fd20 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f237800d800 tx=0x7f237800dcc0 comp rx=0 tx=0).stop 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 --2- 172.21.15.39:0/3818332517 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23840c31b0 0x7f23841ed270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 >> 172.21.15.39:0/3818332517 conn(0x7f23840c4030 msgr2=0x7f2384154ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 shutdown_connections 2024-09-13T16:50:10.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:10.029+0000 7f2383fff640 1 -- 172.21.15.39:0/3818332517 wait complete. 2024-09-13T16:50:10.052 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:50:12.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:12.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59ceb4640 1 Processor -- start 2024-09-13T16:50:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59ceb4640 1 -- start start 2024-09-13T16:50:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59ceb4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff59814a940 0x7ff598153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59ceb4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5980c6830 con 0x7ff598070810 2024-09-13T16:50:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59ceb4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5980c6b10 con 0x7ff5980c6320 2024-09-13T16:50:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59ceb4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5980c6df0 con 0x7ff59814a940 2024-09-13T16:50:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff596d9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff598070810 0x7ff598070c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41672/0 (socket says 172.21.15.39:41672) 2024-09-13T16:50:12.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff596d9d640 1 -- 172.21.15.39:0/385726704 learned_addr learned my addr 172.21.15.39:0/385726704 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:12.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59659c640 1 --2- 172.21.15.39:0/385726704 >> v2:172.21.15.84:3300/0 conn(0x7ff59814a940 0x7ff598153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:12.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538519448 0 0) 0x7ff5980c6830 con 0x7ff598070810 2024-09-13T16:50:12.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff580003530 con 0x7ff598070810 2024-09-13T16:50:12.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14787528 0 0) 0x7ff580003530 con 0x7ff598070810 2024-09-13T16:50:12.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 >> v2:172.21.15.84:3300/0 conn(0x7ff59814a940 msgr2=0x7ff598153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:12.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 --2- 172.21.15.39:0/385726704 >> v2:172.21.15.84:3300/0 conn(0x7ff59814a940 0x7ff598153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 >> v1:172.21.15.84:6789/0 conn(0x7ff5980c6320 legacy=0x7ff5980c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980c70d0 con 0x7ff598070810 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7ff590001a00 con 0x7ff598070810 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 >> v1:172.21.15.39:6789/0 conn(0x7ff598070810 legacy=0x7ff598070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff580003980 0x7ff580003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5800047d0 0x7ff580004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5800052b0 0x7ff58000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980c6df0 con 0x7ff5800047d0 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58000c180 con 0x7ff5800052b0 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58000c340 con 0x7ff580003980 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff595d9b640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5800047d0 0x7ff580004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff596d9d640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5800052b0 0x7ff58000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:12.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59659c640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff580003980 0x7ff580003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff595d9b640 1 -- 172.21.15.39:0/385726704 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff580003980 msgr2=0x7ff580003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff595d9b640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff580003980 0x7ff580003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff595d9b640 1 -- 172.21.15.39:0/385726704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5800052b0 msgr2=0x7ff58000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff595d9b640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5800052b0 0x7ff58000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff595d9b640 1 -- 172.21.15.39:0/385726704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff580003530 con 0x7ff5800047d0 2024-09-13T16:50:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff596d9d640 1 -- 172.21.15.39:0/385726704 reap_dead start 2024-09-13T16:50:12.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.193+0000 7ff59659c640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff580003980 0x7ff580003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff595d9b640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5800047d0 0x7ff580004c50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff58c001750 tx=0x7ff58c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca22375c703bb611 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0052d0 con 0x7ff5800047d0 2024-09-13T16:50:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c0a7050 con 0x7ff5800047d0 2024-09-13T16:50:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59559a640 1 -- 172.21.15.39:0/385726704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c003600 con 0x7ff5800047d0 2024-09-13T16:50:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59ceb4640 1 -- 172.21.15.39:0/385726704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5800047d0 msgr2=0x7ff580004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:12.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59ceb4640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5800047d0 0x7ff580004c50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff58c001750 tx=0x7ff58c01c3c0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59ceb4640 1 -- 172.21.15.39:0/385726704 shutdown_connections 2024-09-13T16:50:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59ceb4640 1 --2- 172.21.15.39:0/385726704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5800047d0 0x7ff580004c50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59ceb4640 1 -- 172.21.15.39:0/385726704 >> 172.21.15.39:0/385726704 conn(0x7ff5980ab460 msgr2=0x7ff5980ab870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59ceb4640 1 -- 172.21.15.39:0/385726704 shutdown_connections 2024-09-13T16:50:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59ceb4640 1 -- 172.21.15.39:0/385726704 wait complete. 2024-09-13T16:50:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59ceb4640 1 Processor -- start 2024-09-13T16:50:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.197+0000 7ff59ceb4640 1 -- start start 2024-09-13T16:50:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59ceb4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5800047d0 0x7ff598149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59ceb4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff598156e00 0x7ff598157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59ceb4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5980c3130 0x7ff5980c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59ceb4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5981ed390 con 0x7ff5980c3130 2024-09-13T16:50:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59ceb4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598151690 con 0x7ff598156e00 2024-09-13T16:50:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59ceb4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5981ecac0 con 0x7ff5800047d0 2024-09-13T16:50:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff596d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5980c3130 0x7ff5980c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff596d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5980c3130 0x7ff5980c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50762/0 (socket says 172.21.15.39:50762) 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff596d9d640 1 -- 172.21.15.39:0/1882642440 learned_addr learned my addr 172.21.15.39:0/1882642440 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff595d9b640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff598156e00 0x7ff598157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59659c640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5800047d0 0x7ff598149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff596d9d640 1 -- 172.21.15.39:0/1882642440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5800047d0 msgr2=0x7ff598149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff596d9d640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5800047d0 0x7ff598149ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff596d9d640 1 -- 172.21.15.39:0/1882642440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff598156e00 msgr2=0x7ff598157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff596d9d640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff598156e00 0x7ff598157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff596d9d640 1 -- 172.21.15.39:0/1882642440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980c03d0 con 0x7ff5980c3130 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59659c640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5800047d0 0x7ff598149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff596d9d640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5980c3130 0x7ff5980c35b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff5900013e0 tx=0x7ff590007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff586ffd640 1 -- 172.21.15.39:0/1882642440 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59000ab40 con 0x7ff5980c3130 2024-09-13T16:50:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff586ffd640 1 -- 172.21.15.39:0/1882642440 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59000aca0 con 0x7ff5980c3130 2024-09-13T16:50:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff598149c00 con 0x7ff5980c3130 2024-09-13T16:50:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff586ffd640 1 -- 172.21.15.39:0/1882642440 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590013630 con 0x7ff5980c3130 2024-09-13T16:50:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5981ed020 con 0x7ff5980c3130 2024-09-13T16:50:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5980c6830 con 0x7ff5980c3130 2024-09-13T16:50:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff586ffd640 1 -- 172.21.15.39:0/1882642440 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff590013790 con 0x7ff5980c3130 2024-09-13T16:50:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff586ffd640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff578066690 0x7ff578068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff586ffd640 1 -- 172.21.15.39:0/1882642440 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6723+0+0 (secure 0 0 0) 0x7ff590084b60 con 0x7ff5980c3130 2024-09-13T16:50:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.201+0000 7ff59659c640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff578066690 0x7ff578068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.205+0000 7ff59659c640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff578066690 0x7ff578068b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5980bccf0 tx=0x7ff588008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:12.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.209+0000 7ff586ffd640 1 -- 172.21.15.39:0/1882642440 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff590051b60 con 0x7ff5980c3130 2024-09-13T16:50:12.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.501+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5980c7810 con 0x7ff5980c3130 2024-09-13T16:50:12.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.501+0000 7ff586ffd640 1 -- 172.21.15.39:0/1882642440 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+5929 (secure 0 0 0) 0x7ff590055180 con 0x7ff5980c3130 2024-09-13T16:50:12.507 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:12.507 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5142,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:49:47.577682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4232":{"gid":4232,"name":"d","rank":0,"incarnation":15,"state":"up:active","state_seq":31,"addr":"172.21.15.84:6835/3540656099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3540656099},{"type":"v1","addr":"172.21.15.84:6835","nonce":3540656099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:12.507 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 19 2024-09-13T16:50:12.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff578066690 msgr2=0x7ff578068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:12.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff578066690 0x7ff578068b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff5980bccf0 tx=0x7ff588008040 comp rx=0 tx=0).stop 2024-09-13T16:50:12.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5980c3130 msgr2=0x7ff5980c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:12.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5980c3130 0x7ff5980c35b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff5900013e0 tx=0x7ff590007a90 comp rx=0 tx=0).stop 2024-09-13T16:50:12.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 shutdown_connections 2024-09-13T16:50:12.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff578066690 0x7ff578068b50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff598156e00 0x7ff598157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5980c3130 0x7ff5980c35b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 --2- 172.21.15.39:0/1882642440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5800047d0 0x7ff598149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:12.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 >> 172.21.15.39:0/1882642440 conn(0x7ff5980ab460 msgr2=0x7ff5980a9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:12.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 shutdown_connections 2024-09-13T16:50:12.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:12.505+0000 7ff59ceb4640 1 -- 172.21.15.39:0/1882642440 wait complete. 2024-09-13T16:50:12.530 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:50:14.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:14.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7903287640 1 Processor -- start 2024-09-13T16:50:14.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7903287640 1 -- start start 2024-09-13T16:50:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7903287640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f78fc071dd0 0x7f78fc154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7903287640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f78fc0c7800 0x7f78fc1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7903287640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc0c5a30 con 0x7f78fc151500 2024-09-13T16:50:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7903287640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc1576a0 con 0x7f78fc071dd0 2024-09-13T16:50:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7903287640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc157980 con 0x7f78fc0c7800 2024-09-13T16:50:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7901023640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f78fc151500 0x7f78fc0c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41686/0 (socket says 172.21.15.39:41686) 2024-09-13T16:50:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7901023640 1 -- 172.21.15.39:0/2769178726 learned_addr learned my addr 172.21.15.39:0/2769178726 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7900822640 1 --2- 172.21.15.39:0/2769178726 >> v2:172.21.15.39:3300/0 conn(0x7f78fc071dd0 0x7f78fc154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7901824640 1 --2- 172.21.15.39:0/2769178726 >> v2:172.21.15.84:3300/0 conn(0x7f78fc0c7800 0x7f78fc1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7900822640 1 -- 172.21.15.39:0/2769178726 >> v2:172.21.15.84:3300/0 conn(0x7f78fc0c7800 msgr2=0x7f78fc1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7900822640 1 --2- 172.21.15.39:0/2769178726 >> v2:172.21.15.84:3300/0 conn(0x7f78fc0c7800 0x7f78fc1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7900822640 1 -- 172.21.15.39:0/2769178726 >> v1:172.21.15.39:6789/0 conn(0x7f78fc151500 legacy=0x7f78fc0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f7900822640 1 -- 172.21.15.39:0/2769178726 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc157c60 con 0x7f78fc071dd0 2024-09-13T16:50:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.689+0000 7f78f3fff640 1 -- 172.21.15.39:0/2769178726 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147650903 0 0) 0x7f78fc0c5a30 con 0x7f78fc151500 2024-09-13T16:50:14.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f7900822640 1 --2- 172.21.15.39:0/2769178726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc071dd0 0x7f78fc154070 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0097c0 tx=0x7f78ec01ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12e75587d7cf804 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:14.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f78f3fff640 1 -- 172.21.15.39:0/2769178726 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0ac620 con 0x7f78fc071dd0 2024-09-13T16:50:14.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f78f3fff640 1 -- 172.21.15.39:0/2769178726 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec0ac780 con 0x7f78fc071dd0 2024-09-13T16:50:14.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f78f3fff640 1 -- 172.21.15.39:0/2769178726 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec0acb10 con 0x7f78fc071dd0 2024-09-13T16:50:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f7903287640 1 -- 172.21.15.39:0/2769178726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc071dd0 msgr2=0x7f78fc154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f7903287640 1 --2- 172.21.15.39:0/2769178726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc071dd0 0x7f78fc154070 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0097c0 tx=0x7f78ec01ce00 comp rx=0 tx=0).stop 2024-09-13T16:50:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f7903287640 1 -- 172.21.15.39:0/2769178726 shutdown_connections 2024-09-13T16:50:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f7903287640 1 --2- 172.21.15.39:0/2769178726 >> v2:172.21.15.84:3300/0 conn(0x7f78fc0c7800 0x7f78fc1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f7903287640 1 --2- 172.21.15.39:0/2769178726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc071dd0 0x7f78fc154070 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f7903287640 1 -- 172.21.15.39:0/2769178726 >> 172.21.15.39:0/2769178726 conn(0x7f78fc0ab530 msgr2=0x7f78fc0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f7903287640 1 -- 172.21.15.39:0/2769178726 shutdown_connections 2024-09-13T16:50:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.693+0000 7f7903287640 1 -- 172.21.15.39:0/2769178726 wait complete. 2024-09-13T16:50:14.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7903287640 1 Processor -- start 2024-09-13T16:50:14.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7903287640 1 -- start start 2024-09-13T16:50:14.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7903287640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78fc0c7800 0x7f78fc1ec450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7903287640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78fc151500 0x7f78fc0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7903287640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc155150 0x7f78fc151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7903287640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1ed450 con 0x7f78fc155150 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7903287640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1f0500 con 0x7f78fc0c7800 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7903287640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc153030 con 0x7f78fc151500 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7901824640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc155150 0x7f78fc151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7901824640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc155150 0x7f78fc151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50780/0 (socket says 172.21.15.39:50780) 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7901824640 1 -- 172.21.15.39:0/2497959632 learned_addr learned my addr 172.21.15.39:0/2497959632 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7900822640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78fc151500 0x7f78fc0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7901023640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78fc0c7800 0x7f78fc1ec450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:14.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7901824640 1 -- 172.21.15.39:0/2497959632 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78fc151500 msgr2=0x7f78fc0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:14.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7901824640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78fc151500 0x7f78fc0ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:14.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7901824640 1 -- 172.21.15.39:0/2497959632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78fc0c7800 msgr2=0x7f78fc1ec450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7901824640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78fc0c7800 0x7f78fc1ec450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.697+0000 7f7901824640 1 -- 172.21.15.39:0/2497959632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec009420 con 0x7f78fc155150 2024-09-13T16:50:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f7901824640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc155150 0x7f78fc151bd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f78f4007e90 tx=0x7f78f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f78f1ffb640 1 -- 172.21.15.39:0/2497959632 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4014620 con 0x7f78fc155150 2024-09-13T16:50:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f78f1ffb640 1 -- 172.21.15.39:0/2497959632 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f4014780 con 0x7f78fc155150 2024-09-13T16:50:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc153b40 con 0x7f78fc155150 2024-09-13T16:50:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f78f1ffb640 1 -- 172.21.15.39:0/2497959632 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4014b30 con 0x7f78fc155150 2024-09-13T16:50:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc153860 con 0x7f78fc155150 2024-09-13T16:50:14.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc0c4280 con 0x7f78fc155150 2024-09-13T16:50:14.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f78f1ffb640 1 -- 172.21.15.39:0/2497959632 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f78f4014c90 con 0x7f78fc155150 2024-09-13T16:50:14.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f78f1ffb640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78e00666c0 0x7f78e0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:14.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f78f1ffb640 1 -- 172.21.15.39:0/2497959632 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f78f4084db0 con 0x7f78fc155150 2024-09-13T16:50:14.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f7901023640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78e00666c0 0x7f78e0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:14.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.701+0000 7f7901023640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78e00666c0 0x7f78e0068b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78f8000f70 tx=0x7f78f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:14.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:14.709+0000 7f78f1ffb640 1 -- 172.21.15.39:0/2497959632 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f78f4051e10 con 0x7f78fc155150 2024-09-13T16:50:15.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.021+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78fc071bb0 con 0x7f78fc155150 2024-09-13T16:50:15.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.021+0000 7f78f1ffb640 1 -- 172.21.15.39:0/2497959632 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5171 (secure 0 0 0) 0x7f78f4055430 con 0x7f78fc155150 2024-09-13T16:50:15.026 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:15.027 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:14.555755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:15.027 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 20 2024-09-13T16:50:15.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78e00666c0 msgr2=0x7f78e0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:15.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78e00666c0 0x7f78e0068b80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78f8000f70 tx=0x7f78f8009040 comp rx=0 tx=0).stop 2024-09-13T16:50:15.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc155150 msgr2=0x7f78fc151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:15.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc155150 0x7f78fc151bd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f78f4007e90 tx=0x7f78f400e4b0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 shutdown_connections 2024-09-13T16:50:15.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78e00666c0 0x7f78e0068b80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78fc155150 0x7f78fc151bd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78fc151500 0x7f78fc0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 --2- 172.21.15.39:0/2497959632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78fc0c7800 0x7f78fc1ec450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 >> 172.21.15.39:0/2497959632 conn(0x7f78fc0ab530 msgr2=0x7f78fc0ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:15.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 shutdown_connections 2024-09-13T16:50:15.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.025+0000 7f7903287640 1 -- 172.21.15.39:0/2497959632 wait complete. 2024-09-13T16:50:15.048 INFO:tasks.mds_thrash.fs.[b]:mds.d down, removed from mdsmap 2024-09-13T16:50:15.048 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T16:50:15.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd97f6f640 1 Processor -- start 2024-09-13T16:50:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd97f6f640 1 -- start start 2024-09-13T16:50:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd97f6f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdd90070b40 0x7fdd900c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd97f6f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd900bcfb0 0x7fdd9015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd97f6f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd90157360 con 0x7fdd900c4260 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd97f6f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd900c80b0 con 0x7fdd90070b40 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd97f6f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd901575b0 con 0x7fdd900bcfb0 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd9550a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdd90070b40 0x7fdd900c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd95d0b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdd900c4260 0x7fdd90070430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41698/0 (socket says 172.21.15.39:41698) 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd95d0b640 1 -- 172.21.15.39:0/1370783621 learned_addr learned my addr 172.21.15.39:0/1370783621 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.165+0000 7fdd9650c640 1 --2- 172.21.15.39:0/1370783621 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd900bcfb0 0x7fdd9015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd9550a640 1 -- 172.21.15.39:0/1370783621 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd900bcfb0 msgr2=0x7fdd9015e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd9550a640 1 --2- 172.21.15.39:0/1370783621 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd900bcfb0 0x7fdd9015e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd9550a640 1 -- 172.21.15.39:0/1370783621 >> v1:172.21.15.39:6789/0 conn(0x7fdd900c4260 legacy=0x7fdd90070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd94d09640 1 -- 172.21.15.39:0/1370783621 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022823094 0 0) 0x7fdd90157360 con 0x7fdd900c4260 2024-09-13T16:50:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd9550a640 1 -- 172.21.15.39:0/1370783621 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd90157890 con 0x7fdd90070b40 2024-09-13T16:50:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd9550a640 1 --2- 172.21.15.39:0/1370783621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd90070b40 0x7fdd900c4640 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdd800097c0 tx=0x7fdd8001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e58b110c87a0ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd94d09640 1 -- 172.21.15.39:0/1370783621 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd900c7330 con 0x7fdd90070b40 2024-09-13T16:50:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd94d09640 1 -- 172.21.15.39:0/1370783621 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd800a7050 con 0x7fdd90070b40 2024-09-13T16:50:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd94d09640 1 -- 172.21.15.39:0/1370783621 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd800ab820 con 0x7fdd90070b40 2024-09-13T16:50:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd97f6f640 1 -- 172.21.15.39:0/1370783621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd90070b40 msgr2=0x7fdd900c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd97f6f640 1 --2- 172.21.15.39:0/1370783621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd90070b40 0x7fdd900c4640 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdd800097c0 tx=0x7fdd8001c380 comp rx=0 tx=0).stop 2024-09-13T16:50:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd97f6f640 1 -- 172.21.15.39:0/1370783621 shutdown_connections 2024-09-13T16:50:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd97f6f640 1 --2- 172.21.15.39:0/1370783621 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd900bcfb0 0x7fdd9015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd97f6f640 1 --2- 172.21.15.39:0/1370783621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd90070b40 0x7fdd900c4640 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd97f6f640 1 -- 172.21.15.39:0/1370783621 >> 172.21.15.39:0/1370783621 conn(0x7fdd90069e10 msgr2=0x7fdd9006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd97f6f640 1 -- 172.21.15.39:0/1370783621 shutdown_connections 2024-09-13T16:50:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.169+0000 7fdd97f6f640 1 -- 172.21.15.39:0/1370783621 wait complete. 2024-09-13T16:50:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 Processor -- start 2024-09-13T16:50:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 -- start start 2024-09-13T16:50:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd900bcfb0 0x7fdd90153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd900c4260 0x7fdd90151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd901521a0 0x7fdd90154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90154660 con 0x7fdd900c4260 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90153080 con 0x7fdd900bcfb0 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd900c0320 con 0x7fdd901521a0 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9550a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd900c4260 0x7fdd90151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9550a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd900c4260 0x7fdd90151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50796/0 (socket says 172.21.15.39:50796) 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9550a640 1 -- 172.21.15.39:0/202225687 learned_addr learned my addr 172.21.15.39:0/202225687 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9650c640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd901521a0 0x7fdd90154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd95d0b640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd900bcfb0 0x7fdd90153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9550a640 1 -- 172.21.15.39:0/202225687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd901521a0 msgr2=0x7fdd90154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9550a640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd901521a0 0x7fdd90154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9550a640 1 -- 172.21.15.39:0/202225687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd900bcfb0 msgr2=0x7fdd90153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9550a640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd900bcfb0 0x7fdd90153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9550a640 1 -- 172.21.15.39:0/202225687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80009420 con 0x7fdd900c4260 2024-09-13T16:50:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd9550a640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd900c4260 0x7fdd90151c60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdd80019fb0 tx=0x7fdd80019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd867fc640 1 -- 172.21.15.39:0/202225687 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd800b4610 con 0x7fdd900c4260 2024-09-13T16:50:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd867fc640 1 -- 172.21.15.39:0/202225687 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd800abe90 con 0x7fdd900c4260 2024-09-13T16:50:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd901540a0 con 0x7fdd900c4260 2024-09-13T16:50:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd867fc640 1 -- 172.21.15.39:0/202225687 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd800b49d0 con 0x7fdd900c4260 2024-09-13T16:50:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd90153290 con 0x7fdd900c4260 2024-09-13T16:50:15.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.173+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd900c4640 con 0x7fdd900c4260 2024-09-13T16:50:15.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.177+0000 7fdd867fc640 1 -- 172.21.15.39:0/202225687 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd800b2040 con 0x7fdd900c4260 2024-09-13T16:50:15.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.177+0000 7fdd867fc640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd740666c0 0x7fdd74068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:15.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.177+0000 7fdd867fc640 1 -- 172.21.15.39:0/202225687 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdd8011eac0 con 0x7fdd900c4260 2024-09-13T16:50:15.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.177+0000 7fdd95d0b640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd740666c0 0x7fdd74068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:15.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.177+0000 7fdd95d0b640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd740666c0 0x7fdd74068b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000f70 tx=0x7fdd8c007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:15.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.181+0000 7fdd867fc640 1 -- 172.21.15.39:0/202225687 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd800ebbe0 con 0x7fdd900c4260 2024-09-13T16:50:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.397+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd900c6e70 con 0x7fdd900c4260 2024-09-13T16:50:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd867fc640 1 -- 172.21.15.39:0/202225687 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5171 (secure 0 0 0) 0x7fdd800ef200 con 0x7fdd900c4260 2024-09-13T16:50:15.404 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:15.404 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:14.555755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 20 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd740666c0 msgr2=0x7fdd74068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd740666c0 0x7fdd74068b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000f70 tx=0x7fdd8c007c10 comp rx=0 tx=0).stop 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd900c4260 msgr2=0x7fdd90151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd900c4260 0x7fdd90151c60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdd80019fb0 tx=0x7fdd80019120 comp rx=0 tx=0).stop 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 shutdown_connections 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd740666c0 0x7fdd74068b80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd901521a0 0x7fdd90154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd900c4260 0x7fdd90151c60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 --2- 172.21.15.39:0/202225687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd900bcfb0 0x7fdd90153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 >> 172.21.15.39:0/202225687 conn(0x7fdd90069e10 msgr2=0x7fdd90145760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:15.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 shutdown_connections 2024-09-13T16:50:15.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:15.401+0000 7fdd97f6f640 1 -- 172.21.15.39:0/202225687 wait complete. 2024-09-13T16:50:15.422 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 20, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 8, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_4264": { "addr": "172.21.15.84:6833/1392588044", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 1392588044, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 1392588044, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4264, "incarnation": 4, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:48:02.679109+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4264 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 20, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5142": { "addr": "172.21.15.39:6835/1572373831", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 1572373831, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 1572373831, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5142, "incarnation": 20, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 28, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:50:14.555755+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5142 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:50:17.423 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:17.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.565+0000 7fdf2a6d1640 1 Processor -- start 2024-09-13T16:50:17.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.565+0000 7fdf2a6d1640 1 -- start start 2024-09-13T16:50:17.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf2a6d1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf240662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:17.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf2a6d1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf240c6270 0x7fdf240bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:17.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf2a6d1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2406ddf0 con 0x7fdf240c2ee0 2024-09-13T16:50:17.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf2a6d1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf240c4710 con 0x7fdf240c6270 2024-09-13T16:50:17.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf2a6d1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf240c49d0 con 0x7fdf24065ee0 2024-09-13T16:50:17.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf28c6e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf240c6270 0x7fdf240bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:17.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf23fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf240662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:17.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf1b7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdf240c2ee0 0x7fdf240c2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:33030/0 (socket says 172.21.15.39:33030) 2024-09-13T16:50:17.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf1b7fe640 1 -- 172.21.15.39:0/1857337298 learned_addr learned my addr 172.21.15.39:0/1857337298 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:17.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf237fe640 1 -- 172.21.15.39:0/1857337298 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418126444 0 0) 0x7fdf2406ddf0 con 0x7fdf240c2ee0 2024-09-13T16:50:17.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf237fe640 1 -- 172.21.15.39:0/1857337298 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf10003530 con 0x7fdf240c2ee0 2024-09-13T16:50:17.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf28c6e640 1 -- 172.21.15.39:0/1857337298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 msgr2=0x7fdf240662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:17.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf28c6e640 1 --2- 172.21.15.39:0/1857337298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf240662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf28c6e640 1 -- 172.21.15.39:0/1857337298 >> v1:172.21.15.84:6789/0 conn(0x7fdf240c2ee0 legacy=0x7fdf240c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:17.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf28c6e640 1 -- 172.21.15.39:0/1857337298 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf24154880 con 0x7fdf240c6270 2024-09-13T16:50:17.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.569+0000 7fdf23fff640 1 --2- 172.21.15.39:0/1857337298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf240662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:17.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf28c6e640 1 --2- 172.21.15.39:0/1857337298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c6270 0x7fdf240bfc40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009d90 tx=0x7fdf1c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d052b0b289d9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:17.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf237fe640 1 -- 172.21.15.39:0/1857337298 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0a58b0 con 0x7fdf240c6270 2024-09-13T16:50:17.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf237fe640 1 -- 172.21.15.39:0/1857337298 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0a5a10 con 0x7fdf240c6270 2024-09-13T16:50:17.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf237fe640 1 -- 172.21.15.39:0/1857337298 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0a5d80 con 0x7fdf240c6270 2024-09-13T16:50:17.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1857337298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c6270 msgr2=0x7fdf240bfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:17.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 --2- 172.21.15.39:0/1857337298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c6270 0x7fdf240bfc40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009d90 tx=0x7fdf1c01c8e0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1857337298 shutdown_connections 2024-09-13T16:50:17.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 --2- 172.21.15.39:0/1857337298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c6270 0x7fdf240bfc40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 --2- 172.21.15.39:0/1857337298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf240662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1857337298 >> 172.21.15.39:0/1857337298 conn(0x7fdf240c3310 msgr2=0x7fdf240c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:17.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1857337298 shutdown_connections 2024-09-13T16:50:17.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1857337298 wait complete. 2024-09-13T16:50:17.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 Processor -- start 2024-09-13T16:50:17.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.573+0000 7fdf2a6d1640 1 -- start start 2024-09-13T16:50:17.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf2a6d1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf241ec600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:17.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf2a6d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c2ee0 0x7fdf24141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:17.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf2a6d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240c6270 0x7fdf2414cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:17.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf2a6d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf24071ad0 con 0x7fdf240c6270 2024-09-13T16:50:17.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf2a6d1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf24151f20 con 0x7fdf240c2ee0 2024-09-13T16:50:17.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf2a6d1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2414c7f0 con 0x7fdf24065ee0 2024-09-13T16:50:17.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf28c6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240c6270 0x7fdf2414cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:17.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf28c6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240c6270 0x7fdf2414cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50810/0 (socket says 172.21.15.39:50810) 2024-09-13T16:50:17.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf28c6e640 1 -- 172.21.15.39:0/1507782270 learned_addr learned my addr 172.21.15.39:0/1507782270 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf23fff640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf241ec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf1b7fe640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c2ee0 0x7fdf24141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf28c6e640 1 -- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 msgr2=0x7fdf241ec600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf28c6e640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf241ec600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf28c6e640 1 -- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c2ee0 msgr2=0x7fdf24141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf28c6e640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c2ee0 0x7fdf24141520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf28c6e640 1 -- 172.21.15.39:0/1507782270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf14009420 con 0x7fdf240c6270 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf1b7fe640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c2ee0 0x7fdf24141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf23fff640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf241ec600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf28c6e640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240c6270 0x7fdf2414cc80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c01a510 tx=0x7fdf1c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf217fa640 1 -- 172.21.15.39:0/1507782270 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0ad640 con 0x7fdf240c6270 2024-09-13T16:50:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf217fa640 1 -- 172.21.15.39:0/1507782270 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0ad7a0 con 0x7fdf240c6270 2024-09-13T16:50:17.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf217fa640 1 -- 172.21.15.39:0/1507782270 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0adb30 con 0x7fdf240c6270 2024-09-13T16:50:17.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c009a70 con 0x7fdf240c6270 2024-09-13T16:50:17.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf24144830 con 0x7fdf240c6270 2024-09-13T16:50:17.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2414c980 con 0x7fdf240c6270 2024-09-13T16:50:17.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf217fa640 1 -- 172.21.15.39:0/1507782270 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf1c0b4070 con 0x7fdf240c6270 2024-09-13T16:50:17.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf217fa640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf04066690 0x7fdf04068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:17.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf217fa640 1 -- 172.21.15.39:0/1507782270 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fdf1c11f110 con 0x7fdf240c6270 2024-09-13T16:50:17.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.577+0000 7fdf23fff640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf04066690 0x7fdf04068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:17.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.581+0000 7fdf23fff640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf04066690 0x7fdf04068b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdf240c5fd0 tx=0x7fdf14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:17.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.581+0000 7fdf217fa640 1 -- 172.21.15.39:0/1507782270 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf1c0ec0e0 con 0x7fdf240c6270 2024-09-13T16:50:17.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf24151980 con 0x7fdf240c6270 2024-09-13T16:50:17.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf217fa640 1 -- 172.21.15.39:0/1507782270 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5171 (secure 0 0 0) 0x7fdf1c0ef700 con 0x7fdf240c6270 2024-09-13T16:50:17.809 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:17.809 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:16.832848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:17.809 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 22 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf04066690 msgr2=0x7fdf04068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf04066690 0x7fdf04068b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdf240c5fd0 tx=0x7fdf14008040 comp rx=0 tx=0).stop 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240c6270 msgr2=0x7fdf2414cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240c6270 0x7fdf2414cc80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c01a510 tx=0x7fdf1c019680 comp rx=0 tx=0).stop 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 shutdown_connections 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf04066690 0x7fdf04068b50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf240c2ee0 0x7fdf24141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240c6270 0x7fdf2414cc80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 --2- 172.21.15.39:0/1507782270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24065ee0 0x7fdf241ec600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 >> 172.21.15.39:0/1507782270 conn(0x7fdf240c3310 msgr2=0x7fdf240c2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:17.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 shutdown_connections 2024-09-13T16:50:17.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:17.805+0000 7fdf2a6d1640 1 -- 172.21.15.39:0/1507782270 wait complete. 2024-09-13T16:50:17.829 INFO:tasks.mds_thrash.fs.[b]:no change 2024-09-13T16:50:18.881 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-13T16:50:18.894 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-13T16:50:19.400 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-13T16:50:19.418 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-13T16:50:19.708 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-13T16:50:19.725 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-13T16:50:19.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:19.874 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-13T16:50:19.887 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-13T16:50:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.961+0000 7f34534c4640 1 Processor -- start 2024-09-13T16:50:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.961+0000 7f34534c4640 1 -- start start 2024-09-13T16:50:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.961+0000 7f34534c4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f344c155160 0x7f344c1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.961+0000 7f34534c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c151bf0 con 0x7f344c14bf30 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.961+0000 7f34534c4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c155660 con 0x7f344c0a5ef0 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.961+0000 7f34534c4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344c155940 con 0x7f344c155160 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.961+0000 7f3451260640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f344c14bf30 0x7f344c06ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41706/0 (socket says 172.21.15.39:41706) 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.961+0000 7f3451260640 1 -- 172.21.15.39:0/341539137 learned_addr learned my addr 172.21.15.39:0/341539137 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.961+0000 7f3450a5f640 1 --2- 172.21.15.39:0/341539137 >> v2:172.21.15.84:3300/0 conn(0x7f344c155160 0x7f344c1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3443fff640 1 -- 172.21.15.39:0/341539137 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796490899 0 0) 0x7f344c151bf0 con 0x7f344c14bf30 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3443fff640 1 -- 172.21.15.39:0/341539137 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3438003530 con 0x7f344c14bf30 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3443fff640 1 -- 172.21.15.39:0/341539137 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425941761 0 0) 0x7f344c155660 con 0x7f344c0a5ef0 2024-09-13T16:50:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3443fff640 1 -- 172.21.15.39:0/341539137 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f344c151bf0 con 0x7f344c0a5ef0 2024-09-13T16:50:19.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3450a5f640 1 -- 172.21.15.39:0/341539137 >> v1:172.21.15.84:6789/0 conn(0x7f344c0a5ef0 legacy=0x7f344c152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:19.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3450a5f640 1 -- 172.21.15.39:0/341539137 >> v1:172.21.15.39:6789/0 conn(0x7f344c14bf30 legacy=0x7f344c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:19.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3450a5f640 1 -- 172.21.15.39:0/341539137 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c155c20 con 0x7f344c155160 2024-09-13T16:50:19.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3450a5f640 1 --2- 172.21.15.39:0/341539137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 0x7f344c1520f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f343c0097c0 tx=0x7f343c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb2c7dd05f40dcf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:19.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3443fff640 1 -- 172.21.15.39:0/341539137 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0ab650 con 0x7f344c155160 2024-09-13T16:50:19.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3443fff640 1 -- 172.21.15.39:0/341539137 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c0ab7b0 con 0x7f344c155160 2024-09-13T16:50:19.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.965+0000 7f3443fff640 1 -- 172.21.15.39:0/341539137 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0abb40 con 0x7f344c155160 2024-09-13T16:50:19.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 -- 172.21.15.39:0/341539137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 msgr2=0x7f344c1520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:19.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 --2- 172.21.15.39:0/341539137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 0x7f344c1520f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f343c0097c0 tx=0x7f343c01c470 comp rx=0 tx=0).stop 2024-09-13T16:50:19.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 -- 172.21.15.39:0/341539137 shutdown_connections 2024-09-13T16:50:19.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 --2- 172.21.15.39:0/341539137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 0x7f344c1520f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:19.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 -- 172.21.15.39:0/341539137 >> 172.21.15.39:0/341539137 conn(0x7f344c0c32a0 msgr2=0x7f344c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:19.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 -- 172.21.15.39:0/341539137 shutdown_connections 2024-09-13T16:50:19.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 -- 172.21.15.39:0/341539137 wait complete. 2024-09-13T16:50:19.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 Processor -- start 2024-09-13T16:50:19.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 -- start start 2024-09-13T16:50:19.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344c0a5ef0 0x7f344c13f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:19.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f344c14bf30 0x7f344c13f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:19.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 0x7f344c1ee390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:19.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c14d8d0 con 0x7f344c0a5ef0 2024-09-13T16:50:19.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c145b30 con 0x7f344c155160 2024-09-13T16:50:19.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f34534c4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c1452d0 con 0x7f344c14bf30 2024-09-13T16:50:19.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451260640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344c0a5ef0 0x7f344c13f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:19.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451260640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344c0a5ef0 0x7f344c13f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50826/0 (socket says 172.21.15.39:50826) 2024-09-13T16:50:19.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451260640 1 -- 172.21.15.39:0/1883497747 learned_addr learned my addr 172.21.15.39:0/1883497747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:19.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451a61640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 0x7f344c1ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:19.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3450a5f640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f344c14bf30 0x7f344c13f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:19.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451260640 1 -- 172.21.15.39:0/1883497747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f344c14bf30 msgr2=0x7f344c13f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:19.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451260640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f344c14bf30 0x7f344c13f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:19.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451260640 1 -- 172.21.15.39:0/1883497747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 msgr2=0x7f344c1ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:19.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451260640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 0x7f344c1ee390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:19.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451260640 1 -- 172.21.15.39:0/1883497747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c009420 con 0x7f344c0a5ef0 2024-09-13T16:50:19.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451a61640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 0x7f344c1ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:19.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.969+0000 7f3451260640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344c0a5ef0 0x7f344c13f490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f344800c420 tx=0x7f344800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:19.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f3441ffb640 1 -- 172.21.15.39:0/1883497747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448005cf0 con 0x7f344c0a5ef0 2024-09-13T16:50:19.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f3441ffb640 1 -- 172.21.15.39:0/1883497747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3448005e50 con 0x7f344c0a5ef0 2024-09-13T16:50:19.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c1ef110 con 0x7f344c0a5ef0 2024-09-13T16:50:19.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f3441ffb640 1 -- 172.21.15.39:0/1883497747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34480072d0 con 0x7f344c0a5ef0 2024-09-13T16:50:19.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c145850 con 0x7f344c0a5ef0 2024-09-13T16:50:19.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c155660 con 0x7f344c0a5ef0 2024-09-13T16:50:19.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f3441ffb640 1 -- 172.21.15.39:0/1883497747 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3448007430 con 0x7f344c0a5ef0 2024-09-13T16:50:19.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f3441ffb640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3430066690 0x7f3430068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:19.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f3441ffb640 1 -- 172.21.15.39:0/1883497747 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f3448083b30 con 0x7f344c0a5ef0 2024-09-13T16:50:19.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f3450a5f640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3430066690 0x7f3430068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:19.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.973+0000 7f3450a5f640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3430066690 0x7f3430068b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f343c0af000 tx=0x7f343c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:19.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:19.981+0000 7f3441ffb640 1 -- 172.21.15.39:0/1883497747 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3448050af0 con 0x7f344c0a5ef0 2024-09-13T16:50:20.049 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-13T16:50:20.061 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-13T16:50:20.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.229+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f344c0c0ad0 con 0x7f344c0a5ef0 2024-09-13T16:50:20.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.229+0000 7f3441ffb640 1 -- 172.21.15.39:0/1883497747 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5172 (secure 0 0 0) 0x7f3448017090 con 0x7f344c0a5ef0 2024-09-13T16:50:20.235 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:20.235 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:20.235 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 24 2024-09-13T16:50:20.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3430066690 msgr2=0x7f3430068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:20.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3430066690 0x7f3430068b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f343c0af000 tx=0x7f343c0a7000 comp rx=0 tx=0).stop 2024-09-13T16:50:20.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344c0a5ef0 msgr2=0x7f344c13f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:20.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344c0a5ef0 0x7f344c13f490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f344800c420 tx=0x7f344800c8e0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 shutdown_connections 2024-09-13T16:50:20.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3430066690 0x7f3430068b50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f344c155160 0x7f344c1ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f344c14bf30 0x7f344c13f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 --2- 172.21.15.39:0/1883497747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344c0a5ef0 0x7f344c13f490 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 >> 172.21.15.39:0/1883497747 conn(0x7f344c0c32a0 msgr2=0x7f344c1520f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:20.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 shutdown_connections 2024-09-13T16:50:20.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.233+0000 7f34534c4640 1 -- 172.21.15.39:0/1883497747 wait complete. 2024-09-13T16:50:20.255 INFO:tasks.mds_thrash.fs.[b]:mds.a has gained rank=0, replacing gid=4232 2024-09-13T16:50:20.255 INFO:tasks.mds_thrash.fs.[b]:waiting for 0 secs before reviving mds.d 2024-09-13T16:50:20.255 INFO:tasks.mds_thrash.fs.[b]:reviving mds.d 2024-09-13T16:50:20.256 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:50:20.256 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:50:20.258 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:50:20.259 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:20.333 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T16:50:20.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.365+0000 7f4e4f4d0640 1 Processor -- start 2024-09-13T16:50:20.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.365+0000 7f4e4f4d0640 1 -- start start 2024-09-13T16:50:20.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4f4d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:20.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4f4d0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4814f590 con 0x7f4e480c7470 2024-09-13T16:50:20.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4f4d0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4806eae0 con 0x7f4e4814f1b0 2024-09-13T16:50:20.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4f4d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4806f660 con 0x7f4e4806e5e0 2024-09-13T16:50:20.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4da6d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4e480c7470 0x7f4e480c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:55672/0 (socket says 172.21.15.39:55672) 2024-09-13T16:50:20.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4da6d640 1 -- 172.21.15.39:0/1405460003 learned_addr learned my addr 172.21.15.39:0/1405460003 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:20.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4ca6b640 1 --2- 172.21.15.39:0/1405460003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:20.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e3ffff640 1 -- 172.21.15.39:0/1405460003 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693806281 0 0) 0x7f4e4814f590 con 0x7f4e480c7470 2024-09-13T16:50:20.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e3ffff640 1 -- 172.21.15.39:0/1405460003 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e34003530 con 0x7f4e480c7470 2024-09-13T16:50:20.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4ca6b640 1 -- 172.21.15.39:0/1405460003 >> v1:172.21.15.84:6789/0 conn(0x7f4e4814f1b0 legacy=0x7f4e480714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:20.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4ca6b640 1 -- 172.21.15.39:0/1405460003 >> v1:172.21.15.39:6789/0 conn(0x7f4e480c7470 legacy=0x7f4e480c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:20.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e3ffff640 1 -- 172.21.15.39:0/1405460003 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986524898 0 0) 0x7f4e4806eae0 con 0x7f4e4814f1b0 2024-09-13T16:50:20.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4ca6b640 1 -- 172.21.15.39:0/1405460003 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4806edc0 con 0x7f4e4806e5e0 2024-09-13T16:50:20.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e4ca6b640 1 --2- 172.21.15.39:0/1405460003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48151bb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e380097f0 tx=0x7f4e3801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1176c42bd2b78c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:20.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e3ffff640 1 -- 172.21.15.39:0/1405460003 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380aa610 con 0x7f4e4806e5e0 2024-09-13T16:50:20.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e3ffff640 1 -- 172.21.15.39:0/1405460003 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e380a6040 con 0x7f4e4806e5e0 2024-09-13T16:50:20.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.369+0000 7f4e3ffff640 1 -- 172.21.15.39:0/1405460003 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380aa990 con 0x7f4e4806e5e0 2024-09-13T16:50:20.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/1405460003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 msgr2=0x7f4e48151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:20.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 --2- 172.21.15.39:0/1405460003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48151bb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e380097f0 tx=0x7f4e3801c4b0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/1405460003 shutdown_connections 2024-09-13T16:50:20.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 --2- 172.21.15.39:0/1405460003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48151bb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/1405460003 >> 172.21.15.39:0/1405460003 conn(0x7f4e480a9720 msgr2=0x7f4e480a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:20.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/1405460003 shutdown_connections 2024-09-13T16:50:20.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/1405460003 wait complete. 2024-09-13T16:50:20.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 Processor -- start 2024-09-13T16:50:20.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- start start 2024-09-13T16:50:20.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:20.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e480c7470 0x7f4e480acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:20.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4814f1b0 0x7f4e481ebd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:20.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48147de0 con 0x7f4e480c7470 2024-09-13T16:50:20.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e481edf00 con 0x7f4e4814f1b0 2024-09-13T16:50:20.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e481ee1e0 con 0x7f4e4806e5e0 2024-09-13T16:50:20.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4ca6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e480c7470 0x7f4e480acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4ca6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e480c7470 0x7f4e480acc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55266/0 (socket says 172.21.15.39:55266) 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4ca6b640 1 -- 172.21.15.39:0/4054098495 learned_addr learned my addr 172.21.15.39:0/4054098495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4d26c640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4da6d640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4814f1b0 0x7f4e481ebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4ca6b640 1 -- 172.21.15.39:0/4054098495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 msgr2=0x7f4e48155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4ca6b640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4ca6b640 1 -- 172.21.15.39:0/4054098495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4814f1b0 msgr2=0x7f4e481ebd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4ca6b640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4814f1b0 0x7f4e481ebd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4ca6b640 1 -- 172.21.15.39:0/4054098495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38009420 con 0x7f4e480c7470 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4d26c640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4ca6b640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e480c7470 0x7f4e480acc00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4e380097f0 tx=0x7f4e3801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e3dffb640 1 -- 172.21.15.39:0/4054098495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380be410 con 0x7f4e480c7470 2024-09-13T16:50:20.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e3dffb640 1 -- 172.21.15.39:0/4054098495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e380be590 con 0x7f4e480c7470 2024-09-13T16:50:20.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e481ee4c0 con 0x7f4e480c7470 2024-09-13T16:50:20.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e3dffb640 1 -- 172.21.15.39:0/4054098495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380be940 con 0x7f4e480c7470 2024-09-13T16:50:20.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.373+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e48155f40 con 0x7f4e480c7470 2024-09-13T16:50:20.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.377+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e481ee9e0 con 0x7f4e480c7470 2024-09-13T16:50:20.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.377+0000 7f4e3dffb640 1 -- 172.21.15.39:0/4054098495 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4e380b4400 con 0x7f4e480c7470 2024-09-13T16:50:20.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.377+0000 7f4e3dffb640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4e2c0666c0 0x7f4e2c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:20.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.377+0000 7f4e3dffb640 1 -- 172.21.15.39:0/4054098495 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f4e3811f0d0 con 0x7f4e480c7470 2024-09-13T16:50:20.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.377+0000 7f4e4d26c640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4e2c0666c0 0x7f4e2c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:20.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.377+0000 7f4e4d26c640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4e2c0666c0 0x7f4e2c068b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4e44007fd0 tx=0x7f4e44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:20.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.381+0000 7f4e3dffb640 1 -- 172.21.15.39:0/4054098495 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4e481ee9e0 con 0x7f4e480c7470 2024-09-13T16:50:20.407 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-13T16:50:20.417 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-13T16:50:20.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e481ee9e0 con 0x7f4e480c7470 2024-09-13T16:50:20.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e3dffb640 1 -- 172.21.15.39:0/4054098495 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5172 (secure 0 0 0) 0x7f4e481ee9e0 con 0x7f4e480c7470 2024-09-13T16:50:20.620 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:20.621 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:20.621 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 24 2024-09-13T16:50:20.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4e2c0666c0 msgr2=0x7f4e2c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:20.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4e2c0666c0 0x7f4e2c068b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4e44007fd0 tx=0x7f4e44009040 comp rx=0 tx=0).stop 2024-09-13T16:50:20.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e480c7470 msgr2=0x7f4e480acc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:20.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e480c7470 0x7f4e480acc00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4e380097f0 tx=0x7f4e3801acf0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 shutdown_connections 2024-09-13T16:50:20.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4e2c0666c0 0x7f4e2c068b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e4814f1b0 0x7f4e481ebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e480c7470 0x7f4e480acc00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 --2- 172.21.15.39:0/4054098495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4e4806e5e0 0x7f4e48155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:20.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 >> 172.21.15.39:0/4054098495 conn(0x7f4e480a9720 msgr2=0x7f4e480a9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:20.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 shutdown_connections 2024-09-13T16:50:20.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:20.617+0000 7f4e4f4d0640 1 -- 172.21.15.39:0/4054098495 wait complete. 2024-09-13T16:50:20.645 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-09-13T16:50:20.667 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-13T16:50:20.679 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-13T16:50:21.096 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-13T16:50:21.108 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-13T16:50:21.340 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-13T16:50:21.354 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-13T16:50:21.499 INFO:tasks.workunit.client.1.smithi084.stderr:ffsb_op.c: In function 'print_op_results': 2024-09-13T16:50:21.499 INFO:tasks.workunit.client.1.smithi084.stderr:ffsb_op.c:83:37: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-13T16:50:21.499 INFO:tasks.workunit.client.1.smithi084.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-13T16:50:21.499 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~ 2024-09-13T16:50:21.499 INFO:tasks.workunit.client.1.smithi084.stderr:ffsb_op.c:86:33: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-13T16:50:21.499 INFO:tasks.workunit.client.1.smithi084.stderr: 86 | sprintf(buf, "NA\0"); 2024-09-13T16:50:21.500 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~ 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr:ffsb_op.c: In function 'print_results': 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr:ffsb_op.c:83:33: warning: '/sec' directive writing 4 bytes into a region of size between 1 and 256 [-Wformat-overflow=] 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~ 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr:In file included from /usr/include/stdio.h:894, 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr: from ffsb_op.c:20: 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr:/usr/include/x86_64-linux-gnu/bits/stdio2.h:38:10: note: '__builtin___sprintf_chk' output between 5 and 260 bytes into a destination of size 256 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr: 38 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr: 39 | __glibc_objsize (__s), __fmt, 2024-09-13T16:50:21.516 INFO:tasks.workunit.client.1.smithi084.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:21.517 INFO:tasks.workunit.client.1.smithi084.stderr: 40 | __va_arg_pack ()); 2024-09-13T16:50:21.517 INFO:tasks.workunit.client.1.smithi084.stderr: | ~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:21.570 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-13T16:50:21.583 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-13T16:50:21.900 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-13T16:50:21.911 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-13T16:50:22.071 INFO:tasks.workunit.client.1.smithi084.stderr:In file included from parser.h:22, 2024-09-13T16:50:22.071 INFO:tasks.workunit.client.1.smithi084.stderr: from parser.c:25: 2024-09-13T16:50:22.072 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c: In function 'init_filesys': 2024-09-13T16:50:22.073 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:22.073 INFO:tasks.workunit.client.1.smithi084.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:22.073 INFO:tasks.workunit.client.1.smithi084.stderr: | ^ 2024-09-13T16:50:22.073 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:22.076 INFO:tasks.workunit.client.1.smithi084.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:22.076 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~ 2024-09-13T16:50:22.076 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:22.076 INFO:tasks.workunit.client.1.smithi084.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:22.076 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:934:17: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr: | ^ 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~ 2024-09-13T16:50:22.077 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:934:17: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr: | ^ 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:22.078 INFO:tasks.workunit.client.1.smithi084.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~ 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:941:17: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:22.079 INFO:tasks.workunit.client.1.smithi084.stderr: | ^ 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~ 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:941:17: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-13T16:50:22.080 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.081 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c: In function 'init_tg_stats': 2024-09-13T16:50:22.081 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:22.081 INFO:tasks.workunit.client.1.smithi084.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:22.081 INFO:tasks.workunit.client.1.smithi084.stderr: | ^ 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~ 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:971:41: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.082 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr: | ^ 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~ 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:971:41: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:22.083 INFO:tasks.workunit.client.1.smithi084.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr: | ^ 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~ 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T16:50:22.084 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:980:41: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr: | ^ 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~ 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr: 36 | container_of(ptr, type, member) 2024-09-13T16:50:22.085 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~ 2024-09-13T16:50:22.086 INFO:tasks.workunit.client.1.smithi084.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-13T16:50:22.086 INFO:tasks.workunit.client.1.smithi084.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T16:50:22.086 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~ 2024-09-13T16:50:22.086 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:980:41: note: in expansion of macro 'list_for_each_entry' 2024-09-13T16:50:22.086 INFO:tasks.workunit.client.1.smithi084.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-13T16:50:22.086 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.135 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c: In function 'get_optstr': 2024-09-13T16:50:22.135 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:206:23: warning: 'strnlen' specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-13T16:50:22.135 INFO:tasks.workunit.client.1.smithi084.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-13T16:50:22.135 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.135 INFO:tasks.workunit.client.1.smithi084.stderr:parser.c:200:14: note: source object declared here 2024-09-13T16:50:22.135 INFO:tasks.workunit.client.1.smithi084.stderr: 200 | char temp[BUFSIZE]; 2024-09-13T16:50:22.135 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~ 2024-09-13T16:50:22.152 INFO:tasks.workunit.client.1.smithi084.stderr:In file included from /usr/include/string.h:535, 2024-09-13T16:50:22.152 INFO:tasks.workunit.client.1.smithi084.stderr: from parser.c:19: 2024-09-13T16:50:22.152 INFO:tasks.workunit.client.1.smithi084.stderr:In function 'memcpy', 2024-09-13T16:50:22.152 INFO:tasks.workunit.client.1.smithi084.stderr: inlined from 'size64_convert' at parser.c:119:2: 2024-09-13T16:50:22.152 INFO:tasks.workunit.client.1.smithi084.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10: warning: '__builtin_memcpy' reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-13T16:50:22.152 INFO:tasks.workunit.client.1.smithi084.stderr: 29 | return __builtin___memcpy_chk (__dest, __src, __len, 2024-09-13T16:50:22.153 INFO:tasks.workunit.client.1.smithi084.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.153 INFO:tasks.workunit.client.1.smithi084.stderr: 30 | __glibc_objsize0 (__dest)); 2024-09-13T16:50:22.153 INFO:tasks.workunit.client.1.smithi084.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T16:50:22.495 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-13T16:50:22.509 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-13T16:50:22.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:22.733 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-13T16:50:22.744 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-13T16:50:22.774 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T16:50:22.769+0000 7fdb7e6fa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:50:22.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.777+0000 7f9cf220f640 1 Processor -- start 2024-09-13T16:50:22.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.777+0000 7f9cf220f640 1 -- start start 2024-09-13T16:50:22.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c0240 0x7f9cec0c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:22.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9cec157d50 0x7f9cec06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:22.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec0c6510 con 0x7f9cec0c6130 2024-09-13T16:50:22.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec0c1c80 con 0x7f9cec157d50 2024-09-13T16:50:22.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec1544f0 con 0x7f9cec0c0240 2024-09-13T16:50:22.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cebfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9cec157d50 0x7f9cec06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:22.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9ceb7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9cec0c6130 0x7f9cec153de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:55680/0 (socket says 172.21.15.39:55680) 2024-09-13T16:50:22.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9ceb7fe640 1 -- 172.21.15.39:0/266955477 learned_addr learned my addr 172.21.15.39:0/266955477 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:22.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9ce3fff640 1 --2- 172.21.15.39:0/266955477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c0240 0x7f9cec0c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:22.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cebfff640 1 -- 172.21.15.39:0/266955477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c0240 msgr2=0x7f9cec0c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:22.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cebfff640 1 --2- 172.21.15.39:0/266955477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c0240 0x7f9cec0c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:22.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cebfff640 1 -- 172.21.15.39:0/266955477 >> v1:172.21.15.39:6789/0 conn(0x7f9cec0c6130 legacy=0x7f9cec153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:22.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9ceaffd640 1 -- 172.21.15.39:0/266955477 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750021298 0 0) 0x7f9cdc001120 con 0x7f9cec0c6130 2024-09-13T16:50:22.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cebfff640 1 -- 172.21.15.39:0/266955477 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec1581e0 con 0x7f9cec157d50 2024-09-13T16:50:22.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cebfff640 1 --2- 172.21.15.39:0/266955477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 0x7f9cec06ae20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009d90 tx=0x7f9ce401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935a56b3527d4d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:22.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9ceaffd640 1 -- 172.21.15.39:0/266955477 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40a5a50 con 0x7f9cec157d50 2024-09-13T16:50:22.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9ceaffd640 1 -- 172.21.15.39:0/266955477 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce40a5bb0 con 0x7f9cec157d50 2024-09-13T16:50:22.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9ceaffd640 1 -- 172.21.15.39:0/266955477 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40ad440 con 0x7f9cec157d50 2024-09-13T16:50:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 -- 172.21.15.39:0/266955477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 msgr2=0x7f9cec06ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 --2- 172.21.15.39:0/266955477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 0x7f9cec06ae20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009d90 tx=0x7f9ce401c970 comp rx=0 tx=0).stop 2024-09-13T16:50:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 -- 172.21.15.39:0/266955477 shutdown_connections 2024-09-13T16:50:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 --2- 172.21.15.39:0/266955477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 0x7f9cec06ae20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 --2- 172.21.15.39:0/266955477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c0240 0x7f9cec0c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:22.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 -- 172.21.15.39:0/266955477 >> 172.21.15.39:0/266955477 conn(0x7f9cec0c32a0 msgr2=0x7f9cec0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:22.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 -- 172.21.15.39:0/266955477 shutdown_connections 2024-09-13T16:50:22.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.781+0000 7f9cf220f640 1 -- 172.21.15.39:0/266955477 wait complete. 2024-09-13T16:50:22.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 Processor -- start 2024-09-13T16:50:22.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 -- start start 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cec0c0240 0x7f9cec06dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c6130 0x7f9cec15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 0x7f9cec15c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec158910 con 0x7f9cec157d50 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec159640 con 0x7f9cec0c0240 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec15c870 con 0x7f9cec0c6130 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cebfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 0x7f9cec15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cebfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 0x7f9cec15c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55276/0 (socket says 172.21.15.39:55276) 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cebfff640 1 -- 172.21.15.39:0/4125480434 learned_addr learned my addr 172.21.15.39:0/4125480434 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9ceb7fe640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cec0c0240 0x7f9cec06dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9ce3fff640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c6130 0x7f9cec15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cebfff640 1 -- 172.21.15.39:0/4125480434 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c6130 msgr2=0x7f9cec15bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cebfff640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c6130 0x7f9cec15bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cebfff640 1 -- 172.21.15.39:0/4125480434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cec0c0240 msgr2=0x7f9cec06dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cebfff640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cec0c0240 0x7f9cec06dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cebfff640 1 -- 172.21.15.39:0/4125480434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4009a70 con 0x7f9cec157d50 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9ceb7fe640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cec0c0240 0x7f9cec06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cebfff640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 0x7f9cec15c330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9ce401b020 tx=0x7f9ce4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9ce8ff9640 1 -- 172.21.15.39:0/4125480434 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40addc0 con 0x7f9cec157d50 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9ce8ff9640 1 -- 172.21.15.39:0/4125480434 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce4009450 con 0x7f9cec157d50 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9ce8ff9640 1 -- 172.21.15.39:0/4125480434 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40b9650 con 0x7f9cec157d50 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec0ab920 con 0x7f9cec157d50 2024-09-13T16:50:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.785+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec1ecab0 con 0x7f9cec157d50 2024-09-13T16:50:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.789+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec06a8c0 con 0x7f9cec157d50 2024-09-13T16:50:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.789+0000 7f9ce8ff9640 1 -- 172.21.15.39:0/4125480434 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ce40b97b0 con 0x7f9cec157d50 2024-09-13T16:50:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.789+0000 7f9ce8ff9640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9ccc066690 0x7f9ccc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.789+0000 7f9ce8ff9640 1 -- 172.21.15.39:0/4125480434 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f9ce411fa80 con 0x7f9cec157d50 2024-09-13T16:50:22.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.789+0000 7f9ceb7fe640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9ccc066690 0x7f9ccc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:22.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.789+0000 7f9ceb7fe640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9ccc066690 0x7f9ccc068b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc000f70 tx=0x7f9cdc007a60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:22.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:22.793+0000 7f9ce8ff9640 1 -- 172.21.15.39:0/4125480434 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ce40ecb50 con 0x7f9cec157d50 2024-09-13T16:50:22.956 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-13T16:50:22.971 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-13T16:50:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.057+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cec0c24c0 con 0x7f9cec157d50 2024-09-13T16:50:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.057+0000 7f9ce8ff9640 1 -- 172.21.15.39:0/4125480434 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f9ce40f0170 con 0x7f9cec157d50 2024-09-13T16:50:23.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:23.063 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 25 2024-09-13T16:50:23.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9ccc066690 msgr2=0x7f9ccc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9ccc066690 0x7f9ccc068b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc000f70 tx=0x7f9cdc007a60 comp rx=0 tx=0).stop 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 msgr2=0x7f9cec15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 0x7f9cec15c330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9ce401b020 tx=0x7f9ce4019710 comp rx=0 tx=0).stop 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 shutdown_connections 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9ccc066690 0x7f9ccc068b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cec157d50 0x7f9cec15c330 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9cec0c6130 0x7f9cec15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 --2- 172.21.15.39:0/4125480434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9cec0c0240 0x7f9cec06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 >> 172.21.15.39:0/4125480434 conn(0x7f9cec0c32a0 msgr2=0x7f9cec06aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 shutdown_connections 2024-09-13T16:50:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:23.061+0000 7f9cf220f640 1 -- 172.21.15.39:0/4125480434 wait complete. 2024-09-13T16:50:23.072 INFO:tasks.workunit.client.1.smithi084.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-13T16:50:23.086 INFO:tasks.mds_thrash.fs.[b]:mds.d reported in up:standby state 2024-09-13T16:50:23.087 INFO:tasks.mds_thrash.fs.[b]:waiting for 35 secs before thrashing 2024-09-13T16:50:23.088 INFO:tasks.workunit.client.1.smithi084.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-13T16:50:23.255 INFO:tasks.workunit.client.1.smithi084.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.1/client.1/tmp/ffsb' 2024-09-13T16:50:23.263 INFO:tasks.workunit.client.1.smithi084.stdout:FFSB version 6.0-RC2 started 2024-09-13T16:50:23.263 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T16:50:23.266 INFO:tasks.workunit.client.1.smithi084.stdout:benchmark time = 300 2024-09-13T16:50:23.266 INFO:tasks.workunit.client.1.smithi084.stdout:ThreadGroup 0 2024-09-13T16:50:23.266 INFO:tasks.workunit.client.1.smithi084.stdout:================ 2024-09-13T16:50:23.266 INFO:tasks.workunit.client.1.smithi084.stdout: num_threads = 32 2024-09-13T16:50:23.266 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T16:50:23.266 INFO:tasks.workunit.client.1.smithi084.stdout: read_random = off 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: read_size = 0 (0B) 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: read_blocksize = 0 (0B) 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: read_skip = off 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: read_skipsize = 0 (0B) 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: write_random = on 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: write_size = 5242880 (5MB) 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: fsync_file = 0 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: write_blocksize = 4096 (4KB) 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: wait time = 0 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: op weights 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: read = 0 (0.00%) 2024-09-13T16:50:23.267 INFO:tasks.workunit.client.1.smithi084.stdout: readall = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: write = 1 (100.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: create = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: append = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: delete = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: metaop = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: createdir = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: stat = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: writeall = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: writeall_fsync = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: open_close = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: write_fsync = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: create_fsync = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: append_fsync = 0 (0.00%) 2024-09-13T16:50:23.268 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout:FileSystem . 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout:========== 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: num_dirs = 0 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: starting files = 128 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: min file size = 104857600 (100MB) 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: max file size = 104857600 (100MB) 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: directio = off 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: alignedio = on 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: bufferedio = off 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T16:50:23.269 INFO:tasks.workunit.client.1.smithi084.stdout: aging is off 2024-09-13T16:50:23.270 INFO:tasks.workunit.client.1.smithi084.stdout: current utilization = 0.00% 2024-09-13T16:50:23.270 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T16:50:23.270 INFO:tasks.workunit.client.1.smithi084.stdout:checking existing fs: . 2024-09-13T16:50:23.270 INFO:tasks.workunit.client.1.smithi084.stdout:recreating new fileset 2024-09-13T16:50:23.270 INFO:tasks.workunit.client.1.smithi084.stderr:opendir: No such file or directory 2024-09-13T16:50:25.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:50:25.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:50:33.720 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f060bdb8640 1 Processor -- start 2024-09-13T16:50:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f060bdb8640 1 -- start start 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f060bdb8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f060406cbc0 0x7f06040bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f060bdb8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f06040611c0 0x7f06041545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f060bdb8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040c00b0 con 0x7f06040c2db0 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f060bdb8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0604153e70 con 0x7f060406cbc0 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f060bdb8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040c0300 con 0x7f06040611c0 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f0609353640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f060406cbc0 0x7f06040bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f0609353640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f060406cbc0 0x7f06040bf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47980/0 (socket says 172.21.15.39:47980) 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f0609353640 1 -- 172.21.15.39:0/362414188 learned_addr learned my addr 172.21.15.39:0/362414188 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f060a355640 1 --2- 172.21.15.39:0/362414188 >> v2:172.21.15.84:3300/0 conn(0x7f06040611c0 0x7f06041545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f0609353640 1 -- 172.21.15.39:0/362414188 >> v2:172.21.15.84:3300/0 conn(0x7f06040611c0 msgr2=0x7f06041545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:33.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f0609353640 1 --2- 172.21.15.39:0/362414188 >> v2:172.21.15.84:3300/0 conn(0x7f06040611c0 0x7f06041545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f0609353640 1 -- 172.21.15.39:0/362414188 >> v1:172.21.15.84:6789/0 conn(0x7f06040c2db0 legacy=0x7f06041512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.869+0000 7f0609353640 1 -- 172.21.15.39:0/362414188 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040c05e0 con 0x7f060406cbc0 2024-09-13T16:50:33.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f0609353640 1 --2- 172.21.15.39:0/362414188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060406cbc0 0x7f06040bf4f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f060406fad0 tx=0x7f05f401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b538a5525efbc86d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:33.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f0608b52640 1 -- 172.21.15.39:0/362414188 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40ab600 con 0x7f060406cbc0 2024-09-13T16:50:33.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f0608b52640 1 -- 172.21.15.39:0/362414188 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f40a7050 con 0x7f060406cbc0 2024-09-13T16:50:33.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f0608b52640 1 -- 172.21.15.39:0/362414188 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40ab980 con 0x7f060406cbc0 2024-09-13T16:50:33.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f060bdb8640 1 -- 172.21.15.39:0/362414188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060406cbc0 msgr2=0x7f06040bf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f060bdb8640 1 --2- 172.21.15.39:0/362414188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060406cbc0 0x7f06040bf4f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f060406fad0 tx=0x7f05f401c400 comp rx=0 tx=0).stop 2024-09-13T16:50:33.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f060bdb8640 1 -- 172.21.15.39:0/362414188 shutdown_connections 2024-09-13T16:50:33.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f060bdb8640 1 --2- 172.21.15.39:0/362414188 >> v2:172.21.15.84:3300/0 conn(0x7f06040611c0 0x7f06041545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f060bdb8640 1 --2- 172.21.15.39:0/362414188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f060406cbc0 0x7f06040bf4f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f060bdb8640 1 -- 172.21.15.39:0/362414188 >> 172.21.15.39:0/362414188 conn(0x7f0604069e10 msgr2=0x7f060406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:33.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f060bdb8640 1 -- 172.21.15.39:0/362414188 shutdown_connections 2024-09-13T16:50:33.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.873+0000 7f060bdb8640 1 -- 172.21.15.39:0/362414188 wait complete. 2024-09-13T16:50:33.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 Processor -- start 2024-09-13T16:50:33.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 -- start start 2024-09-13T16:50:33.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06040611c0 0x7f06041f0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060406cbc0 0x7f06041ed240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06040c2db0 0x7f0604150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06041ed780 con 0x7f06040c2db0 2024-09-13T16:50:33.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06041f0380 con 0x7f06040611c0 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06040c59f0 con 0x7f060406cbc0 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060a355640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06040c2db0 0x7f0604150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060a355640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06040c2db0 0x7f0604150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47988/0 (socket says 172.21.15.39:47988) 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060a355640 1 -- 172.21.15.39:0/3727469250 learned_addr learned my addr 172.21.15.39:0/3727469250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f0609b54640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06040611c0 0x7f06041f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f0609353640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060406cbc0 0x7f06041ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060a355640 1 -- 172.21.15.39:0/3727469250 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060406cbc0 msgr2=0x7f06041ed240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060a355640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060406cbc0 0x7f06041ed240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060a355640 1 -- 172.21.15.39:0/3727469250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06040611c0 msgr2=0x7f06041f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060a355640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06040611c0 0x7f06041f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060a355640 1 -- 172.21.15.39:0/3727469250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f40094d0 con 0x7f06040c2db0 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f0609353640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060406cbc0 0x7f06041ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060a355640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06040c2db0 0x7f0604150220 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f060000d590 tx=0x7f060000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f05fa7fc640 1 -- 172.21.15.39:0/3727469250 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06000125f0 con 0x7f06040c2db0 2024-09-13T16:50:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f05fa7fc640 1 -- 172.21.15.39:0/3727469250 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600012770 con 0x7f06040c2db0 2024-09-13T16:50:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f05fa7fc640 1 -- 172.21.15.39:0/3727469250 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600012b20 con 0x7f06040c2db0 2024-09-13T16:50:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06040c5cd0 con 0x7f06040c2db0 2024-09-13T16:50:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06041f0010 con 0x7f06040c2db0 2024-09-13T16:50:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06041f0510 con 0x7f06040c2db0 2024-09-13T16:50:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.877+0000 7f05fa7fc640 1 -- 172.21.15.39:0/3727469250 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0600012c80 con 0x7f06040c2db0 2024-09-13T16:50:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.881+0000 7f05fa7fc640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05e8066690 0x7f05e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.881+0000 7f05fa7fc640 1 -- 172.21.15.39:0/3727469250 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0600084a00 con 0x7f06040c2db0 2024-09-13T16:50:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.881+0000 7f0609b54640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05e8066690 0x7f05e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.881+0000 7f0609b54640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05e8066690 0x7f05e8068b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0012f0 tx=0x7f05fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:33.885+0000 7f05fa7fc640 1 -- 172.21.15.39:0/3727469250 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f06000519b0 con 0x7f06040c2db0 2024-09-13T16:50:34.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.081+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06041545e0 con 0x7f06040c2db0 2024-09-13T16:50:34.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.081+0000 7f05fa7fc640 1 -- 172.21.15.39:0/3727469250 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f0600054fd0 con 0x7f06040c2db0 2024-09-13T16:50:34.086 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:34.086 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:34.087 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 25 2024-09-13T16:50:34.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05e8066690 msgr2=0x7f05e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05e8066690 0x7f05e8068b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0012f0 tx=0x7f05fc009040 comp rx=0 tx=0).stop 2024-09-13T16:50:34.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06040c2db0 msgr2=0x7f0604150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06040c2db0 0x7f0604150220 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f060000d590 tx=0x7f060000da50 comp rx=0 tx=0).stop 2024-09-13T16:50:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 shutdown_connections 2024-09-13T16:50:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05e8066690 0x7f05e8068b50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06040c2db0 0x7f0604150220 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f060406cbc0 0x7f06041ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 --2- 172.21.15.39:0/3727469250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06040611c0 0x7f06041f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 >> 172.21.15.39:0/3727469250 conn(0x7f0604069e10 msgr2=0x7f060415df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 shutdown_connections 2024-09-13T16:50:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.085+0000 7f060bdb8640 1 -- 172.21.15.39:0/3727469250 wait complete. 2024-09-13T16:50:34.106 INFO:tasks.mds_thrash.fs.[a]:kill mds.b (rank=0) 2024-09-13T16:50:34.106 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:50:34.106 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:50:34.137 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:50:34.138 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.253+0000 7ff9f1d9c640 1 Processor -- start 2024-09-13T16:50:34.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.253+0000 7ff9f1d9c640 1 -- start start 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.253+0000 7ff9f1d9c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff9ec1556d0 0x7ff9ec0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9f1d9c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff9ec0c7610 0x7ff9ec151b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9f1d9c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ec06feb0 con 0x7ff9ec0a72b0 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9f1d9c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ec0aae50 con 0x7ff9ec1556d0 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9f1d9c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ec155ab0 con 0x7ff9ec0c7610 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9eb7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff9ec1556d0 0x7ff9ec0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9eb7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff9ec1556d0 0x7ff9ec0c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47992/0 (socket says 172.21.15.39:47992) 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9ebfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff9ec0a72b0 0x7ff9ec071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42744/0 (socket says 172.21.15.39:42744) 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9eb7fe640 1 -- 172.21.15.39:0/11855673 learned_addr learned my addr 172.21.15.39:0/11855673 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9eaffd640 1 --2- 172.21.15.39:0/11855673 >> v2:172.21.15.84:3300/0 conn(0x7ff9ec0c7610 0x7ff9ec151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9eb7fe640 1 -- 172.21.15.39:0/11855673 >> v2:172.21.15.84:3300/0 conn(0x7ff9ec0c7610 msgr2=0x7ff9ec151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9eb7fe640 1 --2- 172.21.15.39:0/11855673 >> v2:172.21.15.84:3300/0 conn(0x7ff9ec0c7610 0x7ff9ec151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9eb7fe640 1 -- 172.21.15.39:0/11855673 >> v1:172.21.15.39:6789/0 conn(0x7ff9ec0a72b0 legacy=0x7ff9ec071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9eb7fe640 1 -- 172.21.15.39:0/11855673 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0c3c20 con 0x7ff9ec1556d0 2024-09-13T16:50:34.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9ea7fc640 1 -- 172.21.15.39:0/11855673 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14877378 0 0) 0x7ff9ec06feb0 con 0x7ff9ec0a72b0 2024-09-13T16:50:34.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9eb7fe640 1 --2- 172.21.15.39:0/11855673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 0x7ff9ec0c36e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0ab0c0 tx=0x7ff9dc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a17be35b132fa07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:34.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9ea7fc640 1 -- 172.21.15.39:0/11855673 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0ab600 con 0x7ff9ec1556d0 2024-09-13T16:50:34.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9ea7fc640 1 -- 172.21.15.39:0/11855673 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc0a7050 con 0x7ff9ec1556d0 2024-09-13T16:50:34.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.257+0000 7ff9ea7fc640 1 -- 172.21.15.39:0/11855673 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc0ab980 con 0x7ff9ec1556d0 2024-09-13T16:50:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/11855673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 msgr2=0x7ff9ec0c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 --2- 172.21.15.39:0/11855673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 0x7ff9ec0c36e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec0ab0c0 tx=0x7ff9dc01c400 comp rx=0 tx=0).stop 2024-09-13T16:50:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/11855673 shutdown_connections 2024-09-13T16:50:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 --2- 172.21.15.39:0/11855673 >> v2:172.21.15.84:3300/0 conn(0x7ff9ec0c7610 0x7ff9ec151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 --2- 172.21.15.39:0/11855673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 0x7ff9ec0c36e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/11855673 >> 172.21.15.39:0/11855673 conn(0x7ff9ec0a97b0 msgr2=0x7ff9ec0a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/11855673 shutdown_connections 2024-09-13T16:50:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/11855673 wait complete. 2024-09-13T16:50:34.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 Processor -- start 2024-09-13T16:50:34.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- start start 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9ec0a72b0 0x7ff9ec14f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 0x7ff9ec154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9ec153170 0x7ff9ec1ed230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec152d50 con 0x7ff9ec1556d0 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec1544c0 con 0x7ff9ec0a72b0 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0c46a0 con 0x7ff9ec153170 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 0x7ff9ec154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 0x7ff9ec154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47994/0 (socket says 172.21.15.39:47994) 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eaffd640 1 -- 172.21.15.39:0/2374673060 learned_addr learned my addr 172.21.15.39:0/2374673060 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eb7fe640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9ec0a72b0 0x7ff9ec14f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9ebfff640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9ec153170 0x7ff9ec1ed230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eaffd640 1 -- 172.21.15.39:0/2374673060 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9ec153170 msgr2=0x7ff9ec1ed230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eaffd640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9ec153170 0x7ff9ec1ed230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eaffd640 1 -- 172.21.15.39:0/2374673060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9ec0a72b0 msgr2=0x7ff9ec14f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eaffd640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9ec0a72b0 0x7ff9ec14f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eaffd640 1 -- 172.21.15.39:0/2374673060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc0094d0 con 0x7ff9ec1556d0 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9eaffd640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 0x7ff9ec154870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000d5f0 tx=0x7ff9e000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9d3fff640 1 -- 172.21.15.39:0/2374673060 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00125f0 con 0x7ff9ec1556d0 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9d3fff640 1 -- 172.21.15.39:0/2374673060 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e0012750 con 0x7ff9ec1556d0 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ec1edce0 con 0x7ff9ec1556d0 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9d3fff640 1 -- 172.21.15.39:0/2374673060 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0012b00 con 0x7ff9ec1556d0 2024-09-13T16:50:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.261+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec0c4940 con 0x7ff9ec1556d0 2024-09-13T16:50:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.265+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ec152e90 con 0x7ff9ec1556d0 2024-09-13T16:50:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.265+0000 7ff9d3fff640 1 -- 172.21.15.39:0/2374673060 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9e0012c60 con 0x7ff9ec1556d0 2024-09-13T16:50:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.265+0000 7ff9d3fff640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9d8066690 0x7ff9d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.265+0000 7ff9d3fff640 1 -- 172.21.15.39:0/2374673060 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff9e0084b00 con 0x7ff9ec1556d0 2024-09-13T16:50:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.265+0000 7ff9eb7fe640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9d8066690 0x7ff9d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.265+0000 7ff9eb7fe640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9d8066690 0x7ff9d8068b50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0ad000 tx=0x7ff9dc0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:34.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.269+0000 7ff9d3fff640 1 -- 172.21.15.39:0/2374673060 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff9ec152e90 con 0x7ff9ec1556d0 2024-09-13T16:50:34.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9ec152e90 con 0x7ff9ec1556d0 2024-09-13T16:50:34.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9d3fff640 1 -- 172.21.15.39:0/2374673060 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5929 (secure 0 0 0) 0x7ff9ec152e90 con 0x7ff9ec1556d0 2024-09-13T16:50:34.512 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:34.513 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:34.513 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 25 2024-09-13T16:50:34.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9d8066690 msgr2=0x7ff9d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9d8066690 0x7ff9d8068b50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0ad000 tx=0x7ff9dc0a6000 comp rx=0 tx=0).stop 2024-09-13T16:50:34.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 msgr2=0x7ff9ec154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:34.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 0x7ff9ec154870 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000d5f0 tx=0x7ff9e000dab0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 shutdown_connections 2024-09-13T16:50:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9d8066690 0x7ff9d8068b50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9ec153170 0x7ff9ec1ed230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9ec1556d0 0x7ff9ec154870 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.509+0000 7ff9f1d9c640 1 --2- 172.21.15.39:0/2374673060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9ec0a72b0 0x7ff9ec14f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.513+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 >> 172.21.15.39:0/2374673060 conn(0x7ff9ec0a97b0 msgr2=0x7ff9ec06ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.513+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 shutdown_connections 2024-09-13T16:50:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:34.513+0000 7ff9f1d9c640 1 -- 172.21.15.39:0/2374673060 wait complete. 2024-09-13T16:50:34.532 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:50:36.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b51efc640 1 Processor -- start 2024-09-13T16:50:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b51efc640 1 -- start start 2024-09-13T16:50:36.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b51efc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6b4c154bb0 0x7f6b4c0c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b51efc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c153e30 con 0x7f6b4c0c5870 2024-09-13T16:50:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b51efc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c0a7000 con 0x7f6b4c0c2370 2024-09-13T16:50:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b51efc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4c0720f0 con 0x7f6b4c154bb0 2024-09-13T16:50:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b4bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f6b4c0c5870 0x7f6b4c15e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42754/0 (socket says 172.21.15.39:42754) 2024-09-13T16:50:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b4bfff640 1 -- 172.21.15.39:0/180496171 learned_addr learned my addr 172.21.15.39:0/180496171 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b43fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6b4c154bb0 0x7f6b4c0c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b4affd640 1 -- 172.21.15.39:0/180496171 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961254052 0 0) 0x7f6b4c153e30 con 0x7f6b4c0c5870 2024-09-13T16:50:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b4affd640 1 -- 172.21.15.39:0/180496171 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b38003530 con 0x7f6b4c0c5870 2024-09-13T16:50:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b43fff640 1 -- 172.21.15.39:0/180496171 >> v1:172.21.15.84:6789/0 conn(0x7f6b4c0c2370 legacy=0x7f6b4c0c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b43fff640 1 -- 172.21.15.39:0/180496171 >> v1:172.21.15.39:6789/0 conn(0x7f6b4c0c5870 legacy=0x7f6b4c15e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.669+0000 7f6b43fff640 1 -- 172.21.15.39:0/180496171 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c1540d0 con 0x7f6b4c154bb0 2024-09-13T16:50:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b43fff640 1 --2- 172.21.15.39:0/180496171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 0x7f6b4c0c2750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b340097c0 tx=0x7f6b3401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f79f1e30007768e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b4affd640 1 -- 172.21.15.39:0/180496171 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b340a4ea0 con 0x7f6b4c154bb0 2024-09-13T16:50:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b4affd640 1 -- 172.21.15.39:0/180496171 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b340a7050 con 0x7f6b4c154bb0 2024-09-13T16:50:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b4affd640 1 -- 172.21.15.39:0/180496171 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b340ab820 con 0x7f6b4c154bb0 2024-09-13T16:50:36.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b51efc640 1 -- 172.21.15.39:0/180496171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 msgr2=0x7f6b4c0c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b51efc640 1 --2- 172.21.15.39:0/180496171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 0x7f6b4c0c2750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b340097c0 tx=0x7f6b3401c410 comp rx=0 tx=0).stop 2024-09-13T16:50:36.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b51efc640 1 -- 172.21.15.39:0/180496171 shutdown_connections 2024-09-13T16:50:36.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b51efc640 1 --2- 172.21.15.39:0/180496171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 0x7f6b4c0c2750 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b51efc640 1 -- 172.21.15.39:0/180496171 >> 172.21.15.39:0/180496171 conn(0x7f6b4c0a9720 msgr2=0x7f6b4c0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:36.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b51efc640 1 -- 172.21.15.39:0/180496171 shutdown_connections 2024-09-13T16:50:36.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b51efc640 1 -- 172.21.15.39:0/180496171 wait complete. 2024-09-13T16:50:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b51efc640 1 Processor -- start 2024-09-13T16:50:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.673+0000 7f6b51efc640 1 -- start start 2024-09-13T16:50:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b51efc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c0c2370 0x7f6b4c141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b51efc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b4c0c5870 0x7f6b4c1efea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b51efc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 0x7f6b4c1f03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b51efc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c06db50 con 0x7f6b4c154bb0 2024-09-13T16:50:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b51efc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c06de10 con 0x7f6b4c0c2370 2024-09-13T16:50:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b51efc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c06e0f0 con 0x7f6b4c0c5870 2024-09-13T16:50:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 0x7f6b4c1f03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 0x7f6b4c1f03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48004/0 (socket says 172.21.15.39:48004) 2024-09-13T16:50:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4bfff640 1 -- 172.21.15.39:0/3700776417 learned_addr learned my addr 172.21.15.39:0/3700776417 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4b7fe640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c0c2370 0x7f6b4c141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b43fff640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b4c0c5870 0x7f6b4c1efea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4bfff640 1 -- 172.21.15.39:0/3700776417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b4c0c5870 msgr2=0x7f6b4c1efea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4bfff640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b4c0c5870 0x7f6b4c1efea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4bfff640 1 -- 172.21.15.39:0/3700776417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c0c2370 msgr2=0x7f6b4c141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4bfff640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c0c2370 0x7f6b4c141b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4bfff640 1 -- 172.21.15.39:0/3700776417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34009420 con 0x7f6b4c154bb0 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b43fff640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b4c0c5870 0x7f6b4c1efea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4bfff640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 0x7f6b4c1f03e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400cb40 tx=0x7f6b4400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b48ff9640 1 -- 172.21.15.39:0/3700776417 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400aca0 con 0x7f6b4c154bb0 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b48ff9640 1 -- 172.21.15.39:0/3700776417 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4400ae00 con 0x7f6b4c154bb0 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c1415f0 con 0x7f6b4c154bb0 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b48ff9640 1 -- 172.21.15.39:0/3700776417 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44012620 con 0x7f6b4c154bb0 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c1427e0 con 0x7f6b4c154bb0 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c141830 con 0x7f6b4c154bb0 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b48ff9640 1 -- 172.21.15.39:0/3700776417 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b44007070 con 0x7f6b4c154bb0 2024-09-13T16:50:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b48ff9640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b2c0666c0 0x7f6b2c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b48ff9640 1 -- 172.21.15.39:0/3700776417 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f6b440845d0 con 0x7f6b4c154bb0 2024-09-13T16:50:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4b7fe640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b2c0666c0 0x7f6b2c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.677+0000 7f6b4b7fe640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b2c0666c0 0x7f6b2c068b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c000f70 tx=0x7f6b3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:36.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.681+0000 7f6b48ff9640 1 -- 172.21.15.39:0/3700776417 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6b4c141830 con 0x7f6b4c154bb0 2024-09-13T16:50:36.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.885+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b4c141830 con 0x7f6b4c154bb0 2024-09-13T16:50:36.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.885+0000 7f6b48ff9640 1 -- 172.21.15.39:0/3700776417 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f6b4c141830 con 0x7f6b4c154bb0 2024-09-13T16:50:36.890 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:36.890 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:36.890 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 25 2024-09-13T16:50:36.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b2c0666c0 msgr2=0x7f6b2c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b2c0666c0 0x7f6b2c068b80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c000f70 tx=0x7f6b3c009040 comp rx=0 tx=0).stop 2024-09-13T16:50:36.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 msgr2=0x7f6b4c1f03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 0x7f6b4c1f03e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400cb40 tx=0x7f6b4400a060 comp rx=0 tx=0).stop 2024-09-13T16:50:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 shutdown_connections 2024-09-13T16:50:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b2c0666c0 0x7f6b2c068b80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b4c154bb0 0x7f6b4c1f03e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b4c0c5870 0x7f6b4c1efea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 --2- 172.21.15.39:0/3700776417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b4c0c2370 0x7f6b4c141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 >> 172.21.15.39:0/3700776417 conn(0x7f6b4c0a9720 msgr2=0x7f6b4c0c0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 shutdown_connections 2024-09-13T16:50:36.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:36.889+0000 7f6b51efc640 1 -- 172.21.15.39:0/3700776417 wait complete. 2024-09-13T16:50:36.909 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:50:38.910 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.041+0000 7f13a9882640 1 Processor -- start 2024-09-13T16:50:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.041+0000 7f13a9882640 1 -- start start 2024-09-13T16:50:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a9882640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 0x7f13a406f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a9882640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a406d750 con 0x7f13a414f340 2024-09-13T16:50:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a9882640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a4155ce0 con 0x7f13a40c7180 2024-09-13T16:50:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a9882640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a406e410 con 0x7f13a40c26c0 2024-09-13T16:50:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a2ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f13a414f340 0x7f13a4071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42770/0 (socket says 172.21.15.39:42770) 2024-09-13T16:50:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a2ffd640 1 -- 172.21.15.39:0/3445891386 learned_addr learned my addr 172.21.15.39:0/3445891386 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a37fe640 1 --2- 172.21.15.39:0/3445891386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 0x7f13a406f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a1ffb640 1 -- 172.21.15.39:0/3445891386 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265450428 0 0) 0x7f13a406d750 con 0x7f13a414f340 2024-09-13T16:50:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a1ffb640 1 -- 172.21.15.39:0/3445891386 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1390003530 con 0x7f13a414f340 2024-09-13T16:50:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a37fe640 1 -- 172.21.15.39:0/3445891386 >> v1:172.21.15.84:6789/0 conn(0x7f13a40c7180 legacy=0x7f13a406f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a37fe640 1 -- 172.21.15.39:0/3445891386 >> v1:172.21.15.39:6789/0 conn(0x7f13a414f340 legacy=0x7f13a4071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a1ffb640 1 -- 172.21.15.39:0/3445891386 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609295932 0 0) 0x7f1390003530 con 0x7f13a414f340 2024-09-13T16:50:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.045+0000 7f13a37fe640 1 -- 172.21.15.39:0/3445891386 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a406d3e0 con 0x7f13a40c26c0 2024-09-13T16:50:39.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a37fe640 1 --2- 172.21.15.39:0/3445891386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 0x7f13a406f760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f139c009dc0 tx=0x7f139c01c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f5340896818cf18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a1ffb640 1 -- 172.21.15.39:0/3445891386 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c001680 con 0x7f13a40c26c0 2024-09-13T16:50:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a1ffb640 1 -- 172.21.15.39:0/3445891386 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c0a6500 con 0x7f13a40c26c0 2024-09-13T16:50:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a1ffb640 1 -- 172.21.15.39:0/3445891386 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c0a6870 con 0x7f13a40c26c0 2024-09-13T16:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 -- 172.21.15.39:0/3445891386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 msgr2=0x7f13a406f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 --2- 172.21.15.39:0/3445891386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 0x7f13a406f760 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f139c009dc0 tx=0x7f139c01c820 comp rx=0 tx=0).stop 2024-09-13T16:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 -- 172.21.15.39:0/3445891386 shutdown_connections 2024-09-13T16:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 --2- 172.21.15.39:0/3445891386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 0x7f13a406f760 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 -- 172.21.15.39:0/3445891386 >> 172.21.15.39:0/3445891386 conn(0x7f13a40a96b0 msgr2=0x7f13a40a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 -- 172.21.15.39:0/3445891386 shutdown_connections 2024-09-13T16:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 -- 172.21.15.39:0/3445891386 wait complete. 2024-09-13T16:50:39.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 Processor -- start 2024-09-13T16:50:39.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 -- start start 2024-09-13T16:50:39.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 0x7f13a40c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:39.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a40c7180 0x7f13a40c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:39.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a414f340 0x7f13a41eff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40a7000 con 0x7f13a414f340 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40c3ab0 con 0x7f13a40c7180 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a9882640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40c4540 con 0x7f13a40c26c0 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a414f340 0x7f13a41eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a414f340 0x7f13a41eff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48016/0 (socket says 172.21.15.39:48016) 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a37fe640 1 -- 172.21.15.39:0/1981648595 learned_addr learned my addr 172.21.15.39:0/1981648595 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a2ffd640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 0x7f13a40c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.049+0000 7f13a27fc640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a40c7180 0x7f13a40c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a37fe640 1 -- 172.21.15.39:0/1981648595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 msgr2=0x7f13a40c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a37fe640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 0x7f13a40c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a37fe640 1 -- 172.21.15.39:0/1981648595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a40c7180 msgr2=0x7f13a40c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a37fe640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a40c7180 0x7f13a40c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:39.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a37fe640 1 -- 172.21.15.39:0/1981648595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c009a70 con 0x7f13a414f340 2024-09-13T16:50:39.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a37fe640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a414f340 0x7f13a41eff10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f139c0b0040 tx=0x7f139c0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:39.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f139bfff640 1 -- 172.21.15.39:0/1981648595 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c0a6940 con 0x7f13a414f340 2024-09-13T16:50:39.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f139bfff640 1 -- 172.21.15.39:0/1981648595 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c0a6aa0 con 0x7f13a414f340 2024-09-13T16:50:39.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a4148c30 con 0x7f13a414f340 2024-09-13T16:50:39.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f139bfff640 1 -- 172.21.15.39:0/1981648595 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c0a6e10 con 0x7f13a414f340 2024-09-13T16:50:39.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a414dfa0 con 0x7f13a414f340 2024-09-13T16:50:39.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a40c46f0 con 0x7f13a414f340 2024-09-13T16:50:39.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f139bfff640 1 -- 172.21.15.39:0/1981648595 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f139c004040 con 0x7f13a414f340 2024-09-13T16:50:39.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f139bfff640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1384066690 0x7f1384068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f139bfff640 1 -- 172.21.15.39:0/1981648595 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f139c0aa070 con 0x7f13a414f340 2024-09-13T16:50:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a2ffd640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1384066690 0x7f1384068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.053+0000 7f13a2ffd640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1384066690 0x7f1384068b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f138c005890 tx=0x7f138c001d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.057+0000 7f139bfff640 1 -- 172.21.15.39:0/1981648595 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f13a40c46f0 con 0x7f13a414f340 2024-09-13T16:50:39.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.253+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13a40c46b0 con 0x7f13a414f340 2024-09-13T16:50:39.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.253+0000 7f139bfff640 1 -- 172.21.15.39:0/1981648595 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f139c0f03d0 con 0x7f13a414f340 2024-09-13T16:50:39.259 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:39.260 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 25 2024-09-13T16:50:39.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1384066690 msgr2=0x7f1384068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:39.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1384066690 0x7f1384068b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f138c005890 tx=0x7f138c001d10 comp rx=0 tx=0).stop 2024-09-13T16:50:39.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a414f340 msgr2=0x7f13a41eff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:39.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a414f340 0x7f13a41eff10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f139c0b0040 tx=0x7f139c0195c0 comp rx=0 tx=0).stop 2024-09-13T16:50:39.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 shutdown_connections 2024-09-13T16:50:39.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1384066690 0x7f1384068b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:39.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13a40c7180 0x7f13a40c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:39.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13a414f340 0x7f13a41eff10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:39.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 --2- 172.21.15.39:0/1981648595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13a40c26c0 0x7f13a40c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:39.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 >> 172.21.15.39:0/1981648595 conn(0x7f13a40a96b0 msgr2=0x7f13a40c54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:39.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 shutdown_connections 2024-09-13T16:50:39.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:39.257+0000 7f13a9882640 1 -- 172.21.15.39:0/1981648595 wait complete. 2024-09-13T16:50:39.277 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:50:41.279 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:41.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.409+0000 7fa1e5dff640 1 Processor -- start 2024-09-13T16:50:41.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.409+0000 7fa1e5dff640 1 -- start start 2024-09-13T16:50:41.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1e5dff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa1e00c4710 0x7fa1e00c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:41.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1e5dff640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e00c76a0 con 0x7fa1e00c5fd0 2024-09-13T16:50:41.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1e5dff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e0151570 con 0x7fa1e014aa40 2024-09-13T16:50:41.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1e5dff640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e0151b30 con 0x7fa1e00c4710 2024-09-13T16:50:41.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1deffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa1e00c4710 0x7fa1e00c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:41.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1dffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa1e00c5fd0 0x7fa1e015b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:58478/0 (socket says 172.21.15.39:58478) 2024-09-13T16:50:41.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1dffff640 1 -- 172.21.15.39:0/1659953397 learned_addr learned my addr 172.21.15.39:0/1659953397 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:41.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1deffd640 1 -- 172.21.15.39:0/1659953397 >> v1:172.21.15.84:6789/0 conn(0x7fa1e014aa40 legacy=0x7fa1e00c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:41.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1deffd640 1 -- 172.21.15.39:0/1659953397 >> v1:172.21.15.39:6789/0 conn(0x7fa1e00c5fd0 legacy=0x7fa1e015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:41.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1deffd640 1 -- 172.21.15.39:0/1659953397 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e0151850 con 0x7fa1e00c4710 2024-09-13T16:50:41.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1de7fc640 1 -- 172.21.15.39:0/1659953397 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238110431 0 0) 0x7fa1e00c76a0 con 0x7fa1e00c5fd0 2024-09-13T16:50:41.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1deffd640 1 --2- 172.21.15.39:0/1659953397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 0x7fa1e00c5a90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40097c0 tx=0x7fa1d401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd991dc54881cd53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:41.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1de7fc640 1 -- 172.21.15.39:0/1659953397 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40ac640 con 0x7fa1e00c4710 2024-09-13T16:50:41.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1de7fc640 1 -- 172.21.15.39:0/1659953397 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d40ac7a0 con 0x7fa1e00c4710 2024-09-13T16:50:41.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.413+0000 7fa1de7fc640 1 -- 172.21.15.39:0/1659953397 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40acb30 con 0x7fa1e00c4710 2024-09-13T16:50:41.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- 172.21.15.39:0/1659953397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 msgr2=0x7fa1e00c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:41.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 --2- 172.21.15.39:0/1659953397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 0x7fa1e00c5a90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40097c0 tx=0x7fa1d401c3e0 comp rx=0 tx=0).stop 2024-09-13T16:50:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- 172.21.15.39:0/1659953397 shutdown_connections 2024-09-13T16:50:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 --2- 172.21.15.39:0/1659953397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 0x7fa1e00c5a90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- 172.21.15.39:0/1659953397 >> 172.21.15.39:0/1659953397 conn(0x7fa1e00c3310 msgr2=0x7fa1e00c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- 172.21.15.39:0/1659953397 shutdown_connections 2024-09-13T16:50:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- 172.21.15.39:0/1659953397 wait complete. 2024-09-13T16:50:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 Processor -- start 2024-09-13T16:50:41.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- start start 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 0x7fa1e00bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1e00c5fd0 0x7fa1e013d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e014aa40 0x7fa1e013dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0150a90 con 0x7fa1e00c4710 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e01436b0 con 0x7fa1e014aa40 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e015c4b0 con 0x7fa1e00c5fd0 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 0x7fa1e00bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1df7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 0x7fa1e00bfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35644/0 (socket says 172.21.15.39:35644) 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1df7fe640 1 -- 172.21.15.39:0/3989410992 learned_addr learned my addr 172.21.15.39:0/3989410992 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1deffd640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1e00c5fd0 0x7fa1e013d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1dffff640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e014aa40 0x7fa1e013dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1df7fe640 1 -- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1e00c5fd0 msgr2=0x7fa1e013d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1df7fe640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1e00c5fd0 0x7fa1e013d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1df7fe640 1 -- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e014aa40 msgr2=0x7fa1e013dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1df7fe640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e014aa40 0x7fa1e013dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1df7fe640 1 -- 172.21.15.39:0/3989410992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4009420 con 0x7fa1e00c4710 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1deffd640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1e00c5fd0 0x7fa1e013d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1dffff640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e014aa40 0x7fa1e013dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1df7fe640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 0x7fa1e00bfea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c7f0 tx=0x7fa1d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1c7fff640 1 -- 172.21.15.39:0/3989410992 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8003b80 con 0x7fa1e00c4710 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1c7fff640 1 -- 172.21.15.39:0/3989410992 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8003ce0 con 0x7fa1e00c4710 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e015c770 con 0x7fa1e00c4710 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1c7fff640 1 -- 172.21.15.39:0/3989410992 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800aa90 con 0x7fa1e00c4710 2024-09-13T16:50:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.417+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e01ecbc0 con 0x7fa1e00c4710 2024-09-13T16:50:41.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.421+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e013e190 con 0x7fa1e00c4710 2024-09-13T16:50:41.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.421+0000 7fa1c7fff640 1 -- 172.21.15.39:0/3989410992 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1d800abf0 con 0x7fa1e00c4710 2024-09-13T16:50:41.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.421+0000 7fa1c7fff640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1cc066690 0x7fa1cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:41.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.421+0000 7fa1c7fff640 1 -- 172.21.15.39:0/3989410992 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fa1d8084430 con 0x7fa1e00c4710 2024-09-13T16:50:41.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.421+0000 7fa1deffd640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1cc066690 0x7fa1cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:41.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.421+0000 7fa1deffd640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1cc066690 0x7fa1cc068b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40ae000 tx=0x7fa1d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:41.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.425+0000 7fa1c7fff640 1 -- 172.21.15.39:0/3989410992 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa1d80513e0 con 0x7fa1e00c4710 2024-09-13T16:50:41.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.629+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1e014a560 con 0x7fa1e00c4710 2024-09-13T16:50:41.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.629+0000 7fa1c7fff640 1 -- 172.21.15.39:0/3989410992 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5929 (secure 0 0 0) 0x7fa1d8054a00 con 0x7fa1e00c4710 2024-09-13T16:50:41.633 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:41.633 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:41.634 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 25 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1cc066690 msgr2=0x7fa1cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1cc066690 0x7fa1cc068b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40ae000 tx=0x7fa1d4008040 comp rx=0 tx=0).stop 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 msgr2=0x7fa1e00bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 0x7fa1e00bfea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c7f0 tx=0x7fa1d800ccb0 comp rx=0 tx=0).stop 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 shutdown_connections 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1cc066690 0x7fa1cc068b50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1e014aa40 0x7fa1e013dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1e00c5fd0 0x7fa1e013d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 --2- 172.21.15.39:0/3989410992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e00c4710 0x7fa1e00bfea0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 >> 172.21.15.39:0/3989410992 conn(0x7fa1e00c3310 msgr2=0x7fa1e0070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:41.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 shutdown_connections 2024-09-13T16:50:41.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:41.633+0000 7fa1e5dff640 1 -- 172.21.15.39:0/3989410992 wait complete. 2024-09-13T16:50:41.653 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:50:43.654 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.785+0000 7fd6b6b5d640 1 Processor -- start 2024-09-13T16:50:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.785+0000 7fd6b6b5d640 1 -- start start 2024-09-13T16:50:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b6b5d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd6b00c2640 0x7fd6b01525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b6b5d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b006c5d0 con 0x7fd6b00c2260 2024-09-13T16:50:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b6b5d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b006c850 con 0x7fd6b00c63e0 2024-09-13T16:50:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b6b5d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b006d1f0 con 0x7fd6b00c2640 2024-09-13T16:50:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b48f9640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd6b00c2260 0x7fd6b014ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:58482/0 (socket says 172.21.15.39:58482) 2024-09-13T16:50:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b48f9640 1 -- 172.21.15.39:0/3641182685 learned_addr learned my addr 172.21.15.39:0/3641182685 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b50fa640 1 --2- 172.21.15.39:0/3641182685 >> v2:172.21.15.84:3300/0 conn(0x7fd6b00c2640 0x7fd6b01525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418277361 0 0) 0x7fd6b006c5d0 con 0x7fd6b00c2260 2024-09-13T16:50:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd698003530 con 0x7fd6b00c2260 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2290398164 0 0) 0x7fd698003530 con 0x7fd6b00c2260 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 >> v2:172.21.15.84:3300/0 conn(0x7fd6b00c2640 msgr2=0x7fd6b01525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 --2- 172.21.15.39:0/3641182685 >> v2:172.21.15.84:3300/0 conn(0x7fd6b00c2640 0x7fd6b01525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 >> v1:172.21.15.84:6789/0 conn(0x7fd6b00c63e0 legacy=0x7fd6b00c0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b006cb30 con 0x7fd6b00c2260 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fd6a4001680 con 0x7fd6b00c2260 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 >> v1:172.21.15.39:6789/0 conn(0x7fd6b00c2260 legacy=0x7fd6b014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd698003980 0x7fd698003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd698004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd6980052b0 0x7fd69800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b006d1f0 con 0x7fd6980047d0 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69800c180 con 0x7fd698003980 2024-09-13T16:50:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69800c340 con 0x7fd6980052b0 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b50fa640 1 --2- 172.21.15.39:0/3641182685 >> v2:172.21.15.84:3300/0 conn(0x7fd6b00c2640 0x7fd6b01525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6affff640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd698004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b48f9640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd698003980 0x7fd698003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6affff640 1 -- 172.21.15.39:0/3641182685 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd6980052b0 msgr2=0x7fd69800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b50fa640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd6980052b0 0x7fd69800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6affff640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd6980052b0 0x7fd69800b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6affff640 1 -- 172.21.15.39:0/3641182685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd698003980 msgr2=0x7fd698003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6affff640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd698003980 0x7fd698003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6affff640 1 -- 172.21.15.39:0/3641182685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698003530 con 0x7fd6980047d0 2024-09-13T16:50:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.789+0000 7fd6b50fa640 1 -- 172.21.15.39:0/3641182685 reap_dead start 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6affff640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd698004c50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00c6820 tx=0x7fd6a001c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2440cb36a2b849bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00052d0 con 0x7fd6980047d0 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a00a7050 con 0x7fd6980047d0 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6af7fe640 1 -- 172.21.15.39:0/3641182685 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0003600 con 0x7fd6980047d0 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3641182685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 msgr2=0x7fd698004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd698004c50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00c6820 tx=0x7fd6a001c370 comp rx=0 tx=0).stop 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3641182685 shutdown_connections 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 --2- 172.21.15.39:0/3641182685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd698004c50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3641182685 >> 172.21.15.39:0/3641182685 conn(0x7fd6b006a6a0 msgr2=0x7fd6b00c5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3641182685 shutdown_connections 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3641182685 wait complete. 2024-09-13T16:50:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 Processor -- start 2024-09-13T16:50:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 -- start start 2024-09-13T16:50:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd6b00bab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6b00bbd90 0x7fd6b01dde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd6b01dd1c0 0x7fd6b0071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b014dc50 con 0x7fd6980047d0 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01eb490 con 0x7fd6b00bbd90 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b6b5d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01eb770 con 0x7fd6b01dd1c0 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b48f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd6b00bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b48f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd6b00bab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35668/0 (socket says 172.21.15.39:35668) 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b48f9640 1 -- 172.21.15.39:0/3920357249 learned_addr learned my addr 172.21.15.39:0/3920357249 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6affff640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6b00bbd90 0x7fd6b01dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b50fa640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd6b01dd1c0 0x7fd6b0071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b48f9640 1 -- 172.21.15.39:0/3920357249 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd6b01dd1c0 msgr2=0x7fd6b0071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b48f9640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd6b01dd1c0 0x7fd6b0071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b48f9640 1 -- 172.21.15.39:0/3920357249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6b00bbd90 msgr2=0x7fd6b01dde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b48f9640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6b00bbd90 0x7fd6b01dde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b48f9640 1 -- 172.21.15.39:0/3920357249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b014dea0 con 0x7fd6980047d0 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.793+0000 7fd6b48f9640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd6b00bab70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400e780 tx=0x7fd6a400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6ad7fa640 1 -- 172.21.15.39:0/3920357249 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a400c6c0 con 0x7fd6980047d0 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6ad7fa640 1 -- 172.21.15.39:0/3920357249 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a400c820 con 0x7fd6980047d0 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b0072020 con 0x7fd6980047d0 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6ad7fa640 1 -- 172.21.15.39:0/3920357249 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a400cbd0 con 0x7fd6980047d0 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01dee10 con 0x7fd6980047d0 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b01eb930 con 0x7fd6980047d0 2024-09-13T16:50:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6ad7fa640 1 -- 172.21.15.39:0/3920357249 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd6a400cd30 con 0x7fd6980047d0 2024-09-13T16:50:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6ad7fa640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd69c0666c0 0x7fd69c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6ad7fa640 1 -- 172.21.15.39:0/3920357249 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fd6a40842f0 con 0x7fd6980047d0 2024-09-13T16:50:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6affff640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd69c0666c0 0x7fd69c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.797+0000 7fd6affff640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd69c0666c0 0x7fd69c068b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6980057b0 tx=0x7fd6a00011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:43.805+0000 7fd6ad7fa640 1 -- 172.21.15.39:0/3920357249 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd6b01eb930 con 0x7fd6980047d0 2024-09-13T16:50:44.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.001+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6b01eb930 con 0x7fd6980047d0 2024-09-13T16:50:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.001+0000 7fd6ad7fa640 1 -- 172.21.15.39:0/3920357249 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5929 (secure 0 0 0) 0x7fd6b01eb930 con 0x7fd6980047d0 2024-09-13T16:50:44.005 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:44.005 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:44.006 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 25 2024-09-13T16:50:44.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd69c0666c0 msgr2=0x7fd69c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd69c0666c0 0x7fd69c068b80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6980057b0 tx=0x7fd6a00011a0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 msgr2=0x7fd6b00bab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd6b00bab70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400e780 tx=0x7fd6a400ec40 comp rx=0 tx=0).stop 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 shutdown_connections 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd69c0666c0 0x7fd69c068b80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6b00bbd90 0x7fd6b01dde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd6b01dd1c0 0x7fd6b0071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 --2- 172.21.15.39:0/3920357249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6980047d0 0x7fd6b00bab70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 >> 172.21.15.39:0/3920357249 conn(0x7fd6b006a6a0 msgr2=0x7fd6b0153970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 shutdown_connections 2024-09-13T16:50:44.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:44.005+0000 7fd6b6b5d640 1 -- 172.21.15.39:0/3920357249 wait complete. 2024-09-13T16:50:44.023 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:50:46.024 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:46.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff640c48640 1 Processor -- start 2024-09-13T16:50:46.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff640c48640 1 -- start start 2024-09-13T16:50:46.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff640c48640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff63c150f80 0x7ff63c153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff640c48640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff63c153e70 0x7ff63c0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff640c48640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff63c1546c0 con 0x7ff63c0c5330 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff640c48640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63c0c19a0 con 0x7ff63c153e70 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff640c48640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63c0c49f0 con 0x7ff63c150f80 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff63ad9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff63c153e70 0x7ff63c0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff633fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff63c0c5330 0x7ff63c06c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:58488/0 (socket says 172.21.15.39:58488) 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff633fff640 1 -- 172.21.15.39:0/1615881086 learned_addr learned my addr 172.21.15.39:0/1615881086 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff63a59c640 1 --2- 172.21.15.39:0/1615881086 >> v2:172.21.15.84:3300/0 conn(0x7ff63c150f80 0x7ff63c153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff63ad9d640 1 -- 172.21.15.39:0/1615881086 >> v2:172.21.15.84:3300/0 conn(0x7ff63c150f80 msgr2=0x7ff63c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff63ad9d640 1 --2- 172.21.15.39:0/1615881086 >> v2:172.21.15.84:3300/0 conn(0x7ff63c150f80 0x7ff63c153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff63ad9d640 1 -- 172.21.15.39:0/1615881086 >> v1:172.21.15.39:6789/0 conn(0x7ff63c0c5330 legacy=0x7ff63c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff63ad9d640 1 -- 172.21.15.39:0/1615881086 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c1543e0 con 0x7ff63c153e70 2024-09-13T16:50:46.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.157+0000 7ff639d9b640 1 -- 172.21.15.39:0/1615881086 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179282109 0 0) 0x7ff63c1546c0 con 0x7ff63c0c5330 2024-09-13T16:50:46.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff63ad9d640 1 --2- 172.21.15.39:0/1615881086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c153e70 0x7ff63c0c1210 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff634009d90 tx=0x7ff63401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3452192b26b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:46.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff639d9b640 1 -- 172.21.15.39:0/1615881086 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6340a58b0 con 0x7ff63c153e70 2024-09-13T16:50:46.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff639d9b640 1 -- 172.21.15.39:0/1615881086 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6340a5a10 con 0x7ff63c153e70 2024-09-13T16:50:46.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff639d9b640 1 -- 172.21.15.39:0/1615881086 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6340a5da0 con 0x7ff63c153e70 2024-09-13T16:50:46.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 -- 172.21.15.39:0/1615881086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c153e70 msgr2=0x7ff63c0c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 --2- 172.21.15.39:0/1615881086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c153e70 0x7ff63c0c1210 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff634009d90 tx=0x7ff63401c870 comp rx=0 tx=0).stop 2024-09-13T16:50:46.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 -- 172.21.15.39:0/1615881086 shutdown_connections 2024-09-13T16:50:46.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 --2- 172.21.15.39:0/1615881086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c153e70 0x7ff63c0c1210 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 --2- 172.21.15.39:0/1615881086 >> v2:172.21.15.84:3300/0 conn(0x7ff63c150f80 0x7ff63c153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 -- 172.21.15.39:0/1615881086 >> 172.21.15.39:0/1615881086 conn(0x7ff63c069e10 msgr2=0x7ff63c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:46.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 -- 172.21.15.39:0/1615881086 shutdown_connections 2024-09-13T16:50:46.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 -- 172.21.15.39:0/1615881086 wait complete. 2024-09-13T16:50:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 Processor -- start 2024-09-13T16:50:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 -- start start 2024-09-13T16:50:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff63c0c5330 0x7ff63c1eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c150f80 0x7ff63c157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff63c153e70 0x7ff63c156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c159520 con 0x7ff63c150f80 2024-09-13T16:50:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c159800 con 0x7ff63c0c5330 2024-09-13T16:50:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff640c48640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c1ea1e0 con 0x7ff63c153e70 2024-09-13T16:50:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff633fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c150f80 0x7ff63c157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff633fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c150f80 0x7ff63c157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35692/0 (socket says 172.21.15.39:35692) 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff633fff640 1 -- 172.21.15.39:0/234636975 learned_addr learned my addr 172.21.15.39:0/234636975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff63a59c640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff63c0c5330 0x7ff63c1eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff63ad9d640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff63c153e70 0x7ff63c156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff633fff640 1 -- 172.21.15.39:0/234636975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff63c153e70 msgr2=0x7ff63c156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff633fff640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff63c153e70 0x7ff63c156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff633fff640 1 -- 172.21.15.39:0/234636975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff63c0c5330 msgr2=0x7ff63c1eacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff633fff640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff63c0c5330 0x7ff63c1eacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff633fff640 1 -- 172.21.15.39:0/234636975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff634009a70 con 0x7ff63c150f80 2024-09-13T16:50:46.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff63a59c640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff63c0c5330 0x7ff63c1eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:46.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.161+0000 7ff633fff640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c150f80 0x7ff63c157db0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c4b0 tx=0x7ff62400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:46.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff632ffd640 1 -- 172.21.15.39:0/234636975 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240074e0 con 0x7ff63c150f80 2024-09-13T16:50:46.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff632ffd640 1 -- 172.21.15.39:0/234636975 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624007660 con 0x7ff63c150f80 2024-09-13T16:50:46.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff632ffd640 1 -- 172.21.15.39:0/234636975 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624004a30 con 0x7ff63c150f80 2024-09-13T16:50:46.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c1ea4c0 con 0x7ff63c150f80 2024-09-13T16:50:46.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c1582f0 con 0x7ff63c150f80 2024-09-13T16:50:46.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c1ea9e0 con 0x7ff63c150f80 2024-09-13T16:50:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff632ffd640 1 -- 172.21.15.39:0/234636975 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff624004b90 con 0x7ff63c150f80 2024-09-13T16:50:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff632ffd640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff61c0666c0 0x7ff61c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff632ffd640 1 -- 172.21.15.39:0/234636975 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff6240837b0 con 0x7ff63c150f80 2024-09-13T16:50:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff63a59c640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff61c0666c0 0x7ff61c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.165+0000 7ff63a59c640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff61c0666c0 0x7ff61c068b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff62c007a70 tx=0x7ff62c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.169+0000 7ff632ffd640 1 -- 172.21.15.39:0/234636975 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff624050780 con 0x7ff63c150f80 2024-09-13T16:50:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff63c0bfee0 con 0x7ff63c150f80 2024-09-13T16:50:46.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff632ffd640 1 -- 172.21.15.39:0/234636975 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5929 (secure 0 0 0) 0x7ff624053da0 con 0x7ff63c150f80 2024-09-13T16:50:46.372 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:46.372 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:46.372 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 25 2024-09-13T16:50:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff61c0666c0 msgr2=0x7ff61c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff61c0666c0 0x7ff61c068b80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff62c007a70 tx=0x7ff62c008040 comp rx=0 tx=0).stop 2024-09-13T16:50:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c150f80 msgr2=0x7ff63c157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c150f80 0x7ff63c157db0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c4b0 tx=0x7ff62400c970 comp rx=0 tx=0).stop 2024-09-13T16:50:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 shutdown_connections 2024-09-13T16:50:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff61c0666c0 0x7ff61c068b80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff63c153e70 0x7ff63c156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff63c150f80 0x7ff63c157db0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 --2- 172.21.15.39:0/234636975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff63c0c5330 0x7ff63c1eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 >> 172.21.15.39:0/234636975 conn(0x7ff63c069e10 msgr2=0x7ff63c0c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 shutdown_connections 2024-09-13T16:50:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:46.369+0000 7ff640c48640 1 -- 172.21.15.39:0/234636975 wait complete. 2024-09-13T16:50:46.391 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:50:48.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:48.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0662ab9640 1 Processor -- start 2024-09-13T16:50:48.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0662ab9640 1 -- start start 2024-09-13T16:50:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0662ab9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f065c071790 0x7f065c06ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0662ab9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f065c0c2f60 0x7f065c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0662ab9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f065c0c73e0 con 0x7f065c0c6f50 2024-09-13T16:50:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0662ab9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c0c3b30 con 0x7f065c071790 2024-09-13T16:50:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0662ab9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065c0c5f10 con 0x7f065c0c2f60 2024-09-13T16:50:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0660855640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f065c071790 0x7f065c06ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0660855640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f065c071790 0x7f065c06ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35702/0 (socket says 172.21.15.39:35702) 2024-09-13T16:50:48.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0660855640 1 -- 172.21.15.39:0/643767081 learned_addr learned my addr 172.21.15.39:0/643767081 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:48.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0661056640 1 --2- 172.21.15.39:0/643767081 >> v2:172.21.15.84:3300/0 conn(0x7f065c0c2f60 0x7f065c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:48.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0660855640 1 -- 172.21.15.39:0/643767081 >> v2:172.21.15.84:3300/0 conn(0x7f065c0c2f60 msgr2=0x7f065c157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:48.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0660855640 1 --2- 172.21.15.39:0/643767081 >> v2:172.21.15.84:3300/0 conn(0x7f065c0c2f60 0x7f065c157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:48.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0660855640 1 -- 172.21.15.39:0/643767081 >> v1:172.21.15.84:6789/0 conn(0x7f065c0c6f50 legacy=0x7f065c0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:48.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.520+0000 7f0660855640 1 -- 172.21.15.39:0/643767081 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0c2480 con 0x7f065c071790 2024-09-13T16:50:48.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0660855640 1 --2- 172.21.15.39:0/643767081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c071790 0x7f065c06ae70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f065c072640 tx=0x7f064c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f64978c8ca364cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:48.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f065b7fe640 1 -- 172.21.15.39:0/643767081 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0aa5f0 con 0x7f065c071790 2024-09-13T16:50:48.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f065b7fe640 1 -- 172.21.15.39:0/643767081 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c0aa750 con 0x7f065c071790 2024-09-13T16:50:48.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f065b7fe640 1 -- 172.21.15.39:0/643767081 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0aaae0 con 0x7f065c071790 2024-09-13T16:50:48.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 -- 172.21.15.39:0/643767081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c071790 msgr2=0x7f065c06ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:48.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 --2- 172.21.15.39:0/643767081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c071790 0x7f065c06ae70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f065c072640 tx=0x7f064c01c490 comp rx=0 tx=0).stop 2024-09-13T16:50:48.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 -- 172.21.15.39:0/643767081 shutdown_connections 2024-09-13T16:50:48.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 --2- 172.21.15.39:0/643767081 >> v2:172.21.15.84:3300/0 conn(0x7f065c0c2f60 0x7f065c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:48.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 --2- 172.21.15.39:0/643767081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c071790 0x7f065c06ae70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:48.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 -- 172.21.15.39:0/643767081 >> 172.21.15.39:0/643767081 conn(0x7f065c0a96b0 msgr2=0x7f065c0a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:48.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 -- 172.21.15.39:0/643767081 shutdown_connections 2024-09-13T16:50:48.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 -- 172.21.15.39:0/643767081 wait complete. 2024-09-13T16:50:48.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 Processor -- start 2024-09-13T16:50:48.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.524+0000 7f0662ab9640 1 -- start start 2024-09-13T16:50:48.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0662ab9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c071790 0x7f065c1f0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:48.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0662ab9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c2f60 0x7f065c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0662ab9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c151c50 0x7f065c1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0662ab9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c1f0b50 con 0x7f065c0c2f60 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0662ab9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c0abd00 con 0x7f065c151c50 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0662ab9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065c152620 con 0x7f065c071790 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f065bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c2f60 0x7f065c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f065bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c2f60 0x7f065c156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35706/0 (socket says 172.21.15.39:35706) 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f065bfff640 1 -- 172.21.15.39:0/3988120132 learned_addr learned my addr 172.21.15.39:0/3988120132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0660855640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c071790 0x7f065c1f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0661056640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c151c50 0x7f065c1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f065bfff640 1 -- 172.21.15.39:0/3988120132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c071790 msgr2=0x7f065c1f0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f065bfff640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c071790 0x7f065c1f0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:48.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f065bfff640 1 -- 172.21.15.39:0/3988120132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c151c50 msgr2=0x7f065c1520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:48.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f065bfff640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c151c50 0x7f065c1520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:48.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f065bfff640 1 -- 172.21.15.39:0/3988120132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0094d0 con 0x7f065c0c2f60 2024-09-13T16:50:48.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0661056640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c151c50 0x7f065c1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:48.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f065bfff640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c2f60 0x7f065c156430 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f065000c3c0 tx=0x7f065000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:48.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f06597fa640 1 -- 172.21.15.39:0/3988120132 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06500052d0 con 0x7f065c0c2f60 2024-09-13T16:50:48.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f06597fa640 1 -- 172.21.15.39:0/3988120132 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0650005430 con 0x7f065c0c2f60 2024-09-13T16:50:48.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c0c34c0 con 0x7f065c0c2f60 2024-09-13T16:50:48.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f06597fa640 1 -- 172.21.15.39:0/3988120132 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650003840 con 0x7f065c0c2f60 2024-09-13T16:50:48.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c0ad260 con 0x7f065c0c2f60 2024-09-13T16:50:48.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c071b70 con 0x7f065c0c2f60 2024-09-13T16:50:48.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f06597fa640 1 -- 172.21.15.39:0/3988120132 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06500039a0 con 0x7f065c0c2f60 2024-09-13T16:50:48.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f06597fa640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0648066690 0x7f0648068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:48.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f06597fa640 1 -- 172.21.15.39:0/3988120132 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f0650083ab0 con 0x7f065c0c2f60 2024-09-13T16:50:48.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0660855640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0648066690 0x7f0648068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:48.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.528+0000 7f0660855640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0648066690 0x7f0648068b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f064c0ae000 tx=0x7f064c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:48.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.532+0000 7f06597fa640 1 -- 172.21.15.39:0/3988120132 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0650050a60 con 0x7f065c0c2f60 2024-09-13T16:50:48.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f065c06f800 con 0x7f065c0c2f60 2024-09-13T16:50:48.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f06597fa640 1 -- 172.21.15.39:0/3988120132 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f0650054080 con 0x7f065c0c2f60 2024-09-13T16:50:48.733 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:48.733 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:48:02.679109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.84:6833/1392588044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1392588044},{"type":"v1","addr":"172.21.15.84:6833","nonce":1392588044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:48.733 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 25 2024-09-13T16:50:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0648066690 msgr2=0x7f0648068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0648066690 0x7f0648068b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f064c0ae000 tx=0x7f064c0a7000 comp rx=0 tx=0).stop 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c2f60 msgr2=0x7f065c156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c2f60 0x7f065c156430 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f065000c3c0 tx=0x7f065000c880 comp rx=0 tx=0).stop 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 shutdown_connections 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0648066690 0x7f0648068b50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f065c151c50 0x7f065c1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f065c0c2f60 0x7f065c156430 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 --2- 172.21.15.39:0/3988120132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f065c071790 0x7f065c1f0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.728+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 >> 172.21.15.39:0/3988120132 conn(0x7f065c0a96b0 msgr2=0x7f065c0c54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.732+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 shutdown_connections 2024-09-13T16:50:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:48.732+0000 7f0662ab9640 1 -- 172.21.15.39:0/3988120132 wait complete. 2024-09-13T16:50:48.752 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:50:50.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:50.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5da53f5640 1 Processor -- start 2024-09-13T16:50:50.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5da53f5640 1 -- start start 2024-09-13T16:50:50.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5da53f5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5da014e0c0 0x7f5da00a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5da53f5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da00c2780 con 0x7f5da00c4740 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5da53f5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da00c2a60 con 0x7f5da00c1f50 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5da53f5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da00c2d40 con 0x7f5da014e0c0 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5d9f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5da00c4740 0x7f5da00c4b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53554/0 (socket says 172.21.15.39:53554) 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5d9f7fe640 1 -- 172.21.15.39:0/1055802406 learned_addr learned my addr 172.21.15.39:0/1055802406 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5d9effd640 1 --2- 172.21.15.39:0/1055802406 >> v2:172.21.15.39:3300/0 conn(0x7f5da014e0c0 0x7f5da00a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5d9dffb640 1 -- 172.21.15.39:0/1055802406 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434057218 0 0) 0x7f5da00c2780 con 0x7f5da00c4740 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5d9dffb640 1 -- 172.21.15.39:0/1055802406 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d88003530 con 0x7f5da00c4740 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5d9effd640 1 -- 172.21.15.39:0/1055802406 >> v1:172.21.15.84:6789/0 conn(0x7f5da00c1f50 legacy=0x7f5da006cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5d9effd640 1 -- 172.21.15.39:0/1055802406 >> v1:172.21.15.39:6789/0 conn(0x7f5da00c4740 legacy=0x7f5da00c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:50.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.884+0000 7f5d9effd640 1 -- 172.21.15.39:0/1055802406 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da00c3020 con 0x7f5da014e0c0 2024-09-13T16:50:50.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5d9effd640 1 --2- 172.21.15.39:0/1055802406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da014e0c0 0x7f5da00a4e70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5d900097c0 tx=0x7f5d9001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86611d268a29846 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:50.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5d9dffb640 1 -- 172.21.15.39:0/1055802406 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900ac640 con 0x7f5da014e0c0 2024-09-13T16:50:50.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5d9dffb640 1 -- 172.21.15.39:0/1055802406 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d900ac7a0 con 0x7f5da014e0c0 2024-09-13T16:50:50.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5d9dffb640 1 -- 172.21.15.39:0/1055802406 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900acb30 con 0x7f5da014e0c0 2024-09-13T16:50:50.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5da53f5640 1 -- 172.21.15.39:0/1055802406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da014e0c0 msgr2=0x7f5da00a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:50.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5da53f5640 1 --2- 172.21.15.39:0/1055802406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da014e0c0 0x7f5da00a4e70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5d900097c0 tx=0x7f5d9001c3e0 comp rx=0 tx=0).stop 2024-09-13T16:50:50.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5da53f5640 1 -- 172.21.15.39:0/1055802406 shutdown_connections 2024-09-13T16:50:50.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5da53f5640 1 --2- 172.21.15.39:0/1055802406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da014e0c0 0x7f5da00a4e70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:50.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5da53f5640 1 -- 172.21.15.39:0/1055802406 >> 172.21.15.39:0/1055802406 conn(0x7f5da0069e10 msgr2=0x7f5da006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:50.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5da53f5640 1 -- 172.21.15.39:0/1055802406 shutdown_connections 2024-09-13T16:50:50.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.888+0000 7f5da53f5640 1 -- 172.21.15.39:0/1055802406 wait complete. 2024-09-13T16:50:50.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 Processor -- start 2024-09-13T16:50:50.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 -- start start 2024-09-13T16:50:50.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da00c1f50 0x7f5da0071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:50.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5da00c4740 0x7f5da014cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:50.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5da014e0c0 0x7f5da0145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:50.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0151d30 con 0x7f5da00c1f50 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0147320 con 0x7f5da014e0c0 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0151f40 con 0x7f5da00c4740 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da00c1f50 0x7f5da0071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da00c1f50 0x7f5da0071f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54746/0 (socket says 172.21.15.39:54746) 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9effd640 1 -- 172.21.15.39:0/1393095488 learned_addr learned my addr 172.21.15.39:0/1393095488 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9f7fe640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5da014e0c0 0x7f5da0145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9e7fc640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5da00c4740 0x7f5da014cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9effd640 1 -- 172.21.15.39:0/1393095488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5da00c4740 msgr2=0x7f5da014cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9effd640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5da00c4740 0x7f5da014cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9effd640 1 -- 172.21.15.39:0/1393095488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5da014e0c0 msgr2=0x7f5da0145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9effd640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5da014e0c0 0x7f5da0145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:50.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9effd640 1 -- 172.21.15.39:0/1393095488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d90009420 con 0x7f5da00c1f50 2024-09-13T16:50:50.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9e7fc640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5da00c4740 0x7f5da014cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:50.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d9effd640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da00c1f50 0x7f5da0071f90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5d900af040 tx=0x7f5d9001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:50.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d877fe640 1 -- 172.21.15.39:0/1393095488 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900b8650 con 0x7f5da00c1f50 2024-09-13T16:50:50.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d877fe640 1 -- 172.21.15.39:0/1393095488 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d900b87b0 con 0x7f5da00c1f50 2024-09-13T16:50:50.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da01e9bc0 con 0x7f5da00c1f50 2024-09-13T16:50:50.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d877fe640 1 -- 172.21.15.39:0/1393095488 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900b8b60 con 0x7f5da00c1f50 2024-09-13T16:50:50.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da0147800 con 0x7f5da00c1f50 2024-09-13T16:50:50.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da01e9a20 con 0x7f5da00c1f50 2024-09-13T16:50:50.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.892+0000 7f5d877fe640 1 -- 172.21.15.39:0/1393095488 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d900b8cc0 con 0x7f5da00c1f50 2024-09-13T16:50:50.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.896+0000 7f5d877fe640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d8c066690 0x7f5d8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:50.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.896+0000 7f5d877fe640 1 -- 172.21.15.39:0/1393095488 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7f5d9011ef60 con 0x7f5da00c1f50 2024-09-13T16:50:50.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.896+0000 7f5d9e7fc640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d8c066690 0x7f5d8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:50.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.896+0000 7f5d9e7fc640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d8c066690 0x7f5d8c068b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5da01504f0 tx=0x7f5d94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:50.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:50.900+0000 7f5d877fe640 1 -- 172.21.15.39:0/1393095488 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5da01e9a20 con 0x7f5da00c1f50 2024-09-13T16:50:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.092+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5da01e9a20 con 0x7f5da00c1f50 2024-09-13T16:50:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5d877fe640 1 -- 172.21.15.39:0/1393095488 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5178 (secure 0 0 0) 0x7f5da01e9a20 con 0x7f5da00c1f50 2024-09-13T16:50:51.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:51.100 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:50.745880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":9,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d8c066690 msgr2=0x7f5d8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d8c066690 0x7f5d8c068b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5da01504f0 tx=0x7f5d94009040 comp rx=0 tx=0).stop 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da00c1f50 msgr2=0x7f5da0071f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da00c1f50 0x7f5da0071f90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5d900af040 tx=0x7f5d9001ac20 comp rx=0 tx=0).stop 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 shutdown_connections 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d8c066690 0x7f5d8c068b50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5da014e0c0 0x7f5da0145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5da00c4740 0x7f5da014cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 --2- 172.21.15.39:0/1393095488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5da00c1f50 0x7f5da0071f90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 >> 172.21.15.39:0/1393095488 conn(0x7f5da0069e10 msgr2=0x7f5da006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 shutdown_connections 2024-09-13T16:50:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.096+0000 7f5da53f5640 1 -- 172.21.15.39:0/1393095488 wait complete. 2024-09-13T16:50:51.119 INFO:tasks.mds_thrash.fs.[a]:mds.b down, removed from mdsmap 2024-09-13T16:50:51.119 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T16:50:51.119 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:51.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f639d5b3640 1 Processor -- start 2024-09-13T16:50:51.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f639d5b3640 1 -- start start 2024-09-13T16:50:51.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f639d5b3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63980aabc0 0x7f63980aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:51.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f639d5b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63980c40b0 0x7f639815c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:51.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f639d5b3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6398152a80 con 0x7f639806c870 2024-09-13T16:50:51.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f639d5b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63980c30f0 con 0x7f63980c40b0 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f639d5b3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6398151e00 con 0x7f63980aabc0 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f638ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f639806c870 0x7f63980c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53558/0 (socket says 172.21.15.39:53558) 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f638ffff640 1 -- 172.21.15.39:0/213323345 learned_addr learned my addr 172.21.15.39:0/213323345 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f63977fe640 1 --2- 172.21.15.39:0/213323345 >> v2:172.21.15.39:3300/0 conn(0x7f63980c40b0 0x7f639815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f63967fc640 1 -- 172.21.15.39:0/213323345 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609575926 0 0) 0x7f6388001120 con 0x7f639806c870 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f6396ffd640 1 --2- 172.21.15.39:0/213323345 >> v2:172.21.15.84:3300/0 conn(0x7f63980aabc0 0x7f63980aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f63967fc640 1 -- 172.21.15.39:0/213323345 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6384003530 con 0x7f639806c870 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f63977fe640 1 -- 172.21.15.39:0/213323345 >> v2:172.21.15.84:3300/0 conn(0x7f63980aabc0 msgr2=0x7f63980aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f63977fe640 1 --2- 172.21.15.39:0/213323345 >> v2:172.21.15.84:3300/0 conn(0x7f63980aabc0 0x7f63980aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f63977fe640 1 -- 172.21.15.39:0/213323345 >> v1:172.21.15.39:6789/0 conn(0x7f639806c870 legacy=0x7f63980c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f63977fe640 1 -- 172.21.15.39:0/213323345 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63981520e0 con 0x7f63980c40b0 2024-09-13T16:50:51.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.240+0000 7f6396ffd640 1 --2- 172.21.15.39:0/213323345 >> v2:172.21.15.84:3300/0 conn(0x7f63980aabc0 0x7f63980aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:51.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f63977fe640 1 --2- 172.21.15.39:0/213323345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63980c40b0 0x7f639815c470 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f63980c33a0 tx=0x7f639001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411118560e665433 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f63967fc640 1 -- 172.21.15.39:0/213323345 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900a5930 con 0x7f63980c40b0 2024-09-13T16:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f63967fc640 1 -- 172.21.15.39:0/213323345 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63900a5a90 con 0x7f63980c40b0 2024-09-13T16:50:51.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f63967fc640 1 -- 172.21.15.39:0/213323345 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900a5e20 con 0x7f63980c40b0 2024-09-13T16:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 -- 172.21.15.39:0/213323345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63980c40b0 msgr2=0x7f639815c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 --2- 172.21.15.39:0/213323345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63980c40b0 0x7f639815c470 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f63980c33a0 tx=0x7f639001c870 comp rx=0 tx=0).stop 2024-09-13T16:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 -- 172.21.15.39:0/213323345 shutdown_connections 2024-09-13T16:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 --2- 172.21.15.39:0/213323345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63980c40b0 0x7f639815c470 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 --2- 172.21.15.39:0/213323345 >> v2:172.21.15.84:3300/0 conn(0x7f63980aabc0 0x7f63980aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 -- 172.21.15.39:0/213323345 >> 172.21.15.39:0/213323345 conn(0x7f6398069e10 msgr2=0x7f639806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:51.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 -- 172.21.15.39:0/213323345 shutdown_connections 2024-09-13T16:50:51.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 -- 172.21.15.39:0/213323345 wait complete. 2024-09-13T16:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 Processor -- start 2024-09-13T16:50:51.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 -- start start 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f639806c870 0x7f6398154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63980aabc0 0x7f6398070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63980c40b0 0x7f63980712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639806f190 con 0x7f639806c870 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639806f470 con 0x7f63980aabc0 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f639d5b3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6398154780 con 0x7f63980c40b0 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f6396ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f639806c870 0x7f6398154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f6396ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f639806c870 0x7f6398154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54766/0 (socket says 172.21.15.39:54766) 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f6396ffd640 1 -- 172.21.15.39:0/1572885229 learned_addr learned my addr 172.21.15.39:0/1572885229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f638ffff640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63980aabc0 0x7f6398070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f63977fe640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63980c40b0 0x7f63980712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f6396ffd640 1 -- 172.21.15.39:0/1572885229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63980c40b0 msgr2=0x7f63980712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:51.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f6396ffd640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63980c40b0 0x7f63980712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f6396ffd640 1 -- 172.21.15.39:0/1572885229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63980aabc0 msgr2=0x7f6398070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f6396ffd640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63980aabc0 0x7f6398070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.244+0000 7f6396ffd640 1 -- 172.21.15.39:0/1572885229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390009a70 con 0x7f639806c870 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f6396ffd640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f639806c870 0x7f6398154e30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f638000c7f0 tx=0x7f638000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f638f7fe640 1 -- 172.21.15.39:0/1572885229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638000cea0 con 0x7f639806c870 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f638f7fe640 1 -- 172.21.15.39:0/1572885229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6380007a30 con 0x7f639806c870 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6398141fe0 con 0x7f639806c870 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f638f7fe640 1 -- 172.21.15.39:0/1572885229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380007de0 con 0x7f639806c870 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63981ee080 con 0x7f639806c870 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6398141d40 con 0x7f639806c870 2024-09-13T16:50:51.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f638f7fe640 1 -- 172.21.15.39:0/1572885229 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63800113f0 con 0x7f639806c870 2024-09-13T16:50:51.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f638f7fe640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6378066690 0x7f6378068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:51.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f638f7fe640 1 -- 172.21.15.39:0/1572885229 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7f63800847c0 con 0x7f639806c870 2024-09-13T16:50:51.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f638ffff640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6378066690 0x7f6378068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:51.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.248+0000 7f638ffff640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6378066690 0x7f6378068b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f639814c780 tx=0x7f6388009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:51.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.252+0000 7f638f7fe640 1 -- 172.21.15.39:0/1572885229 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6380051600 con 0x7f639806c870 2024-09-13T16:50:51.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.448+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63980aafa0 con 0x7f639806c870 2024-09-13T16:50:51.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.448+0000 7f638f7fe640 1 -- 172.21.15.39:0/1572885229 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5178 (secure 0 0 0) 0x7f6380054c20 con 0x7f639806c870 2024-09-13T16:50:51.454 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:51.454 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:50.745880+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":9,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:51.454 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 27 2024-09-13T16:50:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6378066690 msgr2=0x7f6378068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6378066690 0x7f6378068b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f639814c780 tx=0x7f6388009040 comp rx=0 tx=0).stop 2024-09-13T16:50:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f639806c870 msgr2=0x7f6398154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:51.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f639806c870 0x7f6398154e30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f638000c7f0 tx=0x7f638000ccb0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 shutdown_connections 2024-09-13T16:50:51.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6378066690 0x7f6378068b50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63980aabc0 0x7f6398070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63980c40b0 0x7f63980712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 --2- 172.21.15.39:0/1572885229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f639806c870 0x7f6398154e30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:51.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 >> 172.21.15.39:0/1572885229 conn(0x7f6398069e10 msgr2=0x7f63980c2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:51.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 shutdown_connections 2024-09-13T16:50:51.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:51.452+0000 7f639d5b3640 1 -- 172.21.15.39:0/1572885229 wait complete. 2024-09-13T16:50:51.474 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 27, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 27, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5053": { "addr": "172.21.15.84:6835/1969975539", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 1969975539, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 1969975539, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5053, "incarnation": 26, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:reconnect", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:50:50.745880+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5053 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 24, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5142": { "addr": "172.21.15.39:6835/1572373831", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 1572373831, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 1572373831, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5142, "incarnation": 20, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 28, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:50:18.849824+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5142 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:50:53.476 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:53.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe93c4f8640 1 Processor -- start 2024-09-13T16:50:53.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe93c4f8640 1 -- start start 2024-09-13T16:50:53.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe93c4f8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9340abc50 0x7fe93414f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe93c4f8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9340ab340 0x7fe93414f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe93c4f8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe934156270 con 0x7fe93406cbc0 2024-09-13T16:50:53.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe93c4f8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe934061530 con 0x7fe9340ab340 2024-09-13T16:50:53.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe93c4f8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341554b0 con 0x7fe9340abc50 2024-09-13T16:50:53.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe939a93640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9340ab340 0x7fe93414f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe939a93640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9340ab340 0x7fe93414f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54774/0 (socket says 172.21.15.39:54774) 2024-09-13T16:50:53.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe939a93640 1 -- 172.21.15.39:0/4023859545 learned_addr learned my addr 172.21.15.39:0/4023859545 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:53.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe93a294640 1 --2- 172.21.15.39:0/4023859545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9340abc50 0x7fe93414f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe939a93640 1 -- 172.21.15.39:0/4023859545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9340abc50 msgr2=0x7fe93414f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:53.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe939a93640 1 --2- 172.21.15.39:0/4023859545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9340abc50 0x7fe93414f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe939a93640 1 -- 172.21.15.39:0/4023859545 >> v1:172.21.15.84:6789/0 conn(0x7fe93406cbc0 legacy=0x7fe93406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:53.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe939a93640 1 -- 172.21.15.39:0/4023859545 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe934155790 con 0x7fe9340ab340 2024-09-13T16:50:53.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.616+0000 7fe93a294640 1 --2- 172.21.15.39:0/4023859545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9340abc50 0x7fe93414f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:53.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe939a93640 1 --2- 172.21.15.39:0/4023859545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340ab340 0x7fe93414f640 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe924009870 tx=0x7fe92401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc16848e84737e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:53.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe939292640 1 -- 172.21.15.39:0/4023859545 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9240ad640 con 0x7fe9340ab340 2024-09-13T16:50:53.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe939292640 1 -- 172.21.15.39:0/4023859545 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9240ad7a0 con 0x7fe9340ab340 2024-09-13T16:50:53.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe939292640 1 -- 172.21.15.39:0/4023859545 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9240adb30 con 0x7fe9340ab340 2024-09-13T16:50:53.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4023859545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340ab340 msgr2=0x7fe93414f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe93c4f8640 1 --2- 172.21.15.39:0/4023859545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340ab340 0x7fe93414f640 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe924009870 tx=0x7fe92401c3d0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4023859545 shutdown_connections 2024-09-13T16:50:53.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe93c4f8640 1 --2- 172.21.15.39:0/4023859545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340ab340 0x7fe93414f640 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe93c4f8640 1 --2- 172.21.15.39:0/4023859545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9340abc50 0x7fe93414f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4023859545 >> 172.21.15.39:0/4023859545 conn(0x7fe934069e10 msgr2=0x7fe93406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:53.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4023859545 shutdown_connections 2024-09-13T16:50:53.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.620+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4023859545 wait complete. 2024-09-13T16:50:53.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 Processor -- start 2024-09-13T16:50:53.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 -- start start 2024-09-13T16:50:53.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe93406cbc0 0x7fe934146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9340ab340 0x7fe93414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340abc50 0x7fe934141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe934146370 con 0x7fe9340abc50 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe934146930 con 0x7fe9340ab340 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe934147300 con 0x7fe93406cbc0 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93aa95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340abc50 0x7fe934141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93aa95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340abc50 0x7fe934141940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54786/0 (socket says 172.21.15.39:54786) 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93aa95640 1 -- 172.21.15.39:0/4176562596 learned_addr learned my addr 172.21.15.39:0/4176562596 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe939a93640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9340ab340 0x7fe93414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93a294640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe93406cbc0 0x7fe934146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93aa95640 1 -- 172.21.15.39:0/4176562596 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe93406cbc0 msgr2=0x7fe934146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93aa95640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe93406cbc0 0x7fe934146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93aa95640 1 -- 172.21.15.39:0/4176562596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9340ab340 msgr2=0x7fe93414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93aa95640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9340ab340 0x7fe93414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93aa95640 1 -- 172.21.15.39:0/4176562596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9240094d0 con 0x7fe9340abc50 2024-09-13T16:50:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93aa95640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340abc50 0x7fe934141940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b970 tx=0x7fe92c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe92affd640 1 -- 172.21.15.39:0/4176562596 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00cae0 con 0x7fe9340abc50 2024-09-13T16:50:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe92affd640 1 -- 172.21.15.39:0/4176562596 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c00cc40 con 0x7fe9340abc50 2024-09-13T16:50:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe934147cc0 con 0x7fe9340abc50 2024-09-13T16:50:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe92affd640 1 -- 172.21.15.39:0/4176562596 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c014680 con 0x7fe9340abc50 2024-09-13T16:50:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9341465c0 con 0x7fe9340abc50 2024-09-13T16:50:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.624+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe934146a70 con 0x7fe9340abc50 2024-09-13T16:50:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.628+0000 7fe92affd640 1 -- 172.21.15.39:0/4176562596 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe92c003980 con 0x7fe9340abc50 2024-09-13T16:50:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.628+0000 7fe92affd640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe918066690 0x7fe918068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.628+0000 7fe92affd640 1 -- 172.21.15.39:0/4176562596 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7fe92c084590 con 0x7fe9340abc50 2024-09-13T16:50:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.628+0000 7fe93a294640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe918066690 0x7fe918068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.628+0000 7fe93a294640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe918066690 0x7fe918068b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe93000a9a0 tx=0x7fe930008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:53.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.632+0000 7fe92affd640 1 -- 172.21.15.39:0/4176562596 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe92c051540 con 0x7fe9340abc50 2024-09-13T16:50:53.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.852+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe934150110 con 0x7fe9340abc50 2024-09-13T16:50:53.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.852+0000 7fe92affd640 1 -- 172.21.15.39:0/4176562596 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5176 (secure 0 0 0) 0x7fe92c054b60 con 0x7fe9340abc50 2024-09-13T16:50:53.859 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:53.859 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:53.859 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-13T16:50:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe918066690 msgr2=0x7fe918068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe918066690 0x7fe918068b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe93000a9a0 tx=0x7fe930008040 comp rx=0 tx=0).stop 2024-09-13T16:50:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340abc50 msgr2=0x7fe934141940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340abc50 0x7fe934141940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b970 tx=0x7fe92c00be30 comp rx=0 tx=0).stop 2024-09-13T16:50:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 shutdown_connections 2024-09-13T16:50:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe918066690 0x7fe918068b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9340ab340 0x7fe93414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9340abc50 0x7fe934141940 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 --2- 172.21.15.39:0/4176562596 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe93406cbc0 0x7fe934146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 >> 172.21.15.39:0/4176562596 conn(0x7fe934069e10 msgr2=0x7fe9340aa930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 shutdown_connections 2024-09-13T16:50:53.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:53.856+0000 7fe93c4f8640 1 -- 172.21.15.39:0/4176562596 wait complete. 2024-09-13T16:50:53.881 INFO:tasks.mds_thrash.fs.[a]:mds.d has gained rank=0, replacing gid=4264 2024-09-13T16:50:53.881 INFO:tasks.mds_thrash.fs.[a]:waiting for 3 secs before reviving mds.b 2024-09-13T16:50:55.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:50:55.756 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:50:55.888 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T16:50:55.887+0000 7f6ab94d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:50:56.882 INFO:tasks.mds_thrash.fs.[a]:reviving mds.b 2024-09-13T16:50:56.882 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:50:56.883 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:50:56.885 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:50:56.886 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:56.970 INFO:tasks.ceph.mds.b.smithi084.stdout:starting mds.b at 2024-09-13T16:50:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.028+0000 7efc43113640 1 Processor -- start 2024-09-13T16:50:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.028+0000 7efc43113640 1 -- start start 2024-09-13T16:50:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc43113640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c150f80 0x7efc3c153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc43113640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc3c0c5330 0x7efc3c06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc43113640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c1546c0 con 0x7efc3c153e70 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc43113640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3c0c19a0 con 0x7efc3c0c5330 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc43113640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0c49f0 con 0x7efc3c150f80 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc3bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc3c0c5330 0x7efc3c06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc40eaf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c150f80 0x7efc3c153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc3bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc3c0c5330 0x7efc3c06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54794/0 (socket says 172.21.15.39:54794) 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc3bfff640 1 -- 172.21.15.39:0/1749919763 learned_addr learned my addr 172.21.15.39:0/1749919763 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc3bfff640 1 -- 172.21.15.39:0/1749919763 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c150f80 msgr2=0x7efc3c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc3bfff640 1 --2- 172.21.15.39:0/1749919763 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c150f80 0x7efc3c153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc3b7fe640 1 -- 172.21.15.39:0/1749919763 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437175631 0 0) 0x7efc3c1546c0 con 0x7efc3c153e70 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc3bfff640 1 -- 172.21.15.39:0/1749919763 >> v1:172.21.15.84:6789/0 conn(0x7efc3c153e70 legacy=0x7efc3c06c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc3bfff640 1 -- 172.21.15.39:0/1749919763 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c1543e0 con 0x7efc3c0c5330 2024-09-13T16:50:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.032+0000 7efc3bfff640 1 --2- 172.21.15.39:0/1749919763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c0c5330 0x7efc3c06c090 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efc30009870 tx=0x7efc3001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2617027548e93f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3b7fe640 1 -- 172.21.15.39:0/1749919763 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc300ab600 con 0x7efc3c0c5330 2024-09-13T16:50:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3b7fe640 1 -- 172.21.15.39:0/1749919763 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc300a7050 con 0x7efc3c0c5330 2024-09-13T16:50:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3b7fe640 1 -- 172.21.15.39:0/1749919763 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc300ab980 con 0x7efc3c0c5330 2024-09-13T16:50:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- 172.21.15.39:0/1749919763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c0c5330 msgr2=0x7efc3c06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 --2- 172.21.15.39:0/1749919763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c0c5330 0x7efc3c06c090 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efc30009870 tx=0x7efc3001c4a0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- 172.21.15.39:0/1749919763 shutdown_connections 2024-09-13T16:50:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 --2- 172.21.15.39:0/1749919763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c0c5330 0x7efc3c06c090 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 --2- 172.21.15.39:0/1749919763 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c150f80 0x7efc3c153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- 172.21.15.39:0/1749919763 >> 172.21.15.39:0/1749919763 conn(0x7efc3c069e10 msgr2=0x7efc3c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- 172.21.15.39:0/1749919763 shutdown_connections 2024-09-13T16:50:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- 172.21.15.39:0/1749919763 wait complete. 2024-09-13T16:50:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 Processor -- start 2024-09-13T16:50:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- start start 2024-09-13T16:50:57.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc3c0c5330 0x7efc3c0a9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c150f80 0x7efc3c145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c153e70 0x7efc3c1460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0a60d0 con 0x7efc3c150f80 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0a5630 con 0x7efc3c0c5330 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0aabe0 con 0x7efc3c153e70 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c150f80 0x7efc3c145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc40eaf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc3c0c5330 0x7efc3c0a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c150f80 0x7efc3c145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54806/0 (socket says 172.21.15.39:54806) 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3bfff640 1 -- 172.21.15.39:0/3872856304 learned_addr learned my addr 172.21.15.39:0/3872856304 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc416b0640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c153e70 0x7efc3c1460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3bfff640 1 -- 172.21.15.39:0/3872856304 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c153e70 msgr2=0x7efc3c1460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3bfff640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c153e70 0x7efc3c1460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3bfff640 1 -- 172.21.15.39:0/3872856304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc3c0c5330 msgr2=0x7efc3c0a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3bfff640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc3c0c5330 0x7efc3c0a9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3bfff640 1 -- 172.21.15.39:0/3872856304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300094d0 con 0x7efc3c150f80 2024-09-13T16:50:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc3bfff640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c150f80 0x7efc3c145b70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efc300ae040 tx=0x7efc30019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc397fa640 1 -- 172.21.15.39:0/3872856304 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc300b4440 con 0x7efc3c150f80 2024-09-13T16:50:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc397fa640 1 -- 172.21.15.39:0/3872856304 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc300b45a0 con 0x7efc3c150f80 2024-09-13T16:50:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c0a58d0 con 0x7efc3c150f80 2024-09-13T16:50:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc397fa640 1 -- 172.21.15.39:0/3872856304 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc300b4950 con 0x7efc3c150f80 2024-09-13T16:50:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.036+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c0aaec0 con 0x7efc3c150f80 2024-09-13T16:50:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.040+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c153930 con 0x7efc3c150f80 2024-09-13T16:50:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.040+0000 7efc397fa640 1 -- 172.21.15.39:0/3872856304 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc300b2070 con 0x7efc3c150f80 2024-09-13T16:50:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.040+0000 7efc397fa640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc200666c0 0x7efc20068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.040+0000 7efc397fa640 1 -- 172.21.15.39:0/3872856304 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7efc3011e240 con 0x7efc3c150f80 2024-09-13T16:50:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.040+0000 7efc40eaf640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc200666c0 0x7efc20068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.040+0000 7efc40eaf640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc200666c0 0x7efc20068b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc280045b0 tx=0x7efc2800f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:57.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.044+0000 7efc397fa640 1 -- 172.21.15.39:0/3872856304 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efc300eb250 con 0x7efc3c150f80 2024-09-13T16:50:57.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.272+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc3c0c3c00 con 0x7efc3c150f80 2024-09-13T16:50:57.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.272+0000 7efc397fa640 1 -- 172.21.15.39:0/3872856304 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5176 (secure 0 0 0) 0x7efc300ee870 con 0x7efc3c150f80 2024-09-13T16:50:57.279 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:57.279 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:57.279 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-09-13T16:50:57.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc200666c0 msgr2=0x7efc20068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc200666c0 0x7efc20068b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc280045b0 tx=0x7efc2800f040 comp rx=0 tx=0).stop 2024-09-13T16:50:57.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c150f80 msgr2=0x7efc3c145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:57.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c150f80 0x7efc3c145b70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efc300ae040 tx=0x7efc30019240 comp rx=0 tx=0).stop 2024-09-13T16:50:57.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 shutdown_connections 2024-09-13T16:50:57.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc200666c0 0x7efc20068b80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc3c153e70 0x7efc3c1460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc3c150f80 0x7efc3c145b70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 --2- 172.21.15.39:0/3872856304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc3c0c5330 0x7efc3c0a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:57.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 >> 172.21.15.39:0/3872856304 conn(0x7efc3c069e10 msgr2=0x7efc3c152b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:57.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 shutdown_connections 2024-09-13T16:50:57.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:57.276+0000 7efc43113640 1 -- 172.21.15.39:0/3872856304 wait complete. 2024-09-13T16:50:57.299 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2024-09-13T16:50:58.088 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:58.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd77b3640 1 Processor -- start 2024-09-13T16:50:58.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd77b3640 1 -- start start 2024-09-13T16:50:58.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd77b3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c2370 0x7f3dd00c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd77b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3dd00c5870 0x7f3dd015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd77b3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd0153e30 con 0x7f3dd0154bb0 2024-09-13T16:50:58.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd77b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd00720f0 con 0x7f3dd00c5870 2024-09-13T16:50:58.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd77b3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0151b30 con 0x7f3dd00c2370 2024-09-13T16:50:58.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd5d50640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3dd00c5870 0x7f3dd015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd5d50640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3dd00c5870 0x7f3dd015e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54814/0 (socket says 172.21.15.39:54814) 2024-09-13T16:50:58.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd5d50640 1 -- 172.21.15.39:0/4291125314 learned_addr learned my addr 172.21.15.39:0/4291125314 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:58.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd554f640 1 --2- 172.21.15.39:0/4291125314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c2370 0x7f3dd00c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd5d50640 1 -- 172.21.15.39:0/4291125314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c2370 msgr2=0x7f3dd00c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:58.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd5d50640 1 --2- 172.21.15.39:0/4291125314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c2370 0x7f3dd00c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd5d50640 1 -- 172.21.15.39:0/4291125314 >> v1:172.21.15.84:6789/0 conn(0x7f3dd0154bb0 legacy=0x7f3dd00c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd5d50640 1 -- 172.21.15.39:0/4291125314 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00a7000 con 0x7f3dd00c5870 2024-09-13T16:50:58.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.248+0000 7f3dd554f640 1 --2- 172.21.15.39:0/4291125314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c2370 0x7f3dd00c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:50:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd5d50640 1 --2- 172.21.15.39:0/4291125314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd00c5870 0x7f3dd015e7b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009e50 tx=0x7f3dc801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559af05107c448d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dc7fff640 1 -- 172.21.15.39:0/4291125314 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80a5750 con 0x7f3dd00c5870 2024-09-13T16:50:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dc7fff640 1 -- 172.21.15.39:0/4291125314 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc80a58b0 con 0x7f3dd00c5870 2024-09-13T16:50:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dc7fff640 1 -- 172.21.15.39:0/4291125314 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80a5c40 con 0x7f3dd00c5870 2024-09-13T16:50:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 -- 172.21.15.39:0/4291125314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd00c5870 msgr2=0x7f3dd015e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 --2- 172.21.15.39:0/4291125314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd00c5870 0x7f3dd015e7b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009e50 tx=0x7f3dc801c9a0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 -- 172.21.15.39:0/4291125314 shutdown_connections 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 --2- 172.21.15.39:0/4291125314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd00c5870 0x7f3dd015e7b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 --2- 172.21.15.39:0/4291125314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c2370 0x7f3dd00c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 -- 172.21.15.39:0/4291125314 >> 172.21.15.39:0/4291125314 conn(0x7f3dd00a9720 msgr2=0x7f3dd00a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 -- 172.21.15.39:0/4291125314 shutdown_connections 2024-09-13T16:50:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 -- 172.21.15.39:0/4291125314 wait complete. 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 Processor -- start 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 -- start start 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd00c2370 0x7f3dd0143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c5870 0x7f3dd006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd0154bb0 0x7f3dd006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.252+0000 7f3dd77b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd014ee60 con 0x7f3dd0154bb0 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd77b3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0143fa0 con 0x7f3dd00c2370 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd77b3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd01515f0 con 0x7f3dd00c5870 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd5d50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd0154bb0 0x7f3dd006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd5d50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd0154bb0 0x7f3dd006d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54826/0 (socket says 172.21.15.39:54826) 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd5d50640 1 -- 172.21.15.39:0/852484995 learned_addr learned my addr 172.21.15.39:0/852484995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd554f640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd00c2370 0x7f3dd0143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd4d4e640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c5870 0x7f3dd006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd5d50640 1 -- 172.21.15.39:0/852484995 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c5870 msgr2=0x7f3dd006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd5d50640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c5870 0x7f3dd006d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd5d50640 1 -- 172.21.15.39:0/852484995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd00c2370 msgr2=0x7f3dd0143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd5d50640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd00c2370 0x7f3dd0143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd5d50640 1 -- 172.21.15.39:0/852484995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc8009b30 con 0x7f3dd0154bb0 2024-09-13T16:50:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd5d50640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd0154bb0 0x7f3dd006d690 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80af040 tx=0x7f3dc8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dc5ffb640 1 -- 172.21.15.39:0/852484995 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80b8670 con 0x7f3dd0154bb0 2024-09-13T16:50:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dc5ffb640 1 -- 172.21.15.39:0/852484995 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc80b87d0 con 0x7f3dd0154bb0 2024-09-13T16:50:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd01ecf10 con 0x7f3dd0154bb0 2024-09-13T16:50:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dc5ffb640 1 -- 172.21.15.39:0/852484995 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80b8b60 con 0x7f3dd0154bb0 2024-09-13T16:50:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd01441f0 con 0x7f3dd0154bb0 2024-09-13T16:50:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd00c2750 con 0x7f3dd0154bb0 2024-09-13T16:50:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dc5ffb640 1 -- 172.21.15.39:0/852484995 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3dc80b8cc0 con 0x7f3dd0154bb0 2024-09-13T16:50:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dc5ffb640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db4066690 0x7f3db4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd554f640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db4066690 0x7f3db4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dc5ffb640 1 -- 172.21.15.39:0/852484995 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7f3dc811fad0 con 0x7f3dd0154bb0 2024-09-13T16:50:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.256+0000 7f3dd554f640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db4066690 0x7f3db4068b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009950 tx=0x7f3dcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.264+0000 7f3dc5ffb640 1 -- 172.21.15.39:0/852484995 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3dc80ecad0 con 0x7f3dd0154bb0 2024-09-13T16:50:58.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.492+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dd01444f0 con 0x7f3dd0154bb0 2024-09-13T16:50:58.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.492+0000 7f3dc5ffb640 1 -- 172.21.15.39:0/852484995 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f3dc80f00f0 con 0x7f3dd0154bb0 2024-09-13T16:50:58.499 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:58.499 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:58.500 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-09-13T16:50:58.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db4066690 msgr2=0x7f3db4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db4066690 0x7f3db4068b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009950 tx=0x7f3dcc008040 comp rx=0 tx=0).stop 2024-09-13T16:50:58.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd0154bb0 msgr2=0x7f3dd006d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd0154bb0 0x7f3dd006d690 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80af040 tx=0x7f3dc8019740 comp rx=0 tx=0).stop 2024-09-13T16:50:58.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 shutdown_connections 2024-09-13T16:50:58.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3db4066690 0x7f3db4068b50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd0154bb0 0x7f3dd006d690 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3dd00c5870 0x7f3dd006d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 --2- 172.21.15.39:0/852484995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3dd00c2370 0x7f3dd0143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 >> 172.21.15.39:0/852484995 conn(0x7f3dd00a9720 msgr2=0x7f3dd006a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:58.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 shutdown_connections 2024-09-13T16:50:58.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.496+0000 7f3dd77b3640 1 -- 172.21.15.39:0/852484995 wait complete. 2024-09-13T16:50:58.520 INFO:tasks.mds_thrash.fs.[b]:kill mds.a (rank=0) 2024-09-13T16:50:58.521 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:50:58.521 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:50:58.536 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:50:58.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:58.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.700+0000 7f7888fca640 1 Processor -- start 2024-09-13T16:50:58.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.700+0000 7f7888fca640 1 -- start start 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7888fca640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7884070b40 0x7f78840c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7888fca640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840bcfb0 0x7f788415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7888fca640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7884157360 con 0x7f78840c4260 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7888fca640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78840c80b0 con 0x7f7884070b40 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7888fca640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78841575b0 con 0x7f78840bcfb0 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7881d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7884070b40 0x7f78840c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7881d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7884070b40 0x7f78840c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54842/0 (socket says 172.21.15.39:54842) 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7881d9b640 1 -- 172.21.15.39:0/269499059 learned_addr learned my addr 172.21.15.39:0/269499059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7882d9d640 1 --2- 172.21.15.39:0/269499059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840bcfb0 0x7f788415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7881d9b640 1 -- 172.21.15.39:0/269499059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840bcfb0 msgr2=0x7f788415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:58.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7881d9b640 1 --2- 172.21.15.39:0/269499059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840bcfb0 0x7f788415e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7881d9b640 1 -- 172.21.15.39:0/269499059 >> v1:172.21.15.84:6789/0 conn(0x7f78840c4260 legacy=0x7f7884070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7881d9b640 1 -- 172.21.15.39:0/269499059 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7884157890 con 0x7f7884070b40 2024-09-13T16:50:58.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7882d9d640 1 --2- 172.21.15.39:0/269499059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840bcfb0 0x7f788415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7881d9b640 1 --2- 172.21.15.39:0/269499059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7884070b40 0x7f78840c4640 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7878009870 tx=0x7f787801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87ad1d32ce3b2a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f788159a640 1 -- 172.21.15.39:0/269499059 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78840c7330 con 0x7f7884070b40 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f788159a640 1 -- 172.21.15.39:0/269499059 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78780a7050 con 0x7f7884070b40 2024-09-13T16:50:58.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f788159a640 1 -- 172.21.15.39:0/269499059 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78780ab820 con 0x7f7884070b40 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7888fca640 1 -- 172.21.15.39:0/269499059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7884070b40 msgr2=0x7f78840c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.704+0000 7f7888fca640 1 --2- 172.21.15.39:0/269499059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7884070b40 0x7f78840c4640 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7878009870 tx=0x7f787801c400 comp rx=0 tx=0).stop 2024-09-13T16:50:58.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- 172.21.15.39:0/269499059 shutdown_connections 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 --2- 172.21.15.39:0/269499059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840bcfb0 0x7f788415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 --2- 172.21.15.39:0/269499059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7884070b40 0x7f78840c4640 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- 172.21.15.39:0/269499059 >> 172.21.15.39:0/269499059 conn(0x7f7884069e10 msgr2=0x7f788406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- 172.21.15.39:0/269499059 shutdown_connections 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- 172.21.15.39:0/269499059 wait complete. 2024-09-13T16:50:58.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 Processor -- start 2024-09-13T16:50:58.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- start start 2024-09-13T16:50:58.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78840bcfb0 0x7f78840ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840c4260 0x7f78840bde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f788415e720 0x7f78840bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840bda40 con 0x7f78840bcfb0 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788414b6d0 con 0x7f788415e720 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c9aa0 con 0x7f78840c4260 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f788259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78840bcfb0 0x7f78840ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f788259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78840bcfb0 0x7f78840ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54850/0 (socket says 172.21.15.39:54850) 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f788259c640 1 -- 172.21.15.39:0/3331494359 learned_addr learned my addr 172.21.15.39:0/3331494359 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7882d9d640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f788415e720 0x7f78840bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7881d9b640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840c4260 0x7f78840bde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f788259c640 1 -- 172.21.15.39:0/3331494359 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840c4260 msgr2=0x7f78840bde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f788259c640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840c4260 0x7f78840bde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f788259c640 1 -- 172.21.15.39:0/3331494359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f788415e720 msgr2=0x7f78840bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f788259c640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f788415e720 0x7f78840bfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f788259c640 1 -- 172.21.15.39:0/3331494359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780094d0 con 0x7f78840bcfb0 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f788259c640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78840bcfb0 0x7f78840ca2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f787400c410 tx=0x7f787400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7872ffd640 1 -- 172.21.15.39:0/3331494359 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874011790 con 0x7f78840bcfb0 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7872ffd640 1 -- 172.21.15.39:0/3331494359 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7874011910 con 0x7f78840bcfb0 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78840bd480 con 0x7f78840bcfb0 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7872ffd640 1 -- 172.21.15.39:0/3331494359 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874011d00 con 0x7f78840bcfb0 2024-09-13T16:50:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.708+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78840be360 con 0x7f78840bcfb0 2024-09-13T16:50:58.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.712+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78840c4640 con 0x7f78840bcfb0 2024-09-13T16:50:58.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.712+0000 7f7872ffd640 1 -- 172.21.15.39:0/3331494359 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f787401c460 con 0x7f78840bcfb0 2024-09-13T16:50:58.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.712+0000 7f7872ffd640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7864066690 0x7f7864068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:58.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.712+0000 7f7872ffd640 1 -- 172.21.15.39:0/3331494359 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7f78740844e0 con 0x7f78840bcfb0 2024-09-13T16:50:58.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.712+0000 7f7881d9b640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7864066690 0x7f7864068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:58.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.712+0000 7f7881d9b640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7864066690 0x7f7864068b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f78780ad000 tx=0x7f78780a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:58.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.716+0000 7f7872ffd640 1 -- 172.21.15.39:0/3331494359 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f78740513a0 con 0x7f78840bcfb0 2024-09-13T16:50:58.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.968+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78840c6e70 con 0x7f78840bcfb0 2024-09-13T16:50:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.968+0000 7f7872ffd640 1 -- 172.21.15.39:0/3331494359 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f7874011e60 con 0x7f78840bcfb0 2024-09-13T16:50:58.974 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:58.974 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-09-13T16:50:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7864066690 msgr2=0x7f7864068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7864066690 0x7f7864068b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f78780ad000 tx=0x7f78780a6000 comp rx=0 tx=0).stop 2024-09-13T16:50:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78840bcfb0 msgr2=0x7f78840ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78840bcfb0 0x7f78840ca2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f787400c410 tx=0x7f787400c8d0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 shutdown_connections 2024-09-13T16:50:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7864066690 0x7f7864068b50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f788415e720 0x7f78840bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78840c4260 0x7f78840bde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 --2- 172.21.15.39:0/3331494359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78840bcfb0 0x7f78840ca2f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 >> 172.21.15.39:0/3331494359 conn(0x7f7884069e10 msgr2=0x7f7884150910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 shutdown_connections 2024-09-13T16:50:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:58.972+0000 7f7888fca640 1 -- 172.21.15.39:0/3331494359 wait complete. 2024-09-13T16:50:58.994 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:50:59.300 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:50:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.420+0000 7fb384f9c640 1 Processor -- start 2024-09-13T16:50:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb384f9c640 1 -- start start 2024-09-13T16:50:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb384f9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c6130 0x7fb3800c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb384f9c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb380157d50 0x7fb38006ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb384f9c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3800c6510 con 0x7fb3800c0240 2024-09-13T16:50:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb384f9c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3800c1c80 con 0x7fb380157d50 2024-09-13T16:50:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb384f9c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3801544f0 con 0x7fb3800c6130 2024-09-13T16:50:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb380157d50 0x7fb38006ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c6130 0x7fb3800c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37e59c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c6130 0x7fb3800c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:51772/0 (socket says 172.21.15.39:51772) 2024-09-13T16:50:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37ed9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb380157d50 0x7fb38006ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:41272/0 (socket says 172.21.15.39:41272) 2024-09-13T16:50:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37e59c640 1 -- 172.21.15.39:0/1112274199 learned_addr learned my addr 172.21.15.39:0/1112274199 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37e59c640 1 -- 172.21.15.39:0/1112274199 >> v2:172.21.15.84:3300/0 conn(0x7fb380157d50 msgr2=0x7fb38006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:50:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37e59c640 1 --2- 172.21.15.39:0/1112274199 >> v2:172.21.15.84:3300/0 conn(0x7fb380157d50 0x7fb38006ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37e59c640 1 -- 172.21.15.39:0/1112274199 >> v1:172.21.15.84:6789/0 conn(0x7fb3800c0240 legacy=0x7fb380153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37e59c640 1 -- 172.21.15.39:0/1112274199 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb380158200 con 0x7fb3800c6130 2024-09-13T16:50:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.424+0000 7fb37ed9d640 1 --2- 172.21.15.39:0/1112274199 >> v2:172.21.15.84:3300/0 conn(0x7fb380157d50 0x7fb38006ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:50:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb37e59c640 1 --2- 172.21.15.39:0/1112274199 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c6130 0x7fb3800c24c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb370009870 tx=0x7fb37001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16882e66331cb94 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb37dd9b640 1 -- 172.21.15.39:0/1112274199 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3700a9640 con 0x7fb3800c6130 2024-09-13T16:50:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb37dd9b640 1 -- 172.21.15.39:0/1112274199 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3700a97a0 con 0x7fb3800c6130 2024-09-13T16:50:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb37dd9b640 1 -- 172.21.15.39:0/1112274199 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3700a9b30 con 0x7fb3800c6130 2024-09-13T16:50:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 -- 172.21.15.39:0/1112274199 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c6130 msgr2=0x7fb3800c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 --2- 172.21.15.39:0/1112274199 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c6130 0x7fb3800c24c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb370009870 tx=0x7fb37001c560 comp rx=0 tx=0).stop 2024-09-13T16:50:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 -- 172.21.15.39:0/1112274199 shutdown_connections 2024-09-13T16:50:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 --2- 172.21.15.39:0/1112274199 >> v2:172.21.15.84:3300/0 conn(0x7fb380157d50 0x7fb38006ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 --2- 172.21.15.39:0/1112274199 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c6130 0x7fb3800c24c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 -- 172.21.15.39:0/1112274199 >> 172.21.15.39:0/1112274199 conn(0x7fb3800c32a0 msgr2=0x7fb3800c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 -- 172.21.15.39:0/1112274199 shutdown_connections 2024-09-13T16:50:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 -- 172.21.15.39:0/1112274199 wait complete. 2024-09-13T16:50:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 Processor -- start 2024-09-13T16:50:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.428+0000 7fb384f9c640 1 -- start start 2024-09-13T16:50:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb384f9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c0240 0x7fb38006c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb384f9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3800c6130 0x7fb3800713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb384f9c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb380157d50 0x7fb38006cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb384f9c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38014e6c0 con 0x7fb380157d50 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb384f9c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3800708f0 con 0x7fb3800c6130 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb384f9c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3801f0b60 con 0x7fb3800c0240 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb380157d50 0x7fb38006cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb380157d50 0x7fb38006cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54858/0 (socket says 172.21.15.39:54858) 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37ed9d640 1 -- 172.21.15.39:0/1988876944 learned_addr learned my addr 172.21.15.39:0/1988876944 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb377fff640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3800c6130 0x7fb3800713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37e59c640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c0240 0x7fb38006c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37ed9d640 1 -- 172.21.15.39:0/1988876944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c0240 msgr2=0x7fb38006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37ed9d640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c0240 0x7fb38006c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37ed9d640 1 -- 172.21.15.39:0/1988876944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3800c6130 msgr2=0x7fb3800713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37ed9d640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3800c6130 0x7fb3800713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37ed9d640 1 -- 172.21.15.39:0/1988876944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb378009a70 con 0x7fb380157d50 2024-09-13T16:50:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37ed9d640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb380157d50 0x7fb38006cec0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb37800efc0 tx=0x7fb37800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb376ffd640 1 -- 172.21.15.39:0/1988876944 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378009cd0 con 0x7fb380157d50 2024-09-13T16:50:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb376ffd640 1 -- 172.21.15.39:0/1988876944 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37800f040 con 0x7fb380157d50 2024-09-13T16:50:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3700094d0 con 0x7fb380157d50 2024-09-13T16:50:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb376ffd640 1 -- 172.21.15.39:0/1988876944 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3780039a0 con 0x7fb380157d50 2024-09-13T16:50:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb380070580 con 0x7fb380157d50 2024-09-13T16:50:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3800c24c0 con 0x7fb380157d50 2024-09-13T16:50:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb376ffd640 1 -- 172.21.15.39:0/1988876944 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb378014480 con 0x7fb380157d50 2024-09-13T16:50:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb376ffd640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb3600666c0 0x7fb360068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:50:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb376ffd640 1 -- 172.21.15.39:0/1988876944 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7fb378085130 con 0x7fb380157d50 2024-09-13T16:50:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.432+0000 7fb37e59c640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb3600666c0 0x7fb360068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:50:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.436+0000 7fb37e59c640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb3600666c0 0x7fb360068b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb3700ae000 tx=0x7fb3700b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:50:59.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.440+0000 7fb376ffd640 1 -- 172.21.15.39:0/1988876944 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb378052090 con 0x7fb380157d50 2024-09-13T16:50:59.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.680+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3800c0790 con 0x7fb380157d50 2024-09-13T16:50:59.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.680+0000 7fb376ffd640 1 -- 172.21.15.39:0/1988876944 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fb3780556b0 con 0x7fb380157d50 2024-09-13T16:50:59.686 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:50:59.686 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:50:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-09-13T16:50:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb3600666c0 msgr2=0x7fb360068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb3600666c0 0x7fb360068b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb3700ae000 tx=0x7fb3700b3040 comp rx=0 tx=0).stop 2024-09-13T16:50:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb380157d50 msgr2=0x7fb38006cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:50:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb380157d50 0x7fb38006cec0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb37800efc0 tx=0x7fb37800c490 comp rx=0 tx=0).stop 2024-09-13T16:50:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 shutdown_connections 2024-09-13T16:50:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb3600666c0 0x7fb360068b80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3800c6130 0x7fb3800713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb380157d50 0x7fb38006cec0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 --2- 172.21.15.39:0/1988876944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb3800c0240 0x7fb38006c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:50:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 >> 172.21.15.39:0/1988876944 conn(0x7fb3800c32a0 msgr2=0x7fb3800c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:50:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 shutdown_connections 2024-09-13T16:50:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:50:59.684+0000 7fb384f9c640 1 -- 172.21.15.39:0/1988876944 wait complete. 2024-09-13T16:50:59.705 INFO:tasks.mds_thrash.fs.[a]:mds.b reported in up:standby state 2024-09-13T16:50:59.705 INFO:tasks.mds_thrash.fs.[a]:waiting for 105 secs before thrashing 2024-09-13T16:51:00.995 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8127fb8640 1 Processor -- start 2024-09-13T16:51:01.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8127fb8640 1 -- start start 2024-09-13T16:51:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8127fb8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8120065ee0 0x7f81200662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8127fb8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81200c2ee0 0x7f81200c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8127fb8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812006ddf0 con 0x7f81200c6270 2024-09-13T16:51:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8127fb8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81200c4710 con 0x7f81200c2ee0 2024-09-13T16:51:01.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8127fb8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81200c49d0 con 0x7f8120065ee0 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8126555640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f81200c6270 0x7f81200bfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:45526/0 (socket says 172.21.15.39:45526) 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8125553640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81200c2ee0 0x7f81200c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8126555640 1 -- 172.21.15.39:0/2523858765 learned_addr learned my addr 172.21.15.39:0/2523858765 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8125d54640 1 --2- 172.21.15.39:0/2523858765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8120065ee0 0x7f81200662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8124d52640 1 -- 172.21.15.39:0/2523858765 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587204981 0 0) 0x7f812006ddf0 con 0x7f81200c6270 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8124d52640 1 -- 172.21.15.39:0/2523858765 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f810c003530 con 0x7f81200c6270 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8125553640 1 -- 172.21.15.39:0/2523858765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8120065ee0 msgr2=0x7f81200662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8125553640 1 --2- 172.21.15.39:0/2523858765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8120065ee0 0x7f81200662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.124+0000 7f8125553640 1 -- 172.21.15.39:0/2523858765 >> v1:172.21.15.39:6789/0 conn(0x7f81200c6270 legacy=0x7f81200bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8125553640 1 -- 172.21.15.39:0/2523858765 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8120154880 con 0x7f81200c2ee0 2024-09-13T16:51:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8125553640 1 --2- 172.21.15.39:0/2523858765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 0x7f81200c2220 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f81100097c0 tx=0x7f811001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3676e8ea178fccce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8124d52640 1 -- 172.21.15.39:0/2523858765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100ab600 con 0x7f81200c2ee0 2024-09-13T16:51:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8124d52640 1 -- 172.21.15.39:0/2523858765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81100a7050 con 0x7f81200c2ee0 2024-09-13T16:51:01.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8124d52640 1 -- 172.21.15.39:0/2523858765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100ab980 con 0x7f81200c2ee0 2024-09-13T16:51:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8127fb8640 1 -- 172.21.15.39:0/2523858765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 msgr2=0x7f81200c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8127fb8640 1 --2- 172.21.15.39:0/2523858765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 0x7f81200c2220 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f81100097c0 tx=0x7f811001c410 comp rx=0 tx=0).stop 2024-09-13T16:51:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8127fb8640 1 -- 172.21.15.39:0/2523858765 shutdown_connections 2024-09-13T16:51:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8127fb8640 1 --2- 172.21.15.39:0/2523858765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 0x7f81200c2220 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8127fb8640 1 --2- 172.21.15.39:0/2523858765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8120065ee0 0x7f81200662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8127fb8640 1 -- 172.21.15.39:0/2523858765 >> 172.21.15.39:0/2523858765 conn(0x7f81200c3310 msgr2=0x7f81200c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8127fb8640 1 -- 172.21.15.39:0/2523858765 shutdown_connections 2024-09-13T16:51:01.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.128+0000 7f8127fb8640 1 -- 172.21.15.39:0/2523858765 wait complete. 2024-09-13T16:51:01.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 Processor -- start 2024-09-13T16:51:01.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 -- start start 2024-09-13T16:51:01.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8120065ee0 0x7f8120153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 0x7f812014f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81200c6270 0x7f812014fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8120155150 con 0x7f81200c2ee0 2024-09-13T16:51:01.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812014ff80 con 0x7f8120065ee0 2024-09-13T16:51:01.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812014f070 con 0x7f81200c6270 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125553640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 0x7f812014f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125553640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 0x7f812014f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51718/0 (socket says 172.21.15.39:51718) 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125553640 1 -- 172.21.15.39:0/2224294169 learned_addr learned my addr 172.21.15.39:0/2224294169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125d54640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8120065ee0 0x7f8120153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8126555640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81200c6270 0x7f812014fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125553640 1 -- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81200c6270 msgr2=0x7f812014fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125553640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81200c6270 0x7f812014fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125553640 1 -- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8120065ee0 msgr2=0x7f8120153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125553640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8120065ee0 0x7f8120153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125553640 1 -- 172.21.15.39:0/2224294169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110009420 con 0x7f81200c2ee0 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8126555640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81200c6270 0x7f812014fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125d54640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8120065ee0 0x7f8120153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:01.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8125553640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 0x7f812014f500 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f81100ae040 tx=0x7f81100191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f81167fc640 1 -- 172.21.15.39:0/2224294169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100b4430 con 0x7f81200c2ee0 2024-09-13T16:51:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f81167fc640 1 -- 172.21.15.39:0/2224294169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81100b4590 con 0x7f81200c2ee0 2024-09-13T16:51:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8120155640 con 0x7f81200c2ee0 2024-09-13T16:51:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8120155360 con 0x7f81200c2ee0 2024-09-13T16:51:01.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f81167fc640 1 -- 172.21.15.39:0/2224294169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100b4940 con 0x7f81200c2ee0 2024-09-13T16:51:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.132+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8108005260 con 0x7f81200c2ee0 2024-09-13T16:51:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.136+0000 7f81167fc640 1 -- 172.21.15.39:0/2224294169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f81100b2040 con 0x7f81200c2ee0 2024-09-13T16:51:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.136+0000 7f81167fc640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8104066690 0x7f8104068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.136+0000 7f81167fc640 1 -- 172.21.15.39:0/2224294169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7f811011e280 con 0x7f81200c2ee0 2024-09-13T16:51:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.136+0000 7f8125d54640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8104066690 0x7f8104068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:01.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.136+0000 7f8125d54640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8104066690 0x7f8104068b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f81201583f0 tx=0x7f811c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:01.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.140+0000 7f81167fc640 1 -- 172.21.15.39:0/2224294169 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f81100eb1e0 con 0x7f81200c2ee0 2024-09-13T16:51:01.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.372+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8108005510 con 0x7f81200c2ee0 2024-09-13T16:51:01.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.372+0000 7f81167fc640 1 -- 172.21.15.39:0/2224294169 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f81100ee800 con 0x7f81200c2ee0 2024-09-13T16:51:01.377 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:01.377 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:01.378 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-09-13T16:51:01.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8104066690 msgr2=0x7f8104068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8104066690 0x7f8104068b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f81201583f0 tx=0x7f811c008040 comp rx=0 tx=0).stop 2024-09-13T16:51:01.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 msgr2=0x7f812014f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:01.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 0x7f812014f500 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f81100ae040 tx=0x7f81100191b0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 shutdown_connections 2024-09-13T16:51:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8104066690 0x7f8104068b50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81200c6270 0x7f812014fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81200c2ee0 0x7f812014f500 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 --2- 172.21.15.39:0/2224294169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8120065ee0 0x7f8120153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 >> 172.21.15.39:0/2224294169 conn(0x7f81200c3310 msgr2=0x7f8120069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:01.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 shutdown_connections 2024-09-13T16:51:01.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:01.376+0000 7f8127fb8640 1 -- 172.21.15.39:0/2224294169 wait complete. 2024-09-13T16:51:01.398 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:51:03.399 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:03.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.536+0000 7fc8a16e6640 1 Processor -- start 2024-09-13T16:51:03.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.536+0000 7fc8a16e6640 1 -- start start 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.536+0000 7fc8a16e6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc89c151500 0x7fc89c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.536+0000 7fc8a16e6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc89c0c47c0 0x7fc89c154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.536+0000 7fc8a16e6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0c5a30 con 0x7fc89c071dd0 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.536+0000 7fc8a16e6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c1576a0 con 0x7fc89c151500 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.536+0000 7fc8a16e6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c157980 con 0x7fc89c0c47c0 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc89affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc89c151500 0x7fc89c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc89a7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc89c071dd0 0x7fc89c154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:45536/0 (socket says 172.21.15.39:45536) 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc89a7fc640 1 -- 172.21.15.39:0/2673565959 learned_addr learned my addr 172.21.15.39:0/2673565959 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc89b7fe640 1 --2- 172.21.15.39:0/2673565959 >> v2:172.21.15.84:3300/0 conn(0x7fc89c0c47c0 0x7fc89c154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc899ffb640 1 -- 172.21.15.39:0/2673565959 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337753109 0 0) 0x7fc89c0c5a30 con 0x7fc89c071dd0 2024-09-13T16:51:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc899ffb640 1 -- 172.21.15.39:0/2673565959 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc884003530 con 0x7fc89c071dd0 2024-09-13T16:51:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc89affd640 1 -- 172.21.15.39:0/2673565959 >> v2:172.21.15.84:3300/0 conn(0x7fc89c0c47c0 msgr2=0x7fc89c154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc89affd640 1 --2- 172.21.15.39:0/2673565959 >> v2:172.21.15.84:3300/0 conn(0x7fc89c0c47c0 0x7fc89c154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc89affd640 1 -- 172.21.15.39:0/2673565959 >> v1:172.21.15.39:6789/0 conn(0x7fc89c071dd0 legacy=0x7fc89c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc89affd640 1 -- 172.21.15.39:0/2673565959 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c157c60 con 0x7fc89c151500 2024-09-13T16:51:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc89affd640 1 --2- 172.21.15.39:0/2673565959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c151500 0x7fc89c0c4280 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0097c0 tx=0x7fc88c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199524ce36f2ab16 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc899ffb640 1 -- 172.21.15.39:0/2673565959 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0ac620 con 0x7fc89c151500 2024-09-13T16:51:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc899ffb640 1 -- 172.21.15.39:0/2673565959 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c0ac780 con 0x7fc89c151500 2024-09-13T16:51:03.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc899ffb640 1 -- 172.21.15.39:0/2673565959 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0acaf0 con 0x7fc89c151500 2024-09-13T16:51:03.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc8a16e6640 1 -- 172.21.15.39:0/2673565959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c151500 msgr2=0x7fc89c0c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc8a16e6640 1 --2- 172.21.15.39:0/2673565959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c151500 0x7fc89c0c4280 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0097c0 tx=0x7fc88c01c3a0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc8a16e6640 1 -- 172.21.15.39:0/2673565959 shutdown_connections 2024-09-13T16:51:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc8a16e6640 1 --2- 172.21.15.39:0/2673565959 >> v2:172.21.15.84:3300/0 conn(0x7fc89c0c47c0 0x7fc89c154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc8a16e6640 1 --2- 172.21.15.39:0/2673565959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c151500 0x7fc89c0c4280 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc8a16e6640 1 -- 172.21.15.39:0/2673565959 >> 172.21.15.39:0/2673565959 conn(0x7fc89c0ab530 msgr2=0x7fc89c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc8a16e6640 1 -- 172.21.15.39:0/2673565959 shutdown_connections 2024-09-13T16:51:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.540+0000 7fc8a16e6640 1 -- 172.21.15.39:0/2673565959 wait complete. 2024-09-13T16:51:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 Processor -- start 2024-09-13T16:51:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 -- start start 2024-09-13T16:51:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c071dd0 0x7fc89c152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c0c47c0 0x7fc89c1eca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c151500 0x7fc89c1ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c156a40 con 0x7fc89c0c47c0 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c14d2f0 con 0x7fc89c151500 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1ec540 con 0x7fc89c071dd0 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c0c47c0 0x7fc89c1eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c0c47c0 0x7fc89c1eca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51730/0 (socket says 172.21.15.39:51730) 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89a7fc640 1 -- 172.21.15.39:0/700586609 learned_addr learned my addr 172.21.15.39:0/700586609 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89affd640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c071dd0 0x7fc89c152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89b7fe640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c151500 0x7fc89c1ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89a7fc640 1 -- 172.21.15.39:0/700586609 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c071dd0 msgr2=0x7fc89c152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89a7fc640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c071dd0 0x7fc89c152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89a7fc640 1 -- 172.21.15.39:0/700586609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c151500 msgr2=0x7fc89c1ecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89a7fc640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c151500 0x7fc89c1ecf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89a7fc640 1 -- 172.21.15.39:0/700586609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c009420 con 0x7fc89c0c47c0 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89affd640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c071dd0 0x7fc89c152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc89a7fc640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c0c47c0 0x7fc89c1eca20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc890001210 tx=0x7fc89000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8837fe640 1 -- 172.21.15.39:0/700586609 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890005cf0 con 0x7fc89c0c47c0 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8837fe640 1 -- 172.21.15.39:0/700586609 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc890005e50 con 0x7fc89c0c47c0 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c1f06f0 con 0x7fc89c0c47c0 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8837fe640 1 -- 172.21.15.39:0/700586609 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900072d0 con 0x7fc89c0c47c0 2024-09-13T16:51:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.544+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c0c9320 con 0x7fc89c0c47c0 2024-09-13T16:51:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.548+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c1ec700 con 0x7fc89c0c47c0 2024-09-13T16:51:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.548+0000 7fc8837fe640 1 -- 172.21.15.39:0/700586609 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc890007430 con 0x7fc89c0c47c0 2024-09-13T16:51:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.548+0000 7fc8837fe640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc8880666c0 0x7fc888068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.548+0000 7fc89affd640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc8880666c0 0x7fc888068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.548+0000 7fc8837fe640 1 -- 172.21.15.39:0/700586609 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7fc890084200 con 0x7fc89c0c47c0 2024-09-13T16:51:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.548+0000 7fc89affd640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc8880666c0 0x7fc888068b80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0ae000 tx=0x7fc88c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:03.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.552+0000 7fc8837fe640 1 -- 172.21.15.39:0/700586609 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc8900512f0 con 0x7fc89c0c47c0 2024-09-13T16:51:03.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.764+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc89c0c4280 con 0x7fc89c0c47c0 2024-09-13T16:51:03.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.764+0000 7fc8837fe640 1 -- 172.21.15.39:0/700586609 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fc890054910 con 0x7fc89c0c47c0 2024-09-13T16:51:03.769 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:03.769 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:03.769 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-09-13T16:51:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc8880666c0 msgr2=0x7fc888068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc8880666c0 0x7fc888068b80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0ae000 tx=0x7fc88c0a7000 comp rx=0 tx=0).stop 2024-09-13T16:51:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c0c47c0 msgr2=0x7fc89c1eca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c0c47c0 0x7fc89c1eca20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc890001210 tx=0x7fc89000c8b0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 shutdown_connections 2024-09-13T16:51:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc8880666c0 0x7fc888068b80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c151500 0x7fc89c1ecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c0c47c0 0x7fc89c1eca20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 --2- 172.21.15.39:0/700586609 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c071dd0 0x7fc89c152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:03.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 >> 172.21.15.39:0/700586609 conn(0x7fc89c0ab530 msgr2=0x7fc89c15da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:03.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 shutdown_connections 2024-09-13T16:51:03.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:03.768+0000 7fc8a16e6640 1 -- 172.21.15.39:0/700586609 wait complete. 2024-09-13T16:51:03.793 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:51:05.794 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:05.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.952+0000 7f2960c81640 1 Processor -- start 2024-09-13T16:51:05.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.952+0000 7f2960c81640 1 -- start start 2024-09-13T16:51:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c1556d0 0x7f295c0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f295c0a72b0 0x7f295c071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295c06feb0 con 0x7f295c0c7610 2024-09-13T16:51:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295c0aae50 con 0x7f295c0a72b0 2024-09-13T16:51:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c155ab0 con 0x7f295c1556d0 2024-09-13T16:51:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2959d9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f295c0c7610 0x7f295c151b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:45540/0 (socket says 172.21.15.39:45540) 2024-09-13T16:51:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2959d9b640 1 -- 172.21.15.39:0/2387524238 learned_addr learned my addr 172.21.15.39:0/2387524238 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:51:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295ad9d640 1 --2- 172.21.15.39:0/2387524238 >> v2:172.21.15.39:3300/0 conn(0x7f295c0a72b0 0x7f295c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295a59c640 1 --2- 172.21.15.39:0/2387524238 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c1556d0 0x7f295c0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295959a640 1 -- 172.21.15.39:0/2387524238 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906194340 0 0) 0x7f295c06feb0 con 0x7f295c0c7610 2024-09-13T16:51:05.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295959a640 1 -- 172.21.15.39:0/2387524238 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2944003530 con 0x7f295c0c7610 2024-09-13T16:51:05.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295ad9d640 1 -- 172.21.15.39:0/2387524238 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c1556d0 msgr2=0x7f295c0c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:05.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295ad9d640 1 --2- 172.21.15.39:0/2387524238 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c1556d0 0x7f295c0c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295ad9d640 1 -- 172.21.15.39:0/2387524238 >> v1:172.21.15.39:6789/0 conn(0x7f295c0c7610 legacy=0x7f295c151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295ad9d640 1 -- 172.21.15.39:0/2387524238 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c0c3c20 con 0x7f295c0a72b0 2024-09-13T16:51:05.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295ad9d640 1 --2- 172.21.15.39:0/2387524238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c0a72b0 0x7f295c071530 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2954009d90 tx=0x7f295401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4681124a6dacb7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:05.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295959a640 1 -- 172.21.15.39:0/2387524238 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540a5800 con 0x7f295c0a72b0 2024-09-13T16:51:05.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295959a640 1 -- 172.21.15.39:0/2387524238 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540a5960 con 0x7f295c0a72b0 2024-09-13T16:51:05.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f295959a640 1 -- 172.21.15.39:0/2387524238 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540a5cf0 con 0x7f295c0a72b0 2024-09-13T16:51:05.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 -- 172.21.15.39:0/2387524238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c0a72b0 msgr2=0x7f295c071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 --2- 172.21.15.39:0/2387524238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c0a72b0 0x7f295c071530 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2954009d90 tx=0x7f295401c850 comp rx=0 tx=0).stop 2024-09-13T16:51:05.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 -- 172.21.15.39:0/2387524238 shutdown_connections 2024-09-13T16:51:05.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 --2- 172.21.15.39:0/2387524238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c0a72b0 0x7f295c071530 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 --2- 172.21.15.39:0/2387524238 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c1556d0 0x7f295c0c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 -- 172.21.15.39:0/2387524238 >> 172.21.15.39:0/2387524238 conn(0x7f295c0a97b0 msgr2=0x7f295c0a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:05.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 -- 172.21.15.39:0/2387524238 shutdown_connections 2024-09-13T16:51:05.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.956+0000 7f2960c81640 1 -- 172.21.15.39:0/2387524238 wait complete. 2024-09-13T16:51:05.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 Processor -- start 2024-09-13T16:51:05.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 -- start start 2024-09-13T16:51:05.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c0a72b0 0x7f295c15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c1556d0 0x7f295c14ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c1eccf0 0x7f295c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0bd340 con 0x7f295c1eccf0 2024-09-13T16:51:05.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c14f9f0 con 0x7f295c1556d0 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1ed1b0 con 0x7f295c0a72b0 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c1eccf0 0x7f295c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c1eccf0 0x7f295c14f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51752/0 (socket says 172.21.15.39:51752) 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295ad9d640 1 -- 172.21.15.39:0/3320842895 learned_addr learned my addr 172.21.15.39:0/3320842895 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2959d9b640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c1556d0 0x7f295c14ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295a59c640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c0a72b0 0x7f295c15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295ad9d640 1 -- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c0a72b0 msgr2=0x7f295c15a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295ad9d640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c0a72b0 0x7f295c15a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295ad9d640 1 -- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c1556d0 msgr2=0x7f295c14ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295ad9d640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c1556d0 0x7f295c14ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295ad9d640 1 -- 172.21.15.39:0/3320842895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954009a70 con 0x7f295c1eccf0 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2959d9b640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c1556d0 0x7f295c14ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:05.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295a59c640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c0a72b0 0x7f295c15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:05.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f295ad9d640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c1eccf0 0x7f295c14f150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f29540b0040 tx=0x7f29540195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:05.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f294affd640 1 -- 172.21.15.39:0/3320842895 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540b9860 con 0x7f295c1eccf0 2024-09-13T16:51:05.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f294affd640 1 -- 172.21.15.39:0/3320842895 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540b99c0 con 0x7f295c1eccf0 2024-09-13T16:51:05.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c151c10 con 0x7f295c1eccf0 2024-09-13T16:51:05.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f294affd640 1 -- 172.21.15.39:0/3320842895 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540b9d70 con 0x7f295c1eccf0 2024-09-13T16:51:05.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.960+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c14fc40 con 0x7f295c1eccf0 2024-09-13T16:51:05.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.964+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c14fe30 con 0x7f295c1eccf0 2024-09-13T16:51:05.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.964+0000 7f294affd640 1 -- 172.21.15.39:0/3320842895 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29540ad440 con 0x7f295c1eccf0 2024-09-13T16:51:05.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.964+0000 7f294affd640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f293c066690 0x7f293c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:05.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.964+0000 7f294affd640 1 -- 172.21.15.39:0/3320842895 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7f295411f870 con 0x7f295c1eccf0 2024-09-13T16:51:05.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.964+0000 7f295a59c640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f293c066690 0x7f293c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:05.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.964+0000 7f295a59c640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f293c066690 0x7f293c068b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f294c0097a0 tx=0x7f294c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:05.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:05.968+0000 7f294affd640 1 -- 172.21.15.39:0/3320842895 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f29540ec7d0 con 0x7f295c1eccf0 2024-09-13T16:51:06.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.180+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f295c071530 con 0x7f295c1eccf0 2024-09-13T16:51:06.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.180+0000 7f294affd640 1 -- 172.21.15.39:0/3320842895 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f29540efdf0 con 0x7f295c1eccf0 2024-09-13T16:51:06.186 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:06.186 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:06.186 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-09-13T16:51:06.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f293c066690 msgr2=0x7f293c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f293c066690 0x7f293c068b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f294c0097a0 tx=0x7f294c008040 comp rx=0 tx=0).stop 2024-09-13T16:51:06.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c1eccf0 msgr2=0x7f295c14f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:06.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c1eccf0 0x7f295c14f150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f29540b0040 tx=0x7f29540195f0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 shutdown_connections 2024-09-13T16:51:06.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f293c066690 0x7f293c068b50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f295c1556d0 0x7f295c14ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c1eccf0 0x7f295c14f150 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 --2- 172.21.15.39:0/3320842895 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f295c0a72b0 0x7f295c15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:06.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 >> 172.21.15.39:0/3320842895 conn(0x7f295c0a97b0 msgr2=0x7f295c0ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:06.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 shutdown_connections 2024-09-13T16:51:06.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:06.184+0000 7f2960c81640 1 -- 172.21.15.39:0/3320842895 wait complete. 2024-09-13T16:51:06.207 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:51:08.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:08.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c72de0640 1 Processor -- start 2024-09-13T16:51:08.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c72de0640 1 -- start start 2024-09-13T16:51:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c72de0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7c6c151500 0x7f7c6c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c72de0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c071dd0 0x7f7c6c154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c72de0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7c6c0c47c0 0x7f7c6c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c72de0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c0c5a30 con 0x7f7c6c151500 2024-09-13T16:51:08.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c72de0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c1576a0 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c72de0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c157980 con 0x7f7c6c071dd0 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c70b7c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7c6c151500 0x7f7c6c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c70b7c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7c6c151500 0x7f7c6c0c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51764/0 (socket says 172.21.15.39:51764) 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c70b7c640 1 -- 172.21.15.39:0/3380349044 learned_addr learned my addr 172.21.15.39:0/3380349044 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c6bfff640 1 --2- 172.21.15.39:0/3380349044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c071dd0 0x7f7c6c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c7137d640 1 --2- 172.21.15.39:0/3380349044 >> v2:172.21.15.84:3300/0 conn(0x7f7c6c0c47c0 0x7f7c6c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c70b7c640 1 -- 172.21.15.39:0/3380349044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c071dd0 msgr2=0x7f7c6c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c70b7c640 1 --2- 172.21.15.39:0/3380349044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c071dd0 0x7f7c6c154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c70b7c640 1 -- 172.21.15.39:0/3380349044 >> v2:172.21.15.84:3300/0 conn(0x7f7c6c0c47c0 msgr2=0x7f7c6c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c70b7c640 1 --2- 172.21.15.39:0/3380349044 >> v2:172.21.15.84:3300/0 conn(0x7f7c6c0c47c0 0x7f7c6c1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.336+0000 7f7c70b7c640 1 -- 172.21.15.39:0/3380349044 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c157c60 con 0x7f7c6c151500 2024-09-13T16:51:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c70b7c640 1 --2- 172.21.15.39:0/3380349044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c151500 0x7f7c6c0c4280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0c6260 tx=0x7f7c6001cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aee80fe551ba1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c6b7fe640 1 -- 172.21.15.39:0/3380349044 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600ab640 con 0x7f7c6c151500 2024-09-13T16:51:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c6b7fe640 1 -- 172.21.15.39:0/3380349044 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c600a7070 con 0x7f7c6c151500 2024-09-13T16:51:08.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c6b7fe640 1 -- 172.21.15.39:0/3380349044 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c600ab9c0 con 0x7f7c6c151500 2024-09-13T16:51:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 -- 172.21.15.39:0/3380349044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c151500 msgr2=0x7f7c6c0c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 --2- 172.21.15.39:0/3380349044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c151500 0x7f7c6c0c4280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0c6260 tx=0x7f7c6001cec0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 -- 172.21.15.39:0/3380349044 shutdown_connections 2024-09-13T16:51:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 --2- 172.21.15.39:0/3380349044 >> v2:172.21.15.84:3300/0 conn(0x7f7c6c0c47c0 0x7f7c6c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 --2- 172.21.15.39:0/3380349044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c071dd0 0x7f7c6c154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 --2- 172.21.15.39:0/3380349044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c151500 0x7f7c6c0c4280 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 -- 172.21.15.39:0/3380349044 >> 172.21.15.39:0/3380349044 conn(0x7f7c6c0ab530 msgr2=0x7f7c6c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:08.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 -- 172.21.15.39:0/3380349044 shutdown_connections 2024-09-13T16:51:08.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 -- 172.21.15.39:0/3380349044 wait complete. 2024-09-13T16:51:08.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 Processor -- start 2024-09-13T16:51:08.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.340+0000 7f7c72de0640 1 -- start start 2024-09-13T16:51:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c72de0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c6c071dd0 0x7f7c6c151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c72de0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c0c47c0 0x7f7c6c1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c72de0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c151500 0x7f7c6c0a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c72de0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1ec960 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c72de0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0ad080 con 0x7f7c6c071dd0 2024-09-13T16:51:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c72de0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0aaad0 con 0x7f7c6c151500 2024-09-13T16:51:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c6bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c0c47c0 0x7f7c6c1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c6bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c0c47c0 0x7f7c6c1501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51776/0 (socket says 172.21.15.39:51776) 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c6bfff640 1 -- 172.21.15.39:0/1075612855 learned_addr learned my addr 172.21.15.39:0/1075612855 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c70b7c640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c6c071dd0 0x7f7c6c151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c7137d640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c151500 0x7f7c6c0a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c6bfff640 1 -- 172.21.15.39:0/1075612855 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c151500 msgr2=0x7f7c6c0a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c6bfff640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c151500 0x7f7c6c0a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c6bfff640 1 -- 172.21.15.39:0/1075612855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c6c071dd0 msgr2=0x7f7c6c151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c6bfff640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c6c071dd0 0x7f7c6c151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c6bfff640 1 -- 172.21.15.39:0/1075612855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600094d0 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c6bfff640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c0c47c0 0x7f7c6c1501a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800d800 tx=0x7f7c5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c697fa640 1 -- 172.21.15.39:0/1075612855 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c580096a0 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c697fa640 1 -- 172.21.15.39:0/1075612855 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c58009800 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6c1ec680 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c697fa640 1 -- 172.21.15.39:0/1075612855 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c58009b70 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c0a9920 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6c0a94a0 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c697fa640 1 -- 172.21.15.39:0/1075612855 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7c58009cd0 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c697fa640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7c500666c0 0x7f7c50068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c697fa640 1 -- 172.21.15.39:0/1075612855 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7f7c58084c60 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.344+0000 7f7c70b7c640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7c500666c0 0x7f7c50068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.348+0000 7f7c70b7c640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7c500666c0 0x7f7c50068b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7c600af000 tx=0x7f7c600b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:08.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.352+0000 7f7c697fa640 1 -- 172.21.15.39:0/1075612855 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7c6c0a94a0 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.592+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c6c0a94a0 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.592+0000 7f7c697fa640 1 -- 172.21.15.39:0/1075612855 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f7c6c0a94a0 con 0x7f7c6c0c47c0 2024-09-13T16:51:08.598 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:08.598 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7c500666c0 msgr2=0x7f7c50068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7c500666c0 0x7f7c50068b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7c600af000 tx=0x7f7c600b6040 comp rx=0 tx=0).stop 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c0c47c0 msgr2=0x7f7c6c1501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c0c47c0 0x7f7c6c1501a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7c5800d800 tx=0x7f7c5800dcc0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 shutdown_connections 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7c500666c0 0x7f7c50068b80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7c6c151500 0x7f7c6c0a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c6c0c47c0 0x7f7c6c1501a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 --2- 172.21.15.39:0/1075612855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7c6c071dd0 0x7f7c6c151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 >> 172.21.15.39:0/1075612855 conn(0x7f7c6c0ab530 msgr2=0x7f7c6c0c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 shutdown_connections 2024-09-13T16:51:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:08.596+0000 7f7c72de0640 1 -- 172.21.15.39:0/1075612855 wait complete. 2024-09-13T16:51:08.618 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:51:10.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:10.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03ea668640 1 Processor -- start 2024-09-13T16:51:10.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03ea668640 1 -- start start 2024-09-13T16:51:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03ea668640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03e40c2db0 0x7f03e4061200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03ea668640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e40c2560 0x7f03e414e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03ea668640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e406fb40 con 0x7f03e414e630 2024-09-13T16:51:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03ea668640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e4071730 con 0x7f03e40c2db0 2024-09-13T16:51:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03ea668640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4150da0 con 0x7f03e40c2560 2024-09-13T16:51:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03e37fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e40c2560 0x7f03e414e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03e3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03e40c2db0 0x7f03e4061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03e37fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e40c2560 0x7f03e414e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:53728/0 (socket says 172.21.15.39:53728) 2024-09-13T16:51:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03e37fe640 1 -- 172.21.15.39:0/3147007289 learned_addr learned my addr 172.21.15.39:0/3147007289 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03e3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03e40c2db0 0x7f03e4061200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:57620/0 (socket says 172.21.15.39:57620) 2024-09-13T16:51:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03e3fff640 1 -- 172.21.15.39:0/3147007289 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e40c2560 msgr2=0x7f03e414e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03e3fff640 1 --2- 172.21.15.39:0/3147007289 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e40c2560 0x7f03e414e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03e3fff640 1 -- 172.21.15.39:0/3147007289 >> v1:172.21.15.84:6789/0 conn(0x7f03e414e630 legacy=0x7f03e41545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.764+0000 7f03e3fff640 1 -- 172.21.15.39:0/3147007289 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40706c0 con 0x7f03e40c2db0 2024-09-13T16:51:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03e3fff640 1 --2- 172.21.15.39:0/3147007289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e40c2db0 0x7f03e4061200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03d40097c0 tx=0x7f03d401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3e12b34186961 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03e2ffd640 1 -- 172.21.15.39:0/3147007289 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d40ab600 con 0x7f03e40c2db0 2024-09-13T16:51:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03e2ffd640 1 -- 172.21.15.39:0/3147007289 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d40a7050 con 0x7f03e40c2db0 2024-09-13T16:51:10.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03e2ffd640 1 -- 172.21.15.39:0/3147007289 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d40ab980 con 0x7f03e40c2db0 2024-09-13T16:51:10.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03ea668640 1 -- 172.21.15.39:0/3147007289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e40c2db0 msgr2=0x7f03e4061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03ea668640 1 --2- 172.21.15.39:0/3147007289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e40c2db0 0x7f03e4061200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03d40097c0 tx=0x7f03d401c4a0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03ea668640 1 -- 172.21.15.39:0/3147007289 shutdown_connections 2024-09-13T16:51:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03ea668640 1 --2- 172.21.15.39:0/3147007289 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e40c2560 0x7f03e414e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03ea668640 1 --2- 172.21.15.39:0/3147007289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e40c2db0 0x7f03e4061200 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03ea668640 1 -- 172.21.15.39:0/3147007289 >> 172.21.15.39:0/3147007289 conn(0x7f03e4069e10 msgr2=0x7f03e406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03ea668640 1 -- 172.21.15.39:0/3147007289 shutdown_connections 2024-09-13T16:51:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.768+0000 7f03ea668640 1 -- 172.21.15.39:0/3147007289 wait complete. 2024-09-13T16:51:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 Processor -- start 2024-09-13T16:51:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 -- start start 2024-09-13T16:51:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e40c2560 0x7f03e41ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e40c2db0 0x7f03e41493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e414e630 0x7f03e41466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40c3ea0 con 0x7f03e40c2db0 2024-09-13T16:51:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e414b690 con 0x7f03e40c2560 2024-09-13T16:51:10.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e41ed090 con 0x7f03e414e630 2024-09-13T16:51:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e40c2db0 0x7f03e41493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e40c2560 0x7f03e41ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e40c2db0 0x7f03e41493e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54678/0 (socket says 172.21.15.39:54678) 2024-09-13T16:51:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e37fe640 1 -- 172.21.15.39:0/1143624716 learned_addr learned my addr 172.21.15.39:0/1143624716 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e8c05640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e414e630 0x7f03e41466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e37fe640 1 -- 172.21.15.39:0/1143624716 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e414e630 msgr2=0x7f03e41466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e37fe640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e414e630 0x7f03e41466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e37fe640 1 -- 172.21.15.39:0/1143624716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e40c2560 msgr2=0x7f03e41ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e37fe640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e40c2560 0x7f03e41ecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e37fe640 1 -- 172.21.15.39:0/1143624716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4009420 con 0x7f03e40c2db0 2024-09-13T16:51:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e37fe640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e40c2db0 0x7f03e41493e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f03d800e8c0 tx=0x7f03d800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e0ff9640 1 -- 172.21.15.39:0/1143624716 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8009620 con 0x7f03e40c2db0 2024-09-13T16:51:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e0ff9640 1 -- 172.21.15.39:0/1143624716 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d800c8e0 con 0x7f03e40c2db0 2024-09-13T16:51:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 -- 172.21.15.39:0/1143624716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e4152e80 con 0x7f03e40c2db0 2024-09-13T16:51:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03e0ff9640 1 -- 172.21.15.39:0/1143624716 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800cc90 con 0x7f03e40c2db0 2024-09-13T16:51:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.772+0000 7f03ea668640 1 -- 172.21.15.39:0/1143624716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e4153440 con 0x7f03e40c2db0 2024-09-13T16:51:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.776+0000 7f03ca7fc640 1 -- 172.21.15.39:0/1143624716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e414e0f0 con 0x7f03e40c2db0 2024-09-13T16:51:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.776+0000 7f03e0ff9640 1 -- 172.21.15.39:0/1143624716 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03d8013470 con 0x7f03e40c2db0 2024-09-13T16:51:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.776+0000 7f03e0ff9640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f03d0066690 0x7f03d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.776+0000 7f03e0ff9640 1 -- 172.21.15.39:0/1143624716 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7f03d80845f0 con 0x7f03e40c2db0 2024-09-13T16:51:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.776+0000 7f03e3fff640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f03d0066690 0x7f03d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.776+0000 7f03e3fff640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f03d0066690 0x7f03d0068b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f03d40ad000 tx=0x7f03d40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:10.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.780+0000 7f03e0ff9640 1 -- 172.21.15.39:0/1143624716 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f03d8051600 con 0x7f03e40c2db0 2024-09-13T16:51:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.988+0000 7f03ca7fc640 1 -- 172.21.15.39:0/1143624716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03e40c6f90 con 0x7f03e40c2db0 2024-09-13T16:51:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03e0ff9640 1 -- 172.21.15.39:0/1143624716 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f03d8054c20 con 0x7f03e40c2db0 2024-09-13T16:51:10.995 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:10.995 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:10.996 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-09-13T16:51:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 -- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f03d0066690 msgr2=0x7f03d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f03d0066690 0x7f03d0068b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f03d40ad000 tx=0x7f03d40a6000 comp rx=0 tx=0).stop 2024-09-13T16:51:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 -- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e40c2db0 msgr2=0x7f03e41493e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e40c2db0 0x7f03e41493e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f03d800e8c0 tx=0x7f03d800ed80 comp rx=0 tx=0).stop 2024-09-13T16:51:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 -- 172.21.15.39:0/1143624716 shutdown_connections 2024-09-13T16:51:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f03d0066690 0x7f03d0068b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f03e414e630 0x7f03e41466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e40c2db0 0x7f03e41493e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 --2- 172.21.15.39:0/1143624716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03e40c2560 0x7f03e41ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 -- 172.21.15.39:0/1143624716 >> 172.21.15.39:0/1143624716 conn(0x7f03e4069e10 msgr2=0x7f03e414aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 -- 172.21.15.39:0/1143624716 shutdown_connections 2024-09-13T16:51:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:10.992+0000 7f03ca7fc640 1 -- 172.21.15.39:0/1143624716 wait complete. 2024-09-13T16:51:11.016 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:51:13.017 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:13.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.152+0000 7fc40395f640 1 Processor -- start 2024-09-13T16:51:13.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc40395f640 1 -- start start 2024-09-13T16:51:13.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc40395f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3fc0c1d60 0x7fc3fc155870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:13.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc40395f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc3fc0c5460 0x7fc3fc155db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:13.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc40395f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3fc0c4630 con 0x7fc3fc065ee0 2024-09-13T16:51:13.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc40395f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc0c0a90 con 0x7fc3fc0c5460 2024-09-13T16:51:13.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc40395f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc0c11c0 con 0x7fc3fc0c1d60 2024-09-13T16:51:13.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc4016fb640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc3fc065ee0 0x7fc3fc0c1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40530/0 (socket says 172.21.15.39:40530) 2024-09-13T16:51:13.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc4016fb640 1 -- 172.21.15.39:0/3776883508 learned_addr learned my addr 172.21.15.39:0/3776883508 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:51:13.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc401efc640 1 --2- 172.21.15.39:0/3776883508 >> v2:172.21.15.39:3300/0 conn(0x7fc3fc0c5460 0x7fc3fc155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:13.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc400efa640 1 --2- 172.21.15.39:0/3776883508 >> v2:172.21.15.84:3300/0 conn(0x7fc3fc0c1d60 0x7fc3fc155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:13.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc3f3fff640 1 -- 172.21.15.39:0/3776883508 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641277833 0 0) 0x7fc3fc0c4630 con 0x7fc3fc065ee0 2024-09-13T16:51:13.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc3f3fff640 1 -- 172.21.15.39:0/3776883508 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3e8003530 con 0x7fc3fc065ee0 2024-09-13T16:51:13.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc401efc640 1 -- 172.21.15.39:0/3776883508 >> v2:172.21.15.84:3300/0 conn(0x7fc3fc0c1d60 msgr2=0x7fc3fc155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:13.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc401efc640 1 --2- 172.21.15.39:0/3776883508 >> v2:172.21.15.84:3300/0 conn(0x7fc3fc0c1d60 0x7fc3fc155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc401efc640 1 -- 172.21.15.39:0/3776883508 >> v1:172.21.15.39:6789/0 conn(0x7fc3fc065ee0 legacy=0x7fc3fc0c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:13.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc401efc640 1 -- 172.21.15.39:0/3776883508 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc0c4880 con 0x7fc3fc0c5460 2024-09-13T16:51:13.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.156+0000 7fc400efa640 1 --2- 172.21.15.39:0/3776883508 >> v2:172.21.15.84:3300/0 conn(0x7fc3fc0c1d60 0x7fc3fc155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:13.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc401efc640 1 --2- 172.21.15.39:0/3776883508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 0x7fc3fc155db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009d90 tx=0x7fc3f401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3208b392ea9d287 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:13.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc3f3fff640 1 -- 172.21.15.39:0/3776883508 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40a5930 con 0x7fc3fc0c5460 2024-09-13T16:51:13.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc3f3fff640 1 -- 172.21.15.39:0/3776883508 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f40a5a90 con 0x7fc3fc0c5460 2024-09-13T16:51:13.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc3f3fff640 1 -- 172.21.15.39:0/3776883508 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40a5e20 con 0x7fc3fc0c5460 2024-09-13T16:51:13.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 -- 172.21.15.39:0/3776883508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 msgr2=0x7fc3fc155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:13.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 --2- 172.21.15.39:0/3776883508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 0x7fc3fc155db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009d90 tx=0x7fc3f401c870 comp rx=0 tx=0).stop 2024-09-13T16:51:13.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 -- 172.21.15.39:0/3776883508 shutdown_connections 2024-09-13T16:51:13.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 --2- 172.21.15.39:0/3776883508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 0x7fc3fc155db0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 --2- 172.21.15.39:0/3776883508 >> v2:172.21.15.84:3300/0 conn(0x7fc3fc0c1d60 0x7fc3fc155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 -- 172.21.15.39:0/3776883508 >> 172.21.15.39:0/3776883508 conn(0x7fc3fc0a86b0 msgr2=0x7fc3fc0a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:13.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 -- 172.21.15.39:0/3776883508 shutdown_connections 2024-09-13T16:51:13.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 -- 172.21.15.39:0/3776883508 wait complete. 2024-09-13T16:51:13.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 Processor -- start 2024-09-13T16:51:13.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 -- start start 2024-09-13T16:51:13.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3fc065ee0 0x7fc3fc1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:13.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.160+0000 7fc40395f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3fc0c1d60 0x7fc3fc14ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:13.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc40395f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 0x7fc3fc1f0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:13.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc40395f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc156140 con 0x7fc3fc0c5460 2024-09-13T16:51:13.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc40395f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc155cf0 con 0x7fc3fc065ee0 2024-09-13T16:51:13.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc40395f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc1ecb90 con 0x7fc3fc0c1d60 2024-09-13T16:51:13.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc401efc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 0x7fc3fc1f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:13.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc401efc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 0x7fc3fc1f0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54706/0 (socket says 172.21.15.39:54706) 2024-09-13T16:51:13.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc401efc640 1 -- 172.21.15.39:0/1907359532 learned_addr learned my addr 172.21.15.39:0/1907359532 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:13.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc4016fb640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3fc065ee0 0x7fc3fc1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:13.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc400efa640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3fc0c1d60 0x7fc3fc14ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:13.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc401efc640 1 -- 172.21.15.39:0/1907359532 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3fc0c1d60 msgr2=0x7fc3fc14ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:13.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc401efc640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3fc0c1d60 0x7fc3fc14ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc401efc640 1 -- 172.21.15.39:0/1907359532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3fc065ee0 msgr2=0x7fc3fc1557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:13.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc401efc640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3fc065ee0 0x7fc3fc1557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc401efc640 1 -- 172.21.15.39:0/1907359532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4009a70 con 0x7fc3fc0c5460 2024-09-13T16:51:13.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc401efc640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 0x7fc3fc1f0070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40b0040 tx=0x7fc3f4019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:13.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc3f1ffb640 1 -- 172.21.15.39:0/1907359532 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40add90 con 0x7fc3fc0c5460 2024-09-13T16:51:13.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc3f1ffb640 1 -- 172.21.15.39:0/1907359532 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f4009450 con 0x7fc3fc0c5460 2024-09-13T16:51:13.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc1ece70 con 0x7fc3fc0c5460 2024-09-13T16:51:13.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc3f1ffb640 1 -- 172.21.15.39:0/1907359532 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40b9690 con 0x7fc3fc0c5460 2024-09-13T16:51:13.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc1ed430 con 0x7fc3fc0c5460 2024-09-13T16:51:13.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3fc1ed950 con 0x7fc3fc0c5460 2024-09-13T16:51:13.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc3f1ffb640 1 -- 172.21.15.39:0/1907359532 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc3f40b97f0 con 0x7fc3fc0c5460 2024-09-13T16:51:13.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc3f1ffb640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3e0066690 0x7fc3e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:13.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc3f1ffb640 1 -- 172.21.15.39:0/1907359532 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6895+0+0 (secure 0 0 0) 0x7fc3f411fd00 con 0x7fc3fc0c5460 2024-09-13T16:51:13.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc4016fb640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3e0066690 0x7fc3e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:13.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.164+0000 7fc4016fb640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3e0066690 0x7fc3e0068b50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80011f0 tx=0x7fc3f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:13.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.168+0000 7fc3f1ffb640 1 -- 172.21.15.39:0/1907359532 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc3fc1ed950 con 0x7fc3fc0c5460 2024-09-13T16:51:13.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3fc1ed950 con 0x7fc3fc0c5460 2024-09-13T16:51:13.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc3f1ffb640 1 -- 172.21.15.39:0/1907359532 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fc3fc1ed950 con 0x7fc3fc0c5460 2024-09-13T16:51:13.452 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:13.452 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:50:18.849824+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":28,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5142},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5142":{"gid":5142,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6835/1572373831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1572373831},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572373831}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:13.453 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3e0066690 msgr2=0x7fc3e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3e0066690 0x7fc3e0068b50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80011f0 tx=0x7fc3f8009040 comp rx=0 tx=0).stop 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 msgr2=0x7fc3fc1f0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 0x7fc3fc1f0070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40b0040 tx=0x7fc3f4019610 comp rx=0 tx=0).stop 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 shutdown_connections 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc3e0066690 0x7fc3e0068b50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3fc0c5460 0x7fc3fc1f0070 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3fc0c1d60 0x7fc3fc14ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 --2- 172.21.15.39:0/1907359532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3fc065ee0 0x7fc3fc1557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 >> 172.21.15.39:0/1907359532 conn(0x7fc3fc0a86b0 msgr2=0x7fc3fc0a6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 shutdown_connections 2024-09-13T16:51:13.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:13.448+0000 7fc40395f640 1 -- 172.21.15.39:0/1907359532 wait complete. 2024-09-13T16:51:13.474 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:51:15.475 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b90fd8640 1 Processor -- start 2024-09-13T16:51:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b90fd8640 1 -- start start 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b90fd8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8b8c0c5f40 0x7f8b8c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b90fd8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b8c071ed0 0x7f8b8c1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b90fd8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c0c2e60 con 0x7f8b8c0c5790 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b90fd8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8c0c7470 con 0x7f8b8c0c5f40 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b90fd8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8c0c2570 con 0x7f8b8c071ed0 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b8a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8b8c0c5f40 0x7f8b8c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b8a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8b8c0c5f40 0x7f8b8c156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54716/0 (socket says 172.21.15.39:54716) 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b8a59c640 1 -- 172.21.15.39:0/2300684195 learned_addr learned my addr 172.21.15.39:0/2300684195 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b89d9b640 1 --2- 172.21.15.39:0/2300684195 >> v2:172.21.15.84:3300/0 conn(0x7f8b8c071ed0 0x7f8b8c1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b8a59c640 1 -- 172.21.15.39:0/2300684195 >> v2:172.21.15.84:3300/0 conn(0x7f8b8c071ed0 msgr2=0x7f8b8c1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b8a59c640 1 --2- 172.21.15.39:0/2300684195 >> v2:172.21.15.84:3300/0 conn(0x7f8b8c071ed0 0x7f8b8c1577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b8a59c640 1 -- 172.21.15.39:0/2300684195 >> v1:172.21.15.84:6789/0 conn(0x7f8b8c0c5790 legacy=0x7f8b8c157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.628+0000 7f8b8a59c640 1 -- 172.21.15.39:0/2300684195 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c072360 con 0x7f8b8c0c5f40 2024-09-13T16:51:15.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b8a59c640 1 --2- 172.21.15.39:0/2300684195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c0c5f40 0x7f8b8c156cf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0bec60 tx=0x7f8b7c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c563aa0f0ef0ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:15.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b8959a640 1 -- 172.21.15.39:0/2300684195 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0ab5f0 con 0x7f8b8c0c5f40 2024-09-13T16:51:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b8959a640 1 -- 172.21.15.39:0/2300684195 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c0ab750 con 0x7f8b8c0c5f40 2024-09-13T16:51:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b8959a640 1 -- 172.21.15.39:0/2300684195 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0abae0 con 0x7f8b8c0c5f40 2024-09-13T16:51:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b90fd8640 1 -- 172.21.15.39:0/2300684195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c0c5f40 msgr2=0x7f8b8c156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b90fd8640 1 --2- 172.21.15.39:0/2300684195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c0c5f40 0x7f8b8c156cf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0bec60 tx=0x7f8b7c01c4e0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b90fd8640 1 -- 172.21.15.39:0/2300684195 shutdown_connections 2024-09-13T16:51:15.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b90fd8640 1 --2- 172.21.15.39:0/2300684195 >> v2:172.21.15.84:3300/0 conn(0x7f8b8c071ed0 0x7f8b8c1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b90fd8640 1 --2- 172.21.15.39:0/2300684195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c0c5f40 0x7f8b8c156cf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b90fd8640 1 -- 172.21.15.39:0/2300684195 >> 172.21.15.39:0/2300684195 conn(0x7f8b8c06c020 msgr2=0x7f8b8c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:15.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b90fd8640 1 -- 172.21.15.39:0/2300684195 shutdown_connections 2024-09-13T16:51:15.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.632+0000 7f8b90fd8640 1 -- 172.21.15.39:0/2300684195 wait complete. 2024-09-13T16:51:15.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 Processor -- start 2024-09-13T16:51:15.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 -- start start 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b8c0c5790 0x7f8b8c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b8c0c5f40 0x7f8b8c0bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c1534d0 0x7f8b8c1ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c1eca30 con 0x7f8b8c1534d0 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0ac360 con 0x7f8b8c0c5f40 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0ac080 con 0x7f8b8c0c5790 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c1534d0 0x7f8b8c1ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c1534d0 0x7f8b8c1ed1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54732/0 (socket says 172.21.15.39:54732) 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8ad9d640 1 -- 172.21.15.39:0/210272632 learned_addr learned my addr 172.21.15.39:0/210272632 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8a59c640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b8c0c5790 0x7f8b8c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b89d9b640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b8c0c5f40 0x7f8b8c0bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8ad9d640 1 -- 172.21.15.39:0/210272632 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b8c0c5790 msgr2=0x7f8b8c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8ad9d640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b8c0c5790 0x7f8b8c155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8ad9d640 1 -- 172.21.15.39:0/210272632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b8c0c5f40 msgr2=0x7f8b8c0bdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8ad9d640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b8c0c5f40 0x7f8b8c0bdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8ad9d640 1 -- 172.21.15.39:0/210272632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0094d0 con 0x7f8b8c1534d0 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b8ad9d640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c1534d0 0x7f8b8c1ed1f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400c780 tx=0x7f8b8400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b7affd640 1 -- 172.21.15.39:0/210272632 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84003d90 con 0x7f8b8c1534d0 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b7affd640 1 -- 172.21.15.39:0/210272632 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8400ce30 con 0x7f8b8c1534d0 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c1f0a50 con 0x7f8b8c1534d0 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b7affd640 1 -- 172.21.15.39:0/210272632 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84005290 con 0x7f8b8c1534d0 2024-09-13T16:51:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c1eccf0 con 0x7f8b8c1534d0 2024-09-13T16:51:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b7affd640 1 -- 172.21.15.39:0/210272632 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b840053f0 con 0x7f8b8c1534d0 2024-09-13T16:51:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c156cf0 con 0x7f8b8c1534d0 2024-09-13T16:51:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.636+0000 7f8b7affd640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.640+0000 7f8b7affd640 1 -- 172.21.15.39:0/210272632 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f8b84083d90 con 0x7f8b8c1534d0 2024-09-13T16:51:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.640+0000 7f8b8a59c640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.640+0000 7f8b8a59c640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0af000 tx=0x7f8b7c0b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.644+0000 7f8b7affd640 1 -- 172.21.15.39:0/210272632 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8b84050d40 con 0x7f8b8c1534d0 2024-09-13T16:51:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.896+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b8c156740 con 0x7f8b8c1534d0 2024-09-13T16:51:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b7affd640 1 -- 172.21.15.39:0/210272632 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5178 (secure 0 0 0) 0x7f8b84017070 con 0x7f8b8c1534d0 2024-09-13T16:51:15.903 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:15.903 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:15.582036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-09-13T16:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 msgr2=0x7f8b6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0af000 tx=0x7f8b7c0b6040 comp rx=0 tx=0).stop 2024-09-13T16:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c1534d0 msgr2=0x7f8b8c1ed1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c1534d0 0x7f8b8c1ed1f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400c780 tx=0x7f8b8400cc40 comp rx=0 tx=0).stop 2024-09-13T16:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 shutdown_connections 2024-09-13T16:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b8c0c5f40 0x7f8b8c0bdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c1534d0 0x7f8b8c1ed1f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 --2- 172.21.15.39:0/210272632 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b8c0c5790 0x7f8b8c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 >> 172.21.15.39:0/210272632 conn(0x7f8b8c06c020 msgr2=0x7f8b8c06aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 shutdown_connections 2024-09-13T16:51:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:15.900+0000 7f8b90fd8640 1 -- 172.21.15.39:0/210272632 wait complete. 2024-09-13T16:51:15.923 INFO:tasks.mds_thrash.fs.[b]:mds.a down, removed from mdsmap 2024-09-13T16:51:15.923 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T16:51:15.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:16.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc3be1640 1 Processor -- start 2024-09-13T16:51:16.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc3be1640 1 -- start start 2024-09-13T16:51:16.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc3be1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9fbc14f1b0 0x7f9fbc151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:16.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc3be1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9fbc06e5e0 0x7f9fbc0714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:16.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc3be1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc14f590 con 0x7f9fbc0c7470 2024-09-13T16:51:16.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc3be1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc06f660 con 0x7f9fbc06e5e0 2024-09-13T16:51:16.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc3be1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc06eae0 con 0x7f9fbc14f1b0 2024-09-13T16:51:16.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc117c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9fbc06e5e0 0x7f9fbc0714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:16.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc217e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9fbc0c7470 0x7f9fbc0c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40534/0 (socket says 172.21.15.39:40534) 2024-09-13T16:51:16.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.048+0000 7f9fc217e640 1 -- 172.21.15.39:0/2310714671 learned_addr learned my addr 172.21.15.39:0/2310714671 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:51:16.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc197d640 1 --2- 172.21.15.39:0/2310714671 >> v2:172.21.15.84:3300/0 conn(0x7f9fbc14f1b0 0x7f9fbc151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:16.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc117c640 1 -- 172.21.15.39:0/2310714671 >> v2:172.21.15.84:3300/0 conn(0x7f9fbc14f1b0 msgr2=0x7f9fbc151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:16.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc117c640 1 --2- 172.21.15.39:0/2310714671 >> v2:172.21.15.84:3300/0 conn(0x7f9fbc14f1b0 0x7f9fbc151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc117c640 1 -- 172.21.15.39:0/2310714671 >> v1:172.21.15.39:6789/0 conn(0x7f9fbc0c7470 legacy=0x7f9fbc0c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:16.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc117c640 1 -- 172.21.15.39:0/2310714671 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc06edc0 con 0x7f9fbc06e5e0 2024-09-13T16:51:16.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc197d640 1 --2- 172.21.15.39:0/2310714671 >> v2:172.21.15.84:3300/0 conn(0x7f9fbc14f1b0 0x7f9fbc151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:16.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc117c640 1 --2- 172.21.15.39:0/2310714671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc06e5e0 0x7f9fbc0714e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc071140 tx=0x7f9fac01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6889ce80bfe14d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:16.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc097b640 1 -- 172.21.15.39:0/2310714671 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0ab600 con 0x7f9fbc06e5e0 2024-09-13T16:51:16.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc097b640 1 -- 172.21.15.39:0/2310714671 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac0a7050 con 0x7f9fbc06e5e0 2024-09-13T16:51:16.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.052+0000 7f9fc097b640 1 -- 172.21.15.39:0/2310714671 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0ab980 con 0x7f9fbc06e5e0 2024-09-13T16:51:16.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 -- 172.21.15.39:0/2310714671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc06e5e0 msgr2=0x7f9fbc0714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:16.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 --2- 172.21.15.39:0/2310714671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc06e5e0 0x7f9fbc0714e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc071140 tx=0x7f9fac01c310 comp rx=0 tx=0).stop 2024-09-13T16:51:16.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 -- 172.21.15.39:0/2310714671 shutdown_connections 2024-09-13T16:51:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 --2- 172.21.15.39:0/2310714671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc06e5e0 0x7f9fbc0714e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 --2- 172.21.15.39:0/2310714671 >> v2:172.21.15.84:3300/0 conn(0x7f9fbc14f1b0 0x7f9fbc151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 -- 172.21.15.39:0/2310714671 >> 172.21.15.39:0/2310714671 conn(0x7f9fbc0a9720 msgr2=0x7f9fbc0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 -- 172.21.15.39:0/2310714671 shutdown_connections 2024-09-13T16:51:16.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 -- 172.21.15.39:0/2310714671 wait complete. 2024-09-13T16:51:16.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 Processor -- start 2024-09-13T16:51:16.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 -- start start 2024-09-13T16:51:16.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fbc06e5e0 0x7f9fbc153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:16.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc0c7470 0x7f9fbc0c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:16.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc14f1b0 0x7f9fbc0c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:16.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc154d50 con 0x7f9fbc0c7470 2024-09-13T16:51:16.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc1558c0 con 0x7f9fbc14f1b0 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc3be1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc154f40 con 0x7f9fbc06e5e0 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc117c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc0c7470 0x7f9fbc0c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc117c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc0c7470 0x7f9fbc0c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54742/0 (socket says 172.21.15.39:54742) 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.056+0000 7f9fc117c640 1 -- 172.21.15.39:0/1939759039 learned_addr learned my addr 172.21.15.39:0/1939759039 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc117c640 1 -- 172.21.15.39:0/1939759039 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fbc06e5e0 msgr2=0x7f9fbc153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc117c640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fbc06e5e0 0x7f9fbc153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc117c640 1 -- 172.21.15.39:0/1939759039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc14f1b0 msgr2=0x7f9fbc0c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc117c640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc14f1b0 0x7f9fbc0c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc117c640 1 -- 172.21.15.39:0/1939759039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac009420 con 0x7f9fbc0c7470 2024-09-13T16:51:16.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc117c640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc0c7470 0x7f9fbc0c41c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9fac000c10 tx=0x7f9fac0190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fb27fc640 1 -- 172.21.15.39:0/1939759039 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0aa830 con 0x7f9fbc0c7470 2024-09-13T16:51:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fb27fc640 1 -- 172.21.15.39:0/1939759039 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac0aa990 con 0x7f9fbc0c7470 2024-09-13T16:51:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc155220 con 0x7f9fbc0c7470 2024-09-13T16:51:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fb27fc640 1 -- 172.21.15.39:0/1939759039 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0aad40 con 0x7f9fbc0c7470 2024-09-13T16:51:16.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc154180 con 0x7f9fbc0c7470 2024-09-13T16:51:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc0714e0 con 0x7f9fbc0c7470 2024-09-13T16:51:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fb27fc640 1 -- 172.21.15.39:0/1939759039 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9fac0b2040 con 0x7f9fbc0c7470 2024-09-13T16:51:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fb27fc640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa00666c0 0x7f9fa0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fb27fc640 1 -- 172.21.15.39:0/1939759039 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f9fac11ea70 con 0x7f9fbc0c7470 2024-09-13T16:51:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.060+0000 7f9fc197d640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa00666c0 0x7f9fa0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:16.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.064+0000 7f9fc197d640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa00666c0 0x7f9fa0068b80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc150150 tx=0x7f9fb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:16.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.064+0000 7f9fb27fc640 1 -- 172.21.15.39:0/1939759039 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9fac0eb970 con 0x7f9fbc0c7470 2024-09-13T16:51:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fbc155a50 con 0x7f9fbc0c7470 2024-09-13T16:51:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fb27fc640 1 -- 172.21.15.39:0/1939759039 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5178 (secure 0 0 0) 0x7f9fac0eef90 con 0x7f9fbc0c7470 2024-09-13T16:51:16.356 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:16.356 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:15.582036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:16.356 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 32 2024-09-13T16:51:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa00666c0 msgr2=0x7f9fa0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa00666c0 0x7f9fa0068b80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc150150 tx=0x7f9fb8008040 comp rx=0 tx=0).stop 2024-09-13T16:51:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc0c7470 msgr2=0x7f9fbc0c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc0c7470 0x7f9fbc0c41c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9fac000c10 tx=0x7f9fac0190b0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 shutdown_connections 2024-09-13T16:51:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa00666c0 0x7f9fa0068b80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc14f1b0 0x7f9fbc0c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fbc0c7470 0x7f9fbc0c41c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 --2- 172.21.15.39:0/1939759039 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fbc06e5e0 0x7f9fbc153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:16.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 >> 172.21.15.39:0/1939759039 conn(0x7f9fbc0a9720 msgr2=0x7f9fbc14fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:16.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 shutdown_connections 2024-09-13T16:51:16.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:16.352+0000 7f9fc3be1640 1 -- 172.21.15.39:0/1939759039 wait complete. 2024-09-13T16:51:16.375 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 32, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 29, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5053": { "addr": "172.21.15.84:6835/1969975539", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 1969975539, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 1969975539, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5053, "incarnation": 26, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:50:52.769334+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5053 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 32, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_4814": { "addr": "172.21.15.84:6833/2144264453", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 2144264453, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 2144264453, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4814, "incarnation": 31, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 30, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:51:15.582036+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4814 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:51:17.812 INFO:tasks.ceph.mds.b.smithi084.stderr:2024-09-13T16:51:17.810+0000 7fd621417640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:51:18.376 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:18.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6aacbe640 1 Processor -- start 2024-09-13T16:51:18.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6aacbe640 1 -- start start 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6aacbe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe6a40ab9e0 0x7fe6a40c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6aacbe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe6a4071bb0 0x7fe6a406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6aacbe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a40c0ac0 0x7fe6a406d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6aacbe640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a40abdc0 con 0x7fe6a4071bb0 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6aacbe640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a4150e00 con 0x7fe6a40ab9e0 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6aacbe640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4150250 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a3fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe6a4071bb0 0x7fe6a406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a3fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe6a4071bb0 0x7fe6a406cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54758/0 (socket says 172.21.15.39:54758) 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a3fff640 1 -- 172.21.15.39:0/3441908022 learned_addr learned my addr 172.21.15.39:0/3441908022 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a8a5a640 1 --2- 172.21.15.39:0/3441908022 >> v2:172.21.15.84:3300/0 conn(0x7fe6a40ab9e0 0x7fe6a40c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a925b640 1 --2- 172.21.15.39:0/3441908022 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a40c0ac0 0x7fe6a406d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a3fff640 1 -- 172.21.15.39:0/3441908022 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a40c0ac0 msgr2=0x7fe6a406d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a3fff640 1 --2- 172.21.15.39:0/3441908022 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a40c0ac0 0x7fe6a406d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a3fff640 1 -- 172.21.15.39:0/3441908022 >> v2:172.21.15.84:3300/0 conn(0x7fe6a40ab9e0 msgr2=0x7fe6a40c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a3fff640 1 --2- 172.21.15.39:0/3441908022 >> v2:172.21.15.84:3300/0 conn(0x7fe6a40ab9e0 0x7fe6a40c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.516+0000 7fe6a3fff640 1 -- 172.21.15.39:0/3441908022 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a414a8f0 con 0x7fe6a4071bb0 2024-09-13T16:51:18.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6a3fff640 1 --2- 172.21.15.39:0/3441908022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a4071bb0 0x7fe6a406cbe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6a406b7b0 tx=0x7fe69801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbf465fba4f40fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6a37fe640 1 -- 172.21.15.39:0/3441908022 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698004230 con 0x7fe6a4071bb0 2024-09-13T16:51:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6a37fe640 1 -- 172.21.15.39:0/3441908022 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6980a7050 con 0x7fe6a4071bb0 2024-09-13T16:51:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6a37fe640 1 -- 172.21.15.39:0/3441908022 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698003620 con 0x7fe6a4071bb0 2024-09-13T16:51:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6aacbe640 1 -- 172.21.15.39:0/3441908022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a4071bb0 msgr2=0x7fe6a406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/3441908022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a4071bb0 0x7fe6a406cbe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6a406b7b0 tx=0x7fe69801c520 comp rx=0 tx=0).stop 2024-09-13T16:51:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6aacbe640 1 -- 172.21.15.39:0/3441908022 shutdown_connections 2024-09-13T16:51:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/3441908022 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a40c0ac0 0x7fe6a406d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/3441908022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a4071bb0 0x7fe6a406cbe0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/3441908022 >> v2:172.21.15.84:3300/0 conn(0x7fe6a40ab9e0 0x7fe6a40c2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.520+0000 7fe6aacbe640 1 -- 172.21.15.39:0/3441908022 >> 172.21.15.39:0/3441908022 conn(0x7fe6a4069e10 msgr2=0x7fe6a406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 -- 172.21.15.39:0/3441908022 shutdown_connections 2024-09-13T16:51:18.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 -- 172.21.15.39:0/3441908022 wait complete. 2024-09-13T16:51:18.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 Processor -- start 2024-09-13T16:51:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 -- start start 2024-09-13T16:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a4071bb0 0x7fe6a413eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a40ab9e0 0x7fe6a40bf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a40c0ac0 0x7fe6a413cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a413f410 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a413dce0 con 0x7fe6a40ab9e0 2024-09-13T16:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a41f1080 con 0x7fe6a4071bb0 2024-09-13T16:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a925b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a40c0ac0 0x7fe6a413cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a925b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a40c0ac0 0x7fe6a413cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54770/0 (socket says 172.21.15.39:54770) 2024-09-13T16:51:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a925b640 1 -- 172.21.15.39:0/849002477 learned_addr learned my addr 172.21.15.39:0/849002477 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a8a5a640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a4071bb0 0x7fe6a413eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a3fff640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a40ab9e0 0x7fe6a40bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a925b640 1 -- 172.21.15.39:0/849002477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a4071bb0 msgr2=0x7fe6a413eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a925b640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a4071bb0 0x7fe6a413eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a925b640 1 -- 172.21.15.39:0/849002477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a40ab9e0 msgr2=0x7fe6a40bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a925b640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a40ab9e0 0x7fe6a40bf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a925b640 1 -- 172.21.15.39:0/849002477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6980094d0 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a8a5a640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a4071bb0 0x7fe6a413eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a925b640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a40c0ac0 0x7fe6a413cb20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009fd0 tx=0x7fe69c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a17fa640 1 -- 172.21.15.39:0/849002477 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c014420 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a17fa640 1 -- 172.21.15.39:0/849002477 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c014580 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a41f02c0 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.524+0000 7fe6a17fa640 1 -- 172.21.15.39:0/849002477 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c014930 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.528+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a41f0880 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.528+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a414dd90 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.528+0000 7fe6a17fa640 1 -- 172.21.15.39:0/849002477 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe69c014a90 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.528+0000 7fe6a17fa640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe6880666c0 0x7fe688068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.528+0000 7fe6a17fa640 1 -- 172.21.15.39:0/849002477 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fe69c085a10 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.528+0000 7fe6a8a5a640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe6880666c0 0x7fe688068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.528+0000 7fe6a8a5a640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe6880666c0 0x7fe688068b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6900097c0 tx=0x7fe690008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:18.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.532+0000 7fe6a17fa640 1 -- 172.21.15.39:0/849002477 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe69c0528f0 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.780+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6a40c2b50 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6a17fa640 1 -- 172.21.15.39:0/849002477 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5175 (secure 0 0 0) 0x7fe69c055f10 con 0x7fe6a40c0ac0 2024-09-13T16:51:18.788 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:18.788 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 34 2024-09-13T16:51:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe6880666c0 msgr2=0x7fe688068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe6880666c0 0x7fe688068b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6900097c0 tx=0x7fe690008040 comp rx=0 tx=0).stop 2024-09-13T16:51:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a40c0ac0 msgr2=0x7fe6a413cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a40c0ac0 0x7fe6a413cb20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009fd0 tx=0x7fe69c00e4b0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 shutdown_connections 2024-09-13T16:51:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe6880666c0 0x7fe688068b80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6a40ab9e0 0x7fe6a40bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a40c0ac0 0x7fe6a413cb20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 --2- 172.21.15.39:0/849002477 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe6a4071bb0 0x7fe6a413eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 >> 172.21.15.39:0/849002477 conn(0x7fe6a4069e10 msgr2=0x7fe6a40c5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 shutdown_connections 2024-09-13T16:51:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:18.784+0000 7fe6aacbe640 1 -- 172.21.15.39:0/849002477 wait complete. 2024-09-13T16:51:18.808 INFO:tasks.mds_thrash.fs.[b]:mds.b has gained rank=0, replacing gid=5142 2024-09-13T16:51:18.808 INFO:tasks.mds_thrash.fs.[b]:waiting for 1 secs before reviving mds.a 2024-09-13T16:51:19.809 INFO:tasks.mds_thrash.fs.[b]:reviving mds.a 2024-09-13T16:51:19.809 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:51:19.809 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:51:19.812 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:51:19.812 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:19.897 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T16:51:19.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6aad32640 1 Processor -- start 2024-09-13T16:51:19.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6aad32640 1 -- start start 2024-09-13T16:51:19.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6aad32640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb6a4151550 0x7fb6a406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:19.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6aad32640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb6a406b360 0x7fb6a4153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6aad32640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a41519e0 con 0x7fb6a414acf0 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6aad32640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a40c4e20 con 0x7fb6a406b360 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6aad32640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a4154ab0 con 0x7fb6a4151550 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6a92cf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb6a406b360 0x7fb6a4153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6a8ace640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb6a414acf0 0x7fb6a40c4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40552/0 (socket says 172.21.15.39:40552) 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6a8ace640 1 -- 172.21.15.39:0/1117457412 learned_addr learned my addr 172.21.15.39:0/1117457412 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6a3fff640 1 --2- 172.21.15.39:0/1117457412 >> v2:172.21.15.84:3300/0 conn(0x7fb6a4151550 0x7fb6a406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6a92cf640 1 -- 172.21.15.39:0/1117457412 >> v2:172.21.15.84:3300/0 conn(0x7fb6a4151550 msgr2=0x7fb6a406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6a92cf640 1 --2- 172.21.15.39:0/1117457412 >> v2:172.21.15.84:3300/0 conn(0x7fb6a4151550 0x7fb6a406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6a92cf640 1 -- 172.21.15.39:0/1117457412 >> v1:172.21.15.39:6789/0 conn(0x7fb6a414acf0 legacy=0x7fb6a40c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6a37fe640 1 -- 172.21.15.39:0/1117457412 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247339443 0 0) 0x7fb6a41519e0 con 0x7fb6a414acf0 2024-09-13T16:51:19.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.944+0000 7fb6a92cf640 1 -- 172.21.15.39:0/1117457412 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a4154d90 con 0x7fb6a406b360 2024-09-13T16:51:19.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6a92cf640 1 --2- 172.21.15.39:0/1117457412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 0x7fb6a4153ff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009d90 tx=0x7fb69c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaef3f9656da0ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:19.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6a37fe640 1 -- 172.21.15.39:0/1117457412 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0a5930 con 0x7fb6a406b360 2024-09-13T16:51:19.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6a37fe640 1 -- 172.21.15.39:0/1117457412 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c0a5a90 con 0x7fb6a406b360 2024-09-13T16:51:19.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6a37fe640 1 -- 172.21.15.39:0/1117457412 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0a5e20 con 0x7fb6a406b360 2024-09-13T16:51:19.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 -- 172.21.15.39:0/1117457412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 msgr2=0x7fb6a4153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:19.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 --2- 172.21.15.39:0/1117457412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 0x7fb6a4153ff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009d90 tx=0x7fb69c01c870 comp rx=0 tx=0).stop 2024-09-13T16:51:19.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 -- 172.21.15.39:0/1117457412 shutdown_connections 2024-09-13T16:51:19.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 --2- 172.21.15.39:0/1117457412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 0x7fb6a4153ff0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:19.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 --2- 172.21.15.39:0/1117457412 >> v2:172.21.15.84:3300/0 conn(0x7fb6a4151550 0x7fb6a406ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:19.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 -- 172.21.15.39:0/1117457412 >> 172.21.15.39:0/1117457412 conn(0x7fb6a40c32a0 msgr2=0x7fb6a40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:19.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 -- 172.21.15.39:0/1117457412 shutdown_connections 2024-09-13T16:51:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 -- 172.21.15.39:0/1117457412 wait complete. 2024-09-13T16:51:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 Processor -- start 2024-09-13T16:51:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.948+0000 7fb6aad32640 1 -- start start 2024-09-13T16:51:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6aad32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 0x7fb6a41ee090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6aad32640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6a414acf0 0x7fb6a4148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6aad32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6a4151550 0x7fb6a4148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6aad32640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a4149680 con 0x7fb6a406b360 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6aad32640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a41ebdd0 con 0x7fb6a4151550 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6aad32640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a41ee9e0 con 0x7fb6a414acf0 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a8ace640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 0x7fb6a41ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a8ace640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 0x7fb6a41ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54788/0 (socket says 172.21.15.39:54788) 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a8ace640 1 -- 172.21.15.39:0/2743270283 learned_addr learned my addr 172.21.15.39:0/2743270283 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a3fff640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6a414acf0 0x7fb6a4148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a92cf640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6a4151550 0x7fb6a4148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a8ace640 1 -- 172.21.15.39:0/2743270283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6a414acf0 msgr2=0x7fb6a4148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a8ace640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6a414acf0 0x7fb6a4148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a8ace640 1 -- 172.21.15.39:0/2743270283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6a4151550 msgr2=0x7fb6a4148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a8ace640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6a4151550 0x7fb6a4148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a8ace640 1 -- 172.21.15.39:0/2743270283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c009a70 con 0x7fb6a406b360 2024-09-13T16:51:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a92cf640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6a4151550 0x7fb6a4148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:51:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a8ace640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 0x7fb6a41ee090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb69800c480 tx=0x7fb69800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a17fa640 1 -- 172.21.15.39:0/2743270283 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6980074e0 con 0x7fb6a406b360 2024-09-13T16:51:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a17fa640 1 -- 172.21.15.39:0/2743270283 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb698007660 con 0x7fb6a406b360 2024-09-13T16:51:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a17fa640 1 -- 172.21.15.39:0/2743270283 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb698004a30 con 0x7fb6a406b360 2024-09-13T16:51:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a41ec070 con 0x7fb6a406b360 2024-09-13T16:51:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a4153b60 con 0x7fb6a406b360 2024-09-13T16:51:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a4153ff0 con 0x7fb6a406b360 2024-09-13T16:51:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a17fa640 1 -- 172.21.15.39:0/2743270283 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb698004b90 con 0x7fb6a406b360 2024-09-13T16:51:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a17fa640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb688066690 0x7fb688068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a17fa640 1 -- 172.21.15.39:0/2743270283 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fb698083810 con 0x7fb6a406b360 2024-09-13T16:51:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.952+0000 7fb6a3fff640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb688066690 0x7fb688068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.956+0000 7fb6a3fff640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb688066690 0x7fb688068b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40c4d30 tx=0x7fb690008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:19.960+0000 7fb6a17fa640 1 -- 172.21.15.39:0/2743270283 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb6980541a0 con 0x7fb6a406b360 2024-09-13T16:51:20.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.188+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6a40a5ac0 con 0x7fb6a406b360 2024-09-13T16:51:20.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.188+0000 7fb6a17fa640 1 -- 172.21.15.39:0/2743270283 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5175 (secure 0 0 0) 0x7fb698018090 con 0x7fb6a406b360 2024-09-13T16:51:20.192 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:20.193 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:20.193 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 34 2024-09-13T16:51:20.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.188+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb688066690 msgr2=0x7fb688068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.188+0000 7fb6aad32640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb688066690 0x7fb688068b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40c4d30 tx=0x7fb690008040 comp rx=0 tx=0).stop 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.188+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 msgr2=0x7fb6a41ee090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.188+0000 7fb6aad32640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 0x7fb6a41ee090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb69800c480 tx=0x7fb69800c940 comp rx=0 tx=0).stop 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.192+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 shutdown_connections 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.192+0000 7fb6aad32640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb688066690 0x7fb688068b50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.192+0000 7fb6aad32640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6a4151550 0x7fb6a4148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.192+0000 7fb6aad32640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb6a414acf0 0x7fb6a4148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.192+0000 7fb6aad32640 1 --2- 172.21.15.39:0/2743270283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6a406b360 0x7fb6a41ee090 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.192+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 >> 172.21.15.39:0/2743270283 conn(0x7fb6a40c32a0 msgr2=0x7fb6a406a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.192+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 shutdown_connections 2024-09-13T16:51:20.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:20.192+0000 7fb6aad32640 1 -- 172.21.15.39:0/2743270283 wait complete. 2024-09-13T16:51:20.213 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T16:51:22.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:51:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141f268640 1 Processor -- start 2024-09-13T16:51:22.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141f268640 1 -- start start 2024-09-13T16:51:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141f268640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1418071dd0 0x7f1418154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141f268640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180c5a30 con 0x7f14180c7800 2024-09-13T16:51:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141f268640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14181576a0 con 0x7f1418151500 2024-09-13T16:51:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141f268640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1418157980 con 0x7f1418071dd0 2024-09-13T16:51:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141d805640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f14180c7800 0x7f14181545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41594/0 (socket says 172.21.15.39:41594) 2024-09-13T16:51:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141d805640 1 -- 172.21.15.39:0/187553729 learned_addr learned my addr 172.21.15.39:0/187553729 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:51:22.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141c803640 1 --2- 172.21.15.39:0/187553729 >> v2:172.21.15.84:3300/0 conn(0x7f1418071dd0 0x7f1418154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393542111 0 0) 0x7f14180c5a30 con 0x7f14180c7800 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1404003530 con 0x7f14180c7800 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1043237089 0 0) 0x7f1404003530 con 0x7f14180c7800 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 >> v2:172.21.15.84:3300/0 conn(0x7f1418071dd0 msgr2=0x7f1418154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 --2- 172.21.15.39:0/187553729 >> v2:172.21.15.84:3300/0 conn(0x7f1418071dd0 0x7f1418154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 >> v1:172.21.15.84:6789/0 conn(0x7f1418151500 legacy=0x7f14180c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1418157c60 con 0x7f14180c7800 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1410001a20 con 0x7f14180c7800 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 >> v1:172.21.15.39:6789/0 conn(0x7f14180c7800 legacy=0x7f14181545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f14040039b0 0x7f1404003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14040047d0 0x7f1404004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040051e0 0x7f140400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1418157980 con 0x7f14040051e0 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140400c110 con 0x7f14040047d0 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140400c2d0 con 0x7f14040039b0 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141d805640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040051e0 0x7f140400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141d004640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f14040039b0 0x7f1404003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141c803640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14040047d0 0x7f1404004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141d805640 1 -- 172.21.15.39:0/187553729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f14040039b0 msgr2=0x7f1404003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141d805640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f14040039b0 0x7f1404003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141d805640 1 -- 172.21.15.39:0/187553729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14040047d0 msgr2=0x7f1404004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:51:22.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141d805640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14040047d0 0x7f1404004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.360+0000 7f141d805640 1 -- 172.21.15.39:0/187553729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1404003530 con 0x7f14040051e0 2024-09-13T16:51:22.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f141d805640 1 -- 172.21.15.39:0/187553729 reap_dead start 2024-09-13T16:51:22.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f141d805640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040051e0 0x7f140400b990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f14180c7cb0 tx=0x7f141001fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11be87932073bd58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14100af070 con 0x7f14040051e0 2024-09-13T16:51:22.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14100aa4b0 con 0x7f14040051e0 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f140ffff640 1 -- 172.21.15.39:0/187553729 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14100aa860 con 0x7f14040051e0 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f141f268640 1 -- 172.21.15.39:0/187553729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040051e0 msgr2=0x7f140400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f141f268640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040051e0 0x7f140400b990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f14180c7cb0 tx=0x7f141001fa80 comp rx=0 tx=0).stop 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f141f268640 1 -- 172.21.15.39:0/187553729 shutdown_connections 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f141f268640 1 --2- 172.21.15.39:0/187553729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040051e0 0x7f140400b990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f141f268640 1 -- 172.21.15.39:0/187553729 >> 172.21.15.39:0/187553729 conn(0x7f14180ab530 msgr2=0x7f14180ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f141f268640 1 -- 172.21.15.39:0/187553729 shutdown_connections 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.364+0000 7f141f268640 1 -- 172.21.15.39:0/187553729 wait complete. 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 Processor -- start 2024-09-13T16:51:22.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 -- start start 2024-09-13T16:51:22.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f14040051e0 0x7f1418155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1418147250 0x7f14181476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f141814f380 0x7f141814f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1418147c10 con 0x7f1418147250 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1418147dc0 con 0x7f141814f380 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1418153210 con 0x7f14040051e0 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141c803640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1418147250 0x7f14181476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141c803640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1418147250 0x7f14181476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48262/0 (socket says 172.21.15.39:48262) 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141c803640 1 -- 172.21.15.39:0/684433766 learned_addr learned my addr 172.21.15.39:0/684433766 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141d805640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f141814f380 0x7f141814f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141d004640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f14040051e0 0x7f1418155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141c803640 1 -- 172.21.15.39:0/684433766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f14040051e0 msgr2=0x7f1418155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141c803640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f14040051e0 0x7f1418155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141c803640 1 -- 172.21.15.39:0/684433766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f141814f380 msgr2=0x7f141814f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141c803640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f141814f380 0x7f141814f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141c803640 1 -- 172.21.15.39:0/684433766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1418152a50 con 0x7f1418147250 2024-09-13T16:51:22.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141c803640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1418147250 0x7f14181476d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1408011560 tx=0x7f1408011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f140dffb640 1 -- 172.21.15.39:0/684433766 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408015870 con 0x7f1418147250 2024-09-13T16:51:22.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f140dffb640 1 -- 172.21.15.39:0/684433766 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14080159d0 con 0x7f1418147250 2024-09-13T16:51:22.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1418155400 con 0x7f1418147250 2024-09-13T16:51:22.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f140dffb640 1 -- 172.21.15.39:0/684433766 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408015d40 con 0x7f1418147250 2024-09-13T16:51:22.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14181556e0 con 0x7f1418147250 2024-09-13T16:51:22.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.368+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14180c4280 con 0x7f1418147250 2024-09-13T16:51:22.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.372+0000 7f140dffb640 1 -- 172.21.15.39:0/684433766 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1408026420 con 0x7f1418147250 2024-09-13T16:51:22.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.372+0000 7f140dffb640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13fc066690 0x7f13fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:51:22.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.372+0000 7f140dffb640 1 -- 172.21.15.39:0/684433766 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f1408084d80 con 0x7f1418147250 2024-09-13T16:51:22.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.372+0000 7f141d004640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13fc066690 0x7f13fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:51:22.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.372+0000 7f141d004640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13fc066690 0x7f13fc068b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1414000c10 tx=0x7f1414009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:51:22.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.376+0000 7f140dffb640 1 -- 172.21.15.39:0/684433766 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1408051cd0 con 0x7f1418147250 2024-09-13T16:51:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.596+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1418071bb0 con 0x7f1418147250 2024-09-13T16:51:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.596+0000 7f140dffb640 1 -- 172.21.15.39:0/684433766 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f14080552f0 con 0x7f1418147250 2024-09-13T16:51:22.601 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:51:22.601 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:51:22.601 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:51:22.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13fc066690 msgr2=0x7f13fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13fc066690 0x7f13fc068b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1414000c10 tx=0x7f1414009040 comp rx=0 tx=0).stop 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1418147250 msgr2=0x7f14181476d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1418147250 0x7f14181476d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1408011560 tx=0x7f1408011a20 comp rx=0 tx=0).stop 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 shutdown_connections 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13fc066690 0x7f13fc068b50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f141814f380 0x7f141814f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1418147250 0x7f14181476d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 --2- 172.21.15.39:0/684433766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f14040051e0 0x7f1418155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 >> 172.21.15.39:0/684433766 conn(0x7f14180ab530 msgr2=0x7f14180ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 shutdown_connections 2024-09-13T16:51:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:51:22.600+0000 7f141f268640 1 -- 172.21.15.39:0/684433766 wait complete. 2024-09-13T16:51:22.622 INFO:tasks.mds_thrash.fs.[b]:mds.a reported in up:standby state 2024-09-13T16:51:22.623 INFO:tasks.mds_thrash.fs.[b]:waiting for 44 secs before thrashing 2024-09-13T16:51:25.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:51:25.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:51:55.928 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:51:55.931 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:52:06.624 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd17cd0640 1 Processor -- start 2024-09-13T16:52:06.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd17cd0640 1 -- start start 2024-09-13T16:52:06.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd17cd0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd100bd010 0x7fdd10070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd17cd0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdd100c7860 0x7fdd10070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd17cd0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd10157e80 con 0x7fdd10151750 2024-09-13T16:52:06.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd17cd0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd10151b30 con 0x7fdd100c7860 2024-09-13T16:52:06.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd17cd0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd10158160 con 0x7fdd100bd010 2024-09-13T16:52:06.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd15a6c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdd10151750 0x7fdd100c42e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50088/0 (socket says 172.21.15.39:50088) 2024-09-13T16:52:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd15a6c640 1 -- 172.21.15.39:0/873822767 learned_addr learned my addr 172.21.15.39:0/873822767 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd1626d640 1 --2- 172.21.15.39:0/873822767 >> v2:172.21.15.39:3300/0 conn(0x7fdd100c7860 0x7fdd10070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd1526b640 1 --2- 172.21.15.39:0/873822767 >> v2:172.21.15.84:3300/0 conn(0x7fdd100bd010 0x7fdd10070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd14a6a640 1 -- 172.21.15.39:0/873822767 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335017615 0 0) 0x7fdd10157e80 con 0x7fdd10151750 2024-09-13T16:52:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd14a6a640 1 -- 172.21.15.39:0/873822767 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcfc003530 con 0x7fdd10151750 2024-09-13T16:52:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd1626d640 1 -- 172.21.15.39:0/873822767 >> v2:172.21.15.84:3300/0 conn(0x7fdd100bd010 msgr2=0x7fdd10070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd1626d640 1 --2- 172.21.15.39:0/873822767 >> v2:172.21.15.84:3300/0 conn(0x7fdd100bd010 0x7fdd10070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd1626d640 1 -- 172.21.15.39:0/873822767 >> v1:172.21.15.39:6789/0 conn(0x7fdd10151750 legacy=0x7fdd100c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.768+0000 7fdd1626d640 1 -- 172.21.15.39:0/873822767 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd100c6370 con 0x7fdd100c7860 2024-09-13T16:52:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd1626d640 1 --2- 172.21.15.39:0/873822767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 0x7fdd10070970 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdd100674e0 tx=0x7fdd0801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71e992137378346 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd14a6a640 1 -- 172.21.15.39:0/873822767 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd080a59c0 con 0x7fdd100c7860 2024-09-13T16:52:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd14a6a640 1 -- 172.21.15.39:0/873822767 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd080a5b20 con 0x7fdd100c7860 2024-09-13T16:52:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd14a6a640 1 -- 172.21.15.39:0/873822767 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd080a5e90 con 0x7fdd100c7860 2024-09-13T16:52:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd17cd0640 1 -- 172.21.15.39:0/873822767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 msgr2=0x7fdd10070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd17cd0640 1 --2- 172.21.15.39:0/873822767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 0x7fdd10070970 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdd100674e0 tx=0x7fdd0801c900 comp rx=0 tx=0).stop 2024-09-13T16:52:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd17cd0640 1 -- 172.21.15.39:0/873822767 shutdown_connections 2024-09-13T16:52:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd17cd0640 1 --2- 172.21.15.39:0/873822767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 0x7fdd10070970 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd17cd0640 1 --2- 172.21.15.39:0/873822767 >> v2:172.21.15.84:3300/0 conn(0x7fdd100bd010 0x7fdd10070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd17cd0640 1 -- 172.21.15.39:0/873822767 >> 172.21.15.39:0/873822767 conn(0x7fdd10069e10 msgr2=0x7fdd1006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:06.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd17cd0640 1 -- 172.21.15.39:0/873822767 shutdown_connections 2024-09-13T16:52:06.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.772+0000 7fdd17cd0640 1 -- 172.21.15.39:0/873822767 wait complete. 2024-09-13T16:52:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd17cd0640 1 Processor -- start 2024-09-13T16:52:06.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd17cd0640 1 -- start start 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd17cd0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd100bd010 0x7fdd101542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd17cd0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 0x7fdd1015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd17cd0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd10151750 0x7fdd100bf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd17cd0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100be530 con 0x7fdd100c7860 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd17cd0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100bd490 con 0x7fdd100bd010 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd17cd0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100bd770 con 0x7fdd10151750 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1526b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 0x7fdd1015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1526b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 0x7fdd1015dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44076/0 (socket says 172.21.15.39:44076) 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1526b640 1 -- 172.21.15.39:0/1909499102 learned_addr learned my addr 172.21.15.39:0/1909499102 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:06.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd15a6c640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd100bd010 0x7fdd101542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1626d640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd10151750 0x7fdd100bf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1526b640 1 -- 172.21.15.39:0/1909499102 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd10151750 msgr2=0x7fdd100bf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1526b640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd10151750 0x7fdd100bf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1526b640 1 -- 172.21.15.39:0/1909499102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd100bd010 msgr2=0x7fdd101542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1526b640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd100bd010 0x7fdd101542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1526b640 1 -- 172.21.15.39:0/1909499102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd08009a70 con 0x7fdd100c7860 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1626d640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd10151750 0x7fdd100bf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd1526b640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 0x7fdd1015dd10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000d800 tx=0x7fdd0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd067fc640 1 -- 172.21.15.39:0/1909499102 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000b6c0 con 0x7fdd100c7860 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd067fc640 1 -- 172.21.15.39:0/1909499102 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0000b820 con 0x7fdd100c7860 2024-09-13T16:52:06.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.776+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd100bda50 con 0x7fdd100c7860 2024-09-13T16:52:06.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.780+0000 7fdd067fc640 1 -- 172.21.15.39:0/1909499102 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000bb90 con 0x7fdd100c7860 2024-09-13T16:52:06.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.780+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd100be010 con 0x7fdd100c7860 2024-09-13T16:52:06.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.780+0000 7fdd067fc640 1 -- 172.21.15.39:0/1909499102 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd0000bcf0 con 0x7fdd100c7860 2024-09-13T16:52:06.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.780+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd100be1d0 con 0x7fdd100c7860 2024-09-13T16:52:06.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.780+0000 7fdd067fc640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcf4066690 0x7fdcf4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:06.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.780+0000 7fdd067fc640 1 -- 172.21.15.39:0/1909499102 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fdd00084ba0 con 0x7fdd100c7860 2024-09-13T16:52:06.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.780+0000 7fdd15a6c640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcf4066690 0x7fdcf4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:06.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.780+0000 7fdd15a6c640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcf4066690 0x7fdcf4068b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005890 tx=0x7fdd0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:06.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:06.784+0000 7fdd067fc640 1 -- 172.21.15.39:0/1909499102 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd00051aa0 con 0x7fdd100c7860 2024-09-13T16:52:07.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.036+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd10157e80 con 0x7fdd100c7860 2024-09-13T16:52:07.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.036+0000 7fdd067fc640 1 -- 172.21.15.39:0/1909499102 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fdd000550c0 con 0x7fdd100c7860 2024-09-13T16:52:07.040 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:07.041 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:07.041 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcf4066690 msgr2=0x7fdcf4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcf4066690 0x7fdcf4068b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005890 tx=0x7fdd0c009040 comp rx=0 tx=0).stop 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 msgr2=0x7fdd1015dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 0x7fdd1015dd10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000d800 tx=0x7fdd0000dcc0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 shutdown_connections 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcf4066690 0x7fdcf4068b50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd10151750 0x7fdd100bf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd100c7860 0x7fdd1015dd10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 --2- 172.21.15.39:0/1909499102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd100bd010 0x7fdd101542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 >> 172.21.15.39:0/1909499102 conn(0x7fdd10069e10 msgr2=0x7fdd100c41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 shutdown_connections 2024-09-13T16:52:07.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.040+0000 7fdd17cd0640 1 -- 172.21.15.39:0/1909499102 wait complete. 2024-09-13T16:52:07.062 INFO:tasks.mds_thrash.fs.[b]:kill mds.b (rank=0) 2024-09-13T16:52:07.062 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:52:07.062 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:07.082 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:52:07.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:07.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe339d0e640 1 Processor -- start 2024-09-13T16:52:07.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe339d0e640 1 -- start start 2024-09-13T16:52:07.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe339d0e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe334151500 0x7fe3340c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe339d0e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3340c47c0 0x7fe334154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe339d0e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3340c5a30 con 0x7fe334071dd0 2024-09-13T16:52:07.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe339d0e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3341576a0 con 0x7fe334151500 2024-09-13T16:52:07.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe339d0e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe334157980 con 0x7fe3340c47c0 2024-09-13T16:52:07.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe3337fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe334151500 0x7fe3340c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe333fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3340c47c0 0x7fe334154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe3337fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe334151500 0x7fe3340c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:59476/0 (socket says 172.21.15.39:59476) 2024-09-13T16:52:07.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.200+0000 7fe3337fe640 1 -- 172.21.15.39:0/1607234602 learned_addr learned my addr 172.21.15.39:0/1607234602 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:07.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.204+0000 7fe3337fe640 1 -- 172.21.15.39:0/1607234602 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3340c47c0 msgr2=0x7fe334154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:07.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.204+0000 7fe3337fe640 1 --2- 172.21.15.39:0/1607234602 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3340c47c0 0x7fe334154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.204+0000 7fe3337fe640 1 -- 172.21.15.39:0/1607234602 >> v1:172.21.15.84:6789/0 conn(0x7fe334071dd0 legacy=0x7fe334154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.204+0000 7fe3337fe640 1 -- 172.21.15.39:0/1607234602 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe334157c60 con 0x7fe334151500 2024-09-13T16:52:07.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.204+0000 7fe333fff640 1 --2- 172.21.15.39:0/1607234602 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3340c47c0 0x7fe334154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:07.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.204+0000 7fe3337fe640 1 --2- 172.21.15.39:0/1607234602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe334151500 0x7fe3340c4280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe3240098a0 tx=0x7fe32401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a4144eafd2496b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:07.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.204+0000 7fe332ffd640 1 -- 172.21.15.39:0/1607234602 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3240a6040 con 0x7fe334151500 2024-09-13T16:52:07.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.204+0000 7fe332ffd640 1 -- 172.21.15.39:0/1607234602 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3240aa600 con 0x7fe334151500 2024-09-13T16:52:07.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.204+0000 7fe332ffd640 1 -- 172.21.15.39:0/1607234602 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3240aa990 con 0x7fe334151500 2024-09-13T16:52:07.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- 172.21.15.39:0/1607234602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe334151500 msgr2=0x7fe3340c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 --2- 172.21.15.39:0/1607234602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe334151500 0x7fe3340c4280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe3240098a0 tx=0x7fe32401c480 comp rx=0 tx=0).stop 2024-09-13T16:52:07.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- 172.21.15.39:0/1607234602 shutdown_connections 2024-09-13T16:52:07.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 --2- 172.21.15.39:0/1607234602 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3340c47c0 0x7fe334154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 --2- 172.21.15.39:0/1607234602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe334151500 0x7fe3340c4280 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- 172.21.15.39:0/1607234602 >> 172.21.15.39:0/1607234602 conn(0x7fe3340ab530 msgr2=0x7fe3340ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:07.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- 172.21.15.39:0/1607234602 shutdown_connections 2024-09-13T16:52:07.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- 172.21.15.39:0/1607234602 wait complete. 2024-09-13T16:52:07.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 Processor -- start 2024-09-13T16:52:07.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- start start 2024-09-13T16:52:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe334071dd0 0x7fe3340c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3340c47c0 0x7fe3341472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe334151500 0x7fe3341f0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe334148bf0 con 0x7fe334071dd0 2024-09-13T16:52:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe334147d60 con 0x7fe3340c47c0 2024-09-13T16:52:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe334147fb0 con 0x7fe334151500 2024-09-13T16:52:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe334071dd0 0x7fe3340c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe334071dd0 0x7fe3340c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44084/0 (socket says 172.21.15.39:44084) 2024-09-13T16:52:07.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe3337fe640 1 -- 172.21.15.39:0/43808729 learned_addr learned my addr 172.21.15.39:0/43808729 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:07.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe32bfff640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3340c47c0 0x7fe3341472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe333fff640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe334151500 0x7fe3341f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe3337fe640 1 -- 172.21.15.39:0/43808729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe334151500 msgr2=0x7fe3341f0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe3337fe640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe334151500 0x7fe3341f0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe3337fe640 1 -- 172.21.15.39:0/43808729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3340c47c0 msgr2=0x7fe3341472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe3337fe640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3340c47c0 0x7fe3341472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe3337fe640 1 -- 172.21.15.39:0/43808729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c009a70 con 0x7fe334071dd0 2024-09-13T16:52:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe3337fe640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe334071dd0 0x7fe3340c8cf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe32401a0b0 tx=0x7fe32401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe330ff9640 1 -- 172.21.15.39:0/43808729 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3240b3600 con 0x7fe334071dd0 2024-09-13T16:52:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe330ff9640 1 -- 172.21.15.39:0/43808729 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3240b3760 con 0x7fe334071dd0 2024-09-13T16:52:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3240094d0 con 0x7fe334071dd0 2024-09-13T16:52:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.208+0000 7fe330ff9640 1 -- 172.21.15.39:0/43808729 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3240b3af0 con 0x7fe334071dd0 2024-09-13T16:52:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.212+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe334146da0 con 0x7fe334071dd0 2024-09-13T16:52:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.212+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3340c4280 con 0x7fe334071dd0 2024-09-13T16:52:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.212+0000 7fe330ff9640 1 -- 172.21.15.39:0/43808729 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe3240b3c50 con 0x7fe334071dd0 2024-09-13T16:52:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.212+0000 7fe330ff9640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe314066710 0x7fe314068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.212+0000 7fe330ff9640 1 -- 172.21.15.39:0/43808729 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fe32411ef50 con 0x7fe334071dd0 2024-09-13T16:52:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.212+0000 7fe32bfff640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe314066710 0x7fe314068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.212+0000 7fe32bfff640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe314066710 0x7fe314068bd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe33413fad0 tx=0x7fe31c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:07.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.216+0000 7fe330ff9640 1 -- 172.21.15.39:0/43808729 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe324150db0 con 0x7fe334071dd0 2024-09-13T16:52:07.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.484+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe33414eea0 con 0x7fe334071dd0 2024-09-13T16:52:07.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.484+0000 7fe330ff9640 1 -- 172.21.15.39:0/43808729 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fe32411e090 con 0x7fe334071dd0 2024-09-13T16:52:07.491 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:07.491 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:07.492 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:52:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe314066710 msgr2=0x7fe314068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe314066710 0x7fe314068bd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe33413fad0 tx=0x7fe31c009040 comp rx=0 tx=0).stop 2024-09-13T16:52:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe334071dd0 msgr2=0x7fe3340c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe334071dd0 0x7fe3340c8cf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe32401a0b0 tx=0x7fe32401acc0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 shutdown_connections 2024-09-13T16:52:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe314066710 0x7fe314068bd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3340c47c0 0x7fe3341472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe334151500 0x7fe3341f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 --2- 172.21.15.39:0/43808729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe334071dd0 0x7fe3340c8cf0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 >> 172.21.15.39:0/43808729 conn(0x7fe3340ab530 msgr2=0x7fe3340c6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 shutdown_connections 2024-09-13T16:52:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:07.488+0000 7fe339d0e640 1 -- 172.21.15.39:0/43808729 wait complete. 2024-09-13T16:52:07.515 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:52:09.516 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.684+0000 7fadb3b9f640 1 Processor -- start 2024-09-13T16:52:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.684+0000 7fadb3b9f640 1 -- start start 2024-09-13T16:52:09.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.684+0000 7fadb3b9f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0abc50 0x7fadac14f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.684+0000 7fadb3b9f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fadac0ab340 0x7fadac14f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.684+0000 7fadb3b9f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadac156270 con 0x7fadac06cbc0 2024-09-13T16:52:09.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.684+0000 7fadb3b9f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac061530 con 0x7fadac0ab340 2024-09-13T16:52:09.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.684+0000 7fadb3b9f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac1554b0 con 0x7fadac0abc50 2024-09-13T16:52:09.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb113a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fadac0ab340 0x7fadac14f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb113a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fadac0ab340 0x7fadac14f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44088/0 (socket says 172.21.15.39:44088) 2024-09-13T16:52:09.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb113a640 1 -- 172.21.15.39:0/3276610794 learned_addr learned my addr 172.21.15.39:0/3276610794 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:09.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb193b640 1 --2- 172.21.15.39:0/3276610794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0abc50 0x7fadac14f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb113a640 1 -- 172.21.15.39:0/3276610794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0abc50 msgr2=0x7fadac14f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb113a640 1 --2- 172.21.15.39:0/3276610794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0abc50 0x7fadac14f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb113a640 1 -- 172.21.15.39:0/3276610794 >> v1:172.21.15.84:6789/0 conn(0x7fadac06cbc0 legacy=0x7fadac06d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:09.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb113a640 1 -- 172.21.15.39:0/3276610794 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac155790 con 0x7fadac0ab340 2024-09-13T16:52:09.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb113a640 1 --2- 172.21.15.39:0/3276610794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac0ab340 0x7fadac14f640 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009870 tx=0x7fad9c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b29a04fc711b50d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb0939640 1 -- 172.21.15.39:0/3276610794 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0ab610 con 0x7fadac0ab340 2024-09-13T16:52:09.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb0939640 1 -- 172.21.15.39:0/3276610794 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9c0ab770 con 0x7fadac0ab340 2024-09-13T16:52:09.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb0939640 1 -- 172.21.15.39:0/3276610794 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0abb00 con 0x7fadac0ab340 2024-09-13T16:52:09.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3276610794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac0ab340 msgr2=0x7fadac14f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb3b9f640 1 --2- 172.21.15.39:0/3276610794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac0ab340 0x7fadac14f640 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009870 tx=0x7fad9c01c410 comp rx=0 tx=0).stop 2024-09-13T16:52:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3276610794 shutdown_connections 2024-09-13T16:52:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb3b9f640 1 --2- 172.21.15.39:0/3276610794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac0ab340 0x7fadac14f640 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb3b9f640 1 --2- 172.21.15.39:0/3276610794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0abc50 0x7fadac14f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3276610794 >> 172.21.15.39:0/3276610794 conn(0x7fadac069e10 msgr2=0x7fadac06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.688+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3276610794 shutdown_connections 2024-09-13T16:52:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3276610794 wait complete. 2024-09-13T16:52:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 Processor -- start 2024-09-13T16:52:09.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 -- start start 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac06cbc0 0x7fadac14ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0ab340 0x7fadac0c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadac0abc50 0x7fadac151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac1ee660 con 0x7fadac06cbc0 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0c5c00 con 0x7fadac0abc50 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac150440 con 0x7fadac0ab340 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb193b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac06cbc0 0x7fadac14ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb193b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac06cbc0 0x7fadac14ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44102/0 (socket says 172.21.15.39:44102) 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb193b640 1 -- 172.21.15.39:0/3046970906 learned_addr learned my addr 172.21.15.39:0/3046970906 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb113a640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0ab340 0x7fadac0c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb213c640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadac0abc50 0x7fadac151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb193b640 1 -- 172.21.15.39:0/3046970906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0ab340 msgr2=0x7fadac0c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb193b640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0ab340 0x7fadac0c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb193b640 1 -- 172.21.15.39:0/3046970906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadac0abc50 msgr2=0x7fadac151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb193b640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadac0abc50 0x7fadac151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb193b640 1 -- 172.21.15.39:0/3046970906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c0094d0 con 0x7fadac06cbc0 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb193b640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac06cbc0 0x7fadac14ff00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fada800d860 tx=0x7fada800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fada27fc640 1 -- 172.21.15.39:0/3046970906 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada800de50 con 0x7fadac06cbc0 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fada27fc640 1 -- 172.21.15.39:0/3046970906 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada800b8d0 con 0x7fadac06cbc0 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac0c52d0 con 0x7fadac06cbc0 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fada27fc640 1 -- 172.21.15.39:0/3046970906 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada800bc40 con 0x7fadac06cbc0 2024-09-13T16:52:09.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.692+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac0c5890 con 0x7fadac06cbc0 2024-09-13T16:52:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.696+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac0ac030 con 0x7fadac06cbc0 2024-09-13T16:52:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.696+0000 7fada27fc640 1 -- 172.21.15.39:0/3046970906 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fada801e460 con 0x7fadac06cbc0 2024-09-13T16:52:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.696+0000 7fada27fc640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad90066690 0x7fad90068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.696+0000 7fada27fc640 1 -- 172.21.15.39:0/3046970906 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fada8084b70 con 0x7fadac06cbc0 2024-09-13T16:52:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.696+0000 7fadb113a640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad90066690 0x7fad90068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:09.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.696+0000 7fadb113a640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad90066690 0x7fad90068b50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0af000 tx=0x7fad9c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.700+0000 7fada27fc640 1 -- 172.21.15.39:0/3046970906 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fada8051a70 con 0x7fadac06cbc0 2024-09-13T16:52:09.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.940+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadac0c07a0 con 0x7fadac06cbc0 2024-09-13T16:52:09.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.940+0000 7fada27fc640 1 -- 172.21.15.39:0/3046970906 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fada8055090 con 0x7fadac06cbc0 2024-09-13T16:52:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:09.945 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:09.945 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:52:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad90066690 msgr2=0x7fad90068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad90066690 0x7fad90068b50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0af000 tx=0x7fad9c0a7000 comp rx=0 tx=0).stop 2024-09-13T16:52:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac06cbc0 msgr2=0x7fadac14ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac06cbc0 0x7fadac14ff00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fada800d860 tx=0x7fada800dd20 comp rx=0 tx=0).stop 2024-09-13T16:52:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 shutdown_connections 2024-09-13T16:52:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad90066690 0x7fad90068b50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fadac0abc50 0x7fadac151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fadac0ab340 0x7fadac0c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 --2- 172.21.15.39:0/3046970906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac06cbc0 0x7fadac14ff00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 >> 172.21.15.39:0/3046970906 conn(0x7fadac069e10 msgr2=0x7fadac14f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 shutdown_connections 2024-09-13T16:52:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:09.944+0000 7fadb3b9f640 1 -- 172.21.15.39:0/3046970906 wait complete. 2024-09-13T16:52:09.966 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:52:11.967 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:12.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.104+0000 7efc5466a640 1 Processor -- start 2024-09-13T16:52:12.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.104+0000 7efc5466a640 1 -- start start 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.104+0000 7efc5466a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc4c14a770 0x7efc4c150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c0c6060 0x7efc4c14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c0c4760 con 0x7efc4c06ae20 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c06df80 con 0x7efc4c14a770 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0c1950 con 0x7efc4c0c6060 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc52406640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc4c14a770 0x7efc4c150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc52406640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc4c14a770 0x7efc4c150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52468/0 (socket says 172.21.15.39:52468) 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc52406640 1 -- 172.21.15.39:0/295879466 learned_addr learned my addr 172.21.15.39:0/295879466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc51c05640 1 --2- 172.21.15.39:0/295879466 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c0c6060 0x7efc4c14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:12.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc52406640 1 -- 172.21.15.39:0/295879466 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c0c6060 msgr2=0x7efc4c14ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:12.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc52406640 1 --2- 172.21.15.39:0/295879466 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c0c6060 0x7efc4c14ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc52406640 1 -- 172.21.15.39:0/295879466 >> v1:172.21.15.84:6789/0 conn(0x7efc4c06ae20 legacy=0x7efc4c0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:12.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc52406640 1 -- 172.21.15.39:0/295879466 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c151120 con 0x7efc4c14a770 2024-09-13T16:52:12.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc52406640 1 --2- 172.21.15.39:0/295879466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c14a770 0x7efc4c150ad0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efc44009e50 tx=0x7efc4401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf8fa513e10d0e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:12.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc51404640 1 -- 172.21.15.39:0/295879466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440a56c0 con 0x7efc4c14a770 2024-09-13T16:52:12.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc51404640 1 -- 172.21.15.39:0/295879466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc440a5820 con 0x7efc4c14a770 2024-09-13T16:52:12.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc51404640 1 -- 172.21.15.39:0/295879466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440a5bb0 con 0x7efc4c14a770 2024-09-13T16:52:12.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 -- 172.21.15.39:0/295879466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c14a770 msgr2=0x7efc4c150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:12.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 --2- 172.21.15.39:0/295879466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c14a770 0x7efc4c150ad0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efc44009e50 tx=0x7efc4401c930 comp rx=0 tx=0).stop 2024-09-13T16:52:12.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 -- 172.21.15.39:0/295879466 shutdown_connections 2024-09-13T16:52:12.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 --2- 172.21.15.39:0/295879466 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c0c6060 0x7efc4c14ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 --2- 172.21.15.39:0/295879466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c14a770 0x7efc4c150ad0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 -- 172.21.15.39:0/295879466 >> 172.21.15.39:0/295879466 conn(0x7efc4c0c32a0 msgr2=0x7efc4c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:12.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 -- 172.21.15.39:0/295879466 shutdown_connections 2024-09-13T16:52:12.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.108+0000 7efc5466a640 1 -- 172.21.15.39:0/295879466 wait complete. 2024-09-13T16:52:12.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 Processor -- start 2024-09-13T16:52:12.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 -- start start 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c06ae20 0x7efc4c0c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4c0c6060 0x7efc4c149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c14a770 0x7efc4c149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c1f1010 con 0x7efc4c06ae20 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0bf940 con 0x7efc4c0c6060 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c1f0250 con 0x7efc4c14a770 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52406640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c06ae20 0x7efc4c0c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52406640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c06ae20 0x7efc4c0c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52476/0 (socket says 172.21.15.39:52476) 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52406640 1 -- 172.21.15.39:0/891109496 learned_addr learned my addr 172.21.15.39:0/891109496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:12.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc51c05640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4c0c6060 0x7efc4c149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52c07640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c14a770 0x7efc4c149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52406640 1 -- 172.21.15.39:0/891109496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c14a770 msgr2=0x7efc4c149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52406640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c14a770 0x7efc4c149a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52406640 1 -- 172.21.15.39:0/891109496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4c0c6060 msgr2=0x7efc4c149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52406640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4c0c6060 0x7efc4c149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52406640 1 -- 172.21.15.39:0/891109496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc44009b30 con 0x7efc4c06ae20 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52c07640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c14a770 0x7efc4c149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc52406640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c06ae20 0x7efc4c0c2840 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efc4401afe0 tx=0x7efc440196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc42ffd640 1 -- 172.21.15.39:0/891109496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440b8670 con 0x7efc4c06ae20 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc42ffd640 1 -- 172.21.15.39:0/891109496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc440b87d0 con 0x7efc4c06ae20 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c1f0530 con 0x7efc4c06ae20 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc42ffd640 1 -- 172.21.15.39:0/891109496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440b8b80 con 0x7efc4c06ae20 2024-09-13T16:52:12.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.112+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c1f0af0 con 0x7efc4c06ae20 2024-09-13T16:52:12.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.116+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c0619c0 con 0x7efc4c06ae20 2024-09-13T16:52:12.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.116+0000 7efc42ffd640 1 -- 172.21.15.39:0/891109496 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc440b8ce0 con 0x7efc4c06ae20 2024-09-13T16:52:12.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.116+0000 7efc42ffd640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc300666c0 0x7efc30068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:12.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.116+0000 7efc42ffd640 1 -- 172.21.15.39:0/891109496 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7efc4411f8c0 con 0x7efc4c06ae20 2024-09-13T16:52:12.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.116+0000 7efc51c05640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc300666c0 0x7efc30068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:12.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.116+0000 7efc51c05640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc300666c0 0x7efc30068b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efc4c1edc70 tx=0x7efc48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:12.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.120+0000 7efc42ffd640 1 -- 172.21.15.39:0/891109496 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efc440ec880 con 0x7efc4c06ae20 2024-09-13T16:52:12.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc4c072560 con 0x7efc4c06ae20 2024-09-13T16:52:12.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc42ffd640 1 -- 172.21.15.39:0/891109496 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7efc440efea0 con 0x7efc4c06ae20 2024-09-13T16:52:12.356 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:12.356 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:12.356 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:52:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc300666c0 msgr2=0x7efc30068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc300666c0 0x7efc30068b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efc4c1edc70 tx=0x7efc48008040 comp rx=0 tx=0).stop 2024-09-13T16:52:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c06ae20 msgr2=0x7efc4c0c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c06ae20 0x7efc4c0c2840 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efc4401afe0 tx=0x7efc440196d0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 shutdown_connections 2024-09-13T16:52:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc300666c0 0x7efc30068b80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc4c0c6060 0x7efc4c149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc4c14a770 0x7efc4c149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 --2- 172.21.15.39:0/891109496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc4c06ae20 0x7efc4c0c2840 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:12.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 >> 172.21.15.39:0/891109496 conn(0x7efc4c0c32a0 msgr2=0x7efc4c15bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:12.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 shutdown_connections 2024-09-13T16:52:12.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:12.352+0000 7efc5466a640 1 -- 172.21.15.39:0/891109496 wait complete. 2024-09-13T16:52:12.375 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:52:14.376 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:14.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.504+0000 7f631c676640 1 Processor -- start 2024-09-13T16:52:14.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.504+0000 7f631c676640 1 -- start start 2024-09-13T16:52:14.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631c676640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6314151500 0x7f63140c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:14.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631c676640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6314071dd0 0x7f6314154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631c676640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63140c47c0 0x7f63141545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631c676640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63140c5a30 con 0x7f63140c47c0 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631c676640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63141576a0 con 0x7f6314151500 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631c676640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6314157980 con 0x7f6314071dd0 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631ac13640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63140c47c0 0x7f63141545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631ac13640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63140c47c0 0x7f63141545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52486/0 (socket says 172.21.15.39:52486) 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631ac13640 1 -- 172.21.15.39:0/3918205344 learned_addr learned my addr 172.21.15.39:0/3918205344 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f6319c11640 1 --2- 172.21.15.39:0/3918205344 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6314071dd0 0x7f6314154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631a412640 1 --2- 172.21.15.39:0/3918205344 >> v2:172.21.15.84:3300/0 conn(0x7f6314151500 0x7f63140c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631ac13640 1 -- 172.21.15.39:0/3918205344 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6314071dd0 msgr2=0x7f6314154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631ac13640 1 --2- 172.21.15.39:0/3918205344 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6314071dd0 0x7f6314154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631ac13640 1 -- 172.21.15.39:0/3918205344 >> v2:172.21.15.84:3300/0 conn(0x7f6314151500 msgr2=0x7f63140c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:14.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631ac13640 1 --2- 172.21.15.39:0/3918205344 >> v2:172.21.15.84:3300/0 conn(0x7f6314151500 0x7f63140c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631ac13640 1 -- 172.21.15.39:0/3918205344 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6314157c60 con 0x7f63140c47c0 2024-09-13T16:52:14.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.508+0000 7f631a412640 1 --2- 172.21.15.39:0/3918205344 >> v2:172.21.15.84:3300/0 conn(0x7f6314151500 0x7f63140c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:14.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631ac13640 1 --2- 172.21.15.39:0/3918205344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63140c47c0 0x7f63141545b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f63140a9470 tx=0x7f630c01d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae6b30b1450287d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:14.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f6319410640 1 -- 172.21.15.39:0/3918205344 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c0a57d0 con 0x7f63140c47c0 2024-09-13T16:52:14.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f6319410640 1 -- 172.21.15.39:0/3918205344 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630c0a5930 con 0x7f63140c47c0 2024-09-13T16:52:14.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f6319410640 1 -- 172.21.15.39:0/3918205344 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c0a5cc0 con 0x7f63140c47c0 2024-09-13T16:52:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631c676640 1 -- 172.21.15.39:0/3918205344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63140c47c0 msgr2=0x7f63141545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631c676640 1 --2- 172.21.15.39:0/3918205344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63140c47c0 0x7f63141545b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f63140a9470 tx=0x7f630c01d390 comp rx=0 tx=0).stop 2024-09-13T16:52:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631c676640 1 -- 172.21.15.39:0/3918205344 shutdown_connections 2024-09-13T16:52:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631c676640 1 --2- 172.21.15.39:0/3918205344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63140c47c0 0x7f63141545b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631c676640 1 --2- 172.21.15.39:0/3918205344 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6314071dd0 0x7f6314154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631c676640 1 --2- 172.21.15.39:0/3918205344 >> v2:172.21.15.84:3300/0 conn(0x7f6314151500 0x7f63140c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631c676640 1 -- 172.21.15.39:0/3918205344 >> 172.21.15.39:0/3918205344 conn(0x7f63140ab530 msgr2=0x7f63140ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631c676640 1 -- 172.21.15.39:0/3918205344 shutdown_connections 2024-09-13T16:52:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.512+0000 7f631c676640 1 -- 172.21.15.39:0/3918205344 wait complete. 2024-09-13T16:52:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 Processor -- start 2024-09-13T16:52:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 -- start start 2024-09-13T16:52:14.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6314071dd0 0x7f631414a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:14.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63140c47c0 0x7f6314147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:14.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6314151500 0x7f6314147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631414b7e0 con 0x7f6314071dd0 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6314151fb0 con 0x7f6314151500 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63141ed190 con 0x7f63140c47c0 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631a412640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6314071dd0 0x7f631414a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631a412640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6314071dd0 0x7f631414a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52494/0 (socket says 172.21.15.39:52494) 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631a412640 1 -- 172.21.15.39:0/3867503144 learned_addr learned my addr 172.21.15.39:0/3867503144 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f6319c11640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63140c47c0 0x7f6314147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631ac13640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6314151500 0x7f6314147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631a412640 1 -- 172.21.15.39:0/3867503144 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63140c47c0 msgr2=0x7f6314147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631a412640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63140c47c0 0x7f6314147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631a412640 1 -- 172.21.15.39:0/3867503144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6314151500 msgr2=0x7f6314147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631a412640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6314151500 0x7f6314147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631a412640 1 -- 172.21.15.39:0/3867503144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c009b30 con 0x7f6314071dd0 2024-09-13T16:52:14.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631a412640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6314071dd0 0x7f631414a9d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f631000d800 tx=0x7f631000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f630affd640 1 -- 172.21.15.39:0/3867503144 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63100096c0 con 0x7f6314071dd0 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f630affd640 1 -- 172.21.15.39:0/3867503144 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6310009820 con 0x7f6314071dd0 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6314152250 con 0x7f6314071dd0 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f630affd640 1 -- 172.21.15.39:0/3867503144 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310009bd0 con 0x7f6314071dd0 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631414af10 con 0x7f6314071dd0 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.516+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63140711d0 con 0x7f6314071dd0 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.520+0000 7f630affd640 1 -- 172.21.15.39:0/3867503144 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6310009d30 con 0x7f6314071dd0 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.520+0000 7f630affd640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62f8066690 0x7f62f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.520+0000 7f630affd640 1 -- 172.21.15.39:0/3867503144 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f6310084d60 con 0x7f6314071dd0 2024-09-13T16:52:14.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.520+0000 7f6319c11640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62f8066690 0x7f62f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.520+0000 7f6319c11640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62f8066690 0x7f62f8068b50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f63140c4360 tx=0x7f6304008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:14.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.524+0000 7f630affd640 1 -- 172.21.15.39:0/3867503144 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6310051c60 con 0x7f6314071dd0 2024-09-13T16:52:14.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.744+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6314071bb0 con 0x7f6314071dd0 2024-09-13T16:52:14.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.744+0000 7f630affd640 1 -- 172.21.15.39:0/3867503144 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f6310055280 con 0x7f6314071dd0 2024-09-13T16:52:14.753 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:14.753 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:14.754 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:52:14.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62f8066690 msgr2=0x7f62f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:14.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62f8066690 0x7f62f8068b50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f63140c4360 tx=0x7f6304008040 comp rx=0 tx=0).stop 2024-09-13T16:52:14.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6314071dd0 msgr2=0x7f631414a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:14.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6314071dd0 0x7f631414a9d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f631000d800 tx=0x7f631000dcc0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 shutdown_connections 2024-09-13T16:52:14.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f62f8066690 0x7f62f8068b50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6314151500 0x7f6314147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63140c47c0 0x7f6314147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 --2- 172.21.15.39:0/3867503144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6314071dd0 0x7f631414a9d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:14.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 >> 172.21.15.39:0/3867503144 conn(0x7f63140ab530 msgr2=0x7f631414f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:14.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 shutdown_connections 2024-09-13T16:52:14.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:14.748+0000 7f631c676640 1 -- 172.21.15.39:0/3867503144 wait complete. 2024-09-13T16:52:14.769 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:52:16.770 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f3851f7c640 1 Processor -- start 2024-09-13T16:52:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f3851f7c640 1 -- start start 2024-09-13T16:52:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f3851f7c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f384c065ee0 0x7f384c0662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f3851f7c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c2ee0 0x7f384c0c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f3851f7c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384c06ddf0 con 0x7f384c0c6270 2024-09-13T16:52:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f3851f7c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c0c4710 con 0x7f384c065ee0 2024-09-13T16:52:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f3851f7c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384c0c49d0 con 0x7f384c0c2ee0 2024-09-13T16:52:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f384bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f384c0c6270 0x7f384c0bfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53304/0 (socket says 172.21.15.39:53304) 2024-09-13T16:52:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f384bfff640 1 -- 172.21.15.39:0/3545978415 learned_addr learned my addr 172.21.15.39:0/3545978415 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f384affd640 1 --2- 172.21.15.39:0/3545978415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c2ee0 0x7f384c0c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f384b7fe640 1 --2- 172.21.15.39:0/3545978415 >> v2:172.21.15.84:3300/0 conn(0x7f384c065ee0 0x7f384c0662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.904+0000 7f384a7fc640 1 -- 172.21.15.39:0/3545978415 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304834290 0 0) 0x7f384c06ddf0 con 0x7f384c0c6270 2024-09-13T16:52:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384a7fc640 1 -- 172.21.15.39:0/3545978415 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3834003530 con 0x7f384c0c6270 2024-09-13T16:52:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384affd640 1 -- 172.21.15.39:0/3545978415 >> v2:172.21.15.84:3300/0 conn(0x7f384c065ee0 msgr2=0x7f384c0662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384affd640 1 --2- 172.21.15.39:0/3545978415 >> v2:172.21.15.84:3300/0 conn(0x7f384c065ee0 0x7f384c0662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384affd640 1 -- 172.21.15.39:0/3545978415 >> v1:172.21.15.39:6789/0 conn(0x7f384c0c6270 legacy=0x7f384c0bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:16.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384a7fc640 1 -- 172.21.15.39:0/3545978415 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3659027842 0 0) 0x7f3834003530 con 0x7f384c0c6270 2024-09-13T16:52:16.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384affd640 1 -- 172.21.15.39:0/3545978415 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c154880 con 0x7f384c0c2ee0 2024-09-13T16:52:16.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384affd640 1 --2- 172.21.15.39:0/3545978415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c2ee0 0x7f384c0c2220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f38400097c0 tx=0x7f384001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c17fded7de0207c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:16.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384a7fc640 1 -- 172.21.15.39:0/3545978415 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844001a00 con 0x7f384c0c2ee0 2024-09-13T16:52:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384a7fc640 1 -- 172.21.15.39:0/3545978415 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38400a7050 con 0x7f384c0c2ee0 2024-09-13T16:52:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.908+0000 7f384a7fc640 1 -- 172.21.15.39:0/3545978415 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38400ab600 con 0x7f384c0c2ee0 2024-09-13T16:52:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- 172.21.15.39:0/3545978415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c2ee0 msgr2=0x7f384c0c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 --2- 172.21.15.39:0/3545978415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c2ee0 0x7f384c0c2220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f38400097c0 tx=0x7f384001c330 comp rx=0 tx=0).stop 2024-09-13T16:52:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- 172.21.15.39:0/3545978415 shutdown_connections 2024-09-13T16:52:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 --2- 172.21.15.39:0/3545978415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c2ee0 0x7f384c0c2220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 --2- 172.21.15.39:0/3545978415 >> v2:172.21.15.84:3300/0 conn(0x7f384c065ee0 0x7f384c0662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- 172.21.15.39:0/3545978415 >> 172.21.15.39:0/3545978415 conn(0x7f384c0c3310 msgr2=0x7f384c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- 172.21.15.39:0/3545978415 shutdown_connections 2024-09-13T16:52:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- 172.21.15.39:0/3545978415 wait complete. 2024-09-13T16:52:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 Processor -- start 2024-09-13T16:52:16.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- start start 2024-09-13T16:52:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c065ee0 0x7f384c157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f384c0c2ee0 0x7f384c0ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c6270 0x7f384c1ecd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c155f80 con 0x7f384c065ee0 2024-09-13T16:52:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c14ffc0 con 0x7f384c0c2ee0 2024-09-13T16:52:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384c0a7450 con 0x7f384c0c6270 2024-09-13T16:52:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c065ee0 0x7f384c157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c065ee0 0x7f384c157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52502/0 (socket says 172.21.15.39:52502) 2024-09-13T16:52:16.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384b7fe640 1 -- 172.21.15.39:0/1016771210 learned_addr learned my addr 172.21.15.39:0/1016771210 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:16.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384affd640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f384c0c2ee0 0x7f384c0ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384bfff640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c6270 0x7f384c1ecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384b7fe640 1 -- 172.21.15.39:0/1016771210 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c6270 msgr2=0x7f384c1ecd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:16.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384b7fe640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c6270 0x7f384c1ecd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384b7fe640 1 -- 172.21.15.39:0/1016771210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f384c0c2ee0 msgr2=0x7f384c0ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:16.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384b7fe640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f384c0c2ee0 0x7f384c0ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:16.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384b7fe640 1 -- 172.21.15.39:0/1016771210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3840009420 con 0x7f384c065ee0 2024-09-13T16:52:16.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f384b7fe640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c065ee0 0x7f384c157060 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f383c00e930 tx=0x7f383c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3833fff640 1 -- 172.21.15.39:0/1016771210 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c009690 con 0x7f384c065ee0 2024-09-13T16:52:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3833fff640 1 -- 172.21.15.39:0/1016771210 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c00ca00 con 0x7f384c065ee0 2024-09-13T16:52:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384c1ec570 con 0x7f384c065ee0 2024-09-13T16:52:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3833fff640 1 -- 172.21.15.39:0/1016771210 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c00cd70 con 0x7f384c065ee0 2024-09-13T16:52:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.912+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c1504f0 con 0x7f384c065ee0 2024-09-13T16:52:16.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.916+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384c0662c0 con 0x7f384c065ee0 2024-09-13T16:52:16.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.916+0000 7f3833fff640 1 -- 172.21.15.39:0/1016771210 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f383c013470 con 0x7f384c065ee0 2024-09-13T16:52:16.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.916+0000 7f3833fff640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3838066690 0x7f3838068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:16.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.916+0000 7f3833fff640 1 -- 172.21.15.39:0/1016771210 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f383c0848b0 con 0x7f384c065ee0 2024-09-13T16:52:16.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.916+0000 7f384affd640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3838066690 0x7f3838068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:16.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.916+0000 7f384affd640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3838066690 0x7f3838068b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38400ad000 tx=0x7f38400a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:16.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:16.920+0000 7f3833fff640 1 -- 172.21.15.39:0/1016771210 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f383c051870 con 0x7f384c065ee0 2024-09-13T16:52:17.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f384c06ec90 con 0x7f384c065ee0 2024-09-13T16:52:17.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3833fff640 1 -- 172.21.15.39:0/1016771210 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f383c054e90 con 0x7f384c065ee0 2024-09-13T16:52:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:17.191 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:17.192 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3838066690 msgr2=0x7f3838068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3838066690 0x7f3838068b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f38400ad000 tx=0x7f38400a6000 comp rx=0 tx=0).stop 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c065ee0 msgr2=0x7f384c157060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c065ee0 0x7f384c157060 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f383c00e930 tx=0x7f383c00edf0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 shutdown_connections 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3838066690 0x7f3838068b50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f384c0c2ee0 0x7f384c0ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f384c0c6270 0x7f384c1ecd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 --2- 172.21.15.39:0/1016771210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384c065ee0 0x7f384c157060 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 >> 172.21.15.39:0/1016771210 conn(0x7f384c0c3310 msgr2=0x7f384c06a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:17.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 shutdown_connections 2024-09-13T16:52:17.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:17.188+0000 7f3851f7c640 1 -- 172.21.15.39:0/1016771210 wait complete. 2024-09-13T16:52:17.214 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:52:19.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:19.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.352+0000 7f0a624ad640 1 Processor -- start 2024-09-13T16:52:19.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.352+0000 7f0a624ad640 1 -- start start 2024-09-13T16:52:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a624ad640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0a5c14e4a0 0x7f0a5c06e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a624ad640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0a5c0c4d60 0x7f0a5c06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a624ad640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5c1517e0 con 0x7f0a5c06b360 2024-09-13T16:52:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a624ad640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c150bd0 con 0x7f0a5c14e4a0 2024-09-13T16:52:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a624ad640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c150eb0 con 0x7f0a5c0c4d60 2024-09-13T16:52:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a5bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0a5c14e4a0 0x7f0a5c06e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:19.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a5bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0a5c14e4a0 0x7f0a5c06e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52516/0 (socket says 172.21.15.39:52516) 2024-09-13T16:52:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a5bfff640 1 -- 172.21.15.39:0/3860170496 learned_addr learned my addr 172.21.15.39:0/3860170496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a5b7fe640 1 --2- 172.21.15.39:0/3860170496 >> v2:172.21.15.84:3300/0 conn(0x7f0a5c0c4d60 0x7f0a5c06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a5bfff640 1 -- 172.21.15.39:0/3860170496 >> v2:172.21.15.84:3300/0 conn(0x7f0a5c0c4d60 msgr2=0x7f0a5c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a5bfff640 1 --2- 172.21.15.39:0/3860170496 >> v2:172.21.15.84:3300/0 conn(0x7f0a5c0c4d60 0x7f0a5c06ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a5bfff640 1 -- 172.21.15.39:0/3860170496 >> v1:172.21.15.84:6789/0 conn(0x7f0a5c06b360 legacy=0x7f0a5c06c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:19.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a5bfff640 1 -- 172.21.15.39:0/3860170496 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c151190 con 0x7f0a5c14e4a0 2024-09-13T16:52:19.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.356+0000 7f0a5bfff640 1 --2- 172.21.15.39:0/3860170496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c14e4a0 0x7f0a5c06e030 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0c2940 tx=0x7f0a4c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febea1202f920c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:19.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5affd640 1 -- 172.21.15.39:0/3860170496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0ab600 con 0x7f0a5c14e4a0 2024-09-13T16:52:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5affd640 1 -- 172.21.15.39:0/3860170496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c0a7050 con 0x7f0a5c14e4a0 2024-09-13T16:52:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5affd640 1 -- 172.21.15.39:0/3860170496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0ab980 con 0x7f0a5c14e4a0 2024-09-13T16:52:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 -- 172.21.15.39:0/3860170496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c14e4a0 msgr2=0x7f0a5c06e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 --2- 172.21.15.39:0/3860170496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c14e4a0 0x7f0a5c06e030 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0c2940 tx=0x7f0a4c01c450 comp rx=0 tx=0).stop 2024-09-13T16:52:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 -- 172.21.15.39:0/3860170496 shutdown_connections 2024-09-13T16:52:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 --2- 172.21.15.39:0/3860170496 >> v2:172.21.15.84:3300/0 conn(0x7f0a5c0c4d60 0x7f0a5c06ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 --2- 172.21.15.39:0/3860170496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c14e4a0 0x7f0a5c06e030 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 -- 172.21.15.39:0/3860170496 >> 172.21.15.39:0/3860170496 conn(0x7f0a5c0c32a0 msgr2=0x7f0a5c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 -- 172.21.15.39:0/3860170496 shutdown_connections 2024-09-13T16:52:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 -- 172.21.15.39:0/3860170496 wait complete. 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 Processor -- start 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 -- start start 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c06b360 0x7f0a5c143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5c0c4d60 0x7f0a5c1414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c14e4a0 0x7f0a5c1419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c14ec80 con 0x7f0a5c06b360 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c141f30 con 0x7f0a5c14e4a0 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a624ad640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c14e930 con 0x7f0a5c0c4d60 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c06b360 0x7f0a5c143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c06b360 0x7f0a5c143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52522/0 (socket says 172.21.15.39:52522) 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5bfff640 1 -- 172.21.15.39:0/2198510842 learned_addr learned my addr 172.21.15.39:0/2198510842 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:19.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a60a4a640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c14e4a0 0x7f0a5c1419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5b7fe640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5c0c4d60 0x7f0a5c1414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5bfff640 1 -- 172.21.15.39:0/2198510842 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5c0c4d60 msgr2=0x7f0a5c1414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5bfff640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5c0c4d60 0x7f0a5c1414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5bfff640 1 -- 172.21.15.39:0/2198510842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c14e4a0 msgr2=0x7f0a5c1419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5bfff640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c14e4a0 0x7f0a5c1419f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5bfff640 1 -- 172.21.15.39:0/2198510842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c0094d0 con 0x7f0a5c06b360 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5b7fe640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5c0c4d60 0x7f0a5c1414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.360+0000 7f0a5bfff640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c06b360 0x7f0a5c143640 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0ae040 tx=0x7f0a4c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a58ff9640 1 -- 172.21.15.39:0/2198510842 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0b6470 con 0x7f0a5c06b360 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a58ff9640 1 -- 172.21.15.39:0/2198510842 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c0b65d0 con 0x7f0a5c06b360 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c1ee450 con 0x7f0a5c06b360 2024-09-13T16:52:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a58ff9640 1 -- 172.21.15.39:0/2198510842 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0b6980 con 0x7f0a5c06b360 2024-09-13T16:52:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c150890 con 0x7f0a5c06b360 2024-09-13T16:52:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c14efa0 con 0x7f0a5c06b360 2024-09-13T16:52:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a58ff9640 1 -- 172.21.15.39:0/2198510842 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a4c0b2070 con 0x7f0a5c06b360 2024-09-13T16:52:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a58ff9640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a48066690 0x7f0a48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a58ff9640 1 -- 172.21.15.39:0/2198510842 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7f0a4c11ec00 con 0x7f0a5c06b360 2024-09-13T16:52:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a5b7fe640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a48066690 0x7f0a48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.364+0000 7f0a5b7fe640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a48066690 0x7f0a48068b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0a76f0 tx=0x7f0a50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.368+0000 7f0a58ff9640 1 -- 172.21.15.39:0/2198510842 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0a4c125080 con 0x7f0a5c06b360 2024-09-13T16:52:19.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.628+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a5c06ae20 con 0x7f0a5c06b360 2024-09-13T16:52:19.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.628+0000 7f0a58ff9640 1 -- 172.21.15.39:0/2198510842 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f0a4c0b0020 con 0x7f0a5c06b360 2024-09-13T16:52:19.633 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:19.633 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:19.633 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:52:19.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a48066690 msgr2=0x7f0a48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:19.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a48066690 0x7f0a48068b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0a76f0 tx=0x7f0a50008040 comp rx=0 tx=0).stop 2024-09-13T16:52:19.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c06b360 msgr2=0x7f0a5c143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:19.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c06b360 0x7f0a5c143640 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0ae040 tx=0x7f0a4c0191f0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 shutdown_connections 2024-09-13T16:52:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a48066690 0x7f0a48068b50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a5c14e4a0 0x7f0a5c1419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5c0c4d60 0x7f0a5c1414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 --2- 172.21.15.39:0/2198510842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5c06b360 0x7f0a5c143640 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 >> 172.21.15.39:0/2198510842 conn(0x7f0a5c0c32a0 msgr2=0x7f0a5c06c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 shutdown_connections 2024-09-13T16:52:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:19.632+0000 7f0a624ad640 1 -- 172.21.15.39:0/2198510842 wait complete. 2024-09-13T16:52:19.653 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:52:21.654 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:21.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.775+0000 7fa6d0d9f640 1 Processor -- start 2024-09-13T16:52:21.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.775+0000 7fa6d0d9f640 1 -- start start 2024-09-13T16:52:21.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6d0d9f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa6cc0c2db0 0x7fa6cc061200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:21.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6d0d9f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6cc0c2560 0x7fa6cc14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:21.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6d0d9f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6cc06fb40 con 0x7fa6cc14e630 2024-09-13T16:52:21.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6d0d9f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6cc071730 con 0x7fa6cc0c2db0 2024-09-13T16:52:21.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6d0d9f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6cc150da0 con 0x7fa6cc0c2560 2024-09-13T16:52:21.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6ca59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa6cc0c2db0 0x7fa6cc061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:21.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6ca59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa6cc0c2db0 0x7fa6cc061200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42924/0 (socket says 172.21.15.39:42924) 2024-09-13T16:52:21.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6ca59c640 1 -- 172.21.15.39:0/2507436100 learned_addr learned my addr 172.21.15.39:0/2507436100 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:21.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6c9d9b640 1 --2- 172.21.15.39:0/2507436100 >> v2:172.21.15.84:3300/0 conn(0x7fa6cc0c2560 0x7fa6cc14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:21.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6ca59c640 1 -- 172.21.15.39:0/2507436100 >> v2:172.21.15.84:3300/0 conn(0x7fa6cc0c2560 msgr2=0x7fa6cc14e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:21.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6ca59c640 1 --2- 172.21.15.39:0/2507436100 >> v2:172.21.15.84:3300/0 conn(0x7fa6cc0c2560 0x7fa6cc14e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:21.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6ca59c640 1 -- 172.21.15.39:0/2507436100 >> v1:172.21.15.84:6789/0 conn(0x7fa6cc14e630 legacy=0x7fa6cc1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:21.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6ca59c640 1 -- 172.21.15.39:0/2507436100 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc14ea60 con 0x7fa6cc0c2db0 2024-09-13T16:52:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6ca59c640 1 --2- 172.21.15.39:0/2507436100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc0c2db0 0x7fa6cc061200 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0a9220 tx=0x7fa6bc01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e83c942a21ec0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6c959a640 1 -- 172.21.15.39:0/2507436100 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0a7980 con 0x7fa6cc0c2db0 2024-09-13T16:52:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6c959a640 1 -- 172.21.15.39:0/2507436100 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc0a7ae0 con 0x7fa6cc0c2db0 2024-09-13T16:52:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.779+0000 7fa6c959a640 1 -- 172.21.15.39:0/2507436100 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0a7e70 con 0x7fa6cc0c2db0 2024-09-13T16:52:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/2507436100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc0c2db0 msgr2=0x7fa6cc061200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 --2- 172.21.15.39:0/2507436100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc0c2db0 0x7fa6cc061200 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0a9220 tx=0x7fa6bc01ebb0 comp rx=0 tx=0).stop 2024-09-13T16:52:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/2507436100 shutdown_connections 2024-09-13T16:52:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 --2- 172.21.15.39:0/2507436100 >> v2:172.21.15.84:3300/0 conn(0x7fa6cc0c2560 0x7fa6cc14e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 --2- 172.21.15.39:0/2507436100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc0c2db0 0x7fa6cc061200 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:21.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/2507436100 >> 172.21.15.39:0/2507436100 conn(0x7fa6cc069e10 msgr2=0x7fa6cc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:21.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/2507436100 shutdown_connections 2024-09-13T16:52:21.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/2507436100 wait complete. 2024-09-13T16:52:21.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 Processor -- start 2024-09-13T16:52:21.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 -- start start 2024-09-13T16:52:21.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc0c2560 0x7fa6cc0aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6cc0c2db0 0x7fa6cc1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.783+0000 7fa6d0d9f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc14e630 0x7fa6cc144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6d0d9f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc0aaaf0 con 0x7fa6cc14e630 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6d0d9f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc0ab830 con 0x7fa6cc0c2560 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6d0d9f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc0ab550 con 0x7fa6cc0c2db0 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6cad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc14e630 0x7fa6cc144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6cad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc14e630 0x7fa6cc144020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42928/0 (socket says 172.21.15.39:42928) 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6cad9d640 1 -- 172.21.15.39:0/1482462996 learned_addr learned my addr 172.21.15.39:0/1482462996 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6ca59c640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc0c2560 0x7fa6cc0aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:21.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6c9d9b640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6cc0c2db0 0x7fa6cc1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6cad9d640 1 -- 172.21.15.39:0/1482462996 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6cc0c2db0 msgr2=0x7fa6cc1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6cad9d640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6cc0c2db0 0x7fa6cc1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:21.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6cad9d640 1 -- 172.21.15.39:0/1482462996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc0c2560 msgr2=0x7fa6cc0aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6cad9d640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc0c2560 0x7fa6cc0aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6cad9d640 1 -- 172.21.15.39:0/1482462996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc00b770 con 0x7fa6cc14e630 2024-09-13T16:52:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6cad9d640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc14e630 0x7fa6cc144020 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400b970 tx=0x7fa6c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6baffd640 1 -- 172.21.15.39:0/1482462996 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400cb40 con 0x7fa6cc14e630 2024-09-13T16:52:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6baffd640 1 -- 172.21.15.39:0/1482462996 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c400cca0 con 0x7fa6cc14e630 2024-09-13T16:52:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6cc1ec990 con 0x7fa6cc14e630 2024-09-13T16:52:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6baffd640 1 -- 172.21.15.39:0/1482462996 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4014680 con 0x7fa6cc14e630 2024-09-13T16:52:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6cc1ecf50 con 0x7fa6cc14e630 2024-09-13T16:52:21.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.787+0000 7fa6baffd640 1 -- 172.21.15.39:0/1482462996 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa6c401d3b0 con 0x7fa6cc14e630 2024-09-13T16:52:21.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.791+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6cc1ec7f0 con 0x7fa6cc14e630 2024-09-13T16:52:21.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.791+0000 7fa6baffd640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa6ac066690 0x7fa6ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:21.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.791+0000 7fa6baffd640 1 -- 172.21.15.39:0/1482462996 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fa6c4084420 con 0x7fa6cc14e630 2024-09-13T16:52:21.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.791+0000 7fa6ca59c640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa6ac066690 0x7fa6ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:21.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.791+0000 7fa6ca59c640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa6ac066690 0x7fa6ac068b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0b1000 tx=0x7fa6bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:21.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:21.795+0000 7fa6baffd640 1 -- 172.21.15.39:0/1482462996 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa6cc1ec7f0 con 0x7fa6cc14e630 2024-09-13T16:52:22.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.011+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6cc1ec7f0 con 0x7fa6cc14e630 2024-09-13T16:52:22.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.011+0000 7fa6baffd640 1 -- 172.21.15.39:0/1482462996 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fa6cc1ec7f0 con 0x7fa6cc14e630 2024-09-13T16:52:22.018 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:22.018 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:22.018 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:52:22.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa6ac066690 msgr2=0x7fa6ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:22.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa6ac066690 0x7fa6ac068b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0b1000 tx=0x7fa6bc008040 comp rx=0 tx=0).stop 2024-09-13T16:52:22.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc14e630 msgr2=0x7fa6cc144020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:22.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc14e630 0x7fa6cc144020 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400b970 tx=0x7fa6c400be30 comp rx=0 tx=0).stop 2024-09-13T16:52:22.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 shutdown_connections 2024-09-13T16:52:22.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa6ac066690 0x7fa6ac068b50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:22.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6cc14e630 0x7fa6cc144020 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:22.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6cc0c2db0 0x7fa6cc1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:22.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 --2- 172.21.15.39:0/1482462996 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6cc0c2560 0x7fa6cc0aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:22.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 >> 172.21.15.39:0/1482462996 conn(0x7fa6cc069e10 msgr2=0x7fa6cc070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:22.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 shutdown_connections 2024-09-13T16:52:22.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:22.015+0000 7fa6d0d9f640 1 -- 172.21.15.39:0/1482462996 wait complete. 2024-09-13T16:52:22.042 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:52:24.043 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:24.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.179+0000 7fc6af397640 1 Processor -- start 2024-09-13T16:52:24.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.179+0000 7fc6af397640 1 -- start start 2024-09-13T16:52:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.179+0000 7fc6af397640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc6a814a770 0x7fc6a8150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.179+0000 7fc6af397640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc6a80c6060 0x7fc6a814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:24.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.179+0000 7fc6af397640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a80c4760 con 0x7fc6a806ae20 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.179+0000 7fc6af397640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a806df80 con 0x7fc6a814a770 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.179+0000 7fc6af397640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a80c1950 con 0x7fc6a80c6060 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6ad934640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc6a806ae20 0x7fc6a80713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:35360/0 (socket says 172.21.15.39:35360) 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6ad934640 1 -- 172.21.15.39:0/476910249 learned_addr learned my addr 172.21.15.39:0/476910249 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6ad133640 1 --2- 172.21.15.39:0/476910249 >> v2:172.21.15.39:3300/0 conn(0x7fc6a814a770 0x7fc6a8150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6ac932640 1 --2- 172.21.15.39:0/476910249 >> v2:172.21.15.84:3300/0 conn(0x7fc6a80c6060 0x7fc6a814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc69ffff640 1 -- 172.21.15.39:0/476910249 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481305562 0 0) 0x7fc6a80c4760 con 0x7fc6a806ae20 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc69ffff640 1 -- 172.21.15.39:0/476910249 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc694003530 con 0x7fc6a806ae20 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6ad133640 1 -- 172.21.15.39:0/476910249 >> v2:172.21.15.84:3300/0 conn(0x7fc6a80c6060 msgr2=0x7fc6a814ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6ad133640 1 --2- 172.21.15.39:0/476910249 >> v2:172.21.15.84:3300/0 conn(0x7fc6a80c6060 0x7fc6a814ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6ad133640 1 -- 172.21.15.39:0/476910249 >> v1:172.21.15.39:6789/0 conn(0x7fc6a806ae20 legacy=0x7fc6a80713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6ad133640 1 -- 172.21.15.39:0/476910249 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a8151120 con 0x7fc6a814a770 2024-09-13T16:52:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6ad133640 1 --2- 172.21.15.39:0/476910249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a814a770 0x7fc6a8150ad0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80a94e0 tx=0x7fc6a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9f50b3365bac06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc69ffff640 1 -- 172.21.15.39:0/476910249 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40ab600 con 0x7fc6a814a770 2024-09-13T16:52:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc69ffff640 1 -- 172.21.15.39:0/476910249 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a40a7050 con 0x7fc6a814a770 2024-09-13T16:52:24.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc69ffff640 1 -- 172.21.15.39:0/476910249 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40ab960 con 0x7fc6a814a770 2024-09-13T16:52:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6af397640 1 -- 172.21.15.39:0/476910249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a814a770 msgr2=0x7fc6a8150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6af397640 1 --2- 172.21.15.39:0/476910249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a814a770 0x7fc6a8150ad0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80a94e0 tx=0x7fc6a401c3c0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6af397640 1 -- 172.21.15.39:0/476910249 shutdown_connections 2024-09-13T16:52:24.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6af397640 1 --2- 172.21.15.39:0/476910249 >> v2:172.21.15.84:3300/0 conn(0x7fc6a80c6060 0x7fc6a814ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6af397640 1 --2- 172.21.15.39:0/476910249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a814a770 0x7fc6a8150ad0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6af397640 1 -- 172.21.15.39:0/476910249 >> 172.21.15.39:0/476910249 conn(0x7fc6a80c32a0 msgr2=0x7fc6a80c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6af397640 1 -- 172.21.15.39:0/476910249 shutdown_connections 2024-09-13T16:52:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.183+0000 7fc6af397640 1 -- 172.21.15.39:0/476910249 wait complete. 2024-09-13T16:52:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 Processor -- start 2024-09-13T16:52:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 -- start start 2024-09-13T16:52:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6a806ae20 0x7fc6a815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a80c6060 0x7fc6a81ecd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6a814a770 0x7fc6a81ed260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a80702d0 con 0x7fc6a80c6060 2024-09-13T16:52:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a8154f70 con 0x7fc6a814a770 2024-09-13T16:52:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a814a190 con 0x7fc6a806ae20 2024-09-13T16:52:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ac932640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a80c6060 0x7fc6a81ecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:24.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ac932640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a80c6060 0x7fc6a81ecd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42946/0 (socket says 172.21.15.39:42946) 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ac932640 1 -- 172.21.15.39:0/42259679 learned_addr learned my addr 172.21.15.39:0/42259679 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ad934640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6a814a770 0x7fc6a81ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ad133640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6a806ae20 0x7fc6a815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ac932640 1 -- 172.21.15.39:0/42259679 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6a806ae20 msgr2=0x7fc6a815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ac932640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6a806ae20 0x7fc6a815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ac932640 1 -- 172.21.15.39:0/42259679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6a814a770 msgr2=0x7fc6a81ed260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ac932640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6a814a770 0x7fc6a81ed260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ac932640 1 -- 172.21.15.39:0/42259679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4009420 con 0x7fc6a80c6060 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6ac932640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a80c6060 0x7fc6a81ecd20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e930 tx=0x7fc69800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc69dffb640 1 -- 172.21.15.39:0/42259679 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698009690 con 0x7fc6a80c6060 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc69dffb640 1 -- 172.21.15.39:0/42259679 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69800ca00 con 0x7fc6a80c6060 2024-09-13T16:52:24.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc69dffb640 1 -- 172.21.15.39:0/42259679 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800cd70 con 0x7fc6a80c6060 2024-09-13T16:52:24.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a80704e0 con 0x7fc6a80c6060 2024-09-13T16:52:24.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a8070aa0 con 0x7fc6a80c6060 2024-09-13T16:52:24.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.187+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a80619c0 con 0x7fc6a80c6060 2024-09-13T16:52:24.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.191+0000 7fc69dffb640 1 -- 172.21.15.39:0/42259679 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc698013470 con 0x7fc6a80c6060 2024-09-13T16:52:24.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.191+0000 7fc69dffb640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc68c066690 0x7fc68c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:24.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.191+0000 7fc69dffb640 1 -- 172.21.15.39:0/42259679 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6981+0+0 (secure 0 0 0) 0x7fc6980848b0 con 0x7fc6a80c6060 2024-09-13T16:52:24.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.191+0000 7fc6ad133640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc68c066690 0x7fc68c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:24.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.191+0000 7fc6ad133640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc68c066690 0x7fc68c068b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40ad000 tx=0x7fc6a40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:24.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.195+0000 7fc69dffb640 1 -- 172.21.15.39:0/42259679 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc698051870 con 0x7fc6a80c6060 2024-09-13T16:52:24.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.419+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6a80c11a0 con 0x7fc6a80c6060 2024-09-13T16:52:24.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.419+0000 7fc69dffb640 1 -- 172.21.15.39:0/42259679 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fc698054e90 con 0x7fc6a80c6060 2024-09-13T16:52:24.425 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:24.425 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5418,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:51:17.595973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4814":{"gid":4814,"name":"b","rank":0,"incarnation":31,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6833/2144264453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2144264453},{"type":"v1","addr":"172.21.15.84:6833","nonce":2144264453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:24.426 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-13T16:52:24.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc68c066690 msgr2=0x7fc68c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:24.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc68c066690 0x7fc68c068b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40ad000 tx=0x7fc6a40a6000 comp rx=0 tx=0).stop 2024-09-13T16:52:24.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a80c6060 msgr2=0x7fc6a81ecd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:24.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a80c6060 0x7fc6a81ecd20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e930 tx=0x7fc69800edf0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 shutdown_connections 2024-09-13T16:52:24.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc68c066690 0x7fc68c068b50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6a814a770 0x7fc6a81ed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6a80c6060 0x7fc6a81ecd20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 --2- 172.21.15.39:0/42259679 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6a806ae20 0x7fc6a815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:24.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 >> 172.21.15.39:0/42259679 conn(0x7fc6a80c32a0 msgr2=0x7fc6a814f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:24.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 shutdown_connections 2024-09-13T16:52:24.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:24.423+0000 7fc6af397640 1 -- 172.21.15.39:0/42259679 wait complete. 2024-09-13T16:52:24.444 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:52:26.261 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:52:26.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:52:26.445 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:26.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f168ea3b640 1 Processor -- start 2024-09-13T16:52:26.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f168ea3b640 1 -- start start 2024-09-13T16:52:26.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f168ea3b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16880c1ab0 0x7f16880c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f168ea3b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168814f920 con 0x7f1688155200 2024-09-13T16:52:26.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f168ea3b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16880c1780 con 0x7f168814f540 2024-09-13T16:52:26.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f168ea3b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16880709c0 con 0x7f16880c1ab0 2024-09-13T16:52:26.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f168cfd8640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1688155200 0x7f1688155690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:35366/0 (socket says 172.21.15.39:35366) 2024-09-13T16:52:26.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f168cfd8640 1 -- 172.21.15.39:0/2520355215 learned_addr learned my addr 172.21.15.39:0/2520355215 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:26.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f167ffff640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16880c1ab0 0x7f16880c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721298689 0 0) 0x7f168814f920 con 0x7f1688155200 2024-09-13T16:52:26.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1674003530 con 0x7f1688155200 2024-09-13T16:52:26.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238005336 0 0) 0x7f16880c1780 con 0x7f168814f540 2024-09-13T16:52:26.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f168814f920 con 0x7f168814f540 2024-09-13T16:52:26.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2843682486 0 0) 0x7f1674003530 con 0x7f1688155200 2024-09-13T16:52:26.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16880c1ab0 msgr2=0x7f16880c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:26.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16880c1ab0 0x7f16880c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 >> v1:172.21.15.84:6789/0 conn(0x7f168814f540 legacy=0x7f168806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688071580 con 0x7f1688155200 2024-09-13T16:52:26.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1680001a00 con 0x7f1688155200 2024-09-13T16:52:26.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 >> v1:172.21.15.39:6789/0 conn(0x7f1688155200 legacy=0x7f1688155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f1674003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16740047d0 0x7f1674004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16880c1ab0 0x7f1674005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167400ba40 con 0x7f1674003b40 2024-09-13T16:52:26.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167400bb80 con 0x7f16740047d0 2024-09-13T16:52:26.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167400bd40 con 0x7f16880c1ab0 2024-09-13T16:52:26.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f1687fff640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f1674003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f168cfd8640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16880c1ab0 0x7f1674005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f167ffff640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16740047d0 0x7f1674004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f1687fff640 1 -- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16880c1ab0 msgr2=0x7f1674005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:26.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f1687fff640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16880c1ab0 0x7f1674005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f1687fff640 1 -- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16740047d0 msgr2=0x7f1674004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:26.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f1687fff640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16740047d0 0x7f1674004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f1687fff640 1 -- 172.21.15.39:0/2520355215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1674003530 con 0x7f1674003b40 2024-09-13T16:52:26.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.579+0000 7f167ffff640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16740047d0 0x7f1674004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f1687fff640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f1674003f40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f16780018d0 tx=0x7f167801c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c32058a5f7fb59 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16780052d0 con 0x7f1674003b40 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1678005450 con 0x7f1674003b40 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f16877fe640 1 -- 172.21.15.39:0/2520355215 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678003840 con 0x7f1674003b40 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f168ea3b640 1 -- 172.21.15.39:0/2520355215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 msgr2=0x7f1674003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f168ea3b640 1 --2- 172.21.15.39:0/2520355215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f1674003f40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f16780018d0 tx=0x7f167801c550 comp rx=0 tx=0).stop 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f168cfd8640 1 -- 172.21.15.39:0/2520355215 reap_dead start 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f168ea3b640 1 -- 172.21.15.39:0/2520355215 shutdown_connections 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f168ea3b640 1 -- 172.21.15.39:0/2520355215 >> 172.21.15.39:0/2520355215 conn(0x7f16880c32a0 msgr2=0x7f16880c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f168ea3b640 1 -- 172.21.15.39:0/2520355215 shutdown_connections 2024-09-13T16:52:26.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.583+0000 7f168ea3b640 1 -- 172.21.15.39:0/2520355215 wait complete. 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 Processor -- start 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 -- start start 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f16880c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1688144230 0x7f168806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f168814b490 0x7f1688147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1688144750 con 0x7f1674003b40 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16881f1000 con 0x7f168814b490 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168806ee00 con 0x7f1688144230 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f1687fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f16880c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f1687fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f16880c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42954/0 (socket says 172.21.15.39:42954) 2024-09-13T16:52:26.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f1687fff640 1 -- 172.21.15.39:0/4170257361 learned_addr learned my addr 172.21.15.39:0/4170257361 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f167ffff640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1688144230 0x7f168806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168cfd8640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f168814b490 0x7f1688147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f1687fff640 1 -- 172.21.15.39:0/4170257361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1688144230 msgr2=0x7f168806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f1687fff640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1688144230 0x7f168806e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f1687fff640 1 -- 172.21.15.39:0/4170257361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f168814b490 msgr2=0x7f1688147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f1687fff640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f168814b490 0x7f1688147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f1687fff640 1 -- 172.21.15.39:0/4170257361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688142990 con 0x7f1674003b40 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f1687fff640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f16880c08d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1678005040 tx=0x7f1674005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f16857fa640 1 -- 172.21.15.39:0/4170257361 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1678005190 con 0x7f1674003b40 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f16857fa640 1 -- 172.21.15.39:0/4170257361 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1678003820 con 0x7f1674003b40 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16881f0c20 con 0x7f1674003b40 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f16857fa640 1 -- 172.21.15.39:0/4170257361 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16780052f0 con 0x7f1674003b40 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1688142c70 con 0x7f1674003b40 2024-09-13T16:52:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.587+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16880619c0 con 0x7f1674003b40 2024-09-13T16:52:26.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.591+0000 7f16857fa640 1 -- 172.21.15.39:0/4170257361 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1678003bb0 con 0x7f1674003b40 2024-09-13T16:52:26.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.591+0000 7f16857fa640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16680666c0 0x7f1668068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.591+0000 7f16857fa640 1 -- 172.21.15.39:0/4170257361 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f167811e9a0 con 0x7f1674003b40 2024-09-13T16:52:26.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.591+0000 7f167ffff640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16680666c0 0x7f1668068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.591+0000 7f167ffff640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16680666c0 0x7f1668068b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f16880a8520 tx=0x7f1670008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.595+0000 7f16857fa640 1 -- 172.21.15.39:0/4170257361 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f16780eb7c0 con 0x7f1674003b40 2024-09-13T16:52:26.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.835+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16880619c0 con 0x7f1674003b40 2024-09-13T16:52:26.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.835+0000 7f16857fa640 1 -- 172.21.15.39:0/4170257361 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f1678003e60 con 0x7f1674003b40 2024-09-13T16:52:26.841 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:26.842 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:26.801729+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:26.842 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 38 2024-09-13T16:52:26.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16680666c0 msgr2=0x7f1668068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16680666c0 0x7f1668068b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f16880a8520 tx=0x7f1670008040 comp rx=0 tx=0).stop 2024-09-13T16:52:26.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 msgr2=0x7f16880c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f16880c08d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1678005040 tx=0x7f1674005800 comp rx=0 tx=0).stop 2024-09-13T16:52:26.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 shutdown_connections 2024-09-13T16:52:26.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16680666c0 0x7f1668068b80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f168814b490 0x7f1688147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1688144230 0x7f168806e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 --2- 172.21.15.39:0/4170257361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1674003b40 0x7f16880c08d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 >> 172.21.15.39:0/4170257361 conn(0x7f16880c32a0 msgr2=0x7f16880c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:26.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 shutdown_connections 2024-09-13T16:52:26.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.839+0000 7f168ea3b640 1 -- 172.21.15.39:0/4170257361 wait complete. 2024-09-13T16:52:26.863 INFO:tasks.mds_thrash.fs.[b]:mds.b down, removed from mdsmap 2024-09-13T16:52:26.864 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T16:52:26.864 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:26.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.979+0000 7fcf20b32640 1 Processor -- start 2024-09-13T16:52:26.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.979+0000 7fcf20b32640 1 -- start start 2024-09-13T16:52:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.979+0000 7fcf20b32640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcf1c0a5f10 0x7fcf1c1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.979+0000 7fcf20b32640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1c0c17d0 con 0x7fcf1c06ab90 2024-09-13T16:52:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.979+0000 7fcf20b32640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1c066600 con 0x7fcf1c0bfd20 2024-09-13T16:52:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.979+0000 7fcf20b32640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1c0c1a20 con 0x7fcf1c0a5f10 2024-09-13T16:52:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.979+0000 7fcf1a59c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcf1c06ab90 0x7fcf1c151370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:35372/0 (socket says 172.21.15.39:35372) 2024-09-13T16:52:26.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.979+0000 7fcf1a59c640 1 -- 172.21.15.39:0/3782809103 learned_addr learned my addr 172.21.15.39:0/3782809103 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf19d9b640 1 --2- 172.21.15.39:0/3782809103 >> v2:172.21.15.84:3300/0 conn(0x7fcf1c0a5f10 0x7fcf1c1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919980736 0 0) 0x7fcf1c0c17d0 con 0x7fcf1c06ab90 2024-09-13T16:52:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf04003530 con 0x7fcf1c06ab90 2024-09-13T16:52:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2864589103 0 0) 0x7fcf04003530 con 0x7fcf1c06ab90 2024-09-13T16:52:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 >> v2:172.21.15.84:3300/0 conn(0x7fcf1c0a5f10 msgr2=0x7fcf1c1574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:26.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 --2- 172.21.15.39:0/3782809103 >> v2:172.21.15.84:3300/0 conn(0x7fcf1c0a5f10 0x7fcf1c1574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 >> v1:172.21.15.84:6789/0 conn(0x7fcf1c0bfd20 legacy=0x7fcf1c15e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c066850 con 0x7fcf1c06ab90 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fcf0c001680 con 0x7fcf1c06ab90 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 >> v1:172.21.15.39:6789/0 conn(0x7fcf1c06ab90 legacy=0x7fcf1c151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf04003980 0x7fcf04003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf04004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf04005190 0x7fcf0400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0c1a20 con 0x7fcf040047d0 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0400c130 con 0x7fcf04003980 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0400c2f0 con 0x7fcf04005190 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf19d9b640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf04004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1a59c640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf04003980 0x7fcf04003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1ad9d640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf04005190 0x7fcf0400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf19d9b640 1 -- 172.21.15.39:0/3782809103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf04005190 msgr2=0x7fcf0400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf19d9b640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf04005190 0x7fcf0400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf19d9b640 1 -- 172.21.15.39:0/3782809103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf04003980 msgr2=0x7fcf04003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf19d9b640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf04003980 0x7fcf04003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf19d9b640 1 -- 172.21.15.39:0/3782809103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04003530 con 0x7fcf040047d0 2024-09-13T16:52:26.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1ad9d640 1 -- 172.21.15.39:0/3782809103 reap_dead start 2024-09-13T16:52:26.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.983+0000 7fcf1a59c640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf04003980 0x7fcf04003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf19d9b640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf04004c50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009ae0 tx=0x7fcf1001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d42ea1bd324b1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf100a6450 con 0x7fcf040047d0 2024-09-13T16:52:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf100a65b0 con 0x7fcf040047d0 2024-09-13T16:52:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1959a640 1 -- 172.21.15.39:0/3782809103 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf100a6960 con 0x7fcf040047d0 2024-09-13T16:52:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 -- 172.21.15.39:0/3782809103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 msgr2=0x7fcf04004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf04004c50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009ae0 tx=0x7fcf1001ca70 comp rx=0 tx=0).stop 2024-09-13T16:52:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 -- 172.21.15.39:0/3782809103 shutdown_connections 2024-09-13T16:52:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 --2- 172.21.15.39:0/3782809103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf04004c50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 -- 172.21.15.39:0/3782809103 >> 172.21.15.39:0/3782809103 conn(0x7fcf1c069e10 msgr2=0x7fcf1c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 -- 172.21.15.39:0/3782809103 shutdown_connections 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 -- 172.21.15.39:0/3782809103 wait complete. 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 Processor -- start 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 -- start start 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf1c153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf1c0a7c30 0x7fcf1c141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf1c141e90 0x7fcf1c1f0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c142f00 con 0x7fcf040047d0 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0a6eb0 con 0x7fcf1c141e90 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf20b32640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c142860 con 0x7fcf1c0a7c30 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf1c153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf1c153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42972/0 (socket says 172.21.15.39:42972) 2024-09-13T16:52:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1a59c640 1 -- 172.21.15.39:0/1686595816 learned_addr learned my addr 172.21.15.39:0/1686595816 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf19d9b640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf1c0a7c30 0x7fcf1c141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1ad9d640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf1c141e90 0x7fcf1c1f0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1a59c640 1 -- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf1c0a7c30 msgr2=0x7fcf1c141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1a59c640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf1c0a7c30 0x7fcf1c141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1a59c640 1 -- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf1c141e90 msgr2=0x7fcf1c1f0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1a59c640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf1c141e90 0x7fcf1c1f0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1a59c640 1 -- 172.21.15.39:0/1686595816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c14eef0 con 0x7fcf040047d0 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf19d9b640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf1c0a7c30 0x7fcf1c141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1ad9d640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf1c141e90 0x7fcf1c1f0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.987+0000 7fcf1a59c640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf1c153630 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c000fc0 tx=0x7fcf0c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:26.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf0affd640 1 -- 172.21.15.39:0/1686595816 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00ee00 con 0x7fcf040047d0 2024-09-13T16:52:26.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf0affd640 1 -- 172.21.15.39:0/1686595816 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c00c840 con 0x7fcf040047d0 2024-09-13T16:52:26.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c0c64c0 con 0x7fcf040047d0 2024-09-13T16:52:26.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf0affd640 1 -- 172.21.15.39:0/1686595816 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00cbd0 con 0x7fcf040047d0 2024-09-13T16:52:26.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c06c800 con 0x7fcf040047d0 2024-09-13T16:52:26.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c1f0e40 con 0x7fcf040047d0 2024-09-13T16:52:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf0affd640 1 -- 172.21.15.39:0/1686595816 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf0c00cd30 con 0x7fcf040047d0 2024-09-13T16:52:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf0affd640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcefc066690 0x7fcefc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf0affd640 1 -- 172.21.15.39:0/1686595816 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7fcf0c084440 con 0x7fcf040047d0 2024-09-13T16:52:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf19d9b640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcefc066690 0x7fcefc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.991+0000 7fcf19d9b640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcefc066690 0x7fcefc068b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcf100ad000 tx=0x7fcf100a9040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:26.995+0000 7fcf0affd640 1 -- 172.21.15.39:0/1686595816 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf0c0512f0 con 0x7fcf040047d0 2024-09-13T16:52:27.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.207+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf1c157d80 con 0x7fcf040047d0 2024-09-13T16:52:27.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf0affd640 1 -- 172.21.15.39:0/1686595816 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5176 (secure 0 0 0) 0x7fcf0c054910 con 0x7fcf040047d0 2024-09-13T16:52:27.216 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:27.216 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:26.801729+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:27.218 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 38 2024-09-13T16:52:27.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcefc066690 msgr2=0x7fcefc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:27.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcefc066690 0x7fcefc068b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcf100ad000 tx=0x7fcf100a9040 comp rx=0 tx=0).stop 2024-09-13T16:52:27.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 msgr2=0x7fcf1c153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:27.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf1c153630 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c000fc0 tx=0x7fcf0c00ec10 comp rx=0 tx=0).stop 2024-09-13T16:52:27.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 shutdown_connections 2024-09-13T16:52:27.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcefc066690 0x7fcefc068b50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:27.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf1c141e90 0x7fcf1c1f0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:27.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf1c0a7c30 0x7fcf1c141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:27.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 --2- 172.21.15.39:0/1686595816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf040047d0 0x7fcf1c153630 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:27.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 >> 172.21.15.39:0/1686595816 conn(0x7fcf1c069e10 msgr2=0x7fcf1c0c0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:27.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 shutdown_connections 2024-09-13T16:52:27.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:27.211+0000 7fcf20b32640 1 -- 172.21.15.39:0/1686595816 wait complete. 2024-09-13T16:52:27.238 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 38, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 29, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5053": { "addr": "172.21.15.84:6835/1969975539", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 1969975539, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 1969975539, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5053, "incarnation": 26, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:50:52.769334+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5053 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 38, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5418": { "addr": "172.21.15.39:6835/2065196144", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 2065196144, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 2065196144, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5418, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 19 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:52:26.801729+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5418 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:52:29.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:29.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.391+0000 7f006db19640 1 Processor -- start 2024-09-13T16:52:29.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f006db19640 1 -- start start 2024-09-13T16:52:29.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f006db19640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680bf4f0 0x7f0068153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:29.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f006db19640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00680611c0 0x7f00680bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:29.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f006db19640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00680c1340 con 0x7f0068151860 2024-09-13T16:52:29.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f006db19640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00681578f0 con 0x7f00680611c0 2024-09-13T16:52:29.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f006db19640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00680c1590 con 0x7f00680bf4f0 2024-09-13T16:52:29.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f00677fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0068151860 0x7f0068157d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:35374/0 (socket says 172.21.15.39:35374) 2024-09-13T16:52:29.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f0067fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00680611c0 0x7f00680bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:29.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f00677fe640 1 -- 172.21.15.39:0/3530420095 learned_addr learned my addr 172.21.15.39:0/3530420095 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:29.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f0066ffd640 1 --2- 172.21.15.39:0/3530420095 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680bf4f0 0x7f0068153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:29.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f00667fc640 1 -- 172.21.15.39:0/3530420095 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531687582 0 0) 0x7f00680c1340 con 0x7f0068151860 2024-09-13T16:52:29.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f00667fc640 1 -- 172.21.15.39:0/3530420095 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0050003530 con 0x7f0068151860 2024-09-13T16:52:29.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f0067fff640 1 -- 172.21.15.39:0/3530420095 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680bf4f0 msgr2=0x7f0068153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:29.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f0067fff640 1 --2- 172.21.15.39:0/3530420095 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680bf4f0 0x7f0068153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f0067fff640 1 -- 172.21.15.39:0/3530420095 >> v1:172.21.15.39:6789/0 conn(0x7f0068151860 legacy=0x7f0068157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:29.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f0067fff640 1 -- 172.21.15.39:0/3530420095 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680c1870 con 0x7f00680611c0 2024-09-13T16:52:29.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.395+0000 7f0066ffd640 1 --2- 172.21.15.39:0/3530420095 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680bf4f0 0x7f0068153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:29.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f0067fff640 1 --2- 172.21.15.39:0/3530420095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00680611c0 0x7f00680bfad0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0060009d90 tx=0x7f006001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d2ef24a75b584 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:29.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f00667fc640 1 -- 172.21.15.39:0/3530420095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600a5970 con 0x7f00680611c0 2024-09-13T16:52:29.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f00667fc640 1 -- 172.21.15.39:0/3530420095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00600a5ad0 con 0x7f00680611c0 2024-09-13T16:52:29.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f00667fc640 1 -- 172.21.15.39:0/3530420095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600a5e60 con 0x7f00680611c0 2024-09-13T16:52:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 -- 172.21.15.39:0/3530420095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00680611c0 msgr2=0x7f00680bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 --2- 172.21.15.39:0/3530420095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00680611c0 0x7f00680bfad0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0060009d90 tx=0x7f006001c8b0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 -- 172.21.15.39:0/3530420095 shutdown_connections 2024-09-13T16:52:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 --2- 172.21.15.39:0/3530420095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00680611c0 0x7f00680bfad0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 --2- 172.21.15.39:0/3530420095 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680bf4f0 0x7f0068153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 -- 172.21.15.39:0/3530420095 >> 172.21.15.39:0/3530420095 conn(0x7f0068069e10 msgr2=0x7f006806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:29.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 -- 172.21.15.39:0/3530420095 shutdown_connections 2024-09-13T16:52:29.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 -- 172.21.15.39:0/3530420095 wait complete. 2024-09-13T16:52:29.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 Processor -- start 2024-09-13T16:52:29.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 -- start start 2024-09-13T16:52:29.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680611c0 0x7f00680a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:29.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00680bf4f0 0x7f00680a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:29.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0068151860 0x7f006814a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:29.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006814b1a0 con 0x7f0068151860 2024-09-13T16:52:29.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068153c30 con 0x7f00680bf4f0 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f006db19640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0068154120 con 0x7f00680611c0 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f0067fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0068151860 0x7f006814a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f0067fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0068151860 0x7f006814a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42986/0 (socket says 172.21.15.39:42986) 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.399+0000 7f0067fff640 1 -- 172.21.15.39:0/1691831222 learned_addr learned my addr 172.21.15.39:0/1691831222 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f00677fe640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680611c0 0x7f00680a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f0066ffd640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00680bf4f0 0x7f00680a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f0067fff640 1 -- 172.21.15.39:0/1691831222 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680611c0 msgr2=0x7f00680a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f0067fff640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680611c0 0x7f00680a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f0067fff640 1 -- 172.21.15.39:0/1691831222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00680bf4f0 msgr2=0x7f00680a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f0067fff640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00680bf4f0 0x7f00680a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f0067fff640 1 -- 172.21.15.39:0/1691831222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060009a70 con 0x7f0068151860 2024-09-13T16:52:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f0067fff640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0068151860 0x7f006814a1b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f00600b0040 tx=0x7f0060019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f004ffff640 1 -- 172.21.15.39:0/1691831222 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600add90 con 0x7f0068151860 2024-09-13T16:52:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f004ffff640 1 -- 172.21.15.39:0/1691831222 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0060009450 con 0x7f0068151860 2024-09-13T16:52:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00681521a0 con 0x7f0068151860 2024-09-13T16:52:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f004ffff640 1 -- 172.21.15.39:0/1691831222 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600b9690 con 0x7f0068151860 2024-09-13T16:52:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0068153e40 con 0x7f0068151860 2024-09-13T16:52:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0068152330 con 0x7f0068151860 2024-09-13T16:52:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f004ffff640 1 -- 172.21.15.39:0/1691831222 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00600b97f0 con 0x7f0068151860 2024-09-13T16:52:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f004ffff640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0054066690 0x7f0054068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f004ffff640 1 -- 172.21.15.39:0/1691831222 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f006011fd90 con 0x7f0068151860 2024-09-13T16:52:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f00677fe640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0054066690 0x7f0054068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.403+0000 7f00677fe640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0054066690 0x7f0054068b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f00580011f0 tx=0x7f0058009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:29.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.407+0000 7f004ffff640 1 -- 172.21.15.39:0/1691831222 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f00600ecd60 con 0x7f0068151860 2024-09-13T16:52:29.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.651+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00680bfad0 con 0x7f0068151860 2024-09-13T16:52:29.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.651+0000 7f004ffff640 1 -- 172.21.15.39:0/1691831222 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f00600f0380 con 0x7f0068151860 2024-09-13T16:52:29.658 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:29.658 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 39 2024-09-13T16:52:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0054066690 msgr2=0x7f0054068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0054066690 0x7f0054068b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f00580011f0 tx=0x7f0058009040 comp rx=0 tx=0).stop 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0068151860 msgr2=0x7f006814a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0068151860 0x7f006814a1b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f00600b0040 tx=0x7f0060019650 comp rx=0 tx=0).stop 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 shutdown_connections 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0054066690 0x7f0054068b50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00680bf4f0 0x7f00680a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0068151860 0x7f006814a1b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 --2- 172.21.15.39:0/1691831222 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00680611c0 0x7f00680a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 >> 172.21.15.39:0/1691831222 conn(0x7f0068069e10 msgr2=0x7f00680c2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 shutdown_connections 2024-09-13T16:52:29.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:29.655+0000 7f006db19640 1 -- 172.21.15.39:0/1691831222 wait complete. 2024-09-13T16:52:29.679 INFO:tasks.mds_thrash.fs.[b]:mds.a has gained rank=0, replacing gid=4814 2024-09-13T16:52:29.679 INFO:tasks.mds_thrash.fs.[b]:waiting for 3 secs before reviving mds.b 2024-09-13T16:52:30.636 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T16:52:30.631+0000 7ff66aab4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:52:32.680 INFO:tasks.mds_thrash.fs.[b]:reviving mds.b 2024-09-13T16:52:32.680 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:52:32.680 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:52:32.683 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:52:32.684 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:32.764 INFO:tasks.ceph.mds.b.smithi084.stdout:starting mds.b at 2024-09-13T16:52:32.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.827+0000 7efd4e79c640 1 Processor -- start 2024-09-13T16:52:32.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.827+0000 7efd4e79c640 1 -- start start 2024-09-13T16:52:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd4e79c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efd480c4710 0x7efd480c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd4e79c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efd480c5fd0 0x7efd4815b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd4e79c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd480c76a0 con 0x7efd4814aa40 2024-09-13T16:52:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd4e79c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd48151b30 con 0x7efd480c4710 2024-09-13T16:52:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd4e79c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd48151570 con 0x7efd480c5fd0 2024-09-13T16:52:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd477fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efd480c4710 0x7efd480c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd477fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efd480c4710 0x7efd480c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45490/0 (socket says 172.21.15.39:45490) 2024-09-13T16:52:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd477fe640 1 -- 172.21.15.39:0/3172306079 learned_addr learned my addr 172.21.15.39:0/3172306079 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd4cd39640 1 --2- 172.21.15.39:0/3172306079 >> v2:172.21.15.84:3300/0 conn(0x7efd480c5fd0 0x7efd4815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd477fe640 1 -- 172.21.15.39:0/3172306079 >> v2:172.21.15.84:3300/0 conn(0x7efd480c5fd0 msgr2=0x7efd4815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd477fe640 1 --2- 172.21.15.39:0/3172306079 >> v2:172.21.15.84:3300/0 conn(0x7efd480c5fd0 0x7efd4815b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd477fe640 1 -- 172.21.15.39:0/3172306079 >> v1:172.21.15.39:6789/0 conn(0x7efd4814aa40 legacy=0x7efd480c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.831+0000 7efd477fe640 1 -- 172.21.15.39:0/3172306079 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48151850 con 0x7efd480c4710 2024-09-13T16:52:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd477fe640 1 --2- 172.21.15.39:0/3172306079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 0x7efd480c5a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efd480c24a0 tx=0x7efd3c01e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ed1532ea63ac68 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd46ffd640 1 -- 172.21.15.39:0/3172306079 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0a7860 con 0x7efd480c4710 2024-09-13T16:52:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd46ffd640 1 -- 172.21.15.39:0/3172306079 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c0a79c0 con 0x7efd480c4710 2024-09-13T16:52:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd46ffd640 1 -- 172.21.15.39:0/3172306079 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0a7d50 con 0x7efd480c4710 2024-09-13T16:52:32.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 -- 172.21.15.39:0/3172306079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 msgr2=0x7efd480c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 --2- 172.21.15.39:0/3172306079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 0x7efd480c5a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efd480c24a0 tx=0x7efd3c01e890 comp rx=0 tx=0).stop 2024-09-13T16:52:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 -- 172.21.15.39:0/3172306079 shutdown_connections 2024-09-13T16:52:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 --2- 172.21.15.39:0/3172306079 >> v2:172.21.15.84:3300/0 conn(0x7efd480c5fd0 0x7efd4815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 --2- 172.21.15.39:0/3172306079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 0x7efd480c5a90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 -- 172.21.15.39:0/3172306079 >> 172.21.15.39:0/3172306079 conn(0x7efd480c3310 msgr2=0x7efd480c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 -- 172.21.15.39:0/3172306079 shutdown_connections 2024-09-13T16:52:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 -- 172.21.15.39:0/3172306079 wait complete. 2024-09-13T16:52:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 Processor -- start 2024-09-13T16:52:32.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 -- start start 2024-09-13T16:52:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 0x7efd48144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd480c5fd0 0x7efd4814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efd4814aa40 0x7efd4806a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd481441b0 con 0x7efd480c4710 2024-09-13T16:52:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 0x7efd48144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4814b9b0 con 0x7efd480c5fd0 2024-09-13T16:52:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 0x7efd48144920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45498/0 (socket says 172.21.15.39:45498) 2024-09-13T16:52:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4e79c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4814bc90 con 0x7efd4814aa40 2024-09-13T16:52:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd47fff640 1 -- 172.21.15.39:0/2985916239 learned_addr learned my addr 172.21.15.39:0/2985916239 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd477fe640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd480c5fd0 0x7efd4814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.835+0000 7efd4cd39640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efd4814aa40 0x7efd4806a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd47fff640 1 -- 172.21.15.39:0/2985916239 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efd4814aa40 msgr2=0x7efd4806a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd47fff640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efd4814aa40 0x7efd4806a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd47fff640 1 -- 172.21.15.39:0/2985916239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd480c5fd0 msgr2=0x7efd4814b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd47fff640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd480c5fd0 0x7efd4814b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd47fff640 1 -- 172.21.15.39:0/2985916239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0094d0 con 0x7efd480c4710 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd47fff640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 0x7efd48144920 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c500 tx=0x7efd3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd44ff9640 1 -- 172.21.15.39:0/2985916239 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38005220 con 0x7efd480c4710 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd44ff9640 1 -- 172.21.15.39:0/2985916239 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd38005380 con 0x7efd480c4710 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd48144400 con 0x7efd480c4710 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd44ff9640 1 -- 172.21.15.39:0/2985916239 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38007270 con 0x7efd480c4710 2024-09-13T16:52:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd480723c0 con 0x7efd480c4710 2024-09-13T16:52:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4814df50 con 0x7efd480c4710 2024-09-13T16:52:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd44ff9640 1 -- 172.21.15.39:0/2985916239 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efd380073d0 con 0x7efd480c4710 2024-09-13T16:52:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd44ff9640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efd34066690 0x7efd34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:32.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd477fe640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efd34066690 0x7efd34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:32.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd44ff9640 1 -- 172.21.15.39:0/2985916239 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7efd38083990 con 0x7efd480c4710 2024-09-13T16:52:32.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.839+0000 7efd477fe640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efd34066690 0x7efd34068b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0b1000 tx=0x7efd3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:32.843+0000 7efd44ff9640 1 -- 172.21.15.39:0/2985916239 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efd38050870 con 0x7efd480c4710 2024-09-13T16:52:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.079+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd4814be50 con 0x7efd480c4710 2024-09-13T16:52:33.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd44ff9640 1 -- 172.21.15.39:0/2985916239 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5933 (secure 0 0 0) 0x7efd38053e90 con 0x7efd480c4710 2024-09-13T16:52:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:33.086 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 40 2024-09-13T16:52:33.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efd34066690 msgr2=0x7efd34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:33.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efd34066690 0x7efd34068b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0b1000 tx=0x7efd3c009040 comp rx=0 tx=0).stop 2024-09-13T16:52:33.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 msgr2=0x7efd48144920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:33.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 0x7efd48144920 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c500 tx=0x7efd3800c9c0 comp rx=0 tx=0).stop 2024-09-13T16:52:33.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 shutdown_connections 2024-09-13T16:52:33.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efd34066690 0x7efd34068b50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:33.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efd480c5fd0 0x7efd4814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:33.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efd4814aa40 0x7efd4806a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:33.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 --2- 172.21.15.39:0/2985916239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd480c4710 0x7efd48144920 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:33.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 >> 172.21.15.39:0/2985916239 conn(0x7efd480c3310 msgr2=0x7efd480c0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:33.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 shutdown_connections 2024-09-13T16:52:33.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:33.083+0000 7efd4e79c640 1 -- 172.21.15.39:0/2985916239 wait complete. 2024-09-13T16:52:33.108 INFO:tasks.mds_thrash.fs.[b]:mds.b reported in up:standby state 2024-09-13T16:52:33.108 INFO:tasks.mds_thrash.fs.[b]:waiting for 26 secs before thrashing 2024-09-13T16:52:44.706 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.839+0000 7f8b85308640 1 Processor -- start 2024-09-13T16:52:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.839+0000 7f8b85308640 1 -- start start 2024-09-13T16:52:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b85308640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8b800c4260 0x7f8b800c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b85308640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b80070b40 0x7f8b8015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b85308640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b80157360 con 0x7f8b800bcfb0 2024-09-13T16:52:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b85308640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b800c80b0 con 0x7f8b800c4260 2024-09-13T16:52:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b85308640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b801575b0 con 0x7f8b80070b40 2024-09-13T16:52:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8b800c4260 0x7f8b800c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7e7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8b800bcfb0 0x7f8b80070430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:43586/0 (socket says 172.21.15.39:43586) 2024-09-13T16:52:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7e7fc640 1 -- 172.21.15.39:0/40433341 learned_addr learned my addr 172.21.15.39:0/40433341 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:44.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7f7fe640 1 --2- 172.21.15.39:0/40433341 >> v2:172.21.15.84:3300/0 conn(0x7f8b80070b40 0x7f8b8015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7effd640 1 -- 172.21.15.39:0/40433341 >> v2:172.21.15.84:3300/0 conn(0x7f8b80070b40 msgr2=0x7f8b8015e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:44.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7effd640 1 --2- 172.21.15.39:0/40433341 >> v2:172.21.15.84:3300/0 conn(0x7f8b80070b40 0x7f8b8015e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7effd640 1 -- 172.21.15.39:0/40433341 >> v1:172.21.15.39:6789/0 conn(0x7f8b800bcfb0 legacy=0x7f8b80070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7effd640 1 -- 172.21.15.39:0/40433341 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80157890 con 0x7f8b800c4260 2024-09-13T16:52:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7effd640 1 --2- 172.21.15.39:0/40433341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800c4260 0x7f8b800c4640 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8b800ca990 tx=0x7f8b7001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ca54837fbe0e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7dffb640 1 -- 172.21.15.39:0/40433341 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800c7330 con 0x7f8b800c4260 2024-09-13T16:52:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7dffb640 1 -- 172.21.15.39:0/40433341 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b700a7050 con 0x7f8b800c4260 2024-09-13T16:52:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.843+0000 7f8b7dffb640 1 -- 172.21.15.39:0/40433341 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700ab820 con 0x7f8b800c4260 2024-09-13T16:52:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- 172.21.15.39:0/40433341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800c4260 msgr2=0x7f8b800c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 --2- 172.21.15.39:0/40433341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800c4260 0x7f8b800c4640 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8b800ca990 tx=0x7f8b7001c360 comp rx=0 tx=0).stop 2024-09-13T16:52:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- 172.21.15.39:0/40433341 shutdown_connections 2024-09-13T16:52:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 --2- 172.21.15.39:0/40433341 >> v2:172.21.15.84:3300/0 conn(0x7f8b80070b40 0x7f8b8015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 --2- 172.21.15.39:0/40433341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800c4260 0x7f8b800c4640 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- 172.21.15.39:0/40433341 >> 172.21.15.39:0/40433341 conn(0x7f8b80069e10 msgr2=0x7f8b8006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- 172.21.15.39:0/40433341 shutdown_connections 2024-09-13T16:52:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- 172.21.15.39:0/40433341 wait complete. 2024-09-13T16:52:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 Processor -- start 2024-09-13T16:52:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- start start 2024-09-13T16:52:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800bcfb0 0x7f8b80147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b800c4260 0x7f8b801ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b800ca500 0x7f8b801ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b801ed920 con 0x7f8b800bcfb0 2024-09-13T16:52:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800bd840 con 0x7f8b800c4260 2024-09-13T16:52:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b801ec9a0 con 0x7f8b800ca500 2024-09-13T16:52:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800bcfb0 0x7f8b80147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800bcfb0 0x7f8b80147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41768/0 (socket says 172.21.15.39:41768) 2024-09-13T16:52:44.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7effd640 1 -- 172.21.15.39:0/161602746 learned_addr learned my addr 172.21.15.39:0/161602746 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7f7fe640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b800ca500 0x7f8b801ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7e7fc640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b800c4260 0x7f8b801ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7effd640 1 -- 172.21.15.39:0/161602746 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b800ca500 msgr2=0x7f8b801ed3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7effd640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b800ca500 0x7f8b801ed3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7effd640 1 -- 172.21.15.39:0/161602746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b800c4260 msgr2=0x7f8b801ecea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7effd640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b800c4260 0x7f8b801ecea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7effd640 1 -- 172.21.15.39:0/161602746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70009420 con 0x7f8b800bcfb0 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b7effd640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800bcfb0 0x7f8b80147600 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8b700ae040 tx=0x7f8b7001aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b677fe640 1 -- 172.21.15.39:0/161602746 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700b7650 con 0x7f8b800bcfb0 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b677fe640 1 -- 172.21.15.39:0/161602746 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b700b77d0 con 0x7f8b800bcfb0 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b800bd560 con 0x7f8b800bcfb0 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b677fe640 1 -- 172.21.15.39:0/161602746 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700b7b80 con 0x7f8b800bcfb0 2024-09-13T16:52:44.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.847+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b801f0ca0 con 0x7f8b800bcfb0 2024-09-13T16:52:44.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.851+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b800c4640 con 0x7f8b800bcfb0 2024-09-13T16:52:44.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.851+0000 7f8b677fe640 1 -- 172.21.15.39:0/161602746 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b700b7ce0 con 0x7f8b800bcfb0 2024-09-13T16:52:44.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.851+0000 7f8b677fe640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:44.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.851+0000 7f8b677fe640 1 -- 172.21.15.39:0/161602746 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f8b7011f060 con 0x7f8b800bcfb0 2024-09-13T16:52:44.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.851+0000 7f8b7e7fc640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:44.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.851+0000 7f8b7e7fc640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b80154f00 tx=0x7f8b74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:44.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:44.855+0000 7f8b677fe640 1 -- 172.21.15.39:0/161602746 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8b700ebf10 con 0x7f8b800bcfb0 2024-09-13T16:52:45.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b800c6e70 con 0x7f8b800bcfb0 2024-09-13T16:52:45.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b677fe640 1 -- 172.21.15.39:0/161602746 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f8b700ef530 con 0x7f8b800bcfb0 2024-09-13T16:52:45.076 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:45.076 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:45.076 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 40 2024-09-13T16:52:45.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 msgr2=0x7f8b6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8b80154f00 tx=0x7f8b74009040 comp rx=0 tx=0).stop 2024-09-13T16:52:45.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800bcfb0 msgr2=0x7f8b80147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800bcfb0 0x7f8b80147600 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8b700ae040 tx=0x7f8b7001aba0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 shutdown_connections 2024-09-13T16:52:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b6c066690 0x7f8b6c068b50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b800c4260 0x7f8b801ecea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b800ca500 0x7f8b801ed3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 --2- 172.21.15.39:0/161602746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b800bcfb0 0x7f8b80147600 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 >> 172.21.15.39:0/161602746 conn(0x7f8b80069e10 msgr2=0x7f8b800c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 shutdown_connections 2024-09-13T16:52:45.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.071+0000 7f8b85308640 1 -- 172.21.15.39:0/161602746 wait complete. 2024-09-13T16:52:45.095 INFO:tasks.mds_thrash.fs.[a]:kill mds.d (rank=0) 2024-09-13T16:52:45.096 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:52:45.096 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:45.111 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:52:45.112 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:45.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ffa346640 1 Processor -- start 2024-09-13T16:52:45.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ffa346640 1 -- start start 2024-09-13T16:52:45.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ffa346640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1ff4151500 0x7f1ff40c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ffa346640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4071dd0 0x7f1ff4154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ffa346640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff40c5a30 con 0x7f1ff40c47c0 2024-09-13T16:52:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ffa346640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff41576a0 con 0x7f1ff4151500 2024-09-13T16:52:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ffa346640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4157980 con 0x7f1ff4071dd0 2024-09-13T16:52:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ff37fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4071dd0 0x7f1ff4154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ff3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1ff4151500 0x7f1ff40c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ff37fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4071dd0 0x7f1ff4154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:39404/0 (socket says 172.21.15.39:39404) 2024-09-13T16:52:45.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ff37fe640 1 -- 172.21.15.39:0/4100847373 learned_addr learned my addr 172.21.15.39:0/4100847373 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ff37fe640 1 -- 172.21.15.39:0/4100847373 >> v2:172.21.15.84:3300/0 conn(0x7f1ff4151500 msgr2=0x7f1ff40c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ff37fe640 1 --2- 172.21.15.39:0/4100847373 >> v2:172.21.15.84:3300/0 conn(0x7f1ff4151500 0x7f1ff40c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ff2ffd640 1 -- 172.21.15.39:0/4100847373 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044592909 0 0) 0x7f1ff40c5a30 con 0x7f1ff40c47c0 2024-09-13T16:52:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ff37fe640 1 -- 172.21.15.39:0/4100847373 >> v1:172.21.15.84:6789/0 conn(0x7f1ff40c47c0 legacy=0x7f1ff41545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.231+0000 7f1ff37fe640 1 -- 172.21.15.39:0/4100847373 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4157c60 con 0x7f1ff4071dd0 2024-09-13T16:52:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ff37fe640 1 --2- 172.21.15.39:0/4100847373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4071dd0 0x7f1ff4154070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009870 tx=0x7f1fe801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=263c09fcc677c301 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ff2ffd640 1 -- 172.21.15.39:0/4100847373 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80aa620 con 0x7f1ff4071dd0 2024-09-13T16:52:45.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ff2ffd640 1 -- 172.21.15.39:0/4100847373 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe80aa780 con 0x7f1ff4071dd0 2024-09-13T16:52:45.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ff2ffd640 1 -- 172.21.15.39:0/4100847373 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80aab10 con 0x7f1ff4071dd0 2024-09-13T16:52:45.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 -- 172.21.15.39:0/4100847373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4071dd0 msgr2=0x7f1ff4154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 --2- 172.21.15.39:0/4100847373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4071dd0 0x7f1ff4154070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009870 tx=0x7f1fe801c4a0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 -- 172.21.15.39:0/4100847373 shutdown_connections 2024-09-13T16:52:45.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 --2- 172.21.15.39:0/4100847373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4071dd0 0x7f1ff4154070 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009870 tx=0x7f1fe801c4a0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 --2- 172.21.15.39:0/4100847373 >> v2:172.21.15.84:3300/0 conn(0x7f1ff4151500 0x7f1ff40c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 -- 172.21.15.39:0/4100847373 >> 172.21.15.39:0/4100847373 conn(0x7f1ff40ab530 msgr2=0x7f1ff40ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:45.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 -- 172.21.15.39:0/4100847373 shutdown_connections 2024-09-13T16:52:45.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 -- 172.21.15.39:0/4100847373 wait complete. 2024-09-13T16:52:45.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 Processor -- start 2024-09-13T16:52:45.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.235+0000 7f1ffa346640 1 -- start start 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ffa346640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff40c47c0 0x7f1ff41537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ffa346640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4151500 0x7f1ff40c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ffa346640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ff41ec540 0x7f1ff4151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ffa346640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4155150 con 0x7f1ff40c47c0 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ffa346640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40ab3b0 con 0x7f1ff41ec540 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ffa346640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40aaf20 con 0x7f1ff4151500 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff40c47c0 0x7f1ff41537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff40c47c0 0x7f1ff41537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41782/0 (socket says 172.21.15.39:41782) 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff3fff640 1 -- 172.21.15.39:0/604526609 learned_addr learned my addr 172.21.15.39:0/604526609 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff88e3640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ff41ec540 0x7f1ff4151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff37fe640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4151500 0x7f1ff40c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff3fff640 1 -- 172.21.15.39:0/604526609 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4151500 msgr2=0x7f1ff40c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff3fff640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4151500 0x7f1ff40c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff3fff640 1 -- 172.21.15.39:0/604526609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ff41ec540 msgr2=0x7f1ff4151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff3fff640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ff41ec540 0x7f1ff4151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff3fff640 1 -- 172.21.15.39:0/604526609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80094d0 con 0x7f1ff40c47c0 2024-09-13T16:52:45.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff3fff640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff40c47c0 0x7f1ff41537d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e8a0 tx=0x7f1fe400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff0ff9640 1 -- 172.21.15.39:0/604526609 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4009510 con 0x7f1ff40c47c0 2024-09-13T16:52:45.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff0ff9640 1 -- 172.21.15.39:0/604526609 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe400c8e0 con 0x7f1ff40c47c0 2024-09-13T16:52:45.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff414ff60 con 0x7f1ff40c47c0 2024-09-13T16:52:45.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff0ff9640 1 -- 172.21.15.39:0/604526609 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400cc50 con 0x7f1ff40c47c0 2024-09-13T16:52:45.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff4155360 con 0x7f1ff40c47c0 2024-09-13T16:52:45.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff40c4280 con 0x7f1ff40c47c0 2024-09-13T16:52:45.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff0ff9640 1 -- 172.21.15.39:0/604526609 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1fe4013470 con 0x7f1ff40c47c0 2024-09-13T16:52:45.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff0ff9640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1fe0066690 0x7f1fe0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:45.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff0ff9640 1 -- 172.21.15.39:0/604526609 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f1fe40849f0 con 0x7f1ff40c47c0 2024-09-13T16:52:45.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.239+0000 7f1ff37fe640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1fe0066690 0x7f1fe0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:45.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.243+0000 7f1ff37fe640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1fe0066690 0x7f1fe0068b50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80ae000 tx=0x7f1fe80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:45.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.247+0000 7f1ff0ff9640 1 -- 172.21.15.39:0/604526609 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1fe4051950 con 0x7f1ff40c47c0 2024-09-13T16:52:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.467+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ff4071bb0 con 0x7f1ff40c47c0 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.467+0000 7f1ff0ff9640 1 -- 172.21.15.39:0/604526609 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f1fe4054f70 con 0x7f1ff40c47c0 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 40 2024-09-13T16:52:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1fe0066690 msgr2=0x7f1fe0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1fe0066690 0x7f1fe0068b50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80ae000 tx=0x7f1fe80a7000 comp rx=0 tx=0).stop 2024-09-13T16:52:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff40c47c0 msgr2=0x7f1ff41537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff40c47c0 0x7f1ff41537d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e8a0 tx=0x7f1fe400eca0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 shutdown_connections 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1fe0066690 0x7f1fe0068b50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ff41ec540 0x7f1ff4151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ff4151500 0x7f1ff40c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 --2- 172.21.15.39:0/604526609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff40c47c0 0x7f1ff41537d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 >> 172.21.15.39:0/604526609 conn(0x7f1ff40ab530 msgr2=0x7f1ff40ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 shutdown_connections 2024-09-13T16:52:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:45.471+0000 7f1ffa346640 1 -- 172.21.15.39:0/604526609 wait complete. 2024-09-13T16:52:45.494 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:52:47.495 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:47.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9d696a640 1 Processor -- start 2024-09-13T16:52:47.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9d696a640 1 -- start start 2024-09-13T16:52:47.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9d696a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd9d006c870 0x7fd9d00aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9d696a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd9d00c40b0 0x7fd9d015c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9d696a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d0152a80 con 0x7fd9d00aabc0 2024-09-13T16:52:47.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9d696a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d00c30f0 con 0x7fd9d006c870 2024-09-13T16:52:47.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9d696a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d0151e00 con 0x7fd9d00c40b0 2024-09-13T16:52:47.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9cf7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd9d006c870 0x7fd9d00aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9cffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd9d00aabc0 0x7fd9d00c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:43592/0 (socket says 172.21.15.39:43592) 2024-09-13T16:52:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9cffff640 1 -- 172.21.15.39:0/2989654236 learned_addr learned my addr 172.21.15.39:0/2989654236 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9d4f07640 1 --2- 172.21.15.39:0/2989654236 >> v2:172.21.15.84:3300/0 conn(0x7fd9d00c40b0 0x7fd9d015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9ceffd640 1 -- 172.21.15.39:0/2989654236 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243102238 0 0) 0x7fd9c0001120 con 0x7fd9d00aabc0 2024-09-13T16:52:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9ceffd640 1 -- 172.21.15.39:0/2989654236 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b8003530 con 0x7fd9d00aabc0 2024-09-13T16:52:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9cf7fe640 1 -- 172.21.15.39:0/2989654236 >> v2:172.21.15.84:3300/0 conn(0x7fd9d00c40b0 msgr2=0x7fd9d015c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9cf7fe640 1 --2- 172.21.15.39:0/2989654236 >> v2:172.21.15.84:3300/0 conn(0x7fd9d00c40b0 0x7fd9d015c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9cf7fe640 1 -- 172.21.15.39:0/2989654236 >> v1:172.21.15.39:6789/0 conn(0x7fd9d00aabc0 legacy=0x7fd9d00c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.627+0000 7fd9cf7fe640 1 -- 172.21.15.39:0/2989654236 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d01520e0 con 0x7fd9d006c870 2024-09-13T16:52:47.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9cf7fe640 1 --2- 172.21.15.39:0/2989654236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 0x7fd9d00aafa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd9d006f220 tx=0x7fd9c401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38cfee22d32ac67 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:47.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9ceffd640 1 -- 172.21.15.39:0/2989654236 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40ac620 con 0x7fd9d006c870 2024-09-13T16:52:47.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9ceffd640 1 -- 172.21.15.39:0/2989654236 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c40ac780 con 0x7fd9d006c870 2024-09-13T16:52:47.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9ceffd640 1 -- 172.21.15.39:0/2989654236 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40acaf0 con 0x7fd9d006c870 2024-09-13T16:52:47.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 -- 172.21.15.39:0/2989654236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 msgr2=0x7fd9d00aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 --2- 172.21.15.39:0/2989654236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 0x7fd9d00aafa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd9d006f220 tx=0x7fd9c401c3c0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 -- 172.21.15.39:0/2989654236 shutdown_connections 2024-09-13T16:52:47.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 --2- 172.21.15.39:0/2989654236 >> v2:172.21.15.84:3300/0 conn(0x7fd9d00c40b0 0x7fd9d015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 --2- 172.21.15.39:0/2989654236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 0x7fd9d00aafa0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 -- 172.21.15.39:0/2989654236 >> 172.21.15.39:0/2989654236 conn(0x7fd9d0069e10 msgr2=0x7fd9d006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:47.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 -- 172.21.15.39:0/2989654236 shutdown_connections 2024-09-13T16:52:47.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 -- 172.21.15.39:0/2989654236 wait complete. 2024-09-13T16:52:47.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 Processor -- start 2024-09-13T16:52:47.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 -- start start 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 0x7fd9d0147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9d00aabc0 0x7fd9d0144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9d00c40b0 0x7fd9d0144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00a80d0 con 0x7fd9d006c870 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00a5830 con 0x7fd9d00c40b0 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9d696a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00bf900 con 0x7fd9d00aabc0 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 0x7fd9d0147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 0x7fd9d0147150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41796/0 (socket says 172.21.15.39:41796) 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.631+0000 7fd9cffff640 1 -- 172.21.15.39:0/2051466761 learned_addr learned my addr 172.21.15.39:0/2051466761 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:47.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9cf7fe640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9d00aabc0 0x7fd9d0144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9d4f07640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9d00c40b0 0x7fd9d0144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9cffff640 1 -- 172.21.15.39:0/2051466761 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9d00aabc0 msgr2=0x7fd9d0144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9cffff640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9d00aabc0 0x7fd9d0144a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9cffff640 1 -- 172.21.15.39:0/2051466761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9d00c40b0 msgr2=0x7fd9d0144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9cffff640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9d00c40b0 0x7fd9d0144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9cffff640 1 -- 172.21.15.39:0/2051466761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4009420 con 0x7fd9d006c870 2024-09-13T16:52:47.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9cffff640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 0x7fd9d0147150 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000c700 tx=0x7fd9c000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:47.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9ccff9640 1 -- 172.21.15.39:0/2051466761 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0005ac0 con 0x7fd9d006c870 2024-09-13T16:52:47.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9ccff9640 1 -- 172.21.15.39:0/2051466761 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c0005c60 con 0x7fd9d006c870 2024-09-13T16:52:47.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d00a82b0 con 0x7fd9d006c870 2024-09-13T16:52:47.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9ccff9640 1 -- 172.21.15.39:0/2051466761 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0001d60 con 0x7fd9d006c870 2024-09-13T16:52:47.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d00a54c0 con 0x7fd9d006c870 2024-09-13T16:52:47.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d00a5310 con 0x7fd9d006c870 2024-09-13T16:52:47.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9ccff9640 1 -- 172.21.15.39:0/2051466761 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9c0007070 con 0x7fd9d006c870 2024-09-13T16:52:47.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9ccff9640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9bc066690 0x7fd9bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:47.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9ccff9640 1 -- 172.21.15.39:0/2051466761 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7fd9c00839a0 con 0x7fd9d006c870 2024-09-13T16:52:47.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9cf7fe640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9bc066690 0x7fd9bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:47.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.635+0000 7fd9cf7fe640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9bc066690 0x7fd9bc068b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40ae000 tx=0x7fd9c40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:47.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.639+0000 7fd9ccff9640 1 -- 172.21.15.39:0/2051466761 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd9c0050870 con 0x7fd9d006c870 2024-09-13T16:52:47.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9d0152a80 con 0x7fd9d006c870 2024-09-13T16:52:47.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9ccff9640 1 -- 172.21.15.39:0/2051466761 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fd9d0152a80 con 0x7fd9d006c870 2024-09-13T16:52:47.847 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:47.848 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:47.848 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 40 2024-09-13T16:52:47.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9bc066690 msgr2=0x7fd9bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9bc066690 0x7fd9bc068b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40ae000 tx=0x7fd9c40a7000 comp rx=0 tx=0).stop 2024-09-13T16:52:47.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 msgr2=0x7fd9d0147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:47.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 0x7fd9d0147150 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000c700 tx=0x7fd9c000cbc0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 shutdown_connections 2024-09-13T16:52:47.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9bc066690 0x7fd9bc068b50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9d00c40b0 0x7fd9d0144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9d00aabc0 0x7fd9d0144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 --2- 172.21.15.39:0/2051466761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9d006c870 0x7fd9d0147150 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:47.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 >> 172.21.15.39:0/2051466761 conn(0x7fd9d0069e10 msgr2=0x7fd9d00c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:47.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 shutdown_connections 2024-09-13T16:52:47.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:47.843+0000 7fd9d696a640 1 -- 172.21.15.39:0/2051466761 wait complete. 2024-09-13T16:52:47.867 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:52:49.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:50.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:49.999+0000 7f385a731640 1 Processor -- start 2024-09-13T16:52:50.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f385a731640 1 -- start start 2024-09-13T16:52:50.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f385a731640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f385406fba0 0x7f3854154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f385a731640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38540c33c0 con 0x7f38540c1280 2024-09-13T16:52:50.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f385a731640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38540c0ac0 con 0x7f38540c28d0 2024-09-13T16:52:50.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f385a731640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38540611c0 con 0x7f385406fba0 2024-09-13T16:52:50.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f38537fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f385406fba0 0x7f3854154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f3858cce640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f38540c1280 0x7f385414df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:43608/0 (socket says 172.21.15.39:43608) 2024-09-13T16:52:50.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f3858cce640 1 -- 172.21.15.39:0/2367350377 learned_addr learned my addr 172.21.15.39:0/2367350377 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:50.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f38537fe640 1 -- 172.21.15.39:0/2367350377 >> v1:172.21.15.84:6789/0 conn(0x7f38540c28d0 legacy=0x7f38540c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f38537fe640 1 -- 172.21.15.39:0/2367350377 >> v1:172.21.15.39:6789/0 conn(0x7f38540c1280 legacy=0x7f385414df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f38537fe640 1 -- 172.21.15.39:0/2367350377 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38540613d0 con 0x7f385406fba0 2024-09-13T16:52:50.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.003+0000 7f3852ffd640 1 -- 172.21.15.39:0/2367350377 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321100071 0 0) 0x7f384c0014a0 con 0x7f38540c1280 2024-09-13T16:52:50.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f38537fe640 1 --2- 172.21.15.39:0/2367350377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 0x7f3854154b70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f38480097c0 tx=0x7f384801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855d2c363a90bdf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:50.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f3852ffd640 1 -- 172.21.15.39:0/2367350377 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480ab600 con 0x7f385406fba0 2024-09-13T16:52:50.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f3852ffd640 1 -- 172.21.15.39:0/2367350377 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38480a7050 con 0x7f385406fba0 2024-09-13T16:52:50.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f3852ffd640 1 -- 172.21.15.39:0/2367350377 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480ab980 con 0x7f385406fba0 2024-09-13T16:52:50.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f385a731640 1 -- 172.21.15.39:0/2367350377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 msgr2=0x7f3854154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f385a731640 1 --2- 172.21.15.39:0/2367350377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 0x7f3854154b70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f38480097c0 tx=0x7f384801c410 comp rx=0 tx=0).stop 2024-09-13T16:52:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f385a731640 1 -- 172.21.15.39:0/2367350377 shutdown_connections 2024-09-13T16:52:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f385a731640 1 --2- 172.21.15.39:0/2367350377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 0x7f3854154b70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f385a731640 1 -- 172.21.15.39:0/2367350377 >> 172.21.15.39:0/2367350377 conn(0x7f3854069e10 msgr2=0x7f385406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f385a731640 1 -- 172.21.15.39:0/2367350377 shutdown_connections 2024-09-13T16:52:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f385a731640 1 -- 172.21.15.39:0/2367350377 wait complete. 2024-09-13T16:52:50.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f385a731640 1 Processor -- start 2024-09-13T16:52:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.007+0000 7f385a731640 1 -- start start 2024-09-13T16:52:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f385a731640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 0x7f38541476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f385a731640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38540c1280 0x7f3854144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f385a731640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38540c28d0 0x7f38541ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f385a731640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38541ee350 con 0x7f385406fba0 2024-09-13T16:52:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f385a731640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38541e9e80 con 0x7f38540c1280 2024-09-13T16:52:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f385a731640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3854148400 con 0x7f38540c28d0 2024-09-13T16:52:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3853fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 0x7f38541476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3853fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 0x7f38541476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41812/0 (socket says 172.21.15.39:41812) 2024-09-13T16:52:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3853fff640 1 -- 172.21.15.39:0/2206622233 learned_addr learned my addr 172.21.15.39:0/2206622233 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f38537fe640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38540c1280 0x7f3854144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3858cce640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38540c28d0 0x7f38541ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3853fff640 1 -- 172.21.15.39:0/2206622233 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38540c28d0 msgr2=0x7f38541ea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3853fff640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38540c28d0 0x7f38541ea2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3853fff640 1 -- 172.21.15.39:0/2206622233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38540c1280 msgr2=0x7f3854144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3853fff640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38540c1280 0x7f3854144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3853fff640 1 -- 172.21.15.39:0/2206622233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848009420 con 0x7f385406fba0 2024-09-13T16:52:50.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f38537fe640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38540c1280 0x7f3854144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:50.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3853fff640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 0x7f38541476d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f384400c3f0 tx=0x7f384400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:50.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3850ff9640 1 -- 172.21.15.39:0/2206622233 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38440052d0 con 0x7f385406fba0 2024-09-13T16:52:50.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3850ff9640 1 -- 172.21.15.39:0/2206622233 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3844005430 con 0x7f385406fba0 2024-09-13T16:52:50.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3850ff9640 1 -- 172.21.15.39:0/2206622233 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844003840 con 0x7f385406fba0 2024-09-13T16:52:50.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3854152c90 con 0x7f385406fba0 2024-09-13T16:52:50.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3854148950 con 0x7f385406fba0 2024-09-13T16:52:50.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38541ea120 con 0x7f385406fba0 2024-09-13T16:52:50.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3850ff9640 1 -- 172.21.15.39:0/2206622233 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38440039a0 con 0x7f385406fba0 2024-09-13T16:52:50.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.011+0000 7f3850ff9640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3840066690 0x7f3840068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:50.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.015+0000 7f3850ff9640 1 -- 172.21.15.39:0/2206622233 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f3844083ba0 con 0x7f385406fba0 2024-09-13T16:52:50.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.015+0000 7f38537fe640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3840066690 0x7f3840068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:50.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.015+0000 7f38537fe640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3840066690 0x7f3840068b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f38480ad000 tx=0x7f38480a4f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:50.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.019+0000 7f3850ff9640 1 -- 172.21.15.39:0/2206622233 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3844050a50 con 0x7f385406fba0 2024-09-13T16:52:50.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.215+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3854154b70 con 0x7f385406fba0 2024-09-13T16:52:50.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.215+0000 7f3850ff9640 1 -- 172.21.15.39:0/2206622233 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f3844054070 con 0x7f385406fba0 2024-09-13T16:52:50.220 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:50.220 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 40 2024-09-13T16:52:50.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.215+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3840066690 msgr2=0x7f3840068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.215+0000 7f385a731640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3840066690 0x7f3840068b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f38480ad000 tx=0x7f38480a4f90 comp rx=0 tx=0).stop 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.215+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 msgr2=0x7f38541476d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.215+0000 7f385a731640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 0x7f38541476d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f384400c3f0 tx=0x7f384400c8b0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.219+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 shutdown_connections 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.219+0000 7f385a731640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3840066690 0x7f3840068b50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.219+0000 7f385a731640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38540c1280 0x7f3854144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.219+0000 7f385a731640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38540c28d0 0x7f38541ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.219+0000 7f385a731640 1 --2- 172.21.15.39:0/2206622233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f385406fba0 0x7f38541476d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.219+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 >> 172.21.15.39:0/2206622233 conn(0x7f3854069e10 msgr2=0x7f385414df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.219+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 shutdown_connections 2024-09-13T16:52:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:50.219+0000 7f385a731640 1 -- 172.21.15.39:0/2206622233 wait complete. 2024-09-13T16:52:50.239 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:52:52.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:52.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f961270e640 1 Processor -- start 2024-09-13T16:52:52.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f961270e640 1 -- start start 2024-09-13T16:52:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f961270e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f961270e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f960c0c47c0 0x7f960c154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f961270e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960c0c5a30 con 0x7f960c071dd0 2024-09-13T16:52:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f961270e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c1576a0 con 0x7f960c0c47c0 2024-09-13T16:52:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f961270e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c157980 con 0x7f960c151500 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f9610cab640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f960c0c47c0 0x7f960c154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f960b7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f960c071dd0 0x7f960c154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49622/0 (socket says 172.21.15.39:49622) 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f9610cab640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f960c0c47c0 0x7f960c154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57580/0 (socket says 172.21.15.39:57580) 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f960b7fe640 1 -- 172.21.15.39:0/1019795155 learned_addr learned my addr 172.21.15.39:0/1019795155 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f960bfff640 1 --2- 172.21.15.39:0/1019795155 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f9610cab640 1 -- 172.21.15.39:0/1019795155 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 msgr2=0x7f960c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f9610cab640 1 --2- 172.21.15.39:0/1019795155 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c0c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f9610cab640 1 -- 172.21.15.39:0/1019795155 >> v1:172.21.15.39:6789/0 conn(0x7f960c071dd0 legacy=0x7f960c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f9610cab640 1 -- 172.21.15.39:0/1019795155 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c157c60 con 0x7f960c0c47c0 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f960affd640 1 -- 172.21.15.39:0/1019795155 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929356341 0 0) 0x7f960c0c5a30 con 0x7f960c071dd0 2024-09-13T16:52:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.375+0000 7f960bfff640 1 --2- 172.21.15.39:0/1019795155 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c0c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:52.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f9610cab640 1 --2- 172.21.15.39:0/1019795155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 0x7f960c154780 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9604009d90 tx=0x7f960401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c19050fccd4c03a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:52.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960affd640 1 -- 172.21.15.39:0/1019795155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96040a5990 con 0x7f960c0c47c0 2024-09-13T16:52:52.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960affd640 1 -- 172.21.15.39:0/1019795155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96040a5af0 con 0x7f960c0c47c0 2024-09-13T16:52:52.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960affd640 1 -- 172.21.15.39:0/1019795155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96040a5e80 con 0x7f960c0c47c0 2024-09-13T16:52:52.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 -- 172.21.15.39:0/1019795155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 msgr2=0x7f960c154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 --2- 172.21.15.39:0/1019795155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 0x7f960c154780 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9604009d90 tx=0x7f960401c8b0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 -- 172.21.15.39:0/1019795155 shutdown_connections 2024-09-13T16:52:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 --2- 172.21.15.39:0/1019795155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 0x7f960c154780 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 --2- 172.21.15.39:0/1019795155 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c0c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 -- 172.21.15.39:0/1019795155 >> 172.21.15.39:0/1019795155 conn(0x7f960c0ab530 msgr2=0x7f960c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 -- 172.21.15.39:0/1019795155 shutdown_connections 2024-09-13T16:52:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 -- 172.21.15.39:0/1019795155 wait complete. 2024-09-13T16:52:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 Processor -- start 2024-09-13T16:52:52.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 -- start start 2024-09-13T16:52:52.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c071dd0 0x7f960c1f0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 0x7f960c1ec170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c14a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c14cbf0 con 0x7f960c0c47c0 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c1efdd0 con 0x7f960c071dd0 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f961270e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c156640 con 0x7f960c151500 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 0x7f960c1ec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 0x7f960c1ec170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57594/0 (socket says 172.21.15.39:57594) 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960b7fe640 1 -- 172.21.15.39:0/2433264157 learned_addr learned my addr 172.21.15.39:0/2433264157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960bfff640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c071dd0 0x7f960c1f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f9610cab640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c14a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960b7fe640 1 -- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 msgr2=0x7f960c14a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960b7fe640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c14a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960b7fe640 1 -- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c071dd0 msgr2=0x7f960c1f0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960b7fe640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c071dd0 0x7f960c1f0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960b7fe640 1 -- 172.21.15.39:0/2433264157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9604009a70 con 0x7f960c0c47c0 2024-09-13T16:52:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960bfff640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c071dd0 0x7f960c1f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f9610cab640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c14a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.379+0000 7f960b7fe640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 0x7f960c1ec170 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f960000c4b0 tx=0x7f960000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f9608ff9640 1 -- 172.21.15.39:0/2433264157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96000074e0 con 0x7f960c0c47c0 2024-09-13T16:52:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f9608ff9640 1 -- 172.21.15.39:0/2433264157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600007660 con 0x7f960c0c47c0 2024-09-13T16:52:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f9608ff9640 1 -- 172.21.15.39:0/2433264157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600004a30 con 0x7f960c0c47c0 2024-09-13T16:52:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c156360 con 0x7f960c0c47c0 2024-09-13T16:52:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c1efa60 con 0x7f960c0c47c0 2024-09-13T16:52:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960c1eff40 con 0x7f960c0c47c0 2024-09-13T16:52:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f9608ff9640 1 -- 172.21.15.39:0/2433264157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9600004b90 con 0x7f960c0c47c0 2024-09-13T16:52:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f9608ff9640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95f8066690 0x7f95f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f9608ff9640 1 -- 172.21.15.39:0/2433264157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f96000838c0 con 0x7f960c0c47c0 2024-09-13T16:52:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f960bfff640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95f8066690 0x7f95f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.383+0000 7f960bfff640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95f8066690 0x7f95f8068b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0097a0 tx=0x7f95fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:52.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.387+0000 7f9608ff9640 1 -- 172.21.15.39:0/2433264157 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9600050830 con 0x7f960c0c47c0 2024-09-13T16:52:52.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.587+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f960c0c4280 con 0x7f960c0c47c0 2024-09-13T16:52:52.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.587+0000 7f9608ff9640 1 -- 172.21.15.39:0/2433264157 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f9600018020 con 0x7f960c0c47c0 2024-09-13T16:52:52.593 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:52.593 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:52.594 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 40 2024-09-13T16:52:52.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95f8066690 msgr2=0x7f95f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95f8066690 0x7f95f8068b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0097a0 tx=0x7f95fc008040 comp rx=0 tx=0).stop 2024-09-13T16:52:52.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 msgr2=0x7f960c1ec170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:52.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 0x7f960c1ec170 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f960000c4b0 tx=0x7f960000c970 comp rx=0 tx=0).stop 2024-09-13T16:52:52.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 shutdown_connections 2024-09-13T16:52:52.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95f8066690 0x7f95f8068b50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f960c151500 0x7f960c14a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960c0c47c0 0x7f960c1ec170 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 --2- 172.21.15.39:0/2433264157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c071dd0 0x7f960c1f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:52.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 >> 172.21.15.39:0/2433264157 conn(0x7f960c0ab530 msgr2=0x7f960c0c7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:52.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 shutdown_connections 2024-09-13T16:52:52.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:52.591+0000 7f961270e640 1 -- 172.21.15.39:0/2433264157 wait complete. 2024-09-13T16:52:52.613 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:52:54.614 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:54.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.747+0000 7f22f126d640 1 Processor -- start 2024-09-13T16:52:54.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.747+0000 7f22f126d640 1 -- start start 2024-09-13T16:52:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22f126d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f22ec06c870 0x7f22ec0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22f126d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 0x7f22ec15c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22f126d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec152a80 con 0x7f22ec0aabc0 2024-09-13T16:52:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22f126d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec0c30f0 con 0x7f22ec06c870 2024-09-13T16:52:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22f126d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec151e00 con 0x7f22ec0c40b0 2024-09-13T16:52:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22ea7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f22ec06c870 0x7f22ec0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22ea7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f22ec06c870 0x7f22ec0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57596/0 (socket says 172.21.15.39:57596) 2024-09-13T16:52:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22ea7fc640 1 -- 172.21.15.39:0/3817324592 learned_addr learned my addr 172.21.15.39:0/3817324592 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:54.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22eb7fe640 1 --2- 172.21.15.39:0/3817324592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 0x7f22ec15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22ea7fc640 1 -- 172.21.15.39:0/3817324592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 msgr2=0x7f22ec15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22ea7fc640 1 --2- 172.21.15.39:0/3817324592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 0x7f22ec15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22ea7fc640 1 -- 172.21.15.39:0/3817324592 >> v1:172.21.15.84:6789/0 conn(0x7f22ec0aabc0 legacy=0x7f22ec0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22ea7fc640 1 -- 172.21.15.39:0/3817324592 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec1520e0 con 0x7f22ec06c870 2024-09-13T16:52:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22ea7fc640 1 --2- 172.21.15.39:0/3817324592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 0x7f22ec0aafa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009870 tx=0x7f22e001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad84f0112f60f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22e9ffb640 1 -- 172.21.15.39:0/3817324592 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e00a6040 con 0x7f22ec06c870 2024-09-13T16:52:54.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22e9ffb640 1 -- 172.21.15.39:0/3817324592 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e00aa600 con 0x7f22ec06c870 2024-09-13T16:52:54.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.751+0000 7f22e9ffb640 1 -- 172.21.15.39:0/3817324592 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e00aa990 con 0x7f22ec06c870 2024-09-13T16:52:54.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- 172.21.15.39:0/3817324592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 msgr2=0x7f22ec0aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 --2- 172.21.15.39:0/3817324592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 0x7f22ec0aafa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009870 tx=0x7f22e001c4a0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- 172.21.15.39:0/3817324592 shutdown_connections 2024-09-13T16:52:54.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 --2- 172.21.15.39:0/3817324592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 0x7f22ec15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 --2- 172.21.15.39:0/3817324592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 0x7f22ec0aafa0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- 172.21.15.39:0/3817324592 >> 172.21.15.39:0/3817324592 conn(0x7f22ec069e10 msgr2=0x7f22ec06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:54.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- 172.21.15.39:0/3817324592 shutdown_connections 2024-09-13T16:52:54.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- 172.21.15.39:0/3817324592 wait complete. 2024-09-13T16:52:54.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 Processor -- start 2024-09-13T16:52:54.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- start start 2024-09-13T16:52:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 0x7f22ec071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22ec0aabc0 0x7f22ec0a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 0x7f22ec0a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec14b490 con 0x7f22ec06c870 2024-09-13T16:52:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec150280 con 0x7f22ec0aabc0 2024-09-13T16:52:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec1eb620 con 0x7f22ec0c40b0 2024-09-13T16:52:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 0x7f22ec071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 0x7f22ec071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57598/0 (socket says 172.21.15.39:57598) 2024-09-13T16:52:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eaffd640 1 -- 172.21.15.39:0/1419701899 learned_addr learned my addr 172.21.15.39:0/1419701899 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22ea7fc640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22ec0aabc0 0x7f22ec0a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eb7fe640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 0x7f22ec0a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eaffd640 1 -- 172.21.15.39:0/1419701899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 msgr2=0x7f22ec0a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eaffd640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 0x7f22ec0a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eaffd640 1 -- 172.21.15.39:0/1419701899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22ec0aabc0 msgr2=0x7f22ec0a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eaffd640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22ec0aabc0 0x7f22ec0a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eaffd640 1 -- 172.21.15.39:0/1419701899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00094d0 con 0x7f22ec06c870 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eb7fe640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 0x7f22ec0a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22eaffd640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 0x7f22ec071660 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c530 tx=0x7f22dc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22d37fe640 1 -- 172.21.15.39:0/1419701899 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc005220 con 0x7f22ec06c870 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22d37fe640 1 -- 172.21.15.39:0/1419701899 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc005380 con 0x7f22ec06c870 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec14ae40 con 0x7f22ec06c870 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22d37fe640 1 -- 172.21.15.39:0/1419701899 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc007270 con 0x7f22ec06c870 2024-09-13T16:52:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.755+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec0c0820 con 0x7f22ec06c870 2024-09-13T16:52:54.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.759+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec0aafa0 con 0x7f22ec06c870 2024-09-13T16:52:54.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.759+0000 7f22d37fe640 1 -- 172.21.15.39:0/1419701899 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f22dc0073d0 con 0x7f22ec06c870 2024-09-13T16:52:54.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.759+0000 7f22d37fe640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22d80666c0 0x7f22d8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:54.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.759+0000 7f22d37fe640 1 -- 172.21.15.39:0/1419701899 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f22dc083d10 con 0x7f22ec06c870 2024-09-13T16:52:54.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.759+0000 7f22ea7fc640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22d80666c0 0x7f22d8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:54.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.759+0000 7f22ea7fc640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22d80666c0 0x7f22d8068b80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f22e00ae000 tx=0x7f22e00b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:54.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.763+0000 7f22d37fe640 1 -- 172.21.15.39:0/1419701899 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f22dc050ba0 con 0x7f22ec06c870 2024-09-13T16:52:54.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22ec152a80 con 0x7f22ec06c870 2024-09-13T16:52:54.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22d37fe640 1 -- 172.21.15.39:0/1419701899 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f22ec152a80 con 0x7f22ec06c870 2024-09-13T16:52:54.968 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:54.968 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:54.968 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 40 2024-09-13T16:52:54.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22d80666c0 msgr2=0x7f22d8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22d80666c0 0x7f22d8068b80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f22e00ae000 tx=0x7f22e00b5040 comp rx=0 tx=0).stop 2024-09-13T16:52:54.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 msgr2=0x7f22ec071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:54.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 0x7f22ec071660 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c530 tx=0x7f22dc00c9f0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 shutdown_connections 2024-09-13T16:52:54.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22d80666c0 0x7f22d8068b80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22ec0aabc0 0x7f22ec0a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22ec0c40b0 0x7f22ec0a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 --2- 172.21.15.39:0/1419701899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22ec06c870 0x7f22ec071660 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.963+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 >> 172.21.15.39:0/1419701899 conn(0x7f22ec069e10 msgr2=0x7f22ec1559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.967+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 shutdown_connections 2024-09-13T16:52:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:54.967+0000 7f22f126d640 1 -- 172.21.15.39:0/1419701899 wait complete. 2024-09-13T16:52:54.987 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:52:56.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:52:56.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:52:56.989 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:57.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c4577f640 1 Processor -- start 2024-09-13T16:52:57.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c4577f640 1 -- start start 2024-09-13T16:52:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c4577f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c40151500 0x7f2c400c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c4577f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c400c5a30 con 0x7f2c40071dd0 2024-09-13T16:52:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c4577f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c401576a0 con 0x7f2c400c47c0 2024-09-13T16:52:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c4577f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c40157980 con 0x7f2c40151500 2024-09-13T16:52:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c37fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2c40071dd0 0x7f2c40154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49630/0 (socket says 172.21.15.39:49630) 2024-09-13T16:52:57.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c37fff640 1 -- 172.21.15.39:0/4046753981 learned_addr learned my addr 172.21.15.39:0/4046753981 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3effd640 1 --2- 172.21.15.39:0/4046753981 >> v2:172.21.15.84:3300/0 conn(0x7f2c40151500 0x7f2c400c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861635939 0 0) 0x7f2c400c5a30 con 0x7f2c40071dd0 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c2c003530 con 0x7f2c40071dd0 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2343114049 0 0) 0x7f2c2c003530 con 0x7f2c40071dd0 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 >> v2:172.21.15.84:3300/0 conn(0x7f2c40151500 msgr2=0x7f2c400c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 --2- 172.21.15.39:0/4046753981 >> v2:172.21.15.84:3300/0 conn(0x7f2c40151500 0x7f2c400c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 >> v1:172.21.15.84:6789/0 conn(0x7f2c400c47c0 legacy=0x7f2c40154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40157c60 con 0x7f2c40071dd0 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f2c300016a0 con 0x7f2c40071dd0 2024-09-13T16:52:57.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 >> v1:172.21.15.39:6789/0 conn(0x7f2c40071dd0 legacy=0x7f2c40154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c003980 0x7f2c2c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c2c0047d0 0x7f2c2c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c2c0052b0 0x7f2c2c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c40157980 con 0x7f2c2c0047d0 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c00c180 con 0x7f2c2c0052b0 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c00c340 con 0x7f2c2c003980 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c37fff640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c2c0047d0 0x7f2c2c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3f7fe640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c2c0052b0 0x7f2c2c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3effd640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c003980 0x7f2c2c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c37fff640 1 -- 172.21.15.39:0/4046753981 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c003980 msgr2=0x7f2c2c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c37fff640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c003980 0x7f2c2c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c37fff640 1 -- 172.21.15.39:0/4046753981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c2c0052b0 msgr2=0x7f2c2c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:57.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c37fff640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c2c0052b0 0x7f2c2c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c37fff640 1 -- 172.21.15.39:0/4046753981 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c003530 con 0x7f2c2c0047d0 2024-09-13T16:52:57.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3f7fe640 1 -- 172.21.15.39:0/4046753981 reap_dead start 2024-09-13T16:52:57.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.119+0000 7f2c3effd640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c003980 0x7f2c2c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:57.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c37fff640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c2c0047d0 0x7f2c2c004c50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2c30001ec0 tx=0x7f2c3001d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7f29c75923ec8c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c300058e0 con 0x7f2c2c0047d0 2024-09-13T16:52:57.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c30005a40 con 0x7f2c2c0047d0 2024-09-13T16:52:57.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c3e7fc640 1 -- 172.21.15.39:0/4046753981 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30005df0 con 0x7f2c2c0047d0 2024-09-13T16:52:57.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 -- 172.21.15.39:0/4046753981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c2c0047d0 msgr2=0x7f2c2c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c2c0047d0 0x7f2c2c004c50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2c30001ec0 tx=0x7f2c3001d610 comp rx=0 tx=0).stop 2024-09-13T16:52:57.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 -- 172.21.15.39:0/4046753981 shutdown_connections 2024-09-13T16:52:57.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 --2- 172.21.15.39:0/4046753981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c2c0047d0 0x7f2c2c004c50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 -- 172.21.15.39:0/4046753981 >> 172.21.15.39:0/4046753981 conn(0x7f2c400ab530 msgr2=0x7f2c400ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:57.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 -- 172.21.15.39:0/4046753981 shutdown_connections 2024-09-13T16:52:57.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 -- 172.21.15.39:0/4046753981 wait complete. 2024-09-13T16:52:57.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 Processor -- start 2024-09-13T16:52:57.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 -- start start 2024-09-13T16:52:57.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c0047d0 0x7f2c401500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4014df20 0x7f2c401e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c401e0f60 0x7f2c40147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c401df940 con 0x7f2c4014df20 2024-09-13T16:52:57.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c401487a0 con 0x7f2c401e0f60 2024-09-13T16:52:57.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c4577f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4014e740 con 0x7f2c2c0047d0 2024-09-13T16:52:57.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c37fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4014df20 0x7f2c401e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c37fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4014df20 0x7f2c401e0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57626/0 (socket says 172.21.15.39:57626) 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.123+0000 7f2c37fff640 1 -- 172.21.15.39:0/266087054 learned_addr learned my addr 172.21.15.39:0/266087054 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c3effd640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c0047d0 0x7f2c401500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c3f7fe640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c401e0f60 0x7f2c40147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c37fff640 1 -- 172.21.15.39:0/266087054 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c0047d0 msgr2=0x7f2c401500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c37fff640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c0047d0 0x7f2c401500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c37fff640 1 -- 172.21.15.39:0/266087054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c401e0f60 msgr2=0x7f2c40147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c37fff640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c401e0f60 0x7f2c40147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c37fff640 1 -- 172.21.15.39:0/266087054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c401569d0 con 0x7f2c4014df20 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c37fff640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4014df20 0x7f2c401e0a20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2c300ab040 tx=0x7f2c2c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c377fe640 1 -- 172.21.15.39:0/266087054 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c300a58b0 con 0x7f2c4014df20 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c377fe640 1 -- 172.21.15.39:0/266087054 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c300a5a10 con 0x7f2c4014df20 2024-09-13T16:52:57.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c401566a0 con 0x7f2c4014df20 2024-09-13T16:52:57.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c377fe640 1 -- 172.21.15.39:0/266087054 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c300a5d80 con 0x7f2c4014df20 2024-09-13T16:52:57.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c401f0e80 con 0x7f2c4014df20 2024-09-13T16:52:57.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c40156b60 con 0x7f2c4014df20 2024-09-13T16:52:57.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c377fe640 1 -- 172.21.15.39:0/266087054 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c300047f0 con 0x7f2c4014df20 2024-09-13T16:52:57.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c377fe640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c200666c0 0x7f2c20068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:57.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c377fe640 1 -- 172.21.15.39:0/266087054 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f2c3011d4c0 con 0x7f2c4014df20 2024-09-13T16:52:57.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c3effd640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c200666c0 0x7f2c20068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:57.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.127+0000 7f2c3effd640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c200666c0 0x7f2c20068b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2c401572f0 tx=0x7f2c28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:57.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.131+0000 7f2c377fe640 1 -- 172.21.15.39:0/266087054 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2c300ea2e0 con 0x7f2c4014df20 2024-09-13T16:52:57.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.327+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c40070430 con 0x7f2c4014df20 2024-09-13T16:52:57.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c377fe640 1 -- 172.21.15.39:0/266087054 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f2c300ed900 con 0x7f2c4014df20 2024-09-13T16:52:57.335 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:57.335 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:57.335 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 40 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c200666c0 msgr2=0x7f2c20068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c200666c0 0x7f2c20068b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2c401572f0 tx=0x7f2c28008040 comp rx=0 tx=0).stop 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4014df20 msgr2=0x7f2c401e0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4014df20 0x7f2c401e0a20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2c300ab040 tx=0x7f2c2c005800 comp rx=0 tx=0).stop 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 shutdown_connections 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2c200666c0 0x7f2c20068b80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c401e0f60 0x7f2c40147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c4014df20 0x7f2c401e0a20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 --2- 172.21.15.39:0/266087054 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2c2c0047d0 0x7f2c401500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 >> 172.21.15.39:0/266087054 conn(0x7f2c400ab530 msgr2=0x7f2c401e2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:57.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 shutdown_connections 2024-09-13T16:52:57.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:57.331+0000 7f2c4577f640 1 -- 172.21.15.39:0/266087054 wait complete. 2024-09-13T16:52:57.353 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:52:59.110 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:59.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe1546d640 1 Processor -- start 2024-09-13T16:52:59.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe1546d640 1 -- start start 2024-09-13T16:52:59.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe1546d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efe101516d0 0x7efe101549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe1546d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efe100c4730 0x7efe100c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe1546d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe10154f30 con 0x7efe100c42f0 2024-09-13T16:52:59.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe1546d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe10071720 con 0x7efe101516d0 2024-09-13T16:52:59.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe1546d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe100c4ba0 con 0x7efe100c4730 2024-09-13T16:52:59.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0e7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7efe100c42f0 0x7efe100c6530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49644/0 (socket says 172.21.15.39:49644) 2024-09-13T16:52:59.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0e7fc640 1 -- 172.21.15.39:0/88213097 learned_addr learned my addr 172.21.15.39:0/88213097 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:59.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0effd640 1 --2- 172.21.15.39:0/88213097 >> v2:172.21.15.39:3300/0 conn(0x7efe101516d0 0x7efe101549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0f7fe640 1 --2- 172.21.15.39:0/88213097 >> v2:172.21.15.84:3300/0 conn(0x7efe100c4730 0x7efe100c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0dffb640 1 -- 172.21.15.39:0/88213097 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309326238 0 0) 0x7efe10154f30 con 0x7efe100c42f0 2024-09-13T16:52:59.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0dffb640 1 -- 172.21.15.39:0/88213097 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003530 con 0x7efe100c42f0 2024-09-13T16:52:59.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0effd640 1 -- 172.21.15.39:0/88213097 >> v2:172.21.15.84:3300/0 conn(0x7efe100c4730 msgr2=0x7efe100c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:59.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0effd640 1 --2- 172.21.15.39:0/88213097 >> v2:172.21.15.84:3300/0 conn(0x7efe100c4730 0x7efe100c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0effd640 1 -- 172.21.15.39:0/88213097 >> v1:172.21.15.39:6789/0 conn(0x7efe100c42f0 legacy=0x7efe100c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0effd640 1 -- 172.21.15.39:0/88213097 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10070800 con 0x7efe101516d0 2024-09-13T16:52:59.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.239+0000 7efe0f7fe640 1 --2- 172.21.15.39:0/88213097 >> v2:172.21.15.84:3300/0 conn(0x7efe100c4730 0x7efe100c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe0effd640 1 --2- 172.21.15.39:0/88213097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe101516d0 0x7efe101549f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efe1006b0a0 tx=0x7efe0001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877568c7bc90b6a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe0dffb640 1 -- 172.21.15.39:0/88213097 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe000a5930 con 0x7efe101516d0 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe0dffb640 1 -- 172.21.15.39:0/88213097 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe000a5a90 con 0x7efe101516d0 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe0dffb640 1 -- 172.21.15.39:0/88213097 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe000a5e20 con 0x7efe101516d0 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 -- 172.21.15.39:0/88213097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe101516d0 msgr2=0x7efe101549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 --2- 172.21.15.39:0/88213097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe101516d0 0x7efe101549f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efe1006b0a0 tx=0x7efe0001c870 comp rx=0 tx=0).stop 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 -- 172.21.15.39:0/88213097 shutdown_connections 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 --2- 172.21.15.39:0/88213097 >> v2:172.21.15.84:3300/0 conn(0x7efe100c4730 0x7efe100c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 --2- 172.21.15.39:0/88213097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe101516d0 0x7efe101549f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 -- 172.21.15.39:0/88213097 >> 172.21.15.39:0/88213097 conn(0x7efe100ab500 msgr2=0x7efe100ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:59.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 -- 172.21.15.39:0/88213097 shutdown_connections 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 -- 172.21.15.39:0/88213097 wait complete. 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 Processor -- start 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 -- start start 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe100c42f0 0x7efe100c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.243+0000 7efe1546d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe100c4730 0x7efe10144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe1546d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe101516d0 0x7efe10144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe1546d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe101488b0 con 0x7efe100c4730 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe1546d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c2340 con 0x7efe100c42f0 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe1546d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c3860 con 0x7efe101516d0 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe100c4730 0x7efe10144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe100c4730 0x7efe10144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57636/0 (socket says 172.21.15.39:57636) 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0e7fc640 1 -- 172.21.15.39:0/3918945922 learned_addr learned my addr 172.21.15.39:0/3918945922 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:59.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0effd640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe100c42f0 0x7efe100c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0f7fe640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe101516d0 0x7efe10144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0e7fc640 1 -- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe101516d0 msgr2=0x7efe10144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0e7fc640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe101516d0 0x7efe10144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0e7fc640 1 -- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe100c42f0 msgr2=0x7efe100c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0e7fc640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe100c42f0 0x7efe100c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0e7fc640 1 -- 172.21.15.39:0/3918945922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe00009a70 con 0x7efe100c4730 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0effd640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe100c42f0 0x7efe100c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0f7fe640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe101516d0 0x7efe10144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0e7fc640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe100c4730 0x7efe10144910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c450 tx=0x7efdf800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe077fe640 1 -- 172.21.15.39:0/3918945922 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8005cf0 con 0x7efe100c4730 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe077fe640 1 -- 172.21.15.39:0/3918945922 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf8005e50 con 0x7efe100c4730 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe077fe640 1 -- 172.21.15.39:0/3918945922 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8007270 con 0x7efe100c4730 2024-09-13T16:52:59.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe10158010 con 0x7efe100c4730 2024-09-13T16:52:59.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe100c1860 con 0x7efe100c4730 2024-09-13T16:52:59.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe100c7bb0 con 0x7efe100c4730 2024-09-13T16:52:59.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe077fe640 1 -- 172.21.15.39:0/3918945922 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efdf80073d0 con 0x7efe100c4730 2024-09-13T16:52:59.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe077fe640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdf00666c0 0x7efdf0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe077fe640 1 -- 172.21.15.39:0/3918945922 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7efdf8083a80 con 0x7efe100c4730 2024-09-13T16:52:59.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0effd640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdf00666c0 0x7efdf0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.247+0000 7efe0effd640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdf00666c0 0x7efdf0068b80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efe000af000 tx=0x7efe000a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.251+0000 7efe077fe640 1 -- 172.21.15.39:0/3918945922 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efdf80509e0 con 0x7efe100c4730 2024-09-13T16:52:59.355 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe100c7870 con 0x7efe100c4730 2024-09-13T16:52:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe077fe640 1 -- 172.21.15.39:0/3918945922 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5933 (secure 0 0 0) 0x7efdf8017090 con 0x7efe100c4730 2024-09-13T16:52:59.451 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:59.452 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:50:52.769334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.84:6835/1969975539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1969975539},{"type":"v1","addr":"172.21.15.84:6835","nonce":1969975539}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 40 2024-09-13T16:52:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdf00666c0 msgr2=0x7efdf0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdf00666c0 0x7efdf0068b80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efe000af000 tx=0x7efe000a7070 comp rx=0 tx=0).stop 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe100c4730 msgr2=0x7efe10144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe100c4730 0x7efe10144910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c450 tx=0x7efdf800c910 comp rx=0 tx=0).stop 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 shutdown_connections 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdf00666c0 0x7efdf0068b80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efe101516d0 0x7efe10144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe100c4730 0x7efe10144910 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 --2- 172.21.15.39:0/3918945922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe100c42f0 0x7efe100c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 >> 172.21.15.39:0/3918945922 conn(0x7efe100ab500 msgr2=0x7efe100c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 shutdown_connections 2024-09-13T16:52:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.447+0000 7efe1546d640 1 -- 172.21.15.39:0/3918945922 wait complete. 2024-09-13T16:52:59.470 INFO:tasks.mds_thrash.fs.[b]:kill mds.a (rank=0) 2024-09-13T16:52:59.470 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:52:59.470 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:52:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f765a7a8640 1 Processor -- start 2024-09-13T16:52:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f765a7a8640 1 -- start start 2024-09-13T16:52:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f765a7a8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f76540bf4f0 0x7f7654153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f765a7a8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f76540611c0 0x7f76540bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f765a7a8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76540c1340 con 0x7f7654151860 2024-09-13T16:52:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f765a7a8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76541578f0 con 0x7f76540bf4f0 2024-09-13T16:52:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f765a7a8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76540c1590 con 0x7f76540611c0 2024-09-13T16:52:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f76537fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f76540bf4f0 0x7f7654153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f76537fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f76540bf4f0 0x7f7654153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57638/0 (socket says 172.21.15.39:57638) 2024-09-13T16:52:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f76537fe640 1 -- 172.21.15.39:0/1299926725 learned_addr learned my addr 172.21.15.39:0/1299926725 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f7658d45640 1 --2- 172.21.15.39:0/1299926725 >> v2:172.21.15.84:3300/0 conn(0x7f76540611c0 0x7f76540bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f76537fe640 1 -- 172.21.15.39:0/1299926725 >> v2:172.21.15.84:3300/0 conn(0x7f76540611c0 msgr2=0x7f76540bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f76537fe640 1 --2- 172.21.15.39:0/1299926725 >> v2:172.21.15.84:3300/0 conn(0x7f76540611c0 0x7f76540bfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f76537fe640 1 -- 172.21.15.39:0/1299926725 >> v1:172.21.15.84:6789/0 conn(0x7f7654151860 legacy=0x7f7654157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.471+0000 7f76537fe640 1 -- 172.21.15.39:0/1299926725 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540c1870 con 0x7f76540bf4f0 2024-09-13T16:52:59.480 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:52:59.481 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f76537fe640 1 --2- 172.21.15.39:0/1299926725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 0x7f7654153df0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f76540c0b50 tx=0x7f764801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a391f2def783be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f7652ffd640 1 -- 172.21.15.39:0/1299926725 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480ab620 con 0x7f76540bf4f0 2024-09-13T16:52:59.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f7652ffd640 1 -- 172.21.15.39:0/1299926725 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76480ab780 con 0x7f76540bf4f0 2024-09-13T16:52:59.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f7652ffd640 1 -- 172.21.15.39:0/1299926725 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480abb10 con 0x7f76540bf4f0 2024-09-13T16:52:59.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 -- 172.21.15.39:0/1299926725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 msgr2=0x7f7654153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 --2- 172.21.15.39:0/1299926725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 0x7f7654153df0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f76540c0b50 tx=0x7f764801c420 comp rx=0 tx=0).stop 2024-09-13T16:52:59.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 -- 172.21.15.39:0/1299926725 shutdown_connections 2024-09-13T16:52:59.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 --2- 172.21.15.39:0/1299926725 >> v2:172.21.15.84:3300/0 conn(0x7f76540611c0 0x7f76540bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 --2- 172.21.15.39:0/1299926725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 0x7f7654153df0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 -- 172.21.15.39:0/1299926725 >> 172.21.15.39:0/1299926725 conn(0x7f7654069e10 msgr2=0x7f765406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:59.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 -- 172.21.15.39:0/1299926725 shutdown_connections 2024-09-13T16:52:59.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 -- 172.21.15.39:0/1299926725 wait complete. 2024-09-13T16:52:59.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 Processor -- start 2024-09-13T16:52:59.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.475+0000 7f765a7a8640 1 -- start start 2024-09-13T16:52:59.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f765a7a8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76540611c0 0x7f765414cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f765a7a8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 0x7f7654070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f765a7a8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7654151860 0x7f76540706a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f765a7a8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76540c5c20 con 0x7f76540bf4f0 2024-09-13T16:52:59.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f765a7a8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7654070be0 con 0x7f76540611c0 2024-09-13T16:52:59.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f765a7a8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7654071570 con 0x7f7654151860 2024-09-13T16:52:59.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f76537fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 0x7f7654070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f76537fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 0x7f7654070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57644/0 (socket says 172.21.15.39:57644) 2024-09-13T16:52:59.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f76537fe640 1 -- 172.21.15.39:0/3402305809 learned_addr learned my addr 172.21.15.39:0/3402305809 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:59.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7653fff640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76540611c0 0x7f765414cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7658d45640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7654151860 0x7f76540706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f76537fe640 1 -- 172.21.15.39:0/3402305809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7654151860 msgr2=0x7f76540706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f76537fe640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7654151860 0x7f76540706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f76537fe640 1 -- 172.21.15.39:0/3402305809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76540611c0 msgr2=0x7f765414cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f76537fe640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76540611c0 0x7f765414cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f76537fe640 1 -- 172.21.15.39:0/3402305809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480094d0 con 0x7f76540bf4f0 2024-09-13T16:52:59.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7658d45640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7654151860 0x7f76540706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:52:59.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f76537fe640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 0x7f7654070160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f764801a050 tx=0x7f764801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7650ff9640 1 -- 172.21.15.39:0/3402305809 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480be3f0 con 0x7f76540bf4f0 2024-09-13T16:52:59.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7650ff9640 1 -- 172.21.15.39:0/3402305809 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76480be570 con 0x7f76540bf4f0 2024-09-13T16:52:59.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7654072030 con 0x7f76540bf4f0 2024-09-13T16:52:59.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7650ff9640 1 -- 172.21.15.39:0/3402305809 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480be920 con 0x7f76540bf4f0 2024-09-13T16:52:59.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7654071830 con 0x7f76540bf4f0 2024-09-13T16:52:59.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76540bfad0 con 0x7f76540bf4f0 2024-09-13T16:52:59.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7650ff9640 1 -- 172.21.15.39:0/3402305809 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f76480b43f0 con 0x7f76540bf4f0 2024-09-13T16:52:59.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7650ff9640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7640066690 0x7f7640068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7650ff9640 1 -- 172.21.15.39:0/3402305809 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f764811e3e0 con 0x7f76540bf4f0 2024-09-13T16:52:59.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7653fff640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7640066690 0x7f7640068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.479+0000 7f7653fff640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7640066690 0x7f7640068b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f76440016c0 tx=0x7f7644009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.487+0000 7f7650ff9640 1 -- 172.21.15.39:0/3402305809 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f76480eb200 con 0x7f76540bf4f0 2024-09-13T16:52:59.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.587+0000 7f1e381bf640 1 Processor -- start 2024-09-13T16:52:59.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.587+0000 7f1e381bf640 1 -- start start 2024-09-13T16:52:59.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e381bf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e300c7960 0x7f1e30155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e381bf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e3006fa50 0x7f1e30155f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e381bf640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e300c5e50 con 0x7f1e300c6000 2024-09-13T16:52:59.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e381bf640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3006ff50 con 0x7f1e300c7960 2024-09-13T16:52:59.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e381bf640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e30072000 con 0x7f1e3006fa50 2024-09-13T16:52:59.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e35f5b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e300c7960 0x7f1e30155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e35f5b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e300c7960 0x7f1e30155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57660/0 (socket says 172.21.15.39:57660) 2024-09-13T16:52:59.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e3675c640 1 --1- 172.21.15.39:0/1234526296 >> v1:172.21.15.39:6789/0 conn(0x7f1e300c6000 0x7f1e300c2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49648/0 (socket says 172.21.15.39:49648) 2024-09-13T16:52:59.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e35f5b640 1 -- 172.21.15.39:0/1234526296 learned_addr learned my addr 172.21.15.39:0/1234526296 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:59.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e35f5b640 1 -- 172.21.15.39:0/1234526296 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e3006fa50 msgr2=0x7f1e30155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:59.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e35f5b640 1 --2- 172.21.15.39:0/1234526296 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e3006fa50 0x7f1e30155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e35f5b640 1 -- 172.21.15.39:0/1234526296 >> v1:172.21.15.39:6789/0 conn(0x7f1e300c6000 legacy=0x7f1e300c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e35f5b640 1 -- 172.21.15.39:0/1234526296 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300722e0 con 0x7f1e300c7960 2024-09-13T16:52:59.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.591+0000 7f1e35f5b640 1 --2- 172.21.15.39:0/1234526296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e300c7960 0x7f1e30155a20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009e50 tx=0x7f1e2001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2b8fe4c33d9e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e34f59640 1 -- 172.21.15.39:0/1234526296 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200a5830 con 0x7f1e300c7960 2024-09-13T16:52:59.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e34f59640 1 -- 172.21.15.39:0/1234526296 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e200a5990 con 0x7f1e300c7960 2024-09-13T16:52:59.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e34f59640 1 -- 172.21.15.39:0/1234526296 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200a5d20 con 0x7f1e300c7960 2024-09-13T16:52:59.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 -- 172.21.15.39:0/1234526296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e300c7960 msgr2=0x7f1e30155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 --2- 172.21.15.39:0/1234526296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e300c7960 0x7f1e30155a20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009e50 tx=0x7f1e2001c970 comp rx=0 tx=0).stop 2024-09-13T16:52:59.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 -- 172.21.15.39:0/1234526296 shutdown_connections 2024-09-13T16:52:59.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 --2- 172.21.15.39:0/1234526296 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e3006fa50 0x7f1e30155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 --2- 172.21.15.39:0/1234526296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e300c7960 0x7f1e30155a20 secure :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009e50 tx=0x7f1e2001c970 comp rx=0 tx=0).stop 2024-09-13T16:52:59.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 -- 172.21.15.39:0/1234526296 >> 172.21.15.39:0/1234526296 conn(0x7f1e3006c020 msgr2=0x7f1e3006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:59.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 -- 172.21.15.39:0/1234526296 shutdown_connections 2024-09-13T16:52:59.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 -- 172.21.15.39:0/1234526296 wait complete. 2024-09-13T16:52:59.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 Processor -- start 2024-09-13T16:52:59.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.595+0000 7f1e381bf640 1 -- start start 2024-09-13T16:52:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e381bf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3006fa50 0x7f1e301f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e381bf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e300c6000 0x7f1e3014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e381bf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e30150320 0x7f1e300bd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e381bf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3014eb20 con 0x7f1e3006fa50 2024-09-13T16:52:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e381bf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3014d5c0 con 0x7f1e300c6000 2024-09-13T16:52:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e381bf640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3014d8a0 con 0x7f1e30150320 2024-09-13T16:52:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e35f5b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3006fa50 0x7f1e301f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e35f5b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3006fa50 0x7f1e301f06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57676/0 (socket says 172.21.15.39:57676) 2024-09-13T16:52:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e35f5b640 1 -- 172.21.15.39:0/2391628975 learned_addr learned my addr 172.21.15.39:0/2391628975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e3675c640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e30150320 0x7f1e300bd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e3575a640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e300c6000 0x7f1e3014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e35f5b640 1 -- 172.21.15.39:0/2391628975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e30150320 msgr2=0x7f1e300bd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e35f5b640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e30150320 0x7f1e300bd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e35f5b640 1 -- 172.21.15.39:0/2391628975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e300c6000 msgr2=0x7f1e3014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e35f5b640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e300c6000 0x7f1e3014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e35f5b640 1 -- 172.21.15.39:0/2391628975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e20009b30 con 0x7f1e3006fa50 2024-09-13T16:52:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e35f5b640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3006fa50 0x7f1e301f06a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e2001b020 tx=0x7f1e20019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e2e7fc640 1 -- 172.21.15.39:0/2391628975 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200b8880 con 0x7f1e3006fa50 2024-09-13T16:52:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e2e7fc640 1 -- 172.21.15.39:0/2391628975 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e200b89e0 con 0x7f1e3006fa50 2024-09-13T16:52:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e301efbc0 con 0x7f1e3006fa50 2024-09-13T16:52:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e2e7fc640 1 -- 172.21.15.39:0/2391628975 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200b8d50 con 0x7f1e3006fa50 2024-09-13T16:52:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.599+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e301f0180 con 0x7f1e3006fa50 2024-09-13T16:52:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.603+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e301f0370 con 0x7f1e3006fa50 2024-09-13T16:52:59.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.603+0000 7f1e2e7fc640 1 -- 172.21.15.39:0/2391628975 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e200039a0 con 0x7f1e3006fa50 2024-09-13T16:52:59.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.603+0000 7f1e2e7fc640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e1c0666c0 0x7f1e1c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.603+0000 7f1e2e7fc640 1 -- 172.21.15.39:0/2391628975 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7f1e2011f920 con 0x7f1e3006fa50 2024-09-13T16:52:59.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.603+0000 7f1e3575a640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e1c0666c0 0x7f1e1c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.603+0000 7f1e3575a640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e1c0666c0 0x7f1e1c068b80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e24006fd0 tx=0x7f1e24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.607+0000 7f1e2e7fc640 1 -- 172.21.15.39:0/2391628975 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1e200ec770 con 0x7f1e3006fa50 2024-09-13T16:52:59.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.691+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76540c0f10 con 0x7f76540bf4f0 2024-09-13T16:52:59.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f7650ff9640 1 -- 172.21.15.39:0/3402305809 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f76480ee820 con 0x7f76540bf4f0 2024-09-13T16:52:59.699 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:59.699 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:52:59.591101+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 41 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7640066690 msgr2=0x7f7640068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7640066690 0x7f7640068b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f76440016c0 tx=0x7f7644009040 comp rx=0 tx=0).stop 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 msgr2=0x7f7654070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 0x7f7654070160 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f764801a050 tx=0x7f764801ac60 comp rx=0 tx=0).stop 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 shutdown_connections 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7640066690 0x7f7640068b50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7654151860 0x7f76540706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76540bf4f0 0x7f7654070160 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 --2- 172.21.15.39:0/3402305809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76540611c0 0x7f765414cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 >> 172.21.15.39:0/3402305809 conn(0x7f7654069e10 msgr2=0x7f76540c2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 shutdown_connections 2024-09-13T16:52:59.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.695+0000 7f765a7a8640 1 -- 172.21.15.39:0/3402305809 wait complete. 2024-09-13T16:52:59.716 INFO:tasks.mds_thrash.fs.[a]:mds.d down, removed from mdsmap 2024-09-13T16:52:59.716 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T16:52:59.717 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:52:59.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e300c2570 con 0x7f1e3006fa50 2024-09-13T16:52:59.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e2e7fc640 1 -- 172.21.15.39:0/2391628975 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f1e200efd90 con 0x7f1e3006fa50 2024-09-13T16:52:59.813 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:52:59.813 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:52:59.591101+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:52:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 41 2024-09-13T16:52:59.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e1c0666c0 msgr2=0x7f1e1c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e1c0666c0 0x7f1e1c068b80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e24006fd0 tx=0x7f1e24008040 comp rx=0 tx=0).stop 2024-09-13T16:52:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3006fa50 msgr2=0x7f1e301f06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3006fa50 0x7f1e301f06a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e2001b020 tx=0x7f1e20019710 comp rx=0 tx=0).stop 2024-09-13T16:52:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 shutdown_connections 2024-09-13T16:52:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e1c0666c0 0x7f1e1c068b80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e300c6000 0x7f1e3014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e30150320 0x7f1e300bd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 --2- 172.21.15.39:0/2391628975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3006fa50 0x7f1e301f06a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 >> 172.21.15.39:0/2391628975 conn(0x7f1e3006c020 msgr2=0x7f1e3006ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 shutdown_connections 2024-09-13T16:52:59.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.807+0000 7f1e381bf640 1 -- 172.21.15.39:0/2391628975 wait complete. 2024-09-13T16:52:59.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a93c2640 1 Processor -- start 2024-09-13T16:52:59.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a93c2640 1 -- start start 2024-09-13T16:52:59.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a93c2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f80a414f1b0 0x7f80a4151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a93c2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 0x7f80a40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a93c2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a414f590 con 0x7f80a406e5e0 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a93c2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a406f660 con 0x7f80a414f1b0 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a93c2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a406eae0 con 0x7f80a40c7470 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a27fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f80a406e5e0 0x7f80a40714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49656/0 (socket says 172.21.15.39:49656) 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a27fc640 1 -- 172.21.15.39:0/955033020 learned_addr learned my addr 172.21.15.39:0/955033020 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a37fe640 1 --2- 172.21.15.39:0/955033020 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 0x7f80a40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a2ffd640 1 --2- 172.21.15.39:0/955033020 >> v2:172.21.15.84:3300/0 conn(0x7f80a414f1b0 0x7f80a4151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a1ffb640 1 -- 172.21.15.39:0/955033020 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921203851 0 0) 0x7f80a414f590 con 0x7f80a406e5e0 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a1ffb640 1 -- 172.21.15.39:0/955033020 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f808c003530 con 0x7f80a406e5e0 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a2ffd640 1 -- 172.21.15.39:0/955033020 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 msgr2=0x7f80a40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a2ffd640 1 --2- 172.21.15.39:0/955033020 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 0x7f80a40c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a2ffd640 1 -- 172.21.15.39:0/955033020 >> v1:172.21.15.39:6789/0 conn(0x7f80a406e5e0 legacy=0x7f80a40714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a2ffd640 1 -- 172.21.15.39:0/955033020 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a406edc0 con 0x7f80a414f1b0 2024-09-13T16:52:59.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.819+0000 7f80a37fe640 1 --2- 172.21.15.39:0/955033020 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 0x7f80a40c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:52:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a2ffd640 1 --2- 172.21.15.39:0/955033020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a414f1b0 0x7f80a4151bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f80940097c0 tx=0x7f809401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ec42a2b067c808 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a1ffb640 1 -- 172.21.15.39:0/955033020 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a40c69a0 con 0x7f80a414f1b0 2024-09-13T16:52:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a1ffb640 1 -- 172.21.15.39:0/955033020 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80940ab670 con 0x7f80a414f1b0 2024-09-13T16:52:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a1ffb640 1 -- 172.21.15.39:0/955033020 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80940aba00 con 0x7f80a414f1b0 2024-09-13T16:52:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a93c2640 1 -- 172.21.15.39:0/955033020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a414f1b0 msgr2=0x7f80a4151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a93c2640 1 --2- 172.21.15.39:0/955033020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a414f1b0 0x7f80a4151bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f80940097c0 tx=0x7f809401c470 comp rx=0 tx=0).stop 2024-09-13T16:52:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a93c2640 1 -- 172.21.15.39:0/955033020 shutdown_connections 2024-09-13T16:52:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a93c2640 1 --2- 172.21.15.39:0/955033020 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 0x7f80a40c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a93c2640 1 --2- 172.21.15.39:0/955033020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a414f1b0 0x7f80a4151bb0 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f80940097c0 tx=0x7f809401c470 comp rx=0 tx=0).stop 2024-09-13T16:52:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a93c2640 1 -- 172.21.15.39:0/955033020 >> 172.21.15.39:0/955033020 conn(0x7f80a40a9720 msgr2=0x7f80a40a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:52:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a93c2640 1 -- 172.21.15.39:0/955033020 shutdown_connections 2024-09-13T16:52:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a93c2640 1 -- 172.21.15.39:0/955033020 wait complete. 2024-09-13T16:52:59.830 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:52:59.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.823+0000 7f80a93c2640 1 Processor -- start 2024-09-13T16:52:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 -- start start 2024-09-13T16:52:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a406e5e0 0x7f80a415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 0x7f80a40ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80a40bc990 0x7f80a40ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40bd6a0 con 0x7f80a40bc990 2024-09-13T16:52:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a41ecd10 con 0x7f80a406e5e0 2024-09-13T16:52:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a414cae0 con 0x7f80a40c7470 2024-09-13T16:52:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80a40bc990 0x7f80a40ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a406e5e0 0x7f80a415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80a40bc990 0x7f80a40ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57692/0 (socket says 172.21.15.39:57692) 2024-09-13T16:52:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a37fe640 1 -- 172.21.15.39:0/3209969134 learned_addr learned my addr 172.21.15.39:0/3209969134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:52:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a27fc640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 0x7f80a40ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a37fe640 1 -- 172.21.15.39:0/3209969134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 msgr2=0x7f80a40ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a37fe640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 0x7f80a40ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a37fe640 1 -- 172.21.15.39:0/3209969134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a406e5e0 msgr2=0x7f80a415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:52:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a37fe640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a406e5e0 0x7f80a415a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:52:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a37fe640 1 -- 172.21.15.39:0/3209969134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c009a70 con 0x7f80a40bc990 2024-09-13T16:52:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a37fe640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80a40bc990 0x7f80a40ad7b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f809c00efc0 tx=0x7f809c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f808b7fe640 1 -- 172.21.15.39:0/3209969134 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c009cd0 con 0x7f80a40bc990 2024-09-13T16:52:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f808b7fe640 1 -- 172.21.15.39:0/3209969134 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c00f040 con 0x7f80a40bc990 2024-09-13T16:52:59.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8094009420 con 0x7f80a40bc990 2024-09-13T16:52:59.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f808b7fe640 1 -- 172.21.15.39:0/3209969134 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c0039a0 con 0x7f80a40bc990 2024-09-13T16:52:59.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a406ba50 con 0x7f80a40bc990 2024-09-13T16:52:59.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a40c31d0 con 0x7f80a40bc990 2024-09-13T16:52:59.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f808b7fe640 1 -- 172.21.15.39:0/3209969134 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f809c014480 con 0x7f80a40bc990 2024-09-13T16:52:59.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f808b7fe640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8090066690 0x7f8090068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:52:59.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f808b7fe640 1 -- 172.21.15.39:0/3209969134 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7f809c085170 con 0x7f80a40bc990 2024-09-13T16:52:59.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a2ffd640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8090066690 0x7f8090068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:52:59.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.827+0000 7f80a2ffd640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8090066690 0x7f8090068b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f80940af000 tx=0x7f80940a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:52:59.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:52:59.831+0000 7f808b7fe640 1 -- 172.21.15.39:0/3209969134 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f809c052050 con 0x7f80a40bc990 2024-09-13T16:53:00.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.027+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80a415c1c0 con 0x7f80a40bc990 2024-09-13T16:53:00.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.027+0000 7f808b7fe640 1 -- 172.21.15.39:0/3209969134 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f809c055670 con 0x7f80a40bc990 2024-09-13T16:53:00.034 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:00.034 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:52:59.591101+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:00.034 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 41 2024-09-13T16:53:00.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8090066690 msgr2=0x7f8090068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8090066690 0x7f8090068b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f80940af000 tx=0x7f80940a7000 comp rx=0 tx=0).stop 2024-09-13T16:53:00.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80a40bc990 msgr2=0x7f80a40ad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:00.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80a40bc990 0x7f80a40ad7b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f809c00efc0 tx=0x7f809c00c490 comp rx=0 tx=0).stop 2024-09-13T16:53:00.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 shutdown_connections 2024-09-13T16:53:00.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8090066690 0x7f8090068b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80a40bc990 0x7f80a40ad7b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80a40c7470 0x7f80a40ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 --2- 172.21.15.39:0/3209969134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80a406e5e0 0x7f80a415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:00.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 >> 172.21.15.39:0/3209969134 conn(0x7f80a40a9720 msgr2=0x7f80a40a9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:00.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 shutdown_connections 2024-09-13T16:53:00.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:00.031+0000 7f80a93c2640 1 -- 172.21.15.39:0/3209969134 wait complete. 2024-09-13T16:53:00.053 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 41, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 41, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_4922": { "addr": "172.21.15.84:6833/1355376480", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 1355376480, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 1355376480, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4922, "incarnation": 41, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 32, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:52:59.591101+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4922 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 39, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5418": { "addr": "172.21.15.39:6835/2065196144", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 2065196144, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 2065196144, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5418, "incarnation": 36, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 31, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:52:27.811143+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5418 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:53:01.831 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:01.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa511685640 1 Processor -- start 2024-09-13T16:53:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa511685640 1 -- start start 2024-09-13T16:53:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa511685640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa50c0aabc0 0x7fa50c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa511685640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 0x7fa50c0c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa511685640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50c152a80 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa511685640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50c0c30f0 con 0x7fa50c0aabc0 2024-09-13T16:53:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa511685640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c151e00 con 0x7fa50c06c870 2024-09-13T16:53:01.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa50affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa50c0aabc0 0x7fa50c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa50a7fc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 0x7fa50c0c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa50affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa50c0aabc0 0x7fa50c0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:39362/0 (socket says 172.21.15.39:39362) 2024-09-13T16:53:01.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.963+0000 7fa50affd640 1 -- 172.21.15.39:0/68443969 learned_addr learned my addr 172.21.15.39:0/68443969 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:01.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa50a7fc640 1 -- 172.21.15.39:0/68443969 >> v2:172.21.15.84:3300/0 conn(0x7fa50c0aabc0 msgr2=0x7fa50c0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:01.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa50a7fc640 1 --2- 172.21.15.39:0/68443969 >> v2:172.21.15.84:3300/0 conn(0x7fa50c0aabc0 0x7fa50c0aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa50a7fc640 1 -- 172.21.15.39:0/68443969 >> v1:172.21.15.84:6789/0 conn(0x7fa50c0c3ee0 legacy=0x7fa50c1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa50a7fc640 1 -- 172.21.15.39:0/68443969 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50c1520e0 con 0x7fa50c06c870 2024-09-13T16:53:01.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa50affd640 1 --2- 172.21.15.39:0/68443969 >> v2:172.21.15.84:3300/0 conn(0x7fa50c0aabc0 0x7fa50c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:53:01.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa50a7fc640 1 --2- 172.21.15.39:0/68443969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 0x7fa50c0c39a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009d90 tx=0x7fa4fc01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d84f9752c174a55d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa509ffb640 1 -- 172.21.15.39:0/68443969 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0a5910 con 0x7fa50c06c870 2024-09-13T16:53:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa509ffb640 1 -- 172.21.15.39:0/68443969 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc0a5a70 con 0x7fa50c06c870 2024-09-13T16:53:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa509ffb640 1 -- 172.21.15.39:0/68443969 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0a5e00 con 0x7fa50c06c870 2024-09-13T16:53:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa511685640 1 -- 172.21.15.39:0/68443969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 msgr2=0x7fa50c0c39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa511685640 1 --2- 172.21.15.39:0/68443969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 0x7fa50c0c39a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009d90 tx=0x7fa4fc01c940 comp rx=0 tx=0).stop 2024-09-13T16:53:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa511685640 1 -- 172.21.15.39:0/68443969 shutdown_connections 2024-09-13T16:53:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa511685640 1 --2- 172.21.15.39:0/68443969 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 0x7fa50c0c39a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.967+0000 7fa511685640 1 --2- 172.21.15.39:0/68443969 >> v2:172.21.15.84:3300/0 conn(0x7fa50c0aabc0 0x7fa50c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 -- 172.21.15.39:0/68443969 >> 172.21.15.39:0/68443969 conn(0x7fa50c069e10 msgr2=0x7fa50c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 -- 172.21.15.39:0/68443969 shutdown_connections 2024-09-13T16:53:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 -- 172.21.15.39:0/68443969 wait complete. 2024-09-13T16:53:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 Processor -- start 2024-09-13T16:53:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 -- start start 2024-09-13T16:53:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 0x7fa50c071c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa50c0aabc0 0x7fa50c14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa50c0c3ee0 0x7fa50c1ebd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c1eb340 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c1eb870 con 0x7fa50c0aabc0 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c1eb590 con 0x7fa50c06c870 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa50c0c3ee0 0x7fa50c1ebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa50c0c3ee0 0x7fa50c1ebd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37130/0 (socket says 172.21.15.39:37130) 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50b7fe640 1 -- 172.21.15.39:0/3061630250 learned_addr learned my addr 172.21.15.39:0/3061630250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50a7fc640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa50c0aabc0 0x7fa50c14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50affd640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 0x7fa50c071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50b7fe640 1 -- 172.21.15.39:0/3061630250 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 msgr2=0x7fa50c071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50b7fe640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 0x7fa50c071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50b7fe640 1 -- 172.21.15.39:0/3061630250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa50c0aabc0 msgr2=0x7fa50c14ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50b7fe640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa50c0aabc0 0x7fa50c14ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50b7fe640 1 -- 172.21.15.39:0/3061630250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f40094d0 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa50b7fe640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa50c0c3ee0 0x7fa50c1ebd30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa50400c390 tx=0x7fa50400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa5037fe640 1 -- 172.21.15.39:0/3061630250 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5040052d0 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa5037fe640 1 -- 172.21.15.39:0/3061630250 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa504005430 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc009a70 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa5037fe640 1 -- 172.21.15.39:0/3061630250 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504007240 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.971+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c0bb4b0 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.975+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c0719b0 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.975+0000 7fa5037fe640 1 -- 172.21.15.39:0/3061630250 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa5040073a0 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.975+0000 7fa5037fe640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4ec066690 0x7fa4ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.975+0000 7fa5037fe640 1 -- 172.21.15.39:0/3061630250 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7fa5040838c0 con 0x7fa50c0c3ee0 2024-09-13T16:53:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.975+0000 7fa50affd640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4ec066690 0x7fa4ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.975+0000 7fa50affd640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4ec066690 0x7fa4ec068b50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009c70 tx=0x7fa4f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:01.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:01.979+0000 7fa5037fe640 1 -- 172.21.15.39:0/3061630250 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa50c0719b0 con 0x7fa50c0c3ee0 2024-09-13T16:53:02.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:02.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f179339b640 1 Processor -- start 2024-09-13T16:53:02.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f179339b640 1 -- start start 2024-09-13T16:53:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f179339b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f178c151500 0x7f178c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f179339b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f178c071dd0 0x7f178c154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f179339b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c0c5a30 con 0x7f178c0c47c0 2024-09-13T16:53:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f179339b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c1576a0 con 0x7f178c071dd0 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f179339b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c157980 con 0x7f178c151500 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1791938640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f178c0c47c0 0x7f178c1545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53726/0 (socket says 172.21.15.39:53726) 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1791938640 1 -- 172.21.15.39:0/3331074860 learned_addr learned my addr 172.21.15.39:0/3331074860 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1790936640 1 --2- 172.21.15.39:0/3331074860 >> v2:172.21.15.84:3300/0 conn(0x7f178c071dd0 0x7f178c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1791137640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f178c151500 0x7f178c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487619150 0 0) 0x7f178c0c5a30 con 0x7f178c0c47c0 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1778003530 con 0x7f178c0c47c0 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1360929947 0 0) 0x7f1778003530 con 0x7f178c0c47c0 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f178c151500 msgr2=0x7f178c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f178c151500 0x7f178c0c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 >> v2:172.21.15.84:3300/0 conn(0x7f178c071dd0 msgr2=0x7f178c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:02.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 --2- 172.21.15.39:0/3331074860 >> v2:172.21.15.84:3300/0 conn(0x7f178c071dd0 0x7f178c154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c157c60 con 0x7f178c0c47c0 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1784001a00 con 0x7f178c0c47c0 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 >> v1:172.21.15.39:6789/0 conn(0x7f178c0c47c0 legacy=0x7f178c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f178c151500 0x7f1778003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1778003f00 0x7f17780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1778004d10 0x7f1778005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c1576a0 con 0x7f1778003f00 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c157980 con 0x7f1778004d10 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177800b9e0 con 0x7f178c151500 2024-09-13T16:53:02.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1790936640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1778003f00 0x7f17780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1791938640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1778004d10 0x7f1778005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1790936640 1 -- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f178c151500 msgr2=0x7f1778003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1791137640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f178c151500 0x7f1778003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1790936640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f178c151500 0x7f1778003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1790936640 1 -- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1778004d10 msgr2=0x7f1778005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1790936640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1778004d10 0x7f1778005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1790936640 1 -- 172.21.15.39:0/3331074860 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1778003530 con 0x7f1778003f00 2024-09-13T16:53:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1791938640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1778004d10 0x7f1778005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:02.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.171+0000 7f1791137640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f178c151500 0x7f1778003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:02.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f1790936640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1778003f00 0x7f17780047d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f177c009ae0 tx=0x7f177c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198b098988127a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:02.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0a5630 con 0x7f1778003f00 2024-09-13T16:53:02.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c0a5790 con 0x7f1778003f00 2024-09-13T16:53:02.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f1783fff640 1 -- 172.21.15.39:0/3331074860 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0a5b40 con 0x7f1778003f00 2024-09-13T16:53:02.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 -- 172.21.15.39:0/3331074860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1778003f00 msgr2=0x7f17780047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 --2- 172.21.15.39:0/3331074860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1778003f00 0x7f17780047d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f177c009ae0 tx=0x7f177c01ca40 comp rx=0 tx=0).stop 2024-09-13T16:53:02.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f1791938640 1 -- 172.21.15.39:0/3331074860 reap_dead start 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 -- 172.21.15.39:0/3331074860 shutdown_connections 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 -- 172.21.15.39:0/3331074860 >> 172.21.15.39:0/3331074860 conn(0x7f178c0ab530 msgr2=0x7f178c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 -- 172.21.15.39:0/3331074860 shutdown_connections 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 -- 172.21.15.39:0/3331074860 wait complete. 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 Processor -- start 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 -- start start 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1778003f00 0x7f178c14db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f178c153890 0x7f178c1efff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c1f0530 0x7f178c151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c1ec7a0 con 0x7f178c1f0530 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c15d370 con 0x7f178c153890 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.175+0000 7f179339b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c15a6c0 con 0x7f1778003f00 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791938640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c1f0530 0x7f178c151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791938640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c1f0530 0x7f178c151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37148/0 (socket says 172.21.15.39:37148) 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791938640 1 -- 172.21.15.39:0/3830723833 learned_addr learned my addr 172.21.15.39:0/3830723833 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1790936640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f178c153890 0x7f178c1efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791137640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1778003f00 0x7f178c14db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791938640 1 -- 172.21.15.39:0/3830723833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1778003f00 msgr2=0x7f178c14db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791938640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1778003f00 0x7f178c14db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791938640 1 -- 172.21.15.39:0/3830723833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f178c153890 msgr2=0x7f178c1efff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791938640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f178c153890 0x7f178c1efff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791938640 1 -- 172.21.15.39:0/3830723833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c15a9a0 con 0x7f178c1f0530 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1790936640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f178c153890 0x7f178c1efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791938640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c1f0530 0x7f178c151bd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1784001340 tx=0x7f178400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:02.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa50c152a80 con 0x7fa50c0c3ee0 2024-09-13T16:53:02.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1781ffb640 1 -- 172.21.15.39:0/3830723833 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784004d70 con 0x7f178c1f0530 2024-09-13T16:53:02.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1781ffb640 1 -- 172.21.15.39:0/3830723833 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178400bb80 con 0x7f178c1f0530 2024-09-13T16:53:02.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c15ac80 con 0x7f178c1f0530 2024-09-13T16:53:02.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1781ffb640 1 -- 172.21.15.39:0/3830723833 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784007110 con 0x7f178c1f0530 2024-09-13T16:53:02.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c152c70 con 0x7f178c1f0530 2024-09-13T16:53:02.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7fa5037fe640 1 -- 172.21.15.39:0/3061630250 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5175 (secure 0 0 0) 0x7fa50c152a80 con 0x7fa50c0c3ee0 2024-09-13T16:53:02.189 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:02.189 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:01.612378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:rejoin","state_seq":9,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:02.189 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 43 2024-09-13T16:53:02.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c15ae10 con 0x7f178c1f0530 2024-09-13T16:53:02.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1781ffb640 1 -- 172.21.15.39:0/3830723833 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1784007270 con 0x7f178c1f0530 2024-09-13T16:53:02.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1781ffb640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1770066690 0x7f1770068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:02.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1781ffb640 1 -- 172.21.15.39:0/3830723833 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7f17840850f0 con 0x7f178c1f0530 2024-09-13T16:53:02.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791137640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1770066690 0x7f1770068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:02.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4ec066690 msgr2=0x7fa4ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7fa511685640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4ec066690 0x7fa4ec068b50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009c70 tx=0x7fa4f4008040 comp rx=0 tx=0).stop 2024-09-13T16:53:02.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa50c0c3ee0 msgr2=0x7fa50c1ebd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7fa511685640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa50c0c3ee0 0x7fa50c1ebd30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa50400c390 tx=0x7fa50400c850 comp rx=0 tx=0).stop 2024-09-13T16:53:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.179+0000 7f1791137640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1770066690 0x7f1770068b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f17880099e0 tx=0x7f1788008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:02.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.183+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 shutdown_connections 2024-09-13T16:53:02.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.183+0000 7fa511685640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4ec066690 0x7fa4ec068b50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.183+0000 7fa511685640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa50c0aabc0 0x7fa50c14ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.183+0000 7fa511685640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa50c0c3ee0 0x7fa50c1ebd30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.183+0000 7fa511685640 1 --2- 172.21.15.39:0/3061630250 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa50c06c870 0x7fa50c071c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.183+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 >> 172.21.15.39:0/3061630250 conn(0x7fa50c069e10 msgr2=0x7fa50c1550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:02.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.183+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 shutdown_connections 2024-09-13T16:53:02.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.183+0000 7fa511685640 1 -- 172.21.15.39:0/3061630250 wait complete. 2024-09-13T16:53:02.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.183+0000 7f1781ffb640 1 -- 172.21.15.39:0/3830723833 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1784052020 con 0x7f178c1f0530 2024-09-13T16:53:02.204 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:53:02.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.375+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f178c157320 con 0x7f178c1f0530 2024-09-13T16:53:02.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.375+0000 7f1781ffb640 1 -- 172.21.15.39:0/3830723833 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f1784055640 con 0x7f178c1f0530 2024-09-13T16:53:02.381 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:02.381 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:01.612378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:rejoin","state_seq":9,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 43 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1770066690 msgr2=0x7f1770068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1770066690 0x7f1770068b50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f17880099e0 tx=0x7f1788008040 comp rx=0 tx=0).stop 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c1f0530 msgr2=0x7f178c151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c1f0530 0x7f178c151bd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1784001340 tx=0x7f178400b4a0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 shutdown_connections 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1770066690 0x7f1770068b50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f178c153890 0x7f178c1efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c1f0530 0x7f178c151bd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 --2- 172.21.15.39:0/3830723833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1778003f00 0x7f178c14db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 >> 172.21.15.39:0/3830723833 conn(0x7f178c0ab530 msgr2=0x7f178c14e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 shutdown_connections 2024-09-13T16:53:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:02.379+0000 7f179339b640 1 -- 172.21.15.39:0/3830723833 wait complete. 2024-09-13T16:53:02.399 INFO:tasks.mds_thrash.fs.[a]:no change 2024-09-13T16:53:02.847 INFO:tasks.ceph.mds.b.smithi084.stderr:2024-09-13T16:53:02.845+0000 7f7ebbd71640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:53:04.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:04.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.343+0000 7f2046be1640 1 Processor -- start 2024-09-13T16:53:04.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.343+0000 7f2046be1640 1 -- start start 2024-09-13T16:53:04.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.343+0000 7f2046be1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2040154bb0 0x7f20400c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.343+0000 7f2046be1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f20400c5870 0x7f204015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.343+0000 7f2046be1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2040153e30 con 0x7f20400c2370 2024-09-13T16:53:04.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.343+0000 7f2046be1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20400720f0 con 0x7f2040154bb0 2024-09-13T16:53:04.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.343+0000 7f2046be1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2040151b30 con 0x7f20400c5870 2024-09-13T16:53:04.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2040154bb0 0x7f20400c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f204497d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f20400c2370 0x7f20400c0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53732/0 (socket says 172.21.15.39:53732) 2024-09-13T16:53:04.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f204497d640 1 -- 172.21.15.39:0/3527397821 learned_addr learned my addr 172.21.15.39:0/3527397821 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:04.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f204517e640 1 --2- 172.21.15.39:0/3527397821 >> v2:172.21.15.84:3300/0 conn(0x7f20400c5870 0x7f204015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203ffff640 1 -- 172.21.15.39:0/3527397821 >> v2:172.21.15.84:3300/0 conn(0x7f20400c5870 msgr2=0x7f204015e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:04.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203ffff640 1 --2- 172.21.15.39:0/3527397821 >> v2:172.21.15.84:3300/0 conn(0x7f20400c5870 0x7f204015e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203ffff640 1 -- 172.21.15.39:0/3527397821 >> v1:172.21.15.39:6789/0 conn(0x7f20400c2370 legacy=0x7f20400c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203ffff640 1 -- 172.21.15.39:0/3527397821 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20400a7000 con 0x7f2040154bb0 2024-09-13T16:53:04.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203f7fe640 1 -- 172.21.15.39:0/3527397821 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264090137 0 0) 0x7f2040153e30 con 0x7f20400c2370 2024-09-13T16:53:04.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203ffff640 1 --2- 172.21.15.39:0/3527397821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 0x7f20400c2750 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f20400c4350 tx=0x7f202c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521aaa74e0f07fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203f7fe640 1 -- 172.21.15.39:0/3527397821 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c0ab600 con 0x7f2040154bb0 2024-09-13T16:53:04.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203f7fe640 1 -- 172.21.15.39:0/3527397821 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c0a7050 con 0x7f2040154bb0 2024-09-13T16:53:04.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.347+0000 7f203f7fe640 1 -- 172.21.15.39:0/3527397821 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c0ab980 con 0x7f2040154bb0 2024-09-13T16:53:04.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- 172.21.15.39:0/3527397821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 msgr2=0x7f20400c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 --2- 172.21.15.39:0/3527397821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 0x7f20400c2750 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f20400c4350 tx=0x7f202c01c3f0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- 172.21.15.39:0/3527397821 shutdown_connections 2024-09-13T16:53:04.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 --2- 172.21.15.39:0/3527397821 >> v2:172.21.15.84:3300/0 conn(0x7f20400c5870 0x7f204015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 --2- 172.21.15.39:0/3527397821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 0x7f20400c2750 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- 172.21.15.39:0/3527397821 >> 172.21.15.39:0/3527397821 conn(0x7f20400a9720 msgr2=0x7f20400a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:04.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- 172.21.15.39:0/3527397821 shutdown_connections 2024-09-13T16:53:04.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- 172.21.15.39:0/3527397821 wait complete. 2024-09-13T16:53:04.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 Processor -- start 2024-09-13T16:53:04.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- start start 2024-09-13T16:53:04.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20400c2370 0x7f20401f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20400c5870 0x7f20401ec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 0x7f2040152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204014d260 con 0x7f2040154bb0 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204014c380 con 0x7f20400c5870 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204014c0a0 con 0x7f20400c2370 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204517e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 0x7f2040152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204517e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 0x7f2040152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37160/0 (socket says 172.21.15.39:37160) 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204517e640 1 -- 172.21.15.39:0/1674923817 learned_addr learned my addr 172.21.15.39:0/1674923817 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204497d640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20400c2370 0x7f20401f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f203ffff640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20400c5870 0x7f20401ec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204517e640 1 -- 172.21.15.39:0/1674923817 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20400c2370 msgr2=0x7f20401f07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204517e640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20400c2370 0x7f20401f07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204517e640 1 -- 172.21.15.39:0/1674923817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20400c5870 msgr2=0x7f20401ec6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204517e640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20400c5870 0x7f20401ec6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204517e640 1 -- 172.21.15.39:0/1674923817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c009420 con 0x7f2040154bb0 2024-09-13T16:53:04.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f204517e640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 0x7f2040152e20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2038007e40 tx=0x7f203800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f203d7fa640 1 -- 172.21.15.39:0/1674923817 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038014620 con 0x7f2040154bb0 2024-09-13T16:53:04.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f203d7fa640 1 -- 172.21.15.39:0/1674923817 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203800ee90 con 0x7f2040154bb0 2024-09-13T16:53:04.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- 172.21.15.39:0/1674923817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20401efcf0 con 0x7f2040154bb0 2024-09-13T16:53:04.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f203d7fa640 1 -- 172.21.15.39:0/1674923817 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20380149e0 con 0x7f2040154bb0 2024-09-13T16:53:04.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.351+0000 7f2046be1640 1 -- 172.21.15.39:0/1674923817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20401f02b0 con 0x7f2040154bb0 2024-09-13T16:53:04.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.355+0000 7f202affd640 1 -- 172.21.15.39:0/1674923817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2018005260 con 0x7f2040154bb0 2024-09-13T16:53:04.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.355+0000 7f203d7fa640 1 -- 172.21.15.39:0/1674923817 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2038014b40 con 0x7f2040154bb0 2024-09-13T16:53:04.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.355+0000 7f203d7fa640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2024066690 0x7f2024068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.355+0000 7f203d7fa640 1 -- 172.21.15.39:0/1674923817 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7f20380850a0 con 0x7f2040154bb0 2024-09-13T16:53:04.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.355+0000 7f204497d640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2024066690 0x7f2024068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.355+0000 7f204497d640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2024066690 0x7f2024068b50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2034000f70 tx=0x7f2034007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.359+0000 7f203d7fa640 1 -- 172.21.15.39:0/1674923817 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2038051ef0 con 0x7f2040154bb0 2024-09-13T16:53:04.399 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.527+0000 7fb22d817640 1 Processor -- start 2024-09-13T16:53:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.527+0000 7fb22d817640 1 -- start start 2024-09-13T16:53:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.527+0000 7fb22d817640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb22806aac0 0x7fb2280aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.527+0000 7fb22d817640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2280c3ae0 con 0x7fb2280c21d0 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.527+0000 7fb22d817640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2280abb30 con 0x7fb22814b850 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.527+0000 7fb22d817640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2280ab340 con 0x7fb22806aac0 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.527+0000 7fb226ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb2280c21d0 0x7fb228151c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53746/0 (socket says 172.21.15.39:53746) 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.527+0000 7fb226ffd640 1 -- 172.21.15.39:0/440561352 learned_addr learned my addr 172.21.15.39:0/440561352 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.527+0000 7fb2267fc640 1 --2- 172.21.15.39:0/440561352 >> v2:172.21.15.39:3300/0 conn(0x7fb22806aac0 0x7fb2280aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb225ffb640 1 -- 172.21.15.39:0/440561352 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970136056 0 0) 0x7fb2280c3ae0 con 0x7fb2280c21d0 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb225ffb640 1 -- 172.21.15.39:0/440561352 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb210003530 con 0x7fb2280c21d0 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb2267fc640 1 -- 172.21.15.39:0/440561352 >> v1:172.21.15.84:6789/0 conn(0x7fb22814b850 legacy=0x7fb22815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb2267fc640 1 -- 172.21.15.39:0/440561352 >> v1:172.21.15.39:6789/0 conn(0x7fb2280c21d0 legacy=0x7fb228151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb2267fc640 1 -- 172.21.15.39:0/440561352 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2280abe10 con 0x7fb22806aac0 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb2267fc640 1 --2- 172.21.15.39:0/440561352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 0x7fb2280aae00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0097c0 tx=0x7fb21c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe9d84b03479b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb225ffb640 1 -- 172.21.15.39:0/440561352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0ac640 con 0x7fb22806aac0 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb225ffb640 1 -- 172.21.15.39:0/440561352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c0ac7a0 con 0x7fb22806aac0 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb225ffb640 1 -- 172.21.15.39:0/440561352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0acb30 con 0x7fb22806aac0 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb22d817640 1 -- 172.21.15.39:0/440561352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 msgr2=0x7fb2280aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb22d817640 1 --2- 172.21.15.39:0/440561352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 0x7fb2280aae00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0097c0 tx=0x7fb21c01c3e0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb22d817640 1 -- 172.21.15.39:0/440561352 shutdown_connections 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb22d817640 1 --2- 172.21.15.39:0/440561352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 0x7fb2280aae00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb22d817640 1 -- 172.21.15.39:0/440561352 >> 172.21.15.39:0/440561352 conn(0x7fb228069e10 msgr2=0x7fb22806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb22d817640 1 -- 172.21.15.39:0/440561352 shutdown_connections 2024-09-13T16:53:04.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.531+0000 7fb22d817640 1 -- 172.21.15.39:0/440561352 wait complete. 2024-09-13T16:53:04.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 Processor -- start 2024-09-13T16:53:04.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 -- start start 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 0x7fb22814c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb2280c21d0 0x7fb228141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb22814b850 0x7fb22814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22814caf0 con 0x7fb22806aac0 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22815d7c0 con 0x7fb22814b850 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2281eaf90 con 0x7fb2280c21d0 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb226ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 0x7fb22814c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb226ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 0x7fb22814c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37186/0 (socket says 172.21.15.39:37186) 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb226ffd640 1 -- 172.21.15.39:0/2899265662 learned_addr learned my addr 172.21.15.39:0/2899265662 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb2267fc640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb2280c21d0 0x7fb228141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb2277fe640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb22814b850 0x7fb22814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb226ffd640 1 -- 172.21.15.39:0/2899265662 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb2280c21d0 msgr2=0x7fb228141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb226ffd640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb2280c21d0 0x7fb228141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb226ffd640 1 -- 172.21.15.39:0/2899265662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb22814b850 msgr2=0x7fb22814e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb226ffd640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb22814b850 0x7fb22814e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb226ffd640 1 -- 172.21.15.39:0/2899265662 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c009420 con 0x7fb22806aac0 2024-09-13T16:53:04.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb226ffd640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 0x7fb22814c5b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb21800c4d0 tx=0x7fb21800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb20ffff640 1 -- 172.21.15.39:0/2899265662 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218005cf0 con 0x7fb22806aac0 2024-09-13T16:53:04.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb20ffff640 1 -- 172.21.15.39:0/2899265662 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218005e50 con 0x7fb22806aac0 2024-09-13T16:53:04.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb20ffff640 1 -- 172.21.15.39:0/2899265662 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218007270 con 0x7fb22806aac0 2024-09-13T16:53:04.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2281eb270 con 0x7fb22806aac0 2024-09-13T16:53:04.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.535+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2281438a0 con 0x7fb22806aac0 2024-09-13T16:53:04.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.539+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2280c3560 con 0x7fb22806aac0 2024-09-13T16:53:04.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.539+0000 7fb20ffff640 1 -- 172.21.15.39:0/2899265662 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb2180073d0 con 0x7fb22806aac0 2024-09-13T16:53:04.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.539+0000 7fb20ffff640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb214066690 0x7fb214068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:04.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.539+0000 7fb20ffff640 1 -- 172.21.15.39:0/2899265662 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7fb218083bb0 con 0x7fb22806aac0 2024-09-13T16:53:04.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.539+0000 7fb2267fc640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb214066690 0x7fb214068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:04.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.539+0000 7fb2267fc640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb214066690 0x7fb214068b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0ae000 tx=0x7fb21c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:04.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.543+0000 7fb20ffff640 1 -- 172.21.15.39:0/2899265662 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb218050ab0 con 0x7fb22806aac0 2024-09-13T16:53:04.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.599+0000 7f202affd640 1 -- 172.21.15.39:0/1674923817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2018005510 con 0x7f2040154bb0 2024-09-13T16:53:04.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.599+0000 7f203d7fa640 1 -- 172.21.15.39:0/1674923817 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f2038055510 con 0x7f2040154bb0 2024-09-13T16:53:04.605 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:04.606 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:04.606 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 44 2024-09-13T16:53:04.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 -- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2024066690 msgr2=0x7f2024068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2024066690 0x7f2024068b50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2034000f70 tx=0x7f2034007b80 comp rx=0 tx=0).stop 2024-09-13T16:53:04.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 -- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 msgr2=0x7f2040152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:04.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 0x7f2040152e20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2038007e40 tx=0x7f203800e4b0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 -- 172.21.15.39:0/1674923817 shutdown_connections 2024-09-13T16:53:04.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2024066690 0x7f2024068b50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20400c5870 0x7f20401ec6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040154bb0 0x7f2040152e20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 --2- 172.21.15.39:0/1674923817 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f20400c2370 0x7f20401f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:04.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 -- 172.21.15.39:0/1674923817 >> 172.21.15.39:0/1674923817 conn(0x7f20400a9720 msgr2=0x7f20400a8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:04.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 -- 172.21.15.39:0/1674923817 shutdown_connections 2024-09-13T16:53:04.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.603+0000 7f202affd640 1 -- 172.21.15.39:0/1674923817 wait complete. 2024-09-13T16:53:04.630 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:53:04.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.779+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2280c25b0 con 0x7fb22806aac0 2024-09-13T16:53:05.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.779+0000 7fb20ffff640 1 -- 172.21.15.39:0/2899265662 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5176 (secure 0 0 0) 0x7fb218017090 con 0x7fb22806aac0 2024-09-13T16:53:05.014 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:05.014 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:05.015 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 44 2024-09-13T16:53:05.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb214066690 msgr2=0x7fb214068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb214066690 0x7fb214068b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb21c0ae000 tx=0x7fb21c0a7000 comp rx=0 tx=0).stop 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 msgr2=0x7fb22814c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 0x7fb22814c5b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb21800c4d0 tx=0x7fb21800c990 comp rx=0 tx=0).stop 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 shutdown_connections 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb214066690 0x7fb214068b50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb22814b850 0x7fb22814e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb2280c21d0 0x7fb228141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 --2- 172.21.15.39:0/2899265662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb22806aac0 0x7fb22814c5b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 >> 172.21.15.39:0/2899265662 conn(0x7fb228069e10 msgr2=0x7fb22806d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 shutdown_connections 2024-09-13T16:53:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:04.783+0000 7fb22d817640 1 -- 172.21.15.39:0/2899265662 wait complete. 2024-09-13T16:53:05.017 INFO:tasks.mds_thrash.fs.[a]:mds.b has gained rank=0, replacing gid=5053 2024-09-13T16:53:05.017 INFO:tasks.mds_thrash.fs.[a]:waiting for 0 secs before reviving mds.d 2024-09-13T16:53:05.017 INFO:tasks.mds_thrash.fs.[a]:reviving mds.d 2024-09-13T16:53:05.017 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:53:05.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:53:05.020 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:53:05.020 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:05.096 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T16:53:05.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.163+0000 7fdd4ee7d640 1 Processor -- start 2024-09-13T16:53:05.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.163+0000 7fdd4ee7d640 1 -- start start 2024-09-13T16:53:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd4ee7d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdd481556d0 0x7fdd480c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd4ee7d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4806feb0 con 0x7fdd480c7610 2024-09-13T16:53:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd4ee7d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd480c3c20 con 0x7fdd480a72b0 2024-09-13T16:53:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd4ee7d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd480aae50 con 0x7fdd481556d0 2024-09-13T16:53:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd4cc19640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdd481556d0 0x7fdd480c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd47fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdd480c7610 0x7fdd48151b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53760/0 (socket says 172.21.15.39:53760) 2024-09-13T16:53:05.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd47fff640 1 -- 172.21.15.39:0/1477409084 learned_addr learned my addr 172.21.15.39:0/1477409084 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:05.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd4cc19640 1 -- 172.21.15.39:0/1477409084 >> v1:172.21.15.84:6789/0 conn(0x7fdd480a72b0 legacy=0x7fdd48071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:05.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd4cc19640 1 -- 172.21.15.39:0/1477409084 >> v1:172.21.15.39:6789/0 conn(0x7fdd480c7610 legacy=0x7fdd48151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd4cc19640 1 -- 172.21.15.39:0/1477409084 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd480c6c30 con 0x7fdd481556d0 2024-09-13T16:53:05.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.167+0000 7fdd477fe640 1 -- 172.21.15.39:0/1477409084 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065263949 0 0) 0x7fdd4806feb0 con 0x7fdd480c7610 2024-09-13T16:53:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4cc19640 1 --2- 172.21.15.39:0/1477409084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 0x7fdd480c36e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0097c0 tx=0x7fdd3c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57d914817a68dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd477fe640 1 -- 172.21.15.39:0/1477409084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0ab600 con 0x7fdd481556d0 2024-09-13T16:53:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd477fe640 1 -- 172.21.15.39:0/1477409084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c0a7050 con 0x7fdd481556d0 2024-09-13T16:53:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd477fe640 1 -- 172.21.15.39:0/1477409084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0ab980 con 0x7fdd481556d0 2024-09-13T16:53:05.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/1477409084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 msgr2=0x7fdd480c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 --2- 172.21.15.39:0/1477409084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 0x7fdd480c36e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0097c0 tx=0x7fdd3c01c340 comp rx=0 tx=0).stop 2024-09-13T16:53:05.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/1477409084 shutdown_connections 2024-09-13T16:53:05.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 --2- 172.21.15.39:0/1477409084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 0x7fdd480c36e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/1477409084 >> 172.21.15.39:0/1477409084 conn(0x7fdd480a97b0 msgr2=0x7fdd480a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:05.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/1477409084 shutdown_connections 2024-09-13T16:53:05.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/1477409084 wait complete. 2024-09-13T16:53:05.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 Processor -- start 2024-09-13T16:53:05.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 -- start start 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd480a72b0 0x7fdd4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 0x7fdd481ed750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd480a93b0 0x7fdd4814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4806ec40 con 0x7fdd481556d0 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4806f170 con 0x7fdd480a93b0 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4ee7d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481ee0a0 con 0x7fdd480a72b0 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 0x7fdd481ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 0x7fdd481ed750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37200/0 (socket says 172.21.15.39:37200) 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd47fff640 1 -- 172.21.15.39:0/898675232 learned_addr learned my addr 172.21.15.39:0/898675232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4cc19640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd480a72b0 0x7fdd4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4d41a640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd480a93b0 0x7fdd4814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd47fff640 1 -- 172.21.15.39:0/898675232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd480a72b0 msgr2=0x7fdd4806d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd47fff640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd480a72b0 0x7fdd4806d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd47fff640 1 -- 172.21.15.39:0/898675232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd480a93b0 msgr2=0x7fdd4814f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd47fff640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd480a93b0 0x7fdd4814f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd47fff640 1 -- 172.21.15.39:0/898675232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009420 con 0x7fdd481556d0 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.171+0000 7fdd4cc19640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd480a72b0 0x7fdd4806d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd47fff640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 0x7fdd481ed750 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdd340011f0 tx=0x7fdd3400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd457fa640 1 -- 172.21.15.39:0/898675232 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340074e0 con 0x7fdd481556d0 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd457fa640 1 -- 172.21.15.39:0/898675232 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd34007640 con 0x7fdd481556d0 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd457fa640 1 -- 172.21.15.39:0/898675232 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34004a10 con 0x7fdd481556d0 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4806ee90 con 0x7fdd481556d0 2024-09-13T16:53:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd481edc90 con 0x7fdd481556d0 2024-09-13T16:53:05.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd480c36e0 con 0x7fdd481556d0 2024-09-13T16:53:05.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd457fa640 1 -- 172.21.15.39:0/898675232 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdd34004b70 con 0x7fdd481556d0 2024-09-13T16:53:05.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd457fa640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd2c066690 0x7fdd2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:05.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd457fa640 1 -- 172.21.15.39:0/898675232 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7fdd34083970 con 0x7fdd481556d0 2024-09-13T16:53:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd4cc19640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd2c066690 0x7fdd2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.175+0000 7fdd4cc19640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd2c066690 0x7fdd2c068b50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0ad000 tx=0x7fdd3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:05.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.179+0000 7fdd457fa640 1 -- 172.21.15.39:0/898675232 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd340507c0 con 0x7fdd481556d0 2024-09-13T16:53:05.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.387+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd480722c0 con 0x7fdd481556d0 2024-09-13T16:53:05.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd457fa640 1 -- 172.21.15.39:0/898675232 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5176 (secure 0 0 0) 0x7fdd34018020 con 0x7fdd481556d0 2024-09-13T16:53:05.394 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:05.394 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:05.395 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 44 2024-09-13T16:53:05.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd2c066690 msgr2=0x7fdd2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd2c066690 0x7fdd2c068b50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0ad000 tx=0x7fdd3c008040 comp rx=0 tx=0).stop 2024-09-13T16:53:05.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 msgr2=0x7fdd481ed750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:05.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 0x7fdd481ed750 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdd340011f0 tx=0x7fdd3400c9b0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 shutdown_connections 2024-09-13T16:53:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdd2c066690 0x7fdd2c068b50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd480a93b0 0x7fdd4814f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd481556d0 0x7fdd481ed750 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 --2- 172.21.15.39:0/898675232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdd480a72b0 0x7fdd4806d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 >> 172.21.15.39:0/898675232 conn(0x7fdd480a97b0 msgr2=0x7fdd481548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 shutdown_connections 2024-09-13T16:53:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:05.391+0000 7fdd4ee7d640 1 -- 172.21.15.39:0/898675232 wait complete. 2024-09-13T16:53:05.417 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-09-13T16:53:06.632 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f802db1d640 1 Processor -- start 2024-09-13T16:53:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f802db1d640 1 -- start start 2024-09-13T16:53:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f802db1d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8028151500 0x7f80280c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f802db1d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8028071dd0 0x7f8028154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f802db1d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80280c5a30 con 0x7f80280c47c0 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f802db1d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80281576a0 con 0x7f8028151500 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f802db1d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8028157980 con 0x7f8028071dd0 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f80277fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8028151500 0x7f80280c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f8027fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f80280c47c0 0x7f80281545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53766/0 (socket says 172.21.15.39:53766) 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f8027fff640 1 -- 172.21.15.39:0/3233583565 learned_addr learned my addr 172.21.15.39:0/3233583565 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f801ffff640 1 --2- 172.21.15.39:0/3233583565 >> v2:172.21.15.84:3300/0 conn(0x7f8028071dd0 0x7f8028154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f80277fe640 1 -- 172.21.15.39:0/3233583565 >> v2:172.21.15.84:3300/0 conn(0x7f8028071dd0 msgr2=0x7f8028154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f80277fe640 1 --2- 172.21.15.39:0/3233583565 >> v2:172.21.15.84:3300/0 conn(0x7f8028071dd0 0x7f8028154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f80277fe640 1 -- 172.21.15.39:0/3233583565 >> v1:172.21.15.39:6789/0 conn(0x7f80280c47c0 legacy=0x7f80281545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f80277fe640 1 -- 172.21.15.39:0/3233583565 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028157c60 con 0x7f8028151500 2024-09-13T16:53:06.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.771+0000 7f8026ffd640 1 -- 172.21.15.39:0/3233583565 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2193927686 0 0) 0x7f80280c5a30 con 0x7f80280c47c0 2024-09-13T16:53:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f80277fe640 1 --2- 172.21.15.39:0/3233583565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028151500 0x7f80280c4280 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f80180097c0 tx=0x7f801801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab0d15c52404e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f8026ffd640 1 -- 172.21.15.39:0/3233583565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80180ac620 con 0x7f8028151500 2024-09-13T16:53:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f8026ffd640 1 -- 172.21.15.39:0/3233583565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80180ac780 con 0x7f8028151500 2024-09-13T16:53:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f8026ffd640 1 -- 172.21.15.39:0/3233583565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80180acb10 con 0x7f8028151500 2024-09-13T16:53:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f802db1d640 1 -- 172.21.15.39:0/3233583565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028151500 msgr2=0x7f80280c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f802db1d640 1 --2- 172.21.15.39:0/3233583565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028151500 0x7f80280c4280 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f80180097c0 tx=0x7f801801c3a0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f802db1d640 1 -- 172.21.15.39:0/3233583565 shutdown_connections 2024-09-13T16:53:06.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f802db1d640 1 --2- 172.21.15.39:0/3233583565 >> v2:172.21.15.84:3300/0 conn(0x7f8028071dd0 0x7f8028154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f802db1d640 1 --2- 172.21.15.39:0/3233583565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028151500 0x7f80280c4280 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f802db1d640 1 -- 172.21.15.39:0/3233583565 >> 172.21.15.39:0/3233583565 conn(0x7f80280ab530 msgr2=0x7f80280ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:06.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f802db1d640 1 -- 172.21.15.39:0/3233583565 shutdown_connections 2024-09-13T16:53:06.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.775+0000 7f802db1d640 1 -- 172.21.15.39:0/3233583565 wait complete. 2024-09-13T16:53:06.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 Processor -- start 2024-09-13T16:53:06.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 -- start start 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028071dd0 0x7f80281f0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80280c47c0 0x7f8028070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8028151500 0x7f80280a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80281482d0 con 0x7f8028071dd0 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80281eff10 con 0x7f8028151500 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802814c650 con 0x7f80280c47c0 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f80277fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028071dd0 0x7f80281f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f80277fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028071dd0 0x7f80281f0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37222/0 (socket says 172.21.15.39:37222) 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f80277fe640 1 -- 172.21.15.39:0/3122372259 learned_addr learned my addr 172.21.15.39:0/3122372259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f8027fff640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8028151500 0x7f80280a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f80277fe640 1 -- 172.21.15.39:0/3122372259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80280c47c0 msgr2=0x7f8028070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f801ffff640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80280c47c0 0x7f8028070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f80277fe640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80280c47c0 0x7f8028070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f80277fe640 1 -- 172.21.15.39:0/3122372259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8028151500 msgr2=0x7f80280a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f80277fe640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8028151500 0x7f80280a93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f80277fe640 1 -- 172.21.15.39:0/3122372259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8018009420 con 0x7f8028071dd0 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f8027fff640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8028151500 0x7f80280a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f80277fe640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028071dd0 0x7f80281f0960 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f80180af040 tx=0x7f8018019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f8024ff9640 1 -- 172.21.15.39:0/3122372259 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80180b5410 con 0x7f8028071dd0 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f8024ff9640 1 -- 172.21.15.39:0/3122372259 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80180b5570 con 0x7f8028071dd0 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80281f0160 con 0x7f8028071dd0 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f8024ff9640 1 -- 172.21.15.39:0/3122372259 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80180b5920 con 0x7f8028071dd0 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8028153ac0 con 0x7f8028071dd0 2024-09-13T16:53:06.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.779+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80280c4280 con 0x7f8028071dd0 2024-09-13T16:53:06.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.783+0000 7f8024ff9640 1 -- 172.21.15.39:0/3122372259 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80180b3040 con 0x7f8028071dd0 2024-09-13T16:53:06.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.783+0000 7f8024ff9640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8008066690 0x7f8008068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:06.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.783+0000 7f8024ff9640 1 -- 172.21.15.39:0/3122372259 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7f801811eb50 con 0x7f8028071dd0 2024-09-13T16:53:06.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.783+0000 7f801ffff640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8008066690 0x7f8008068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:06.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.783+0000 7f801ffff640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8008066690 0x7f8008068b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f80280c3d70 tx=0x7f8010008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:06.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:06.787+0000 7f8024ff9640 1 -- 172.21.15.39:0/3122372259 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f801811ee60 con 0x7f8028071dd0 2024-09-13T16:53:07.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8028071bb0 con 0x7f8028071dd0 2024-09-13T16:53:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f8024ff9640 1 -- 172.21.15.39:0/3122372259 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f80180eba80 con 0x7f8028071dd0 2024-09-13T16:53:07.015 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:07.015 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:07.016 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 45 2024-09-13T16:53:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8008066690 msgr2=0x7f8008068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8008066690 0x7f8008068b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f80280c3d70 tx=0x7f8010008040 comp rx=0 tx=0).stop 2024-09-13T16:53:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028071dd0 msgr2=0x7f80281f0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028071dd0 0x7f80281f0960 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f80180af040 tx=0x7f8018019140 comp rx=0 tx=0).stop 2024-09-13T16:53:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 shutdown_connections 2024-09-13T16:53:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8008066690 0x7f8008068b50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8028151500 0x7f80280a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80280c47c0 0x7f8028070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 --2- 172.21.15.39:0/3122372259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8028071dd0 0x7f80281f0960 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 >> 172.21.15.39:0/3122372259 conn(0x7f80280ab530 msgr2=0x7f8028060b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 shutdown_connections 2024-09-13T16:53:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.011+0000 7f802db1d640 1 -- 172.21.15.39:0/3122372259 wait complete. 2024-09-13T16:53:07.035 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:53:07.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:07.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e2d92640 1 Processor -- start 2024-09-13T16:53:07.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e2d92640 1 -- start start 2024-09-13T16:53:07.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e2d92640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59dc14a770 0x7f59dc150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e2d92640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59dc0c6060 0x7f59dc14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e2d92640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc06ae20 0x7f59dc06b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e2d92640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59dc06df80 con 0x7f59dc14a770 2024-09-13T16:53:07.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e2d92640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59dc0c1950 con 0x7f59dc0c6060 2024-09-13T16:53:07.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e2d92640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0c4760 con 0x7f59dc06ae20 2024-09-13T16:53:07.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e0b2e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59dc14a770 0x7f59dc150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e0b2e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59dc14a770 0x7f59dc150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37234/0 (socket says 172.21.15.39:37234) 2024-09-13T16:53:07.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e0b2e640 1 -- 172.21.15.39:0/1078355091 learned_addr learned my addr 172.21.15.39:0/1078355091 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:07.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e132f640 1 --2- 172.21.15.39:0/1078355091 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc06ae20 0x7f59dc06b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59dbfff640 1 --2- 172.21.15.39:0/1078355091 >> v2:172.21.15.84:3300/0 conn(0x7f59dc0c6060 0x7f59dc14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e0b2e640 1 -- 172.21.15.39:0/1078355091 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc06ae20 msgr2=0x7f59dc06b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e0b2e640 1 --2- 172.21.15.39:0/1078355091 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc06ae20 0x7f59dc06b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e0b2e640 1 -- 172.21.15.39:0/1078355091 >> v2:172.21.15.84:3300/0 conn(0x7f59dc0c6060 msgr2=0x7f59dc14ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e0b2e640 1 --2- 172.21.15.39:0/1078355091 >> v2:172.21.15.84:3300/0 conn(0x7f59dc0c6060 0x7f59dc14ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.543+0000 7f59e0b2e640 1 -- 172.21.15.39:0/1078355091 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc151120 con 0x7f59dc14a770 2024-09-13T16:53:07.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e0b2e640 1 --2- 172.21.15.39:0/1078355091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 0x7f59dc150ad0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0a94e0 tx=0x7f59d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f637268b6592a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59db7fe640 1 -- 172.21.15.39:0/1078355091 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d00ab620 con 0x7f59dc14a770 2024-09-13T16:53:07.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59db7fe640 1 -- 172.21.15.39:0/1078355091 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d00ab780 con 0x7f59dc14a770 2024-09-13T16:53:07.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59db7fe640 1 -- 172.21.15.39:0/1078355091 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d00abb10 con 0x7f59dc14a770 2024-09-13T16:53:07.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 -- 172.21.15.39:0/1078355091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 msgr2=0x7f59dc150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 --2- 172.21.15.39:0/1078355091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 0x7f59dc150ad0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0a94e0 tx=0x7f59d001c460 comp rx=0 tx=0).stop 2024-09-13T16:53:07.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 -- 172.21.15.39:0/1078355091 shutdown_connections 2024-09-13T16:53:07.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 --2- 172.21.15.39:0/1078355091 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc06ae20 0x7f59dc06b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 --2- 172.21.15.39:0/1078355091 >> v2:172.21.15.84:3300/0 conn(0x7f59dc0c6060 0x7f59dc14ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 --2- 172.21.15.39:0/1078355091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 0x7f59dc150ad0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 -- 172.21.15.39:0/1078355091 >> 172.21.15.39:0/1078355091 conn(0x7f59dc0c32a0 msgr2=0x7f59dc0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 -- 172.21.15.39:0/1078355091 shutdown_connections 2024-09-13T16:53:07.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 -- 172.21.15.39:0/1078355091 wait complete. 2024-09-13T16:53:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.547+0000 7f59e2d92640 1 Processor -- start 2024-09-13T16:53:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 -- start start 2024-09-13T16:53:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59dc06ae20 0x7f59dc150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc0c6060 0x7f59dc0a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 0x7f59dc1e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc149d00 con 0x7f59dc14a770 2024-09-13T16:53:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc1f0170 con 0x7f59dc06ae20 2024-09-13T16:53:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0ac580 con 0x7f59dc0c6060 2024-09-13T16:53:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e132f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 0x7f59dc1e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e132f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 0x7f59dc1e38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37238/0 (socket says 172.21.15.39:37238) 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e132f640 1 -- 172.21.15.39:0/3048688377 learned_addr learned my addr 172.21.15.39:0/3048688377 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e0b2e640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59dc06ae20 0x7f59dc150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59dbfff640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc0c6060 0x7f59dc0a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e132f640 1 -- 172.21.15.39:0/3048688377 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc0c6060 msgr2=0x7f59dc0a9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e132f640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc0c6060 0x7f59dc0a9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e132f640 1 -- 172.21.15.39:0/3048688377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59dc06ae20 msgr2=0x7f59dc150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e132f640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59dc06ae20 0x7f59dc150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e132f640 1 -- 172.21.15.39:0/3048688377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d00094d0 con 0x7f59dc14a770 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e132f640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 0x7f59dc1e38f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f59d4007e90 tx=0x7f59d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59d97fa640 1 -- 172.21.15.39:0/3048688377 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4014420 con 0x7f59dc14a770 2024-09-13T16:53:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59d97fa640 1 -- 172.21.15.39:0/3048688377 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4014580 con 0x7f59dc14a770 2024-09-13T16:53:07.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc1f06d0 con 0x7f59dc14a770 2024-09-13T16:53:07.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59d97fa640 1 -- 172.21.15.39:0/3048688377 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4014930 con 0x7f59dc14a770 2024-09-13T16:53:07.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc154ef0 con 0x7f59dc14a770 2024-09-13T16:53:07.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc0619c0 con 0x7f59dc14a770 2024-09-13T16:53:07.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.551+0000 7f59d97fa640 1 -- 172.21.15.39:0/3048688377 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59d4014a90 con 0x7f59dc14a770 2024-09-13T16:53:07.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.555+0000 7f59d97fa640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59c0066690 0x7f59c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:07.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.555+0000 7f59d97fa640 1 -- 172.21.15.39:0/3048688377 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7f59d4084fa0 con 0x7f59dc14a770 2024-09-13T16:53:07.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.555+0000 7f59e0b2e640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59c0066690 0x7f59c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:07.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.555+0000 7f59e0b2e640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59c0066690 0x7f59c0068b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f59d00af000 tx=0x7f59d00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:07.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.559+0000 7f59d97fa640 1 -- 172.21.15.39:0/3048688377 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59d40b6db0 con 0x7f59dc14a770 2024-09-13T16:53:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.807+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59dc14ab50 con 0x7f59dc14a770 2024-09-13T16:53:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.807+0000 7f59d97fa640 1 -- 172.21.15.39:0/3048688377 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f59d4055860 con 0x7f59dc14a770 2024-09-13T16:53:07.812 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:07.812 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:07.812 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 45 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59c0066690 msgr2=0x7f59c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59c0066690 0x7f59c0068b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f59d00af000 tx=0x7f59d00a7000 comp rx=0 tx=0).stop 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 msgr2=0x7f59dc1e38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 0x7f59dc1e38f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f59d4007e90 tx=0x7f59d400e4b0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 shutdown_connections 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59c0066690 0x7f59c0068b50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59dc14a770 0x7f59dc1e38f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59dc0c6060 0x7f59dc0a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 --2- 172.21.15.39:0/3048688377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59dc06ae20 0x7f59dc150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:07.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 >> 172.21.15.39:0/3048688377 conn(0x7f59dc0c32a0 msgr2=0x7f59dc0702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:07.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 shutdown_connections 2024-09-13T16:53:07.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:07.811+0000 7f59e2d92640 1 -- 172.21.15.39:0/3048688377 wait complete. 2024-09-13T16:53:07.837 INFO:tasks.mds_thrash.fs.[a]:mds.d reported in up:standby state 2024-09-13T16:53:07.837 INFO:tasks.mds_thrash.fs.[a]:waiting for 48 secs before thrashing 2024-09-13T16:53:09.036 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b3377d640 1 Processor -- start 2024-09-13T16:53:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b3377d640 1 -- start start 2024-09-13T16:53:09.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b3377d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6b2c0c4260 0x7f6b2c0c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:09.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b3377d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c0bcfb0 0x7f6b2c070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:09.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b3377d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c157360 con 0x7f6b2c0c77e0 2024-09-13T16:53:09.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b3377d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c80b0 con 0x7f6b2c0c4260 2024-09-13T16:53:09.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b3377d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c1575b0 con 0x7f6b2c0bcfb0 2024-09-13T16:53:09.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b31d1a640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f6b2c0c77e0 0x7f6b2c153df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53778/0 (socket says 172.21.15.39:53778) 2024-09-13T16:53:09.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b31d1a640 1 -- 172.21.15.39:0/2047575003 learned_addr learned my addr 172.21.15.39:0/2047575003 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:09.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b31519640 1 --2- 172.21.15.39:0/2047575003 >> v2:172.21.15.84:3300/0 conn(0x7f6b2c0c4260 0x7f6b2c0c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.155+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059106092 0 0) 0x7f6b2c157360 con 0x7f6b2c0c77e0 2024-09-13T16:53:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b30d18640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c0bcfb0 0x7f6b2c070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b18003530 con 0x7f6b2c0c77e0 2024-09-13T16:53:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2120365052 0 0) 0x7f6b18003530 con 0x7f6b2c0c77e0 2024-09-13T16:53:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c0bcfb0 msgr2=0x7f6b2c070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:09.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c0bcfb0 0x7f6b2c070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 >> v2:172.21.15.84:3300/0 conn(0x7f6b2c0c4260 msgr2=0x7f6b2c0c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:09.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 --2- 172.21.15.39:0/2047575003 >> v2:172.21.15.84:3300/0 conn(0x7f6b2c0c4260 0x7f6b2c0c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c157890 con 0x7f6b2c0c77e0 2024-09-13T16:53:09.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f6b24001a00 con 0x7f6b2c0c77e0 2024-09-13T16:53:09.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 >> v1:172.21.15.39:6789/0 conn(0x7f6b2c0c77e0 legacy=0x7f6b2c153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c0bcfb0 0x7f6b18003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b18003f00 0x7f6b180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b18004d10 0x7f6b18005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c80b0 con 0x7f6b18004d10 2024-09-13T16:53:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c1575b0 con 0x7f6b18003f00 2024-09-13T16:53:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1800ba70 con 0x7f6b2c0bcfb0 2024-09-13T16:53:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b31d1a640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b18004d10 0x7f6b18005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b31519640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c0bcfb0 0x7f6b18003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b30d18640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b18003f00 0x7f6b180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b31d1a640 1 -- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c0bcfb0 msgr2=0x7f6b18003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b31d1a640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c0bcfb0 0x7f6b18003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b31d1a640 1 -- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b18003f00 msgr2=0x7f6b180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b31d1a640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b18003f00 0x7f6b180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b31d1a640 1 -- 172.21.15.39:0/2047575003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b18003530 con 0x7f6b18004d10 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b31d1a640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b18004d10 0x7f6b18005190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0c7bf0 tx=0x7f6b2401d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bc0b240c116d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0c7330 con 0x7f6b18004d10 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b240a8070 con 0x7f6b18004d10 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b23fff640 1 -- 172.21.15.39:0/2047575003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c154d10 con 0x7f6b18004d10 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b3377d640 1 -- 172.21.15.39:0/2047575003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b18004d10 msgr2=0x7f6b18005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b3377d640 1 --2- 172.21.15.39:0/2047575003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b18004d10 0x7f6b18005190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0c7bf0 tx=0x7f6b2401d3c0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b31d1a640 1 -- 172.21.15.39:0/2047575003 reap_dead start 2024-09-13T16:53:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b3377d640 1 -- 172.21.15.39:0/2047575003 shutdown_connections 2024-09-13T16:53:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b3377d640 1 -- 172.21.15.39:0/2047575003 >> 172.21.15.39:0/2047575003 conn(0x7f6b2c069e10 msgr2=0x7f6b2c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b3377d640 1 -- 172.21.15.39:0/2047575003 shutdown_connections 2024-09-13T16:53:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.159+0000 7f6b3377d640 1 -- 172.21.15.39:0/2047575003 wait complete. 2024-09-13T16:53:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 Processor -- start 2024-09-13T16:53:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 -- start start 2024-09-13T16:53:09.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c156200 0x7f6b2c148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:09.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b2c147ce0 0x7f6b2c148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:09.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c1ec940 0x7f6b2c1ecdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:09.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c14b920 con 0x7f6b2c147ce0 2024-09-13T16:53:09.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c156fb0 con 0x7f6b2c1ec940 2024-09-13T16:53:09.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c14a600 con 0x7f6b2c156200 2024-09-13T16:53:09.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b30d18640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b2c147ce0 0x7f6b2c148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:09.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b30d18640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b2c147ce0 0x7f6b2c148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37254/0 (socket says 172.21.15.39:37254) 2024-09-13T16:53:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b30d18640 1 -- 172.21.15.39:0/3422004809 learned_addr learned my addr 172.21.15.39:0/3422004809 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b31519640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c156200 0x7f6b2c148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b31d1a640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c1ec940 0x7f6b2c1ecdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b30d18640 1 -- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c156200 msgr2=0x7f6b2c148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b30d18640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c156200 0x7f6b2c148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b30d18640 1 -- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c1ec940 msgr2=0x7f6b2c1ecdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b30d18640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c1ec940 0x7f6b2c1ecdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b30d18640 1 -- 172.21.15.39:0/3422004809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c14b410 con 0x7f6b2c147ce0 2024-09-13T16:53:09.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b31519640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c156200 0x7f6b2c148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b31d1a640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c1ec940 0x7f6b2c1ecdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b30d18640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b2c147ce0 0x7f6b2c148160 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c011590 tx=0x7f6b1c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b21ffb640 1 -- 172.21.15.39:0/3422004809 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c015930 con 0x7f6b2c147ce0 2024-09-13T16:53:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b21ffb640 1 -- 172.21.15.39:0/3422004809 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c015a90 con 0x7f6b2c147ce0 2024-09-13T16:53:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b21ffb640 1 -- 172.21.15.39:0/3422004809 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c015e40 con 0x7f6b2c147ce0 2024-09-13T16:53:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c0c0420 con 0x7f6b2c147ce0 2024-09-13T16:53:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.163+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c0c00b0 con 0x7f6b2c147ce0 2024-09-13T16:53:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.167+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c0bff10 con 0x7f6b2c147ce0 2024-09-13T16:53:09.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.167+0000 7f6b21ffb640 1 -- 172.21.15.39:0/3422004809 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b1c01d400 con 0x7f6b2c147ce0 2024-09-13T16:53:09.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.167+0000 7f6b21ffb640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b10066690 0x7f6b10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:09.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.167+0000 7f6b31519640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b10066690 0x7f6b10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:09.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.167+0000 7f6b21ffb640 1 -- 172.21.15.39:0/3422004809 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7f6b1c084f40 con 0x7f6b2c147ce0 2024-09-13T16:53:09.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.167+0000 7f6b31519640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b10066690 0x7f6b10068b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0af900 tx=0x7f6b28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:09.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.171+0000 7f6b21ffb640 1 -- 172.21.15.39:0/3422004809 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6b2c0bff10 con 0x7f6b2c147ce0 2024-09-13T16:53:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.375+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b2c0bff10 con 0x7f6b2c147ce0 2024-09-13T16:53:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.375+0000 7f6b21ffb640 1 -- 172.21.15.39:0/3422004809 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f6b2c0bff10 con 0x7f6b2c147ce0 2024-09-13T16:53:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:09.380 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 45 2024-09-13T16:53:09.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b10066690 msgr2=0x7f6b10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:09.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b10066690 0x7f6b10068b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0af900 tx=0x7f6b28008040 comp rx=0 tx=0).stop 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b2c147ce0 msgr2=0x7f6b2c148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b2c147ce0 0x7f6b2c148160 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c011590 tx=0x7f6b1c009f90 comp rx=0 tx=0).stop 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 shutdown_connections 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6b10066690 0x7f6b10068b50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b2c1ec940 0x7f6b2c1ecdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b2c147ce0 0x7f6b2c148160 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 --2- 172.21.15.39:0/3422004809 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b2c156200 0x7f6b2c148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 >> 172.21.15.39:0/3422004809 conn(0x7f6b2c069e10 msgr2=0x7f6b2c1584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 shutdown_connections 2024-09-13T16:53:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:09.379+0000 7f6b3377d640 1 -- 172.21.15.39:0/3422004809 wait complete. 2024-09-13T16:53:09.401 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:53:11.402 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:11.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f05671ef640 1 Processor -- start 2024-09-13T16:53:11.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f05671ef640 1 -- start start 2024-09-13T16:53:11.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f05671ef640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f05600c7a90 0x7f0560157840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:11.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f05671ef640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0560157d80 0x7f056015e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:11.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f05671ef640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0560154ed0 con 0x7f0560151960 2024-09-13T16:53:11.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f05671ef640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05600c51f0 con 0x7f05600c7a90 2024-09-13T16:53:11.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f05671ef640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05600c7330 con 0x7f0560157d80 2024-09-13T16:53:11.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f055ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f05600c7a90 0x7f0560157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:11.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f0564f8b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0560151960 0x7f05600c62e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:47986/0 (socket says 172.21.15.39:47986) 2024-09-13T16:53:11.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f0564f8b640 1 -- 172.21.15.39:0/367770899 learned_addr learned my addr 172.21.15.39:0/367770899 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:11.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f056578c640 1 --2- 172.21.15.39:0/367770899 >> v2:172.21.15.84:3300/0 conn(0x7f0560157d80 0x7f056015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:11.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f055ffff640 1 -- 172.21.15.39:0/367770899 >> v2:172.21.15.84:3300/0 conn(0x7f0560157d80 msgr2=0x7f056015e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:11.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f055ffff640 1 --2- 172.21.15.39:0/367770899 >> v2:172.21.15.84:3300/0 conn(0x7f0560157d80 0x7f056015e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f055ffff640 1 -- 172.21.15.39:0/367770899 >> v1:172.21.15.39:6789/0 conn(0x7f0560151960 legacy=0x7f05600c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:11.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.547+0000 7f055ffff640 1 -- 172.21.15.39:0/367770899 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560154350 con 0x7f05600c7a90 2024-09-13T16:53:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f055ffff640 1 --2- 172.21.15.39:0/367770899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 0x7f0560157840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f05600c8090 tx=0x7f054c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35ea34026e7a544 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f055f7fe640 1 -- 172.21.15.39:0/367770899 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c0ab600 con 0x7f05600c7a90 2024-09-13T16:53:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f055f7fe640 1 -- 172.21.15.39:0/367770899 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c0a7050 con 0x7f05600c7a90 2024-09-13T16:53:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f055f7fe640 1 -- 172.21.15.39:0/367770899 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c0ab980 con 0x7f05600c7a90 2024-09-13T16:53:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 -- 172.21.15.39:0/367770899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 msgr2=0x7f0560157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 --2- 172.21.15.39:0/367770899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 0x7f0560157840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f05600c8090 tx=0x7f054c01c3f0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 -- 172.21.15.39:0/367770899 shutdown_connections 2024-09-13T16:53:11.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 --2- 172.21.15.39:0/367770899 >> v2:172.21.15.84:3300/0 conn(0x7f0560157d80 0x7f056015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 --2- 172.21.15.39:0/367770899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 0x7f0560157840 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 -- 172.21.15.39:0/367770899 >> 172.21.15.39:0/367770899 conn(0x7f05600ab500 msgr2=0x7f05600ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:11.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 -- 172.21.15.39:0/367770899 shutdown_connections 2024-09-13T16:53:11.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 -- 172.21.15.39:0/367770899 wait complete. 2024-09-13T16:53:11.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 Processor -- start 2024-09-13T16:53:11.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.551+0000 7f05671ef640 1 -- start start 2024-09-13T16:53:11.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f05671ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 0x7f056014a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f05671ef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0560151960 0x7f0560141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f05671ef640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0560157d80 0x7f0560141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f05671ef640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05600ab380 con 0x7f05600c7a90 2024-09-13T16:53:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f05671ef640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05601424f0 con 0x7f0560151960 2024-09-13T16:53:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f05671ef640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056014af10 con 0x7f0560157d80 2024-09-13T16:53:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f0564f8b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 0x7f056014a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f0564f8b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 0x7f056014a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48726/0 (socket says 172.21.15.39:48726) 2024-09-13T16:53:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f0564f8b640 1 -- 172.21.15.39:0/1835723597 learned_addr learned my addr 172.21.15.39:0/1835723597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f055ffff640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0560151960 0x7f0560141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f056578c640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0560157d80 0x7f0560141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f0564f8b640 1 -- 172.21.15.39:0/1835723597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0560157d80 msgr2=0x7f0560141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f0564f8b640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0560157d80 0x7f0560141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f0564f8b640 1 -- 172.21.15.39:0/1835723597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0560151960 msgr2=0x7f0560141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f0564f8b640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0560151960 0x7f0560141a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f0564f8b640 1 -- 172.21.15.39:0/1835723597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c009420 con 0x7f05600c7a90 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f056578c640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0560157d80 0x7f0560141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f0564f8b640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 0x7f056014a3e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f055400c530 tx=0x7f055400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f055d7fa640 1 -- 172.21.15.39:0/1835723597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554005cf0 con 0x7f05600c7a90 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f055d7fa640 1 -- 172.21.15.39:0/1835723597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0554005e50 con 0x7f05600c7a90 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056014bd90 con 0x7f05600c7a90 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f055d7fa640 1 -- 172.21.15.39:0/1835723597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554007270 con 0x7f05600c7a90 2024-09-13T16:53:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05600aae60 con 0x7f05600c7a90 2024-09-13T16:53:11.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0560153f00 con 0x7f05600c7a90 2024-09-13T16:53:11.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.555+0000 7f055d7fa640 1 -- 172.21.15.39:0/1835723597 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f05540073d0 con 0x7f05600c7a90 2024-09-13T16:53:11.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.559+0000 7f055d7fa640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0544066690 0x7f0544068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:11.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.559+0000 7f055d7fa640 1 -- 172.21.15.39:0/1835723597 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7f0554083ae0 con 0x7f05600c7a90 2024-09-13T16:53:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.559+0000 7f055ffff640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0544066690 0x7f0544068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.559+0000 7f055ffff640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0544066690 0x7f0544068b50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f054c0ad000 tx=0x7f054c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:11.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.563+0000 7f055d7fa640 1 -- 172.21.15.39:0/1835723597 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f05540509e0 con 0x7f05600c7a90 2024-09-13T16:53:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.811+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05600c7870 con 0x7f05600c7a90 2024-09-13T16:53:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.811+0000 7f055d7fa640 1 -- 172.21.15.39:0/1835723597 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f0554054000 con 0x7f05600c7a90 2024-09-13T16:53:11.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:11.817 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:11.818 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 45 2024-09-13T16:53:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0544066690 msgr2=0x7f0544068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0544066690 0x7f0544068b50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f054c0ad000 tx=0x7f054c008040 comp rx=0 tx=0).stop 2024-09-13T16:53:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 msgr2=0x7f056014a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 0x7f056014a3e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f055400c530 tx=0x7f055400c9f0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 shutdown_connections 2024-09-13T16:53:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0544066690 0x7f0544068b50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0560151960 0x7f0560141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0560157d80 0x7f0560141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 --2- 172.21.15.39:0/1835723597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05600c7a90 0x7f056014a3e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 >> 172.21.15.39:0/1835723597 conn(0x7f05600ab500 msgr2=0x7f05600ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 shutdown_connections 2024-09-13T16:53:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:11.815+0000 7f05671ef640 1 -- 172.21.15.39:0/1835723597 wait complete. 2024-09-13T16:53:11.837 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:53:13.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:13.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.979+0000 7f87d9e61640 1 Processor -- start 2024-09-13T16:53:13.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.979+0000 7f87d9e61640 1 -- start start 2024-09-13T16:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d9e61640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f87d40c4710 0x7f87d40c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d9e61640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f87d40c5fd0 0x7f87d415b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d9e61640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d40c76a0 con 0x7f87d414aa40 2024-09-13T16:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d9e61640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d4151b30 con 0x7f87d40c5fd0 2024-09-13T16:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d9e61640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d4151570 con 0x7f87d40c4710 2024-09-13T16:53:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d3fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f87d40c5fd0 0x7f87d415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d3fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f87d40c5fd0 0x7f87d415b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48738/0 (socket says 172.21.15.39:48738) 2024-09-13T16:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d3fff640 1 -- 172.21.15.39:0/2145138950 learned_addr learned my addr 172.21.15.39:0/2145138950 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d3fff640 1 -- 172.21.15.39:0/2145138950 >> v2:172.21.15.84:3300/0 conn(0x7f87d40c4710 msgr2=0x7f87d40c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d3fff640 1 --2- 172.21.15.39:0/2145138950 >> v2:172.21.15.84:3300/0 conn(0x7f87d40c4710 0x7f87d40c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d3fff640 1 -- 172.21.15.39:0/2145138950 >> v1:172.21.15.84:6789/0 conn(0x7f87d414aa40 legacy=0x7f87d40c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:13.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.983+0000 7f87d3fff640 1 -- 172.21.15.39:0/2145138950 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4151850 con 0x7f87d40c5fd0 2024-09-13T16:53:13.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d3fff640 1 --2- 172.21.15.39:0/2145138950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 0x7f87d415b470 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f87d4072610 tx=0x7f87cc01eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3daf48be86e54523 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d27fc640 1 -- 172.21.15.39:0/2145138950 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00b2d0 con 0x7f87d40c5fd0 2024-09-13T16:53:13.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d27fc640 1 -- 172.21.15.39:0/2145138950 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc00b430 con 0x7f87d40c5fd0 2024-09-13T16:53:13.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d27fc640 1 -- 172.21.15.39:0/2145138950 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0a7ae0 con 0x7f87d40c5fd0 2024-09-13T16:53:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 -- 172.21.15.39:0/2145138950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 msgr2=0x7f87d415b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 --2- 172.21.15.39:0/2145138950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 0x7f87d415b470 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f87d4072610 tx=0x7f87cc01eaf0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 -- 172.21.15.39:0/2145138950 shutdown_connections 2024-09-13T16:53:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 --2- 172.21.15.39:0/2145138950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 0x7f87d415b470 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 --2- 172.21.15.39:0/2145138950 >> v2:172.21.15.84:3300/0 conn(0x7f87d40c4710 0x7f87d40c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 -- 172.21.15.39:0/2145138950 >> 172.21.15.39:0/2145138950 conn(0x7f87d40c3310 msgr2=0x7f87d40c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 -- 172.21.15.39:0/2145138950 shutdown_connections 2024-09-13T16:53:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 -- 172.21.15.39:0/2145138950 wait complete. 2024-09-13T16:53:13.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 Processor -- start 2024-09-13T16:53:13.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.987+0000 7f87d9e61640 1 -- start start 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d9e61640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f87d40c4710 0x7f87d4149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d9e61640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 0x7f87d4143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d9e61640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87d414aa40 0x7f87d4144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d9e61640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4155010 con 0x7f87d40c5fd0 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d9e61640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4152640 con 0x7f87d414aa40 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d9e61640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d41439a0 con 0x7f87d40c4710 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 0x7f87d4143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 0x7f87d4143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48752/0 (socket says 172.21.15.39:48752) 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d2ffd640 1 -- 172.21.15.39:0/4153734480 learned_addr learned my addr 172.21.15.39:0/4153734480 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d3fff640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87d414aa40 0x7f87d4144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d37fe640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f87d40c4710 0x7f87d4149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d2ffd640 1 -- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f87d40c4710 msgr2=0x7f87d4149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d2ffd640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f87d40c4710 0x7f87d4149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d2ffd640 1 -- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87d414aa40 msgr2=0x7f87d4144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d2ffd640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87d414aa40 0x7f87d4144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d2ffd640 1 -- 172.21.15.39:0/4153734480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc00bd40 con 0x7f87d40c5fd0 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d37fe640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f87d40c4710 0x7f87d4149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d3fff640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87d414aa40 0x7f87d4144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d2ffd640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 0x7f87d4143e30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b440 tx=0x7f87c800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87bbfff640 1 -- 172.21.15.39:0/4153734480 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8011890 con 0x7f87d40c5fd0 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87bbfff640 1 -- 172.21.15.39:0/4153734480 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8011a10 con 0x7f87d40c5fd0 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87bbfff640 1 -- 172.21.15.39:0/4153734480 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8011dc0 con 0x7f87d40c5fd0 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d4152080 con 0x7f87d40c5fd0 2024-09-13T16:53:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d4154410 con 0x7f87d40c5fd0 2024-09-13T16:53:13.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d4143b50 con 0x7f87d40c5fd0 2024-09-13T16:53:13.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87bbfff640 1 -- 172.21.15.39:0/4153734480 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f87c801a400 con 0x7f87d40c5fd0 2024-09-13T16:53:13.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87bbfff640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f87c0066690 0x7f87c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87bbfff640 1 -- 172.21.15.39:0/4153734480 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 7153+0+0 (secure 0 0 0) 0x7f87c8084af0 con 0x7f87d40c5fd0 2024-09-13T16:53:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.991+0000 7f87d37fe640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f87c0066690 0x7f87c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.995+0000 7f87d37fe640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f87c0066690 0x7f87c0068b50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87c40016e0 tx=0x7f87c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:14.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:13.999+0000 7f87bbfff640 1 -- 172.21.15.39:0/4153734480 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f87d4143b50 con 0x7f87d40c5fd0 2024-09-13T16:53:14.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.295+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87d4143b10 con 0x7f87d40c5fd0 2024-09-13T16:53:14.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.295+0000 7f87bbfff640 1 -- 172.21.15.39:0/4153734480 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f87c8015090 con 0x7f87d40c5fd0 2024-09-13T16:53:14.302 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:14.302 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:52:27.811143+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":31,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6835/2065196144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2065196144},{"type":"v1","addr":"172.21.15.39:6835","nonce":2065196144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:14.302 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 45 2024-09-13T16:53:14.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f87c0066690 msgr2=0x7f87c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:14.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f87c0066690 0x7f87c0068b50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87c40016e0 tx=0x7f87c4009040 comp rx=0 tx=0).stop 2024-09-13T16:53:14.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 msgr2=0x7f87d4143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:14.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 0x7f87d4143e30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f87c800b440 tx=0x7f87c800b900 comp rx=0 tx=0).stop 2024-09-13T16:53:14.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 shutdown_connections 2024-09-13T16:53:14.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f87c0066690 0x7f87c0068b50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:14.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87d414aa40 0x7f87d4144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:14.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87d40c5fd0 0x7f87d4143e30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:14.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 --2- 172.21.15.39:0/4153734480 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f87d40c4710 0x7f87d4149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:14.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 >> 172.21.15.39:0/4153734480 conn(0x7f87d40c3310 msgr2=0x7f87d406a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:14.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 shutdown_connections 2024-09-13T16:53:14.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:14.299+0000 7f87d9e61640 1 -- 172.21.15.39:0/4153734480 wait complete. 2024-09-13T16:53:14.323 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:53:16.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:16.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.463+0000 7fae215ba640 1 Processor -- start 2024-09-13T16:53:16.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.463+0000 7fae215ba640 1 -- start start 2024-09-13T16:53:16.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae215ba640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c151860 0x7fae1c157d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae215ba640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fae1c0611c0 0x7fae1c0bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae215ba640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c0c1340 con 0x7fae1c0bf4f0 2024-09-13T16:53:16.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae215ba640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c1578f0 con 0x7fae1c0611c0 2024-09-13T16:53:16.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae215ba640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1582c0 con 0x7fae1c151860 2024-09-13T16:53:16.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae1b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fae1c0611c0 0x7fae1c0bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae1b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fae1c0611c0 0x7fae1c0bfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48758/0 (socket says 172.21.15.39:48758) 2024-09-13T16:53:16.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae1b7fe640 1 -- 172.21.15.39:0/1664815443 learned_addr learned my addr 172.21.15.39:0/1664815443 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:16.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae1affd640 1 --2- 172.21.15.39:0/1664815443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c151860 0x7fae1c157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae1b7fe640 1 -- 172.21.15.39:0/1664815443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c151860 msgr2=0x7fae1c157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:16.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae1b7fe640 1 --2- 172.21.15.39:0/1664815443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c151860 0x7fae1c157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae1b7fe640 1 -- 172.21.15.39:0/1664815443 >> v1:172.21.15.84:6789/0 conn(0x7fae1c0bf4f0 legacy=0x7fae1c153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.467+0000 7fae1b7fe640 1 -- 172.21.15.39:0/1664815443 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c0c1590 con 0x7fae1c0611c0 2024-09-13T16:53:16.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae1b7fe640 1 --2- 172.21.15.39:0/1664815443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 0x7fae1c0bfad0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fae14009e50 tx=0x7fae1401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b161cb1cd1375086 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:16.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae19ffb640 1 -- 172.21.15.39:0/1664815443 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140a5830 con 0x7fae1c0611c0 2024-09-13T16:53:16.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae19ffb640 1 -- 172.21.15.39:0/1664815443 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae140a5990 con 0x7fae1c0611c0 2024-09-13T16:53:16.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae19ffb640 1 -- 172.21.15.39:0/1664815443 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140a5d20 con 0x7fae1c0611c0 2024-09-13T16:53:16.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 -- 172.21.15.39:0/1664815443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 msgr2=0x7fae1c0bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 --2- 172.21.15.39:0/1664815443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 0x7fae1c0bfad0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fae14009e50 tx=0x7fae1401c970 comp rx=0 tx=0).stop 2024-09-13T16:53:16.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 -- 172.21.15.39:0/1664815443 shutdown_connections 2024-09-13T16:53:16.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 --2- 172.21.15.39:0/1664815443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 0x7fae1c0bfad0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 --2- 172.21.15.39:0/1664815443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c151860 0x7fae1c157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 -- 172.21.15.39:0/1664815443 >> 172.21.15.39:0/1664815443 conn(0x7fae1c069e10 msgr2=0x7fae1c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:16.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 -- 172.21.15.39:0/1664815443 shutdown_connections 2024-09-13T16:53:16.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 -- 172.21.15.39:0/1664815443 wait complete. 2024-09-13T16:53:16.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 Processor -- start 2024-09-13T16:53:16.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.471+0000 7fae215ba640 1 -- start start 2024-09-13T16:53:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae215ba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 0x7fae1c14e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae215ba640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c0bf4f0 0x7fae1c1f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae215ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae1c151860 0x7fae1c1f0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae215ba640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0ab7e0 con 0x7fae1c0611c0 2024-09-13T16:53:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae215ba640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1efc30 con 0x7fae1c151860 2024-09-13T16:53:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae215ba640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0aa330 con 0x7fae1c0bf4f0 2024-09-13T16:53:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 0x7fae1c14e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 0x7fae1c14e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48772/0 (socket says 172.21.15.39:48772) 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1affd640 1 -- 172.21.15.39:0/4014081035 learned_addr learned my addr 172.21.15.39:0/4014081035 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1a7fc640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c0bf4f0 0x7fae1c1f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1b7fe640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae1c151860 0x7fae1c1f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1affd640 1 -- 172.21.15.39:0/4014081035 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c0bf4f0 msgr2=0x7fae1c1f00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1affd640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c0bf4f0 0x7fae1c1f00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1affd640 1 -- 172.21.15.39:0/4014081035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae1c151860 msgr2=0x7fae1c1f0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1affd640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae1c151860 0x7fae1c1f0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1affd640 1 -- 172.21.15.39:0/4014081035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14009b30 con 0x7fae1c0611c0 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1affd640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 0x7fae1c14e470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00d800 tx=0x7fae0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae037fe640 1 -- 172.21.15.39:0/4014081035 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c00b6c0 con 0x7fae1c0611c0 2024-09-13T16:53:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae037fe640 1 -- 172.21.15.39:0/4014081035 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c00b820 con 0x7fae1c0611c0 2024-09-13T16:53:16.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c14b850 con 0x7fae1c0611c0 2024-09-13T16:53:16.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae037fe640 1 -- 172.21.15.39:0/4014081035 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c00bb90 con 0x7fae1c0611c0 2024-09-13T16:53:16.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c0ab500 con 0x7fae1c0611c0 2024-09-13T16:53:16.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c0bfad0 con 0x7fae1c0611c0 2024-09-13T16:53:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae037fe640 1 -- 172.21.15.39:0/4014081035 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae0c00bcf0 con 0x7fae1c0611c0 2024-09-13T16:53:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae037fe640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae08066690 0x7fae08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae037fe640 1 -- 172.21.15.39:0/4014081035 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7fae0c084cf0 con 0x7fae1c0611c0 2024-09-13T16:53:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.475+0000 7fae1a7fc640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae08066690 0x7fae08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.479+0000 7fae1a7fc640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae08066690 0x7fae08068b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fae1c156f60 tx=0x7fae10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.483+0000 7fae037fe640 1 -- 172.21.15.39:0/4014081035 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fae0c089070 con 0x7fae1c0611c0 2024-09-13T16:53:16.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae1c0c0f10 con 0x7fae1c0611c0 2024-09-13T16:53:16.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae037fe640 1 -- 172.21.15.39:0/4014081035 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5178 (secure 0 0 0) 0x7fae0c051af0 con 0x7fae1c0611c0 2024-09-13T16:53:16.691 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:16.691 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:15.737152+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:16.691 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 47 2024-09-13T16:53:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae08066690 msgr2=0x7fae08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae08066690 0x7fae08068b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fae1c156f60 tx=0x7fae10009040 comp rx=0 tx=0).stop 2024-09-13T16:53:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 msgr2=0x7fae1c14e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 0x7fae1c14e470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00d800 tx=0x7fae0c00dcc0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 shutdown_connections 2024-09-13T16:53:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae08066690 0x7fae08068b50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae1c151860 0x7fae1c1f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae1c0bf4f0 0x7fae1c1f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 --2- 172.21.15.39:0/4014081035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae1c0611c0 0x7fae1c14e470 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 >> 172.21.15.39:0/4014081035 conn(0x7fae1c069e10 msgr2=0x7fae1c070160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 shutdown_connections 2024-09-13T16:53:16.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.687+0000 7fae215ba640 1 -- 172.21.15.39:0/4014081035 wait complete. 2024-09-13T16:53:16.710 INFO:tasks.mds_thrash.fs.[b]:mds.a down, removed from mdsmap 2024-09-13T16:53:16.710 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T16:53:16.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4943676640 1 Processor -- start 2024-09-13T16:53:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4943676640 1 -- start start 2024-09-13T16:53:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4943676640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c06ae20 0x7f493c0c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4943676640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f493c157bb0 0x7f493c158000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4943676640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c0c7610 con 0x7f493c071790 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4943676640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493c158540 con 0x7f493c157bb0 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4943676640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0c7330 con 0x7f493c06ae20 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4941412640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f493c071790 0x7f493c0c2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48000/0 (socket says 172.21.15.39:48000) 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4941412640 1 -- 172.21.15.39:0/3024038030 learned_addr learned my addr 172.21.15.39:0/3024038030 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4940c11640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c06ae20 0x7f493c0c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4941c13640 1 --2- 172.21.15.39:0/3024038030 >> v2:172.21.15.84:3300/0 conn(0x7f493c157bb0 0x7f493c158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872091372 0 0) 0x7f493c0c7610 con 0x7f493c071790 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4928003530 con 0x7f493c071790 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681837041 0 0) 0x7f4928003530 con 0x7f493c071790 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c06ae20 msgr2=0x7f493c0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c06ae20 0x7f493c0c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 >> v2:172.21.15.84:3300/0 conn(0x7f493c157bb0 msgr2=0x7f493c158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 --2- 172.21.15.39:0/3024038030 >> v2:172.21.15.84:3300/0 conn(0x7f493c157bb0 0x7f493c158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c0c2410 con 0x7f493c071790 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4938001680 con 0x7f493c071790 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 >> v1:172.21.15.39:6789/0 conn(0x7f493c071790 legacy=0x7f493c0c2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4928003980 0x7f4928003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49280047d0 0x7f4928004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c06ae20 0x7f4928005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c158540 con 0x7f49280047d0 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0c7330 con 0x7f4928003980 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492800b9c0 con 0x7f493c06ae20 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4940c11640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49280047d0 0x7f4928004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4941412640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4928003980 0x7f4928003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4941c13640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c06ae20 0x7f4928005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4940c11640 1 -- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c06ae20 msgr2=0x7f4928005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4940c11640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c06ae20 0x7f4928005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4940c11640 1 -- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4928003980 msgr2=0x7f4928003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4940c11640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4928003980 0x7f4928003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.835+0000 7f4940c11640 1 -- 172.21.15.39:0/3024038030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928003530 con 0x7f49280047d0 2024-09-13T16:53:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4940c11640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49280047d0 0x7f4928004c50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f492c009ae0 tx=0x7f492c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f265b2161ba46da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c0a5530 con 0x7f49280047d0 2024-09-13T16:53:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c0a5690 con 0x7f49280047d0 2024-09-13T16:53:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4933fff640 1 -- 172.21.15.39:0/3024038030 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c0a5a40 con 0x7f49280047d0 2024-09-13T16:53:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4943676640 1 -- 172.21.15.39:0/3024038030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49280047d0 msgr2=0x7f4928004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4943676640 1 --2- 172.21.15.39:0/3024038030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49280047d0 0x7f4928004c50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f492c009ae0 tx=0x7f492c01c9e0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4941c13640 1 -- 172.21.15.39:0/3024038030 reap_dead start 2024-09-13T16:53:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4943676640 1 -- 172.21.15.39:0/3024038030 shutdown_connections 2024-09-13T16:53:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4943676640 1 -- 172.21.15.39:0/3024038030 >> 172.21.15.39:0/3024038030 conn(0x7f493c0a96b0 msgr2=0x7f493c0a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4943676640 1 -- 172.21.15.39:0/3024038030 shutdown_connections 2024-09-13T16:53:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4943676640 1 -- 172.21.15.39:0/3024038030 wait complete. 2024-09-13T16:53:16.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4943676640 1 Processor -- start 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.839+0000 7f4943676640 1 -- start start 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4943676640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49280047d0 0x7f493c15ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4943676640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c155390 0x7f493c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4943676640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f493c1419c0 0x7f493c141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4943676640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c1ed170 con 0x7f493c1419c0 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4943676640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c155d50 con 0x7f49280047d0 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4943676640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c1ec530 con 0x7f493c155390 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941c13640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f493c1419c0 0x7f493c141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941c13640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f493c1419c0 0x7f493c141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48794/0 (socket says 172.21.15.39:48794) 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941c13640 1 -- 172.21.15.39:0/455264172 learned_addr learned my addr 172.21.15.39:0/455264172 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4940c11640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c155390 0x7f493c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941412640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49280047d0 0x7f493c15ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941c13640 1 -- 172.21.15.39:0/455264172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c155390 msgr2=0x7f493c155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941c13640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c155390 0x7f493c155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941c13640 1 -- 172.21.15.39:0/455264172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49280047d0 msgr2=0x7f493c15ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941c13640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49280047d0 0x7f493c15ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941c13640 1 -- 172.21.15.39:0/455264172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c1ec7d0 con 0x7f493c1419c0 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4940c11640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c155390 0x7f493c155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941c13640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f493c1419c0 0x7f493c141e40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f493c06fd20 tx=0x7f49340115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4931ffb640 1 -- 172.21.15.39:0/455264172 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400e070 con 0x7f493c1419c0 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4931ffb640 1 -- 172.21.15.39:0/455264172 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934011e40 con 0x7f493c1419c0 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c1ecab0 con 0x7f493c1419c0 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4931ffb640 1 -- 172.21.15.39:0/455264172 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340039a0 con 0x7f493c1419c0 2024-09-13T16:53:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c15e310 con 0x7f493c1419c0 2024-09-13T16:53:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c071b70 con 0x7f493c1419c0 2024-09-13T16:53:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4931ffb640 1 -- 172.21.15.39:0/455264172 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4934003b20 con 0x7f493c1419c0 2024-09-13T16:53:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4931ffb640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4920066690 0x7f4920068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4931ffb640 1 -- 172.21.15.39:0/455264172 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7f4934084e20 con 0x7f493c1419c0 2024-09-13T16:53:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.843+0000 7f4941412640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4920066690 0x7f4920068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.847+0000 7f4941412640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4920066690 0x7f4920068b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4938000f70 tx=0x7f4938009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:16.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:16.851+0000 7f4931ffb640 1 -- 172.21.15.39:0/455264172 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4934051c20 con 0x7f493c1419c0 2024-09-13T16:53:17.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.123+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f493c15e790 con 0x7f493c1419c0 2024-09-13T16:53:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.123+0000 7f4931ffb640 1 -- 172.21.15.39:0/455264172 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f4934084090 con 0x7f493c1419c0 2024-09-13T16:53:17.129 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:17.129 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:16.747138+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:rejoin","state_seq":5,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 48 2024-09-13T16:53:17.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4920066690 msgr2=0x7f4920068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:17.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4920066690 0x7f4920068b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4938000f70 tx=0x7f4938009040 comp rx=0 tx=0).stop 2024-09-13T16:53:17.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f493c1419c0 msgr2=0x7f493c141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:17.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f493c1419c0 0x7f493c141e40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f493c06fd20 tx=0x7f49340115a0 comp rx=0 tx=0).stop 2024-09-13T16:53:17.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 shutdown_connections 2024-09-13T16:53:17.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4920066690 0x7f4920068b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:17.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f493c1419c0 0x7f493c141e40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:17.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f493c155390 0x7f493c155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:17.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 --2- 172.21.15.39:0/455264172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49280047d0 0x7f493c15ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:17.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 >> 172.21.15.39:0/455264172 conn(0x7f493c0a96b0 msgr2=0x7f493c152e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:17.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 shutdown_connections 2024-09-13T16:53:17.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:17.127+0000 7f4943676640 1 -- 172.21.15.39:0/455264172 wait complete. 2024-09-13T16:53:17.156 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 48, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 44, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_4922": { "addr": "172.21.15.84:6833/1355376480", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 1355376480, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 1355376480, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4922, "incarnation": 41, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 32, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:53:02.615795+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4922 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 48, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5362": { "addr": "172.21.15.84:6835/3600322972", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 3600322972, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 3600322972, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5362, "incarnation": 46, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:rejoin", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 33, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:53:16.747138+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5362 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:53:19.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71feceb640 1 Processor -- start 2024-09-13T16:53:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71feceb640 1 -- start start 2024-09-13T16:53:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71feceb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 0x7f71f80aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71feceb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71f806aac0 0x7f71f8151c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71feceb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f80c3ae0 con 0x7f71f814b850 2024-09-13T16:53:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71feceb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f80ab340 con 0x7f71f806aac0 2024-09-13T16:53:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71feceb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f80abb30 con 0x7f71f80c21d0 2024-09-13T16:53:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71f7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71f806aac0 0x7f71f8151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71fd288640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f71f814b850 0x7f71f8155230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48004/0 (socket says 172.21.15.39:48004) 2024-09-13T16:53:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71fd288640 1 -- 172.21.15.39:0/767407949 learned_addr learned my addr 172.21.15.39:0/767407949 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.303+0000 7f71fca87640 1 --2- 172.21.15.39:0/767407949 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 0x7f71f80aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:19.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f77fe640 1 -- 172.21.15.39:0/767407949 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262828143 0 0) 0x7f71f80c3ae0 con 0x7f71f814b850 2024-09-13T16:53:19.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f77fe640 1 -- 172.21.15.39:0/767407949 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71e8003530 con 0x7f71f814b850 2024-09-13T16:53:19.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f7fff640 1 -- 172.21.15.39:0/767407949 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 msgr2=0x7f71f80aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:19.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f7fff640 1 --2- 172.21.15.39:0/767407949 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 0x7f71f80aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f7fff640 1 -- 172.21.15.39:0/767407949 >> v1:172.21.15.39:6789/0 conn(0x7f71f814b850 legacy=0x7f71f8155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:19.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f7fff640 1 -- 172.21.15.39:0/767407949 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f80abe10 con 0x7f71f806aac0 2024-09-13T16:53:19.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f7fff640 1 --2- 172.21.15.39:0/767407949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f806aac0 0x7f71f8151c40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f71e40097c0 tx=0x7f71e401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecb35ca63469a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:19.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f77fe640 1 -- 172.21.15.39:0/767407949 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40ab600 con 0x7f71f806aac0 2024-09-13T16:53:19.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f77fe640 1 -- 172.21.15.39:0/767407949 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e40a7050 con 0x7f71f806aac0 2024-09-13T16:53:19.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71f77fe640 1 -- 172.21.15.39:0/767407949 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e40ab980 con 0x7f71f806aac0 2024-09-13T16:53:19.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71feceb640 1 -- 172.21.15.39:0/767407949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f806aac0 msgr2=0x7f71f8151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:19.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71feceb640 1 --2- 172.21.15.39:0/767407949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f806aac0 0x7f71f8151c40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f71e40097c0 tx=0x7f71e401c3e0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71feceb640 1 -- 172.21.15.39:0/767407949 shutdown_connections 2024-09-13T16:53:19.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71feceb640 1 --2- 172.21.15.39:0/767407949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f806aac0 0x7f71f8151c40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71feceb640 1 --2- 172.21.15.39:0/767407949 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 0x7f71f80aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71feceb640 1 -- 172.21.15.39:0/767407949 >> 172.21.15.39:0/767407949 conn(0x7f71f8069e10 msgr2=0x7f71f806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:19.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71feceb640 1 -- 172.21.15.39:0/767407949 shutdown_connections 2024-09-13T16:53:19.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.307+0000 7f71feceb640 1 -- 172.21.15.39:0/767407949 wait complete. 2024-09-13T16:53:19.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 Processor -- start 2024-09-13T16:53:19.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 -- start start 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f806aac0 0x7f71f81ee1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 0x7f71f80bf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f814b850 0x7f71f80c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81edd20 con 0x7f71f814b850 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f80c01f0 con 0x7f71f806aac0 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81ed710 con 0x7f71f80c21d0 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fd288640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f814b850 0x7f71f80c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fd288640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f814b850 0x7f71f80c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48818/0 (socket says 172.21.15.39:48818) 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fd288640 1 -- 172.21.15.39:0/3873135603 learned_addr learned my addr 172.21.15.39:0/3873135603 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71f7fff640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 0x7f71f80bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fca87640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f806aac0 0x7f71f81ee1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fd288640 1 -- 172.21.15.39:0/3873135603 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 msgr2=0x7f71f80bf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:19.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fd288640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 0x7f71f80bf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fd288640 1 -- 172.21.15.39:0/3873135603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f806aac0 msgr2=0x7f71f81ee1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:19.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fd288640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f806aac0 0x7f71f81ee1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fd288640 1 -- 172.21.15.39:0/3873135603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e4009420 con 0x7f71f814b850 2024-09-13T16:53:19.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71f7fff640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 0x7f71f80bf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:19.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71fd288640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f814b850 0x7f71f80c4950 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f71f000caa0 tx=0x7f71f000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:19.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71f57fa640 1 -- 172.21.15.39:0/3873135603 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f000ac00 con 0x7f71f814b850 2024-09-13T16:53:19.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71f57fa640 1 -- 172.21.15.39:0/3873135603 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f000ad60 con 0x7f71f814b850 2024-09-13T16:53:19.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f81ed9b0 con 0x7f71f814b850 2024-09-13T16:53:19.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71f57fa640 1 -- 172.21.15.39:0/3873135603 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0012620 con 0x7f71f814b850 2024-09-13T16:53:19.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.311+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f814f6e0 con 0x7f71f814b850 2024-09-13T16:53:19.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.315+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f80c3560 con 0x7f71f814b850 2024-09-13T16:53:19.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.315+0000 7f71f57fa640 1 -- 172.21.15.39:0/3873135603 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f71f0007070 con 0x7f71f814b850 2024-09-13T16:53:19.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.315+0000 7f71f57fa640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f71dc0666c0 0x7f71dc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:19.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.315+0000 7f71f57fa640 1 -- 172.21.15.39:0/3873135603 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7f71f00845c0 con 0x7f71f814b850 2024-09-13T16:53:19.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.315+0000 7f71fca87640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f71dc0666c0 0x7f71dc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:19.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.315+0000 7f71fca87640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f71dc0666c0 0x7f71dc068b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009b60 tx=0x7f71ec009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:19.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.319+0000 7f71f57fa640 1 -- 172.21.15.39:0/3873135603 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f71f0051470 con 0x7f71f814b850 2024-09-13T16:53:19.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.531+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71f80aaf50 con 0x7f71f814b850 2024-09-13T16:53:19.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.531+0000 7f71f57fa640 1 -- 172.21.15.39:0/3873135603 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f71f0054a90 con 0x7f71f814b850 2024-09-13T16:53:19.532 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:19.532 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:19.532 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 49 2024-09-13T16:53:19.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f71dc0666c0 msgr2=0x7f71dc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:19.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f71dc0666c0 0x7f71dc068b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009b60 tx=0x7f71ec009190 comp rx=0 tx=0).stop 2024-09-13T16:53:19.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f814b850 msgr2=0x7f71f80c4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:19.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f814b850 0x7f71f80c4950 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f71f000caa0 tx=0x7f71f000cf60 comp rx=0 tx=0).stop 2024-09-13T16:53:19.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 shutdown_connections 2024-09-13T16:53:19.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f71dc0666c0 0x7f71dc068b80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f814b850 0x7f71f80c4950 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71f80c21d0 0x7f71f80bf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 --2- 172.21.15.39:0/3873135603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71f806aac0 0x7f71f81ee1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:19.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 >> 172.21.15.39:0/3873135603 conn(0x7f71f8069e10 msgr2=0x7f71f8155040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:19.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 shutdown_connections 2024-09-13T16:53:19.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:19.535+0000 7f71feceb640 1 -- 172.21.15.39:0/3873135603 wait complete. 2024-09-13T16:53:19.555 INFO:tasks.mds_thrash.fs.[b]:mds.d has gained rank=0, replacing gid=5418 2024-09-13T16:53:19.555 INFO:tasks.mds_thrash.fs.[b]:waiting for 1 secs before reviving mds.a 2024-09-13T16:53:20.556 INFO:tasks.mds_thrash.fs.[b]:reviving mds.a 2024-09-13T16:53:20.557 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:53:20.557 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:53:20.559 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:53:20.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:20.622 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T16:53:20.651 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T16:53:20.645+0000 7fd9f625d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:53:20.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f207f6640 1 Processor -- start 2024-09-13T16:53:20.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f207f6640 1 -- start start 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f207f6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f180c1810 0x7f5f1806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f207f6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5f181576e0 0x7f5f18157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f207f6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f180c15d0 con 0x7f5f1806b080 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f207f6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1806d0b0 con 0x7f5f181576e0 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f207f6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180c1f00 con 0x7f5f180c1810 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f1e592640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5f1806b080 0x7f5f18151390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33108/0 (socket says 172.21.15.39:33108) 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f1e592640 1 -- 172.21.15.39:0/4033466840 learned_addr learned my addr 172.21.15.39:0/4033466840 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f1dd91640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f180c1810 0x7f5f1806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f1ed93640 1 --2- 172.21.15.39:0/4033466840 >> v2:172.21.15.84:3300/0 conn(0x7f5f181576e0 0x7f5f18157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124575097 0 0) 0x7f5f180c15d0 con 0x7f5f1806b080 2024-09-13T16:53:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.687+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f04003530 con 0x7f5f1806b080 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3192918156 0 0) 0x7f5f04003530 con 0x7f5f1806b080 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f180c1810 msgr2=0x7f5f1806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f180c1810 0x7f5f1806e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 >> v2:172.21.15.84:3300/0 conn(0x7f5f181576e0 msgr2=0x7f5f18157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 --2- 172.21.15.39:0/4033466840 >> v2:172.21.15.84:3300/0 conn(0x7f5f181576e0 0x7f5f18157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f180c21a0 con 0x7f5f1806b080 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f5f14001680 con 0x7f5f1806b080 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 >> v1:172.21.15.39:6789/0 conn(0x7f5f1806b080 legacy=0x7f5f18151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f04003980 0x7f5f04003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f04004370 0x7f5f04004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f180c1810 0x7f5f04004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180c1f00 con 0x7f5f04004370 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0400b990 con 0x7f5f04003980 2024-09-13T16:53:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0400bad0 con 0x7f5f180c1810 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1dd91640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f04004370 0x7f5f04004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1e592640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f04003980 0x7f5f04003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1ed93640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f180c1810 0x7f5f04004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1dd91640 1 -- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f180c1810 msgr2=0x7f5f04004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1dd91640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f180c1810 0x7f5f04004fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1dd91640 1 -- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f04003980 msgr2=0x7f5f04003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1dd91640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f04003980 0x7f5f04003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1dd91640 1 -- 172.21.15.39:0/4033466840 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f180c21a0 con 0x7f5f04004370 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1e592640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f04003980 0x7f5f04003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1dd91640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f04004370 0x7f5f04004850 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009ae0 tx=0x7f5f0801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ef298d5967e127 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080a5520 con 0x7f5f04004370 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f080a5680 con 0x7f5f04004370 2024-09-13T16:53:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1d590640 1 -- 172.21.15.39:0/4033466840 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080a5a30 con 0x7f5f04004370 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f207f6640 1 -- 172.21.15.39:0/4033466840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f04004370 msgr2=0x7f5f04004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f207f6640 1 --2- 172.21.15.39:0/4033466840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f04004370 0x7f5f04004850 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009ae0 tx=0x7f5f0801c9e0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f1ed93640 1 -- 172.21.15.39:0/4033466840 reap_dead start 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f207f6640 1 -- 172.21.15.39:0/4033466840 shutdown_connections 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f207f6640 1 -- 172.21.15.39:0/4033466840 >> 172.21.15.39:0/4033466840 conn(0x7f5f180c32a0 msgr2=0x7f5f180c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f207f6640 1 -- 172.21.15.39:0/4033466840 shutdown_connections 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.691+0000 7f5f207f6640 1 -- 172.21.15.39:0/4033466840 wait complete. 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 Processor -- start 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 -- start start 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f04004370 0x7f5f180bca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f180bcfb0 0x7f5f1814a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f1814aed0 0x7f5f181efe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180c07d0 con 0x7f5f180bcfb0 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180bd430 con 0x7f5f1814aed0 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180bd730 con 0x7f5f04004370 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1dd91640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f180bcfb0 0x7f5f1814a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1dd91640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f180bcfb0 0x7f5f1814a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35570/0 (socket says 172.21.15.39:35570) 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1dd91640 1 -- 172.21.15.39:0/1910238225 learned_addr learned my addr 172.21.15.39:0/1910238225 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1e592640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f04004370 0x7f5f180bca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1ed93640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f1814aed0 0x7f5f181efe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1dd91640 1 -- 172.21.15.39:0/1910238225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f04004370 msgr2=0x7f5f180bca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1dd91640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f04004370 0x7f5f180bca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1dd91640 1 -- 172.21.15.39:0/1910238225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f1814aed0 msgr2=0x7f5f181efe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1dd91640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f1814aed0 0x7f5f181efe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1dd91640 1 -- 172.21.15.39:0/1910238225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1806ea90 con 0x7f5f180bcfb0 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1ed93640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f1814aed0 0x7f5f181efe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f1dd91640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f180bcfb0 0x7f5f1814a990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5f080b0040 tx=0x7f5f04005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f0effd640 1 -- 172.21.15.39:0/1910238225 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0800f000 con 0x7f5f180bcfb0 2024-09-13T16:53:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f0effd640 1 -- 172.21.15.39:0/1910238225 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08004020 con 0x7f5f180bcfb0 2024-09-13T16:53:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f0effd640 1 -- 172.21.15.39:0/1910238225 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080043d0 con 0x7f5f180bcfb0 2024-09-13T16:53:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f181ed4e0 con 0x7f5f180bcfb0 2024-09-13T16:53:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.695+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f18150650 con 0x7f5f180bcfb0 2024-09-13T16:53:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.699+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f18150810 con 0x7f5f180bcfb0 2024-09-13T16:53:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.699+0000 7f5f0effd640 1 -- 172.21.15.39:0/1910238225 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f080b4040 con 0x7f5f180bcfb0 2024-09-13T16:53:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.699+0000 7f5f0effd640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5efc066690 0x7f5efc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.699+0000 7f5f0effd640 1 -- 172.21.15.39:0/1910238225 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7f5f0811e6d0 con 0x7f5f180bcfb0 2024-09-13T16:53:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.699+0000 7f5f1e592640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5efc066690 0x7f5efc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:20.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.699+0000 7f5f1e592640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5efc066690 0x7f5efc068b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f140017f0 tx=0x7f5f14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:20.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.703+0000 7f5f0effd640 1 -- 172.21.15.39:0/1910238225 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5f080eb6a0 con 0x7f5f180bcfb0 2024-09-13T16:53:20.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f18157af0 con 0x7f5f180bcfb0 2024-09-13T16:53:20.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f0effd640 1 -- 172.21.15.39:0/1910238225 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f5f080eecc0 con 0x7f5f180bcfb0 2024-09-13T16:53:20.948 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:20.948 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:20.948 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 50 2024-09-13T16:53:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5efc066690 msgr2=0x7f5efc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5efc066690 0x7f5efc068b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5f140017f0 tx=0x7f5f14009040 comp rx=0 tx=0).stop 2024-09-13T16:53:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f180bcfb0 msgr2=0x7f5f1814a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f180bcfb0 0x7f5f1814a990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5f080b0040 tx=0x7f5f04005800 comp rx=0 tx=0).stop 2024-09-13T16:53:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 shutdown_connections 2024-09-13T16:53:20.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5efc066690 0x7f5efc068b50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f1814aed0 0x7f5f181efe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f180bcfb0 0x7f5f1814a990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 --2- 172.21.15.39:0/1910238225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f04004370 0x7f5f180bca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:20.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 >> 172.21.15.39:0/1910238225 conn(0x7f5f180c32a0 msgr2=0x7f5f180c53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:20.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 shutdown_connections 2024-09-13T16:53:20.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:20.947+0000 7f5f207f6640 1 -- 172.21.15.39:0/1910238225 wait complete. 2024-09-13T16:53:20.966 INFO:tasks.mds_thrash.fs.[b]:mds.a reported in up:standby state 2024-09-13T16:53:20.966 INFO:tasks.mds_thrash.fs.[b]:waiting for 98 secs before thrashing 2024-09-13T16:53:26.397 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:53:26.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:53:55.839 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:55.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1652ff4640 1 Processor -- start 2024-09-13T16:53:55.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1652ff4640 1 -- start start 2024-09-13T16:53:55.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1652ff4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c06fba0 0x7f164c154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:55.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1652ff4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f164c0c1280 0x7f164c0c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:55.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1652ff4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164c0c33c0 con 0x7f164c0c28d0 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1652ff4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164c0611c0 con 0x7f164c0c1280 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1652ff4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164c0c0ac0 con 0x7f164c06fba0 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1651591640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f164c0c1280 0x7f164c0c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1651591640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f164c0c1280 0x7f164c0c16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59522/0 (socket says 172.21.15.39:59522) 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1651591640 1 -- 172.21.15.39:0/3092636182 learned_addr learned my addr 172.21.15.39:0/3092636182 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f164bfff640 1 --2- 172.21.15.39:0/3092636182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c06fba0 0x7f164c154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1651591640 1 -- 172.21.15.39:0/3092636182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c06fba0 msgr2=0x7f164c154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1651591640 1 --2- 172.21.15.39:0/3092636182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c06fba0 0x7f164c154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1651591640 1 -- 172.21.15.39:0/3092636182 >> v1:172.21.15.84:6789/0 conn(0x7f164c0c28d0 legacy=0x7f164c0c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:55.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.983+0000 7f1651591640 1 -- 172.21.15.39:0/3092636182 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c0613d0 con 0x7f164c0c1280 2024-09-13T16:53:55.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1651591640 1 --2- 172.21.15.39:0/3092636182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c0c1280 0x7f164c0c16f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1644009e50 tx=0x7f164401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e250631b8df772ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:55.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f164b7fe640 1 -- 172.21.15.39:0/3092636182 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440a64a0 con 0x7f164c0c1280 2024-09-13T16:53:55.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f164b7fe640 1 -- 172.21.15.39:0/3092636182 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16440a6600 con 0x7f164c0c1280 2024-09-13T16:53:55.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f164b7fe640 1 -- 172.21.15.39:0/3092636182 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440a6990 con 0x7f164c0c1280 2024-09-13T16:53:55.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 -- 172.21.15.39:0/3092636182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c0c1280 msgr2=0x7f164c0c16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:55.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 --2- 172.21.15.39:0/3092636182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c0c1280 0x7f164c0c16f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1644009e50 tx=0x7f164401c6e0 comp rx=0 tx=0).stop 2024-09-13T16:53:55.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 -- 172.21.15.39:0/3092636182 shutdown_connections 2024-09-13T16:53:55.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 --2- 172.21.15.39:0/3092636182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c0c1280 0x7f164c0c16f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:55.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 --2- 172.21.15.39:0/3092636182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c06fba0 0x7f164c154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:55.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 -- 172.21.15.39:0/3092636182 >> 172.21.15.39:0/3092636182 conn(0x7f164c069e10 msgr2=0x7f164c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:55.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 -- 172.21.15.39:0/3092636182 shutdown_connections 2024-09-13T16:53:55.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 -- 172.21.15.39:0/3092636182 wait complete. 2024-09-13T16:53:55.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 Processor -- start 2024-09-13T16:53:55.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 -- start start 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c06fba0 0x7f164c153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c0c1280 0x7f164c144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f164c0c28d0 0x7f164c0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c15c910 con 0x7f164c06fba0 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c148340 con 0x7f164c0c28d0 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1652ff4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164c153eb0 con 0x7f164c0c1280 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1650d90640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c06fba0 0x7f164c153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1650d90640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c06fba0 0x7f164c153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59538/0 (socket says 172.21.15.39:59538) 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1650d90640 1 -- 172.21.15.39:0/426265823 learned_addr learned my addr 172.21.15.39:0/426265823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1651591640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f164c0c28d0 0x7f164c0c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:55.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f164bfff640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c0c1280 0x7f164c144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1650d90640 1 -- 172.21.15.39:0/426265823 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c0c1280 msgr2=0x7f164c144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1650d90640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c0c1280 0x7f164c144490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1650d90640 1 -- 172.21.15.39:0/426265823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f164c0c28d0 msgr2=0x7f164c0c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1650d90640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f164c0c28d0 0x7f164c0c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1650d90640 1 -- 172.21.15.39:0/426265823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1644009b30 con 0x7f164c06fba0 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.987+0000 7f1651591640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f164c0c28d0 0x7f164c0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f1650d90640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c06fba0 0x7f164c153630 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f164000c3c0 tx=0x7f164000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f16497fa640 1 -- 172.21.15.39:0/426265823 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16400052d0 con 0x7f164c06fba0 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f16497fa640 1 -- 172.21.15.39:0/426265823 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1640005430 con 0x7f164c06fba0 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f16497fa640 1 -- 172.21.15.39:0/426265823 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640003840 con 0x7f164c06fba0 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164c14d970 con 0x7f164c06fba0 2024-09-13T16:53:55.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164c1540e0 con 0x7f164c06fba0 2024-09-13T16:53:55.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164c0abd70 con 0x7f164c06fba0 2024-09-13T16:53:55.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f16497fa640 1 -- 172.21.15.39:0/426265823 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f16400039a0 con 0x7f164c06fba0 2024-09-13T16:53:55.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f16497fa640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16300666c0 0x7f1630068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:55.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f16497fa640 1 -- 172.21.15.39:0/426265823 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7f1640083ab0 con 0x7f164c06fba0 2024-09-13T16:53:55.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f164bfff640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16300666c0 0x7f1630068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:55.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.991+0000 7f164bfff640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16300666c0 0x7f1630068b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f164c14b570 tx=0x7f1638008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:55.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:55.999+0000 7f16497fa640 1 -- 172.21.15.39:0/426265823 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f16400508b0 con 0x7f164c06fba0 2024-09-13T16:53:56.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.259+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f164c0c33c0 con 0x7f164c06fba0 2024-09-13T16:53:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.259+0000 7f16497fa640 1 -- 172.21.15.39:0/426265823 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f164c0c33c0 con 0x7f164c06fba0 2024-09-13T16:53:56.260 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:56.260 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:56.261 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 50 2024-09-13T16:53:56.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16300666c0 msgr2=0x7f1630068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16300666c0 0x7f1630068b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f164c14b570 tx=0x7f1638008040 comp rx=0 tx=0).stop 2024-09-13T16:53:56.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c06fba0 msgr2=0x7f164c153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c06fba0 0x7f164c153630 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f164000c3c0 tx=0x7f164000c880 comp rx=0 tx=0).stop 2024-09-13T16:53:56.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 shutdown_connections 2024-09-13T16:53:56.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f16300666c0 0x7f1630068b80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f164c0c28d0 0x7f164c0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f164c0c1280 0x7f164c144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 --2- 172.21.15.39:0/426265823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f164c06fba0 0x7f164c153630 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 >> 172.21.15.39:0/426265823 conn(0x7f164c069e10 msgr2=0x7f164c148a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:56.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 shutdown_connections 2024-09-13T16:53:56.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.263+0000 7f1652ff4640 1 -- 172.21.15.39:0/426265823 wait complete. 2024-09-13T16:53:56.282 INFO:tasks.mds_thrash.fs.[a]:kill mds.b (rank=0) 2024-09-13T16:53:56.282 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:53:56.282 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:53:56.297 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:53:56.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:56.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.419+0000 7fbfc2876640 1 Processor -- start 2024-09-13T16:53:56.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfc2876640 1 -- start start 2024-09-13T16:53:56.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfc2876640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbfbc071dd0 0x7fbfbc154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfc2876640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc0c5a30 con 0x7fbfbc0c7800 2024-09-13T16:53:56.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfc2876640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc1576a0 con 0x7fbfbc151500 2024-09-13T16:53:56.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfc2876640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc157980 con 0x7fbfbc071dd0 2024-09-13T16:53:56.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfb3fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbfbc071dd0 0x7fbfbc154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfb3fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbfbc071dd0 0x7fbfbc154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59540/0 (socket says 172.21.15.39:59540) 2024-09-13T16:53:56.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfb3fff640 1 -- 172.21.15.39:0/2903159370 learned_addr learned my addr 172.21.15.39:0/2903159370 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:56.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfb3fff640 1 -- 172.21.15.39:0/2903159370 >> v1:172.21.15.84:6789/0 conn(0x7fbfbc151500 legacy=0x7fbfbc0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfb3fff640 1 -- 172.21.15.39:0/2903159370 >> v1:172.21.15.39:6789/0 conn(0x7fbfbc0c7800 legacy=0x7fbfbc1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfb3fff640 1 -- 172.21.15.39:0/2903159370 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc157c60 con 0x7fbfbc071dd0 2024-09-13T16:53:56.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.423+0000 7fbfbb7fe640 1 -- 172.21.15.39:0/2903159370 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621993137 0 0) 0x7fbfbc0c5a30 con 0x7fbfbc0c7800 2024-09-13T16:53:56.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfb3fff640 1 --2- 172.21.15.39:0/2903159370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc071dd0 0x7fbfbc154070 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40098a0 tx=0x7fbfa401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d628fafee413c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:56.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfbb7fe640 1 -- 172.21.15.39:0/2903159370 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40a6040 con 0x7fbfbc071dd0 2024-09-13T16:53:56.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfbb7fe640 1 -- 172.21.15.39:0/2903159370 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa40aa600 con 0x7fbfbc071dd0 2024-09-13T16:53:56.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfbb7fe640 1 -- 172.21.15.39:0/2903159370 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40aa990 con 0x7fbfbc071dd0 2024-09-13T16:53:56.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfc2876640 1 -- 172.21.15.39:0/2903159370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc071dd0 msgr2=0x7fbfbc154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfc2876640 1 --2- 172.21.15.39:0/2903159370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc071dd0 0x7fbfbc154070 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40098a0 tx=0x7fbfa401c480 comp rx=0 tx=0).stop 2024-09-13T16:53:56.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfc2876640 1 -- 172.21.15.39:0/2903159370 shutdown_connections 2024-09-13T16:53:56.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfc2876640 1 --2- 172.21.15.39:0/2903159370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc071dd0 0x7fbfbc154070 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfc2876640 1 -- 172.21.15.39:0/2903159370 >> 172.21.15.39:0/2903159370 conn(0x7fbfbc0ab530 msgr2=0x7fbfbc0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:56.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfc2876640 1 -- 172.21.15.39:0/2903159370 shutdown_connections 2024-09-13T16:53:56.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfc2876640 1 -- 172.21.15.39:0/2903159370 wait complete. 2024-09-13T16:53:56.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfc2876640 1 Processor -- start 2024-09-13T16:53:56.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.427+0000 7fbfc2876640 1 -- start start 2024-09-13T16:53:56.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc2876640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfbc071dd0 0x7fbfbc156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc2876640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfbc0c7800 0x7fbfbc14b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc2876640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc151500 0x7fbfbc147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc2876640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc148ad0 con 0x7fbfbc151500 2024-09-13T16:53:56.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc2876640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc149fa0 con 0x7fbfbc071dd0 2024-09-13T16:53:56.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc2876640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc1ec420 con 0x7fbfbc0c7800 2024-09-13T16:53:56.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc0e13640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc151500 0x7fbfbc147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc0e13640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc151500 0x7fbfbc147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59548/0 (socket says 172.21.15.39:59548) 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc0e13640 1 -- 172.21.15.39:0/2378417828 learned_addr learned my addr 172.21.15.39:0/2378417828 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfbbfff640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfbc071dd0 0x7fbfbc156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfb3fff640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfbc0c7800 0x7fbfbc14b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc0e13640 1 -- 172.21.15.39:0/2378417828 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfbc0c7800 msgr2=0x7fbfbc14b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc0e13640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfbc0c7800 0x7fbfbc14b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc0e13640 1 -- 172.21.15.39:0/2378417828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfbc071dd0 msgr2=0x7fbfbc156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc0e13640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfbc071dd0 0x7fbfbc156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc0e13640 1 -- 172.21.15.39:0/2378417828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40094d0 con 0x7fbfbc151500 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc0e13640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc151500 0x7fbfbc147070 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400c710 tx=0x7fbfb400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:56.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfb97fa640 1 -- 172.21.15.39:0/2378417828 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4003b80 con 0x7fbfbc151500 2024-09-13T16:53:56.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfb97fa640 1 -- 172.21.15.39:0/2378417828 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb4003ce0 con 0x7fbfbc151500 2024-09-13T16:53:56.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc148130 con 0x7fbfbc151500 2024-09-13T16:53:56.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfb97fa640 1 -- 172.21.15.39:0/2378417828 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400aa90 con 0x7fbfbc151500 2024-09-13T16:53:56.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc1484c0 con 0x7fbfbc151500 2024-09-13T16:53:56.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc148c10 con 0x7fbfbc151500 2024-09-13T16:53:56.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfb97fa640 1 -- 172.21.15.39:0/2378417828 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbfb400abf0 con 0x7fbfbc151500 2024-09-13T16:53:56.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfb97fa640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf9c0666c0 0x7fbf9c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:56.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfb97fa640 1 -- 172.21.15.39:0/2378417828 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7fbfb4084450 con 0x7fbfbc151500 2024-09-13T16:53:56.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.431+0000 7fbfbbfff640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf9c0666c0 0x7fbf9c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:56.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.435+0000 7fbfbbfff640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf9c0666c0 0x7fbf9c068b80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbfac007fd0 tx=0x7fbfac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:56.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.439+0000 7fbfb97fa640 1 -- 172.21.15.39:0/2378417828 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbfbc148c10 con 0x7fbfbc151500 2024-09-13T16:53:56.443 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:53:56.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:53:56.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.651+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfbc148c10 con 0x7fbfbc151500 2024-09-13T16:53:56.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.651+0000 7fbfb97fa640 1 -- 172.21.15.39:0/2378417828 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fbfbc148c10 con 0x7fbfbc151500 2024-09-13T16:53:56.653 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:56.653 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:56.654 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 50 2024-09-13T16:53:56.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf9c0666c0 msgr2=0x7fbf9c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf9c0666c0 0x7fbf9c068b80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbfac007fd0 tx=0x7fbfac009040 comp rx=0 tx=0).stop 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc151500 msgr2=0x7fbfbc147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc151500 0x7fbfbc147070 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400c710 tx=0x7fbfb400cbd0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 shutdown_connections 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf9c0666c0 0x7fbf9c068b80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfbc151500 0x7fbfbc147070 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfbc0c7800 0x7fbfbc14b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 --2- 172.21.15.39:0/2378417828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfbc071dd0 0x7fbfbc156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 >> 172.21.15.39:0/2378417828 conn(0x7fbfbc0ab530 msgr2=0x7fbfbc150610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 shutdown_connections 2024-09-13T16:53:56.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:56.655+0000 7fbfc2876640 1 -- 172.21.15.39:0/2378417828 wait complete. 2024-09-13T16:53:56.674 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:53:58.675 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:53:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f99c1640 1 Processor -- start 2024-09-13T16:53:58.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f99c1640 1 -- start start 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f99c1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f74f40c1790 0x7f74f4157f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f99c1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f74f406d370 0x7f74f40c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f99c1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f4157370 con 0x7f74f40c20d0 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f99c1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f40c04a0 con 0x7f74f40c1790 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f99c1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f4153df0 con 0x7f74f406d370 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f27fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f74f40c1790 0x7f74f4157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f27fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f74f40c1790 0x7f74f4157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59560/0 (socket says 172.21.15.39:59560) 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f27fc640 1 -- 172.21.15.39:0/1642651054 learned_addr learned my addr 172.21.15.39:0/1642651054 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f37fe640 1 --2- 172.21.15.39:0/1642651054 >> v2:172.21.15.84:3300/0 conn(0x7f74f406d370 0x7f74f40c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f27fc640 1 -- 172.21.15.39:0/1642651054 >> v2:172.21.15.84:3300/0 conn(0x7f74f406d370 msgr2=0x7f74f40c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:53:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f27fc640 1 --2- 172.21.15.39:0/1642651054 >> v2:172.21.15.84:3300/0 conn(0x7f74f406d370 0x7f74f40c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f27fc640 1 -- 172.21.15.39:0/1642651054 >> v1:172.21.15.84:6789/0 conn(0x7f74f40c20d0 legacy=0x7f74f4157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.807+0000 7f74f27fc640 1 -- 172.21.15.39:0/1642651054 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f406c700 con 0x7f74f40c1790 2024-09-13T16:53:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f27fc640 1 --2- 172.21.15.39:0/1642651054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f40c1790 0x7f74f4157f10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74f4061a80 tx=0x7f74dc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c093a69f38f4dcfc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f1ffb640 1 -- 172.21.15.39:0/1642651054 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0aa650 con 0x7f74f40c1790 2024-09-13T16:53:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f1ffb640 1 -- 172.21.15.39:0/1642651054 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc0aa7b0 con 0x7f74f40c1790 2024-09-13T16:53:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f1ffb640 1 -- 172.21.15.39:0/1642651054 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0aab40 con 0x7f74f40c1790 2024-09-13T16:53:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 -- 172.21.15.39:0/1642651054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f40c1790 msgr2=0x7f74f4157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 --2- 172.21.15.39:0/1642651054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f40c1790 0x7f74f4157f10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74f4061a80 tx=0x7f74dc01c4f0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 -- 172.21.15.39:0/1642651054 shutdown_connections 2024-09-13T16:53:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 --2- 172.21.15.39:0/1642651054 >> v2:172.21.15.84:3300/0 conn(0x7f74f406d370 0x7f74f40c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 --2- 172.21.15.39:0/1642651054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f40c1790 0x7f74f4157f10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 -- 172.21.15.39:0/1642651054 >> 172.21.15.39:0/1642651054 conn(0x7f74f4069e10 msgr2=0x7f74f406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 -- 172.21.15.39:0/1642651054 shutdown_connections 2024-09-13T16:53:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 -- 172.21.15.39:0/1642651054 wait complete. 2024-09-13T16:53:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 Processor -- start 2024-09-13T16:53:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 -- start start 2024-09-13T16:53:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f406d370 0x7f74f414aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74f40c1790 0x7f74f414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40c20d0 0x7f74f41ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f414bde0 con 0x7f74f406d370 2024-09-13T16:53:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f414c5b0 con 0x7f74f40c20d0 2024-09-13T16:53:58.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f99c1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f4159610 con 0x7f74f40c1790 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.811+0000 7f74f2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f406d370 0x7f74f414aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f406d370 0x7f74f414aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59566/0 (socket says 172.21.15.39:59566) 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f2ffd640 1 -- 172.21.15.39:0/876026905 learned_addr learned my addr 172.21.15.39:0/876026905 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f37fe640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40c20d0 0x7f74f41ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f27fc640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74f40c1790 0x7f74f414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f2ffd640 1 -- 172.21.15.39:0/876026905 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74f40c1790 msgr2=0x7f74f414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f2ffd640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74f40c1790 0x7f74f414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f2ffd640 1 -- 172.21.15.39:0/876026905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40c20d0 msgr2=0x7f74f41ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f2ffd640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40c20d0 0x7f74f41ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f2ffd640 1 -- 172.21.15.39:0/876026905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc0094d0 con 0x7f74f406d370 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f2ffd640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f406d370 0x7f74f414aa90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c500 tx=0x7f74e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74ebfff640 1 -- 172.21.15.39:0/876026905 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4005220 con 0x7f74f406d370 2024-09-13T16:53:58.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74ebfff640 1 -- 172.21.15.39:0/876026905 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e4005380 con 0x7f74f406d370 2024-09-13T16:53:58.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f41f0540 con 0x7f74f406d370 2024-09-13T16:53:58.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74ebfff640 1 -- 172.21.15.39:0/876026905 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4007270 con 0x7f74f406d370 2024-09-13T16:53:58.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f414c2d0 con 0x7f74f406d370 2024-09-13T16:53:58.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f414dce0 con 0x7f74f406d370 2024-09-13T16:53:58.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74ebfff640 1 -- 172.21.15.39:0/876026905 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74e40073d0 con 0x7f74f406d370 2024-09-13T16:53:58.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74ebfff640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74d4066690 0x7f74d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:53:58.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74ebfff640 1 -- 172.21.15.39:0/876026905 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7f74e4083a40 con 0x7f74f406d370 2024-09-13T16:53:58.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f27fc640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74d4066690 0x7f74d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:53:58.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.815+0000 7f74f27fc640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74d4066690 0x7f74d4068b50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0ae000 tx=0x7f74dc0b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:53:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:58.819+0000 7f74ebfff640 1 -- 172.21.15.39:0/876026905 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74f414dce0 con 0x7f74f406d370 2024-09-13T16:53:59.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74f40c24b0 con 0x7f74f406d370 2024-09-13T16:53:59.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74ebfff640 1 -- 172.21.15.39:0/876026905 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f74e4085e60 con 0x7f74f406d370 2024-09-13T16:53:59.020 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:53:59.020 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:53:59.020 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 50 2024-09-13T16:53:59.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74d4066690 msgr2=0x7f74d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74d4066690 0x7f74d4068b50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0ae000 tx=0x7f74dc0b5040 comp rx=0 tx=0).stop 2024-09-13T16:53:59.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f406d370 msgr2=0x7f74f414aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:53:59.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f406d370 0x7f74f414aa90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f74e400c500 tx=0x7f74e400c9c0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 shutdown_connections 2024-09-13T16:53:59.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74d4066690 0x7f74d4068b50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74f40c20d0 0x7f74f41ef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74f40c1790 0x7f74f414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 --2- 172.21.15.39:0/876026905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74f406d370 0x7f74f414aa90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:53:59.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 >> 172.21.15.39:0/876026905 conn(0x7f74f4069e10 msgr2=0x7f74f4150df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:53:59.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 shutdown_connections 2024-09-13T16:53:59.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:53:59.019+0000 7f74f99c1640 1 -- 172.21.15.39:0/876026905 wait complete. 2024-09-13T16:53:59.038 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:54:01.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.170+0000 7fdbf6c06640 1 Processor -- start 2024-09-13T16:54:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.170+0000 7fdbf6c06640 1 -- start start 2024-09-13T16:54:01.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf6c06640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdbf006c870 0x7fdbf00aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf6c06640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdbf00c40b0 0x7fdbf015c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf6c06640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf0152a80 con 0x7fdbf00aabc0 2024-09-13T16:54:01.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf6c06640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf00c30f0 con 0x7fdbf00c40b0 2024-09-13T16:54:01.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf6c06640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf0151e00 con 0x7fdbf006c870 2024-09-13T16:54:01.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf51a3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdbf00c40b0 0x7fdbf015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf51a3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdbf00c40b0 0x7fdbf015c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41030/0 (socket says 172.21.15.39:41030) 2024-09-13T16:54:01.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf51a3640 1 -- 172.21.15.39:0/2119317522 learned_addr learned my addr 172.21.15.39:0/2119317522 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:01.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbeffff640 1 --2- 172.21.15.39:0/2119317522 >> v2:172.21.15.84:3300/0 conn(0x7fdbf006c870 0x7fdbf00aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf51a3640 1 -- 172.21.15.39:0/2119317522 >> v2:172.21.15.84:3300/0 conn(0x7fdbf006c870 msgr2=0x7fdbf00aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:01.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf51a3640 1 --2- 172.21.15.39:0/2119317522 >> v2:172.21.15.84:3300/0 conn(0x7fdbf006c870 0x7fdbf00aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf51a3640 1 -- 172.21.15.39:0/2119317522 >> v1:172.21.15.84:6789/0 conn(0x7fdbf00aabc0 legacy=0x7fdbf00c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbf51a3640 1 -- 172.21.15.39:0/2119317522 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf01520e0 con 0x7fdbf00c40b0 2024-09-13T16:54:01.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.174+0000 7fdbef7fe640 1 -- 172.21.15.39:0/2119317522 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54452555 0 0) 0x7fdbe4001120 con 0x7fdbf00aabc0 2024-09-13T16:54:01.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf51a3640 1 --2- 172.21.15.39:0/2119317522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00c40b0 0x7fdbf015c470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00bfe70 tx=0x7fdbe801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5091f385fa06b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbef7fe640 1 -- 172.21.15.39:0/2119317522 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80a57f0 con 0x7fdbf00c40b0 2024-09-13T16:54:01.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbef7fe640 1 -- 172.21.15.39:0/2119317522 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe80a5950 con 0x7fdbf00c40b0 2024-09-13T16:54:01.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbef7fe640 1 -- 172.21.15.39:0/2119317522 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80a5ce0 con 0x7fdbf00c40b0 2024-09-13T16:54:01.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- 172.21.15.39:0/2119317522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00c40b0 msgr2=0x7fdbf015c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 --2- 172.21.15.39:0/2119317522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00c40b0 0x7fdbf015c470 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdbf00bfe70 tx=0x7fdbe801c930 comp rx=0 tx=0).stop 2024-09-13T16:54:01.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- 172.21.15.39:0/2119317522 shutdown_connections 2024-09-13T16:54:01.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 --2- 172.21.15.39:0/2119317522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00c40b0 0x7fdbf015c470 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 --2- 172.21.15.39:0/2119317522 >> v2:172.21.15.84:3300/0 conn(0x7fdbf006c870 0x7fdbf00aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- 172.21.15.39:0/2119317522 >> 172.21.15.39:0/2119317522 conn(0x7fdbf0069e10 msgr2=0x7fdbf006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:01.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- 172.21.15.39:0/2119317522 shutdown_connections 2024-09-13T16:54:01.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- 172.21.15.39:0/2119317522 wait complete. 2024-09-13T16:54:01.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 Processor -- start 2024-09-13T16:54:01.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- start start 2024-09-13T16:54:01.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbf006c870 0x7fdbf01eca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00aabc0 0x7fdbf0143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdbf00c40b0 0x7fdbf0143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf015c7f0 con 0x7fdbf00aabc0 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf00a5b40 con 0x7fdbf006c870 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf01ed6f0 con 0x7fdbf00c40b0 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbeffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00aabc0 0x7fdbf0143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbeffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00aabc0 0x7fdbf0143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41044/0 (socket says 172.21.15.39:41044) 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbeffff640 1 -- 172.21.15.39:0/3721011231 learned_addr learned my addr 172.21.15.39:0/3721011231 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf49a2640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbf006c870 0x7fdbf01eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf51a3640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdbf00c40b0 0x7fdbf0143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbeffff640 1 -- 172.21.15.39:0/3721011231 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdbf00c40b0 msgr2=0x7fdbf0143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbeffff640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdbf00c40b0 0x7fdbf0143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbeffff640 1 -- 172.21.15.39:0/3721011231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbf006c870 msgr2=0x7fdbf01eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbeffff640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbf006c870 0x7fdbf01eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbeffff640 1 -- 172.21.15.39:0/3721011231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe8009b30 con 0x7fdbf00aabc0 2024-09-13T16:54:01.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbeffff640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00aabc0 0x7fdbf0143a90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e930 tx=0x7fdbe000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbed7fa640 1 -- 172.21.15.39:0/3721011231 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0009690 con 0x7fdbf00aabc0 2024-09-13T16:54:01.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbed7fa640 1 -- 172.21.15.39:0/3721011231 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe000ca00 con 0x7fdbf00aabc0 2024-09-13T16:54:01.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.178+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf0155c90 con 0x7fdbf00aabc0 2024-09-13T16:54:01.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.182+0000 7fdbed7fa640 1 -- 172.21.15.39:0/3721011231 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000cd70 con 0x7fdbf00aabc0 2024-09-13T16:54:01.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.182+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf014e000 con 0x7fdbf00aabc0 2024-09-13T16:54:01.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.182+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf00aafa0 con 0x7fdbf00aabc0 2024-09-13T16:54:01.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.182+0000 7fdbed7fa640 1 -- 172.21.15.39:0/3721011231 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdbe0013470 con 0x7fdbf00aabc0 2024-09-13T16:54:01.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.182+0000 7fdbed7fa640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdbdc066690 0x7fdbdc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:01.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.182+0000 7fdbf49a2640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdbdc066690 0x7fdbdc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:01.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.182+0000 7fdbed7fa640 1 -- 172.21.15.39:0/3721011231 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7fdbe0084960 con 0x7fdbf00aabc0 2024-09-13T16:54:01.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.182+0000 7fdbf49a2640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdbdc066690 0x7fdbdc068b50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001380 tx=0x7fdbe4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:01.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.186+0000 7fdbed7fa640 1 -- 172.21.15.39:0/3721011231 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdbe0051820 con 0x7fdbf00aabc0 2024-09-13T16:54:01.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.386+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbf006c650 con 0x7fdbf00aabc0 2024-09-13T16:54:01.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.386+0000 7fdbed7fa640 1 -- 172.21.15.39:0/3721011231 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fdbe0054e40 con 0x7fdbf00aabc0 2024-09-13T16:54:01.387 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:01.388 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 50 2024-09-13T16:54:01.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.386+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdbdc066690 msgr2=0x7fdbdc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.386+0000 7fdbf6c06640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdbdc066690 0x7fdbdc068b50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001380 tx=0x7fdbe4009040 comp rx=0 tx=0).stop 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.386+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00aabc0 msgr2=0x7fdbf0143a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.386+0000 7fdbf6c06640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00aabc0 0x7fdbf0143a90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e930 tx=0x7fdbe000edf0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.390+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 shutdown_connections 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.390+0000 7fdbf6c06640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdbdc066690 0x7fdbdc068b50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.390+0000 7fdbf6c06640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdbf00c40b0 0x7fdbf0143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.390+0000 7fdbf6c06640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf00aabc0 0x7fdbf0143a90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.390+0000 7fdbf6c06640 1 --2- 172.21.15.39:0/3721011231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbf006c870 0x7fdbf01eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.390+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 >> 172.21.15.39:0/3721011231 conn(0x7fdbf0069e10 msgr2=0x7fdbf00c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.390+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 shutdown_connections 2024-09-13T16:54:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:01.390+0000 7fdbf6c06640 1 -- 172.21.15.39:0/3721011231 wait complete. 2024-09-13T16:54:01.407 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:54:03.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:03.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.538+0000 7f13d3c81640 1 Processor -- start 2024-09-13T16:54:03.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.538+0000 7f13d3c81640 1 -- start start 2024-09-13T16:54:03.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d3c81640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f13cc0aabc0 0x7f13cc0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d3c81640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 0x7f13cc15c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d3c81640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13cc152a80 con 0x7f13cc06c870 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d3c81640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13cc0c30f0 con 0x7f13cc0aabc0 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d3c81640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc151e00 con 0x7f13cc0c40b0 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d121c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f13cc06c870 0x7f13cc0c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51324/0 (socket says 172.21.15.39:51324) 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d121c640 1 -- 172.21.15.39:0/2845796959 learned_addr learned my addr 172.21.15.39:0/2845796959 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d1a1d640 1 --2- 172.21.15.39:0/2845796959 >> v2:172.21.15.39:3300/0 conn(0x7f13cc0aabc0 0x7f13cc0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d221e640 1 --2- 172.21.15.39:0/2845796959 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 0x7f13cc15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d0a1b640 1 -- 172.21.15.39:0/2845796959 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183970208 0 0) 0x7f13c8001120 con 0x7f13cc06c870 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d0a1b640 1 -- 172.21.15.39:0/2845796959 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13b8003530 con 0x7f13cc06c870 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d1a1d640 1 -- 172.21.15.39:0/2845796959 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 msgr2=0x7f13cc15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d1a1d640 1 --2- 172.21.15.39:0/2845796959 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 0x7f13cc15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d1a1d640 1 -- 172.21.15.39:0/2845796959 >> v1:172.21.15.39:6789/0 conn(0x7f13cc06c870 legacy=0x7f13cc0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d1a1d640 1 -- 172.21.15.39:0/2845796959 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc1520e0 con 0x7f13cc0aabc0 2024-09-13T16:54:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d1a1d640 1 --2- 172.21.15.39:0/2845796959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 0x7f13cc0aafa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0097c0 tx=0x7f13bc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7474d825755372 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d0a1b640 1 -- 172.21.15.39:0/2845796959 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0ac640 con 0x7f13cc0aabc0 2024-09-13T16:54:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d0a1b640 1 -- 172.21.15.39:0/2845796959 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc0ac7a0 con 0x7f13cc0aabc0 2024-09-13T16:54:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d0a1b640 1 -- 172.21.15.39:0/2845796959 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0acb30 con 0x7f13cc0aabc0 2024-09-13T16:54:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d3c81640 1 -- 172.21.15.39:0/2845796959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 msgr2=0x7f13cc0aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.542+0000 7f13d3c81640 1 --2- 172.21.15.39:0/2845796959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 0x7f13cc0aafa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0097c0 tx=0x7f13bc01c3e0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- 172.21.15.39:0/2845796959 shutdown_connections 2024-09-13T16:54:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 --2- 172.21.15.39:0/2845796959 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 0x7f13cc15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 --2- 172.21.15.39:0/2845796959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 0x7f13cc0aafa0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- 172.21.15.39:0/2845796959 >> 172.21.15.39:0/2845796959 conn(0x7f13cc069e10 msgr2=0x7f13cc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- 172.21.15.39:0/2845796959 shutdown_connections 2024-09-13T16:54:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- 172.21.15.39:0/2845796959 wait complete. 2024-09-13T16:54:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 Processor -- start 2024-09-13T16:54:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- start start 2024-09-13T16:54:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13cc06c870 0x7f13cc0a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 0x7f13cc0bee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 0x7f13cc1ec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc1efd80 con 0x7f13cc0aabc0 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc1ecd60 con 0x7f13cc06c870 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc1ed040 con 0x7f13cc0c40b0 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d121c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 0x7f13cc0bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d121c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 0x7f13cc0bee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41064/0 (socket says 172.21.15.39:41064) 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d121c640 1 -- 172.21.15.39:0/3330290887 learned_addr learned my addr 172.21.15.39:0/3330290887 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d1a1d640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13cc06c870 0x7f13cc0a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d221e640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 0x7f13cc1ec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d121c640 1 -- 172.21.15.39:0/3330290887 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 msgr2=0x7f13cc1ec4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d121c640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 0x7f13cc1ec4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d121c640 1 -- 172.21.15.39:0/3330290887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13cc06c870 msgr2=0x7f13cc0a89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d121c640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13cc06c870 0x7f13cc0a89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d121c640 1 -- 172.21.15.39:0/3330290887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc009420 con 0x7f13cc0aabc0 2024-09-13T16:54:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d1a1d640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13cc06c870 0x7f13cc0a89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:54:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d121c640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 0x7f13cc0bee90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f13c800c4d0 tx=0x7f13c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13c27fc640 1 -- 172.21.15.39:0/3330290887 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8005cf0 con 0x7f13cc0aabc0 2024-09-13T16:54:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13c27fc640 1 -- 172.21.15.39:0/3330290887 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c8005e50 con 0x7f13cc0aabc0 2024-09-13T16:54:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13c27fc640 1 -- 172.21.15.39:0/3330290887 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8007290 con 0x7f13cc0aabc0 2024-09-13T16:54:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13cc1ed320 con 0x7f13cc0aabc0 2024-09-13T16:54:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc0bf3d0 con 0x7f13cc0aabc0 2024-09-13T16:54:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.546+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13cc1eff30 con 0x7f13cc0aabc0 2024-09-13T16:54:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.550+0000 7f13c27fc640 1 -- 172.21.15.39:0/3330290887 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f13c80073f0 con 0x7f13cc0aabc0 2024-09-13T16:54:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.550+0000 7f13c27fc640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13b0066690 0x7f13b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.550+0000 7f13c27fc640 1 -- 172.21.15.39:0/3330290887 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7f13c8083b50 con 0x7f13cc0aabc0 2024-09-13T16:54:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.550+0000 7f13d1a1d640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13b0066690 0x7f13b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.550+0000 7f13d1a1d640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13b0066690 0x7f13b0068b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0ae000 tx=0x7f13bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.554+0000 7f13c27fc640 1 -- 172.21.15.39:0/3330290887 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f13c8050950 con 0x7f13cc0aabc0 2024-09-13T16:54:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.746+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13cc152a80 con 0x7f13cc0aabc0 2024-09-13T16:54:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.746+0000 7f13c27fc640 1 -- 172.21.15.39:0/3330290887 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f13cc152a80 con 0x7f13cc0aabc0 2024-09-13T16:54:03.748 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:03.748 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 50 2024-09-13T16:54:03.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13b0066690 msgr2=0x7f13b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13b0066690 0x7f13b0068b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0ae000 tx=0x7f13bc008040 comp rx=0 tx=0).stop 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 msgr2=0x7f13cc0bee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 0x7f13cc0bee90 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f13c800c4d0 tx=0x7f13c800c990 comp rx=0 tx=0).stop 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 shutdown_connections 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13b0066690 0x7f13b0068b50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13cc0c40b0 0x7f13cc1ec4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13cc0aabc0 0x7f13cc0bee90 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 --2- 172.21.15.39:0/3330290887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13cc06c870 0x7f13cc0a89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 >> 172.21.15.39:0/3330290887 conn(0x7f13cc069e10 msgr2=0x7f13cc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 shutdown_connections 2024-09-13T16:54:03.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:03.750+0000 7f13d3c81640 1 -- 172.21.15.39:0/3330290887 wait complete. 2024-09-13T16:54:03.767 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:54:05.767 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.898+0000 7fb2199fd640 1 Processor -- start 2024-09-13T16:54:05.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2199fd640 1 -- start start 2024-09-13T16:54:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2199fd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2140aafa0 0x7fb21415d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2199fd640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb214152a80 con 0x7fb21406c870 2024-09-13T16:54:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2199fd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2140ab410 con 0x7fb2140aabc0 2024-09-13T16:54:05.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2199fd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2140c30f0 con 0x7fb2140aafa0 2024-09-13T16:54:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2127fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb21406c870 0x7fb2140c40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51338/0 (socket says 172.21.15.39:51338) 2024-09-13T16:54:05.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2127fc640 1 -- 172.21.15.39:0/997395788 learned_addr learned my addr 172.21.15.39:0/997395788 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2137fe640 1 --2- 172.21.15.39:0/997395788 >> v2:172.21.15.84:3300/0 conn(0x7fb2140aafa0 0x7fb21415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567858640 0 0) 0x7fb20c0014a0 con 0x7fb21406c870 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f4003530 con 0x7fb21406c870 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 342209418 0 0) 0x7fb1f4003530 con 0x7fb21406c870 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 >> v2:172.21.15.84:3300/0 conn(0x7fb2140aafa0 msgr2=0x7fb21415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 --2- 172.21.15.39:0/997395788 >> v2:172.21.15.84:3300/0 conn(0x7fb2140aafa0 0x7fb21415d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 >> v1:172.21.15.84:6789/0 conn(0x7fb2140aabc0 legacy=0x7fb2140c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb214151e00 con 0x7fb21406c870 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fb20c001b90 con 0x7fb21406c870 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 >> v1:172.21.15.39:6789/0 conn(0x7fb21406c870 legacy=0x7fb2140c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb1f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f40047d0 0x7fb1f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f40052b0 0x7fb1f400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2140c30f0 con 0x7fb1f4003980 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f400c180 con 0x7fb1f40052b0 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f400c340 con 0x7fb1f40047d0 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb212ffd640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb1f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2127fc640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f40047d0 0x7fb1f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2137fe640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f40052b0 0x7fb1f400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb212ffd640 1 -- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f40047d0 msgr2=0x7fb1f4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb212ffd640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f40047d0 0x7fb1f4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb212ffd640 1 -- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f40052b0 msgr2=0x7fb1f400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb212ffd640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f40052b0 0x7fb1f400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb212ffd640 1 -- 172.21.15.39:0/997395788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f4003530 con 0x7fb1f4003980 2024-09-13T16:54:05.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2137fe640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1f40052b0 0x7fb1f400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:54:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2137fe640 1 -- 172.21.15.39:0/997395788 reap_dead start 2024-09-13T16:54:05.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.902+0000 7fb2127fc640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1f40047d0 0x7fb1f4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:54:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb212ffd640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb1f4003da0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc001750 tx=0x7fb1fc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511367965d9f4ec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0052d0 con 0x7fb1f4003980 2024-09-13T16:54:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc0a7050 con 0x7fb1f4003980 2024-09-13T16:54:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb211ffb640 1 -- 172.21.15.39:0/997395788 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc003600 con 0x7fb1f4003980 2024-09-13T16:54:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 -- 172.21.15.39:0/997395788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 msgr2=0x7fb1f4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb1f4003da0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc001750 tx=0x7fb1fc01c390 comp rx=0 tx=0).stop 2024-09-13T16:54:05.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 -- 172.21.15.39:0/997395788 shutdown_connections 2024-09-13T16:54:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 --2- 172.21.15.39:0/997395788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb1f4003da0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 -- 172.21.15.39:0/997395788 >> 172.21.15.39:0/997395788 conn(0x7fb214069e10 msgr2=0x7fb21406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 -- 172.21.15.39:0/997395788 shutdown_connections 2024-09-13T16:54:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 -- 172.21.15.39:0/997395788 wait complete. 2024-09-13T16:54:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 Processor -- start 2024-09-13T16:54:05.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 -- start start 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb214071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb214148bc0 0x7fb2140719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2140a61c0 0x7fb2140a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2140710b0 con 0x7fb1f4003980 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21406fba0 con 0x7fb2140a61c0 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2199fd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb214148970 con 0x7fb214148bc0 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb212ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb214071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb212ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb214071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41084/0 (socket says 172.21.15.39:41084) 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb212ffd640 1 -- 172.21.15.39:0/3250772975 learned_addr learned my addr 172.21.15.39:0/3250772975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2137fe640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2140a61c0 0x7fb2140a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb212ffd640 1 -- 172.21.15.39:0/3250772975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb214148bc0 msgr2=0x7fb2140719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb2127fc640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb214148bc0 0x7fb2140719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb212ffd640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb214148bc0 0x7fb2140719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb212ffd640 1 -- 172.21.15.39:0/3250772975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2140a61c0 msgr2=0x7fb2140a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb212ffd640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2140a61c0 0x7fb2140a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.906+0000 7fb212ffd640 1 -- 172.21.15.39:0/3250772975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21406fdf0 con 0x7fb1f4003980 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb212ffd640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb214071460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc01c1c0 tx=0x7fb1f4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb20bfff640 1 -- 172.21.15.39:0/3250772975 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0072f0 con 0x7fb1f4003980 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb20bfff640 1 -- 172.21.15.39:0/3250772975 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc007450 con 0x7fb1f4003980 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb20bfff640 1 -- 172.21.15.39:0/3250772975 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0067e0 con 0x7fb1f4003980 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb214148690 con 0x7fb1f4003980 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2140a7930 con 0x7fb1f4003980 2024-09-13T16:54:05.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2140c4190 con 0x7fb1f4003980 2024-09-13T16:54:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb20bfff640 1 -- 172.21.15.39:0/3250772975 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb1fc006940 con 0x7fb1f4003980 2024-09-13T16:54:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb20bfff640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1f80666c0 0x7fb1f8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb20bfff640 1 -- 172.21.15.39:0/3250772975 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7fb1fc11e6c0 con 0x7fb1f4003980 2024-09-13T16:54:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb2127fc640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1f80666c0 0x7fb1f8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:05.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.910+0000 7fb2127fc640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1f80666c0 0x7fb1f8068b80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb20c001640 tx=0x7fb20c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:05.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:05.914+0000 7fb20bfff640 1 -- 172.21.15.39:0/3250772975 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb1fc0eb430 con 0x7fb1f4003980 2024-09-13T16:54:06.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.110+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb21406c650 con 0x7fb1f4003980 2024-09-13T16:54:06.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb20bfff640 1 -- 172.21.15.39:0/3250772975 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fb1fc0eea50 con 0x7fb1f4003980 2024-09-13T16:54:06.113 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:06.114 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:06.114 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 50 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1f80666c0 msgr2=0x7fb1f8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1f80666c0 0x7fb1f8068b80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb20c001640 tx=0x7fb20c009040 comp rx=0 tx=0).stop 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 msgr2=0x7fb214071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb214071460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc01c1c0 tx=0x7fb1f4005800 comp rx=0 tx=0).stop 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 shutdown_connections 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1f80666c0 0x7fb1f8068b80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2140a61c0 0x7fb2140a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb214148bc0 0x7fb2140719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 --2- 172.21.15.39:0/3250772975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f4003980 0x7fb214071460 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 >> 172.21.15.39:0/3250772975 conn(0x7fb214069e10 msgr2=0x7fb214155850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:06.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 shutdown_connections 2024-09-13T16:54:06.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:06.114+0000 7fb2199fd640 1 -- 172.21.15.39:0/3250772975 wait complete. 2024-09-13T16:54:06.133 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:54:08.134 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:08.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.270+0000 7f1f16b46640 1 Processor -- start 2024-09-13T16:54:08.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.270+0000 7f1f16b46640 1 -- start start 2024-09-13T16:54:08.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f16b46640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f10151500 0x7f1f100c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f16b46640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f100c5a30 con 0x7f1f10071dd0 2024-09-13T16:54:08.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f16b46640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f101576a0 con 0x7f1f100c47c0 2024-09-13T16:54:08.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f16b46640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10157980 con 0x7f1f10151500 2024-09-13T16:54:08.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1f10071dd0 0x7f1f10154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51352/0 (socket says 172.21.15.39:51352) 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0ffff640 1 -- 172.21.15.39:0/243781766 learned_addr learned my addr 172.21.15.39:0/243781766 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f148e2640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f10151500 0x7f1f100c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091272898 0 0) 0x7f1f100c5a30 con 0x7f1f10071dd0 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef8003530 con 0x7f1f10071dd0 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1560008738 0 0) 0x7f1ef8003530 con 0x7f1f10071dd0 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f10151500 msgr2=0x7f1f100c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f10151500 0x7f1f100c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 >> v1:172.21.15.84:6789/0 conn(0x7f1f100c47c0 legacy=0x7f1f10154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f10157c60 con 0x7f1f10071dd0 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1f04001680 con 0x7f1f10071dd0 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 >> v1:172.21.15.39:6789/0 conn(0x7f1f10071dd0 legacy=0x7f1f10154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ef80039b0 0x7f1ef8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef80047d0 0x7f1ef8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f10151500 0x7f1ef80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10157980 con 0x7f1ef80047d0 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef800bac0 con 0x7f1ef80039b0 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef800bc80 con 0x7f1f10151500 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0ffff640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef80047d0 0x7f1ef8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f148e2640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ef80039b0 0x7f1ef8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0ffff640 1 -- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f10151500 msgr2=0x7f1ef80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0ffff640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f10151500 0x7f1ef80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0ffff640 1 -- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ef80039b0 msgr2=0x7f1ef8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0ffff640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ef80039b0 0x7f1ef8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.274+0000 7f1f0ffff640 1 -- 172.21.15.39:0/243781766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8003530 con 0x7f1ef80047d0 2024-09-13T16:54:08.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef80047d0 0x7f1ef8004c50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400c210 tx=0x7f1f0401d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2df8862976a98a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:08.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f040a5910 con 0x7f1ef80047d0 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f040a5a90 con 0x7f1ef80047d0 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0f7fe640 1 -- 172.21.15.39:0/243781766 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f040a5e40 con 0x7f1ef80047d0 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- 172.21.15.39:0/243781766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef80047d0 msgr2=0x7f1ef8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 --2- 172.21.15.39:0/243781766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ef80047d0 0x7f1ef8004c50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400c210 tx=0x7f1f0401d670 comp rx=0 tx=0).stop 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f150e3640 1 -- 172.21.15.39:0/243781766 reap_dead start 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- 172.21.15.39:0/243781766 shutdown_connections 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- 172.21.15.39:0/243781766 >> 172.21.15.39:0/243781766 conn(0x7f1f100ab530 msgr2=0x7f1f100ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- 172.21.15.39:0/243781766 shutdown_connections 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- 172.21.15.39:0/243781766 wait complete. 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 Processor -- start 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- start start 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ef80047d0 0x7f1f101525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f101f0e00 0x7f1f1014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f1014b320 0x7f1f10146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101f1280 con 0x7f1f101f0e00 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101f03b0 con 0x7f1ef80047d0 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101f0970 con 0x7f1f1014b320 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f101f0e00 0x7f1f1014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f101f0e00 0x7f1f1014fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41098/0 (socket says 172.21.15.39:41098) 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 -- 172.21.15.39:0/3598339737 learned_addr learned my addr 172.21.15.39:0/3598339737 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f148e2640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ef80047d0 0x7f1f101525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f150e3640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f1014b320 0x7f1f10146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 -- 172.21.15.39:0/3598339737 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f1014b320 msgr2=0x7f1f10146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f1014b320 0x7f1f10146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 -- 172.21.15.39:0/3598339737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ef80047d0 msgr2=0x7f1f101525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ef80047d0 0x7f1f101525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 -- 172.21.15.39:0/3598339737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f10153a10 con 0x7f1f101f0e00 2024-09-13T16:54:08.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0ffff640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f101f0e00 0x7f1f1014fa30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001940 tx=0x7f1ef8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:08.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0d7fa640 1 -- 172.21.15.39:0/3598339737 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f040a5910 con 0x7f1f101f0e00 2024-09-13T16:54:08.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0d7fa640 1 -- 172.21.15.39:0/3598339737 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f040b3070 con 0x7f1f101f0e00 2024-09-13T16:54:08.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1014ce00 con 0x7f1f101f0e00 2024-09-13T16:54:08.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.278+0000 7f1f0d7fa640 1 -- 172.21.15.39:0/3598339737 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04001640 con 0x7f1f101f0e00 2024-09-13T16:54:08.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.282+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f101f0600 con 0x7f1f101f0e00 2024-09-13T16:54:08.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.282+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1014d2b0 con 0x7f1f101f0e00 2024-09-13T16:54:08.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.282+0000 7f1f0d7fa640 1 -- 172.21.15.39:0/3598339737 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f04007040 con 0x7f1f101f0e00 2024-09-13T16:54:08.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.282+0000 7f1f0d7fa640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1efc066690 0x7f1efc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:08.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.282+0000 7f1f0d7fa640 1 -- 172.21.15.39:0/3598339737 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 7239+0+0 (secure 0 0 0) 0x7f1f040b5070 con 0x7f1f101f0e00 2024-09-13T16:54:08.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.282+0000 7f1f148e2640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1efc066690 0x7f1efc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:08.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.282+0000 7f1f148e2640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1efc066690 0x7f1efc068b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009a50 tx=0x7f1f00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:08.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.286+0000 7f1f0d7fa640 1 -- 172.21.15.39:0/3598339737 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f040eb700 con 0x7f1f101f0e00 2024-09-13T16:54:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.482+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f101f0790 con 0x7f1f101f0e00 2024-09-13T16:54:08.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.482+0000 7f1f0d7fa640 1 -- 172.21.15.39:0/3598339737 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f1f040eed20 con 0x7f1f101f0e00 2024-09-13T16:54:08.485 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:08.485 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:53:02.615795+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":32,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4922":{"gid":4922,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6833/1355376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1355376480},{"type":"v1","addr":"172.21.15.84:6833","nonce":1355376480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 50 2024-09-13T16:54:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1efc066690 msgr2=0x7f1efc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1efc066690 0x7f1efc068b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009a50 tx=0x7f1f00008040 comp rx=0 tx=0).stop 2024-09-13T16:54:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f101f0e00 msgr2=0x7f1f1014fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f101f0e00 0x7f1f1014fa30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001940 tx=0x7f1ef8005800 comp rx=0 tx=0).stop 2024-09-13T16:54:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 shutdown_connections 2024-09-13T16:54:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1efc066690 0x7f1efc068b50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f1014b320 0x7f1f10146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f101f0e00 0x7f1f1014fa30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 --2- 172.21.15.39:0/3598339737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ef80047d0 0x7f1f101525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 >> 172.21.15.39:0/3598339737 conn(0x7f1f100ab530 msgr2=0x7f1f1014e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 shutdown_connections 2024-09-13T16:54:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:08.486+0000 7f1f16b46640 1 -- 172.21.15.39:0/3598339737 wait complete. 2024-09-13T16:54:08.504 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:54:10.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:10.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb90d2640 1 Processor -- start 2024-09-13T16:54:10.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb90d2640 1 -- start start 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb90d2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faeb4151500 0x7faeb40c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb90d2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4071dd0 0x7faeb4154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb90d2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faeb40c47c0 0x7faeb41545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb90d2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb40c5a30 con 0x7faeb40c47c0 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb90d2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb41576a0 con 0x7faeb4151500 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb90d2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4157980 con 0x7faeb4071dd0 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb359e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faeb40c47c0 0x7faeb41545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb359e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faeb40c47c0 0x7faeb41545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57232/0 (socket says 172.21.15.39:57232) 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb359e640 1 -- 172.21.15.39:0/43082894 learned_addr learned my addr 172.21.15.39:0/43082894 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:10.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb2d9d640 1 --2- 172.21.15.39:0/43082894 >> v2:172.21.15.84:3300/0 conn(0x7faeb4151500 0x7faeb40c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb259c640 1 --2- 172.21.15.39:0/43082894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4071dd0 0x7faeb4154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb359e640 1 -- 172.21.15.39:0/43082894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4071dd0 msgr2=0x7faeb4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:10.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb359e640 1 --2- 172.21.15.39:0/43082894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4071dd0 0x7faeb4154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb359e640 1 -- 172.21.15.39:0/43082894 >> v2:172.21.15.84:3300/0 conn(0x7faeb4151500 msgr2=0x7faeb40c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:10.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb359e640 1 --2- 172.21.15.39:0/43082894 >> v2:172.21.15.84:3300/0 conn(0x7faeb4151500 0x7faeb40c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.638+0000 7faeb359e640 1 -- 172.21.15.39:0/43082894 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4157c60 con 0x7faeb40c47c0 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb359e640 1 --2- 172.21.15.39:0/43082894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 0x7faeb41545b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faeb40a9470 tx=0x7faeac01d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992a243e56bcde38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb1d9b640 1 -- 172.21.15.39:0/43082894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0a57d0 con 0x7faeb40c47c0 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb1d9b640 1 -- 172.21.15.39:0/43082894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac0a5930 con 0x7faeb40c47c0 2024-09-13T16:54:10.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb1d9b640 1 -- 172.21.15.39:0/43082894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0a5cc0 con 0x7faeb40c47c0 2024-09-13T16:54:10.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 -- 172.21.15.39:0/43082894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 msgr2=0x7faeb41545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 --2- 172.21.15.39:0/43082894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 0x7faeb41545b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faeb40a9470 tx=0x7faeac01d390 comp rx=0 tx=0).stop 2024-09-13T16:54:10.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 -- 172.21.15.39:0/43082894 shutdown_connections 2024-09-13T16:54:10.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 --2- 172.21.15.39:0/43082894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 0x7faeb41545b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 --2- 172.21.15.39:0/43082894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4071dd0 0x7faeb4154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 --2- 172.21.15.39:0/43082894 >> v2:172.21.15.84:3300/0 conn(0x7faeb4151500 0x7faeb40c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 -- 172.21.15.39:0/43082894 >> 172.21.15.39:0/43082894 conn(0x7faeb40ab530 msgr2=0x7faeb40ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:10.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 -- 172.21.15.39:0/43082894 shutdown_connections 2024-09-13T16:54:10.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 -- 172.21.15.39:0/43082894 wait complete. 2024-09-13T16:54:10.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 Processor -- start 2024-09-13T16:54:10.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.642+0000 7faeb90d2640 1 -- start start 2024-09-13T16:54:10.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb90d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb4071dd0 0x7faeb414a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb90d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 0x7faeb4147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb90d2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4151500 0x7faeb4147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb90d2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb414b7e0 con 0x7faeb40c47c0 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb90d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4151fb0 con 0x7faeb4071dd0 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb90d2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb41ed190 con 0x7faeb4151500 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 0x7faeb4147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 0x7faeb4147430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57238/0 (socket says 172.21.15.39:57238) 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb259c640 1 -- 172.21.15.39:0/384604178 learned_addr learned my addr 172.21.15.39:0/384604178 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb2d9d640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb4071dd0 0x7faeb414a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb359e640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4151500 0x7faeb4147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb259c640 1 -- 172.21.15.39:0/384604178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4151500 msgr2=0x7faeb4147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb259c640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4151500 0x7faeb4147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb259c640 1 -- 172.21.15.39:0/384604178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb4071dd0 msgr2=0x7faeb414a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb259c640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb4071dd0 0x7faeb414a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb259c640 1 -- 172.21.15.39:0/384604178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac009b30 con 0x7faeb40c47c0 2024-09-13T16:54:10.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb259c640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 0x7faeb4147430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faea400d800 tx=0x7faea400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:10.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7fae9b7fe640 1 -- 172.21.15.39:0/384604178 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40096a0 con 0x7faeb40c47c0 2024-09-13T16:54:10.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7fae9b7fe640 1 -- 172.21.15.39:0/384604178 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea4009800 con 0x7faeb40c47c0 2024-09-13T16:54:10.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb4152250 con 0x7faeb40c47c0 2024-09-13T16:54:10.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7fae9b7fe640 1 -- 172.21.15.39:0/384604178 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4009b70 con 0x7faeb40c47c0 2024-09-13T16:54:10.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb414af10 con 0x7faeb40c47c0 2024-09-13T16:54:10.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb40711d0 con 0x7faeb40c47c0 2024-09-13T16:54:10.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7fae9b7fe640 1 -- 172.21.15.39:0/384604178 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faea4009cd0 con 0x7faeb40c47c0 2024-09-13T16:54:10.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7fae9b7fe640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faea0066690 0x7faea0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7fae9b7fe640 1 -- 172.21.15.39:0/384604178 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7faea40853e0 con 0x7faeb40c47c0 2024-09-13T16:54:10.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb2d9d640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faea0066690 0x7faea0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.646+0000 7faeb2d9d640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faea0066690 0x7faea0068b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faeb414ef50 tx=0x7faea8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:10.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.650+0000 7fae9b7fe640 1 -- 172.21.15.39:0/384604178 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faea4052170 con 0x7faeb40c47c0 2024-09-13T16:54:10.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.850+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faeb4071bb0 con 0x7faeb40c47c0 2024-09-13T16:54:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.850+0000 7fae9b7fe640 1 -- 172.21.15.39:0/384604178 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5178 (secure 0 0 0) 0x7faea4055790 con 0x7faeb40c47c0 2024-09-13T16:54:10.853 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:10.853 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:10.621508+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 52 2024-09-13T16:54:10.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faea0066690 msgr2=0x7faea0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faea0066690 0x7faea0068b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faeb414ef50 tx=0x7faea8008040 comp rx=0 tx=0).stop 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 msgr2=0x7faeb4147430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 0x7faeb4147430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7faea400d800 tx=0x7faea400dcc0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 shutdown_connections 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faea0066690 0x7faea0068b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faeb4151500 0x7faeb4147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb40c47c0 0x7faeb4147430 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 --2- 172.21.15.39:0/384604178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb4071dd0 0x7faeb414a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 >> 172.21.15.39:0/384604178 conn(0x7faeb40ab530 msgr2=0x7faeb414f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 shutdown_connections 2024-09-13T16:54:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.854+0000 7faeb90d2640 1 -- 172.21.15.39:0/384604178 wait complete. 2024-09-13T16:54:10.872 INFO:tasks.mds_thrash.fs.[a]:mds.b down, removed from mdsmap 2024-09-13T16:54:10.872 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T16:54:10.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:10.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.986+0000 7f2848040640 1 Processor -- start 2024-09-13T16:54:10.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.986+0000 7f2848040640 1 -- start start 2024-09-13T16:54:10.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2848040640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2840151620 0x7f2840153e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2848040640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f28400720e0 0x7f28400c6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2848040640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2840157420 con 0x7f2840157b90 2024-09-13T16:54:10.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2848040640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2840071660 con 0x7f2840151620 2024-09-13T16:54:10.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2848040640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2840157700 con 0x7f28400720e0 2024-09-13T16:54:10.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2845ddc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2840151620 0x7f2840153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f28465dd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2840157b90 0x7f28400c68f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52860/0 (socket says 172.21.15.39:52860) 2024-09-13T16:54:10.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f28465dd640 1 -- 172.21.15.39:0/1351162837 learned_addr learned my addr 172.21.15.39:0/1351162837 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:54:10.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f28455db640 1 --2- 172.21.15.39:0/1351162837 >> v2:172.21.15.84:3300/0 conn(0x7f28400720e0 0x7f28400c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2845ddc640 1 -- 172.21.15.39:0/1351162837 >> v2:172.21.15.84:3300/0 conn(0x7f28400720e0 msgr2=0x7f28400c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:10.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2845ddc640 1 --2- 172.21.15.39:0/1351162837 >> v2:172.21.15.84:3300/0 conn(0x7f28400720e0 0x7f28400c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2845ddc640 1 -- 172.21.15.39:0/1351162837 >> v1:172.21.15.39:6789/0 conn(0x7f2840157b90 legacy=0x7f28400c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2844dda640 1 -- 172.21.15.39:0/1351162837 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773648549 0 0) 0x7f2840157420 con 0x7f2840157b90 2024-09-13T16:54:10.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f2845ddc640 1 -- 172.21.15.39:0/1351162837 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28400a7330 con 0x7f2840151620 2024-09-13T16:54:10.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.990+0000 7f28455db640 1 --2- 172.21.15.39:0/1351162837 >> v2:172.21.15.84:3300/0 conn(0x7f28400720e0 0x7f28400c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2845ddc640 1 --2- 172.21.15.39:0/1351162837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 0x7f2840153e30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f284006f6f0 tx=0x7f283c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1212331689679c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2844dda640 1 -- 172.21.15.39:0/1351162837 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0a5840 con 0x7f2840151620 2024-09-13T16:54:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2844dda640 1 -- 172.21.15.39:0/1351162837 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c0a59a0 con 0x7f2840151620 2024-09-13T16:54:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2844dda640 1 -- 172.21.15.39:0/1351162837 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0a5d30 con 0x7f2840151620 2024-09-13T16:54:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 -- 172.21.15.39:0/1351162837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 msgr2=0x7f2840153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 --2- 172.21.15.39:0/1351162837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 0x7f2840153e30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f284006f6f0 tx=0x7f283c01c870 comp rx=0 tx=0).stop 2024-09-13T16:54:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 -- 172.21.15.39:0/1351162837 shutdown_connections 2024-09-13T16:54:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 --2- 172.21.15.39:0/1351162837 >> v2:172.21.15.84:3300/0 conn(0x7f28400720e0 0x7f28400c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 --2- 172.21.15.39:0/1351162837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 0x7f2840153e30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 -- 172.21.15.39:0/1351162837 >> 172.21.15.39:0/1351162837 conn(0x7f28400c7820 msgr2=0x7f28400c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 -- 172.21.15.39:0/1351162837 shutdown_connections 2024-09-13T16:54:10.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 -- 172.21.15.39:0/1351162837 wait complete. 2024-09-13T16:54:10.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 Processor -- start 2024-09-13T16:54:10.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 -- start start 2024-09-13T16:54:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28400720e0 0x7f28401ed800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 0x7f28400ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2840157b90 0x7f28400ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284015d5b0 con 0x7f2840151620 2024-09-13T16:54:10.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2840159040 con 0x7f28400720e0 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2848040640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28400aab10 con 0x7f2840157b90 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28455db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 0x7f28400ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28455db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 0x7f28400ac0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57258/0 (socket says 172.21.15.39:57258) 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28455db640 1 -- 172.21.15.39:0/1081768643 learned_addr learned my addr 172.21.15.39:0/1081768643 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2845ddc640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28400720e0 0x7f28401ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28465dd640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2840157b90 0x7f28400ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28455db640 1 -- 172.21.15.39:0/1081768643 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2840157b90 msgr2=0x7f28400ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28455db640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2840157b90 0x7f28400ab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28455db640 1 -- 172.21.15.39:0/1081768643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28400720e0 msgr2=0x7f28401ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28455db640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28400720e0 0x7f28401ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28455db640 1 -- 172.21.15.39:0/1081768643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c009a70 con 0x7f2840151620 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f2845ddc640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28400720e0 0x7f28401ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:10.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.994+0000 7f28455db640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 0x7f28400ac0a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f283000e840 tx=0x7f283000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f28367fc640 1 -- 172.21.15.39:0/1081768643 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000c870 con 0x7f2840151620 2024-09-13T16:54:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f28367fc640 1 -- 172.21.15.39:0/1081768643 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283000c9d0 con 0x7f2840151620 2024-09-13T16:54:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28401eceb0 con 0x7f2840151620 2024-09-13T16:54:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f28367fc640 1 -- 172.21.15.39:0/1081768643 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000cd60 con 0x7f2840151620 2024-09-13T16:54:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28400aad60 con 0x7f2840151620 2024-09-13T16:54:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28400c6130 con 0x7f2840151620 2024-09-13T16:54:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f28367fc640 1 -- 172.21.15.39:0/1081768643 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2830013410 con 0x7f2840151620 2024-09-13T16:54:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f28367fc640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28240666c0 0x7f2824068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f28367fc640 1 -- 172.21.15.39:0/1081768643 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7f2830084550 con 0x7f2840151620 2024-09-13T16:54:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f2845ddc640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28240666c0 0x7f2824068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:11.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:10.998+0000 7f2845ddc640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28240666c0 0x7f2824068b80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f283c0af000 tx=0x7f283c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:11.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.002+0000 7f28367fc640 1 -- 172.21.15.39:0/1081768643 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2830051270 con 0x7f2840151620 2024-09-13T16:54:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.202+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2840071420 con 0x7f2840151620 2024-09-13T16:54:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.202+0000 7f28367fc640 1 -- 172.21.15.39:0/1081768643 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5178 (secure 0 0 0) 0x7f2830054890 con 0x7f2840151620 2024-09-13T16:54:11.205 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:11.205 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:10.621508+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:11.205 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 52 2024-09-13T16:54:11.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28240666c0 msgr2=0x7f2824068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:11.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28240666c0 0x7f2824068b80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f283c0af000 tx=0x7f283c0a7070 comp rx=0 tx=0).stop 2024-09-13T16:54:11.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 msgr2=0x7f28400ac0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:11.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 0x7f28400ac0a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f283000e840 tx=0x7f283000ed00 comp rx=0 tx=0).stop 2024-09-13T16:54:11.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 shutdown_connections 2024-09-13T16:54:11.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f28240666c0 0x7f2824068b80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:11.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2840157b90 0x7f28400ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:11.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2840151620 0x7f28400ac0a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:11.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 --2- 172.21.15.39:0/1081768643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28400720e0 0x7f28401ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:11.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 >> 172.21.15.39:0/1081768643 conn(0x7f28400c7820 msgr2=0x7f284006b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:11.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 shutdown_connections 2024-09-13T16:54:11.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:11.206+0000 7f2848040640 1 -- 172.21.15.39:0/1081768643 wait complete. 2024-09-13T16:54:11.225 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 52, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 52, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5703": { "addr": "172.21.15.39:6835/4094657757", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 4094657757, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 4094657757, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5703, "incarnation": 51, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 34, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:54:10.621508+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5703 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 49, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5362": { "addr": "172.21.15.84:6835/3600322972", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 3600322972, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 3600322972, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5362, "incarnation": 46, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 33, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:53:17.755716+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5362 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:54:13.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:13.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9daa63640 1 Processor -- start 2024-09-13T16:54:13.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9daa63640 1 -- start start 2024-09-13T16:54:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9daa63640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 0x7fc9d40c0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9daa63640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d40706f0 con 0x7fc9d40c2db0 2024-09-13T16:54:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9daa63640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d41510d0 con 0x7fc9d406fd50 2024-09-13T16:54:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9daa63640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c17f0 con 0x7fc9d4071f90 2024-09-13T16:54:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9d3fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc9d40c2db0 0x7fc9d40c2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52868/0 (socket says 172.21.15.39:52868) 2024-09-13T16:54:13.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9d3fff640 1 -- 172.21.15.39:0/1295822055 learned_addr learned my addr 172.21.15.39:0/1295822055 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:54:13.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9d37fe640 1 --2- 172.21.15.39:0/1295822055 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 0x7fc9d40c0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9d2ffd640 1 -- 172.21.15.39:0/1295822055 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273113649 0 0) 0x7fc9d40706f0 con 0x7fc9d40c2db0 2024-09-13T16:54:13.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9d2ffd640 1 -- 172.21.15.39:0/1295822055 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9bc003530 con 0x7fc9d40c2db0 2024-09-13T16:54:13.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9d2ffd640 1 -- 172.21.15.39:0/1295822055 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2898845094 0 0) 0x7fc9bc003530 con 0x7fc9d40c2db0 2024-09-13T16:54:13.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9d37fe640 1 -- 172.21.15.39:0/1295822055 >> v1:172.21.15.84:6789/0 conn(0x7fc9d406fd50 legacy=0x7fc9d414d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9d37fe640 1 -- 172.21.15.39:0/1295822055 >> v1:172.21.15.39:6789/0 conn(0x7fc9d40c2db0 legacy=0x7fc9d40c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.358+0000 7fc9d37fe640 1 -- 172.21.15.39:0/1295822055 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d41513b0 con 0x7fc9d4071f90 2024-09-13T16:54:13.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9d37fe640 1 --2- 172.21.15.39:0/1295822055 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 0x7fc9d40c0d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800a990 tx=0x7fc9c801d740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76dfbfbddbcd691a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:13.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9d2ffd640 1 -- 172.21.15.39:0/1295822055 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4001680 con 0x7fc9d4071f90 2024-09-13T16:54:13.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9d2ffd640 1 -- 172.21.15.39:0/1295822055 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c80a7430 con 0x7fc9d4071f90 2024-09-13T16:54:13.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9d2ffd640 1 -- 172.21.15.39:0/1295822055 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80a77a0 con 0x7fc9d4071f90 2024-09-13T16:54:13.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9daa63640 1 -- 172.21.15.39:0/1295822055 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 msgr2=0x7fc9d40c0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9daa63640 1 --2- 172.21.15.39:0/1295822055 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 0x7fc9d40c0d60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800a990 tx=0x7fc9c801d740 comp rx=0 tx=0).stop 2024-09-13T16:54:13.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9daa63640 1 -- 172.21.15.39:0/1295822055 shutdown_connections 2024-09-13T16:54:13.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9daa63640 1 --2- 172.21.15.39:0/1295822055 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 0x7fc9d40c0d60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9daa63640 1 -- 172.21.15.39:0/1295822055 >> 172.21.15.39:0/1295822055 conn(0x7fc9d4069e10 msgr2=0x7fc9d406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9daa63640 1 -- 172.21.15.39:0/1295822055 shutdown_connections 2024-09-13T16:54:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.362+0000 7fc9daa63640 1 -- 172.21.15.39:0/1295822055 wait complete. 2024-09-13T16:54:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 Processor -- start 2024-09-13T16:54:13.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 -- start start 2024-09-13T16:54:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d406fd50 0x7fc9d40abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 0x7fc9d4143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9d40c2db0 0x7fc9d4146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d41ebe80 con 0x7fc9d406fd50 2024-09-13T16:54:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d41ec3b0 con 0x7fc9d40c2db0 2024-09-13T16:54:13.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d41466f0 con 0x7fc9d4071f90 2024-09-13T16:54:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d406fd50 0x7fc9d40abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d406fd50 0x7fc9d40abf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57270/0 (socket says 172.21.15.39:57270) 2024-09-13T16:54:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d3fff640 1 -- 172.21.15.39:0/3479137720 learned_addr learned my addr 172.21.15.39:0/3479137720 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d3fff640 1 -- 172.21.15.39:0/3479137720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 msgr2=0x7fc9d4143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d3fff640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 0x7fc9d4143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d3fff640 1 -- 172.21.15.39:0/3479137720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9d40c2db0 msgr2=0x7fc9d4146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d3fff640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9d40c2db0 0x7fc9d4146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d3fff640 1 -- 172.21.15.39:0/3479137720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c800a5c0 con 0x7fc9d406fd50 2024-09-13T16:54:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d3fff640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d406fd50 0x7fc9d40abf00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c700 tx=0x7fc9c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d0ff9640 1 -- 172.21.15.39:0/3479137720 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4005ae0 con 0x7fc9d406fd50 2024-09-13T16:54:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d0ff9640 1 -- 172.21.15.39:0/3479137720 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4005c60 con 0x7fc9d406fd50 2024-09-13T16:54:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9d0ff9640 1 -- 172.21.15.39:0/3479137720 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4007070 con 0x7fc9d406fd50 2024-09-13T16:54:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d41ec0d0 con 0x7fc9d406fd50 2024-09-13T16:54:13.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.366+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d41444f0 con 0x7fc9d406fd50 2024-09-13T16:54:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.370+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d40c0d60 con 0x7fc9d406fd50 2024-09-13T16:54:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.370+0000 7fc9d0ff9640 1 -- 172.21.15.39:0/3479137720 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc9c40071d0 con 0x7fc9d406fd50 2024-09-13T16:54:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.370+0000 7fc9d0ff9640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9c006eed0 0x7fc9c0071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.370+0000 7fc9d0ff9640 1 -- 172.21.15.39:0/3479137720 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7fc9c4083e30 con 0x7fc9d406fd50 2024-09-13T16:54:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.370+0000 7fc9d37fe640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9c006eed0 0x7fc9c0071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.370+0000 7fc9d37fe640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9c006eed0 0x7fc9c0071390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80af000 tx=0x7fc9c80b4040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:13.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.374+0000 7fc9d0ff9640 1 -- 172.21.15.39:0/3479137720 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc9c4050be0 con 0x7fc9d406fd50 2024-09-13T16:54:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9d4150550 con 0x7fc9d406fd50 2024-09-13T16:54:13.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9d0ff9640 1 -- 172.21.15.39:0/3479137720 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5175 (secure 0 0 0) 0x7fc9c4017020 con 0x7fc9d406fd50 2024-09-13T16:54:13.619 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:13.619 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 54 2024-09-13T16:54:13.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9c006eed0 msgr2=0x7fc9c0071390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9c006eed0 0x7fc9c0071390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80af000 tx=0x7fc9c80b4040 comp rx=0 tx=0).stop 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d406fd50 msgr2=0x7fc9d40abf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d406fd50 0x7fc9d40abf00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c700 tx=0x7fc9c400cbc0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 shutdown_connections 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9c006eed0 0x7fc9c0071390 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9d40c2db0 0x7fc9d4146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9d4071f90 0x7fc9d4143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 --2- 172.21.15.39:0/3479137720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d406fd50 0x7fc9d40abf00 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 >> 172.21.15.39:0/3479137720 conn(0x7fc9d4069e10 msgr2=0x7fc9d415b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 shutdown_connections 2024-09-13T16:54:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:13.618+0000 7fc9daa63640 1 -- 172.21.15.39:0/3479137720 wait complete. 2024-09-13T16:54:13.639 INFO:tasks.mds_thrash.fs.[a]:mds.a has gained rank=0, replacing gid=4922 2024-09-13T16:54:13.639 INFO:tasks.mds_thrash.fs.[a]:waiting for 4 secs before reviving mds.b 2024-09-13T16:54:15.786 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T16:54:15.786+0000 7fc430b49640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:54:17.639 INFO:tasks.mds_thrash.fs.[a]:reviving mds.b 2024-09-13T16:54:17.640 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:54:17.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:54:17.642 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:54:17.643 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:17.720 INFO:tasks.ceph.mds.b.smithi084.stdout:starting mds.b at 2024-09-13T16:54:17.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.790+0000 7f31eb229640 1 Processor -- start 2024-09-13T16:54:17.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.790+0000 7f31eb229640 1 -- start start 2024-09-13T16:54:17.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31eb229640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c42f0 0x7f31e40c4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31eb229640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e4070800 con 0x7f31e40c6530 2024-09-13T16:54:17.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31eb229640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e4070aa0 con 0x7f31e41516d0 2024-09-13T16:54:17.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31eb229640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e4071720 con 0x7f31e40c42f0 2024-09-13T16:54:17.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31e97c6640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f31e40c6530 0x7f31e40c69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52870/0 (socket says 172.21.15.39:52870) 2024-09-13T16:54:17.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31e97c6640 1 -- 172.21.15.39:0/3940241467 learned_addr learned my addr 172.21.15.39:0/3940241467 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:54:17.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31e3fff640 1 --2- 172.21.15.39:0/3940241467 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c42f0 0x7f31e40c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31e37fe640 1 -- 172.21.15.39:0/3940241467 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697414077 0 0) 0x7f31e4070800 con 0x7f31e40c6530 2024-09-13T16:54:17.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31e37fe640 1 -- 172.21.15.39:0/3940241467 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d4003530 con 0x7f31e40c6530 2024-09-13T16:54:17.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31e3fff640 1 -- 172.21.15.39:0/3940241467 >> v1:172.21.15.84:6789/0 conn(0x7f31e41516d0 legacy=0x7f31e41549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31e3fff640 1 -- 172.21.15.39:0/3940241467 >> v1:172.21.15.39:6789/0 conn(0x7f31e40c6530 legacy=0x7f31e40c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31e3fff640 1 -- 172.21.15.39:0/3940241467 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e4070d80 con 0x7f31e40c42f0 2024-09-13T16:54:17.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.794+0000 7f31e3fff640 1 --2- 172.21.15.39:0/3940241467 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c42f0 0x7f31e40c4730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f31d00097f0 tx=0x7f31d001c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ab950938de53d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31e37fe640 1 -- 172.21.15.39:0/3940241467 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00a8050 con 0x7f31e40c42f0 2024-09-13T16:54:17.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31e37fe640 1 -- 172.21.15.39:0/3940241467 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d00ac5f0 con 0x7f31e40c42f0 2024-09-13T16:54:17.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31e37fe640 1 -- 172.21.15.39:0/3940241467 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00ac980 con 0x7f31e40c42f0 2024-09-13T16:54:17.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31eb229640 1 -- 172.21.15.39:0/3940241467 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c42f0 msgr2=0x7f31e40c4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31eb229640 1 --2- 172.21.15.39:0/3940241467 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c42f0 0x7f31e40c4730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f31d00097f0 tx=0x7f31d001c390 comp rx=0 tx=0).stop 2024-09-13T16:54:17.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31eb229640 1 -- 172.21.15.39:0/3940241467 shutdown_connections 2024-09-13T16:54:17.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31eb229640 1 --2- 172.21.15.39:0/3940241467 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c42f0 0x7f31e40c4730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31eb229640 1 -- 172.21.15.39:0/3940241467 >> 172.21.15.39:0/3940241467 conn(0x7f31e40ab500 msgr2=0x7f31e40ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:17.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31eb229640 1 -- 172.21.15.39:0/3940241467 shutdown_connections 2024-09-13T16:54:17.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31eb229640 1 -- 172.21.15.39:0/3940241467 wait complete. 2024-09-13T16:54:17.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31eb229640 1 Processor -- start 2024-09-13T16:54:17.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.798+0000 7f31eb229640 1 -- start start 2024-09-13T16:54:17.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31eb229640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e40c42f0 0x7f31e4158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31eb229640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c6530 0x7f31e414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31eb229640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e41516d0 0x7f31e414af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31eb229640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e41ec4b0 con 0x7f31e41516d0 2024-09-13T16:54:17.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31eb229640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e414fbd0 con 0x7f31e40c42f0 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31eb229640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c3530 con 0x7f31e40c6530 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e97c6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e41516d0 0x7f31e414af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e8fc5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e40c42f0 0x7f31e4158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c6530 0x7f31e414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e97c6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e41516d0 0x7f31e414af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57276/0 (socket says 172.21.15.39:57276) 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e8fc5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e40c42f0 0x7f31e4158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:40326/0 (socket says 172.21.15.39:40326) 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e97c6640 1 -- 172.21.15.39:0/3406310943 learned_addr learned my addr 172.21.15.39:0/3406310943 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e97c6640 1 -- 172.21.15.39:0/3406310943 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c6530 msgr2=0x7f31e414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e97c6640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c6530 0x7f31e414f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e97c6640 1 -- 172.21.15.39:0/3406310943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e40c42f0 msgr2=0x7f31e4158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e97c6640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e40c42f0 0x7f31e4158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e97c6640 1 -- 172.21.15.39:0/3406310943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0009420 con 0x7f31e41516d0 2024-09-13T16:54:17.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e8fc5640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e40c42f0 0x7f31e4158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:54:17.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e97c6640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e41516d0 0x7f31e414af30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00cb40 tx=0x7f31dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e17fa640 1 -- 172.21.15.39:0/3406310943 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc00aca0 con 0x7f31e41516d0 2024-09-13T16:54:17.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e17fa640 1 -- 172.21.15.39:0/3406310943 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc00ae00 con 0x7f31e41516d0 2024-09-13T16:54:17.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e40c3250 con 0x7f31e41516d0 2024-09-13T16:54:17.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e17fa640 1 -- 172.21.15.39:0/3406310943 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc012620 con 0x7f31e41516d0 2024-09-13T16:54:17.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e41556e0 con 0x7f31e41516d0 2024-09-13T16:54:17.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e40c4730 con 0x7f31e41516d0 2024-09-13T16:54:17.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.802+0000 7f31e17fa640 1 -- 172.21.15.39:0/3406310943 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f31dc007070 con 0x7f31e41516d0 2024-09-13T16:54:17.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.806+0000 7f31e17fa640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f31c8066690 0x7f31c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:17.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.806+0000 7f31e8fc5640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f31c8066690 0x7f31c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:17.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.806+0000 7f31e17fa640 1 -- 172.21.15.39:0/3406310943 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7f31dc084760 con 0x7f31e41516d0 2024-09-13T16:54:17.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.806+0000 7f31e8fc5640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f31c8066690 0x7f31c8068b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f31d8007fd0 tx=0x7f31d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:17.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:17.810+0000 7f31e17fa640 1 -- 172.21.15.39:0/3406310943 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f31dc051530 con 0x7f31e41516d0 2024-09-13T16:54:18.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.030+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31e40bcbd0 con 0x7f31e41516d0 2024-09-13T16:54:18.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.030+0000 7f31e17fa640 1 -- 172.21.15.39:0/3406310943 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f31dc054b50 con 0x7f31e41516d0 2024-09-13T16:54:18.033 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:18.033 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5069,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:18.033 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 55 2024-09-13T16:54:18.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f31c8066690 msgr2=0x7f31c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f31c8066690 0x7f31c8068b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f31d8007fd0 tx=0x7f31d8009040 comp rx=0 tx=0).stop 2024-09-13T16:54:18.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e41516d0 msgr2=0x7f31e414af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:18.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e41516d0 0x7f31e414af30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00cb40 tx=0x7f31dc00a060 comp rx=0 tx=0).stop 2024-09-13T16:54:18.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 shutdown_connections 2024-09-13T16:54:18.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f31c8066690 0x7f31c8068b50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e41516d0 0x7f31e414af30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f31e40c6530 0x7f31e414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 --2- 172.21.15.39:0/3406310943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e40c42f0 0x7f31e4158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:18.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 >> 172.21.15.39:0/3406310943 conn(0x7f31e40ab500 msgr2=0x7f31e414cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:18.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 shutdown_connections 2024-09-13T16:54:18.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:18.034+0000 7f31eb229640 1 -- 172.21.15.39:0/3406310943 wait complete. 2024-09-13T16:54:18.055 INFO:tasks.mds_thrash.fs.[a]:mds.b reported in up:standby state 2024-09-13T16:54:18.056 INFO:tasks.mds_thrash.fs.[a]:waiting for 92 secs before thrashing 2024-09-13T16:54:26.519 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:54:26.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:54:56.565 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:54:56.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:54:58.968 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c3408640 1 Processor -- start 2024-09-13T16:54:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c3408640 1 -- start start 2024-09-13T16:54:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c3408640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa8bc151860 0x7fa8bc157d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c3408640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8bc0bf4f0 0x7fa8bc153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c3408640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8bc0c1340 con 0x7fa8bc0611c0 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c3408640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8bc1578f0 con 0x7fa8bc151860 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c3408640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8bc1582c0 con 0x7fa8bc0bf4f0 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c11a4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa8bc151860 0x7fa8bc157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c11a4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa8bc151860 0x7fa8bc157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48560/0 (socket says 172.21.15.39:48560) 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c11a4640 1 -- 172.21.15.39:0/1922226774 learned_addr learned my addr 172.21.15.39:0/1922226774 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c09a3640 1 --2- 172.21.15.39:0/1922226774 >> v2:172.21.15.84:3300/0 conn(0x7fa8bc0bf4f0 0x7fa8bc153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c11a4640 1 -- 172.21.15.39:0/1922226774 >> v2:172.21.15.84:3300/0 conn(0x7fa8bc0bf4f0 msgr2=0x7fa8bc153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c11a4640 1 --2- 172.21.15.39:0/1922226774 >> v2:172.21.15.84:3300/0 conn(0x7fa8bc0bf4f0 0x7fa8bc153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c11a4640 1 -- 172.21.15.39:0/1922226774 >> v1:172.21.15.84:6789/0 conn(0x7fa8bc0611c0 legacy=0x7fa8bc0bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:54:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.114+0000 7fa8c11a4640 1 -- 172.21.15.39:0/1922226774 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc0c1590 con 0x7fa8bc151860 2024-09-13T16:54:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c11a4640 1 --2- 172.21.15.39:0/1922226774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 0x7fa8bc157d80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc06b8b0 tx=0x7fa8b801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee996317761202b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8b3fff640 1 -- 172.21.15.39:0/1922226774 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80ab620 con 0x7fa8bc151860 2024-09-13T16:54:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8b3fff640 1 -- 172.21.15.39:0/1922226774 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b80ab780 con 0x7fa8bc151860 2024-09-13T16:54:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8b3fff640 1 -- 172.21.15.39:0/1922226774 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80abb10 con 0x7fa8bc151860 2024-09-13T16:54:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 -- 172.21.15.39:0/1922226774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 msgr2=0x7fa8bc157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 --2- 172.21.15.39:0/1922226774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 0x7fa8bc157d80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc06b8b0 tx=0x7fa8b801c420 comp rx=0 tx=0).stop 2024-09-13T16:54:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 -- 172.21.15.39:0/1922226774 shutdown_connections 2024-09-13T16:54:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 --2- 172.21.15.39:0/1922226774 >> v2:172.21.15.84:3300/0 conn(0x7fa8bc0bf4f0 0x7fa8bc153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 --2- 172.21.15.39:0/1922226774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 0x7fa8bc157d80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 -- 172.21.15.39:0/1922226774 >> 172.21.15.39:0/1922226774 conn(0x7fa8bc069e10 msgr2=0x7fa8bc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 -- 172.21.15.39:0/1922226774 shutdown_connections 2024-09-13T16:54:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 -- 172.21.15.39:0/1922226774 wait complete. 2024-09-13T16:54:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 Processor -- start 2024-09-13T16:54:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.118+0000 7fa8c3408640 1 -- start start 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c3408640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa8bc0611c0 0x7fa8bc144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c3408640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc0bf4f0 0x7fa8bc1ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c3408640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 0x7fa8bc1ee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c3408640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc1468d0 con 0x7fa8bc151860 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c3408640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc146e00 con 0x7fa8bc0bf4f0 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c3408640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc1f0a70 con 0x7fa8bc0611c0 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c19a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 0x7fa8bc1ee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c19a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 0x7fa8bc1ee7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48568/0 (socket says 172.21.15.39:48568) 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c19a5640 1 -- 172.21.15.39:0/2886510299 learned_addr learned my addr 172.21.15.39:0/2886510299 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c09a3640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc0bf4f0 0x7fa8bc1ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c11a4640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa8bc0611c0 0x7fa8bc144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c19a5640 1 -- 172.21.15.39:0/2886510299 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa8bc0611c0 msgr2=0x7fa8bc144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c19a5640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa8bc0611c0 0x7fa8bc144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c19a5640 1 -- 172.21.15.39:0/2886510299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc0bf4f0 msgr2=0x7fa8bc1ee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c19a5640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc0bf4f0 0x7fa8bc1ee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c19a5640 1 -- 172.21.15.39:0/2886510299 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b80094d0 con 0x7fa8bc151860 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c09a3640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc0bf4f0 0x7fa8bc1ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c19a5640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 0x7fa8bc1ee7e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400b9a0 tx=0x7fa8b400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8b1ffb640 1 -- 172.21.15.39:0/2886510299 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b400cba0 con 0x7fa8bc151860 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8b1ffb640 1 -- 172.21.15.39:0/2886510299 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b400cd00 con 0x7fa8bc151860 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc1f0d50 con 0x7fa8bc151860 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8b1ffb640 1 -- 172.21.15.39:0/2886510299 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4014680 con 0x7fa8bc151860 2024-09-13T16:54:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8bc148180 con 0x7fa8bc151860 2024-09-13T16:54:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8bc1edfc0 con 0x7fa8bc151860 2024-09-13T16:54:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8b1ffb640 1 -- 172.21.15.39:0/2886510299 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa8b401d3b0 con 0x7fa8bc151860 2024-09-13T16:54:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8b1ffb640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa8a00666c0 0x7fa8a0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8b1ffb640 1 -- 172.21.15.39:0/2886510299 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7fa8b40846e0 con 0x7fa8bc151860 2024-09-13T16:54:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.122+0000 7fa8c11a4640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa8a00666c0 0x7fa8a0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.126+0000 7fa8c11a4640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa8a00666c0 0x7fa8a0068b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80af000 tx=0x7fa8b80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:59.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.130+0000 7fa8b1ffb640 1 -- 172.21.15.39:0/2886510299 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa8bc1edfc0 con 0x7fa8bc151860 2024-09-13T16:54:59.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.338+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8bc1edfc0 con 0x7fa8bc151860 2024-09-13T16:54:59.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.338+0000 7fa8b1ffb640 1 -- 172.21.15.39:0/2886510299 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fa8bc1edfc0 con 0x7fa8bc151860 2024-09-13T16:54:59.341 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:59.341 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5069,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:59.341 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 55 2024-09-13T16:54:59.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa8a00666c0 msgr2=0x7fa8a0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa8a00666c0 0x7fa8a0068b80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80af000 tx=0x7fa8b80a7000 comp rx=0 tx=0).stop 2024-09-13T16:54:59.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 msgr2=0x7fa8bc1ee7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 0x7fa8bc1ee7e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400b9a0 tx=0x7fa8b400be60 comp rx=0 tx=0).stop 2024-09-13T16:54:59.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 shutdown_connections 2024-09-13T16:54:59.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa8a00666c0 0x7fa8a0068b80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8bc0bf4f0 0x7fa8bc1ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc151860 0x7fa8bc1ee7e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 --2- 172.21.15.39:0/2886510299 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa8bc0611c0 0x7fa8bc144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 >> 172.21.15.39:0/2886510299 conn(0x7fa8bc069e10 msgr2=0x7fa8bc154570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:59.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 shutdown_connections 2024-09-13T16:54:59.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.346+0000 7fa8c3408640 1 -- 172.21.15.39:0/2886510299 wait complete. 2024-09-13T16:54:59.366 INFO:tasks.mds_thrash.fs.[b]:kill mds.d (rank=0) 2024-09-13T16:54:59.367 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:54:59.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:54:59.390 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:54:59.390 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:54:59.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.518+0000 7f102bb7e640 1 Processor -- start 2024-09-13T16:54:59.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.518+0000 7f102bb7e640 1 -- start start 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102bb7e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1024065ee0 0x7f102414f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102bb7e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102406ddf0 con 0x7f10240c0a90 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102bb7e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10240c5950 con 0x7f10240c2ef0 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102bb7e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10240c53d0 con 0x7f1024065ee0 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102991a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1024065ee0 0x7f102414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102991a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1024065ee0 0x7f102414f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48570/0 (socket says 172.21.15.39:48570) 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102991a640 1 -- 172.21.15.39:0/393677075 learned_addr learned my addr 172.21.15.39:0/393677075 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102991a640 1 -- 172.21.15.39:0/393677075 >> v1:172.21.15.84:6789/0 conn(0x7f10240c2ef0 legacy=0x7f102415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102991a640 1 -- 172.21.15.39:0/393677075 >> v1:172.21.15.39:6789/0 conn(0x7f10240c0a90 legacy=0x7f10240c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f1028918640 1 -- 172.21.15.39:0/393677075 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956374619 0 0) 0x7f102406ddf0 con 0x7f10240c0a90 2024-09-13T16:54:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102991a640 1 -- 172.21.15.39:0/393677075 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102406e040 con 0x7f1024065ee0 2024-09-13T16:54:59.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f102991a640 1 --2- 172.21.15.39:0/393677075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 0x7f102414f230 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1020009870 tx=0x7f102001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5e0e996aac25f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:59.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f1028918640 1 -- 172.21.15.39:0/393677075 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200a95f0 con 0x7f1024065ee0 2024-09-13T16:54:59.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f1028918640 1 -- 172.21.15.39:0/393677075 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10200a9750 con 0x7f1024065ee0 2024-09-13T16:54:59.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.522+0000 7f1028918640 1 -- 172.21.15.39:0/393677075 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200a9ae0 con 0x7f1024065ee0 2024-09-13T16:54:59.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- 172.21.15.39:0/393677075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 msgr2=0x7f102414f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 --2- 172.21.15.39:0/393677075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 0x7f102414f230 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1020009870 tx=0x7f102001c530 comp rx=0 tx=0).stop 2024-09-13T16:54:59.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- 172.21.15.39:0/393677075 shutdown_connections 2024-09-13T16:54:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 --2- 172.21.15.39:0/393677075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 0x7f102414f230 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- 172.21.15.39:0/393677075 >> 172.21.15.39:0/393677075 conn(0x7f10240c3310 msgr2=0x7f10240c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- 172.21.15.39:0/393677075 shutdown_connections 2024-09-13T16:54:59.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- 172.21.15.39:0/393677075 wait complete. 2024-09-13T16:54:59.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 Processor -- start 2024-09-13T16:54:59.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- start start 2024-09-13T16:54:59.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 0x7f102413dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10240c0a90 0x7f10241ee130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10240c2ef0 0x7f10241ee670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10241edca0 con 0x7f1024065ee0 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10241412a0 con 0x7f10240c2ef0 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10241eebb0 con 0x7f10240c0a90 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102991a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 0x7f102413dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102991a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 0x7f102413dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48586/0 (socket says 172.21.15.39:48586) 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102991a640 1 -- 172.21.15.39:0/3082434280 learned_addr learned my addr 172.21.15.39:0/3082434280 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102a11b640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10240c2ef0 0x7f10241ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f1029119640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10240c0a90 0x7f10241ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102991a640 1 -- 172.21.15.39:0/3082434280 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10240c0a90 msgr2=0x7f10241ee130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102991a640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10240c0a90 0x7f10241ee130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102991a640 1 -- 172.21.15.39:0/3082434280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10240c2ef0 msgr2=0x7f10241ee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102991a640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10240c2ef0 0x7f10241ee670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102991a640 1 -- 172.21.15.39:0/3082434280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200094d0 con 0x7f1024065ee0 2024-09-13T16:54:59.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102a11b640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10240c2ef0 0x7f10241ee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:54:59.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102991a640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 0x7f102413dac0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f102001a160 tx=0x7f10200192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:59.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f101a7fc640 1 -- 172.21.15.39:0/3082434280 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200b1800 con 0x7f1024065ee0 2024-09-13T16:54:59.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f101a7fc640 1 -- 172.21.15.39:0/3082434280 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10200b1960 con 0x7f1024065ee0 2024-09-13T16:54:59.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102406abf0 con 0x7f1024065ee0 2024-09-13T16:54:59.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f101a7fc640 1 -- 172.21.15.39:0/3082434280 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200b1d10 con 0x7f1024065ee0 2024-09-13T16:54:59.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.526+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102413ffc0 con 0x7f1024065ee0 2024-09-13T16:54:59.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.530+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1024066350 con 0x7f1024065ee0 2024-09-13T16:54:59.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.530+0000 7f101a7fc640 1 -- 172.21.15.39:0/3082434280 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f10200c43f0 con 0x7f1024065ee0 2024-09-13T16:54:59.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.530+0000 7f101a7fc640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10080666c0 0x7f1008068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:54:59.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.530+0000 7f101a7fc640 1 -- 172.21.15.39:0/3082434280 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7f102011f240 con 0x7f1024065ee0 2024-09-13T16:54:59.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.530+0000 7f1029119640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10080666c0 0x7f1008068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:54:59.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.530+0000 7f1029119640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10080666c0 0x7f1008068b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f10240c7650 tx=0x7f1014009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:54:59.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.534+0000 7f101a7fc640 1 -- 172.21.15.39:0/3082434280 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f10200ebff0 con 0x7f1024065ee0 2024-09-13T16:54:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.766+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1024066870 con 0x7f1024065ee0 2024-09-13T16:54:59.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.766+0000 7f101a7fc640 1 -- 172.21.15.39:0/3082434280 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f10200ef610 con 0x7f1024065ee0 2024-09-13T16:54:59.767 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:54:59.768 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5069,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:54:59.768 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 55 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10080666c0 msgr2=0x7f1008068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10080666c0 0x7f1008068b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f10240c7650 tx=0x7f1014009040 comp rx=0 tx=0).stop 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 msgr2=0x7f102413dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 0x7f102413dac0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f102001a160 tx=0x7f10200192d0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 shutdown_connections 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10080666c0 0x7f1008068b80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10240c2ef0 0x7f10241ee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10240c0a90 0x7f10241ee130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 --2- 172.21.15.39:0/3082434280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1024065ee0 0x7f102413dac0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 >> 172.21.15.39:0/3082434280 conn(0x7f10240c3310 msgr2=0x7f102406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 shutdown_connections 2024-09-13T16:54:59.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:54:59.770+0000 7f102bb7e640 1 -- 172.21.15.39:0/3082434280 wait complete. 2024-09-13T16:54:59.787 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:55:01.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb91100640 1 Processor -- start 2024-09-13T16:55:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb91100640 1 -- start start 2024-09-13T16:55:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb91100640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c151500 0x7fdb8c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb91100640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdb8c071dd0 0x7fdb8c154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb91100640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c0c5a30 con 0x7fdb8c0c47c0 2024-09-13T16:55:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb91100640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c1576a0 con 0x7fdb8c071dd0 2024-09-13T16:55:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb91100640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c157980 con 0x7fdb8c151500 2024-09-13T16:55:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb8259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdb8c071dd0 0x7fdb8c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb8259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdb8c071dd0 0x7fdb8c154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34914/0 (socket says 172.21.15.39:34914) 2024-09-13T16:55:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb8b59e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdb8c0c47c0 0x7fdb8c1545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:39078/0 (socket says 172.21.15.39:39078) 2024-09-13T16:55:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb8259c640 1 -- 172.21.15.39:0/2704115050 learned_addr learned my addr 172.21.15.39:0/2704115050 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:01.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb8ad9d640 1 --2- 172.21.15.39:0/2704115050 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c151500 0x7fdb8c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:01.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb8259c640 1 -- 172.21.15.39:0/2704115050 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c151500 msgr2=0x7fdb8c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:01.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb8259c640 1 --2- 172.21.15.39:0/2704115050 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c151500 0x7fdb8c0c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:01.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb8259c640 1 -- 172.21.15.39:0/2704115050 >> v1:172.21.15.39:6789/0 conn(0x7fdb8c0c47c0 legacy=0x7fdb8c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:01.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.926+0000 7fdb8259c640 1 -- 172.21.15.39:0/2704115050 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c157c60 con 0x7fdb8c071dd0 2024-09-13T16:55:01.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb8259c640 1 --2- 172.21.15.39:0/2704115050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c071dd0 0x7fdb8c154070 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009870 tx=0x7fdb7401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0682a7c2bdd742a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:01.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb8a59c640 1 -- 172.21.15.39:0/2704115050 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb740aa620 con 0x7fdb8c071dd0 2024-09-13T16:55:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb8a59c640 1 -- 172.21.15.39:0/2704115050 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb740aa780 con 0x7fdb8c071dd0 2024-09-13T16:55:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb8a59c640 1 -- 172.21.15.39:0/2704115050 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb740aab10 con 0x7fdb8c071dd0 2024-09-13T16:55:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 -- 172.21.15.39:0/2704115050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c071dd0 msgr2=0x7fdb8c154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 --2- 172.21.15.39:0/2704115050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c071dd0 0x7fdb8c154070 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009870 tx=0x7fdb7401c4a0 comp rx=0 tx=0).stop 2024-09-13T16:55:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 -- 172.21.15.39:0/2704115050 shutdown_connections 2024-09-13T16:55:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 --2- 172.21.15.39:0/2704115050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c071dd0 0x7fdb8c154070 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 --2- 172.21.15.39:0/2704115050 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c151500 0x7fdb8c0c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 -- 172.21.15.39:0/2704115050 >> 172.21.15.39:0/2704115050 conn(0x7fdb8c0ab530 msgr2=0x7fdb8c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 -- 172.21.15.39:0/2704115050 shutdown_connections 2024-09-13T16:55:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 -- 172.21.15.39:0/2704115050 wait complete. 2024-09-13T16:55:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 Processor -- start 2024-09-13T16:55:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 -- start start 2024-09-13T16:55:01.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb8c071dd0 0x7fdb8c147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c0c47c0 0x7fdb8c14ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c151500 0x7fdb8c14fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c14a9d0 con 0x7fdb8c151500 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c14b630 con 0x7fdb8c071dd0 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb91100640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c14bc40 con 0x7fdb8c0c47c0 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb8b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c151500 0x7fdb8c14fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb8b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c151500 0x7fdb8c14fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34918/0 (socket says 172.21.15.39:34918) 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.930+0000 7fdb8b59e640 1 -- 172.21.15.39:0/2258369322 learned_addr learned my addr 172.21.15.39:0/2258369322 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb8ad9d640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb8c071dd0 0x7fdb8c147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb8259c640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c0c47c0 0x7fdb8c14ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb8b59e640 1 -- 172.21.15.39:0/2258369322 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c0c47c0 msgr2=0x7fdb8c14ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb8b59e640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c0c47c0 0x7fdb8c14ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb8b59e640 1 -- 172.21.15.39:0/2258369322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb8c071dd0 msgr2=0x7fdb8c147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb8b59e640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb8c071dd0 0x7fdb8c147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb8b59e640 1 -- 172.21.15.39:0/2258369322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb740094d0 con 0x7fdb8c151500 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb8ad9d640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb8c071dd0 0x7fdb8c147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb8b59e640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c151500 0x7fdb8c14fb30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400c7a0 tx=0x7fdb8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb83fff640 1 -- 172.21.15.39:0/2258369322 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400a640 con 0x7fdb8c151500 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb83fff640 1 -- 172.21.15.39:0/2258369322 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8400a7a0 con 0x7fdb8c151500 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c14bf20 con 0x7fdb8c151500 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb83fff640 1 -- 172.21.15.39:0/2258369322 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400ab10 con 0x7fdb8c151500 2024-09-13T16:55:01.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c14b3a0 con 0x7fdb8c151500 2024-09-13T16:55:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c14ab40 con 0x7fdb8c151500 2024-09-13T16:55:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb83fff640 1 -- 172.21.15.39:0/2258369322 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb8400ac70 con 0x7fdb8c151500 2024-09-13T16:55:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb83fff640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb6c066690 0x7fdb6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.934+0000 7fdb83fff640 1 -- 172.21.15.39:0/2258369322 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7fdb84083ed0 con 0x7fdb8c151500 2024-09-13T16:55:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.938+0000 7fdb8ad9d640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb6c066690 0x7fdb6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.938+0000 7fdb8ad9d640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb6c066690 0x7fdb6c068b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0097a0 tx=0x7fdb7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:01.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:01.938+0000 7fdb83fff640 1 -- 172.21.15.39:0/2258369322 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdb84050d30 con 0x7fdb8c151500 2024-09-13T16:55:02.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb8c0c4280 con 0x7fdb8c151500 2024-09-13T16:55:02.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb83fff640 1 -- 172.21.15.39:0/2258369322 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fdb84017020 con 0x7fdb8c151500 2024-09-13T16:55:02.174 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:02.174 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5069,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:02.175 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 55 2024-09-13T16:55:02.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb6c066690 msgr2=0x7fdb6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb6c066690 0x7fdb6c068b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0097a0 tx=0x7fdb7c008040 comp rx=0 tx=0).stop 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c151500 msgr2=0x7fdb8c14fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c151500 0x7fdb8c14fb30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400c7a0 tx=0x7fdb8400cc60 comp rx=0 tx=0).stop 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 shutdown_connections 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb6c066690 0x7fdb6c068b50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c151500 0x7fdb8c14fb30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdb8c0c47c0 0x7fdb8c14ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 --2- 172.21.15.39:0/2258369322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb8c071dd0 0x7fdb8c147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.174+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 >> 172.21.15.39:0/2258369322 conn(0x7fdb8c0ab530 msgr2=0x7fdb8c0ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.178+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 shutdown_connections 2024-09-13T16:55:02.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:02.178+0000 7fdb91100640 1 -- 172.21.15.39:0/2258369322 wait complete. 2024-09-13T16:55:02.195 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:55:04.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:04.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfd0b7b640 1 Processor -- start 2024-09-13T16:55:04.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfd0b7b640 1 -- start start 2024-09-13T16:55:04.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfd0b7b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcfcc071660 0x7fcfcc157b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:04.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfd0b7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcfcc06cf80 0x7fcfcc153a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:04.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfd0b7b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc0c0480 con 0x7fcfcc072310 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfd0b7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc0c0760 con 0x7fcfcc06cf80 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfd0b7b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc157030 con 0x7fcfcc071660 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfcaffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcfcc06cf80 0x7fcfcc153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfcaffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcfcc06cf80 0x7fcfcc153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34922/0 (socket says 172.21.15.39:34922) 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfcaffd640 1 -- 172.21.15.39:0/2520229930 learned_addr learned my addr 172.21.15.39:0/2520229930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfcb7fe640 1 --2- 172.21.15.39:0/2520229930 >> v2:172.21.15.84:3300/0 conn(0x7fcfcc071660 0x7fcfcc157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfcaffd640 1 -- 172.21.15.39:0/2520229930 >> v2:172.21.15.84:3300/0 conn(0x7fcfcc071660 msgr2=0x7fcfcc157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/2520229930 >> v2:172.21.15.84:3300/0 conn(0x7fcfcc071660 0x7fcfcc157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfcaffd640 1 -- 172.21.15.39:0/2520229930 >> v1:172.21.15.84:6789/0 conn(0x7fcfcc072310 legacy=0x7fcfcc0c0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.358+0000 7fcfcaffd640 1 -- 172.21.15.39:0/2520229930 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc157310 con 0x7fcfcc06cf80 2024-09-13T16:55:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/2520229930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc06cf80 0x7fcfcc153a40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009870 tx=0x7fcfc001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4e01dbcf8181d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfca7fc640 1 -- 172.21.15.39:0/2520229930 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00ab640 con 0x7fcfcc06cf80 2024-09-13T16:55:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfca7fc640 1 -- 172.21.15.39:0/2520229930 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc00a7070 con 0x7fcfcc06cf80 2024-09-13T16:55:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfca7fc640 1 -- 172.21.15.39:0/2520229930 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00ab9c0 con 0x7fcfcc06cf80 2024-09-13T16:55:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/2520229930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc06cf80 msgr2=0x7fcfcc153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfd0b7b640 1 --2- 172.21.15.39:0/2520229930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc06cf80 0x7fcfcc153a40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009870 tx=0x7fcfc001c420 comp rx=0 tx=0).stop 2024-09-13T16:55:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/2520229930 shutdown_connections 2024-09-13T16:55:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfd0b7b640 1 --2- 172.21.15.39:0/2520229930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc06cf80 0x7fcfcc153a40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfd0b7b640 1 --2- 172.21.15.39:0/2520229930 >> v2:172.21.15.84:3300/0 conn(0x7fcfcc071660 0x7fcfcc157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/2520229930 >> 172.21.15.39:0/2520229930 conn(0x7fcfcc0c30c0 msgr2=0x7fcfcc0c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/2520229930 shutdown_connections 2024-09-13T16:55:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.362+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/2520229930 wait complete. 2024-09-13T16:55:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 Processor -- start 2024-09-13T16:55:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 -- start start 2024-09-13T16:55:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc06cf80 0x7fcfcc0a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc071660 0x7fcfcc0ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc072310 0x7fcfcc1effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0abc30 con 0x7fcfcc071660 2024-09-13T16:55:04.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc1f04f0 con 0x7fcfcc072310 2024-09-13T16:55:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0c5de0 con 0x7fcfcc06cf80 2024-09-13T16:55:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc071660 0x7fcfcc0ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc071660 0x7fcfcc0ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34926/0 (socket says 172.21.15.39:34926) 2024-09-13T16:55:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcaffd640 1 -- 172.21.15.39:0/824698449 learned_addr learned my addr 172.21.15.39:0/824698449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcb7fe640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc06cf80 0x7fcfcc0a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcbfff640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc072310 0x7fcfcc1effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcaffd640 1 -- 172.21.15.39:0/824698449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc06cf80 msgr2=0x7fcfcc0a9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc06cf80 0x7fcfcc0a9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcaffd640 1 -- 172.21.15.39:0/824698449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc072310 msgr2=0x7fcfcc1effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc072310 0x7fcfcc1effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcaffd640 1 -- 172.21.15.39:0/824698449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00094d0 con 0x7fcfcc071660 2024-09-13T16:55:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcbfff640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc072310 0x7fcfcc1effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:55:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc071660 0x7fcfcc0ab6f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00ad040 tx=0x7fcfc001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfb3fff640 1 -- 172.21.15.39:0/824698449 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00be3f0 con 0x7fcfcc071660 2024-09-13T16:55:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfb3fff640 1 -- 172.21.15.39:0/824698449 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc00be570 con 0x7fcfcc071660 2024-09-13T16:55:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc0c60c0 con 0x7fcfcc071660 2024-09-13T16:55:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfb3fff640 1 -- 172.21.15.39:0/824698449 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00be920 con 0x7fcfcc071660 2024-09-13T16:55:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc1efa90 con 0x7fcfcc071660 2024-09-13T16:55:04.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.366+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc1efc80 con 0x7fcfcc071660 2024-09-13T16:55:04.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.370+0000 7fcfb3fff640 1 -- 172.21.15.39:0/824698449 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcfc00b43f0 con 0x7fcfcc071660 2024-09-13T16:55:04.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.370+0000 7fcfb3fff640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb8066690 0x7fcfb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:04.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.370+0000 7fcfb3fff640 1 -- 172.21.15.39:0/824698449 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7fcfc011ecc0 con 0x7fcfcc071660 2024-09-13T16:55:04.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.370+0000 7fcfcb7fe640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb8066690 0x7fcfb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:04.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.370+0000 7fcfcb7fe640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb8066690 0x7fcfb8068b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b9d0 tx=0x7fcfbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.374+0000 7fcfb3fff640 1 -- 172.21.15.39:0/824698449 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcfc00eba90 con 0x7fcfcc071660 2024-09-13T16:55:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfcc153a40 con 0x7fcfcc071660 2024-09-13T16:55:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfb3fff640 1 -- 172.21.15.39:0/824698449 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5932 (secure 0 0 0) 0x7fcfc00ef0b0 con 0x7fcfcc071660 2024-09-13T16:55:04.590 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:04.590 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5069,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:04.590 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 55 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb8066690 msgr2=0x7fcfb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb8066690 0x7fcfb8068b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b9d0 tx=0x7fcfbc008040 comp rx=0 tx=0).stop 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc071660 msgr2=0x7fcfcc0ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc071660 0x7fcfcc0ab6f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00ad040 tx=0x7fcfc001ac60 comp rx=0 tx=0).stop 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 shutdown_connections 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb8066690 0x7fcfb8068b50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc072310 0x7fcfcc1effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc071660 0x7fcfcc0ab6f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 --2- 172.21.15.39:0/824698449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc06cf80 0x7fcfcc0a9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 >> 172.21.15.39:0/824698449 conn(0x7fcfcc0c30c0 msgr2=0x7fcfcc0c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 shutdown_connections 2024-09-13T16:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:04.590+0000 7fcfd0b7b640 1 -- 172.21.15.39:0/824698449 wait complete. 2024-09-13T16:55:04.609 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:55:06.609 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:06.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb28b98640 1 Processor -- start 2024-09-13T16:55:06.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb28b98640 1 -- start start 2024-09-13T16:55:06.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb28b98640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb24071410 0x7feb240ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:06.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb28b98640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 0x7feb240c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:06.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb28b98640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feb240c6df0 0x7feb2414eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:06.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb28b98640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb240ab8b0 con 0x7feb240c6df0 2024-09-13T16:55:06.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb28b98640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb241523e0 con 0x7feb24071410 2024-09-13T16:55:06.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb28b98640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb241526c0 con 0x7feb24072320 2024-09-13T16:55:06.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb23fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feb240c6df0 0x7feb2414eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:06.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb23fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feb240c6df0 0x7feb2414eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34938/0 (socket says 172.21.15.39:34938) 2024-09-13T16:55:06.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb23fff640 1 -- 172.21.15.39:0/310015482 learned_addr learned my addr 172.21.15.39:0/310015482 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:06.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb237fe640 1 --2- 172.21.15.39:0/310015482 >> v2:172.21.15.84:3300/0 conn(0x7feb24071410 0x7feb240ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:06.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb22ffd640 1 --2- 172.21.15.39:0/310015482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 0x7feb240c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:06.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb23fff640 1 -- 172.21.15.39:0/310015482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 msgr2=0x7feb240c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:06.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb23fff640 1 --2- 172.21.15.39:0/310015482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 0x7feb240c5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:06.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb23fff640 1 -- 172.21.15.39:0/310015482 >> v2:172.21.15.84:3300/0 conn(0x7feb24071410 msgr2=0x7feb240ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:06.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb23fff640 1 --2- 172.21.15.39:0/310015482 >> v2:172.21.15.84:3300/0 conn(0x7feb24071410 0x7feb240ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:06.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb23fff640 1 -- 172.21.15.39:0/310015482 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb240abae0 con 0x7feb240c6df0 2024-09-13T16:55:06.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.762+0000 7feb237fe640 1 --2- 172.21.15.39:0/310015482 >> v2:172.21.15.84:3300/0 conn(0x7feb24071410 0x7feb240ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:55:06.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb23fff640 1 --2- 172.21.15.39:0/310015482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 0x7feb2414eca0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7feb240c5090 tx=0x7feb1c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209fb57ba8094cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:06.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb227fc640 1 -- 172.21.15.39:0/310015482 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0a56f0 con 0x7feb240c6df0 2024-09-13T16:55:06.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb227fc640 1 -- 172.21.15.39:0/310015482 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c0a5850 con 0x7feb240c6df0 2024-09-13T16:55:06.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb227fc640 1 -- 172.21.15.39:0/310015482 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0a5be0 con 0x7feb240c6df0 2024-09-13T16:55:06.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 -- 172.21.15.39:0/310015482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 msgr2=0x7feb2414eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:06.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 --2- 172.21.15.39:0/310015482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 0x7feb2414eca0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7feb240c5090 tx=0x7feb1c01c960 comp rx=0 tx=0).stop 2024-09-13T16:55:06.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 -- 172.21.15.39:0/310015482 shutdown_connections 2024-09-13T16:55:06.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 --2- 172.21.15.39:0/310015482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 0x7feb2414eca0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:06.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 --2- 172.21.15.39:0/310015482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 0x7feb240c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 --2- 172.21.15.39:0/310015482 >> v2:172.21.15.84:3300/0 conn(0x7feb24071410 0x7feb240ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 -- 172.21.15.39:0/310015482 >> 172.21.15.39:0/310015482 conn(0x7feb240a92e0 msgr2=0x7feb240a96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 -- 172.21.15.39:0/310015482 shutdown_connections 2024-09-13T16:55:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 -- 172.21.15.39:0/310015482 wait complete. 2024-09-13T16:55:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 Processor -- start 2024-09-13T16:55:06.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 -- start start 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb24071410 0x7feb2414a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 0x7feb2406e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 0x7feb24149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb241ee740 con 0x7feb240c6df0 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb241edb30 con 0x7feb24071410 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.766+0000 7feb28b98640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb241ede10 con 0x7feb24072320 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb23fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 0x7feb24149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb23fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 0x7feb24149810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34948/0 (socket says 172.21.15.39:34948) 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb23fff640 1 -- 172.21.15.39:0/1253237126 learned_addr learned my addr 172.21.15.39:0/1253237126 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb22ffd640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 0x7feb2406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb237fe640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb24071410 0x7feb2414a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb23fff640 1 -- 172.21.15.39:0/1253237126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 msgr2=0x7feb2406e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb23fff640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 0x7feb2406e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb23fff640 1 -- 172.21.15.39:0/1253237126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb24071410 msgr2=0x7feb2414a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb23fff640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb24071410 0x7feb2414a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb23fff640 1 -- 172.21.15.39:0/1253237126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c009b30 con 0x7feb240c6df0 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb237fe640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb24071410 0x7feb2414a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb23fff640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 0x7feb24149810 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0af040 tx=0x7feb1c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb0bfff640 1 -- 172.21.15.39:0/1253237126 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0b8670 con 0x7feb240c6df0 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb0bfff640 1 -- 172.21.15.39:0/1253237126 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c0b87d0 con 0x7feb240c6df0 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb0bfff640 1 -- 172.21.15.39:0/1253237126 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0b8b80 con 0x7feb240c6df0 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb241ee0f0 con 0x7feb240c6df0 2024-09-13T16:55:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2414ad90 con 0x7feb240c6df0 2024-09-13T16:55:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb24155b50 con 0x7feb240c6df0 2024-09-13T16:55:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb0bfff640 1 -- 172.21.15.39:0/1253237126 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feb1c0b8ce0 con 0x7feb240c6df0 2024-09-13T16:55:06.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb0bfff640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb10066690 0x7feb10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:06.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb0bfff640 1 -- 172.21.15.39:0/1253237126 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7feb1c11fa90 con 0x7feb240c6df0 2024-09-13T16:55:06.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.770+0000 7feb237fe640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb10066690 0x7feb10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:06.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.774+0000 7feb237fe640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb10066690 0x7feb10068b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feb14006b20 tx=0x7feb14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:06.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:06.778+0000 7feb0bfff640 1 -- 172.21.15.39:0/1253237126 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feb1c0ec840 con 0x7feb240c6df0 2024-09-13T16:55:07.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.026+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb240aa5e0 con 0x7feb240c6df0 2024-09-13T16:55:07.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.026+0000 7feb0bfff640 1 -- 172.21.15.39:0/1253237126 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5932 (secure 0 0 0) 0x7feb1c0efe60 con 0x7feb240c6df0 2024-09-13T16:55:07.027 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:07.027 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5069,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:07.027 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 55 2024-09-13T16:55:07.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb10066690 msgr2=0x7feb10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:07.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb10066690 0x7feb10068b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feb14006b20 tx=0x7feb14008040 comp rx=0 tx=0).stop 2024-09-13T16:55:07.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 msgr2=0x7feb24149810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:07.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 0x7feb24149810 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0af040 tx=0x7feb1c019700 comp rx=0 tx=0).stop 2024-09-13T16:55:07.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 shutdown_connections 2024-09-13T16:55:07.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb10066690 0x7feb10068b50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:07.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240c6df0 0x7feb24149810 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:07.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb24072320 0x7feb2406e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:07.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 --2- 172.21.15.39:0/1253237126 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb24071410 0x7feb2414a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:07.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 >> 172.21.15.39:0/1253237126 conn(0x7feb240a92e0 msgr2=0x7feb240a96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:07.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 shutdown_connections 2024-09-13T16:55:07.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:07.030+0000 7feb28b98640 1 -- 172.21.15.39:0/1253237126 wait complete. 2024-09-13T16:55:07.046 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:55:09.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:09.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.178+0000 7f1d15fd8640 1 Processor -- start 2024-09-13T16:55:09.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.178+0000 7f1d15fd8640 1 -- start start 2024-09-13T16:55:09.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d15fd8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d100c2190 0x7f1d100c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:09.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d15fd8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d100c54a0 0x7f1d10151010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:09.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d15fd8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d10151a00 con 0x7f1d10151550 2024-09-13T16:55:09.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d15fd8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d10154d20 con 0x7f1d100c2190 2024-09-13T16:55:09.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d15fd8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d10154110 con 0x7f1d100c54a0 2024-09-13T16:55:09.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d157d7640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1d10151550 0x7f1d10071450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:39082/0 (socket says 172.21.15.39:39082) 2024-09-13T16:55:09.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d157d7640 1 -- 172.21.15.39:0/978229472 learned_addr learned my addr 172.21.15.39:0/978229472 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:55:09.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d14fd6640 1 --2- 172.21.15.39:0/978229472 >> v2:172.21.15.84:3300/0 conn(0x7f1d100c2190 0x7f1d100c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:09.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0ffff640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d100c54a0 0x7f1d10151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:09.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3837134215 0 0) 0x7f1d080014a0 con 0x7f1d10151550 2024-09-13T16:55:09.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d00003530 con 0x7f1d10151550 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2461923180 0 0) 0x7f1d00003530 con 0x7f1d10151550 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d100c54a0 msgr2=0x7f1d10151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d100c54a0 0x7f1d10151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 >> v2:172.21.15.84:3300/0 conn(0x7f1d100c2190 msgr2=0x7f1d100c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 --2- 172.21.15.39:0/978229472 >> v2:172.21.15.84:3300/0 conn(0x7f1d100c2190 0x7f1d100c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d101543f0 con 0x7f1d10151550 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1d08001b90 con 0x7f1d10151550 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 >> v1:172.21.15.39:6789/0 conn(0x7f1d10151550 legacy=0x7f1d10071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d100c54a0 0x7f1d00003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d00003f00 0x7f1d000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d00004d10 0x7f1d00005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:09.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d10154d20 con 0x7f1d00004d10 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d10154110 con 0x7f1d00003f00 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0000b9e0 con 0x7f1d100c54a0 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d0ffff640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d00003f00 0x7f1d000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d157d7640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d00004d10 0x7f1d00005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d14fd6640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d100c54a0 0x7f1d00003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d157d7640 1 -- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d100c54a0 msgr2=0x7f1d00003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d157d7640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d100c54a0 0x7f1d00003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d157d7640 1 -- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d00003f00 msgr2=0x7f1d000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d157d7640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d00003f00 0x7f1d000047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.182+0000 7f1d157d7640 1 -- 172.21.15.39:0/978229472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d00003530 con 0x7f1d00004d10 2024-09-13T16:55:09.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d157d7640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d00004d10 0x7f1d00005190 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800a860 tx=0x7f1d0801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ae2eed7c5e6633 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:09.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d080aa6f0 con 0x7f1d00004d10 2024-09-13T16:55:09.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d080aa870 con 0x7f1d00004d10 2024-09-13T16:55:09.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0f7fe640 1 -- 172.21.15.39:0/978229472 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d080aac60 con 0x7f1d00004d10 2024-09-13T16:55:09.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 -- 172.21.15.39:0/978229472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d00004d10 msgr2=0x7f1d00005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:09.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 --2- 172.21.15.39:0/978229472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d00004d10 0x7f1d00005190 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800a860 tx=0x7f1d0801fac0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d157d7640 1 -- 172.21.15.39:0/978229472 reap_dead start 2024-09-13T16:55:09.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 -- 172.21.15.39:0/978229472 shutdown_connections 2024-09-13T16:55:09.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 -- 172.21.15.39:0/978229472 >> 172.21.15.39:0/978229472 conn(0x7f1d100a92e0 msgr2=0x7f1d100a96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:09.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 -- 172.21.15.39:0/978229472 shutdown_connections 2024-09-13T16:55:09.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 -- 172.21.15.39:0/978229472 wait complete. 2024-09-13T16:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 Processor -- start 2024-09-13T16:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 -- start start 2024-09-13T16:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d101491d0 0x7f1d10146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d10146e20 0x7f1d101467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d100ac680 0x7f1d101eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100ac380 con 0x7f1d10146e20 2024-09-13T16:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100ab540 con 0x7f1d100ac680 2024-09-13T16:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d15fd8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100aaf80 con 0x7f1d101491d0 2024-09-13T16:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d10146e20 0x7f1d101467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d10146e20 0x7f1d101467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34970/0 (socket says 172.21.15.39:34970) 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0ffff640 1 -- 172.21.15.39:0/3052250251 learned_addr learned my addr 172.21.15.39:0/3052250251 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d157d7640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d100ac680 0x7f1d101eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d14fd6640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d101491d0 0x7f1d10146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0ffff640 1 -- 172.21.15.39:0/3052250251 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d101491d0 msgr2=0x7f1d10146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0ffff640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d101491d0 0x7f1d10146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0ffff640 1 -- 172.21.15.39:0/3052250251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d100ac680 msgr2=0x7f1d101eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0ffff640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d100ac680 0x7f1d101eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.186+0000 7f1d0ffff640 1 -- 172.21.15.39:0/3052250251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d100ab260 con 0x7f1d10146e20 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d0ffff640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d10146e20 0x7f1d101467c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0118b0 tx=0x7f1cfc011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d0d7fa640 1 -- 172.21.15.39:0/3052250251 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00f9d0 con 0x7f1d10146e20 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d0d7fa640 1 -- 172.21.15.39:0/3052250251 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc00fb30 con 0x7f1d10146e20 2024-09-13T16:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d101ea890 con 0x7f1d10146e20 2024-09-13T16:55:09.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d0d7fa640 1 -- 172.21.15.39:0/3052250251 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00fec0 con 0x7f1d10146e20 2024-09-13T16:55:09.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d101495b0 con 0x7f1d10146e20 2024-09-13T16:55:09.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d100acb50 con 0x7f1d10146e20 2024-09-13T16:55:09.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d0d7fa640 1 -- 172.21.15.39:0/3052250251 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1cfc014440 con 0x7f1d10146e20 2024-09-13T16:55:09.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d0d7fa640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cf40666c0 0x7f1cf4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:09.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d0d7fa640 1 -- 172.21.15.39:0/3052250251 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7f1cfc084f20 con 0x7f1d10146e20 2024-09-13T16:55:09.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d14fd6640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cf40666c0 0x7f1cf4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:09.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.190+0000 7f1d14fd6640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cf40666c0 0x7f1cf4068b80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009b00 tx=0x7f1d04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:09.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.198+0000 7f1d0d7fa640 1 -- 172.21.15.39:0/3052250251 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1cfc051cd0 con 0x7f1d10146e20 2024-09-13T16:55:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.426+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d101ea6f0 con 0x7f1d10146e20 2024-09-13T16:55:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.426+0000 7f1d0d7fa640 1 -- 172.21.15.39:0/3052250251 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f1d101ea6f0 con 0x7f1d10146e20 2024-09-13T16:55:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:09.427 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5069,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 55 2024-09-13T16:55:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cf40666c0 msgr2=0x7f1cf4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cf40666c0 0x7f1cf4068b80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009b00 tx=0x7f1d04008040 comp rx=0 tx=0).stop 2024-09-13T16:55:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d10146e20 msgr2=0x7f1d101467c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d10146e20 0x7f1d101467c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0118b0 tx=0x7f1cfc011d70 comp rx=0 tx=0).stop 2024-09-13T16:55:09.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 shutdown_connections 2024-09-13T16:55:09.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cf40666c0 0x7f1cf4068b80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d100ac680 0x7f1d101eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d10146e20 0x7f1d101467c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 --2- 172.21.15.39:0/3052250251 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d101491d0 0x7f1d10146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:09.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 >> 172.21.15.39:0/3052250251 conn(0x7f1d100a92e0 msgr2=0x7f1d10149aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:09.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 shutdown_connections 2024-09-13T16:55:09.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:09.430+0000 7f1d15fd8640 1 -- 172.21.15.39:0/3052250251 wait complete. 2024-09-13T16:55:09.455 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:55:11.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:11.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e693bc640 1 Processor -- start 2024-09-13T16:55:11.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e693bc640 1 -- start start 2024-09-13T16:55:11.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e693bc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151500 0x7f6e640c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:11.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e693bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e640c5a30 con 0x7f6e64071dd0 2024-09-13T16:55:11.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e693bc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e641576a0 con 0x7f6e640c47c0 2024-09-13T16:55:11.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e693bc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64157980 con 0x7f6e64151500 2024-09-13T16:55:11.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e5bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f6e64071dd0 0x7f6e64154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37040/0 (socket says 172.21.15.39:37040) 2024-09-13T16:55:11.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e5bfff640 1 -- 172.21.15.39:0/1829769675 learned_addr learned my addr 172.21.15.39:0/1829769675 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:55:11.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e62ffd640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151500 0x7f6e640c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494371403 0 0) 0x7f6e640c5a30 con 0x7f6e64071dd0 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e50003530 con 0x7f6e64071dd0 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2061748808 0 0) 0x7f6e50003530 con 0x7f6e64071dd0 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151500 msgr2=0x7f6e640c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151500 0x7f6e640c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 >> v1:172.21.15.84:6789/0 conn(0x7f6e640c47c0 legacy=0x7f6e64154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64157c60 con 0x7f6e64071dd0 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f6e4c001680 con 0x7f6e64071dd0 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 >> v1:172.21.15.39:6789/0 conn(0x7f6e64071dd0 legacy=0x7f6e64154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e500039b0 0x7f6e50003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151500 0x7f6e500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e50005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64157980 con 0x7f6e50004d10 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5000ba40 con 0x7f6e500039b0 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5000bc00 con 0x7f6e64151500 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e637fe640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e50005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e62ffd640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e500039b0 0x7f6e50003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e5bfff640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151500 0x7f6e500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e637fe640 1 -- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151500 msgr2=0x7f6e500047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e637fe640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151500 0x7f6e500047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e637fe640 1 -- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e500039b0 msgr2=0x7f6e50003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e637fe640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e500039b0 0x7f6e50003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e637fe640 1 -- 172.21.15.39:0/1829769675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50003530 con 0x7f6e50004d10 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e62ffd640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e500039b0 0x7f6e50003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:55:11.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.594+0000 7f6e5bfff640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151500 0x7f6e500047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:55:11.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e637fe640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e50005190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c001f60 tx=0x7f6e5c01d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e1ef34e8b8af52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c0a5740 con 0x7f6e50004d10 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c0a58a0 con 0x7f6e50004d10 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e627fc640 1 -- 172.21.15.39:0/1829769675 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c0a5c50 con 0x7f6e50004d10 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e693bc640 1 -- 172.21.15.39:0/1829769675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 msgr2=0x7f6e50005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e693bc640 1 --2- 172.21.15.39:0/1829769675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e50005190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c001f60 tx=0x7f6e5c01d2b0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e637fe640 1 -- 172.21.15.39:0/1829769675 reap_dead start 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e693bc640 1 -- 172.21.15.39:0/1829769675 shutdown_connections 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e693bc640 1 -- 172.21.15.39:0/1829769675 >> 172.21.15.39:0/1829769675 conn(0x7f6e640ab530 msgr2=0x7f6e640ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e693bc640 1 -- 172.21.15.39:0/1829769675 shutdown_connections 2024-09-13T16:55:11.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e693bc640 1 -- 172.21.15.39:0/1829769675 wait complete. 2024-09-13T16:55:11.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e693bc640 1 Processor -- start 2024-09-13T16:55:11.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.598+0000 7f6e693bc640 1 -- start start 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e693bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e64152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e693bc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151e00 0x7f6e641525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e693bc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e64149200 0x7f6e64149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e693bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64149bc0 con 0x7f6e50004d10 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e693bc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64153650 con 0x7f6e64149200 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e693bc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64153b80 con 0x7f6e64151e00 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e62ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e64152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e62ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e64152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39892/0 (socket says 172.21.15.39:39892) 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e62ffd640 1 -- 172.21.15.39:0/3289599172 learned_addr learned my addr 172.21.15.39:0/3289599172 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e637fe640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e64149200 0x7f6e64149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e5bfff640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151e00 0x7f6e641525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:11.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e62ffd640 1 -- 172.21.15.39:0/3289599172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151e00 msgr2=0x7f6e641525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e62ffd640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151e00 0x7f6e641525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e62ffd640 1 -- 172.21.15.39:0/3289599172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e64149200 msgr2=0x7f6e64149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e62ffd640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e64149200 0x7f6e64149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e62ffd640 1 -- 172.21.15.39:0/3289599172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e641ed3d0 con 0x7f6e50004d10 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e62ffd640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e64152df0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009160 tx=0x7f6e54011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e5b7fe640 1 -- 172.21.15.39:0/3289599172 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54004220 con 0x7f6e50004d10 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e5b7fe640 1 -- 172.21.15.39:0/3289599172 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e54004380 con 0x7f6e50004d10 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e641ed6b0 con 0x7f6e50004d10 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e5b7fe640 1 -- 172.21.15.39:0/3289599172 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54015890 con 0x7f6e50004d10 2024-09-13T16:55:11.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6414c000 con 0x7f6e50004d10 2024-09-13T16:55:11.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.602+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e64154070 con 0x7f6e50004d10 2024-09-13T16:55:11.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.606+0000 7f6e5b7fe640 1 -- 172.21.15.39:0/3289599172 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e540159f0 con 0x7f6e50004d10 2024-09-13T16:55:11.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.606+0000 7f6e5b7fe640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e440666c0 0x7f6e44068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:11.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.606+0000 7f6e5b7fe640 1 -- 172.21.15.39:0/3289599172 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7f6e54085000 con 0x7f6e50004d10 2024-09-13T16:55:11.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.606+0000 7f6e5bfff640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e440666c0 0x7f6e44068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:11.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.606+0000 7f6e5bfff640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e440666c0 0x7f6e44068b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6e640713c0 tx=0x7f6e4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:11.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.610+0000 7f6e5b7fe640 1 -- 172.21.15.39:0/3289599172 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6e54051e90 con 0x7f6e50004d10 2024-09-13T16:55:11.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.846+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e64071bb0 con 0x7f6e50004d10 2024-09-13T16:55:11.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.846+0000 7f6e5b7fe640 1 -- 172.21.15.39:0/3289599172 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f6e54022070 con 0x7f6e50004d10 2024-09-13T16:55:11.848 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:11.848 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5069,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:11.849 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 55 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e440666c0 msgr2=0x7f6e44068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e440666c0 0x7f6e44068b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6e640713c0 tx=0x7f6e4c009040 comp rx=0 tx=0).stop 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 msgr2=0x7f6e64152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e64152df0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009160 tx=0x7f6e54011a60 comp rx=0 tx=0).stop 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 shutdown_connections 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e440666c0 0x7f6e44068b80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e64149200 0x7f6e64149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e64151e00 0x7f6e641525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 --2- 172.21.15.39:0/3289599172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e50004d10 0x7f6e64152df0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 >> 172.21.15.39:0/3289599172 conn(0x7f6e640ab530 msgr2=0x7f6e64150410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 shutdown_connections 2024-09-13T16:55:11.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:11.850+0000 7f6e693bc640 1 -- 172.21.15.39:0/3289599172 wait complete. 2024-09-13T16:55:11.869 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:55:13.870 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:14.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f97502f5640 1 Processor -- start 2024-09-13T16:55:14.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f97502f5640 1 -- start start 2024-09-13T16:55:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f97502f5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f97481556d0 0x7f97480c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f97502f5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f97480c7610 0x7f9748151b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f97502f5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974806feb0 con 0x7f97480a72b0 2024-09-13T16:55:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f97502f5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97480aae50 con 0x7f97481556d0 2024-09-13T16:55:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f97502f5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9748155ab0 con 0x7f97480c7610 2024-09-13T16:55:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f974e091640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f97481556d0 0x7f97480c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:14.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f974e892640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f97480a72b0 0x7f9748071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37044/0 (socket says 172.21.15.39:37044) 2024-09-13T16:55:14.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f974e091640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f97481556d0 0x7f97480c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39906/0 (socket says 172.21.15.39:39906) 2024-09-13T16:55:14.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f974e892640 1 -- 172.21.15.39:0/3264124702 learned_addr learned my addr 172.21.15.39:0/3264124702 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:55:14.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f974d890640 1 --2- 172.21.15.39:0/3264124702 >> v2:172.21.15.84:3300/0 conn(0x7f97480c7610 0x7f9748151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:14.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f974e091640 1 -- 172.21.15.39:0/3264124702 >> v2:172.21.15.84:3300/0 conn(0x7f97480c7610 msgr2=0x7f9748151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:14.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f974e091640 1 --2- 172.21.15.39:0/3264124702 >> v2:172.21.15.84:3300/0 conn(0x7f97480c7610 0x7f9748151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:14.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f974e091640 1 -- 172.21.15.39:0/3264124702 >> v1:172.21.15.39:6789/0 conn(0x7f97480a72b0 legacy=0x7f9748071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:14.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.026+0000 7f974e091640 1 -- 172.21.15.39:0/3264124702 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480c3c20 con 0x7f97481556d0 2024-09-13T16:55:14.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f974e091640 1 --2- 172.21.15.39:0/3264124702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97481556d0 0x7f97480c36e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f97480ab0c0 tx=0x7f974401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da4c7b0e3d1ac39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f974d08f640 1 -- 172.21.15.39:0/3264124702 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97440ab600 con 0x7f97481556d0 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f974d08f640 1 -- 172.21.15.39:0/3264124702 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97440a7050 con 0x7f97481556d0 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f974d08f640 1 -- 172.21.15.39:0/3264124702 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97440ab980 con 0x7f97481556d0 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f97502f5640 1 -- 172.21.15.39:0/3264124702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97481556d0 msgr2=0x7f97480c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f97502f5640 1 --2- 172.21.15.39:0/3264124702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97481556d0 0x7f97480c36e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f97480ab0c0 tx=0x7f974401c360 comp rx=0 tx=0).stop 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f97502f5640 1 -- 172.21.15.39:0/3264124702 shutdown_connections 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f97502f5640 1 --2- 172.21.15.39:0/3264124702 >> v2:172.21.15.84:3300/0 conn(0x7f97480c7610 0x7f9748151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f97502f5640 1 --2- 172.21.15.39:0/3264124702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97481556d0 0x7f97480c36e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f97502f5640 1 -- 172.21.15.39:0/3264124702 >> 172.21.15.39:0/3264124702 conn(0x7f97480a97b0 msgr2=0x7f97480a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f97502f5640 1 -- 172.21.15.39:0/3264124702 shutdown_connections 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.030+0000 7f97502f5640 1 -- 172.21.15.39:0/3264124702 wait complete. 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 Processor -- start 2024-09-13T16:55:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 -- start start 2024-09-13T16:55:14.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97480a72b0 0x7f974815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:14.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97481556d0 0x7f97481efb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:14.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9748154bf0 0x7f97481f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:14.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97481f0610 con 0x7f9748154bf0 2024-09-13T16:55:14.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97480a8220 con 0x7f97481556d0 2024-09-13T16:55:14.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97481f0820 con 0x7f97480a72b0 2024-09-13T16:55:14.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e892640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9748154bf0 0x7f97481f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e892640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9748154bf0 0x7f97481f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39914/0 (socket says 172.21.15.39:39914) 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e892640 1 -- 172.21.15.39:0/1825875564 learned_addr learned my addr 172.21.15.39:0/1825875564 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e091640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97480a72b0 0x7f974815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974d890640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97481556d0 0x7f97481efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e892640 1 -- 172.21.15.39:0/1825875564 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97480a72b0 msgr2=0x7f974815a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e892640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97480a72b0 0x7f974815a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e892640 1 -- 172.21.15.39:0/1825875564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97481556d0 msgr2=0x7f97481efb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e892640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97481556d0 0x7f97481efb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e892640 1 -- 172.21.15.39:0/1825875564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9744009420 con 0x7f9748154bf0 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974d890640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97481556d0 0x7f97481efb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f974e892640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9748154bf0 0x7f97481f00d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f974000ca80 tx=0x7f974000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:14.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f973effd640 1 -- 172.21.15.39:0/1825875564 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974000ac30 con 0x7f9748154bf0 2024-09-13T16:55:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f973effd640 1 -- 172.21.15.39:0/1825875564 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974000ad90 con 0x7f9748154bf0 2024-09-13T16:55:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97480a8470 con 0x7f9748154bf0 2024-09-13T16:55:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f973effd640 1 -- 172.21.15.39:0/1825875564 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740012600 con 0x7f9748154bf0 2024-09-13T16:55:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97480a8f70 con 0x7f9748154bf0 2024-09-13T16:55:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.034+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9748152040 con 0x7f9748154bf0 2024-09-13T16:55:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.038+0000 7f973effd640 1 -- 172.21.15.39:0/1825875564 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9740007070 con 0x7f9748154bf0 2024-09-13T16:55:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.038+0000 7f973effd640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f972c066690 0x7f972c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.038+0000 7f973effd640 1 -- 172.21.15.39:0/1825875564 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 7325+0+0 (secure 0 0 0) 0x7f9740084540 con 0x7f9748154bf0 2024-09-13T16:55:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.038+0000 7f974e091640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f972c066690 0x7f972c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:14.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.038+0000 7f974e091640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f972c066690 0x7f972c068b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f97440ad000 tx=0x7f97440a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:14.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.042+0000 7f973effd640 1 -- 172.21.15.39:0/1825875564 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9740051320 con 0x7f9748154bf0 2024-09-13T16:55:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.274+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9748151b80 con 0x7f9748154bf0 2024-09-13T16:55:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.274+0000 7f973effd640 1 -- 172.21.15.39:0/1825875564 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5932 (secure 0 0 0) 0x7f9740054940 con 0x7f9748154bf0 2024-09-13T16:55:14.275 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:14.275 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5069,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:53:17.755716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5362":{"gid":5362,"name":"d","rank":0,"incarnation":46,"state":"up:active","state_seq":6,"addr":"172.21.15.84:6835/3600322972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3600322972},{"type":"v1","addr":"172.21.15.84:6835","nonce":3600322972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 55 2024-09-13T16:55:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f972c066690 msgr2=0x7f972c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f972c066690 0x7f972c068b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f97440ad000 tx=0x7f97440a6000 comp rx=0 tx=0).stop 2024-09-13T16:55:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9748154bf0 msgr2=0x7f97481f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9748154bf0 0x7f97481f00d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f974000ca80 tx=0x7f974000cf40 comp rx=0 tx=0).stop 2024-09-13T16:55:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 shutdown_connections 2024-09-13T16:55:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f972c066690 0x7f972c068b50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97481556d0 0x7f97481efb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9748154bf0 0x7f97481f00d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 --2- 172.21.15.39:0/1825875564 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97480a72b0 0x7f974815a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 >> 172.21.15.39:0/1825875564 conn(0x7f97480a97b0 msgr2=0x7f9748150770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 shutdown_connections 2024-09-13T16:55:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:14.278+0000 7f97502f5640 1 -- 172.21.15.39:0/1825875564 wait complete. 2024-09-13T16:55:14.294 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:55:16.295 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:16.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.442+0000 7f8d7732d640 1 Processor -- start 2024-09-13T16:55:16.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d7732d640 1 -- start start 2024-09-13T16:55:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d7732d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d700bf4f0 0x7f8d70153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d7732d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d700611c0 0x7f8d700bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d7732d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700c1340 con 0x7f8d70151860 2024-09-13T16:55:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d7732d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d701578f0 con 0x7f8d700bf4f0 2024-09-13T16:55:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d7732d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700c1590 con 0x7f8d700611c0 2024-09-13T16:55:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d748c8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d700bf4f0 0x7f8d70153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d758ca640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d700611c0 0x7f8d700bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d748c8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d700bf4f0 0x7f8d70153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54900/0 (socket says 172.21.15.39:54900) 2024-09-13T16:55:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d748c8640 1 -- 172.21.15.39:0/1683119622 learned_addr learned my addr 172.21.15.39:0/1683119622 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d748c8640 1 -- 172.21.15.39:0/1683119622 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d700611c0 msgr2=0x7f8d700bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d748c8640 1 --2- 172.21.15.39:0/1683119622 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d700611c0 0x7f8d700bfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d748c8640 1 -- 172.21.15.39:0/1683119622 >> v1:172.21.15.84:6789/0 conn(0x7f8d70151860 legacy=0x7f8d70157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.446+0000 7f8d748c8640 1 -- 172.21.15.39:0/1683119622 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700c1870 con 0x7f8d700bf4f0 2024-09-13T16:55:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d748c8640 1 --2- 172.21.15.39:0/1683119622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d700bf4f0 0x7f8d70153df0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d6001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90a0222bbfc9065 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d67fff640 1 -- 172.21.15.39:0/1683119622 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600a6040 con 0x7f8d700bf4f0 2024-09-13T16:55:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d67fff640 1 -- 172.21.15.39:0/1683119622 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d600aa600 con 0x7f8d700bf4f0 2024-09-13T16:55:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d67fff640 1 -- 172.21.15.39:0/1683119622 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600aa990 con 0x7f8d700bf4f0 2024-09-13T16:55:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 -- 172.21.15.39:0/1683119622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d700bf4f0 msgr2=0x7f8d70153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 --2- 172.21.15.39:0/1683119622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d700bf4f0 0x7f8d70153df0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d6001c4a0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 -- 172.21.15.39:0/1683119622 shutdown_connections 2024-09-13T16:55:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 --2- 172.21.15.39:0/1683119622 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d700611c0 0x7f8d700bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 --2- 172.21.15.39:0/1683119622 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d700bf4f0 0x7f8d70153df0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 -- 172.21.15.39:0/1683119622 >> 172.21.15.39:0/1683119622 conn(0x7f8d70069e10 msgr2=0x7f8d7006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 -- 172.21.15.39:0/1683119622 shutdown_connections 2024-09-13T16:55:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 -- 172.21.15.39:0/1683119622 wait complete. 2024-09-13T16:55:16.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 Processor -- start 2024-09-13T16:55:16.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.450+0000 7f8d7732d640 1 -- start start 2024-09-13T16:55:16.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d7732d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d700611c0 0x7f8d70070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d7732d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700bf4f0 0x7f8d7014a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d7732d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d70151860 0x7f8d701eca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d7732d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70143ad0 con 0x7f8d700bf4f0 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d7732d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70071c50 con 0x7f8d700611c0 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d7732d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70071f30 con 0x7f8d70151860 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d748c8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700bf4f0 0x7f8d7014a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d748c8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700bf4f0 0x7f8d7014a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39924/0 (socket says 172.21.15.39:39924) 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d748c8640 1 -- 172.21.15.39:0/4262444260 learned_addr learned my addr 172.21.15.39:0/4262444260 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d750c9640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d700611c0 0x7f8d70070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d758ca640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d70151860 0x7f8d701eca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d748c8640 1 -- 172.21.15.39:0/4262444260 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d70151860 msgr2=0x7f8d701eca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d748c8640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d70151860 0x7f8d701eca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d748c8640 1 -- 172.21.15.39:0/4262444260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d700611c0 msgr2=0x7f8d70070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d748c8640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d700611c0 0x7f8d70070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d748c8640 1 -- 172.21.15.39:0/4262444260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d600094d0 con 0x7f8d700bf4f0 2024-09-13T16:55:16.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d748c8640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700bf4f0 0x7f8d7014a710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8d6001ab50 tx=0x7f8d6001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:16.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d65ffb640 1 -- 172.21.15.39:0/4262444260 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600b3600 con 0x7f8d700bf4f0 2024-09-13T16:55:16.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d65ffb640 1 -- 172.21.15.39:0/4262444260 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d600b3780 con 0x7f8d700bf4f0 2024-09-13T16:55:16.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d70070200 con 0x7f8d700bf4f0 2024-09-13T16:55:16.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d65ffb640 1 -- 172.21.15.39:0/4262444260 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600b3b30 con 0x7f8d700bf4f0 2024-09-13T16:55:16.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d700707c0 con 0x7f8d700bf4f0 2024-09-13T16:55:16.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d700709b0 con 0x7f8d700bf4f0 2024-09-13T16:55:16.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.454+0000 7f8d65ffb640 1 -- 172.21.15.39:0/4262444260 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8d600b3c90 con 0x7f8d700bf4f0 2024-09-13T16:55:16.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.458+0000 7f8d65ffb640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d540666c0 0x7f8d54068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.458+0000 7f8d65ffb640 1 -- 172.21.15.39:0/4262444260 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7f8d6011efe0 con 0x7f8d700bf4f0 2024-09-13T16:55:16.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.458+0000 7f8d750c9640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d540666c0 0x7f8d54068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.458+0000 7f8d750c9640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d540666c0 0x7f8d54068b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c000f70 tx=0x7f8d6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:16.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.462+0000 7f8d65ffb640 1 -- 172.21.15.39:0/4262444260 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8d600ebca0 con 0x7f8d700bf4f0 2024-09-13T16:55:16.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.698+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d700bfad0 con 0x7f8d700bf4f0 2024-09-13T16:55:16.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.698+0000 7f8d65ffb640 1 -- 172.21.15.39:0/4262444260 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f8d600ef2c0 con 0x7f8d700bf4f0 2024-09-13T16:55:16.698 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:16.699 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:16.635145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:16.699 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 58 2024-09-13T16:55:16.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d540666c0 msgr2=0x7f8d54068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d540666c0 0x7f8d54068b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c000f70 tx=0x7f8d6c009040 comp rx=0 tx=0).stop 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700bf4f0 msgr2=0x7f8d7014a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700bf4f0 0x7f8d7014a710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8d6001ab50 tx=0x7f8d6001ace0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 shutdown_connections 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d540666c0 0x7f8d54068b80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d70151860 0x7f8d701eca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d700bf4f0 0x7f8d7014a710 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 --2- 172.21.15.39:0/4262444260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d700611c0 0x7f8d70070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 >> 172.21.15.39:0/4262444260 conn(0x7f8d70069e10 msgr2=0x7f8d700c28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 shutdown_connections 2024-09-13T16:55:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.702+0000 7f8d7732d640 1 -- 172.21.15.39:0/4262444260 wait complete. 2024-09-13T16:55:16.722 INFO:tasks.mds_thrash.fs.[b]:mds.d down, removed from mdsmap 2024-09-13T16:55:16.722 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T16:55:16.722 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.838+0000 7f85e4b88640 1 Processor -- start 2024-09-13T16:55:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.838+0000 7f85e4b88640 1 -- start start 2024-09-13T16:55:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85e4b88640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e006fa50 0x7f85e0156130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85e4b88640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e00c5e50 con 0x7f85e00c6000 2024-09-13T16:55:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85e4b88640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e006ff50 con 0x7f85e00c7960 2024-09-13T16:55:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85e4b88640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0072000 con 0x7f85e006fa50 2024-09-13T16:55:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85ded9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f85e00c6000 0x7f85e00c2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37052/0 (socket says 172.21.15.39:37052) 2024-09-13T16:55:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85ded9d640 1 -- 172.21.15.39:0/3500188974 learned_addr learned my addr 172.21.15.39:0/3500188974 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:55:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85ddd9b640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e006fa50 0x7f85e0156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721124922 0 0) 0x7f85e00c5e50 con 0x7f85e00c6000 2024-09-13T16:55:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85c8003530 con 0x7f85e00c6000 2024-09-13T16:55:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2119352326 0 0) 0x7f85c8003530 con 0x7f85e00c6000 2024-09-13T16:55:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e006fa50 msgr2=0x7f85e0156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e006fa50 0x7f85e0156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 >> v1:172.21.15.84:6789/0 conn(0x7f85e00c7960 legacy=0x7f85e0155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e00722e0 con 0x7f85e00c6000 2024-09-13T16:55:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f85d8001a00 con 0x7f85e00c6000 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 >> v1:172.21.15.39:6789/0 conn(0x7f85e00c6000 legacy=0x7f85e00c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85c80047d0 0x7f85c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e006fa50 0x7f85c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0072000 con 0x7f85c80039b0 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c800baf0 con 0x7f85c80047d0 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80100c0 con 0x7f85e006fa50 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85de59c640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85ddd9b640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85c80047d0 0x7f85c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85ded9d640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e006fa50 0x7f85c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85de59c640 1 -- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e006fa50 msgr2=0x7f85c80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85de59c640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e006fa50 0x7f85c80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85de59c640 1 -- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85c80047d0 msgr2=0x7f85c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85de59c640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85c80047d0 0x7f85c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85de59c640 1 -- 172.21.15.39:0/3500188974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8003530 con 0x7f85c80039b0 2024-09-13T16:55:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.842+0000 7f85ded9d640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e006fa50 0x7f85c80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:55:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85de59c640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85c8003dd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f85d0001780 tx=0x7f85d001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835e3601aee0b33a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d00052d0 con 0x7f85c80039b0 2024-09-13T16:55:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d00a7050 con 0x7f85c80039b0 2024-09-13T16:55:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85dd59a640 1 -- 172.21.15.39:0/3500188974 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0003600 con 0x7f85c80039b0 2024-09-13T16:55:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 -- 172.21.15.39:0/3500188974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 msgr2=0x7f85c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 --2- 172.21.15.39:0/3500188974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85c8003dd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f85d0001780 tx=0x7f85d001c3f0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85ded9d640 1 -- 172.21.15.39:0/3500188974 reap_dead start 2024-09-13T16:55:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 -- 172.21.15.39:0/3500188974 shutdown_connections 2024-09-13T16:55:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 -- 172.21.15.39:0/3500188974 >> 172.21.15.39:0/3500188974 conn(0x7f85e006c020 msgr2=0x7f85e006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:16.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 -- 172.21.15.39:0/3500188974 shutdown_connections 2024-09-13T16:55:16.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 -- 172.21.15.39:0/3500188974 wait complete. 2024-09-13T16:55:16.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 Processor -- start 2024-09-13T16:55:16.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 -- start start 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85e01e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e00ac9f0 0x7f85e00a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e01eb830 0x7f85e014f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e00abdf0 con 0x7f85c80039b0 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e00ac0d0 con 0x7f85e00ac9f0 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85e4b88640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e01e8610 con 0x7f85e01eb830 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85de59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85e01e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85de59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85e01e8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39946/0 (socket says 172.21.15.39:39946) 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.846+0000 7f85de59c640 1 -- 172.21.15.39:0/1952365107 learned_addr learned my addr 172.21.15.39:0/1952365107 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ddd9b640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e00ac9f0 0x7f85e00a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ded9d640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e01eb830 0x7f85e014f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85de59c640 1 -- 172.21.15.39:0/1952365107 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e01eb830 msgr2=0x7f85e014f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85de59c640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e01eb830 0x7f85e014f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85de59c640 1 -- 172.21.15.39:0/1952365107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e00ac9f0 msgr2=0x7f85e00a89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85de59c640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e00ac9f0 0x7f85e00a89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85de59c640 1 -- 172.21.15.39:0/1952365107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e01e88f0 con 0x7f85c80039b0 2024-09-13T16:55:16.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85de59c640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85e01e8e10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f85d0005040 tx=0x7f85c8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:16.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ceffd640 1 -- 172.21.15.39:0/1952365107 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0007180 con 0x7f85c80039b0 2024-09-13T16:55:16.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ceffd640 1 -- 172.21.15.39:0/1952365107 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d00072e0 con 0x7f85c80039b0 2024-09-13T16:55:16.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ceffd640 1 -- 172.21.15.39:0/1952365107 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0007670 con 0x7f85c80039b0 2024-09-13T16:55:16.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e00a9e00 con 0x7f85c80039b0 2024-09-13T16:55:16.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e01570e0 con 0x7f85c80039b0 2024-09-13T16:55:16.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e01e8a80 con 0x7f85c80039b0 2024-09-13T16:55:16.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ceffd640 1 -- 172.21.15.39:0/1952365107 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85d00067e0 con 0x7f85c80039b0 2024-09-13T16:55:16.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ceffd640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85c0066690 0x7f85c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:16.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ceffd640 1 -- 172.21.15.39:0/1952365107 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7f85d011eb10 con 0x7f85c80039b0 2024-09-13T16:55:16.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ddd9b640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85c0066690 0x7f85c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:16.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.850+0000 7f85ddd9b640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85c0066690 0x7f85c0068b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f85e006bf90 tx=0x7f85d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:16.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:16.858+0000 7f85ceffd640 1 -- 172.21.15.39:0/1952365107 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f85d00eb860 con 0x7f85c80039b0 2024-09-13T16:55:17.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.114+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85e00c2570 con 0x7f85c80039b0 2024-09-13T16:55:17.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.114+0000 7f85ceffd640 1 -- 172.21.15.39:0/1952365107 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f85d00eee80 con 0x7f85c80039b0 2024-09-13T16:55:17.116 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:17.117 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:16.635145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 58 2024-09-13T16:55:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85c0066690 msgr2=0x7f85c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85c0066690 0x7f85c0068b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f85e006bf90 tx=0x7f85d4008040 comp rx=0 tx=0).stop 2024-09-13T16:55:17.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 msgr2=0x7f85e01e8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:17.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85e01e8e10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f85d0005040 tx=0x7f85c8005800 comp rx=0 tx=0).stop 2024-09-13T16:55:17.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 shutdown_connections 2024-09-13T16:55:17.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85c0066690 0x7f85c0068b50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:17.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e00ac9f0 0x7f85e00a89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:17.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85e01eb830 0x7f85e014f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 --2- 172.21.15.39:0/1952365107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85c80039b0 0x7f85e01e8e10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 >> 172.21.15.39:0/1952365107 conn(0x7f85e006c020 msgr2=0x7f85e006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 shutdown_connections 2024-09-13T16:55:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:17.118+0000 7f85e4b88640 1 -- 172.21.15.39:0/1952365107 wait complete. 2024-09-13T16:55:17.136 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 58, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 54, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5703": { "addr": "172.21.15.39:6835/4094657757", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 4094657757, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 4094657757, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5703, "incarnation": 51, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 34, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:54:12.632566+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5703 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 58, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5069": { "addr": "172.21.15.84:6833/2740003107", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 2740003107, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 2740003107, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5069, "incarnation": 56, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:rejoin", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 35, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:55:16.635145+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5069 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:55:17.803 INFO:tasks.ceph.mds.b.smithi084.stderr:2024-09-13T16:55:17.800+0000 7fd508306640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:55:19.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:19.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f541d9be640 1 Processor -- start 2024-09-13T16:55:19.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f541d9be640 1 -- start start 2024-09-13T16:55:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f541d9be640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 0x7f5418157d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f541d9be640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f54180bf4f0 0x7f5418153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f541d9be640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54180611c0 0x7f54180bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f541d9be640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54180c1340 con 0x7f54180bf4f0 2024-09-13T16:55:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f541d9be640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54181578f0 con 0x7f54180611c0 2024-09-13T16:55:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f541d9be640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54181582c0 con 0x7f5418151860 2024-09-13T16:55:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f54167fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f54180bf4f0 0x7f5418153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f54167fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f54180bf4f0 0x7f5418153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39956/0 (socket says 172.21.15.39:39956) 2024-09-13T16:55:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f54167fc640 1 -- 172.21.15.39:0/3992797467 learned_addr learned my addr 172.21.15.39:0/3992797467 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f5416ffd640 1 --2- 172.21.15.39:0/3992797467 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 0x7f5418157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.286+0000 7f54177fe640 1 --2- 172.21.15.39:0/3992797467 >> v2:172.21.15.84:3300/0 conn(0x7f54180611c0 0x7f54180bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f54167fc640 1 -- 172.21.15.39:0/3992797467 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 msgr2=0x7f5418157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f54167fc640 1 --2- 172.21.15.39:0/3992797467 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 0x7f5418157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f54167fc640 1 -- 172.21.15.39:0/3992797467 >> v2:172.21.15.84:3300/0 conn(0x7f54180611c0 msgr2=0x7f54180bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f54167fc640 1 --2- 172.21.15.39:0/3992797467 >> v2:172.21.15.84:3300/0 conn(0x7f54180611c0 0x7f54180bfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f54167fc640 1 -- 172.21.15.39:0/3992797467 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54180c1590 con 0x7f54180bf4f0 2024-09-13T16:55:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f54177fe640 1 --2- 172.21.15.39:0/3992797467 >> v2:172.21.15.84:3300/0 conn(0x7f54180611c0 0x7f54180bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:55:19.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f54167fc640 1 --2- 172.21.15.39:0/3992797467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180bf4f0 0x7f5418153df0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f54180bf910 tx=0x7f540c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c29d0e90a000a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f5415ffb640 1 -- 172.21.15.39:0/3992797467 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c0ab620 con 0x7f54180bf4f0 2024-09-13T16:55:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f5415ffb640 1 -- 172.21.15.39:0/3992797467 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c0ab780 con 0x7f54180bf4f0 2024-09-13T16:55:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f5415ffb640 1 -- 172.21.15.39:0/3992797467 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c0abb10 con 0x7f54180bf4f0 2024-09-13T16:55:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f541d9be640 1 -- 172.21.15.39:0/3992797467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180bf4f0 msgr2=0x7f5418153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f541d9be640 1 --2- 172.21.15.39:0/3992797467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180bf4f0 0x7f5418153df0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f54180bf910 tx=0x7f540c01c460 comp rx=0 tx=0).stop 2024-09-13T16:55:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f541d9be640 1 -- 172.21.15.39:0/3992797467 shutdown_connections 2024-09-13T16:55:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f541d9be640 1 --2- 172.21.15.39:0/3992797467 >> v2:172.21.15.84:3300/0 conn(0x7f54180611c0 0x7f54180bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f541d9be640 1 --2- 172.21.15.39:0/3992797467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180bf4f0 0x7f5418153df0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f541d9be640 1 --2- 172.21.15.39:0/3992797467 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 0x7f5418157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f541d9be640 1 -- 172.21.15.39:0/3992797467 >> 172.21.15.39:0/3992797467 conn(0x7f5418069e10 msgr2=0x7f541806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f541d9be640 1 -- 172.21.15.39:0/3992797467 shutdown_connections 2024-09-13T16:55:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.290+0000 7f541d9be640 1 -- 172.21.15.39:0/3992797467 wait complete. 2024-09-13T16:55:19.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 Processor -- start 2024-09-13T16:55:19.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 -- start start 2024-09-13T16:55:19.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180611c0 0x7f5418156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:19.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54180bf4f0 0x7f5418152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:19.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 0x7f5418070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:19.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5418155ee0 con 0x7f54180611c0 2024-09-13T16:55:19.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54180720c0 con 0x7f54180bf4f0 2024-09-13T16:55:19.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5418071b90 con 0x7f5418151860 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f5416ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180611c0 0x7f5418156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f5416ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180611c0 0x7f5418156ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39958/0 (socket says 172.21.15.39:39958) 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f5416ffd640 1 -- 172.21.15.39:0/2529666415 learned_addr learned my addr 172.21.15.39:0/2529666415 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f54167fc640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54180bf4f0 0x7f5418152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f54177fe640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 0x7f5418070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f5416ffd640 1 -- 172.21.15.39:0/2529666415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 msgr2=0x7f5418070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f5416ffd640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 0x7f5418070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f5416ffd640 1 -- 172.21.15.39:0/2529666415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54180bf4f0 msgr2=0x7f5418152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f5416ffd640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54180bf4f0 0x7f5418152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f5416ffd640 1 -- 172.21.15.39:0/2529666415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0094d0 con 0x7f54180611c0 2024-09-13T16:55:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f5416ffd640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180611c0 0x7f5418156ce0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f540800d5f0 tx=0x7f540800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f53fffff640 1 -- 172.21.15.39:0/2529666415 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54080125f0 con 0x7f54180611c0 2024-09-13T16:55:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f53fffff640 1 -- 172.21.15.39:0/2529666415 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5408012750 con 0x7f54180611c0 2024-09-13T16:55:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54180717b0 con 0x7f54180611c0 2024-09-13T16:55:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f53fffff640 1 -- 172.21.15.39:0/2529666415 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408012ae0 con 0x7f54180611c0 2024-09-13T16:55:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.294+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5418071de0 con 0x7f54180611c0 2024-09-13T16:55:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.298+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541806d970 con 0x7f54180611c0 2024-09-13T16:55:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.298+0000 7f53fffff640 1 -- 172.21.15.39:0/2529666415 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5408012c40 con 0x7f54180611c0 2024-09-13T16:55:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.298+0000 7f53fffff640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5404066690 0x7f5404068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.298+0000 7f53fffff640 1 -- 172.21.15.39:0/2529666415 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7f5408084ce0 con 0x7f54180611c0 2024-09-13T16:55:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.298+0000 7f54167fc640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5404066690 0x7f5404068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.298+0000 7f54167fc640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5404066690 0x7f5404068b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f540c0af000 tx=0x7f540c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:19.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.302+0000 7f53fffff640 1 -- 172.21.15.39:0/2529666415 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5408051a30 con 0x7f54180611c0 2024-09-13T16:55:19.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.510+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5418153b60 con 0x7f54180611c0 2024-09-13T16:55:19.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.510+0000 7f53fffff640 1 -- 172.21.15.39:0/2529666415 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f5408055050 con 0x7f54180611c0 2024-09-13T16:55:19.512 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:19.512 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:19.512 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 59 2024-09-13T16:55:19.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5404066690 msgr2=0x7f5404068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:19.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5404066690 0x7f5404068b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f540c0af000 tx=0x7f540c0a7000 comp rx=0 tx=0).stop 2024-09-13T16:55:19.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180611c0 msgr2=0x7f5418156ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:19.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180611c0 0x7f5418156ce0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f540800d5f0 tx=0x7f540800dab0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 shutdown_connections 2024-09-13T16:55:19.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5404066690 0x7f5404068b50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54180bf4f0 0x7f5418152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5418151860 0x7f5418070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 --2- 172.21.15.39:0/2529666415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54180611c0 0x7f5418156ce0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 >> 172.21.15.39:0/2529666415 conn(0x7f5418069e10 msgr2=0x7f541814ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 shutdown_connections 2024-09-13T16:55:19.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:19.514+0000 7f541d9be640 1 -- 172.21.15.39:0/2529666415 wait complete. 2024-09-13T16:55:19.531 INFO:tasks.mds_thrash.fs.[b]:mds.b has gained rank=0, replacing gid=5362 2024-09-13T16:55:19.531 INFO:tasks.mds_thrash.fs.[b]:waiting for 5 secs before reviving mds.d 2024-09-13T16:55:24.532 INFO:tasks.mds_thrash.fs.[b]:reviving mds.d 2024-09-13T16:55:24.532 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:55:24.532 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:55:24.535 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:55:24.535 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:24.618 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T16:55:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b7bfb640 1 Processor -- start 2024-09-13T16:55:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b7bfb640 1 -- start start 2024-09-13T16:55:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b7bfb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f81b006d2d0 0x7f81b015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b7bfb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b0156270 con 0x7f81b00ab340 2024-09-13T16:55:24.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b7bfb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b01554b0 con 0x7f81b00abc50 2024-09-13T16:55:24.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b7bfb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b0061530 con 0x7f81b006d2d0 2024-09-13T16:55:24.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b5196640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f81b00ab340 0x7f81b006cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:45522/0 (socket says 172.21.15.39:45522) 2024-09-13T16:55:24.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b5196640 1 -- 172.21.15.39:0/3316595506 learned_addr learned my addr 172.21.15.39:0/3316595506 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:55:24.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b6198640 1 --2- 172.21.15.39:0/3316595506 >> v2:172.21.15.84:3300/0 conn(0x7f81b006d2d0 0x7f81b015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:24.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b4995640 1 -- 172.21.15.39:0/3316595506 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647796947 0 0) 0x7f81b0156270 con 0x7f81b00ab340 2024-09-13T16:55:24.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b4995640 1 -- 172.21.15.39:0/3316595506 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f819c003530 con 0x7f81b00ab340 2024-09-13T16:55:24.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b6198640 1 -- 172.21.15.39:0/3316595506 >> v1:172.21.15.84:6789/0 conn(0x7f81b00abc50 legacy=0x7f81b014f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:24.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b6198640 1 -- 172.21.15.39:0/3316595506 >> v1:172.21.15.39:6789/0 conn(0x7f81b00ab340 legacy=0x7f81b006cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:24.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.690+0000 7f81b6198640 1 -- 172.21.15.39:0/3316595506 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0155790 con 0x7f81b006d2d0 2024-09-13T16:55:24.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b6198640 1 --2- 172.21.15.39:0/3316595506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 0x7f81b015d6b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009d90 tx=0x7f81a801c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d92b997a163d485 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:24.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b4995640 1 -- 172.21.15.39:0/3316595506 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80a58e0 con 0x7f81b006d2d0 2024-09-13T16:55:24.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b4995640 1 -- 172.21.15.39:0/3316595506 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a80a5a40 con 0x7f81b006d2d0 2024-09-13T16:55:24.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b4995640 1 -- 172.21.15.39:0/3316595506 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80a5dd0 con 0x7f81b006d2d0 2024-09-13T16:55:24.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b7bfb640 1 -- 172.21.15.39:0/3316595506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 msgr2=0x7f81b015d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:24.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b7bfb640 1 --2- 172.21.15.39:0/3316595506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 0x7f81b015d6b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009d90 tx=0x7f81a801c820 comp rx=0 tx=0).stop 2024-09-13T16:55:24.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b7bfb640 1 -- 172.21.15.39:0/3316595506 shutdown_connections 2024-09-13T16:55:24.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b7bfb640 1 --2- 172.21.15.39:0/3316595506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 0x7f81b015d6b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:24.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b7bfb640 1 -- 172.21.15.39:0/3316595506 >> 172.21.15.39:0/3316595506 conn(0x7f81b0069e10 msgr2=0x7f81b006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:24.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b7bfb640 1 -- 172.21.15.39:0/3316595506 shutdown_connections 2024-09-13T16:55:24.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b7bfb640 1 -- 172.21.15.39:0/3316595506 wait complete. 2024-09-13T16:55:24.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b7bfb640 1 Processor -- start 2024-09-13T16:55:24.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.694+0000 7f81b7bfb640 1 -- start start 2024-09-13T16:55:24.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b7bfb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 0x7f81b0144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b7bfb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b00ab340 0x7f81b015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b7bfb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81b00abc50 0x7f81b015d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b7bfb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b00708d0 con 0x7f81b00ab340 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b7bfb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0146a20 con 0x7f81b006d2d0 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b7bfb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0146f20 con 0x7f81b00abc50 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5196640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b00ab340 0x7f81b015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5196640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b00ab340 0x7f81b015cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45286/0 (socket says 172.21.15.39:45286) 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5196640 1 -- 172.21.15.39:0/1267167211 learned_addr learned my addr 172.21.15.39:0/1267167211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5196640 1 -- 172.21.15.39:0/1267167211 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81b00abc50 msgr2=0x7f81b015d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:24.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5997640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 0x7f81b0144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5196640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81b00abc50 0x7f81b015d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5196640 1 -- 172.21.15.39:0/1267167211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 msgr2=0x7f81b0144450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5196640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 0x7f81b0144450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5196640 1 -- 172.21.15.39:0/1267167211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a8009a70 con 0x7f81b00ab340 2024-09-13T16:55:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5997640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 0x7f81b0144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:55:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b5196640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b00ab340 0x7f81b015cd60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c420 tx=0x7f81a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81a67fc640 1 -- 172.21.15.39:0/1267167211 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0005cf0 con 0x7f81b00ab340 2024-09-13T16:55:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81a67fc640 1 -- 172.21.15.39:0/1267167211 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a0005e50 con 0x7f81b00ab340 2024-09-13T16:55:24.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b01ecdb0 con 0x7f81b00ab340 2024-09-13T16:55:24.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b0147200 con 0x7f81b00ab340 2024-09-13T16:55:24.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81a67fc640 1 -- 172.21.15.39:0/1267167211 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a00072d0 con 0x7f81b00ab340 2024-09-13T16:55:24.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8198005260 con 0x7f81b00ab340 2024-09-13T16:55:24.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.698+0000 7f81a67fc640 1 -- 172.21.15.39:0/1267167211 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f81a0007430 con 0x7f81b00ab340 2024-09-13T16:55:24.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.702+0000 7f81a67fc640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81940666c0 0x7f8194068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:24.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.702+0000 7f81a67fc640 1 -- 172.21.15.39:0/1267167211 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7f81a0083d10 con 0x7f81b00ab340 2024-09-13T16:55:24.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.702+0000 7f81b5997640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81940666c0 0x7f8194068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:24.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.702+0000 7f81b5997640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81940666c0 0x7f8194068b80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000f70 tx=0x7f81ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:24.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:24.706+0000 7f81a67fc640 1 -- 172.21.15.39:0/1267167211 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f81a0050a80 con 0x7f81b00ab340 2024-09-13T16:55:25.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.002+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8198005510 con 0x7f81b00ab340 2024-09-13T16:55:25.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81a67fc640 1 -- 172.21.15.39:0/1267167211 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f81a0017090 con 0x7f81b00ab340 2024-09-13T16:55:25.005 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:25.005 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5539,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:25.005 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 60 2024-09-13T16:55:25.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81940666c0 msgr2=0x7f8194068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:25.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81940666c0 0x7f8194068b80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000f70 tx=0x7f81ac009040 comp rx=0 tx=0).stop 2024-09-13T16:55:25.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b00ab340 msgr2=0x7f81b015cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:25.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b00ab340 0x7f81b015cd60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c420 tx=0x7f81a000c8e0 comp rx=0 tx=0).stop 2024-09-13T16:55:25.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 shutdown_connections 2024-09-13T16:55:25.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81940666c0 0x7f8194068b80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:25.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f81b00abc50 0x7f81b015d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:25.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b00ab340 0x7f81b015cd60 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:25.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 --2- 172.21.15.39:0/1267167211 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f81b006d2d0 0x7f81b0144450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:25.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 >> 172.21.15.39:0/1267167211 conn(0x7f81b0069e10 msgr2=0x7f81b006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:25.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 shutdown_connections 2024-09-13T16:55:25.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:25.006+0000 7f81b7bfb640 1 -- 172.21.15.39:0/1267167211 wait complete. 2024-09-13T16:55:25.024 INFO:tasks.mds_thrash.fs.[b]:mds.d reported in up:standby state 2024-09-13T16:55:25.024 INFO:tasks.mds_thrash.fs.[b]:waiting for 94 secs before thrashing 2024-09-13T16:55:26.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:55:26.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:55:50.057 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:50.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f59167b3640 1 Processor -- start 2024-09-13T16:55:50.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f59167b3640 1 -- start start 2024-09-13T16:55:50.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f59167b3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59100c2db0 0x7f5910061200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f59167b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f591014e800 0x7f59101545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f59167b3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591006fb40 con 0x7f59100c2560 2024-09-13T16:55:50.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f59167b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5910071730 con 0x7f591014e800 2024-09-13T16:55:50.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f59167b3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5910150da0 con 0x7f59100c2db0 2024-09-13T16:55:50.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f5914d50640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f591014e800 0x7f59101545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f5914d50640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f591014e800 0x7f59101545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45890/0 (socket says 172.21.15.39:45890) 2024-09-13T16:55:50.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f5914d50640 1 -- 172.21.15.39:0/3813187257 learned_addr learned my addr 172.21.15.39:0/3813187257 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:50.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f590ffff640 1 --2- 172.21.15.39:0/3813187257 >> v2:172.21.15.84:3300/0 conn(0x7f59100c2db0 0x7f5910061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f5914d50640 1 -- 172.21.15.39:0/3813187257 >> v2:172.21.15.84:3300/0 conn(0x7f59100c2db0 msgr2=0x7f5910061200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:50.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f5914d50640 1 --2- 172.21.15.39:0/3813187257 >> v2:172.21.15.84:3300/0 conn(0x7f59100c2db0 0x7f5910061200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f5914d50640 1 -- 172.21.15.39:0/3813187257 >> v1:172.21.15.84:6789/0 conn(0x7f59100c2560 legacy=0x7f591014e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.205+0000 7f5914d50640 1 -- 172.21.15.39:0/3813187257 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59100706c0 con 0x7f591014e800 2024-09-13T16:55:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.209+0000 7f5914d50640 1 --2- 172.21.15.39:0/3813187257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 0x7f59101545f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5910067310 tx=0x7f590801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55e2bcafda54717 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.209+0000 7f590effd640 1 -- 172.21.15.39:0/3813187257 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59080a5730 con 0x7f591014e800 2024-09-13T16:55:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.209+0000 7f590effd640 1 -- 172.21.15.39:0/3813187257 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59080a5890 con 0x7f591014e800 2024-09-13T16:55:50.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.209+0000 7f590effd640 1 -- 172.21.15.39:0/3813187257 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59080a5c20 con 0x7f591014e800 2024-09-13T16:55:50.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.213+0000 7f59167b3640 1 -- 172.21.15.39:0/3813187257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 msgr2=0x7f59101545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.213+0000 7f59167b3640 1 --2- 172.21.15.39:0/3813187257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 0x7f59101545f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5910067310 tx=0x7f590801ca10 comp rx=0 tx=0).stop 2024-09-13T16:55:50.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.213+0000 7f59167b3640 1 -- 172.21.15.39:0/3813187257 shutdown_connections 2024-09-13T16:55:50.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.213+0000 7f59167b3640 1 --2- 172.21.15.39:0/3813187257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 0x7f59101545f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.213+0000 7f59167b3640 1 --2- 172.21.15.39:0/3813187257 >> v2:172.21.15.84:3300/0 conn(0x7f59100c2db0 0x7f5910061200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.213+0000 7f59167b3640 1 -- 172.21.15.39:0/3813187257 >> 172.21.15.39:0/3813187257 conn(0x7f5910069e10 msgr2=0x7f591006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:50.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 -- 172.21.15.39:0/3813187257 shutdown_connections 2024-09-13T16:55:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 -- 172.21.15.39:0/3813187257 wait complete. 2024-09-13T16:55:50.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 Processor -- start 2024-09-13T16:55:50.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 -- start start 2024-09-13T16:55:50.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59100c2560 0x7f591014a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59100c2db0 0x7f5910143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 0x7f591014d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59100c4230 con 0x7f591014e800 2024-09-13T16:55:50.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591014abc0 con 0x7f59100c2db0 2024-09-13T16:55:50.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f59167b3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5910143540 con 0x7f59100c2560 2024-09-13T16:55:50.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f5914d50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 0x7f591014d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f5914d50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 0x7f591014d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45900/0 (socket says 172.21.15.39:45900) 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f5914d50640 1 -- 172.21.15.39:0/1408037962 learned_addr learned my addr 172.21.15.39:0/1408037962 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f590ffff640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59100c2560 0x7f591014a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f590f7fe640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59100c2db0 0x7f5910143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f5914d50640 1 -- 172.21.15.39:0/1408037962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59100c2560 msgr2=0x7f591014a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f5914d50640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59100c2560 0x7f591014a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f5914d50640 1 -- 172.21.15.39:0/1408037962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59100c2db0 msgr2=0x7f5910143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f5914d50640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59100c2db0 0x7f5910143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f5914d50640 1 -- 172.21.15.39:0/1408037962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5908009b30 con 0x7f591014e800 2024-09-13T16:55:50.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.217+0000 7f5914d50640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 0x7f591014d240 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f59080af040 tx=0x7f59080197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:50.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f590cff9640 1 -- 172.21.15.39:0/1408037962 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59080b8670 con 0x7f591014e800 2024-09-13T16:55:50.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f590cff9640 1 -- 172.21.15.39:0/1408037962 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59080b87d0 con 0x7f591014e800 2024-09-13T16:55:50.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591014a8e0 con 0x7f591014e800 2024-09-13T16:55:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f590cff9640 1 -- 172.21.15.39:0/1408037962 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59080b8b80 con 0x7f591014e800 2024-09-13T16:55:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5910144e80 con 0x7f591014e800 2024-09-13T16:55:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5910061200 con 0x7f591014e800 2024-09-13T16:55:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f590cff9640 1 -- 172.21.15.39:0/1408037962 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59080b8ce0 con 0x7f591014e800 2024-09-13T16:55:50.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f590cff9640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58fc066690 0x7f58fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f590cff9640 1 -- 172.21.15.39:0/1408037962 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7f590811fac0 con 0x7f591014e800 2024-09-13T16:55:50.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f590ffff640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58fc066690 0x7f58fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.221+0000 7f590ffff640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58fc066690 0x7f58fc068b50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5900009b60 tx=0x7f5900009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:50.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.225+0000 7f590cff9640 1 -- 172.21.15.39:0/1408037962 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59080ec9e0 con 0x7f591014e800 2024-09-13T16:55:50.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.453+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59100c0fb0 con 0x7f591014e800 2024-09-13T16:55:50.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.453+0000 7f590cff9640 1 -- 172.21.15.39:0/1408037962 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f5908009510 con 0x7f591014e800 2024-09-13T16:55:50.456 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:50.456 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5539,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:50.456 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 60 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58fc066690 msgr2=0x7f58fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58fc066690 0x7f58fc068b50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5900009b60 tx=0x7f5900009190 comp rx=0 tx=0).stop 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 msgr2=0x7f591014d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 0x7f591014d240 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f59080af040 tx=0x7f59080197b0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 shutdown_connections 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58fc066690 0x7f58fc068b50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59100c2db0 0x7f5910143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591014e800 0x7f591014d240 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 --2- 172.21.15.39:0/1408037962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59100c2560 0x7f591014a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 >> 172.21.15.39:0/1408037962 conn(0x7f5910069e10 msgr2=0x7f5910070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:50.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 shutdown_connections 2024-09-13T16:55:50.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.457+0000 7f59167b3640 1 -- 172.21.15.39:0/1408037962 wait complete. 2024-09-13T16:55:50.475 INFO:tasks.mds_thrash.fs.[a]:kill mds.a (rank=0) 2024-09-13T16:55:50.475 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:55:50.475 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:55:50.488 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:55:50.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.669+0000 7f5b037fa640 1 Processor -- start 2024-09-13T16:55:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.669+0000 7f5b037fa640 1 -- start start 2024-09-13T16:55:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.669+0000 7f5b037fa640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5afc0c2ea0 0x7f5afc0ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.669+0000 7f5b037fa640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5afc0ac300 0x7f5afc155920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.669+0000 7f5b037fa640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5afc071ef0 con 0x7f5afc0c3280 2024-09-13T16:55:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.669+0000 7f5b037fa640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc072420 con 0x7f5afc0c2ea0 2024-09-13T16:55:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.669+0000 7f5b037fa640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc0ab760 con 0x7f5afc0ac300 2024-09-13T16:55:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b00d95640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5afc0c3280 0x7f5afc0abbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54454/0 (socket says 172.21.15.39:54454) 2024-09-13T16:55:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b00d95640 1 -- 172.21.15.39:0/1278607406 learned_addr learned my addr 172.21.15.39:0/1278607406 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:55:50.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b01596640 1 --2- 172.21.15.39:0/1278607406 >> v2:172.21.15.39:3300/0 conn(0x7f5afc0c2ea0 0x7f5afc0ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b01d97640 1 --2- 172.21.15.39:0/1278607406 >> v2:172.21.15.84:3300/0 conn(0x7f5afc0ac300 0x7f5afc155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5af3fff640 1 -- 172.21.15.39:0/1278607406 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126039008 0 0) 0x7f5afc071ef0 con 0x7f5afc0c3280 2024-09-13T16:55:50.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5af3fff640 1 -- 172.21.15.39:0/1278607406 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae8003530 con 0x7f5afc0c3280 2024-09-13T16:55:50.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b01596640 1 -- 172.21.15.39:0/1278607406 >> v2:172.21.15.84:3300/0 conn(0x7f5afc0ac300 msgr2=0x7f5afc155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:50.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b01596640 1 --2- 172.21.15.39:0/1278607406 >> v2:172.21.15.84:3300/0 conn(0x7f5afc0ac300 0x7f5afc155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b01596640 1 -- 172.21.15.39:0/1278607406 >> v1:172.21.15.39:6789/0 conn(0x7f5afc0c3280 legacy=0x7f5afc0abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b01596640 1 -- 172.21.15.39:0/1278607406 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0aa570 con 0x7f5afc0c2ea0 2024-09-13T16:55:50.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b01596640 1 --2- 172.21.15.39:0/1278607406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c2ea0 0x7f5afc0ac730 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0c4010 tx=0x7f5af801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d014a0f8b47c392 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5af3fff640 1 -- 172.21.15.39:0/1278607406 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af80ab600 con 0x7f5afc0c2ea0 2024-09-13T16:55:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5af3fff640 1 -- 172.21.15.39:0/1278607406 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af80a7050 con 0x7f5afc0c2ea0 2024-09-13T16:55:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5af3fff640 1 -- 172.21.15.39:0/1278607406 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af80ab980 con 0x7f5afc0c2ea0 2024-09-13T16:55:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b037fa640 1 -- 172.21.15.39:0/1278607406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c2ea0 msgr2=0x7f5afc0ac730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b037fa640 1 --2- 172.21.15.39:0/1278607406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c2ea0 0x7f5afc0ac730 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0c4010 tx=0x7f5af801c3c0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b037fa640 1 -- 172.21.15.39:0/1278607406 shutdown_connections 2024-09-13T16:55:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b037fa640 1 --2- 172.21.15.39:0/1278607406 >> v2:172.21.15.84:3300/0 conn(0x7f5afc0ac300 0x7f5afc155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b037fa640 1 --2- 172.21.15.39:0/1278607406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c2ea0 0x7f5afc0ac730 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b037fa640 1 -- 172.21.15.39:0/1278607406 >> 172.21.15.39:0/1278607406 conn(0x7f5afc0a96b0 msgr2=0x7f5afc0a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b037fa640 1 -- 172.21.15.39:0/1278607406 shutdown_connections 2024-09-13T16:55:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.673+0000 7f5b037fa640 1 -- 172.21.15.39:0/1278607406 wait complete. 2024-09-13T16:55:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b037fa640 1 Processor -- start 2024-09-13T16:55:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b037fa640 1 -- start start 2024-09-13T16:55:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b037fa640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5afc0ac300 0x7f5afc06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b037fa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5afc0c2ea0 0x7f5afc0a8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b037fa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c3280 0x7f5afc146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b037fa640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc153120 con 0x7f5afc0c3280 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b037fa640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc06dde0 con 0x7f5afc0c2ea0 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b037fa640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1ec4b0 con 0x7f5afc0ac300 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01d97640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c3280 0x7f5afc146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01d97640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c3280 0x7f5afc146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45922/0 (socket says 172.21.15.39:45922) 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01d97640 1 -- 172.21.15.39:0/988647822 learned_addr learned my addr 172.21.15.39:0/988647822 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b00d95640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5afc0c2ea0 0x7f5afc0a8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01596640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5afc0ac300 0x7f5afc06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01d97640 1 -- 172.21.15.39:0/988647822 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5afc0ac300 msgr2=0x7f5afc06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01d97640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5afc0ac300 0x7f5afc06e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01d97640 1 -- 172.21.15.39:0/988647822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5afc0c2ea0 msgr2=0x7f5afc0a8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01d97640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5afc0c2ea0 0x7f5afc0a8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01d97640 1 -- 172.21.15.39:0/988647822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af8009420 con 0x7f5afc0c3280 2024-09-13T16:55:50.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b00d95640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5afc0c2ea0 0x7f5afc0a8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:55:50.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5b01d97640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c3280 0x7f5afc146650 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5af4007e40 tx=0x7f5af400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:50.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5af1ffb640 1 -- 172.21.15.39:0/988647822 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4014620 con 0x7f5afc0c3280 2024-09-13T16:55:50.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5af1ffb640 1 -- 172.21.15.39:0/988647822 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af400ee90 con 0x7f5afc0c3280 2024-09-13T16:55:50.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.677+0000 7f5af1ffb640 1 -- 172.21.15.39:0/988647822 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af40149e0 con 0x7f5afc0c3280 2024-09-13T16:55:50.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.681+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc1ec790 con 0x7f5afc0c3280 2024-09-13T16:55:50.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.681+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc1ecd50 con 0x7f5afc0c3280 2024-09-13T16:55:50.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.681+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc04e220 con 0x7f5afc0c3280 2024-09-13T16:55:50.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.681+0000 7f5af1ffb640 1 -- 172.21.15.39:0/988647822 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5af4014b40 con 0x7f5afc0c3280 2024-09-13T16:55:50.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.681+0000 7f5af1ffb640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ae0066690 0x7f5ae0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:50.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.681+0000 7f5af1ffb640 1 -- 172.21.15.39:0/988647822 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7f5af40851f0 con 0x7f5afc0c3280 2024-09-13T16:55:50.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.681+0000 7f5b01596640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ae0066690 0x7f5ae0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:50.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.681+0000 7f5b01596640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ae0066690 0x7f5ae0068b50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5af80ad000 tx=0x7f5af8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.689+0000 7f5af1ffb640 1 -- 172.21.15.39:0/988647822 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5af4051eb0 con 0x7f5afc0c3280 2024-09-13T16:55:50.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.909+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5afc1ed270 con 0x7f5afc0c3280 2024-09-13T16:55:50.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5af1ffb640 1 -- 172.21.15.39:0/988647822 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f5afc1ed270 con 0x7f5afc0c3280 2024-09-13T16:55:50.913 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:50.913 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5539,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:50.913 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 60 2024-09-13T16:55:50.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ae0066690 msgr2=0x7f5ae0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ae0066690 0x7f5ae0068b50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5af80ad000 tx=0x7f5af8008040 comp rx=0 tx=0).stop 2024-09-13T16:55:50.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c3280 msgr2=0x7f5afc146650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:50.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c3280 0x7f5afc146650 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5af4007e40 tx=0x7f5af400e4b0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 shutdown_connections 2024-09-13T16:55:50.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ae0066690 0x7f5ae0068b50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5afc0c2ea0 0x7f5afc0a8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5afc0c3280 0x7f5afc146650 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 --2- 172.21.15.39:0/988647822 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5afc0ac300 0x7f5afc06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:50.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 >> 172.21.15.39:0/988647822 conn(0x7f5afc0a96b0 msgr2=0x7f5afc1539b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:50.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 shutdown_connections 2024-09-13T16:55:50.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:50.913+0000 7f5b037fa640 1 -- 172.21.15.39:0/988647822 wait complete. 2024-09-13T16:55:50.938 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:55:52.939 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:53.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad2580c640 1 Processor -- start 2024-09-13T16:55:53.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad2580c640 1 -- start start 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad2580c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 0x7fad201577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad2580c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fad200c0be0 0x7fad20157d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad2580c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad20157330 con 0x7fad200c2690 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad2580c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad200c4a20 con 0x7fad200c0be0 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad2580c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad20158240 con 0x7fad200c6290 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad1f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fad200c0be0 0x7fad20157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad1f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fad200c0be0 0x7fad20157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45932/0 (socket says 172.21.15.39:45932) 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad1f7fe640 1 -- 172.21.15.39:0/3618950214 learned_addr learned my addr 172.21.15.39:0/3618950214 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad1e7fc640 1 --2- 172.21.15.39:0/3618950214 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 0x7fad201577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad1f7fe640 1 -- 172.21.15.39:0/3618950214 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 msgr2=0x7fad201577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:53.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad1f7fe640 1 --2- 172.21.15.39:0/3618950214 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 0x7fad201577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad1f7fe640 1 -- 172.21.15.39:0/3618950214 >> v1:172.21.15.84:6789/0 conn(0x7fad200c2690 legacy=0x7fad2006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:53.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.081+0000 7fad1f7fe640 1 -- 172.21.15.39:0/3618950214 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200c4740 con 0x7fad200c0be0 2024-09-13T16:55:53.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1f7fe640 1 --2- 172.21.15.39:0/3618950214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c0be0 0x7fad20157d00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad18009e50 tx=0x7fad1801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624732e9d7a3e10c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:53.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1dffb640 1 -- 172.21.15.39:0/3618950214 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad180a5830 con 0x7fad200c0be0 2024-09-13T16:55:53.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1dffb640 1 -- 172.21.15.39:0/3618950214 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad180a5990 con 0x7fad200c0be0 2024-09-13T16:55:53.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1dffb640 1 -- 172.21.15.39:0/3618950214 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad180a5d20 con 0x7fad200c0be0 2024-09-13T16:55:53.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 -- 172.21.15.39:0/3618950214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c0be0 msgr2=0x7fad20157d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:53.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 --2- 172.21.15.39:0/3618950214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c0be0 0x7fad20157d00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad18009e50 tx=0x7fad1801c970 comp rx=0 tx=0).stop 2024-09-13T16:55:53.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 -- 172.21.15.39:0/3618950214 shutdown_connections 2024-09-13T16:55:53.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 --2- 172.21.15.39:0/3618950214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c0be0 0x7fad20157d00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 --2- 172.21.15.39:0/3618950214 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 0x7fad201577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 -- 172.21.15.39:0/3618950214 >> 172.21.15.39:0/3618950214 conn(0x7fad200c32a0 msgr2=0x7fad200c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:53.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 -- 172.21.15.39:0/3618950214 shutdown_connections 2024-09-13T16:55:53.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 -- 172.21.15.39:0/3618950214 wait complete. 2024-09-13T16:55:53.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 Processor -- start 2024-09-13T16:55:53.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 -- start start 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad200c0be0 0x7fad201504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c2690 0x7fad20149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 0x7fad20149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad20148d70 con 0x7fad200c2690 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2014b070 con 0x7fad200c0be0 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad2580c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2014ad70 con 0x7fad200c6290 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c2690 0x7fad20149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c2690 0x7fad20149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45944/0 (socket says 172.21.15.39:45944) 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1e7fc640 1 -- 172.21.15.39:0/3117613176 learned_addr learned my addr 172.21.15.39:0/3117613176 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1effd640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad200c0be0 0x7fad201504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1f7fe640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 0x7fad20149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1e7fc640 1 -- 172.21.15.39:0/3117613176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 msgr2=0x7fad20149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1e7fc640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 0x7fad20149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1e7fc640 1 -- 172.21.15.39:0/3117613176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad200c0be0 msgr2=0x7fad201504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1e7fc640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad200c0be0 0x7fad201504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1e7fc640 1 -- 172.21.15.39:0/3117613176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad18009b30 con 0x7fad200c2690 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.085+0000 7fad1e7fc640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c2690 0x7fad20149860 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fad1400d830 tx=0x7fad1400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad07fff640 1 -- 172.21.15.39:0/3117613176 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14004030 con 0x7fad200c2690 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad07fff640 1 -- 172.21.15.39:0/3117613176 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad14004190 con 0x7fad200c2690 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad20156da0 con 0x7fad200c2690 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad07fff640 1 -- 172.21.15.39:0/3117613176 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14004520 con 0x7fad200c2690 2024-09-13T16:55:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad201509e0 con 0x7fad200c2690 2024-09-13T16:55:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2014b220 con 0x7fad200c2690 2024-09-13T16:55:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad07fff640 1 -- 172.21.15.39:0/3117613176 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad1400b6c0 con 0x7fad200c2690 2024-09-13T16:55:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad07fff640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad0c0666c0 0x7fad0c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad07fff640 1 -- 172.21.15.39:0/3117613176 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7fad14084950 con 0x7fad200c2690 2024-09-13T16:55:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad1effd640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad0c0666c0 0x7fad0c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:53.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.089+0000 7fad1effd640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad0c0666c0 0x7fad0c068b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fad10001650 tx=0x7fad10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.093+0000 7fad07fff640 1 -- 172.21.15.39:0/3117613176 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fad2014b220 con 0x7fad200c2690 2024-09-13T16:55:53.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad2014b220 con 0x7fad200c2690 2024-09-13T16:55:53.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad07fff640 1 -- 172.21.15.39:0/3117613176 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5930 (secure 0 0 0) 0x7fad2014b220 con 0x7fad200c2690 2024-09-13T16:55:53.294 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:53.294 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5539,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:53.294 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 60 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad0c0666c0 msgr2=0x7fad0c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad0c0666c0 0x7fad0c068b80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fad10001650 tx=0x7fad10009040 comp rx=0 tx=0).stop 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c2690 msgr2=0x7fad20149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c2690 0x7fad20149860 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fad1400d830 tx=0x7fad1400dcf0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 shutdown_connections 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fad0c0666c0 0x7fad0c068b80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fad200c6290 0x7fad20149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad200c2690 0x7fad20149860 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 --2- 172.21.15.39:0/3117613176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad200c0be0 0x7fad201504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 >> 172.21.15.39:0/3117613176 conn(0x7fad200c32a0 msgr2=0x7fad200a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.293+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 shutdown_connections 2024-09-13T16:55:53.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:53.297+0000 7fad2580c640 1 -- 172.21.15.39:0/3117613176 wait complete. 2024-09-13T16:55:53.313 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:55:55.314 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:55.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef3a970640 1 Processor -- start 2024-09-13T16:55:55.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef3a970640 1 -- start start 2024-09-13T16:55:55.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef3a970640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fef34071dd0 0x7fef34154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef3a970640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef340c5a30 con 0x7fef340c7800 2024-09-13T16:55:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef3a970640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef341576a0 con 0x7fef34151500 2024-09-13T16:55:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef3a970640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef34157980 con 0x7fef34071dd0 2024-09-13T16:55:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef337fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fef34071dd0 0x7fef34154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef38f0d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fef340c7800 0x7fef341545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54460/0 (socket says 172.21.15.39:54460) 2024-09-13T16:55:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef38f0d640 1 -- 172.21.15.39:0/3596535805 learned_addr learned my addr 172.21.15.39:0/3596535805 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:55:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef337fe640 1 -- 172.21.15.39:0/3596535805 >> v1:172.21.15.84:6789/0 conn(0x7fef34151500 legacy=0x7fef340c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef337fe640 1 -- 172.21.15.39:0/3596535805 >> v1:172.21.15.39:6789/0 conn(0x7fef340c7800 legacy=0x7fef341545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef32ffd640 1 -- 172.21.15.39:0/3596535805 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451265413 0 0) 0x7fef340c5a30 con 0x7fef340c7800 2024-09-13T16:55:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.449+0000 7fef337fe640 1 -- 172.21.15.39:0/3596535805 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef34157c60 con 0x7fef34071dd0 2024-09-13T16:55:55.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 --2- 172.21.15.39:0/3596535805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34071dd0 0x7fef34154070 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fef280097f0 tx=0x7fef2801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b589c8adfba1789 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:55.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef32ffd640 1 -- 172.21.15.39:0/3596535805 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280ab600 con 0x7fef34071dd0 2024-09-13T16:55:55.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef32ffd640 1 -- 172.21.15.39:0/3596535805 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef280ab760 con 0x7fef34071dd0 2024-09-13T16:55:55.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef32ffd640 1 -- 172.21.15.39:0/3596535805 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280abaf0 con 0x7fef34071dd0 2024-09-13T16:55:55.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- 172.21.15.39:0/3596535805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34071dd0 msgr2=0x7fef34154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:55.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 --2- 172.21.15.39:0/3596535805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34071dd0 0x7fef34154070 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fef280097f0 tx=0x7fef2801c3e0 comp rx=0 tx=0).stop 2024-09-13T16:55:55.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- 172.21.15.39:0/3596535805 shutdown_connections 2024-09-13T16:55:55.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 --2- 172.21.15.39:0/3596535805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34071dd0 0x7fef34154070 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:55.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- 172.21.15.39:0/3596535805 >> 172.21.15.39:0/3596535805 conn(0x7fef340ab530 msgr2=0x7fef340ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:55.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- 172.21.15.39:0/3596535805 shutdown_connections 2024-09-13T16:55:55.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- 172.21.15.39:0/3596535805 wait complete. 2024-09-13T16:55:55.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 Processor -- start 2024-09-13T16:55:55.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- start start 2024-09-13T16:55:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef340c7800 0x7fef340c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34151500 0x7fef341ec910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef3414e750 0x7fef341ece50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef341538a0 con 0x7fef34151500 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef34151c70 con 0x7fef340c7800 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef341ec350 con 0x7fef3414e750 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34151500 0x7fef341ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34151500 0x7fef341ec910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45960/0 (socket says 172.21.15.39:45960) 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 -- 172.21.15.39:0/1504477456 learned_addr learned my addr 172.21.15.39:0/1504477456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef33fff640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef340c7800 0x7fef340c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef38f0d640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef3414e750 0x7fef341ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 -- 172.21.15.39:0/1504477456 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef3414e750 msgr2=0x7fef341ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef3414e750 0x7fef341ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 -- 172.21.15.39:0/1504477456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef340c7800 msgr2=0x7fef340c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef340c7800 0x7fef340c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 -- 172.21.15.39:0/1504477456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef28009420 con 0x7fef34151500 2024-09-13T16:55:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef33fff640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef340c7800 0x7fef340c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:55:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef337fe640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34151500 0x7fef341ec910 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fef280097f0 tx=0x7fef2801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef30ff9640 1 -- 172.21.15.39:0/1504477456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28004230 con 0x7fef34151500 2024-09-13T16:55:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef30ff9640 1 -- 172.21.15.39:0/1504477456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef28004390 con 0x7fef34151500 2024-09-13T16:55:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef34151ee0 con 0x7fef34151500 2024-09-13T16:55:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef30ff9640 1 -- 172.21.15.39:0/1504477456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28003860 con 0x7fef34151500 2024-09-13T16:55:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.453+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef341524a0 con 0x7fef34151500 2024-09-13T16:55:55.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.457+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef340c4280 con 0x7fef34151500 2024-09-13T16:55:55.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.457+0000 7fef30ff9640 1 -- 172.21.15.39:0/1504477456 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef280039c0 con 0x7fef34151500 2024-09-13T16:55:55.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.457+0000 7fef30ff9640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef20066690 0x7fef20068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:55.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.457+0000 7fef30ff9640 1 -- 172.21.15.39:0/1504477456 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7fef2811f0b0 con 0x7fef34151500 2024-09-13T16:55:55.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.457+0000 7fef33fff640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef20066690 0x7fef20068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:55.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.457+0000 7fef33fff640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef20066690 0x7fef20068b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef340aa730 tx=0x7fef24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:55.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.461+0000 7fef30ff9640 1 -- 172.21.15.39:0/1504477456 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fef280ebe20 con 0x7fef34151500 2024-09-13T16:55:55.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef34157320 con 0x7fef34151500 2024-09-13T16:55:55.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef30ff9640 1 -- 172.21.15.39:0/1504477456 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5930 (secure 0 0 0) 0x7fef280c4090 con 0x7fef34151500 2024-09-13T16:55:55.662 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:55.662 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5539,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:55.662 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 60 2024-09-13T16:55:55.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef20066690 msgr2=0x7fef20068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:55.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef20066690 0x7fef20068b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef340aa730 tx=0x7fef24009040 comp rx=0 tx=0).stop 2024-09-13T16:55:55.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34151500 msgr2=0x7fef341ec910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:55.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34151500 0x7fef341ec910 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fef280097f0 tx=0x7fef2801ac20 comp rx=0 tx=0).stop 2024-09-13T16:55:55.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 shutdown_connections 2024-09-13T16:55:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fef20066690 0x7fef20068b50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef3414e750 0x7fef341ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef34151500 0x7fef341ec910 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 --2- 172.21.15.39:0/1504477456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef340c7800 0x7fef340c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 >> 172.21.15.39:0/1504477456 conn(0x7fef340ab530 msgr2=0x7fef340c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 shutdown_connections 2024-09-13T16:55:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:55.661+0000 7fef3a970640 1 -- 172.21.15.39:0/1504477456 wait complete. 2024-09-13T16:55:55.680 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:55:56.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:55:56.775 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:55:57.681 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:55:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f387120c640 1 Processor -- start 2024-09-13T16:55:57.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f387120c640 1 -- start start 2024-09-13T16:55:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f387120c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 0x7f386c0662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f387120c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f386c0c2ee0 0x7f386c0c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f387120c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c06ddf0 con 0x7f386c0c6270 2024-09-13T16:55:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f387120c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0c4710 con 0x7f386c0c2ee0 2024-09-13T16:55:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f387120c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0c49d0 con 0x7f386c065ee0 2024-09-13T16:55:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f386a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f386c0c2ee0 0x7f386c0c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f386b59e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f386c0c6270 0x7f386c0bfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54470/0 (socket says 172.21.15.39:54470) 2024-09-13T16:55:57.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f386b59e640 1 -- 172.21.15.39:0/1523018617 learned_addr learned my addr 172.21.15.39:0/1523018617 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:55:57.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f386ad9d640 1 --2- 172.21.15.39:0/1523018617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 0x7f386c0662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:57.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f386a59c640 1 -- 172.21.15.39:0/1523018617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 msgr2=0x7f386c0662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:55:57.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f386a59c640 1 --2- 172.21.15.39:0/1523018617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 0x7f386c0662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:57.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f386a59c640 1 -- 172.21.15.39:0/1523018617 >> v1:172.21.15.39:6789/0 conn(0x7f386c0c6270 legacy=0x7f386c0bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:57.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f3869d9b640 1 -- 172.21.15.39:0/1523018617 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301194686 0 0) 0x7f386c06ddf0 con 0x7f386c0c6270 2024-09-13T16:55:57.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f386a59c640 1 -- 172.21.15.39:0/1523018617 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c154880 con 0x7f386c0c2ee0 2024-09-13T16:55:57.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.817+0000 7f386ad9d640 1 --2- 172.21.15.39:0/1523018617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 0x7f386c0662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:55:57.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f386a59c640 1 --2- 172.21.15.39:0/1523018617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c2ee0 0x7f386c0c2220 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f38600097c0 tx=0x7f386001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a68cdd703a1445c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:57.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f3869d9b640 1 -- 172.21.15.39:0/1523018617 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600ab600 con 0x7f386c0c2ee0 2024-09-13T16:55:57.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f3869d9b640 1 -- 172.21.15.39:0/1523018617 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38600a7050 con 0x7f386c0c2ee0 2024-09-13T16:55:57.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f3869d9b640 1 -- 172.21.15.39:0/1523018617 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600ab980 con 0x7f386c0c2ee0 2024-09-13T16:55:57.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f387120c640 1 -- 172.21.15.39:0/1523018617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c2ee0 msgr2=0x7f386c0c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:57.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f387120c640 1 --2- 172.21.15.39:0/1523018617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c2ee0 0x7f386c0c2220 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f38600097c0 tx=0x7f386001c410 comp rx=0 tx=0).stop 2024-09-13T16:55:57.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f387120c640 1 -- 172.21.15.39:0/1523018617 shutdown_connections 2024-09-13T16:55:57.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f387120c640 1 --2- 172.21.15.39:0/1523018617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c2ee0 0x7f386c0c2220 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:57.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f387120c640 1 --2- 172.21.15.39:0/1523018617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 0x7f386c0662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:57.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f387120c640 1 -- 172.21.15.39:0/1523018617 >> 172.21.15.39:0/1523018617 conn(0x7f386c0c3310 msgr2=0x7f386c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:57.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f387120c640 1 -- 172.21.15.39:0/1523018617 shutdown_connections 2024-09-13T16:55:57.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.821+0000 7f387120c640 1 -- 172.21.15.39:0/1523018617 wait complete. 2024-09-13T16:55:57.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 Processor -- start 2024-09-13T16:55:57.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 -- start start 2024-09-13T16:55:57.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 0x7f386c1ef770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:57.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f386c0c2ee0 0x7f386c0a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:57.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c6270 0x7f386c0aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c14cb30 con 0x7f386c0c6270 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c14c660 con 0x7f386c0c2ee0 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c14c890 con 0x7f386c065ee0 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c6270 0x7f386c0aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c6270 0x7f386c0aaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45966/0 (socket says 172.21.15.39:45966) 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386b59e640 1 -- 172.21.15.39:0/1160721235 learned_addr learned my addr 172.21.15.39:0/1160721235 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386ad9d640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 0x7f386c1ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386a59c640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f386c0c2ee0 0x7f386c0a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386b59e640 1 -- 172.21.15.39:0/1160721235 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 msgr2=0x7f386c1ef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386b59e640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 0x7f386c1ef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:57.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386b59e640 1 -- 172.21.15.39:0/1160721235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f386c0c2ee0 msgr2=0x7f386c0a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:57.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386b59e640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f386c0c2ee0 0x7f386c0a9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:57.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386b59e640 1 -- 172.21.15.39:0/1160721235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860009420 con 0x7f386c0c6270 2024-09-13T16:55:57.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f386b59e640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c6270 0x7f386c0aaa60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f386400c780 tx=0x7f386400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:57.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f38537fe640 1 -- 172.21.15.39:0/1160721235 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386400ce30 con 0x7f386c0c6270 2024-09-13T16:55:57.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f38537fe640 1 -- 172.21.15.39:0/1160721235 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386400a640 con 0x7f386c0c6270 2024-09-13T16:55:57.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c1f00c0 con 0x7f386c0c6270 2024-09-13T16:55:57.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f38537fe640 1 -- 172.21.15.39:0/1160721235 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386400a9f0 con 0x7f386c0c6270 2024-09-13T16:55:57.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.825+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c1efe80 con 0x7f386c0c6270 2024-09-13T16:55:57.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.829+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c1f02b0 con 0x7f386c0c6270 2024-09-13T16:55:57.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.829+0000 7f38537fe640 1 -- 172.21.15.39:0/1160721235 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f386400ab50 con 0x7f386c0c6270 2024-09-13T16:55:57.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.829+0000 7f38537fe640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:55:57.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.829+0000 7f38537fe640 1 -- 172.21.15.39:0/1160721235 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7f3864083a00 con 0x7f386c0c6270 2024-09-13T16:55:57.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.829+0000 7f386ad9d640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:55:57.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.829+0000 7f386ad9d640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f386c150980 tx=0x7f385c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:55:57.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:57.833+0000 7f38537fe640 1 -- 172.21.15.39:0/1160721235 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3864050800 con 0x7f386c0c6270 2024-09-13T16:55:58.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.029+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f386c0662c0 con 0x7f386c0c6270 2024-09-13T16:55:58.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.029+0000 7f38537fe640 1 -- 172.21.15.39:0/1160721235 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f3864018020 con 0x7f386c0c6270 2024-09-13T16:55:58.032 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:55:58.033 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5539,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:55:58.033 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 60 2024-09-13T16:55:58.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 msgr2=0x7f3858068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f386c150980 tx=0x7f385c008040 comp rx=0 tx=0).stop 2024-09-13T16:55:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c6270 msgr2=0x7f386c0aaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:55:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c6270 0x7f386c0aaa60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f386400c780 tx=0x7f386400cc40 comp rx=0 tx=0).stop 2024-09-13T16:55:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 shutdown_connections 2024-09-13T16:55:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3858066690 0x7f3858068b50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f386c0c2ee0 0x7f386c0a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f386c0c6270 0x7f386c0aaa60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 --2- 172.21.15.39:0/1160721235 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f386c065ee0 0x7f386c1ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:55:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 >> 172.21.15.39:0/1160721235 conn(0x7f386c0c3310 msgr2=0x7f386c0a6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:55:58.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 shutdown_connections 2024-09-13T16:55:58.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:55:58.033+0000 7f387120c640 1 -- 172.21.15.39:0/1160721235 wait complete. 2024-09-13T16:55:58.051 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:56:00.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:00.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9342c2b640 1 Processor -- start 2024-09-13T16:56:00.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9342c2b640 1 -- start start 2024-09-13T16:56:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9342c2b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f933c070e30 0x7f933c0611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9342c2b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f933c1548a0 0x7f933c0c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9342c2b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c154d10 con 0x7f933c151650 2024-09-13T16:56:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9342c2b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c157670 con 0x7f933c070e30 2024-09-13T16:56:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9342c2b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c061880 con 0x7f933c1548a0 2024-09-13T16:56:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9333fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f933c070e30 0x7f933c0611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f93409c7640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f933c151650 0x7f933c157a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50622/0 (socket says 172.21.15.39:50622) 2024-09-13T16:56:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9333fff640 1 --2- 172.21.15.39:0/2945952894 >> v2:172.21.15.39:3300/0 conn(0x7f933c070e30 0x7f933c0611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46154/0 (socket says 172.21.15.39:46154) 2024-09-13T16:56:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f93409c7640 1 -- 172.21.15.39:0/2945952894 learned_addr learned my addr 172.21.15.39:0/2945952894 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:56:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f93411c8640 1 --2- 172.21.15.39:0/2945952894 >> v2:172.21.15.84:3300/0 conn(0x7f933c1548a0 0x7f933c0c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9333fff640 1 -- 172.21.15.39:0/2945952894 >> v2:172.21.15.84:3300/0 conn(0x7f933c1548a0 msgr2=0x7f933c0c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9333fff640 1 --2- 172.21.15.39:0/2945952894 >> v2:172.21.15.84:3300/0 conn(0x7f933c1548a0 0x7f933c0c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9333fff640 1 -- 172.21.15.39:0/2945952894 >> v1:172.21.15.39:6789/0 conn(0x7f933c151650 legacy=0x7f933c157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.185+0000 7f9333fff640 1 -- 172.21.15.39:0/2945952894 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c0abc00 con 0x7f933c070e30 2024-09-13T16:56:00.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9333fff640 1 --2- 172.21.15.39:0/2945952894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 0x7f933c0611c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f933c06e630 tx=0x7f932c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cff35655a35eac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:00.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f93337fe640 1 -- 172.21.15.39:0/2945952894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0ab600 con 0x7f933c070e30 2024-09-13T16:56:00.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f93337fe640 1 -- 172.21.15.39:0/2945952894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c0a7050 con 0x7f933c070e30 2024-09-13T16:56:00.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f93337fe640 1 -- 172.21.15.39:0/2945952894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0ab980 con 0x7f933c070e30 2024-09-13T16:56:00.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 -- 172.21.15.39:0/2945952894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 msgr2=0x7f933c0611c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:00.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 --2- 172.21.15.39:0/2945952894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 0x7f933c0611c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f933c06e630 tx=0x7f932c01c3f0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 -- 172.21.15.39:0/2945952894 shutdown_connections 2024-09-13T16:56:00.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 --2- 172.21.15.39:0/2945952894 >> v2:172.21.15.84:3300/0 conn(0x7f933c1548a0 0x7f933c0c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 --2- 172.21.15.39:0/2945952894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 0x7f933c0611c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 -- 172.21.15.39:0/2945952894 >> 172.21.15.39:0/2945952894 conn(0x7f933c0c2a50 msgr2=0x7f933c0c2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:00.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 -- 172.21.15.39:0/2945952894 shutdown_connections 2024-09-13T16:56:00.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 -- 172.21.15.39:0/2945952894 wait complete. 2024-09-13T16:56:00.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 Processor -- start 2024-09-13T16:56:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 -- start start 2024-09-13T16:56:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 0x7f933c155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f933c151650 0x7f933c1ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c1548a0 0x7f933c1ecb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0ab700 con 0x7f933c070e30 2024-09-13T16:56:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0c5700 con 0x7f933c1548a0 2024-09-13T16:56:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.189+0000 7f9342c2b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0aadd0 con 0x7f933c151650 2024-09-13T16:56:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93409c7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 0x7f933c155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93409c7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 0x7f933c155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46158/0 (socket says 172.21.15.39:46158) 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93409c7640 1 -- 172.21.15.39:0/2857446090 learned_addr learned my addr 172.21.15.39:0/2857446090 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93411c8640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c1548a0 0x7f933c1ecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f9333fff640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f933c151650 0x7f933c1ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93409c7640 1 -- 172.21.15.39:0/2857446090 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f933c151650 msgr2=0x7f933c1ec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93409c7640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f933c151650 0x7f933c1ec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93409c7640 1 -- 172.21.15.39:0/2857446090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c1548a0 msgr2=0x7f933c1ecb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93409c7640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c1548a0 0x7f933c1ecb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93409c7640 1 -- 172.21.15.39:0/2857446090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c009420 con 0x7f933c070e30 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93409c7640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 0x7f933c155450 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f933800c400 tx=0x7f933800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93317fa640 1 -- 172.21.15.39:0/2857446090 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338005cf0 con 0x7f933c070e30 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93317fa640 1 -- 172.21.15.39:0/2857446090 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338005e50 con 0x7f933c070e30 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c0ab0b0 con 0x7f933c070e30 2024-09-13T16:56:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93317fa640 1 -- 172.21.15.39:0/2857446090 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93380072d0 con 0x7f933c070e30 2024-09-13T16:56:00.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c155990 con 0x7f933c070e30 2024-09-13T16:56:00.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c0c1960 con 0x7f933c070e30 2024-09-13T16:56:00.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93317fa640 1 -- 172.21.15.39:0/2857446090 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9338007430 con 0x7f933c070e30 2024-09-13T16:56:00.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93317fa640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93280666c0 0x7f9328068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:00.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f93317fa640 1 -- 172.21.15.39:0/2857446090 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7f9338083bd0 con 0x7f933c070e30 2024-09-13T16:56:00.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f9333fff640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93280666c0 0x7f9328068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:00.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.193+0000 7f9333fff640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93280666c0 0x7f9328068b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f932c0ad000 tx=0x7f932c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:00.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.197+0000 7f93317fa640 1 -- 172.21.15.39:0/2857446090 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f93380509d0 con 0x7f933c070e30 2024-09-13T16:56:00.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.401+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f933c0c2470 con 0x7f933c070e30 2024-09-13T16:56:00.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.401+0000 7f93317fa640 1 -- 172.21.15.39:0/2857446090 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f9338053ff0 con 0x7f933c070e30 2024-09-13T16:56:00.405 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:00.405 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5539,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:00.405 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 60 2024-09-13T16:56:00.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93280666c0 msgr2=0x7f9328068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:00.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93280666c0 0x7f9328068b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f932c0ad000 tx=0x7f932c008040 comp rx=0 tx=0).stop 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 msgr2=0x7f933c155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 0x7f933c155450 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f933800c400 tx=0x7f933800c8c0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 shutdown_connections 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f93280666c0 0x7f9328068b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f933c1548a0 0x7f933c1ecb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f933c151650 0x7f933c1ec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 --2- 172.21.15.39:0/2857446090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f933c070e30 0x7f933c155450 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 >> 172.21.15.39:0/2857446090 conn(0x7f933c0c2a50 msgr2=0x7f933c15df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 shutdown_connections 2024-09-13T16:56:00.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:00.405+0000 7f9342c2b640 1 -- 172.21.15.39:0/2857446090 wait complete. 2024-09-13T16:56:00.424 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:56:02.425 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:02.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f331cb640 1 Processor -- start 2024-09-13T16:56:02.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f331cb640 1 -- start start 2024-09-13T16:56:02.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f331cb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0aabc0 0x7f2f2c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:02.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f331cb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2f2c0c40b0 0x7f2f2c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f331cb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c152a80 con 0x7f2f2c06c870 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f331cb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2c0c30f0 con 0x7f2f2c0c40b0 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f331cb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c151e00 con 0x7f2f2c0aabc0 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f31768640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2f2c0c40b0 0x7f2f2c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f31768640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2f2c0c40b0 0x7f2f2c15c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46166/0 (socket says 172.21.15.39:46166) 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f31768640 1 -- 172.21.15.39:0/3628847644 learned_addr learned my addr 172.21.15.39:0/3628847644 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f30f67640 1 --2- 172.21.15.39:0/3628847644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0aabc0 0x7f2f2c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f31768640 1 -- 172.21.15.39:0/3628847644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0aabc0 msgr2=0x7f2f2c0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f31768640 1 --2- 172.21.15.39:0/3628847644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0aabc0 0x7f2f2c0aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f31768640 1 -- 172.21.15.39:0/3628847644 >> v1:172.21.15.84:6789/0 conn(0x7f2f2c06c870 legacy=0x7f2f2c0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f31768640 1 -- 172.21.15.39:0/3628847644 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c1520e0 con 0x7f2f2c0c40b0 2024-09-13T16:56:02.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.557+0000 7f2f30f67640 1 --2- 172.21.15.39:0/3628847644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0aabc0 0x7f2f2c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:56:02.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f31768640 1 --2- 172.21.15.39:0/3628847644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c0c40b0 0x7f2f2c15c470 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009e50 tx=0x7f2f2401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96089b644603bdfe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:02.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f2b7fe640 1 -- 172.21.15.39:0/3628847644 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f240a5840 con 0x7f2f2c0c40b0 2024-09-13T16:56:02.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f2b7fe640 1 -- 172.21.15.39:0/3628847644 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f240a59a0 con 0x7f2f2c0c40b0 2024-09-13T16:56:02.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f2b7fe640 1 -- 172.21.15.39:0/3628847644 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f240a5d30 con 0x7f2f2c0c40b0 2024-09-13T16:56:02.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 -- 172.21.15.39:0/3628847644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c0c40b0 msgr2=0x7f2f2c15c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:02.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 --2- 172.21.15.39:0/3628847644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c0c40b0 0x7f2f2c15c470 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009e50 tx=0x7f2f2401c9c0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 -- 172.21.15.39:0/3628847644 shutdown_connections 2024-09-13T16:56:02.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 --2- 172.21.15.39:0/3628847644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c0c40b0 0x7f2f2c15c470 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 --2- 172.21.15.39:0/3628847644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0aabc0 0x7f2f2c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 -- 172.21.15.39:0/3628847644 >> 172.21.15.39:0/3628847644 conn(0x7f2f2c069e10 msgr2=0x7f2f2c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:02.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 -- 172.21.15.39:0/3628847644 shutdown_connections 2024-09-13T16:56:02.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 -- 172.21.15.39:0/3628847644 wait complete. 2024-09-13T16:56:02.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 Processor -- start 2024-09-13T16:56:02.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 -- start start 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c06c870 0x7f2f2c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f2c0aabc0 0x7f2f2c149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0c40b0 0x7f2f2c14a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0a7ad0 con 0x7f2f2c06c870 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c149950 con 0x7f2f2c0aabc0 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f331cb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0a7ce0 con 0x7f2f2c0c40b0 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f30f67640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c06c870 0x7f2f2c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f30f67640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c06c870 0x7f2f2c150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46170/0 (socket says 172.21.15.39:46170) 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f30f67640 1 -- 172.21.15.39:0/45958371 learned_addr learned my addr 172.21.15.39:0/45958371 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f2bfff640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f2c0aabc0 0x7f2f2c149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f31768640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0c40b0 0x7f2f2c14a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f30f67640 1 -- 172.21.15.39:0/45958371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0c40b0 msgr2=0x7f2f2c14a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f30f67640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0c40b0 0x7f2f2c14a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f30f67640 1 -- 172.21.15.39:0/45958371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f2c0aabc0 msgr2=0x7f2f2c149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f30f67640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f2c0aabc0 0x7f2f2c149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f30f67640 1 -- 172.21.15.39:0/45958371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24009b30 con 0x7f2f2c06c870 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f2bfff640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f2c0aabc0 0x7f2f2c149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.561+0000 7f2f30f67640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c06c870 0x7f2f2c150d40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c7c0 tx=0x7f2f1400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f297fa640 1 -- 172.21.15.39:0/45958371 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14007a30 con 0x7f2f2c06c870 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f297fa640 1 -- 172.21.15.39:0/45958371 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14007b90 con 0x7f2f2c06c870 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c0a7fc0 con 0x7f2f2c06c870 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f297fa640 1 -- 172.21.15.39:0/45958371 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400ce70 con 0x7f2f2c06c870 2024-09-13T16:56:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c151280 con 0x7f2f2c06c870 2024-09-13T16:56:02.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c0aafa0 con 0x7f2f2c06c870 2024-09-13T16:56:02.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f297fa640 1 -- 172.21.15.39:0/45958371 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f140113f0 con 0x7f2f2c06c870 2024-09-13T16:56:02.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f297fa640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f0c066690 0x7f2f0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:02.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f297fa640 1 -- 172.21.15.39:0/45958371 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 7411+0+0 (secure 0 0 0) 0x7f2f14084c50 con 0x7f2f2c06c870 2024-09-13T16:56:02.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f2bfff640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f0c066690 0x7f2f0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:02.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.565+0000 7f2f2bfff640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f0c066690 0x7f2f0c068b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0c51a0 tx=0x7f2f1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:02.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.569+0000 7f2f297fa640 1 -- 172.21.15.39:0/45958371 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2f1408a020 con 0x7f2f2c06c870 2024-09-13T16:56:02.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.765+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f2c06c650 con 0x7f2f2c06c870 2024-09-13T16:56:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.765+0000 7f2f297fa640 1 -- 172.21.15.39:0/45958371 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f2f140519a0 con 0x7f2f2c06c870 2024-09-13T16:56:02.769 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:02.769 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5539,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:54:12.632566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5703":{"gid":5703,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":16,"addr":"172.21.15.39:6835/4094657757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4094657757},{"type":"v1","addr":"172.21.15.39:6835","nonce":4094657757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:02.769 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 60 2024-09-13T16:56:02.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f0c066690 msgr2=0x7f2f0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:02.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f0c066690 0x7f2f0c068b50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0c51a0 tx=0x7f2f1c009040 comp rx=0 tx=0).stop 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c06c870 msgr2=0x7f2f2c150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c06c870 0x7f2f2c150d40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c7c0 tx=0x7f2f1400cc80 comp rx=0 tx=0).stop 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 shutdown_connections 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f0c066690 0x7f2f0c068b50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f2c0aabc0 0x7f2f2c149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f2c0c40b0 0x7f2f2c14a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 --2- 172.21.15.39:0/45958371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2c06c870 0x7f2f2c150d40 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 >> 172.21.15.39:0/45958371 conn(0x7f2f2c069e10 msgr2=0x7f2f2c1557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 shutdown_connections 2024-09-13T16:56:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:02.769+0000 7f2f331cb640 1 -- 172.21.15.39:0/45958371 wait complete. 2024-09-13T16:56:02.788 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:56:04.789 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:04.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.917+0000 7efc346d2640 1 Processor -- start 2024-09-13T16:56:04.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.917+0000 7efc346d2640 1 -- start start 2024-09-13T16:56:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc346d2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efc2c14f340 0x7efc2c071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc346d2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc2c0c7180 0x7efc2c06f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc346d2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2c06d750 con 0x7efc2c0c26c0 2024-09-13T16:56:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc346d2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2c06e410 con 0x7efc2c0c7180 2024-09-13T16:56:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc346d2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2c0a7000 con 0x7efc2c14f340 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc32c6f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7efc2c0c26c0 0x7efc2c06f590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50628/0 (socket says 172.21.15.39:50628) 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc32c6f640 1 -- 172.21.15.39:0/3716534640 learned_addr learned my addr 172.21.15.39:0/3716534640 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc31c6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc2c0c7180 0x7efc2c06f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc3246e640 1 --2- 172.21.15.39:0/3716534640 >> v2:172.21.15.84:3300/0 conn(0x7efc2c14f340 0x7efc2c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc3146c640 1 -- 172.21.15.39:0/3716534640 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308006257 0 0) 0x7efc2c06d750 con 0x7efc2c0c26c0 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc3146c640 1 -- 172.21.15.39:0/3716534640 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc18003530 con 0x7efc2c0c26c0 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc31c6d640 1 -- 172.21.15.39:0/3716534640 >> v2:172.21.15.84:3300/0 conn(0x7efc2c14f340 msgr2=0x7efc2c071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc31c6d640 1 --2- 172.21.15.39:0/3716534640 >> v2:172.21.15.84:3300/0 conn(0x7efc2c14f340 0x7efc2c071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc31c6d640 1 -- 172.21.15.39:0/3716534640 >> v1:172.21.15.39:6789/0 conn(0x7efc2c0c26c0 legacy=0x7efc2c06f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc31c6d640 1 -- 172.21.15.39:0/3716534640 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c155ce0 con 0x7efc2c0c7180 2024-09-13T16:56:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.921+0000 7efc31c6d640 1 --2- 172.21.15.39:0/3716534640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c0c7180 0x7efc2c06f050 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0c6ac0 tx=0x7efc1c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1189320a49a251 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc3146c640 1 -- 172.21.15.39:0/3716534640 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0ab600 con 0x7efc2c0c7180 2024-09-13T16:56:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc3146c640 1 -- 172.21.15.39:0/3716534640 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c0a7050 con 0x7efc2c0c7180 2024-09-13T16:56:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc3146c640 1 -- 172.21.15.39:0/3716534640 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0ab980 con 0x7efc2c0c7180 2024-09-13T16:56:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- 172.21.15.39:0/3716534640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c0c7180 msgr2=0x7efc2c06f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 --2- 172.21.15.39:0/3716534640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c0c7180 0x7efc2c06f050 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0c6ac0 tx=0x7efc1c01c3f0 comp rx=0 tx=0).stop 2024-09-13T16:56:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- 172.21.15.39:0/3716534640 shutdown_connections 2024-09-13T16:56:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 --2- 172.21.15.39:0/3716534640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c0c7180 0x7efc2c06f050 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 --2- 172.21.15.39:0/3716534640 >> v2:172.21.15.84:3300/0 conn(0x7efc2c14f340 0x7efc2c071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- 172.21.15.39:0/3716534640 >> 172.21.15.39:0/3716534640 conn(0x7efc2c0a96b0 msgr2=0x7efc2c0a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- 172.21.15.39:0/3716534640 shutdown_connections 2024-09-13T16:56:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- 172.21.15.39:0/3716534640 wait complete. 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 Processor -- start 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- start start 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2c0c26c0 0x7efc2c0c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2c0c7180 0x7efc2c0a8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c14f340 0x7efc2c1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c154a50 con 0x7efc2c14f340 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c152fd0 con 0x7efc2c0c26c0 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c0a8d50 con 0x7efc2c0c7180 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc32c6f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c14f340 0x7efc2c1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc32c6f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c14f340 0x7efc2c1510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46176/0 (socket says 172.21.15.39:46176) 2024-09-13T16:56:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc32c6f640 1 -- 172.21.15.39:0/3516059874 learned_addr learned my addr 172.21.15.39:0/3516059874 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc31c6d640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2c0c7180 0x7efc2c0a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc3246e640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2c0c26c0 0x7efc2c0c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc32c6f640 1 -- 172.21.15.39:0/3516059874 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2c0c7180 msgr2=0x7efc2c0a8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc32c6f640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2c0c7180 0x7efc2c0a8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc32c6f640 1 -- 172.21.15.39:0/3516059874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2c0c26c0 msgr2=0x7efc2c0c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc32c6f640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2c0c26c0 0x7efc2c0c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc32c6f640 1 -- 172.21.15.39:0/3516059874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c009420 con 0x7efc2c14f340 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc32c6f640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c14f340 0x7efc2c1510d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c3e0 tx=0x7efc2400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc22ffd640 1 -- 172.21.15.39:0/3516059874 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24005cf0 con 0x7efc2c14f340 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc22ffd640 1 -- 172.21.15.39:0/3516059874 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc24005e50 con 0x7efc2c14f340 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2c153270 con 0x7efc2c14f340 2024-09-13T16:56:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc22ffd640 1 -- 172.21.15.39:0/3516059874 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc240072d0 con 0x7efc2c14f340 2024-09-13T16:56:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.925+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2c0c4cd0 con 0x7efc2c14f340 2024-09-13T16:56:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.929+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2c154bc0 con 0x7efc2c14f340 2024-09-13T16:56:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.929+0000 7efc22ffd640 1 -- 172.21.15.39:0/3516059874 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc24007430 con 0x7efc2c14f340 2024-09-13T16:56:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.929+0000 7efc22ffd640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc10066690 0x7efc10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.929+0000 7efc22ffd640 1 -- 172.21.15.39:0/3516059874 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7efc24083d20 con 0x7efc2c14f340 2024-09-13T16:56:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.929+0000 7efc3246e640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc10066690 0x7efc10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.929+0000 7efc3246e640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc10066690 0x7efc10068b50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efc2c15b950 tx=0x7efc2800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:04.933+0000 7efc22ffd640 1 -- 172.21.15.39:0/3516059874 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efc24050ad0 con 0x7efc2c14f340 2024-09-13T16:56:05.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.133+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc2c071530 con 0x7efc2c14f340 2024-09-13T16:56:05.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.133+0000 7efc22ffd640 1 -- 172.21.15.39:0/3516059874 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5172 (secure 0 0 0) 0x7efc240540f0 con 0x7efc2c14f340 2024-09-13T16:56:05.136 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:05.136 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:04.631875+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:05.136 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 61 2024-09-13T16:56:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc10066690 msgr2=0x7efc10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc10066690 0x7efc10068b50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efc2c15b950 tx=0x7efc2800b040 comp rx=0 tx=0).stop 2024-09-13T16:56:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c14f340 msgr2=0x7efc2c1510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c14f340 0x7efc2c1510d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c3e0 tx=0x7efc2400c8a0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 shutdown_connections 2024-09-13T16:56:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc10066690 0x7efc10068b50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc2c14f340 0x7efc2c1510d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc2c0c7180 0x7efc2c0a8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 --2- 172.21.15.39:0/3516059874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc2c0c26c0 0x7efc2c0c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 >> 172.21.15.39:0/3516059874 conn(0x7efc2c0a96b0 msgr2=0x7efc2c0a9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 shutdown_connections 2024-09-13T16:56:05.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.137+0000 7efc346d2640 1 -- 172.21.15.39:0/3516059874 wait complete. 2024-09-13T16:56:05.155 INFO:tasks.mds_thrash.fs.[a]:mds.a down, removed from mdsmap 2024-09-13T16:56:05.156 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T16:56:05.156 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:05.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.281+0000 7f94c206e640 1 Processor -- start 2024-09-13T16:56:05.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.281+0000 7f94c206e640 1 -- start start 2024-09-13T16:56:05.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94c206e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94bc0c20d0 0x7f94bc0c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:05.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94c206e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc0c1790 0x7f94bc157800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:05.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94c206e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc157370 con 0x7f94bc06d370 2024-09-13T16:56:05.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94c206e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc0c04a0 con 0x7f94bc0c20d0 2024-09-13T16:56:05.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94c206e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc153df0 con 0x7f94bc0c1790 2024-09-13T16:56:05.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94bb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94bc0c20d0 0x7f94bc0c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:05.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc0c1790 0x7f94bc157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:05.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94bbfff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f94bc06d370 0x7f94bc157d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:33478/0 (socket says 172.21.15.39:33478) 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94bb7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94bc0c20d0 0x7f94bc0c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:38706/0 (socket says 172.21.15.39:38706) 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94bbfff640 1 -- 172.21.15.39:0/4273648524 learned_addr learned my addr 172.21.15.39:0/4273648524 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94bb7fe640 1 -- 172.21.15.39:0/4273648524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc0c1790 msgr2=0x7f94bc157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94bb7fe640 1 --2- 172.21.15.39:0/4273648524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc0c1790 0x7f94bc157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94bb7fe640 1 -- 172.21.15.39:0/4273648524 >> v1:172.21.15.84:6789/0 conn(0x7f94bc06d370 legacy=0x7f94bc157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94bb7fe640 1 -- 172.21.15.39:0/4273648524 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc06c700 con 0x7f94bc0c20d0 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94bb7fe640 1 --2- 172.21.15.39:0/4273648524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 0x7f94bc0c24b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f94a40097c0 tx=0x7f94a401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e0edf0aac702f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94baffd640 1 -- 172.21.15.39:0/4273648524 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40ab670 con 0x7f94bc0c20d0 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94baffd640 1 -- 172.21.15.39:0/4273648524 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a40ab7d0 con 0x7f94bc0c20d0 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.285+0000 7f94baffd640 1 -- 172.21.15.39:0/4273648524 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40abb60 con 0x7f94bc0c20d0 2024-09-13T16:56:05.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- 172.21.15.39:0/4273648524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 msgr2=0x7f94bc0c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:05.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 --2- 172.21.15.39:0/4273648524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 0x7f94bc0c24b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f94a40097c0 tx=0x7f94a401c470 comp rx=0 tx=0).stop 2024-09-13T16:56:05.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- 172.21.15.39:0/4273648524 shutdown_connections 2024-09-13T16:56:05.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 --2- 172.21.15.39:0/4273648524 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc0c1790 0x7f94bc157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 --2- 172.21.15.39:0/4273648524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 0x7f94bc0c24b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- 172.21.15.39:0/4273648524 >> 172.21.15.39:0/4273648524 conn(0x7f94bc069e10 msgr2=0x7f94bc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:05.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- 172.21.15.39:0/4273648524 shutdown_connections 2024-09-13T16:56:05.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- 172.21.15.39:0/4273648524 wait complete. 2024-09-13T16:56:05.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 Processor -- start 2024-09-13T16:56:05.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- start start 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc06d370 0x7f94bc1f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94bc0c1790 0x7f94bc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 0x7f94bc1498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc072320 con 0x7f94bc0c1790 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1f08c0 con 0x7f94bc0c20d0 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc14ab30 con 0x7f94bc06d370 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94bc0c1790 0x7f94bc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94bc0c1790 0x7f94bc15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46192/0 (socket says 172.21.15.39:46192) 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b3fff640 1 -- 172.21.15.39:0/3942056360 learned_addr learned my addr 172.21.15.39:0/3942056360 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:05.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94bbfff640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 0x7f94bc1498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94bb7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc06d370 0x7f94bc1f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b3fff640 1 -- 172.21.15.39:0/3942056360 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc06d370 msgr2=0x7f94bc1f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b3fff640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc06d370 0x7f94bc1f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b3fff640 1 -- 172.21.15.39:0/3942056360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 msgr2=0x7f94bc1498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b3fff640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 0x7f94bc1498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b3fff640 1 -- 172.21.15.39:0/3942056360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a4009420 con 0x7f94bc0c1790 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94bbfff640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 0x7f94bc1498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94bb7fe640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc06d370 0x7f94bc1f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b3fff640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94bc0c1790 0x7f94bc15ced0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00e900 tx=0x7f94ac00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:05.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b8ff9640 1 -- 172.21.15.39:0/3942056360 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac009690 con 0x7f94bc0c1790 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b8ff9640 1 -- 172.21.15.39:0/3942056360 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac00ca00 con 0x7f94bc0c1790 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc0c4680 con 0x7f94bc0c1790 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94b8ff9640 1 -- 172.21.15.39:0/3942056360 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00cd90 con 0x7f94bc0c1790 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.289+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc0c4c40 con 0x7f94bc0c1790 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.293+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc0c24b0 con 0x7f94bc0c1790 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.293+0000 7f94b8ff9640 1 -- 172.21.15.39:0/3942056360 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94ac013470 con 0x7f94bc0c1790 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.293+0000 7f94b8ff9640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f949c0666c0 0x7f949c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.293+0000 7f94b8ff9640 1 -- 172.21.15.39:0/3942056360 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f94ac084900 con 0x7f94bc0c1790 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.293+0000 7f94bb7fe640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f949c0666c0 0x7f949c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:05.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.293+0000 7f94bb7fe640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f949c0666c0 0x7f949c068b80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94a40af000 tx=0x7f94a40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:05.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.297+0000 7f94b8ff9640 1 -- 172.21.15.39:0/3942056360 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f94ac0516b0 con 0x7f94bc0c1790 2024-09-13T16:56:05.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.497+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94bc072570 con 0x7f94bc0c1790 2024-09-13T16:56:05.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.497+0000 7f94b8ff9640 1 -- 172.21.15.39:0/3942056360 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5172 (secure 0 0 0) 0x7f94ac054cd0 con 0x7f94bc0c1790 2024-09-13T16:56:05.499 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:05.499 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:04.631875+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:05.500 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 61 2024-09-13T16:56:05.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f949c0666c0 msgr2=0x7f949c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:05.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f949c0666c0 0x7f949c068b80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f94a40af000 tx=0x7f94a40a7000 comp rx=0 tx=0).stop 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94bc0c1790 msgr2=0x7f94bc15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94bc0c1790 0x7f94bc15ced0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00e900 tx=0x7f94ac00edc0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 shutdown_connections 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f949c0666c0 0x7f949c068b80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94bc0c20d0 0x7f94bc1498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94bc0c1790 0x7f94bc15ced0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 --2- 172.21.15.39:0/3942056360 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94bc06d370 0x7f94bc1f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 >> 172.21.15.39:0/3942056360 conn(0x7f94bc069e10 msgr2=0x7f94bc0ab270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 shutdown_connections 2024-09-13T16:56:05.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:05.501+0000 7f94c206e640 1 -- 172.21.15.39:0/3942056360 wait complete. 2024-09-13T16:56:05.520 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 61, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 61, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5539": { "addr": "172.21.15.84:6835/661906662", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 661906662, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 661906662, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5539, "incarnation": 61, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:56:04.631875+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5539 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 59, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5069": { "addr": "172.21.15.84:6833/2740003107", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 2740003107, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 2740003107, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5069, "incarnation": 56, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 18 } }, "last_failure": 0, "last_failure_osd_epoch": 35, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:55:17.642071+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5069 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:56:07.521 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:07.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fbbeda640 1 Processor -- start 2024-09-13T16:56:07.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fbbeda640 1 -- start start 2024-09-13T16:56:07.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fbbeda640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb406d370 0x7f8fb4157f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:07.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fbbeda640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb4157370 con 0x7f8fb40c1790 2024-09-13T16:56:07.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fbbeda640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb4153df0 con 0x7f8fb40c20d0 2024-09-13T16:56:07.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fbbeda640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40c04a0 con 0x7f8fb406d370 2024-09-13T16:56:07.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fb9475640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8fb40c1790 0x7f8fb415c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50638/0 (socket says 172.21.15.39:50638) 2024-09-13T16:56:07.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fb9475640 1 -- 172.21.15.39:0/3848845178 learned_addr learned my addr 172.21.15.39:0/3848845178 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:56:07.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fba477640 1 --2- 172.21.15.39:0/3848845178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb406d370 0x7f8fb4157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:07.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fb8c74640 1 -- 172.21.15.39:0/3848845178 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074485246 0 0) 0x7f8fb4157370 con 0x7f8fb40c1790 2024-09-13T16:56:07.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fb8c74640 1 -- 172.21.15.39:0/3848845178 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa0003530 con 0x7f8fb40c1790 2024-09-13T16:56:07.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fb8c74640 1 -- 172.21.15.39:0/3848845178 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293769269 0 0) 0x7f8fb4153df0 con 0x7f8fb40c20d0 2024-09-13T16:56:07.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fb8c74640 1 -- 172.21.15.39:0/3848845178 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb4157370 con 0x7f8fb40c20d0 2024-09-13T16:56:07.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fba477640 1 -- 172.21.15.39:0/3848845178 >> v1:172.21.15.84:6789/0 conn(0x7f8fb40c20d0 legacy=0x7f8fb4157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:07.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fba477640 1 -- 172.21.15.39:0/3848845178 >> v1:172.21.15.39:6789/0 conn(0x7f8fb40c1790 legacy=0x7f8fb415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:07.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.657+0000 7f8fba477640 1 -- 172.21.15.39:0/3848845178 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb406c700 con 0x7f8fb406d370 2024-09-13T16:56:07.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fba477640 1 --2- 172.21.15.39:0/3848845178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb406d370 0x7f8fb4157f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009dc0 tx=0x7f8fac01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e8356e4a7a6645 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:07.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb8c74640 1 -- 172.21.15.39:0/3848845178 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0a5a40 con 0x7f8fb406d370 2024-09-13T16:56:07.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb8c74640 1 -- 172.21.15.39:0/3848845178 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac0a5ba0 con 0x7f8fb406d370 2024-09-13T16:56:07.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb8c74640 1 -- 172.21.15.39:0/3848845178 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0ad440 con 0x7f8fb406d370 2024-09-13T16:56:07.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 -- 172.21.15.39:0/3848845178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb406d370 msgr2=0x7f8fb4157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:07.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 --2- 172.21.15.39:0/3848845178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb406d370 0x7f8fb4157f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009dc0 tx=0x7f8fac01c960 comp rx=0 tx=0).stop 2024-09-13T16:56:07.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 -- 172.21.15.39:0/3848845178 shutdown_connections 2024-09-13T16:56:07.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 --2- 172.21.15.39:0/3848845178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb406d370 0x7f8fb4157f10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:07.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 -- 172.21.15.39:0/3848845178 >> 172.21.15.39:0/3848845178 conn(0x7f8fb4069e10 msgr2=0x7f8fb406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:07.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 -- 172.21.15.39:0/3848845178 shutdown_connections 2024-09-13T16:56:07.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 -- 172.21.15.39:0/3848845178 wait complete. 2024-09-13T16:56:07.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 Processor -- start 2024-09-13T16:56:07.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 -- start start 2024-09-13T16:56:07.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fb406d370 0x7f8fb4145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb40c1790 0x7f8fb414ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb40c20d0 0x7f8fb4070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4144250 con 0x7f8fb406d370 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4145990 con 0x7f8fb40c20d0 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fbbeda640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb406f670 con 0x7f8fb40c1790 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb9c76640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fb406d370 0x7f8fb4145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb9c76640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fb406d370 0x7f8fb4145e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46206/0 (socket says 172.21.15.39:46206) 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb9c76640 1 -- 172.21.15.39:0/1543396805 learned_addr learned my addr 172.21.15.39:0/1543396805 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fba477640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb40c20d0 0x7f8fb4070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb9475640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb40c1790 0x7f8fb414ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb9c76640 1 -- 172.21.15.39:0/1543396805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb40c1790 msgr2=0x7f8fb414ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb9c76640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb40c1790 0x7f8fb414ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:07.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb9c76640 1 -- 172.21.15.39:0/1543396805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb40c20d0 msgr2=0x7f8fb4070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:07.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb9c76640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb40c20d0 0x7f8fb4070170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:07.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.661+0000 7f8fb9c76640 1 -- 172.21.15.39:0/1543396805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac009a70 con 0x7f8fb406d370 2024-09-13T16:56:07.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8fb9c76640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fb406d370 0x7f8fb4145e20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400c490 tx=0x7f8fa400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:07.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8faa7fc640 1 -- 172.21.15.39:0/1543396805 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa40052d0 con 0x7f8fb406d370 2024-09-13T16:56:07.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8faa7fc640 1 -- 172.21.15.39:0/1543396805 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa4005430 con 0x7f8fb406d370 2024-09-13T16:56:07.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb4146360 con 0x7f8fb406d370 2024-09-13T16:56:07.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8faa7fc640 1 -- 172.21.15.39:0/1543396805 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4003840 con 0x7f8fb406d370 2024-09-13T16:56:07.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb4145620 con 0x7f8fb406d370 2024-09-13T16:56:07.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb4151310 con 0x7f8fb406d370 2024-09-13T16:56:07.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8faa7fc640 1 -- 172.21.15.39:0/1543396805 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8fa40039a0 con 0x7f8fb406d370 2024-09-13T16:56:07.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8faa7fc640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8f980666c0 0x7f8f98068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:07.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8faa7fc640 1 -- 172.21.15.39:0/1543396805 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f8fa4083db0 con 0x7f8fb406d370 2024-09-13T16:56:07.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8fb9475640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8f980666c0 0x7f8f98068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.665+0000 7f8fb9475640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8f980666c0 0x7f8f98068b80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8fb406fa80 tx=0x7f8fb0007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:07.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.669+0000 7f8faa7fc640 1 -- 172.21.15.39:0/1543396805 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8fa4050b60 con 0x7f8fb406d370 2024-09-13T16:56:07.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.873+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fb4157800 con 0x7f8fb406d370 2024-09-13T16:56:07.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.873+0000 7f8faa7fc640 1 -- 172.21.15.39:0/1543396805 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+5173 (secure 0 0 0) 0x7f8fa4054180 con 0x7f8fb406d370 2024-09-13T16:56:07.875 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:07.875 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:06.883896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:07.876 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 63 2024-09-13T16:56:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8f980666c0 msgr2=0x7f8f98068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8f980666c0 0x7f8f98068b80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8fb406fa80 tx=0x7f8fb0007d40 comp rx=0 tx=0).stop 2024-09-13T16:56:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fb406d370 msgr2=0x7f8fb4145e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fb406d370 0x7f8fb4145e20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400c490 tx=0x7f8fa400c950 comp rx=0 tx=0).stop 2024-09-13T16:56:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 shutdown_connections 2024-09-13T16:56:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8f980666c0 0x7f8f98068b80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fb40c20d0 0x7f8fb4070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8fb40c1790 0x7f8fb414ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 --2- 172.21.15.39:0/1543396805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fb406d370 0x7f8fb4145e20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 >> 172.21.15.39:0/1543396805 conn(0x7f8fb4069e10 msgr2=0x7f8fb41531f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 shutdown_connections 2024-09-13T16:56:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:07.877+0000 7f8fbbeda640 1 -- 172.21.15.39:0/1543396805 wait complete. 2024-09-13T16:56:07.895 INFO:tasks.mds_thrash.fs.[a]:no change 2024-09-13T16:56:09.799 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T16:56:09.799+0000 7f5e120c9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:56:09.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a2cfb640 1 Processor -- start 2024-09-13T16:56:10.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a2cfb640 1 -- start start 2024-09-13T16:56:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a2cfb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 0x7fc29c151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:10.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a2cfb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc29c0c4710 0x7fc29c06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a2cfb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc29c0c4b70 0x7fc29c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a2cfb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29c154ab0 con 0x7fc29c0c4b70 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a2cfb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29c154d90 con 0x7fc29c0c4710 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a2cfb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c150d50 con 0x7fc29c14acf0 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a1298640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc29c0c4b70 0x7fc29c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a1298640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc29c0c4b70 0x7fc29c153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46222/0 (socket says 172.21.15.39:46222) 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a1298640 1 -- 172.21.15.39:0/2531318151 learned_addr learned my addr 172.21.15.39:0/2531318151 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a0a97640 1 --2- 172.21.15.39:0/2531318151 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 0x7fc29c151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc293fff640 1 --2- 172.21.15.39:0/2531318151 >> v2:172.21.15.84:3300/0 conn(0x7fc29c0c4710 0x7fc29c06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a1298640 1 -- 172.21.15.39:0/2531318151 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 msgr2=0x7fc29c151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:10.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a1298640 1 --2- 172.21.15.39:0/2531318151 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 0x7fc29c151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a1298640 1 -- 172.21.15.39:0/2531318151 >> v2:172.21.15.84:3300/0 conn(0x7fc29c0c4710 msgr2=0x7fc29c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:10.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a1298640 1 --2- 172.21.15.39:0/2531318151 >> v2:172.21.15.84:3300/0 conn(0x7fc29c0c4710 0x7fc29c06ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.033+0000 7fc2a1298640 1 -- 172.21.15.39:0/2531318151 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c151030 con 0x7fc29c0c4b70 2024-09-13T16:56:10.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a1298640 1 --2- 172.21.15.39:0/2531318151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 0x7fc29c153ff0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc29400d150 tx=0x7fc29401eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c4fe6b0522a1dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:10.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2937fe640 1 -- 172.21.15.39:0/2531318151 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29400b2d0 con 0x7fc29c0c4b70 2024-09-13T16:56:10.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2937fe640 1 -- 172.21.15.39:0/2531318151 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29400b430 con 0x7fc29c0c4b70 2024-09-13T16:56:10.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2937fe640 1 -- 172.21.15.39:0/2531318151 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2940a7bd0 con 0x7fc29c0c4b70 2024-09-13T16:56:10.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2531318151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 msgr2=0x7fc29c153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2531318151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 0x7fc29c153ff0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc29400d150 tx=0x7fc29401eb00 comp rx=0 tx=0).stop 2024-09-13T16:56:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2531318151 shutdown_connections 2024-09-13T16:56:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2531318151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 0x7fc29c153ff0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2531318151 >> v2:172.21.15.84:3300/0 conn(0x7fc29c0c4710 0x7fc29c06ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2531318151 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 0x7fc29c151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2531318151 >> 172.21.15.39:0/2531318151 conn(0x7fc29c0c32a0 msgr2=0x7fc29c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:10.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2531318151 shutdown_connections 2024-09-13T16:56:10.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.037+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2531318151 wait complete. 2024-09-13T16:56:10.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 Processor -- start 2024-09-13T16:56:10.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 -- start start 2024-09-13T16:56:10.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc29c0c4710 0x7fc29c152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 0x7fc29c0bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 0x7fc29c1ed240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0a7f60 con 0x7fc29c0c4b70 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0a7750 con 0x7fc29c0c4710 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c152fd0 con 0x7fc29c14acf0 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc293fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 0x7fc29c0bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc293fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 0x7fc29c0bfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47480/0 (socket says 172.21.15.39:47480) 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc293fff640 1 -- 172.21.15.39:0/2731455361 learned_addr learned my addr 172.21.15.39:0/2731455361 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a1298640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 0x7fc29c1ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a0a97640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc29c0c4710 0x7fc29c152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc293fff640 1 -- 172.21.15.39:0/2731455361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 msgr2=0x7fc29c1ed240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:10.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc293fff640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 0x7fc29c1ed240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc293fff640 1 -- 172.21.15.39:0/2731455361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc29c0c4710 msgr2=0x7fc29c152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc293fff640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc29c0c4710 0x7fc29c152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc293fff640 1 -- 172.21.15.39:0/2731455361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29400bd40 con 0x7fc29c0c4b70 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc293fff640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 0x7fc29c0bfe30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00d800 tx=0x7fc28c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2917fa640 1 -- 172.21.15.39:0/2731455361 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c018070 con 0x7fc29c0c4b70 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2917fa640 1 -- 172.21.15.39:0/2731455361 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c004020 con 0x7fc29c0c4b70 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c1eccf0 con 0x7fc29c0c4b70 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2917fa640 1 -- 172.21.15.39:0/2731455361 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0041d0 con 0x7fc29c0c4b70 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c0a79a0 con 0x7fc29c0c4b70 2024-09-13T16:56:10.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c1f02c0 con 0x7fc29c0c4b70 2024-09-13T16:56:10.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.041+0000 7fc2917fa640 1 -- 172.21.15.39:0/2731455361 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc28c004330 con 0x7fc29c0c4b70 2024-09-13T16:56:10.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.045+0000 7fc2917fa640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2880666c0 0x7fc288068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:10.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.045+0000 7fc2917fa640 1 -- 172.21.15.39:0/2731455361 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fc28c084670 con 0x7fc29c0c4b70 2024-09-13T16:56:10.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.045+0000 7fc2a0a97640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2880666c0 0x7fc288068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:10.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.045+0000 7fc2a0a97640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2880666c0 0x7fc288068b80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc29c1eab30 tx=0x7fc298008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:10.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.049+0000 7fc2917fa640 1 -- 172.21.15.39:0/2731455361 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc28c051370 con 0x7fc29c0c4b70 2024-09-13T16:56:10.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.253+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc29c06ae20 con 0x7fc29c0c4b70 2024-09-13T16:56:10.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2917fa640 1 -- 172.21.15.39:0/2731455361 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5173 (secure 0 0 0) 0x7fc28c054990 con 0x7fc29c0c4b70 2024-09-13T16:56:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:10.257 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:10.257 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 64 2024-09-13T16:56:10.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2880666c0 msgr2=0x7fc288068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:10.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2880666c0 0x7fc288068b80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc29c1eab30 tx=0x7fc298008040 comp rx=0 tx=0).stop 2024-09-13T16:56:10.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 msgr2=0x7fc29c0bfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:10.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 0x7fc29c0bfe30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00d800 tx=0x7fc28c00dcc0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 shutdown_connections 2024-09-13T16:56:10.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2880666c0 0x7fc288068b80 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc29c1eab30 tx=0x7fc298008040 comp rx=0 tx=0).stop 2024-09-13T16:56:10.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc29c14acf0 0x7fc29c1ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc29c0c4b70 0x7fc29c0bfe30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 --2- 172.21.15.39:0/2731455361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc29c0c4710 0x7fc29c152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:10.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 >> 172.21.15.39:0/2731455361 conn(0x7fc29c0c32a0 msgr2=0x7fc29c14f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:10.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 shutdown_connections 2024-09-13T16:56:10.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:10.257+0000 7fc2a2cfb640 1 -- 172.21.15.39:0/2731455361 wait complete. 2024-09-13T16:56:10.278 INFO:tasks.mds_thrash.fs.[a]:mds.d has gained rank=0, replacing gid=5703 2024-09-13T16:56:10.278 INFO:tasks.mds_thrash.fs.[a]:waiting for 4 secs before reviving mds.a 2024-09-13T16:56:14.279 INFO:tasks.mds_thrash.fs.[a]:reviving mds.a 2024-09-13T16:56:14.279 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:56:14.279 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:56:14.282 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:56:14.282 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:14.358 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T16:56:14.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.449+0000 7ff9cf45d640 1 Processor -- start 2024-09-13T16:56:14.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.449+0000 7ff9cf45d640 1 -- start start 2024-09-13T16:56:14.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.449+0000 7ff9cf45d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff9c80bf4f0 0x7ff9c8153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:14.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.449+0000 7ff9cf45d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c80c1340 con 0x7ff9c80611c0 2024-09-13T16:56:14.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.449+0000 7ff9cf45d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c81578f0 con 0x7ff9c8151860 2024-09-13T16:56:14.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.449+0000 7ff9cf45d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c80c1590 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cc9f8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff9c80bf4f0 0x7ff9c8153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:14.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cc9f8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff9c80bf4f0 0x7ff9c8153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47512/0 (socket says 172.21.15.39:47512) 2024-09-13T16:56:14.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cc9f8640 1 -- 172.21.15.39:0/1197401066 learned_addr learned my addr 172.21.15.39:0/1197401066 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:14.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cd9fa640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff9c80611c0 0x7ff9c80bfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:58938/0 (socket says 172.21.15.39:58938) 2024-09-13T16:56:14.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cc9f8640 1 -- 172.21.15.39:0/1197401066 >> v1:172.21.15.84:6789/0 conn(0x7ff9c8151860 legacy=0x7ff9c8157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:14.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cc9f8640 1 -- 172.21.15.39:0/1197401066 >> v1:172.21.15.39:6789/0 conn(0x7ff9c80611c0 legacy=0x7ff9c80bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:14.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cc9f8640 1 -- 172.21.15.39:0/1197401066 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c80c1870 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9bffff640 1 -- 172.21.15.39:0/1197401066 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206789151 0 0) 0x7ff9c80c1340 con 0x7ff9c80611c0 2024-09-13T16:56:14.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cc9f8640 1 --2- 172.21.15.39:0/1197401066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 0x7ff9c8153df0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009870 tx=0x7ff9b801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e14a8afe466b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:14.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9bffff640 1 -- 172.21.15.39:0/1197401066 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80a6040 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9bffff640 1 -- 172.21.15.39:0/1197401066 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b80aa600 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9bffff640 1 -- 172.21.15.39:0/1197401066 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80aa990 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cf45d640 1 -- 172.21.15.39:0/1197401066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 msgr2=0x7ff9c8153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cf45d640 1 --2- 172.21.15.39:0/1197401066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 0x7ff9c8153df0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009870 tx=0x7ff9b801c4a0 comp rx=0 tx=0).stop 2024-09-13T16:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cf45d640 1 -- 172.21.15.39:0/1197401066 shutdown_connections 2024-09-13T16:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cf45d640 1 --2- 172.21.15.39:0/1197401066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 0x7ff9c8153df0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cf45d640 1 -- 172.21.15.39:0/1197401066 >> 172.21.15.39:0/1197401066 conn(0x7ff9c8069e10 msgr2=0x7ff9c806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cf45d640 1 -- 172.21.15.39:0/1197401066 shutdown_connections 2024-09-13T16:56:14.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.453+0000 7ff9cf45d640 1 -- 172.21.15.39:0/1197401066 wait complete. 2024-09-13T16:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 Processor -- start 2024-09-13T16:56:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 -- start start 2024-09-13T16:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9c80611c0 0x7ff9c80aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 0x7ff9c81ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:14.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c8151860 0x7ff9c81edc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c80aa220 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c8156390 con 0x7ff9c8151860 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c80a9480 con 0x7ff9c80611c0 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cc9f8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 0x7ff9c81ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cc9f8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 0x7ff9c81ed6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47528/0 (socket says 172.21.15.39:47528) 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cc9f8640 1 -- 172.21.15.39:0/3635972729 learned_addr learned my addr 172.21.15.39:0/3635972729 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cd9fa640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c8151860 0x7ff9c81edc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cd1f9640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9c80611c0 0x7ff9c80aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cc9f8640 1 -- 172.21.15.39:0/3635972729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9c80611c0 msgr2=0x7ff9c80aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cc9f8640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9c80611c0 0x7ff9c80aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cc9f8640 1 -- 172.21.15.39:0/3635972729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c8151860 msgr2=0x7ff9c81edc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cc9f8640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c8151860 0x7ff9c81edc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:14.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cc9f8640 1 -- 172.21.15.39:0/3635972729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80094d0 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cc9f8640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 0x7ff9c81ed6e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80190b0 tx=0x7ff9b801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:14.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9bdffb640 1 -- 172.21.15.39:0/3635972729 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80aae10 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9bdffb640 1 -- 172.21.15.39:0/3635972729 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b80b7070 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c8155a60 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9bdffb640 1 -- 172.21.15.39:0/3635972729 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80bd600 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.457+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c8156020 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.461+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c8156520 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.461+0000 7ff9bdffb640 1 -- 172.21.15.39:0/3635972729 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9b80bd760 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.461+0000 7ff9bdffb640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9ac0666c0 0x7ff9ac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:14.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.461+0000 7ff9bdffb640 1 -- 172.21.15.39:0/3635972729 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7ff9b811ecb0 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.461+0000 7ff9cd1f9640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9ac0666c0 0x7ff9ac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:14.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.461+0000 7ff9cd1f9640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9ac0666c0 0x7ff9ac068b80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4000f70 tx=0x7ff9c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:14.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.465+0000 7ff9bdffb640 1 -- 172.21.15.39:0/3635972729 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff9b80eb880 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.717+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9c8153df0 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.717+0000 7ff9bdffb640 1 -- 172.21.15.39:0/3635972729 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5173 (secure 0 0 0) 0x7ff9b80b9090 con 0x7ff9c80bf4f0 2024-09-13T16:56:14.719 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:14.720 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:14.720 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 64 2024-09-13T16:56:14.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9ac0666c0 msgr2=0x7ff9ac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:14.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9ac0666c0 0x7ff9ac068b80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4000f70 tx=0x7ff9c4009040 comp rx=0 tx=0).stop 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 msgr2=0x7ff9c81ed6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 0x7ff9c81ed6e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80190b0 tx=0x7ff9b801ace0 comp rx=0 tx=0).stop 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 shutdown_connections 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff9ac0666c0 0x7ff9ac068b80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9c8151860 0x7ff9c81edc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c80bf4f0 0x7ff9c81ed6e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 --2- 172.21.15.39:0/3635972729 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff9c80611c0 0x7ff9c80aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 >> 172.21.15.39:0/3635972729 conn(0x7ff9c8069e10 msgr2=0x7ff9c8153640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 shutdown_connections 2024-09-13T16:56:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:14.721+0000 7ff9cf45d640 1 -- 172.21.15.39:0/3635972729 wait complete. 2024-09-13T16:56:14.742 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T16:56:16.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b894db640 1 Processor -- start 2024-09-13T16:56:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b894db640 1 -- start start 2024-09-13T16:56:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b894db640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 0x7f1b840c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b894db640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1b840bcfb0 0x7f1b84070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b894db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1b840c77e0 0x7f1b84070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:16.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b894db640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b84157360 con 0x7f1b840c77e0 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b894db640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b840c80b0 con 0x7f1b840bcfb0 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b894db640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841575b0 con 0x7f1b840c4260 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b837fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1b840c77e0 0x7f1b84070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b837fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1b840c77e0 0x7f1b84070970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47540/0 (socket says 172.21.15.39:47540) 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b837fe640 1 -- 172.21.15.39:0/623109089 learned_addr learned my addr 172.21.15.39:0/623109089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b82ffd640 1 --2- 172.21.15.39:0/623109089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 0x7f1b840c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b827fc640 1 --2- 172.21.15.39:0/623109089 >> v2:172.21.15.84:3300/0 conn(0x7f1b840bcfb0 0x7f1b84070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b837fe640 1 -- 172.21.15.39:0/623109089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 msgr2=0x7f1b840c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b837fe640 1 --2- 172.21.15.39:0/623109089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 0x7f1b840c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:16.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b837fe640 1 -- 172.21.15.39:0/623109089 >> v2:172.21.15.84:3300/0 conn(0x7f1b840bcfb0 msgr2=0x7f1b84070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:16.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b837fe640 1 --2- 172.21.15.39:0/623109089 >> v2:172.21.15.84:3300/0 conn(0x7f1b840bcfb0 0x7f1b84070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:16.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b837fe640 1 -- 172.21.15.39:0/623109089 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b84157890 con 0x7f1b840c77e0 2024-09-13T16:56:16.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.881+0000 7f1b82ffd640 1 --2- 172.21.15.39:0/623109089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 0x7f1b840c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:56:16.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b837fe640 1 --2- 172.21.15.39:0/623109089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840c77e0 0x7f1b84070970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1b840ca9e0 tx=0x7f1b7c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1469bf99039d03d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:16.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b81ffb640 1 -- 172.21.15.39:0/623109089 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840c7330 con 0x7f1b840c77e0 2024-09-13T16:56:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b81ffb640 1 -- 172.21.15.39:0/623109089 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c0a5660 con 0x7f1b840c77e0 2024-09-13T16:56:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b81ffb640 1 -- 172.21.15.39:0/623109089 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0a59f0 con 0x7f1b840c77e0 2024-09-13T16:56:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 -- 172.21.15.39:0/623109089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840c77e0 msgr2=0x7f1b84070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:16.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 --2- 172.21.15.39:0/623109089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840c77e0 0x7f1b84070970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1b840ca9e0 tx=0x7f1b7c01c8d0 comp rx=0 tx=0).stop 2024-09-13T16:56:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 -- 172.21.15.39:0/623109089 shutdown_connections 2024-09-13T16:56:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 --2- 172.21.15.39:0/623109089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840c77e0 0x7f1b84070970 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 --2- 172.21.15.39:0/623109089 >> v2:172.21.15.84:3300/0 conn(0x7f1b840bcfb0 0x7f1b84070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 --2- 172.21.15.39:0/623109089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 0x7f1b840c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 -- 172.21.15.39:0/623109089 >> 172.21.15.39:0/623109089 conn(0x7f1b84069e10 msgr2=0x7f1b8406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 -- 172.21.15.39:0/623109089 shutdown_connections 2024-09-13T16:56:16.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 -- 172.21.15.39:0/623109089 wait complete. 2024-09-13T16:56:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 Processor -- start 2024-09-13T16:56:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 -- start start 2024-09-13T16:56:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840bcfb0 0x7f1b8414b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 0x7f1b8414ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b840c77e0 0x7f1b8414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841f0a30 con 0x7f1b840bcfb0 2024-09-13T16:56:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8414b600 con 0x7f1b840c77e0 2024-09-13T16:56:16.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b894db640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841f0750 con 0x7f1b840c4260 2024-09-13T16:56:16.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b82ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840bcfb0 0x7f1b8414b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:16.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b82ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840bcfb0 0x7f1b8414b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47550/0 (socket says 172.21.15.39:47550) 2024-09-13T16:56:16.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b82ffd640 1 -- 172.21.15.39:0/1737017153 learned_addr learned my addr 172.21.15.39:0/1737017153 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:16.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b827fc640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 0x7f1b8414ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:16.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b837fe640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b840c77e0 0x7f1b8414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:16.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b82ffd640 1 -- 172.21.15.39:0/1737017153 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 msgr2=0x7f1b8414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:16.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b82ffd640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 0x7f1b8414ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:16.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b82ffd640 1 -- 172.21.15.39:0/1737017153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b840c77e0 msgr2=0x7f1b8414cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b82ffd640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b840c77e0 0x7f1b8414cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.885+0000 7f1b82ffd640 1 -- 172.21.15.39:0/1737017153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c009b30 con 0x7f1b840bcfb0 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b82ffd640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840bcfb0 0x7f1b8414b0c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e810 tx=0x7f1b7400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b6b7fe640 1 -- 172.21.15.39:0/1737017153 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400c8d0 con 0x7f1b840bcfb0 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b6b7fe640 1 -- 172.21.15.39:0/1737017153 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7400ca30 con 0x7f1b840bcfb0 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8414a8c0 con 0x7f1b840bcfb0 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b6b7fe640 1 -- 172.21.15.39:0/1737017153 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400cdc0 con 0x7f1b840bcfb0 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8414d980 con 0x7f1b840bcfb0 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b840c4640 con 0x7f1b840bcfb0 2024-09-13T16:56:16.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b6b7fe640 1 -- 172.21.15.39:0/1737017153 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b74013410 con 0x7f1b840bcfb0 2024-09-13T16:56:16.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b6b7fe640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b70066690 0x7f1b70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:16.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b6b7fe640 1 -- 172.21.15.39:0/1737017153 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f1b740845e0 con 0x7f1b840bcfb0 2024-09-13T16:56:16.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b827fc640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b70066690 0x7f1b70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:16.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.889+0000 7f1b827fc640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b70066690 0x7f1b70068b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b84158460 tx=0x7f1b78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:16.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:16.893+0000 7f1b6b7fe640 1 -- 172.21.15.39:0/1737017153 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b740512e0 con 0x7f1b840bcfb0 2024-09-13T16:56:17.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.129+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b840c6e70 con 0x7f1b840bcfb0 2024-09-13T16:56:17.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.129+0000 7f1b6b7fe640 1 -- 172.21.15.39:0/1737017153 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f1b74054900 con 0x7f1b840bcfb0 2024-09-13T16:56:17.131 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:17.131 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5955,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:17.131 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 65 2024-09-13T16:56:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b70066690 msgr2=0x7f1b70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b70066690 0x7f1b70068b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b84158460 tx=0x7f1b78008040 comp rx=0 tx=0).stop 2024-09-13T16:56:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840bcfb0 msgr2=0x7f1b8414b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840bcfb0 0x7f1b8414b0c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e810 tx=0x7f1b7400ecd0 comp rx=0 tx=0).stop 2024-09-13T16:56:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 shutdown_connections 2024-09-13T16:56:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1b70066690 0x7f1b70068b50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b840c77e0 0x7f1b8414cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b840c4260 0x7f1b8414ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 --2- 172.21.15.39:0/1737017153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b840bcfb0 0x7f1b8414b0c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 >> 172.21.15.39:0/1737017153 conn(0x7f1b84069e10 msgr2=0x7f1b84153640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 shutdown_connections 2024-09-13T16:56:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:17.133+0000 7f1b894db640 1 -- 172.21.15.39:0/1737017153 wait complete. 2024-09-13T16:56:17.151 INFO:tasks.mds_thrash.fs.[a]:mds.a reported in up:standby state 2024-09-13T16:56:17.151 INFO:tasks.mds_thrash.fs.[a]:waiting for 89 secs before thrashing 2024-09-13T16:56:26.925 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:56:26.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:56:57.021 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:56:57.025 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:56:59.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa73ebf640 1 Processor -- start 2024-09-13T16:56:59.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa73ebf640 1 -- start start 2024-09-13T16:56:59.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa73ebf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faa6c157f50 0x7faa6c15f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa73ebf640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c154ed0 con 0x7faa6c151960 2024-09-13T16:56:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa73ebf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c0c7330 con 0x7faa6c0c7a90 2024-09-13T16:56:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa73ebf640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6c0c51f0 con 0x7faa6c157f50 2024-09-13T16:56:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa7245c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faa6c157f50 0x7faa6c15f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa71c5b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7faa6c151960 0x7faa6c0c62e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:46324/0 (socket says 172.21.15.39:46324) 2024-09-13T16:56:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa71c5b640 1 -- 172.21.15.39:0/1784792000 learned_addr learned my addr 172.21.15.39:0/1784792000 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:56:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa7245c640 1 -- 172.21.15.39:0/1784792000 >> v1:172.21.15.84:6789/0 conn(0x7faa6c0c7a90 legacy=0x7faa6c157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa7245c640 1 -- 172.21.15.39:0/1784792000 >> v1:172.21.15.39:6789/0 conn(0x7faa6c151960 legacy=0x7faa6c0c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.172+0000 7faa7245c640 1 -- 172.21.15.39:0/1784792000 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c1583e0 con 0x7faa6c157f50 2024-09-13T16:56:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa7245c640 1 --2- 172.21.15.39:0/1784792000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 0x7faa6c15f890 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7faa64009d90 tx=0x7faa6401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5dd6716bded15b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa70c59640 1 -- 172.21.15.39:0/1784792000 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa640a5850 con 0x7faa6c157f50 2024-09-13T16:56:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa70c59640 1 -- 172.21.15.39:0/1784792000 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa640a59b0 con 0x7faa6c157f50 2024-09-13T16:56:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa70c59640 1 -- 172.21.15.39:0/1784792000 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa640a5d40 con 0x7faa6c157f50 2024-09-13T16:56:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 -- 172.21.15.39:0/1784792000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 msgr2=0x7faa6c15f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 --2- 172.21.15.39:0/1784792000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 0x7faa6c15f890 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7faa64009d90 tx=0x7faa6401c8a0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 -- 172.21.15.39:0/1784792000 shutdown_connections 2024-09-13T16:56:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 --2- 172.21.15.39:0/1784792000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 0x7faa6c15f890 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 -- 172.21.15.39:0/1784792000 >> 172.21.15.39:0/1784792000 conn(0x7faa6c0ab500 msgr2=0x7faa6c0ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 -- 172.21.15.39:0/1784792000 shutdown_connections 2024-09-13T16:56:59.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 -- 172.21.15.39:0/1784792000 wait complete. 2024-09-13T16:56:59.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 Processor -- start 2024-09-13T16:56:59.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 -- start start 2024-09-13T16:56:59.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa6c0c7a90 0x7faa6c1ec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c151960 0x7faa6c150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 0x7faa6c0ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0adf60 con 0x7faa6c157f50 2024-09-13T16:56:59.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0aed60 con 0x7faa6c151960 2024-09-13T16:56:59.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa73ebf640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0af2e0 con 0x7faa6c0c7a90 2024-09-13T16:56:59.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa7245c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 0x7faa6c0ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa7245c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 0x7faa6c0ad070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37698/0 (socket says 172.21.15.39:37698) 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.176+0000 7faa7245c640 1 -- 172.21.15.39:0/1503039053 learned_addr learned my addr 172.21.15.39:0/1503039053 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa71c5b640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa6c0c7a90 0x7faa6c1ec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa7145a640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c151960 0x7faa6c150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa7245c640 1 -- 172.21.15.39:0/1503039053 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa6c0c7a90 msgr2=0x7faa6c1ec6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa7245c640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa6c0c7a90 0x7faa6c1ec6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa7245c640 1 -- 172.21.15.39:0/1503039053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c151960 msgr2=0x7faa6c150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa7245c640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c151960 0x7faa6c150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa7245c640 1 -- 172.21.15.39:0/1503039053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa64009a70 con 0x7faa6c157f50 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa7145a640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c151960 0x7faa6c150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa7245c640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 0x7faa6c0ad070 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7faa6401a4d0 tx=0x7faa64019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa627fc640 1 -- 172.21.15.39:0/1503039053 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa640b9860 con 0x7faa6c157f50 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa627fc640 1 -- 172.21.15.39:0/1503039053 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa640a5840 con 0x7faa6c157f50 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa627fc640 1 -- 172.21.15.39:0/1503039053 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa640b9bc0 con 0x7faa6c157f50 2024-09-13T16:56:59.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c156da0 con 0x7faa6c157f50 2024-09-13T16:56:59.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c0aef70 con 0x7faa6c157f50 2024-09-13T16:56:59.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c1572c0 con 0x7faa6c157f50 2024-09-13T16:56:59.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa627fc640 1 -- 172.21.15.39:0/1503039053 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa640b9d20 con 0x7faa6c157f50 2024-09-13T16:56:59.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa627fc640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa50066690 0x7faa50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa627fc640 1 -- 172.21.15.39:0/1503039053 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7faa64120070 con 0x7faa6c157f50 2024-09-13T16:56:59.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa71c5b640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa50066690 0x7faa50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.180+0000 7faa71c5b640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa50066690 0x7faa50068b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faa68005890 tx=0x7faa68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:59.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.184+0000 7faa627fc640 1 -- 172.21.15.39:0/1503039053 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faa640ecdc0 con 0x7faa6c157f50 2024-09-13T16:56:59.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.404+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa6c153ca0 con 0x7faa6c157f50 2024-09-13T16:56:59.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.404+0000 7faa627fc640 1 -- 172.21.15.39:0/1503039053 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5930 (secure 0 0 0) 0x7faa640f03e0 con 0x7faa6c157f50 2024-09-13T16:56:59.406 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:59.406 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5955,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:59.406 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 65 2024-09-13T16:56:59.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa50066690 msgr2=0x7faa50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa50066690 0x7faa50068b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faa68005890 tx=0x7faa68009040 comp rx=0 tx=0).stop 2024-09-13T16:56:59.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 msgr2=0x7faa6c0ad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 0x7faa6c0ad070 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7faa6401a4d0 tx=0x7faa64019640 comp rx=0 tx=0).stop 2024-09-13T16:56:59.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 shutdown_connections 2024-09-13T16:56:59.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa50066690 0x7faa50068b50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa6c151960 0x7faa6c150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa6c157f50 0x7faa6c0ad070 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 --2- 172.21.15.39:0/1503039053 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa6c0c7a90 0x7faa6c1ec6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 >> 172.21.15.39:0/1503039053 conn(0x7faa6c0ab500 msgr2=0x7faa6c070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:59.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 shutdown_connections 2024-09-13T16:56:59.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.408+0000 7faa73ebf640 1 -- 172.21.15.39:0/1503039053 wait complete. 2024-09-13T16:56:59.426 INFO:tasks.mds_thrash.fs.[b]:kill mds.b (rank=0) 2024-09-13T16:56:59.427 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:56:59.427 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:56:59.441 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:56:59.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:56:59.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32ed3f640 1 Processor -- start 2024-09-13T16:56:59.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32ed3f640 1 -- start start 2024-09-13T16:56:59.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32ed3f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe328071dd0 0x7fe328154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32ed3f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3280c7800 0x7fe3281545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32ed3f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3280c5a30 con 0x7fe328151500 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32ed3f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3281576a0 con 0x7fe328071dd0 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32ed3f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe328157980 con 0x7fe3280c7800 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32cadb640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe328151500 0x7fe3280c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:46332/0 (socket says 172.21.15.39:46332) 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32cadb640 1 -- 172.21.15.39:0/3746753745 learned_addr learned my addr 172.21.15.39:0/3746753745 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe327fff640 1 --2- 172.21.15.39:0/3746753745 >> v2:172.21.15.84:3300/0 conn(0x7fe328071dd0 0x7fe328154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe32d2dc640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3280c7800 0x7fe3281545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62636418 0 0) 0x7fe3280c5a30 con 0x7fe328151500 2024-09-13T16:56:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe318003530 con 0x7fe328151500 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4278463320 0 0) 0x7fe318003530 con 0x7fe328151500 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3280c7800 msgr2=0x7fe3281545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe3277fe640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3280c7800 0x7fe3281545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 >> v2:172.21.15.84:3300/0 conn(0x7fe328071dd0 msgr2=0x7fe328154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe3277fe640 1 --2- 172.21.15.39:0/3746753745 >> v2:172.21.15.84:3300/0 conn(0x7fe328071dd0 0x7fe328154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.572+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328157c60 con 0x7fe328151500 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fe31c0016a0 con 0x7fe328151500 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 >> v1:172.21.15.39:6789/0 conn(0x7fe328151500 legacy=0x7fe3280c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe318003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3280c7800 0x7fe3180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe318004d10 0x7fe318005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281576a0 con 0x7fe318003980 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe328157980 con 0x7fe318004d10 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31800b990 con 0x7fe3280c7800 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe327fff640 1 --2- 172.21.15.39:0/3746753745 >> v2:172.21.15.84:3300/0 conn(0x7fe328071dd0 0x7fe328154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32cadb640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe318003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe327fff640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3280c7800 0x7fe3180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32d2dc640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe318004d10 0x7fe318005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32cadb640 1 -- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3280c7800 msgr2=0x7fe3180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32cadb640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe3280c7800 0x7fe3180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32cadb640 1 -- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe318004d10 msgr2=0x7fe318005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32cadb640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe318004d10 0x7fe318005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32cadb640 1 -- 172.21.15.39:0/3746753745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe318003530 con 0x7fe318003980 2024-09-13T16:56:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32cadb640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe318003da0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe3280c5bc0 tx=0x7fe31c01d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a60e9e937ebcda server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328154dc0 con 0x7fe318003980 2024-09-13T16:56:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c0a58f0 con 0x7fe318003980 2024-09-13T16:56:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe3277fe640 1 -- 172.21.15.39:0/3746753745 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0a5cc0 con 0x7fe318003980 2024-09-13T16:56:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32ed3f640 1 -- 172.21.15.39:0/3746753745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 msgr2=0x7fe318003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32ed3f640 1 --2- 172.21.15.39:0/3746753745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe318003da0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe3280c5bc0 tx=0x7fe31c01d650 comp rx=0 tx=0).stop 2024-09-13T16:56:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32d2dc640 1 -- 172.21.15.39:0/3746753745 reap_dead start 2024-09-13T16:56:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32ed3f640 1 -- 172.21.15.39:0/3746753745 shutdown_connections 2024-09-13T16:56:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32ed3f640 1 -- 172.21.15.39:0/3746753745 >> 172.21.15.39:0/3746753745 conn(0x7fe3280ab530 msgr2=0x7fe3280ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32ed3f640 1 -- 172.21.15.39:0/3746753745 shutdown_connections 2024-09-13T16:56:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.576+0000 7fe32ed3f640 1 -- 172.21.15.39:0/3746753745 wait complete. 2024-09-13T16:56:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 Processor -- start 2024-09-13T16:56:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 -- start start 2024-09-13T16:56:59.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe3281ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe32814a270 0x7fe328156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe328151bd0 0x7fe3281483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32814a7c0 con 0x7fe318003980 2024-09-13T16:56:59.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281ec130 con 0x7fe328151bd0 2024-09-13T16:56:59.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281ec410 con 0x7fe32814a270 2024-09-13T16:56:59.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32cadb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe3281ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32cadb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe3281ecc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37714/0 (socket says 172.21.15.39:37714) 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32cadb640 1 -- 172.21.15.39:0/830936833 learned_addr learned my addr 172.21.15.39:0/830936833 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32d2dc640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe328151bd0 0x7fe3281483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe327fff640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe32814a270 0x7fe328156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32cadb640 1 -- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe32814a270 msgr2=0x7fe328156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32cadb640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe32814a270 0x7fe328156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32cadb640 1 -- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe328151bd0 msgr2=0x7fe3281483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32cadb640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe328151bd0 0x7fe3281483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32cadb640 1 -- 172.21.15.39:0/830936833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3281ec6f0 con 0x7fe318003980 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe327fff640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe32814a270 0x7fe328156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32d2dc640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe328151bd0 0x7fe3281483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:56:59.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32cadb640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe3281ecc10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007040 tx=0x7fe318005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:59.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe3257fa640 1 -- 172.21.15.39:0/830936833 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c001660 con 0x7fe318003980 2024-09-13T16:56:59.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe3257fa640 1 -- 172.21.15.39:0/830936833 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c0047f0 con 0x7fe318003980 2024-09-13T16:56:59.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3280c85e0 con 0x7fe318003980 2024-09-13T16:56:59.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe3257fa640 1 -- 172.21.15.39:0/830936833 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c004b80 con 0x7fe318003980 2024-09-13T16:56:59.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.580+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe328153a00 con 0x7fe318003980 2024-09-13T16:56:59.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.584+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3281ec270 con 0x7fe318003980 2024-09-13T16:56:59.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.584+0000 7fe3257fa640 1 -- 172.21.15.39:0/830936833 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe31c0b5070 con 0x7fe318003980 2024-09-13T16:56:59.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.584+0000 7fe3257fa640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe30c066690 0x7fe30c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:56:59.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.584+0000 7fe3257fa640 1 -- 172.21.15.39:0/830936833 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fe31c11e490 con 0x7fe318003980 2024-09-13T16:56:59.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.584+0000 7fe327fff640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe30c066690 0x7fe30c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:56:59.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.584+0000 7fe327fff640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe30c066690 0x7fe30c068b50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe328071180 tx=0x7fe314008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:56:59.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.588+0000 7fe3257fa640 1 -- 172.21.15.39:0/830936833 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe3281ec270 con 0x7fe318003980 2024-09-13T16:56:59.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.828+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3281ec270 con 0x7fe318003980 2024-09-13T16:56:59.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.828+0000 7fe3257fa640 1 -- 172.21.15.39:0/830936833 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5930 (secure 0 0 0) 0x7fe3281ec270 con 0x7fe318003980 2024-09-13T16:56:59.832 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:56:59.832 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5955,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:56:59.833 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 65 2024-09-13T16:56:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe30c066690 msgr2=0x7fe30c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe30c066690 0x7fe30c068b50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe328071180 tx=0x7fe314008040 comp rx=0 tx=0).stop 2024-09-13T16:56:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 msgr2=0x7fe3281ecc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:56:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe3281ecc10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007040 tx=0x7fe318005800 comp rx=0 tx=0).stop 2024-09-13T16:56:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 shutdown_connections 2024-09-13T16:56:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe30c066690 0x7fe30c068b50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe328151bd0 0x7fe3281483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe32814a270 0x7fe328156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 --2- 172.21.15.39:0/830936833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe318003980 0x7fe3281ecc10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:56:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 >> 172.21.15.39:0/830936833 conn(0x7fe3280ab530 msgr2=0x7fe3280c7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:56:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 shutdown_connections 2024-09-13T16:56:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:56:59.832+0000 7fe32ed3f640 1 -- 172.21.15.39:0/830936833 wait complete. 2024-09-13T16:56:59.855 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:57:01.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.992+0000 7f4d08e6e640 1 Processor -- start 2024-09-13T16:57:01.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.992+0000 7f4d08e6e640 1 -- start start 2024-09-13T16:57:01.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.992+0000 7f4d08e6e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d04070b40 0x7f4d040c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:01.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.992+0000 7f4d08e6e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4d040bcfb0 0x7f4d0415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:01.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.992+0000 7f4d08e6e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d04157360 con 0x7f4d040c4260 2024-09-13T16:57:01.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.992+0000 7f4d08e6e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d040c80b0 con 0x7f4d040bcfb0 2024-09-13T16:57:01.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.992+0000 7f4d08e6e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d041575b0 con 0x7f4d04070b40 2024-09-13T16:57:01.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d0259c640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4d040c4260 0x7f4d04070430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:52090/0 (socket says 172.21.15.39:52090) 2024-09-13T16:57:01.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d0259c640 1 -- 172.21.15.39:0/2865428261 learned_addr learned my addr 172.21.15.39:0/2865428261 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:01.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d01d9b640 1 --2- 172.21.15.39:0/2865428261 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d04070b40 0x7f4d040c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:01.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d02d9d640 1 --2- 172.21.15.39:0/2865428261 >> v2:172.21.15.84:3300/0 conn(0x7f4d040bcfb0 0x7f4d0415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:01.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d01d9b640 1 -- 172.21.15.39:0/2865428261 >> v2:172.21.15.84:3300/0 conn(0x7f4d040bcfb0 msgr2=0x7f4d0415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:01.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d01d9b640 1 --2- 172.21.15.39:0/2865428261 >> v2:172.21.15.84:3300/0 conn(0x7f4d040bcfb0 0x7f4d0415e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:01.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d01d9b640 1 -- 172.21.15.39:0/2865428261 >> v1:172.21.15.84:6789/0 conn(0x7f4d040c4260 legacy=0x7f4d04070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:01.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d01d9b640 1 -- 172.21.15.39:0/2865428261 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d04157890 con 0x7f4d04070b40 2024-09-13T16:57:01.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d02d9d640 1 --2- 172.21.15.39:0/2865428261 >> v2:172.21.15.84:3300/0 conn(0x7f4d040bcfb0 0x7f4d0415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:57:01.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d01d9b640 1 --2- 172.21.15.39:0/2865428261 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d04070b40 0x7f4d040c4640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80097c0 tx=0x7f4cf801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db52cced44d0369d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:02.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d0159a640 1 -- 172.21.15.39:0/2865428261 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d040c7330 con 0x7f4d04070b40 2024-09-13T16:57:02.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d0159a640 1 -- 172.21.15.39:0/2865428261 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf80a7050 con 0x7f4d04070b40 2024-09-13T16:57:02.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:01.996+0000 7f4d0159a640 1 -- 172.21.15.39:0/2865428261 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf80ab820 con 0x7f4d04070b40 2024-09-13T16:57:02.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- 172.21.15.39:0/2865428261 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d04070b40 msgr2=0x7f4d040c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:02.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 --2- 172.21.15.39:0/2865428261 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d04070b40 0x7f4d040c4640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80097c0 tx=0x7f4cf801c380 comp rx=0 tx=0).stop 2024-09-13T16:57:02.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- 172.21.15.39:0/2865428261 shutdown_connections 2024-09-13T16:57:02.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 --2- 172.21.15.39:0/2865428261 >> v2:172.21.15.84:3300/0 conn(0x7f4d040bcfb0 0x7f4d0415e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:02.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 --2- 172.21.15.39:0/2865428261 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d04070b40 0x7f4d040c4640 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:02.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- 172.21.15.39:0/2865428261 >> 172.21.15.39:0/2865428261 conn(0x7f4d04069e10 msgr2=0x7f4d0406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:02.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- 172.21.15.39:0/2865428261 shutdown_connections 2024-09-13T16:57:02.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- 172.21.15.39:0/2865428261 wait complete. 2024-09-13T16:57:02.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 Processor -- start 2024-09-13T16:57:02.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- start start 2024-09-13T16:57:02.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d040bcfb0 0x7f4d041473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:02.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d040c4260 0x7f4d041542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:02.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d041547e0 0x7f4d04151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0414f4e0 con 0x7f4d040bcfb0 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0414b410 con 0x7f4d040c4260 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040be1c0 con 0x7f4d041547e0 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d0259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d040bcfb0 0x7f4d041473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d0259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d040bcfb0 0x7f4d041473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55830/0 (socket says 172.21.15.39:55830) 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d0259c640 1 -- 172.21.15.39:0/146029364 learned_addr learned my addr 172.21.15.39:0/146029364 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d01d9b640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d040c4260 0x7f4d041542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d02d9d640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d041547e0 0x7f4d04151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d0259c640 1 -- 172.21.15.39:0/146029364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d041547e0 msgr2=0x7f4d04151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d0259c640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d041547e0 0x7f4d04151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d0259c640 1 -- 172.21.15.39:0/146029364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d040c4260 msgr2=0x7f4d041542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d0259c640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d040c4260 0x7f4d041542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d0259c640 1 -- 172.21.15.39:0/146029364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc009a70 con 0x7f4d040bcfb0 2024-09-13T16:57:02.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d0259c640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d040bcfb0 0x7f4d041473f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400c370 tx=0x7f4cf400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:02.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4cf2ffd640 1 -- 172.21.15.39:0/146029364 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4015e30 con 0x7f4d040bcfb0 2024-09-13T16:57:02.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4cf2ffd640 1 -- 172.21.15.39:0/146029364 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf40058c0 con 0x7f4d040bcfb0 2024-09-13T16:57:02.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4cf2ffd640 1 -- 172.21.15.39:0/146029364 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4005c70 con 0x7f4d040bcfb0 2024-09-13T16:57:02.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf8009420 con 0x7f4d040bcfb0 2024-09-13T16:57:02.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.000+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d040be460 con 0x7f4d040bcfb0 2024-09-13T16:57:02.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.004+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d040c4640 con 0x7f4d040bcfb0 2024-09-13T16:57:02.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.004+0000 7f4cf2ffd640 1 -- 172.21.15.39:0/146029364 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4cf401b020 con 0x7f4d040bcfb0 2024-09-13T16:57:02.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.004+0000 7f4cf2ffd640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ce4066690 0x7f4ce4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:02.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.004+0000 7f4cf2ffd640 1 -- 172.21.15.39:0/146029364 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f4cf408c550 con 0x7f4d040bcfb0 2024-09-13T16:57:02.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.004+0000 7f4d01d9b640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ce4066690 0x7f4ce4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:02.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.004+0000 7f4d01d9b640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ce4066690 0x7f4ce4068b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80ad000 tx=0x7f4cf8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:02.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.008+0000 7f4cf2ffd640 1 -- 172.21.15.39:0/146029364 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4cf4059350 con 0x7f4d040bcfb0 2024-09-13T16:57:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.284+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d040c6e70 con 0x7f4d040bcfb0 2024-09-13T16:57:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.284+0000 7f4cf2ffd640 1 -- 172.21.15.39:0/146029364 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f4cf405c970 con 0x7f4d040bcfb0 2024-09-13T16:57:02.287 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:02.287 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5955,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 65 2024-09-13T16:57:02.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ce4066690 msgr2=0x7f4ce4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:02.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ce4066690 0x7f4ce4068b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80ad000 tx=0x7f4cf8008040 comp rx=0 tx=0).stop 2024-09-13T16:57:02.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d040bcfb0 msgr2=0x7f4d041473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:02.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d040bcfb0 0x7f4d041473f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400c370 tx=0x7f4cf400c830 comp rx=0 tx=0).stop 2024-09-13T16:57:02.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 shutdown_connections 2024-09-13T16:57:02.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ce4066690 0x7f4ce4068b50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:02.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d040c4260 0x7f4d041542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:02.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d041547e0 0x7f4d04151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:02.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 --2- 172.21.15.39:0/146029364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d040bcfb0 0x7f4d041473f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:02.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 >> 172.21.15.39:0/146029364 conn(0x7f4d04069e10 msgr2=0x7f4d040c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:02.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 shutdown_connections 2024-09-13T16:57:02.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:02.288+0000 7f4d08e6e640 1 -- 172.21.15.39:0/146029364 wait complete. 2024-09-13T16:57:02.312 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:57:04.313 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.452+0000 7fcc8497e640 1 Processor -- start 2024-09-13T16:57:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.452+0000 7fcc8497e640 1 -- start start 2024-09-13T16:57:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc8497e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcc8014aa40 0x7fcc800c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc8497e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc800c76a0 con 0x7fcc800c4710 2024-09-13T16:57:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc8497e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc80151570 con 0x7fcc800c1750 2024-09-13T16:57:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc8497e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc80151b30 con 0x7fcc8014aa40 2024-09-13T16:57:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7e59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcc8014aa40 0x7fcc800c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7ed9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcc800c4710 0x7fcc8015b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34390/0 (socket says 172.21.15.39:34390) 2024-09-13T16:57:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7ed9d640 1 -- 172.21.15.39:0/2801234324 learned_addr learned my addr 172.21.15.39:0/2801234324 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7e59c640 1 -- 172.21.15.39:0/2801234324 >> v1:172.21.15.84:6789/0 conn(0x7fcc800c1750 legacy=0x7fcc800c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7e59c640 1 -- 172.21.15.39:0/2801234324 >> v1:172.21.15.39:6789/0 conn(0x7fcc800c4710 legacy=0x7fcc8015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:04.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7d59a640 1 -- 172.21.15.39:0/2801234324 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020252828 0 0) 0x7fcc800c76a0 con 0x7fcc800c4710 2024-09-13T16:57:04.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7e59c640 1 -- 172.21.15.39:0/2801234324 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80151850 con 0x7fcc8014aa40 2024-09-13T16:57:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7e59c640 1 --2- 172.21.15.39:0/2801234324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 0x7fcc800c1210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000ba60 tx=0x7fcc7001ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ee8e133faa7a44 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7d59a640 1 -- 172.21.15.39:0/2801234324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc700a7a40 con 0x7fcc8014aa40 2024-09-13T16:57:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7d59a640 1 -- 172.21.15.39:0/2801234324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc700a7ba0 con 0x7fcc8014aa40 2024-09-13T16:57:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.456+0000 7fcc7d59a640 1 -- 172.21.15.39:0/2801234324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc700af440 con 0x7fcc8014aa40 2024-09-13T16:57:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- 172.21.15.39:0/2801234324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 msgr2=0x7fcc800c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 --2- 172.21.15.39:0/2801234324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 0x7fcc800c1210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000ba60 tx=0x7fcc7001ea70 comp rx=0 tx=0).stop 2024-09-13T16:57:04.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- 172.21.15.39:0/2801234324 shutdown_connections 2024-09-13T16:57:04.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 --2- 172.21.15.39:0/2801234324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 0x7fcc800c1210 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:04.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- 172.21.15.39:0/2801234324 >> 172.21.15.39:0/2801234324 conn(0x7fcc800c3310 msgr2=0x7fcc800c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:04.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- 172.21.15.39:0/2801234324 shutdown_connections 2024-09-13T16:57:04.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- 172.21.15.39:0/2801234324 wait complete. 2024-09-13T16:57:04.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 Processor -- start 2024-09-13T16:57:04.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- start start 2024-09-13T16:57:04.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcc800c1750 0x7fcc800716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:04.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc800c4710 0x7fcc8013e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:04.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 0x7fcc8013ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8013f350 con 0x7fcc8014aa40 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80145350 con 0x7fcc800c4710 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8014f990 con 0x7fcc800c1750 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 0x7fcc8013ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 0x7fcc8013ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55846/0 (socket says 172.21.15.39:55846) 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7ed9d640 1 -- 172.21.15.39:0/754478853 learned_addr learned my addr 172.21.15.39:0/754478853 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7dd9b640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc800c4710 0x7fcc8013e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7e59c640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcc800c1750 0x7fcc800716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7ed9d640 1 -- 172.21.15.39:0/754478853 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcc800c1750 msgr2=0x7fcc800716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7ed9d640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcc800c1750 0x7fcc800716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7ed9d640 1 -- 172.21.15.39:0/754478853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc800c4710 msgr2=0x7fcc8013e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:04.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7ed9d640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc800c4710 0x7fcc8013e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:04.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7ed9d640 1 -- 172.21.15.39:0/754478853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7000b6c0 con 0x7fcc8014aa40 2024-09-13T16:57:04.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7dd9b640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc800c4710 0x7fcc8013e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:57:04.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc7ed9d640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 0x7fcc8013ee10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800c780 tx=0x7fcc7800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:04.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc6effd640 1 -- 172.21.15.39:0/754478853 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800ce30 con 0x7fcc8014aa40 2024-09-13T16:57:04.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc6effd640 1 -- 172.21.15.39:0/754478853 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7800a640 con 0x7fcc8014aa40 2024-09-13T16:57:04.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc801455c0 con 0x7fcc8014aa40 2024-09-13T16:57:04.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.460+0000 7fcc6effd640 1 -- 172.21.15.39:0/754478853 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800aa30 con 0x7fcc8014aa40 2024-09-13T16:57:04.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.464+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc80154960 con 0x7fcc8014aa40 2024-09-13T16:57:04.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.464+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc80145ae0 con 0x7fcc8014aa40 2024-09-13T16:57:04.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.464+0000 7fcc6effd640 1 -- 172.21.15.39:0/754478853 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcc7800ab90 con 0x7fcc8014aa40 2024-09-13T16:57:04.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.464+0000 7fcc6effd640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcc600666c0 0x7fcc60068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:04.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.464+0000 7fcc6effd640 1 -- 172.21.15.39:0/754478853 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fcc78084040 con 0x7fcc8014aa40 2024-09-13T16:57:04.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.464+0000 7fcc7e59c640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcc600666c0 0x7fcc60068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:04.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.464+0000 7fcc7e59c640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcc600666c0 0x7fcc60068b80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc700b1000 tx=0x7fcc700a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:04.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.468+0000 7fcc6effd640 1 -- 172.21.15.39:0/754478853 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcc80145ae0 con 0x7fcc8014aa40 2024-09-13T16:57:04.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.688+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc801547c0 con 0x7fcc8014aa40 2024-09-13T16:57:04.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.688+0000 7fcc6effd640 1 -- 172.21.15.39:0/754478853 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5930 (secure 0 0 0) 0x7fcc801547c0 con 0x7fcc8014aa40 2024-09-13T16:57:04.692 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:04.692 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5955,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:04.692 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 65 2024-09-13T16:57:04.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcc600666c0 msgr2=0x7fcc60068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:04.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcc600666c0 0x7fcc60068b80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc700b1000 tx=0x7fcc700a9070 comp rx=0 tx=0).stop 2024-09-13T16:57:04.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 msgr2=0x7fcc8013ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:04.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 0x7fcc8013ee10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800c780 tx=0x7fcc7800cc40 comp rx=0 tx=0).stop 2024-09-13T16:57:04.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 shutdown_connections 2024-09-13T16:57:04.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcc600666c0 0x7fcc60068b80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:04.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcc800c4710 0x7fcc8013e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:04.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc8014aa40 0x7fcc8013ee10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:04.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 --2- 172.21.15.39:0/754478853 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcc800c1750 0x7fcc800716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:04.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 >> 172.21.15.39:0/754478853 conn(0x7fcc800c3310 msgr2=0x7fcc8014e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:04.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 shutdown_connections 2024-09-13T16:57:04.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:04.692+0000 7fcc8497e640 1 -- 172.21.15.39:0/754478853 wait complete. 2024-09-13T16:57:04.711 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:57:06.712 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.856+0000 7fa08abc1640 1 Processor -- start 2024-09-13T16:57:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.856+0000 7fa08abc1640 1 -- start start 2024-09-13T16:57:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa08abc1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 0x7fa0840c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa08abc1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa0840a7120 0x7fa0840714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa08abc1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0840c2d30 con 0x7fa0840719e0 2024-09-13T16:57:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa08abc1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa084071e10 con 0x7fa0840a7120 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa08abc1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08406fe70 con 0x7fa0840c4470 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa0840a7120 0x7fa0840714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa0840a7120 0x7fa0840714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55854/0 (socket says 172.21.15.39:55854) 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07bfff640 1 -- 172.21.15.39:0/79151408 learned_addr learned my addr 172.21.15.39:0/79151408 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa08895d640 1 --2- 172.21.15.39:0/79151408 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 0x7fa0840c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07bfff640 1 -- 172.21.15.39:0/79151408 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 msgr2=0x7fa0840c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07bfff640 1 --2- 172.21.15.39:0/79151408 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 0x7fa0840c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07bfff640 1 -- 172.21.15.39:0/79151408 >> v1:172.21.15.84:6789/0 conn(0x7fa0840719e0 legacy=0x7fa08406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07bfff640 1 -- 172.21.15.39:0/79151408 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08406fb90 con 0x7fa0840a7120 2024-09-13T16:57:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa08895d640 1 --2- 172.21.15.39:0/79151408 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 0x7fa0840c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:06.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07bfff640 1 --2- 172.21.15.39:0/79151408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840a7120 0x7fa0840714a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa074009870 tx=0x7fa07401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fef0746abd1e258 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:06.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07b7fe640 1 -- 172.21.15.39:0/79151408 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0740ac640 con 0x7fa0840a7120 2024-09-13T16:57:06.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07b7fe640 1 -- 172.21.15.39:0/79151408 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0740ac7a0 con 0x7fa0840a7120 2024-09-13T16:57:06.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.860+0000 7fa07b7fe640 1 -- 172.21.15.39:0/79151408 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0740acb30 con 0x7fa0840a7120 2024-09-13T16:57:06.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 -- 172.21.15.39:0/79151408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840a7120 msgr2=0x7fa0840714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 --2- 172.21.15.39:0/79151408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840a7120 0x7fa0840714a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa074009870 tx=0x7fa07401c520 comp rx=0 tx=0).stop 2024-09-13T16:57:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 -- 172.21.15.39:0/79151408 shutdown_connections 2024-09-13T16:57:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 --2- 172.21.15.39:0/79151408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840a7120 0x7fa0840714a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 --2- 172.21.15.39:0/79151408 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 0x7fa0840c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 -- 172.21.15.39:0/79151408 >> 172.21.15.39:0/79151408 conn(0x7fa08406c020 msgr2=0x7fa08406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 -- 172.21.15.39:0/79151408 shutdown_connections 2024-09-13T16:57:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 -- 172.21.15.39:0/79151408 wait complete. 2024-09-13T16:57:06.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 Processor -- start 2024-09-13T16:57:06.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 -- start start 2024-09-13T16:57:06.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840719e0 0x7fa08414d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:06.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0840a7120 0x7fa0841eb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:06.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 0x7fa0841ebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:06.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa084157810 con 0x7fa0840719e0 2024-09-13T16:57:06.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0841ea300 con 0x7fa0840a7120 2024-09-13T16:57:06.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08abc1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0841eda10 con 0x7fa0840c4470 2024-09-13T16:57:06.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08895d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840719e0 0x7fa08414d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08895d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840719e0 0x7fa08414d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55856/0 (socket says 172.21.15.39:55856) 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa08895d640 1 -- 172.21.15.39:0/1214522314 learned_addr learned my addr 172.21.15.39:0/1214522314 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.864+0000 7fa07bfff640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0840a7120 0x7fa0841eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08915e640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 0x7fa0841ebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08895d640 1 -- 172.21.15.39:0/1214522314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 msgr2=0x7fa0841ebcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08895d640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 0x7fa0841ebcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08895d640 1 -- 172.21.15.39:0/1214522314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0840a7120 msgr2=0x7fa0841eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08895d640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0840a7120 0x7fa0841eb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08895d640 1 -- 172.21.15.39:0/1214522314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0740094d0 con 0x7fa0840719e0 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08895d640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840719e0 0x7fa08414d060 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa08000c7c0 tx=0x7fa08000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa0797fa640 1 -- 172.21.15.39:0/1214522314 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080007a30 con 0x7fa0840719e0 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa0797fa640 1 -- 172.21.15.39:0/1214522314 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa080007b90 con 0x7fa0840719e0 2024-09-13T16:57:06.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0841ea5c0 con 0x7fa0840719e0 2024-09-13T16:57:06.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa0797fa640 1 -- 172.21.15.39:0/1214522314 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08000ce70 con 0x7fa0840719e0 2024-09-13T16:57:06.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0841eab80 con 0x7fa0840719e0 2024-09-13T16:57:06.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0840c5340 con 0x7fa0840719e0 2024-09-13T16:57:06.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa0797fa640 1 -- 172.21.15.39:0/1214522314 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa0800113f0 con 0x7fa0840719e0 2024-09-13T16:57:06.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa0797fa640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa070066690 0x7fa070068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:06.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa0797fa640 1 -- 172.21.15.39:0/1214522314 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fa080084c90 con 0x7fa0840719e0 2024-09-13T16:57:06.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa07bfff640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa070066690 0x7fa070068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:06.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.868+0000 7fa07bfff640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa070066690 0x7fa070068b50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa0740ae000 tx=0x7fa074008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:06.872+0000 7fa0797fa640 1 -- 172.21.15.39:0/1214522314 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa080051990 con 0x7fa0840719e0 2024-09-13T16:57:07.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.116+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa084155630 con 0x7fa0840719e0 2024-09-13T16:57:07.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.116+0000 7fa0797fa640 1 -- 172.21.15.39:0/1214522314 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5930 (secure 0 0 0) 0x7fa08001e090 con 0x7fa0840719e0 2024-09-13T16:57:07.119 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:07.119 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5955,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:07.120 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 65 2024-09-13T16:57:07.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa070066690 msgr2=0x7fa070068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:07.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa070066690 0x7fa070068b50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa0740ae000 tx=0x7fa074008040 comp rx=0 tx=0).stop 2024-09-13T16:57:07.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840719e0 msgr2=0x7fa08414d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840719e0 0x7fa08414d060 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa08000c7c0 tx=0x7fa08000cc80 comp rx=0 tx=0).stop 2024-09-13T16:57:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 shutdown_connections 2024-09-13T16:57:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa070066690 0x7fa070068b50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0840a7120 0x7fa0841eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0840c4470 0x7fa0841ebcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 --2- 172.21.15.39:0/1214522314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0840719e0 0x7fa08414d060 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 >> 172.21.15.39:0/1214522314 conn(0x7fa08406c020 msgr2=0x7fa0840c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 shutdown_connections 2024-09-13T16:57:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:07.120+0000 7fa08abc1640 1 -- 172.21.15.39:0/1214522314 wait complete. 2024-09-13T16:57:07.138 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:57:09.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:09.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3f256640 1 Processor -- start 2024-09-13T16:57:09.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3f256640 1 -- start start 2024-09-13T16:57:09.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3f256640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6d3814aa40 0x7f6d380c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3f256640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c1750 0x7f6d380c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3f256640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6d380c4710 0x7f6d380c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3f256640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d38151b30 con 0x7f6d380c4710 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3f256640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d380c6510 con 0x7f6d3814aa40 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3f256640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380c76a0 con 0x7f6d380c1750 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3d7f3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6d380c4710 0x7f6d380c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3d7f3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6d380c4710 0x7f6d380c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55868/0 (socket says 172.21.15.39:55868) 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3d7f3640 1 -- 172.21.15.39:0/4188930955 learned_addr learned my addr 172.21.15.39:0/4188930955 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3cff2640 1 --2- 172.21.15.39:0/4188930955 >> v2:172.21.15.84:3300/0 conn(0x7f6d3814aa40 0x7f6d380c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d37fff640 1 --2- 172.21.15.39:0/4188930955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c1750 0x7f6d380c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3d7f3640 1 -- 172.21.15.39:0/4188930955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c1750 msgr2=0x7f6d380c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:09.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3d7f3640 1 --2- 172.21.15.39:0/4188930955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c1750 0x7f6d380c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3d7f3640 1 -- 172.21.15.39:0/4188930955 >> v2:172.21.15.84:3300/0 conn(0x7f6d3814aa40 msgr2=0x7f6d380c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:09.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3d7f3640 1 --2- 172.21.15.39:0/4188930955 >> v2:172.21.15.84:3300/0 conn(0x7f6d3814aa40 0x7f6d380c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.296+0000 7f6d3d7f3640 1 -- 172.21.15.39:0/4188930955 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d38151570 con 0x7f6d380c4710 2024-09-13T16:57:09.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3cff2640 1 --2- 172.21.15.39:0/4188930955 >> v2:172.21.15.84:3300/0 conn(0x7f6d3814aa40 0x7f6d380c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:09.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3d7f3640 1 --2- 172.21.15.39:0/4188930955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c4710 0x7f6d380c5fd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6d3806fde0 tx=0x7f6d3001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a14eca43f4d75f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:09.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d377fe640 1 -- 172.21.15.39:0/4188930955 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300a57d0 con 0x7f6d380c4710 2024-09-13T16:57:09.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d377fe640 1 -- 172.21.15.39:0/4188930955 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d300a5930 con 0x7f6d380c4710 2024-09-13T16:57:09.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d377fe640 1 -- 172.21.15.39:0/4188930955 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300a5d00 con 0x7f6d380c4710 2024-09-13T16:57:09.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 -- 172.21.15.39:0/4188930955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c4710 msgr2=0x7f6d380c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:09.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 --2- 172.21.15.39:0/4188930955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c4710 0x7f6d380c5fd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6d3806fde0 tx=0x7f6d3001c950 comp rx=0 tx=0).stop 2024-09-13T16:57:09.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 -- 172.21.15.39:0/4188930955 shutdown_connections 2024-09-13T16:57:09.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 --2- 172.21.15.39:0/4188930955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c4710 0x7f6d380c5fd0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 --2- 172.21.15.39:0/4188930955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c1750 0x7f6d380c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 --2- 172.21.15.39:0/4188930955 >> v2:172.21.15.84:3300/0 conn(0x7f6d3814aa40 0x7f6d380c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 -- 172.21.15.39:0/4188930955 >> 172.21.15.39:0/4188930955 conn(0x7f6d380c3310 msgr2=0x7f6d380c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:09.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 -- 172.21.15.39:0/4188930955 shutdown_connections 2024-09-13T16:57:09.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 -- 172.21.15.39:0/4188930955 wait complete. 2024-09-13T16:57:09.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 Processor -- start 2024-09-13T16:57:09.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.300+0000 7f6d3f256640 1 -- start start 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3f256640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c1750 0x7f6d38149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3f256640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c4710 0x7f6d381ec690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3f256640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d3814aa40 0x7f6d381ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3f256640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380aa5f0 con 0x7f6d380c1750 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3f256640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380aa840 con 0x7f6d3814aa40 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3f256640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d381ed240 con 0x7f6d380c4710 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3cff2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c1750 0x7f6d38149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3cff2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c1750 0x7f6d38149a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55878/0 (socket says 172.21.15.39:55878) 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3cff2640 1 -- 172.21.15.39:0/1275314714 learned_addr learned my addr 172.21.15.39:0/1275314714 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3d7f3640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d3814aa40 0x7f6d381ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:09.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d37fff640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c4710 0x7f6d381ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3cff2640 1 -- 172.21.15.39:0/1275314714 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c4710 msgr2=0x7f6d381ec690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3cff2640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c4710 0x7f6d381ec690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3cff2640 1 -- 172.21.15.39:0/1275314714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d3814aa40 msgr2=0x7f6d381ecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3cff2640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d3814aa40 0x7f6d381ecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3cff2640 1 -- 172.21.15.39:0/1275314714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d30009b30 con 0x7f6d380c1750 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d37fff640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c4710 0x7f6d381ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3cff2640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c1750 0x7f6d38149a30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c7f0 tx=0x7f6d2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d357fa640 1 -- 172.21.15.39:0/1275314714 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c007820 con 0x7f6d380c1750 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d357fa640 1 -- 172.21.15.39:0/1275314714 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c007980 con 0x7f6d380c1750 2024-09-13T16:57:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3814b1d0 con 0x7f6d380c1750 2024-09-13T16:57:09.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d357fa640 1 -- 172.21.15.39:0/1275314714 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c007d30 con 0x7f6d380c1750 2024-09-13T16:57:09.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d380aa2c0 con 0x7f6d380c1750 2024-09-13T16:57:09.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.304+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d381ed410 con 0x7f6d380c1750 2024-09-13T16:57:09.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.308+0000 7f6d357fa640 1 -- 172.21.15.39:0/1275314714 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d2c024480 con 0x7f6d380c1750 2024-09-13T16:57:09.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.308+0000 7f6d357fa640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6d1c066690 0x7f6d1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:09.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.308+0000 7f6d357fa640 1 -- 172.21.15.39:0/1275314714 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f6d2c084f00 con 0x7f6d380c1750 2024-09-13T16:57:09.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.308+0000 7f6d37fff640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6d1c066690 0x7f6d1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.308+0000 7f6d37fff640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6d1c066690 0x7f6d1c068b50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6d380720a0 tx=0x7f6d24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:09.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.312+0000 7f6d357fa640 1 -- 172.21.15.39:0/1275314714 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6d2c051ce0 con 0x7f6d380c1750 2024-09-13T16:57:09.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.608+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d3815b250 con 0x7f6d380c1750 2024-09-13T16:57:09.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.608+0000 7f6d357fa640 1 -- 172.21.15.39:0/1275314714 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f6d2c01f070 con 0x7f6d380c1750 2024-09-13T16:57:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:09.611 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5955,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:09.611 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 65 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6d1c066690 msgr2=0x7f6d1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6d1c066690 0x7f6d1c068b50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6d380720a0 tx=0x7f6d24008040 comp rx=0 tx=0).stop 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c1750 msgr2=0x7f6d38149a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c1750 0x7f6d38149a30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c7f0 tx=0x7f6d2c00ccb0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 shutdown_connections 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6d1c066690 0x7f6d1c068b50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d3814aa40 0x7f6d381ecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d380c4710 0x7f6d381ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 --2- 172.21.15.39:0/1275314714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d380c1750 0x7f6d38149a30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:09.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 >> 172.21.15.39:0/1275314714 conn(0x7f6d380c3310 msgr2=0x7f6d380c38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 shutdown_connections 2024-09-13T16:57:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:09.612+0000 7f6d3f256640 1 -- 172.21.15.39:0/1275314714 wait complete. 2024-09-13T16:57:09.632 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:57:11.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.768+0000 7f406096a640 1 Processor -- start 2024-09-13T16:57:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.768+0000 7f406096a640 1 -- start start 2024-09-13T16:57:11.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f406096a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f405c14aa40 0x7f405c0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:11.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f406096a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f405c0c4710 0x7f405c15b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:11.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f406096a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405c0c76a0 con 0x7f405c0c1750 2024-09-13T16:57:11.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f406096a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405c151b30 con 0x7f405c14aa40 2024-09-13T16:57:11.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f406096a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405c151570 con 0x7f405c0c4710 2024-09-13T16:57:11.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f405a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f405c14aa40 0x7f405c0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:11.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f4053fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f405c0c1750 0x7f405c0c5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50180/0 (socket says 172.21.15.39:50180) 2024-09-13T16:57:11.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f4053fff640 1 -- 172.21.15.39:0/2354647077 learned_addr learned my addr 172.21.15.39:0/2354647077 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:11.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f405ad9d640 1 --2- 172.21.15.39:0/2354647077 >> v2:172.21.15.84:3300/0 conn(0x7f405c0c4710 0x7f405c15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f4059d9b640 1 -- 172.21.15.39:0/2354647077 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185453244 0 0) 0x7f405c0c76a0 con 0x7f405c0c1750 2024-09-13T16:57:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f405a59c640 1 -- 172.21.15.39:0/2354647077 >> v2:172.21.15.84:3300/0 conn(0x7f405c0c4710 msgr2=0x7f405c15b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f405a59c640 1 --2- 172.21.15.39:0/2354647077 >> v2:172.21.15.84:3300/0 conn(0x7f405c0c4710 0x7f405c15b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f405a59c640 1 -- 172.21.15.39:0/2354647077 >> v1:172.21.15.39:6789/0 conn(0x7f405c0c1750 legacy=0x7f405c0c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f405a59c640 1 -- 172.21.15.39:0/2354647077 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c151850 con 0x7f405c14aa40 2024-09-13T16:57:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f405a59c640 1 --2- 172.21.15.39:0/2354647077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 0x7f405c0c1210 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f405c0c24a0 tx=0x7f404c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc820a7e28f8e4af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f4059d9b640 1 -- 172.21.15.39:0/2354647077 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0ac620 con 0x7f405c14aa40 2024-09-13T16:57:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f4059d9b640 1 -- 172.21.15.39:0/2354647077 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c0ac780 con 0x7f405c14aa40 2024-09-13T16:57:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.772+0000 7f4059d9b640 1 -- 172.21.15.39:0/2354647077 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404c0acb10 con 0x7f405c14aa40 2024-09-13T16:57:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- 172.21.15.39:0/2354647077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 msgr2=0x7f405c0c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 --2- 172.21.15.39:0/2354647077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 0x7f405c0c1210 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f405c0c24a0 tx=0x7f404c01c3c0 comp rx=0 tx=0).stop 2024-09-13T16:57:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- 172.21.15.39:0/2354647077 shutdown_connections 2024-09-13T16:57:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 --2- 172.21.15.39:0/2354647077 >> v2:172.21.15.84:3300/0 conn(0x7f405c0c4710 0x7f405c15b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 --2- 172.21.15.39:0/2354647077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 0x7f405c0c1210 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:11.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- 172.21.15.39:0/2354647077 >> 172.21.15.39:0/2354647077 conn(0x7f405c0c3310 msgr2=0x7f405c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:11.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- 172.21.15.39:0/2354647077 shutdown_connections 2024-09-13T16:57:11.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- 172.21.15.39:0/2354647077 wait complete. 2024-09-13T16:57:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 Processor -- start 2024-09-13T16:57:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- start start 2024-09-13T16:57:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f405c0c1750 0x7f405c0bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405c0c4710 0x7f405c1eec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 0x7f405c1ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c0a97e0 con 0x7f405c14aa40 2024-09-13T16:57:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1ef700 con 0x7f405c0c4710 2024-09-13T16:57:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405c1f0770 con 0x7f405c0c1750 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 0x7f405c1ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 0x7f405c1ef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45344/0 (socket says 172.21.15.39:45344) 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405ad9d640 1 -- 172.21.15.39:0/4210183934 learned_addr learned my addr 172.21.15.39:0/4210183934 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f4053fff640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405c0c4710 0x7f405c1eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405a59c640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f405c0c1750 0x7f405c0bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405ad9d640 1 -- 172.21.15.39:0/4210183934 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f405c0c1750 msgr2=0x7f405c0bfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405ad9d640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f405c0c1750 0x7f405c0bfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405ad9d640 1 -- 172.21.15.39:0/4210183934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405c0c4710 msgr2=0x7f405c1eec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405ad9d640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405c0c4710 0x7f405c1eec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405ad9d640 1 -- 172.21.15.39:0/4210183934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c009420 con 0x7f405c14aa40 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f4053fff640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405c0c4710 0x7f405c1eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:57:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f405ad9d640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 0x7f405c1ef1c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4054007d20 tx=0x7f405400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f4052ffd640 1 -- 172.21.15.39:0/4210183934 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054014420 con 0x7f405c14aa40 2024-09-13T16:57:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f4052ffd640 1 -- 172.21.15.39:0/4210183934 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40540145a0 con 0x7f405c14aa40 2024-09-13T16:57:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c071890 con 0x7f405c14aa40 2024-09-13T16:57:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f4052ffd640 1 -- 172.21.15.39:0/4210183934 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054014950 con 0x7f405c14aa40 2024-09-13T16:57:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.776+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405c071e50 con 0x7f405c14aa40 2024-09-13T16:57:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.780+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405c14ae20 con 0x7f405c14aa40 2024-09-13T16:57:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.780+0000 7f4052ffd640 1 -- 172.21.15.39:0/4210183934 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4054014ab0 con 0x7f405c14aa40 2024-09-13T16:57:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.780+0000 7f4052ffd640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f403c066690 0x7f403c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.780+0000 7f4052ffd640 1 -- 172.21.15.39:0/4210183934 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f4054085110 con 0x7f405c14aa40 2024-09-13T16:57:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.780+0000 7f405a59c640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f403c066690 0x7f403c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.780+0000 7f405a59c640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f403c066690 0x7f403c068b50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f404c0ae000 tx=0x7f404c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:11.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:11.784+0000 7f4052ffd640 1 -- 172.21.15.39:0/4210183934 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4054051e10 con 0x7f405c14aa40 2024-09-13T16:57:12.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.040+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f405c0c2170 con 0x7f405c14aa40 2024-09-13T16:57:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.040+0000 7f4052ffd640 1 -- 172.21.15.39:0/4210183934 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f4054055430 con 0x7f405c14aa40 2024-09-13T16:57:12.044 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5955,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:12.045 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 65 2024-09-13T16:57:12.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f403c066690 msgr2=0x7f403c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:12.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f403c066690 0x7f403c068b50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f404c0ae000 tx=0x7f404c0a7000 comp rx=0 tx=0).stop 2024-09-13T16:57:12.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 msgr2=0x7f405c1ef1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 0x7f405c1ef1c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4054007d20 tx=0x7f405400e4b0 comp rx=0 tx=0).stop 2024-09-13T16:57:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 shutdown_connections 2024-09-13T16:57:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f403c066690 0x7f403c068b50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f405c0c4710 0x7f405c1eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405c14aa40 0x7f405c1ef1c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 --2- 172.21.15.39:0/4210183934 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f405c0c1750 0x7f405c0bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 >> 172.21.15.39:0/4210183934 conn(0x7f405c0c3310 msgr2=0x7f405c147cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 shutdown_connections 2024-09-13T16:57:12.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:12.044+0000 7f406096a640 1 -- 172.21.15.39:0/4210183934 wait complete. 2024-09-13T16:57:12.065 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:57:14.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.192+0000 7fb97e8c9640 1 Processor -- start 2024-09-13T16:57:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.192+0000 7fb97e8c9640 1 -- start start 2024-09-13T16:57:14.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb97e8c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb978071390 0x7fb978154100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb97e8c9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb978071ac0 0x7fb9780c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb97e8c9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9780c1890 con 0x7fb9780c2540 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb97e8c9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9780a9d40 con 0x7fb978071ac0 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb97e8c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb978061410 con 0x7fb978071390 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb9777fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb978071ac0 0x7fb9780c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb97ce66640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb9780c2540 0x7fb9780c6050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50194/0 (socket says 172.21.15.39:50194) 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb97ce66640 1 -- 172.21.15.39:0/806892580 learned_addr learned my addr 172.21.15.39:0/806892580 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb977fff640 1 --2- 172.21.15.39:0/806892580 >> v2:172.21.15.84:3300/0 conn(0x7fb978071390 0x7fb978154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb9777fe640 1 -- 172.21.15.39:0/806892580 >> v2:172.21.15.84:3300/0 conn(0x7fb978071390 msgr2=0x7fb978154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb9777fe640 1 --2- 172.21.15.39:0/806892580 >> v2:172.21.15.84:3300/0 conn(0x7fb978071390 0x7fb978154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb9777fe640 1 -- 172.21.15.39:0/806892580 >> v1:172.21.15.39:6789/0 conn(0x7fb9780c2540 legacy=0x7fb9780c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb9777fe640 1 -- 172.21.15.39:0/806892580 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9780616b0 con 0x7fb978071ac0 2024-09-13T16:57:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb976ffd640 1 -- 172.21.15.39:0/806892580 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234364723 0 0) 0x7fb9780c1890 con 0x7fb9780c2540 2024-09-13T16:57:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb977fff640 1 --2- 172.21.15.39:0/806892580 >> v2:172.21.15.84:3300/0 conn(0x7fb978071390 0x7fb978154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:14.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb9777fe640 1 --2- 172.21.15.39:0/806892580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071ac0 0x7fb9780c2000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb97806dde0 tx=0x7fb96c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786751f0accdc66a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:14.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb976ffd640 1 -- 172.21.15.39:0/806892580 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0ac640 con 0x7fb978071ac0 2024-09-13T16:57:14.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb976ffd640 1 -- 172.21.15.39:0/806892580 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c0ac7a0 con 0x7fb978071ac0 2024-09-13T16:57:14.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.196+0000 7fb976ffd640 1 -- 172.21.15.39:0/806892580 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0acb30 con 0x7fb978071ac0 2024-09-13T16:57:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 -- 172.21.15.39:0/806892580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071ac0 msgr2=0x7fb9780c2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 --2- 172.21.15.39:0/806892580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071ac0 0x7fb9780c2000 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb97806dde0 tx=0x7fb96c01c3c0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 -- 172.21.15.39:0/806892580 shutdown_connections 2024-09-13T16:57:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 --2- 172.21.15.39:0/806892580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071ac0 0x7fb9780c2000 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 --2- 172.21.15.39:0/806892580 >> v2:172.21.15.84:3300/0 conn(0x7fb978071390 0x7fb978154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 -- 172.21.15.39:0/806892580 >> 172.21.15.39:0/806892580 conn(0x7fb9780a8860 msgr2=0x7fb97806bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 -- 172.21.15.39:0/806892580 shutdown_connections 2024-09-13T16:57:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 -- 172.21.15.39:0/806892580 wait complete. 2024-09-13T16:57:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 Processor -- start 2024-09-13T16:57:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 -- start start 2024-09-13T16:57:14.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071390 0x7fb9781507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:14.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb978071ac0 0x7fb97813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:14.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97814e980 0x7fb97813ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:14.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97814f170 con 0x7fb978071390 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97814ee40 con 0x7fb97814e980 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97e8c9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb978146b80 con 0x7fb978071ac0 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb977fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071390 0x7fb9781507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb977fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071390 0x7fb9781507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45372/0 (socket says 172.21.15.39:45372) 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb977fff640 1 -- 172.21.15.39:0/187333010 learned_addr learned my addr 172.21.15.39:0/187333010 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97ce66640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97814e980 0x7fb97813ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb977fff640 1 -- 172.21.15.39:0/187333010 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb978071ac0 msgr2=0x7fb97813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb9777fe640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb978071ac0 0x7fb97813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb977fff640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb978071ac0 0x7fb97813fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb977fff640 1 -- 172.21.15.39:0/187333010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97814e980 msgr2=0x7fb97813ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb977fff640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97814e980 0x7fb97813ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb977fff640 1 -- 172.21.15.39:0/187333010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c009420 con 0x7fb978071390 2024-09-13T16:57:14.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb97ce66640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97814e980 0x7fb97813ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:14.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb977fff640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071390 0x7fb9781507a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb96800c7f0 tx=0x7fb96800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:14.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.200+0000 7fb974ff9640 1 -- 172.21.15.39:0/187333010 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968007820 con 0x7fb978071390 2024-09-13T16:57:14.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb974ff9640 1 -- 172.21.15.39:0/187333010 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb968007980 con 0x7fb978071390 2024-09-13T16:57:14.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9781468a0 con 0x7fb978071390 2024-09-13T16:57:14.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb974ff9640 1 -- 172.21.15.39:0/187333010 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968007d30 con 0x7fb978071390 2024-09-13T16:57:14.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97815b840 con 0x7fb978071390 2024-09-13T16:57:14.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb978072010 con 0x7fb978071390 2024-09-13T16:57:14.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb974ff9640 1 -- 172.21.15.39:0/187333010 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb9680113f0 con 0x7fb978071390 2024-09-13T16:57:14.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb974ff9640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9640666c0 0x7fb964068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:14.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb974ff9640 1 -- 172.21.15.39:0/187333010 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fb968084b60 con 0x7fb978071390 2024-09-13T16:57:14.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb9777fe640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9640666c0 0x7fb964068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:14.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.204+0000 7fb9777fe640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9640666c0 0x7fb964068b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0ae000 tx=0x7fb96c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:14.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.212+0000 7fb974ff9640 1 -- 172.21.15.39:0/187333010 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb9680517d0 con 0x7fb978071390 2024-09-13T16:57:14.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb978146e60 con 0x7fb978071390 2024-09-13T16:57:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb974ff9640 1 -- 172.21.15.39:0/187333010 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5930 (secure 0 0 0) 0x7fb968054df0 con 0x7fb978071390 2024-09-13T16:57:14.425 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:14.425 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5955,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:55:17.642071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5069},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5069":{"gid":5069,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6833/2740003107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2740003107},{"type":"v1","addr":"172.21.15.84:6833","nonce":2740003107}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:14.426 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 65 2024-09-13T16:57:14.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9640666c0 msgr2=0x7fb964068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:14.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9640666c0 0x7fb964068b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0ae000 tx=0x7fb96c008040 comp rx=0 tx=0).stop 2024-09-13T16:57:14.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071390 msgr2=0x7fb9781507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071390 0x7fb9781507a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb96800c7f0 tx=0x7fb96800ccb0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 shutdown_connections 2024-09-13T16:57:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb9640666c0 0x7fb964068b80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb97814e980 0x7fb97813ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb978071ac0 0x7fb97813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 --2- 172.21.15.39:0/187333010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb978071390 0x7fb9781507a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 >> 172.21.15.39:0/187333010 conn(0x7fb9780a8860 msgr2=0x7fb978152280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 shutdown_connections 2024-09-13T16:57:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:14.424+0000 7fb97e8c9640 1 -- 172.21.15.39:0/187333010 wait complete. 2024-09-13T16:57:14.443 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:57:16.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.596+0000 7f64f7e8c640 1 Processor -- start 2024-09-13T16:57:16.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.596+0000 7f64f7e8c640 1 -- start start 2024-09-13T16:57:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f7e8c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f64f014e800 0x7f64f01545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f7e8c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f006fb40 con 0x7f64f00c2560 2024-09-13T16:57:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f7e8c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f00706c0 con 0x7f64f00c2db0 2024-09-13T16:57:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f7e8c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f0071730 con 0x7f64f014e800 2024-09-13T16:57:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f5427640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f64f00c2560 0x7f64f014e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50198/0 (socket says 172.21.15.39:50198) 2024-09-13T16:57:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f5427640 1 -- 172.21.15.39:0/113672104 learned_addr learned my addr 172.21.15.39:0/113672104 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f6429640 1 --2- 172.21.15.39:0/113672104 >> v2:172.21.15.84:3300/0 conn(0x7f64f014e800 0x7f64f01545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149400980 0 0) 0x7f64f006fb40 con 0x7f64f00c2560 2024-09-13T16:57:16.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64dc005740 con 0x7f64f00c2560 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 55950639 0 0) 0x7f64dc005740 con 0x7f64f00c2560 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 >> v2:172.21.15.84:3300/0 conn(0x7f64f014e800 msgr2=0x7f64f01545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 --2- 172.21.15.39:0/113672104 >> v2:172.21.15.84:3300/0 conn(0x7f64f014e800 0x7f64f01545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 >> v1:172.21.15.84:6789/0 conn(0x7f64f00c2db0 legacy=0x7f64f0061200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f006fd90 con 0x7f64f00c2560 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f64e0001680 con 0x7f64f00c2560 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 >> v1:172.21.15.39:6789/0 conn(0x7f64f00c2560 legacy=0x7f64f014e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64dc005b90 0x7f64dc005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64dc006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64dc0073a0 0x7f64dc00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc00e210 con 0x7f64dc0069e0 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc00e380 con 0x7f64dc0073a0 2024-09-13T16:57:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc00e540 con 0x7f64dc005b90 2024-09-13T16:57:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f5427640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64dc006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f6429640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64dc0073a0 0x7f64dc00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f5427640 1 -- 172.21.15.39:0/113672104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64dc005b90 msgr2=0x7f64dc005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f5427640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64dc005b90 0x7f64dc005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f5427640 1 -- 172.21.15.39:0/113672104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64dc0073a0 msgr2=0x7f64dc00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f5427640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64dc0073a0 0x7f64dc00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f6429640 1 -- 172.21.15.39:0/113672104 reap_dead start 2024-09-13T16:57:16.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f5427640 1 -- 172.21.15.39:0/113672104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc005740 con 0x7f64dc0069e0 2024-09-13T16:57:16.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.600+0000 7f64f5427640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64dc006e60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c240 tx=0x7f64e001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df53d09271fe1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:16.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0005ae0 con 0x7f64dc0069e0 2024-09-13T16:57:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e00ab070 con 0x7f64dc0069e0 2024-09-13T16:57:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f4c26640 1 -- 172.21.15.39:0/113672104 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0007070 con 0x7f64dc0069e0 2024-09-13T16:57:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- 172.21.15.39:0/113672104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 msgr2=0x7f64dc006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64dc006e60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c240 tx=0x7f64e001f390 comp rx=0 tx=0).stop 2024-09-13T16:57:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- 172.21.15.39:0/113672104 shutdown_connections 2024-09-13T16:57:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 --2- 172.21.15.39:0/113672104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64dc006e60 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- 172.21.15.39:0/113672104 >> 172.21.15.39:0/113672104 conn(0x7f64f0069e10 msgr2=0x7f64f006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- 172.21.15.39:0/113672104 shutdown_connections 2024-09-13T16:57:16.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- 172.21.15.39:0/113672104 wait complete. 2024-09-13T16:57:16.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 Processor -- start 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- start start 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64f014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64f0145050 0x7f64f00aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f01e84d0 0x7f64f0144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0154140 con 0x7f64dc0069e0 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f00aaa20 con 0x7f64f01e84d0 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f01e9100 con 0x7f64f0145050 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5c28640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64f014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5c28640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64f014ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45388/0 (socket says 172.21.15.39:45388) 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5c28640 1 -- 172.21.15.39:0/4272377312 learned_addr learned my addr 172.21.15.39:0/4272377312 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5427640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64f0145050 0x7f64f00aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f6429640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f01e84d0 0x7f64f0144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5c28640 1 -- 172.21.15.39:0/4272377312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64f0145050 msgr2=0x7f64f00aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5c28640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64f0145050 0x7f64f00aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5c28640 1 -- 172.21.15.39:0/4272377312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f01e84d0 msgr2=0x7f64f0144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5c28640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f01e84d0 0x7f64f0144490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5c28640 1 -- 172.21.15.39:0/4272377312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f01e8e20 con 0x7f64dc0069e0 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f5c28640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64f014ec10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c400 tx=0x7f64ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64e67fc640 1 -- 172.21.15.39:0/4272377312 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0052d0 con 0x7f64dc0069e0 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64e67fc640 1 -- 172.21.15.39:0/4272377312 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec005430 con 0x7f64dc0069e0 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f00aa6b0 con 0x7f64dc0069e0 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64e67fc640 1 -- 172.21.15.39:0/4272377312 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec007260 con 0x7f64dc0069e0 2024-09-13T16:57:16.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.604+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f0152130 con 0x7f64dc0069e0 2024-09-13T16:57:16.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.608+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f014e0f0 con 0x7f64dc0069e0 2024-09-13T16:57:16.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.608+0000 7f64e67fc640 1 -- 172.21.15.39:0/4272377312 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64ec0073c0 con 0x7f64dc0069e0 2024-09-13T16:57:16.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.608+0000 7f64e67fc640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64d4066690 0x7f64d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.608+0000 7f64e67fc640 1 -- 172.21.15.39:0/4272377312 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7f64ec083c90 con 0x7f64dc0069e0 2024-09-13T16:57:16.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.608+0000 7f64f5427640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64d4066690 0x7f64d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.608+0000 7f64f5427640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64d4066690 0x7f64d4068b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64e00af000 tx=0x7f64e00aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:16.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.612+0000 7f64e67fc640 1 -- 172.21.15.39:0/4272377312 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f64ec0508b0 con 0x7f64dc0069e0 2024-09-13T16:57:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.836+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64f00c13d0 con 0x7f64dc0069e0 2024-09-13T16:57:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64e67fc640 1 -- 172.21.15.39:0/4272377312 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f64ec053ed0 con 0x7f64dc0069e0 2024-09-13T16:57:16.841 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:16.841 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:15.956320+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 67 2024-09-13T16:57:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64d4066690 msgr2=0x7f64d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64d4066690 0x7f64d4068b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64e00af000 tx=0x7f64e00aa000 comp rx=0 tx=0).stop 2024-09-13T16:57:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 msgr2=0x7f64f014ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64f014ec10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c400 tx=0x7f64ec00c8c0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 shutdown_connections 2024-09-13T16:57:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64d4066690 0x7f64d4068b50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64f01e84d0 0x7f64f0144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64f0145050 0x7f64f00aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 --2- 172.21.15.39:0/4272377312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64dc0069e0 0x7f64f014ec10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 >> 172.21.15.39:0/4272377312 conn(0x7f64f0069e10 msgr2=0x7f64f014d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 shutdown_connections 2024-09-13T16:57:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.840+0000 7f64f7e8c640 1 -- 172.21.15.39:0/4272377312 wait complete. 2024-09-13T16:57:16.861 INFO:tasks.mds_thrash.fs.[b]:mds.b down, removed from mdsmap 2024-09-13T16:57:16.861 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T16:57:16.861 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:16.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a9f692640 1 Processor -- start 2024-09-13T16:57:16.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a9f692640 1 -- start start 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a9f692640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1a9806cbc0 0x7f1a980bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a9f692640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a980c00b0 con 0x7f1a980611c0 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a9f692640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a98153e70 con 0x7f1a980c2db0 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a9f692640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a980c0300 con 0x7f1a9806cbc0 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a9dc2f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1a980611c0 0x7f1a981545e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50204/0 (socket says 172.21.15.39:50204) 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a9dc2f640 1 -- 172.21.15.39:0/1573774616 learned_addr learned my addr 172.21.15.39:0/1573774616 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a9cc2d640 1 --2- 172.21.15.39:0/1573774616 >> v2:172.21.15.84:3300/0 conn(0x7f1a9806cbc0 0x7f1a980bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223839906 0 0) 0x7f1a980c00b0 con 0x7f1a980611c0 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a84003530 con 0x7f1a980611c0 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 589179323 0 0) 0x7f1a84003530 con 0x7f1a980611c0 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 >> v2:172.21.15.84:3300/0 conn(0x7f1a9806cbc0 msgr2=0x7f1a980bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 --2- 172.21.15.39:0/1573774616 >> v2:172.21.15.84:3300/0 conn(0x7f1a9806cbc0 0x7f1a980bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 >> v1:172.21.15.84:6789/0 conn(0x7f1a980c2db0 legacy=0x7f1a981512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a980c05e0 con 0x7f1a980611c0 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1a90001a00 con 0x7f1a980611c0 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 >> v1:172.21.15.39:6789/0 conn(0x7f1a980611c0 legacy=0x7f1a981545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a84003980 0x7f1a84003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a840047d0 0x7f1a84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840052b0 0x7f1a8400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980c0300 con 0x7f1a840052b0 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8400c180 con 0x7f1a840047d0 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.980+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8400c340 con 0x7f1a84003980 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9dc2f640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840052b0 0x7f1a8400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9cc2d640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a840047d0 0x7f1a84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9d42e640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a84003980 0x7f1a84003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9dc2f640 1 -- 172.21.15.39:0/1573774616 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a84003980 msgr2=0x7f1a84003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:16.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9dc2f640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a84003980 0x7f1a84003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9dc2f640 1 -- 172.21.15.39:0/1573774616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a840047d0 msgr2=0x7f1a84004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:16.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9dc2f640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a840047d0 0x7f1a84004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9dc2f640 1 -- 172.21.15.39:0/1573774616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84003530 con 0x7f1a840052b0 2024-09-13T16:57:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9dc2f640 1 -- 172.21.15.39:0/1573774616 reap_dead start 2024-09-13T16:57:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9dc2f640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840052b0 0x7f1a8400b990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000a800 tx=0x7f1a9001d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fe41ac651d562b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a900072d0 con 0x7f1a840052b0 2024-09-13T16:57:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a900a8070 con 0x7f1a840052b0 2024-09-13T16:57:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a8ffff640 1 -- 172.21.15.39:0/1573774616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90007670 con 0x7f1a840052b0 2024-09-13T16:57:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9f692640 1 -- 172.21.15.39:0/1573774616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840052b0 msgr2=0x7f1a8400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9f692640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840052b0 0x7f1a8400b990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000a800 tx=0x7f1a9001d320 comp rx=0 tx=0).stop 2024-09-13T16:57:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9f692640 1 -- 172.21.15.39:0/1573774616 shutdown_connections 2024-09-13T16:57:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9f692640 1 --2- 172.21.15.39:0/1573774616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840052b0 0x7f1a8400b990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9f692640 1 -- 172.21.15.39:0/1573774616 >> 172.21.15.39:0/1573774616 conn(0x7f1a98069e10 msgr2=0x7f1a9806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9f692640 1 -- 172.21.15.39:0/1573774616 shutdown_connections 2024-09-13T16:57:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.984+0000 7f1a9f692640 1 -- 172.21.15.39:0/1573774616 wait complete. 2024-09-13T16:57:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 Processor -- start 2024-09-13T16:57:16.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 -- start start 2024-09-13T16:57:16.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a840052b0 0x7f1a980a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a980a9f40 0x7f1a98157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a980aa3d0 0x7f1a98157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a981ed530 con 0x7f1a980a9f40 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980aa920 con 0x7f1a980aa3d0 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a98157ba0 con 0x7f1a840052b0 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9cc2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a980a9f40 0x7f1a98157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9cc2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a980a9f40 0x7f1a98157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45418/0 (socket says 172.21.15.39:45418) 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9cc2d640 1 -- 172.21.15.39:0/2687388447 learned_addr learned my addr 172.21.15.39:0/2687388447 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9d42e640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a840052b0 0x7f1a980a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9dc2f640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a980aa3d0 0x7f1a98157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9cc2d640 1 -- 172.21.15.39:0/2687388447 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a840052b0 msgr2=0x7f1a980a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9cc2d640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a840052b0 0x7f1a980a9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9cc2d640 1 -- 172.21.15.39:0/2687388447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a980aa3d0 msgr2=0x7f1a98157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9cc2d640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a980aa3d0 0x7f1a98157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:16.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9cc2d640 1 -- 172.21.15.39:0/2687388447 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a98157f80 con 0x7f1a980a9f40 2024-09-13T16:57:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9dc2f640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a980aa3d0 0x7f1a98157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9cc2d640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a980a9f40 0x7f1a98157120 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1a88011560 tx=0x7f1a88011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a8dffb640 1 -- 172.21.15.39:0/2687388447 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88015930 con 0x7f1a980a9f40 2024-09-13T16:57:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a8dffb640 1 -- 172.21.15.39:0/2687388447 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a88015a90 con 0x7f1a980a9f40 2024-09-13T16:57:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a981ecf90 con 0x7f1a980a9f40 2024-09-13T16:57:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a8dffb640 1 -- 172.21.15.39:0/2687388447 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a88015e20 con 0x7f1a980a9f40 2024-09-13T16:57:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.988+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a981ed250 con 0x7f1a980a9f40 2024-09-13T16:57:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.992+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a981552c0 con 0x7f1a980a9f40 2024-09-13T16:57:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.992+0000 7f1a8dffb640 1 -- 172.21.15.39:0/2687388447 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1a8801d400 con 0x7f1a980a9f40 2024-09-13T16:57:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.992+0000 7f1a8dffb640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a7c066690 0x7f1a7c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.992+0000 7f1a8dffb640 1 -- 172.21.15.39:0/2687388447 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7f1a88085060 con 0x7f1a980a9f40 2024-09-13T16:57:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.992+0000 7f1a9d42e640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a7c066690 0x7f1a7c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.992+0000 7f1a9d42e640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a7c066690 0x7f1a7c068b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a94000c10 tx=0x7f1a94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:17.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:16.996+0000 7f1a8dffb640 1 -- 172.21.15.39:0/2687388447 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1a88051dc0 con 0x7f1a980a9f40 2024-09-13T16:57:17.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.204+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a981512b0 con 0x7f1a980a9f40 2024-09-13T16:57:17.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.204+0000 7f1a8dffb640 1 -- 172.21.15.39:0/2687388447 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5173 (secure 0 0 0) 0x7f1a880553e0 con 0x7f1a980a9f40 2024-09-13T16:57:17.207 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:17.207 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:16.966490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:17.208 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 68 2024-09-13T16:57:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a7c066690 msgr2=0x7f1a7c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a7c066690 0x7f1a7c068b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a94000c10 tx=0x7f1a94009040 comp rx=0 tx=0).stop 2024-09-13T16:57:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a980a9f40 msgr2=0x7f1a98157120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a980a9f40 0x7f1a98157120 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1a88011560 tx=0x7f1a88011a20 comp rx=0 tx=0).stop 2024-09-13T16:57:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 shutdown_connections 2024-09-13T16:57:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a7c066690 0x7f1a7c068b50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a980aa3d0 0x7f1a98157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a980a9f40 0x7f1a98157120 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 --2- 172.21.15.39:0/2687388447 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a840052b0 0x7f1a980a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:17.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 >> 172.21.15.39:0/2687388447 conn(0x7f1a98069e10 msgr2=0x7f1a9806cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:17.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 shutdown_connections 2024-09-13T16:57:17.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:17.208+0000 7f1a9f692640 1 -- 172.21.15.39:0/2687388447 wait complete. 2024-09-13T16:57:17.227 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 68, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 64, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5539": { "addr": "172.21.15.84:6835/661906662", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 661906662, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 661906662, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5539, "incarnation": 61, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 36, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:56:07.896181+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5539 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 68, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5955": { "addr": "172.21.15.39:6835/1334276441", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 1334276441, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 1334276441, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5955, "incarnation": 66, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 18 } }, "last_failure": 0, "last_failure_osd_epoch": 37, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:57:16.966490+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5955 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:57:19.228 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.364+0000 7f803a75d640 1 Processor -- start 2024-09-13T16:57:19.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.364+0000 7f803a75d640 1 -- start start 2024-09-13T16:57:19.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f803a75d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f80340c6130 0x7f80340c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f803a75d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8034157d50 0x7f803406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f803a75d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340c6510 con 0x7f80340c0240 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f803a75d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80340c1c80 con 0x7f80340c6130 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f803a75d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80341544f0 con 0x7f8034157d50 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f8033fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f80340c6130 0x7f80340c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f8033fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f80340c6130 0x7f80340c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45420/0 (socket says 172.21.15.39:45420) 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f8033fff640 1 -- 172.21.15.39:0/1742508354 learned_addr learned my addr 172.21.15.39:0/1742508354 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f8038cfa640 1 --2- 172.21.15.39:0/1742508354 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8034157d50 0x7f803406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f8033fff640 1 -- 172.21.15.39:0/1742508354 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8034157d50 msgr2=0x7f803406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f8033fff640 1 --2- 172.21.15.39:0/1742508354 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8034157d50 0x7f803406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f8033fff640 1 -- 172.21.15.39:0/1742508354 >> v1:172.21.15.84:6789/0 conn(0x7f80340c0240 legacy=0x7f8034153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.368+0000 7f8033fff640 1 -- 172.21.15.39:0/1742508354 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80341581e0 con 0x7f80340c6130 2024-09-13T16:57:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f8033fff640 1 --2- 172.21.15.39:0/1742508354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 0x7f80340c24c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8024009870 tx=0x7f802401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f756a4b4d5d3e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:19.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f8032ffd640 1 -- 172.21.15.39:0/1742508354 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240a9640 con 0x7f80340c6130 2024-09-13T16:57:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f8032ffd640 1 -- 172.21.15.39:0/1742508354 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80240a97a0 con 0x7f80340c6130 2024-09-13T16:57:19.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f8032ffd640 1 -- 172.21.15.39:0/1742508354 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80240a9b30 con 0x7f80340c6130 2024-09-13T16:57:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 -- 172.21.15.39:0/1742508354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 msgr2=0x7f80340c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 --2- 172.21.15.39:0/1742508354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 0x7f80340c24c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8024009870 tx=0x7f802401c560 comp rx=0 tx=0).stop 2024-09-13T16:57:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 -- 172.21.15.39:0/1742508354 shutdown_connections 2024-09-13T16:57:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 --2- 172.21.15.39:0/1742508354 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8034157d50 0x7f803406ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 --2- 172.21.15.39:0/1742508354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 0x7f80340c24c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 -- 172.21.15.39:0/1742508354 >> 172.21.15.39:0/1742508354 conn(0x7f80340c32a0 msgr2=0x7f80340c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 -- 172.21.15.39:0/1742508354 shutdown_connections 2024-09-13T16:57:19.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 -- 172.21.15.39:0/1742508354 wait complete. 2024-09-13T16:57:19.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 Processor -- start 2024-09-13T16:57:19.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 -- start start 2024-09-13T16:57:19.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80340c0240 0x7f803406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 0x7f803415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8034157d50 0x7f80341ec520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8034158860 con 0x7f80340c6130 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80341585c0 con 0x7f8034157d50 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f803a75d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803415e070 con 0x7f80340c0240 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f80337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 0x7f803415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f80337fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 0x7f803415d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45430/0 (socket says 172.21.15.39:45430) 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.372+0000 7f80337fe640 1 -- 172.21.15.39:0/1970589375 learned_addr learned my addr 172.21.15.39:0/1970589375 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8033fff640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80340c0240 0x7f803406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8038cfa640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8034157d50 0x7f80341ec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f80337fe640 1 -- 172.21.15.39:0/1970589375 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80340c0240 msgr2=0x7f803406d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f80337fe640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80340c0240 0x7f803406d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f80337fe640 1 -- 172.21.15.39:0/1970589375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8034157d50 msgr2=0x7f80341ec520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f80337fe640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8034157d50 0x7f80341ec520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f80337fe640 1 -- 172.21.15.39:0/1970589375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80240094d0 con 0x7f80340c6130 2024-09-13T16:57:19.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f80337fe640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 0x7f803415d2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f802800c500 tx=0x7f802800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8030ff9640 1 -- 172.21.15.39:0/1970589375 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028005220 con 0x7f80340c6130 2024-09-13T16:57:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8030ff9640 1 -- 172.21.15.39:0/1970589375 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8028005380 con 0x7f80340c6130 2024-09-13T16:57:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803415e350 con 0x7f80340c6130 2024-09-13T16:57:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8030ff9640 1 -- 172.21.15.39:0/1970589375 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028007270 con 0x7f80340c6130 2024-09-13T16:57:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80340a77d0 con 0x7f80340c6130 2024-09-13T16:57:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80340c24c0 con 0x7f80340c6130 2024-09-13T16:57:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8030ff9640 1 -- 172.21.15.39:0/1970589375 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80280073d0 con 0x7f80340c6130 2024-09-13T16:57:19.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8030ff9640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8020066690 0x7f8020068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8033fff640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8020066690 0x7f8020068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8030ff9640 1 -- 172.21.15.39:0/1970589375 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7f80280839f0 con 0x7f80340c6130 2024-09-13T16:57:19.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.376+0000 7f8033fff640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8020066690 0x7f8020068b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f80240ae000 tx=0x7f80240b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:19.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.380+0000 7f8030ff9640 1 -- 172.21.15.39:0/1970589375 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f80280506b0 con 0x7f80340c6130 2024-09-13T16:57:19.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.632+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8034153ad0 con 0x7f80340c6130 2024-09-13T16:57:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.632+0000 7f8030ff9640 1 -- 172.21.15.39:0/1970589375 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+5173 (secure 0 0 0) 0x7f8028053cd0 con 0x7f80340c6130 2024-09-13T16:57:19.636 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:19.636 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:19.636 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 69 2024-09-13T16:57:19.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8020066690 msgr2=0x7f8020068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8020066690 0x7f8020068b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f80240ae000 tx=0x7f80240b3040 comp rx=0 tx=0).stop 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 msgr2=0x7f803415d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 0x7f803415d2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f802800c500 tx=0x7f802800c9c0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 shutdown_connections 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8020066690 0x7f8020068b50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8034157d50 0x7f80341ec520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80340c6130 0x7f803415d2f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 --2- 172.21.15.39:0/1970589375 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80340c0240 0x7f803406d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 >> 172.21.15.39:0/1970589375 conn(0x7f80340c32a0 msgr2=0x7f80340c4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 shutdown_connections 2024-09-13T16:57:19.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.636+0000 7f803a75d640 1 -- 172.21.15.39:0/1970589375 wait complete. 2024-09-13T16:57:19.663 INFO:tasks.mds_thrash.fs.[b]:mds.a has gained rank=0, replacing gid=5069 2024-09-13T16:57:19.664 INFO:tasks.mds_thrash.fs.[b]:waiting for 0 secs before reviving mds.b 2024-09-13T16:57:19.664 INFO:tasks.mds_thrash.fs.[b]:reviving mds.b 2024-09-13T16:57:19.664 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T16:57:19.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T16:57:19.666 INFO:tasks.ceph.mds.b:Started 2024-09-13T16:57:19.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:19.742 INFO:tasks.ceph.mds.b.smithi084.stdout:starting mds.b at 2024-09-13T16:57:19.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa56a094640 1 Processor -- start 2024-09-13T16:57:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa56a094640 1 -- start start 2024-09-13T16:57:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa56a094640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa56414bda0 0x7fa56406f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa56a094640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5641550a0 0x7fa564155510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa56a094640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56414f6d0 con 0x7fa5640c0e40 2024-09-13T16:57:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa56a094640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa564072480 con 0x7fa56414bda0 2024-09-13T16:57:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa56a094640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56414c180 con 0x7fa5641550a0 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa5637fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa56414bda0 0x7fa56406f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa562ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa5640c0e40 0x7fa5640c1260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50206/0 (socket says 172.21.15.39:50206) 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa562ffd640 1 -- 172.21.15.39:0/4096723366 learned_addr learned my addr 172.21.15.39:0/4096723366 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa563fff640 1 --2- 172.21.15.39:0/4096723366 >> v2:172.21.15.84:3300/0 conn(0x7fa5641550a0 0x7fa564155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa5627fc640 1 -- 172.21.15.39:0/4096723366 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846537715 0 0) 0x7fa56414f6d0 con 0x7fa5640c0e40 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa5627fc640 1 -- 172.21.15.39:0/4096723366 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa54c003530 con 0x7fa5640c0e40 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa5637fe640 1 -- 172.21.15.39:0/4096723366 >> v2:172.21.15.84:3300/0 conn(0x7fa5641550a0 msgr2=0x7fa564155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa5637fe640 1 --2- 172.21.15.39:0/4096723366 >> v2:172.21.15.84:3300/0 conn(0x7fa5641550a0 0x7fa564155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa5637fe640 1 -- 172.21.15.39:0/4096723366 >> v1:172.21.15.39:6789/0 conn(0x7fa5640c0e40 legacy=0x7fa5640c1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.788+0000 7fa5637fe640 1 -- 172.21.15.39:0/4096723366 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa564071ec0 con 0x7fa56414bda0 2024-09-13T16:57:19.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa5637fe640 1 --2- 172.21.15.39:0/4096723366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56414bda0 0x7fa56406f6e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa5640bff80 tx=0x7fa55801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69aff36a966d90f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:19.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa5627fc640 1 -- 172.21.15.39:0/4096723366 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580ab600 con 0x7fa56414bda0 2024-09-13T16:57:19.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa5627fc640 1 -- 172.21.15.39:0/4096723366 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5580a7050 con 0x7fa56414bda0 2024-09-13T16:57:19.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa5627fc640 1 -- 172.21.15.39:0/4096723366 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580ab980 con 0x7fa56414bda0 2024-09-13T16:57:19.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 -- 172.21.15.39:0/4096723366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56414bda0 msgr2=0x7fa56406f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 --2- 172.21.15.39:0/4096723366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56414bda0 0x7fa56406f6e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa5640bff80 tx=0x7fa55801c3c0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 -- 172.21.15.39:0/4096723366 shutdown_connections 2024-09-13T16:57:19.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 --2- 172.21.15.39:0/4096723366 >> v2:172.21.15.84:3300/0 conn(0x7fa5641550a0 0x7fa564155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 --2- 172.21.15.39:0/4096723366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56414bda0 0x7fa56406f6e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 -- 172.21.15.39:0/4096723366 >> 172.21.15.39:0/4096723366 conn(0x7fa5640a7800 msgr2=0x7fa5640a7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:19.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 -- 172.21.15.39:0/4096723366 shutdown_connections 2024-09-13T16:57:19.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 -- 172.21.15.39:0/4096723366 wait complete. 2024-09-13T16:57:19.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 Processor -- start 2024-09-13T16:57:19.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.792+0000 7fa56a094640 1 -- start start 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa56a094640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5640c0e40 0x7fa5640a6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa56a094640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56414bda0 0x7fa56414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa56a094640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5641550a0 0x7fa56414d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa56a094640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56414f530 con 0x7fa5641550a0 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa56a094640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56414c7f0 con 0x7fa5640c0e40 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa56a094640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56414ef70 con 0x7fa56414bda0 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa563fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5641550a0 0x7fa56414d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa5637fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5640c0e40 0x7fa5640a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa563fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5641550a0 0x7fa56414d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45454/0 (socket says 172.21.15.39:45454) 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa563fff640 1 -- 172.21.15.39:0/3763456870 learned_addr learned my addr 172.21.15.39:0/3763456870 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa562ffd640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56414bda0 0x7fa56414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa563fff640 1 -- 172.21.15.39:0/3763456870 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56414bda0 msgr2=0x7fa56414e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa563fff640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56414bda0 0x7fa56414e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa563fff640 1 -- 172.21.15.39:0/3763456870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5640c0e40 msgr2=0x7fa5640a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa563fff640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5640c0e40 0x7fa5640a6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa563fff640 1 -- 172.21.15.39:0/3763456870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558009420 con 0x7fa5641550a0 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa5637fe640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5640c0e40 0x7fa5640a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa563fff640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5641550a0 0x7fa56414d1f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00efc0 tx=0x7fa55c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa54bfff640 1 -- 172.21.15.39:0/3763456870 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c014480 con 0x7fa5641550a0 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa54bfff640 1 -- 172.21.15.39:0/3763456870 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c00f040 con 0x7fa5641550a0 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56414f250 con 0x7fa5641550a0 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa54bfff640 1 -- 172.21.15.39:0/3763456870 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c014820 con 0x7fa5641550a0 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56414c3c0 con 0x7fa5641550a0 2024-09-13T16:57:19.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5640bf9a0 con 0x7fa5641550a0 2024-09-13T16:57:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa54bfff640 1 -- 172.21.15.39:0/3763456870 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa55c014980 con 0x7fa5641550a0 2024-09-13T16:57:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa54bfff640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa550066690 0x7fa550068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa54bfff640 1 -- 172.21.15.39:0/3763456870 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7fa55c085280 con 0x7fa5641550a0 2024-09-13T16:57:19.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.796+0000 7fa5637fe640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa550066690 0x7fa550068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:19.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.800+0000 7fa5637fe640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa550066690 0x7fa550068b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa5580ad000 tx=0x7fa558008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:19.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:19.804+0000 7fa54bfff640 1 -- 172.21.15.39:0/3763456870 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa5640bf9a0 con 0x7fa5641550a0 2024-09-13T16:57:19.925 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T16:57:19.924+0000 7f78c13cc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:57:20.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.052+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5640bf9a0 con 0x7fa5641550a0 2024-09-13T16:57:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.052+0000 7fa54bfff640 1 -- 172.21.15.39:0/3763456870 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5930 (secure 0 0 0) 0x7fa5640bf9a0 con 0x7fa5641550a0 2024-09-13T16:57:20.055 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:20.055 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 70 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa550066690 msgr2=0x7fa550068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa550066690 0x7fa550068b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa5580ad000 tx=0x7fa558008040 comp rx=0 tx=0).stop 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5641550a0 msgr2=0x7fa56414d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5641550a0 0x7fa56414d1f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00efc0 tx=0x7fa55c00c490 comp rx=0 tx=0).stop 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 shutdown_connections 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa550066690 0x7fa550068b50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5641550a0 0x7fa56414d1f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56414bda0 0x7fa56414e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 --2- 172.21.15.39:0/3763456870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5640c0e40 0x7fa5640a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 >> 172.21.15.39:0/3763456870 conn(0x7fa5640a7800 msgr2=0x7fa5640c1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 shutdown_connections 2024-09-13T16:57:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:20.056+0000 7fa56a094640 1 -- 172.21.15.39:0/3763456870 wait complete. 2024-09-13T16:57:20.078 INFO:tasks.mds_thrash.fs.[b]:mds.b reported in up:standby state 2024-09-13T16:57:20.079 INFO:tasks.mds_thrash.fs.[b]:waiting for 87 secs before thrashing 2024-09-13T16:57:27.071 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:57:27.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:57:46.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:46.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.287+0000 7f4455a1f640 1 Processor -- start 2024-09-13T16:57:46.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.287+0000 7f4455a1f640 1 -- start start 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f4455a1f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f445006e680 0x7f4450157df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f4455a1f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44500c15d0 con 0x7f44500c1810 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f4455a1f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44500c1f00 con 0x7f445006b080 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f4455a1f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445006d0b0 con 0x7f445006e680 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f444e7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f44500c1810 0x7f44501576e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44158/0 (socket says 172.21.15.39:44158) 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f444e7fc640 1 -- 172.21.15.39:0/693652350 learned_addr learned my addr 172.21.15.39:0/693652350 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f444f7fe640 1 --2- 172.21.15.39:0/693652350 >> v2:172.21.15.39:3300/0 conn(0x7f445006e680 0x7f4450157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f444dffb640 1 -- 172.21.15.39:0/693652350 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458500323 0 0) 0x7f44500c15d0 con 0x7f44500c1810 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f444dffb640 1 -- 172.21.15.39:0/693652350 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4438003530 con 0x7f44500c1810 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f444f7fe640 1 -- 172.21.15.39:0/693652350 >> v1:172.21.15.84:6789/0 conn(0x7f445006b080 legacy=0x7f4450151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f444f7fe640 1 -- 172.21.15.39:0/693652350 >> v1:172.21.15.39:6789/0 conn(0x7f44500c1810 legacy=0x7f44501576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.291+0000 7f444f7fe640 1 -- 172.21.15.39:0/693652350 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44500c21a0 con 0x7f445006e680 2024-09-13T16:57:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444f7fe640 1 --2- 172.21.15.39:0/693652350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006e680 0x7f4450157df0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4448009d90 tx=0x7f444801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38c63c77da757a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444dffb640 1 -- 172.21.15.39:0/693652350 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480a5920 con 0x7f445006e680 2024-09-13T16:57:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444dffb640 1 -- 172.21.15.39:0/693652350 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44480a5a80 con 0x7f445006e680 2024-09-13T16:57:46.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444dffb640 1 -- 172.21.15.39:0/693652350 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480a5df0 con 0x7f445006e680 2024-09-13T16:57:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 -- 172.21.15.39:0/693652350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006e680 msgr2=0x7f4450157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 --2- 172.21.15.39:0/693652350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006e680 0x7f4450157df0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4448009d90 tx=0x7f444801c970 comp rx=0 tx=0).stop 2024-09-13T16:57:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 -- 172.21.15.39:0/693652350 shutdown_connections 2024-09-13T16:57:46.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 --2- 172.21.15.39:0/693652350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006e680 0x7f4450157df0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 -- 172.21.15.39:0/693652350 >> 172.21.15.39:0/693652350 conn(0x7f44500c32a0 msgr2=0x7f44500c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 -- 172.21.15.39:0/693652350 shutdown_connections 2024-09-13T16:57:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 -- 172.21.15.39:0/693652350 wait complete. 2024-09-13T16:57:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 Processor -- start 2024-09-13T16:57:46.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 -- start start 2024-09-13T16:57:46.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006b080 0x7f44501f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445006e680 0x7f44500bef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44500c1810 0x7f44501eceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4450157000 con 0x7f445006b080 2024-09-13T16:57:46.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44501ec780 con 0x7f44500c1810 2024-09-13T16:57:46.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f4455a1f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44501f0c40 con 0x7f445006e680 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006b080 0x7f44501f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006b080 0x7f44501f0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36154/0 (socket says 172.21.15.39:36154) 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444effd640 1 -- 172.21.15.39:0/802127001 learned_addr learned my addr 172.21.15.39:0/802127001 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444e7fc640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445006e680 0x7f44500bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444f7fe640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44500c1810 0x7f44501eceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444effd640 1 -- 172.21.15.39:0/802127001 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445006e680 msgr2=0x7f44500bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444effd640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445006e680 0x7f44500bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444effd640 1 -- 172.21.15.39:0/802127001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44500c1810 msgr2=0x7f44501eceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444effd640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44500c1810 0x7f44501eceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444effd640 1 -- 172.21.15.39:0/802127001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4448009a70 con 0x7f445006b080 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.295+0000 7f444effd640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006b080 0x7f44501f0700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f444000c500 tx=0x7f444000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f4437fff640 1 -- 172.21.15.39:0/802127001 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440005220 con 0x7f445006b080 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f4437fff640 1 -- 172.21.15.39:0/802127001 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440005380 con 0x7f445006b080 2024-09-13T16:57:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44501eca20 con 0x7f445006b080 2024-09-13T16:57:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f4437fff640 1 -- 172.21.15.39:0/802127001 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440007270 con 0x7f445006b080 2024-09-13T16:57:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4450156760 con 0x7f445006b080 2024-09-13T16:57:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44500c5f70 con 0x7f445006b080 2024-09-13T16:57:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f4437fff640 1 -- 172.21.15.39:0/802127001 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44400073d0 con 0x7f445006b080 2024-09-13T16:57:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f4437fff640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f443c066690 0x7f443c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f4437fff640 1 -- 172.21.15.39:0/802127001 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7f4440083e20 con 0x7f445006b080 2024-09-13T16:57:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f444e7fc640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f443c066690 0x7f443c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.299+0000 7f444e7fc640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f443c066690 0x7f443c068b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f44501ea5c0 tx=0x7f4444009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:46.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.303+0000 7f4437fff640 1 -- 172.21.15.39:0/802127001 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4440050a50 con 0x7f445006b080 2024-09-13T16:57:46.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f445006b460 con 0x7f445006b080 2024-09-13T16:57:46.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4437fff640 1 -- 172.21.15.39:0/802127001 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f4440054070 con 0x7f445006b080 2024-09-13T16:57:46.529 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:46.529 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 70 2024-09-13T16:57:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f443c066690 msgr2=0x7f443c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f443c066690 0x7f443c068b50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f44501ea5c0 tx=0x7f4444009040 comp rx=0 tx=0).stop 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006b080 msgr2=0x7f44501f0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006b080 0x7f44501f0700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f444000c500 tx=0x7f444000c9c0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 shutdown_connections 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f443c066690 0x7f443c068b50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f44500c1810 0x7f44501eceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445006e680 0x7f44500bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 --2- 172.21.15.39:0/802127001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445006b080 0x7f44501f0700 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 >> 172.21.15.39:0/802127001 conn(0x7f44500c32a0 msgr2=0x7f4450150870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 shutdown_connections 2024-09-13T16:57:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.527+0000 7f4455a1f640 1 -- 172.21.15.39:0/802127001 wait complete. 2024-09-13T16:57:46.552 INFO:tasks.mds_thrash.fs.[a]:kill mds.d (rank=0) 2024-09-13T16:57:46.552 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:57:46.552 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:57:46.567 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:57:46.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:46.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f0296335640 1 Processor -- start 2024-09-13T16:57:46.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f0296335640 1 -- start start 2024-09-13T16:57:46.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f0296335640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0290065ee0 0x7f029014f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f0296335640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c0a90 0x7f02900c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f0296335640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f02900c2ef0 0x7f02900c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f0296335640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02900c53d0 con 0x7f0290065ee0 2024-09-13T16:57:46.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f0296335640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029006ddf0 con 0x7f02900c2ef0 2024-09-13T16:57:46.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f0296335640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900c5950 con 0x7f02900c0a90 2024-09-13T16:57:46.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f028ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0290065ee0 0x7f029014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f028ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0290065ee0 0x7f029014f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36164/0 (socket says 172.21.15.39:36164) 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f028ffff640 1 -- 172.21.15.39:0/405960183 learned_addr learned my addr 172.21.15.39:0/405960183 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f02948d2640 1 --2- 172.21.15.39:0/405960183 >> v2:172.21.15.84:3300/0 conn(0x7f02900c2ef0 0x7f02900c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f0287fff640 1 --2- 172.21.15.39:0/405960183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c0a90 0x7f02900c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f028ffff640 1 -- 172.21.15.39:0/405960183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c0a90 msgr2=0x7f02900c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f028ffff640 1 --2- 172.21.15.39:0/405960183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c0a90 0x7f02900c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f028ffff640 1 -- 172.21.15.39:0/405960183 >> v2:172.21.15.84:3300/0 conn(0x7f02900c2ef0 msgr2=0x7f02900c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f028ffff640 1 --2- 172.21.15.39:0/405960183 >> v2:172.21.15.84:3300/0 conn(0x7f02900c2ef0 0x7f02900c22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f028ffff640 1 -- 172.21.15.39:0/405960183 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900c2820 con 0x7f0290065ee0 2024-09-13T16:57:46.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.687+0000 7f02948d2640 1 --2- 172.21.15.39:0/405960183 >> v2:172.21.15.84:3300/0 conn(0x7f02900c2ef0 0x7f02900c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:57:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f028ffff640 1 --2- 172.21.15.39:0/405960183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 0x7f029014f230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f02900c2a10 tx=0x7f028001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3d719fa4ecd67 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f028f7fe640 1 -- 172.21.15.39:0/405960183 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800aa610 con 0x7f0290065ee0 2024-09-13T16:57:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f028f7fe640 1 -- 172.21.15.39:0/405960183 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02800a6040 con 0x7f0290065ee0 2024-09-13T16:57:46.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f028f7fe640 1 -- 172.21.15.39:0/405960183 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800aa990 con 0x7f0290065ee0 2024-09-13T16:57:46.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 -- 172.21.15.39:0/405960183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 msgr2=0x7f029014f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 --2- 172.21.15.39:0/405960183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 0x7f029014f230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f02900c2a10 tx=0x7f028001c4f0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 -- 172.21.15.39:0/405960183 shutdown_connections 2024-09-13T16:57:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 --2- 172.21.15.39:0/405960183 >> v2:172.21.15.84:3300/0 conn(0x7f02900c2ef0 0x7f02900c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 --2- 172.21.15.39:0/405960183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c0a90 0x7f02900c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 --2- 172.21.15.39:0/405960183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 0x7f029014f230 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 -- 172.21.15.39:0/405960183 >> 172.21.15.39:0/405960183 conn(0x7f02900c3310 msgr2=0x7f02900c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 -- 172.21.15.39:0/405960183 shutdown_connections 2024-09-13T16:57:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 -- 172.21.15.39:0/405960183 wait complete. 2024-09-13T16:57:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 Processor -- start 2024-09-13T16:57:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.691+0000 7f0296335640 1 -- start start 2024-09-13T16:57:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0296335640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 0x7f029014fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0296335640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02900c0a90 0x7f029014bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0296335640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c2ef0 0x7f0290153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0296335640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02901533e0 con 0x7f0290065ee0 2024-09-13T16:57:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0296335640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0290146c70 con 0x7f02900c0a90 2024-09-13T16:57:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0296335640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0290154ae0 con 0x7f02900c2ef0 2024-09-13T16:57:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 0x7f029014fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 0x7f029014fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36178/0 (socket says 172.21.15.39:36178) 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028ffff640 1 -- 172.21.15.39:0/3451551011 learned_addr learned my addr 172.21.15.39:0/3451551011 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0287fff640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02900c0a90 0x7f029014bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f02948d2640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c2ef0 0x7f0290153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028ffff640 1 -- 172.21.15.39:0/3451551011 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c2ef0 msgr2=0x7f0290153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028ffff640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c2ef0 0x7f0290153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028ffff640 1 -- 172.21.15.39:0/3451551011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02900c0a90 msgr2=0x7f029014bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028ffff640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02900c0a90 0x7f029014bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028ffff640 1 -- 172.21.15.39:0/3451551011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02800094d0 con 0x7f0290065ee0 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028ffff640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 0x7f029014fe80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f028001aba0 tx=0x7f028001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028d7fa640 1 -- 172.21.15.39:0/3451551011 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800bd5f0 con 0x7f0290065ee0 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028d7fa640 1 -- 172.21.15.39:0/3451551011 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02800bd750 con 0x7f0290065ee0 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028d7fa640 1 -- 172.21.15.39:0/3451551011 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800bdac0 con 0x7f0290065ee0 2024-09-13T16:57:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0290146ec0 con 0x7f0290065ee0 2024-09-13T16:57:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029014c080 con 0x7f0290065ee0 2024-09-13T16:57:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0290153520 con 0x7f0290065ee0 2024-09-13T16:57:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.695+0000 7f028d7fa640 1 -- 172.21.15.39:0/3451551011 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f02800b33f0 con 0x7f0290065ee0 2024-09-13T16:57:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.699+0000 7f028d7fa640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0270066690 0x7f0270068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.699+0000 7f028d7fa640 1 -- 172.21.15.39:0/3451551011 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7f028011f730 con 0x7f0290065ee0 2024-09-13T16:57:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.699+0000 7f0287fff640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0270066690 0x7f0270068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.699+0000 7f0287fff640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0270066690 0x7f0270068b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f029014e2d0 tx=0x7f0278008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:46.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.703+0000 7f028d7fa640 1 -- 172.21.15.39:0/3451551011 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0290153520 con 0x7f0290065ee0 2024-09-13T16:57:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.919+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0290153520 con 0x7f0290065ee0 2024-09-13T16:57:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.919+0000 7f028d7fa640 1 -- 172.21.15.39:0/3451551011 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f0290153520 con 0x7f0290065ee0 2024-09-13T16:57:46.924 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:46.924 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 70 2024-09-13T16:57:46.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0270066690 msgr2=0x7f0270068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0270066690 0x7f0270068b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f029014e2d0 tx=0x7f0278008040 comp rx=0 tx=0).stop 2024-09-13T16:57:46.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 msgr2=0x7f029014fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:46.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 0x7f029014fe80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f028001aba0 tx=0x7f028001ad30 comp rx=0 tx=0).stop 2024-09-13T16:57:46.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 shutdown_connections 2024-09-13T16:57:46.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0270066690 0x7f0270068b50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02900c0a90 0x7f029014bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02900c2ef0 0x7f0290153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 --2- 172.21.15.39:0/3451551011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0290065ee0 0x7f029014fe80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:46.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 >> 172.21.15.39:0/3451551011 conn(0x7f02900c3310 msgr2=0x7f02900c4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:46.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 shutdown_connections 2024-09-13T16:57:46.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:46.923+0000 7f0296335640 1 -- 172.21.15.39:0/3451551011 wait complete. 2024-09-13T16:57:46.944 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:57:48.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:49.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.079+0000 7f1cecd18640 1 Processor -- start 2024-09-13T16:57:49.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.079+0000 7f1cecd18640 1 -- start start 2024-09-13T16:57:49.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1cecd18640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce80aabc0 0x7f1ce80aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:49.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1cecd18640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1ce806c870 0x7f1ce80c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:49.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1cecd18640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1ce80c3ee0 0x7f1ce81550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1cecd18640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce8152a80 con 0x7f1ce806c870 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1cecd18640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80c30f0 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1cecd18640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8151e00 con 0x7f1ce80aabc0 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce5d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1ce806c870 0x7f1ce80c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce5d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1ce806c870 0x7f1ce80c39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36186/0 (socket says 172.21.15.39:36186) 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce5d9b640 1 -- 172.21.15.39:0/2543148692 learned_addr learned my addr 172.21.15.39:0/2543148692 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce6d9d640 1 --2- 172.21.15.39:0/2543148692 >> v2:172.21.15.84:3300/0 conn(0x7f1ce80c3ee0 0x7f1ce81550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce659c640 1 --2- 172.21.15.39:0/2543148692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce80aabc0 0x7f1ce80aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce5d9b640 1 -- 172.21.15.39:0/2543148692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce80aabc0 msgr2=0x7f1ce80aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce5d9b640 1 --2- 172.21.15.39:0/2543148692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce80aabc0 0x7f1ce80aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce5d9b640 1 -- 172.21.15.39:0/2543148692 >> v2:172.21.15.84:3300/0 conn(0x7f1ce80c3ee0 msgr2=0x7f1ce81550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:49.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce5d9b640 1 --2- 172.21.15.39:0/2543148692 >> v2:172.21.15.84:3300/0 conn(0x7f1ce80c3ee0 0x7f1ce81550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce5d9b640 1 -- 172.21.15.39:0/2543148692 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce81520e0 con 0x7f1ce806c870 2024-09-13T16:57:49.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce5d9b640 1 --2- 172.21.15.39:0/2543148692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce806c870 0x7f1ce80c39a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1ce806c5f0 tx=0x7f1cd801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dd67703fefd39e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:49.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce559a640 1 -- 172.21.15.39:0/2543148692 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd80a5750 con 0x7f1ce806c870 2024-09-13T16:57:49.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce559a640 1 -- 172.21.15.39:0/2543148692 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd80a58b0 con 0x7f1ce806c870 2024-09-13T16:57:49.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.083+0000 7f1ce559a640 1 -- 172.21.15.39:0/2543148692 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd80a5c40 con 0x7f1ce806c870 2024-09-13T16:57:49.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- 172.21.15.39:0/2543148692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce806c870 msgr2=0x7f1ce80c39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:49.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 --2- 172.21.15.39:0/2543148692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce806c870 0x7f1ce80c39a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1ce806c5f0 tx=0x7f1cd801c9c0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- 172.21.15.39:0/2543148692 shutdown_connections 2024-09-13T16:57:49.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 --2- 172.21.15.39:0/2543148692 >> v2:172.21.15.84:3300/0 conn(0x7f1ce80c3ee0 0x7f1ce81550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 --2- 172.21.15.39:0/2543148692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce806c870 0x7f1ce80c39a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 --2- 172.21.15.39:0/2543148692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce80aabc0 0x7f1ce80aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- 172.21.15.39:0/2543148692 >> 172.21.15.39:0/2543148692 conn(0x7f1ce8069e10 msgr2=0x7f1ce806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:49.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- 172.21.15.39:0/2543148692 shutdown_connections 2024-09-13T16:57:49.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- 172.21.15.39:0/2543148692 wait complete. 2024-09-13T16:57:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 Processor -- start 2024-09-13T16:57:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- start start 2024-09-13T16:57:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce806c870 0x7f1ce80a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ce80aabc0 0x7f1ce8149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce80c3ee0 0x7f1ce81498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce806fcb0 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce814cb00 con 0x7f1ce80aabc0 2024-09-13T16:57:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8149e00 con 0x7f1ce806c870 2024-09-13T16:57:49.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce80c3ee0 0x7f1ce81498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:49.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce80c3ee0 0x7f1ce81498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36198/0 (socket says 172.21.15.39:36198) 2024-09-13T16:57:49.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce6d9d640 1 -- 172.21.15.39:0/1067039061 learned_addr learned my addr 172.21.15.39:0/1067039061 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:49.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce659c640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce806c870 0x7f1ce80a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:49.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce5d9b640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ce80aabc0 0x7f1ce8149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:49.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce6d9d640 1 -- 172.21.15.39:0/1067039061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce806c870 msgr2=0x7f1ce80a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:49.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce6d9d640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce806c870 0x7f1ce80a9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce6d9d640 1 -- 172.21.15.39:0/1067039061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ce80aabc0 msgr2=0x7f1ce8149380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce6d9d640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ce80aabc0 0x7f1ce8149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce6d9d640 1 -- 172.21.15.39:0/1067039061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd8009b30 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce659c640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce806c870 0x7f1ce80a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1ce6d9d640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce80c3ee0 0x7f1ce81498c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000d560 tx=0x7f1ce000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cdeffd640 1 -- 172.21.15.39:0/1067039061 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00125f0 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cdeffd640 1 -- 172.21.15.39:0/1067039061 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce0012770 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cdeffd640 1 -- 172.21.15.39:0/1067039061 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0012b20 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce814cdc0 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.087+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce814d380 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.091+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce81550e0 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.091+0000 7f1cdeffd640 1 -- 172.21.15.39:0/1067039061 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ce0012c80 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.091+0000 7f1cdeffd640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cc8066690 0x7f1cc8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.091+0000 7f1cdeffd640 1 -- 172.21.15.39:0/1067039061 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7f1ce0084d40 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.091+0000 7f1ce659c640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cc8066690 0x7f1cc8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.091+0000 7f1ce659c640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cc8066690 0x7f1cc8068b50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000b840 tx=0x7f1cd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:49.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.095+0000 7f1cdeffd640 1 -- 172.21.15.39:0/1067039061 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1ce00519f0 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.299+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ce806c650 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.299+0000 7f1cdeffd640 1 -- 172.21.15.39:0/1067039061 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f1ce0055010 con 0x7f1ce80c3ee0 2024-09-13T16:57:49.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:49.303 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 70 2024-09-13T16:57:49.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cc8066690 msgr2=0x7f1cc8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:49.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cc8066690 0x7f1cc8068b50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000b840 tx=0x7f1cd0008040 comp rx=0 tx=0).stop 2024-09-13T16:57:49.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce80c3ee0 msgr2=0x7f1ce81498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:49.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce80c3ee0 0x7f1ce81498c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000d560 tx=0x7f1ce000da20 comp rx=0 tx=0).stop 2024-09-13T16:57:49.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 shutdown_connections 2024-09-13T16:57:49.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1cc8066690 0x7f1cc8068b50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1ce80aabc0 0x7f1ce8149380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ce80c3ee0 0x7f1ce81498c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 --2- 172.21.15.39:0/1067039061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1ce806c870 0x7f1ce80a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:49.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 >> 172.21.15.39:0/1067039061 conn(0x7f1ce8069e10 msgr2=0x7f1ce81549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:49.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 shutdown_connections 2024-09-13T16:57:49.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:49.303+0000 7f1cecd18640 1 -- 172.21.15.39:0/1067039061 wait complete. 2024-09-13T16:57:49.321 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:57:51.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.455+0000 7f8cb7382640 1 Processor -- start 2024-09-13T16:57:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.455+0000 7f8cb7382640 1 -- start start 2024-09-13T16:57:51.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb7382640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8cb00aabc0 0x7f8cb00aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:51.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb7382640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8cb006c870 0x7f8cb00c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:51.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb7382640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8cb00c3ee0 0x7f8cb01550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:51.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb7382640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb0152a80 con 0x7f8cb00c3ee0 2024-09-13T16:57:51.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb7382640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb00c30f0 con 0x7f8cb00aabc0 2024-09-13T16:57:51.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb7382640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0151e00 con 0x7f8cb006c870 2024-09-13T16:57:51.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb591f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8cb00c3ee0 0x7f8cb01550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:51.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb591f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8cb00c3ee0 0x7f8cb01550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42032/0 (socket says 172.21.15.39:42032) 2024-09-13T16:57:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb591f640 1 -- 172.21.15.39:0/1476828952 learned_addr learned my addr 172.21.15.39:0/1476828952 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb511e640 1 --2- 172.21.15.39:0/1476828952 >> v2:172.21.15.84:3300/0 conn(0x7f8cb00aabc0 0x7f8cb00aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb491d640 1 --2- 172.21.15.39:0/1476828952 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8cb006c870 0x7f8cb00c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb591f640 1 -- 172.21.15.39:0/1476828952 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8cb006c870 msgr2=0x7f8cb00c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb591f640 1 --2- 172.21.15.39:0/1476828952 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8cb006c870 0x7f8cb00c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb591f640 1 -- 172.21.15.39:0/1476828952 >> v2:172.21.15.84:3300/0 conn(0x7f8cb00aabc0 msgr2=0x7f8cb00aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb591f640 1 --2- 172.21.15.39:0/1476828952 >> v2:172.21.15.84:3300/0 conn(0x7f8cb00aabc0 0x7f8cb00aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb591f640 1 -- 172.21.15.39:0/1476828952 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb01520e0 con 0x7f8cb00c3ee0 2024-09-13T16:57:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb511e640 1 --2- 172.21.15.39:0/1476828952 >> v2:172.21.15.84:3300/0 conn(0x7f8cb00aabc0 0x7f8cb00aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:57:51.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8cb591f640 1 --2- 172.21.15.39:0/1476828952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb00c3ee0 0x7f8cb01550e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00c33a0 tx=0x7f8ca801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706d6c847e2ae75e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8ca7fff640 1 -- 172.21.15.39:0/1476828952 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80a5840 con 0x7f8cb00c3ee0 2024-09-13T16:57:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8ca7fff640 1 -- 172.21.15.39:0/1476828952 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca80a59a0 con 0x7f8cb00c3ee0 2024-09-13T16:57:51.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.459+0000 7f8ca7fff640 1 -- 172.21.15.39:0/1476828952 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80a5d30 con 0x7f8cb00c3ee0 2024-09-13T16:57:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- 172.21.15.39:0/1476828952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb00c3ee0 msgr2=0x7f8cb01550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 --2- 172.21.15.39:0/1476828952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb00c3ee0 0x7f8cb01550e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00c33a0 tx=0x7f8ca801c9c0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- 172.21.15.39:0/1476828952 shutdown_connections 2024-09-13T16:57:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 --2- 172.21.15.39:0/1476828952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb00c3ee0 0x7f8cb01550e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 --2- 172.21.15.39:0/1476828952 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8cb006c870 0x7f8cb00c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 --2- 172.21.15.39:0/1476828952 >> v2:172.21.15.84:3300/0 conn(0x7f8cb00aabc0 0x7f8cb00aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- 172.21.15.39:0/1476828952 >> 172.21.15.39:0/1476828952 conn(0x7f8cb0069e10 msgr2=0x7f8cb006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- 172.21.15.39:0/1476828952 shutdown_connections 2024-09-13T16:57:51.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- 172.21.15.39:0/1476828952 wait complete. 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 Processor -- start 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- start start 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb006c870 0x7f8cb014ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8cb00aabc0 0x7f8cb014aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cb00c3ee0 0x7f8cb014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0070310 con 0x7f8cb006c870 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0072630 con 0x7f8cb00c3ee0 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb015b3c0 con 0x7f8cb00aabc0 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb511e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb006c870 0x7f8cb014ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:51.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb511e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb006c870 0x7f8cb014ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42042/0 (socket says 172.21.15.39:42042) 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb511e640 1 -- 172.21.15.39:0/3273707288 learned_addr learned my addr 172.21.15.39:0/3273707288 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb591f640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cb00c3ee0 0x7f8cb014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb511e640 1 -- 172.21.15.39:0/3273707288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8cb00aabc0 msgr2=0x7f8cb014aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb511e640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8cb00aabc0 0x7f8cb014aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb511e640 1 -- 172.21.15.39:0/3273707288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cb00c3ee0 msgr2=0x7f8cb014af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb511e640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cb00c3ee0 0x7f8cb014af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb511e640 1 -- 172.21.15.39:0/3273707288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca8009b30 con 0x7f8cb006c870 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb591f640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cb00c3ee0 0x7f8cb014af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb511e640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb006c870 0x7f8cb014ebc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00e810 tx=0x7f8cac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8ca5ffb640 1 -- 172.21.15.39:0/3273707288 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00c8d0 con 0x7f8cb006c870 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8ca5ffb640 1 -- 172.21.15.39:0/3273707288 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac00ca30 con 0x7f8cb006c870 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb01f0c40 con 0x7f8cb006c870 2024-09-13T16:57:51.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8ca5ffb640 1 -- 172.21.15.39:0/3273707288 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00cdc0 con 0x7f8cb006c870 2024-09-13T16:57:51.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.463+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb01f0160 con 0x7f8cb006c870 2024-09-13T16:57:51.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.467+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb01550e0 con 0x7f8cb006c870 2024-09-13T16:57:51.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.467+0000 7f8ca5ffb640 1 -- 172.21.15.39:0/3273707288 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8cac013410 con 0x7f8cb006c870 2024-09-13T16:57:51.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.467+0000 7f8ca5ffb640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c94066690 0x7f8c94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:51.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.467+0000 7f8ca5ffb640 1 -- 172.21.15.39:0/3273707288 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7f8cac084670 con 0x7f8cb006c870 2024-09-13T16:57:51.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.467+0000 7f8cb491d640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c94066690 0x7f8c94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:51.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.467+0000 7f8cb491d640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c94066690 0x7f8c94068b50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0006aa0 tx=0x7f8ca0006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:51.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.471+0000 7f8ca5ffb640 1 -- 172.21.15.39:0/3273707288 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8cac051320 con 0x7f8cb006c870 2024-09-13T16:57:51.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.667+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cb006c650 con 0x7f8cb006c870 2024-09-13T16:57:51.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.667+0000 7f8ca5ffb640 1 -- 172.21.15.39:0/3273707288 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f8cac054940 con 0x7f8cb006c870 2024-09-13T16:57:51.674 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:51.674 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 70 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c94066690 msgr2=0x7f8c94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c94066690 0x7f8c94068b50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0006aa0 tx=0x7f8ca0006a30 comp rx=0 tx=0).stop 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb006c870 msgr2=0x7f8cb014ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb006c870 0x7f8cb014ebc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00e810 tx=0x7f8cac00ecd0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 shutdown_connections 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c94066690 0x7f8c94068b50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8cb00c3ee0 0x7f8cb014af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8cb00aabc0 0x7f8cb014aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 --2- 172.21.15.39:0/3273707288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb006c870 0x7f8cb014ebc0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 >> 172.21.15.39:0/3273707288 conn(0x7f8cb0069e10 msgr2=0x7f8cb006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:51.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 shutdown_connections 2024-09-13T16:57:51.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:51.671+0000 7f8cb7382640 1 -- 172.21.15.39:0/3273707288 wait complete. 2024-09-13T16:57:51.689 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:57:53.690 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:53.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84dfeb0640 1 Processor -- start 2024-09-13T16:57:53.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84dfeb0640 1 -- start start 2024-09-13T16:57:53.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84dfeb0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f84d8150f80 0x7f84d8153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:53.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84dfeb0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d80c5330 0x7f84d806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:53.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84dfeb0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d81546c0 con 0x7f84d8153e70 2024-09-13T16:57:53.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84dfeb0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d80c19a0 con 0x7f84d8150f80 2024-09-13T16:57:53.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84dfeb0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80c49f0 con 0x7f84d80c5330 2024-09-13T16:57:53.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84ddc4c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f84d8150f80 0x7f84d8153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84de44d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f84d8153e70 0x7f84d806c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41766/0 (socket says 172.21.15.39:41766) 2024-09-13T16:57:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84de44d640 1 -- 172.21.15.39:0/2121185392 learned_addr learned my addr 172.21.15.39:0/2121185392 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84dd44b640 1 --2- 172.21.15.39:0/2121185392 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d80c5330 0x7f84d806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84ddc4c640 1 -- 172.21.15.39:0/2121185392 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d80c5330 msgr2=0x7f84d806c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84dcc4a640 1 -- 172.21.15.39:0/2121185392 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712173501 0 0) 0x7f84d81546c0 con 0x7f84d8153e70 2024-09-13T16:57:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84ddc4c640 1 --2- 172.21.15.39:0/2121185392 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d80c5330 0x7f84d806c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84ddc4c640 1 -- 172.21.15.39:0/2121185392 >> v1:172.21.15.39:6789/0 conn(0x7f84d8153e70 legacy=0x7f84d806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:53.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.827+0000 7f84ddc4c640 1 -- 172.21.15.39:0/2121185392 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d81543e0 con 0x7f84d8150f80 2024-09-13T16:57:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84ddc4c640 1 --2- 172.21.15.39:0/2121185392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 0x7f84d8153930 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f84d40097c0 tx=0x7f84d401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cd5d14e600b723 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:53.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dcc4a640 1 -- 172.21.15.39:0/2121185392 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d40ab600 con 0x7f84d8150f80 2024-09-13T16:57:53.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dcc4a640 1 -- 172.21.15.39:0/2121185392 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d40a7050 con 0x7f84d8150f80 2024-09-13T16:57:53.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dcc4a640 1 -- 172.21.15.39:0/2121185392 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d40ab980 con 0x7f84d8150f80 2024-09-13T16:57:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2121185392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 msgr2=0x7f84d8153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 --2- 172.21.15.39:0/2121185392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 0x7f84d8153930 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f84d40097c0 tx=0x7f84d401c3e0 comp rx=0 tx=0).stop 2024-09-13T16:57:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2121185392 shutdown_connections 2024-09-13T16:57:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 --2- 172.21.15.39:0/2121185392 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d80c5330 0x7f84d806c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 --2- 172.21.15.39:0/2121185392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 0x7f84d8153930 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2121185392 >> 172.21.15.39:0/2121185392 conn(0x7f84d8069e10 msgr2=0x7f84d806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2121185392 shutdown_connections 2024-09-13T16:57:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2121185392 wait complete. 2024-09-13T16:57:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 Processor -- start 2024-09-13T16:57:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 -- start start 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d80c5330 0x7f84d8142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 0x7f84d8158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d8153e70 0x7f84d8151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80aa4e0 con 0x7f84d8150f80 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80ab1c0 con 0x7f84d80c5330 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dfeb0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d81ed2b0 con 0x7f84d8153e70 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dd44b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 0x7f84d8158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dd44b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 0x7f84d8158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42056/0 (socket says 172.21.15.39:42056) 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.831+0000 7f84dd44b640 1 -- 172.21.15.39:0/2723697757 learned_addr learned my addr 172.21.15.39:0/2723697757 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84de44d640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d8153e70 0x7f84d8151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:53.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ddc4c640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d80c5330 0x7f84d8142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84dd44b640 1 -- 172.21.15.39:0/2723697757 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d8153e70 msgr2=0x7f84d8151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84dd44b640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d8153e70 0x7f84d8151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84dd44b640 1 -- 172.21.15.39:0/2723697757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d80c5330 msgr2=0x7f84d8142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84dd44b640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d80c5330 0x7f84d8142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84dd44b640 1 -- 172.21.15.39:0/2723697757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d4009420 con 0x7f84d8150f80 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ddc4c640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d80c5330 0x7f84d8142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84dd44b640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 0x7f84d8158e30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f84c800d800 tx=0x7f84c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ce7fc640 1 -- 172.21.15.39:0/2723697757 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c800b6c0 con 0x7f84d8150f80 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ce7fc640 1 -- 172.21.15.39:0/2723697757 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c800b820 con 0x7f84d8150f80 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d80aa890 con 0x7f84d8150f80 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ce7fc640 1 -- 172.21.15.39:0/2723697757 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c800bb90 con 0x7f84d8150f80 2024-09-13T16:57:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d80aae50 con 0x7f84d8150f80 2024-09-13T16:57:53.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d8153930 con 0x7f84d8150f80 2024-09-13T16:57:53.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ce7fc640 1 -- 172.21.15.39:0/2723697757 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f84c800bcf0 con 0x7f84d8150f80 2024-09-13T16:57:53.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ce7fc640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84bc066690 0x7f84bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:53.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ce7fc640 1 -- 172.21.15.39:0/2723697757 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7f84c8084df0 con 0x7f84d8150f80 2024-09-13T16:57:53.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ddc4c640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84bc066690 0x7f84bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:53.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.835+0000 7f84ddc4c640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84bc066690 0x7f84bc068b50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f84d40ad000 tx=0x7f84d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:53.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:53.839+0000 7f84ce7fc640 1 -- 172.21.15.39:0/2723697757 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f84c8051aa0 con 0x7f84d8150f80 2024-09-13T16:57:54.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84d80c3c00 con 0x7f84d8150f80 2024-09-13T16:57:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84ce7fc640 1 -- 172.21.15.39:0/2723697757 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5930 (secure 0 0 0) 0x7f84c80550c0 con 0x7f84d8150f80 2024-09-13T16:57:54.042 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:54.042 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:54.042 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 70 2024-09-13T16:57:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84bc066690 msgr2=0x7f84bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84bc066690 0x7f84bc068b50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f84d40ad000 tx=0x7f84d4008040 comp rx=0 tx=0).stop 2024-09-13T16:57:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 msgr2=0x7f84d8158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 0x7f84d8158e30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f84c800d800 tx=0x7f84c800dcc0 comp rx=0 tx=0).stop 2024-09-13T16:57:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 shutdown_connections 2024-09-13T16:57:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84bc066690 0x7f84bc068b50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84d8153e70 0x7f84d8151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d8150f80 0x7f84d8158e30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 --2- 172.21.15.39:0/2723697757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84d80c5330 0x7f84d8142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 >> 172.21.15.39:0/2723697757 conn(0x7f84d8069e10 msgr2=0x7f84d80609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 shutdown_connections 2024-09-13T16:57:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:54.039+0000 7f84dfeb0640 1 -- 172.21.15.39:0/2723697757 wait complete. 2024-09-13T16:57:54.062 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:57:56.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:56.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.195+0000 7fe2e5faf640 1 Processor -- start 2024-09-13T16:57:56.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.195+0000 7fe2e5faf640 1 -- start start 2024-09-13T16:57:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2e5faf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe2e0071dd0 0x7fe2e0154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2e5faf640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e00c5a30 con 0x7fe2e00c7800 2024-09-13T16:57:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2e5faf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e01576a0 con 0x7fe2e0151500 2024-09-13T16:57:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2e5faf640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e0157980 con 0x7fe2e0071dd0 2024-09-13T16:57:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2d6ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe2e0071dd0 0x7fe2e0154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2d6ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe2e0071dd0 0x7fe2e0154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42066/0 (socket says 172.21.15.39:42066) 2024-09-13T16:57:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2d6ffd640 1 -- 172.21.15.39:0/4249801379 learned_addr learned my addr 172.21.15.39:0/4249801379 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2d6ffd640 1 -- 172.21.15.39:0/4249801379 >> v1:172.21.15.84:6789/0 conn(0x7fe2e0151500 legacy=0x7fe2e00c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2d6ffd640 1 -- 172.21.15.39:0/4249801379 >> v1:172.21.15.39:6789/0 conn(0x7fe2e00c7800 legacy=0x7fe2e01545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2d6ffd640 1 -- 172.21.15.39:0/4249801379 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0157c60 con 0x7fe2e0071dd0 2024-09-13T16:57:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2d6ffd640 1 --2- 172.21.15.39:0/4249801379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 0x7fe2e0154070 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80098a0 tx=0x7fe2c801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aea25ff79220dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2deffd640 1 -- 172.21.15.39:0/4249801379 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80ab660 con 0x7fe2e0071dd0 2024-09-13T16:57:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2deffd640 1 -- 172.21.15.39:0/4249801379 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c80ab7c0 con 0x7fe2e0071dd0 2024-09-13T16:57:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.199+0000 7fe2deffd640 1 -- 172.21.15.39:0/4249801379 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80abb50 con 0x7fe2e0071dd0 2024-09-13T16:57:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- 172.21.15.39:0/4249801379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 msgr2=0x7fe2e0154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 --2- 172.21.15.39:0/4249801379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 0x7fe2e0154070 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80098a0 tx=0x7fe2c801c440 comp rx=0 tx=0).stop 2024-09-13T16:57:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- 172.21.15.39:0/4249801379 shutdown_connections 2024-09-13T16:57:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 --2- 172.21.15.39:0/4249801379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 0x7fe2e0154070 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- 172.21.15.39:0/4249801379 >> 172.21.15.39:0/4249801379 conn(0x7fe2e00ab530 msgr2=0x7fe2e00ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- 172.21.15.39:0/4249801379 shutdown_connections 2024-09-13T16:57:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- 172.21.15.39:0/4249801379 wait complete. 2024-09-13T16:57:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 Processor -- start 2024-09-13T16:57:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- start start 2024-09-13T16:57:56.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 0x7fe2e00aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:56.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe2e00c7800 0x7fe2e0147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:56.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e0151500 0x7fe2e0151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:56.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0153c20 con 0x7fe2e0071dd0 2024-09-13T16:57:56.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0147710 con 0x7fe2e0151500 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0153940 con 0x7fe2e00c7800 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2df7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 0x7fe2e00aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2df7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 0x7fe2e00aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42078/0 (socket says 172.21.15.39:42078) 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2df7fe640 1 -- 172.21.15.39:0/2527432888 learned_addr learned my addr 172.21.15.39:0/2527432888 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2d6ffd640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe2e00c7800 0x7fe2e0147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2dffff640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e0151500 0x7fe2e0151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2df7fe640 1 -- 172.21.15.39:0/2527432888 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe2e00c7800 msgr2=0x7fe2e0147f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2df7fe640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe2e00c7800 0x7fe2e0147f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2df7fe640 1 -- 172.21.15.39:0/2527432888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e0151500 msgr2=0x7fe2e0151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2df7fe640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e0151500 0x7fe2e0151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2df7fe640 1 -- 172.21.15.39:0/2527432888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80094d0 con 0x7fe2e0071dd0 2024-09-13T16:57:56.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2dffff640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e0151500 0x7fe2e0151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:56.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2df7fe640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 0x7fe2e00aabb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000b440 tx=0x7fe2d000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:56.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2dcff9640 1 -- 172.21.15.39:0/2527432888 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0011890 con 0x7fe2e0071dd0 2024-09-13T16:57:56.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2dcff9640 1 -- 172.21.15.39:0/2527432888 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d00119f0 con 0x7fe2e0071dd0 2024-09-13T16:57:56.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2dcff9640 1 -- 172.21.15.39:0/2527432888 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0011da0 con 0x7fe2e0071dd0 2024-09-13T16:57:56.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e00a9540 con 0x7fe2e0071dd0 2024-09-13T16:57:56.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.203+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e0148470 con 0x7fe2e0071dd0 2024-09-13T16:57:56.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.207+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e00a93a0 con 0x7fe2e0071dd0 2024-09-13T16:57:56.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.207+0000 7fe2dcff9640 1 -- 172.21.15.39:0/2527432888 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2d0004020 con 0x7fe2e0071dd0 2024-09-13T16:57:56.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.207+0000 7fe2dcff9640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2c0066690 0x7fe2c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:56.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.207+0000 7fe2dcff9640 1 -- 172.21.15.39:0/2527432888 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7fe2d0084ae0 con 0x7fe2e0071dd0 2024-09-13T16:57:56.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.207+0000 7fe2d6ffd640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2c0066690 0x7fe2c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:56.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.207+0000 7fe2d6ffd640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2c0066690 0x7fe2c0068b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80af000 tx=0x7fe2c80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:56.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.211+0000 7fe2dcff9640 1 -- 172.21.15.39:0/2527432888 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe2d0051870 con 0x7fe2e0071dd0 2024-09-13T16:57:56.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.407+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2e00a93a0 con 0x7fe2e0071dd0 2024-09-13T16:57:56.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2dcff9640 1 -- 172.21.15.39:0/2527432888 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5930 (secure 0 0 0) 0x7fe2e00a93a0 con 0x7fe2e0071dd0 2024-09-13T16:57:56.413 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:56.413 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:56.413 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 70 2024-09-13T16:57:56.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2c0066690 msgr2=0x7fe2c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:56.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2c0066690 0x7fe2c0068b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80af000 tx=0x7fe2c80a7000 comp rx=0 tx=0).stop 2024-09-13T16:57:56.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 msgr2=0x7fe2e00aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 0x7fe2e00aabb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000b440 tx=0x7fe2d000b900 comp rx=0 tx=0).stop 2024-09-13T16:57:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 shutdown_connections 2024-09-13T16:57:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe2c0066690 0x7fe2c0068b50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2e0151500 0x7fe2e0151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe2e00c7800 0x7fe2e0147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 --2- 172.21.15.39:0/2527432888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e0071dd0 0x7fe2e00aabb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 >> 172.21.15.39:0/2527432888 conn(0x7fe2e00ab530 msgr2=0x7fe2e00ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 shutdown_connections 2024-09-13T16:57:56.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:56.411+0000 7fe2e5faf640 1 -- 172.21.15.39:0/2527432888 wait complete. 2024-09-13T16:57:56.433 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:57:57.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:57:57.338 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:57:58.434 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:57:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf72712640 1 Processor -- start 2024-09-13T16:57:58.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf72712640 1 -- start start 2024-09-13T16:57:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf72712640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 0x7fdf6c154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf72712640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdf6c0c7800 0x7fdf6c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf72712640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6c0c5a30 con 0x7fdf6c151500 2024-09-13T16:57:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf72712640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6c1576a0 con 0x7fdf6c0c7800 2024-09-13T16:57:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf72712640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c157980 con 0x7fdf6c071dd0 2024-09-13T16:57:58.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf6bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdf6c151500 0x7fdf6c0c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41786/0 (socket says 172.21.15.39:41786) 2024-09-13T16:57:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf6bfff640 1 -- 172.21.15.39:0/2432958878 learned_addr learned my addr 172.21.15.39:0/2432958878 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:57:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf70caf640 1 --2- 172.21.15.39:0/2432958878 >> v2:172.21.15.39:3300/0 conn(0x7fdf6c0c7800 0x7fdf6c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf6b7fe640 1 --2- 172.21.15.39:0/2432958878 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 0x7fdf6c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf6affd640 1 -- 172.21.15.39:0/2432958878 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523061034 0 0) 0x7fdf6c0c5a30 con 0x7fdf6c151500 2024-09-13T16:57:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf6affd640 1 -- 172.21.15.39:0/2432958878 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf54003530 con 0x7fdf6c151500 2024-09-13T16:57:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf70caf640 1 -- 172.21.15.39:0/2432958878 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 msgr2=0x7fdf6c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:57:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf70caf640 1 --2- 172.21.15.39:0/2432958878 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 0x7fdf6c154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf70caf640 1 -- 172.21.15.39:0/2432958878 >> v1:172.21.15.39:6789/0 conn(0x7fdf6c151500 legacy=0x7fdf6c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:58.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.567+0000 7fdf70caf640 1 -- 172.21.15.39:0/2432958878 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c157c60 con 0x7fdf6c0c7800 2024-09-13T16:57:58.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf70caf640 1 --2- 172.21.15.39:0/2432958878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c0c7800 0x7fdf6c1545b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009d90 tx=0x7fdf6401d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c1a8a32d46c2c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:58.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf6affd640 1 -- 172.21.15.39:0/2432958878 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640a5990 con 0x7fdf6c0c7800 2024-09-13T16:57:58.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf6affd640 1 -- 172.21.15.39:0/2432958878 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf640a5af0 con 0x7fdf6c0c7800 2024-09-13T16:57:58.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf6affd640 1 -- 172.21.15.39:0/2432958878 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640a5e80 con 0x7fdf6c0c7800 2024-09-13T16:57:58.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 -- 172.21.15.39:0/2432958878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c0c7800 msgr2=0x7fdf6c1545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:58.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 --2- 172.21.15.39:0/2432958878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c0c7800 0x7fdf6c1545b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009d90 tx=0x7fdf6401d310 comp rx=0 tx=0).stop 2024-09-13T16:57:58.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 -- 172.21.15.39:0/2432958878 shutdown_connections 2024-09-13T16:57:58.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 --2- 172.21.15.39:0/2432958878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c0c7800 0x7fdf6c1545b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:58.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 --2- 172.21.15.39:0/2432958878 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 0x7fdf6c154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:58.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 -- 172.21.15.39:0/2432958878 >> 172.21.15.39:0/2432958878 conn(0x7fdf6c0ab530 msgr2=0x7fdf6c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:58.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 -- 172.21.15.39:0/2432958878 shutdown_connections 2024-09-13T16:57:58.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 -- 172.21.15.39:0/2432958878 wait complete. 2024-09-13T16:57:58.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 Processor -- start 2024-09-13T16:57:58.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 -- start start 2024-09-13T16:57:58.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 0x7fdf6c0c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:58.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf6c0c7800 0x7fdf6c155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:58.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c151500 0x7fdf6c1efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:58.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c14b0e0 con 0x7fdf6c151500 2024-09-13T16:57:58.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c14b900 con 0x7fdf6c0c7800 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf72712640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c14ab20 con 0x7fdf6c071dd0 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf70caf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c151500 0x7fdf6c1efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf70caf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c151500 0x7fdf6c1efb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42094/0 (socket says 172.21.15.39:42094) 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf70caf640 1 -- 172.21.15.39:0/3503921452 learned_addr learned my addr 172.21.15.39:0/3503921452 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf6bfff640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 0x7fdf6c0c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf6b7fe640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf6c0c7800 0x7fdf6c155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf70caf640 1 -- 172.21.15.39:0/3503921452 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 msgr2=0x7fdf6c0c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf70caf640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 0x7fdf6c0c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf70caf640 1 -- 172.21.15.39:0/3503921452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf6c0c7800 msgr2=0x7fdf6c155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf70caf640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf6c0c7800 0x7fdf6c155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf70caf640 1 -- 172.21.15.39:0/3503921452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64009a70 con 0x7fdf6c151500 2024-09-13T16:57:58.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.571+0000 7fdf6b7fe640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf6c0c7800 0x7fdf6c155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:57:58.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf70caf640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c151500 0x7fdf6c1efb80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdf6401a4e0 tx=0x7fdf64019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:58.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf68ff9640 1 -- 172.21.15.39:0/3503921452 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640be070 con 0x7fdf6c151500 2024-09-13T16:57:58.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf68ff9640 1 -- 172.21.15.39:0/3503921452 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64009450 con 0x7fdf6c151500 2024-09-13T16:57:58.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf68ff9640 1 -- 172.21.15.39:0/3503921452 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640addd0 con 0x7fdf6c151500 2024-09-13T16:57:58.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6c14ae00 con 0x7fdf6c151500 2024-09-13T16:57:58.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c153a40 con 0x7fdf6c151500 2024-09-13T16:57:58.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6c0c6290 con 0x7fdf6c151500 2024-09-13T16:57:58.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf68ff9640 1 -- 172.21.15.39:0/3503921452 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf640b4040 con 0x7fdf6c151500 2024-09-13T16:57:58.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf68ff9640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf58066690 0x7fdf58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:57:58.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf68ff9640 1 -- 172.21.15.39:0/3503921452 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7583+0+0 (secure 0 0 0) 0x7fdf6411f290 con 0x7fdf6c151500 2024-09-13T16:57:58.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf6bfff640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf58066690 0x7fdf58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:57:58.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.575+0000 7fdf6bfff640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf58066690 0x7fdf58068b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c005890 tx=0x7fdf5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:57:58.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.579+0000 7fdf68ff9640 1 -- 172.21.15.39:0/3503921452 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf640ec000 con 0x7fdf6c151500 2024-09-13T16:57:58.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.783+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf6c0c4280 con 0x7fdf6c151500 2024-09-13T16:57:58.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.783+0000 7fdf68ff9640 1 -- 172.21.15.39:0/3503921452 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5930 (secure 0 0 0) 0x7fdf640ef620 con 0x7fdf6c151500 2024-09-13T16:57:58.786 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:57:58.786 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:56:07.896181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":36,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6835/661906662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":661906662},{"type":"v1","addr":"172.21.15.84:6835","nonce":661906662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:57:58.787 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 70 2024-09-13T16:57:58.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf58066690 msgr2=0x7fdf58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf58066690 0x7fdf58068b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c005890 tx=0x7fdf5c009040 comp rx=0 tx=0).stop 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c151500 msgr2=0x7fdf6c1efb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c151500 0x7fdf6c1efb80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdf6401a4e0 tx=0x7fdf64019650 comp rx=0 tx=0).stop 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 shutdown_connections 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf58066690 0x7fdf58068b50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf6c0c7800 0x7fdf6c155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf6c151500 0x7fdf6c1efb80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 --2- 172.21.15.39:0/3503921452 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf6c071dd0 0x7fdf6c0c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 >> 172.21.15.39:0/3503921452 conn(0x7fdf6c0ab530 msgr2=0x7fdf6c0ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:57:58.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 shutdown_connections 2024-09-13T16:57:58.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:57:58.787+0000 7fdf72712640 1 -- 172.21.15.39:0/3503921452 wait complete. 2024-09-13T16:57:58.805 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:58:00.806 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:00.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76fd9e640 1 Processor -- start 2024-09-13T16:58:00.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76fd9e640 1 -- start start 2024-09-13T16:58:00.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76fd9e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc768155a20 0x7fc7680c0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:00.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76fd9e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7680c7800 0x7fc7680c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:00.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76fd9e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7680be830 con 0x7fc7681595d0 2024-09-13T16:58:00.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76fd9e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7680c91a0 con 0x7fc768155a20 2024-09-13T16:58:00.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76fd9e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc768155e00 con 0x7fc7680c7800 2024-09-13T16:58:00.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76db3a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc768155a20 0x7fc7680c0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:00.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76d339640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc7681595d0 0x7fc768159a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48674/0 (socket says 172.21.15.39:48674) 2024-09-13T16:58:00.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76d339640 1 -- 172.21.15.39:0/1084938355 learned_addr learned my addr 172.21.15.39:0/1084938355 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:58:00.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76e33b640 1 --2- 172.21.15.39:0/1084938355 >> v2:172.21.15.84:3300/0 conn(0x7fc7680c7800 0x7fc7680c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:00.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76db3a640 1 -- 172.21.15.39:0/1084938355 >> v2:172.21.15.84:3300/0 conn(0x7fc7680c7800 msgr2=0x7fc7680c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:00.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76db3a640 1 --2- 172.21.15.39:0/1084938355 >> v2:172.21.15.84:3300/0 conn(0x7fc7680c7800 0x7fc7680c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:00.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76db3a640 1 -- 172.21.15.39:0/1084938355 >> v1:172.21.15.39:6789/0 conn(0x7fc7681595d0 legacy=0x7fc768159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:00.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76cb38640 1 -- 172.21.15.39:0/1084938355 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683919305 0 0) 0x7fc7680be830 con 0x7fc7681595d0 2024-09-13T16:58:00.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.943+0000 7fc76db3a640 1 -- 172.21.15.39:0/1084938355 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7680c9660 con 0x7fc768155a20 2024-09-13T16:58:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.947+0000 7fc76db3a640 1 --2- 172.21.15.39:0/1084938355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc768155a20 0x7fc7680c0300 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc7680c6d30 tx=0x7fc75c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33eaf8c7bf3afa5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:00.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.947+0000 7fc76cb38640 1 -- 172.21.15.39:0/1084938355 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0a5850 con 0x7fc768155a20 2024-09-13T16:58:00.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.947+0000 7fc76cb38640 1 -- 172.21.15.39:0/1084938355 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c0a59b0 con 0x7fc768155a20 2024-09-13T16:58:00.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.947+0000 7fc76cb38640 1 -- 172.21.15.39:0/1084938355 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0a5d40 con 0x7fc768155a20 2024-09-13T16:58:00.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 -- 172.21.15.39:0/1084938355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc768155a20 msgr2=0x7fc7680c0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:00.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 --2- 172.21.15.39:0/1084938355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc768155a20 0x7fc7680c0300 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc7680c6d30 tx=0x7fc75c01c8a0 comp rx=0 tx=0).stop 2024-09-13T16:58:00.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 -- 172.21.15.39:0/1084938355 shutdown_connections 2024-09-13T16:58:00.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 --2- 172.21.15.39:0/1084938355 >> v2:172.21.15.84:3300/0 conn(0x7fc7680c7800 0x7fc7680c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:00.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 --2- 172.21.15.39:0/1084938355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc768155a20 0x7fc7680c0300 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:00.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 -- 172.21.15.39:0/1084938355 >> 172.21.15.39:0/1084938355 conn(0x7fc7680ab530 msgr2=0x7fc7680ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:00.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 -- 172.21.15.39:0/1084938355 shutdown_connections 2024-09-13T16:58:00.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 -- 172.21.15.39:0/1084938355 wait complete. 2024-09-13T16:58:00.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 Processor -- start 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 -- start start 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7680c7800 0x7fc7680c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7681595d0 0x7fc768157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc76815a410 0x7fc76814f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681f18d0 con 0x7fc76815a410 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc768157a90 con 0x7fc7681595d0 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76fd9e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7681f08c0 con 0x7fc7680c7800 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76e33b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc76815a410 0x7fc76814f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76e33b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc76815a410 0x7fc76814f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53234/0 (socket says 172.21.15.39:53234) 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76e33b640 1 -- 172.21.15.39:0/3726895846 learned_addr learned my addr 172.21.15.39:0/3726895846 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76db3a640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7680c7800 0x7fc7680c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:00.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76d339640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7681595d0 0x7fc768157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:00.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76e33b640 1 -- 172.21.15.39:0/3726895846 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7680c7800 msgr2=0x7fc7680c0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:00.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76e33b640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7680c7800 0x7fc7680c0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:00.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76e33b640 1 -- 172.21.15.39:0/3726895846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7681595d0 msgr2=0x7fc768157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:00.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76e33b640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7681595d0 0x7fc768157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:00.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76e33b640 1 -- 172.21.15.39:0/3726895846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c009a70 con 0x7fc76815a410 2024-09-13T16:58:00.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.951+0000 7fc76e33b640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc76815a410 0x7fc76814f760 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc76000d590 tx=0x7fc76000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:00.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc75a7fc640 1 -- 172.21.15.39:0/3726895846 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7600125f0 con 0x7fc76815a410 2024-09-13T16:58:00.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc768158300 con 0x7fc76815a410 2024-09-13T16:58:00.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc75a7fc640 1 -- 172.21.15.39:0/3726895846 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc760012770 con 0x7fc76815a410 2024-09-13T16:58:00.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc75a7fc640 1 -- 172.21.15.39:0/3726895846 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760012b20 con 0x7fc76815a410 2024-09-13T16:58:00.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7681f10d0 con 0x7fc76815a410 2024-09-13T16:58:00.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7680c0300 con 0x7fc76815a410 2024-09-13T16:58:00.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc75a7fc640 1 -- 172.21.15.39:0/3726895846 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc760012c80 con 0x7fc76815a410 2024-09-13T16:58:00.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc75a7fc640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc74c066690 0x7fc74c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:00.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc75a7fc640 1 -- 172.21.15.39:0/3726895846 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7fc760084c20 con 0x7fc76815a410 2024-09-13T16:58:00.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.955+0000 7fc76db3a640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc74c066690 0x7fc74c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:00.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.959+0000 7fc76db3a640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc74c066690 0x7fc74c068b50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0af000 tx=0x7fc75c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:00.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:00.963+0000 7fc75a7fc640 1 -- 172.21.15.39:0/3726895846 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc7600518a0 con 0x7fc76815a410 2024-09-13T16:58:01.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.159+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7680c8f80 con 0x7fc76815a410 2024-09-13T16:58:01.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.159+0000 7fc75a7fc640 1 -- 172.21.15.39:0/3726895846 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+5179 (secure 0 0 0) 0x7fc760054ec0 con 0x7fc76815a410 2024-09-13T16:58:01.163 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:01.163 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:00.950181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:01.163 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 72 2024-09-13T16:58:01.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc74c066690 msgr2=0x7fc74c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:01.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc74c066690 0x7fc74c068b50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0af000 tx=0x7fc75c0a7070 comp rx=0 tx=0).stop 2024-09-13T16:58:01.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc76815a410 msgr2=0x7fc76814f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc76815a410 0x7fc76814f760 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc76000d590 tx=0x7fc76000da50 comp rx=0 tx=0).stop 2024-09-13T16:58:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 shutdown_connections 2024-09-13T16:58:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc74c066690 0x7fc74c068b50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7681595d0 0x7fc768157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc76815a410 0x7fc76814f760 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 --2- 172.21.15.39:0/3726895846 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7680c7800 0x7fc7680c0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 >> 172.21.15.39:0/3726895846 conn(0x7fc7680ab530 msgr2=0x7fc768162270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 shutdown_connections 2024-09-13T16:58:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.163+0000 7fc76fd9e640 1 -- 172.21.15.39:0/3726895846 wait complete. 2024-09-13T16:58:01.183 INFO:tasks.mds_thrash.fs.[a]:mds.d down, removed from mdsmap 2024-09-13T16:58:01.183 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T16:58:01.183 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.299+0000 7fdf11a20640 1 Processor -- start 2024-09-13T16:58:01.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.299+0000 7fdf11a20640 1 -- start start 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf11a20640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf0c0aafa0 0x7fdf0c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf11a20640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c152a80 con 0x7fdf0c0aabc0 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf11a20640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0ab410 con 0x7fdf0c06c870 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf11a20640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c0c30f0 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf0affd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdf0c0aabc0 0x7fdf0c0c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48682/0 (socket says 172.21.15.39:48682) 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf0affd640 1 -- 172.21.15.39:0/3376778070 learned_addr learned my addr 172.21.15.39:0/3376778070 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf0b7fe640 1 --2- 172.21.15.39:0/3376778070 >> v2:172.21.15.84:3300/0 conn(0x7fdf0c0aafa0 0x7fdf0c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf09ffb640 1 -- 172.21.15.39:0/3376778070 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803626565 0 0) 0x7fdefc001120 con 0x7fdf0c0aabc0 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf09ffb640 1 -- 172.21.15.39:0/3376778070 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef4003530 con 0x7fdf0c0aabc0 2024-09-13T16:58:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf0b7fe640 1 -- 172.21.15.39:0/3376778070 >> v1:172.21.15.84:6789/0 conn(0x7fdf0c06c870 legacy=0x7fdf0c0c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf0b7fe640 1 -- 172.21.15.39:0/3376778070 >> v1:172.21.15.39:6789/0 conn(0x7fdf0c0aabc0 legacy=0x7fdf0c0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:01.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf0b7fe640 1 -- 172.21.15.39:0/3376778070 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c151e00 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf0b7fe640 1 --2- 172.21.15.39:0/3376778070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0aafa0 0x7fdf0c15d480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdf040097c0 tx=0x7fdf0401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb087dcc48d78430 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:01.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf09ffb640 1 -- 172.21.15.39:0/3376778070 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04004230 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf09ffb640 1 -- 172.21.15.39:0/3376778070 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf040a7050 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.303+0000 7fdf09ffb640 1 -- 172.21.15.39:0/3376778070 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04003620 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- 172.21.15.39:0/3376778070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0aafa0 msgr2=0x7fdf0c15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:01.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 --2- 172.21.15.39:0/3376778070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0aafa0 0x7fdf0c15d480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdf040097c0 tx=0x7fdf0401c350 comp rx=0 tx=0).stop 2024-09-13T16:58:01.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- 172.21.15.39:0/3376778070 shutdown_connections 2024-09-13T16:58:01.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 --2- 172.21.15.39:0/3376778070 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0aafa0 0x7fdf0c15d480 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- 172.21.15.39:0/3376778070 >> 172.21.15.39:0/3376778070 conn(0x7fdf0c069e10 msgr2=0x7fdf0c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:01.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- 172.21.15.39:0/3376778070 shutdown_connections 2024-09-13T16:58:01.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- 172.21.15.39:0/3376778070 wait complete. 2024-09-13T16:58:01.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 Processor -- start 2024-09-13T16:58:01.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- start start 2024-09-13T16:58:01.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c06c870 0x7fdf0c1ec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c0aabc0 0x7fdf0c0a7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0aafa0 0x7fdf0c1f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1ecdf0 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c14c6a0 con 0x7fdf0c06c870 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c14c0e0 con 0x7fdf0c0aabc0 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0aafa0 0x7fdf0c1f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0aafa0 0x7fdf0c1f06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53246/0 (socket says 172.21.15.39:53246) 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0b7fe640 1 -- 172.21.15.39:0/3261232851 learned_addr learned my addr 172.21.15.39:0/3261232851 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0a7fc640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c0aabc0 0x7fdf0c0a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0affd640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c06c870 0x7fdf0c1ec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0b7fe640 1 -- 172.21.15.39:0/3261232851 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c0aabc0 msgr2=0x7fdf0c0a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0b7fe640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c0aabc0 0x7fdf0c0a7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0b7fe640 1 -- 172.21.15.39:0/3261232851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c06c870 msgr2=0x7fdf0c1ec8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0b7fe640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c06c870 0x7fdf0c1ec8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0b7fe640 1 -- 172.21.15.39:0/3261232851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04009420 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0a7fc640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c0aabc0 0x7fdf0c0a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf0b7fe640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0aafa0 0x7fdf0c1f06c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdf040097c0 tx=0x7fdf040190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdef3fff640 1 -- 172.21.15.39:0/3261232851 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04003a30 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdef3fff640 1 -- 172.21.15.39:0/3261232851 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf04003b90 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c14c3c0 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdef3fff640 1 -- 172.21.15.39:0/3261232851 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040ab440 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.307+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c1f01a0 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.311+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c1f0390 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.311+0000 7fdef3fff640 1 -- 172.21.15.39:0/3261232851 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf040b2050 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.311+0000 7fdef3fff640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef8066690 0x7fdef8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:01.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.311+0000 7fdef3fff640 1 -- 172.21.15.39:0/3261232851 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7fdf0411ec10 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.311+0000 7fdf0affd640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef8066690 0x7fdef8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:01.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.311+0000 7fdf0affd640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef8066690 0x7fdef8068b50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdefc001280 tx=0x7fdefc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:01.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.315+0000 7fdef3fff640 1 -- 172.21.15.39:0/3261232851 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf040eb980 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.515+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf0c0c39a0 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.515+0000 7fdef3fff640 1 -- 172.21.15.39:0/3261232851 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+5179 (secure 0 0 0) 0x7fdf040eefa0 con 0x7fdf0c0aafa0 2024-09-13T16:58:01.518 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:00.950181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:01.518 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 72 2024-09-13T16:58:01.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef8066690 msgr2=0x7fdef8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:01.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef8066690 0x7fdef8068b50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdefc001280 tx=0x7fdefc009040 comp rx=0 tx=0).stop 2024-09-13T16:58:01.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0aafa0 msgr2=0x7fdf0c1f06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:01.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0aafa0 0x7fdf0c1f06c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdf040097c0 tx=0x7fdf040190f0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 shutdown_connections 2024-09-13T16:58:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef8066690 0x7fdef8068b50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0aafa0 0x7fdf0c1f06c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c0aabc0 0x7fdf0c0a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 --2- 172.21.15.39:0/3261232851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c06c870 0x7fdf0c1ec8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 >> 172.21.15.39:0/3261232851 conn(0x7fdf0c069e10 msgr2=0x7fdf0c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 shutdown_connections 2024-09-13T16:58:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:01.519+0000 7fdf11a20640 1 -- 172.21.15.39:0/3261232851 wait complete. 2024-09-13T16:58:01.539 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 72, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 72, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5671": { "addr": "172.21.15.84:6833/3122796453", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 3122796453, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 3122796453, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5671, "incarnation": 71, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 12 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:58:00.950181+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5671 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 69, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5955": { "addr": "172.21.15.39:6835/1334276441", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 1334276441, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 1334276441, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5955, "incarnation": 66, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 19 } }, "last_failure": 0, "last_failure_osd_epoch": 37, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:57:17.975025+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5955 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:58:03.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:03.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.675+0000 7f63ea815640 1 Processor -- start 2024-09-13T16:58:03.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.675+0000 7f63ea815640 1 -- start start 2024-09-13T16:58:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63ea815640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63e4071fd0 0x7f63e414e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63ea815640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40ab2c0 0x7f63e41541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63ea815640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e40611c0 con 0x7f63e40c1e30 2024-09-13T16:58:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63ea815640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e40ab080 con 0x7f63e4071fd0 2024-09-13T16:58:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63ea815640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4151750 con 0x7f63e40ab2c0 2024-09-13T16:58:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e3fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f63e40c1e30 0x7f63e414de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48692/0 (socket says 172.21.15.39:48692) 2024-09-13T16:58:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e3fff640 1 -- 172.21.15.39:0/647732081 learned_addr learned my addr 172.21.15.39:0/647732081 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:58:03.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e37fe640 1 --2- 172.21.15.39:0/647732081 >> v2:172.21.15.39:3300/0 conn(0x7f63e4071fd0 0x7f63e414e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e8db2640 1 --2- 172.21.15.39:0/647732081 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40ab2c0 0x7f63e41541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e37fe640 1 -- 172.21.15.39:0/647732081 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40ab2c0 msgr2=0x7f63e41541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e37fe640 1 --2- 172.21.15.39:0/647732081 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40ab2c0 0x7f63e41541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e37fe640 1 -- 172.21.15.39:0/647732081 >> v1:172.21.15.39:6789/0 conn(0x7f63e40c1e30 legacy=0x7f63e414de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e2ffd640 1 -- 172.21.15.39:0/647732081 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557647494 0 0) 0x7f63e40611c0 con 0x7f63e40c1e30 2024-09-13T16:58:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e37fe640 1 -- 172.21.15.39:0/647732081 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e4151a30 con 0x7f63e4071fd0 2024-09-13T16:58:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e37fe640 1 --2- 172.21.15.39:0/647732081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 0x7f63e414e560 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097c0 tx=0x7f63d801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727c25669f0800ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.679+0000 7f63e2ffd640 1 -- 172.21.15.39:0/647732081 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ab600 con 0x7f63e4071fd0 2024-09-13T16:58:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e2ffd640 1 -- 172.21.15.39:0/647732081 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d80a7050 con 0x7f63e4071fd0 2024-09-13T16:58:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e2ffd640 1 -- 172.21.15.39:0/647732081 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80ab980 con 0x7f63e4071fd0 2024-09-13T16:58:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 -- 172.21.15.39:0/647732081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 msgr2=0x7f63e414e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 --2- 172.21.15.39:0/647732081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 0x7f63e414e560 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097c0 tx=0x7f63d801c410 comp rx=0 tx=0).stop 2024-09-13T16:58:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 -- 172.21.15.39:0/647732081 shutdown_connections 2024-09-13T16:58:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 --2- 172.21.15.39:0/647732081 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40ab2c0 0x7f63e41541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 --2- 172.21.15.39:0/647732081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 0x7f63e414e560 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 -- 172.21.15.39:0/647732081 >> 172.21.15.39:0/647732081 conn(0x7f63e4069e10 msgr2=0x7f63e406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 -- 172.21.15.39:0/647732081 shutdown_connections 2024-09-13T16:58:03.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 -- 172.21.15.39:0/647732081 wait complete. 2024-09-13T16:58:03.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 Processor -- start 2024-09-13T16:58:03.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 -- start start 2024-09-13T16:58:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 0x7f63e406c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63e40ab2c0 0x7f63e41edf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40c1e30 0x7f63e41ea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40c5f20 con 0x7f63e4071fd0 2024-09-13T16:58:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e41ee490 con 0x7f63e40ab2c0 2024-09-13T16:58:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63ea815640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e41408c0 con 0x7f63e40c1e30 2024-09-13T16:58:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 0x7f63e406c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 0x7f63e406c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53268/0 (socket says 172.21.15.39:53268) 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e3fff640 1 -- 172.21.15.39:0/2254714061 learned_addr learned my addr 172.21.15.39:0/2254714061 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e8db2640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40c1e30 0x7f63e41ea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e37fe640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63e40ab2c0 0x7f63e41edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e3fff640 1 -- 172.21.15.39:0/2254714061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40c1e30 msgr2=0x7f63e41ea7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e3fff640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40c1e30 0x7f63e41ea7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e3fff640 1 -- 172.21.15.39:0/2254714061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63e40ab2c0 msgr2=0x7f63e41edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e3fff640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63e40ab2c0 0x7f63e41edf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e3fff640 1 -- 172.21.15.39:0/2254714061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8009420 con 0x7f63e4071fd0 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.683+0000 7f63e3fff640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 0x7f63e406c520 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f63d40011f0 tx=0x7f63d400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63e0ff9640 1 -- 172.21.15.39:0/2254714061 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d40074e0 con 0x7f63e4071fd0 2024-09-13T16:58:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63e0ff9640 1 -- 172.21.15.39:0/2254714061 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d4007640 con 0x7f63e4071fd0 2024-09-13T16:58:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e41ea030 con 0x7f63e4071fd0 2024-09-13T16:58:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63e0ff9640 1 -- 172.21.15.39:0/2254714061 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4004a10 con 0x7f63e4071fd0 2024-09-13T16:58:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e4153120 con 0x7f63e4071fd0 2024-09-13T16:58:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e4152bc0 con 0x7f63e4071fd0 2024-09-13T16:58:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63e0ff9640 1 -- 172.21.15.39:0/2254714061 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63d4004b70 con 0x7f63e4071fd0 2024-09-13T16:58:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63e0ff9640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63d00666e0 0x7f63d0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63e0ff9640 1 -- 172.21.15.39:0/2254714061 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7f63d4083e60 con 0x7f63e4071fd0 2024-09-13T16:58:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63e37fe640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63d00666e0 0x7f63d0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.687+0000 7f63e37fe640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63d00666e0 0x7f63d0068ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f63d80ad000 tx=0x7f63d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:03.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.691+0000 7f63e0ff9640 1 -- 172.21.15.39:0/2254714061 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f63d4050b90 con 0x7f63e4071fd0 2024-09-13T16:58:03.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.931+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63e40a9380 con 0x7f63e4071fd0 2024-09-13T16:58:03.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63e0ff9640 1 -- 172.21.15.39:0/2254714061 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f63d4018070 con 0x7f63e4071fd0 2024-09-13T16:58:03.937 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:03.937 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:03.937 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 74 2024-09-13T16:58:03.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63d00666e0 msgr2=0x7f63d0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:03.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63d00666e0 0x7f63d0068ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f63d80ad000 tx=0x7f63d8008040 comp rx=0 tx=0).stop 2024-09-13T16:58:03.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 msgr2=0x7f63e406c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:03.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 0x7f63e406c520 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f63d40011f0 tx=0x7f63d400c9b0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 shutdown_connections 2024-09-13T16:58:03.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63d00666e0 0x7f63d0068ba0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63e40ab2c0 0x7f63e41edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63e40c1e30 0x7f63e41ea7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 --2- 172.21.15.39:0/2254714061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e4071fd0 0x7f63e406c520 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 >> 172.21.15.39:0/2254714061 conn(0x7f63e4069e10 msgr2=0x7f63e414b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 shutdown_connections 2024-09-13T16:58:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:03.935+0000 7f63ea815640 1 -- 172.21.15.39:0/2254714061 wait complete. 2024-09-13T16:58:03.959 INFO:tasks.mds_thrash.fs.[a]:mds.b has gained rank=0, replacing gid=5539 2024-09-13T16:58:03.959 INFO:tasks.mds_thrash.fs.[a]:waiting for 9 secs before reviving mds.d 2024-09-13T16:58:04.989 INFO:tasks.ceph.mds.b.smithi084.stderr:2024-09-13T16:58:04.990+0000 7fe6f7b4e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:58:12.960 INFO:tasks.mds_thrash.fs.[a]:reviving mds.d 2024-09-13T16:58:12.961 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T16:58:12.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T16:58:12.964 INFO:tasks.ceph.mds.d:Started 2024-09-13T16:58:12.964 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:13.047 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T16:58:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea324d640 1 Processor -- start 2024-09-13T16:58:13.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea324d640 1 -- start start 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea324d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fae9c151500 0x7fae9c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea324d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fae9c071dd0 0x7fae9c154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea324d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c0c47c0 0x7fae9c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea324d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c0c5a30 con 0x7fae9c071dd0 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea324d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c1576a0 con 0x7fae9c151500 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea324d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c157980 con 0x7fae9c0c47c0 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7fae9bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fae9c071dd0 0x7fae9c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7fae9bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fae9c071dd0 0x7fae9c154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52740/0 (socket says 172.21.15.39:52740) 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7fae9bfff640 1 -- 172.21.15.39:0/4271473599 learned_addr learned my addr 172.21.15.39:0/4271473599 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:13.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea17ea640 1 --2- 172.21.15.39:0/4271473599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c0c47c0 0x7fae9c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.111+0000 7faea0fe9640 1 --2- 172.21.15.39:0/4271473599 >> v2:172.21.15.84:3300/0 conn(0x7fae9c151500 0x7fae9c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7fae9bfff640 1 -- 172.21.15.39:0/4271473599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c0c47c0 msgr2=0x7fae9c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7fae9bfff640 1 --2- 172.21.15.39:0/4271473599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c0c47c0 0x7fae9c1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7fae9bfff640 1 -- 172.21.15.39:0/4271473599 >> v2:172.21.15.84:3300/0 conn(0x7fae9c151500 msgr2=0x7fae9c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7fae9bfff640 1 --2- 172.21.15.39:0/4271473599 >> v2:172.21.15.84:3300/0 conn(0x7fae9c151500 0x7fae9c0c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7fae9bfff640 1 -- 172.21.15.39:0/4271473599 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c157c60 con 0x7fae9c071dd0 2024-09-13T16:58:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7fae9bfff640 1 --2- 172.21.15.39:0/4271473599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c071dd0 0x7fae9c154070 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fae88009870 tx=0x7fae8801cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2f96bb8be64075 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7fae9b7fe640 1 -- 172.21.15.39:0/4271473599 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880ab640 con 0x7fae9c071dd0 2024-09-13T16:58:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7fae9b7fe640 1 -- 172.21.15.39:0/4271473599 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae880a7070 con 0x7fae9c071dd0 2024-09-13T16:58:13.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7fae9b7fe640 1 -- 172.21.15.39:0/4271473599 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880ab9c0 con 0x7fae9c071dd0 2024-09-13T16:58:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7faea324d640 1 -- 172.21.15.39:0/4271473599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c071dd0 msgr2=0x7fae9c154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7faea324d640 1 --2- 172.21.15.39:0/4271473599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c071dd0 0x7fae9c154070 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fae88009870 tx=0x7fae8801cec0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7faea324d640 1 -- 172.21.15.39:0/4271473599 shutdown_connections 2024-09-13T16:58:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7faea324d640 1 --2- 172.21.15.39:0/4271473599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c0c47c0 0x7fae9c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7faea324d640 1 --2- 172.21.15.39:0/4271473599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c071dd0 0x7fae9c154070 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7faea324d640 1 --2- 172.21.15.39:0/4271473599 >> v2:172.21.15.84:3300/0 conn(0x7fae9c151500 0x7fae9c0c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7faea324d640 1 -- 172.21.15.39:0/4271473599 >> 172.21.15.39:0/4271473599 conn(0x7fae9c0ab530 msgr2=0x7fae9c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7faea324d640 1 -- 172.21.15.39:0/4271473599 shutdown_connections 2024-09-13T16:58:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.115+0000 7faea324d640 1 -- 172.21.15.39:0/4271473599 wait complete. 2024-09-13T16:58:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 Processor -- start 2024-09-13T16:58:13.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 -- start start 2024-09-13T16:58:13.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c071dd0 0x7fae9c14bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:13.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c0c47c0 0x7fae9c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae9c151500 0x7fae9c0aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0acdf0 con 0x7fae9c0c47c0 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c1ed890 con 0x7fae9c151500 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c156ee0 con 0x7fae9c071dd0 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae9bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c0c47c0 0x7fae9c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae9bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c0c47c0 0x7fae9c152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52746/0 (socket says 172.21.15.39:52746) 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae9bfff640 1 -- 172.21.15.39:0/3764876405 learned_addr learned my addr 172.21.15.39:0/3764876405 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea0fe9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c071dd0 0x7fae9c14bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea17ea640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae9c151500 0x7fae9c0aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae9bfff640 1 -- 172.21.15.39:0/3764876405 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c071dd0 msgr2=0x7fae9c14bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae9bfff640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c071dd0 0x7fae9c14bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae9bfff640 1 -- 172.21.15.39:0/3764876405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae9c151500 msgr2=0x7fae9c0aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:13.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae9bfff640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae9c151500 0x7fae9c0aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae9bfff640 1 -- 172.21.15.39:0/3764876405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae880094d0 con 0x7fae9c0c47c0 2024-09-13T16:58:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae9bfff640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c0c47c0 0x7fae9c152c20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fae8801cd00 tx=0x7fae8801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae997fa640 1 -- 172.21.15.39:0/3764876405 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880abe80 con 0x7fae9c0c47c0 2024-09-13T16:58:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae997fa640 1 -- 172.21.15.39:0/3764876405 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae880be3f0 con 0x7fae9c0c47c0 2024-09-13T16:58:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c1eddc0 con 0x7fae9c0c47c0 2024-09-13T16:58:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7fae997fa640 1 -- 172.21.15.39:0/3764876405 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880be780 con 0x7fae9c0c47c0 2024-09-13T16:58:13.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c1edae0 con 0x7fae9c0c47c0 2024-09-13T16:58:13.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.119+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c157070 con 0x7fae9c0c47c0 2024-09-13T16:58:13.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.123+0000 7fae997fa640 1 -- 172.21.15.39:0/3764876405 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae880c7440 con 0x7fae9c0c47c0 2024-09-13T16:58:13.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.123+0000 7fae997fa640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae80066690 0x7fae80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:13.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.123+0000 7fae997fa640 1 -- 172.21.15.39:0/3764876405 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7fae8811f0f0 con 0x7fae9c0c47c0 2024-09-13T16:58:13.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.123+0000 7faea0fe9640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae80066690 0x7fae80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:13.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.123+0000 7faea0fe9640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae80066690 0x7fae80068b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fae900079c0 tx=0x7fae90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:13.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.127+0000 7fae997fa640 1 -- 172.21.15.39:0/3764876405 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fae880ebdf0 con 0x7fae9c0c47c0 2024-09-13T16:58:13.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae9c15e2a0 con 0x7fae9c0c47c0 2024-09-13T16:58:13.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7fae997fa640 1 -- 172.21.15.39:0/3764876405 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5176 (secure 0 0 0) 0x7fae880ef410 con 0x7fae9c0c47c0 2024-09-13T16:58:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:13.352 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:13.353 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 74 2024-09-13T16:58:13.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae80066690 msgr2=0x7fae80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:13.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae80066690 0x7fae80068b50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fae900079c0 tx=0x7fae90008040 comp rx=0 tx=0).stop 2024-09-13T16:58:13.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c0c47c0 msgr2=0x7fae9c152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:13.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c0c47c0 0x7fae9c152c20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fae8801cd00 tx=0x7fae8801aca0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 shutdown_connections 2024-09-13T16:58:13.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fae80066690 0x7fae80068b50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fae9c151500 0x7fae9c0aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae9c0c47c0 0x7fae9c152c20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 --2- 172.21.15.39:0/3764876405 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fae9c071dd0 0x7fae9c14bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:13.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 >> 172.21.15.39:0/3764876405 conn(0x7fae9c0ab530 msgr2=0x7fae9c0c8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:13.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 shutdown_connections 2024-09-13T16:58:13.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:13.347+0000 7faea324d640 1 -- 172.21.15.39:0/3764876405 wait complete. 2024-09-13T16:58:13.370 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-09-13T16:58:15.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:15.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec51dbe640 1 Processor -- start 2024-09-13T16:58:15.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec51dbe640 1 -- start start 2024-09-13T16:58:15.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec51dbe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fec4c151500 0x7fec4c0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:15.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec51dbe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 0x7fec4c154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:15.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec51dbe640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4c0c5a30 con 0x7fec4c071dd0 2024-09-13T16:58:15.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec51dbe640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4c1576a0 con 0x7fec4c151500 2024-09-13T16:58:15.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec51dbe640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c157980 con 0x7fec4c0c47c0 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec42ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fec4c071dd0 0x7fec4c154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49242/0 (socket says 172.21.15.39:49242) 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec42ffd640 1 -- 172.21.15.39:0/573601894 learned_addr learned my addr 172.21.15.39:0/573601894 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec4b7fe640 1 --2- 172.21.15.39:0/573601894 >> v2:172.21.15.39:3300/0 conn(0x7fec4c151500 0x7fec4c0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec4bfff640 1 --2- 172.21.15.39:0/573601894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 0x7fec4c154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec4affd640 1 -- 172.21.15.39:0/573601894 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925639221 0 0) 0x7fec4c0c5a30 con 0x7fec4c071dd0 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec4affd640 1 -- 172.21.15.39:0/573601894 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec38003530 con 0x7fec4c071dd0 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec4b7fe640 1 -- 172.21.15.39:0/573601894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 msgr2=0x7fec4c154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec4b7fe640 1 --2- 172.21.15.39:0/573601894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 0x7fec4c154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec4b7fe640 1 -- 172.21.15.39:0/573601894 >> v1:172.21.15.39:6789/0 conn(0x7fec4c071dd0 legacy=0x7fec4c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:15.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.519+0000 7fec4b7fe640 1 -- 172.21.15.39:0/573601894 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c157c60 con 0x7fec4c151500 2024-09-13T16:58:15.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4b7fe640 1 --2- 172.21.15.39:0/573601894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c151500 0x7fec4c0c4280 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0097f0 tx=0x7fec3c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93f168254c4146e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:15.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4affd640 1 -- 172.21.15.39:0/573601894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0ab600 con 0x7fec4c151500 2024-09-13T16:58:15.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4affd640 1 -- 172.21.15.39:0/573601894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c0ab760 con 0x7fec4c151500 2024-09-13T16:58:15.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4affd640 1 -- 172.21.15.39:0/573601894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0abaf0 con 0x7fec4c151500 2024-09-13T16:58:15.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 -- 172.21.15.39:0/573601894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c151500 msgr2=0x7fec4c0c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:15.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 --2- 172.21.15.39:0/573601894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c151500 0x7fec4c0c4280 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0097f0 tx=0x7fec3c01c3e0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 -- 172.21.15.39:0/573601894 shutdown_connections 2024-09-13T16:58:15.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 --2- 172.21.15.39:0/573601894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 0x7fec4c154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 --2- 172.21.15.39:0/573601894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c151500 0x7fec4c0c4280 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 -- 172.21.15.39:0/573601894 >> 172.21.15.39:0/573601894 conn(0x7fec4c0ab530 msgr2=0x7fec4c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:15.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 -- 172.21.15.39:0/573601894 shutdown_connections 2024-09-13T16:58:15.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 -- 172.21.15.39:0/573601894 wait complete. 2024-09-13T16:58:15.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 Processor -- start 2024-09-13T16:58:15.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 -- start start 2024-09-13T16:58:15.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c071dd0 0x7fec4c14b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:15.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 0x7fec4c1ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:15.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4c151500 0x7fec4c1efc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:15.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c14c610 con 0x7fec4c071dd0 2024-09-13T16:58:15.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c152dc0 con 0x7fec4c151500 2024-09-13T16:58:15.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec51dbe640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c1530a0 con 0x7fec4c0c47c0 2024-09-13T16:58:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c071dd0 0x7fec4c14b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c071dd0 0x7fec4c14b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52762/0 (socket says 172.21.15.39:52762) 2024-09-13T16:58:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4b7fe640 1 -- 172.21.15.39:0/272952472 learned_addr learned my addr 172.21.15.39:0/272952472 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec42ffd640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 0x7fec4c1ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4bfff640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4c151500 0x7fec4c1efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4b7fe640 1 -- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 msgr2=0x7fec4c1ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4b7fe640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 0x7fec4c1ef750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4b7fe640 1 -- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4c151500 msgr2=0x7fec4c1efc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4b7fe640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4c151500 0x7fec4c1efc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4b7fe640 1 -- 172.21.15.39:0/272952472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c009420 con 0x7fec4c071dd0 2024-09-13T16:58:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec4bfff640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4c151500 0x7fec4c1efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:58:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.523+0000 7fec42ffd640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 0x7fec4c1ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:58:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec4b7fe640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c071dd0 0x7fec4c14b540 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0ad040 tx=0x7fec3c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec48ff9640 1 -- 172.21.15.39:0/272952472 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0bf610 con 0x7fec4c071dd0 2024-09-13T16:58:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec48ff9640 1 -- 172.21.15.39:0/272952472 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c0bf790 con 0x7fec4c071dd0 2024-09-13T16:58:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c153380 con 0x7fec4c071dd0 2024-09-13T16:58:15.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec48ff9640 1 -- 172.21.15.39:0/272952472 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0bfb40 con 0x7fec4c071dd0 2024-09-13T16:58:15.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c153940 con 0x7fec4c071dd0 2024-09-13T16:58:15.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c0c4280 con 0x7fec4c071dd0 2024-09-13T16:58:15.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec48ff9640 1 -- 172.21.15.39:0/272952472 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec3c0bfca0 con 0x7fec4c071dd0 2024-09-13T16:58:15.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec48ff9640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec2c066690 0x7fec2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:15.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec48ff9640 1 -- 172.21.15.39:0/272952472 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7fec3c11f400 con 0x7fec4c071dd0 2024-09-13T16:58:15.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec42ffd640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec2c066690 0x7fec2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:15.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.527+0000 7fec42ffd640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec2c066690 0x7fec2c068b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fec4c1ed2a0 tx=0x7fec34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:15.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.531+0000 7fec48ff9640 1 -- 172.21.15.39:0/272952472 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fec3c0ec170 con 0x7fec4c071dd0 2024-09-13T16:58:15.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.739+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec4c0c3c90 con 0x7fec4c071dd0 2024-09-13T16:58:15.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.739+0000 7fec48ff9640 1 -- 172.21.15.39:0/272952472 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fec3c0ef790 con 0x7fec4c071dd0 2024-09-13T16:58:15.743 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:15.743 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:15.743 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-13T16:58:15.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec2c066690 msgr2=0x7fec2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:15.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec2c066690 0x7fec2c068b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fec4c1ed2a0 tx=0x7fec34009040 comp rx=0 tx=0).stop 2024-09-13T16:58:15.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c071dd0 msgr2=0x7fec4c14b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:15.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c071dd0 0x7fec4c14b540 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0ad040 tx=0x7fec3c01ac20 comp rx=0 tx=0).stop 2024-09-13T16:58:15.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 shutdown_connections 2024-09-13T16:58:15.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec2c066690 0x7fec2c068b50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec4c151500 0x7fec4c1efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fec4c0c47c0 0x7fec4c1ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 --2- 172.21.15.39:0/272952472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec4c071dd0 0x7fec4c14b540 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:15.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 >> 172.21.15.39:0/272952472 conn(0x7fec4c0ab530 msgr2=0x7fec4c070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:15.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 shutdown_connections 2024-09-13T16:58:15.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:15.743+0000 7fec51dbe640 1 -- 172.21.15.39:0/272952472 wait complete. 2024-09-13T16:58:15.764 INFO:tasks.mds_thrash.fs.[a]:mds.d reported in up:standby state 2024-09-13T16:58:15.764 INFO:tasks.mds_thrash.fs.[a]:waiting for 85 secs before thrashing 2024-09-13T16:58:27.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:58:27.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:58:47.079 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:47.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c81dd640 1 Processor -- start 2024-09-13T16:58:47.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c81dd640 1 -- start start 2024-09-13T16:58:47.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c81dd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f11c00c4710 0x7f11c00c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c81dd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c015b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c81dd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c00c76a0 con 0x7f11c014aa40 2024-09-13T16:58:47.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c81dd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c0151b30 con 0x7f11c00c4710 2024-09-13T16:58:47.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c81dd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0151570 con 0x7f11c00c5fd0 2024-09-13T16:58:47.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c677a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c015b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c5778640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f11c00c4710 0x7f11c00c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c677a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c015b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:43700/0 (socket says 172.21.15.39:43700) 2024-09-13T16:58:47.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.219+0000 7f11c677a640 1 -- 172.21.15.39:0/4189915885 learned_addr learned my addr 172.21.15.39:0/4189915885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:47.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.223+0000 7f11c5778640 1 -- 172.21.15.39:0/4189915885 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 msgr2=0x7f11c015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:47.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.223+0000 7f11c5778640 1 --2- 172.21.15.39:0/4189915885 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c015b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.223+0000 7f11c5778640 1 -- 172.21.15.39:0/4189915885 >> v1:172.21.15.84:6789/0 conn(0x7f11c014aa40 legacy=0x7f11c00c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.223+0000 7f11c5778640 1 -- 172.21.15.39:0/4189915885 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0151850 con 0x7f11c00c4710 2024-09-13T16:58:47.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.223+0000 7f11c677a640 1 --2- 172.21.15.39:0/4189915885 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c015b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:58:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.223+0000 7f11c5778640 1 --2- 172.21.15.39:0/4189915885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c00c4710 0x7f11c00c5a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097c0 tx=0x7f11b001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3915fc4f40785b56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.223+0000 7f11c4f77640 1 -- 172.21.15.39:0/4189915885 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b00ab600 con 0x7f11c00c4710 2024-09-13T16:58:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.223+0000 7f11c4f77640 1 -- 172.21.15.39:0/4189915885 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b00a7050 con 0x7f11c00c4710 2024-09-13T16:58:47.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.223+0000 7f11c4f77640 1 -- 172.21.15.39:0/4189915885 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b00ab980 con 0x7f11c00c4710 2024-09-13T16:58:47.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- 172.21.15.39:0/4189915885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c00c4710 msgr2=0x7f11c00c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 --2- 172.21.15.39:0/4189915885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c00c4710 0x7f11c00c5a90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097c0 tx=0x7f11b001c3e0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- 172.21.15.39:0/4189915885 shutdown_connections 2024-09-13T16:58:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 --2- 172.21.15.39:0/4189915885 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c015b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 --2- 172.21.15.39:0/4189915885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c00c4710 0x7f11c00c5a90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- 172.21.15.39:0/4189915885 >> 172.21.15.39:0/4189915885 conn(0x7f11c00c3310 msgr2=0x7f11c00c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- 172.21.15.39:0/4189915885 shutdown_connections 2024-09-13T16:58:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- 172.21.15.39:0/4189915885 wait complete. 2024-09-13T16:58:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 Processor -- start 2024-09-13T16:58:47.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- start start 2024-09-13T16:58:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11c00c4710 0x7f11c00bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c0148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c014aa40 0x7f11c0148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0148fa0 con 0x7f11c00c4710 2024-09-13T16:58:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c00719c0 con 0x7f11c014aa40 2024-09-13T16:58:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0071c10 con 0x7f11c00c5fd0 2024-09-13T16:58:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5f79640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11c00c4710 0x7f11c00bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5f79640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11c00c4710 0x7f11c00bfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47362/0 (socket says 172.21.15.39:47362) 2024-09-13T16:58:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5f79640 1 -- 172.21.15.39:0/4174395202 learned_addr learned my addr 172.21.15.39:0/4174395202 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5778640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c0148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c677a640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c014aa40 0x7f11c0148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5f79640 1 -- 172.21.15.39:0/4174395202 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 msgr2=0x7f11c0148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5f79640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c0148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5f79640 1 -- 172.21.15.39:0/4174395202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c014aa40 msgr2=0x7f11c0148a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5f79640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c014aa40 0x7f11c0148a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5f79640 1 -- 172.21.15.39:0/4174395202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b800bd40 con 0x7f11c00c4710 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c677a640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c014aa40 0x7f11c0148a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c5f79640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11c00c4710 0x7f11c00bfea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00c500 tx=0x7f11bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11b67fc640 1 -- 172.21.15.39:0/4174395202 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc005220 con 0x7f11c00c4710 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11b67fc640 1 -- 172.21.15.39:0/4174395202 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc005380 con 0x7f11c00c4710 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b0009420 con 0x7f11c00c4710 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11b67fc640 1 -- 172.21.15.39:0/4174395202 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc007270 con 0x7f11c00c4710 2024-09-13T16:58:47.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.227+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c00721d0 con 0x7f11c00c4710 2024-09-13T16:58:47.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.231+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c00a8140 con 0x7f11c00c4710 2024-09-13T16:58:47.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.231+0000 7f11b67fc640 1 -- 172.21.15.39:0/4174395202 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f11bc0073d0 con 0x7f11c00c4710 2024-09-13T16:58:47.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.231+0000 7f11b67fc640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f11a4066690 0x7f11a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.231+0000 7f11b67fc640 1 -- 172.21.15.39:0/4174395202 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7f11bc084280 con 0x7f11c00c4710 2024-09-13T16:58:47.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.231+0000 7f11c5778640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f11a4066690 0x7f11a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.231+0000 7f11c5778640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f11a4066690 0x7f11a4068b50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f11b00ad000 tx=0x7f11b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:47.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.235+0000 7f11b67fc640 1 -- 172.21.15.39:0/4174395202 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f11bc050db0 con 0x7f11c00c4710 2024-09-13T16:58:47.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.451+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11c006b830 con 0x7f11c00c4710 2024-09-13T16:58:47.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.451+0000 7f11b67fc640 1 -- 172.21.15.39:0/4174395202 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f11bc0543d0 con 0x7f11c00c4710 2024-09-13T16:58:47.450 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:47.450 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:47.451 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-13T16:58:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f11a4066690 msgr2=0x7f11a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f11a4066690 0x7f11a4068b50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f11b00ad000 tx=0x7f11b0008040 comp rx=0 tx=0).stop 2024-09-13T16:58:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11c00c4710 msgr2=0x7f11c00bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11c00c4710 0x7f11c00bfea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00c500 tx=0x7f11bc00c9c0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 shutdown_connections 2024-09-13T16:58:47.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f11a4066690 0x7f11a4068b50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11c014aa40 0x7f11c0148a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f11c00c5fd0 0x7f11c0148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 --2- 172.21.15.39:0/4174395202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11c00c4710 0x7f11c00bfea0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 >> 172.21.15.39:0/4174395202 conn(0x7f11c00c3310 msgr2=0x7f11c00c0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:47.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 shutdown_connections 2024-09-13T16:58:47.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.455+0000 7f11c81dd640 1 -- 172.21.15.39:0/4174395202 wait complete. 2024-09-13T16:58:47.472 INFO:tasks.mds_thrash.fs.[b]:kill mds.a (rank=0) 2024-09-13T16:58:47.472 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T16:58:47.473 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:58:47.485 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T16:58:47.485 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:47.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7ee056640 1 Processor -- start 2024-09-13T16:58:47.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7ee056640 1 -- start start 2024-09-13T16:58:47.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7ee056640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7e814a770 0x7fd7e8150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7ee056640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd7e814ab50 0x7fd7e80713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7ee056640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e80c4760 con 0x7fd7e80c6060 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7ee056640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e806df80 con 0x7fd7e814a770 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7ee056640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e80c1950 con 0x7fd7e814ab50 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7e77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7e814a770 0x7fd7e8150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7e77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7e814a770 0x7fd7e8150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47378/0 (socket says 172.21.15.39:47378) 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7e77fe640 1 -- 172.21.15.39:0/116104406 learned_addr learned my addr 172.21.15.39:0/116104406 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7e7fff640 1 --2- 172.21.15.39:0/116104406 >> v2:172.21.15.84:3300/0 conn(0x7fd7e814ab50 0x7fd7e80713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7e77fe640 1 -- 172.21.15.39:0/116104406 >> v2:172.21.15.84:3300/0 conn(0x7fd7e814ab50 msgr2=0x7fd7e80713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7e77fe640 1 --2- 172.21.15.39:0/116104406 >> v2:172.21.15.84:3300/0 conn(0x7fd7e814ab50 0x7fd7e80713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7e77fe640 1 -- 172.21.15.39:0/116104406 >> v1:172.21.15.84:6789/0 conn(0x7fd7e80c6060 legacy=0x7fd7e806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.659+0000 7fd7e77fe640 1 -- 172.21.15.39:0/116104406 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e814afa0 con 0x7fd7e814a770 2024-09-13T16:58:47.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7e77fe640 1 --2- 172.21.15.39:0/116104406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814a770 0x7fd7e8150ad0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8155160 tx=0x7fd7d801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fd2c609ee7a05b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7e67fc640 1 -- 172.21.15.39:0/116104406 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d80ab620 con 0x7fd7e814a770 2024-09-13T16:58:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7e67fc640 1 -- 172.21.15.39:0/116104406 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d80ab780 con 0x7fd7e814a770 2024-09-13T16:58:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7e67fc640 1 -- 172.21.15.39:0/116104406 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d80abb10 con 0x7fd7e814a770 2024-09-13T16:58:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 -- 172.21.15.39:0/116104406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814a770 msgr2=0x7fd7e8150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 --2- 172.21.15.39:0/116104406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814a770 0x7fd7e8150ad0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8155160 tx=0x7fd7d801c420 comp rx=0 tx=0).stop 2024-09-13T16:58:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 -- 172.21.15.39:0/116104406 shutdown_connections 2024-09-13T16:58:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 --2- 172.21.15.39:0/116104406 >> v2:172.21.15.84:3300/0 conn(0x7fd7e814ab50 0x7fd7e80713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 --2- 172.21.15.39:0/116104406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814a770 0x7fd7e8150ad0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 -- 172.21.15.39:0/116104406 >> 172.21.15.39:0/116104406 conn(0x7fd7e80c32a0 msgr2=0x7fd7e80c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 -- 172.21.15.39:0/116104406 shutdown_connections 2024-09-13T16:58:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 -- 172.21.15.39:0/116104406 wait complete. 2024-09-13T16:58:47.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 Processor -- start 2024-09-13T16:58:47.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.663+0000 7fd7ee056640 1 -- start start 2024-09-13T16:58:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7ee056640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e80c6060 0x7fd7e814de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7ee056640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7e814a770 0x7fd7e8148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7ee056640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814ab50 0x7fd7e8070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7ee056640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81edaf0 con 0x7fd7e814ab50 2024-09-13T16:58:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7ee056640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81eddd0 con 0x7fd7e80c6060 2024-09-13T16:58:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7ee056640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81ee0b0 con 0x7fd7e814a770 2024-09-13T16:58:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814ab50 0x7fd7e8070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814ab50 0x7fd7e8070ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47382/0 (socket says 172.21.15.39:47382) 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e7fff640 1 -- 172.21.15.39:0/699792633 learned_addr learned my addr 172.21.15.39:0/699792633 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e77fe640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e80c6060 0x7fd7e814de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e6ffd640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7e814a770 0x7fd7e8148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e7fff640 1 -- 172.21.15.39:0/699792633 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7e814a770 msgr2=0x7fd7e8148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e7fff640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7e814a770 0x7fd7e8148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e7fff640 1 -- 172.21.15.39:0/699792633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e80c6060 msgr2=0x7fd7e814de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e7fff640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e80c6060 0x7fd7e814de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e7fff640 1 -- 172.21.15.39:0/699792633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80094d0 con 0x7fd7e814ab50 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e6ffd640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7e814a770 0x7fd7e8148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e7fff640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814ab50 0x7fd7e8070ad0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0007e40 tx=0x7fd7e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7cffff640 1 -- 172.21.15.39:0/699792633 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0014620 con 0x7fd7e814ab50 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7cffff640 1 -- 172.21.15.39:0/699792633 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e000ee90 con 0x7fd7e814ab50 2024-09-13T16:58:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e81ee390 con 0x7fd7e814ab50 2024-09-13T16:58:47.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7cffff640 1 -- 172.21.15.39:0/699792633 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e00149e0 con 0x7fd7e814ab50 2024-09-13T16:58:47.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e8148aa0 con 0x7fd7e814ab50 2024-09-13T16:58:47.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e80619c0 con 0x7fd7e814ab50 2024-09-13T16:58:47.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7cffff640 1 -- 172.21.15.39:0/699792633 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd7e0014b40 con 0x7fd7e814ab50 2024-09-13T16:58:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7cffff640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7d40666c0 0x7fd7d4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7cffff640 1 -- 172.21.15.39:0/699792633 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7fd7e00851c0 con 0x7fd7e814ab50 2024-09-13T16:58:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.667+0000 7fd7e77fe640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7d40666c0 0x7fd7d4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.671+0000 7fd7e77fe640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7d40666c0 0x7fd7d4068b80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80af000 tx=0x7fd7d80b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:47.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.675+0000 7fd7cffff640 1 -- 172.21.15.39:0/699792633 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd7e0051e10 con 0x7fd7e814ab50 2024-09-13T16:58:47.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.923+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7e806b4b0 con 0x7fd7e814ab50 2024-09-13T16:58:47.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7cffff640 1 -- 172.21.15.39:0/699792633 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fd7e0055430 con 0x7fd7e814ab50 2024-09-13T16:58:47.924 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:47.924 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:47.925 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-13T16:58:47.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7d40666c0 msgr2=0x7fd7d4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7d40666c0 0x7fd7d4068b80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80af000 tx=0x7fd7d80b6040 comp rx=0 tx=0).stop 2024-09-13T16:58:47.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814ab50 msgr2=0x7fd7e8070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:47.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814ab50 0x7fd7e8070ad0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0007e40 tx=0x7fd7e000e4b0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 shutdown_connections 2024-09-13T16:58:47.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7d40666c0 0x7fd7d4068b80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7e814ab50 0x7fd7e8070ad0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7e814a770 0x7fd7e8148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 --2- 172.21.15.39:0/699792633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7e80c6060 0x7fd7e814de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:47.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 >> 172.21.15.39:0/699792633 conn(0x7fd7e80c32a0 msgr2=0x7fd7e8150950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:47.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 shutdown_connections 2024-09-13T16:58:47.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:47.927+0000 7fd7ee056640 1 -- 172.21.15.39:0/699792633 wait complete. 2024-09-13T16:58:47.946 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:58:49.946 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:50.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.099+0000 7f2d3616d640 1 Processor -- start 2024-09-13T16:58:50.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.099+0000 7f2d3616d640 1 -- start start 2024-09-13T16:58:50.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d3616d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2d300c6060 0x7f2d30150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:50.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d3616d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d300c4760 con 0x7f2d3014ab50 2024-09-13T16:58:50.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d3616d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3014afa0 con 0x7f2d3014a770 2024-09-13T16:58:50.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d3616d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d3006df80 con 0x7f2d300c6060 2024-09-13T16:58:50.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d2effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2d300c6060 0x7f2d30150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:50.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d2ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2d3014ab50 0x7f2d300713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56888/0 (socket says 172.21.15.39:56888) 2024-09-13T16:58:50.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d2ffff640 1 -- 172.21.15.39:0/2980432419 learned_addr learned my addr 172.21.15.39:0/2980432419 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:58:50.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d2e7fc640 1 -- 172.21.15.39:0/2980432419 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392288535 0 0) 0x7f2d300c4760 con 0x7f2d3014ab50 2024-09-13T16:58:50.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d2e7fc640 1 -- 172.21.15.39:0/2980432419 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d18003530 con 0x7f2d3014ab50 2024-09-13T16:58:50.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d2effd640 1 -- 172.21.15.39:0/2980432419 >> v1:172.21.15.84:6789/0 conn(0x7f2d3014a770 legacy=0x7f2d3006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:50.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d2effd640 1 -- 172.21.15.39:0/2980432419 >> v1:172.21.15.39:6789/0 conn(0x7f2d3014ab50 legacy=0x7f2d300713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:50.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d2effd640 1 -- 172.21.15.39:0/2980432419 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30151120 con 0x7f2d300c6060 2024-09-13T16:58:50.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.103+0000 7f2d2effd640 1 --2- 172.21.15.39:0/2980432419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d300c6060 0x7f2d30150ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2d240097c0 tx=0x7f2d2401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc6cd3eac7f00cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:50.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d2e7fc640 1 -- 172.21.15.39:0/2980432419 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d240a4ea0 con 0x7f2d300c6060 2024-09-13T16:58:50.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d2e7fc640 1 -- 172.21.15.39:0/2980432419 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d240a7050 con 0x7f2d300c6060 2024-09-13T16:58:50.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d2e7fc640 1 -- 172.21.15.39:0/2980432419 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d240ab820 con 0x7f2d300c6060 2024-09-13T16:58:50.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 -- 172.21.15.39:0/2980432419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d300c6060 msgr2=0x7f2d30150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:50.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 --2- 172.21.15.39:0/2980432419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d300c6060 0x7f2d30150ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2d240097c0 tx=0x7f2d2401c410 comp rx=0 tx=0).stop 2024-09-13T16:58:50.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 -- 172.21.15.39:0/2980432419 shutdown_connections 2024-09-13T16:58:50.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 --2- 172.21.15.39:0/2980432419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d300c6060 0x7f2d30150ad0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:50.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 -- 172.21.15.39:0/2980432419 >> 172.21.15.39:0/2980432419 conn(0x7f2d300c32a0 msgr2=0x7f2d300c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:50.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 -- 172.21.15.39:0/2980432419 shutdown_connections 2024-09-13T16:58:50.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 -- 172.21.15.39:0/2980432419 wait complete. 2024-09-13T16:58:50.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 Processor -- start 2024-09-13T16:58:50.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 -- start start 2024-09-13T16:58:50.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d300c6060 0x7f2d301f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:50.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d3014a770 0x7f2d300beef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:50.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3014ab50 0x7f2d30149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:50.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d301f0dd0 con 0x7f2d3014ab50 2024-09-13T16:58:50.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3015b330 con 0x7f2d3014a770 2024-09-13T16:58:50.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d3616d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3015b610 con 0x7f2d300c6060 2024-09-13T16:58:50.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d2ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3014ab50 0x7f2d30149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:50.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.107+0000 7f2d2ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3014ab50 0x7f2d30149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43690/0 (socket says 172.21.15.39:43690) 2024-09-13T16:58:50.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2ffff640 1 -- 172.21.15.39:0/2800016317 learned_addr learned my addr 172.21.15.39:0/2800016317 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:50.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2effd640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d3014a770 0x7f2d300beef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:50.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2f7fe640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d300c6060 0x7f2d301f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:50.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2ffff640 1 -- 172.21.15.39:0/2800016317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d300c6060 msgr2=0x7f2d301f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2ffff640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d300c6060 0x7f2d301f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2ffff640 1 -- 172.21.15.39:0/2800016317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d3014a770 msgr2=0x7f2d300beef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2ffff640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d3014a770 0x7f2d300beef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2ffff640 1 -- 172.21.15.39:0/2800016317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d24009420 con 0x7f2d3014ab50 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2ffff640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3014ab50 0x7f2d30149c40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800cb40 tx=0x7f2d2800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d17fff640 1 -- 172.21.15.39:0/2800016317 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800aca0 con 0x7f2d3014ab50 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d17fff640 1 -- 172.21.15.39:0/2800016317 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2800ae00 con 0x7f2d3014ab50 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3015b8f0 con 0x7f2d3014ab50 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d17fff640 1 -- 172.21.15.39:0/2800016317 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28012620 con 0x7f2d3014ab50 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d301f10b0 con 0x7f2d3014ab50 2024-09-13T16:58:50.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d300619c0 con 0x7f2d3014ab50 2024-09-13T16:58:50.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d17fff640 1 -- 172.21.15.39:0/2800016317 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d28007070 con 0x7f2d3014ab50 2024-09-13T16:58:50.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d17fff640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d1c066690 0x7f2d1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:50.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d17fff640 1 -- 172.21.15.39:0/2800016317 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7f2d28084970 con 0x7f2d3014ab50 2024-09-13T16:58:50.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.111+0000 7f2d2f7fe640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d1c066690 0x7f2d1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:50.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.115+0000 7f2d2f7fe640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d1c066690 0x7f2d1c068b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d20007fd0 tx=0x7f2d20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:50.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.119+0000 7f2d17fff640 1 -- 172.21.15.39:0/2800016317 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2d28051540 con 0x7f2d3014ab50 2024-09-13T16:58:50.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.335+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d3015bb30 con 0x7f2d3014ab50 2024-09-13T16:58:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.335+0000 7f2d17fff640 1 -- 172.21.15.39:0/2800016317 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f2d3015bb30 con 0x7f2d3014ab50 2024-09-13T16:58:50.335 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:50.335 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-13T16:58:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d1c066690 msgr2=0x7f2d1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d1c066690 0x7f2d1c068b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2d20007fd0 tx=0x7f2d20009040 comp rx=0 tx=0).stop 2024-09-13T16:58:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3014ab50 msgr2=0x7f2d30149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3014ab50 0x7f2d30149c40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800cb40 tx=0x7f2d2800a060 comp rx=0 tx=0).stop 2024-09-13T16:58:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 shutdown_connections 2024-09-13T16:58:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d1c066690 0x7f2d1c068b50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d3014a770 0x7f2d300beef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:50.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3014ab50 0x7f2d30149c40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:50.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 --2- 172.21.15.39:0/2800016317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d300c6060 0x7f2d301f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:50.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 >> 172.21.15.39:0/2800016317 conn(0x7f2d300c32a0 msgr2=0x7f2d300c1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:50.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 shutdown_connections 2024-09-13T16:58:50.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:50.339+0000 7f2d3616d640 1 -- 172.21.15.39:0/2800016317 wait complete. 2024-09-13T16:58:50.356 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:58:52.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:52.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7139a71640 1 Processor -- start 2024-09-13T16:58:52.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7139a71640 1 -- start start 2024-09-13T16:58:52.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7139a71640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71341556d0 0x7f71340c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:52.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7139a71640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71340a72b0 0x7f7134071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:52.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7139a71640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713406feb0 con 0x7f71340c7610 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7139a71640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71340aae50 con 0x7f71341556d0 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7139a71640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7134155ab0 con 0x7f71340a72b0 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f71337fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71341556d0 0x7f71340c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7132ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f71340c7610 0x7f7134151b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56900/0 (socket says 172.21.15.39:56900) 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f71337fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71341556d0 0x7f71340c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43702/0 (socket says 172.21.15.39:43702) 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7132ffd640 1 -- 172.21.15.39:0/4274776785 learned_addr learned my addr 172.21.15.39:0/4274776785 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f7133fff640 1 --2- 172.21.15.39:0/4274776785 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71340a72b0 0x7f7134071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f71327fc640 1 -- 172.21.15.39:0/4274776785 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537503874 0 0) 0x7f713406feb0 con 0x7f71340c7610 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f71337fe640 1 -- 172.21.15.39:0/4274776785 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71340a72b0 msgr2=0x7f7134071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f71337fe640 1 --2- 172.21.15.39:0/4274776785 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71340a72b0 0x7f7134071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f71337fe640 1 -- 172.21.15.39:0/4274776785 >> v1:172.21.15.39:6789/0 conn(0x7f71340c7610 legacy=0x7f7134151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.495+0000 7f71337fe640 1 -- 172.21.15.39:0/4274776785 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340c3c20 con 0x7f71341556d0 2024-09-13T16:58:52.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f71337fe640 1 --2- 172.21.15.39:0/4274776785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71341556d0 0x7f71340c36e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f71240097c0 tx=0x7f712401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11e8788d2310b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:52.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f71327fc640 1 -- 172.21.15.39:0/4274776785 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71240ab600 con 0x7f71341556d0 2024-09-13T16:58:52.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f71327fc640 1 -- 172.21.15.39:0/4274776785 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71240a7050 con 0x7f71341556d0 2024-09-13T16:58:52.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f71327fc640 1 -- 172.21.15.39:0/4274776785 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71240ab980 con 0x7f71341556d0 2024-09-13T16:58:52.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 -- 172.21.15.39:0/4274776785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71341556d0 msgr2=0x7f71340c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:52.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 --2- 172.21.15.39:0/4274776785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71341556d0 0x7f71340c36e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f71240097c0 tx=0x7f712401c380 comp rx=0 tx=0).stop 2024-09-13T16:58:52.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 -- 172.21.15.39:0/4274776785 shutdown_connections 2024-09-13T16:58:52.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 --2- 172.21.15.39:0/4274776785 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71340a72b0 0x7f7134071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 --2- 172.21.15.39:0/4274776785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71341556d0 0x7f71340c36e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 -- 172.21.15.39:0/4274776785 >> 172.21.15.39:0/4274776785 conn(0x7f71340a97b0 msgr2=0x7f71340a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:52.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 -- 172.21.15.39:0/4274776785 shutdown_connections 2024-09-13T16:58:52.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 -- 172.21.15.39:0/4274776785 wait complete. 2024-09-13T16:58:52.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 Processor -- start 2024-09-13T16:58:52.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 -- start start 2024-09-13T16:58:52.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71340a72b0 0x7f71340bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:52.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71341556d0 0x7f7134150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:52.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7134151dd0 0x7f713406b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340bd520 con 0x7f7134151dd0 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71341ed160 con 0x7f71340a72b0 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7139a71640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340bcb50 con 0x7f71341556d0 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7133fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7134151dd0 0x7f713406b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7133fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7134151dd0 0x7f713406b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43718/0 (socket says 172.21.15.39:43718) 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.499+0000 7f7133fff640 1 -- 172.21.15.39:0/4025478323 learned_addr learned my addr 172.21.15.39:0/4025478323 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f71337fe640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71340a72b0 0x7f71340bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7132ffd640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71341556d0 0x7f7134150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7133fff640 1 -- 172.21.15.39:0/4025478323 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71341556d0 msgr2=0x7f7134150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7133fff640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71341556d0 0x7f7134150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7133fff640 1 -- 172.21.15.39:0/4025478323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71340a72b0 msgr2=0x7f71340bcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7133fff640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71340a72b0 0x7f71340bcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7133fff640 1 -- 172.21.15.39:0/4025478323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7124009420 con 0x7f7134151dd0 2024-09-13T16:58:52.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7133fff640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7134151dd0 0x7f713406b5f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f712c009fd0 tx=0x7f712c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:52.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f711bfff640 1 -- 172.21.15.39:0/4025478323 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c014420 con 0x7f7134151dd0 2024-09-13T16:58:52.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f711bfff640 1 -- 172.21.15.39:0/4025478323 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c014580 con 0x7f7134151dd0 2024-09-13T16:58:52.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71340be2e0 con 0x7f7134151dd0 2024-09-13T16:58:52.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f711bfff640 1 -- 172.21.15.39:0/4025478323 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c014930 con 0x7f7134151dd0 2024-09-13T16:58:52.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71341ed650 con 0x7f7134151dd0 2024-09-13T16:58:52.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71340a82c0 con 0x7f7134151dd0 2024-09-13T16:58:52.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f711bfff640 1 -- 172.21.15.39:0/4025478323 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f712c014a90 con 0x7f7134151dd0 2024-09-13T16:58:52.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f711bfff640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7120066690 0x7f7120068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:52.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f711bfff640 1 -- 172.21.15.39:0/4025478323 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7f712c085180 con 0x7f7134151dd0 2024-09-13T16:58:52.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f71337fe640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7120066690 0x7f7120068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:52.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.503+0000 7f71337fe640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7120066690 0x7f7120068b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f71240ad000 tx=0x7f71240a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:52.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.507+0000 7f711bfff640 1 -- 172.21.15.39:0/4025478323 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f712c051dd0 con 0x7f7134151dd0 2024-09-13T16:58:52.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71340bccc0 con 0x7f7134151dd0 2024-09-13T16:58:52.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f711bfff640 1 -- 172.21.15.39:0/4025478323 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f712c0553f0 con 0x7f7134151dd0 2024-09-13T16:58:52.737 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:52.737 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:52.738 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-13T16:58:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7120066690 msgr2=0x7f7120068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7120066690 0x7f7120068b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f71240ad000 tx=0x7f71240a6000 comp rx=0 tx=0).stop 2024-09-13T16:58:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7134151dd0 msgr2=0x7f713406b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7134151dd0 0x7f713406b5f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f712c009fd0 tx=0x7f712c00e4b0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 shutdown_connections 2024-09-13T16:58:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7120066690 0x7f7120068b50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7134151dd0 0x7f713406b5f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f71341556d0 0x7f7134150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 --2- 172.21.15.39:0/4025478323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71340a72b0 0x7f71340bcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 >> 172.21.15.39:0/4025478323 conn(0x7f71340a97b0 msgr2=0x7f71340a7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 shutdown_connections 2024-09-13T16:58:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:52.739+0000 7f7139a71640 1 -- 172.21.15.39:0/4025478323 wait complete. 2024-09-13T16:58:52.756 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:58:54.757 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:54.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe9715cf640 1 Processor -- start 2024-09-13T16:58:54.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe9715cf640 1 -- start start 2024-09-13T16:58:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe9715cf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe96c0c5f40 0x7fe96c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe9715cf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c071ed0 0x7fe96c1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe9715cf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c0c2e60 con 0x7fe96c0c5790 2024-09-13T16:58:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe9715cf640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96c0c7470 con 0x7fe96c0c5f40 2024-09-13T16:58:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe9715cf640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0c2570 con 0x7fe96c071ed0 2024-09-13T16:58:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe96affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe96c0c5f40 0x7fe96c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe96affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe96c0c5f40 0x7fe96c156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43722/0 (socket says 172.21.15.39:43722) 2024-09-13T16:58:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe96affd640 1 -- 172.21.15.39:0/3424206811 learned_addr learned my addr 172.21.15.39:0/3424206811 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe96a7fc640 1 --2- 172.21.15.39:0/3424206811 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c071ed0 0x7fe96c1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe96affd640 1 -- 172.21.15.39:0/3424206811 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c071ed0 msgr2=0x7fe96c1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe96affd640 1 --2- 172.21.15.39:0/3424206811 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c071ed0 0x7fe96c1577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe96affd640 1 -- 172.21.15.39:0/3424206811 >> v1:172.21.15.84:6789/0 conn(0x7fe96c0c5790 legacy=0x7fe96c157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe96affd640 1 -- 172.21.15.39:0/3424206811 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c072360 con 0x7fe96c0c5f40 2024-09-13T16:58:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.903+0000 7fe96a7fc640 1 --2- 172.21.15.39:0/3424206811 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c071ed0 0x7fe96c1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:58:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe96affd640 1 --2- 172.21.15.39:0/3424206811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 0x7fe96c156cf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe960009870 tx=0x7fe96001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfa1f96e8782985 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe969ffb640 1 -- 172.21.15.39:0/3424206811 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600ab670 con 0x7fe96c0c5f40 2024-09-13T16:58:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe969ffb640 1 -- 172.21.15.39:0/3424206811 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9600ab7d0 con 0x7fe96c0c5f40 2024-09-13T16:58:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe969ffb640 1 -- 172.21.15.39:0/3424206811 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9600abb60 con 0x7fe96c0c5f40 2024-09-13T16:58:54.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe9715cf640 1 -- 172.21.15.39:0/3424206811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 msgr2=0x7fe96c156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:54.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe9715cf640 1 --2- 172.21.15.39:0/3424206811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 0x7fe96c156cf0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe960009870 tx=0x7fe96001c560 comp rx=0 tx=0).stop 2024-09-13T16:58:54.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe9715cf640 1 -- 172.21.15.39:0/3424206811 shutdown_connections 2024-09-13T16:58:54.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe9715cf640 1 --2- 172.21.15.39:0/3424206811 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c071ed0 0x7fe96c1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:54.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe9715cf640 1 --2- 172.21.15.39:0/3424206811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 0x7fe96c156cf0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:54.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe9715cf640 1 -- 172.21.15.39:0/3424206811 >> 172.21.15.39:0/3424206811 conn(0x7fe96c06c020 msgr2=0x7fe96c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe9715cf640 1 -- 172.21.15.39:0/3424206811 shutdown_connections 2024-09-13T16:58:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.907+0000 7fe9715cf640 1 -- 172.21.15.39:0/3424206811 wait complete. 2024-09-13T16:58:54.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 Processor -- start 2024-09-13T16:58:54.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 -- start start 2024-09-13T16:58:54.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c0c5790 0x7fe96c152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:54.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 0x7fe96c0bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:54.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe96c1edce0 0x7fe96c1f0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:54.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c1efd80 con 0x7fe96c0c5f40 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c150f30 con 0x7fe96c1edce0 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c151210 con 0x7fe96c0c5790 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 0x7fe96c0bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 0x7fe96c0bdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43724/0 (socket says 172.21.15.39:43724) 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96a7fc640 1 -- 172.21.15.39:0/2955847404 learned_addr learned my addr 172.21.15.39:0/2955847404 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96affd640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c0c5790 0x7fe96c152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96b7fe640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe96c1edce0 0x7fe96c1f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96a7fc640 1 -- 172.21.15.39:0/2955847404 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c0c5790 msgr2=0x7fe96c152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96a7fc640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c0c5790 0x7fe96c152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96a7fc640 1 -- 172.21.15.39:0/2955847404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe96c1edce0 msgr2=0x7fe96c1f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96a7fc640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe96c1edce0 0x7fe96c1f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96a7fc640 1 -- 172.21.15.39:0/2955847404 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600094d0 con 0x7fe96c0c5f40 2024-09-13T16:58:54.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe96a7fc640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 0x7fe96c0bdb60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00c7f0 tx=0x7fe95c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:54.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9537fe640 1 -- 172.21.15.39:0/2955847404 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c007820 con 0x7fe96c0c5f40 2024-09-13T16:58:54.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9537fe640 1 -- 172.21.15.39:0/2955847404 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c007980 con 0x7fe96c0c5f40 2024-09-13T16:58:54.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c1514f0 con 0x7fe96c0c5f40 2024-09-13T16:58:54.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9537fe640 1 -- 172.21.15.39:0/2955847404 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c007d10 con 0x7fe96c0c5f40 2024-09-13T16:58:54.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c153400 con 0x7fe96c0c5f40 2024-09-13T16:58:54.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.911+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c156cf0 con 0x7fe96c0c5f40 2024-09-13T16:58:54.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.915+0000 7fe9537fe640 1 -- 172.21.15.39:0/2955847404 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe95c0113f0 con 0x7fe96c0c5f40 2024-09-13T16:58:54.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.915+0000 7fe9537fe640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe9580666c0 0x7fe958068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:54.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.915+0000 7fe9537fe640 1 -- 172.21.15.39:0/2955847404 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7fe95c084b60 con 0x7fe96c0c5f40 2024-09-13T16:58:54.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.915+0000 7fe96affd640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe9580666c0 0x7fe958068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:54.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.915+0000 7fe96affd640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe9580666c0 0x7fe958068b80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe9600af000 tx=0x7fe9600a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:54.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:54.919+0000 7fe9537fe640 1 -- 172.21.15.39:0/2955847404 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe95c0517b0 con 0x7fe96c0c5f40 2024-09-13T16:58:55.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.215+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe96c0c1f70 con 0x7fe96c0c5f40 2024-09-13T16:58:55.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.215+0000 7fe9537fe640 1 -- 172.21.15.39:0/2955847404 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fe95c054dd0 con 0x7fe96c0c5f40 2024-09-13T16:58:55.215 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:55.215 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:55.215 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-13T16:58:55.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe9580666c0 msgr2=0x7fe958068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:55.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe9580666c0 0x7fe958068b80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe9600af000 tx=0x7fe9600a7000 comp rx=0 tx=0).stop 2024-09-13T16:58:55.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 msgr2=0x7fe96c0bdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:55.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 0x7fe96c0bdb60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00c7f0 tx=0x7fe95c00ccb0 comp rx=0 tx=0).stop 2024-09-13T16:58:55.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 shutdown_connections 2024-09-13T16:58:55.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe9580666c0 0x7fe958068b80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:55.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe96c1edce0 0x7fe96c1f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:55.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe96c0c5f40 0x7fe96c0bdb60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:55.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 --2- 172.21.15.39:0/2955847404 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe96c0c5790 0x7fe96c152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:55.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 >> 172.21.15.39:0/2955847404 conn(0x7fe96c06c020 msgr2=0x7fe96c0a75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:55.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 shutdown_connections 2024-09-13T16:58:55.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:55.219+0000 7fe9715cf640 1 -- 172.21.15.39:0/2955847404 wait complete. 2024-09-13T16:58:55.236 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:58:57.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:57.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f905df69640 1 Processor -- start 2024-09-13T16:58:57.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f905df69640 1 -- start start 2024-09-13T16:58:57.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f905df69640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058157c90 0x7f90580c0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:57.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f905df69640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90581581e0 con 0x7f90580c31b0 2024-09-13T16:58:57.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f905df69640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9058154ce0 con 0x7f9058151330 2024-09-13T16:58:57.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f905df69640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c6780 con 0x7f9058157c90 2024-09-13T16:58:57.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f9057fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f90580c31b0 0x7f90580c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56906/0 (socket says 172.21.15.39:56906) 2024-09-13T16:58:57.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f9057fff640 1 -- 172.21.15.39:0/1094588176 learned_addr learned my addr 172.21.15.39:0/1094588176 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:58:57.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f905d768640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058157c90 0x7f90580c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:57.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222430682 0 0) 0x7f90581581e0 con 0x7f90580c31b0 2024-09-13T16:58:57.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9048003530 con 0x7f90580c31b0 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1444332322 0 0) 0x7f9048003530 con 0x7f90580c31b0 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058157c90 msgr2=0x7f90580c0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058157c90 0x7f90580c0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 >> v1:172.21.15.84:6789/0 conn(0x7f9058151330 legacy=0x7f90581545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90580c6a60 con 0x7f90580c31b0 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f9044001680 con 0x7f90580c31b0 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 >> v1:172.21.15.39:6789/0 conn(0x7f90580c31b0 legacy=0x7f90580c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f9048003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9048004360 0x7f9048004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058157c90 0x7f9048005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:57.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c6780 con 0x7f90480039b0 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904800ba40 con 0x7f9048004360 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.391+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904800bc00 con 0x7f9058157c90 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905cf67640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f9048003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f9057fff640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9048004360 0x7f9048004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905d768640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058157c90 0x7f9048005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905cf67640 1 -- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058157c90 msgr2=0x7f9048005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905cf67640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058157c90 0x7f9048005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905cf67640 1 -- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9048004360 msgr2=0x7f9048004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905cf67640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9048004360 0x7f9048004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905cf67640 1 -- 172.21.15.39:0/1094588176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9048003530 con 0x7f90480039b0 2024-09-13T16:58:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905cf67640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f9048003d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f904c001910 tx=0x7f904c01c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac1afe6b9011f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0aa5f0 con 0x7f90480039b0 2024-09-13T16:58:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c0aa770 con 0x7f90480039b0 2024-09-13T16:58:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f90577fe640 1 -- 172.21.15.39:0/1094588176 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0aab20 con 0x7f90480039b0 2024-09-13T16:58:57.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905df69640 1 -- 172.21.15.39:0/1094588176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 msgr2=0x7f9048003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:57.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905df69640 1 --2- 172.21.15.39:0/1094588176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f9048003d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f904c001910 tx=0x7f904c01c550 comp rx=0 tx=0).stop 2024-09-13T16:58:57.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905d768640 1 -- 172.21.15.39:0/1094588176 reap_dead start 2024-09-13T16:58:57.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905df69640 1 -- 172.21.15.39:0/1094588176 shutdown_connections 2024-09-13T16:58:57.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905df69640 1 -- 172.21.15.39:0/1094588176 >> 172.21.15.39:0/1094588176 conn(0x7f90580c4030 msgr2=0x7f90580c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:57.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905df69640 1 -- 172.21.15.39:0/1094588176 shutdown_connections 2024-09-13T16:58:57.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.395+0000 7f905df69640 1 -- 172.21.15.39:0/1094588176 wait complete. 2024-09-13T16:58:57.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 Processor -- start 2024-09-13T16:58:57.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 -- start start 2024-09-13T16:58:57.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f90581ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058156910 0x7f90581f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806d6e0 0x7f905806db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90581eff30 con 0x7f90480039b0 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90581ed510 con 0x7f905806d6e0 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90581588e0 con 0x7f9058156910 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905cf67640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f90581ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905cf67640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f90581ecfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43750/0 (socket says 172.21.15.39:43750) 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905cf67640 1 -- 172.21.15.39:0/1758529752 learned_addr learned my addr 172.21.15.39:0/1758529752 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f9057fff640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058156910 0x7f90581f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905cf67640 1 -- 172.21.15.39:0/1758529752 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058156910 msgr2=0x7f90581f03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:57.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905cf67640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058156910 0x7f90581f03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905d768640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806d6e0 0x7f905806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905cf67640 1 -- 172.21.15.39:0/1758529752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806d6e0 msgr2=0x7f905806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905cf67640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806d6e0 0x7f905806db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905cf67640 1 -- 172.21.15.39:0/1758529752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058158bc0 con 0x7f90480039b0 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f9057fff640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058156910 0x7f90581f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905cf67640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f90581ecfd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f904c0ac040 tx=0x7f9048005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f90557fa640 1 -- 172.21.15.39:0/1758529752 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0aa5f0 con 0x7f90480039b0 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f90557fa640 1 -- 172.21.15.39:0/1758529752 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c0b8070 con 0x7f90480039b0 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90581efbc0 con 0x7f90480039b0 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f90557fa640 1 -- 172.21.15.39:0/1758529752 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0052c0 con 0x7f90480039b0 2024-09-13T16:58:57.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9058158ea0 con 0x7f90480039b0 2024-09-13T16:58:57.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.399+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90580c4f90 con 0x7f90480039b0 2024-09-13T16:58:57.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.403+0000 7f90557fa640 1 -- 172.21.15.39:0/1758529752 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f904c0ba070 con 0x7f90480039b0 2024-09-13T16:58:57.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.403+0000 7f90557fa640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f903c066690 0x7f903c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:57.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.403+0000 7f90557fa640 1 -- 172.21.15.39:0/1758529752 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7f904c0a7070 con 0x7f90480039b0 2024-09-13T16:58:57.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.403+0000 7f9057fff640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f903c066690 0x7f903c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:57.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.403+0000 7f9057fff640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f903c066690 0x7f903c068b50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9044001210 tx=0x7f9044001d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:57.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.407+0000 7f90557fa640 1 -- 172.21.15.39:0/1758529752 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f904c0eb700 con 0x7f90480039b0 2024-09-13T16:58:57.521 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:58:57.523 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:58:57.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.619+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90580c2710 con 0x7f90480039b0 2024-09-13T16:58:57.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.619+0000 7f90557fa640 1 -- 172.21.15.39:0/1758529752 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f904c0c1090 con 0x7f90480039b0 2024-09-13T16:58:57.617 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:58:57.617 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:58:57.618 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-13T16:58:57.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.619+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f903c066690 msgr2=0x7f903c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:57.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.619+0000 7f905df69640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f903c066690 0x7f903c068b50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9044001210 tx=0x7f9044001d10 comp rx=0 tx=0).stop 2024-09-13T16:58:57.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.619+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 msgr2=0x7f90581ecfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:57.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.619+0000 7f905df69640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f90581ecfd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f904c0ac040 tx=0x7f9048005800 comp rx=0 tx=0).stop 2024-09-13T16:58:57.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.623+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 shutdown_connections 2024-09-13T16:58:57.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.623+0000 7f905df69640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f903c066690 0x7f903c068b50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:57.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.623+0000 7f905df69640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f905806d6e0 0x7f905806db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:57.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.623+0000 7f905df69640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9058156910 0x7f90581f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:57.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.623+0000 7f905df69640 1 --2- 172.21.15.39:0/1758529752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90480039b0 0x7f90581ecfd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:57.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.623+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 >> 172.21.15.39:0/1758529752 conn(0x7f90580c4030 msgr2=0x7f9058154760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:57.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.623+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 shutdown_connections 2024-09-13T16:58:57.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:57.623+0000 7f905df69640 1 -- 172.21.15.39:0/1758529752 wait complete. 2024-09-13T16:58:57.636 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:58:59.637 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:58:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.778+0000 7fd41f117640 1 Processor -- start 2024-09-13T16:58:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.778+0000 7fd41f117640 1 -- start start 2024-09-13T16:58:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41f117640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd418151500 0x7fd4180c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41f117640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4180c5a30 con 0x7fd4180c47c0 2024-09-13T16:58:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41f117640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4181576a0 con 0x7fd418071dd0 2024-09-13T16:58:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41f117640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd418157980 con 0x7fd418151500 2024-09-13T16:58:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41d6b4640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd4180c47c0 0x7fd418154780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56920/0 (socket says 172.21.15.39:56920) 2024-09-13T16:58:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41d6b4640 1 -- 172.21.15.39:0/3162076106 learned_addr learned my addr 172.21.15.39:0/3162076106 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:58:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41ceb3640 1 --2- 172.21.15.39:0/3162076106 >> v2:172.21.15.84:3300/0 conn(0x7fd418151500 0x7fd4180c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137367597 0 0) 0x7fd4180c5a30 con 0x7fd4180c47c0 2024-09-13T16:58:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd400003530 con 0x7fd4180c47c0 2024-09-13T16:58:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2702003233 0 0) 0x7fd400003530 con 0x7fd4180c47c0 2024-09-13T16:58:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 >> v2:172.21.15.84:3300/0 conn(0x7fd418151500 msgr2=0x7fd4180c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 --2- 172.21.15.39:0/3162076106 >> v2:172.21.15.84:3300/0 conn(0x7fd418151500 0x7fd4180c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:59.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 >> v1:172.21.15.84:6789/0 conn(0x7fd418071dd0 legacy=0x7fd418154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd418157c60 con 0x7fd4180c47c0 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fd4100016a0 con 0x7fd4180c47c0 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 >> v1:172.21.15.39:6789/0 conn(0x7fd4180c47c0 legacy=0x7fd418154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd400003980 0x7fd400003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41ceb3640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd400003980 0x7fd400003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4000047d0 0x7fd400004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd400005190 0x7fd40000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418157980 con 0x7fd400003980 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40000c090 con 0x7fd400005190 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40000c250 con 0x7fd4000047d0 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41ceb3640 1 -- 172.21.15.39:0/3162076106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4000047d0 msgr2=0x7fd400004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41ceb3640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4000047d0 0x7fd400004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41ceb3640 1 -- 172.21.15.39:0/3162076106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd400005190 msgr2=0x7fd40000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:58:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41d6b4640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd400005190 0x7fd40000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41ceb3640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd400005190 0x7fd40000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41ceb3640 1 -- 172.21.15.39:0/3162076106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd418157c60 con 0x7fd400003980 2024-09-13T16:58:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41d6b4640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd400005190 0x7fd40000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:58:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.782+0000 7fd41d6b4640 1 -- 172.21.15.39:0/3162076106 reap_dead start 2024-09-13T16:58:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41ceb3640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd400003980 0x7fd400003da0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd408009fc0 tx=0x7fd40801d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96392f242fa9b3b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080a5c60 con 0x7fd400003980 2024-09-13T16:58:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40800f040 con 0x7fd400003980 2024-09-13T16:58:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd4177fe640 1 -- 172.21.15.39:0/3162076106 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080ad3f0 con 0x7fd400003980 2024-09-13T16:58:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41f117640 1 -- 172.21.15.39:0/3162076106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd400003980 msgr2=0x7fd400003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41f117640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd400003980 0x7fd400003da0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd408009fc0 tx=0x7fd40801d6b0 comp rx=0 tx=0).stop 2024-09-13T16:58:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41f117640 1 -- 172.21.15.39:0/3162076106 shutdown_connections 2024-09-13T16:58:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41f117640 1 --2- 172.21.15.39:0/3162076106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd400003980 0x7fd400003da0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41f117640 1 -- 172.21.15.39:0/3162076106 >> 172.21.15.39:0/3162076106 conn(0x7fd4180ab530 msgr2=0x7fd4180ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:58:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41f117640 1 -- 172.21.15.39:0/3162076106 shutdown_connections 2024-09-13T16:58:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41f117640 1 -- 172.21.15.39:0/3162076106 wait complete. 2024-09-13T16:58:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41f117640 1 Processor -- start 2024-09-13T16:58:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.786+0000 7fd41f117640 1 -- start start 2024-09-13T16:58:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41f117640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd41814dd90 0x7fd4180bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41f117640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4180bf710 0x7fd41814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41f117640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd41814e6b0 0x7fd4181ed220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41f117640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41814eb30 con 0x7fd41814e6b0 2024-09-13T16:58:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41f117640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180c2d10 con 0x7fd4180bf710 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41f117640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4181ecd20 con 0x7fd41814dd90 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41d6b4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd41814e6b0 0x7fd4181ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41d6b4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd41814e6b0 0x7fd4181ed220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43766/0 (socket says 172.21.15.39:43766) 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41d6b4640 1 -- 172.21.15.39:0/3462881978 learned_addr learned my addr 172.21.15.39:0/3462881978 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41ceb3640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd41814dd90 0x7fd4180bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd417fff640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4180bf710 0x7fd41814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41d6b4640 1 -- 172.21.15.39:0/3462881978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd41814dd90 msgr2=0x7fd4180bf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41d6b4640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd41814dd90 0x7fd4180bf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41d6b4640 1 -- 172.21.15.39:0/3462881978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4180bf710 msgr2=0x7fd41814e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41d6b4640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4180bf710 0x7fd41814e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41d6b4640 1 -- 172.21.15.39:0/3462881978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd418156560 con 0x7fd41814e6b0 2024-09-13T16:58:59.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41d6b4640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd41814e6b0 0x7fd4181ed220 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd4100011f0 tx=0x7fd41000db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd4157fa640 1 -- 172.21.15.39:0/3462881978 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4100072d0 con 0x7fd41814e6b0 2024-09-13T16:58:59.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd4157fa640 1 -- 172.21.15.39:0/3462881978 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd410007450 con 0x7fd41814e6b0 2024-09-13T16:58:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd418156840 con 0x7fd41814e6b0 2024-09-13T16:58:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd4157fa640 1 -- 172.21.15.39:0/3462881978 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4100047f0 con 0x7fd41814e6b0 2024-09-13T16:58:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd418156e00 con 0x7fd41814e6b0 2024-09-13T16:58:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4181ecf40 con 0x7fd41814e6b0 2024-09-13T16:58:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd4157fa640 1 -- 172.21.15.39:0/3462881978 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd410004970 con 0x7fd41814e6b0 2024-09-13T16:58:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd4157fa640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd404066690 0x7fd404068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:58:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd4157fa640 1 -- 172.21.15.39:0/3462881978 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7fd410084450 con 0x7fd41814e6b0 2024-09-13T16:58:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.790+0000 7fd41ceb3640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd404066690 0x7fd404068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:58:59.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.794+0000 7fd41ceb3640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd404066690 0x7fd404068b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd4080ae000 tx=0x7fd40801cc50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:58:59.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:58:59.798+0000 7fd4157fa640 1 -- 172.21.15.39:0/3462881978 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd4181ecf40 con 0x7fd41814e6b0 2024-09-13T16:59:00.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.042+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4181ecee0 con 0x7fd41814e6b0 2024-09-13T16:59:00.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.042+0000 7fd4157fa640 1 -- 172.21.15.39:0/3462881978 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fd4100510a0 con 0x7fd41814e6b0 2024-09-13T16:59:00.043 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:00.043 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:00.044 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-13T16:59:00.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd404066690 msgr2=0x7fd404068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:00.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd404066690 0x7fd404068b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd4080ae000 tx=0x7fd40801cc50 comp rx=0 tx=0).stop 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd41814e6b0 msgr2=0x7fd4181ed220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd41814e6b0 0x7fd4181ed220 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd4100011f0 tx=0x7fd41000db80 comp rx=0 tx=0).stop 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 shutdown_connections 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd404066690 0x7fd404068b50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4180bf710 0x7fd41814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd41814e6b0 0x7fd4181ed220 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 --2- 172.21.15.39:0/3462881978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd41814dd90 0x7fd4180bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 >> 172.21.15.39:0/3462881978 conn(0x7fd4180ab530 msgr2=0x7fd418070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 shutdown_connections 2024-09-13T16:59:00.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:00.046+0000 7fd41f117640 1 -- 172.21.15.39:0/3462881978 wait complete. 2024-09-13T16:59:00.063 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:59:02.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:02.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bf19d640 1 Processor -- start 2024-09-13T16:59:02.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bf19d640 1 -- start start 2024-09-13T16:59:02.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bf19d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80c41e0 0x7f27b80c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:02.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bf19d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f27b80bcf10 0x7f27b8070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:02.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bf19d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b80c6270 con 0x7f27b80c7760 2024-09-13T16:59:02.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bf19d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b80c6b10 con 0x7f27b80bcf10 2024-09-13T16:59:02.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bf19d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b81580a0 con 0x7f27b80c41e0 2024-09-13T16:59:02.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bd73a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f27b80bcf10 0x7f27b8070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:02.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27b7fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f27b80c7760 0x7f27b8070600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54104/0 (socket says 172.21.15.39:54104) 2024-09-13T16:59:02.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27b7fff640 1 -- 172.21.15.39:0/2104309553 learned_addr learned my addr 172.21.15.39:0/2104309553 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:02.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bcf39640 1 --2- 172.21.15.39:0/2104309553 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80c41e0 0x7f27b80c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:02.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bd73a640 1 -- 172.21.15.39:0/2104309553 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80c41e0 msgr2=0x7f27b80c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:02.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bd73a640 1 --2- 172.21.15.39:0/2104309553 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80c41e0 0x7f27b80c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bd73a640 1 -- 172.21.15.39:0/2104309553 >> v1:172.21.15.39:6789/0 conn(0x7f27b80c7760 legacy=0x7f27b8070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:02.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27bd73a640 1 -- 172.21.15.39:0/2104309553 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b80c64c0 con 0x7f27b80bcf10 2024-09-13T16:59:02.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.210+0000 7f27b77fe640 1 -- 172.21.15.39:0/2104309553 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906791943 0 0) 0x7f27b80c6270 con 0x7f27b80c7760 2024-09-13T16:59:02.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27bd73a640 1 --2- 172.21.15.39:0/2104309553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80bcf10 0x7f27b8070d10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009d90 tx=0x7f27b001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800e78766bd58301 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:02.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27b77fe640 1 -- 172.21.15.39:0/2104309553 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b00a5820 con 0x7f27b80bcf10 2024-09-13T16:59:02.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27b77fe640 1 -- 172.21.15.39:0/2104309553 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b00a5980 con 0x7f27b80bcf10 2024-09-13T16:59:02.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27b77fe640 1 -- 172.21.15.39:0/2104309553 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b00a5d10 con 0x7f27b80bcf10 2024-09-13T16:59:02.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27bf19d640 1 -- 172.21.15.39:0/2104309553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80bcf10 msgr2=0x7f27b8070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:02.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27bf19d640 1 --2- 172.21.15.39:0/2104309553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80bcf10 0x7f27b8070d10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009d90 tx=0x7f27b001c850 comp rx=0 tx=0).stop 2024-09-13T16:59:02.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27bf19d640 1 -- 172.21.15.39:0/2104309553 shutdown_connections 2024-09-13T16:59:02.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27bf19d640 1 --2- 172.21.15.39:0/2104309553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80bcf10 0x7f27b8070d10 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27bf19d640 1 --2- 172.21.15.39:0/2104309553 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80c41e0 0x7f27b80c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27bf19d640 1 -- 172.21.15.39:0/2104309553 >> 172.21.15.39:0/2104309553 conn(0x7f27b80ab430 msgr2=0x7f27b80ab840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:02.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27bf19d640 1 -- 172.21.15.39:0/2104309553 shutdown_connections 2024-09-13T16:59:02.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.214+0000 7f27bf19d640 1 -- 172.21.15.39:0/2104309553 wait complete. 2024-09-13T16:59:02.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 Processor -- start 2024-09-13T16:59:02.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 -- start start 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80bcf10 0x7f27b8153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27b80c41e0 0x7f27b814a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80c7760 0x7f27b814ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80c0ef0 con 0x7f27b80c7760 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b81ecdc0 con 0x7f27b80c41e0 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b81ed2f0 con 0x7f27b80bcf10 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bd73a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80c7760 0x7f27b814ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bd73a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80c7760 0x7f27b814ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56566/0 (socket says 172.21.15.39:56566) 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bd73a640 1 -- 172.21.15.39:0/1129222898 learned_addr learned my addr 172.21.15.39:0/1129222898 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bcf39640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80bcf10 0x7f27b8153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27b7fff640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27b80c41e0 0x7f27b814a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:02.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bd73a640 1 -- 172.21.15.39:0/1129222898 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80bcf10 msgr2=0x7f27b8153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bd73a640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80bcf10 0x7f27b8153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bd73a640 1 -- 172.21.15.39:0/1129222898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27b80c41e0 msgr2=0x7f27b814a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bd73a640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27b80c41e0 0x7f27b814a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bd73a640 1 -- 172.21.15.39:0/1129222898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0009a70 con 0x7f27b80c7760 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27b7fff640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27b80c41e0 0x7f27b814a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bd73a640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80c7760 0x7f27b814ae30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f27b00b0040 tx=0x7f27b00195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27b57fa640 1 -- 172.21.15.39:0/1129222898 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b00b9860 con 0x7f27b80c7760 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27b57fa640 1 -- 172.21.15.39:0/1129222898 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b00b99c0 con 0x7f27b80c7760 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b81524e0 con 0x7f27b80c7760 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27b57fa640 1 -- 172.21.15.39:0/1129222898 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b00b9d70 con 0x7f27b80c7760 2024-09-13T16:59:02.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b80bff30 con 0x7f27b80c7760 2024-09-13T16:59:02.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b8152670 con 0x7f27b80c7760 2024-09-13T16:59:02.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27b57fa640 1 -- 172.21.15.39:0/1129222898 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27b00ad440 con 0x7f27b80c7760 2024-09-13T16:59:02.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.218+0000 7f27b57fa640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f279c066690 0x7f279c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:02.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.222+0000 7f27b57fa640 1 -- 172.21.15.39:0/1129222898 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7f27b011fb50 con 0x7f27b80c7760 2024-09-13T16:59:02.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.222+0000 7f27bcf39640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f279c066690 0x7f279c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:02.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.222+0000 7f27bcf39640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f279c066690 0x7f279c068b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00b9d0 tx=0x7f27ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:02.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.226+0000 7f27b57fa640 1 -- 172.21.15.39:0/1129222898 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27b00ec720 con 0x7f27b80c7760 2024-09-13T16:59:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.442+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27b80c45c0 con 0x7f27b80c7760 2024-09-13T16:59:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.442+0000 7f27b57fa640 1 -- 172.21.15.39:0/1129222898 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f27b00efd40 con 0x7f27b80c7760 2024-09-13T16:59:02.443 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:02.444 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:57:17.975025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":37,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5955},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5955":{"gid":5955,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1334276441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1334276441},{"type":"v1","addr":"172.21.15.39:6835","nonce":1334276441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:02.444 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f279c066690 msgr2=0x7f279c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f279c066690 0x7f279c068b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00b9d0 tx=0x7f27ac008040 comp rx=0 tx=0).stop 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80c7760 msgr2=0x7f27b814ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80c7760 0x7f27b814ae30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f27b00b0040 tx=0x7f27b00195f0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 shutdown_connections 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f279c066690 0x7f279c068b50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27b80c41e0 0x7f27b814a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27b80c7760 0x7f27b814ae30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 --2- 172.21.15.39:0/1129222898 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27b80bcf10 0x7f27b8153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 >> 172.21.15.39:0/1129222898 conn(0x7f27b80ab430 msgr2=0x7f27b814e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 shutdown_connections 2024-09-13T16:59:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:02.446+0000 7f27bf19d640 1 -- 172.21.15.39:0/1129222898 wait complete. 2024-09-13T16:59:02.463 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T16:59:04.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:04.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3be1985640 1 Processor -- start 2024-09-13T16:59:04.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3be1985640 1 -- start start 2024-09-13T16:59:04.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3be1985640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3bdc0c1810 0x7f3bdc06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:04.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3be1985640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc1576e0 0x7f3bdc157b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:04.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3be1985640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bdc0c15d0 con 0x7f3bdc06b080 2024-09-13T16:59:04.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3be1985640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bdc06d0b0 con 0x7f3bdc0c1810 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3be1985640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0c1f00 con 0x7f3bdc1576e0 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bdaffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f3bdc06b080 0x7f3bdc151390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54112/0 (socket says 172.21.15.39:54112) 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bdaffd640 1 -- 172.21.15.39:0/3345154450 learned_addr learned my addr 172.21.15.39:0/3345154450 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bdb7fe640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc1576e0 0x7f3bdc157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43674068 0 0) 0x7f3bdc0c15d0 con 0x7f3bdc06b080 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bc8003530 con 0x7f3bdc06b080 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bd3fff640 1 --2- 172.21.15.39:0/3345154450 >> v2:172.21.15.84:3300/0 conn(0x7f3bdc0c1810 0x7f3bdc06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3259062818 0 0) 0x7f3bc8003530 con 0x7f3bdc06b080 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc1576e0 msgr2=0x7f3bdc157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bda7fc640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc1576e0 0x7f3bdc157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 >> v2:172.21.15.84:3300/0 conn(0x7f3bdc0c1810 msgr2=0x7f3bdc06e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bda7fc640 1 --2- 172.21.15.39:0/3345154450 >> v2:172.21.15.84:3300/0 conn(0x7f3bdc0c1810 0x7f3bdc06e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc0c21a0 con 0x7f3bdc06b080 2024-09-13T16:59:04.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.614+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f3bcc0016a0 con 0x7f3bdc06b080 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bd3fff640 1 --2- 172.21.15.39:0/3345154450 >> v2:172.21.15.84:3300/0 conn(0x7f3bdc0c1810 0x7f3bdc06e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 >> v1:172.21.15.39:6789/0 conn(0x7f3bdc06b080 legacy=0x7f3bdc151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc8003980 0x7f3bc8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc1576e0 0x7f3bc80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bc8004d10 0x7f3bc8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc0c1f00 con 0x7f3bc8004d10 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc800b990 con 0x7f3bc8003980 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc800bb00 con 0x7f3bdc1576e0 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdb7fe640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bc8004d10 0x7f3bc8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bd3fff640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc1576e0 0x7f3bc80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdaffd640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc8003980 0x7f3bc8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdb7fe640 1 -- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc1576e0 msgr2=0x7f3bc80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdb7fe640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc1576e0 0x7f3bc80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdb7fe640 1 -- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc8003980 msgr2=0x7f3bc8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:04.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdb7fe640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc8003980 0x7f3bc8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdb7fe640 1 -- 172.21.15.39:0/3345154450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8003530 con 0x7f3bc8004d10 2024-09-13T16:59:04.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdaffd640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bc8003980 0x7f3bc8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:59:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdb7fe640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bc8004d10 0x7f3bc8005190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009fc0 tx=0x7f3bd401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a582b3eab9dbb83f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40a5c90 con 0x7f3bc8004d10 2024-09-13T16:59:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd400f040 con 0x7f3bc8004d10 2024-09-13T16:59:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bda7fc640 1 -- 172.21.15.39:0/3345154450 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40ad440 con 0x7f3bc8004d10 2024-09-13T16:59:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3be1985640 1 -- 172.21.15.39:0/3345154450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bc8004d10 msgr2=0x7f3bc8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3be1985640 1 --2- 172.21.15.39:0/3345154450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bc8004d10 0x7f3bc8005190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009fc0 tx=0x7f3bd401cc80 comp rx=0 tx=0).stop 2024-09-13T16:59:04.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3bdb7fe640 1 -- 172.21.15.39:0/3345154450 reap_dead start 2024-09-13T16:59:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3be1985640 1 -- 172.21.15.39:0/3345154450 shutdown_connections 2024-09-13T16:59:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3be1985640 1 -- 172.21.15.39:0/3345154450 >> 172.21.15.39:0/3345154450 conn(0x7f3bdc0c32a0 msgr2=0x7f3bdc0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3be1985640 1 -- 172.21.15.39:0/3345154450 shutdown_connections 2024-09-13T16:59:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.618+0000 7f3be1985640 1 -- 172.21.15.39:0/3345154450 wait complete. 2024-09-13T16:59:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 Processor -- start 2024-09-13T16:59:04.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 -- start start 2024-09-13T16:59:04.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bdc147000 0x7f3bdc14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc14d800 0x7f3bdc14ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bdc155840 0x7f3bdc155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc14c340 con 0x7f3bdc147000 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc1473e0 con 0x7f3bdc155840 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc14ccd0 con 0x7f3bdc14d800 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bdc147000 0x7f3bdc14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bdc147000 0x7f3bdc14c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56584/0 (socket says 172.21.15.39:56584) 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdaffd640 1 -- 172.21.15.39:0/3688481849 learned_addr learned my addr 172.21.15.39:0/3688481849 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdb7fe640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bdc155840 0x7f3bdc155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdaffd640 1 -- 172.21.15.39:0/3688481849 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc14d800 msgr2=0x7f3bdc14ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:04.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdaffd640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc14d800 0x7f3bdc14ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdaffd640 1 -- 172.21.15.39:0/3688481849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bdc155840 msgr2=0x7f3bdc155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdaffd640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bdc155840 0x7f3bdc155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdaffd640 1 -- 172.21.15.39:0/3688481849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc1566e0 con 0x7f3bdc147000 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdb7fe640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bdc155840 0x7f3bdc155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bdaffd640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bdc147000 0x7f3bdc14c790 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00d880 tx=0x7f3bcc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bd8ff9640 1 -- 172.21.15.39:0/3688481849 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc004cc0 con 0x7f3bdc147000 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bd8ff9640 1 -- 172.21.15.39:0/3688481849 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc004e20 con 0x7f3bdc147000 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bdc14e170 con 0x7f3bdc147000 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3bd8ff9640 1 -- 172.21.15.39:0/3688481849 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0072d0 con 0x7f3bdc147000 2024-09-13T16:59:04.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bdc14e730 con 0x7f3bdc147000 2024-09-13T16:59:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.622+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bdc151a20 con 0x7f3bdc147000 2024-09-13T16:59:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.626+0000 7f3bd8ff9640 1 -- 172.21.15.39:0/3688481849 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3bcc007430 con 0x7f3bdc147000 2024-09-13T16:59:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.626+0000 7f3bd8ff9640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bbc0666c0 0x7f3bbc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.626+0000 7f3bd8ff9640 1 -- 172.21.15.39:0/3688481849 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 7669+0+0 (secure 0 0 0) 0x7f3bcc084720 con 0x7f3bdc147000 2024-09-13T16:59:04.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.626+0000 7f3bd3fff640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bbc0666c0 0x7f3bbc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:04.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.626+0000 7f3bd3fff640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bbc0666c0 0x7f3bbc068b80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006f10 tx=0x7f3bc4006e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:04.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.630+0000 7f3bd8ff9640 1 -- 172.21.15.39:0/3688481849 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3bcc0512e0 con 0x7f3bdc147000 2024-09-13T16:59:04.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.870+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bdc06b460 con 0x7f3bdc147000 2024-09-13T16:59:04.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.870+0000 7f3bd8ff9640 1 -- 172.21.15.39:0/3688481849 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f3bcc054900 con 0x7f3bdc147000 2024-09-13T16:59:04.870 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:04.870 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:04.678462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:04.871 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 76 2024-09-13T16:59:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bbc0666c0 msgr2=0x7f3bbc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bbc0666c0 0x7f3bbc068b80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006f10 tx=0x7f3bc4006e40 comp rx=0 tx=0).stop 2024-09-13T16:59:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bdc147000 msgr2=0x7f3bdc14c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:04.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bdc147000 0x7f3bdc14c790 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00d880 tx=0x7f3bcc00dd40 comp rx=0 tx=0).stop 2024-09-13T16:59:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 shutdown_connections 2024-09-13T16:59:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3bbc0666c0 0x7f3bbc068b80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3bdc155840 0x7f3bdc155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3bdc14d800 0x7f3bdc14ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 --2- 172.21.15.39:0/3688481849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3bdc147000 0x7f3bdc14c790 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 >> 172.21.15.39:0/3688481849 conn(0x7f3bdc0c32a0 msgr2=0x7f3bdc0c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 shutdown_connections 2024-09-13T16:59:04.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:04.874+0000 7f3be1985640 1 -- 172.21.15.39:0/3688481849 wait complete. 2024-09-13T16:59:04.890 INFO:tasks.mds_thrash.fs.[b]:mds.a down, removed from mdsmap 2024-09-13T16:59:04.890 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T16:59:04.890 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e7353640 1 Processor -- start 2024-09-13T16:59:05.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e7353640 1 -- start start 2024-09-13T16:59:05.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e7353640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61e014f1b0 0x7f61e0151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:05.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e7353640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f61e006e5e0 0x7f61e00714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:05.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e7353640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e014f590 con 0x7f61e00c7470 2024-09-13T16:59:05.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e7353640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e006f660 con 0x7f61e006e5e0 2024-09-13T16:59:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e7353640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e006eae0 con 0x7f61e014f1b0 2024-09-13T16:59:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e48ee640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f61e006e5e0 0x7f61e00714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e48ee640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f61e006e5e0 0x7f61e00714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56596/0 (socket says 172.21.15.39:56596) 2024-09-13T16:59:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e48ee640 1 -- 172.21.15.39:0/995443913 learned_addr learned my addr 172.21.15.39:0/995443913 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e50ef640 1 --2- 172.21.15.39:0/995443913 >> v2:172.21.15.84:3300/0 conn(0x7f61e014f1b0 0x7f61e0151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e48ee640 1 -- 172.21.15.39:0/995443913 >> v2:172.21.15.84:3300/0 conn(0x7f61e014f1b0 msgr2=0x7f61e0151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e48ee640 1 --2- 172.21.15.39:0/995443913 >> v2:172.21.15.84:3300/0 conn(0x7f61e014f1b0 0x7f61e0151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e48ee640 1 -- 172.21.15.39:0/995443913 >> v1:172.21.15.84:6789/0 conn(0x7f61e00c7470 legacy=0x7f61e00c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e48ee640 1 -- 172.21.15.39:0/995443913 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e006edc0 con 0x7f61e006e5e0 2024-09-13T16:59:05.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61e48ee640 1 --2- 172.21.15.39:0/995443913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 0x7f61e00714e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f61e0071140 tx=0x7f61d001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38ff02a8f83f129 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:05.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.018+0000 7f61d7fff640 1 -- 172.21.15.39:0/995443913 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d00aa610 con 0x7f61e006e5e0 2024-09-13T16:59:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61d7fff640 1 -- 172.21.15.39:0/995443913 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d00a6040 con 0x7f61e006e5e0 2024-09-13T16:59:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61d7fff640 1 -- 172.21.15.39:0/995443913 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d00aa990 con 0x7f61e006e5e0 2024-09-13T16:59:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 -- 172.21.15.39:0/995443913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 msgr2=0x7f61e00714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 --2- 172.21.15.39:0/995443913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 0x7f61e00714e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f61e0071140 tx=0x7f61d001c4b0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 -- 172.21.15.39:0/995443913 shutdown_connections 2024-09-13T16:59:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 --2- 172.21.15.39:0/995443913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 0x7f61e00714e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 --2- 172.21.15.39:0/995443913 >> v2:172.21.15.84:3300/0 conn(0x7f61e014f1b0 0x7f61e0151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 -- 172.21.15.39:0/995443913 >> 172.21.15.39:0/995443913 conn(0x7f61e00a9720 msgr2=0x7f61e00a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 -- 172.21.15.39:0/995443913 shutdown_connections 2024-09-13T16:59:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 -- 172.21.15.39:0/995443913 wait complete. 2024-09-13T16:59:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 Processor -- start 2024-09-13T16:59:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 -- start start 2024-09-13T16:59:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 0x7f61e014c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f61e00c7470 0x7f61e00c1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e014f1b0 0x7f61e006a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e01ee140 con 0x7f61e006e5e0 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e006b150 con 0x7f61e014f1b0 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e7353640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e006b430 con 0x7f61e00c7470 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e50ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 0x7f61e014c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e50ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 0x7f61e014c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56598/0 (socket says 172.21.15.39:56598) 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e50ef640 1 -- 172.21.15.39:0/114667908 learned_addr learned my addr 172.21.15.39:0/114667908 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e58f0640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e014f1b0 0x7f61e006a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e48ee640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f61e00c7470 0x7f61e00c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e50ef640 1 -- 172.21.15.39:0/114667908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f61e00c7470 msgr2=0x7f61e00c1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e50ef640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f61e00c7470 0x7f61e00c1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e50ef640 1 -- 172.21.15.39:0/114667908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e014f1b0 msgr2=0x7f61e006a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e50ef640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e014f1b0 0x7f61e006a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e50ef640 1 -- 172.21.15.39:0/114667908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00094d0 con 0x7f61e006e5e0 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e58f0640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e014f1b0 0x7f61e006a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.022+0000 7f61e50ef640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 0x7f61e014c0b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00d5f0 tx=0x7f61dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61d5ffb640 1 -- 172.21.15.39:0/114667908 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0125f0 con 0x7f61e006e5e0 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61d5ffb640 1 -- 172.21.15.39:0/114667908 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc012750 con 0x7f61e006e5e0 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e006b710 con 0x7f61e006e5e0 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61d5ffb640 1 -- 172.21.15.39:0/114667908 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc012ae0 con 0x7f61e006e5e0 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e00c1540 con 0x7f61e006e5e0 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e00714e0 con 0x7f61e006e5e0 2024-09-13T16:59:05.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61d5ffb640 1 -- 172.21.15.39:0/114667908 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61dc012c40 con 0x7f61e006e5e0 2024-09-13T16:59:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61d5ffb640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f61c40666c0 0x7f61c4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61d5ffb640 1 -- 172.21.15.39:0/114667908 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f61dc084de0 con 0x7f61e006e5e0 2024-09-13T16:59:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61e48ee640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f61c40666c0 0x7f61c4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.026+0000 7f61e48ee640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f61c40666c0 0x7f61c4068b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61d00ae000 tx=0x7f61d00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:05.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.030+0000 7f61d5ffb640 1 -- 172.21.15.39:0/114667908 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f61dc0519e0 con 0x7f61e006e5e0 2024-09-13T16:59:05.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.274+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61e01520c0 con 0x7f61e006e5e0 2024-09-13T16:59:05.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.274+0000 7f61d5ffb640 1 -- 172.21.15.39:0/114667908 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f61dc007380 con 0x7f61e006e5e0 2024-09-13T16:59:05.274 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:05.275 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:04.678462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:05.275 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 76 2024-09-13T16:59:05.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f61c40666c0 msgr2=0x7f61c4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:05.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f61c40666c0 0x7f61c4068b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f61d00ae000 tx=0x7f61d00a7000 comp rx=0 tx=0).stop 2024-09-13T16:59:05.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 msgr2=0x7f61e014c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:05.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 0x7f61e014c0b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00d5f0 tx=0x7f61dc00dab0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 shutdown_connections 2024-09-13T16:59:05.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f61c40666c0 0x7f61c4068b80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e014f1b0 0x7f61e006a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f61e00c7470 0x7f61e00c1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 --2- 172.21.15.39:0/114667908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e006e5e0 0x7f61e014c0b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:05.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 >> 172.21.15.39:0/114667908 conn(0x7f61e00a9720 msgr2=0x7f61e00c3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:05.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 shutdown_connections 2024-09-13T16:59:05.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:05.278+0000 7f61e7353640 1 -- 172.21.15.39:0/114667908 wait complete. 2024-09-13T16:59:05.296 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 76, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 74, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5671": { "addr": "172.21.15.84:6833/3122796453", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 3122796453, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 3122796453, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5671, "incarnation": 71, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 38, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:58:02.976157+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5671 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 76, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5282": { "addr": "172.21.15.84:6835/1326467723", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 1326467723, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 1326467723, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5282, "incarnation": 76, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 39, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:59:04.678462+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5282 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:59:07.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:07.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f682cf87640 1 Processor -- start 2024-09-13T16:59:07.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f682cf87640 1 -- start start 2024-09-13T16:59:07.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f682cf87640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6828071bb0 0x7f682806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:07.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f682cf87640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f68280c0ac0 0x7f682806d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:07.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f682cf87640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6828150e00 con 0x7f68280ab9e0 2024-09-13T16:59:07.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f682cf87640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68280abdc0 con 0x7f68280c0ac0 2024-09-13T16:59:07.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f682cf87640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6828150250 con 0x7f6828071bb0 2024-09-13T16:59:07.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f6826d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f68280c0ac0 0x7f682806d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:07.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f6826d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f68280c0ac0 0x7f682806d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56610/0 (socket says 172.21.15.39:56610) 2024-09-13T16:59:07.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f6826d9d640 1 -- 172.21.15.39:0/536859692 learned_addr learned my addr 172.21.15.39:0/536859692 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:07.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f6825d9b640 1 --2- 172.21.15.39:0/536859692 >> v2:172.21.15.84:3300/0 conn(0x7f6828071bb0 0x7f682806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:07.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f6826d9d640 1 -- 172.21.15.39:0/536859692 >> v2:172.21.15.84:3300/0 conn(0x7f6828071bb0 msgr2=0x7f682806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:07.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f6826d9d640 1 --2- 172.21.15.39:0/536859692 >> v2:172.21.15.84:3300/0 conn(0x7f6828071bb0 0x7f682806cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f6826d9d640 1 -- 172.21.15.39:0/536859692 >> v1:172.21.15.84:6789/0 conn(0x7f68280ab9e0 legacy=0x7f68280c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:07.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.446+0000 7f6826d9d640 1 -- 172.21.15.39:0/536859692 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682814a8f0 con 0x7f68280c0ac0 2024-09-13T16:59:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f6826d9d640 1 --2- 172.21.15.39:0/536859692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 0x7f682806d120 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f68280c3920 tx=0x7f682001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1c899c0eadacf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682559a640 1 -- 172.21.15.39:0/536859692 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68200a56e0 con 0x7f68280c0ac0 2024-09-13T16:59:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682559a640 1 -- 172.21.15.39:0/536859692 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68200a5840 con 0x7f68280c0ac0 2024-09-13T16:59:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682559a640 1 -- 172.21.15.39:0/536859692 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68200a5bd0 con 0x7f68280c0ac0 2024-09-13T16:59:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682cf87640 1 -- 172.21.15.39:0/536859692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 msgr2=0x7f682806d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682cf87640 1 --2- 172.21.15.39:0/536859692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 0x7f682806d120 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f68280c3920 tx=0x7f682001c9b0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682cf87640 1 -- 172.21.15.39:0/536859692 shutdown_connections 2024-09-13T16:59:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682cf87640 1 --2- 172.21.15.39:0/536859692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 0x7f682806d120 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682cf87640 1 --2- 172.21.15.39:0/536859692 >> v2:172.21.15.84:3300/0 conn(0x7f6828071bb0 0x7f682806cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682cf87640 1 -- 172.21.15.39:0/536859692 >> 172.21.15.39:0/536859692 conn(0x7f6828069e10 msgr2=0x7f682806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682cf87640 1 -- 172.21.15.39:0/536859692 shutdown_connections 2024-09-13T16:59:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.450+0000 7f682cf87640 1 -- 172.21.15.39:0/536859692 wait complete. 2024-09-13T16:59:07.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 Processor -- start 2024-09-13T16:59:07.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 -- start start 2024-09-13T16:59:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6828071bb0 0x7f68281ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68280ab9e0 0x7f68280bf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 0x7f68280a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6828145ba0 con 0x7f68280c0ac0 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68280a92a0 con 0x7f68280ab9e0 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6828143e00 con 0x7f6828071bb0 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6826d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 0x7f68280a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6826d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 0x7f68280a9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56614/0 (socket says 172.21.15.39:56614) 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6826d9d640 1 -- 172.21.15.39:0/1699907220 learned_addr learned my addr 172.21.15.39:0/1699907220 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682659c640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6828071bb0 0x7f68281ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6825d9b640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68280ab9e0 0x7f68280bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6826d9d640 1 -- 172.21.15.39:0/1699907220 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6828071bb0 msgr2=0x7f68281ef7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6826d9d640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6828071bb0 0x7f68281ef7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6826d9d640 1 -- 172.21.15.39:0/1699907220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68280ab9e0 msgr2=0x7f68280bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6826d9d640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68280ab9e0 0x7f68280bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6826d9d640 1 -- 172.21.15.39:0/1699907220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820009b30 con 0x7f68280c0ac0 2024-09-13T16:59:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6826d9d640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 0x7f68280a9780 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f682001a5e0 tx=0x7f6820019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6816ffd640 1 -- 172.21.15.39:0/1699907220 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68200b8670 con 0x7f68280c0ac0 2024-09-13T16:59:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6816ffd640 1 -- 172.21.15.39:0/1699907220 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68200b87d0 con 0x7f68280c0ac0 2024-09-13T16:59:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6816ffd640 1 -- 172.21.15.39:0/1699907220 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68200b8b60 con 0x7f68280c0ac0 2024-09-13T16:59:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6828145300 con 0x7f68280c0ac0 2024-09-13T16:59:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68281458c0 con 0x7f68280c0ac0 2024-09-13T16:59:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68280a9460 con 0x7f68280c0ac0 2024-09-13T16:59:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.454+0000 7f6816ffd640 1 -- 172.21.15.39:0/1699907220 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f68200b8cc0 con 0x7f68280c0ac0 2024-09-13T16:59:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.458+0000 7f6816ffd640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6808066690 0x7f6808068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.458+0000 7f6816ffd640 1 -- 172.21.15.39:0/1699907220 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f682011fe00 con 0x7f68280c0ac0 2024-09-13T16:59:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.458+0000 7f682659c640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6808066690 0x7f6808068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.458+0000 7f682659c640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6808066690 0x7f6808068b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f68280ac640 tx=0x7f6818009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:07.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.462+0000 7f6816ffd640 1 -- 172.21.15.39:0/1699907220 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f68200eca00 con 0x7f68280c0ac0 2024-09-13T16:59:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68280c2b50 con 0x7f68280c0ac0 2024-09-13T16:59:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f6816ffd640 1 -- 172.21.15.39:0/1699907220 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f68200f0020 con 0x7f68280c0ac0 2024-09-13T16:59:07.713 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:07.713 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:06.904404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:07.714 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 78 2024-09-13T16:59:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6808066690 msgr2=0x7f6808068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6808066690 0x7f6808068b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f68280ac640 tx=0x7f6818009040 comp rx=0 tx=0).stop 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 msgr2=0x7f68280a9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 0x7f68280a9780 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f682001a5e0 tx=0x7f6820019750 comp rx=0 tx=0).stop 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 shutdown_connections 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6808066690 0x7f6808068b50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68280ab9e0 0x7f68280bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68280c0ac0 0x7f68280a9780 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 --2- 172.21.15.39:0/1699907220 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6828071bb0 0x7f68281ef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.714+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 >> 172.21.15.39:0/1699907220 conn(0x7f6828069e10 msgr2=0x7f682814ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.718+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 shutdown_connections 2024-09-13T16:59:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:07.718+0000 7f682cf87640 1 -- 172.21.15.39:0/1699907220 wait complete. 2024-09-13T16:59:07.734 INFO:tasks.mds_thrash.fs.[b]:no change 2024-09-13T16:59:08.894 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T16:59:08.893+0000 7f261cc18640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T16:59:09.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f5084b50640 1 Processor -- start 2024-09-13T16:59:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f5084b50640 1 -- start start 2024-09-13T16:59:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f5084b50640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f508006b080 0x7f508006e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f5084b50640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f50801576e0 0x7f5080157b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f5084b50640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50800c15d0 con 0x7f50800c1810 2024-09-13T16:59:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f5084b50640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508006d0b0 con 0x7f508006b080 2024-09-13T16:59:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f5084b50640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50800c1f00 con 0x7f50801576e0 2024-09-13T16:59:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507e59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f508006b080 0x7f508006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507dd9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f50800c1810 0x7f5080151390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54116/0 (socket says 172.21.15.39:54116) 2024-09-13T16:59:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507dd9b640 1 -- 172.21.15.39:0/1245151121 learned_addr learned my addr 172.21.15.39:0/1245151121 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507ed9d640 1 --2- 172.21.15.39:0/1245151121 >> v2:172.21.15.84:3300/0 conn(0x7f50801576e0 0x7f5080157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507e59c640 1 -- 172.21.15.39:0/1245151121 >> v2:172.21.15.84:3300/0 conn(0x7f50801576e0 msgr2=0x7f5080157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507e59c640 1 --2- 172.21.15.39:0/1245151121 >> v2:172.21.15.84:3300/0 conn(0x7f50801576e0 0x7f5080157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507e59c640 1 -- 172.21.15.39:0/1245151121 >> v1:172.21.15.39:6789/0 conn(0x7f50800c1810 legacy=0x7f5080151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507d59a640 1 -- 172.21.15.39:0/1245151121 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106037739 0 0) 0x7f50800c15d0 con 0x7f50800c1810 2024-09-13T16:59:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507e59c640 1 -- 172.21.15.39:0/1245151121 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800c21a0 con 0x7f508006b080 2024-09-13T16:59:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.886+0000 7f507e59c640 1 --2- 172.21.15.39:0/1245151121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 0x7f508006e680 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f50800a94a0 tx=0x7f507001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f83b6a7e2a709d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507d59a640 1 -- 172.21.15.39:0/1245151121 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50700ad660 con 0x7f508006b080 2024-09-13T16:59:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507d59a640 1 -- 172.21.15.39:0/1245151121 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50700ad7c0 con 0x7f508006b080 2024-09-13T16:59:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507d59a640 1 -- 172.21.15.39:0/1245151121 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50700adb50 con 0x7f508006b080 2024-09-13T16:59:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- 172.21.15.39:0/1245151121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 msgr2=0x7f508006e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:09.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 --2- 172.21.15.39:0/1245151121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 0x7f508006e680 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f50800a94a0 tx=0x7f507001c450 comp rx=0 tx=0).stop 2024-09-13T16:59:09.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- 172.21.15.39:0/1245151121 shutdown_connections 2024-09-13T16:59:09.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 --2- 172.21.15.39:0/1245151121 >> v2:172.21.15.84:3300/0 conn(0x7f50801576e0 0x7f5080157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:09.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 --2- 172.21.15.39:0/1245151121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 0x7f508006e680 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:09.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- 172.21.15.39:0/1245151121 >> 172.21.15.39:0/1245151121 conn(0x7f50800c32a0 msgr2=0x7f50800c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:09.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- 172.21.15.39:0/1245151121 shutdown_connections 2024-09-13T16:59:09.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- 172.21.15.39:0/1245151121 wait complete. 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 Processor -- start 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- start start 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 0x7f50801efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50800c1810 0x7f5080071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50801576e0 0x7f5080156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50800a7be0 con 0x7f508006b080 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50800ab520 con 0x7f50800c1810 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508006a910 con 0x7f50801576e0 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 0x7f50801efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 0x7f50801efcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56632/0 (socket says 172.21.15.39:56632) 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507e59c640 1 -- 172.21.15.39:0/3073220630 learned_addr learned my addr 172.21.15.39:0/3073220630 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507dd9b640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50800c1810 0x7f5080071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507ed9d640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50801576e0 0x7f5080156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:09.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507e59c640 1 -- 172.21.15.39:0/3073220630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50801576e0 msgr2=0x7f5080156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:09.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507e59c640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50801576e0 0x7f5080156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:09.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507e59c640 1 -- 172.21.15.39:0/3073220630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50800c1810 msgr2=0x7f5080071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:09.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507e59c640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50800c1810 0x7f5080071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:09.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507e59c640 1 -- 172.21.15.39:0/3073220630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070009420 con 0x7f508006b080 2024-09-13T16:59:09.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507dd9b640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50800c1810 0x7f5080071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:59:09.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f507e59c640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 0x7f50801efcb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f50700b0040 tx=0x7f50700191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:09.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f506effd640 1 -- 172.21.15.39:0/3073220630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50700b6610 con 0x7f508006b080 2024-09-13T16:59:09.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f506effd640 1 -- 172.21.15.39:0/3073220630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50700b6790 con 0x7f508006b080 2024-09-13T16:59:09.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50800abd80 con 0x7f508006b080 2024-09-13T16:59:09.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f506effd640 1 -- 172.21.15.39:0/3073220630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50700b6b40 con 0x7f508006b080 2024-09-13T16:59:09.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.890+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50800ab770 con 0x7f508006b080 2024-09-13T16:59:09.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.894+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50800619c0 con 0x7f508006b080 2024-09-13T16:59:09.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.894+0000 7f506effd640 1 -- 172.21.15.39:0/3073220630 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f50700b4040 con 0x7f508006b080 2024-09-13T16:59:09.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.894+0000 7f506effd640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5060066690 0x7f5060068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:09.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.894+0000 7f506effd640 1 -- 172.21.15.39:0/3073220630 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f507011ee70 con 0x7f508006b080 2024-09-13T16:59:09.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.894+0000 7f507dd9b640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5060066690 0x7f5060068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:09.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.894+0000 7f507dd9b640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5060066690 0x7f5060068b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5074007fd0 tx=0x7f5074007c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:09.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:09.898+0000 7f506effd640 1 -- 172.21.15.39:0/3073220630 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f50700ebb10 con 0x7f508006b080 2024-09-13T16:59:10.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.162+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50800ab960 con 0x7f508006b080 2024-09-13T16:59:10.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.166+0000 7f506effd640 1 -- 172.21.15.39:0/3073220630 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f50700ef130 con 0x7f508006b080 2024-09-13T16:59:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:10.164 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:10.165 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 79 2024-09-13T16:59:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5060066690 msgr2=0x7f5060068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5060066690 0x7f5060068b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5074007fd0 tx=0x7f5074007c20 comp rx=0 tx=0).stop 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 msgr2=0x7f50801efcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 0x7f50801efcb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f50700b0040 tx=0x7f50700191f0 comp rx=0 tx=0).stop 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 shutdown_connections 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5060066690 0x7f5060068b50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f50800c1810 0x7f5080071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50801576e0 0x7f5080156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 --2- 172.21.15.39:0/3073220630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508006b080 0x7f50801efcb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 >> 172.21.15.39:0/3073220630 conn(0x7f50800c32a0 msgr2=0x7f50800c53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 shutdown_connections 2024-09-13T16:59:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:10.170+0000 7f5084b50640 1 -- 172.21.15.39:0/3073220630 wait complete. 2024-09-13T16:59:10.186 INFO:tasks.mds_thrash.fs.[b]:mds.d has gained rank=0, replacing gid=5955 2024-09-13T16:59:10.186 INFO:tasks.mds_thrash.fs.[b]:waiting for 6 secs before reviving mds.a 2024-09-13T16:59:16.188 INFO:tasks.mds_thrash.fs.[b]:reviving mds.a 2024-09-13T16:59:16.189 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T16:59:16.189 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T16:59:16.192 INFO:tasks.ceph.mds.a:Started 2024-09-13T16:59:16.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:16.267 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T16:59:16.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f16a0251640 1 Processor -- start 2024-09-13T16:59:16.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f16a0251640 1 -- start start 2024-09-13T16:59:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f16a0251640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1698154bb0 0x7f16980c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f16a0251640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16980c5870 0x7f169815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f16a0251640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1698153e30 con 0x7f16980c2370 2024-09-13T16:59:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f16a0251640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16980720f0 con 0x7f1698154bb0 2024-09-13T16:59:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f16a0251640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1698151b30 con 0x7f16980c5870 2024-09-13T16:59:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169d7ec640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1698154bb0 0x7f16980c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169d7ec640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1698154bb0 0x7f16980c2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49386/0 (socket says 172.21.15.39:49386) 2024-09-13T16:59:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169d7ec640 1 -- 172.21.15.39:0/495727806 learned_addr learned my addr 172.21.15.39:0/495727806 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169dfed640 1 --1- 172.21.15.39:0/495727806 >> v1:172.21.15.39:6789/0 conn(0x7f16980c2370 0x7f16980c0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:55192/0 (socket says 172.21.15.39:55192) 2024-09-13T16:59:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169e7ee640 1 --2- 172.21.15.39:0/495727806 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16980c5870 0x7f169815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169cfeb640 1 -- 172.21.15.39:0/495727806 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848443176 0 0) 0x7f1698153e30 con 0x7f16980c2370 2024-09-13T16:59:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169cfeb640 1 -- 172.21.15.39:0/495727806 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1684003530 con 0x7f16980c2370 2024-09-13T16:59:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169d7ec640 1 -- 172.21.15.39:0/495727806 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16980c5870 msgr2=0x7f169815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169d7ec640 1 --2- 172.21.15.39:0/495727806 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16980c5870 0x7f169815e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169d7ec640 1 -- 172.21.15.39:0/495727806 >> v1:172.21.15.39:6789/0 conn(0x7f16980c2370 legacy=0x7f16980c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.334+0000 7f169d7ec640 1 -- 172.21.15.39:0/495727806 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16980a7000 con 0x7f1698154bb0 2024-09-13T16:59:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f169d7ec640 1 --2- 172.21.15.39:0/495727806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1698154bb0 0x7f16980c2750 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1688009870 tx=0x7f168801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd052d1d8998963a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f169cfeb640 1 -- 172.21.15.39:0/495727806 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16880ac620 con 0x7f1698154bb0 2024-09-13T16:59:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f169cfeb640 1 -- 172.21.15.39:0/495727806 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16880ac780 con 0x7f1698154bb0 2024-09-13T16:59:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f169cfeb640 1 -- 172.21.15.39:0/495727806 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16880acb10 con 0x7f1698154bb0 2024-09-13T16:59:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f16a0251640 1 -- 172.21.15.39:0/495727806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1698154bb0 msgr2=0x7f16980c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f16a0251640 1 --2- 172.21.15.39:0/495727806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1698154bb0 0x7f16980c2750 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1688009870 tx=0x7f168801c4d0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f16a0251640 1 -- 172.21.15.39:0/495727806 shutdown_connections 2024-09-13T16:59:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f16a0251640 1 --2- 172.21.15.39:0/495727806 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f16980c5870 0x7f169815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f16a0251640 1 --2- 172.21.15.39:0/495727806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1698154bb0 0x7f16980c2750 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f16a0251640 1 -- 172.21.15.39:0/495727806 >> 172.21.15.39:0/495727806 conn(0x7f16980a9720 msgr2=0x7f16980a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f16a0251640 1 -- 172.21.15.39:0/495727806 shutdown_connections 2024-09-13T16:59:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.338+0000 7f16a0251640 1 -- 172.21.15.39:0/495727806 wait complete. 2024-09-13T16:59:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 Processor -- start 2024-09-13T16:59:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 -- start start 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16980c2370 0x7f16981534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16980c5870 0x7f16980c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1698154bb0 0x7f169806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169814c110 con 0x7f16980c2370 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169814c360 con 0x7f16980c5870 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169806ee30 con 0x7f1698154bb0 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169dfed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16980c2370 0x7f16981534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169dfed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16980c2370 0x7f16981534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49388/0 (socket says 172.21.15.39:49388) 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169dfed640 1 -- 172.21.15.39:0/1660475225 learned_addr learned my addr 172.21.15.39:0/1660475225 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169e7ee640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1698154bb0 0x7f169806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169d7ec640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16980c5870 0x7f16980c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169dfed640 1 -- 172.21.15.39:0/1660475225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1698154bb0 msgr2=0x7f169806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169dfed640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1698154bb0 0x7f169806d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169dfed640 1 -- 172.21.15.39:0/1660475225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16980c5870 msgr2=0x7f16980c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169dfed640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16980c5870 0x7f16980c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169dfed640 1 -- 172.21.15.39:0/1660475225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16880094d0 con 0x7f16980c2370 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f169dfed640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16980c2370 0x7f16981534c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f169400c500 tx=0x7f169400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f168e7fc640 1 -- 172.21.15.39:0/1660475225 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694007500 con 0x7f16980c2370 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f168e7fc640 1 -- 172.21.15.39:0/1660475225 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694007660 con 0x7f16980c2370 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f168e7fc640 1 -- 172.21.15.39:0/1660475225 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694004a10 con 0x7f16980c2370 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16981ef970 con 0x7f16980c2370 2024-09-13T16:59:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16981eff30 con 0x7f16980c2370 2024-09-13T16:59:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.342+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16981f0120 con 0x7f16980c2370 2024-09-13T16:59:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.346+0000 7f168e7fc640 1 -- 172.21.15.39:0/1660475225 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1694004b70 con 0x7f16980c2370 2024-09-13T16:59:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.346+0000 7f168e7fc640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f167c066690 0x7f167c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.346+0000 7f168e7fc640 1 -- 172.21.15.39:0/1660475225 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f1694083bc0 con 0x7f16980c2370 2024-09-13T16:59:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.346+0000 7f169d7ec640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f167c066690 0x7f167c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:16.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.346+0000 7f169d7ec640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f167c066690 0x7f167c068b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f16880ae000 tx=0x7f1688008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:16.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.350+0000 7f168e7fc640 1 -- 172.21.15.39:0/1660475225 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f16940507c0 con 0x7f16980c2370 2024-09-13T16:59:16.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.586+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16980c2750 con 0x7f16980c2370 2024-09-13T16:59:16.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.586+0000 7f168e7fc640 1 -- 172.21.15.39:0/1660475225 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f1694018090 con 0x7f16980c2370 2024-09-13T16:59:16.585 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:16.585 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:16.585 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 79 2024-09-13T16:59:16.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f167c066690 msgr2=0x7f167c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:16.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f167c066690 0x7f167c068b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f16880ae000 tx=0x7f1688008040 comp rx=0 tx=0).stop 2024-09-13T16:59:16.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16980c2370 msgr2=0x7f16981534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:16.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16980c2370 0x7f16981534c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f169400c500 tx=0x7f169400c9c0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 shutdown_connections 2024-09-13T16:59:16.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f167c066690 0x7f167c068b50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16980c5870 0x7f16980c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1698154bb0 0x7f169806d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 --2- 172.21.15.39:0/1660475225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16980c2370 0x7f16981534c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:16.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 >> 172.21.15.39:0/1660475225 conn(0x7f16980a9720 msgr2=0x7f1698150f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:16.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 shutdown_connections 2024-09-13T16:59:16.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:16.590+0000 7f16a0251640 1 -- 172.21.15.39:0/1660475225 wait complete. 2024-09-13T16:59:16.607 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T16:59:18.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.762+0000 7f2fdb7f0640 1 Processor -- start 2024-09-13T16:59:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.762+0000 7f2fdb7f0640 1 -- start start 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fdb7f0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4151500 0x7f2fd40c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fdb7f0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fd40c47c0 0x7f2fd4154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fdb7f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd40c5a30 con 0x7f2fd4071dd0 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fdb7f0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd41576a0 con 0x7f2fd40c47c0 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fdb7f0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4157980 con 0x7f2fd4151500 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd8d8b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2fd4071dd0 0x7f2fd4154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:55206/0 (socket says 172.21.15.39:55206) 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd8d8b640 1 -- 172.21.15.39:0/2802812418 learned_addr learned my addr 172.21.15.39:0/2802812418 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd9d8d640 1 --2- 172.21.15.39:0/2802812418 >> v2:172.21.15.84:3300/0 conn(0x7f2fd40c47c0 0x7f2fd4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd958c640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4151500 0x7f2fd40c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316178537 0 0) 0x7f2fd40c5a30 con 0x7f2fd4071dd0 2024-09-13T16:59:18.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc0003530 con 0x7f2fd4071dd0 2024-09-13T16:59:18.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 285092471 0 0) 0x7f2fc0003530 con 0x7f2fd4071dd0 2024-09-13T16:59:18.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4151500 msgr2=0x7f2fd40c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:18.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4151500 0x7f2fd40c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:18.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 >> v2:172.21.15.84:3300/0 conn(0x7f2fd40c47c0 msgr2=0x7f2fd4154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:18.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 --2- 172.21.15.39:0/2802812418 >> v2:172.21.15.84:3300/0 conn(0x7f2fd40c47c0 0x7f2fd4154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:18.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4157c60 con 0x7f2fd4071dd0 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f2fc4001680 con 0x7f2fd4071dd0 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 >> v1:172.21.15.39:6789/0 conn(0x7f2fd4071dd0 legacy=0x7f2fd4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4151500 0x7f2fc0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc0003f00 0x7f2fc00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fc0004d10 0x7f2fc0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41576a0 con 0x7f2fc0004d10 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4157980 con 0x7f2fc0003f00 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc000b9e0 con 0x7f2fd4151500 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd9d8d640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fc0004d10 0x7f2fc0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd8d8b640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc0003f00 0x7f2fc00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd958c640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4151500 0x7f2fc0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd9d8d640 1 -- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4151500 msgr2=0x7f2fc0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd9d8d640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4151500 0x7f2fc0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:18.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd9d8d640 1 -- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc0003f00 msgr2=0x7f2fc00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:18.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd9d8d640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc0003f00 0x7f2fc00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:18.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.766+0000 7f2fd9d8d640 1 -- 172.21.15.39:0/2802812418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc0003530 con 0x7f2fc0004d10 2024-09-13T16:59:18.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fd9d8d640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fc0004d10 0x7f2fc0005190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e170 tx=0x7f2fcc01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c3cc86f442984f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:18.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0a5ba0 con 0x7f2fc0004d10 2024-09-13T16:59:18.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc00f040 con 0x7f2fc0004d10 2024-09-13T16:59:18.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fcbfff640 1 -- 172.21.15.39:0/2802812418 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0ad440 con 0x7f2fc0004d10 2024-09-13T16:59:18.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/2802812418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fc0004d10 msgr2=0x7f2fc0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:18.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fdb7f0640 1 --2- 172.21.15.39:0/2802812418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fc0004d10 0x7f2fc0005190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e170 tx=0x7f2fcc01cca0 comp rx=0 tx=0).stop 2024-09-13T16:59:18.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fd9d8d640 1 -- 172.21.15.39:0/2802812418 reap_dead start 2024-09-13T16:59:18.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/2802812418 shutdown_connections 2024-09-13T16:59:18.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/2802812418 >> 172.21.15.39:0/2802812418 conn(0x7f2fd40ab530 msgr2=0x7f2fd40ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:18.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/2802812418 shutdown_connections 2024-09-13T16:59:18.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/2802812418 wait complete. 2024-09-13T16:59:18.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fdb7f0640 1 Processor -- start 2024-09-13T16:59:18.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.770+0000 7f2fdb7f0640 1 -- start start 2024-09-13T16:59:18.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fdb7f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd414c690 0x7f2fd41ecde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:18.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fdb7f0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4156f20 0x7f2fd4147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fdb7f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd4152de0 0x7f2fd4147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fdb7f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41480d0 con 0x7f2fd4152de0 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fdb7f0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4151d70 con 0x7f2fd414c690 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fdb7f0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4152050 con 0x7f2fd4156f20 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd9d8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd4152de0 0x7f2fd4147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd9d8d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd4152de0 0x7f2fd4147b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49414/0 (socket says 172.21.15.39:49414) 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd9d8d640 1 -- 172.21.15.39:0/1763192134 learned_addr learned my addr 172.21.15.39:0/1763192134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd8d8b640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4156f20 0x7f2fd4147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd958c640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd414c690 0x7f2fd41ecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd9d8d640 1 -- 172.21.15.39:0/1763192134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4156f20 msgr2=0x7f2fd4147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd9d8d640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4156f20 0x7f2fd4147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:18.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd9d8d640 1 -- 172.21.15.39:0/1763192134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd414c690 msgr2=0x7f2fd41ecde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:18.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd9d8d640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd414c690 0x7f2fd41ecde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:18.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd9d8d640 1 -- 172.21.15.39:0/1763192134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd41559f0 con 0x7f2fd4152de0 2024-09-13T16:59:18.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd9d8d640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd4152de0 0x7f2fd4147b90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e170 tx=0x7f2fcc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:18.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fc9ffb640 1 -- 172.21.15.39:0/1763192134 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0bc070 con 0x7f2fd4152de0 2024-09-13T16:59:18.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fc9ffb640 1 -- 172.21.15.39:0/1763192134 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc0039a0 con 0x7f2fd4152de0 2024-09-13T16:59:18.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd4155cd0 con 0x7f2fd4152de0 2024-09-13T16:59:18.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fc9ffb640 1 -- 172.21.15.39:0/1763192134 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc003b10 con 0x7f2fd4152de0 2024-09-13T16:59:18.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd40708f0 con 0x7f2fd4152de0 2024-09-13T16:59:18.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd41521c0 con 0x7f2fd4152de0 2024-09-13T16:59:18.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fc9ffb640 1 -- 172.21.15.39:0/1763192134 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2fcc0b4040 con 0x7f2fd4152de0 2024-09-13T16:59:18.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fc9ffb640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fb80666c0 0x7f2fb8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:18.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fc9ffb640 1 -- 172.21.15.39:0/1763192134 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f2fcc11eb90 con 0x7f2fd4152de0 2024-09-13T16:59:18.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.774+0000 7f2fd958c640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fb80666c0 0x7f2fb8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:18.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.778+0000 7f2fd958c640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fb80666c0 0x7f2fb8068b80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009ae0 tx=0x7f2fd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:18.782+0000 7f2fc9ffb640 1 -- 172.21.15.39:0/1763192134 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2fcc0eb720 con 0x7f2fd4152de0 2024-09-13T16:59:19.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fd41518e0 con 0x7f2fd4152de0 2024-09-13T16:59:19.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fc9ffb640 1 -- 172.21.15.39:0/1763192134 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f2fcc0b2020 con 0x7f2fd4152de0 2024-09-13T16:59:19.001 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:19.001 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:19.002 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:19.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fb80666c0 msgr2=0x7f2fb8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:19.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fb80666c0 0x7f2fb8068b80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009ae0 tx=0x7f2fd0008040 comp rx=0 tx=0).stop 2024-09-13T16:59:19.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd4152de0 msgr2=0x7f2fd4147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:19.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd4152de0 0x7f2fd4147b90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00e170 tx=0x7f2fcc009750 comp rx=0 tx=0).stop 2024-09-13T16:59:19.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 shutdown_connections 2024-09-13T16:59:19.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fb80666c0 0x7f2fb8068b80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:19.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fd4152de0 0x7f2fd4147b90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:19.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2fd4156f20 0x7f2fd4147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:19.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 --2- 172.21.15.39:0/1763192134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fd414c690 0x7f2fd41ecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:19.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 >> 172.21.15.39:0/1763192134 conn(0x7f2fd40ab530 msgr2=0x7f2fd40c74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:19.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 shutdown_connections 2024-09-13T16:59:19.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:19.002+0000 7f2fdb7f0640 1 -- 172.21.15.39:0/1763192134 wait complete. 2024-09-13T16:59:19.025 INFO:tasks.mds_thrash.fs.[b]:mds.a reported in up:standby state 2024-09-13T16:59:19.026 INFO:tasks.mds_thrash.fs.[b]:waiting for 30 secs before thrashing 2024-09-13T16:59:27.966 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:59:27.969 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:59:40.766 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:40.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.918+0000 7f89a16e9640 1 Processor -- start 2024-09-13T16:59:40.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.918+0000 7f89a16e9640 1 -- start start 2024-09-13T16:59:40.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.918+0000 7f89a16e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f899c071660 0x7f899c157b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:40.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.918+0000 7f89a16e9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c06cf80 0x7f899c153a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:40.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.918+0000 7f89a16e9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c0c0480 con 0x7f899c072310 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.918+0000 7f89a16e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899c0c0760 con 0x7f899c071660 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.918+0000 7f89a16e9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c157030 con 0x7f899c06cf80 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f899c071660 0x7f899c157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f899c071660 0x7f899c157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45234/0 (socket says 172.21.15.39:45234) 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899bfff640 1 -- 172.21.15.39:0/132946568 learned_addr learned my addr 172.21.15.39:0/132946568 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f89a0ee8640 1 --1- 172.21.15.39:0/132946568 >> v1:172.21.15.39:6789/0 conn(0x7f899c072310 0x7f899c0c0fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37702/0 (socket says 172.21.15.39:37702) 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899b7fe640 1 --2- 172.21.15.39:0/132946568 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c06cf80 0x7f899c153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899bfff640 1 -- 172.21.15.39:0/132946568 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c06cf80 msgr2=0x7f899c153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899bfff640 1 --2- 172.21.15.39:0/132946568 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c06cf80 0x7f899c153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899bfff640 1 -- 172.21.15.39:0/132946568 >> v1:172.21.15.39:6789/0 conn(0x7f899c072310 legacy=0x7f899c0c0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899bfff640 1 -- 172.21.15.39:0/132946568 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c157310 con 0x7f899c071660 2024-09-13T16:59:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899affd640 1 -- 172.21.15.39:0/132946568 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140009806 0 0) 0x7f899c0c0480 con 0x7f899c072310 2024-09-13T16:59:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899bfff640 1 --2- 172.21.15.39:0/132946568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c071660 0x7f899c157b10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f898c009870 tx=0x7f898c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4484eb31d2c79db0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899affd640 1 -- 172.21.15.39:0/132946568 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c0aa620 con 0x7f899c071660 2024-09-13T16:59:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899affd640 1 -- 172.21.15.39:0/132946568 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c0aa780 con 0x7f899c071660 2024-09-13T16:59:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.922+0000 7f899affd640 1 -- 172.21.15.39:0/132946568 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c0aab10 con 0x7f899c071660 2024-09-13T16:59:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- 172.21.15.39:0/132946568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c071660 msgr2=0x7f899c157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 --2- 172.21.15.39:0/132946568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c071660 0x7f899c157b10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f898c009870 tx=0x7f898c01c4a0 comp rx=0 tx=0).stop 2024-09-13T16:59:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- 172.21.15.39:0/132946568 shutdown_connections 2024-09-13T16:59:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 --2- 172.21.15.39:0/132946568 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c06cf80 0x7f899c153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 --2- 172.21.15.39:0/132946568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c071660 0x7f899c157b10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- 172.21.15.39:0/132946568 >> 172.21.15.39:0/132946568 conn(0x7f899c0c30c0 msgr2=0x7f899c0c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- 172.21.15.39:0/132946568 shutdown_connections 2024-09-13T16:59:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- 172.21.15.39:0/132946568 wait complete. 2024-09-13T16:59:40.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 Processor -- start 2024-09-13T16:59:40.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- start start 2024-09-13T16:59:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899c06cf80 0x7f899c1efc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c071660 0x7f899c14d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c072310 0x7f899c1ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0a9ed0 con 0x7f899c072310 2024-09-13T16:59:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0aaed0 con 0x7f899c06cf80 2024-09-13T16:59:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c0a95a0 con 0x7f899c071660 2024-09-13T16:59:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a0ee8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c072310 0x7f899c1ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a0ee8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c072310 0x7f899c1ebeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45252/0 (socket says 172.21.15.39:45252) 2024-09-13T16:59:40.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a0ee8640 1 -- 172.21.15.39:0/2787536382 learned_addr learned my addr 172.21.15.39:0/2787536382 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f899bfff640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899c06cf80 0x7f899c1efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f899b7fe640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c071660 0x7f899c14d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a0ee8640 1 -- 172.21.15.39:0/2787536382 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c071660 msgr2=0x7f899c14d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a0ee8640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c071660 0x7f899c14d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a0ee8640 1 -- 172.21.15.39:0/2787536382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899c06cf80 msgr2=0x7f899c1efc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a0ee8640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899c06cf80 0x7f899c1efc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a0ee8640 1 -- 172.21.15.39:0/2787536382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c0094d0 con 0x7f899c072310 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f899bfff640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899c06cf80 0x7f899c1efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a0ee8640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c072310 0x7f899c1ebeb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f899400c700 tx=0x7f899400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f8998ff9640 1 -- 172.21.15.39:0/2787536382 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899400ce10 con 0x7f899c072310 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f8998ff9640 1 -- 172.21.15.39:0/2787536382 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899400a640 con 0x7f899c072310 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c0a9880 con 0x7f899c072310 2024-09-13T16:59:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f8998ff9640 1 -- 172.21.15.39:0/2787536382 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899400aa70 con 0x7f899c072310 2024-09-13T16:59:40.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.926+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c14d720 con 0x7f899c072310 2024-09-13T16:59:40.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.930+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c0aa080 con 0x7f899c072310 2024-09-13T16:59:40.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.930+0000 7f8998ff9640 1 -- 172.21.15.39:0/2787536382 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f899400abd0 con 0x7f899c072310 2024-09-13T16:59:40.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.930+0000 7f8998ff9640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8988066690 0x7f8988068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:40.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.930+0000 7f8998ff9640 1 -- 172.21.15.39:0/2787536382 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f8994084120 con 0x7f899c072310 2024-09-13T16:59:40.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.930+0000 7f899bfff640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8988066690 0x7f8988068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:40.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.930+0000 7f899bfff640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8988066690 0x7f8988068b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f898c0ae000 tx=0x7f898c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:40.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:40.934+0000 7f8998ff9640 1 -- 172.21.15.39:0/2787536382 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8994050c90 con 0x7f899c072310 2024-09-13T16:59:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.174+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f899c153a40 con 0x7f899c072310 2024-09-13T16:59:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.174+0000 7f8998ff9640 1 -- 172.21.15.39:0/2787536382 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f8994018090 con 0x7f899c072310 2024-09-13T16:59:41.175 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:41.175 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8988066690 msgr2=0x7f8988068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8988066690 0x7f8988068b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f898c0ae000 tx=0x7f898c0a7000 comp rx=0 tx=0).stop 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c072310 msgr2=0x7f899c1ebeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c072310 0x7f899c1ebeb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f899400c700 tx=0x7f899400cbc0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 shutdown_connections 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8988066690 0x7f8988068b50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c072310 0x7f899c1ebeb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f899c071660 0x7f899c14d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 --2- 172.21.15.39:0/2787536382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899c06cf80 0x7f899c1efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 >> 172.21.15.39:0/2787536382 conn(0x7f899c0c30c0 msgr2=0x7f899c0c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 shutdown_connections 2024-09-13T16:59:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.178+0000 7f89a16e9640 1 -- 172.21.15.39:0/2787536382 wait complete. 2024-09-13T16:59:41.195 INFO:tasks.mds_thrash.fs.[a]:kill mds.b (rank=0) 2024-09-13T16:59:41.196 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T16:59:41.196 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:59:41.216 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T16:59:41.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:41.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e960dc640 1 Processor -- start 2024-09-13T16:59:41.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e960dc640 1 -- start start 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e960dc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e901556d0 0x7f3e900c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e960dc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e900a72b0 0x7f3e90071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e960dc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9006feb0 con 0x7f3e900c7610 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e960dc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e900aae50 con 0x7f3e901556d0 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e960dc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90155ab0 con 0x7f3e900a72b0 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e8f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e901556d0 0x7f3e900c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e8ffff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e900a72b0 0x7f3e90071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e8f7fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e901556d0 0x7f3e900c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:48902/0 (socket says 172.21.15.39:48902) 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e87fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3e900c7610 0x7f3e90151b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:51466/0 (socket says 172.21.15.39:51466) 2024-09-13T16:59:41.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.338+0000 7f3e8f7fe640 1 -- 172.21.15.39:0/3750311432 learned_addr learned my addr 172.21.15.39:0/3750311432 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:41.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e8ffff640 1 -- 172.21.15.39:0/3750311432 >> v2:172.21.15.84:3300/0 conn(0x7f3e901556d0 msgr2=0x7f3e900c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:41.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e8ffff640 1 --2- 172.21.15.39:0/3750311432 >> v2:172.21.15.84:3300/0 conn(0x7f3e901556d0 0x7f3e900c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e8ffff640 1 -- 172.21.15.39:0/3750311432 >> v1:172.21.15.84:6789/0 conn(0x7f3e900c7610 legacy=0x7f3e90151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:41.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e8ffff640 1 -- 172.21.15.39:0/3750311432 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900c3c20 con 0x7f3e900a72b0 2024-09-13T16:59:41.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e8ffff640 1 --2- 172.21.15.39:0/3750311432 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e900a72b0 0x7f3e90071530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009d90 tx=0x7f3e8801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7b0f37b45f80d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:41.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e8effd640 1 -- 172.21.15.39:0/3750311432 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e880a5800 con 0x7f3e900a72b0 2024-09-13T16:59:41.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e8effd640 1 -- 172.21.15.39:0/3750311432 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e880a5960 con 0x7f3e900a72b0 2024-09-13T16:59:41.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e8effd640 1 -- 172.21.15.39:0/3750311432 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e880a5cf0 con 0x7f3e900a72b0 2024-09-13T16:59:41.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e960dc640 1 -- 172.21.15.39:0/3750311432 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e900a72b0 msgr2=0x7f3e90071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:41.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e960dc640 1 --2- 172.21.15.39:0/3750311432 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e900a72b0 0x7f3e90071530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009d90 tx=0x7f3e8801c850 comp rx=0 tx=0).stop 2024-09-13T16:59:41.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e960dc640 1 -- 172.21.15.39:0/3750311432 shutdown_connections 2024-09-13T16:59:41.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e960dc640 1 --2- 172.21.15.39:0/3750311432 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e900a72b0 0x7f3e90071530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e960dc640 1 --2- 172.21.15.39:0/3750311432 >> v2:172.21.15.84:3300/0 conn(0x7f3e901556d0 0x7f3e900c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.342+0000 7f3e960dc640 1 -- 172.21.15.39:0/3750311432 >> 172.21.15.39:0/3750311432 conn(0x7f3e900a97b0 msgr2=0x7f3e900a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:41.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 -- 172.21.15.39:0/3750311432 shutdown_connections 2024-09-13T16:59:41.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 -- 172.21.15.39:0/3750311432 wait complete. 2024-09-13T16:59:41.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 Processor -- start 2024-09-13T16:59:41.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 -- start start 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e900a72b0 0x7f3e901534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e901556d0 0x7f3e900c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e901539e0 0x7f3e900c1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9015b280 con 0x7f3e900a72b0 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90156da0 con 0x7f3e901539e0 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901ed6f0 con 0x7f3e901556d0 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e900a72b0 0x7f3e901534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e900a72b0 0x7f3e901534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45262/0 (socket says 172.21.15.39:45262) 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8f7fe640 1 -- 172.21.15.39:0/151199525 learned_addr learned my addr 172.21.15.39:0/151199525 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8ffff640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e901539e0 0x7f3e900c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e87fff640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e901556d0 0x7f3e900c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8f7fe640 1 -- 172.21.15.39:0/151199525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e901556d0 msgr2=0x7f3e900c13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:41.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8f7fe640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e901556d0 0x7f3e900c13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8f7fe640 1 -- 172.21.15.39:0/151199525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e901539e0 msgr2=0x7f3e900c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8f7fe640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e901539e0 0x7f3e900c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8f7fe640 1 -- 172.21.15.39:0/151199525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88009a70 con 0x7f3e900a72b0 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e87fff640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e901556d0 0x7f3e900c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8f7fe640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e900a72b0 0x7f3e901534a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000e890 tx=0x7f3e8000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8cff9640 1 -- 172.21.15.39:0/151199525 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80009620 con 0x7f3e900a72b0 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8cff9640 1 -- 172.21.15.39:0/151199525 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8000c940 con 0x7f3e900a72b0 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9015afa0 con 0x7f3e900a72b0 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e8cff9640 1 -- 172.21.15.39:0/151199525 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8000ccf0 con 0x7f3e900a72b0 2024-09-13T16:59:41.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.346+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e90156a30 con 0x7f3e900a72b0 2024-09-13T16:59:41.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.350+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e90156bf0 con 0x7f3e900a72b0 2024-09-13T16:59:41.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.350+0000 7f3e8cff9640 1 -- 172.21.15.39:0/151199525 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e80013470 con 0x7f3e900a72b0 2024-09-13T16:59:41.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.350+0000 7f3e8cff9640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e70066690 0x7f3e70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:41.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.350+0000 7f3e8cff9640 1 -- 172.21.15.39:0/151199525 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f3e80084a00 con 0x7f3e900a72b0 2024-09-13T16:59:41.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.350+0000 7f3e87fff640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e70066690 0x7f3e70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:41.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.350+0000 7f3e87fff640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e70066690 0x7f3e70068b50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3e901eb2b0 tx=0x7f3e78007d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:41.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.358+0000 7f3e8cff9640 1 -- 172.21.15.39:0/151199525 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3e80051630 con 0x7f3e900a72b0 2024-09-13T16:59:41.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e901565b0 con 0x7f3e900a72b0 2024-09-13T16:59:41.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e8cff9640 1 -- 172.21.15.39:0/151199525 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f3e901565b0 con 0x7f3e900a72b0 2024-09-13T16:59:41.557 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:41.557 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:41.557 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e70066690 msgr2=0x7f3e70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e70066690 0x7f3e70068b50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3e901eb2b0 tx=0x7f3e78007d70 comp rx=0 tx=0).stop 2024-09-13T16:59:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e900a72b0 msgr2=0x7f3e901534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e900a72b0 0x7f3e901534a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000e890 tx=0x7f3e8000ed50 comp rx=0 tx=0).stop 2024-09-13T16:59:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 shutdown_connections 2024-09-13T16:59:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e70066690 0x7f3e70068b50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e901539e0 0x7f3e900c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3e901556d0 0x7f3e900c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 --2- 172.21.15.39:0/151199525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e900a72b0 0x7f3e901534a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:41.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 >> 172.21.15.39:0/151199525 conn(0x7f3e900a97b0 msgr2=0x7f3e900a9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:41.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 shutdown_connections 2024-09-13T16:59:41.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:41.558+0000 7f3e960dc640 1 -- 172.21.15.39:0/151199525 wait complete. 2024-09-13T16:59:41.576 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:59:43.578 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12b6295640 1 Processor -- start 2024-09-13T16:59:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12b6295640 1 -- start start 2024-09-13T16:59:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12b6295640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f12b0151860 0x7f12b0157d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12b6295640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12b00bf4f0 0x7f12b0153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12b6295640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b00c1340 con 0x7f12b00611c0 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12b6295640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b01578f0 con 0x7f12b0151860 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12b6295640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b01582c0 con 0x7f12b00bf4f0 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12affff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f12b0151860 0x7f12b0157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12affff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f12b0151860 0x7f12b0157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45278/0 (socket says 172.21.15.39:45278) 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12affff640 1 -- 172.21.15.39:0/793461646 learned_addr learned my addr 172.21.15.39:0/793461646 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12affff640 1 -- 172.21.15.39:0/793461646 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12b00bf4f0 msgr2=0x7f12b0153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12affff640 1 --2- 172.21.15.39:0/793461646 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12b00bf4f0 0x7f12b0153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12affff640 1 -- 172.21.15.39:0/793461646 >> v1:172.21.15.84:6789/0 conn(0x7f12b00611c0 legacy=0x7f12b00bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.714+0000 7f12affff640 1 -- 172.21.15.39:0/793461646 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b00c1590 con 0x7f12b0151860 2024-09-13T16:59:43.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12affff640 1 --2- 172.21.15.39:0/793461646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 0x7f12b0157d80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009e50 tx=0x7f12a801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3498273c9010063c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:43.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12aeffd640 1 -- 172.21.15.39:0/793461646 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80a57b0 con 0x7f12b0151860 2024-09-13T16:59:43.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12aeffd640 1 -- 172.21.15.39:0/793461646 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a80a5910 con 0x7f12b0151860 2024-09-13T16:59:43.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12aeffd640 1 -- 172.21.15.39:0/793461646 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80a5ca0 con 0x7f12b0151860 2024-09-13T16:59:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 -- 172.21.15.39:0/793461646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 msgr2=0x7f12b0157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 --2- 172.21.15.39:0/793461646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 0x7f12b0157d80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009e50 tx=0x7f12a801c930 comp rx=0 tx=0).stop 2024-09-13T16:59:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 -- 172.21.15.39:0/793461646 shutdown_connections 2024-09-13T16:59:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 --2- 172.21.15.39:0/793461646 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12b00bf4f0 0x7f12b0153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 --2- 172.21.15.39:0/793461646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 0x7f12b0157d80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 -- 172.21.15.39:0/793461646 >> 172.21.15.39:0/793461646 conn(0x7f12b0069e10 msgr2=0x7f12b006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 -- 172.21.15.39:0/793461646 shutdown_connections 2024-09-13T16:59:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 -- 172.21.15.39:0/793461646 wait complete. 2024-09-13T16:59:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 Processor -- start 2024-09-13T16:59:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 -- start start 2024-09-13T16:59:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b00611c0 0x7f12b01494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12b00bf4f0 0x7f12b0150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 0x7f12b01eccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b014acb0 con 0x7f12b0151860 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b014af90 con 0x7f12b00611c0 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b6295640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b014b270 con 0x7f12b00bf4f0 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b4832640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 0x7f12b01eccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b4832640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 0x7f12b01eccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45284/0 (socket says 172.21.15.39:45284) 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.718+0000 7f12b4832640 1 -- 172.21.15.39:0/828916114 learned_addr learned my addr 172.21.15.39:0/828916114 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12affff640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b00611c0 0x7f12b01494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12af7fe640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12b00bf4f0 0x7f12b0150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12b4832640 1 -- 172.21.15.39:0/828916114 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12b00bf4f0 msgr2=0x7f12b0150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12b4832640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12b00bf4f0 0x7f12b0150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12b4832640 1 -- 172.21.15.39:0/828916114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b00611c0 msgr2=0x7f12b01494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12b4832640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b00611c0 0x7f12b01494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12b4832640 1 -- 172.21.15.39:0/828916114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8009b30 con 0x7f12b0151860 2024-09-13T16:59:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12b4832640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 0x7f12b01eccc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f12a000b410 tx=0x7f12a000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12acff9640 1 -- 172.21.15.39:0/828916114 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0011830 con 0x7f12b0151860 2024-09-13T16:59:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12acff9640 1 -- 172.21.15.39:0/828916114 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a0011990 con 0x7f12b0151860 2024-09-13T16:59:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b01ec7a0 con 0x7f12b0151860 2024-09-13T16:59:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12acff9640 1 -- 172.21.15.39:0/828916114 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0011d40 con 0x7f12b0151860 2024-09-13T16:59:43.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b01ec400 con 0x7f12b0151860 2024-09-13T16:59:43.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12acff9640 1 -- 172.21.15.39:0/828916114 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f12a001a400 con 0x7f12b0151860 2024-09-13T16:59:43.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b01ec990 con 0x7f12b0151860 2024-09-13T16:59:43.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12acff9640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f129c0666c0 0x7f129c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:43.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12affff640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f129c0666c0 0x7f129c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:43.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12acff9640 1 -- 172.21.15.39:0/828916114 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f12a00849e0 con 0x7f12b0151860 2024-09-13T16:59:43.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.722+0000 7f12affff640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f129c0666c0 0x7f129c068b80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12a80ae000 tx=0x7f12a80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:43.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.726+0000 7f12acff9640 1 -- 172.21.15.39:0/828916114 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f12a00515e0 con 0x7f12b0151860 2024-09-13T16:59:43.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.926+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12b0153df0 con 0x7f12b0151860 2024-09-13T16:59:43.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.926+0000 7f12acff9640 1 -- 172.21.15.39:0/828916114 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f12a0015090 con 0x7f12b0151860 2024-09-13T16:59:43.925 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:43.925 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:43.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.926+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f129c0666c0 msgr2=0x7f129c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:43.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.926+0000 7f12b6295640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f129c0666c0 0x7f129c068b80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12a80ae000 tx=0x7f12a80a7070 comp rx=0 tx=0).stop 2024-09-13T16:59:43.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.926+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 msgr2=0x7f12b01eccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:43.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.926+0000 7f12b6295640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 0x7f12b01eccc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f12a000b410 tx=0x7f12a000b8d0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.930+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 shutdown_connections 2024-09-13T16:59:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.930+0000 7f12b6295640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f129c0666c0 0x7f129c068b80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.930+0000 7f12b6295640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12b0151860 0x7f12b01eccc0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.930+0000 7f12b6295640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12b00bf4f0 0x7f12b0150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.930+0000 7f12b6295640 1 --2- 172.21.15.39:0/828916114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12b00611c0 0x7f12b01494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.930+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 >> 172.21.15.39:0/828916114 conn(0x7f12b0069e10 msgr2=0x7f12b0151e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.930+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 shutdown_connections 2024-09-13T16:59:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:43.930+0000 7f12b6295640 1 -- 172.21.15.39:0/828916114 wait complete. 2024-09-13T16:59:43.945 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:59:45.946 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa05b4a9640 1 Processor -- start 2024-09-13T16:59:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa05b4a9640 1 -- start start 2024-09-13T16:59:46.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa05b4a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa0540c21d0 0x7fa0540aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:46.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa05b4a9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540c3ae0 con 0x7fa05406aac0 2024-09-13T16:59:46.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa05b4a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540abb30 con 0x7fa05414b850 2024-09-13T16:59:46.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa05b4a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0540ab340 con 0x7fa0540c21d0 2024-09-13T16:59:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa058a44640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa05406aac0 0x7fa054151c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37704/0 (socket says 172.21.15.39:37704) 2024-09-13T16:59:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa058a44640 1 -- 172.21.15.39:0/1408275162 learned_addr learned my addr 172.21.15.39:0/1408275162 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa059245640 1 --2- 172.21.15.39:0/1408275162 >> v2:172.21.15.84:3300/0 conn(0x7fa0540c21d0 0x7fa0540aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448212778 0 0) 0x7fa0540c3ae0 con 0x7fa05406aac0 2024-09-13T16:59:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa03c003530 con 0x7fa05406aac0 2024-09-13T16:59:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3019389895 0 0) 0x7fa03c003530 con 0x7fa05406aac0 2024-09-13T16:59:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 >> v2:172.21.15.84:3300/0 conn(0x7fa0540c21d0 msgr2=0x7fa0540aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 --2- 172.21.15.39:0/1408275162 >> v2:172.21.15.84:3300/0 conn(0x7fa0540c21d0 0x7fa0540aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 >> v1:172.21.15.84:6789/0 conn(0x7fa05414b850 legacy=0x7fa05415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540abe10 con 0x7fa05406aac0 2024-09-13T16:59:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fa048001680 con 0x7fa05406aac0 2024-09-13T16:59:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 >> v1:172.21.15.39:6789/0 conn(0x7fa05406aac0 legacy=0x7fa054151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa03c003980 0x7fa03c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa03c0047d0 0x7fa03c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa03c005190 0x7fa03c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540ab340 con 0x7fa03c003980 2024-09-13T16:59:46.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c00c130 con 0x7fa03c005190 2024-09-13T16:59:46.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c00c2f0 con 0x7fa03c0047d0 2024-09-13T16:59:46.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa059245640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa03c003980 0x7fa03c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:46.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa059a46640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa03c005190 0x7fa03c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:46.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa058a44640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa03c0047d0 0x7fa03c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:46.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa059245640 1 -- 172.21.15.39:0/1408275162 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa03c0047d0 msgr2=0x7fa03c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:46.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa059245640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa03c0047d0 0x7fa03c004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa059245640 1 -- 172.21.15.39:0/1408275162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa03c005190 msgr2=0x7fa03c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:46.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa059245640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa03c005190 0x7fa03c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa059245640 1 -- 172.21.15.39:0/1408275162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c003530 con 0x7fa03c003980 2024-09-13T16:59:46.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.086+0000 7fa059a46640 1 -- 172.21.15.39:0/1408275162 reap_dead start 2024-09-13T16:59:46.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa059245640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa03c003980 0x7fa03c003da0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009fc0 tx=0x7fa04c01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bf179e08546dab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0a5b50 con 0x7fa03c003980 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c00f040 con 0x7fa03c003980 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa053fff640 1 -- 172.21.15.39:0/1408275162 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0ad440 con 0x7fa03c003980 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa05b4a9640 1 -- 172.21.15.39:0/1408275162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa03c003980 msgr2=0x7fa03c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa05b4a9640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa03c003980 0x7fa03c003da0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009fc0 tx=0x7fa04c01cc50 comp rx=0 tx=0).stop 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa05b4a9640 1 -- 172.21.15.39:0/1408275162 shutdown_connections 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa05b4a9640 1 --2- 172.21.15.39:0/1408275162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa03c003980 0x7fa03c003da0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa05b4a9640 1 -- 172.21.15.39:0/1408275162 >> 172.21.15.39:0/1408275162 conn(0x7fa054069e10 msgr2=0x7fa05406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa05b4a9640 1 -- 172.21.15.39:0/1408275162 shutdown_connections 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.090+0000 7fa05b4a9640 1 -- 172.21.15.39:0/1408275162 wait complete. 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 Processor -- start 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 -- start start 2024-09-13T16:59:46.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa03c003980 0x7fa054150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa054060c00 0x7fa05414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa05414d880 0x7fa05414a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540bf940 con 0x7fa054060c00 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05414cdc0 con 0x7fa05414d880 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540bfb90 con 0x7fa03c003980 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa058a44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa054060c00 0x7fa05414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa058a44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa054060c00 0x7fa05414c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45300/0 (socket says 172.21.15.39:45300) 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa058a44640 1 -- 172.21.15.39:0/3649441068 learned_addr learned my addr 172.21.15.39:0/3649441068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa059a46640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa05414d880 0x7fa05414a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa059245640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa03c003980 0x7fa054150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa058a44640 1 -- 172.21.15.39:0/3649441068 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa03c003980 msgr2=0x7fa054150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa058a44640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa03c003980 0x7fa054150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa058a44640 1 -- 172.21.15.39:0/3649441068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa05414d880 msgr2=0x7fa05414a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:46.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa058a44640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa05414d880 0x7fa05414a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa058a44640 1 -- 172.21.15.39:0/3649441068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540bfe70 con 0x7fa054060c00 2024-09-13T16:59:46.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa059a46640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa05414d880 0x7fa05414a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:59:46.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa058a44640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa054060c00 0x7fa05414c1f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa04800d830 tx=0x7fa04800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:46.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa051ffb640 1 -- 172.21.15.39:0/3649441068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0480074e0 con 0x7fa054060c00 2024-09-13T16:59:46.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa051ffb640 1 -- 172.21.15.39:0/3649441068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa048007640 con 0x7fa054060c00 2024-09-13T16:59:46.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05414d080 con 0x7fa054060c00 2024-09-13T16:59:46.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa051ffb640 1 -- 172.21.15.39:0/3649441068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0480049f0 con 0x7fa054060c00 2024-09-13T16:59:46.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05414c730 con 0x7fa054060c00 2024-09-13T16:59:46.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0540c3560 con 0x7fa054060c00 2024-09-13T16:59:46.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa051ffb640 1 -- 172.21.15.39:0/3649441068 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa048004b50 con 0x7fa054060c00 2024-09-13T16:59:46.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.094+0000 7fa051ffb640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa0400666c0 0x7fa040068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:46.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.098+0000 7fa051ffb640 1 -- 172.21.15.39:0/3649441068 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7fa048084320 con 0x7fa054060c00 2024-09-13T16:59:46.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.098+0000 7fa059245640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa0400666c0 0x7fa040068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:46.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.098+0000 7fa059245640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa0400666c0 0x7fa040068b80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0af000 tx=0x7fa04c004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:46.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.102+0000 7fa051ffb640 1 -- 172.21.15.39:0/3649441068 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa048050fe0 con 0x7fa054060c00 2024-09-13T16:59:46.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.298+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa05414d530 con 0x7fa054060c00 2024-09-13T16:59:46.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa051ffb640 1 -- 172.21.15.39:0/3649441068 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fa048054600 con 0x7fa054060c00 2024-09-13T16:59:46.299 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:46.300 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:46.300 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa0400666c0 msgr2=0x7fa040068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:46.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa0400666c0 0x7fa040068b80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0af000 tx=0x7fa04c004ac0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa054060c00 msgr2=0x7fa05414c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa054060c00 0x7fa05414c1f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa04800d830 tx=0x7fa04800dcf0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 shutdown_connections 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa0400666c0 0x7fa040068b80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa05414d880 0x7fa05414a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa054060c00 0x7fa05414c1f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 --2- 172.21.15.39:0/3649441068 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa03c003980 0x7fa054150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 >> 172.21.15.39:0/3649441068 conn(0x7fa054069e10 msgr2=0x7fa0541e03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 shutdown_connections 2024-09-13T16:59:46.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:46.302+0000 7fa05b4a9640 1 -- 172.21.15.39:0/3649441068 wait complete. 2024-09-13T16:59:46.318 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:59:48.319 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:48.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307c229640 1 Processor -- start 2024-09-13T16:59:48.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307c229640 1 -- start start 2024-09-13T16:59:48.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307c229640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f307406c870 0x7f30740aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:48.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307c229640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f307415c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:48.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307c229640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3074152a80 con 0x7f30740aabc0 2024-09-13T16:59:48.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307c229640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30740c30f0 con 0x7f307406c870 2024-09-13T16:59:48.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307c229640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3074151e00 con 0x7f30740c40b0 2024-09-13T16:59:48.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f30797c4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f307406c870 0x7f30740aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:48.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307a7c6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f307415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:48.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307a7c6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f307415c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:52914/0 (socket says 172.21.15.39:52914) 2024-09-13T16:59:48.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307a7c6640 1 -- 172.21.15.39:0/2923449903 learned_addr learned my addr 172.21.15.39:0/2923449903 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:48.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f3079fc5640 1 --1- 172.21.15.39:0/2923449903 >> v1:172.21.15.84:6789/0 conn(0x7f30740aabc0 0x7f30740c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:51486/0 (socket says 172.21.15.39:51486) 2024-09-13T16:59:48.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f30797c4640 1 -- 172.21.15.39:0/2923449903 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 msgr2=0x7f307415c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:48.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f30797c4640 1 --2- 172.21.15.39:0/2923449903 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f307415c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f30797c4640 1 -- 172.21.15.39:0/2923449903 >> v1:172.21.15.84:6789/0 conn(0x7f30740aabc0 legacy=0x7f30740c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:48.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f30797c4640 1 -- 172.21.15.39:0/2923449903 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30741520e0 con 0x7f307406c870 2024-09-13T16:59:48.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.458+0000 7f307a7c6640 1 --2- 172.21.15.39:0/2923449903 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f307415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:59:48.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f30797c4640 1 --2- 172.21.15.39:0/2923449903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307406c870 0x7f30740aafa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f30640097c0 tx=0x7f306401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d06415f2f42d177d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:48.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f3078fc3640 1 -- 172.21.15.39:0/2923449903 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30640ac640 con 0x7f307406c870 2024-09-13T16:59:48.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f3078fc3640 1 -- 172.21.15.39:0/2923449903 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30640ac7a0 con 0x7f307406c870 2024-09-13T16:59:48.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f3078fc3640 1 -- 172.21.15.39:0/2923449903 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30640acb30 con 0x7f307406c870 2024-09-13T16:59:48.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 -- 172.21.15.39:0/2923449903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307406c870 msgr2=0x7f30740aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:48.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 --2- 172.21.15.39:0/2923449903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307406c870 0x7f30740aafa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f30640097c0 tx=0x7f306401c3e0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 -- 172.21.15.39:0/2923449903 shutdown_connections 2024-09-13T16:59:48.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 --2- 172.21.15.39:0/2923449903 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f307415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 --2- 172.21.15.39:0/2923449903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f307406c870 0x7f30740aafa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 -- 172.21.15.39:0/2923449903 >> 172.21.15.39:0/2923449903 conn(0x7f3074069e10 msgr2=0x7f307406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:48.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 -- 172.21.15.39:0/2923449903 shutdown_connections 2024-09-13T16:59:48.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 -- 172.21.15.39:0/2923449903 wait complete. 2024-09-13T16:59:48.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 Processor -- start 2024-09-13T16:59:48.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.462+0000 7f307c229640 1 -- start start 2024-09-13T16:59:48.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307c229640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f307406c870 0x7f307406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:48.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307c229640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30740aabc0 0x7f30741ed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:48.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307c229640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f30741edde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:48.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307c229640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740bfa70 con 0x7f307406c870 2024-09-13T16:59:48.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307c229640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307414bd60 con 0x7f30740aabc0 2024-09-13T16:59:48.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307c229640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30740bf790 con 0x7f30740c40b0 2024-09-13T16:59:48.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f3079fc5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f307406c870 0x7f307406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f3079fc5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f307406c870 0x7f307406f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45304/0 (socket says 172.21.15.39:45304) 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f3079fc5640 1 -- 172.21.15.39:0/2590964163 learned_addr learned my addr 172.21.15.39:0/2590964163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f30797c4640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30740aabc0 0x7f30741ed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307a7c6640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f30741edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f3079fc5640 1 -- 172.21.15.39:0/2590964163 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 msgr2=0x7f30741edde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f3079fc5640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f30741edde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f3079fc5640 1 -- 172.21.15.39:0/2590964163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30740aabc0 msgr2=0x7f30741ed8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f3079fc5640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30740aabc0 0x7f30741ed8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f3079fc5640 1 -- 172.21.15.39:0/2590964163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c009b30 con 0x7f307406c870 2024-09-13T16:59:48.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307a7c6640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f30741edde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:59:48.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f3079fc5640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f307406c870 0x7f307406f250 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f307000c420 tx=0x7f307000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:48.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f306a7fc640 1 -- 172.21.15.39:0/2590964163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700074e0 con 0x7f307406c870 2024-09-13T16:59:48.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f306a7fc640 1 -- 172.21.15.39:0/2590964163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070007640 con 0x7f307406c870 2024-09-13T16:59:48.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3064009420 con 0x7f307406c870 2024-09-13T16:59:48.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f306a7fc640 1 -- 172.21.15.39:0/2590964163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070004a30 con 0x7f307406c870 2024-09-13T16:59:48.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30741ee730 con 0x7f307406c870 2024-09-13T16:59:48.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307414bf20 con 0x7f307406c870 2024-09-13T16:59:48.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f306a7fc640 1 -- 172.21.15.39:0/2590964163 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3070004b90 con 0x7f307406c870 2024-09-13T16:59:48.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f306a7fc640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30580666c0 0x7f3058068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:48.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f306a7fc640 1 -- 172.21.15.39:0/2590964163 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f3070084260 con 0x7f307406c870 2024-09-13T16:59:48.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f30797c4640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30580666c0 0x7f3058068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:48.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.466+0000 7f30797c4640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30580666c0 0x7f3058068b80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f30640ae000 tx=0x7f3064008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:48.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.470+0000 7f306a7fc640 1 -- 172.21.15.39:0/2590964163 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3070050dd0 con 0x7f307406c870 2024-09-13T16:59:48.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.670+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3074152a80 con 0x7f307406c870 2024-09-13T16:59:48.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.670+0000 7f306a7fc640 1 -- 172.21.15.39:0/2590964163 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f3074152a80 con 0x7f307406c870 2024-09-13T16:59:48.669 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:48.669 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:48.670 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:48.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30580666c0 msgr2=0x7f3058068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30580666c0 0x7f3058068b80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f30640ae000 tx=0x7f3064008040 comp rx=0 tx=0).stop 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f307406c870 msgr2=0x7f307406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f307406c870 0x7f307406f250 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f307000c420 tx=0x7f307000c8e0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 shutdown_connections 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f30580666c0 0x7f3058068b80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30740aabc0 0x7f30741ed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f30740c40b0 0x7f30741edde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 --2- 172.21.15.39:0/2590964163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f307406c870 0x7f307406f250 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 >> 172.21.15.39:0/2590964163 conn(0x7f3074069e10 msgr2=0x7f30740c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 shutdown_connections 2024-09-13T16:59:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:48.674+0000 7f307c229640 1 -- 172.21.15.39:0/2590964163 wait complete. 2024-09-13T16:59:48.688 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:59:49.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:49.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.162+0000 7f8861339640 1 Processor -- start 2024-09-13T16:59:49.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.162+0000 7f8861339640 1 -- start start 2024-09-13T16:59:49.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f8861339640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f885c06c870 0x7f885c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f8861339640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f885c0c40b0 0x7f885c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f8861339640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885c152a80 con 0x7f885c0aabc0 2024-09-13T16:59:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f8861339640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c0c30f0 con 0x7f885c06c870 2024-09-13T16:59:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f8861339640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c151e00 con 0x7f885c0c40b0 2024-09-13T16:59:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885a7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f885c06c870 0x7f885c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885a7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f885c06c870 0x7f885c0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45312/0 (socket says 172.21.15.39:45312) 2024-09-13T16:59:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885a7fc640 1 -- 172.21.15.39:0/3226664153 learned_addr learned my addr 172.21.15.39:0/3226664153 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885affd640 1 --1- 172.21.15.39:0/3226664153 >> v1:172.21.15.39:6789/0 conn(0x7f885c0aabc0 0x7f885c0c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37718/0 (socket says 172.21.15.39:37718) 2024-09-13T16:59:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885b7fe640 1 --2- 172.21.15.39:0/3226664153 >> v2:172.21.15.84:3300/0 conn(0x7f885c0c40b0 0x7f885c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885a7fc640 1 -- 172.21.15.39:0/3226664153 >> v2:172.21.15.84:3300/0 conn(0x7f885c0c40b0 msgr2=0x7f885c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:49.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885a7fc640 1 --2- 172.21.15.39:0/3226664153 >> v2:172.21.15.84:3300/0 conn(0x7f885c0c40b0 0x7f885c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885a7fc640 1 -- 172.21.15.39:0/3226664153 >> v1:172.21.15.39:6789/0 conn(0x7f885c0aabc0 legacy=0x7f885c0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885a7fc640 1 -- 172.21.15.39:0/3226664153 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c1520e0 con 0x7f885c06c870 2024-09-13T16:59:49.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f885a7fc640 1 --2- 172.21.15.39:0/3226664153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c06c870 0x7f885c0aafa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f885c06f220 tx=0x7f884c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b509885035872b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:49.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f8859ffb640 1 -- 172.21.15.39:0/3226664153 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0ab660 con 0x7f885c06c870 2024-09-13T16:59:49.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f8859ffb640 1 -- 172.21.15.39:0/3226664153 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c0ab7c0 con 0x7f885c06c870 2024-09-13T16:59:49.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.166+0000 7f8859ffb640 1 -- 172.21.15.39:0/3226664153 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0abb50 con 0x7f885c06c870 2024-09-13T16:59:49.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- 172.21.15.39:0/3226664153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c06c870 msgr2=0x7f885c0aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 --2- 172.21.15.39:0/3226664153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c06c870 0x7f885c0aafa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f885c06f220 tx=0x7f884c01c460 comp rx=0 tx=0).stop 2024-09-13T16:59:49.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- 172.21.15.39:0/3226664153 shutdown_connections 2024-09-13T16:59:49.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 --2- 172.21.15.39:0/3226664153 >> v2:172.21.15.84:3300/0 conn(0x7f885c0c40b0 0x7f885c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 --2- 172.21.15.39:0/3226664153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c06c870 0x7f885c0aafa0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- 172.21.15.39:0/3226664153 >> 172.21.15.39:0/3226664153 conn(0x7f885c069e10 msgr2=0x7f885c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:49.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- 172.21.15.39:0/3226664153 shutdown_connections 2024-09-13T16:59:49.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- 172.21.15.39:0/3226664153 wait complete. 2024-09-13T16:59:49.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 Processor -- start 2024-09-13T16:59:49.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- start start 2024-09-13T16:59:49.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885c06c870 0x7f885c14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c0aabc0 0x7f885c150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f885c0c40b0 0x7f885c141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c14aa90 con 0x7f885c0aabc0 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c14a720 con 0x7f885c06c870 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c1edc90 con 0x7f885c0c40b0 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c0aabc0 0x7f885c150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c0aabc0 0x7f885c150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45324/0 (socket says 172.21.15.39:45324) 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885a7fc640 1 -- 172.21.15.39:0/2989180387 learned_addr learned my addr 172.21.15.39:0/2989180387 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885affd640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885c06c870 0x7f885c14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885b7fe640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f885c0c40b0 0x7f885c141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885a7fc640 1 -- 172.21.15.39:0/2989180387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f885c0c40b0 msgr2=0x7f885c141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885a7fc640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f885c0c40b0 0x7f885c141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885a7fc640 1 -- 172.21.15.39:0/2989180387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885c06c870 msgr2=0x7f885c14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885a7fc640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885c06c870 0x7f885c14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885a7fc640 1 -- 172.21.15.39:0/2989180387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c0094d0 con 0x7f885c0aabc0 2024-09-13T16:59:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f885a7fc640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c0aabc0 0x7f885c150c10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f884c019070 tx=0x7f884c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:49.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f88437fe640 1 -- 172.21.15.39:0/2989180387 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0b4800 con 0x7f885c0aabc0 2024-09-13T16:59:49.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f88437fe640 1 -- 172.21.15.39:0/2989180387 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c0b4960 con 0x7f885c0aabc0 2024-09-13T16:59:49.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c14b760 con 0x7f885c0aabc0 2024-09-13T16:59:49.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f88437fe640 1 -- 172.21.15.39:0/2989180387 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0b4cd0 con 0x7f885c0aabc0 2024-09-13T16:59:49.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.170+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c1edea0 con 0x7f885c0aabc0 2024-09-13T16:59:49.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.174+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885c0c3e60 con 0x7f885c0aabc0 2024-09-13T16:59:49.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.174+0000 7f88437fe640 1 -- 172.21.15.39:0/2989180387 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f884c0a9070 con 0x7f885c0aabc0 2024-09-13T16:59:49.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.174+0000 7f88437fe640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8848066690 0x7f8848068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.174+0000 7f88437fe640 1 -- 172.21.15.39:0/2989180387 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f884c11f440 con 0x7f885c0aabc0 2024-09-13T16:59:49.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.174+0000 7f885affd640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8848066690 0x7f8848068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.174+0000 7f885affd640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8848066690 0x7f8848068b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8850000f70 tx=0x7f8850007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:49.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.178+0000 7f88437fe640 1 -- 172.21.15.39:0/2989180387 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f884c0ec160 con 0x7f885c0aabc0 2024-09-13T16:59:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.374+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f885c06c650 con 0x7f885c0aabc0 2024-09-13T16:59:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.374+0000 7f88437fe640 1 -- 172.21.15.39:0/2989180387 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f884c0ef780 con 0x7f885c0aabc0 2024-09-13T16:59:49.374 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:49.374 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8848066690 msgr2=0x7f8848068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8848066690 0x7f8848068b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8850000f70 tx=0x7f8850007cb0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c0aabc0 msgr2=0x7f885c150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c0aabc0 0x7f885c150c10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f884c019070 tx=0x7f884c01aca0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 shutdown_connections 2024-09-13T16:59:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8848066690 0x7f8848068b50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f885c0c40b0 0x7f885c141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f885c0aabc0 0x7f885c150c10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 --2- 172.21.15.39:0/2989180387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f885c06c870 0x7f885c14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 >> 172.21.15.39:0/2989180387 conn(0x7f885c069e10 msgr2=0x7f885c14ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 shutdown_connections 2024-09-13T16:59:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.378+0000 7f8861339640 1 -- 172.21.15.39:0/2989180387 wait complete. 2024-09-13T16:59:49.392 INFO:tasks.mds_thrash.fs.[b]:kill mds.d (rank=0) 2024-09-13T16:59:49.392 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T16:59:49.392 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T16:59:49.407 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T16:59:49.407 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:49.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.518+0000 7f3623cac640 1 Processor -- start 2024-09-13T16:59:49.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.518+0000 7f3623cac640 1 -- start start 2024-09-13T16:59:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3623cac640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 0x7f361c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3623cac640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f361c0c1f50 0x7f361c06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3623cac640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c0c2780 con 0x7f361c06d300 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3623cac640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c0c2a60 con 0x7f361c0c1f50 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3623cac640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0c2d40 con 0x7f361c14e0c0 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3621247640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f361c0c1f50 0x7f361c06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3621247640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f361c0c1f50 0x7f361c06cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45336/0 (socket says 172.21.15.39:45336) 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3621247640 1 -- 172.21.15.39:0/2723454783 learned_addr learned my addr 172.21.15.39:0/2723454783 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3622249640 1 --1- 172.21.15.39:0/2723454783 >> v1:172.21.15.39:6789/0 conn(0x7f361c06d300 0x7f361c0c4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37724/0 (socket says 172.21.15.39:37724) 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3621a48640 1 --2- 172.21.15.39:0/2723454783 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 0x7f361c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3621247640 1 -- 172.21.15.39:0/2723454783 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 msgr2=0x7f361c0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3621247640 1 --2- 172.21.15.39:0/2723454783 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 0x7f361c0a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3621247640 1 -- 172.21.15.39:0/2723454783 >> v1:172.21.15.39:6789/0 conn(0x7f361c06d300 legacy=0x7f361c0c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3621247640 1 -- 172.21.15.39:0/2723454783 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c0c3020 con 0x7f361c0c1f50 2024-09-13T16:59:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3620a46640 1 -- 172.21.15.39:0/2723454783 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674091488 0 0) 0x7f361c0c2780 con 0x7f361c06d300 2024-09-13T16:59:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3621247640 1 --2- 172.21.15.39:0/2723454783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 0x7f361c06cdc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f360c009870 tx=0x7f360c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2709465807ae08c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3620a46640 1 -- 172.21.15.39:0/2723454783 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0ab600 con 0x7f361c0c1f50 2024-09-13T16:59:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3620a46640 1 -- 172.21.15.39:0/2723454783 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c0a7050 con 0x7f361c0c1f50 2024-09-13T16:59:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.522+0000 7f3620a46640 1 -- 172.21.15.39:0/2723454783 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0ab980 con 0x7f361c0c1f50 2024-09-13T16:59:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- 172.21.15.39:0/2723454783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 msgr2=0x7f361c06cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 --2- 172.21.15.39:0/2723454783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 0x7f361c06cdc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f360c009870 tx=0x7f360c01c4a0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- 172.21.15.39:0/2723454783 shutdown_connections 2024-09-13T16:59:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 --2- 172.21.15.39:0/2723454783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 0x7f361c06cdc0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 --2- 172.21.15.39:0/2723454783 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 0x7f361c0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- 172.21.15.39:0/2723454783 >> 172.21.15.39:0/2723454783 conn(0x7f361c069e10 msgr2=0x7f361c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- 172.21.15.39:0/2723454783 shutdown_connections 2024-09-13T16:59:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- 172.21.15.39:0/2723454783 wait complete. 2024-09-13T16:59:49.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 Processor -- start 2024-09-13T16:59:49.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- start start 2024-09-13T16:59:49.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f361c06d300 0x7f361c14d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 0x7f361c14fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 0x7f361c071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c14d880 con 0x7f361c0c1f50 2024-09-13T16:59:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c150900 con 0x7f361c06d300 2024-09-13T16:59:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1e8350 con 0x7f361c14e0c0 2024-09-13T16:59:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621247640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 0x7f361c14fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621247640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 0x7f361c14fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45352/0 (socket says 172.21.15.39:45352) 2024-09-13T16:59:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621247640 1 -- 172.21.15.39:0/555526902 learned_addr learned my addr 172.21.15.39:0/555526902 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3622249640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 0x7f361c071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621a48640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f361c06d300 0x7f361c14d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621247640 1 -- 172.21.15.39:0/555526902 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 msgr2=0x7f361c071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621247640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 0x7f361c071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621247640 1 -- 172.21.15.39:0/555526902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f361c06d300 msgr2=0x7f361c14d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621247640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f361c06d300 0x7f361c14d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621247640 1 -- 172.21.15.39:0/555526902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c0094d0 con 0x7f361c0c1f50 2024-09-13T16:59:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3622249640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 0x7f361c071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:59:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3621247640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 0x7f361c14fd20 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f360c0ae040 tx=0x7f360c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f36127fc640 1 -- 172.21.15.39:0/555526902 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0b4420 con 0x7f361c0c1f50 2024-09-13T16:59:49.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f36127fc640 1 -- 172.21.15.39:0/555526902 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c0b4580 con 0x7f361c0c1f50 2024-09-13T16:59:49.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c1e9320 con 0x7f361c0c1f50 2024-09-13T16:59:49.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f36127fc640 1 -- 172.21.15.39:0/555526902 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0b4930 con 0x7f361c0c1f50 2024-09-13T16:59:49.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c1e8560 con 0x7f361c0c1f50 2024-09-13T16:59:49.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.526+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c06c840 con 0x7f361c0c1f50 2024-09-13T16:59:49.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.530+0000 7f36127fc640 1 -- 172.21.15.39:0/555526902 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f360c0b2070 con 0x7f361c0c1f50 2024-09-13T16:59:49.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.530+0000 7f36127fc640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3600066690 0x7f3600068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:49.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.530+0000 7f36127fc640 1 -- 172.21.15.39:0/555526902 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f360c11e4f0 con 0x7f361c0c1f50 2024-09-13T16:59:49.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.530+0000 7f3621a48640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3600066690 0x7f3600068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:49.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.530+0000 7f3621a48640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3600066690 0x7f3600068b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f36180096f0 tx=0x7f3618008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.534+0000 7f36127fc640 1 -- 172.21.15.39:0/555526902 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f360c0eb0f0 con 0x7f361c0c1f50 2024-09-13T16:59:49.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.722+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f361c06a570 con 0x7f361c0c1f50 2024-09-13T16:59:49.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.726+0000 7f36127fc640 1 -- 172.21.15.39:0/555526902 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f360c0ee710 con 0x7f361c0c1f50 2024-09-13T16:59:49.724 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:49.725 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:49.725 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:49.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3600066690 msgr2=0x7f3600068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3600066690 0x7f3600068b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f36180096f0 tx=0x7f3618008040 comp rx=0 tx=0).stop 2024-09-13T16:59:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 msgr2=0x7f361c14fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 0x7f361c14fd20 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f360c0ae040 tx=0x7f360c019240 comp rx=0 tx=0).stop 2024-09-13T16:59:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 shutdown_connections 2024-09-13T16:59:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3600066690 0x7f3600068b50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f361c14e0c0 0x7f361c071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f361c0c1f50 0x7f361c14fd20 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 --2- 172.21.15.39:0/555526902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f361c06d300 0x7f361c14d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 >> 172.21.15.39:0/555526902 conn(0x7f361c069e10 msgr2=0x7f361c148760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:49.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 shutdown_connections 2024-09-13T16:59:49.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:49.730+0000 7f3623cac640 1 -- 172.21.15.39:0/555526902 wait complete. 2024-09-13T16:59:49.744 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:59:50.689 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:50.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046bfff640 1 Processor -- start 2024-09-13T16:59:50.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046bfff640 1 -- start start 2024-09-13T16:59:50.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0464151650 0x7f0464070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:50.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f04641548a0 0x7f04640c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:50.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046bfff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0464061630 con 0x7f04640611c0 2024-09-13T16:59:50.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046bfff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0464157670 con 0x7f0464151650 2024-09-13T16:59:50.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046bfff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0464061880 con 0x7f04641548a0 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f0469d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0464151650 0x7f0464070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046a59c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f04641548a0 0x7f04640c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046959a640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f04640611c0 0x7f0464157a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:36642/0 (socket says 172.21.15.39:36642) 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f0469d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0464151650 0x7f0464070e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:40964/0 (socket says 172.21.15.39:40964) 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046959a640 1 -- 172.21.15.39:0/887153804 learned_addr learned my addr 172.21.15.39:0/887153804 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046a59c640 1 --2- 172.21.15.39:0/887153804 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f04641548a0 0x7f04640c1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:60322/0 (socket says 172.21.15.39:60322) 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f0469d9b640 1 -- 172.21.15.39:0/887153804 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f04641548a0 msgr2=0x7f04640c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f0469d9b640 1 --2- 172.21.15.39:0/887153804 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f04641548a0 0x7f04640c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f0469d9b640 1 -- 172.21.15.39:0/887153804 >> v1:172.21.15.84:6789/0 conn(0x7f04640611c0 legacy=0x7f0464157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f0469d9b640 1 -- 172.21.15.39:0/887153804 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0464154d30 con 0x7f0464151650 2024-09-13T16:59:50.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.826+0000 7f046a59c640 1 --2- 172.21.15.39:0/887153804 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f04641548a0 0x7f04640c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T16:59:50.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f0469d9b640 1 --2- 172.21.15.39:0/887153804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0464151650 0x7f0464070e30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f04600097c0 tx=0x7f046001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0283db40338ba82 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:50.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f0468d99640 1 -- 172.21.15.39:0/887153804 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600ab600 con 0x7f0464151650 2024-09-13T16:59:50.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f0468d99640 1 -- 172.21.15.39:0/887153804 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04600a7050 con 0x7f0464151650 2024-09-13T16:59:50.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f0468d99640 1 -- 172.21.15.39:0/887153804 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600ab980 con 0x7f0464151650 2024-09-13T16:59:50.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 -- 172.21.15.39:0/887153804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0464151650 msgr2=0x7f0464070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:50.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 --2- 172.21.15.39:0/887153804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0464151650 0x7f0464070e30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f04600097c0 tx=0x7f046001c410 comp rx=0 tx=0).stop 2024-09-13T16:59:50.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 -- 172.21.15.39:0/887153804 shutdown_connections 2024-09-13T16:59:50.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 --2- 172.21.15.39:0/887153804 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f04641548a0 0x7f04640c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:50.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 --2- 172.21.15.39:0/887153804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0464151650 0x7f0464070e30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:50.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 -- 172.21.15.39:0/887153804 >> 172.21.15.39:0/887153804 conn(0x7f04640c2a50 msgr2=0x7f04640c2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:50.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 -- 172.21.15.39:0/887153804 shutdown_connections 2024-09-13T16:59:50.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 -- 172.21.15.39:0/887153804 wait complete. 2024-09-13T16:59:50.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 Processor -- start 2024-09-13T16:59:50.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.830+0000 7f046bfff640 1 -- start start 2024-09-13T16:59:50.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04640611c0 0x7f046414f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:50.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0464151650 0x7f04641564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04641548a0 0x7f04641ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046bfff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04641ee450 con 0x7f04640611c0 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046bfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0464150970 con 0x7f04641548a0 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046bfff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0464150690 con 0x7f0464151650 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f0469d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04640611c0 0x7f046414f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f0469d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04640611c0 0x7f046414f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33036/0 (socket says 172.21.15.39:33036) 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f0469d9b640 1 -- 172.21.15.39:0/2999516502 learned_addr learned my addr 172.21.15.39:0/2999516502 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046959a640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0464151650 0x7f04641564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046a59c640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04641548a0 0x7f04641ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f0469d9b640 1 -- 172.21.15.39:0/2999516502 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0464151650 msgr2=0x7f04641564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f0469d9b640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0464151650 0x7f04641564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f0469d9b640 1 -- 172.21.15.39:0/2999516502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04641548a0 msgr2=0x7f04641ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f0469d9b640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04641548a0 0x7f04641ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f0469d9b640 1 -- 172.21.15.39:0/2999516502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c009a70 con 0x7f04640611c0 2024-09-13T16:59:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f0469d9b640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04640611c0 0x7f046414f750 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f04600ae040 tx=0x7f04600191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f045a7fc640 1 -- 172.21.15.39:0/2999516502 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600b4430 con 0x7f04640611c0 2024-09-13T16:59:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f045a7fc640 1 -- 172.21.15.39:0/2999516502 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04600b4590 con 0x7f04640611c0 2024-09-13T16:59:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f045a7fc640 1 -- 172.21.15.39:0/2999516502 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600b4940 con 0x7f04640611c0 2024-09-13T16:59:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0460009420 con 0x7f04640611c0 2024-09-13T16:59:50.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04641ee070 con 0x7f04640611c0 2024-09-13T16:59:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04641ee590 con 0x7f04640611c0 2024-09-13T16:59:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f045a7fc640 1 -- 172.21.15.39:0/2999516502 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f04600b2040 con 0x7f04640611c0 2024-09-13T16:59:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f045a7fc640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0448066690 0x7f0448068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f045a7fc640 1 -- 172.21.15.39:0/2999516502 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f046011e800 con 0x7f04640611c0 2024-09-13T16:59:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046959a640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0448066690 0x7f0448068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.834+0000 7f046959a640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0448066690 0x7f0448068b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f046414e0e0 tx=0x7f0454009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:50.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:50.838+0000 7f045a7fc640 1 -- 172.21.15.39:0/2999516502 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f04641ee590 con 0x7f04640611c0 2024-09-13T16:59:51.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.030+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04640c1960 con 0x7f04640611c0 2024-09-13T16:59:51.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.030+0000 7f045a7fc640 1 -- 172.21.15.39:0/2999516502 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f04600eb360 con 0x7f04640611c0 2024-09-13T16:59:51.029 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:51.029 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:51.029 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:51.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.030+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0448066690 msgr2=0x7f0448068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:51.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.030+0000 7f046bfff640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0448066690 0x7f0448068b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f046414e0e0 tx=0x7f0454009040 comp rx=0 tx=0).stop 2024-09-13T16:59:51.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.030+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04640611c0 msgr2=0x7f046414f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:51.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.030+0000 7f046bfff640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04640611c0 0x7f046414f750 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f04600ae040 tx=0x7f04600191b0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.030+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 shutdown_connections 2024-09-13T16:59:51.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.034+0000 7f046bfff640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0448066690 0x7f0448068b50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.034+0000 7f046bfff640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04641548a0 0x7f04641ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.034+0000 7f046bfff640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0464151650 0x7f04641564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.034+0000 7f046bfff640 1 --2- 172.21.15.39:0/2999516502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04640611c0 0x7f046414f750 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.034+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 >> 172.21.15.39:0/2999516502 conn(0x7f04640c2a50 msgr2=0x7f046414a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:51.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.034+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 shutdown_connections 2024-09-13T16:59:51.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.034+0000 7f046bfff640 1 -- 172.21.15.39:0/2999516502 wait complete. 2024-09-13T16:59:51.046 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:59:51.745 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:51.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104d0db640 1 Processor -- start 2024-09-13T16:59:51.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104d0db640 1 -- start start 2024-09-13T16:59:51.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104d0db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f10480aa9a0 0x7f1048154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:51.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104d0db640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f104806b540 0x7f104806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:51.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104d0db640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10480aad80 con 0x7f10480c1680 2024-09-13T16:59:51.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104d0db640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10480c0b80 con 0x7f10480aa9a0 2024-09-13T16:59:51.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104d0db640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10480c10d0 con 0x7f104806b540 2024-09-13T16:59:51.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f1046d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f10480aa9a0 0x7f1048154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:51.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104659c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f10480c1680 0x7f10480c47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53784/0 (socket says 172.21.15.39:53784) 2024-09-13T16:59:51.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104659c640 1 -- 172.21.15.39:0/777454039 learned_addr learned my addr 172.21.15.39:0/777454039 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:51.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f104759e640 1 --2- 172.21.15.39:0/777454039 >> v2:172.21.15.84:3300/0 conn(0x7f104806b540 0x7f104806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:51.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f1045d9b640 1 -- 172.21.15.39:0/777454039 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49935649 0 0) 0x7f10300014a0 con 0x7f10480c1680 2024-09-13T16:59:51.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f1046d9d640 1 -- 172.21.15.39:0/777454039 >> v2:172.21.15.84:3300/0 conn(0x7f104806b540 msgr2=0x7f104806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:51.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.882+0000 7f1046d9d640 1 --2- 172.21.15.39:0/777454039 >> v2:172.21.15.84:3300/0 conn(0x7f104806b540 0x7f104806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f1046d9d640 1 -- 172.21.15.39:0/777454039 >> v1:172.21.15.39:6789/0 conn(0x7f10480c1680 legacy=0x7f10480c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:51.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f1046d9d640 1 -- 172.21.15.39:0/777454039 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1048155000 con 0x7f10480aa9a0 2024-09-13T16:59:51.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f1046d9d640 1 --2- 172.21.15.39:0/777454039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10480aa9a0 0x7f1048154ac0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f10480c3250 tx=0x7f104001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8ad9b8d98e5002 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:51.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f1045d9b640 1 -- 172.21.15.39:0/777454039 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400ab600 con 0x7f10480aa9a0 2024-09-13T16:59:51.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f1045d9b640 1 -- 172.21.15.39:0/777454039 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10400a7050 con 0x7f10480aa9a0 2024-09-13T16:59:51.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f1045d9b640 1 -- 172.21.15.39:0/777454039 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400ab980 con 0x7f10480aa9a0 2024-09-13T16:59:51.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f104d0db640 1 -- 172.21.15.39:0/777454039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10480aa9a0 msgr2=0x7f1048154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:51.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f104d0db640 1 --2- 172.21.15.39:0/777454039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10480aa9a0 0x7f1048154ac0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f10480c3250 tx=0x7f104001c3f0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f104d0db640 1 -- 172.21.15.39:0/777454039 shutdown_connections 2024-09-13T16:59:51.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f104d0db640 1 --2- 172.21.15.39:0/777454039 >> v2:172.21.15.84:3300/0 conn(0x7f104806b540 0x7f104806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f104d0db640 1 --2- 172.21.15.39:0/777454039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10480aa9a0 0x7f1048154ac0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f104d0db640 1 -- 172.21.15.39:0/777454039 >> 172.21.15.39:0/777454039 conn(0x7f104806a6a0 msgr2=0x7f10480c5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:51.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f104d0db640 1 -- 172.21.15.39:0/777454039 shutdown_connections 2024-09-13T16:59:51.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.886+0000 7f104d0db640 1 -- 172.21.15.39:0/777454039 wait complete. 2024-09-13T16:59:51.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 Processor -- start 2024-09-13T16:59:51.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 -- start start 2024-09-13T16:59:51.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f104806b540 0x7f10481444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:51.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10480aa9a0 0x7f1048148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:51.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10480c1680 0x7f10481edc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:51.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10481ed820 con 0x7f104806b540 2024-09-13T16:59:51.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10481485e0 con 0x7f10480aa9a0 2024-09-13T16:59:51.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1048071f90 con 0x7f10480c1680 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f1046d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f104806b540 0x7f10481444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f1046d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f104806b540 0x7f10481444e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33040/0 (socket says 172.21.15.39:33040) 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f1046d9d640 1 -- 172.21.15.39:0/1734212759 learned_addr learned my addr 172.21.15.39:0/1734212759 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104759e640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10480c1680 0x7f10481edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104659c640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10480aa9a0 0x7f1048148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f1046d9d640 1 -- 172.21.15.39:0/1734212759 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10480c1680 msgr2=0x7f10481edc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f1046d9d640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10480c1680 0x7f10481edc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f1046d9d640 1 -- 172.21.15.39:0/1734212759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10480aa9a0 msgr2=0x7f1048148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f1046d9d640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10480aa9a0 0x7f1048148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f1046d9d640 1 -- 172.21.15.39:0/1734212759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1040009420 con 0x7f104806b540 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f1046d9d640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f104806b540 0x7f10481444e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f10400ae040 tx=0x7f1040019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f103f7fe640 1 -- 172.21.15.39:0/1734212759 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400b4840 con 0x7f104806b540 2024-09-13T16:59:51.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f103f7fe640 1 -- 172.21.15.39:0/1734212759 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10400b49c0 con 0x7f104806b540 2024-09-13T16:59:51.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f103f7fe640 1 -- 172.21.15.39:0/1734212759 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400b4d70 con 0x7f104806b540 2024-09-13T16:59:51.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1048070f50 con 0x7f104806b540 2024-09-13T16:59:51.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1048070990 con 0x7f104806b540 2024-09-13T16:59:51.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104806b970 con 0x7f104806b540 2024-09-13T16:59:51.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f103f7fe640 1 -- 172.21.15.39:0/1734212759 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f10400b2040 con 0x7f104806b540 2024-09-13T16:59:51.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f103f7fe640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1028066690 0x7f1028068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:51.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f103f7fe640 1 -- 172.21.15.39:0/1734212759 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f104011ecc0 con 0x7f104806b540 2024-09-13T16:59:51.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.890+0000 7f104659c640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1028066690 0x7f1028068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:51.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.894+0000 7f104659c640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1028066690 0x7f1028068b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f10480c2d00 tx=0x7f1030005ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:51.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:51.898+0000 7f103f7fe640 1 -- 172.21.15.39:0/1734212759 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f10400eb8c0 con 0x7f104806b540 2024-09-13T16:59:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10480bfcc0 con 0x7f104806b540 2024-09-13T16:59:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f103f7fe640 1 -- 172.21.15.39:0/1734212759 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f10400b8090 con 0x7f104806b540 2024-09-13T16:59:52.088 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:52.088 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:52.089 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1028066690 msgr2=0x7f1028068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1028066690 0x7f1028068b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f10480c2d00 tx=0x7f1030005ec0 comp rx=0 tx=0).stop 2024-09-13T16:59:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f104806b540 msgr2=0x7f10481444e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f104806b540 0x7f10481444e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f10400ae040 tx=0x7f1040019190 comp rx=0 tx=0).stop 2024-09-13T16:59:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 shutdown_connections 2024-09-13T16:59:52.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1028066690 0x7f1028068b50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:52.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10480aa9a0 0x7f1048148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:52.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10480c1680 0x7f10481edc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:52.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 --2- 172.21.15.39:0/1734212759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f104806b540 0x7f10481444e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:52.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.090+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 >> 172.21.15.39:0/1734212759 conn(0x7f104806a6a0 msgr2=0x7f10480c1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:52.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.094+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 shutdown_connections 2024-09-13T16:59:52.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:52.094+0000 7f104d0db640 1 -- 172.21.15.39:0/1734212759 wait complete. 2024-09-13T16:59:52.107 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:59:53.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a9c92640 1 Processor -- start 2024-09-13T16:59:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a9c92640 1 -- start start 2024-09-13T16:59:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a9c92640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33a40c7800 0x7f33a4154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a9c92640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a40c5a30 con 0x7f33a4071dd0 2024-09-13T16:59:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a9c92640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a41576a0 con 0x7f33a4151500 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a9c92640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a4157980 con 0x7f33a40c7800 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f339bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f33a4071dd0 0x7f33a4154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53786/0 (socket says 172.21.15.39:53786) 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f339bfff640 1 -- 172.21.15.39:0/3964712962 learned_addr learned my addr 172.21.15.39:0/3964712962 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a3fff640 1 --2- 172.21.15.39:0/3964712962 >> v2:172.21.15.84:3300/0 conn(0x7f33a40c7800 0x7f33a4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486308262 0 0) 0x7f33a40c5a30 con 0x7f33a4071dd0 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3390003530 con 0x7f33a4071dd0 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 301102809 0 0) 0x7f3390003530 con 0x7f33a4071dd0 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 >> v2:172.21.15.84:3300/0 conn(0x7f33a40c7800 msgr2=0x7f33a4154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 --2- 172.21.15.39:0/3964712962 >> v2:172.21.15.84:3300/0 conn(0x7f33a40c7800 0x7f33a4154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 >> v1:172.21.15.84:6789/0 conn(0x7f33a4151500 legacy=0x7f33a40c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4157c60 con 0x7f33a4071dd0 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f338c0016a0 con 0x7f33a4071dd0 2024-09-13T16:59:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 >> v1:172.21.15.39:6789/0 conn(0x7f33a4071dd0 legacy=0x7f33a4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f3390003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33900047d0 0x7f3390004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3390005190 0x7f339000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4157980 con 0x7f3390003980 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339000c130 con 0x7f33900047d0 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339000c2f0 con 0x7f3390005190 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a37fe640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f3390003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f339bfff640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33900047d0 0x7f3390004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a3fff640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3390005190 0x7f339000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a37fe640 1 -- 172.21.15.39:0/3964712962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3390005190 msgr2=0x7f339000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a37fe640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3390005190 0x7f339000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a37fe640 1 -- 172.21.15.39:0/3964712962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33900047d0 msgr2=0x7f3390004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a37fe640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33900047d0 0x7f3390004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a37fe640 1 -- 172.21.15.39:0/3964712962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390003530 con 0x7f3390003980 2024-09-13T16:59:53.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.186+0000 7f33a3fff640 1 -- 172.21.15.39:0/3964712962 reap_dead start 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a37fe640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f3390003da0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3394001780 tx=0x7f339401ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96835574b6b3f8c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33940050d0 con 0x7f3390003980 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33940a7050 con 0x7f3390003980 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a2ffd640 1 -- 172.21.15.39:0/3964712962 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394005490 con 0x7f3390003980 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 -- 172.21.15.39:0/3964712962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 msgr2=0x7f3390003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f3390003da0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3394001780 tx=0x7f339401ce50 comp rx=0 tx=0).stop 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 -- 172.21.15.39:0/3964712962 shutdown_connections 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 --2- 172.21.15.39:0/3964712962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f3390003da0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 -- 172.21.15.39:0/3964712962 >> 172.21.15.39:0/3964712962 conn(0x7f33a40ab530 msgr2=0x7f33a40ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 -- 172.21.15.39:0/3964712962 shutdown_connections 2024-09-13T16:59:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 -- 172.21.15.39:0/3964712962 wait complete. 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 Processor -- start 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 -- start start 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f33a4149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.190+0000 7f33a9c92640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33a41538a0 0x7f33a40aa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a9c92640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a4149f50 0x7f33a40aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a9c92640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4149a00 con 0x7f3390003980 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a9c92640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a414ee60 con 0x7f33a4149f50 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a9c92640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41efdb0 con 0x7f33a41538a0 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f33a4149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f33a4149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33056/0 (socket says 172.21.15.39:33056) 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a37fe640 1 -- 172.21.15.39:0/38637424 learned_addr learned my addr 172.21.15.39:0/38637424 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a3fff640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a4149f50 0x7f33a40aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f339bfff640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33a41538a0 0x7f33a40aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a37fe640 1 -- 172.21.15.39:0/38637424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33a41538a0 msgr2=0x7f33a40aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a37fe640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33a41538a0 0x7f33a40aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a37fe640 1 -- 172.21.15.39:0/38637424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a4149f50 msgr2=0x7f33a40aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a37fe640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a4149f50 0x7f33a40aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a37fe640 1 -- 172.21.15.39:0/38637424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a41f0090 con 0x7f3390003980 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f339bfff640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33a41538a0 0x7f33a40aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a37fe640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f33a4149260 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3394005040 tx=0x7f3390005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a0ff9640 1 -- 172.21.15.39:0/38637424 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394007100 con 0x7f3390003980 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a0ff9640 1 -- 172.21.15.39:0/38637424 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3394007260 con 0x7f3390003980 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a0ff9640 1 -- 172.21.15.39:0/38637424 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33940075f0 con 0x7f3390003980 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a4156870 con 0x7f3390003980 2024-09-13T16:59:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a4149c50 con 0x7f3390003980 2024-09-13T16:59:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a4156a30 con 0x7f3390003980 2024-09-13T16:59:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a0ff9640 1 -- 172.21.15.39:0/38637424 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f33940067e0 con 0x7f3390003980 2024-09-13T16:59:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a0ff9640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33840666c0 0x7f3384068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f33a0ff9640 1 -- 172.21.15.39:0/38637424 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f339411eb90 con 0x7f3390003980 2024-09-13T16:59:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f339bfff640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33840666c0 0x7f3384068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.194+0000 7f339bfff640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33840666c0 0x7f3384068b80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f338c001ec0 tx=0x7f338c001d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.198+0000 7f33a0ff9640 1 -- 172.21.15.39:0/38637424 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f33940eb790 con 0x7f3390003980 2024-09-13T16:59:53.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.390+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33a4154580 con 0x7f3390003980 2024-09-13T16:59:53.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.390+0000 7f33a0ff9640 1 -- 172.21.15.39:0/38637424 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7f33940eedb0 con 0x7f3390003980 2024-09-13T16:59:53.390 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:53.390 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:53.390 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33840666c0 msgr2=0x7f3384068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33840666c0 0x7f3384068b80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f338c001ec0 tx=0x7f338c001d90 comp rx=0 tx=0).stop 2024-09-13T16:59:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 msgr2=0x7f33a4149260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f33a4149260 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3394005040 tx=0x7f3390005800 comp rx=0 tx=0).stop 2024-09-13T16:59:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 shutdown_connections 2024-09-13T16:59:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33840666c0 0x7f3384068b80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33a4149f50 0x7f33a40aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33a41538a0 0x7f33a40aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 --2- 172.21.15.39:0/38637424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3390003980 0x7f33a4149260 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 >> 172.21.15.39:0/38637424 conn(0x7f33a40ab530 msgr2=0x7f33a414a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 shutdown_connections 2024-09-13T16:59:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:53.394+0000 7f33a9c92640 1 -- 172.21.15.39:0/38637424 wait complete. 2024-09-13T16:59:53.408 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T16:59:54.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:54.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.246+0000 7fe77fa4d640 1 Processor -- start 2024-09-13T16:59:54.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.246+0000 7fe77fa4d640 1 -- start start 2024-09-13T16:59:54.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7780a4e70 0x7fe77806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:54.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe77806d300 0x7fe7780c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:54.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7780c2780 con 0x7fe77814e0c0 2024-09-13T16:59:54.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780c2a60 con 0x7fe7780a4e70 2024-09-13T16:59:54.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780c2d40 con 0x7fe77806d300 2024-09-13T16:59:54.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77cfe8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7780a4e70 0x7fe77806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:54.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77cfe8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7780a4e70 0x7fe77806cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33064/0 (socket says 172.21.15.39:33064) 2024-09-13T16:59:54.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77cfe8640 1 -- 172.21.15.39:0/2967192075 learned_addr learned my addr 172.21.15.39:0/2967192075 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:54.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77dfea640 1 --2- 172.21.15.39:0/2967192075 >> v2:172.21.15.84:3300/0 conn(0x7fe77806d300 0x7fe7780c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:54.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77cfe8640 1 -- 172.21.15.39:0/2967192075 >> v2:172.21.15.84:3300/0 conn(0x7fe77806d300 msgr2=0x7fe7780c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:54.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77cfe8640 1 --2- 172.21.15.39:0/2967192075 >> v2:172.21.15.84:3300/0 conn(0x7fe77806d300 0x7fe7780c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77cfe8640 1 -- 172.21.15.39:0/2967192075 >> v1:172.21.15.84:6789/0 conn(0x7fe77814e0c0 legacy=0x7fe7780c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:54.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77cfe8640 1 -- 172.21.15.39:0/2967192075 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780c3020 con 0x7fe7780a4e70 2024-09-13T16:59:54.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77cfe8640 1 --2- 172.21.15.39:0/2967192075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 0x7fe77806cdc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe7780c3f40 tx=0x7fe76801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b7121a613234d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:54.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe76ffff640 1 -- 172.21.15.39:0/2967192075 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7780c5490 con 0x7fe7780a4e70 2024-09-13T16:59:54.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe76ffff640 1 -- 172.21.15.39:0/2967192075 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7680a7050 con 0x7fe7780a4e70 2024-09-13T16:59:54.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe76ffff640 1 -- 172.21.15.39:0/2967192075 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680ab820 con 0x7fe7780a4e70 2024-09-13T16:59:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 -- 172.21.15.39:0/2967192075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 msgr2=0x7fe77806cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 --2- 172.21.15.39:0/2967192075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 0x7fe77806cdc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe7780c3f40 tx=0x7fe76801c450 comp rx=0 tx=0).stop 2024-09-13T16:59:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 -- 172.21.15.39:0/2967192075 shutdown_connections 2024-09-13T16:59:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 --2- 172.21.15.39:0/2967192075 >> v2:172.21.15.84:3300/0 conn(0x7fe77806d300 0x7fe7780c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 --2- 172.21.15.39:0/2967192075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 0x7fe77806cdc0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.250+0000 7fe77fa4d640 1 -- 172.21.15.39:0/2967192075 >> 172.21.15.39:0/2967192075 conn(0x7fe778069e10 msgr2=0x7fe77806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:54.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 -- 172.21.15.39:0/2967192075 shutdown_connections 2024-09-13T16:59:54.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 -- 172.21.15.39:0/2967192075 wait complete. 2024-09-13T16:59:54.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 Processor -- start 2024-09-13T16:59:54.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 -- start start 2024-09-13T16:59:54.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77806d300 0x7fe7780a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:54.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 0x7fe778070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:54.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe77814e0c0 0x7fe778140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:54.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe778071c80 con 0x7fe7780a4e70 2024-09-13T16:59:54.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781edb40 con 0x7fe77814e0c0 2024-09-13T16:59:54.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77806f780 con 0x7fe77806d300 2024-09-13T16:59:54.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77cfe8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 0x7fe778070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77cfe8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 0x7fe778070fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33072/0 (socket says 172.21.15.39:33072) 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77cfe8640 1 -- 172.21.15.39:0/668495045 learned_addr learned my addr 172.21.15.39:0/668495045 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77dfea640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe77814e0c0 0x7fe778140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77d7e9640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77806d300 0x7fe7780a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77cfe8640 1 -- 172.21.15.39:0/668495045 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77806d300 msgr2=0x7fe7780a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77cfe8640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77806d300 0x7fe7780a6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77cfe8640 1 -- 172.21.15.39:0/668495045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe77814e0c0 msgr2=0x7fe778140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77cfe8640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe77814e0c0 0x7fe778140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77cfe8640 1 -- 172.21.15.39:0/668495045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7680094d0 con 0x7fe7780a4e70 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77dfea640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe77814e0c0 0x7fe778140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77cfe8640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 0x7fe778070fc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe76801a080 tx=0x7fe7680191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:54.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe76dffb640 1 -- 172.21.15.39:0/668495045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680b37d0 con 0x7fe7780a4e70 2024-09-13T16:59:54.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe76dffb640 1 -- 172.21.15.39:0/668495045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7680b3930 con 0x7fe7780a4e70 2024-09-13T16:59:54.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe778071f20 con 0x7fe7780a4e70 2024-09-13T16:59:54.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe76dffb640 1 -- 172.21.15.39:0/668495045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680b3ce0 con 0x7fe7780a4e70 2024-09-13T16:59:54.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7780a72a0 con 0x7fe7780a4e70 2024-09-13T16:59:54.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.254+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7780c4740 con 0x7fe7780a4e70 2024-09-13T16:59:54.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.258+0000 7fe76dffb640 1 -- 172.21.15.39:0/668495045 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7680b2070 con 0x7fe7780a4e70 2024-09-13T16:59:54.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.258+0000 7fe76dffb640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe75c0666c0 0x7fe75c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:54.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.258+0000 7fe76dffb640 1 -- 172.21.15.39:0/668495045 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7755+0+0 (secure 0 0 0) 0x7fe76811ec40 con 0x7fe7780a4e70 2024-09-13T16:59:54.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.258+0000 7fe77d7e9640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe75c0666c0 0x7fe75c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:54.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.258+0000 7fe77d7e9640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe75c0666c0 0x7fe75c068b80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe77400b810 tx=0x7fe774008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:54.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.262+0000 7fe76dffb640 1 -- 172.21.15.39:0/668495045 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe7680eb840 con 0x7fe7780a4e70 2024-09-13T16:59:54.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.458+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7780c4740 con 0x7fe7780a4e70 2024-09-13T16:59:54.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.458+0000 7fe76dffb640 1 -- 172.21.15.39:0/668495045 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5933 (secure 0 0 0) 0x7fe7680b0020 con 0x7fe7780a4e70 2024-09-13T16:59:54.457 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:54.457 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:58:02.976157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":38,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5671},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5671":{"gid":5671,"name":"b","rank":0,"incarnation":71,"state":"up:active","state_seq":14,"addr":"172.21.15.84:6833/3122796453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":3122796453},{"type":"v1","addr":"172.21.15.84:6833","nonce":3122796453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:54.458 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 80 2024-09-13T16:59:54.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.458+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe75c0666c0 msgr2=0x7fe75c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:54.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.458+0000 7fe77fa4d640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe75c0666c0 0x7fe75c068b80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe77400b810 tx=0x7fe774008040 comp rx=0 tx=0).stop 2024-09-13T16:59:54.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.458+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 msgr2=0x7fe778070fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:54.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.458+0000 7fe77fa4d640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 0x7fe778070fc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe76801a080 tx=0x7fe7680191f0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.462+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 shutdown_connections 2024-09-13T16:59:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.462+0000 7fe77fa4d640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe75c0666c0 0x7fe75c068b80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.462+0000 7fe77fa4d640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe77814e0c0 0x7fe778140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.462+0000 7fe77fa4d640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7780a4e70 0x7fe778070fc0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.462+0000 7fe77fa4d640 1 --2- 172.21.15.39:0/668495045 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77806d300 0x7fe7780a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.462+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 >> 172.21.15.39:0/668495045 conn(0x7fe778069e10 msgr2=0x7fe77814c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.462+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 shutdown_connections 2024-09-13T16:59:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:54.462+0000 7fe77fa4d640 1 -- 172.21.15.39:0/668495045 wait complete. 2024-09-13T16:59:54.477 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:59:55.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.542+0000 7f5477e08640 1 Processor -- start 2024-09-13T16:59:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.542+0000 7f5477e08640 1 -- start start 2024-09-13T16:59:55.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f5477e08640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f54700c42c0 0x7f54700c4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f5477e08640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700704c0 0x7f5470070930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f5477e08640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54701575b0 con 0x7f5470151850 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f5477e08640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5470151c30 con 0x7f54700c42c0 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f5477e08640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5470157820 con 0x7f54700704c0 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f54753a3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f54700c42c0 0x7f54700c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f54753a3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f54700c42c0 0x7f54700c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33084/0 (socket says 172.21.15.39:33084) 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f54753a3640 1 -- 172.21.15.39:0/3694858629 learned_addr learned my addr 172.21.15.39:0/3694858629 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f54763a5640 1 --2- 172.21.15.39:0/3694858629 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700704c0 0x7f5470070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f54753a3640 1 -- 172.21.15.39:0/3694858629 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700704c0 msgr2=0x7f5470070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f54753a3640 1 --2- 172.21.15.39:0/3694858629 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700704c0 0x7f5470070930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f54753a3640 1 -- 172.21.15.39:0/3694858629 >> v1:172.21.15.84:6789/0 conn(0x7f5470151850 legacy=0x7f54700c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:55.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f54753a3640 1 -- 172.21.15.39:0/3694858629 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470157b00 con 0x7f54700c42c0 2024-09-13T16:59:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f54753a3640 1 --2- 172.21.15.39:0/3694858629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54700c42c0 0x7f54700c4710 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a9365d2fbbf6c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f5474ba2640 1 -- 172.21.15.39:0/3694858629 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600ab650 con 0x7f54700c42c0 2024-09-13T16:59:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f5474ba2640 1 -- 172.21.15.39:0/3694858629 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54600ab7b0 con 0x7f54700c42c0 2024-09-13T16:59:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.546+0000 7f5474ba2640 1 -- 172.21.15.39:0/3694858629 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600abb40 con 0x7f54700c42c0 2024-09-13T16:59:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- 172.21.15.39:0/3694858629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54700c42c0 msgr2=0x7f54700c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 --2- 172.21.15.39:0/3694858629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54700c42c0 0x7f54700c4710 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546001c560 comp rx=0 tx=0).stop 2024-09-13T16:59:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- 172.21.15.39:0/3694858629 shutdown_connections 2024-09-13T16:59:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 --2- 172.21.15.39:0/3694858629 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700704c0 0x7f5470070930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 --2- 172.21.15.39:0/3694858629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54700c42c0 0x7f54700c4710 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- 172.21.15.39:0/3694858629 >> 172.21.15.39:0/3694858629 conn(0x7f54700ab4d0 msgr2=0x7f54700ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- 172.21.15.39:0/3694858629 shutdown_connections 2024-09-13T16:59:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- 172.21.15.39:0/3694858629 wait complete. 2024-09-13T16:59:55.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 Processor -- start 2024-09-13T16:59:55.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- start start 2024-09-13T16:59:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54700704c0 0x7f54700ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700c42c0 0x7f54700aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5470151850 0x7f54700aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5470154d30 con 0x7f5470151850 2024-09-13T16:59:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54700af4f0 con 0x7f54700704c0 2024-09-13T16:59:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54701543d0 con 0x7f54700c42c0 2024-09-13T16:59:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54763a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5470151850 0x7f54700aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54763a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5470151850 0x7f54700aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33094/0 (socket says 172.21.15.39:33094) 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54763a5640 1 -- 172.21.15.39:0/3261995877 learned_addr learned my addr 172.21.15.39:0/3261995877 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5475ba4640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54700704c0 0x7f54700ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54753a3640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700c42c0 0x7f54700aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54763a5640 1 -- 172.21.15.39:0/3261995877 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700c42c0 msgr2=0x7f54700aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54763a5640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700c42c0 0x7f54700aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54763a5640 1 -- 172.21.15.39:0/3261995877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54700704c0 msgr2=0x7f54700ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54763a5640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54700704c0 0x7f54700ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54763a5640 1 -- 172.21.15.39:0/3261995877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600094d0 con 0x7f5470151850 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54763a5640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5470151850 0x7f54700aefb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5468007cf0 tx=0x7f546800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54667fc640 1 -- 172.21.15.39:0/3261995877 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468014420 con 0x7f5470151850 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54667fc640 1 -- 172.21.15.39:0/3261995877 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54680145a0 con 0x7f5470151850 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5470151e10 con 0x7f5470151850 2024-09-13T16:59:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f54667fc640 1 -- 172.21.15.39:0/3261995877 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468014950 con 0x7f5470151850 2024-09-13T16:59:55.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.550+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5470154950 con 0x7f5470151850 2024-09-13T16:59:55.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.554+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5470070930 con 0x7f5470151850 2024-09-13T16:59:55.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.554+0000 7f54667fc640 1 -- 172.21.15.39:0/3261995877 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5468014ab0 con 0x7f5470151850 2024-09-13T16:59:55.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.554+0000 7f54667fc640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5454066690 0x7f5454068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.554+0000 7f5475ba4640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5454066690 0x7f5454068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.554+0000 7f54667fc640 1 -- 172.21.15.39:0/3261995877 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7841+0+0 (secure 0 0 0) 0x7f5468085200 con 0x7f5470151850 2024-09-13T16:59:55.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.554+0000 7f5475ba4640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5454066690 0x7f5454068b50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f54701f1310 tx=0x7f546c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:55.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.558+0000 7f54667fc640 1 -- 172.21.15.39:0/3261995877 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5468051da0 con 0x7f5470151850 2024-09-13T16:59:55.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.754+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54700c7010 con 0x7f5470151850 2024-09-13T16:59:55.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.754+0000 7f54667fc640 1 -- 172.21.15.39:0/3261995877 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f54680553c0 con 0x7f5470151850 2024-09-13T16:59:55.754 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:55.754 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:54.691555+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:55.755 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 81 2024-09-13T16:59:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5454066690 msgr2=0x7f5454068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5454066690 0x7f5454068b50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f54701f1310 tx=0x7f546c009040 comp rx=0 tx=0).stop 2024-09-13T16:59:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5470151850 msgr2=0x7f54700aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5470151850 0x7f54700aefb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5468007cf0 tx=0x7f546800e4b0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 shutdown_connections 2024-09-13T16:59:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5454066690 0x7f5454068b50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5470151850 0x7f54700aefb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54700c42c0 0x7f54700aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 --2- 172.21.15.39:0/3261995877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54700704c0 0x7f54700ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 >> 172.21.15.39:0/3261995877 conn(0x7f54700ab4d0 msgr2=0x7f54700ab8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:55.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 shutdown_connections 2024-09-13T16:59:55.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.758+0000 7f5477e08640 1 -- 172.21.15.39:0/3261995877 wait complete. 2024-09-13T16:59:55.774 INFO:tasks.mds_thrash.fs.[a]:mds.b down, removed from mdsmap 2024-09-13T16:59:55.774 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T16:59:55.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:55.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.894+0000 7f15f5f4b640 1 Processor -- start 2024-09-13T16:59:55.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15f5f4b640 1 -- start start 2024-09-13T16:59:55.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15f5f4b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f15f0150f80 0x7f15f0153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15f5f4b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f00c5330 0x7f15f006c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15f5f4b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15f0153e70 0x7f15f006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15f5f4b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f00c19a0 con 0x7f15f0150f80 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15f5f4b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f01546c0 con 0x7f15f0153e70 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15f5f4b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f00c49f0 con 0x7f15f00c5330 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ef7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f15f0150f80 0x7f15f0153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ef7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f15f0150f80 0x7f15f0153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33104/0 (socket says 172.21.15.39:33104) 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ef7fe640 1 -- 172.21.15.39:0/868753073 learned_addr learned my addr 172.21.15.39:0/868753073 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15eeffd640 1 --2- 172.21.15.39:0/868753073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f00c5330 0x7f15f006c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15effff640 1 --2- 172.21.15.39:0/868753073 >> v2:172.21.15.84:3300/0 conn(0x7f15f0153e70 0x7f15f006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ef7fe640 1 -- 172.21.15.39:0/868753073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f00c5330 msgr2=0x7f15f006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ef7fe640 1 --2- 172.21.15.39:0/868753073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f00c5330 0x7f15f006c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ef7fe640 1 -- 172.21.15.39:0/868753073 >> v2:172.21.15.84:3300/0 conn(0x7f15f0153e70 msgr2=0x7f15f006c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:55.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ef7fe640 1 --2- 172.21.15.39:0/868753073 >> v2:172.21.15.84:3300/0 conn(0x7f15f0153e70 0x7f15f006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ef7fe640 1 -- 172.21.15.39:0/868753073 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f01543e0 con 0x7f15f0150f80 2024-09-13T16:59:55.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15eeffd640 1 --2- 172.21.15.39:0/868753073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f00c5330 0x7f15f006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:59:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ef7fe640 1 --2- 172.21.15.39:0/868753073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f0150f80 0x7f15f0153930 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f15f00a8940 tx=0x7f15e001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b86194c13de99a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ee7fc640 1 -- 172.21.15.39:0/868753073 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00ab620 con 0x7f15f0150f80 2024-09-13T16:59:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.898+0000 7f15ee7fc640 1 -- 172.21.15.39:0/868753073 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e00ab780 con 0x7f15f0150f80 2024-09-13T16:59:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ee7fc640 1 -- 172.21.15.39:0/868753073 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00abb10 con 0x7f15f0150f80 2024-09-13T16:59:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- 172.21.15.39:0/868753073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f0150f80 msgr2=0x7f15f0153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/868753073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f0150f80 0x7f15f0153930 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f15f00a8940 tx=0x7f15e001c460 comp rx=0 tx=0).stop 2024-09-13T16:59:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- 172.21.15.39:0/868753073 shutdown_connections 2024-09-13T16:59:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/868753073 >> v2:172.21.15.84:3300/0 conn(0x7f15f0153e70 0x7f15f006c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/868753073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f00c5330 0x7f15f006c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/868753073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f0150f80 0x7f15f0153930 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- 172.21.15.39:0/868753073 >> 172.21.15.39:0/868753073 conn(0x7f15f0069e10 msgr2=0x7f15f006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- 172.21.15.39:0/868753073 shutdown_connections 2024-09-13T16:59:55.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- 172.21.15.39:0/868753073 wait complete. 2024-09-13T16:59:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 Processor -- start 2024-09-13T16:59:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- start start 2024-09-13T16:59:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f00c5330 0x7f15f00706a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15f0150f80 0x7f15f01f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f0153e70 0x7f15f00bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f006f730 con 0x7f15f00c5330 2024-09-13T16:59:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f006fa10 con 0x7f15f0150f80 2024-09-13T16:59:55.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f01efb60 con 0x7f15f0153e70 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ef7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f00c5330 0x7f15f00706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ef7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f00c5330 0x7f15f00706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33106/0 (socket says 172.21.15.39:33106) 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ef7fe640 1 -- 172.21.15.39:0/2883754796 learned_addr learned my addr 172.21.15.39:0/2883754796 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15eeffd640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15f0150f80 0x7f15f01f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15effff640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f0153e70 0x7f15f00bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ef7fe640 1 -- 172.21.15.39:0/2883754796 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f0153e70 msgr2=0x7f15f00bf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ef7fe640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f0153e70 0x7f15f00bf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ef7fe640 1 -- 172.21.15.39:0/2883754796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15f0150f80 msgr2=0x7f15f01f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ef7fe640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15f0150f80 0x7f15f01f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ef7fe640 1 -- 172.21.15.39:0/2883754796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e00094d0 con 0x7f15f00c5330 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15ef7fe640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f00c5330 0x7f15f00706a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f15e001ab10 tx=0x7f15e001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15d7fff640 1 -- 172.21.15.39:0/2883754796 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00be3f0 con 0x7f15f00c5330 2024-09-13T16:59:55.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15d7fff640 1 -- 172.21.15.39:0/2883754796 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e00be570 con 0x7f15f00c5330 2024-09-13T16:59:55.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f01efe40 con 0x7f15f00c5330 2024-09-13T16:59:55.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15d7fff640 1 -- 172.21.15.39:0/2883754796 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00be920 con 0x7f15f00c5330 2024-09-13T16:59:55.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.902+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f00bfb20 con 0x7f15f00c5330 2024-09-13T16:59:55.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.906+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f00c3c00 con 0x7f15f00c5330 2024-09-13T16:59:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.906+0000 7f15d7fff640 1 -- 172.21.15.39:0/2883754796 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f15e00b43f0 con 0x7f15f00c5330 2024-09-13T16:59:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.906+0000 7f15d7fff640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15dc066690 0x7f15dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.906+0000 7f15d7fff640 1 -- 172.21.15.39:0/2883754796 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7841+0+0 (secure 0 0 0) 0x7f15e011e770 con 0x7f15f00c5330 2024-09-13T16:59:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.906+0000 7f15eeffd640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15dc066690 0x7f15dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.906+0000 7f15eeffd640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15dc066690 0x7f15dc068b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006fd0 tx=0x7f15e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:55.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:55.910+0000 7f15d7fff640 1 -- 172.21.15.39:0/2883754796 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15e00eb280 con 0x7f15f00c5330 2024-09-13T16:59:56.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15f01538b0 con 0x7f15f00c5330 2024-09-13T16:59:56.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15d7fff640 1 -- 172.21.15.39:0/2883754796 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5179 (secure 0 0 0) 0x7f15e00ee8a0 con 0x7f15f00c5330 2024-09-13T16:59:56.105 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:56.105 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:55.963934+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:56.105 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 82 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15dc066690 msgr2=0x7f15dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15dc066690 0x7f15dc068b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006fd0 tx=0x7f15e4008040 comp rx=0 tx=0).stop 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f00c5330 msgr2=0x7f15f00706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f00c5330 0x7f15f00706a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f15e001ab10 tx=0x7f15e001aca0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 shutdown_connections 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f15dc066690 0x7f15dc068b50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15f0150f80 0x7f15f01f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15f0153e70 0x7f15f00bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 --2- 172.21.15.39:0/2883754796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f00c5330 0x7f15f00706a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 >> 172.21.15.39:0/2883754796 conn(0x7f15f0069e10 msgr2=0x7f15f0152e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 shutdown_connections 2024-09-13T16:59:56.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.106+0000 7f15f5f4b640 1 -- 172.21.15.39:0/2883754796 wait complete. 2024-09-13T16:59:56.123 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 82, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 82, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6243": { "addr": "172.21.15.39:6835/4246622083", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 4246622083, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 4246622083, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6243, "incarnation": 81, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 40, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:59:55.963934+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6243 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 79, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5282": { "addr": "172.21.15.84:6835/1326467723", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 1326467723, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 1326467723, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5282, "incarnation": 76, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 39, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T16:59:07.921669+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5282 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T16:59:56.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:56.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f0552f06640 1 Processor -- start 2024-09-13T16:59:56.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f0552f06640 1 -- start start 2024-09-13T16:59:56.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f0552f06640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c06fa50 0x7f054c156130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:56.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f0552f06640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f054c0c6000 0x7f054c0c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:56.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f0552f06640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054c0c5e50 con 0x7f054c0c7960 2024-09-13T16:59:56.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f0552f06640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c06ff50 con 0x7f054c0c6000 2024-09-13T16:59:56.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f0552f06640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c072000 con 0x7f054c06fa50 2024-09-13T16:59:56.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f05514a3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f054c0c6000 0x7f054c0c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:56.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f05514a3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f054c0c6000 0x7f054c0c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33108/0 (socket says 172.21.15.39:33108) 2024-09-13T16:59:56.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f05514a3640 1 -- 172.21.15.39:0/74974894 learned_addr learned my addr 172.21.15.39:0/74974894 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:56.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f054bfff640 1 --2- 172.21.15.39:0/74974894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c06fa50 0x7f054c156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:56.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f05514a3640 1 -- 172.21.15.39:0/74974894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c06fa50 msgr2=0x7f054c156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:56.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f05514a3640 1 --2- 172.21.15.39:0/74974894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c06fa50 0x7f054c156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f05514a3640 1 -- 172.21.15.39:0/74974894 >> v1:172.21.15.84:6789/0 conn(0x7f054c0c7960 legacy=0x7f054c155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:56.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.610+0000 7f05514a3640 1 -- 172.21.15.39:0/74974894 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c0722e0 con 0x7f054c0c6000 2024-09-13T16:59:56.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f05514a3640 1 --2- 172.21.15.39:0/74974894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c0c6000 0x7f054c0c2570 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0544009870 tx=0x7f054401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ca0a2aaf9c5007 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:56.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f054b7fe640 1 -- 172.21.15.39:0/74974894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440aa620 con 0x7f054c0c6000 2024-09-13T16:59:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f054b7fe640 1 -- 172.21.15.39:0/74974894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05440aa780 con 0x7f054c0c6000 2024-09-13T16:59:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f054b7fe640 1 -- 172.21.15.39:0/74974894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440aab10 con 0x7f054c0c6000 2024-09-13T16:59:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 -- 172.21.15.39:0/74974894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c0c6000 msgr2=0x7f054c0c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 --2- 172.21.15.39:0/74974894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c0c6000 0x7f054c0c2570 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0544009870 tx=0x7f054401c4a0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 -- 172.21.15.39:0/74974894 shutdown_connections 2024-09-13T16:59:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 --2- 172.21.15.39:0/74974894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c0c6000 0x7f054c0c2570 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 --2- 172.21.15.39:0/74974894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c06fa50 0x7f054c156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 -- 172.21.15.39:0/74974894 >> 172.21.15.39:0/74974894 conn(0x7f054c06c020 msgr2=0x7f054c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 -- 172.21.15.39:0/74974894 shutdown_connections 2024-09-13T16:59:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 -- 172.21.15.39:0/74974894 wait complete. 2024-09-13T16:59:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 Processor -- start 2024-09-13T16:59:56.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.614+0000 7f0552f06640 1 -- start start 2024-09-13T16:59:56.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0552f06640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c06fa50 0x7f054c146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:56.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0552f06640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c0c6000 0x7f054c146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:56.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0552f06640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c0c7960 0x7f054c150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:56.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0552f06640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c14ff40 con 0x7f054c06fa50 2024-09-13T16:59:56.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0552f06640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c14f910 con 0x7f054c0c6000 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0552f06640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1ee2e0 con 0x7f054c0c7960 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0550ca2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c06fa50 0x7f054c146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0550ca2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c06fa50 0x7f054c146930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33120/0 (socket says 172.21.15.39:33120) 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0550ca2640 1 -- 172.21.15.39:0/245971221 learned_addr learned my addr 172.21.15.39:0/245971221 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f05514a3640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c0c7960 0x7f054c150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f054bfff640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c0c6000 0x7f054c146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0550ca2640 1 -- 172.21.15.39:0/245971221 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c0c7960 msgr2=0x7f054c150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0550ca2640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c0c7960 0x7f054c150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0550ca2640 1 -- 172.21.15.39:0/245971221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c0c6000 msgr2=0x7f054c146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0550ca2640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c0c6000 0x7f054c146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0550ca2640 1 -- 172.21.15.39:0/245971221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440094d0 con 0x7f054c06fa50 2024-09-13T16:59:56.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f05514a3640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c0c7960 0x7f054c150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T16:59:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0550ca2640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c06fa50 0x7f054c146930 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f053c00c7a0 tx=0x7f053c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f05497fa640 1 -- 172.21.15.39:0/245971221 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c003b80 con 0x7f054c06fa50 2024-09-13T16:59:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f05497fa640 1 -- 172.21.15.39:0/245971221 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c003ce0 con 0x7f054c06fa50 2024-09-13T16:59:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c1ee580 con 0x7f054c06fa50 2024-09-13T16:59:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f05497fa640 1 -- 172.21.15.39:0/245971221 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00aa90 con 0x7f054c06fa50 2024-09-13T16:59:56.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c1473b0 con 0x7f054c06fa50 2024-09-13T16:59:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c0c2570 con 0x7f054c06fa50 2024-09-13T16:59:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f05497fa640 1 -- 172.21.15.39:0/245971221 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f053c00abf0 con 0x7f054c06fa50 2024-09-13T16:59:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f05497fa640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f052c066690 0x7f052c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f05497fa640 1 -- 172.21.15.39:0/245971221 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7841+0+0 (secure 0 0 0) 0x7f053c084750 con 0x7f054c06fa50 2024-09-13T16:59:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f054bfff640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f052c066690 0x7f052c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:56.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.618+0000 7f054bfff640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f052c066690 0x7f052c068b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f054c0bfec0 tx=0x7f0534008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:56.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.622+0000 7f05497fa640 1 -- 172.21.15.39:0/245971221 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f053c0512f0 con 0x7f054c06fa50 2024-09-13T16:59:56.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f054c0c02a0 con 0x7f054c06fa50 2024-09-13T16:59:56.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f05497fa640 1 -- 172.21.15.39:0/245971221 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5179 (secure 0 0 0) 0x7f053c054910 con 0x7f054c06fa50 2024-09-13T16:59:56.817 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:56.817 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:55.963934+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:56.817 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 82 2024-09-13T16:59:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f052c066690 msgr2=0x7f052c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f052c066690 0x7f052c068b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f054c0bfec0 tx=0x7f0534008040 comp rx=0 tx=0).stop 2024-09-13T16:59:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c06fa50 msgr2=0x7f054c146930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c06fa50 0x7f054c146930 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f053c00c7a0 tx=0x7f053c00cc60 comp rx=0 tx=0).stop 2024-09-13T16:59:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 shutdown_connections 2024-09-13T16:59:56.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f052c066690 0x7f052c068b50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f054c0c6000 0x7f054c146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f054c0c7960 0x7f054c150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 --2- 172.21.15.39:0/245971221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054c06fa50 0x7f054c146930 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:56.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 >> 172.21.15.39:0/245971221 conn(0x7f054c06c020 msgr2=0x7f054c06c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:56.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.818+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 shutdown_connections 2024-09-13T16:59:56.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:56.822+0000 7f0552f06640 1 -- 172.21.15.39:0/245971221 wait complete. 2024-09-13T16:59:56.837 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T16:59:58.010 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:59:58.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T16:59:58.124 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:58.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.254+0000 7fbdbc7b9640 1 Processor -- start 2024-09-13T16:59:58.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.254+0000 7fbdbc7b9640 1 -- start start 2024-09-13T16:59:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbc7b9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbdb4157530 0x7fbdb40c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbc7b9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb406f840 0x7fbdb40a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbc7b9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb40c3d60 con 0x7fbdb4157910 2024-09-13T16:59:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbc7b9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb406be40 con 0x7fbdb4157530 2024-09-13T16:59:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbc7b9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb406b960 con 0x7fbdb406f840 2024-09-13T16:59:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9d54640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbdb4157910 0x7fbdb4071f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53798/0 (socket says 172.21.15.39:53798) 2024-09-13T16:59:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9d54640 1 -- 172.21.15.39:0/1515951946 learned_addr learned my addr 172.21.15.39:0/1515951946 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbad56640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb406f840 0x7fbdb40a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdba555640 1 --2- 172.21.15.39:0/1515951946 >> v2:172.21.15.84:3300/0 conn(0x7fbdb4157530 0x7fbdb40c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986872425 0 0) 0x7fbdb40c3d60 con 0x7fbdb4157910 2024-09-13T16:59:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda0003530 con 0x7fbdb4157910 2024-09-13T16:59:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232579457 0 0) 0x7fbda0003530 con 0x7fbdb4157910 2024-09-13T16:59:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb406f840 msgr2=0x7fbdb40a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb406f840 0x7fbdb40a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 >> v2:172.21.15.84:3300/0 conn(0x7fbdb4157530 msgr2=0x7fbdb40c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 --2- 172.21.15.39:0/1515951946 >> v2:172.21.15.84:3300/0 conn(0x7fbdb4157530 0x7fbdb40c2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb40c4000 con 0x7fbdb4157910 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fbda4001680 con 0x7fbdb4157910 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 >> v1:172.21.15.39:6789/0 conn(0x7fbdb4157910 legacy=0x7fbdb4071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb406f840 0x7fbda0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbda0003f00 0x7fbda00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda0004d10 0x7fbda0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb406be40 con 0x7fbda0004d10 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb406b960 con 0x7fbda0003f00 2024-09-13T16:59:58.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda000b9c0 con 0x7fbdb406f840 2024-09-13T16:59:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbad56640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda0004d10 0x7fbda0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdba555640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb406f840 0x7fbda0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdb9d54640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbda0003f00 0x7fbda00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbad56640 1 -- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb406f840 msgr2=0x7fbda0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbad56640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb406f840 0x7fbda0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbad56640 1 -- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbda0003f00 msgr2=0x7fbda00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbad56640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbda0003f00 0x7fbda00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.258+0000 7fbdbad56640 1 -- 172.21.15.39:0/1515951946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda0003530 con 0x7fbda0004d10 2024-09-13T16:59:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbad56640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda0004d10 0x7fbda0005190 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009fc0 tx=0x7fbdac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a19819a58bafd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0a5c90 con 0x7fbda0004d10 2024-09-13T16:59:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac00f040 con 0x7fbda0004d10 2024-09-13T16:59:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdb9553640 1 -- 172.21.15.39:0/1515951946 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0ad440 con 0x7fbda0004d10 2024-09-13T16:59:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/1515951946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda0004d10 msgr2=0x7fbda0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbc7b9640 1 --2- 172.21.15.39:0/1515951946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbda0004d10 0x7fbda0005190 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009fc0 tx=0x7fbdac01cc80 comp rx=0 tx=0).stop 2024-09-13T16:59:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbad56640 1 -- 172.21.15.39:0/1515951946 reap_dead start 2024-09-13T16:59:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/1515951946 shutdown_connections 2024-09-13T16:59:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/1515951946 >> 172.21.15.39:0/1515951946 conn(0x7fbdb40a97b0 msgr2=0x7fbdb40a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/1515951946 shutdown_connections 2024-09-13T16:59:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/1515951946 wait complete. 2024-09-13T16:59:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbc7b9640 1 Processor -- start 2024-09-13T16:59:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.262+0000 7fbdbc7b9640 1 -- start start 2024-09-13T16:59:58.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbc7b9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb4146020 0x7fbdb4155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbc7b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdb406f130 0x7fbdb40aa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbc7b9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdb41e9830 0x7fbdb4155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbc7b9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4156380 con 0x7fbdb41e9830 2024-09-13T16:59:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbc7b9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb406de80 con 0x7fbdb406f130 2024-09-13T16:59:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbc7b9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb406e160 con 0x7fbdb4146020 2024-09-13T16:59:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbad56640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdb41e9830 0x7fbdb4155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbad56640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdb41e9830 0x7fbdb4155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33132/0 (socket says 172.21.15.39:33132) 2024-09-13T16:59:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbad56640 1 -- 172.21.15.39:0/2840480169 learned_addr learned my addr 172.21.15.39:0/2840480169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdb9d54640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdb406f130 0x7fbdb40aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdba555640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb4146020 0x7fbdb4155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbad56640 1 -- 172.21.15.39:0/2840480169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb4146020 msgr2=0x7fbdb4155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbad56640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb4146020 0x7fbdb4155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbad56640 1 -- 172.21.15.39:0/2840480169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdb406f130 msgr2=0x7fbdb40aa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbad56640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdb406f130 0x7fbdb40aa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbad56640 1 -- 172.21.15.39:0/2840480169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb41e9030 con 0x7fbdb41e9830 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbad56640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdb41e9830 0x7fbdb4155220 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbdac002240 tx=0x7fbdac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdaaffd640 1 -- 172.21.15.39:0/2840480169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0039a0 con 0x7fbdb41e9830 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdaaffd640 1 -- 172.21.15.39:0/2840480169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac003b00 con 0x7fbdb41e9830 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb41e9310 con 0x7fbdb41e9830 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdaaffd640 1 -- 172.21.15.39:0/2840480169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac003e70 con 0x7fbdb41e9830 2024-09-13T16:59:58.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb41465d0 con 0x7fbdb41e9830 2024-09-13T16:59:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb40c2400 con 0x7fbdb41e9830 2024-09-13T16:59:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdaaffd640 1 -- 172.21.15.39:0/2840480169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbdac0b4040 con 0x7fbdb41e9830 2024-09-13T16:59:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdaaffd640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbd980666c0 0x7fbd98068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdaaffd640 1 -- 172.21.15.39:0/2840480169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7841+0+0 (secure 0 0 0) 0x7fbdac11e7e0 con 0x7fbdb41e9830 2024-09-13T16:59:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.266+0000 7fbdba555640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbd980666c0 0x7fbd98068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.270+0000 7fbdba555640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbd980666c0 0x7fbd98068b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4154db0 tx=0x7fbdb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:58.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.274+0000 7fbdaaffd640 1 -- 172.21.15.39:0/2840480169 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbdac0eb550 con 0x7fbdb41e9830 2024-09-13T16:59:58.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdb406dbb0 con 0x7fbdb41e9830 2024-09-13T16:59:58.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdaaffd640 1 -- 172.21.15.39:0/2840480169 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5176 (secure 0 0 0) 0x7fbdac0b2090 con 0x7fbdb41e9830 2024-09-13T16:59:58.516 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:58.516 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:58.516 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 84 2024-09-13T16:59:58.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbd980666c0 msgr2=0x7fbd98068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbd980666c0 0x7fbd98068b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4154db0 tx=0x7fbdb0008040 comp rx=0 tx=0).stop 2024-09-13T16:59:58.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdb41e9830 msgr2=0x7fbdb4155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdb41e9830 0x7fbdb4155220 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbdac002240 tx=0x7fbdac009750 comp rx=0 tx=0).stop 2024-09-13T16:59:58.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 shutdown_connections 2024-09-13T16:59:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbd980666c0 0x7fbd98068b80 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4154db0 tx=0x7fbdb0008040 comp rx=0 tx=0).stop 2024-09-13T16:59:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdb406f130 0x7fbdb40aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdb41e9830 0x7fbdb4155220 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 --2- 172.21.15.39:0/2840480169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbdb4146020 0x7fbdb4155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 >> 172.21.15.39:0/2840480169 conn(0x7fbdb40a97b0 msgr2=0x7fbdb40c6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 shutdown_connections 2024-09-13T16:59:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.518+0000 7fbdbc7b9640 1 -- 172.21.15.39:0/2840480169 wait complete. 2024-09-13T16:59:58.538 INFO:tasks.mds_thrash.fs.[a]:mds.a has gained rank=0, replacing gid=5671 2024-09-13T16:59:58.538 INFO:tasks.mds_thrash.fs.[a]:waiting for 3 secs before reviving mds.b 2024-09-13T16:59:58.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T16:59:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75ba41c640 1 Processor -- start 2024-09-13T16:59:58.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75ba41c640 1 -- start start 2024-09-13T16:59:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75ba41c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f75b40c2ee0 0x7f75b40c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75ba41c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b40c6270 0x7f75b40c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75ba41c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b406ddf0 con 0x7f75b4065ee0 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75ba41c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b40c4710 con 0x7f75b40c2ee0 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75ba41c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40c49d0 con 0x7f75b40c6270 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b3fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f75b4065ee0 0x7f75b40662c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53808/0 (socket says 172.21.15.39:53808) 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b3fff640 1 -- 172.21.15.39:0/1611524789 learned_addr learned my addr 172.21.15.39:0/1611524789 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b37fe640 1 --2- 172.21.15.39:0/1611524789 >> v2:172.21.15.84:3300/0 conn(0x7f75b40c2ee0 0x7f75b40c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15750509 0 0) 0x7f75b406ddf0 con 0x7f75b4065ee0 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f759c003530 con 0x7f75b4065ee0 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b89b9640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b40c6270 0x7f75b40c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 632180049 0 0) 0x7f759c003530 con 0x7f75b4065ee0 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b40c6270 msgr2=0x7f75b40c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b40c6270 0x7f75b40c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 >> v2:172.21.15.84:3300/0 conn(0x7f75b40c2ee0 msgr2=0x7f75b40c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 --2- 172.21.15.39:0/1611524789 >> v2:172.21.15.84:3300/0 conn(0x7f75b40c2ee0 0x7f75b40c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b4154880 con 0x7f75b4065ee0 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f75a4001680 con 0x7f75b4065ee0 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 >> v1:172.21.15.39:6789/0 conn(0x7f75b4065ee0 legacy=0x7f75b40662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f759c003980 0x7f759c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047d0 0x7f759c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b40c6270 0x7f759c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40c4710 con 0x7f759c003980 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40c49d0 con 0x7f759c0047d0 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c00b990 con 0x7f75b40c6270 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b3fff640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f759c003980 0x7f759c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b37fe640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047d0 0x7f759c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b89b9640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b40c6270 0x7f759c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b3fff640 1 -- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b40c6270 msgr2=0x7f759c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b3fff640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b40c6270 0x7f759c005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b3fff640 1 -- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047d0 msgr2=0x7f759c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T16:59:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b3fff640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047d0 0x7f759c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b3fff640 1 -- 172.21.15.39:0/1611524789 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c003530 con 0x7f759c003980 2024-09-13T16:59:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.974+0000 7f75b37fe640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c0047d0 0x7f759c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:59:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75b3fff640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f759c003980 0x7f759c003da0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c240 tx=0x7f75a401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94be971f066ef014 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a40058e0 con 0x7f759c003980 2024-09-13T16:59:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a4005a40 con 0x7f759c003980 2024-09-13T16:59:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75b2ffd640 1 -- 172.21.15.39:0/1611524789 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4005df0 con 0x7f759c003980 2024-09-13T16:59:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75ba41c640 1 -- 172.21.15.39:0/1611524789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f759c003980 msgr2=0x7f759c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75ba41c640 1 --2- 172.21.15.39:0/1611524789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f759c003980 0x7f759c003da0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c240 tx=0x7f75a401cc40 comp rx=0 tx=0).stop 2024-09-13T16:59:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75b89b9640 1 -- 172.21.15.39:0/1611524789 reap_dead start 2024-09-13T16:59:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75ba41c640 1 -- 172.21.15.39:0/1611524789 shutdown_connections 2024-09-13T16:59:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75ba41c640 1 -- 172.21.15.39:0/1611524789 >> 172.21.15.39:0/1611524789 conn(0x7f75b40c3310 msgr2=0x7f75b40c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75ba41c640 1 -- 172.21.15.39:0/1611524789 shutdown_connections 2024-09-13T16:59:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.978+0000 7f75ba41c640 1 -- 172.21.15.39:0/1611524789 wait complete. 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 Processor -- start 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 -- start start 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c003980 0x7f75b40716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b414bcd0 0x7f75b40abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b4157f10 0x7f75b40ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b41507f0 con 0x7f75b414bcd0 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b4157850 con 0x7f759c003980 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b41ecf70 con 0x7f75b4157f10 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b414bcd0 0x7f75b40abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b414bcd0 0x7f75b40abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33144/0 (socket says 172.21.15.39:33144) 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b37fe640 1 -- 172.21.15.39:0/2561782052 learned_addr learned my addr 172.21.15.39:0/2561782052 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b3fff640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c003980 0x7f75b40716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b89b9640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b4157f10 0x7f75b40ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b37fe640 1 -- 172.21.15.39:0/2561782052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b4157f10 msgr2=0x7f75b40ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b37fe640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b4157f10 0x7f75b40ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b37fe640 1 -- 172.21.15.39:0/2561782052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c003980 msgr2=0x7f75b40716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b37fe640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c003980 0x7f75b40716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b37fe640 1 -- 172.21.15.39:0/2561782052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b4072070 con 0x7f75b414bcd0 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b89b9640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b4157f10 0x7f75b40ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b37fe640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b414bcd0 0x7f75b40abf90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009160 tx=0x7f75a8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b0ff9640 1 -- 172.21.15.39:0/2561782052 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8004220 con 0x7f75b414bcd0 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b0ff9640 1 -- 172.21.15.39:0/2561782052 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a8004380 con 0x7f75b414bcd0 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b414b9f0 con 0x7f75b414bcd0 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b0ff9640 1 -- 172.21.15.39:0/2561782052 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8015890 con 0x7f75b414bcd0 2024-09-13T16:59:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b4071bf0 con 0x7f75b414bcd0 2024-09-13T16:59:58.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b40c2220 con 0x7f75b414bcd0 2024-09-13T16:59:58.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.982+0000 7f75b0ff9640 1 -- 172.21.15.39:0/2561782052 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75a80159f0 con 0x7f75b414bcd0 2024-09-13T16:59:58.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.986+0000 7f75b0ff9640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75a0066690 0x7f75a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T16:59:58.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.986+0000 7f75b0ff9640 1 -- 172.21.15.39:0/2561782052 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7841+0+0 (secure 0 0 0) 0x7f75a8084c90 con 0x7f75b414bcd0 2024-09-13T16:59:58.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.986+0000 7f75b3fff640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75a0066690 0x7f75a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T16:59:58.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.986+0000 7f75b3fff640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75a0066690 0x7f75a0068b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f75a40aa000 tx=0x7f75a4000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T16:59:58.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:58.990+0000 7f75b0ff9640 1 -- 172.21.15.39:0/2561782052 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f75a80518e0 con 0x7f75b414bcd0 2024-09-13T16:59:59.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75b40008e0 con 0x7f75b414bcd0 2024-09-13T16:59:59.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75b0ff9640 1 -- 172.21.15.39:0/2561782052 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f75a8022070 con 0x7f75b414bcd0 2024-09-13T16:59:59.225 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T16:59:59.225 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T16:59:59.225 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 84 2024-09-13T16:59:59.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75a0066690 msgr2=0x7f75a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:59.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75a0066690 0x7f75a0068b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f75a40aa000 tx=0x7f75a4000ff0 comp rx=0 tx=0).stop 2024-09-13T16:59:59.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b414bcd0 msgr2=0x7f75b40abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T16:59:59.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b414bcd0 0x7f75b40abf90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009160 tx=0x7f75a8011a60 comp rx=0 tx=0).stop 2024-09-13T16:59:59.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 shutdown_connections 2024-09-13T16:59:59.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75a0066690 0x7f75a0068b50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:59.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75b4157f10 0x7f75b40ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:59.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b414bcd0 0x7f75b40abf90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:59.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 --2- 172.21.15.39:0/2561782052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f759c003980 0x7f75b40716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T16:59:59.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.226+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 >> 172.21.15.39:0/2561782052 conn(0x7f75b40c3310 msgr2=0x7f75b4153e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T16:59:59.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.230+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 shutdown_connections 2024-09-13T16:59:59.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T16:59:59.230+0000 7f75ba41c640 1 -- 172.21.15.39:0/2561782052 wait complete. 2024-09-13T16:59:59.246 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:00:01.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:00:01.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d8c133640 1 Processor -- start 2024-09-13T17:00:01.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d8c133640 1 -- start start 2024-09-13T17:00:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d8c133640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d840c2690 0x7f0d8406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d8c133640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d840c6290 0x7f0d841577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d8c133640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c0be0 0x7f0d84157d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d8c133640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d840c4a20 con 0x7f0d840c2690 2024-09-13T17:00:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d8c133640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d84157330 con 0x7f0d840c6290 2024-09-13T17:00:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d8c133640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84158240 con 0x7f0d840c0be0 2024-09-13T17:00:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d89ecf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d840c2690 0x7f0d8406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d89ecf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d840c2690 0x7f0d8406ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51820/0 (socket says 172.21.15.39:51820) 2024-09-13T17:00:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d89ecf640 1 -- 172.21.15.39:0/2798515061 learned_addr learned my addr 172.21.15.39:0/2798515061 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d896ce640 1 --2- 172.21.15.39:0/2798515061 >> v2:172.21.15.84:3300/0 conn(0x7f0d840c6290 0x7f0d841577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.390+0000 7f0d8a6d0640 1 --2- 172.21.15.39:0/2798515061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c0be0 0x7f0d84157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d89ecf640 1 -- 172.21.15.39:0/2798515061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c0be0 msgr2=0x7f0d84157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d89ecf640 1 --2- 172.21.15.39:0/2798515061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c0be0 0x7f0d84157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d89ecf640 1 -- 172.21.15.39:0/2798515061 >> v2:172.21.15.84:3300/0 conn(0x7f0d840c6290 msgr2=0x7f0d841577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d89ecf640 1 --2- 172.21.15.39:0/2798515061 >> v2:172.21.15.84:3300/0 conn(0x7f0d840c6290 0x7f0d841577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d89ecf640 1 -- 172.21.15.39:0/2798515061 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d840c4740 con 0x7f0d840c2690 2024-09-13T17:00:01.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d89ecf640 1 --2- 172.21.15.39:0/2798515061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c2690 0x7f0d8406ae20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0d84070340 tx=0x7f0d8001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87182762e92561bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:01.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d88ecd640 1 -- 172.21.15.39:0/2798515061 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800ab620 con 0x7f0d840c2690 2024-09-13T17:00:01.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d88ecd640 1 -- 172.21.15.39:0/2798515061 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d800ab780 con 0x7f0d840c2690 2024-09-13T17:00:01.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d88ecd640 1 -- 172.21.15.39:0/2798515061 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800abb10 con 0x7f0d840c2690 2024-09-13T17:00:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d8c133640 1 -- 172.21.15.39:0/2798515061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c2690 msgr2=0x7f0d8406ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2798515061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c2690 0x7f0d8406ae20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0d84070340 tx=0x7f0d8001c460 comp rx=0 tx=0).stop 2024-09-13T17:00:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d8c133640 1 -- 172.21.15.39:0/2798515061 shutdown_connections 2024-09-13T17:00:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2798515061 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c0be0 0x7f0d84157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2798515061 >> v2:172.21.15.84:3300/0 conn(0x7f0d840c6290 0x7f0d841577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2798515061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c2690 0x7f0d8406ae20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d8c133640 1 -- 172.21.15.39:0/2798515061 >> 172.21.15.39:0/2798515061 conn(0x7f0d840c32a0 msgr2=0x7f0d840c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:01.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d8c133640 1 -- 172.21.15.39:0/2798515061 shutdown_connections 2024-09-13T17:00:01.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.394+0000 7f0d8c133640 1 -- 172.21.15.39:0/2798515061 wait complete. 2024-09-13T17:00:01.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 Processor -- start 2024-09-13T17:00:01.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 -- start start 2024-09-13T17:00:01.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c0be0 0x7f0d841f0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c2690 0x7f0d840a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d840c6290 0x7f0d84158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840acbc0 con 0x7f0d840c0be0 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d841f0b40 con 0x7f0d840c6290 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d841efbb0 con 0x7f0d840c2690 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d89ecf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c0be0 0x7f0d841f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d89ecf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c0be0 0x7f0d841f0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51822/0 (socket says 172.21.15.39:51822) 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d89ecf640 1 -- 172.21.15.39:0/2769642927 learned_addr learned my addr 172.21.15.39:0/2769642927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d896ce640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c2690 0x7f0d840a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8a6d0640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d840c6290 0x7f0d84158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d89ecf640 1 -- 172.21.15.39:0/2769642927 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c2690 msgr2=0x7f0d840a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d89ecf640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c2690 0x7f0d840a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d89ecf640 1 -- 172.21.15.39:0/2769642927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d840c6290 msgr2=0x7f0d84158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d89ecf640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d840c6290 0x7f0d84158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d89ecf640 1 -- 172.21.15.39:0/2769642927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800094d0 con 0x7f0d840c0be0 2024-09-13T17:00:01.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d89ecf640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c0be0 0x7f0d841f0600 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0d8001ab10 tx=0x7f0d8001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:01.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d7a7fc640 1 -- 172.21.15.39:0/2769642927 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800be3f0 con 0x7f0d840c0be0 2024-09-13T17:00:01.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d7a7fc640 1 -- 172.21.15.39:0/2769642927 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d800be550 con 0x7f0d840c0be0 2024-09-13T17:00:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d840ac8e0 con 0x7f0d840c0be0 2024-09-13T17:00:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d7a7fc640 1 -- 172.21.15.39:0/2769642927 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800be8e0 con 0x7f0d840c0be0 2024-09-13T17:00:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d841efe00 con 0x7f0d840c0be0 2024-09-13T17:00:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.398+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8406ae20 con 0x7f0d840c0be0 2024-09-13T17:00:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.402+0000 7f0d7a7fc640 1 -- 172.21.15.39:0/2769642927 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d800c7440 con 0x7f0d840c0be0 2024-09-13T17:00:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.402+0000 7f0d7a7fc640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d680666c0 0x7f0d68068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.402+0000 7f0d7a7fc640 1 -- 172.21.15.39:0/2769642927 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7841+0+0 (secure 0 0 0) 0x7f0d8011f430 con 0x7f0d840c0be0 2024-09-13T17:00:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.402+0000 7f0d896ce640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d680666c0 0x7f0d68068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.402+0000 7f0d896ce640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d680666c0 0x7f0d68068b80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0d84072690 tx=0x7f0d74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:01.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.406+0000 7f0d7a7fc640 1 -- 172.21.15.39:0/2769642927 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d800ec0b0 con 0x7f0d840c0be0 2024-09-13T17:00:01.539 INFO:tasks.mds_thrash.fs.[a]:reviving mds.b 2024-09-13T17:00:01.539 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:00:01.539 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:00:01.542 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:00:01.542 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:00:01.620 INFO:tasks.ceph.mds.b.smithi084.stdout:starting mds.b at 2024-09-13T17:00:01.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.650+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d840c2d90 con 0x7f0d840c0be0 2024-09-13T17:00:01.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.650+0000 7f0d7a7fc640 1 -- 172.21.15.39:0/2769642927 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5176 (secure 0 0 0) 0x7f0d800ef6d0 con 0x7f0d840c0be0 2024-09-13T17:00:01.649 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:00:01.649 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:00:01.650 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 84 2024-09-13T17:00:01.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d680666c0 msgr2=0x7f0d68068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d680666c0 0x7f0d68068b80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0d84072690 tx=0x7f0d74008040 comp rx=0 tx=0).stop 2024-09-13T17:00:01.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c0be0 msgr2=0x7f0d841f0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c0be0 0x7f0d841f0600 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0d8001ab10 tx=0x7f0d8001aca0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 shutdown_connections 2024-09-13T17:00:01.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d680666c0 0x7f0d68068b80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d840c6290 0x7f0d84158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d840c2690 0x7f0d840a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 --2- 172.21.15.39:0/2769642927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840c0be0 0x7f0d841f0600 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 >> 172.21.15.39:0/2769642927 conn(0x7f0d840c32a0 msgr2=0x7f0d840c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:01.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 shutdown_connections 2024-09-13T17:00:01.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.654+0000 7f0d8c133640 1 -- 172.21.15.39:0/2769642927 wait complete. 2024-09-13T17:00:01.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.662+0000 7fde5d357640 1 Processor -- start 2024-09-13T17:00:01.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.662+0000 7fde5d357640 1 -- start start 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde5d357640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fde5806fa50 0x7fde58156130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde5d357640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fde580c6000 0x7fde580c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde5d357640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde580c5e50 con 0x7fde580c7960 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde5d357640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5806ff50 con 0x7fde580c6000 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde5d357640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde58072000 con 0x7fde5806fa50 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde577fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fde580c6000 0x7fde580c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde56ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fde580c7960 0x7fde58155a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44214/0 (socket says 172.21.15.39:44214) 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde56ffd640 1 -- 172.21.15.39:0/3183889182 learned_addr learned my addr 172.21.15.39:0/3183889182 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde567fc640 1 --2- 172.21.15.39:0/3183889182 >> v2:172.21.15.84:3300/0 conn(0x7fde5806fa50 0x7fde58156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde577fe640 1 -- 172.21.15.39:0/3183889182 >> v2:172.21.15.84:3300/0 conn(0x7fde5806fa50 msgr2=0x7fde58156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde577fe640 1 --2- 172.21.15.39:0/3183889182 >> v2:172.21.15.84:3300/0 conn(0x7fde5806fa50 0x7fde58156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde577fe640 1 -- 172.21.15.39:0/3183889182 >> v1:172.21.15.39:6789/0 conn(0x7fde580c7960 legacy=0x7fde58155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde55ffb640 1 -- 172.21.15.39:0/3183889182 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919660782 0 0) 0x7fde580c5e50 con 0x7fde580c7960 2024-09-13T17:00:01.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde577fe640 1 -- 172.21.15.39:0/3183889182 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580722e0 con 0x7fde580c6000 2024-09-13T17:00:01.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde577fe640 1 --2- 172.21.15.39:0/3183889182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde580c6000 0x7fde580c2570 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fde580c3d50 tx=0x7fde5001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c72b7d14d4eacb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:01.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde55ffb640 1 -- 172.21.15.39:0/3183889182 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde500a58c0 con 0x7fde580c6000 2024-09-13T17:00:01.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde55ffb640 1 -- 172.21.15.39:0/3183889182 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde500a5a20 con 0x7fde580c6000 2024-09-13T17:00:01.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.666+0000 7fde55ffb640 1 -- 172.21.15.39:0/3183889182 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde500a5db0 con 0x7fde580c6000 2024-09-13T17:00:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 -- 172.21.15.39:0/3183889182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde580c6000 msgr2=0x7fde580c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 --2- 172.21.15.39:0/3183889182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde580c6000 0x7fde580c2570 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fde580c3d50 tx=0x7fde5001c7e0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 -- 172.21.15.39:0/3183889182 shutdown_connections 2024-09-13T17:00:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 --2- 172.21.15.39:0/3183889182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde580c6000 0x7fde580c2570 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 --2- 172.21.15.39:0/3183889182 >> v2:172.21.15.84:3300/0 conn(0x7fde5806fa50 0x7fde58156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 -- 172.21.15.39:0/3183889182 >> 172.21.15.39:0/3183889182 conn(0x7fde5806c020 msgr2=0x7fde5806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 -- 172.21.15.39:0/3183889182 shutdown_connections 2024-09-13T17:00:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 -- 172.21.15.39:0/3183889182 wait complete. 2024-09-13T17:00:01.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 Processor -- start 2024-09-13T17:00:01.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 -- start start 2024-09-13T17:00:01.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde5806fa50 0x7fde580c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fde580c6000 0x7fde5806ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde580c7960 0x7fde5814f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde580c1330 con 0x7fde5806fa50 2024-09-13T17:00:01.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde580c1520 con 0x7fde580c7960 2024-09-13T17:00:01.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde5d357640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde581ec1b0 con 0x7fde580c6000 2024-09-13T17:00:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde56ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde5806fa50 0x7fde580c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde56ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde5806fa50 0x7fde580c0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51840/0 (socket says 172.21.15.39:51840) 2024-09-13T17:00:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde56ffd640 1 -- 172.21.15.39:0/3554293284 learned_addr learned my addr 172.21.15.39:0/3554293284 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde577fe640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde580c7960 0x7fde5814f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde567fc640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fde580c6000 0x7fde5806ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde56ffd640 1 -- 172.21.15.39:0/3554293284 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fde580c6000 msgr2=0x7fde5806ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde56ffd640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fde580c6000 0x7fde5806ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde56ffd640 1 -- 172.21.15.39:0/3554293284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde580c7960 msgr2=0x7fde5814f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde56ffd640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde580c7960 0x7fde5814f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.670+0000 7fde56ffd640 1 -- 172.21.15.39:0/3554293284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde50009a70 con 0x7fde5806fa50 2024-09-13T17:00:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde56ffd640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde5806fa50 0x7fde580c0df0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c480 tx=0x7fde4800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde3f7fe640 1 -- 172.21.15.39:0/3554293284 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde480074e0 con 0x7fde5806fa50 2024-09-13T17:00:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde3f7fe640 1 -- 172.21.15.39:0/3554293284 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde48007660 con 0x7fde5806fa50 2024-09-13T17:00:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde581ec490 con 0x7fde5806fa50 2024-09-13T17:00:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde3f7fe640 1 -- 172.21.15.39:0/3554293284 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48004a30 con 0x7fde5806fa50 2024-09-13T17:00:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde581eca50 con 0x7fde5806fa50 2024-09-13T17:00:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde581ecc40 con 0x7fde5806fa50 2024-09-13T17:00:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde3f7fe640 1 -- 172.21.15.39:0/3554293284 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde48004b90 con 0x7fde5806fa50 2024-09-13T17:00:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde3f7fe640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fde44066690 0x7fde44068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde3f7fe640 1 -- 172.21.15.39:0/3554293284 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7841+0+0 (secure 0 0 0) 0x7fde48083dd0 con 0x7fde5806fa50 2024-09-13T17:00:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde567fc640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fde44066690 0x7fde44068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.674+0000 7fde567fc640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fde44066690 0x7fde44068b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fde58068430 tx=0x7fde4c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:01.675 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:00:01.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.678+0000 7fde3f7fe640 1 -- 172.21.15.39:0/3554293284 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fde480544b0 con 0x7fde5806fa50 2024-09-13T17:00:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.878+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde580c2570 con 0x7fde5806fa50 2024-09-13T17:00:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde3f7fe640 1 -- 172.21.15.39:0/3554293284 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5176 (secure 0 0 0) 0x7fde48018070 con 0x7fde5806fa50 2024-09-13T17:00:01.879 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:00:01.879 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T16:59:07.921669+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5282":{"gid":5282,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":16,"addr":"172.21.15.84:6835/1326467723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1326467723},{"type":"v1","addr":"172.21.15.84:6835","nonce":1326467723}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:00:01.880 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 84 2024-09-13T17:00:01.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fde44066690 msgr2=0x7fde44068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fde44066690 0x7fde44068b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fde58068430 tx=0x7fde4c009190 comp rx=0 tx=0).stop 2024-09-13T17:00:01.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde5806fa50 msgr2=0x7fde580c0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:01.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde5806fa50 0x7fde580c0df0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c480 tx=0x7fde4800c940 comp rx=0 tx=0).stop 2024-09-13T17:00:01.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 shutdown_connections 2024-09-13T17:00:01.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fde44066690 0x7fde44068b50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fde580c7960 0x7fde5814f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fde580c6000 0x7fde5806ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 --2- 172.21.15.39:0/3554293284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde5806fa50 0x7fde580c0df0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:01.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 >> 172.21.15.39:0/3554293284 conn(0x7fde5806c020 msgr2=0x7fde580be400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:01.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 shutdown_connections 2024-09-13T17:00:01.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:01.882+0000 7fde5d357640 1 -- 172.21.15.39:0/3554293284 wait complete. 2024-09-13T17:00:01.899 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2024-09-13T17:00:01.957 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T17:00:01.958+0000 7fa5070ee640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:00:03.677 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:00:03.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.830+0000 7f8d78cc6640 1 Processor -- start 2024-09-13T17:00:03.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.830+0000 7f8d78cc6640 1 -- start start 2024-09-13T17:00:03.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d78cc6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d74151500 0x7f8d740c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:03.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d78cc6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8d74071dd0 0x7f8d74154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:03.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d78cc6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d740c47c0 0x7f8d741545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d78cc6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d740c5a30 con 0x7f8d74071dd0 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d78cc6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d741576a0 con 0x7f8d740c47c0 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d78cc6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74157980 con 0x7f8d74151500 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d6bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8d74071dd0 0x7f8d74154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d6bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8d74071dd0 0x7f8d74154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51854/0 (socket says 172.21.15.39:51854) 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d6bfff640 1 -- 172.21.15.39:0/3432196618 learned_addr learned my addr 172.21.15.39:0/3432196618 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d72d9d640 1 --2- 172.21.15.39:0/3432196618 >> v2:172.21.15.84:3300/0 conn(0x7f8d740c47c0 0x7f8d741545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d7259c640 1 --2- 172.21.15.39:0/3432196618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d74151500 0x7f8d740c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d6bfff640 1 -- 172.21.15.39:0/3432196618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d74151500 msgr2=0x7f8d740c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d6bfff640 1 --2- 172.21.15.39:0/3432196618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d74151500 0x7f8d740c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d6bfff640 1 -- 172.21.15.39:0/3432196618 >> v2:172.21.15.84:3300/0 conn(0x7f8d740c47c0 msgr2=0x7f8d741545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d6bfff640 1 --2- 172.21.15.39:0/3432196618 >> v2:172.21.15.84:3300/0 conn(0x7f8d740c47c0 0x7f8d741545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:03.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d6bfff640 1 -- 172.21.15.39:0/3432196618 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74157c60 con 0x7f8d74071dd0 2024-09-13T17:00:03.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d6bfff640 1 --2- 172.21.15.39:0/3432196618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 0x7f8d74154070 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8d74153d60 tx=0x7f8d5c01cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea04335ccb82f06a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:03.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d71d9b640 1 -- 172.21.15.39:0/3432196618 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c0ab640 con 0x7f8d74071dd0 2024-09-13T17:00:03.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d71d9b640 1 -- 172.21.15.39:0/3432196618 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c0a7070 con 0x7f8d74071dd0 2024-09-13T17:00:03.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.834+0000 7f8d71d9b640 1 -- 172.21.15.39:0/3432196618 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c0ab9c0 con 0x7f8d74071dd0 2024-09-13T17:00:03.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 -- 172.21.15.39:0/3432196618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 msgr2=0x7f8d74154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:03.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/3432196618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 0x7f8d74154070 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8d74153d60 tx=0x7f8d5c01cec0 comp rx=0 tx=0).stop 2024-09-13T17:00:03.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 -- 172.21.15.39:0/3432196618 shutdown_connections 2024-09-13T17:00:03.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/3432196618 >> v2:172.21.15.84:3300/0 conn(0x7f8d740c47c0 0x7f8d741545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:03.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/3432196618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 0x7f8d74154070 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:03.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/3432196618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d74151500 0x7f8d740c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:03.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 -- 172.21.15.39:0/3432196618 >> 172.21.15.39:0/3432196618 conn(0x7f8d740ab530 msgr2=0x7f8d740ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:03.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 -- 172.21.15.39:0/3432196618 shutdown_connections 2024-09-13T17:00:03.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 -- 172.21.15.39:0/3432196618 wait complete. 2024-09-13T17:00:03.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 Processor -- start 2024-09-13T17:00:03.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 -- start start 2024-09-13T17:00:03.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 0x7f8d7414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:03.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d740c47c0 0x7f8d741470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:03.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d74151500 0x7f8d74147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:03.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74147b60 con 0x7f8d74071dd0 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d74149870 con 0x7f8d74151500 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d78cc6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d741483f0 con 0x7f8d740c47c0 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d7259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 0x7f8d7414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d7259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 0x7f8d7414ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51870/0 (socket says 172.21.15.39:51870) 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d7259c640 1 -- 172.21.15.39:0/1818835865 learned_addr learned my addr 172.21.15.39:0/1818835865 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d72d9d640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d74151500 0x7f8d74147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d6bfff640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d740c47c0 0x7f8d741470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d7259c640 1 -- 172.21.15.39:0/1818835865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d740c47c0 msgr2=0x7f8d741470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d7259c640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d740c47c0 0x7f8d741470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d7259c640 1 -- 172.21.15.39:0/1818835865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d74151500 msgr2=0x7f8d74147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d7259c640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d74151500 0x7f8d74147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:03.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.838+0000 7f8d7259c640 1 -- 172.21.15.39:0/1818835865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c0094d0 con 0x7f8d74071dd0 2024-09-13T17:00:03.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d7259c640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 0x7f8d7414ea70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d5f0 tx=0x7f8d6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:03.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d6affd640 1 -- 172.21.15.39:0/1818835865 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640125f0 con 0x7f8d74071dd0 2024-09-13T17:00:03.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d6affd640 1 -- 172.21.15.39:0/1818835865 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d64012750 con 0x7f8d74071dd0 2024-09-13T17:00:03.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d741ed6d0 con 0x7f8d74071dd0 2024-09-13T17:00:03.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d6affd640 1 -- 172.21.15.39:0/1818835865 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64012ae0 con 0x7f8d74071dd0 2024-09-13T17:00:03.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d74148110 con 0x7f8d74071dd0 2024-09-13T17:00:03.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d740c74d0 con 0x7f8d74071dd0 2024-09-13T17:00:03.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d6affd640 1 -- 172.21.15.39:0/1818835865 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8d64012c40 con 0x7f8d74071dd0 2024-09-13T17:00:03.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d6affd640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d54066690 0x7f8d54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:03.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d6affd640 1 -- 172.21.15.39:0/1818835865 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f8d64084eb0 con 0x7f8d74071dd0 2024-09-13T17:00:03.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d6bfff640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d54066690 0x7f8d54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:03.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.842+0000 7f8d6bfff640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d54066690 0x7f8d54068b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0af000 tx=0x7f8d5c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:03.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:03.846+0000 7f8d6affd640 1 -- 172.21.15.39:0/1818835865 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8d64051a00 con 0x7f8d74071dd0 2024-09-13T17:00:03.900 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:00:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b71982640 1 Processor -- start 2024-09-13T17:00:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b71982640 1 -- start start 2024-09-13T17:00:04.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b71982640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c0c2ee0 0x7f8b6c0c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b71982640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b6c0c6270 0x7f8b6c0c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b71982640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b6c06ddf0 con 0x7f8b6c065ee0 2024-09-13T17:00:04.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b71982640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6c0c4710 con 0x7f8b6c0c6270 2024-09-13T17:00:04.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b71982640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0c49d0 con 0x7f8b6c0c2ee0 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b6affd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8b6c065ee0 0x7f8b6c0662c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44226/0 (socket says 172.21.15.39:44226) 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b6affd640 1 -- 172.21.15.39:0/1123564841 learned_addr learned my addr 172.21.15.39:0/1123564841 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b6a7fc640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c0c2ee0 0x7f8b6c0c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b6b7fe640 1 --2- 172.21.15.39:0/1123564841 >> v2:172.21.15.84:3300/0 conn(0x7f8b6c0c6270 0x7f8b6c0c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498731805 0 0) 0x7f8b6c06ddf0 con 0x7f8b6c065ee0 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b54003530 con 0x7f8b6c065ee0 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2713364974 0 0) 0x7f8b54003530 con 0x7f8b6c065ee0 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c0c2ee0 msgr2=0x7f8b6c0c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c0c2ee0 0x7f8b6c0c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 >> v2:172.21.15.84:3300/0 conn(0x7f8b6c0c6270 msgr2=0x7f8b6c0c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 --2- 172.21.15.39:0/1123564841 >> v2:172.21.15.84:3300/0 conn(0x7f8b6c0c6270 0x7f8b6c0c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c154880 con 0x7f8b6c065ee0 2024-09-13T17:00:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f8b5c001680 con 0x7f8b6c065ee0 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.034+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 >> v1:172.21.15.39:6789/0 conn(0x7f8b6c065ee0 legacy=0x7f8b6c0662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b69ffb640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54003980 0x7f8b54003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b69ffb640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c0c2ee0 0x7f8b540047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b69ffb640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54004d10 0x7f8b54005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0c4710 con 0x7f8b54003980 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0c49d0 con 0x7f8b54004d10 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5400b9c0 con 0x7f8b6c0c2ee0 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6affd640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54003980 0x7f8b54003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6a7fc640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c0c2ee0 0x7f8b540047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6b7fe640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54004d10 0x7f8b54005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6affd640 1 -- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c0c2ee0 msgr2=0x7f8b540047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:04.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6affd640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c0c2ee0 0x7f8b540047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6affd640 1 -- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54004d10 msgr2=0x7f8b54005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:04.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6affd640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54004d10 0x7f8b54005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6affd640 1 -- 172.21.15.39:0/1123564841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b54003530 con 0x7f8b54003980 2024-09-13T17:00:04.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6b7fe640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54004d10 0x7f8b54005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:00:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b6affd640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54003980 0x7f8b54003da0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c210 tx=0x7f8b5c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fac3068a490ea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c0058e0 con 0x7f8b54003980 2024-09-13T17:00:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c005a40 con 0x7f8b54003980 2024-09-13T17:00:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.038+0000 7f8b69ffb640 1 -- 172.21.15.39:0/1123564841 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c005df0 con 0x7f8b54003980 2024-09-13T17:00:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 -- 172.21.15.39:0/1123564841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54003980 msgr2=0x7f8b54003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 --2- 172.21.15.39:0/1123564841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b54003980 0x7f8b54003da0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00c210 tx=0x7f8b5c01cc60 comp rx=0 tx=0).stop 2024-09-13T17:00:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b6b7fe640 1 -- 172.21.15.39:0/1123564841 reap_dead start 2024-09-13T17:00:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 -- 172.21.15.39:0/1123564841 shutdown_connections 2024-09-13T17:00:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 -- 172.21.15.39:0/1123564841 >> 172.21.15.39:0/1123564841 conn(0x7f8b6c0c3310 msgr2=0x7f8b6c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 -- 172.21.15.39:0/1123564841 shutdown_connections 2024-09-13T17:00:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 -- 172.21.15.39:0/1123564841 wait complete. 2024-09-13T17:00:04.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 Processor -- start 2024-09-13T17:00:04.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 -- start start 2024-09-13T17:00:04.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54003980 0x7f8b6c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c1ecb70 0x7f8b6c14a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b6c1f0850 0x7f8b6c071840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1456f0 con 0x7f8b6c1f0850 2024-09-13T17:00:04.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1ecff0 con 0x7f8b54003980 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b71982640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1f00e0 con 0x7f8b6c1ecb70 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b6b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b6c1f0850 0x7f8b6c071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b6b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b6c1f0850 0x7f8b6c071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51878/0 (socket says 172.21.15.39:51878) 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.042+0000 7f8b6b7fe640 1 -- 172.21.15.39:0/1284427218 learned_addr learned my addr 172.21.15.39:0/1284427218 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b6a7fc640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c1ecb70 0x7f8b6c14a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b6affd640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54003980 0x7f8b6c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b6b7fe640 1 -- 172.21.15.39:0/1284427218 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c1ecb70 msgr2=0x7f8b6c14a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b6b7fe640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c1ecb70 0x7f8b6c14a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b6b7fe640 1 -- 172.21.15.39:0/1284427218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54003980 msgr2=0x7f8b6c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b6b7fe640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54003980 0x7f8b6c15a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b6b7fe640 1 -- 172.21.15.39:0/1284427218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c1f03c0 con 0x7f8b6c1f0850 2024-09-13T17:00:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b6b7fe640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b6c1f0850 0x7f8b6c071840 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0c2ca0 tx=0x7f8b640115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:04.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b53fff640 1 -- 172.21.15.39:0/1284427218 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400e030 con 0x7f8b6c1f0850 2024-09-13T17:00:04.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b53fff640 1 -- 172.21.15.39:0/1284427218 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b64011b50 con 0x7f8b6c1f0850 2024-09-13T17:00:04.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c152a50 con 0x7f8b6c1f0850 2024-09-13T17:00:04.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c150c70 con 0x7f8b6c1f0850 2024-09-13T17:00:04.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b53fff640 1 -- 172.21.15.39:0/1284427218 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64016470 con 0x7f8b6c1f0850 2024-09-13T17:00:04.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6c14ded0 con 0x7f8b6c1f0850 2024-09-13T17:00:04.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b53fff640 1 -- 172.21.15.39:0/1284427218 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b640165d0 con 0x7f8b6c1f0850 2024-09-13T17:00:04.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b53fff640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b580666c0 0x7f8b58068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b53fff640 1 -- 172.21.15.39:0/1284427218 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f8b6400f070 con 0x7f8b6c1f0850 2024-09-13T17:00:04.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.046+0000 7f8b6affd640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b580666c0 0x7f8b58068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.050+0000 7f8b6affd640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b580666c0 0x7f8b58068b80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0aa000 tx=0x7f8b5c000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.054+0000 7f8b53fff640 1 -- 172.21.15.39:0/1284427218 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8b64051f60 con 0x7f8b6c1f0850 2024-09-13T17:00:04.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.086+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d7414fda0 con 0x7f8d74071dd0 2024-09-13T17:00:04.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.086+0000 7f8d6affd640 1 -- 172.21.15.39:0/1818835865 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f8d64055020 con 0x7f8d74071dd0 2024-09-13T17:00:04.085 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:00:04.086 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:04.047909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:00:04.086 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 88 2024-09-13T17:00:04.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d54066690 msgr2=0x7f8d54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d54066690 0x7f8d54068b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0af000 tx=0x7f8d5c004740 comp rx=0 tx=0).stop 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 msgr2=0x7f8d7414ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 0x7f8d7414ea70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400d5f0 tx=0x7f8d6400dab0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 shutdown_connections 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8d54066690 0x7f8d54068b50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d74151500 0x7f8d74147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8d740c47c0 0x7f8d741470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 --2- 172.21.15.39:0/1818835865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d74071dd0 0x7f8d7414ea70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 >> 172.21.15.39:0/1818835865 conn(0x7f8d740ab530 msgr2=0x7f8d740a94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 shutdown_connections 2024-09-13T17:00:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.090+0000 7f8d78cc6640 1 -- 172.21.15.39:0/1818835865 wait complete. 2024-09-13T17:00:04.107 INFO:tasks.mds_thrash.fs.[b]:mds.d down, removed from mdsmap 2024-09-13T17:00:04.107 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:00:04.107 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:00:04.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.230+0000 7ffbc0bca640 1 Processor -- start 2024-09-13T17:00:04.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.230+0000 7ffbc0bca640 1 -- start start 2024-09-13T17:00:04.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.230+0000 7ffbc0bca640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc06b080 0x7ffbbc06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.230+0000 7ffbc0bca640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ffbbc1576e0 0x7ffbbc157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.230+0000 7ffbc0bca640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc0c15d0 con 0x7ffbbc0c1810 2024-09-13T17:00:04.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.230+0000 7ffbc0bca640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbbc06d0b0 con 0x7ffbbc1576e0 2024-09-13T17:00:04.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.230+0000 7ffbc0bca640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0c1f00 con 0x7ffbbc06b080 2024-09-13T17:00:04.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb9d9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ffbbc0c1810 0x7ffbbc151390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44234/0 (socket says 172.21.15.39:44234) 2024-09-13T17:00:04.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb9d9b640 1 -- 172.21.15.39:0/1302081106 learned_addr learned my addr 172.21.15.39:0/1302081106 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:00:04.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbba59c640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc06b080 0x7ffbbc06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbbad9d640 1 --2- 172.21.15.39:0/1302081106 >> v2:172.21.15.84:3300/0 conn(0x7ffbbc1576e0 0x7ffbbc157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591393085 0 0) 0x7ffbbc0c15d0 con 0x7ffbbc0c1810 2024-09-13T17:00:04.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba4003530 con 0x7ffbbc0c1810 2024-09-13T17:00:04.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1902029587 0 0) 0x7ffba4003530 con 0x7ffbbc0c1810 2024-09-13T17:00:04.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc06b080 msgr2=0x7ffbbc06e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc06b080 0x7ffbbc06e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 >> v2:172.21.15.84:3300/0 conn(0x7ffbbc1576e0 msgr2=0x7ffbbc157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 --2- 172.21.15.39:0/1302081106 >> v2:172.21.15.84:3300/0 conn(0x7ffbbc1576e0 0x7ffbbc157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc0c21a0 con 0x7ffbbc0c1810 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7ffbb00016a0 con 0x7ffbbc0c1810 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 >> v1:172.21.15.39:6789/0 conn(0x7ffbbc0c1810 legacy=0x7ffbbc151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba4003980 0x7ffba4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffba4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc06b080 0x7ffba4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0c1f00 con 0x7ffba40047d0 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba400b990 con 0x7ffba4003980 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba400bad0 con 0x7ffbbc06b080 2024-09-13T17:00:04.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb9d9b640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffba4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbbad9d640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc06b080 0x7ffba4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbba59c640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba4003980 0x7ffba4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb9d9b640 1 -- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc06b080 msgr2=0x7ffba4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb9d9b640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc06b080 0x7ffba4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb9d9b640 1 -- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba4003980 msgr2=0x7ffba4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb9d9b640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffba4003980 0x7ffba4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.234+0000 7ffbb9d9b640 1 -- 172.21.15.39:0/1302081106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4003530 con 0x7ffba40047d0 2024-09-13T17:00:04.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbb9d9b640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffba4004c50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c290 tx=0x7ffbb001f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777014ffe126d4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:04.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00058e0 con 0x7ffba40047d0 2024-09-13T17:00:04.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb0005a40 con 0x7ffba40047d0 2024-09-13T17:00:04.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbb959a640 1 -- 172.21.15.39:0/1302081106 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0005df0 con 0x7ffba40047d0 2024-09-13T17:00:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 -- 172.21.15.39:0/1302081106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 msgr2=0x7ffba4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 --2- 172.21.15.39:0/1302081106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffba4004c50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c290 tx=0x7ffbb001f450 comp rx=0 tx=0).stop 2024-09-13T17:00:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbbad9d640 1 -- 172.21.15.39:0/1302081106 reap_dead start 2024-09-13T17:00:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 -- 172.21.15.39:0/1302081106 shutdown_connections 2024-09-13T17:00:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 -- 172.21.15.39:0/1302081106 >> 172.21.15.39:0/1302081106 conn(0x7ffbbc0c32a0 msgr2=0x7ffbbc0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:04.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 -- 172.21.15.39:0/1302081106 shutdown_connections 2024-09-13T17:00:04.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 -- 172.21.15.39:0/1302081106 wait complete. 2024-09-13T17:00:04.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 Processor -- start 2024-09-13T17:00:04.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 -- start start 2024-09-13T17:00:04.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffbbc0bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbbc148ce0 0x7ffbbc0c0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc146a80 0x7ffbbc146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc147440 con 0x7ffba40047d0 2024-09-13T17:00:04.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0bff40 con 0x7ffbbc148ce0 2024-09-13T17:00:04.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.238+0000 7ffbc0bca640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc1ecb90 con 0x7ffbbc146a80 2024-09-13T17:00:04.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbba59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffbbc0bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbba59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffbbc0bfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51896/0 (socket says 172.21.15.39:51896) 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbba59c640 1 -- 172.21.15.39:0/2291669979 learned_addr learned my addr 172.21.15.39:0/2291669979 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbb9d9b640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbbc148ce0 0x7ffbbc0c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbbad9d640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc146a80 0x7ffbbc146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbba59c640 1 -- 172.21.15.39:0/2291669979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc146a80 msgr2=0x7ffbbc146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbba59c640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc146a80 0x7ffbbc146f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbba59c640 1 -- 172.21.15.39:0/2291669979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbbc148ce0 msgr2=0x7ffbbc0c0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbba59c640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbbc148ce0 0x7ffbbc0c0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbba59c640 1 -- 172.21.15.39:0/2291669979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc1ece70 con 0x7ffba40047d0 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbb9d9b640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbbc148ce0 0x7ffbbc0c0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbba59c640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffbbc0bfa00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ffbac011560 tx=0x7ffbac011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbaaffd640 1 -- 172.21.15.39:0/2291669979 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac015930 con 0x7ffba40047d0 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbaaffd640 1 -- 172.21.15.39:0/2291669979 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac015a90 con 0x7ffba40047d0 2024-09-13T17:00:04.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbaaffd640 1 -- 172.21.15.39:0/2291669979 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac015e20 con 0x7ffba40047d0 2024-09-13T17:00:04.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc1ed150 con 0x7ffba40047d0 2024-09-13T17:00:04.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc1ed710 con 0x7ffba40047d0 2024-09-13T17:00:04.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc06b460 con 0x7ffba40047d0 2024-09-13T17:00:04.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbaaffd640 1 -- 172.21.15.39:0/2291669979 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffbac00f460 con 0x7ffba40047d0 2024-09-13T17:00:04.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbaaffd640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffb9c0666c0 0x7ffb9c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:04.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbaaffd640 1 -- 172.21.15.39:0/2291669979 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7ffbac0850b0 con 0x7ffba40047d0 2024-09-13T17:00:04.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbb9d9b640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffb9c0666c0 0x7ffb9c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:04.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.242+0000 7ffbb9d9b640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffb9c0666c0 0x7ffb9c068b80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00ae000 tx=0x7ffbb00aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:04.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.250+0000 7ffbaaffd640 1 -- 172.21.15.39:0/2291669979 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffbac051c00 con 0x7ffba40047d0 2024-09-13T17:00:04.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.294+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b6c152250 con 0x7f8b6c1f0850 2024-09-13T17:00:04.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.294+0000 7f8b53fff640 1 -- 172.21.15.39:0/1284427218 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f8b64055580 con 0x7f8b6c1f0850 2024-09-13T17:00:04.295 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:00:04.295 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:04.047909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:00:04.295 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 88 2024-09-13T17:00:04.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b580666c0 msgr2=0x7f8b58068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b580666c0 0x7f8b58068b80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0aa000 tx=0x7f8b5c000ff0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b6c1f0850 msgr2=0x7f8b6c071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b6c1f0850 0x7f8b6c071840 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c0c2ca0 tx=0x7f8b640115a0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 shutdown_connections 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b580666c0 0x7f8b58068b80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b6c1f0850 0x7f8b6c071840 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b6c1ecb70 0x7f8b6c14a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 --2- 172.21.15.39:0/1284427218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b54003980 0x7f8b6c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 >> 172.21.15.39:0/1284427218 conn(0x7f8b6c0c3310 msgr2=0x7f8b6c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 shutdown_connections 2024-09-13T17:00:04.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.298+0000 7f8b71982640 1 -- 172.21.15.39:0/1284427218 wait complete. 2024-09-13T17:00:04.318 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2024-09-13T17:00:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.518+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbbc151750 con 0x7ffba40047d0 2024-09-13T17:00:04.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbaaffd640 1 -- 172.21.15.39:0/2291669979 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+5175 (secure 0 0 0) 0x7ffbac055220 con 0x7ffba40047d0 2024-09-13T17:00:04.519 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:00:04.519 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:04.047909+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:00:04.520 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 88 2024-09-13T17:00:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffb9c0666c0 msgr2=0x7ffb9c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffb9c0666c0 0x7ffb9c068b80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00ae000 tx=0x7ffbb00aa070 comp rx=0 tx=0).stop 2024-09-13T17:00:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 msgr2=0x7ffbbc0bfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffbbc0bfa00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ffbac011560 tx=0x7ffbac011a20 comp rx=0 tx=0).stop 2024-09-13T17:00:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 shutdown_connections 2024-09-13T17:00:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffb9c0666c0 0x7ffb9c068b80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffbbc148ce0 0x7ffbbc0c0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffbbc146a80 0x7ffbbc146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 --2- 172.21.15.39:0/2291669979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffba40047d0 0x7ffbbc0bfa00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 >> 172.21.15.39:0/2291669979 conn(0x7ffbbc0c32a0 msgr2=0x7ffbbc157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 shutdown_connections 2024-09-13T17:00:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:04.522+0000 7ffbc0bca640 1 -- 172.21.15.39:0/2291669979 wait complete. 2024-09-13T17:00:04.543 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 88, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 84, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6243": { "addr": "172.21.15.39:6835/4246622083", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 4246622083, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 4246622083, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6243, "incarnation": 81, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 40, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T16:59:57.987543+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6243 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 88, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5423": { "addr": "172.21.15.84:6833/2638959497", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 2638959497, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 2638959497, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5423, "incarnation": 86, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:rejoin", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 41, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:00:04.047909+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5423 } } } ], "standbys": [ { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:00:06.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:00:06.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f67a0983640 1 Processor -- start 2024-09-13T17:00:06.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f67a0983640 1 -- start start 2024-09-13T17:00:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f67a0983640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f679c14e0c0 0x7f679c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f67a0983640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c0c1f50 0x7f679c06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f67a0983640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679c0c2780 con 0x7f679c06d300 2024-09-13T17:00:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f67a0983640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679c0c2a60 con 0x7f679c14e0c0 2024-09-13T17:00:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f67a0983640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c0c2d40 con 0x7f679c0c1f50 2024-09-13T17:00:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f679a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f679c14e0c0 0x7f679c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f679a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f679c14e0c0 0x7f679c0a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51912/0 (socket says 172.21.15.39:51912) 2024-09-13T17:00:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f679a59c640 1 -- 172.21.15.39:0/2717501413 learned_addr learned my addr 172.21.15.39:0/2717501413 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f6791d9b640 1 --2- 172.21.15.39:0/2717501413 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c0c1f50 0x7f679c06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f679a59c640 1 -- 172.21.15.39:0/2717501413 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c0c1f50 msgr2=0x7f679c06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f679a59c640 1 --2- 172.21.15.39:0/2717501413 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c0c1f50 0x7f679c06cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f679a59c640 1 -- 172.21.15.39:0/2717501413 >> v1:172.21.15.84:6789/0 conn(0x7f679c06d300 legacy=0x7f679c0c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.458+0000 7f679a59c640 1 -- 172.21.15.39:0/2717501413 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c0c3020 con 0x7f679c14e0c0 2024-09-13T17:00:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f679a59c640 1 --2- 172.21.15.39:0/2717501413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c14e0c0 0x7f679c0a4e70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f678c009870 tx=0x7f678c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431bb3a412f024d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f6799d9b640 1 -- 172.21.15.39:0/2717501413 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c0ac620 con 0x7f679c14e0c0 2024-09-13T17:00:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f6799d9b640 1 -- 172.21.15.39:0/2717501413 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c0ac780 con 0x7f679c14e0c0 2024-09-13T17:00:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f6799d9b640 1 -- 172.21.15.39:0/2717501413 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c0acb10 con 0x7f679c14e0c0 2024-09-13T17:00:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 -- 172.21.15.39:0/2717501413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c14e0c0 msgr2=0x7f679c0a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 --2- 172.21.15.39:0/2717501413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c14e0c0 0x7f679c0a4e70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f678c009870 tx=0x7f678c01c4a0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 -- 172.21.15.39:0/2717501413 shutdown_connections 2024-09-13T17:00:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 --2- 172.21.15.39:0/2717501413 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c0c1f50 0x7f679c06cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 --2- 172.21.15.39:0/2717501413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c14e0c0 0x7f679c0a4e70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 -- 172.21.15.39:0/2717501413 >> 172.21.15.39:0/2717501413 conn(0x7f679c069e10 msgr2=0x7f679c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 -- 172.21.15.39:0/2717501413 shutdown_connections 2024-09-13T17:00:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 -- 172.21.15.39:0/2717501413 wait complete. 2024-09-13T17:00:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 Processor -- start 2024-09-13T17:00:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 -- start start 2024-09-13T17:00:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f679c06d300 0x7f679c0a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c0c1f50 0x7f679c14e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c14e0c0 0x7f679c14ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c15dce0 con 0x7f679c0c1f50 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c159b30 con 0x7f679c06d300 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.462+0000 7f67a0983640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c159890 con 0x7f679c14e0c0 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f6791d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c0c1f50 0x7f679c14e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f6791d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c0c1f50 0x7f679c14e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51916/0 (socket says 172.21.15.39:51916) 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f6791d9b640 1 -- 172.21.15.39:0/3105445625 learned_addr learned my addr 172.21.15.39:0/3105445625 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f679a59c640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f679c06d300 0x7f679c0a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f679ad9d640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c14e0c0 0x7f679c14ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f6791d9b640 1 -- 172.21.15.39:0/3105445625 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c14e0c0 msgr2=0x7f679c14ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f6791d9b640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c14e0c0 0x7f679c14ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f6791d9b640 1 -- 172.21.15.39:0/3105445625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f679c06d300 msgr2=0x7f679c0a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f6791d9b640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f679c06d300 0x7f679c0a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f6791d9b640 1 -- 172.21.15.39:0/3105445625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c0094d0 con 0x7f679c0c1f50 2024-09-13T17:00:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f6791d9b640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c0c1f50 0x7f679c14e7b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f678400d830 tx=0x7f678400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f67937fe640 1 -- 172.21.15.39:0/3105445625 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400b6c0 con 0x7f679c0c1f50 2024-09-13T17:00:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f67937fe640 1 -- 172.21.15.39:0/3105445625 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678400b820 con 0x7f679c0c1f50 2024-09-13T17:00:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c15d690 con 0x7f679c0c1f50 2024-09-13T17:00:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f67937fe640 1 -- 172.21.15.39:0/3105445625 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400bb90 con 0x7f679c0c1f50 2024-09-13T17:00:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c159490 con 0x7f679c0c1f50 2024-09-13T17:00:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679c15de90 con 0x7f679c0c1f50 2024-09-13T17:00:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f67937fe640 1 -- 172.21.15.39:0/3105445625 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f678401e460 con 0x7f679c0c1f50 2024-09-13T17:00:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f67937fe640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f677c066690 0x7f677c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f67937fe640 1 -- 172.21.15.39:0/3105445625 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f6784085080 con 0x7f679c0c1f50 2024-09-13T17:00:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f679a59c640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f677c066690 0x7f677c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.466+0000 7f679a59c640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f677c066690 0x7f677c068b50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f678c0ae000 tx=0x7f678c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:06.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.470+0000 7f67937fe640 1 -- 172.21.15.39:0/3105445625 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f679c15de90 con 0x7f679c0c1f50 2024-09-13T17:00:06.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:00:06.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.670+0000 7f5501abc640 1 Processor -- start 2024-09-13T17:00:06.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.670+0000 7f5501abc640 1 -- start start 2024-09-13T17:00:06.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.670+0000 7f5501abc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54fc0c4260 0x7f54fc0c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.670+0000 7f5501abc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f54fc070b40 0x7f54fc15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.670+0000 7f5501abc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc157360 con 0x7f54fc0bcfb0 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.670+0000 7f5501abc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc0c80b0 con 0x7f54fc070b40 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.670+0000 7f5501abc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc1575b0 con 0x7f54fc0c4260 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.670+0000 7f54f3fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f54fc0bcfb0 0x7f54fc070430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44242/0 (socket says 172.21.15.39:44242) 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54f3fff640 1 -- 172.21.15.39:0/3087758479 learned_addr learned my addr 172.21.15.39:0/3087758479 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54fbfff640 1 --2- 172.21.15.39:0/3087758479 >> v2:172.21.15.39:3300/0 conn(0x7f54fc070b40 0x7f54fc15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54fb7fe640 1 --2- 172.21.15.39:0/3087758479 >> v2:172.21.15.84:3300/0 conn(0x7f54fc0c4260 0x7f54fc0c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54faffd640 1 -- 172.21.15.39:0/3087758479 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191864652 0 0) 0x7f54fc157360 con 0x7f54fc0bcfb0 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54faffd640 1 -- 172.21.15.39:0/3087758479 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54e8003530 con 0x7f54fc0bcfb0 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54fbfff640 1 -- 172.21.15.39:0/3087758479 >> v2:172.21.15.84:3300/0 conn(0x7f54fc0c4260 msgr2=0x7f54fc0c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54fbfff640 1 --2- 172.21.15.39:0/3087758479 >> v2:172.21.15.84:3300/0 conn(0x7f54fc0c4260 0x7f54fc0c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54fbfff640 1 -- 172.21.15.39:0/3087758479 >> v1:172.21.15.39:6789/0 conn(0x7f54fc0bcfb0 legacy=0x7f54fc070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54fbfff640 1 -- 172.21.15.39:0/3087758479 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc157890 con 0x7f54fc070b40 2024-09-13T17:00:06.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54fbfff640 1 --2- 172.21.15.39:0/3087758479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc070b40 0x7f54fc15e6f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0c6240 tx=0x7f54f401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1063ed065f2d29d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:06.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54faffd640 1 -- 172.21.15.39:0/3087758479 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc0c7330 con 0x7f54fc070b40 2024-09-13T17:00:06.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54faffd640 1 -- 172.21.15.39:0/3087758479 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f40a57e0 con 0x7f54fc070b40 2024-09-13T17:00:06.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f54faffd640 1 -- 172.21.15.39:0/3087758479 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f40a5b70 con 0x7f54fc070b40 2024-09-13T17:00:06.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f5501abc640 1 -- 172.21.15.39:0/3087758479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc070b40 msgr2=0x7f54fc15e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f5501abc640 1 --2- 172.21.15.39:0/3087758479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc070b40 0x7f54fc15e6f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0c6240 tx=0x7f54f401c810 comp rx=0 tx=0).stop 2024-09-13T17:00:06.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.674+0000 7f5501abc640 1 -- 172.21.15.39:0/3087758479 shutdown_connections 2024-09-13T17:00:06.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 --2- 172.21.15.39:0/3087758479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc070b40 0x7f54fc15e6f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 --2- 172.21.15.39:0/3087758479 >> v2:172.21.15.84:3300/0 conn(0x7f54fc0c4260 0x7f54fc0c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 -- 172.21.15.39:0/3087758479 >> 172.21.15.39:0/3087758479 conn(0x7f54fc069e10 msgr2=0x7f54fc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:06.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 -- 172.21.15.39:0/3087758479 shutdown_connections 2024-09-13T17:00:06.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 -- 172.21.15.39:0/3087758479 wait complete. 2024-09-13T17:00:06.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 Processor -- start 2024-09-13T17:00:06.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 -- start start 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54fc0bcfb0 0x7f54fc0bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc0c4260 0x7f54fc0bd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54fc0bdb20 0x7f54fc0bdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc15e720 con 0x7f54fc0c4260 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0aaaa0 con 0x7f54fc0bdb20 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc15e910 con 0x7f54fc0bcfb0 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc0c4260 0x7f54fc0bd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc0c4260 0x7f54fc0bd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51932/0 (socket says 172.21.15.39:51932) 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f3fff640 1 -- 172.21.15.39:0/3928150288 learned_addr learned my addr 172.21.15.39:0/3928150288 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54fbfff640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54fc0bdb20 0x7f54fc0bdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54fb7fe640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54fc0bcfb0 0x7f54fc0bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f3fff640 1 -- 172.21.15.39:0/3928150288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54fc0bcfb0 msgr2=0x7f54fc0bfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f3fff640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54fc0bcfb0 0x7f54fc0bfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f3fff640 1 -- 172.21.15.39:0/3928150288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54fc0bdb20 msgr2=0x7f54fc0bdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f3fff640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54fc0bdb20 0x7f54fc0bdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f3fff640 1 -- 172.21.15.39:0/3928150288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f4009a70 con 0x7f54fc0c4260 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54fb7fe640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54fc0bcfb0 0x7f54fc0bfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f3fff640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc0c4260 0x7f54fc0bd5e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f54e400c490 tx=0x7f54e400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f8ff9640 1 -- 172.21.15.39:0/3928150288 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40074e0 con 0x7f54fc0c4260 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f8ff9640 1 -- 172.21.15.39:0/3928150288 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e4007640 con 0x7f54fc0c4260 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc0aad60 con 0x7f54fc0c4260 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f54f8ff9640 1 -- 172.21.15.39:0/3928150288 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4004a30 con 0x7f54fc0c4260 2024-09-13T17:00:06.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.678+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc14b6b0 con 0x7f54fc0c4260 2024-09-13T17:00:06.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.682+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc0c4640 con 0x7f54fc0c4260 2024-09-13T17:00:06.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.682+0000 7f54f8ff9640 1 -- 172.21.15.39:0/3928150288 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54e4004b90 con 0x7f54fc0c4260 2024-09-13T17:00:06.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.682+0000 7f54f8ff9640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f54dc066690 0x7f54dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:06.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.682+0000 7f54f8ff9640 1 -- 172.21.15.39:0/3928150288 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f54e4083dc0 con 0x7f54fc0c4260 2024-09-13T17:00:06.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.682+0000 7f54fb7fe640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f54dc066690 0x7f54dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:06.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.682+0000 7f54fb7fe640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f54dc066690 0x7f54dc068b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0c04c0 tx=0x7f54ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:06.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.686+0000 7f54f8ff9640 1 -- 172.21.15.39:0/3928150288 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f54e40509c0 con 0x7f54fc0c4260 2024-09-13T17:00:06.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.710+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f679c0a4e70 con 0x7f679c0c1f50 2024-09-13T17:00:06.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.710+0000 7f67937fe640 1 -- 172.21.15.39:0/3105445625 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f6784051bd0 con 0x7f679c0c1f50 2024-09-13T17:00:06.710 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:00:06.710 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:00:06.710 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 89 2024-09-13T17:00:06.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f677c066690 msgr2=0x7f677c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f677c066690 0x7f677c068b50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f678c0ae000 tx=0x7f678c0a7000 comp rx=0 tx=0).stop 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c0c1f50 msgr2=0x7f679c14e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c0c1f50 0x7f679c14e7b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f678400d830 tx=0x7f678400dcf0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 shutdown_connections 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f677c066690 0x7f677c068b50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f679c14e0c0 0x7f679c14ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679c0c1f50 0x7f679c14e7b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 --2- 172.21.15.39:0/3105445625 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f679c06d300 0x7f679c0a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 >> 172.21.15.39:0/3105445625 conn(0x7f679c069e10 msgr2=0x7f679c14b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 shutdown_connections 2024-09-13T17:00:06.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.714+0000 7f67a0983640 1 -- 172.21.15.39:0/3105445625 wait complete. 2024-09-13T17:00:06.739 INFO:tasks.mds_thrash.fs.[a]:mds.b reported in up:active state 2024-09-13T17:00:06.740 INFO:tasks.mds_thrash.fs.[a]:waiting for 63 secs before thrashing 2024-09-13T17:00:06.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.910+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54fc0c6e70 con 0x7f54fc0c4260 2024-09-13T17:00:06.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.910+0000 7f54f8ff9640 1 -- 172.21.15.39:0/3928150288 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f54e4081e60 con 0x7f54fc0c4260 2024-09-13T17:00:06.910 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:00:06.910 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:00:06.910 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 89 2024-09-13T17:00:06.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f54dc066690 msgr2=0x7f54dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f54dc066690 0x7f54dc068b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0c04c0 tx=0x7f54ec008040 comp rx=0 tx=0).stop 2024-09-13T17:00:06.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc0c4260 msgr2=0x7f54fc0bd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:06.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc0c4260 0x7f54fc0bd5e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f54e400c490 tx=0x7f54e400c950 comp rx=0 tx=0).stop 2024-09-13T17:00:06.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 shutdown_connections 2024-09-13T17:00:06.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f54dc066690 0x7f54dc068b50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54fc0bdb20 0x7f54fc0bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc0c4260 0x7f54fc0bd5e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 --2- 172.21.15.39:0/3928150288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f54fc0bcfb0 0x7f54fc0bfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:06.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 >> 172.21.15.39:0/3928150288 conn(0x7f54fc069e10 msgr2=0x7f54fc155d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:06.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 shutdown_connections 2024-09-13T17:00:06.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:06.914+0000 7f5501abc640 1 -- 172.21.15.39:0/3928150288 wait complete. 2024-09-13T17:00:06.934 INFO:tasks.mds_thrash.fs.[b]:mds.b has gained rank=0, replacing gid=5282 2024-09-13T17:00:06.934 INFO:tasks.mds_thrash.fs.[b]:waiting for 5 secs before reviving mds.d 2024-09-13T17:00:07.032 INFO:tasks.ceph.mds.b.smithi084.stderr:2024-09-13T17:00:07.032+0000 7fe90acc9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:00:11.934 INFO:tasks.mds_thrash.fs.[b]:reviving mds.d 2024-09-13T17:00:11.935 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T17:00:11.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T17:00:11.938 INFO:tasks.ceph.mds.d:Started 2024-09-13T17:00:11.938 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:00:12.018 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T17:00:12.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.078+0000 7f25771fa640 1 Processor -- start 2024-09-13T17:00:12.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f25771fa640 1 -- start start 2024-09-13T17:00:12.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f25771fa640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f25700c21d0 0x7f25700aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:12.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f25771fa640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f257014b850 0x7f257014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:12.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f25771fa640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25700c3ae0 con 0x7f257006aac0 2024-09-13T17:00:12.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f25771fa640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25700ab340 con 0x7f25700c21d0 2024-09-13T17:00:12.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f25771fa640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25700abb30 con 0x7f257014b850 2024-09-13T17:00:12.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2574f96640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f25700c21d0 0x7f25700aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:12.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2575797640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f257014b850 0x7f257014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:12.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2574f96640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f25700c21d0 0x7f25700aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49234/0 (socket says 172.21.15.39:49234) 2024-09-13T17:00:12.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2575797640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f257014b850 0x7f257014bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:43546/0 (socket says 172.21.15.39:43546) 2024-09-13T17:00:12.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2574f96640 1 -- 172.21.15.39:0/152446831 learned_addr learned my addr 172.21.15.39:0/152446831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:12.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2574f96640 1 -- 172.21.15.39:0/152446831 >> v2:172.21.15.84:3300/0 conn(0x7f257014b850 msgr2=0x7f257014bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:12.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2574f96640 1 --2- 172.21.15.39:0/152446831 >> v2:172.21.15.84:3300/0 conn(0x7f257014b850 0x7f257014bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:12.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2574f96640 1 -- 172.21.15.39:0/152446831 >> v1:172.21.15.39:6789/0 conn(0x7f257006aac0 legacy=0x7f2570151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:12.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f256f7fe640 1 -- 172.21.15.39:0/152446831 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706070233 0 0) 0x7f25700c3ae0 con 0x7f257006aac0 2024-09-13T17:00:12.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2574f96640 1 -- 172.21.15.39:0/152446831 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25700abe10 con 0x7f25700c21d0 2024-09-13T17:00:12.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.082+0000 7f2575797640 1 --2- 172.21.15.39:0/152446831 >> v2:172.21.15.84:3300/0 conn(0x7f257014b850 0x7f257014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:00:12.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f2574f96640 1 --2- 172.21.15.39:0/152446831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25700c21d0 0x7f25700aae00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f257006de50 tx=0x7f255c01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e1e6b2605da65 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:12.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f256f7fe640 1 -- 172.21.15.39:0/152446831 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c00d0d0 con 0x7f25700c21d0 2024-09-13T17:00:12.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f256f7fe640 1 -- 172.21.15.39:0/152446831 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c00d230 con 0x7f25700c21d0 2024-09-13T17:00:12.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f256f7fe640 1 -- 172.21.15.39:0/152446831 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c00b670 con 0x7f25700c21d0 2024-09-13T17:00:12.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f25771fa640 1 -- 172.21.15.39:0/152446831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25700c21d0 msgr2=0x7f25700aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:12.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f25771fa640 1 --2- 172.21.15.39:0/152446831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25700c21d0 0x7f25700aae00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f257006de50 tx=0x7f255c01c370 comp rx=0 tx=0).stop 2024-09-13T17:00:12.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f25771fa640 1 -- 172.21.15.39:0/152446831 shutdown_connections 2024-09-13T17:00:12.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f25771fa640 1 --2- 172.21.15.39:0/152446831 >> v2:172.21.15.84:3300/0 conn(0x7f257014b850 0x7f257014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:12.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f25771fa640 1 --2- 172.21.15.39:0/152446831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25700c21d0 0x7f25700aae00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:12.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f25771fa640 1 -- 172.21.15.39:0/152446831 >> 172.21.15.39:0/152446831 conn(0x7f2570069e10 msgr2=0x7f257006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:12.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f25771fa640 1 -- 172.21.15.39:0/152446831 shutdown_connections 2024-09-13T17:00:12.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.086+0000 7f25771fa640 1 -- 172.21.15.39:0/152446831 wait complete. 2024-09-13T17:00:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 Processor -- start 2024-09-13T17:00:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 -- start start 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f257006aac0 0x7f25701eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25700c21d0 0x7f25701e9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f257014b850 0x7f25701e9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2570148c70 con 0x7f257006aac0 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25701534b0 con 0x7f25700c21d0 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25701531d0 con 0x7f257014b850 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2574f96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f257006aac0 0x7f25701eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2574f96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f257006aac0 0x7f25701eb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49244/0 (socket says 172.21.15.39:49244) 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2574f96640 1 -- 172.21.15.39:0/739069682 learned_addr learned my addr 172.21.15.39:0/739069682 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2575797640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f257014b850 0x7f25701e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f256ffff640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25700c21d0 0x7f25701e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2574f96640 1 -- 172.21.15.39:0/739069682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f257014b850 msgr2=0x7f25701e9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2574f96640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f257014b850 0x7f25701e9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2574f96640 1 -- 172.21.15.39:0/739069682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25700c21d0 msgr2=0x7f25701e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2574f96640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25700c21d0 0x7f25701e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2574f96640 1 -- 172.21.15.39:0/739069682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2568009a70 con 0x7f257006aac0 2024-09-13T17:00:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f256ffff640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25700c21d0 0x7f25701e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:00:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f2574f96640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f257006aac0 0x7f25701eb0c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f255c0ae040 tx=0x7f255c019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f256d7fa640 1 -- 172.21.15.39:0/739069682 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c00d0d0 con 0x7f257006aac0 2024-09-13T17:00:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f256d7fa640 1 -- 172.21.15.39:0/739069682 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c00be10 con 0x7f257006aac0 2024-09-13T17:00:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c00d970 con 0x7f257006aac0 2024-09-13T17:00:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f256d7fa640 1 -- 172.21.15.39:0/739069682 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c006ee0 con 0x7f257006aac0 2024-09-13T17:00:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25701ebc80 con 0x7f257006aac0 2024-09-13T17:00:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.090+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25701e8d60 con 0x7f257006aac0 2024-09-13T17:00:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.094+0000 7f256d7fa640 1 -- 172.21.15.39:0/739069682 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f255c0b2040 con 0x7f257006aac0 2024-09-13T17:00:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.094+0000 7f256d7fa640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25540666c0 0x7f2554068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.094+0000 7f256d7fa640 1 -- 172.21.15.39:0/739069682 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f255c11e950 con 0x7f257006aac0 2024-09-13T17:00:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.094+0000 7f256ffff640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25540666c0 0x7f2554068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.094+0000 7f256ffff640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25540666c0 0x7f2554068b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2570156310 tx=0x7f2564009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.098+0000 7f256d7fa640 1 -- 172.21.15.39:0/739069682 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f25701e8d60 con 0x7f257006aac0 2024-09-13T17:00:12.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.314+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25701e8d60 con 0x7f257006aac0 2024-09-13T17:00:12.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f256d7fa640 1 -- 172.21.15.39:0/739069682 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5175 (secure 0 0 0) 0x7f255c0f2450 con 0x7f257006aac0 2024-09-13T17:00:12.315 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:00:12.316 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:00:12.316 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 89 2024-09-13T17:00:12.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25540666c0 msgr2=0x7f2554068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25540666c0 0x7f2554068b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2570156310 tx=0x7f2564009040 comp rx=0 tx=0).stop 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f257006aac0 msgr2=0x7f25701eb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f257006aac0 0x7f25701eb0c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f255c0ae040 tx=0x7f255c019110 comp rx=0 tx=0).stop 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 shutdown_connections 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25540666c0 0x7f2554068b80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25700c21d0 0x7f25701e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f257014b850 0x7f25701e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 --2- 172.21.15.39:0/739069682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f257006aac0 0x7f25701eb0c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 >> 172.21.15.39:0/739069682 conn(0x7f2570069e10 msgr2=0x7f257006d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 shutdown_connections 2024-09-13T17:00:12.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:12.318+0000 7f25771fa640 1 -- 172.21.15.39:0/739069682 wait complete. 2024-09-13T17:00:12.345 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-09-13T17:00:14.346 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:00:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.486+0000 7fca7ce50640 1 Processor -- start 2024-09-13T17:00:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.486+0000 7fca7ce50640 1 -- start start 2024-09-13T17:00:14.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca7ce50640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 0x7fca78156130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:14.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca7ce50640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fca780c6000 0x7fca780c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:14.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca7ce50640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca780c5e50 con 0x7fca780c7960 2024-09-13T17:00:14.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca7ce50640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7806ff50 con 0x7fca780c6000 2024-09-13T17:00:14.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca7ce50640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca78072000 con 0x7fca7806fa50 2024-09-13T17:00:14.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca76d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fca780c6000 0x7fca780c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:14.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca76d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fca780c6000 0x7fca780c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49256/0 (socket says 172.21.15.39:49256) 2024-09-13T17:00:14.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca76d9d640 1 -- 172.21.15.39:0/1654382034 learned_addr learned my addr 172.21.15.39:0/1654382034 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:14.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca75d9b640 1 --2- 172.21.15.39:0/1654382034 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 0x7fca78156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:14.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca76d9d640 1 -- 172.21.15.39:0/1654382034 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 msgr2=0x7fca78156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:00:14.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca76d9d640 1 --2- 172.21.15.39:0/1654382034 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 0x7fca78156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca76d9d640 1 -- 172.21.15.39:0/1654382034 >> v1:172.21.15.84:6789/0 conn(0x7fca780c7960 legacy=0x7fca78155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca76d9d640 1 -- 172.21.15.39:0/1654382034 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca780722e0 con 0x7fca780c6000 2024-09-13T17:00:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.490+0000 7fca75d9b640 1 --2- 172.21.15.39:0/1654382034 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 0x7fca78156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:00:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca76d9d640 1 --2- 172.21.15.39:0/1654382034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 0x7fca780c2570 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fca70009e50 tx=0x7fca7001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bb399d949cfa4b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7559a640 1 -- 172.21.15.39:0/1654382034 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca700a5850 con 0x7fca780c6000 2024-09-13T17:00:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7559a640 1 -- 172.21.15.39:0/1654382034 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca700a59b0 con 0x7fca780c6000 2024-09-13T17:00:14.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7559a640 1 -- 172.21.15.39:0/1654382034 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca700a5d40 con 0x7fca780c6000 2024-09-13T17:00:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 -- 172.21.15.39:0/1654382034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 msgr2=0x7fca780c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 --2- 172.21.15.39:0/1654382034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 0x7fca780c2570 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fca70009e50 tx=0x7fca7001c970 comp rx=0 tx=0).stop 2024-09-13T17:00:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 -- 172.21.15.39:0/1654382034 shutdown_connections 2024-09-13T17:00:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 --2- 172.21.15.39:0/1654382034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 0x7fca780c2570 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 --2- 172.21.15.39:0/1654382034 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 0x7fca78156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 -- 172.21.15.39:0/1654382034 >> 172.21.15.39:0/1654382034 conn(0x7fca7806c020 msgr2=0x7fca7806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 -- 172.21.15.39:0/1654382034 shutdown_connections 2024-09-13T17:00:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 -- 172.21.15.39:0/1654382034 wait complete. 2024-09-13T17:00:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 Processor -- start 2024-09-13T17:00:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 -- start start 2024-09-13T17:00:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 0x7fca78155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:14.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 0x7fca780c0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca780c7960 0x7fca781ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca781ef570 con 0x7fca780c6000 2024-09-13T17:00:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7806b3d0 con 0x7fca780c7960 2024-09-13T17:00:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7ce50640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7806b6b0 con 0x7fca7806fa50 2024-09-13T17:00:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca75d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 0x7fca780c0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca75d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 0x7fca780c0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49262/0 (socket says 172.21.15.39:49262) 2024-09-13T17:00:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca75d9b640 1 -- 172.21.15.39:0/3366026963 learned_addr learned my addr 172.21.15.39:0/3366026963 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:00:14.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca76d9d640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca780c7960 0x7fca781ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:14.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca7659c640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 0x7fca78155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:14.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca75d9b640 1 -- 172.21.15.39:0/3366026963 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 msgr2=0x7fca78155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:14.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca75d9b640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 0x7fca78155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca75d9b640 1 -- 172.21.15.39:0/3366026963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca780c7960 msgr2=0x7fca781ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:14.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca75d9b640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca780c7960 0x7fca781ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.494+0000 7fca75d9b640 1 -- 172.21.15.39:0/3366026963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca70009b30 con 0x7fca780c6000 2024-09-13T17:00:14.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca75d9b640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 0x7fca780c0ee0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00e810 tx=0x7fca6c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:14.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca66ffd640 1 -- 172.21.15.39:0/3366026963 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00c8d0 con 0x7fca780c6000 2024-09-13T17:00:14.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca66ffd640 1 -- 172.21.15.39:0/3366026963 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c00ca30 con 0x7fca780c6000 2024-09-13T17:00:14.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7806b990 con 0x7fca780c6000 2024-09-13T17:00:14.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca66ffd640 1 -- 172.21.15.39:0/3366026963 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00cdc0 con 0x7fca780c6000 2024-09-13T17:00:14.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca781ec000 con 0x7fca780c6000 2024-09-13T17:00:14.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca781ef6e0 con 0x7fca780c6000 2024-09-13T17:00:14.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca66ffd640 1 -- 172.21.15.39:0/3366026963 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fca6c013410 con 0x7fca780c6000 2024-09-13T17:00:14.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca66ffd640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fca58066690 0x7fca58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:00:14.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca66ffd640 1 -- 172.21.15.39:0/3366026963 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7fca6c084800 con 0x7fca780c6000 2024-09-13T17:00:14.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca7659c640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fca58066690 0x7fca58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:00:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.498+0000 7fca7659c640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fca58066690 0x7fca58068b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fca68000f70 tx=0x7fca68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:00:14.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.502+0000 7fca66ffd640 1 -- 172.21.15.39:0/3366026963 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fca781ef6e0 con 0x7fca780c6000 2024-09-13T17:00:14.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca781ef6b0 con 0x7fca780c6000 2024-09-13T17:00:14.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca66ffd640 1 -- 172.21.15.39:0/3366026963 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7fca6c051370 con 0x7fca780c6000 2024-09-13T17:00:14.724 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:00:14.725 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:00:14.725 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:00:14.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fca58066690 msgr2=0x7fca58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:14.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fca58066690 0x7fca58068b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fca68000f70 tx=0x7fca68009040 comp rx=0 tx=0).stop 2024-09-13T17:00:14.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 msgr2=0x7fca780c0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:00:14.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 0x7fca780c0ee0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00e810 tx=0x7fca6c00ecd0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 shutdown_connections 2024-09-13T17:00:14.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fca58066690 0x7fca58068b50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fca780c7960 0x7fca781ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca780c6000 0x7fca780c0ee0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 --2- 172.21.15.39:0/3366026963 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fca7806fa50 0x7fca78155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:00:14.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 >> 172.21.15.39:0/3366026963 conn(0x7fca7806c020 msgr2=0x7fca780bf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:00:14.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 shutdown_connections 2024-09-13T17:00:14.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:00:14.726+0000 7fca7ce50640 1 -- 172.21.15.39:0/3366026963 wait complete. 2024-09-13T17:00:14.744 INFO:tasks.mds_thrash.fs.[b]:mds.d reported in up:standby state 2024-09-13T17:00:14.744 INFO:tasks.mds_thrash.fs.[b]:waiting for 55 secs before thrashing 2024-09-13T17:00:28.056 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:00:28.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:00:58.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:00:58.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:01:09.741 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:09.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f864703f640 1 Processor -- start 2024-09-13T17:01:09.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f864703f640 1 -- start start 2024-09-13T17:01:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f864703f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86400c2db0 0x7f86400bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f864703f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f864006cbc0 0x7f86401512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f864703f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f86401517f0 0x7f86401545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f864703f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86400c00b0 con 0x7f864006cbc0 2024-09-13T17:01:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f864703f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8640153e70 con 0x7f86401517f0 2024-09-13T17:01:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f864703f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86400c0300 con 0x7f86400c2db0 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f863ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f864006cbc0 0x7f86401512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f863ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f864006cbc0 0x7f86401512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34306/0 (socket says 172.21.15.39:34306) 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f863ffff640 1 -- 172.21.15.39:0/1349687619 learned_addr learned my addr 172.21.15.39:0/1349687619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f8644ddb640 1 --2- 172.21.15.39:0/1349687619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86400c2db0 0x7f86400bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f86455dc640 1 --2- 172.21.15.39:0/1349687619 >> v2:172.21.15.84:3300/0 conn(0x7f86401517f0 0x7f86401545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f863ffff640 1 -- 172.21.15.39:0/1349687619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86400c2db0 msgr2=0x7f86400bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f863ffff640 1 --2- 172.21.15.39:0/1349687619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86400c2db0 0x7f86400bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f863ffff640 1 -- 172.21.15.39:0/1349687619 >> v2:172.21.15.84:3300/0 conn(0x7f86401517f0 msgr2=0x7f86401545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f863ffff640 1 --2- 172.21.15.39:0/1349687619 >> v2:172.21.15.84:3300/0 conn(0x7f86401517f0 0x7f86401545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f863ffff640 1 -- 172.21.15.39:0/1349687619 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400c05e0 con 0x7f864006cbc0 2024-09-13T17:01:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f8644ddb640 1 --2- 172.21.15.39:0/1349687619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86400c2db0 0x7f86400bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:01:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.873+0000 7f86455dc640 1 --2- 172.21.15.39:0/1349687619 >> v2:172.21.15.84:3300/0 conn(0x7f86401517f0 0x7f86401545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f863ffff640 1 --2- 172.21.15.39:0/1349687619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 0x7f86401512b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f86400c6050 tx=0x7f862c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04e08e1e51cf95d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f863f7fe640 1 -- 172.21.15.39:0/1349687619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0ab600 con 0x7f864006cbc0 2024-09-13T17:01:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f863f7fe640 1 -- 172.21.15.39:0/1349687619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c0a7050 con 0x7f864006cbc0 2024-09-13T17:01:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f863f7fe640 1 -- 172.21.15.39:0/1349687619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0ab980 con 0x7f864006cbc0 2024-09-13T17:01:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f864703f640 1 -- 172.21.15.39:0/1349687619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 msgr2=0x7f86401512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f864703f640 1 --2- 172.21.15.39:0/1349687619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 0x7f86401512b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f86400c6050 tx=0x7f862c01c400 comp rx=0 tx=0).stop 2024-09-13T17:01:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f864703f640 1 -- 172.21.15.39:0/1349687619 shutdown_connections 2024-09-13T17:01:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f864703f640 1 --2- 172.21.15.39:0/1349687619 >> v2:172.21.15.84:3300/0 conn(0x7f86401517f0 0x7f86401545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f864703f640 1 --2- 172.21.15.39:0/1349687619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 0x7f86401512b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f864703f640 1 --2- 172.21.15.39:0/1349687619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86400c2db0 0x7f86400bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f864703f640 1 -- 172.21.15.39:0/1349687619 >> 172.21.15.39:0/1349687619 conn(0x7f8640069e10 msgr2=0x7f864006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f864703f640 1 -- 172.21.15.39:0/1349687619 shutdown_connections 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.877+0000 7f864703f640 1 -- 172.21.15.39:0/1349687619 wait complete. 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 Processor -- start 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 -- start start 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 0x7f86401446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400c2db0 0x7f86401f0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86401517f0 0x7f86401f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8640150ad0 con 0x7f864006cbc0 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864014f8d0 con 0x7f86400c2db0 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864014fbb0 con 0x7f86401517f0 2024-09-13T17:01:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f8644ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 0x7f86401446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f8644ddb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 0x7f86401446f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34308/0 (socket says 172.21.15.39:34308) 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f8644ddb640 1 -- 172.21.15.39:0/3567161536 learned_addr learned my addr 172.21.15.39:0/3567161536 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f863ffff640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400c2db0 0x7f86401f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f86455dc640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86401517f0 0x7f86401f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f8644ddb640 1 -- 172.21.15.39:0/3567161536 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86401517f0 msgr2=0x7f86401f0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f8644ddb640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86401517f0 0x7f86401f0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f8644ddb640 1 -- 172.21.15.39:0/3567161536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400c2db0 msgr2=0x7f86401f0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f8644ddb640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400c2db0 0x7f86401f0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f8644ddb640 1 -- 172.21.15.39:0/3567161536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0094d0 con 0x7f864006cbc0 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f8644ddb640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 0x7f86401446f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f863400e810 tx=0x7f863400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f863d7fa640 1 -- 172.21.15.39:0/3567161536 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634004030 con 0x7f864006cbc0 2024-09-13T17:01:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f863d7fa640 1 -- 172.21.15.39:0/3567161536 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8634004190 con 0x7f864006cbc0 2024-09-13T17:01:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86401500f0 con 0x7f864006cbc0 2024-09-13T17:01:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f863d7fa640 1 -- 172.21.15.39:0/3567161536 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634004520 con 0x7f864006cbc0 2024-09-13T17:01:09.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.881+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8640150760 con 0x7f864006cbc0 2024-09-13T17:01:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.885+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864015e250 con 0x7f864006cbc0 2024-09-13T17:01:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.885+0000 7f863d7fa640 1 -- 172.21.15.39:0/3567161536 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f863400c6d0 con 0x7f864006cbc0 2024-09-13T17:01:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.885+0000 7f863d7fa640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8624066690 0x7f8624068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.885+0000 7f863d7fa640 1 -- 172.21.15.39:0/3567161536 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f863408b350 con 0x7f864006cbc0 2024-09-13T17:01:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.885+0000 7f863ffff640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8624066690 0x7f8624068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.885+0000 7f863ffff640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8624066690 0x7f8624068b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f862c0ad000 tx=0x7f862c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:09.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.889+0000 7f863d7fa640 1 -- 172.21.15.39:0/3567161536 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f863404adb0 con 0x7f864006cbc0 2024-09-13T17:01:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf3a0a6640 1 Processor -- start 2024-09-13T17:01:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf3a0a6640 1 -- start start 2024-09-13T17:01:09.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf3a0a6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdf340c6c20 0x7fdf340c7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf3a0a6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3415b9f0 0x7fdf3415be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf3a0a6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf34159070 con 0x7fdf34155b60 2024-09-13T17:01:09.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf3a0a6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf340c63d0 con 0x7fdf340c6c20 2024-09-13T17:01:09.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf3a0a6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34158480 con 0x7fdf3415b9f0 2024-09-13T17:01:09.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf2bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdf340c6c20 0x7fdf340c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf2bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdf340c6c20 0x7fdf340c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34310/0 (socket says 172.21.15.39:34310) 2024-09-13T17:01:09.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf2bfff640 1 -- 172.21.15.39:0/4048091831 learned_addr learned my addr 172.21.15.39:0/4048091831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:09.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf33fff640 1 --2- 172.21.15.39:0/4048091831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3415b9f0 0x7fdf3415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf2bfff640 1 -- 172.21.15.39:0/4048091831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3415b9f0 msgr2=0x7fdf3415be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:09.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf2bfff640 1 --2- 172.21.15.39:0/4048091831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3415b9f0 0x7fdf3415be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf2bfff640 1 -- 172.21.15.39:0/4048091831 >> v1:172.21.15.84:6789/0 conn(0x7fdf34155b60 legacy=0x7fdf340a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.921+0000 7fdf2bfff640 1 -- 172.21.15.39:0/4048091831 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34158740 con 0x7fdf340c6c20 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf2bfff640 1 --2- 172.21.15.39:0/4048091831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf340c6c20 0x7fdf340c7060 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009870 tx=0x7fdf1c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378f7accfb51228c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf32ffd640 1 -- 172.21.15.39:0/4048091831 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0ab600 con 0x7fdf340c6c20 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf32ffd640 1 -- 172.21.15.39:0/4048091831 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0a7050 con 0x7fdf340c6c20 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf32ffd640 1 -- 172.21.15.39:0/4048091831 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0ab980 con 0x7fdf340c6c20 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/4048091831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf340c6c20 msgr2=0x7fdf340c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 --2- 172.21.15.39:0/4048091831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf340c6c20 0x7fdf340c7060 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009870 tx=0x7fdf1c01c440 comp rx=0 tx=0).stop 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/4048091831 shutdown_connections 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 --2- 172.21.15.39:0/4048091831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3415b9f0 0x7fdf3415be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 --2- 172.21.15.39:0/4048091831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf340c6c20 0x7fdf340c7060 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/4048091831 >> 172.21.15.39:0/4048091831 conn(0x7fdf3406c020 msgr2=0x7fdf3406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:09.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/4048091831 shutdown_connections 2024-09-13T17:01:09.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/4048091831 wait complete. 2024-09-13T17:01:09.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 Processor -- start 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.925+0000 7fdf3a0a6640 1 -- start start 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf3a0a6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf340c6c20 0x7fdf3406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf3a0a6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf34155b60 0x7fdf340a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf3a0a6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf3415b9f0 0x7fdf340a7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf3a0a6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3406af80 con 0x7fdf3415b9f0 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf3a0a6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340be830 con 0x7fdf340c6c20 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf3a0a6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340be5e0 con 0x7fdf34155b60 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf33fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf3415b9f0 0x7fdf340a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf33fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf3415b9f0 0x7fdf340a7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34326/0 (socket says 172.21.15.39:34326) 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf33fff640 1 -- 172.21.15.39:0/2085251784 learned_addr learned my addr 172.21.15.39:0/2085251784 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:09.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf2bfff640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf34155b60 0x7fdf340a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf337fe640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf340c6c20 0x7fdf3406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf33fff640 1 -- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf34155b60 msgr2=0x7fdf340a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf33fff640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf34155b60 0x7fdf340a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf33fff640 1 -- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf340c6c20 msgr2=0x7fdf3406ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf33fff640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf340c6c20 0x7fdf3406ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf33fff640 1 -- 172.21.15.39:0/2085251784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0094d0 con 0x7fdf3415b9f0 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf2bfff640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf34155b60 0x7fdf340a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf337fe640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf340c6c20 0x7fdf3406ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf33fff640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf3415b9f0 0x7fdf340a7990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00efc0 tx=0x7fdf2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf30ff9640 1 -- 172.21.15.39:0/2085251784 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c009ba0 con 0x7fdf3415b9f0 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf30ff9640 1 -- 172.21.15.39:0/2085251784 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c00f040 con 0x7fdf3415b9f0 2024-09-13T17:01:09.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf340bda20 con 0x7fdf3415b9f0 2024-09-13T17:01:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf30ff9640 1 -- 172.21.15.39:0/2085251784 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0146c0 con 0x7fdf3415b9f0 2024-09-13T17:01:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf340bdd00 con 0x7fdf3415b9f0 2024-09-13T17:01:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf18005260 con 0x7fdf3415b9f0 2024-09-13T17:01:09.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf30ff9640 1 -- 172.21.15.39:0/2085251784 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf2c014820 con 0x7fdf3415b9f0 2024-09-13T17:01:09.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf30ff9640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf14066690 0x7fdf14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:09.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf30ff9640 1 -- 172.21.15.39:0/2085251784 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7fdf2c085780 con 0x7fdf3415b9f0 2024-09-13T17:01:09.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.929+0000 7fdf337fe640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf14066690 0x7fdf14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:09.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.933+0000 7fdf337fe640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf14066690 0x7fdf14068b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf3406f500 tx=0x7fdf24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:09.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:09.937+0000 7fdf30ff9640 1 -- 172.21.15.39:0/2085251784 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf2c0522d0 con 0x7fdf3415b9f0 2024-09-13T17:01:10.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8640158220 con 0x7f864006cbc0 2024-09-13T17:01:10.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f863d7fa640 1 -- 172.21.15.39:0/3567161536 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f8634088060 con 0x7f864006cbc0 2024-09-13T17:01:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:10.128 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:10.128 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8624066690 msgr2=0x7f8624068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8624066690 0x7f8624068b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f862c0ad000 tx=0x7f862c008040 comp rx=0 tx=0).stop 2024-09-13T17:01:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 msgr2=0x7f86401446f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 0x7f86401446f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f863400e810 tx=0x7f863400ecd0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 shutdown_connections 2024-09-13T17:01:10.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8624066690 0x7f8624068b50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400c2db0 0x7f86401f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f86401517f0 0x7f86401f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 --2- 172.21.15.39:0/3567161536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f864006cbc0 0x7f86401446f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 >> 172.21.15.39:0/3567161536 conn(0x7f8640069e10 msgr2=0x7f86400c09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:10.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 shutdown_connections 2024-09-13T17:01:10.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.129+0000 7f864703f640 1 -- 172.21.15.39:0/3567161536 wait complete. 2024-09-13T17:01:10.150 INFO:tasks.mds_thrash.fs.[a]:kill mds.a (rank=0) 2024-09-13T17:01:10.151 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:01:10.151 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:01:10.161 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:01:10.161 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.197+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf180050d0 con 0x7fdf3415b9f0 2024-09-13T17:01:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf30ff9640 1 -- 172.21.15.39:0/2085251784 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7fdf2c018090 con 0x7fdf3415b9f0 2024-09-13T17:01:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:10.199 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf14066690 msgr2=0x7fdf14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf14066690 0x7fdf14068b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf3406f500 tx=0x7fdf24009040 comp rx=0 tx=0).stop 2024-09-13T17:01:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf3415b9f0 msgr2=0x7fdf340a7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf3415b9f0 0x7fdf340a7990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00efc0 tx=0x7fdf2c00c490 comp rx=0 tx=0).stop 2024-09-13T17:01:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 shutdown_connections 2024-09-13T17:01:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf14066690 0x7fdf14068b50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf3415b9f0 0x7fdf340a7990 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf34155b60 0x7fdf340a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 --2- 172.21.15.39:0/2085251784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf340c6c20 0x7fdf3406ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 >> 172.21.15.39:0/2085251784 conn(0x7fdf3406c020 msgr2=0x7fdf3406c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 shutdown_connections 2024-09-13T17:01:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.201+0000 7fdf3a0a6640 1 -- 172.21.15.39:0/2085251784 wait complete. 2024-09-13T17:01:10.223 INFO:tasks.mds_thrash.fs.[b]:kill mds.b (rank=0) 2024-09-13T17:01:10.224 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:01:10.224 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:01:10.237 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:01:10.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc40d2640 1 Processor -- start 2024-09-13T17:01:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc40d2640 1 -- start start 2024-09-13T17:01:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc40d2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2bbc151500 0x7f2bbc0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc40d2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc0c5a30 con 0x7f2bbc0c47c0 2024-09-13T17:01:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc40d2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc1576a0 con 0x7f2bbc071dd0 2024-09-13T17:01:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc40d2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc157980 con 0x7f2bbc151500 2024-09-13T17:01:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc1e6e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2bbc151500 0x7f2bbc0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc266f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2bbc0c47c0 0x7f2bbc154780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40986/0 (socket says 172.21.15.39:40986) 2024-09-13T17:01:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc266f640 1 -- 172.21.15.39:0/1061360186 learned_addr learned my addr 172.21.15.39:0/1061360186 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc0e6c640 1 -- 172.21.15.39:0/1061360186 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527775132 0 0) 0x7f2bbc0c5a30 con 0x7f2bbc0c47c0 2024-09-13T17:01:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc0e6c640 1 -- 172.21.15.39:0/1061360186 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba8003530 con 0x7f2bbc0c47c0 2024-09-13T17:01:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc1e6e640 1 -- 172.21.15.39:0/1061360186 >> v1:172.21.15.84:6789/0 conn(0x7f2bbc071dd0 legacy=0x7f2bbc154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc0e6c640 1 -- 172.21.15.39:0/1061360186 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348083072 0 0) 0x7f2bbc1576a0 con 0x7f2bbc071dd0 2024-09-13T17:01:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc1e6e640 1 -- 172.21.15.39:0/1061360186 >> v1:172.21.15.39:6789/0 conn(0x7f2bbc0c47c0 legacy=0x7f2bbc154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.329+0000 7f2bc1e6e640 1 -- 172.21.15.39:0/1061360186 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc157c60 con 0x7f2bbc151500 2024-09-13T17:01:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc1e6e640 1 --2- 172.21.15.39:0/1061360186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 0x7f2bbc0c4280 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80097f0 tx=0x7f2bb801ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f159348c6b598a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc0e6c640 1 -- 172.21.15.39:0/1061360186 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80ab600 con 0x7f2bbc151500 2024-09-13T17:01:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc0e6c640 1 -- 172.21.15.39:0/1061360186 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb80ab760 con 0x7f2bbc151500 2024-09-13T17:01:10.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc0e6c640 1 -- 172.21.15.39:0/1061360186 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80abaf0 con 0x7f2bbc151500 2024-09-13T17:01:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1061360186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 msgr2=0x7f2bbc0c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc40d2640 1 --2- 172.21.15.39:0/1061360186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 0x7f2bbc0c4280 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80097f0 tx=0x7f2bb801ce40 comp rx=0 tx=0).stop 2024-09-13T17:01:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1061360186 shutdown_connections 2024-09-13T17:01:10.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc40d2640 1 --2- 172.21.15.39:0/1061360186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 0x7f2bbc0c4280 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1061360186 >> 172.21.15.39:0/1061360186 conn(0x7f2bbc0ab530 msgr2=0x7f2bbc0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1061360186 shutdown_connections 2024-09-13T17:01:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.333+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1061360186 wait complete. 2024-09-13T17:01:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 Processor -- start 2024-09-13T17:01:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 -- start start 2024-09-13T17:01:10.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2bbc071dd0 0x7f2bbc1ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bbc0c47c0 0x7f2bbc153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 0x7f2bbc1ec960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc1ec180 con 0x7f2bbc151500 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc1567f0 con 0x7f2bbc0c47c0 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc156ad0 con 0x7f2bbc071dd0 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc266f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 0x7f2bbc1ec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc266f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 0x7f2bbc1ec960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43080/0 (socket says 172.21.15.39:43080) 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc266f640 1 -- 172.21.15.39:0/1661891351 learned_addr learned my addr 172.21.15.39:0/1661891351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc166d640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bbc0c47c0 0x7f2bbc153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc1e6e640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2bbc071dd0 0x7f2bbc1ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc266f640 1 -- 172.21.15.39:0/1661891351 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2bbc071dd0 msgr2=0x7f2bbc1ef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc266f640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2bbc071dd0 0x7f2bbc1ef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc266f640 1 -- 172.21.15.39:0/1661891351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bbc0c47c0 msgr2=0x7f2bbc153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc266f640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bbc0c47c0 0x7f2bbc153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc266f640 1 -- 172.21.15.39:0/1661891351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8009420 con 0x7f2bbc151500 2024-09-13T17:01:10.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc266f640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 0x7f2bbc1ec960 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400cb30 tx=0x7f2bb400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:10.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bb27fc640 1 -- 172.21.15.39:0/1661891351 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400aca0 con 0x7f2bbc151500 2024-09-13T17:01:10.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bb27fc640 1 -- 172.21.15.39:0/1661891351 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb400ae00 con 0x7f2bbc151500 2024-09-13T17:01:10.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc156db0 con 0x7f2bbc151500 2024-09-13T17:01:10.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bb27fc640 1 -- 172.21.15.39:0/1661891351 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4012600 con 0x7f2bbc151500 2024-09-13T17:01:10.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc1520f0 con 0x7f2bbc151500 2024-09-13T17:01:10.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc156ff0 con 0x7f2bbc151500 2024-09-13T17:01:10.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.337+0000 7f2bb27fc640 1 -- 172.21.15.39:0/1661891351 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2bb4012820 con 0x7f2bbc151500 2024-09-13T17:01:10.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.341+0000 7f2bb27fc640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2ba0066690 0x7f2ba0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.341+0000 7f2bb27fc640 1 -- 172.21.15.39:0/1661891351 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f2bb4084850 con 0x7f2bbc151500 2024-09-13T17:01:10.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.341+0000 7f2bc1e6e640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2ba0066690 0x7f2ba0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.341+0000 7f2bc1e6e640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2ba0066690 0x7f2ba0068b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80af000 tx=0x7f2bb8004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:10.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.345+0000 7f2bb27fc640 1 -- 172.21.15.39:0/1661891351 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2bb4051450 con 0x7f2bbc151500 2024-09-13T17:01:10.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2dbd676640 1 Processor -- start 2024-09-13T17:01:10.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2dbd676640 1 -- start start 2024-09-13T17:01:10.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2dbd676640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db8157910 0x7f2db80c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2dbd676640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db80c3d60 con 0x7f2db806f840 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2dbd676640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db806b960 con 0x7f2db8157530 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2dbd676640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db806be40 con 0x7f2db8157910 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db77fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2db806f840 0x7f2db80a6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40990/0 (socket says 172.21.15.39:40990) 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db77fe640 1 -- 172.21.15.39:0/3682451130 learned_addr learned my addr 172.21.15.39:0/3682451130 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db67fc640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db8157910 0x7f2db80c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15881399 0 0) 0x7f2db80c3d60 con 0x7f2db806f840 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da4003530 con 0x7f2db806f840 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720845146 0 0) 0x7f2db806b960 con 0x7f2db8157530 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db80c3d60 con 0x7f2db8157530 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1111521505 0 0) 0x7f2da4003530 con 0x7f2db806f840 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db8157910 msgr2=0x7f2db80c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:10.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db8157910 0x7f2db80c2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 >> v1:172.21.15.84:6789/0 conn(0x7f2db8157530 legacy=0x7f2db8071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db80c4000 con 0x7f2db806f840 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f2da8001a00 con 0x7f2db806f840 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 >> v1:172.21.15.39:6789/0 conn(0x7f2db806f840 legacy=0x7f2db80a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da4003b40 0x7f2da4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db8157910 0x7f2da40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2da4004d10 0x7f2da4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db806be40 con 0x7f2da4004d10 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da400ba40 con 0x7f2da4003b40 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da400bbb0 con 0x7f2db8157910 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db77fe640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2da4004d10 0x7f2da4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db67fc640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db8157910 0x7f2da40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db6ffd640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da4003b40 0x7f2da4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db77fe640 1 -- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db8157910 msgr2=0x7f2da40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db77fe640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db8157910 0x7f2da40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db77fe640 1 -- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da4003b40 msgr2=0x7f2da4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db77fe640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da4003b40 0x7f2da4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.405+0000 7f2db77fe640 1 -- 172.21.15.39:0/3682451130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da4003530 con 0x7f2da4004d10 2024-09-13T17:01:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.409+0000 7f2db77fe640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2da4004d10 0x7f2da4005190 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2da800a7c0 tx=0x7f2da801d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ed613c9762cc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.409+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80a5ea0 con 0x7f2da4004d10 2024-09-13T17:01:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.409+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da80a8070 con 0x7f2da4004d10 2024-09-13T17:01:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.409+0000 7f2db5ffb640 1 -- 172.21.15.39:0/3682451130 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80ad890 con 0x7f2da4004d10 2024-09-13T17:01:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.409+0000 7f2dbd676640 1 -- 172.21.15.39:0/3682451130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2da4004d10 msgr2=0x7f2da4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.409+0000 7f2dbd676640 1 --2- 172.21.15.39:0/3682451130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2da4004d10 0x7f2da4005190 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2da800a7c0 tx=0x7f2da801d2c0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.409+0000 7f2db77fe640 1 -- 172.21.15.39:0/3682451130 reap_dead start 2024-09-13T17:01:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.409+0000 7f2dbd676640 1 -- 172.21.15.39:0/3682451130 shutdown_connections 2024-09-13T17:01:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 -- 172.21.15.39:0/3682451130 >> 172.21.15.39:0/3682451130 conn(0x7f2db80a97b0 msgr2=0x7f2db80a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 -- 172.21.15.39:0/3682451130 shutdown_connections 2024-09-13T17:01:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 -- 172.21.15.39:0/3682451130 wait complete. 2024-09-13T17:01:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 Processor -- start 2024-09-13T17:01:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 -- start start 2024-09-13T17:01:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da4004d10 0x7f2db806e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db814dd10 0x7f2db814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db81ec4b0 0x7f2db81ec930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8152810 con 0x7f2db81ec4b0 2024-09-13T17:01:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8151bd0 con 0x7f2da4004d10 2024-09-13T17:01:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db80acb90 con 0x7f2db814dd10 2024-09-13T17:01:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db81ec4b0 0x7f2db81ec930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db81ec4b0 0x7f2db81ec930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43090/0 (socket says 172.21.15.39:43090) 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db77fe640 1 -- 172.21.15.39:0/562376431 learned_addr learned my addr 172.21.15.39:0/562376431 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db6ffd640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da4004d10 0x7f2db806e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db67fc640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db814dd10 0x7f2db814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db77fe640 1 -- 172.21.15.39:0/562376431 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db814dd10 msgr2=0x7f2db814e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db77fe640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db814dd10 0x7f2db814e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db77fe640 1 -- 172.21.15.39:0/562376431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da4004d10 msgr2=0x7f2db806e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db77fe640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da4004d10 0x7f2db806e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db77fe640 1 -- 172.21.15.39:0/562376431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db80ace70 con 0x7f2db81ec4b0 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2db77fe640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db81ec4b0 0x7f2db81ec930 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2da801db60 tx=0x7f2da4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2daf7fe640 1 -- 172.21.15.39:0/562376431 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80a5ea0 con 0x7f2db81ec4b0 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2daf7fe640 1 -- 172.21.15.39:0/562376431 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da8001f60 con 0x7f2db81ec4b0 2024-09-13T17:01:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db8151e20 con 0x7f2db81ec4b0 2024-09-13T17:01:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2daf7fe640 1 -- 172.21.15.39:0/562376431 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8001a00 con 0x7f2db81ec4b0 2024-09-13T17:01:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.413+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db8152450 con 0x7f2db81ec4b0 2024-09-13T17:01:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.417+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db80c2400 con 0x7f2db81ec4b0 2024-09-13T17:01:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.417+0000 7f2daf7fe640 1 -- 172.21.15.39:0/562376431 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2da80aa030 con 0x7f2db81ec4b0 2024-09-13T17:01:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.417+0000 7f2daf7fe640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d980666c0 0x7f2d98068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.417+0000 7f2db6ffd640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d980666c0 0x7f2d98068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.417+0000 7f2daf7fe640 1 -- 172.21.15.39:0/562376431 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f2da811e7c0 con 0x7f2db81ec4b0 2024-09-13T17:01:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.417+0000 7f2db6ffd640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d980666c0 0x7f2d98068b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2db806f790 tx=0x7f2db0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:10.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.421+0000 7f2daf7fe640 1 -- 172.21.15.39:0/562376431 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2da80eb340 con 0x7f2db81ec4b0 2024-09-13T17:01:10.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.597+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bbc0c4280 con 0x7f2bbc151500 2024-09-13T17:01:10.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.597+0000 7f2bb27fc640 1 -- 172.21.15.39:0/1661891351 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f2bb4017090 con 0x7f2bbc151500 2024-09-13T17:01:10.598 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:10.598 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:10.598 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:10.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2ba0066690 msgr2=0x7f2ba0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2ba0066690 0x7f2ba0068b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80af000 tx=0x7f2bb8004740 comp rx=0 tx=0).stop 2024-09-13T17:01:10.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 msgr2=0x7f2bbc1ec960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 0x7f2bbc1ec960 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400cb30 tx=0x7f2bb400a060 comp rx=0 tx=0).stop 2024-09-13T17:01:10.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 shutdown_connections 2024-09-13T17:01:10.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2ba0066690 0x7f2ba0068b50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bbc0c47c0 0x7f2bbc153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bbc151500 0x7f2bbc1ec960 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 --2- 172.21.15.39:0/1661891351 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2bbc071dd0 0x7f2bbc1ef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 >> 172.21.15.39:0/1661891351 conn(0x7f2bbc0ab530 msgr2=0x7f2bbc146f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:10.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 shutdown_connections 2024-09-13T17:01:10.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.601+0000 7f2bc40d2640 1 -- 172.21.15.39:0/1661891351 wait complete. 2024-09-13T17:01:10.618 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:01:10.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.657+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2db80c2950 con 0x7f2db81ec4b0 2024-09-13T17:01:10.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.657+0000 7f2daf7fe640 1 -- 172.21.15.39:0/562376431 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f2da80ee960 con 0x7f2db81ec4b0 2024-09-13T17:01:10.658 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:10.658 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:10.658 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d980666c0 msgr2=0x7f2d98068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d980666c0 0x7f2d98068b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2db806f790 tx=0x7f2db0009040 comp rx=0 tx=0).stop 2024-09-13T17:01:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db81ec4b0 msgr2=0x7f2db81ec930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db81ec4b0 0x7f2db81ec930 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2da801db60 tx=0x7f2da4005800 comp rx=0 tx=0).stop 2024-09-13T17:01:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 shutdown_connections 2024-09-13T17:01:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d980666c0 0x7f2d98068b80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db81ec4b0 0x7f2db81ec930 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db814dd10 0x7f2db814e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 --2- 172.21.15.39:0/562376431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2da4004d10 0x7f2db806e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 >> 172.21.15.39:0/562376431 conn(0x7f2db80a97b0 msgr2=0x7f2db80c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 shutdown_connections 2024-09-13T17:01:10.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:10.661+0000 7f2dbd676640 1 -- 172.21.15.39:0/562376431 wait complete. 2024-09-13T17:01:10.677 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:01:12.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:12.678 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:12.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0868f54640 1 Processor -- start 2024-09-13T17:01:12.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0868f54640 1 -- start start 2024-09-13T17:01:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0868f54640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f08640c28d0 0x7f0864154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0868f54640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640c1280 0x7f08640c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0868f54640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08640c33c0 con 0x7f086406fba0 2024-09-13T17:01:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0868f54640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08640611c0 con 0x7f08640c28d0 2024-09-13T17:01:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0868f54640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08640c0ac0 con 0x7f08640c1280 2024-09-13T17:01:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0861d9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f086406fba0 0x7f08640c2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41004/0 (socket says 172.21.15.39:41004) 2024-09-13T17:01:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0861d9b640 1 -- 172.21.15.39:0/104761044 learned_addr learned my addr 172.21.15.39:0/104761044 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0862d9d640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640c1280 0x7f08640c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086259c640 1 --2- 172.21.15.39:0/104761044 >> v2:172.21.15.84:3300/0 conn(0x7f08640c28d0 0x7f0864154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176940550 0 0) 0x7f0858001120 con 0x7f086406fba0 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f084c003530 con 0x7f086406fba0 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1888163312 0 0) 0x7f084c003530 con 0x7f086406fba0 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640c1280 msgr2=0x7f08640c16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640c1280 0x7f08640c16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 >> v2:172.21.15.84:3300/0 conn(0x7f08640c28d0 msgr2=0x7f0864154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 --2- 172.21.15.39:0/104761044 >> v2:172.21.15.84:3300/0 conn(0x7f08640c28d0 0x7f0864154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640613d0 con 0x7f086406fba0 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f0858001830 con 0x7f086406fba0 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0862d9d640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640c1280 0x7f08640c16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 >> v1:172.21.15.39:6789/0 conn(0x7f086406fba0 legacy=0x7f08640c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f084c003980 0x7f084c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640c1280 0x7f084c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f084c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08640611c0 con 0x7f084c004d10 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08640c0ac0 con 0x7f084c003980 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084c00b990 con 0x7f08640c1280 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0862d9d640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f084c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086259c640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f084c003980 0x7f084c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0861d9b640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640c1280 0x7f084c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0862d9d640 1 -- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640c1280 msgr2=0x7f084c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0862d9d640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640c1280 0x7f084c0047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0862d9d640 1 -- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f084c003980 msgr2=0x7f084c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0862d9d640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f084c003980 0x7f084c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f0862d9d640 1 -- 172.21.15.39:0/104761044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c009a70 con 0x7f084c004d10 2024-09-13T17:01:12.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.753+0000 7f086259c640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f084c003980 0x7f084c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:12.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f0862d9d640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f084c005190 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f085c00e350 tx=0x7f085c01cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9b546c4d1a5a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:12.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c0a5e10 con 0x7f084c004d10 2024-09-13T17:01:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c00f040 con 0x7f084c004d10 2024-09-13T17:01:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f086159a640 1 -- 172.21.15.39:0/104761044 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c0ad640 con 0x7f084c004d10 2024-09-13T17:01:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f0868f54640 1 -- 172.21.15.39:0/104761044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 msgr2=0x7f084c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f0868f54640 1 --2- 172.21.15.39:0/104761044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f084c005190 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f085c00e350 tx=0x7f085c01cd60 comp rx=0 tx=0).stop 2024-09-13T17:01:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f0862d9d640 1 -- 172.21.15.39:0/104761044 reap_dead start 2024-09-13T17:01:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f0868f54640 1 -- 172.21.15.39:0/104761044 shutdown_connections 2024-09-13T17:01:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f0868f54640 1 -- 172.21.15.39:0/104761044 >> 172.21.15.39:0/104761044 conn(0x7f0864069e10 msgr2=0x7f086406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f0868f54640 1 -- 172.21.15.39:0/104761044 shutdown_connections 2024-09-13T17:01:12.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.757+0000 7f0868f54640 1 -- 172.21.15.39:0/104761044 wait complete. 2024-09-13T17:01:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 Processor -- start 2024-09-13T17:01:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 -- start start 2024-09-13T17:01:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f08640aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640a95e0 0x7f08641ed110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08641e9220 0x7f08640bf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0864152970 con 0x7f084c004d10 2024-09-13T17:01:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641ed650 con 0x7f08641e9220 2024-09-13T17:01:12.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086414d230 con 0x7f08640a95e0 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f086259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f08640aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f086259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f08640aa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43110/0 (socket says 172.21.15.39:43110) 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f086259c640 1 -- 172.21.15.39:0/3445981449 learned_addr learned my addr 172.21.15.39:0/3445981449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0861d9b640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640a95e0 0x7f08641ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0862d9d640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08641e9220 0x7f08640bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f086259c640 1 -- 172.21.15.39:0/3445981449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640a95e0 msgr2=0x7f08641ed110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f086259c640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640a95e0 0x7f08641ed110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f086259c640 1 -- 172.21.15.39:0/3445981449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08641e9220 msgr2=0x7f08640bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f086259c640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08641e9220 0x7f08640bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f086259c640 1 -- 172.21.15.39:0/3445981449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086414d510 con 0x7f084c004d10 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f086259c640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f08640aa7a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0854010950 tx=0x7f0854010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0852ffd640 1 -- 172.21.15.39:0/3445981449 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400ec20 con 0x7f084c004d10 2024-09-13T17:01:12.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0852ffd640 1 -- 172.21.15.39:0/3445981449 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085400ed80 con 0x7f084c004d10 2024-09-13T17:01:12.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08641ec910 con 0x7f084c004d10 2024-09-13T17:01:12.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0852ffd640 1 -- 172.21.15.39:0/3445981449 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854004220 con 0x7f084c004d10 2024-09-13T17:01:12.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08640bfc90 con 0x7f084c004d10 2024-09-13T17:01:12.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08640c1a70 con 0x7f084c004d10 2024-09-13T17:01:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0852ffd640 1 -- 172.21.15.39:0/3445981449 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0854004380 con 0x7f084c004d10 2024-09-13T17:01:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.761+0000 7f0852ffd640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f08440666c0 0x7f0844068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.765+0000 7f0852ffd640 1 -- 172.21.15.39:0/3445981449 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f08540849b0 con 0x7f084c004d10 2024-09-13T17:01:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.765+0000 7f0861d9b640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f08440666c0 0x7f0844068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.765+0000 7f0861d9b640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f08440666c0 0x7f0844068b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0858001960 tx=0x7f08580012a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:12.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.769+0000 7f0852ffd640 1 -- 172.21.15.39:0/3445981449 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0854051500 con 0x7f084c004d10 2024-09-13T17:01:12.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f7781c64640 1 Processor -- start 2024-09-13T17:01:12.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f7781c64640 1 -- start start 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f7781c64640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f777c0713f0 0x7f777c0717d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f7781c64640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c06fba0 0x7f777c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f7781c64640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f777c06d500 0x7f777c15b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f7781c64640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777c070290 con 0x7f777c06d500 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f7781c64640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777c072180 con 0x7f777c0713f0 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f7781c64640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c158bd0 con 0x7f777c06fba0 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f777c06d500 0x7f777c15b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f777c06d500 0x7f777c15b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43112/0 (socket says 172.21.15.39:43112) 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777bfff640 1 -- 172.21.15.39:0/3661794029 learned_addr learned my addr 172.21.15.39:0/3661794029 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777affd640 1 --2- 172.21.15.39:0/3661794029 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c06fba0 0x7f777c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777b7fe640 1 --2- 172.21.15.39:0/3661794029 >> v2:172.21.15.84:3300/0 conn(0x7f777c0713f0 0x7f777c0717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777bfff640 1 -- 172.21.15.39:0/3661794029 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c06fba0 msgr2=0x7f777c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:12.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777bfff640 1 --2- 172.21.15.39:0/3661794029 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c06fba0 0x7f777c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777bfff640 1 -- 172.21.15.39:0/3661794029 >> v2:172.21.15.84:3300/0 conn(0x7f777c0713f0 msgr2=0x7f777c0717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:12.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777bfff640 1 --2- 172.21.15.39:0/3661794029 >> v2:172.21.15.84:3300/0 conn(0x7f777c0713f0 0x7f777c0717d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.813+0000 7f777bfff640 1 -- 172.21.15.39:0/3661794029 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c061520 con 0x7f777c06d500 2024-09-13T17:01:12.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777bfff640 1 --2- 172.21.15.39:0/3661794029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 0x7f777c15b790 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f777c0c0cc0 tx=0x7f776c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79cd06751fbfdf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:12.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777a7fc640 1 -- 172.21.15.39:0/3661794029 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c0ab600 con 0x7f777c06d500 2024-09-13T17:01:12.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777a7fc640 1 -- 172.21.15.39:0/3661794029 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c0a7050 con 0x7f777c06d500 2024-09-13T17:01:12.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777a7fc640 1 -- 172.21.15.39:0/3661794029 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c0ab980 con 0x7f777c06d500 2024-09-13T17:01:12.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- 172.21.15.39:0/3661794029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 msgr2=0x7f777c15b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:12.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 --2- 172.21.15.39:0/3661794029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 0x7f777c15b790 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f777c0c0cc0 tx=0x7f776c01c520 comp rx=0 tx=0).stop 2024-09-13T17:01:12.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- 172.21.15.39:0/3661794029 shutdown_connections 2024-09-13T17:01:12.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 --2- 172.21.15.39:0/3661794029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 0x7f777c15b790 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 --2- 172.21.15.39:0/3661794029 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c06fba0 0x7f777c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 --2- 172.21.15.39:0/3661794029 >> v2:172.21.15.84:3300/0 conn(0x7f777c0713f0 0x7f777c0717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- 172.21.15.39:0/3661794029 >> 172.21.15.39:0/3661794029 conn(0x7f777c069e10 msgr2=0x7f777c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:12.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- 172.21.15.39:0/3661794029 shutdown_connections 2024-09-13T17:01:12.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- 172.21.15.39:0/3661794029 wait complete. 2024-09-13T17:01:12.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 Processor -- start 2024-09-13T17:01:12.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- start start 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 0x7f777c0a9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c06fba0 0x7f777c1572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c0713f0 0x7f777c1f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c1f4460 con 0x7f777c06d500 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c0c2ad0 con 0x7f777c06fba0 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c0c2db0 con 0x7f777c0713f0 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 0x7f777c0a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 0x7f777c0a9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43120/0 (socket says 172.21.15.39:43120) 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777b7fe640 1 -- 172.21.15.39:0/3876544267 learned_addr learned my addr 172.21.15.39:0/3876544267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777bfff640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c0713f0 0x7f777c1f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777affd640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c06fba0 0x7f777c1572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777b7fe640 1 -- 172.21.15.39:0/3876544267 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c0713f0 msgr2=0x7f777c1f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777b7fe640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c0713f0 0x7f777c1f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777b7fe640 1 -- 172.21.15.39:0/3876544267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c06fba0 msgr2=0x7f777c1572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777b7fe640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c06fba0 0x7f777c1572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777b7fe640 1 -- 172.21.15.39:0/3876544267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c0094d0 con 0x7f777c06d500 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777bfff640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c0713f0 0x7f777c1f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f777b7fe640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 0x7f777c0a9170 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f777400d5f0 tx=0x7f777400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7773fff640 1 -- 172.21.15.39:0/3876544267 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740125f0 con 0x7f777c06d500 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7773fff640 1 -- 172.21.15.39:0/3876544267 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7774012750 con 0x7f777c06d500 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777c0c3090 con 0x7f777c06d500 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7773fff640 1 -- 172.21.15.39:0/3876544267 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774012ae0 con 0x7f777c06d500 2024-09-13T17:01:12.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.817+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c1577e0 con 0x7f777c06d500 2024-09-13T17:01:12.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.821+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777c15b790 con 0x7f777c06d500 2024-09-13T17:01:12.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.821+0000 7f7773fff640 1 -- 172.21.15.39:0/3876544267 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7774012c40 con 0x7f777c06d500 2024-09-13T17:01:12.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.821+0000 7f7773fff640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f775c0666c0 0x7f775c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:12.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.821+0000 7f777affd640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f775c0666c0 0x7f775c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:12.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.821+0000 7f7773fff640 1 -- 172.21.15.39:0/3876544267 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f7774084e60 con 0x7f777c06d500 2024-09-13T17:01:12.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.821+0000 7f777affd640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f775c0666c0 0x7f775c068b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f777c0ac640 tx=0x7f776400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:12.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.825+0000 7f7773fff640 1 -- 172.21.15.39:0/3876544267 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f77740519b0 con 0x7f777c06d500 2024-09-13T17:01:12.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.965+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0864071800 con 0x7f084c004d10 2024-09-13T17:01:12.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.965+0000 7f0852ffd640 1 -- 172.21.15.39:0/3445981449 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f0854054b20 con 0x7f084c004d10 2024-09-13T17:01:12.965 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:12.965 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:12.965 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f08440666c0 msgr2=0x7f0844068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f08440666c0 0x7f0844068b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0858001960 tx=0x7f08580012a0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 msgr2=0x7f08640aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f08640aa7a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0854010950 tx=0x7f0854010e10 comp rx=0 tx=0).stop 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 shutdown_connections 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f08440666c0 0x7f0844068b80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f08641e9220 0x7f08640bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f08640a95e0 0x7f08641ed110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 --2- 172.21.15.39:0/3445981449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084c004d10 0x7f08640aa7a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 >> 172.21.15.39:0/3445981449 conn(0x7f0864069e10 msgr2=0x7f0864149a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 shutdown_connections 2024-09-13T17:01:12.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:12.969+0000 7f0868f54640 1 -- 172.21.15.39:0/3445981449 wait complete. 2024-09-13T17:01:12.983 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:01:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.025+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f777c0c3950 con 0x7f777c06d500 2024-09-13T17:01:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.025+0000 7f7773fff640 1 -- 172.21.15.39:0/3876544267 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f7774007380 con 0x7f777c06d500 2024-09-13T17:01:13.025 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:13.026 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f775c0666c0 msgr2=0x7f775c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f775c0666c0 0x7f775c068b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f777c0ac640 tx=0x7f776400b040 comp rx=0 tx=0).stop 2024-09-13T17:01:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 msgr2=0x7f777c0a9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 0x7f777c0a9170 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f777400d5f0 tx=0x7f777400dab0 comp rx=0 tx=0).stop 2024-09-13T17:01:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 shutdown_connections 2024-09-13T17:01:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f775c0666c0 0x7f775c068b80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f777c06fba0 0x7f777c1572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f777c0713f0 0x7f777c1f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 --2- 172.21.15.39:0/3876544267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f777c06d500 0x7f777c0a9170 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 >> 172.21.15.39:0/3876544267 conn(0x7f777c069e10 msgr2=0x7f777c14b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 shutdown_connections 2024-09-13T17:01:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:13.029+0000 7f7781c64640 1 -- 172.21.15.39:0/3876544267 wait complete. 2024-09-13T17:01:13.044 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:01:14.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:15.045 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ef58640 1 Processor -- start 2024-09-13T17:01:15.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ef58640 1 -- start start 2024-09-13T17:01:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ef58640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2d581556d0 0x7f2d580c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ef58640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2d580a72b0 0x7f2d58071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ef58640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5806feb0 con 0x7f2d580c7610 2024-09-13T17:01:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ef58640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d580aae50 con 0x7f2d581556d0 2024-09-13T17:01:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ef58640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d58155ab0 con 0x7f2d580a72b0 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ccf4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2d581556d0 0x7f2d580c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d57fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2d580c7610 0x7f2d58151b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41016/0 (socket says 172.21.15.39:41016) 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d57fff640 1 -- 172.21.15.39:0/3594888403 learned_addr learned my addr 172.21.15.39:0/3594888403 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5d4f5640 1 --2- 172.21.15.39:0/3594888403 >> v2:172.21.15.84:3300/0 conn(0x7f2d580a72b0 0x7f2d58071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ccf4640 1 -- 172.21.15.39:0/3594888403 >> v2:172.21.15.84:3300/0 conn(0x7f2d580a72b0 msgr2=0x7f2d58071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d577fe640 1 -- 172.21.15.39:0/3594888403 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13737726 0 0) 0x7f2d5806feb0 con 0x7f2d580c7610 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ccf4640 1 --2- 172.21.15.39:0/3594888403 >> v2:172.21.15.84:3300/0 conn(0x7f2d580a72b0 0x7f2d58071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ccf4640 1 -- 172.21.15.39:0/3594888403 >> v1:172.21.15.39:6789/0 conn(0x7f2d580c7610 legacy=0x7f2d58151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5ccf4640 1 -- 172.21.15.39:0/3594888403 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d580c3c20 con 0x7f2d581556d0 2024-09-13T17:01:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.121+0000 7f2d5d4f5640 1 --2- 172.21.15.39:0/3594888403 >> v2:172.21.15.84:3300/0 conn(0x7f2d580a72b0 0x7f2d58071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ccf4640 1 --2- 172.21.15.39:0/3594888403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d581556d0 0x7f2d580c36e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2d580714c0 tx=0x7f2d4801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e58c70eb3a1c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d577fe640 1 -- 172.21.15.39:0/3594888403 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d480ab600 con 0x7f2d581556d0 2024-09-13T17:01:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d577fe640 1 -- 172.21.15.39:0/3594888403 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d480a7050 con 0x7f2d581556d0 2024-09-13T17:01:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d577fe640 1 -- 172.21.15.39:0/3594888403 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d480ab980 con 0x7f2d581556d0 2024-09-13T17:01:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 -- 172.21.15.39:0/3594888403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d581556d0 msgr2=0x7f2d580c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 --2- 172.21.15.39:0/3594888403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d581556d0 0x7f2d580c36e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2d580714c0 tx=0x7f2d4801c360 comp rx=0 tx=0).stop 2024-09-13T17:01:15.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 -- 172.21.15.39:0/3594888403 shutdown_connections 2024-09-13T17:01:15.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 --2- 172.21.15.39:0/3594888403 >> v2:172.21.15.84:3300/0 conn(0x7f2d580a72b0 0x7f2d58071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 --2- 172.21.15.39:0/3594888403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d581556d0 0x7f2d580c36e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 -- 172.21.15.39:0/3594888403 >> 172.21.15.39:0/3594888403 conn(0x7f2d580a97b0 msgr2=0x7f2d580a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:15.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 -- 172.21.15.39:0/3594888403 shutdown_connections 2024-09-13T17:01:15.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 -- 172.21.15.39:0/3594888403 wait complete. 2024-09-13T17:01:15.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 Processor -- start 2024-09-13T17:01:15.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 -- start start 2024-09-13T17:01:15.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d580a72b0 0x7f2d5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d581556d0 0x7f2d5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d580c4c20 0x7f2d5806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5815aed0 con 0x7f2d580a72b0 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5806e830 con 0x7f2d581556d0 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.125+0000 7f2d5ef58640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58151540 con 0x7f2d580c4c20 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ccf4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d580a72b0 0x7f2d5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ccf4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d580a72b0 0x7f2d5806d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43138/0 (socket says 172.21.15.39:43138) 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ccf4640 1 -- 172.21.15.39:0/4110967254 learned_addr learned my addr 172.21.15.39:0/4110967254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ccf4640 1 -- 172.21.15.39:0/4110967254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d580c4c20 msgr2=0x7f2d5806ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5d4f5640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d580c4c20 0x7f2d5806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d57fff640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d581556d0 0x7f2d5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ccf4640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d580c4c20 0x7f2d5806ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ccf4640 1 -- 172.21.15.39:0/4110967254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d581556d0 msgr2=0x7f2d5806e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ccf4640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d581556d0 0x7f2d5806e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ccf4640 1 -- 172.21.15.39:0/4110967254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d48009420 con 0x7f2d580a72b0 2024-09-13T17:01:15.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5d4f5640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d580c4c20 0x7f2d5806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:15.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ccf4640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d580a72b0 0x7f2d5806d190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2d48019f90 tx=0x7f2d48019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:15.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d557fa640 1 -- 172.21.15.39:0/4110967254 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48004230 con 0x7f2d580a72b0 2024-09-13T17:01:15.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d557fa640 1 -- 172.21.15.39:0/4110967254 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d48004390 con 0x7f2d580a72b0 2024-09-13T17:01:15.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5806f340 con 0x7f2d580a72b0 2024-09-13T17:01:15.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d557fa640 1 -- 172.21.15.39:0/4110967254 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48003820 con 0x7f2d580a72b0 2024-09-13T17:01:15.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d58151750 con 0x7f2d580a72b0 2024-09-13T17:01:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d58155f40 con 0x7f2d580a72b0 2024-09-13T17:01:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d557fa640 1 -- 172.21.15.39:0/4110967254 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d480b2040 con 0x7f2d580a72b0 2024-09-13T17:01:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d557fa640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d44066690 0x7f2d44068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d557fa640 1 -- 172.21.15.39:0/4110967254 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f2d4811e8b0 con 0x7f2d580a72b0 2024-09-13T17:01:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d57fff640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d44066690 0x7f2d44068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.129+0000 7f2d57fff640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d44066690 0x7f2d44068b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2d580ac020 tx=0x7f2d4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:15.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.137+0000 7f2d557fa640 1 -- 172.21.15.39:0/4110967254 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2d480eb400 con 0x7f2d580a72b0 2024-09-13T17:01:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.173+0000 7f152f5d0640 1 Processor -- start 2024-09-13T17:01:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.173+0000 7f152f5d0640 1 -- start start 2024-09-13T17:01:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152f5d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15280c2370 0x7f15280c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152f5d0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1528153e30 con 0x7f15280c5870 2024-09-13T17:01:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152f5d0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15280a7000 con 0x7f1528154bb0 2024-09-13T17:01:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152f5d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15280720f0 con 0x7f15280c2370 2024-09-13T17:01:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152db6d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f15280c5870 0x7f152815e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41032/0 (socket says 172.21.15.39:41032) 2024-09-13T17:01:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152db6d640 1 -- 172.21.15.39:0/1541823319 learned_addr learned my addr 172.21.15.39:0/1541823319 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152d36c640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15280c2370 0x7f15280c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217724846 0 0) 0x7f1528153e30 con 0x7f15280c5870 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1514003530 con 0x7f15280c5870 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3866232228 0 0) 0x7f1514003530 con 0x7f15280c5870 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15280c2370 msgr2=0x7f15280c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15280c2370 0x7f15280c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 >> v1:172.21.15.84:6789/0 conn(0x7f1528154bb0 legacy=0x7f15280c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15281540d0 con 0x7f15280c5870 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1520001a00 con 0x7f15280c5870 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 >> v1:172.21.15.39:6789/0 conn(0x7f15280c5870 legacy=0x7f152815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15280c2370 0x7f15140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1514003f30 0x7f15140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f1514005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280720f0 con 0x7f1514004d10 2024-09-13T17:01:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151400bb20 con 0x7f1514003f30 2024-09-13T17:01:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151400bce0 con 0x7f15280c2370 2024-09-13T17:01:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152db6d640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f1514005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152cb6b640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1514003f30 0x7f15140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152d36c640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15280c2370 0x7f15140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152db6d640 1 -- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15280c2370 msgr2=0x7f15140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152db6d640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15280c2370 0x7f15140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152db6d640 1 -- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1514003f30 msgr2=0x7f15140047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152db6d640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1514003f30 0x7f15140047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152db6d640 1 -- 172.21.15.39:0/1541823319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514003530 con 0x7f1514004d10 2024-09-13T17:01:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f152db6d640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f1514005190 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f152000a830 tx=0x7f152001fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a3c05b64c792e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200aa6f0 con 0x7f1514004d10 2024-09-13T17:01:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15200aa870 con 0x7f1514004d10 2024-09-13T17:01:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.177+0000 7f151ffff640 1 -- 172.21.15.39:0/1541823319 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200aac20 con 0x7f1514004d10 2024-09-13T17:01:15.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 -- 172.21.15.39:0/1541823319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 msgr2=0x7f1514005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 --2- 172.21.15.39:0/1541823319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f1514005190 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f152000a830 tx=0x7f152001fac0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152db6d640 1 -- 172.21.15.39:0/1541823319 reap_dead start 2024-09-13T17:01:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 -- 172.21.15.39:0/1541823319 shutdown_connections 2024-09-13T17:01:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 -- 172.21.15.39:0/1541823319 >> 172.21.15.39:0/1541823319 conn(0x7f15280a9720 msgr2=0x7f15280a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 -- 172.21.15.39:0/1541823319 shutdown_connections 2024-09-13T17:01:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 -- 172.21.15.39:0/1541823319 wait complete. 2024-09-13T17:01:15.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 Processor -- start 2024-09-13T17:01:15.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 -- start start 2024-09-13T17:01:15.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f15281ed720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152806ebb0 0x7f152806a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15281deba0 0x7f15281e9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15281eccd0 con 0x7f1514004d10 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152814b1d0 con 0x7f152806ebb0 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152f5d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152814b4b0 con 0x7f15281deba0 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152d36c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f15281ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152d36c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f15281ed720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43164/0 (socket says 172.21.15.39:43164) 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.181+0000 7f152d36c640 1 -- 172.21.15.39:0/3552529780 learned_addr learned my addr 172.21.15.39:0/3552529780 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152cb6b640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152806ebb0 0x7f152806a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152db6d640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15281deba0 0x7f15281e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152d36c640 1 -- 172.21.15.39:0/3552529780 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15281deba0 msgr2=0x7f15281e9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152d36c640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15281deba0 0x7f15281e9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152d36c640 1 -- 172.21.15.39:0/3552529780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152806ebb0 msgr2=0x7f152806a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152d36c640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152806ebb0 0x7f152806a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152d36c640 1 -- 172.21.15.39:0/3552529780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15281ecf20 con 0x7f1514004d10 2024-09-13T17:01:15.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152d36c640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f15281ed720 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f15240115e0 tx=0x7f1524011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:15.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f151dffb640 1 -- 172.21.15.39:0/3552529780 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152400f8f0 con 0x7f1514004d10 2024-09-13T17:01:15.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f151dffb640 1 -- 172.21.15.39:0/3552529780 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152400fa50 con 0x7f1514004d10 2024-09-13T17:01:15.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f151dffb640 1 -- 172.21.15.39:0/3552529780 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152400fdc0 con 0x7f1514004d10 2024-09-13T17:01:15.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15281ed200 con 0x7f1514004d10 2024-09-13T17:01:15.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152814da20 con 0x7f1514004d10 2024-09-13T17:01:15.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15280a76d0 con 0x7f1514004d10 2024-09-13T17:01:15.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f151dffb640 1 -- 172.21.15.39:0/3552529780 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1524015460 con 0x7f1514004d10 2024-09-13T17:01:15.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f151dffb640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f150c0666c0 0x7f150c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:15.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f151dffb640 1 -- 172.21.15.39:0/3552529780 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f1524084ea0 con 0x7f1514004d10 2024-09-13T17:01:15.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152cb6b640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f150c0666c0 0x7f150c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:15.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.185+0000 7f152cb6b640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f150c0666c0 0x7f150c068b80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1528061840 tx=0x7f1518009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.189+0000 7f151dffb640 1 -- 172.21.15.39:0/3552529780 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15240519f0 con 0x7f1514004d10 2024-09-13T17:01:15.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.337+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d581ea9b0 con 0x7f2d580a72b0 2024-09-13T17:01:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.337+0000 7f2d557fa640 1 -- 172.21.15.39:0/4110967254 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f2d480c2020 con 0x7f2d580a72b0 2024-09-13T17:01:15.337 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:15.337 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:15.337 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d44066690 msgr2=0x7f2d44068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d44066690 0x7f2d44068b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2d580ac020 tx=0x7f2d4c009040 comp rx=0 tx=0).stop 2024-09-13T17:01:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d580a72b0 msgr2=0x7f2d5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d580a72b0 0x7f2d5806d190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2d48019f90 tx=0x7f2d48019100 comp rx=0 tx=0).stop 2024-09-13T17:01:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 shutdown_connections 2024-09-13T17:01:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2d44066690 0x7f2d44068b50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2d581556d0 0x7f2d5806e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2d580c4c20 0x7f2d5806ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 --2- 172.21.15.39:0/4110967254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d580a72b0 0x7f2d5806d190 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 >> 172.21.15.39:0/4110967254 conn(0x7f2d580a97b0 msgr2=0x7f2d580a8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 shutdown_connections 2024-09-13T17:01:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.341+0000 7f2d5ef58640 1 -- 172.21.15.39:0/4110967254 wait complete. 2024-09-13T17:01:15.356 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:01:15.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.385+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f152806be00 con 0x7f1514004d10 2024-09-13T17:01:15.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.385+0000 7f151dffb640 1 -- 172.21.15.39:0/3552529780 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f1524022020 con 0x7f1514004d10 2024-09-13T17:01:15.384 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:15.385 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:15.385 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:15.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f150c0666c0 msgr2=0x7f150c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f150c0666c0 0x7f150c068b80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1528061840 tx=0x7f1518009040 comp rx=0 tx=0).stop 2024-09-13T17:01:15.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 msgr2=0x7f15281ed720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:15.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f15281ed720 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f15240115e0 tx=0x7f1524011aa0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 shutdown_connections 2024-09-13T17:01:15.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f150c0666c0 0x7f150c068b80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f152806ebb0 0x7f152806a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15281deba0 0x7f15281e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 --2- 172.21.15.39:0/3552529780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1514004d10 0x7f15281ed720 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:15.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 >> 172.21.15.39:0/3552529780 conn(0x7f15280a9720 msgr2=0x7f15280a8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:15.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 shutdown_connections 2024-09-13T17:01:15.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:15.389+0000 7f152f5d0640 1 -- 172.21.15.39:0/3552529780 wait complete. 2024-09-13T17:01:15.403 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:01:17.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:17.404 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:17.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.493+0000 7fa1d1615640 1 Processor -- start 2024-09-13T17:01:17.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.493+0000 7fa1d1615640 1 -- start start 2024-09-13T17:01:17.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1d1615640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 0x7fa1cc151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1d1615640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa1cc06e5e0 0x7fa1cc0714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1d1615640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1cc14f590 con 0x7fa1cc0c7470 2024-09-13T17:01:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1d1615640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1cc06f660 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1d1615640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc06eae0 con 0x7fa1cc14f1b0 2024-09-13T17:01:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1caffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 0x7fa1cc151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1cb7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa1cc0c7470 0x7fa1cc0c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:60750/0 (socket says 172.21.15.39:60750) 2024-09-13T17:01:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1ca7fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa1cc06e5e0 0x7fa1cc0714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1cb7fe640 1 -- 172.21.15.39:0/2996371185 learned_addr learned my addr 172.21.15.39:0/2996371185 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1ca7fc640 1 -- 172.21.15.39:0/2996371185 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 msgr2=0x7fa1cc151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1ca7fc640 1 --2- 172.21.15.39:0/2996371185 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 0x7fa1cc151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1ca7fc640 1 -- 172.21.15.39:0/2996371185 >> v1:172.21.15.84:6789/0 conn(0x7fa1cc0c7470 legacy=0x7fa1cc0c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.497+0000 7fa1ca7fc640 1 -- 172.21.15.39:0/2996371185 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc06edc0 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1ca7fc640 1 --2- 172.21.15.39:0/2996371185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc06e5e0 0x7fa1cc0714e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00097c0 tx=0x7fa1c001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6118f898586f4fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1c9ffb640 1 -- 172.21.15.39:0/2996371185 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00ab670 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1c9ffb640 1 -- 172.21.15.39:0/2996371185 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c00ab7d0 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1c9ffb640 1 -- 172.21.15.39:0/2996371185 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00abb60 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 -- 172.21.15.39:0/2996371185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc06e5e0 msgr2=0x7fa1cc0714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 --2- 172.21.15.39:0/2996371185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc06e5e0 0x7fa1cc0714e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00097c0 tx=0x7fa1c001c470 comp rx=0 tx=0).stop 2024-09-13T17:01:17.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 -- 172.21.15.39:0/2996371185 shutdown_connections 2024-09-13T17:01:17.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 --2- 172.21.15.39:0/2996371185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc06e5e0 0x7fa1cc0714e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 --2- 172.21.15.39:0/2996371185 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 0x7fa1cc151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 -- 172.21.15.39:0/2996371185 >> 172.21.15.39:0/2996371185 conn(0x7fa1cc0a9720 msgr2=0x7fa1cc0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:17.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 -- 172.21.15.39:0/2996371185 shutdown_connections 2024-09-13T17:01:17.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 -- 172.21.15.39:0/2996371185 wait complete. 2024-09-13T17:01:17.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 Processor -- start 2024-09-13T17:01:17.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 -- start start 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1cc06e5e0 0x7fa1cc0c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc0c7470 0x7fa1cc144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 0x7fa1cc145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0c3790 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0c2de0 con 0x7fa1cc0c7470 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1d1615640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc0c3490 con 0x7fa1cc14f1b0 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1caffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1cc06e5e0 0x7fa1cc0c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1caffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1cc06e5e0 0x7fa1cc0c28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43174/0 (socket says 172.21.15.39:43174) 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1caffd640 1 -- 172.21.15.39:0/1220100044 learned_addr learned my addr 172.21.15.39:0/1220100044 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1cb7fe640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 0x7fa1cc145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1ca7fc640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc0c7470 0x7fa1cc144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1caffd640 1 -- 172.21.15.39:0/1220100044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 msgr2=0x7fa1cc145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1caffd640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 0x7fa1cc145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1caffd640 1 -- 172.21.15.39:0/1220100044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc0c7470 msgr2=0x7fa1cc144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1caffd640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc0c7470 0x7fa1cc144bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1caffd640 1 -- 172.21.15.39:0/1220100044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c0009420 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.501+0000 7fa1ca7fc640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc0c7470 0x7fa1cc144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1caffd640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1cc06e5e0 0x7fa1cc0c28a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00e8d0 tx=0x7fa1bc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1b37fe640 1 -- 172.21.15.39:0/1220100044 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc009510 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1b37fe640 1 -- 172.21.15.39:0/1220100044 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc00c8e0 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc0ace40 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1b37fe640 1 -- 172.21.15.39:0/1220100044 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc00cc50 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc0ad400 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1b37fe640 1 -- 172.21.15.39:0/1220100044 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1bc013470 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1cc0714e0 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1b37fe640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1b8066690 0x7fa1b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1ca7fc640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1b8066690 0x7fa1b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1b37fe640 1 -- 172.21.15.39:0/1220100044 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7fa1bc084a60 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.505+0000 7fa1ca7fc640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1b8066690 0x7fa1b8068b50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00af000 tx=0x7fa1c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:17.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.509+0000 7fa1b37fe640 1 -- 172.21.15.39:0/1220100044 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa1bc0515b0 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.525+0000 7f479f1e1640 1 Processor -- start 2024-09-13T17:01:17.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.525+0000 7f479f1e1640 1 -- start start 2024-09-13T17:01:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479f1e1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479814f1b0 0x7f4798151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479f1e1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f479806e5e0 0x7f47980714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479f1e1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479814f590 con 0x7f47980c7470 2024-09-13T17:01:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479f1e1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479806f660 con 0x7f479806e5e0 2024-09-13T17:01:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479f1e1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479806eae0 con 0x7f479814f1b0 2024-09-13T17:01:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479d77e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f47980c7470 0x7f47980c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41034/0 (socket says 172.21.15.39:41034) 2024-09-13T17:01:17.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479d77e640 1 -- 172.21.15.39:0/1554966956 learned_addr learned my addr 172.21.15.39:0/1554966956 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f4797fff640 1 --2- 172.21.15.39:0/1554966956 >> v2:172.21.15.84:3300/0 conn(0x7f479806e5e0 0x7f47980714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479cf7d640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479814f1b0 0x7f4798151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56930686 0 0) 0x7f479814f590 con 0x7f47980c7470 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4788003530 con 0x7f47980c7470 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3427581956 0 0) 0x7f4788003530 con 0x7f47980c7470 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479814f1b0 msgr2=0x7f4798151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479814f1b0 0x7f4798151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 >> v2:172.21.15.84:3300/0 conn(0x7f479806e5e0 msgr2=0x7f47980714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 --2- 172.21.15.39:0/1554966956 >> v2:172.21.15.84:3300/0 conn(0x7f479806e5e0 0x7f47980714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479806edc0 con 0x7f47980c7470 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4790001a00 con 0x7f47980c7470 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 >> v1:172.21.15.39:6789/0 conn(0x7f47980c7470 legacy=0x7f47980c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479814f1b0 0x7f4788003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4788003f50 0x7f47880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4788004d10 0x7f4788005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479806f660 con 0x7f4788004d10 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479806eae0 con 0x7f4788003f50 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478800b990 con 0x7f479814f1b0 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479d77e640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4788004d10 0x7f4788005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479cf7d640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479814f1b0 0x7f4788003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f4797fff640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4788003f50 0x7f47880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479d77e640 1 -- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479814f1b0 msgr2=0x7f4788003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479d77e640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479814f1b0 0x7f4788003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479d77e640 1 -- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4788003f50 msgr2=0x7f47880047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479d77e640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4788003f50 0x7f47880047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479d77e640 1 -- 172.21.15.39:0/1554966956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4788003530 con 0x7f4788004d10 2024-09-13T17:01:17.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.529+0000 7f479cf7d640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479814f1b0 0x7f4788003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:17.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479d77e640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4788004d10 0x7f4788005190 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f479000a7c0 tx=0x7f479001d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f342f39114016693 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:17.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47900ad630 con 0x7f4788004d10 2024-09-13T17:01:17.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47900a8070 con 0x7f4788004d10 2024-09-13T17:01:17.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f47977fe640 1 -- 172.21.15.39:0/1554966956 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47900ada30 con 0x7f4788004d10 2024-09-13T17:01:17.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- 172.21.15.39:0/1554966956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4788004d10 msgr2=0x7f4788005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 --2- 172.21.15.39:0/1554966956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4788004d10 0x7f4788005190 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f479000a7c0 tx=0x7f479001d2a0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479d77e640 1 -- 172.21.15.39:0/1554966956 reap_dead start 2024-09-13T17:01:17.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- 172.21.15.39:0/1554966956 shutdown_connections 2024-09-13T17:01:17.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- 172.21.15.39:0/1554966956 >> 172.21.15.39:0/1554966956 conn(0x7f47980a9720 msgr2=0x7f47980a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- 172.21.15.39:0/1554966956 shutdown_connections 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- 172.21.15.39:0/1554966956 wait complete. 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 Processor -- start 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- start start 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980c1000 0x7f47980ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47981ecb20 0x7f47980abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479806b150 0x7f479806b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4798153110 con 0x7f47981ecb20 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479814e540 con 0x7f47980c1000 2024-09-13T17:01:17.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47980ac2e0 con 0x7f479806b150 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f4797fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47981ecb20 0x7f47980abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f4797fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47981ecb20 0x7f47980abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43192/0 (socket says 172.21.15.39:43192) 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f4797fff640 1 -- 172.21.15.39:0/600864666 learned_addr learned my addr 172.21.15.39:0/600864666 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479cf7d640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980c1000 0x7f47980ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479d77e640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479806b150 0x7f479806b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f4797fff640 1 -- 172.21.15.39:0/600864666 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479806b150 msgr2=0x7f479806b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f4797fff640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479806b150 0x7f479806b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f4797fff640 1 -- 172.21.15.39:0/600864666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980c1000 msgr2=0x7f47980ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f4797fff640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980c1000 0x7f47980ab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f4797fff640 1 -- 172.21.15.39:0/600864666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798149150 con 0x7f47981ecb20 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479cf7d640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980c1000 0x7f47980ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f4797fff640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47981ecb20 0x7f47980abda0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f478c011590 tx=0x7f478c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:17.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f47957fa640 1 -- 172.21.15.39:0/600864666 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c015870 con 0x7f47981ecb20 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f47957fa640 1 -- 172.21.15.39:0/600864666 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c0159d0 con 0x7f47981ecb20 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4798148540 con 0x7f47981ecb20 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f47957fa640 1 -- 172.21.15.39:0/600864666 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c015d40 con 0x7f47981ecb20 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.533+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4798148b00 con 0x7f47981ecb20 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.537+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47981492e0 con 0x7f47981ecb20 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.537+0000 7f47957fa640 1 -- 172.21.15.39:0/600864666 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f478c026420 con 0x7f47981ecb20 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.537+0000 7f47957fa640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f477c0666c0 0x7f477c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.537+0000 7f479cf7d640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f477c0666c0 0x7f477c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.537+0000 7f47957fa640 1 -- 172.21.15.39:0/600864666 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f478c085150 con 0x7f47981ecb20 2024-09-13T17:01:17.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.537+0000 7f479cf7d640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f477c0666c0 0x7f477c068b80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4784009b60 tx=0x7f4784008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:17.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.541+0000 7f47957fa640 1 -- 172.21.15.39:0/600864666 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f478c051c80 con 0x7f47981ecb20 2024-09-13T17:01:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1cc0c2050 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1b37fe640 1 -- 172.21.15.39:0/1220100044 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7fa1bc017090 con 0x7fa1cc06e5e0 2024-09-13T17:01:17.708 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:17.708 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:17.708 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1b8066690 msgr2=0x7fa1b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1b8066690 0x7fa1b8068b50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00af000 tx=0x7fa1c0008040 comp rx=0 tx=0).stop 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1cc06e5e0 msgr2=0x7fa1cc0c28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1cc06e5e0 0x7fa1cc0c28a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00e8d0 tx=0x7fa1bc00ecd0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 shutdown_connections 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa1b8066690 0x7fa1b8068b50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc0c7470 0x7fa1cc144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa1cc14f1b0 0x7fa1cc145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 --2- 172.21.15.39:0/1220100044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1cc06e5e0 0x7fa1cc0c28a0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 >> 172.21.15.39:0/1220100044 conn(0x7fa1cc0a9720 msgr2=0x7fa1cc148eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 shutdown_connections 2024-09-13T17:01:17.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.709+0000 7fa1d1615640 1 -- 172.21.15.39:0/1220100044 wait complete. 2024-09-13T17:01:17.727 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:01:17.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.737+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f479814f590 con 0x7f47981ecb20 2024-09-13T17:01:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.737+0000 7f47957fa640 1 -- 172.21.15.39:0/600864666 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f478c0552a0 con 0x7f47981ecb20 2024-09-13T17:01:17.737 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:17.737 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f477c0666c0 msgr2=0x7f477c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f477c0666c0 0x7f477c068b80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4784009b60 tx=0x7f4784008040 comp rx=0 tx=0).stop 2024-09-13T17:01:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47981ecb20 msgr2=0x7f47980abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47981ecb20 0x7f47980abda0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f478c011590 tx=0x7f478c011a50 comp rx=0 tx=0).stop 2024-09-13T17:01:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 shutdown_connections 2024-09-13T17:01:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f477c0666c0 0x7f477c068b80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f479806b150 0x7f479806b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47981ecb20 0x7f47980abda0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 --2- 172.21.15.39:0/600864666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47980c1000 0x7f47980ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 >> 172.21.15.39:0/600864666 conn(0x7f47980a9720 msgr2=0x7f47980c5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 shutdown_connections 2024-09-13T17:01:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:17.741+0000 7f479f1e1640 1 -- 172.21.15.39:0/600864666 wait complete. 2024-09-13T17:01:17.755 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:01:19.729 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:19.756 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:19.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0363899640 1 Processor -- start 2024-09-13T17:01:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0363899640 1 -- start start 2024-09-13T17:01:19.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0363899640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c0611c0 0x7f035c0bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0363899640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c0c00b0 con 0x7f035c0c2db0 2024-09-13T17:01:19.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0363899640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c153e70 con 0x7f035c06cbc0 2024-09-13T17:01:19.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0363899640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0c0300 con 0x7f035c0611c0 2024-09-13T17:01:19.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0361635640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f035c0c2db0 0x7f035c1512b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41050/0 (socket says 172.21.15.39:41050) 2024-09-13T17:01:19.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0361635640 1 -- 172.21.15.39:0/1661777451 learned_addr learned my addr 172.21.15.39:0/1661777451 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0361e36640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c0611c0 0x7f035c0bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025351396 0 0) 0x7f035c0c00b0 con 0x7f035c0c2db0 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0348003530 con 0x7f035c0c2db0 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4117320497 0 0) 0x7f0348003530 con 0x7f035c0c2db0 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c0611c0 msgr2=0x7f035c0bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c0611c0 0x7f035c0bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 >> v1:172.21.15.84:6789/0 conn(0x7f035c06cbc0 legacy=0x7f035c1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0c05e0 con 0x7f035c0c2db0 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f0358001680 con 0x7f035c0c2db0 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 >> v1:172.21.15.39:6789/0 conn(0x7f035c0c2db0 legacy=0x7f035c1512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:19.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03480039b0 0x7f0348003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f0348004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c0611c0 0x7f0348005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0c0300 con 0x7f03480047d0 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034800bb50 con 0x7f03480039b0 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034800bd10 con 0x7f035c0611c0 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0360e34640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f0348004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0361e36640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c0611c0 0x7f0348005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0361635640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03480039b0 0x7f0348003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0360e34640 1 -- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c0611c0 msgr2=0x7f0348005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0360e34640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c0611c0 0x7f0348005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0360e34640 1 -- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03480039b0 msgr2=0x7f0348003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0360e34640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03480039b0 0x7f0348003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.869+0000 7f0360e34640 1 -- 172.21.15.39:0/1661777451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0348003530 con 0x7f03480047d0 2024-09-13T17:01:19.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0360e34640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f0348004c50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f034c001780 tx=0x7f034c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63cac4c11b4d5165 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c0052d0 con 0x7f03480047d0 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c0a7050 con 0x7f03480047d0 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0353fff640 1 -- 172.21.15.39:0/1661777451 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c003600 con 0x7f03480047d0 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 -- 172.21.15.39:0/1661777451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 msgr2=0x7f0348004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 --2- 172.21.15.39:0/1661777451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f0348004c50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f034c001780 tx=0x7f034c01c420 comp rx=0 tx=0).stop 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0361e36640 1 -- 172.21.15.39:0/1661777451 reap_dead start 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 -- 172.21.15.39:0/1661777451 shutdown_connections 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 -- 172.21.15.39:0/1661777451 >> 172.21.15.39:0/1661777451 conn(0x7f035c069e10 msgr2=0x7f035c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 -- 172.21.15.39:0/1661777451 shutdown_connections 2024-09-13T17:01:19.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 -- 172.21.15.39:0/1661777451 wait complete. 2024-09-13T17:01:19.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 Processor -- start 2024-09-13T17:01:19.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 -- start start 2024-09-13T17:01:19.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f035c0c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f035c157f10 0x7f035c140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c140d50 0x7f035c14c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0c5920 con 0x7f03480047d0 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c1ec720 con 0x7f035c157f10 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0363899640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c1eca00 con 0x7f035c140d50 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0361635640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f035c0c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0361635640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f035c0c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43210/0 (socket says 172.21.15.39:43210) 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0361635640 1 -- 172.21.15.39:0/2077985715 learned_addr learned my addr 172.21.15.39:0/2077985715 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0360e34640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f035c157f10 0x7f035c140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.873+0000 7f0361e36640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c140d50 0x7f035c14c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0361635640 1 -- 172.21.15.39:0/2077985715 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c140d50 msgr2=0x7f035c14c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0361635640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c140d50 0x7f035c14c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0361635640 1 -- 172.21.15.39:0/2077985715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f035c157f10 msgr2=0x7f035c140810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0361635640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f035c157f10 0x7f035c140810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0361635640 1 -- 172.21.15.39:0/2077985715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c1ecce0 con 0x7f03480047d0 2024-09-13T17:01:19.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0361635640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f035c0c4c60 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f035800d7a0 tx=0x7f035800dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:19.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0351ffb640 1 -- 172.21.15.39:0/2077985715 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03580072d0 con 0x7f03480047d0 2024-09-13T17:01:19.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0351ffb640 1 -- 172.21.15.39:0/2077985715 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0358007430 con 0x7f03480047d0 2024-09-13T17:01:19.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0351ffb640 1 -- 172.21.15.39:0/2077985715 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03580047f0 con 0x7f03480047d0 2024-09-13T17:01:19.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035c1ecfc0 con 0x7f03480047d0 2024-09-13T17:01:19.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c0c51a0 con 0x7f03480047d0 2024-09-13T17:01:19.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c14c3a0 con 0x7f03480047d0 2024-09-13T17:01:19.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0351ffb640 1 -- 172.21.15.39:0/2077985715 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0358004950 con 0x7f03480047d0 2024-09-13T17:01:19.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0351ffb640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0340066690 0x7f0340068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0351ffb640 1 -- 172.21.15.39:0/2077985715 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f0358084250 con 0x7f03480047d0 2024-09-13T17:01:19.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0360e34640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0340066690 0x7f0340068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.877+0000 7f0360e34640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0340066690 0x7f0340068b50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f034c001750 tx=0x7f034c0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:19.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.881+0000 7f0351ffb640 1 -- 172.21.15.39:0/2077985715 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f035c14c3a0 con 0x7f03480047d0 2024-09-13T17:01:19.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8fba3b640 1 Processor -- start 2024-09-13T17:01:19.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8fba3b640 1 -- start start 2024-09-13T17:01:19.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8fba3b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40aaff0 0x7fb8f406d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8fba3b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8f41573e0 0x7fb8f4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8fba3b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f406d130 con 0x7fb8f40c2630 2024-09-13T17:01:19.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8fba3b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f4153df0 con 0x7fb8f41573e0 2024-09-13T17:01:19.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8fba3b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f406cb90 con 0x7fb8f40aaff0 2024-09-13T17:01:19.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8f9fd8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb8f41573e0 0x7fb8f4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8f97d7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40aaff0 0x7fb8f406d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8f97d7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40aaff0 0x7fb8f406d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:51824/0 (socket says 172.21.15.39:51824) 2024-09-13T17:01:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8f97d7640 1 -- 172.21.15.39:0/1759027506 learned_addr learned my addr 172.21.15.39:0/1759027506 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8f97d7640 1 -- 172.21.15.39:0/1759027506 >> v2:172.21.15.84:3300/0 conn(0x7fb8f41573e0 msgr2=0x7fb8f4157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8f97d7640 1 --2- 172.21.15.39:0/1759027506 >> v2:172.21.15.84:3300/0 conn(0x7fb8f41573e0 0x7fb8f4157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8f97d7640 1 -- 172.21.15.39:0/1759027506 >> v1:172.21.15.84:6789/0 conn(0x7fb8f40c2630 legacy=0x7fb8f4154520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:19.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.901+0000 7fb8f97d7640 1 -- 172.21.15.39:0/1759027506 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f40c0e30 con 0x7fb8f40aaff0 2024-09-13T17:01:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8f97d7640 1 --2- 172.21.15.39:0/1759027506 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40aaff0 0x7fb8f406d800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009870 tx=0x7fb8f001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ff8a1cab8aa6a99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:19.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8ebfff640 1 -- 172.21.15.39:0/1759027506 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00ac410 con 0x7fb8f40aaff0 2024-09-13T17:01:19.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8ebfff640 1 -- 172.21.15.39:0/1759027506 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f00ac570 con 0x7fb8f40aaff0 2024-09-13T17:01:19.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8ebfff640 1 -- 172.21.15.39:0/1759027506 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00ac900 con 0x7fb8f40aaff0 2024-09-13T17:01:19.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1759027506 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40aaff0 msgr2=0x7fb8f406d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:19.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 --2- 172.21.15.39:0/1759027506 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40aaff0 0x7fb8f406d800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009870 tx=0x7fb8f001c4a0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1759027506 shutdown_connections 2024-09-13T17:01:19.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 --2- 172.21.15.39:0/1759027506 >> v2:172.21.15.84:3300/0 conn(0x7fb8f41573e0 0x7fb8f4157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 --2- 172.21.15.39:0/1759027506 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40aaff0 0x7fb8f406d800 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1759027506 >> 172.21.15.39:0/1759027506 conn(0x7fb8f40a8860 msgr2=0x7fb8f406bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:19.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1759027506 shutdown_connections 2024-09-13T17:01:19.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1759027506 wait complete. 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 Processor -- start 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.905+0000 7fb8fba3b640 1 -- start start 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8fba3b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8f40aaff0 0x7fb8f4147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8fba3b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40c2630 0x7fb8f406fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8fba3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f41573e0 0x7fb8f40c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8fba3b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40c3ec0 con 0x7fb8f40aaff0 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8fba3b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40c41a0 con 0x7fb8f41573e0 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8fba3b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f40c4480 con 0x7fb8f40c2630 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f97d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8f40aaff0 0x7fb8f4147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f97d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8f40aaff0 0x7fb8f4147b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43218/0 (socket says 172.21.15.39:43218) 2024-09-13T17:01:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f97d7640 1 -- 172.21.15.39:0/1857578771 learned_addr learned my addr 172.21.15.39:0/1857578771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f8fd6640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40c2630 0x7fb8f406fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f9fd8640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f41573e0 0x7fb8f40c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f97d7640 1 -- 172.21.15.39:0/1857578771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40c2630 msgr2=0x7fb8f406fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f97d7640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40c2630 0x7fb8f406fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f97d7640 1 -- 172.21.15.39:0/1857578771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f41573e0 msgr2=0x7fb8f40c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f97d7640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f41573e0 0x7fb8f40c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f97d7640 1 -- 172.21.15.39:0/1857578771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f00094d0 con 0x7fb8f40aaff0 2024-09-13T17:01:19.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f97d7640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8f40aaff0 0x7fb8f4147b30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb8f001ab50 tx=0x7fb8f0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:19.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8e9ffb640 1 -- 172.21.15.39:0/1857578771 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00ac410 con 0x7fb8f40aaff0 2024-09-13T17:01:19.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8e9ffb640 1 -- 172.21.15.39:0/1857578771 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f00ace00 con 0x7fb8f40aaff0 2024-09-13T17:01:19.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8e9ffb640 1 -- 172.21.15.39:0/1857578771 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00b7850 con 0x7fb8f40aaff0 2024-09-13T17:01:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f4071ab0 con 0x7fb8f40aaff0 2024-09-13T17:01:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f40702b0 con 0x7fb8f40aaff0 2024-09-13T17:01:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f4071cd0 con 0x7fb8f40aaff0 2024-09-13T17:01:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8e9ffb640 1 -- 172.21.15.39:0/1857578771 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb8f00b3040 con 0x7fb8f40aaff0 2024-09-13T17:01:19.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8e9ffb640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8d8066690 0x7fb8d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:19.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8e9ffb640 1 -- 172.21.15.39:0/1857578771 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7fb8f011ee80 con 0x7fb8f40aaff0 2024-09-13T17:01:19.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f8fd6640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8d8066690 0x7fb8d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:19.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.909+0000 7fb8f8fd6640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8d8066690 0x7fb8d8068b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4151740 tx=0x7fb8e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:19.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:19.913+0000 7fb8e9ffb640 1 -- 172.21.15.39:0/1857578771 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb8f00eba80 con 0x7fb8f40aaff0 2024-09-13T17:01:20.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.081+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f035c14c3a0 con 0x7f03480047d0 2024-09-13T17:01:20.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.081+0000 7f0351ffb640 1 -- 172.21.15.39:0/2077985715 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f035c14c3a0 con 0x7f03480047d0 2024-09-13T17:01:20.080 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:20.080 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:20.081 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0340066690 msgr2=0x7f0340068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0340066690 0x7f0340068b50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f034c001750 tx=0x7f034c0011a0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 msgr2=0x7f035c0c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f035c0c4c60 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f035800d7a0 tx=0x7f035800dba0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 shutdown_connections 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0340066690 0x7f0340068b50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f035c157f10 0x7f035c140810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f035c140d50 0x7f035c14c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 --2- 172.21.15.39:0/2077985715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03480047d0 0x7f035c0c4c60 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 >> 172.21.15.39:0/2077985715 conn(0x7f035c069e10 msgr2=0x7f035c0701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 shutdown_connections 2024-09-13T17:01:20.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.085+0000 7f0363899640 1 -- 172.21.15.39:0/2077985715 wait complete. 2024-09-13T17:01:20.099 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:01:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.109+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8f40c4000 con 0x7fb8f40aaff0 2024-09-13T17:01:20.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.109+0000 7fb8e9ffb640 1 -- 172.21.15.39:0/1857578771 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7fb8f40c4000 con 0x7fb8f40aaff0 2024-09-13T17:01:20.109 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:20.109 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:20.109 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8d8066690 msgr2=0x7fb8d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8d8066690 0x7fb8d8068b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4151740 tx=0x7fb8e4009040 comp rx=0 tx=0).stop 2024-09-13T17:01:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8f40aaff0 msgr2=0x7fb8f4147b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8f40aaff0 0x7fb8f4147b30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb8f001ab50 tx=0x7fb8f0019240 comp rx=0 tx=0).stop 2024-09-13T17:01:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 shutdown_connections 2024-09-13T17:01:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8d8066690 0x7fb8d8068b50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8f41573e0 0x7fb8f40c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8f40c2630 0x7fb8f406fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 --2- 172.21.15.39:0/1857578771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8f40aaff0 0x7fb8f4147b30 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 >> 172.21.15.39:0/1857578771 conn(0x7fb8f40a8860 msgr2=0x7fb8f40aa180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 shutdown_connections 2024-09-13T17:01:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:20.113+0000 7fb8fba3b640 1 -- 172.21.15.39:0/1857578771 wait complete. 2024-09-13T17:01:20.126 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:01:22.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:22.127 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:22.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.237+0000 7f7807539640 1 Processor -- start 2024-09-13T17:01:22.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.237+0000 7f7807539640 1 -- start start 2024-09-13T17:01:22.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.237+0000 7f7807539640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f780014f340 0x7f7800071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.237+0000 7f7807539640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f78000c26c0 0x7f780006f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.237+0000 7f7807539640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780006d750 con 0x7f78000c7180 2024-09-13T17:01:22.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.237+0000 7f7807539640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780006e410 con 0x7f78000c26c0 2024-09-13T17:01:22.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.237+0000 7f7807539640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78000a7000 con 0x7f780014f340 2024-09-13T17:01:22.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.237+0000 7f7805ad6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f78000c26c0 0x7f780006f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.237+0000 7f7805ad6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f78000c26c0 0x7f780006f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47596/0 (socket says 172.21.15.39:47596) 2024-09-13T17:01:22.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f7805ad6640 1 -- 172.21.15.39:0/1131488942 learned_addr learned my addr 172.21.15.39:0/1131488942 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:22.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f78052d5640 1 --2- 172.21.15.39:0/1131488942 >> v2:172.21.15.84:3300/0 conn(0x7f780014f340 0x7f7800071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f7805ad6640 1 -- 172.21.15.39:0/1131488942 >> v2:172.21.15.84:3300/0 conn(0x7f780014f340 msgr2=0x7f7800071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:22.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f7805ad6640 1 --2- 172.21.15.39:0/1131488942 >> v2:172.21.15.84:3300/0 conn(0x7f780014f340 0x7f7800071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f7805ad6640 1 -- 172.21.15.39:0/1131488942 >> v1:172.21.15.84:6789/0 conn(0x7f78000c7180 legacy=0x7f780006f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:22.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f7805ad6640 1 -- 172.21.15.39:0/1131488942 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7800155ce0 con 0x7f78000c26c0 2024-09-13T17:01:22.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f78052d5640 1 --2- 172.21.15.39:0/1131488942 >> v2:172.21.15.84:3300/0 conn(0x7f780014f340 0x7f7800071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:22.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f7805ad6640 1 --2- 172.21.15.39:0/1131488942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c26c0 0x7f780006f760 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f78000c6ac0 tx=0x7f77f801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40772e7d9f6b4eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:22.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f77f3fff640 1 -- 172.21.15.39:0/1131488942 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f80a56f0 con 0x7f78000c26c0 2024-09-13T17:01:22.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f77f3fff640 1 -- 172.21.15.39:0/1131488942 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f80a5850 con 0x7f78000c26c0 2024-09-13T17:01:22.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.241+0000 7f77f3fff640 1 -- 172.21.15.39:0/1131488942 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f80a5be0 con 0x7f78000c26c0 2024-09-13T17:01:22.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- 172.21.15.39:0/1131488942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c26c0 msgr2=0x7f780006f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 --2- 172.21.15.39:0/1131488942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c26c0 0x7f780006f760 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f78000c6ac0 tx=0x7f77f801c920 comp rx=0 tx=0).stop 2024-09-13T17:01:22.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- 172.21.15.39:0/1131488942 shutdown_connections 2024-09-13T17:01:22.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 --2- 172.21.15.39:0/1131488942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c26c0 0x7f780006f760 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 --2- 172.21.15.39:0/1131488942 >> v2:172.21.15.84:3300/0 conn(0x7f780014f340 0x7f7800071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- 172.21.15.39:0/1131488942 >> 172.21.15.39:0/1131488942 conn(0x7f78000a96b0 msgr2=0x7f78000a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:22.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- 172.21.15.39:0/1131488942 shutdown_connections 2024-09-13T17:01:22.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- 172.21.15.39:0/1131488942 wait complete. 2024-09-13T17:01:22.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 Processor -- start 2024-09-13T17:01:22.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- start start 2024-09-13T17:01:22.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78000c26c0 0x7f78000c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c7180 0x7f78000c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f780014f340 0x7f7800148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78000ab290 con 0x7f78000c7180 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78000ad6a0 con 0x7f780014f340 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78000ab4e0 con 0x7f78000c26c0 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7804ad4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c7180 0x7f78000c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7804ad4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c7180 0x7f78000c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47612/0 (socket says 172.21.15.39:47612) 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7804ad4640 1 -- 172.21.15.39:0/2562928232 learned_addr learned my addr 172.21.15.39:0/2562928232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f78052d5640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78000c26c0 0x7f78000c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7805ad6640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f780014f340 0x7f7800148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7804ad4640 1 -- 172.21.15.39:0/2562928232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78000c26c0 msgr2=0x7f78000c14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7804ad4640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78000c26c0 0x7f78000c14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7804ad4640 1 -- 172.21.15.39:0/2562928232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f780014f340 msgr2=0x7f7800148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7804ad4640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f780014f340 0x7f7800148d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7804ad4640 1 -- 172.21.15.39:0/2562928232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f8009b30 con 0x7f78000c7180 2024-09-13T17:01:22.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7804ad4640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c7180 0x7f78000c49f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f77f400c450 tx=0x7f77f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:22.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f77f1ffb640 1 -- 172.21.15.39:0/2562928232 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f4012830 con 0x7f78000c7180 2024-09-13T17:01:22.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f77f1ffb640 1 -- 172.21.15.39:0/2562928232 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f40129b0 con 0x7f78000c7180 2024-09-13T17:01:22.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78000ab7c0 con 0x7f78000c7180 2024-09-13T17:01:22.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f77f1ffb640 1 -- 172.21.15.39:0/2562928232 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f4012d60 con 0x7f78000c7180 2024-09-13T17:01:22.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78000c4f30 con 0x7f78000c7180 2024-09-13T17:01:22.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.245+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7800071530 con 0x7f78000c7180 2024-09-13T17:01:22.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.249+0000 7f77f1ffb640 1 -- 172.21.15.39:0/2562928232 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f77f401c460 con 0x7f78000c7180 2024-09-13T17:01:22.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.249+0000 7f77f1ffb640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77e4066690 0x7f77e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.249+0000 7f77f1ffb640 1 -- 172.21.15.39:0/2562928232 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f77f4084880 con 0x7f78000c7180 2024-09-13T17:01:22.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.249+0000 7f78052d5640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77e4066690 0x7f77e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.249+0000 7f78052d5640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77e4066690 0x7f77e4068b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f77fc006fd0 tx=0x7f77fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:22.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.257+0000 7f77f1ffb640 1 -- 172.21.15.39:0/2562928232 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f77f40513d0 con 0x7f78000c7180 2024-09-13T17:01:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8236af7640 1 Processor -- start 2024-09-13T17:01:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8236af7640 1 -- start start 2024-09-13T17:01:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8236af7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8230157530 0x7f82300c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8236af7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 0x7f8230071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8236af7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f823006f840 0x7f82300a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8236af7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823006be40 con 0x7f8230157530 2024-09-13T17:01:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8236af7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82300c3d60 con 0x7f823006f840 2024-09-13T17:01:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8236af7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823006b960 con 0x7f8230157910 2024-09-13T17:01:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8234893640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8230157530 0x7f82300c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8234893640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8230157530 0x7f82300c2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47616/0 (socket says 172.21.15.39:47616) 2024-09-13T17:01:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8234893640 1 -- 172.21.15.39:0/3026054559 learned_addr learned my addr 172.21.15.39:0/3026054559 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f822ffff640 1 --2- 172.21.15.39:0/3026054559 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 0x7f8230071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8235094640 1 --2- 172.21.15.39:0/3026054559 >> v2:172.21.15.84:3300/0 conn(0x7f823006f840 0x7f82300a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8234893640 1 -- 172.21.15.39:0/3026054559 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 msgr2=0x7f8230071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8234893640 1 --2- 172.21.15.39:0/3026054559 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 0x7f8230071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8234893640 1 -- 172.21.15.39:0/3026054559 >> v2:172.21.15.84:3300/0 conn(0x7f823006f840 msgr2=0x7f82300a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8234893640 1 --2- 172.21.15.39:0/3026054559 >> v2:172.21.15.84:3300/0 conn(0x7f823006f840 0x7f82300a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8234893640 1 -- 172.21.15.39:0/3026054559 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300c4000 con 0x7f8230157530 2024-09-13T17:01:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.269+0000 7f8235094640 1 --2- 172.21.15.39:0/3026054559 >> v2:172.21.15.84:3300/0 conn(0x7f823006f840 0x7f82300a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8234893640 1 --2- 172.21.15.39:0/3026054559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 0x7f82300c2400 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f823006fd20 tx=0x7f822401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f1830bafdf6af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f822f7fe640 1 -- 172.21.15.39:0/3026054559 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82240a89d0 con 0x7f8230157530 2024-09-13T17:01:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f822f7fe640 1 -- 172.21.15.39:0/3026054559 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82240a8b30 con 0x7f8230157530 2024-09-13T17:01:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f822f7fe640 1 -- 172.21.15.39:0/3026054559 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82240b0440 con 0x7f8230157530 2024-09-13T17:01:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 -- 172.21.15.39:0/3026054559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 msgr2=0x7f82300c2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 --2- 172.21.15.39:0/3026054559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 0x7f82300c2400 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f823006fd20 tx=0x7f822401ed10 comp rx=0 tx=0).stop 2024-09-13T17:01:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 -- 172.21.15.39:0/3026054559 shutdown_connections 2024-09-13T17:01:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 --2- 172.21.15.39:0/3026054559 >> v2:172.21.15.84:3300/0 conn(0x7f823006f840 0x7f82300a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 --2- 172.21.15.39:0/3026054559 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 0x7f8230071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 --2- 172.21.15.39:0/3026054559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 0x7f82300c2400 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 -- 172.21.15.39:0/3026054559 >> 172.21.15.39:0/3026054559 conn(0x7f82300a97b0 msgr2=0x7f82300a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 -- 172.21.15.39:0/3026054559 shutdown_connections 2024-09-13T17:01:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 -- 172.21.15.39:0/3026054559 wait complete. 2024-09-13T17:01:22.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 Processor -- start 2024-09-13T17:01:22.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.273+0000 7f8236af7640 1 -- start start 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8236af7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f823006f840 0x7f82301ecac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8236af7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 0x7f82301ed000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8236af7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 0x7f823014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8236af7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300acbc0 con 0x7f8230157530 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8236af7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82301ed540 con 0x7f823006f840 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8236af7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8230151f70 con 0x7f8230157910 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 0x7f82301ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 0x7f82301ed000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47618/0 (socket says 172.21.15.39:47618) 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822ffff640 1 -- 172.21.15.39:0/3648283419 learned_addr learned my addr 172.21.15.39:0/3648283419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8234893640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f823006f840 0x7f82301ecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8235094640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 0x7f823014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822ffff640 1 -- 172.21.15.39:0/3648283419 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 msgr2=0x7f823014deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822ffff640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 0x7f823014deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822ffff640 1 -- 172.21.15.39:0/3648283419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f823006f840 msgr2=0x7f82301ecac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822ffff640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f823006f840 0x7f82301ecac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822ffff640 1 -- 172.21.15.39:0/3648283419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240094d0 con 0x7f8230157530 2024-09-13T17:01:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822ffff640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 0x7f82301ed000 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f822000c7f0 tx=0x7f822000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822d7fa640 1 -- 172.21.15.39:0/3648283419 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220004230 con 0x7f8230157530 2024-09-13T17:01:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822d7fa640 1 -- 172.21.15.39:0/3648283419 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8220004390 con 0x7f8230157530 2024-09-13T17:01:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822d7fa640 1 -- 172.21.15.39:0/3648283419 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220003860 con 0x7f8230157530 2024-09-13T17:01:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8230152250 con 0x7f8230157530 2024-09-13T17:01:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8230150d20 con 0x7f8230157530 2024-09-13T17:01:22.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82300a6ad0 con 0x7f8230157530 2024-09-13T17:01:22.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822d7fa640 1 -- 172.21.15.39:0/3648283419 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8220003bd0 con 0x7f8230157530 2024-09-13T17:01:22.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822d7fa640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f821c0666c0 0x7f821c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:22.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8234893640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f821c0666c0 0x7f821c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:22.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f822d7fa640 1 -- 172.21.15.39:0/3648283419 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7f8220084a40 con 0x7f8230157530 2024-09-13T17:01:22.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.277+0000 7f8234893640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f821c0666c0 0x7f821c068b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82240b2000 tx=0x7f8224008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:22.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.281+0000 7f822d7fa640 1 -- 172.21.15.39:0/3648283419 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8220086e60 con 0x7f8230157530 2024-09-13T17:01:22.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.449+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7800066b00 con 0x7f78000c7180 2024-09-13T17:01:22.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.449+0000 7f77f1ffb640 1 -- 172.21.15.39:0/2562928232 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f7800066b00 con 0x7f78000c7180 2024-09-13T17:01:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:22.449 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:22.449 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:22.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77e4066690 msgr2=0x7f77e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77e4066690 0x7f77e4068b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f77fc006fd0 tx=0x7f77fc008040 comp rx=0 tx=0).stop 2024-09-13T17:01:22.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c7180 msgr2=0x7f78000c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c7180 0x7f78000c49f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f77f400c450 tx=0x7f77f400c910 comp rx=0 tx=0).stop 2024-09-13T17:01:22.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 shutdown_connections 2024-09-13T17:01:22.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f77e4066690 0x7f77e4068b50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f780014f340 0x7f7800148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78000c7180 0x7f78000c49f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 --2- 172.21.15.39:0/2562928232 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78000c26c0 0x7f78000c14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 >> 172.21.15.39:0/2562928232 conn(0x7f78000a96b0 msgr2=0x7f78000a8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:22.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 shutdown_connections 2024-09-13T17:01:22.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.453+0000 7f7807539640 1 -- 172.21.15.39:0/2562928232 wait complete. 2024-09-13T17:01:22.467 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:01:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.477+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8230151650 con 0x7f8230157530 2024-09-13T17:01:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.477+0000 7f822d7fa640 1 -- 172.21.15.39:0/3648283419 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5929 (secure 0 0 0) 0x7f822001e020 con 0x7f8230157530 2024-09-13T17:01:22.477 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:22.477 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5938,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T16:59:57.987543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"a","rank":0,"incarnation":81,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6835/4246622083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4246622083},{"type":"v1","addr":"172.21.15.39:6835","nonce":4246622083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:00:05.058924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5423":{"gid":5423,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6833/2638959497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2638959497},{"type":"v1","addr":"172.21.15.84:6833","nonce":2638959497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:22.477 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-13T17:01:22.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f821c0666c0 msgr2=0x7f821c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f821c0666c0 0x7f821c068b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82240b2000 tx=0x7f8224008040 comp rx=0 tx=0).stop 2024-09-13T17:01:22.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 msgr2=0x7f82301ed000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:22.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 0x7f82301ed000 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f822000c7f0 tx=0x7f822000ccb0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 shutdown_connections 2024-09-13T17:01:22.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f821c0666c0 0x7f821c068b80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8230157910 0x7f823014deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8230157530 0x7f82301ed000 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 --2- 172.21.15.39:0/3648283419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f823006f840 0x7f82301ecac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:22.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 >> 172.21.15.39:0/3648283419 conn(0x7f82300a97b0 msgr2=0x7f823006a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:22.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 shutdown_connections 2024-09-13T17:01:22.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:22.481+0000 7f8236af7640 1 -- 172.21.15.39:0/3648283419 wait complete. 2024-09-13T17:01:22.493 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:01:24.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:24.495 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:24.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feeed3ba640 1 Processor -- start 2024-09-13T17:01:24.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feeed3ba640 1 -- start start 2024-09-13T17:01:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feeed3ba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feee80c7960 0x7feee8155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feeed3ba640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee80c6000 0x7feee80c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feeed3ba640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee80c5e50 con 0x7feee806fa50 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feeed3ba640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee806ff50 con 0x7feee80c7960 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feeed3ba640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee8072000 con 0x7feee80c6000 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee67fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7feee806fa50 0x7feee8155f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44162/0 (socket says 172.21.15.39:44162) 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee67fc640 1 -- 172.21.15.39:0/4133968312 learned_addr learned my addr 172.21.15.39:0/4133968312 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee77fe640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee80c6000 0x7feee80c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee6ffd640 1 --2- 172.21.15.39:0/4133968312 >> v2:172.21.15.84:3300/0 conn(0x7feee80c7960 0x7feee8155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569945684 0 0) 0x7feee80c5e50 con 0x7feee806fa50 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed4003530 con 0x7feee806fa50 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3109720279 0 0) 0x7feed4003530 con 0x7feee806fa50 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee80c6000 msgr2=0x7feee80c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee80c6000 0x7feee80c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 >> v2:172.21.15.84:3300/0 conn(0x7feee80c7960 msgr2=0x7feee8155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 --2- 172.21.15.39:0/4133968312 >> v2:172.21.15.84:3300/0 conn(0x7feee80c7960 0x7feee8155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80722e0 con 0x7feee806fa50 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7feed0001680 con 0x7feee806fa50 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 >> v1:172.21.15.39:6789/0 conn(0x7feee806fa50 legacy=0x7feee8155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee80c6000 0x7feed4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed4003f00 0x7feed40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feed4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee806ff50 con 0x7feed4004d10 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee8072000 con 0x7feed4003f00 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed400b9e0 con 0x7feee80c6000 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee77fe640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feed4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee6ffd640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee80c6000 0x7feed4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee77fe640 1 -- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee80c6000 msgr2=0x7feed4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee77fe640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee80c6000 0x7feed4003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee77fe640 1 -- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed4003f00 msgr2=0x7feed40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee77fe640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed4003f00 0x7feed40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.605+0000 7feee77fe640 1 -- 172.21.15.39:0/4133968312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed4003530 con 0x7feed4004d10 2024-09-13T17:01:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee77fe640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feed4005190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feed8009b10 tx=0x7feed801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642a402a130db921 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed80a5660 con 0x7feed4004d10 2024-09-13T17:01:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed80a57c0 con 0x7feed4004d10 2024-09-13T17:01:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee5ffb640 1 -- 172.21.15.39:0/4133968312 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed80a5b70 con 0x7feed4004d10 2024-09-13T17:01:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 -- 172.21.15.39:0/4133968312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 msgr2=0x7feed4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 --2- 172.21.15.39:0/4133968312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feed4005190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feed8009b10 tx=0x7feed801ca70 comp rx=0 tx=0).stop 2024-09-13T17:01:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee77fe640 1 -- 172.21.15.39:0/4133968312 reap_dead start 2024-09-13T17:01:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 -- 172.21.15.39:0/4133968312 shutdown_connections 2024-09-13T17:01:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 -- 172.21.15.39:0/4133968312 >> 172.21.15.39:0/4133968312 conn(0x7feee806c020 msgr2=0x7feee806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 -- 172.21.15.39:0/4133968312 shutdown_connections 2024-09-13T17:01:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 -- 172.21.15.39:0/4133968312 wait complete. 2024-09-13T17:01:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 Processor -- start 2024-09-13T17:01:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 -- start start 2024-09-13T17:01:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feee8150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee8151040 0x7feee80c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee814d1e0 0x7feee80a8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80c06a0 con 0x7feed4004d10 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80bf0a0 con 0x7feee8151040 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feeed3ba640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee80bd610 con 0x7feee814d1e0 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feee8150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feee8150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47628/0 (socket says 172.21.15.39:47628) 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee6ffd640 1 -- 172.21.15.39:0/2192922473 learned_addr learned my addr 172.21.15.39:0/2192922473 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee67fc640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee8151040 0x7feee80c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.609+0000 7feee77fe640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee814d1e0 0x7feee80a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feee6ffd640 1 -- 172.21.15.39:0/2192922473 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee814d1e0 msgr2=0x7feee80a8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feee6ffd640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee814d1e0 0x7feee80a8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feee6ffd640 1 -- 172.21.15.39:0/2192922473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee8151040 msgr2=0x7feee80c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feee6ffd640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee8151040 0x7feee80c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feee6ffd640 1 -- 172.21.15.39:0/2192922473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80bedc0 con 0x7feed4004d10 2024-09-13T17:01:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feee6ffd640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feee8150b00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feee000f3d0 tx=0x7feee000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feedf7fe640 1 -- 172.21.15.39:0/2192922473 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0016620 con 0x7feed4004d10 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feedf7fe640 1 -- 172.21.15.39:0/2192922473 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee00167a0 con 0x7feed4004d10 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee80bd330 con 0x7feed4004d10 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feedf7fe640 1 -- 172.21.15.39:0/2192922473 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0016b50 con 0x7feed4004d10 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee80aa0e0 con 0x7feed4004d10 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee80c0460 con 0x7feed4004d10 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feedf7fe640 1 -- 172.21.15.39:0/2192922473 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feee0016cb0 con 0x7feed4004d10 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feedf7fe640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feec80666c0 0x7feec8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feedf7fe640 1 -- 172.21.15.39:0/2192922473 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7feee0085920 con 0x7feed4004d10 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feee67fc640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feec80666c0 0x7feec8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.613+0000 7feee67fc640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feec80666c0 0x7feec8068b80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feee814cc60 tx=0x7feed0001500 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:24.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.621+0000 7feedf7fe640 1 -- 172.21.15.39:0/2192922473 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feee0052470 con 0x7feed4004d10 2024-09-13T17:01:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff691e29640 1 Processor -- start 2024-09-13T17:01:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff691e29640 1 -- start start 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff691e29640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c154bb0 0x7ff68c0c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff691e29640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68c153e30 con 0x7ff68c0c2370 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff691e29640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68c0a7000 con 0x7ff68c0c5870 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff691e29640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0720f0 con 0x7ff68c154bb0 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68b7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff68c0c2370 0x7ff68c0c0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44170/0 (socket says 172.21.15.39:44170) 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68b7fe640 1 -- 172.21.15.39:0/3988310805 learned_addr learned my addr 172.21.15.39:0/3988310805 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68affd640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c154bb0 0x7ff68c0c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206998752 0 0) 0x7ff68c153e30 con 0x7ff68c0c2370 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff674003530 con 0x7ff68c0c2370 2024-09-13T17:01:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397407321 0 0) 0x7ff674003530 con 0x7ff68c0c2370 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c154bb0 msgr2=0x7ff68c0c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c154bb0 0x7ff68c0c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 >> v1:172.21.15.84:6789/0 conn(0x7ff68c0c5870 legacy=0x7ff68c15e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c1540d0 con 0x7ff68c0c2370 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7ff67c001680 con 0x7ff68c0c2370 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 >> v1:172.21.15.39:6789/0 conn(0x7ff68c0c2370 legacy=0x7ff68c0c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6740039b0 0x7ff674003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c154bb0 0x7ff6740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff674004d10 0x7ff674005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0720f0 con 0x7ff674004d10 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67400bac0 con 0x7ff6740039b0 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67400bc80 con 0x7ff68c154bb0 2024-09-13T17:01:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68bfff640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff674004d10 0x7ff674005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68b7fe640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6740039b0 0x7ff674003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68affd640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c154bb0 0x7ff6740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68bfff640 1 -- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c154bb0 msgr2=0x7ff6740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68bfff640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c154bb0 0x7ff6740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68bfff640 1 -- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6740039b0 msgr2=0x7ff674003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68bfff640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6740039b0 0x7ff674003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.637+0000 7ff68bfff640 1 -- 172.21.15.39:0/3988310805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674003530 con 0x7ff674004d10 2024-09-13T17:01:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff68bfff640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff674004d10 0x7ff674005190 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff684001e70 tx=0x7ff68401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7d963d4030ae38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684003b80 con 0x7ff674004d10 2024-09-13T17:01:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff684003ce0 con 0x7ff674004d10 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff68a7fc640 1 -- 172.21.15.39:0/3988310805 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6840052c0 con 0x7ff674004d10 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 -- 172.21.15.39:0/3988310805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff674004d10 msgr2=0x7ff674005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 --2- 172.21.15.39:0/3988310805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff674004d10 0x7ff674005190 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff684001e70 tx=0x7ff68401c780 comp rx=0 tx=0).stop 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff68bfff640 1 -- 172.21.15.39:0/3988310805 reap_dead start 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 -- 172.21.15.39:0/3988310805 shutdown_connections 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 -- 172.21.15.39:0/3988310805 >> 172.21.15.39:0/3988310805 conn(0x7ff68c0a9720 msgr2=0x7ff68c0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 -- 172.21.15.39:0/3988310805 shutdown_connections 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 -- 172.21.15.39:0/3988310805 wait complete. 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 Processor -- start 2024-09-13T17:01:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 -- start start 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c150bf0 0x7ff68c150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c1df660 0x7ff68c1dfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff68c0ac2d0 0x7ff68c0ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0ad960 con 0x7ff68c150bf0 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0ab240 con 0x7ff68c0ac2d0 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff691e29640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0ab660 con 0x7ff68c1df660 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff68b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c150bf0 0x7ff68c150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff68b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c150bf0 0x7ff68c150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47646/0 (socket says 172.21.15.39:47646) 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.641+0000 7ff68b7fe640 1 -- 172.21.15.39:0/3728616129 learned_addr learned my addr 172.21.15.39:0/3728616129 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68affd640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c1df660 0x7ff68c1dfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68bfff640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff68c0ac2d0 0x7ff68c0ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68b7fe640 1 -- 172.21.15.39:0/3728616129 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c1df660 msgr2=0x7ff68c1dfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68b7fe640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c1df660 0x7ff68c1dfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68b7fe640 1 -- 172.21.15.39:0/3728616129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff68c0ac2d0 msgr2=0x7ff68c0ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68b7fe640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff68c0ac2d0 0x7ff68c0ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68b7fe640 1 -- 172.21.15.39:0/3728616129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0aaea0 con 0x7ff68c150bf0 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68bfff640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff68c0ac2d0 0x7ff68c0ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68b7fe640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c150bf0 0x7ff68c150fd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0017b0 tx=0x7ff67c012900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff673fff640 1 -- 172.21.15.39:0/3728616129 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c014810 con 0x7ff68c150bf0 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff673fff640 1 -- 172.21.15.39:0/3728616129 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c014970 con 0x7ff68c150bf0 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68c1ea9f0 con 0x7ff68c150bf0 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff673fff640 1 -- 172.21.15.39:0/3728616129 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c014d20 con 0x7ff68c150bf0 2024-09-13T17:01:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c151510 con 0x7ff68c150bf0 2024-09-13T17:01:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68c0c2a70 con 0x7ff68c150bf0 2024-09-13T17:01:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff673fff640 1 -- 172.21.15.39:0/3728616129 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff67c0153e0 con 0x7ff68c150bf0 2024-09-13T17:01:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff673fff640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff678066690 0x7ff678068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff673fff640 1 -- 172.21.15.39:0/3728616129 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7927+0+0 (secure 0 0 0) 0x7ff67c0876c0 con 0x7ff68c150bf0 2024-09-13T17:01:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68affd640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff678066690 0x7ff678068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.645+0000 7ff68affd640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff678066690 0x7ff678068b50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0c1610 tx=0x7ff680008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:24.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.649+0000 7ff673fff640 1 -- 172.21.15.39:0/3728616129 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff67c054210 con 0x7ff68c150bf0 2024-09-13T17:01:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.837+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feee8155a20 con 0x7feed4004d10 2024-09-13T17:01:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feedf7fe640 1 -- 172.21.15.39:0/2192922473 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4416 (secure 0 0 0) 0x7feee0055a90 con 0x7feed4004d10 2024-09-13T17:01:24.839 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:24.839 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:24.710080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:24.710080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 91 2024-09-13T17:01:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feec80666c0 msgr2=0x7feec8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feec80666c0 0x7feec8068b80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feee814cc60 tx=0x7feed0001500 comp rx=0 tx=0).stop 2024-09-13T17:01:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 msgr2=0x7feee8150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feee8150b00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feee000f3d0 tx=0x7feee000f890 comp rx=0 tx=0).stop 2024-09-13T17:01:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 shutdown_connections 2024-09-13T17:01:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feec80666c0 0x7feec8068b80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feee8151040 0x7feee80c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feee814d1e0 0x7feee80a8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 --2- 172.21.15.39:0/2192922473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feed4004d10 0x7feee8150b00 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 >> 172.21.15.39:0/2192922473 conn(0x7feee806c020 msgr2=0x7feee8155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 shutdown_connections 2024-09-13T17:01:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.841+0000 7feeed3ba640 1 -- 172.21.15.39:0/2192922473 wait complete. 2024-09-13T17:01:24.856 INFO:tasks.mds_thrash.fs.[a]:mds.a down, removed from mdsmap 2024-09-13T17:01:24.857 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:01:24.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:24.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.901+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff68c06be00 con 0x7ff68c150bf0 2024-09-13T17:01:24.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.901+0000 7ff673fff640 1 -- 172.21.15.39:0/3728616129 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4416 (secure 0 0 0) 0x7ff67c01c090 con 0x7ff68c150bf0 2024-09-13T17:01:24.901 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:24.901 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:24.710080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:24.710080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:24.901 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 91 2024-09-13T17:01:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff678066690 msgr2=0x7ff678068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff678066690 0x7ff678068b50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0c1610 tx=0x7ff680008040 comp rx=0 tx=0).stop 2024-09-13T17:01:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c150bf0 msgr2=0x7ff68c150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c150bf0 0x7ff68c150fd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0017b0 tx=0x7ff67c012900 comp rx=0 tx=0).stop 2024-09-13T17:01:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 shutdown_connections 2024-09-13T17:01:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff678066690 0x7ff678068b50 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0c1610 tx=0x7ff680008040 comp rx=0 tx=0).stop 2024-09-13T17:01:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff68c0ac2d0 0x7ff68c0ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff68c1df660 0x7ff68c1dfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 --2- 172.21.15.39:0/3728616129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c150bf0 0x7ff68c150fd0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 >> 172.21.15.39:0/3728616129 conn(0x7ff68c0a9720 msgr2=0x7ff68c0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 shutdown_connections 2024-09-13T17:01:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.905+0000 7ff691e29640 1 -- 172.21.15.39:0/3728616129 wait complete. 2024-09-13T17:01:24.920 INFO:tasks.mds_thrash.fs.[b]:mds.b down, removed from mdsmap 2024-09-13T17:01:24.920 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:01:24.920 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:24.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb74221d640 1 Processor -- start 2024-09-13T17:01:24.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb74221d640 1 -- start start 2024-09-13T17:01:24.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb74221d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb73c155910 0x7fb73c0c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb74221d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73c14f920 con 0x7fb73c0c1ab0 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb74221d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73c0c1780 con 0x7fb73c14f540 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb74221d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73c0709c0 con 0x7fb73c155910 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb73affd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb73c0c1ab0 0x7fb73c155200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44180/0 (socket says 172.21.15.39:44180) 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb73affd640 1 -- 172.21.15.39:0/1751636173 learned_addr learned my addr 172.21.15.39:0/1751636173 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb73bfff640 1 --2- 172.21.15.39:0/1751636173 >> v2:172.21.15.84:3300/0 conn(0x7fb73c155910 0x7fb73c0c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb73a7fc640 1 -- 172.21.15.39:0/1751636173 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085411732 0 0) 0x7fb73c14f920 con 0x7fb73c0c1ab0 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb73a7fc640 1 -- 172.21.15.39:0/1751636173 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb724003530 con 0x7fb73c0c1ab0 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb73a7fc640 1 -- 172.21.15.39:0/1751636173 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2309730470 0 0) 0x7fb724003530 con 0x7fb73c0c1ab0 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb73bfff640 1 -- 172.21.15.39:0/1751636173 >> v1:172.21.15.84:6789/0 conn(0x7fb73c14f540 legacy=0x7fb73c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb73bfff640 1 -- 172.21.15.39:0/1751636173 >> v1:172.21.15.39:6789/0 conn(0x7fb73c0c1ab0 legacy=0x7fb73c155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.965+0000 7fb73bfff640 1 -- 172.21.15.39:0/1751636173 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c071580 con 0x7fb73c155910 2024-09-13T17:01:24.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb73bfff640 1 --2- 172.21.15.39:0/1751636173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73c155910 0x7fb73c0c11a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb734009d90 tx=0x7fb73401c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f03ad1538a4254f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:24.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb73a7fc640 1 -- 172.21.15.39:0/1751636173 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730001680 con 0x7fb73c155910 2024-09-13T17:01:24.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb73a7fc640 1 -- 172.21.15.39:0/1751636173 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7340a5590 con 0x7fb73c155910 2024-09-13T17:01:24.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb73a7fc640 1 -- 172.21.15.39:0/1751636173 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340a5920 con 0x7fb73c155910 2024-09-13T17:01:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb74221d640 1 -- 172.21.15.39:0/1751636173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73c155910 msgr2=0x7fb73c0c11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb74221d640 1 --2- 172.21.15.39:0/1751636173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73c155910 0x7fb73c0c11a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb734009d90 tx=0x7fb73401c710 comp rx=0 tx=0).stop 2024-09-13T17:01:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb74221d640 1 -- 172.21.15.39:0/1751636173 shutdown_connections 2024-09-13T17:01:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb74221d640 1 --2- 172.21.15.39:0/1751636173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73c155910 0x7fb73c0c11a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb74221d640 1 -- 172.21.15.39:0/1751636173 >> 172.21.15.39:0/1751636173 conn(0x7fb73c0c32a0 msgr2=0x7fb73c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb74221d640 1 -- 172.21.15.39:0/1751636173 shutdown_connections 2024-09-13T17:01:24.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.969+0000 7fb74221d640 1 -- 172.21.15.39:0/1751636173 wait complete. 2024-09-13T17:01:24.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 Processor -- start 2024-09-13T17:01:24.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 -- start start 2024-09-13T17:01:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73c0c1ab0 0x7fb73c0bbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb73c14f540 0x7fb73c141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb73c155910 0x7fb73c1ed950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0ac090 con 0x7fb73c14f540 2024-09-13T17:01:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c149390 con 0x7fb73c0c1ab0 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c148dd0 con 0x7fb73c155910 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb73c14f540 0x7fb73c141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb73c14f540 0x7fb73c141ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47650/0 (socket says 172.21.15.39:47650) 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73affd640 1 -- 172.21.15.39:0/2292851258 learned_addr learned my addr 172.21.15.39:0/2292851258 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73b7fe640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73c0c1ab0 0x7fb73c0bbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73bfff640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb73c155910 0x7fb73c1ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73affd640 1 -- 172.21.15.39:0/2292851258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb73c155910 msgr2=0x7fb73c1ed950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73affd640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb73c155910 0x7fb73c1ed950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73affd640 1 -- 172.21.15.39:0/2292851258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73c0c1ab0 msgr2=0x7fb73c0bbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73affd640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73c0c1ab0 0x7fb73c0bbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73affd640 1 -- 172.21.15.39:0/2292851258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734009a70 con 0x7fb73c14f540 2024-09-13T17:01:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb73affd640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb73c14f540 0x7fb73c141ac0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb730007fd0 tx=0x7fb73000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb723fff640 1 -- 172.21.15.39:0/2292851258 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300072d0 con 0x7fb73c14f540 2024-09-13T17:01:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb723fff640 1 -- 172.21.15.39:0/2292851258 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb730007430 con 0x7fb73c14f540 2024-09-13T17:01:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c1490b0 con 0x7fb73c14f540 2024-09-13T17:01:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb723fff640 1 -- 172.21.15.39:0/2292851258 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300047f0 con 0x7fb73c14f540 2024-09-13T17:01:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c0bc420 con 0x7fb73c14f540 2024-09-13T17:01:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.973+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73c0619c0 con 0x7fb73c14f540 2024-09-13T17:01:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.977+0000 7fb723fff640 1 -- 172.21.15.39:0/2292851258 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb73000a640 con 0x7fb73c14f540 2024-09-13T17:01:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.977+0000 7fb723fff640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb728066690 0x7fb728068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.977+0000 7fb723fff640 1 -- 172.21.15.39:0/2292851258 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7fb730084320 con 0x7fb73c14f540 2024-09-13T17:01:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.977+0000 7fb73b7fe640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb728066690 0x7fb728068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.977+0000 7fb73b7fe640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb728066690 0x7fb728068b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb73c1e7f40 tx=0x7fb72c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:24.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:24.981+0000 7fb723fff640 1 -- 172.21.15.39:0/2292851258 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb730050f50 con 0x7fb73c14f540 2024-09-13T17:01:25.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0fa577a640 1 Processor -- start 2024-09-13T17:01:25.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0fa577a640 1 -- start start 2024-09-13T17:01:25.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0fa577a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0fa01556d0 0x7f0fa00c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0fa577a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fa00c7610 0x7f0fa0151b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0fa577a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa006feb0 con 0x7f0fa00a72b0 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0fa577a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa00aae50 con 0x7f0fa01556d0 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0fa577a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa0155ab0 con 0x7f0fa00c7610 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0fa00a72b0 0x7f0fa0071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44186/0 (socket says 172.21.15.39:44186) 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9f7fe640 1 -- 172.21.15.39:0/3522952835 learned_addr learned my addr 172.21.15.39:0/3522952835 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9effd640 1 --2- 172.21.15.39:0/3522952835 >> v2:172.21.15.39:3300/0 conn(0x7f0fa01556d0 0x7f0fa00c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9e7fc640 1 --2- 172.21.15.39:0/3522952835 >> v2:172.21.15.84:3300/0 conn(0x7f0fa00c7610 0x7f0fa0151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9dffb640 1 -- 172.21.15.39:0/3522952835 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405693601 0 0) 0x7f0fa006feb0 con 0x7f0fa00a72b0 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9dffb640 1 -- 172.21.15.39:0/3522952835 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f88003530 con 0x7f0fa00a72b0 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9effd640 1 -- 172.21.15.39:0/3522952835 >> v2:172.21.15.84:3300/0 conn(0x7f0fa00c7610 msgr2=0x7f0fa0151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9effd640 1 --2- 172.21.15.39:0/3522952835 >> v2:172.21.15.84:3300/0 conn(0x7f0fa00c7610 0x7f0fa0151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9effd640 1 -- 172.21.15.39:0/3522952835 >> v1:172.21.15.39:6789/0 conn(0x7f0fa00a72b0 legacy=0x7f0fa0071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:25.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.073+0000 7f0f9effd640 1 -- 172.21.15.39:0/3522952835 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa00c3c20 con 0x7f0fa01556d0 2024-09-13T17:01:25.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9effd640 1 --2- 172.21.15.39:0/3522952835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa01556d0 0x7f0fa00c36e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00ab0c0 tx=0x7f0f9401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d578e9bf1493b9ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:25.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9dffb640 1 -- 172.21.15.39:0/3522952835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f940ab600 con 0x7f0fa01556d0 2024-09-13T17:01:25.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9dffb640 1 -- 172.21.15.39:0/3522952835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f940a7050 con 0x7f0fa01556d0 2024-09-13T17:01:25.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9dffb640 1 -- 172.21.15.39:0/3522952835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f940ab980 con 0x7f0fa01556d0 2024-09-13T17:01:25.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 -- 172.21.15.39:0/3522952835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa01556d0 msgr2=0x7f0fa00c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 --2- 172.21.15.39:0/3522952835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa01556d0 0x7f0fa00c36e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00ab0c0 tx=0x7f0f9401c360 comp rx=0 tx=0).stop 2024-09-13T17:01:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 -- 172.21.15.39:0/3522952835 shutdown_connections 2024-09-13T17:01:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 --2- 172.21.15.39:0/3522952835 >> v2:172.21.15.84:3300/0 conn(0x7f0fa00c7610 0x7f0fa0151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 --2- 172.21.15.39:0/3522952835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa01556d0 0x7f0fa00c36e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 -- 172.21.15.39:0/3522952835 >> 172.21.15.39:0/3522952835 conn(0x7f0fa00a97b0 msgr2=0x7f0fa00a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 -- 172.21.15.39:0/3522952835 shutdown_connections 2024-09-13T17:01:25.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 -- 172.21.15.39:0/3522952835 wait complete. 2024-09-13T17:01:25.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 Processor -- start 2024-09-13T17:01:25.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 -- start start 2024-09-13T17:01:25.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0fa00a72b0 0x7f0fa01ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fa01556d0 0x7f0fa00abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa006f150 0x7f0fa01efab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa01ef620 con 0x7f0fa006f150 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa01ed020 con 0x7f0fa01556d0 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0fa577a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa006a750 con 0x7f0fa00a72b0 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa006f150 0x7f0fa01efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa006f150 0x7f0fa01efab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47662/0 (socket says 172.21.15.39:47662) 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9f7fe640 1 -- 172.21.15.39:0/1361506566 learned_addr learned my addr 172.21.15.39:0/1361506566 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9e7fc640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fa01556d0 0x7f0fa00abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:25.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9effd640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0fa00a72b0 0x7f0fa01ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9f7fe640 1 -- 172.21.15.39:0/1361506566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0fa00a72b0 msgr2=0x7f0fa01ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9f7fe640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0fa00a72b0 0x7f0fa01ecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9f7fe640 1 -- 172.21.15.39:0/1361506566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fa01556d0 msgr2=0x7f0fa00abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9f7fe640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fa01556d0 0x7f0fa00abc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9f7fe640 1 -- 172.21.15.39:0/1361506566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f94009420 con 0x7f0fa006f150 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.077+0000 7f0f9effd640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0fa00a72b0 0x7f0fa01ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0f9f7fe640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa006f150 0x7f0fa01efab0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800caa0 tx=0x7f0f9800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0f877fe640 1 -- 172.21.15.39:0/1361506566 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800ac00 con 0x7f0fa006f150 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0f877fe640 1 -- 172.21.15.39:0/1361506566 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9800ad60 con 0x7f0fa006f150 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa0153a90 con 0x7f0fa006f150 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0f877fe640 1 -- 172.21.15.39:0/1361506566 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98012620 con 0x7f0fa006f150 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa0154050 con 0x7f0fa006f150 2024-09-13T17:01:25.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa0066ac0 con 0x7f0fa006f150 2024-09-13T17:01:25.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0f877fe640 1 -- 172.21.15.39:0/1361506566 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f98007070 con 0x7f0fa006f150 2024-09-13T17:01:25.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0f877fe640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0f8c066690 0x7f0f8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:25.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0f877fe640 1 -- 172.21.15.39:0/1361506566 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7f0f98084ad0 con 0x7f0fa006f150 2024-09-13T17:01:25.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0f9effd640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0f8c066690 0x7f0f8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:25.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.081+0000 7f0f9effd640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0f8c066690 0x7f0f8c068b50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f940ad000 tx=0x7f0f94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:25.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.085+0000 7f0f877fe640 1 -- 172.21.15.39:0/1361506566 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0f980514e0 con 0x7f0fa006f150 2024-09-13T17:01:25.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.181+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb73c0c51b0 con 0x7fb73c14f540 2024-09-13T17:01:25.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb723fff640 1 -- 172.21.15.39:0/2292851258 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4416 (secure 0 0 0) 0x7fb730018070 con 0x7fb73c14f540 2024-09-13T17:01:25.182 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:25.182 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:24.710080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:24.710080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:25.183 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 91 2024-09-13T17:01:25.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb728066690 msgr2=0x7fb728068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb728066690 0x7fb728068b50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb73c1e7f40 tx=0x7fb72c009040 comp rx=0 tx=0).stop 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb73c14f540 msgr2=0x7fb73c141ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb73c14f540 0x7fb73c141ac0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb730007fd0 tx=0x7fb73000cbe0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 shutdown_connections 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb728066690 0x7fb728068b50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb73c155910 0x7fb73c1ed950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb73c14f540 0x7fb73c141ac0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 --2- 172.21.15.39:0/2292851258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb73c0c1ab0 0x7fb73c0bbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 >> 172.21.15.39:0/2292851258 conn(0x7fb73c0c32a0 msgr2=0x7fb73c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 shutdown_connections 2024-09-13T17:01:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.185+0000 7fb74221d640 1 -- 172.21.15.39:0/2292851258 wait complete. 2024-09-13T17:01:25.201 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 91, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 91, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_4226": { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 91, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:01:24.710080+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4226 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 91, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5938": { "addr": "172.21.15.84:6835/828722052", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 828722052, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 828722052, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5938, "incarnation": 91, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:01:24.710080+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5938 } } } ], "standbys": [] } 2024-09-13T17:01:25.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.273+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fa0151b80 con 0x7f0fa006f150 2024-09-13T17:01:25.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.273+0000 7f0f877fe640 1 -- 172.21.15.39:0/1361506566 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4416 (secure 0 0 0) 0x7f0f98017090 con 0x7f0fa006f150 2024-09-13T17:01:25.274 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:25.274 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:24.710080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:24.710080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:25.275 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 91 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0f8c066690 msgr2=0x7f0f8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0f8c066690 0x7f0f8c068b50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0f940ad000 tx=0x7f0f94008040 comp rx=0 tx=0).stop 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa006f150 msgr2=0x7f0fa01efab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa006f150 0x7f0fa01efab0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800caa0 tx=0x7f0f9800cf60 comp rx=0 tx=0).stop 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 shutdown_connections 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0f8c066690 0x7f0f8c068b50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fa01556d0 0x7f0fa00abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa006f150 0x7f0fa01efab0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 --2- 172.21.15.39:0/1361506566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0fa00a72b0 0x7f0fa01ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 >> 172.21.15.39:0/1361506566 conn(0x7f0fa00a97b0 msgr2=0x7f0fa00ac9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 shutdown_connections 2024-09-13T17:01:25.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:25.277+0000 7f0fa577a640 1 -- 172.21.15.39:0/1361506566 wait complete. 2024-09-13T17:01:25.292 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 91, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 91, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_4226": { "addr": "172.21.15.39:6837/835951775", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 835951775, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 835951775, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4226, "incarnation": 91, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:01:24.710080+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4226 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 91, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5938": { "addr": "172.21.15.84:6835/828722052", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 828722052, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 828722052, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5938, "incarnation": 91, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:01:24.710080+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5938 } } } ], "standbys": [] } 2024-09-13T17:01:27.201 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:27.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:27.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.333+0000 7f65f0092640 1 Processor -- start 2024-09-13T17:01:27.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65f0092640 1 -- start start 2024-09-13T17:01:27.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65f0092640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f65e80c2ee0 0x7f65e80c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65f0092640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e806ddf0 con 0x7f65e80c6270 2024-09-13T17:01:27.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65f0092640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e80c4710 con 0x7f65e8065ee0 2024-09-13T17:01:27.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65f0092640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e80c49d0 con 0x7f65e80c2ee0 2024-09-13T17:01:27.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65ee62f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f65e80c6270 0x7f65e80bfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44198/0 (socket says 172.21.15.39:44198) 2024-09-13T17:01:27.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65ee62f640 1 -- 172.21.15.39:0/3496173982 learned_addr learned my addr 172.21.15.39:0/3496173982 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:27.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65ed62d640 1 --2- 172.21.15.39:0/3496173982 >> v2:172.21.15.39:3300/0 conn(0x7f65e80c2ee0 0x7f65e80c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65ece2c640 1 -- 172.21.15.39:0/3496173982 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214979947 0 0) 0x7f65e806ddf0 con 0x7f65e80c6270 2024-09-13T17:01:27.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65ece2c640 1 -- 172.21.15.39:0/3496173982 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d4003530 con 0x7f65e80c6270 2024-09-13T17:01:27.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65ed62d640 1 -- 172.21.15.39:0/3496173982 >> v1:172.21.15.84:6789/0 conn(0x7f65e8065ee0 legacy=0x7f65e80662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65ed62d640 1 -- 172.21.15.39:0/3496173982 >> v1:172.21.15.39:6789/0 conn(0x7f65e80c6270 legacy=0x7f65e80bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65ed62d640 1 -- 172.21.15.39:0/3496173982 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e8154880 con 0x7f65e80c2ee0 2024-09-13T17:01:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.337+0000 7f65ed62d640 1 --2- 172.21.15.39:0/3496173982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 0x7f65e80c2220 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0097c0 tx=0x7f65dc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c31fbc01a5c1fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ece2c640 1 -- 172.21.15.39:0/3496173982 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0a7070 con 0x7f65e80c2ee0 2024-09-13T17:01:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ece2c640 1 -- 172.21.15.39:0/3496173982 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e806e490 con 0x7f65e80c2ee0 2024-09-13T17:01:27.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ece2c640 1 -- 172.21.15.39:0/3496173982 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0ab650 con 0x7f65e80c2ee0 2024-09-13T17:01:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- 172.21.15.39:0/3496173982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 msgr2=0x7f65e80c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 --2- 172.21.15.39:0/3496173982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 0x7f65e80c2220 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0097c0 tx=0x7f65dc01c410 comp rx=0 tx=0).stop 2024-09-13T17:01:27.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- 172.21.15.39:0/3496173982 shutdown_connections 2024-09-13T17:01:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 --2- 172.21.15.39:0/3496173982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 0x7f65e80c2220 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- 172.21.15.39:0/3496173982 >> 172.21.15.39:0/3496173982 conn(0x7f65e80c3310 msgr2=0x7f65e80c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- 172.21.15.39:0/3496173982 shutdown_connections 2024-09-13T17:01:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- 172.21.15.39:0/3496173982 wait complete. 2024-09-13T17:01:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 Processor -- start 2024-09-13T17:01:27.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- start start 2024-09-13T17:01:27.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f65e8065ee0 0x7f65e80a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 0x7f65e81ec470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e80c6270 0x7f65e806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e814d000 con 0x7f65e80c2ee0 2024-09-13T17:01:27.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e81ec9b0 con 0x7f65e80c6270 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e81efdb0 con 0x7f65e8065ee0 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ed62d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 0x7f65e81ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ed62d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 0x7f65e81ec470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47682/0 (socket says 172.21.15.39:47682) 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ed62d640 1 -- 172.21.15.39:0/2258591094 learned_addr learned my addr 172.21.15.39:0/2258591094 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ee62f640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e80c6270 0x7f65e806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ede2e640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f65e8065ee0 0x7f65e80a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ed62d640 1 -- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f65e8065ee0 msgr2=0x7f65e80a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ed62d640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f65e8065ee0 0x7f65e80a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ed62d640 1 -- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e80c6270 msgr2=0x7f65e806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ed62d640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e80c6270 0x7f65e806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ed62d640 1 -- 172.21.15.39:0/2258591094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc009420 con 0x7f65e80c2ee0 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ede2e640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f65e8065ee0 0x7f65e80a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ee62f640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e80c6270 0x7f65e806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:27.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65ed62d640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 0x7f65e81ec470 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0ad040 tx=0x7f65dc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:27.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65da7fc640 1 -- 172.21.15.39:0/2258591094 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0bf610 con 0x7f65e80c2ee0 2024-09-13T17:01:27.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65da7fc640 1 -- 172.21.15.39:0/2258591094 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc0bf7b0 con 0x7f65e80c2ee0 2024-09-13T17:01:27.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65da7fc640 1 -- 172.21.15.39:0/2258591094 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0bfb60 con 0x7f65e80c2ee0 2024-09-13T17:01:27.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e80c05e0 con 0x7f65e80c2ee0 2024-09-13T17:01:27.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.341+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e81550c0 con 0x7f65e80c2ee0 2024-09-13T17:01:27.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.345+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e80c2220 con 0x7f65e80c2ee0 2024-09-13T17:01:27.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.345+0000 7f65da7fc640 1 -- 172.21.15.39:0/2258591094 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65dc0b5400 con 0x7f65e80c2ee0 2024-09-13T17:01:27.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.345+0000 7f65da7fc640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65cc066690 0x7f65cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.345+0000 7f65da7fc640 1 -- 172.21.15.39:0/2258591094 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7f65dc11f2f0 con 0x7f65e80c2ee0 2024-09-13T17:01:27.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.345+0000 7f65ede2e640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65cc066690 0x7f65cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.345+0000 7f65ede2e640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65cc066690 0x7f65cc068b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f65e0007fd0 tx=0x7f65e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:27.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.349+0000 7f65da7fc640 1 -- 172.21.15.39:0/2258591094 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65dc0ebe40 con 0x7f65e80c2ee0 2024-09-13T17:01:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.417+0000 7ff42e8cf640 1 Processor -- start 2024-09-13T17:01:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.417+0000 7ff42e8cf640 1 -- start start 2024-09-13T17:01:27.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.417+0000 7ff42e8cf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff4280c6290 0x7ff4281577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.417+0000 7ff42e8cf640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff428157330 con 0x7ff4280c0be0 2024-09-13T17:01:27.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.417+0000 7ff42e8cf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4280c4740 con 0x7ff4280c2690 2024-09-13T17:01:27.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.417+0000 7ff42e8cf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4280c4a20 con 0x7ff4280c6290 2024-09-13T17:01:27.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff42ce6c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff4280c0be0 0x7ff42815e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44210/0 (socket says 172.21.15.39:44210) 2024-09-13T17:01:27.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff42ce6c640 1 -- 172.21.15.39:0/1396560196 learned_addr learned my addr 172.21.15.39:0/1396560196 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:27.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff4277fe640 1 --2- 172.21.15.39:0/1396560196 >> v2:172.21.15.84:3300/0 conn(0x7ff4280c6290 0x7ff4281577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292806257 0 0) 0x7ff428157330 con 0x7ff4280c0be0 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff410003530 con 0x7ff4280c0be0 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856728158 0 0) 0x7ff4280c4740 con 0x7ff4280c2690 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff428157330 con 0x7ff4280c2690 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3604516828 0 0) 0x7ff410003530 con 0x7ff4280c0be0 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 >> v2:172.21.15.84:3300/0 conn(0x7ff4280c6290 msgr2=0x7ff4281577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 --2- 172.21.15.39:0/1396560196 >> v2:172.21.15.84:3300/0 conn(0x7ff4280c6290 0x7ff4281577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 >> v1:172.21.15.84:6789/0 conn(0x7ff4280c2690 legacy=0x7ff42806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff428154880 con 0x7ff4280c0be0 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7ff420001680 con 0x7ff4280c0be0 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 >> v1:172.21.15.39:6789/0 conn(0x7ff4280c0be0 legacy=0x7ff42815e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff410003b10 0x7ff410003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4100047d0 0x7ff410004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff410005190 0x7ff41000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4280c4a20 con 0x7ff410005190 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41000c0c0 con 0x7ff4100047d0 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41000c280 con 0x7ff410003b10 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff42ce6c640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff410005190 0x7ff41000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff4277fe640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4100047d0 0x7ff410004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff427fff640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff410003b10 0x7ff410003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff42ce6c640 1 -- 172.21.15.39:0/1396560196 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff410003b10 msgr2=0x7ff410003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff42ce6c640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff410003b10 0x7ff410003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff42ce6c640 1 -- 172.21.15.39:0/1396560196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4100047d0 msgr2=0x7ff410004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff42ce6c640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4100047d0 0x7ff410004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff42ce6c640 1 -- 172.21.15.39:0/1396560196 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410003530 con 0x7ff410005190 2024-09-13T17:01:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.421+0000 7ff4277fe640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4100047d0 0x7ff410004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:01:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42ce6c640 1 -- 172.21.15.39:0/1396560196 reap_dead start 2024-09-13T17:01:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42ce6c640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff410005190 0x7ff41000b990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff42000c160 tx=0x7ff42001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c920bc3fe3ce894c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff420005ae0 con 0x7ff410005190 2024-09-13T17:01:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff420005c60 con 0x7ff410005190 2024-09-13T17:01:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff426ffd640 1 -- 172.21.15.39:0/1396560196 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4200a58c0 con 0x7ff410005190 2024-09-13T17:01:27.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 -- 172.21.15.39:0/1396560196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff410005190 msgr2=0x7ff41000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff410005190 0x7ff41000b990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff42000c160 tx=0x7ff42001cbc0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 -- 172.21.15.39:0/1396560196 shutdown_connections 2024-09-13T17:01:27.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 --2- 172.21.15.39:0/1396560196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff410005190 0x7ff41000b990 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 -- 172.21.15.39:0/1396560196 >> 172.21.15.39:0/1396560196 conn(0x7ff4280c32a0 msgr2=0x7ff4280c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:27.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 -- 172.21.15.39:0/1396560196 shutdown_connections 2024-09-13T17:01:27.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 -- 172.21.15.39:0/1396560196 wait complete. 2024-09-13T17:01:27.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 Processor -- start 2024-09-13T17:01:27.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 -- start start 2024-09-13T17:01:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff410005190 0x7ff428152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff428071e10 0x7ff4281e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4281e9520 0x7ff42814dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4281e99a0 con 0x7ff4281e9520 2024-09-13T17:01:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff428072380 con 0x7ff428071e10 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42e8cf640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4281ec550 con 0x7ff410005190 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42ce6c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4281e9520 0x7ff42814dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42ce6c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4281e9520 0x7ff42814dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47706/0 (socket says 172.21.15.39:47706) 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.425+0000 7ff42ce6c640 1 -- 172.21.15.39:0/3332207880 learned_addr learned my addr 172.21.15.39:0/3332207880 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff427fff640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff410005190 0x7ff428152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff4277fe640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff428071e10 0x7ff4281e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff42ce6c640 1 -- 172.21.15.39:0/3332207880 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff410005190 msgr2=0x7ff428152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff42ce6c640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff410005190 0x7ff428152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff42ce6c640 1 -- 172.21.15.39:0/3332207880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff428071e10 msgr2=0x7ff4281e8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff42ce6c640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff428071e10 0x7ff4281e8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff42ce6c640 1 -- 172.21.15.39:0/3332207880 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4281ec830 con 0x7ff4281e9520 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff42ce6c640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4281e9520 0x7ff42814dcf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff420001ec0 tx=0x7ff410005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:27.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff424ff9640 1 -- 172.21.15.39:0/3332207880 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4200a5ac0 con 0x7ff4281e9520 2024-09-13T17:01:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff424ff9640 1 -- 172.21.15.39:0/3332207880 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4200a5c20 con 0x7ff4281e9520 2024-09-13T17:01:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4281ecb10 con 0x7ff4281e9520 2024-09-13T17:01:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff424ff9640 1 -- 172.21.15.39:0/3332207880 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4200070d0 con 0x7ff4281e9520 2024-09-13T17:01:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4280aaec0 con 0x7ff4281e9520 2024-09-13T17:01:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4280c4740 con 0x7ff4281e9520 2024-09-13T17:01:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff424ff9640 1 -- 172.21.15.39:0/3332207880 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff420007230 con 0x7ff4281e9520 2024-09-13T17:01:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff424ff9640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff414066690 0x7ff414068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff424ff9640 1 -- 172.21.15.39:0/3332207880 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7ff42011e6b0 con 0x7ff4281e9520 2024-09-13T17:01:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.429+0000 7ff427fff640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff414066690 0x7ff414068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.433+0000 7ff427fff640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff414066690 0x7ff414068b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff4180059b0 tx=0x7ff418009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:27.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.433+0000 7ff424ff9640 1 -- 172.21.15.39:0/3332207880 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff4200eb230 con 0x7ff4281e9520 2024-09-13T17:01:27.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.545+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e81efef0 con 0x7f65e80c2ee0 2024-09-13T17:01:27.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65da7fc640 1 -- 172.21.15.39:0/2258591094 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4419 (secure 0 0 0) 0x7f65dc0c4090 con 0x7f65e80c2ee0 2024-09-13T17:01:27.547 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:27.547 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:26.971995+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:rejoin","state_seq":205,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:26.971995+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:27.547 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 93 2024-09-13T17:01:27.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65cc066690 msgr2=0x7f65cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65cc066690 0x7f65cc068b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f65e0007fd0 tx=0x7f65e0009040 comp rx=0 tx=0).stop 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 msgr2=0x7f65e81ec470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 0x7f65e81ec470 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0ad040 tx=0x7f65dc01ac50 comp rx=0 tx=0).stop 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 shutdown_connections 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65cc066690 0x7f65cc068b50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e80c6270 0x7f65e806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e80c2ee0 0x7f65e81ec470 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 --2- 172.21.15.39:0/2258591094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f65e8065ee0 0x7f65e80a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 >> 172.21.15.39:0/2258591094 conn(0x7f65e80c3310 msgr2=0x7f65e80bfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 shutdown_connections 2024-09-13T17:01:27.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.549+0000 7f65f0092640 1 -- 172.21.15.39:0/2258591094 wait complete. 2024-09-13T17:01:27.566 INFO:tasks.mds_thrash.fs.[a]:no change 2024-09-13T17:01:27.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.625+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff428153480 con 0x7ff4281e9520 2024-09-13T17:01:27.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff424ff9640 1 -- 172.21.15.39:0/3332207880 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4419 (secure 0 0 0) 0x7ff4200b2070 con 0x7ff4281e9520 2024-09-13T17:01:27.627 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:27.627 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:26.971995+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:rejoin","state_seq":205,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:26.971995+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:27.627 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 93 2024-09-13T17:01:27.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff414066690 msgr2=0x7ff414068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff414066690 0x7ff414068b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff4180059b0 tx=0x7ff418009040 comp rx=0 tx=0).stop 2024-09-13T17:01:27.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4281e9520 msgr2=0x7ff42814dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:27.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4281e9520 0x7ff42814dcf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff420001ec0 tx=0x7ff410005800 comp rx=0 tx=0).stop 2024-09-13T17:01:27.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 shutdown_connections 2024-09-13T17:01:27.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff414066690 0x7ff414068b50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff428071e10 0x7ff4281e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4281e9520 0x7ff42814dcf0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 --2- 172.21.15.39:0/3332207880 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff410005190 0x7ff428152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:27.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 >> 172.21.15.39:0/3332207880 conn(0x7ff4280c32a0 msgr2=0x7ff42806a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:27.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 shutdown_connections 2024-09-13T17:01:27.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:27.629+0000 7ff42e8cf640 1 -- 172.21.15.39:0/3332207880 wait complete. 2024-09-13T17:01:27.645 INFO:tasks.mds_thrash.fs.[b]:no change 2024-09-13T17:01:28.355 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:01:28.358 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:01:29.168 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-13T17:01:29.169+0000 7f4de3d3f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:01:29.568 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:29.646 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff798e7b640 1 Processor -- start 2024-09-13T17:01:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff798e7b640 1 -- start start 2024-09-13T17:01:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff798e7b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff7940c2db0 0x7ff794061200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff798e7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff7940c2560 0x7ff79414e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff798e7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79406fb40 con 0x7ff79414e630 2024-09-13T17:01:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff798e7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff794071730 con 0x7ff7940c2560 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff798e7b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff794150da0 con 0x7ff7940c2db0 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff791d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff7940c2560 0x7ff79414e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff791d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff7940c2560 0x7ff79414e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47714/0 (socket says 172.21.15.39:47714) 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff791d9b640 1 -- 172.21.15.39:0/3849825878 learned_addr learned my addr 172.21.15.39:0/3849825878 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff79259c640 1 --2- 172.21.15.39:0/3849825878 >> v2:172.21.15.84:3300/0 conn(0x7ff7940c2db0 0x7ff794061200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff791d9b640 1 -- 172.21.15.39:0/3849825878 >> v2:172.21.15.84:3300/0 conn(0x7ff7940c2db0 msgr2=0x7ff794061200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff791d9b640 1 --2- 172.21.15.39:0/3849825878 >> v2:172.21.15.84:3300/0 conn(0x7ff7940c2db0 0x7ff794061200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff791d9b640 1 -- 172.21.15.39:0/3849825878 >> v1:172.21.15.39:6789/0 conn(0x7ff79414e630 legacy=0x7ff7941545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff79159a640 1 -- 172.21.15.39:0/3849825878 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807208427 0 0) 0x7ff79406fb40 con 0x7ff79414e630 2024-09-13T17:01:29.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.725+0000 7ff791d9b640 1 -- 172.21.15.39:0/3849825878 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79414ea60 con 0x7ff7940c2560 2024-09-13T17:01:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff791d9b640 1 --2- 172.21.15.39:0/3849825878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7940c2560 0x7ff79414e0f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff794067310 tx=0x7ff78801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c5ba0d19602c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff79159a640 1 -- 172.21.15.39:0/3849825878 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7880a79c0 con 0x7ff7940c2560 2024-09-13T17:01:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff79159a640 1 -- 172.21.15.39:0/3849825878 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7880a7b20 con 0x7ff7940c2560 2024-09-13T17:01:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff79159a640 1 -- 172.21.15.39:0/3849825878 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7880af440 con 0x7ff7940c2560 2024-09-13T17:01:29.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 -- 172.21.15.39:0/3849825878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7940c2560 msgr2=0x7ff79414e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:29.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 --2- 172.21.15.39:0/3849825878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7940c2560 0x7ff79414e0f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff794067310 tx=0x7ff78801ebf0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 -- 172.21.15.39:0/3849825878 shutdown_connections 2024-09-13T17:01:29.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 --2- 172.21.15.39:0/3849825878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7940c2560 0x7ff79414e0f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 --2- 172.21.15.39:0/3849825878 >> v2:172.21.15.84:3300/0 conn(0x7ff7940c2db0 0x7ff794061200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 -- 172.21.15.39:0/3849825878 >> 172.21.15.39:0/3849825878 conn(0x7ff794069e10 msgr2=0x7ff79406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:29.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 -- 172.21.15.39:0/3849825878 shutdown_connections 2024-09-13T17:01:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 -- 172.21.15.39:0/3849825878 wait complete. 2024-09-13T17:01:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 Processor -- start 2024-09-13T17:01:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.729+0000 7ff798e7b640 1 -- start start 2024-09-13T17:01:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff798e7b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7940c2560 0x7ff7941effe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff798e7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7940c2db0 0x7ff794146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff798e7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79414e630 0x7ff7941501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff798e7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79414c780 con 0x7ff79414e630 2024-09-13T17:01:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff798e7b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79414fd40 con 0x7ff7940c2db0 2024-09-13T17:01:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff798e7b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79414afe0 con 0x7ff7940c2560 2024-09-13T17:01:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff792d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79414e630 0x7ff7941501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff792d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79414e630 0x7ff7941501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47724/0 (socket says 172.21.15.39:47724) 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff792d9d640 1 -- 172.21.15.39:0/4071895520 learned_addr learned my addr 172.21.15.39:0/4071895520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff791d9b640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7940c2db0 0x7ff794146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff79259c640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7940c2560 0x7ff7941effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff792d9d640 1 -- 172.21.15.39:0/4071895520 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7940c2560 msgr2=0x7ff7941effe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff792d9d640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7940c2560 0x7ff7941effe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff792d9d640 1 -- 172.21.15.39:0/4071895520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7940c2db0 msgr2=0x7ff794146240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff792d9d640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7940c2db0 0x7ff794146240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff792d9d640 1 -- 172.21.15.39:0/4071895520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78800b770 con 0x7ff79414e630 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff79259c640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7940c2560 0x7ff7941effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff792d9d640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79414e630 0x7ff7941501d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff78c002240 tx=0x7ff78c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff782ffd640 1 -- 172.21.15.39:0/4071895520 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c003db0 con 0x7ff79414e630 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff782ffd640 1 -- 172.21.15.39:0/4071895520 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c00ce30 con 0x7ff79414e630 2024-09-13T17:01:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79414c1c0 con 0x7ff79414e630 2024-09-13T17:01:29.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff782ffd640 1 -- 172.21.15.39:0/4071895520 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c005270 con 0x7ff79414e630 2024-09-13T17:01:29.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7940aaca0 con 0x7ff79414e630 2024-09-13T17:01:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79414e0f0 con 0x7ff79414e630 2024-09-13T17:01:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff782ffd640 1 -- 172.21.15.39:0/4071895520 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff78c0053d0 con 0x7ff79414e630 2024-09-13T17:01:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff782ffd640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff774066690 0x7ff774068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff782ffd640 1 -- 172.21.15.39:0/4071895520 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7ff78c084220 con 0x7ff79414e630 2024-09-13T17:01:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.733+0000 7ff79259c640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff774066690 0x7ff774068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.737+0000 7ff79259c640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff774066690 0x7ff774068b50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff794072290 tx=0x7ff784008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:29.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.741+0000 7ff782ffd640 1 -- 172.21.15.39:0/4071895520 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff78c050d70 con 0x7ff79414e630 2024-09-13T17:01:29.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d2dace640 1 Processor -- start 2024-09-13T17:01:29.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d2dace640 1 -- start start 2024-09-13T17:01:29.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d2dace640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9d280c7800 0x7f9d28154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d2dace640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d280c5a30 con 0x7f9d28071dd0 2024-09-13T17:01:29.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d2dace640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d281576a0 con 0x7f9d28151500 2024-09-13T17:01:29.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d2dace640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d28157980 con 0x7f9d280c7800 2024-09-13T17:01:29.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d1ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9d28071dd0 0x7f9d28154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44232/0 (socket says 172.21.15.39:44232) 2024-09-13T17:01:29.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d1ffff640 1 -- 172.21.15.39:0/3737244758 learned_addr learned my addr 172.21.15.39:0/3737244758 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:29.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d27fff640 1 --2- 172.21.15.39:0/3737244758 >> v2:172.21.15.84:3300/0 conn(0x7f9d280c7800 0x7f9d28154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d26ffd640 1 -- 172.21.15.39:0/3737244758 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731030646 0 0) 0x7f9d280c5a30 con 0x7f9d28071dd0 2024-09-13T17:01:29.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d26ffd640 1 -- 172.21.15.39:0/3737244758 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d14003530 con 0x7f9d28071dd0 2024-09-13T17:01:29.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d27fff640 1 -- 172.21.15.39:0/3737244758 >> v1:172.21.15.84:6789/0 conn(0x7f9d28151500 legacy=0x7f9d280c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:29.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d26ffd640 1 -- 172.21.15.39:0/3737244758 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1550673751 0 0) 0x7f9d14003530 con 0x7f9d28071dd0 2024-09-13T17:01:29.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d27fff640 1 -- 172.21.15.39:0/3737244758 >> v1:172.21.15.39:6789/0 conn(0x7f9d28071dd0 legacy=0x7f9d28154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:29.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.793+0000 7f9d27fff640 1 -- 172.21.15.39:0/3737244758 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d28157c60 con 0x7f9d280c7800 2024-09-13T17:01:29.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d27fff640 1 --2- 172.21.15.39:0/3737244758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 0x7f9d28154780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009dc0 tx=0x7f9d2001c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=872dcfad0485e536 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:29.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d26ffd640 1 -- 172.21.15.39:0/3737244758 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d100016a0 con 0x7f9d280c7800 2024-09-13T17:01:29.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d26ffd640 1 -- 172.21.15.39:0/3737244758 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d200a5650 con 0x7f9d280c7800 2024-09-13T17:01:29.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d26ffd640 1 -- 172.21.15.39:0/3737244758 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200a59c0 con 0x7f9d280c7800 2024-09-13T17:01:29.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d2dace640 1 -- 172.21.15.39:0/3737244758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 msgr2=0x7f9d28154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:29.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d2dace640 1 --2- 172.21.15.39:0/3737244758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 0x7f9d28154780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009dc0 tx=0x7f9d2001c7b0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d2dace640 1 -- 172.21.15.39:0/3737244758 shutdown_connections 2024-09-13T17:01:29.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d2dace640 1 --2- 172.21.15.39:0/3737244758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 0x7f9d28154780 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d2dace640 1 -- 172.21.15.39:0/3737244758 >> 172.21.15.39:0/3737244758 conn(0x7f9d280ab530 msgr2=0x7f9d280ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:29.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d2dace640 1 -- 172.21.15.39:0/3737244758 shutdown_connections 2024-09-13T17:01:29.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d2dace640 1 -- 172.21.15.39:0/3737244758 wait complete. 2024-09-13T17:01:29.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d2dace640 1 Processor -- start 2024-09-13T17:01:29.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.797+0000 7f9d2dace640 1 -- start start 2024-09-13T17:01:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d2dace640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d28071dd0 0x7f9d28156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d2dace640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 0x7f9d28151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d2dace640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d28151500 0x7f9d28152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d2dace640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2814a410 con 0x7f9d28151500 2024-09-13T17:01:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d2dace640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2814a630 con 0x7f9d280c7800 2024-09-13T17:01:29.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d2dace640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d280ab1c0 con 0x7f9d28071dd0 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d27fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d28151500 0x7f9d28152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d27fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d28151500 0x7f9d28152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47726/0 (socket says 172.21.15.39:47726) 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d27fff640 1 -- 172.21.15.39:0/2482940255 learned_addr learned my addr 172.21.15.39:0/2482940255 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d277fe640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d28071dd0 0x7f9d28156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d1ffff640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 0x7f9d28151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d27fff640 1 -- 172.21.15.39:0/2482940255 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d28071dd0 msgr2=0x7f9d28156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d27fff640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d28071dd0 0x7f9d28156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d27fff640 1 -- 172.21.15.39:0/2482940255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 msgr2=0x7f9d28151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d27fff640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 0x7f9d28151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d27fff640 1 -- 172.21.15.39:0/2482940255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d20009a70 con 0x7f9d28151500 2024-09-13T17:01:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d1ffff640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 0x7f9d28151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:01:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d27fff640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d28151500 0x7f9d28152110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9d200193c0 tx=0x7f9d20019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d24ff9640 1 -- 172.21.15.39:0/2482940255 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200a59d0 con 0x7f9d28151500 2024-09-13T17:01:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d24ff9640 1 -- 172.21.15.39:0/2482940255 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d200a5b30 con 0x7f9d28151500 2024-09-13T17:01:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d28147d10 con 0x7f9d28151500 2024-09-13T17:01:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d24ff9640 1 -- 172.21.15.39:0/2482940255 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200a5ea0 con 0x7f9d28151500 2024-09-13T17:01:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d280aae40 con 0x7f9d28151500 2024-09-13T17:01:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d24ff9640 1 -- 172.21.15.39:0/2482940255 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d200b4070 con 0x7f9d28151500 2024-09-13T17:01:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.801+0000 7f9d24ff9640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d08066690 0x7f9d08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.805+0000 7f9d277fe640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d08066690 0x7f9d08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.805+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c005260 con 0x7f9d28151500 2024-09-13T17:01:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.805+0000 7f9d24ff9640 1 -- 172.21.15.39:0/2482940255 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7f9d2011f630 con 0x7f9d28151500 2024-09-13T17:01:29.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.805+0000 7f9d277fe640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d08066690 0x7f9d08068b50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9d18007fd0 tx=0x7f9d18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.809+0000 7f9d24ff9640 1 -- 172.21.15.39:0/2482940255 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9d200ec0d0 con 0x7f9d28151500 2024-09-13T17:01:29.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.997+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff79414fe80 con 0x7ff79414e630 2024-09-13T17:01:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:29.997+0000 7ff782ffd640 1 -- 172.21.15.39:0/4071895520 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4419 (secure 0 0 0) 0x7ff78c017070 con 0x7ff79414e630 2024-09-13T17:01:29.998 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:29.998 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 94 2024-09-13T17:01:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff774066690 msgr2=0x7ff774068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff774066690 0x7ff774068b50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff794072290 tx=0x7ff784008040 comp rx=0 tx=0).stop 2024-09-13T17:01:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79414e630 msgr2=0x7ff7941501d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79414e630 0x7ff7941501d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff78c002240 tx=0x7ff78c00cbe0 comp rx=0 tx=0).stop 2024-09-13T17:01:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 shutdown_connections 2024-09-13T17:01:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff774066690 0x7ff774068b50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7940c2db0 0x7ff794146240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79414e630 0x7ff7941501d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 --2- 172.21.15.39:0/4071895520 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7940c2560 0x7ff7941effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 >> 172.21.15.39:0/4071895520 conn(0x7ff794069e10 msgr2=0x7ff79406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 shutdown_connections 2024-09-13T17:01:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.001+0000 7ff798e7b640 1 -- 172.21.15.39:0/4071895520 wait complete. 2024-09-13T17:01:30.023 INFO:tasks.mds_thrash.fs.[a]:mds.c has gained rank=0, replacing gid=6243 2024-09-13T17:01:30.023 INFO:tasks.mds_thrash.fs.[a]:waiting for 7 secs before reviving mds.a 2024-09-13T17:01:30.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.073+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d0c005510 con 0x7f9d28151500 2024-09-13T17:01:30.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.073+0000 7f9d24ff9640 1 -- 172.21.15.39:0/2482940255 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4419 (secure 0 0 0) 0x7f9d200bb020 con 0x7f9d28151500 2024-09-13T17:01:30.074 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:30.074 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:30.074 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 94 2024-09-13T17:01:30.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d08066690 msgr2=0x7f9d08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:30.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d08066690 0x7f9d08068b50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9d18007fd0 tx=0x7f9d18009040 comp rx=0 tx=0).stop 2024-09-13T17:01:30.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d28151500 msgr2=0x7f9d28152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:30.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d28151500 0x7f9d28152110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9d200193c0 tx=0x7f9d20019550 comp rx=0 tx=0).stop 2024-09-13T17:01:30.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 shutdown_connections 2024-09-13T17:01:30.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d08066690 0x7f9d08068b50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:30.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d280c7800 0x7f9d28151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:30.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d28151500 0x7f9d28152110 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:30.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 --2- 172.21.15.39:0/2482940255 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d28071dd0 0x7f9d28156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:30.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 >> 172.21.15.39:0/2482940255 conn(0x7f9d280ab530 msgr2=0x7f9d281469b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:30.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 shutdown_connections 2024-09-13T17:01:30.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:30.077+0000 7f9d2dace640 1 -- 172.21.15.39:0/2482940255 wait complete. 2024-09-13T17:01:30.095 INFO:tasks.mds_thrash.fs.[b]:mds.d has gained rank=0, replacing gid=5423 2024-09-13T17:01:30.095 INFO:tasks.mds_thrash.fs.[b]:waiting for 8 secs before reviving mds.b 2024-09-13T17:01:32.940 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T17:01:32.942+0000 7fd2496f4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:01:37.025 INFO:tasks.mds_thrash.fs.[a]:reviving mds.a 2024-09-13T17:01:37.025 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:01:37.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:01:37.027 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:01:37.028 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:37.108 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T17:01:37.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33fa23a640 1 Processor -- start 2024-09-13T17:01:37.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33fa23a640 1 -- start start 2024-09-13T17:01:37.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33fa23a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33f414acf0 0x7f33f4151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33fa23a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f33f40c4710 0x7f33f406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33fa23a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f4154ab0 con 0x7f33f40c4ba0 2024-09-13T17:01:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33fa23a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f4154d90 con 0x7f33f40c4710 2024-09-13T17:01:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33fa23a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f4150d50 con 0x7f33f414acf0 2024-09-13T17:01:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33f2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f33f40c4710 0x7f33f406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33f2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f33f40c4710 0x7f33f406ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50000/0 (socket says 172.21.15.39:50000) 2024-09-13T17:01:37.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33f2ffd640 1 -- 172.21.15.39:0/3698342147 learned_addr learned my addr 172.21.15.39:0/3698342147 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33f37fe640 1 --2- 172.21.15.39:0/3698342147 >> v2:172.21.15.84:3300/0 conn(0x7f33f414acf0 0x7f33f4151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33f2ffd640 1 -- 172.21.15.39:0/3698342147 >> v2:172.21.15.84:3300/0 conn(0x7f33f414acf0 msgr2=0x7f33f4151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33f2ffd640 1 --2- 172.21.15.39:0/3698342147 >> v2:172.21.15.84:3300/0 conn(0x7f33f414acf0 0x7f33f4151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33f2ffd640 1 -- 172.21.15.39:0/3698342147 >> v1:172.21.15.84:6789/0 conn(0x7f33f40c4ba0 legacy=0x7f33f4153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.169+0000 7f33f2ffd640 1 -- 172.21.15.39:0/3698342147 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4151030 con 0x7f33f40c4710 2024-09-13T17:01:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33f2ffd640 1 --2- 172.21.15.39:0/3698342147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 0x7f33f406ae20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f33e800bb10 tx=0x7f33e801eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3378d7fbbb3def server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33f27fc640 1 -- 172.21.15.39:0/3698342147 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e80a7af0 con 0x7f33f40c4710 2024-09-13T17:01:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33f27fc640 1 -- 172.21.15.39:0/3698342147 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e80a7c50 con 0x7f33f40c4710 2024-09-13T17:01:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33f27fc640 1 -- 172.21.15.39:0/3698342147 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e80af440 con 0x7f33f40c4710 2024-09-13T17:01:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 -- 172.21.15.39:0/3698342147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 msgr2=0x7f33f406ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 --2- 172.21.15.39:0/3698342147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 0x7f33f406ae20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f33e800bb10 tx=0x7f33e801eb00 comp rx=0 tx=0).stop 2024-09-13T17:01:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 -- 172.21.15.39:0/3698342147 shutdown_connections 2024-09-13T17:01:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 --2- 172.21.15.39:0/3698342147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 0x7f33f406ae20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 --2- 172.21.15.39:0/3698342147 >> v2:172.21.15.84:3300/0 conn(0x7f33f414acf0 0x7f33f4151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 -- 172.21.15.39:0/3698342147 >> 172.21.15.39:0/3698342147 conn(0x7f33f40c32a0 msgr2=0x7f33f40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 -- 172.21.15.39:0/3698342147 shutdown_connections 2024-09-13T17:01:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 -- 172.21.15.39:0/3698342147 wait complete. 2024-09-13T17:01:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 Processor -- start 2024-09-13T17:01:37.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.173+0000 7f33fa23a640 1 -- start start 2024-09-13T17:01:37.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33fa23a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 0x7f33f41ed840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33fa23a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f40c4ba0 0x7f33f4070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33fa23a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33f414acf0 0x7f33f40bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33fa23a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f41432f0 con 0x7f33f40c4710 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33fa23a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4071cb0 con 0x7f33f40c4ba0 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33fa23a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f41ecd60 con 0x7f33f414acf0 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 0x7f33f41ed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 0x7f33f41ed840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50012/0 (socket says 172.21.15.39:50012) 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f37fe640 1 -- 172.21.15.39:0/557803286 learned_addr learned my addr 172.21.15.39:0/557803286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f2ffd640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f40c4ba0 0x7f33f4070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f3fff640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33f414acf0 0x7f33f40bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f37fe640 1 -- 172.21.15.39:0/557803286 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33f414acf0 msgr2=0x7f33f40bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f37fe640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33f414acf0 0x7f33f40bfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f37fe640 1 -- 172.21.15.39:0/557803286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f40c4ba0 msgr2=0x7f33f4070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f37fe640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f40c4ba0 0x7f33f4070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f37fe640 1 -- 172.21.15.39:0/557803286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e800b770 con 0x7f33f40c4710 2024-09-13T17:01:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f37fe640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 0x7f33f41ed840 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e870 tx=0x7f33e400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33dbfff640 1 -- 172.21.15.39:0/557803286 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4009510 con 0x7f33f40c4710 2024-09-13T17:01:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33dbfff640 1 -- 172.21.15.39:0/557803286 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e400eec0 con 0x7f33f40c4710 2024-09-13T17:01:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f4154880 con 0x7f33f40c4710 2024-09-13T17:01:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f41ed320 con 0x7f33f40c4710 2024-09-13T17:01:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33dbfff640 1 -- 172.21.15.39:0/557803286 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e400cb40 con 0x7f33f40c4710 2024-09-13T17:01:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d4005260 con 0x7f33f40c4710 2024-09-13T17:01:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33dbfff640 1 -- 172.21.15.39:0/557803286 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f33e4013470 con 0x7f33f40c4710 2024-09-13T17:01:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33dbfff640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33e0066710 0x7f33e0068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33dbfff640 1 -- 172.21.15.39:0/557803286 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7f33e4084db0 con 0x7f33f40c4710 2024-09-13T17:01:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.177+0000 7f33f2ffd640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33e0066710 0x7f33e0068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:37.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.181+0000 7f33f2ffd640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33e0066710 0x7f33e0068bd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f33e80b1000 tx=0x7f33e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.185+0000 7f33dbfff640 1 -- 172.21.15.39:0/557803286 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f33e4051930 con 0x7f33f40c4710 2024-09-13T17:01:37.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.393+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33d4005510 con 0x7f33f40c4710 2024-09-13T17:01:37.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33dbfff640 1 -- 172.21.15.39:0/557803286 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4419 (secure 0 0 0) 0x7f33e4021090 con 0x7f33f40c4710 2024-09-13T17:01:37.395 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:37.395 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:37.395 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 94 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33e0066710 msgr2=0x7f33e0068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33e0066710 0x7f33e0068bd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f33e80b1000 tx=0x7f33e8008040 comp rx=0 tx=0).stop 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 msgr2=0x7f33f41ed840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 0x7f33f41ed840 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f33e400e870 tx=0x7f33e400ec70 comp rx=0 tx=0).stop 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 shutdown_connections 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33e0066710 0x7f33e0068bd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f40c4ba0 0x7f33f4070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33f414acf0 0x7f33f40bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 --2- 172.21.15.39:0/557803286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f40c4710 0x7f33f41ed840 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 >> 172.21.15.39:0/557803286 conn(0x7f33f40c32a0 msgr2=0x7f33f4069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 shutdown_connections 2024-09-13T17:01:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:37.397+0000 7f33fa23a640 1 -- 172.21.15.39:0/557803286 wait complete. 2024-09-13T17:01:37.415 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T17:01:38.096 INFO:tasks.mds_thrash.fs.[b]:reviving mds.b 2024-09-13T17:01:38.096 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:01:38.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:01:38.099 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:01:38.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:38.184 INFO:tasks.ceph.mds.b.smithi084.stdout:starting mds.b at 2024-09-13T17:01:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa924053640 1 Processor -- start 2024-09-13T17:01:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa924053640 1 -- start start 2024-09-13T17:01:38.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa924053640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa91c0c7800 0x7fa91c154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:38.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa924053640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c0c5a30 con 0x7fa91c151500 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa924053640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c1576a0 con 0x7fa91c071dd0 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa924053640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91c157980 con 0x7fa91c0c7800 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa921def640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa91c151500 0x7fa91c0c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54968/0 (socket says 172.21.15.39:54968) 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa921def640 1 -- 172.21.15.39:0/2017162071 learned_addr learned my addr 172.21.15.39:0/2017162071 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa9225f0640 1 --2- 172.21.15.39:0/2017162071 >> v2:172.21.15.84:3300/0 conn(0x7fa91c0c7800 0x7fa91c154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa920ded640 1 -- 172.21.15.39:0/2017162071 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855424663 0 0) 0x7fa91c0c5a30 con 0x7fa91c151500 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa920ded640 1 -- 172.21.15.39:0/2017162071 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa908003530 con 0x7fa91c151500 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa9225f0640 1 -- 172.21.15.39:0/2017162071 >> v1:172.21.15.84:6789/0 conn(0x7fa91c071dd0 legacy=0x7fa91c154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa920ded640 1 -- 172.21.15.39:0/2017162071 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717229013 0 0) 0x7fa91c1576a0 con 0x7fa91c071dd0 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa9225f0640 1 -- 172.21.15.39:0/2017162071 >> v1:172.21.15.39:6789/0 conn(0x7fa91c151500 legacy=0x7fa91c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:38.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.221+0000 7fa9225f0640 1 -- 172.21.15.39:0/2017162071 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c157c60 con 0x7fa91c0c7800 2024-09-13T17:01:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa9225f0640 1 --2- 172.21.15.39:0/2017162071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c0c7800 0x7fa91c154780 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa914009dc0 tx=0x7fa91401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a074dbeaae534310 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa920ded640 1 -- 172.21.15.39:0/2017162071 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9140a5970 con 0x7fa91c0c7800 2024-09-13T17:01:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa920ded640 1 -- 172.21.15.39:0/2017162071 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9140a5ad0 con 0x7fa91c0c7800 2024-09-13T17:01:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa920ded640 1 -- 172.21.15.39:0/2017162071 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9140a5e40 con 0x7fa91c0c7800 2024-09-13T17:01:38.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa924053640 1 -- 172.21.15.39:0/2017162071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c0c7800 msgr2=0x7fa91c154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:38.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa924053640 1 --2- 172.21.15.39:0/2017162071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c0c7800 0x7fa91c154780 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa914009dc0 tx=0x7fa91401c890 comp rx=0 tx=0).stop 2024-09-13T17:01:38.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa924053640 1 -- 172.21.15.39:0/2017162071 shutdown_connections 2024-09-13T17:01:38.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa924053640 1 --2- 172.21.15.39:0/2017162071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c0c7800 0x7fa91c154780 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:38.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa924053640 1 -- 172.21.15.39:0/2017162071 >> 172.21.15.39:0/2017162071 conn(0x7fa91c0ab530 msgr2=0x7fa91c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:38.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa924053640 1 -- 172.21.15.39:0/2017162071 shutdown_connections 2024-09-13T17:01:38.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.225+0000 7fa924053640 1 -- 172.21.15.39:0/2017162071 wait complete. 2024-09-13T17:01:38.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 Processor -- start 2024-09-13T17:01:38.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 -- start start 2024-09-13T17:01:38.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa91c071dd0 0x7fa91c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:38.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa91c0c7800 0x7fa91c14c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:38.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c151500 0x7fa91c14cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:38.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c14d3e0 con 0x7fa91c0c7800 2024-09-13T17:01:38.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c156700 con 0x7fa91c151500 2024-09-13T17:01:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c153b80 con 0x7fa91c071dd0 2024-09-13T17:01:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9225f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c151500 0x7fa91c14cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9215ee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa91c0c7800 0x7fa91c14c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9225f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c151500 0x7fa91c14cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:32852/0 (socket says 172.21.15.39:32852) 2024-09-13T17:01:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9225f0640 1 -- 172.21.15.39:0/2766642357 learned_addr learned my addr 172.21.15.39:0/2766642357 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9215ee640 1 -- 172.21.15.39:0/2766642357 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa91c071dd0 msgr2=0x7fa91c152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9215ee640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa91c071dd0 0x7fa91c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9215ee640 1 -- 172.21.15.39:0/2766642357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c151500 msgr2=0x7fa91c14cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9215ee640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c151500 0x7fa91c14cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9215ee640 1 -- 172.21.15.39:0/2766642357 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa914009a70 con 0x7fa91c0c7800 2024-09-13T17:01:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9225f0640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c151500 0x7fa91c14cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:01:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9215ee640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa91c0c7800 0x7fa91c14c700 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0011f0 tx=0x7fa90c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:38.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9127fc640 1 -- 172.21.15.39:0/2766642357 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0054e0 con 0x7fa91c0c7800 2024-09-13T17:01:38.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9127fc640 1 -- 172.21.15.39:0/2766642357 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c003600 con 0x7fa91c0c7800 2024-09-13T17:01:38.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c1563d0 con 0x7fa91c0c7800 2024-09-13T17:01:38.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa9127fc640 1 -- 172.21.15.39:0/2766642357 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c003a30 con 0x7fa91c0c7800 2024-09-13T17:01:38.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.229+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c1ecbe0 con 0x7fa91c0c7800 2024-09-13T17:01:38.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.233+0000 7fa9127fc640 1 -- 172.21.15.39:0/2766642357 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa90c003b90 con 0x7fa91c0c7800 2024-09-13T17:01:38.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.233+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa904005260 con 0x7fa91c0c7800 2024-09-13T17:01:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.233+0000 7fa9127fc640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa900066690 0x7fa900068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.233+0000 7fa9127fc640 1 -- 172.21.15.39:0/2766642357 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7fa90c083fa0 con 0x7fa91c0c7800 2024-09-13T17:01:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.233+0000 7fa921def640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa900066690 0x7fa900068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:38.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.233+0000 7fa921def640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa900066690 0x7fa900068b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa918007fd0 tx=0x7fa918007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:38.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.237+0000 7fa9127fc640 1 -- 172.21.15.39:0/2766642357 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa90c0b5db0 con 0x7fa91c0c7800 2024-09-13T17:01:38.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa904005510 con 0x7fa91c0c7800 2024-09-13T17:01:38.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa9127fc640 1 -- 172.21.15.39:0/2766642357 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5172 (secure 0 0 0) 0x7fa90c017070 con 0x7fa91c0c7800 2024-09-13T17:01:38.451 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:38.451 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:38.451 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 95 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa900066690 msgr2=0x7fa900068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa900066690 0x7fa900068b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa918007fd0 tx=0x7fa918007900 comp rx=0 tx=0).stop 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa91c0c7800 msgr2=0x7fa91c14c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa91c0c7800 0x7fa91c14c700 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0011f0 tx=0x7fa90c00c8c0 comp rx=0 tx=0).stop 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 shutdown_connections 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa900066690 0x7fa900068b50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa91c151500 0x7fa91c14cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa91c0c7800 0x7fa91c14c700 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 --2- 172.21.15.39:0/2766642357 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa91c071dd0 0x7fa91c152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 >> 172.21.15.39:0/2766642357 conn(0x7fa91c0ab530 msgr2=0x7fa91c0c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 shutdown_connections 2024-09-13T17:01:38.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:38.453+0000 7fa924053640 1 -- 172.21.15.39:0/2766642357 wait complete. 2024-09-13T17:01:38.475 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2024-09-13T17:01:39.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:39.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f44615f9640 1 Processor -- start 2024-09-13T17:01:39.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f44615f9640 1 -- start start 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f44615f9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c1515b0 0x7f445c0bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f44615f9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f445c157330 0x7f445c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f44615f9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445c0c08a0 con 0x7f445c0c0400 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f44615f9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c153d60 con 0x7f445c157330 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f44615f9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c154860 con 0x7f445c1515b0 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f445b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f445c157330 0x7f445c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f445a7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f445c0c0400 0x7f445c06ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54984/0 (socket says 172.21.15.39:54984) 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f445a7fc640 1 -- 172.21.15.39:0/934113713 learned_addr learned my addr 172.21.15.39:0/934113713 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f445affd640 1 --2- 172.21.15.39:0/934113713 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c1515b0 0x7f445c0bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f445b7fe640 1 -- 172.21.15.39:0/934113713 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c1515b0 msgr2=0x7f445c0bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f445b7fe640 1 --2- 172.21.15.39:0/934113713 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c1515b0 0x7f445c0bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f445b7fe640 1 -- 172.21.15.39:0/934113713 >> v1:172.21.15.39:6789/0 conn(0x7f445c0c0400 legacy=0x7f445c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f4459ffb640 1 -- 172.21.15.39:0/934113713 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358619632 0 0) 0x7f444c001120 con 0x7f445c0c0400 2024-09-13T17:01:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f445b7fe640 1 -- 172.21.15.39:0/934113713 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c154cf0 con 0x7f445c157330 2024-09-13T17:01:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.561+0000 7f445affd640 1 --2- 172.21.15.39:0/934113713 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c1515b0 0x7f445c0bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:01:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f445b7fe640 1 --2- 172.21.15.39:0/934113713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c157330 0x7f445c1577c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f44500097c0 tx=0x7f445001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecb757f23f1ad9a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f4459ffb640 1 -- 172.21.15.39:0/934113713 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500ab600 con 0x7f445c157330 2024-09-13T17:01:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f4459ffb640 1 -- 172.21.15.39:0/934113713 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44500a7050 con 0x7f445c157330 2024-09-13T17:01:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f4459ffb640 1 -- 172.21.15.39:0/934113713 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500ab980 con 0x7f445c157330 2024-09-13T17:01:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 -- 172.21.15.39:0/934113713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c157330 msgr2=0x7f445c1577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 --2- 172.21.15.39:0/934113713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c157330 0x7f445c1577c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f44500097c0 tx=0x7f445001c410 comp rx=0 tx=0).stop 2024-09-13T17:01:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 -- 172.21.15.39:0/934113713 shutdown_connections 2024-09-13T17:01:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 --2- 172.21.15.39:0/934113713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c157330 0x7f445c1577c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 --2- 172.21.15.39:0/934113713 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c1515b0 0x7f445c0bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 -- 172.21.15.39:0/934113713 >> 172.21.15.39:0/934113713 conn(0x7f445c0c32a0 msgr2=0x7f445c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 -- 172.21.15.39:0/934113713 shutdown_connections 2024-09-13T17:01:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 -- 172.21.15.39:0/934113713 wait complete. 2024-09-13T17:01:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 Processor -- start 2024-09-13T17:01:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 -- start start 2024-09-13T17:01:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c0c0400 0x7f445c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c1515b0 0x7f445c14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f445c157330 0x7f445c1efd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c1503f0 con 0x7f445c1515b0 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c06c920 con 0x7f445c157330 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f44615f9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c1f0af0 con 0x7f445c0c0400 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.565+0000 7f445a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c1515b0 0x7f445c14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c1515b0 0x7f445c14f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50044/0 (socket says 172.21.15.39:50044) 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445a7fc640 1 -- 172.21.15.39:0/3841690500 learned_addr learned my addr 172.21.15.39:0/3841690500 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445affd640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c0c0400 0x7f445c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445b7fe640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f445c157330 0x7f445c1efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445a7fc640 1 -- 172.21.15.39:0/3841690500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c0c0400 msgr2=0x7f445c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445a7fc640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c0c0400 0x7f445c06c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445a7fc640 1 -- 172.21.15.39:0/3841690500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f445c157330 msgr2=0x7f445c1efd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445a7fc640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f445c157330 0x7f445c1efd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445a7fc640 1 -- 172.21.15.39:0/3841690500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4450009420 con 0x7f445c1515b0 2024-09-13T17:01:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445a7fc640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c1515b0 0x7f445c14f960 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f444c00c3e0 tx=0x7f444c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f44437fe640 1 -- 172.21.15.39:0/3841690500 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0074e0 con 0x7f445c1515b0 2024-09-13T17:01:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f44437fe640 1 -- 172.21.15.39:0/3841690500 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c007660 con 0x7f445c1515b0 2024-09-13T17:01:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c1567d0 con 0x7f445c1515b0 2024-09-13T17:01:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f44437fe640 1 -- 172.21.15.39:0/3841690500 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c004a30 con 0x7f445c1515b0 2024-09-13T17:01:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c06e400 con 0x7f445c1515b0 2024-09-13T17:01:39.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c1f0ca0 con 0x7f445c1515b0 2024-09-13T17:01:39.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f44437fe640 1 -- 172.21.15.39:0/3841690500 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f444c004b90 con 0x7f445c1515b0 2024-09-13T17:01:39.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f44437fe640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44480666c0 0x7f4448068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:39.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f44437fe640 1 -- 172.21.15.39:0/3841690500 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7f444c084590 con 0x7f445c1515b0 2024-09-13T17:01:39.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445affd640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44480666c0 0x7f4448068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:39.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.569+0000 7f445affd640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44480666c0 0x7f4448068b80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4454009fd0 tx=0x7f44540097f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.577+0000 7f44437fe640 1 -- 172.21.15.39:0/3841690500 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f444c054ac0 con 0x7f445c1515b0 2024-09-13T17:01:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.825+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f445c0c08a0 con 0x7f445c1515b0 2024-09-13T17:01:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.825+0000 7f44437fe640 1 -- 172.21.15.39:0/3841690500 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7f445c0c08a0 con 0x7f445c1515b0 2024-09-13T17:01:39.825 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:39.825 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:01:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44480666c0 msgr2=0x7f4448068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44480666c0 0x7f4448068b80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4454009fd0 tx=0x7f44540097f0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c1515b0 msgr2=0x7f445c14f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c1515b0 0x7f445c14f960 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f444c00c3e0 tx=0x7f444c00c8a0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 shutdown_connections 2024-09-13T17:01:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44480666c0 0x7f4448068b80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f445c157330 0x7f445c1efd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f445c1515b0 0x7f445c14f960 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 --2- 172.21.15.39:0/3841690500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f445c0c0400 0x7f445c06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 >> 172.21.15.39:0/3841690500 conn(0x7f445c0c32a0 msgr2=0x7f445c1527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 shutdown_connections 2024-09-13T17:01:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:39.829+0000 7f44615f9640 1 -- 172.21.15.39:0/3841690500 wait complete. 2024-09-13T17:01:39.845 INFO:tasks.mds_thrash.fs.[a]:mds.a reported in up:standby state 2024-09-13T17:01:39.845 INFO:tasks.mds_thrash.fs.[a]:waiting for 63 secs before thrashing 2024-09-13T17:01:40.476 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:01:40.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.617+0000 7fbfb727a640 1 Processor -- start 2024-09-13T17:01:40.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.617+0000 7fbfb727a640 1 -- start start 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.617+0000 7fbfb727a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbfb00c2690 0x7fbfb006ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb727a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfb00c0be0 0x7fbfb015e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb727a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb0157330 con 0x7fbfb00c6290 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb727a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb00c4a20 con 0x7fbfb00c2690 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb727a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb00c4740 con 0x7fbfb00c0be0 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb5016640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbfb00c2690 0x7fbfb006ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb5016640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbfb00c2690 0x7fbfb006ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35890/0 (socket says 172.21.15.39:35890) 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb5016640 1 -- 172.21.15.39:0/2487783667 learned_addr learned my addr 172.21.15.39:0/2487783667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb5817640 1 --2- 172.21.15.39:0/2487783667 >> v2:172.21.15.84:3300/0 conn(0x7fbfb00c0be0 0x7fbfb015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb5016640 1 -- 172.21.15.39:0/2487783667 >> v2:172.21.15.84:3300/0 conn(0x7fbfb00c0be0 msgr2=0x7fbfb015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:01:40.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb5016640 1 --2- 172.21.15.39:0/2487783667 >> v2:172.21.15.84:3300/0 conn(0x7fbfb00c0be0 0x7fbfb015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb5016640 1 -- 172.21.15.39:0/2487783667 >> v1:172.21.15.84:6789/0 conn(0x7fbfb00c6290 legacy=0x7fbfb01577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:40.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb5016640 1 -- 172.21.15.39:0/2487783667 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb0154880 con 0x7fbfb00c2690 2024-09-13T17:01:40.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfb5016640 1 --2- 172.21.15.39:0/2487783667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 0x7fbfb006ae20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00c4000 tx=0x7fbfac01cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1de57a711238518 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:40.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfa7fff640 1 -- 172.21.15.39:0/2487783667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0ab680 con 0x7fbfb00c2690 2024-09-13T17:01:40.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfa7fff640 1 -- 172.21.15.39:0/2487783667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac0ab7e0 con 0x7fbfb00c2690 2024-09-13T17:01:40.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.621+0000 7fbfa7fff640 1 -- 172.21.15.39:0/2487783667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac0abb70 con 0x7fbfb00c2690 2024-09-13T17:01:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 -- 172.21.15.39:0/2487783667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 msgr2=0x7fbfb006ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 --2- 172.21.15.39:0/2487783667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 0x7fbfb006ae20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00c4000 tx=0x7fbfac01cec0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 -- 172.21.15.39:0/2487783667 shutdown_connections 2024-09-13T17:01:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 --2- 172.21.15.39:0/2487783667 >> v2:172.21.15.84:3300/0 conn(0x7fbfb00c0be0 0x7fbfb015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 --2- 172.21.15.39:0/2487783667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 0x7fbfb006ae20 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 -- 172.21.15.39:0/2487783667 >> 172.21.15.39:0/2487783667 conn(0x7fbfb00c32a0 msgr2=0x7fbfb00c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 -- 172.21.15.39:0/2487783667 shutdown_connections 2024-09-13T17:01:40.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 -- 172.21.15.39:0/2487783667 wait complete. 2024-09-13T17:01:40.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 Processor -- start 2024-09-13T17:01:40.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 -- start start 2024-09-13T17:01:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb00c0be0 0x7fbfb01f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 0x7fbfb0071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfb00c6290 0x7fbfb014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb0148e50 con 0x7fbfb00c2690 2024-09-13T17:01:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb014cfc0 con 0x7fbfb00c0be0 2024-09-13T17:01:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb727a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb01f1240 con 0x7fbfb00c6290 2024-09-13T17:01:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb4815640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 0x7fbfb0071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb4815640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 0x7fbfb0071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35896/0 (socket says 172.21.15.39:35896) 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb4815640 1 -- 172.21.15.39:0/1501140529 learned_addr learned my addr 172.21.15.39:0/1501140529 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb5817640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfb00c6290 0x7fbfb014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb5016640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb00c0be0 0x7fbfb01f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb4815640 1 -- 172.21.15.39:0/1501140529 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfb00c6290 msgr2=0x7fbfb014d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb4815640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfb00c6290 0x7fbfb014d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb4815640 1 -- 172.21.15.39:0/1501140529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb00c0be0 msgr2=0x7fbfb01f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb4815640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb00c0be0 0x7fbfb01f0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb4815640 1 -- 172.21.15.39:0/1501140529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac0094d0 con 0x7fbfb00c2690 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.625+0000 7fbfb5016640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb00c0be0 0x7fbfb01f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfb4815640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 0x7fbfb0071620 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000c500 tx=0x7fbfa000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfa5ffb640 1 -- 172.21.15.39:0/1501140529 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0005220 con 0x7fbfb00c2690 2024-09-13T17:01:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfa5ffb640 1 -- 172.21.15.39:0/1501140529 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa0005380 con 0x7fbfb00c2690 2024-09-13T17:01:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb0156830 con 0x7fbfb00c2690 2024-09-13T17:01:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfa5ffb640 1 -- 172.21.15.39:0/1501140529 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0007270 con 0x7fbfb00c2690 2024-09-13T17:01:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb0156df0 con 0x7fbfb00c2690 2024-09-13T17:01:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb00a6330 con 0x7fbfb00c2690 2024-09-13T17:01:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfa5ffb640 1 -- 172.21.15.39:0/1501140529 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbfa00073d0 con 0x7fbfb00c2690 2024-09-13T17:01:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfa5ffb640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf94066690 0x7fbf94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:01:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfa5ffb640 1 -- 172.21.15.39:0/1501140529 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7fbfa00840d0 con 0x7fbfb00c2690 2024-09-13T17:01:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfb5016640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf94066690 0x7fbf94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:01:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.629+0000 7fbfb5016640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf94066690 0x7fbf94068b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0af000 tx=0x7fbfac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:01:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.637+0000 7fbfa5ffb640 1 -- 172.21.15.39:0/1501140529 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbfa0050af0 con 0x7fbfb00c2690 2024-09-13T17:01:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfb006ae20 con 0x7fbfb00c2690 2024-09-13T17:01:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfa5ffb640 1 -- 172.21.15.39:0/1501140529 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7fbfa0054110 con 0x7fbfb00c2690 2024-09-13T17:01:40.868 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:01:40.868 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:01:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:01:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf94066690 msgr2=0x7fbf94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf94066690 0x7fbf94068b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0af000 tx=0x7fbfac008040 comp rx=0 tx=0).stop 2024-09-13T17:01:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 msgr2=0x7fbfb0071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:01:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 0x7fbfb0071620 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000c500 tx=0x7fbfa000c9c0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 shutdown_connections 2024-09-13T17:01:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf94066690 0x7fbf94068b50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfb00c6290 0x7fbfb014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb00c2690 0x7fbfb0071620 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 --2- 172.21.15.39:0/1501140529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb00c0be0 0x7fbfb01f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:01:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 >> 172.21.15.39:0/1501140529 conn(0x7fbfb00c32a0 msgr2=0x7fbfb00a68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:01:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 shutdown_connections 2024-09-13T17:01:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:01:40.869+0000 7fbfb727a640 1 -- 172.21.15.39:0/1501140529 wait complete. 2024-09-13T17:01:40.890 INFO:tasks.mds_thrash.fs.[b]:mds.b reported in up:standby state 2024-09-13T17:01:40.890 INFO:tasks.mds_thrash.fs.[b]:waiting for 83 secs before thrashing 2024-09-13T17:01:58.509 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:01:58.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:02:28.560 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:02:28.564 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:02:42.846 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:02:42.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d08b8f640 1 Processor -- start 2024-09-13T17:02:42.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d08b8f640 1 -- start start 2024-09-13T17:02:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d08b8f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0406bc20 0x7f6d04151480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d08b8f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6d0406cf60 0x7f6d0406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d08b8f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d041519c0 con 0x7f6d0414de70 2024-09-13T17:02:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d08b8f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0406cb40 con 0x7f6d0406cf60 2024-09-13T17:02:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d08b8f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d040aa2d0 con 0x7f6d0406bc20 2024-09-13T17:02:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d02d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6d0406cf60 0x7f6d0406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d01d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0406bc20 0x7f6d04151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d02d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6d0406cf60 0x7f6d0406d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:46410/0 (socket says 172.21.15.39:46410) 2024-09-13T17:02:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d02d9d640 1 -- 172.21.15.39:0/4013272356 learned_addr learned my addr 172.21.15.39:0/4013272356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d02d9d640 1 -- 172.21.15.39:0/4013272356 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0406bc20 msgr2=0x7f6d04151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:42.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d02d9d640 1 --2- 172.21.15.39:0/4013272356 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0406bc20 0x7f6d04151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d02d9d640 1 -- 172.21.15.39:0/4013272356 >> v1:172.21.15.84:6789/0 conn(0x7f6d0414de70 legacy=0x7f6d040a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:42.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.984+0000 7f6d02d9d640 1 -- 172.21.15.39:0/4013272356 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d040614d0 con 0x7f6d0406cf60 2024-09-13T17:02:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d02d9d640 1 --2- 172.21.15.39:0/4013272356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d0406cf60 0x7f6d0406d3b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6d04151b50 tx=0x7f6cfc01eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a8e5d8fa72e1cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d0159a640 1 -- 172.21.15.39:0/4013272356 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0a7760 con 0x7f6d0406cf60 2024-09-13T17:02:42.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d0159a640 1 -- 172.21.15.39:0/4013272356 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc0a78c0 con 0x7f6d0406cf60 2024-09-13T17:02:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d0159a640 1 -- 172.21.15.39:0/4013272356 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0a7c50 con 0x7f6d0406cf60 2024-09-13T17:02:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 -- 172.21.15.39:0/4013272356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d0406cf60 msgr2=0x7f6d0406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:42.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 --2- 172.21.15.39:0/4013272356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d0406cf60 0x7f6d0406d3b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6d04151b50 tx=0x7f6cfc01eb90 comp rx=0 tx=0).stop 2024-09-13T17:02:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 -- 172.21.15.39:0/4013272356 shutdown_connections 2024-09-13T17:02:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 --2- 172.21.15.39:0/4013272356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d0406cf60 0x7f6d0406d3b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 --2- 172.21.15.39:0/4013272356 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0406bc20 0x7f6d04151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 -- 172.21.15.39:0/4013272356 >> 172.21.15.39:0/4013272356 conn(0x7f6d040a8a60 msgr2=0x7f6d040c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 -- 172.21.15.39:0/4013272356 shutdown_connections 2024-09-13T17:02:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 -- 172.21.15.39:0/4013272356 wait complete. 2024-09-13T17:02:42.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 Processor -- start 2024-09-13T17:02:42.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.988+0000 7f6d08b8f640 1 -- start start 2024-09-13T17:02:42.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d08b8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d0406bc20 0x7f6d04070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:42.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d08b8f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d0406cf60 0x7f6d041e94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d08b8f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0414de70 0x7f6d041e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d08b8f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0414fee0 con 0x7f6d0406bc20 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d08b8f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d041501c0 con 0x7f6d0406cf60 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d08b8f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d041e9f60 con 0x7f6d0414de70 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d0259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d0406bc20 0x7f6d04070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d0259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d0406bc20 0x7f6d04070ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36078/0 (socket says 172.21.15.39:36078) 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d0259c640 1 -- 172.21.15.39:0/3365015651 learned_addr learned my addr 172.21.15.39:0/3365015651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d01d9b640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d0406cf60 0x7f6d041e94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d02d9d640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0414de70 0x7f6d041e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d0259c640 1 -- 172.21.15.39:0/3365015651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0414de70 msgr2=0x7f6d041e9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d0259c640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0414de70 0x7f6d041e9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:42.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d0259c640 1 -- 172.21.15.39:0/3365015651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d0406cf60 msgr2=0x7f6d041e94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:42.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d0259c640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d0406cf60 0x7f6d041e94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:42.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d0259c640 1 -- 172.21.15.39:0/3365015651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc009b30 con 0x7f6d0406bc20 2024-09-13T17:02:42.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d02d9d640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0414de70 0x7f6d041e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:02:42.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d0259c640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d0406bc20 0x7f6d04070ec0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400c3c0 tx=0x7f6cf400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:42.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6cf2ffd640 1 -- 172.21.15.39:0/3365015651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40052d0 con 0x7f6d0406bc20 2024-09-13T17:02:42.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6cf2ffd640 1 -- 172.21.15.39:0/3365015651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf4005430 con 0x7f6d0406bc20 2024-09-13T17:02:42.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d04147c10 con 0x7f6d0406bc20 2024-09-13T17:02:42.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6cf2ffd640 1 -- 172.21.15.39:0/3365015651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4003840 con 0x7f6d0406bc20 2024-09-13T17:02:42.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d04147eb0 con 0x7f6d0406bc20 2024-09-13T17:02:42.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d04151480 con 0x7f6d0406bc20 2024-09-13T17:02:42.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6cf2ffd640 1 -- 172.21.15.39:0/3365015651 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6cf40039a0 con 0x7f6d0406bc20 2024-09-13T17:02:42.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.992+0000 7f6cf2ffd640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6ce4066690 0x7f6ce4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:42.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.996+0000 7f6cf2ffd640 1 -- 172.21.15.39:0/3365015651 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7f6cf4083f80 con 0x7f6d0406bc20 2024-09-13T17:02:42.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.996+0000 7f6d01d9b640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6ce4066690 0x7f6ce4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:42.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:42.996+0000 7f6d01d9b640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6ce4066690 0x7f6ce4068b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6d041eb540 tx=0x7f6cf8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:42.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.000+0000 7f6cf2ffd640 1 -- 172.21.15.39:0/3365015651 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6cf40b5db0 con 0x7f6d0406bc20 2024-09-13T17:02:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.288+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d04071400 con 0x7f6d0406bc20 2024-09-13T17:02:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.288+0000 7f6cf2ffd640 1 -- 172.21.15.39:0/3365015651 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7f6cf4017020 con 0x7f6d0406bc20 2024-09-13T17:02:43.288 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:02:43.288 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:02:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:02:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6ce4066690 msgr2=0x7f6ce4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6ce4066690 0x7f6ce4068b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6d041eb540 tx=0x7f6cf8009190 comp rx=0 tx=0).stop 2024-09-13T17:02:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d0406bc20 msgr2=0x7f6d04070ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d0406bc20 0x7f6d04070ec0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400c3c0 tx=0x7f6cf400c880 comp rx=0 tx=0).stop 2024-09-13T17:02:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 shutdown_connections 2024-09-13T17:02:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6ce4066690 0x7f6ce4068b50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d0406cf60 0x7f6d041e94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6d0414de70 0x7f6d041e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 --2- 172.21.15.39:0/3365015651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d0406bc20 0x7f6d04070ec0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 >> 172.21.15.39:0/3365015651 conn(0x7f6d040a8a60 msgr2=0x7f6d0406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:43.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 shutdown_connections 2024-09-13T17:02:43.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.292+0000 7f6d08b8f640 1 -- 172.21.15.39:0/3365015651 wait complete. 2024-09-13T17:02:43.310 INFO:tasks.mds_thrash.fs.[a]:kill mds.c (rank=0) 2024-09-13T17:02:43.311 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:02:43.311 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:02:43.321 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:02:43.321 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:02:43.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4d1a76640 1 Processor -- start 2024-09-13T17:02:43.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4d1a76640 1 -- start start 2024-09-13T17:02:43.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4d1a76640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4cc071bb0 0x7fb4cc06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:43.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4d1a76640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 0x7fb4cc06d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:43.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4d1a76640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc150e00 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4d1a76640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc0abdc0 con 0x7fb4cc071bb0 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4d1a76640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc150250 con 0x7fb4cc0c0ac0 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4c2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4cc071bb0 0x7fb4cc06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4c2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4cc071bb0 0x7fb4cc06cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36082/0 (socket says 172.21.15.39:36082) 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4c2ffd640 1 -- 172.21.15.39:0/2745078028 learned_addr learned my addr 172.21.15.39:0/2745078028 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4cbfff640 1 --2- 172.21.15.39:0/2745078028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 0x7fb4cc06d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4c2ffd640 1 -- 172.21.15.39:0/2745078028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 msgr2=0x7fb4cc06d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4c2ffd640 1 --2- 172.21.15.39:0/2745078028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 0x7fb4cc06d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4c2ffd640 1 -- 172.21.15.39:0/2745078028 >> v1:172.21.15.84:6789/0 conn(0x7fb4cc0ab9e0 legacy=0x7fb4cc0c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:43.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.484+0000 7fb4c2ffd640 1 -- 172.21.15.39:0/2745078028 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc14a8f0 con 0x7fb4cc071bb0 2024-09-13T17:02:43.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4c2ffd640 1 --2- 172.21.15.39:0/2745078028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc071bb0 0x7fb4cc06cbe0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009870 tx=0x7fb4b401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98962719716bc1ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:43.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4caffd640 1 -- 172.21.15.39:0/2745078028 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40ac630 con 0x7fb4cc071bb0 2024-09-13T17:02:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4caffd640 1 -- 172.21.15.39:0/2745078028 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b40ac790 con 0x7fb4cc071bb0 2024-09-13T17:02:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4caffd640 1 -- 172.21.15.39:0/2745078028 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40acb20 con 0x7fb4cc071bb0 2024-09-13T17:02:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4d1a76640 1 -- 172.21.15.39:0/2745078028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc071bb0 msgr2=0x7fb4cc06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:43.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4d1a76640 1 --2- 172.21.15.39:0/2745078028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc071bb0 0x7fb4cc06cbe0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009870 tx=0x7fb4b401c560 comp rx=0 tx=0).stop 2024-09-13T17:02:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4d1a76640 1 -- 172.21.15.39:0/2745078028 shutdown_connections 2024-09-13T17:02:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4d1a76640 1 --2- 172.21.15.39:0/2745078028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 0x7fb4cc06d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4d1a76640 1 --2- 172.21.15.39:0/2745078028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc071bb0 0x7fb4cc06cbe0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4d1a76640 1 -- 172.21.15.39:0/2745078028 >> 172.21.15.39:0/2745078028 conn(0x7fb4cc069e10 msgr2=0x7fb4cc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4d1a76640 1 -- 172.21.15.39:0/2745078028 shutdown_connections 2024-09-13T17:02:43.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.488+0000 7fb4d1a76640 1 -- 172.21.15.39:0/2745078028 wait complete. 2024-09-13T17:02:43.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 Processor -- start 2024-09-13T17:02:43.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 -- start start 2024-09-13T17:02:43.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4cc071bb0 0x7fb4cc14cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:43.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc0ab9e0 0x7fb4cc151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:43.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 0x7fb4cc14c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc14ae20 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc145b40 con 0x7fb4cc071bb0 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc14ab40 con 0x7fb4cc0c0ac0 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc0ab9e0 0x7fb4cc151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc0ab9e0 0x7fb4cc151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36098/0 (socket says 172.21.15.39:36098) 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c2ffd640 1 -- 172.21.15.39:0/3715538793 learned_addr learned my addr 172.21.15.39:0/3715538793 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4cb7fe640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4cc071bb0 0x7fb4cc14cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4cbfff640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 0x7fb4cc14c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c2ffd640 1 -- 172.21.15.39:0/3715538793 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 msgr2=0x7fb4cc14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c2ffd640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 0x7fb4cc14c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c2ffd640 1 -- 172.21.15.39:0/3715538793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4cc071bb0 msgr2=0x7fb4cc14cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:43.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c2ffd640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4cc071bb0 0x7fb4cc14cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c2ffd640 1 -- 172.21.15.39:0/3715538793 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b40094d0 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c2ffd640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc0ab9e0 0x7fb4cc151970 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40af040 tx=0x7fb4b4019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:43.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c8ff9640 1 -- 172.21.15.39:0/3715538793 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40b7670 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c8ff9640 1 -- 172.21.15.39:0/3715538793 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b40b77f0 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4c8ff9640 1 -- 172.21.15.39:0/3715538793 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40b7ba0 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc1468e0 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.492+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc14d530 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.496+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc14bc20 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.496+0000 7fb4c8ff9640 1 -- 172.21.15.39:0/3715538793 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4b40b3070 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.496+0000 7fb4c8ff9640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4ac066690 0x7fb4ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:43.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.496+0000 7fb4c8ff9640 1 -- 172.21.15.39:0/3715538793 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7fb4b411f230 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.496+0000 7fb4cb7fe640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4ac066690 0x7fb4ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:43.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.496+0000 7fb4cb7fe640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4ac066690 0x7fb4ac068b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc15db60 tx=0x7fb4bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:43.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.500+0000 7fb4c8ff9640 1 -- 172.21.15.39:0/3715538793 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb4cc14bc20 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.764+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4cc14c000 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4c8ff9640 1 -- 172.21.15.39:0/3715538793 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7fb4cc14c000 con 0x7fb4cc0ab9e0 2024-09-13T17:02:43.766 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:02:43.766 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:02:43.766 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4ac066690 msgr2=0x7fb4ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4ac066690 0x7fb4ac068b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc15db60 tx=0x7fb4bc009040 comp rx=0 tx=0).stop 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc0ab9e0 msgr2=0x7fb4cc151970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc0ab9e0 0x7fb4cc151970 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40af040 tx=0x7fb4b4019300 comp rx=0 tx=0).stop 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 shutdown_connections 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4ac066690 0x7fb4ac068b50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4cc0c0ac0 0x7fb4cc14c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4cc0ab9e0 0x7fb4cc151970 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 --2- 172.21.15.39:0/3715538793 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4cc071bb0 0x7fb4cc14cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 >> 172.21.15.39:0/3715538793 conn(0x7fb4cc069e10 msgr2=0x7fb4cc14f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:43.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 shutdown_connections 2024-09-13T17:02:43.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:43.768+0000 7fb4d1a76640 1 -- 172.21.15.39:0/3715538793 wait complete. 2024-09-13T17:02:43.787 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:02:45.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:02:45.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f185b496640 1 Processor -- start 2024-09-13T17:02:45.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f185b496640 1 -- start start 2024-09-13T17:02:45.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f185b496640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f185406fa50 0x7f1854156130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:45.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f185b496640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540c5e50 con 0x7f18540c6000 2024-09-13T17:02:45.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f185b496640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185406ff50 con 0x7f18540c7960 2024-09-13T17:02:45.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f185b496640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1854072000 con 0x7f185406fa50 2024-09-13T17:02:45.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f1859a33640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f18540c6000 0x7f18540c2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37330/0 (socket says 172.21.15.39:37330) 2024-09-13T17:02:45.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f1859a33640 1 -- 172.21.15.39:0/4009282709 learned_addr learned my addr 172.21.15.39:0/4009282709 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:02:45.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743438087 0 0) 0x7f18540c5e50 con 0x7f18540c6000 2024-09-13T17:02:45.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1840003530 con 0x7f18540c6000 2024-09-13T17:02:45.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.932+0000 7f1858a31640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f185406fa50 0x7f1854156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:45.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2214951377 0 0) 0x7f1840003530 con 0x7f18540c6000 2024-09-13T17:02:45.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f185406fa50 msgr2=0x7f1854156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:45.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f185406fa50 0x7f1854156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:45.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 >> v1:172.21.15.84:6789/0 conn(0x7f18540c7960 legacy=0x7f1854155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540722e0 con 0x7f18540c6000 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f184c001a00 con 0x7f18540c6000 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 >> v1:172.21.15.39:6789/0 conn(0x7f18540c6000 legacy=0x7f18540c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400039b0 0x7f1840003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18400047d0 0x7f1840004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f185406fa50 0x7f18400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1854072000 con 0x7f18400047d0 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184000baf0 con 0x7f18400039b0 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184000bcb0 con 0x7f185406fa50 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f1858a31640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18400047d0 0x7f1840004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:45.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f1859232640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400039b0 0x7f1840003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:45.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f1858a31640 1 -- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f185406fa50 msgr2=0x7f18400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:45.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f1858a31640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f185406fa50 0x7f18400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:45.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f1858a31640 1 -- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400039b0 msgr2=0x7f1840003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:45.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f1858a31640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400039b0 0x7f1840003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:45.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f1858a31640 1 -- 172.21.15.39:0/4009282709 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1840003530 con 0x7f18400047d0 2024-09-13T17:02:45.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f1859232640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400039b0 0x7f1840003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:02:45.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f1858a31640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18400047d0 0x7f1840004c50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1844009ae0 tx=0x7f184401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699741426c1f9947 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:45.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18440a5560 con 0x7f18400047d0 2024-09-13T17:02:45.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18440a56c0 con 0x7f18400047d0 2024-09-13T17:02:45.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f184bfff640 1 -- 172.21.15.39:0/4009282709 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18440a5a70 con 0x7f18400047d0 2024-09-13T17:02:45.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.936+0000 7f185b496640 1 -- 172.21.15.39:0/4009282709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18400047d0 msgr2=0x7f1840004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:45.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 --2- 172.21.15.39:0/4009282709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18400047d0 0x7f1840004c50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1844009ae0 tx=0x7f184401ca80 comp rx=0 tx=0).stop 2024-09-13T17:02:45.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1859a33640 1 -- 172.21.15.39:0/4009282709 reap_dead start 2024-09-13T17:02:45.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- 172.21.15.39:0/4009282709 shutdown_connections 2024-09-13T17:02:45.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- 172.21.15.39:0/4009282709 >> 172.21.15.39:0/4009282709 conn(0x7f185406c020 msgr2=0x7f185406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:45.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- 172.21.15.39:0/4009282709 shutdown_connections 2024-09-13T17:02:45.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- 172.21.15.39:0/4009282709 wait complete. 2024-09-13T17:02:45.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 Processor -- start 2024-09-13T17:02:45.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- start start 2024-09-13T17:02:45.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400047d0 0x7f18541ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:45.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1854157b60 0x7f18540aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18540aaf90 0x7f18541522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18541576d0 con 0x7f1854157b60 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1854158260 con 0x7f18400047d0 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18541eeb20 con 0x7f18540aaf90 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1858a31640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1854157b60 0x7f18540aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1858a31640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1854157b60 0x7f18540aaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36102/0 (socket says 172.21.15.39:36102) 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1858a31640 1 -- 172.21.15.39:0/1799816095 learned_addr learned my addr 172.21.15.39:0/1799816095 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1859232640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400047d0 0x7f18541ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1859a33640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18540aaf90 0x7f18541522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1858a31640 1 -- 172.21.15.39:0/1799816095 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18540aaf90 msgr2=0x7f18541522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1858a31640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18540aaf90 0x7f18541522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1858a31640 1 -- 172.21.15.39:0/1799816095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400047d0 msgr2=0x7f18541ee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:45.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1858a31640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400047d0 0x7f18541ee2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1858a31640 1 -- 172.21.15.39:0/1799816095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18541eee00 con 0x7f1854157b60 2024-09-13T17:02:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1858a31640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1854157b60 0x7f18540aaa50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1844009ae0 tx=0x7f1840005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1849ffb640 1 -- 172.21.15.39:0/1799816095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18440a5e30 con 0x7f1854157b60 2024-09-13T17:02:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1849ffb640 1 -- 172.21.15.39:0/1799816095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1844004020 con 0x7f1854157b60 2024-09-13T17:02:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18540a7d60 con 0x7f1854157b60 2024-09-13T17:02:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f1849ffb640 1 -- 172.21.15.39:0/1799816095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844004390 con 0x7f1854157b60 2024-09-13T17:02:45.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.940+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1854158540 con 0x7f1854157b60 2024-09-13T17:02:45.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.944+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1854157880 con 0x7f1854157b60 2024-09-13T17:02:45.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.944+0000 7f1849ffb640 1 -- 172.21.15.39:0/1799816095 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18440b4040 con 0x7f1854157b60 2024-09-13T17:02:45.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.944+0000 7f1849ffb640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1838066690 0x7f1838068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:45.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.944+0000 7f1849ffb640 1 -- 172.21.15.39:0/1799816095 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7f184411ec60 con 0x7f1854157b60 2024-09-13T17:02:45.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.944+0000 7f1859232640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1838066690 0x7f1838068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:45.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.944+0000 7f1859232640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1838066690 0x7f1838068b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1854153e60 tx=0x7f1850009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:45.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:45.948+0000 7f1849ffb640 1 -- 172.21.15.39:0/1799816095 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1854157880 con 0x7f1854157b60 2024-09-13T17:02:46.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.180+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18541576d0 con 0x7f1854157b60 2024-09-13T17:02:46.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.180+0000 7f1849ffb640 1 -- 172.21.15.39:0/1799816095 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7f18440b2090 con 0x7f1854157b60 2024-09-13T17:02:46.179 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:02:46.179 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:02:46.180 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.180+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1838066690 msgr2=0x7f1838068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.180+0000 7f185b496640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1838066690 0x7f1838068b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1854153e60 tx=0x7f1850009040 comp rx=0 tx=0).stop 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.180+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1854157b60 msgr2=0x7f18540aaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.180+0000 7f185b496640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1854157b60 0x7f18540aaa50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1844009ae0 tx=0x7f1840005800 comp rx=0 tx=0).stop 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.184+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 shutdown_connections 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.184+0000 7f185b496640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1838066690 0x7f1838068b50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.184+0000 7f185b496640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18540aaf90 0x7f18541522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.184+0000 7f185b496640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1854157b60 0x7f18540aaa50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.184+0000 7f185b496640 1 --2- 172.21.15.39:0/1799816095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18400047d0 0x7f18541ee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.184+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 >> 172.21.15.39:0/1799816095 conn(0x7f185406c020 msgr2=0x7f1854153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.184+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 shutdown_connections 2024-09-13T17:02:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:46.184+0000 7f185b496640 1 -- 172.21.15.39:0/1799816095 wait complete. 2024-09-13T17:02:46.198 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:02:48.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:02:48.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b0a2d3640 1 Processor -- start 2024-09-13T17:02:48.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b0a2d3640 1 -- start start 2024-09-13T17:02:48.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b0a2d3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0406d020 0x7f6b040a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b0a2d3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6b04151460 0x7f6b0414db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b0a2d3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b04154100 con 0x7f6b0414e070 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b0a2d3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b040a8c50 con 0x7f6b04151460 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b0a2d3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041543e0 con 0x7f6b0406d020 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b037fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6b04151460 0x7f6b0414db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b08870640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f6b0414e070 0x7f6b040c4530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37344/0 (socket says 172.21.15.39:37344) 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b037fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6b04151460 0x7f6b0414db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36108/0 (socket says 172.21.15.39:36108) 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b08870640 1 -- 172.21.15.39:0/3468950243 learned_addr learned my addr 172.21.15.39:0/3468950243 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b03fff640 1 --2- 172.21.15.39:0/3468950243 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0406d020 0x7f6b040a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b037fe640 1 -- 172.21.15.39:0/3468950243 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0406d020 msgr2=0x7f6b040a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b037fe640 1 --2- 172.21.15.39:0/3468950243 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0406d020 0x7f6b040a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b037fe640 1 -- 172.21.15.39:0/3468950243 >> v1:172.21.15.39:6789/0 conn(0x7f6b0414e070 legacy=0x7f6b040c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:48.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b02ffd640 1 -- 172.21.15.39:0/3468950243 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542302748 0 0) 0x7f6b04154100 con 0x7f6b0414e070 2024-09-13T17:02:48.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.336+0000 7f6b037fe640 1 -- 172.21.15.39:0/3468950243 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b041546c0 con 0x7f6b04151460 2024-09-13T17:02:48.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b037fe640 1 --2- 172.21.15.39:0/3468950243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b04151460 0x7f6b0414db30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6af80097c0 tx=0x7f6af801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296207dfa05e52da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:48.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b02ffd640 1 -- 172.21.15.39:0/3468950243 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af80ac640 con 0x7f6b04151460 2024-09-13T17:02:48.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b02ffd640 1 -- 172.21.15.39:0/3468950243 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af80ac7a0 con 0x7f6b04151460 2024-09-13T17:02:48.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b02ffd640 1 -- 172.21.15.39:0/3468950243 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af80acb30 con 0x7f6b04151460 2024-09-13T17:02:48.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/3468950243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b04151460 msgr2=0x7f6b0414db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:48.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 --2- 172.21.15.39:0/3468950243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b04151460 0x7f6b0414db30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6af80097c0 tx=0x7f6af801c3e0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/3468950243 shutdown_connections 2024-09-13T17:02:48.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 --2- 172.21.15.39:0/3468950243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b04151460 0x7f6b0414db30 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 --2- 172.21.15.39:0/3468950243 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0406d020 0x7f6b040a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/3468950243 >> 172.21.15.39:0/3468950243 conn(0x7f6b04069e10 msgr2=0x7f6b0406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:48.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/3468950243 shutdown_connections 2024-09-13T17:02:48.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/3468950243 wait complete. 2024-09-13T17:02:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 Processor -- start 2024-09-13T17:02:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- start start 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b0406d020 0x7f6b0406faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0414e070 0x7f6b04145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b04151460 0x7f6b0414a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041ed7c0 con 0x7f6b0406d020 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041ecbb0 con 0x7f6b04151460 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b041ece90 con 0x7f6b0414e070 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b03fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b0406d020 0x7f6b0406faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b03fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b0406d020 0x7f6b0406faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36118/0 (socket says 172.21.15.39:36118) 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b03fff640 1 -- 172.21.15.39:0/1485789283 learned_addr learned my addr 172.21.15.39:0/1485789283 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b037fe640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0414e070 0x7f6b04145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b08870640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b04151460 0x7f6b0414a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b03fff640 1 -- 172.21.15.39:0/1485789283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0414e070 msgr2=0x7f6b04145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b03fff640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0414e070 0x7f6b04145560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b03fff640 1 -- 172.21.15.39:0/1485789283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b04151460 msgr2=0x7f6b0414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b03fff640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b04151460 0x7f6b0414a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b03fff640 1 -- 172.21.15.39:0/1485789283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af8009420 con 0x7f6b0406d020 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b03fff640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b0406d020 0x7f6b0406faf0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6af400d800 tx=0x7f6af400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b00ff9640 1 -- 172.21.15.39:0/1485789283 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400b6c0 con 0x7f6b0406d020 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b00ff9640 1 -- 172.21.15.39:0/1485789283 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af400b820 con 0x7f6b0406d020 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b041ed170 con 0x7f6b0406d020 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.340+0000 7f6b00ff9640 1 -- 172.21.15.39:0/1485789283 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400bb90 con 0x7f6b0406d020 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.344+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b040a8760 con 0x7f6b0406d020 2024-09-13T17:02:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.344+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b04072600 con 0x7f6b0406d020 2024-09-13T17:02:48.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.344+0000 7f6b00ff9640 1 -- 172.21.15.39:0/1485789283 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6af400bcf0 con 0x7f6b0406d020 2024-09-13T17:02:48.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.344+0000 7f6b00ff9640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6af0066690 0x7f6af0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:48.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.344+0000 7f6b00ff9640 1 -- 172.21.15.39:0/1485789283 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7f6af4085050 con 0x7f6b0406d020 2024-09-13T17:02:48.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.344+0000 7f6b037fe640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6af0066690 0x7f6af0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:48.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.344+0000 7f6b037fe640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6af0066690 0x7f6af0068b50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6af80ae000 tx=0x7f6af80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.348+0000 7f6b00ff9640 1 -- 172.21.15.39:0/1485789283 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6af4051af0 con 0x7f6b0406d020 2024-09-13T17:02:48.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.544+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b041464c0 con 0x7f6b0406d020 2024-09-13T17:02:48.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.544+0000 7f6b00ff9640 1 -- 172.21.15.39:0/1485789283 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7f6af4055110 con 0x7f6b0406d020 2024-09-13T17:02:48.544 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:02:48.544 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:02:48.544 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:02:48.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6af0066690 msgr2=0x7f6af0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:48.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6af0066690 0x7f6af0068b50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6af80ae000 tx=0x7f6af80a7000 comp rx=0 tx=0).stop 2024-09-13T17:02:48.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b0406d020 msgr2=0x7f6b0406faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:48.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b0406d020 0x7f6b0406faf0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6af400d800 tx=0x7f6af400dcc0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 shutdown_connections 2024-09-13T17:02:48.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6af0066690 0x7f6af0068b50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b04151460 0x7f6b0414a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6b0414e070 0x7f6b04145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 --2- 172.21.15.39:0/1485789283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b0406d020 0x7f6b0406faf0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:48.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 >> 172.21.15.39:0/1485789283 conn(0x7f6b04069e10 msgr2=0x7f6b0406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:48.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 shutdown_connections 2024-09-13T17:02:48.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:48.548+0000 7f6b0a2d3640 1 -- 172.21.15.39:0/1485789283 wait complete. 2024-09-13T17:02:48.563 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:02:50.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:02:50.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb9d580640 1 Processor -- start 2024-09-13T17:02:50.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb9d580640 1 -- start start 2024-09-13T17:02:50.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb9d580640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb98151500 0x7fbb980c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:50.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb9d580640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbb98071dd0 0x7fbb98154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:50.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb9d580640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbb980c47c0 0x7fbb981545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:50.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb9d580640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb980c5a30 con 0x7fbb98071dd0 2024-09-13T17:02:50.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb9d580640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb981576a0 con 0x7fbb980c47c0 2024-09-13T17:02:50.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb9d580640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98157980 con 0x7fbb98151500 2024-09-13T17:02:50.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb967fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbb98071dd0 0x7fbb98154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:50.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb967fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbb98071dd0 0x7fbb98154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39506/0 (socket says 172.21.15.39:39506) 2024-09-13T17:02:50.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb967fc640 1 -- 172.21.15.39:0/1245683637 learned_addr learned my addr 172.21.15.39:0/1245683637 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:50.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb977fe640 1 --2- 172.21.15.39:0/1245683637 >> v2:172.21.15.84:3300/0 conn(0x7fbb980c47c0 0x7fbb981545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:50.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb96ffd640 1 --2- 172.21.15.39:0/1245683637 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb98151500 0x7fbb980c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:50.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb967fc640 1 -- 172.21.15.39:0/1245683637 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb98151500 msgr2=0x7fbb980c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:50.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb967fc640 1 --2- 172.21.15.39:0/1245683637 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb98151500 0x7fbb980c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb967fc640 1 -- 172.21.15.39:0/1245683637 >> v2:172.21.15.84:3300/0 conn(0x7fbb980c47c0 msgr2=0x7fbb981545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:50.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb967fc640 1 --2- 172.21.15.39:0/1245683637 >> v2:172.21.15.84:3300/0 conn(0x7fbb980c47c0 0x7fbb981545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.700+0000 7fbb967fc640 1 -- 172.21.15.39:0/1245683637 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb98157c60 con 0x7fbb98071dd0 2024-09-13T17:02:50.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb967fc640 1 --2- 172.21.15.39:0/1245683637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 0x7fbb98154070 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fbb98153d60 tx=0x7fbb8c01cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72889c0683562b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:50.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb95ffb640 1 -- 172.21.15.39:0/1245683637 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0ab640 con 0x7fbb98071dd0 2024-09-13T17:02:50.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb95ffb640 1 -- 172.21.15.39:0/1245683637 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c0a7070 con 0x7fbb98071dd0 2024-09-13T17:02:50.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb95ffb640 1 -- 172.21.15.39:0/1245683637 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0ab9c0 con 0x7fbb98071dd0 2024-09-13T17:02:50.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 -- 172.21.15.39:0/1245683637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 msgr2=0x7fbb98154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:50.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1245683637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 0x7fbb98154070 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fbb98153d60 tx=0x7fbb8c01cec0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 -- 172.21.15.39:0/1245683637 shutdown_connections 2024-09-13T17:02:50.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1245683637 >> v2:172.21.15.84:3300/0 conn(0x7fbb980c47c0 0x7fbb981545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1245683637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 0x7fbb98154070 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1245683637 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb98151500 0x7fbb980c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 -- 172.21.15.39:0/1245683637 >> 172.21.15.39:0/1245683637 conn(0x7fbb980ab530 msgr2=0x7fbb980ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:50.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 -- 172.21.15.39:0/1245683637 shutdown_connections 2024-09-13T17:02:50.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 -- 172.21.15.39:0/1245683637 wait complete. 2024-09-13T17:02:50.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 Processor -- start 2024-09-13T17:02:50.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 -- start start 2024-09-13T17:02:50.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 0x7fbb98152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:50.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb980c47c0 0x7fbb98147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:50.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb98151500 0x7fbb9814a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:50.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9814bf60 con 0x7fbb98071dd0 2024-09-13T17:02:50.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98149bb0 con 0x7fbb98151500 2024-09-13T17:02:50.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb9d580640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98148640 con 0x7fbb980c47c0 2024-09-13T17:02:50.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb96ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 0x7fbb98152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:50.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb96ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 0x7fbb98152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39520/0 (socket says 172.21.15.39:39520) 2024-09-13T17:02:50.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb96ffd640 1 -- 172.21.15.39:0/1388049784 learned_addr learned my addr 172.21.15.39:0/1388049784 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:50.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb967fc640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb980c47c0 0x7fbb98147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:50.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb977fe640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb98151500 0x7fbb9814a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:50.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb96ffd640 1 -- 172.21.15.39:0/1388049784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb980c47c0 msgr2=0x7fbb98147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:50.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb96ffd640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb980c47c0 0x7fbb98147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb96ffd640 1 -- 172.21.15.39:0/1388049784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb98151500 msgr2=0x7fbb9814a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:50.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb96ffd640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb98151500 0x7fbb9814a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb96ffd640 1 -- 172.21.15.39:0/1388049784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c0094d0 con 0x7fbb98071dd0 2024-09-13T17:02:50.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb967fc640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb980c47c0 0x7fbb98147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:02:50.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.704+0000 7fbb96ffd640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 0x7fbb98152e80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbb88007e70 tx=0x7fbb8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:50.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb7f7fe640 1 -- 172.21.15.39:0/1388049784 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88014620 con 0x7fbb98071dd0 2024-09-13T17:02:50.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb7f7fe640 1 -- 172.21.15.39:0/1388049784 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb88014780 con 0x7fbb98071dd0 2024-09-13T17:02:50.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb98148920 con 0x7fbb98071dd0 2024-09-13T17:02:50.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb7f7fe640 1 -- 172.21.15.39:0/1388049784 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88014b30 con 0x7fbb98071dd0 2024-09-13T17:02:50.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb98148ee0 con 0x7fbb98071dd0 2024-09-13T17:02:50.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb98071170 con 0x7fbb98071dd0 2024-09-13T17:02:50.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb7f7fe640 1 -- 172.21.15.39:0/1388049784 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb88014c90 con 0x7fbb98071dd0 2024-09-13T17:02:50.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb7f7fe640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb84066690 0x7fbb84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:50.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb7f7fe640 1 -- 172.21.15.39:0/1388049784 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7fbb88086210 con 0x7fbb98071dd0 2024-09-13T17:02:50.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb967fc640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb84066690 0x7fbb84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:50.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.708+0000 7fbb967fc640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb84066690 0x7fbb84068b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0af000 tx=0x7fbb8c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:50.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.712+0000 7fbb7f7fe640 1 -- 172.21.15.39:0/1388049784 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbb88052d90 con 0x7fbb98071dd0 2024-09-13T17:02:50.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.912+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb980c3a00 con 0x7fbb98071dd0 2024-09-13T17:02:50.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.912+0000 7fbb7f7fe640 1 -- 172.21.15.39:0/1388049784 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7fbb880563b0 con 0x7fbb98071dd0 2024-09-13T17:02:50.912 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:02:50.912 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:02:50.913 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb84066690 msgr2=0x7fbb84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb84066690 0x7fbb84068b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0af000 tx=0x7fbb8c004740 comp rx=0 tx=0).stop 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 msgr2=0x7fbb98152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 0x7fbb98152e80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbb88007e70 tx=0x7fbb8800e4b0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 shutdown_connections 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbb84066690 0x7fbb84068b50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb98151500 0x7fbb9814a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbb980c47c0 0x7fbb98147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 --2- 172.21.15.39:0/1388049784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb98071dd0 0x7fbb98152e80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 >> 172.21.15.39:0/1388049784 conn(0x7fbb980ab530 msgr2=0x7fbb980ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 shutdown_connections 2024-09-13T17:02:50.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:50.916+0000 7fbb9d580640 1 -- 172.21.15.39:0/1388049784 wait complete. 2024-09-13T17:02:50.931 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:02:52.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:02:53.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe779233640 1 Processor -- start 2024-09-13T17:02:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe779233640 1 -- start start 2024-09-13T17:02:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe779233640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7740aabc0 0x7fe7740aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe779233640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7740c40b0 0x7fe77415c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe779233640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe774152a80 con 0x7fe77406c870 2024-09-13T17:02:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe779233640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7740c30f0 con 0x7fe7740aabc0 2024-09-13T17:02:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe779233640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe774151e00 con 0x7fe7740c40b0 2024-09-13T17:02:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe77359e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7740c40b0 0x7fe77415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe772d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7740aabc0 0x7fe7740aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe77359e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7740c40b0 0x7fe77415c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:59184/0 (socket says 172.21.15.39:59184) 2024-09-13T17:02:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe77359e640 1 -- 172.21.15.39:0/2258103858 learned_addr learned my addr 172.21.15.39:0/2258103858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe772d9d640 1 -- 172.21.15.39:0/2258103858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7740c40b0 msgr2=0x7fe77415c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:53.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe772d9d640 1 --2- 172.21.15.39:0/2258103858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7740c40b0 0x7fe77415c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe772d9d640 1 -- 172.21.15.39:0/2258103858 >> v1:172.21.15.84:6789/0 conn(0x7fe77406c870 legacy=0x7fe7740c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:53.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe772d9d640 1 -- 172.21.15.39:0/2258103858 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7741520e0 con 0x7fe7740aabc0 2024-09-13T17:02:53.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.068+0000 7fe77359e640 1 --2- 172.21.15.39:0/2258103858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7740c40b0 0x7fe77415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:02:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe772d9d640 1 --2- 172.21.15.39:0/2258103858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7740aabc0 0x7fe7740aafa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7680097c0 tx=0x7fe76801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd0baa2d3d779ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe771d9b640 1 -- 172.21.15.39:0/2258103858 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680ac640 con 0x7fe7740aabc0 2024-09-13T17:02:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe771d9b640 1 -- 172.21.15.39:0/2258103858 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7680ac7a0 con 0x7fe7740aabc0 2024-09-13T17:02:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe771d9b640 1 -- 172.21.15.39:0/2258103858 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680acb30 con 0x7fe7740aabc0 2024-09-13T17:02:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe779233640 1 -- 172.21.15.39:0/2258103858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7740aabc0 msgr2=0x7fe7740aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe779233640 1 --2- 172.21.15.39:0/2258103858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7740aabc0 0x7fe7740aafa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7680097c0 tx=0x7fe76801c3e0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe779233640 1 -- 172.21.15.39:0/2258103858 shutdown_connections 2024-09-13T17:02:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe779233640 1 --2- 172.21.15.39:0/2258103858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7740c40b0 0x7fe77415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe779233640 1 --2- 172.21.15.39:0/2258103858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7740aabc0 0x7fe7740aafa0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe779233640 1 -- 172.21.15.39:0/2258103858 >> 172.21.15.39:0/2258103858 conn(0x7fe774069e10 msgr2=0x7fe77406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe779233640 1 -- 172.21.15.39:0/2258103858 shutdown_connections 2024-09-13T17:02:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.072+0000 7fe779233640 1 -- 172.21.15.39:0/2258103858 wait complete. 2024-09-13T17:02:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 Processor -- start 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 -- start start 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77406c870 0x7fe7740bee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7740aabc0 0x7fe7741ec350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7740c40b0 0x7fe7741ec890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740a93c0 con 0x7fe7740c40b0 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7741f0040 con 0x7fe7740aabc0 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7740a95a0 con 0x7fe77406c870 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7740c40b0 0x7fe7741ec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7740c40b0 0x7fe7741ec890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39530/0 (socket says 172.21.15.39:39530) 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77359e640 1 -- 172.21.15.39:0/2630479978 learned_addr learned my addr 172.21.15.39:0/2630479978 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe772d9d640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77406c870 0x7fe7740bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77259c640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7740aabc0 0x7fe7741ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77359e640 1 -- 172.21.15.39:0/2630479978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77406c870 msgr2=0x7fe7740bee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77359e640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77406c870 0x7fe7740bee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77359e640 1 -- 172.21.15.39:0/2630479978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7740aabc0 msgr2=0x7fe7741ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77359e640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7740aabc0 0x7fe7741ec350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77359e640 1 -- 172.21.15.39:0/2630479978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c0094d0 con 0x7fe7740c40b0 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe772d9d640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77406c870 0x7fe7740bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe77359e640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7740c40b0 0x7fe7741ec890 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00e960 tx=0x7fe76c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe75b7fe640 1 -- 172.21.15.39:0/2630479978 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c009680 con 0x7fe7740c40b0 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe768009420 con 0x7fe7740c40b0 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe75b7fe640 1 -- 172.21.15.39:0/2630479978 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c00ca00 con 0x7fe7740c40b0 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe75b7fe640 1 -- 172.21.15.39:0/2630479978 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00cd90 con 0x7fe7740c40b0 2024-09-13T17:02:53.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7741efcd0 con 0x7fe7740c40b0 2024-09-13T17:02:53.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7740aafa0 con 0x7fe7740c40b0 2024-09-13T17:02:53.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe75b7fe640 1 -- 172.21.15.39:0/2630479978 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe76c013470 con 0x7fe7740c40b0 2024-09-13T17:02:53.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe75b7fe640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe760066690 0x7fe760068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:53.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe75b7fe640 1 -- 172.21.15.39:0/2630479978 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7fe76c084980 con 0x7fe7740c40b0 2024-09-13T17:02:53.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.076+0000 7fe772d9d640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe760066690 0x7fe760068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:53.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.080+0000 7fe772d9d640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe760066690 0x7fe760068b50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe7680ae000 tx=0x7fe7680a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:53.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.084+0000 7fe75b7fe640 1 -- 172.21.15.39:0/2630479978 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe76c0513a0 con 0x7fe7740c40b0 2024-09-13T17:02:53.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.280+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe77406c650 con 0x7fe7740c40b0 2024-09-13T17:02:53.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.280+0000 7fe75b7fe640 1 -- 172.21.15.39:0/2630479978 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7fe76c0549c0 con 0x7fe7740c40b0 2024-09-13T17:02:53.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:02:53.281 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:02:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:02:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe760066690 msgr2=0x7fe760068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe760066690 0x7fe760068b50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe7680ae000 tx=0x7fe7680a7000 comp rx=0 tx=0).stop 2024-09-13T17:02:53.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7740c40b0 msgr2=0x7fe7741ec890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:53.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7740c40b0 0x7fe7741ec890 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00e960 tx=0x7fe76c00ee20 comp rx=0 tx=0).stop 2024-09-13T17:02:53.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 shutdown_connections 2024-09-13T17:02:53.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe760066690 0x7fe760068b50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7740aabc0 0x7fe7741ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7740c40b0 0x7fe7741ec890 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 --2- 172.21.15.39:0/2630479978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe77406c870 0x7fe7740bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:53.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 >> 172.21.15.39:0/2630479978 conn(0x7fe774069e10 msgr2=0x7fe774152ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:53.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 shutdown_connections 2024-09-13T17:02:53.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:53.284+0000 7fe779233640 1 -- 172.21.15.39:0/2630479978 wait complete. 2024-09-13T17:02:53.298 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:02:55.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:02:55.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.432+0000 7f24e5fed640 1 Processor -- start 2024-09-13T17:02:55.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24e5fed640 1 -- start start 2024-09-13T17:02:55.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24e5fed640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f24e00c5330 0x7f24e00c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:55.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24e5fed640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e01546c0 con 0x7f24e006c090 2024-09-13T17:02:55.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24e5fed640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e00c49f0 con 0x7f24e0150f80 2024-09-13T17:02:55.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24e5fed640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e00c19a0 con 0x7f24e00c5330 2024-09-13T17:02:55.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24dffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f24e006c090 0x7f24e006c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40236/0 (socket says 172.21.15.39:40236) 2024-09-13T17:02:55.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24dffff640 1 -- 172.21.15.39:0/3928274132 learned_addr learned my addr 172.21.15.39:0/3928274132 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:02:55.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24deffd640 1 --2- 172.21.15.39:0/3928274132 >> v2:172.21.15.84:3300/0 conn(0x7f24e00c5330 0x7f24e00c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:55.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144343035 0 0) 0x7f24e01546c0 con 0x7f24e006c090 2024-09-13T17:02:55.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24cc003450 con 0x7f24e006c090 2024-09-13T17:02:55.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 870489519 0 0) 0x7f24cc003450 con 0x7f24e006c090 2024-09-13T17:02:55.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 >> v2:172.21.15.84:3300/0 conn(0x7f24e00c5330 msgr2=0x7f24e00c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:55.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 --2- 172.21.15.39:0/3928274132 >> v2:172.21.15.84:3300/0 conn(0x7f24e00c5330 0x7f24e00c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 >> v1:172.21.15.84:6789/0 conn(0x7f24e0150f80 legacy=0x7f24e0153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:55.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e01543e0 con 0x7f24e006c090 2024-09-13T17:02:55.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f24d8001a00 con 0x7f24e006c090 2024-09-13T17:02:55.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 >> v1:172.21.15.39:6789/0 conn(0x7f24e006c090 legacy=0x7f24e006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:55.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24cc003aa0 0x7f24cc003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24cc008860 0x7f24cc00acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24cc00da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e00c19a0 con 0x7f24cc00b570 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc00e310 con 0x7f24cc003aa0 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc00e4d0 con 0x7f24cc008860 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24dffff640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24cc00da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24df7fe640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24cc003aa0 0x7f24cc003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24deffd640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24cc008860 0x7f24cc00acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24dffff640 1 -- 172.21.15.39:0/3928274132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24cc008860 msgr2=0x7f24cc00acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:55.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24dffff640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24cc008860 0x7f24cc00acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24dffff640 1 -- 172.21.15.39:0/3928274132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24cc003aa0 msgr2=0x7f24cc003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:55.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24dffff640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24cc003aa0 0x7f24cc003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24dffff640 1 -- 172.21.15.39:0/3928274132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc003450 con 0x7f24cc00b570 2024-09-13T17:02:55.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.436+0000 7f24deffd640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24cc008860 0x7f24cc00acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:02:55.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24dffff640 1 -- 172.21.15.39:0/3928274132 reap_dead start 2024-09-13T17:02:55.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24dffff640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24cc00da30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f24d800a830 tx=0x7f24d801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25158edac080140 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:55.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d80aa6f0 con 0x7f24cc00b570 2024-09-13T17:02:55.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d80aa890 con 0x7f24cc00b570 2024-09-13T17:02:55.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24de7fc640 1 -- 172.21.15.39:0/3928274132 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d80aac40 con 0x7f24cc00b570 2024-09-13T17:02:55.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24e5fed640 1 -- 172.21.15.39:0/3928274132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 msgr2=0x7f24cc00da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:55.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24e5fed640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24cc00da30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f24d800a830 tx=0x7f24d801fac0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24e5fed640 1 -- 172.21.15.39:0/3928274132 shutdown_connections 2024-09-13T17:02:55.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24e5fed640 1 --2- 172.21.15.39:0/3928274132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24cc00da30 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24e5fed640 1 -- 172.21.15.39:0/3928274132 >> 172.21.15.39:0/3928274132 conn(0x7f24e0069e10 msgr2=0x7f24e006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:55.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24e5fed640 1 -- 172.21.15.39:0/3928274132 shutdown_connections 2024-09-13T17:02:55.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.440+0000 7f24e5fed640 1 -- 172.21.15.39:0/3928274132 wait complete. 2024-09-13T17:02:55.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 Processor -- start 2024-09-13T17:02:55.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 -- start start 2024-09-13T17:02:55.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24e014f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:55.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24e014fee0 0x7f24e01e5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24e01e57a0 0x7f24e0152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e01e4e50 con 0x7f24cc00b570 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e00a6880 con 0x7f24e014fee0 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0070320 con 0x7f24e01e57a0 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24df7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24e014f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24df7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24e014f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39562/0 (socket says 172.21.15.39:39562) 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24df7fe640 1 -- 172.21.15.39:0/1865338531 learned_addr learned my addr 172.21.15.39:0/1865338531 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24dffff640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24e01e57a0 0x7f24e0152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24deffd640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24e014fee0 0x7f24e01e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24df7fe640 1 -- 172.21.15.39:0/1865338531 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24e01e57a0 msgr2=0x7f24e0152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24df7fe640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24e01e57a0 0x7f24e0152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24df7fe640 1 -- 172.21.15.39:0/1865338531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24e014fee0 msgr2=0x7f24e01e5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24df7fe640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24e014fee0 0x7f24e01e5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24df7fe640 1 -- 172.21.15.39:0/1865338531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00a5510 con 0x7f24cc00b570 2024-09-13T17:02:55.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24deffd640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24e014fee0 0x7f24e01e5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24df7fe640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24e014f9a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f24c800f570 tx=0x7f24c800fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24d7fff640 1 -- 172.21.15.39:0/1865338531 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8005320 con 0x7f24cc00b570 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24d7fff640 1 -- 172.21.15.39:0/1865338531 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c8005480 con 0x7f24cc00b570 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e01e91b0 con 0x7f24cc00b570 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24d7fff640 1 -- 172.21.15.39:0/1865338531 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8007250 con 0x7f24cc00b570 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e00a5ab0 con 0x7f24cc00b570 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e00c2690 con 0x7f24cc00b570 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24d7fff640 1 -- 172.21.15.39:0/1865338531 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f24c80073b0 con 0x7f24cc00b570 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24d7fff640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f24c0066690 0x7f24c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24d7fff640 1 -- 172.21.15.39:0/1865338531 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7f24c8083ff0 con 0x7f24cc00b570 2024-09-13T17:02:55.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.444+0000 7f24deffd640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f24c0066690 0x7f24c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:55.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.448+0000 7f24deffd640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f24c0066690 0x7f24c0068b50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f24e0150ca0 tx=0x7f24d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:55.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.452+0000 7f24d7fff640 1 -- 172.21.15.39:0/1865338531 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f24c8050ac0 con 0x7f24cc00b570 2024-09-13T17:02:55.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.644+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24e00c3c00 con 0x7f24cc00b570 2024-09-13T17:02:55.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24d7fff640 1 -- 172.21.15.39:0/1865338531 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7f24c80540e0 con 0x7f24cc00b570 2024-09-13T17:02:55.646 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:02:55.646 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:02:55.646 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:02:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f24c0066690 msgr2=0x7f24c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f24c0066690 0x7f24c0068b50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f24e0150ca0 tx=0x7f24d0008040 comp rx=0 tx=0).stop 2024-09-13T17:02:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 msgr2=0x7f24e014f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24e014f9a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f24c800f570 tx=0x7f24c800fa30 comp rx=0 tx=0).stop 2024-09-13T17:02:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 shutdown_connections 2024-09-13T17:02:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f24c0066690 0x7f24c0068b50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24e014fee0 0x7f24e01e5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24e01e57a0 0x7f24e0152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 --2- 172.21.15.39:0/1865338531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc00b570 0x7f24e014f9a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 >> 172.21.15.39:0/1865338531 conn(0x7f24e0069e10 msgr2=0x7f24e014ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 shutdown_connections 2024-09-13T17:02:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:55.648+0000 7f24e5fed640 1 -- 172.21.15.39:0/1865338531 wait complete. 2024-09-13T17:02:55.667 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:02:57.668 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:02:57.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.804+0000 7fa754a82640 1 Processor -- start 2024-09-13T17:02:57.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.804+0000 7fa754a82640 1 -- start start 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa754a82640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa7500c20d0 0x7fa7500c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa754a82640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa75006d370 0x7fa750157f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa754a82640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa750157370 con 0x7fa7500c1790 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa754a82640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7500c04a0 con 0x7fa75006d370 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa754a82640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa750153df0 con 0x7fa7500c20d0 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74ed9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa75006d370 0x7fa750157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74dd9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa7500c1790 0x7fa750157800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40242/0 (socket says 172.21.15.39:40242) 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74dd9b640 1 -- 172.21.15.39:0/496243757 learned_addr learned my addr 172.21.15.39:0/496243757 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74e59c640 1 --2- 172.21.15.39:0/496243757 >> v2:172.21.15.84:3300/0 conn(0x7fa7500c20d0 0x7fa7500c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74ed9d640 1 -- 172.21.15.39:0/496243757 >> v2:172.21.15.84:3300/0 conn(0x7fa7500c20d0 msgr2=0x7fa7500c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74ed9d640 1 --2- 172.21.15.39:0/496243757 >> v2:172.21.15.84:3300/0 conn(0x7fa7500c20d0 0x7fa7500c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74ed9d640 1 -- 172.21.15.39:0/496243757 >> v1:172.21.15.39:6789/0 conn(0x7fa7500c1790 legacy=0x7fa750157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:57.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74ed9d640 1 -- 172.21.15.39:0/496243757 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75006c700 con 0x7fa75006d370 2024-09-13T17:02:57.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74ed9d640 1 --2- 172.21.15.39:0/496243757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa75006d370 0x7fa750157f10 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa7500c2f50 tx=0x7fa74801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877112c0034a01f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:57.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74d59a640 1 -- 172.21.15.39:0/496243757 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480ab650 con 0x7fa75006d370 2024-09-13T17:02:57.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74d59a640 1 -- 172.21.15.39:0/496243757 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7480ab7b0 con 0x7fa75006d370 2024-09-13T17:02:57.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.808+0000 7fa74d59a640 1 -- 172.21.15.39:0/496243757 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480abb40 con 0x7fa75006d370 2024-09-13T17:02:57.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- 172.21.15.39:0/496243757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa75006d370 msgr2=0x7fa750157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:57.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 --2- 172.21.15.39:0/496243757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa75006d370 0x7fa750157f10 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa7500c2f50 tx=0x7fa74801c450 comp rx=0 tx=0).stop 2024-09-13T17:02:57.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- 172.21.15.39:0/496243757 shutdown_connections 2024-09-13T17:02:57.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 --2- 172.21.15.39:0/496243757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa75006d370 0x7fa750157f10 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:57.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 --2- 172.21.15.39:0/496243757 >> v2:172.21.15.84:3300/0 conn(0x7fa7500c20d0 0x7fa7500c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:57.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- 172.21.15.39:0/496243757 >> 172.21.15.39:0/496243757 conn(0x7fa750069e10 msgr2=0x7fa75006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:57.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- 172.21.15.39:0/496243757 shutdown_connections 2024-09-13T17:02:57.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- 172.21.15.39:0/496243757 wait complete. 2024-09-13T17:02:57.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 Processor -- start 2024-09-13T17:02:57.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- start start 2024-09-13T17:02:57.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa75006d370 0x7fa7500716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:57.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7500c1790 0x7fa750159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:57.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7500c20d0 0x7fa75014a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:57.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75014e2b0 con 0x7fa7500c1790 2024-09-13T17:02:57.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa750158b40 con 0x7fa7500c20d0 2024-09-13T17:02:57.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa750158580 con 0x7fa75006d370 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa74dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7500c1790 0x7fa750159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa74dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7500c1790 0x7fa750159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39576/0 (socket says 172.21.15.39:39576) 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa74dd9b640 1 -- 172.21.15.39:0/2351088459 learned_addr learned my addr 172.21.15.39:0/2351088459 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa74dd9b640 1 -- 172.21.15.39:0/2351088459 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa75006d370 msgr2=0x7fa7500716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa74dd9b640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa75006d370 0x7fa7500716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa74dd9b640 1 -- 172.21.15.39:0/2351088459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7500c20d0 msgr2=0x7fa75014a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa74dd9b640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7500c20d0 0x7fa75014a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa74dd9b640 1 -- 172.21.15.39:0/2351088459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa748009420 con 0x7fa7500c1790 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa74dd9b640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7500c1790 0x7fa750159360 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa74000cbd0 tx=0x7fa74000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:57.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa746ffd640 1 -- 172.21.15.39:0/2351088459 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74000ad40 con 0x7fa7500c1790 2024-09-13T17:02:57.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa746ffd640 1 -- 172.21.15.39:0/2351088459 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa740012400 con 0x7fa7500c1790 2024-09-13T17:02:57.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa746ffd640 1 -- 172.21.15.39:0/2351088459 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74000aea0 con 0x7fa7500c1790 2024-09-13T17:02:57.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa750158860 con 0x7fa7500c1790 2024-09-13T17:02:57.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa750071be0 con 0x7fa7500c1790 2024-09-13T17:02:57.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.812+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75014e3f0 con 0x7fa7500c1790 2024-09-13T17:02:57.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.816+0000 7fa746ffd640 1 -- 172.21.15.39:0/2351088459 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa740007070 con 0x7fa7500c1790 2024-09-13T17:02:57.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.816+0000 7fa746ffd640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7300666c0 0x7fa730068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:02:57.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.816+0000 7fa746ffd640 1 -- 172.21.15.39:0/2351088459 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 8099+0+0 (secure 0 0 0) 0x7fa740084b00 con 0x7fa7500c1790 2024-09-13T17:02:57.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.816+0000 7fa74e59c640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7300666c0 0x7fa730068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:02:57.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.816+0000 7fa74e59c640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7300666c0 0x7fa730068b80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa7380098f0 tx=0x7fa738008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:02:57.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:57.820+0000 7fa746ffd640 1 -- 172.21.15.39:0/2351088459 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa75014e3f0 con 0x7fa7500c1790 2024-09-13T17:02:58.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7500c24b0 con 0x7fa7500c1790 2024-09-13T17:02:58.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa746ffd640 1 -- 172.21.15.39:0/2351088459 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5926 (secure 0 0 0) 0x7fa740017020 con 0x7fa7500c1790 2024-09-13T17:02:58.015 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:02:58.015 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6585,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4226":{"gid":4226,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":206,"addr":"172.21.15.39:6837/835951775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":835951775},{"type":"v1","addr":"172.21.15.39:6837","nonce":835951775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:02:58.016 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-13T17:02:58.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7300666c0 msgr2=0x7fa730068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7300666c0 0x7fa730068b80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa7380098f0 tx=0x7fa738008040 comp rx=0 tx=0).stop 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7500c1790 msgr2=0x7fa750159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7500c1790 0x7fa750159360 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa74000cbd0 tx=0x7fa74000a430 comp rx=0 tx=0).stop 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 shutdown_connections 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7300666c0 0x7fa730068b80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7500c20d0 0x7fa75014a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7500c1790 0x7fa750159360 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 --2- 172.21.15.39:0/2351088459 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa75006d370 0x7fa7500716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 >> 172.21.15.39:0/2351088459 conn(0x7fa750069e10 msgr2=0x7fa750156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 shutdown_connections 2024-09-13T17:02:58.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:02:58.016+0000 7fa754a82640 1 -- 172.21.15.39:0/2351088459 wait complete. 2024-09-13T17:02:58.034 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:02:58.624 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:02:58.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:03:00.035 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfeeebe640 1 Processor -- start 2024-09-13T17:03:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfeeebe640 1 -- start start 2024-09-13T17:03:00.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfeeebe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdfe80aabc0 0x7fdfe80aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfeeebe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdfe806c870 0x7fdfe80c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfeeebe640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe8152a80 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfeeebe640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe80c30f0 con 0x7fdfe80aabc0 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfeeebe640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe8151e00 con 0x7fdfe806c870 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfecc5a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdfe80aabc0 0x7fdfe80aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfecc5a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdfe80aabc0 0x7fdfe80aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59030/0 (socket says 172.21.15.39:59030) 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfecc5a640 1 -- 172.21.15.39:0/756147937 learned_addr learned my addr 172.21.15.39:0/756147937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfe7fff640 1 --2- 172.21.15.39:0/756147937 >> v2:172.21.15.84:3300/0 conn(0x7fdfe806c870 0x7fdfe80c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfecc5a640 1 -- 172.21.15.39:0/756147937 >> v2:172.21.15.84:3300/0 conn(0x7fdfe806c870 msgr2=0x7fdfe80c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfecc5a640 1 --2- 172.21.15.39:0/756147937 >> v2:172.21.15.84:3300/0 conn(0x7fdfe806c870 0x7fdfe80c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfecc5a640 1 -- 172.21.15.39:0/756147937 >> v1:172.21.15.39:6789/0 conn(0x7fdfe80c3ee0 legacy=0x7fdfe81550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.172+0000 7fdfecc5a640 1 -- 172.21.15.39:0/756147937 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe81520e0 con 0x7fdfe80aabc0 2024-09-13T17:03:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfecc5a640 1 --2- 172.21.15.39:0/756147937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80aabc0 0x7fdfe80aafa0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80bfe70 tx=0x7fdfd801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf0eebc67d34d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfe77fe640 1 -- 172.21.15.39:0/756147937 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80ab600 con 0x7fdfe80aabc0 2024-09-13T17:03:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfe77fe640 1 -- 172.21.15.39:0/756147937 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd80a7050 con 0x7fdfe80aabc0 2024-09-13T17:03:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfe77fe640 1 -- 172.21.15.39:0/756147937 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80ab980 con 0x7fdfe80aabc0 2024-09-13T17:03:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 -- 172.21.15.39:0/756147937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80aabc0 msgr2=0x7fdfe80aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 --2- 172.21.15.39:0/756147937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80aabc0 0x7fdfe80aafa0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80bfe70 tx=0x7fdfd801c460 comp rx=0 tx=0).stop 2024-09-13T17:03:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 -- 172.21.15.39:0/756147937 shutdown_connections 2024-09-13T17:03:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 --2- 172.21.15.39:0/756147937 >> v2:172.21.15.84:3300/0 conn(0x7fdfe806c870 0x7fdfe80c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 --2- 172.21.15.39:0/756147937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80aabc0 0x7fdfe80aafa0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 -- 172.21.15.39:0/756147937 >> 172.21.15.39:0/756147937 conn(0x7fdfe8069e10 msgr2=0x7fdfe806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 -- 172.21.15.39:0/756147937 shutdown_connections 2024-09-13T17:03:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 -- 172.21.15.39:0/756147937 wait complete. 2024-09-13T17:03:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 Processor -- start 2024-09-13T17:03:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 -- start start 2024-09-13T17:03:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfe806c870 0x7fdfe814aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe80aabc0 0x7fdfe80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80c3ee0 0x7fdfe81efde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80a8d90 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe814e980 con 0x7fdfe80aabc0 2024-09-13T17:03:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfeeebe640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe814cb00 con 0x7fdfe806c870 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfed45b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80c3ee0 0x7fdfe81efde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfed45b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80c3ee0 0x7fdfe81efde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59040/0 (socket says 172.21.15.39:59040) 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfed45b640 1 -- 172.21.15.39:0/1591554890 learned_addr learned my addr 172.21.15.39:0/1591554890 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfecc5a640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfe806c870 0x7fdfe814aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfe7fff640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe80aabc0 0x7fdfe80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfed45b640 1 -- 172.21.15.39:0/1591554890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfe806c870 msgr2=0x7fdfe814aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfed45b640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfe806c870 0x7fdfe814aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfed45b640 1 -- 172.21.15.39:0/1591554890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe80aabc0 msgr2=0x7fdfe80a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfed45b640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe80aabc0 0x7fdfe80a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.176+0000 7fdfed45b640 1 -- 172.21.15.39:0/1591554890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd80094d0 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfed45b640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80c3ee0 0x7fdfe81efde0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000c770 tx=0x7fdfe000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfe57fa640 1 -- 172.21.15.39:0/1591554890 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0003b80 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfe57fa640 1 -- 172.21.15.39:0/1591554890 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0003ce0 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfe57fa640 1 -- 172.21.15.39:0/1591554890 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000aa90 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe80a8740 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe80a8180 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe806c650 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfe57fa640 1 -- 172.21.15.39:0/1591554890 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdfe000abf0 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfe57fa640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdfd4066690 0x7fdfd4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfe57fa640 1 -- 172.21.15.39:0/1591554890 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7fdfe0084650 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfecc5a640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdfd4066690 0x7fdfd4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.180+0000 7fdfecc5a640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdfd4066690 0x7fdfd4068b50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80ad000 tx=0x7fdfd80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.184+0000 7fdfe57fa640 1 -- 172.21.15.39:0/1591554890 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdfe0051100 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.384+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfe8152a80 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.384+0000 7fdfe57fa640 1 -- 172.21.15.39:0/1591554890 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+5170 (secure 0 0 0) 0x7fdfe8152a80 con 0x7fdfe80c3ee0 2024-09-13T17:03:00.385 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:00.385 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:02:59.731281+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:00.385 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 97 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdfd4066690 msgr2=0x7fdfd4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdfd4066690 0x7fdfd4068b50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80ad000 tx=0x7fdfd80a6000 comp rx=0 tx=0).stop 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80c3ee0 msgr2=0x7fdfe81efde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80c3ee0 0x7fdfe81efde0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000c770 tx=0x7fdfe000cc30 comp rx=0 tx=0).stop 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 shutdown_connections 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdfd4066690 0x7fdfd4068b50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdfe80aabc0 0x7fdfe80a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe80c3ee0 0x7fdfe81efde0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 --2- 172.21.15.39:0/1591554890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdfe806c870 0x7fdfe814aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 >> 172.21.15.39:0/1591554890 conn(0x7fdfe8069e10 msgr2=0x7fdfe8154ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:00.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 shutdown_connections 2024-09-13T17:03:00.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.388+0000 7fdfeeebe640 1 -- 172.21.15.39:0/1591554890 wait complete. 2024-09-13T17:03:00.403 INFO:tasks.mds_thrash.fs.[a]:mds.c down, removed from mdsmap 2024-09-13T17:03:00.403 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:03:00.404 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:00.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eb1d80640 1 Processor -- start 2024-09-13T17:03:00.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eb1d80640 1 -- start start 2024-09-13T17:03:00.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eb1d80640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3eac0aabc0 0x7f3eac0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eb1d80640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3eac06c870 0x7f3eac0c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eb1d80640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac152a80 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eb1d80640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac0c30f0 con 0x7f3eac06c870 2024-09-13T17:03:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eb1d80640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac151e00 con 0x7f3eac0aabc0 2024-09-13T17:03:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eaaffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3eac06c870 0x7f3eac0c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eabfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f3eac0c3ee0 0x7f3eac1550e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53340/0 (socket says 172.21.15.39:53340) 2024-09-13T17:03:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eabfff640 1 -- 172.21.15.39:0/3428510388 learned_addr learned my addr 172.21.15.39:0/3428510388 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eab7fe640 1 --2- 172.21.15.39:0/3428510388 >> v2:172.21.15.84:3300/0 conn(0x7f3eac0aabc0 0x7f3eac0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eaaffd640 1 -- 172.21.15.39:0/3428510388 >> v2:172.21.15.84:3300/0 conn(0x7f3eac0aabc0 msgr2=0x7f3eac0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:00.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eaaffd640 1 --2- 172.21.15.39:0/3428510388 >> v2:172.21.15.84:3300/0 conn(0x7f3eac0aabc0 0x7f3eac0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eaaffd640 1 -- 172.21.15.39:0/3428510388 >> v1:172.21.15.39:6789/0 conn(0x7f3eac0c3ee0 legacy=0x7f3eac1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eaaffd640 1 -- 172.21.15.39:0/3428510388 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac1520e0 con 0x7f3eac06c870 2024-09-13T17:03:00.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.520+0000 7f3eaa7fc640 1 -- 172.21.15.39:0/3428510388 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366240001 0 0) 0x7f3e9c001120 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eaaffd640 1 --2- 172.21.15.39:0/3428510388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac06c870 0x7f3eac0c39a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0c33a0 tx=0x7f3e9401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0301bcd0c25b8c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:00.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eaa7fc640 1 -- 172.21.15.39:0/3428510388 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940ab600 con 0x7f3eac06c870 2024-09-13T17:03:00.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eaa7fc640 1 -- 172.21.15.39:0/3428510388 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e940a7050 con 0x7f3eac06c870 2024-09-13T17:03:00.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eaa7fc640 1 -- 172.21.15.39:0/3428510388 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940ab980 con 0x7f3eac06c870 2024-09-13T17:03:00.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3428510388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac06c870 msgr2=0x7f3eac0c39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 --2- 172.21.15.39:0/3428510388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac06c870 0x7f3eac0c39a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0c33a0 tx=0x7f3e9401c3c0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3428510388 shutdown_connections 2024-09-13T17:03:00.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 --2- 172.21.15.39:0/3428510388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac06c870 0x7f3eac0c39a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 --2- 172.21.15.39:0/3428510388 >> v2:172.21.15.84:3300/0 conn(0x7f3eac0aabc0 0x7f3eac0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3428510388 >> 172.21.15.39:0/3428510388 conn(0x7f3eac069e10 msgr2=0x7f3eac06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:00.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3428510388 shutdown_connections 2024-09-13T17:03:00.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3428510388 wait complete. 2024-09-13T17:03:00.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 Processor -- start 2024-09-13T17:03:00.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.524+0000 7f3eb1d80640 1 -- start start 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eb1d80640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac06c870 0x7f3eac0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eb1d80640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac0aabc0 0x7f3eac14bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eb1d80640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac0c3ee0 0x7f3eac14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eb1d80640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0bb820 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eb1d80640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0bba00 con 0x7f3eac0aabc0 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eb1d80640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0bb1f0 con 0x7f3eac06c870 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eabfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac0c3ee0 0x7f3eac14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eabfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac0c3ee0 0x7f3eac14c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59050/0 (socket says 172.21.15.39:59050) 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eabfff640 1 -- 172.21.15.39:0/3999790814 learned_addr learned my addr 172.21.15.39:0/3999790814 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:00.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eab7fe640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac06c870 0x7f3eac0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eaaffd640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac0aabc0 0x7f3eac14bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eabfff640 1 -- 172.21.15.39:0/3999790814 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac06c870 msgr2=0x7f3eac0c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eabfff640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac06c870 0x7f3eac0c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eabfff640 1 -- 172.21.15.39:0/3999790814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac0aabc0 msgr2=0x7f3eac14bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eabfff640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac0aabc0 0x7f3eac14bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eabfff640 1 -- 172.21.15.39:0/3999790814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e94009420 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eab7fe640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac06c870 0x7f3eac0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:00.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eabfff640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac0c3ee0 0x7f3eac14c180 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00c7b0 tx=0x7f3e9c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:00.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3ea3fff640 1 -- 172.21.15.39:0/3999790814 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c0072d0 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3ea3fff640 1 -- 172.21.15.39:0/3999790814 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c007450 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac1eb4e0 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3ea3fff640 1 -- 172.21.15.39:0/3999790814 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c001d20 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac0bb440 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac0c39a0 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3ea3fff640 1 -- 172.21.15.39:0/3999790814 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e9c0047f0 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3ea3fff640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c066690 0x7f3e8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:00.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3ea3fff640 1 -- 172.21.15.39:0/3999790814 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7f3e9c083ee0 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eab7fe640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c066690 0x7f3e8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:00.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.528+0000 7f3eab7fe640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c066690 0x7f3e8c068b50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400e4a0 tx=0x7f3ea4007650 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:00.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.532+0000 7f3ea3fff640 1 -- 172.21.15.39:0/3999790814 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3e9c0508e0 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.720+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eac152a80 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.720+0000 7f3ea3fff640 1 -- 172.21.15.39:0/3999790814 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+5170 (secure 0 0 0) 0x7f3eac152a80 con 0x7f3eac0c3ee0 2024-09-13T17:03:00.720 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:00.720 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:02:59.731281+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:00.721 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 97 2024-09-13T17:03:00.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c066690 msgr2=0x7f3e8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c066690 0x7f3e8c068b50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400e4a0 tx=0x7f3ea4007650 comp rx=0 tx=0).stop 2024-09-13T17:03:00.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac0c3ee0 msgr2=0x7f3eac14c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:00.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac0c3ee0 0x7f3eac14c180 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00c7b0 tx=0x7f3e9c00cc70 comp rx=0 tx=0).stop 2024-09-13T17:03:00.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 shutdown_connections 2024-09-13T17:03:00.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3e8c066690 0x7f3e8c068b50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3eac0aabc0 0x7f3eac14bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eac0c3ee0 0x7f3eac14c180 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 --2- 172.21.15.39:0/3999790814 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3eac06c870 0x7f3eac0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:00.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 >> 172.21.15.39:0/3999790814 conn(0x7f3eac069e10 msgr2=0x7f3eac155030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:00.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 shutdown_connections 2024-09-13T17:03:00.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:00.724+0000 7f3eb1d80640 1 -- 172.21.15.39:0/3999790814 wait complete. 2024-09-13T17:03:00.739 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 97, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 97, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6585": { "addr": "172.21.15.39:6835/611669592", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 611669592, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 611669592, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6585, "incarnation": 97, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 43, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:02:59.731281+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6585 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 94, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_5938": { "addr": "172.21.15.84:6835/828722052", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 828722052, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 828722052, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5938, "incarnation": 91, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 22 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:01:27.989091+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5938 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/440028459", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 440028459, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 440028459, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 96, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5561, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:03:02.740 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:02.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.876+0000 7f5875538640 1 Processor -- start 2024-09-13T17:03:02.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.876+0000 7f5875538640 1 -- start start 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f5875538640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f587014f1b0 0x7f5870151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f5875538640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587014f590 con 0x7f58700c7470 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f5875538640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587006eae0 con 0x7f587006e5e0 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f5875538640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587006f660 con 0x7f587014f1b0 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f58700c7470 0x7f58700c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53342/0 (socket says 172.21.15.39:53342) 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586f7fe640 1 -- 172.21.15.39:0/3152816440 learned_addr learned my addr 172.21.15.39:0/3152816440 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586effd640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f587014f1b0 0x7f5870151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298068285 0 0) 0x7f587014f590 con 0x7f58700c7470 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5858003530 con 0x7f58700c7470 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1091048295 0 0) 0x7f5858003530 con 0x7f58700c7470 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f587014f1b0 msgr2=0x7f5870151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:02.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f587014f1b0 0x7f5870151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 >> v1:172.21.15.84:6789/0 conn(0x7f587006e5e0 legacy=0x7f58700714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587006edc0 con 0x7f58700c7470 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f5860001680 con 0x7f58700c7470 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 >> v1:172.21.15.39:6789/0 conn(0x7f58700c7470 legacy=0x7f58700c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58580039b0 0x7f5858003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f587014f1b0 0x7f58580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5858004d10 0x7f5858005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587006f660 con 0x7f5858004d10 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585800ba40 con 0x7f58580039b0 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585800bc00 con 0x7f587014f1b0 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586f7fe640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5858004d10 0x7f5858005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586e7fc640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f587014f1b0 0x7f58580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:02.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586effd640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58580039b0 0x7f5858003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:02.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586f7fe640 1 -- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f587014f1b0 msgr2=0x7f58580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:02.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586f7fe640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f587014f1b0 0x7f58580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:02.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586f7fe640 1 -- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58580039b0 msgr2=0x7f5858003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:02.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586f7fe640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58580039b0 0x7f5858003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:02.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586f7fe640 1 -- 172.21.15.39:0/3152816440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858003530 con 0x7f5858004d10 2024-09-13T17:03:02.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586e7fc640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f587014f1b0 0x7f58580047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:02.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.880+0000 7f586effd640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58580039b0 0x7f5858003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586f7fe640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5858004d10 0x7f5858005190 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f586000c210 tx=0x7f586001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a67bc1f176c643 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58600a5b40 con 0x7f5858004d10 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58600a5ca0 con 0x7f5858004d10 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586dffb640 1 -- 172.21.15.39:0/3152816440 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860005b20 con 0x7f5858004d10 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 -- 172.21.15.39:0/3152816440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5858004d10 msgr2=0x7f5858005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 --2- 172.21.15.39:0/3152816440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5858004d10 0x7f5858005190 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f586000c210 tx=0x7f586001cc40 comp rx=0 tx=0).stop 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586f7fe640 1 -- 172.21.15.39:0/3152816440 reap_dead start 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 -- 172.21.15.39:0/3152816440 shutdown_connections 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 -- 172.21.15.39:0/3152816440 >> 172.21.15.39:0/3152816440 conn(0x7f58700a9720 msgr2=0x7f58700a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 -- 172.21.15.39:0/3152816440 shutdown_connections 2024-09-13T17:03:02.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 -- 172.21.15.39:0/3152816440 wait complete. 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 Processor -- start 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 -- start start 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58700c3ac0 0x7f58700aa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58700ac420 0x7f58701e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58701e7fc0 0x7f58701e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58701e5050 con 0x7f58700ac420 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58700acc30 con 0x7f58700c3ac0 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f5875538640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58701e7b30 con 0x7f58701e7fc0 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58700ac420 0x7f58701e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58700ac420 0x7f58701e45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59072/0 (socket says 172.21.15.39:59072) 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586e7fc640 1 -- 172.21.15.39:0/3556961137 learned_addr learned my addr 172.21.15.39:0/3556961137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586effd640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58700c3ac0 0x7f58700aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:02.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586f7fe640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58701e7fc0 0x7f58701e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:02.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586e7fc640 1 -- 172.21.15.39:0/3556961137 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58701e7fc0 msgr2=0x7f58701e4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:02.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586e7fc640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58701e7fc0 0x7f58701e4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:02.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586e7fc640 1 -- 172.21.15.39:0/3556961137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58700c3ac0 msgr2=0x7f58700aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:02.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586e7fc640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58700c3ac0 0x7f58700aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:02.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586e7fc640 1 -- 172.21.15.39:0/3556961137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700ac950 con 0x7f58700ac420 2024-09-13T17:03:02.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.884+0000 7f586e7fc640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58700ac420 0x7f58701e45d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f586400e510 tx=0x7f586400e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:02.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f58577fe640 1 -- 172.21.15.39:0/3556961137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864017070 con 0x7f58700ac420 2024-09-13T17:03:02.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f58577fe640 1 -- 172.21.15.39:0/3556961137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58640052e0 con 0x7f58700ac420 2024-09-13T17:03:02.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58701e74e0 con 0x7f58700ac420 2024-09-13T17:03:02.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f58577fe640 1 -- 172.21.15.39:0/3556961137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864005490 con 0x7f58700ac420 2024-09-13T17:03:02.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58700aba90 con 0x7f58700ac420 2024-09-13T17:03:02.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58701e7340 con 0x7f58700ac420 2024-09-13T17:03:02.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f58577fe640 1 -- 172.21.15.39:0/3556961137 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5864007020 con 0x7f58700ac420 2024-09-13T17:03:02.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f58577fe640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f585c0666c0 0x7f585c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:02.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f58577fe640 1 -- 172.21.15.39:0/3556961137 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7f5864083f90 con 0x7f58700ac420 2024-09-13T17:03:02.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f586effd640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f585c0666c0 0x7f585c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:02.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.888+0000 7f586effd640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f585c0666c0 0x7f585c068b80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5868009fd0 tx=0x7f586800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:02.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:02.892+0000 7f58577fe640 1 -- 172.21.15.39:0/3556961137 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f58701e7340 con 0x7f58700ac420 2024-09-13T17:03:03.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.100+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58701e7340 con 0x7f58700ac420 2024-09-13T17:03:03.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f58577fe640 1 -- 172.21.15.39:0/3556961137 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+5173 (secure 0 0 0) 0x7f58701e7340 con 0x7f58700ac420 2024-09-13T17:03:03.102 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:03.102 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:03.102 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 100 2024-09-13T17:03:03.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f585c0666c0 msgr2=0x7f585c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:03.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f585c0666c0 0x7f585c068b80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5868009fd0 tx=0x7f586800b040 comp rx=0 tx=0).stop 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58700ac420 msgr2=0x7f58701e45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58700ac420 0x7f58701e45d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f586400e510 tx=0x7f586400e9d0 comp rx=0 tx=0).stop 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 shutdown_connections 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f585c0666c0 0x7f585c068b80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58701e7fc0 0x7f58701e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58700ac420 0x7f58701e45d0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 --2- 172.21.15.39:0/3556961137 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58700c3ac0 0x7f58700aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 >> 172.21.15.39:0/3556961137 conn(0x7f58700a9720 msgr2=0x7f5870152280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 shutdown_connections 2024-09-13T17:03:03.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:03.104+0000 7f5875538640 1 -- 172.21.15.39:0/3556961137 wait complete. 2024-09-13T17:03:03.124 INFO:tasks.mds_thrash.fs.[a]:mds.a has gained rank=0, replacing gid=4226 2024-09-13T17:03:03.124 INFO:tasks.mds_thrash.fs.[a]:waiting for 4 secs before reviving mds.c 2024-09-13T17:03:03.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:04.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4efaa8640 1 Processor -- start 2024-09-13T17:03:04.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4efaa8640 1 -- start start 2024-09-13T17:03:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4efaa8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e806b080 0x7fb4e806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4efaa8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4e81576e0 0x7fb4e8157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4efaa8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e80c15d0 con 0x7fb4e80c1810 2024-09-13T17:03:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4efaa8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e806d0b0 con 0x7fb4e81576e0 2024-09-13T17:03:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4efaa8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e80c1f00 con 0x7fb4e806b080 2024-09-13T17:03:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4ed043640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb4e80c1810 0x7fb4e8151390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53348/0 (socket says 172.21.15.39:53348) 2024-09-13T17:03:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4ed043640 1 -- 172.21.15.39:0/3031802613 learned_addr learned my addr 172.21.15.39:0/3031802613 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4ed844640 1 --2- 172.21.15.39:0/3031802613 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e806b080 0x7fb4e806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4ee045640 1 --2- 172.21.15.39:0/3031802613 >> v2:172.21.15.84:3300/0 conn(0x7fb4e81576e0 0x7fb4e8157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4ed844640 1 -- 172.21.15.39:0/3031802613 >> v2:172.21.15.84:3300/0 conn(0x7fb4e81576e0 msgr2=0x7fb4e8157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4ed844640 1 --2- 172.21.15.39:0/3031802613 >> v2:172.21.15.84:3300/0 conn(0x7fb4e81576e0 0x7fb4e8157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4ed844640 1 -- 172.21.15.39:0/3031802613 >> v1:172.21.15.39:6789/0 conn(0x7fb4e80c1810 legacy=0x7fb4e8151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4ed844640 1 -- 172.21.15.39:0/3031802613 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e80c21a0 con 0x7fb4e806b080 2024-09-13T17:03:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.032+0000 7fb4ee045640 1 --2- 172.21.15.39:0/3031802613 >> v2:172.21.15.84:3300/0 conn(0x7fb4e81576e0 0x7fb4e8157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:03:04.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.036+0000 7fb4ed844640 1 --2- 172.21.15.39:0/3031802613 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e806b080 0x7fb4e806e680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40097c0 tx=0x7fb4e401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f9eddacef5bfd18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.036+0000 7fb4ec842640 1 -- 172.21.15.39:0/3031802613 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40ad680 con 0x7fb4e806b080 2024-09-13T17:03:04.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.036+0000 7fb4ec842640 1 -- 172.21.15.39:0/3031802613 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e40ad7e0 con 0x7fb4e806b080 2024-09-13T17:03:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ec842640 1 -- 172.21.15.39:0/3031802613 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40adb70 con 0x7fb4e806b080 2024-09-13T17:03:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- 172.21.15.39:0/3031802613 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e806b080 msgr2=0x7fb4e806e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 --2- 172.21.15.39:0/3031802613 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e806b080 0x7fb4e806e680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40097c0 tx=0x7fb4e401c470 comp rx=0 tx=0).stop 2024-09-13T17:03:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- 172.21.15.39:0/3031802613 shutdown_connections 2024-09-13T17:03:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 --2- 172.21.15.39:0/3031802613 >> v2:172.21.15.84:3300/0 conn(0x7fb4e81576e0 0x7fb4e8157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 --2- 172.21.15.39:0/3031802613 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e806b080 0x7fb4e806e680 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- 172.21.15.39:0/3031802613 >> 172.21.15.39:0/3031802613 conn(0x7fb4e80c32a0 msgr2=0x7fb4e80c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- 172.21.15.39:0/3031802613 shutdown_connections 2024-09-13T17:03:04.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- 172.21.15.39:0/3031802613 wait complete. 2024-09-13T17:03:04.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 Processor -- start 2024-09-13T17:03:04.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- start start 2024-09-13T17:03:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4e806b080 0x7fb4e814c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e80c1810 0x7fb4e81499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e81576e0 0x7fb4e81550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e814b0a0 con 0x7fb4e806b080 2024-09-13T17:03:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e814e520 con 0x7fb4e80c1810 2024-09-13T17:03:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e814e940 con 0x7fb4e81576e0 2024-09-13T17:03:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed844640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4e806b080 0x7fb4e814c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed844640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4e806b080 0x7fb4e814c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59082/0 (socket says 172.21.15.39:59082) 2024-09-13T17:03:04.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed844640 1 -- 172.21.15.39:0/948052893 learned_addr learned my addr 172.21.15.39:0/948052893 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ee045640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e81576e0 0x7fb4e81550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed043640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e80c1810 0x7fb4e81499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed844640 1 -- 172.21.15.39:0/948052893 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e81576e0 msgr2=0x7fb4e81550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed844640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e81576e0 0x7fb4e81550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed844640 1 -- 172.21.15.39:0/948052893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e80c1810 msgr2=0x7fb4e81499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed844640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e80c1810 0x7fb4e81499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed844640 1 -- 172.21.15.39:0/948052893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0009a70 con 0x7fb4e806b080 2024-09-13T17:03:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ee045640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e81576e0 0x7fb4e81550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:04.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4ed844640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4e806b080 0x7fb4e814c990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb4e401a0a0 tx=0x7fb4e4019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:04.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4de7fc640 1 -- 172.21.15.39:0/948052893 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40ad680 con 0x7fb4e806b080 2024-09-13T17:03:04.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4de7fc640 1 -- 172.21.15.39:0/948052893 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e40add60 con 0x7fb4e806b080 2024-09-13T17:03:04.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.040+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e4009420 con 0x7fb4e806b080 2024-09-13T17:03:04.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.044+0000 7fb4de7fc640 1 -- 172.21.15.39:0/948052893 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e40b6650 con 0x7fb4e806b080 2024-09-13T17:03:04.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.044+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e814ced0 con 0x7fb4e806b080 2024-09-13T17:03:04.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.044+0000 7fb4de7fc640 1 -- 172.21.15.39:0/948052893 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4e40b4040 con 0x7fb4e806b080 2024-09-13T17:03:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.044+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e80619c0 con 0x7fb4e806b080 2024-09-13T17:03:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.044+0000 7fb4de7fc640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4cc0666e0 0x7fb4cc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.044+0000 7fb4de7fc640 1 -- 172.21.15.39:0/948052893 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7fb4e411e2a0 con 0x7fb4e806b080 2024-09-13T17:03:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.044+0000 7fb4ed043640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4cc0666e0 0x7fb4cc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.044+0000 7fb4ed043640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4cc0666e0 0x7fb4cc068ba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8155050 tx=0x7fb4d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:04.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.048+0000 7fb4de7fc640 1 -- 172.21.15.39:0/948052893 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb4e40eabd0 con 0x7fb4e806b080 2024-09-13T17:03:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.268+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4e8067130 con 0x7fb4e806b080 2024-09-13T17:03:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.268+0000 7fb4de7fc640 1 -- 172.21.15.39:0/948052893 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+5173 (secure 0 0 0) 0x7fb4e40ee1f0 con 0x7fb4e806b080 2024-09-13T17:03:04.269 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:04.269 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:04.269 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 100 2024-09-13T17:03:04.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4cc0666e0 msgr2=0x7fb4cc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4cc0666e0 0x7fb4cc068ba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8155050 tx=0x7fb4d8009040 comp rx=0 tx=0).stop 2024-09-13T17:03:04.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4e806b080 msgr2=0x7fb4e814c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4e806b080 0x7fb4e814c990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb4e401a0a0 tx=0x7fb4e4019210 comp rx=0 tx=0).stop 2024-09-13T17:03:04.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 shutdown_connections 2024-09-13T17:03:04.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4cc0666e0 0x7fb4cc068ba0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4e80c1810 0x7fb4e81499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4e81576e0 0x7fb4e81550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 --2- 172.21.15.39:0/948052893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4e806b080 0x7fb4e814c990 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 >> 172.21.15.39:0/948052893 conn(0x7fb4e80c32a0 msgr2=0x7fb4e8065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:04.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 shutdown_connections 2024-09-13T17:03:04.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.272+0000 7fb4efaa8640 1 -- 172.21.15.39:0/948052893 wait complete. 2024-09-13T17:03:04.293 INFO:tasks.mds_thrash.fs.[b]:kill mds.d (rank=0) 2024-09-13T17:03:04.293 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T17:03:04.294 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:03:04.306 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T17:03:04.306 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:04.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e771c640 1 Processor -- start 2024-09-13T17:03:04.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e771c640 1 -- start start 2024-09-13T17:03:04.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e771c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 0x7ff7e0153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e771c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff7e00611c0 0x7ff7e00bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e771c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e00c1340 con 0x7ff7e0151860 2024-09-13T17:03:04.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e771c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e01578f0 con 0x7ff7e00611c0 2024-09-13T17:03:04.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e771c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e00c1590 con 0x7ff7e00bf4f0 2024-09-13T17:03:04.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e5cb9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff7e00611c0 0x7ff7e00bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e54b8640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff7e0151860 0x7ff7e0157d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53356/0 (socket says 172.21.15.39:53356) 2024-09-13T17:03:04.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e54b8640 1 -- 172.21.15.39:0/1342718940 learned_addr learned my addr 172.21.15.39:0/1342718940 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:04.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e4cb7640 1 --2- 172.21.15.39:0/1342718940 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 0x7ff7e0153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e5cb9640 1 -- 172.21.15.39:0/1342718940 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 msgr2=0x7ff7e0153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:04.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e5cb9640 1 --2- 172.21.15.39:0/1342718940 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 0x7ff7e0153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e5cb9640 1 -- 172.21.15.39:0/1342718940 >> v1:172.21.15.39:6789/0 conn(0x7ff7e0151860 legacy=0x7ff7e0157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.428+0000 7ff7e5cb9640 1 -- 172.21.15.39:0/1342718940 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00c1870 con 0x7ff7e00611c0 2024-09-13T17:03:04.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 --2- 172.21.15.39:0/1342718940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e00611c0 0x7ff7e00bfad0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80097c0 tx=0x7ff7d801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76e1e0c3f378c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:04.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7d7fff640 1 -- 172.21.15.39:0/1342718940 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80ac640 con 0x7ff7e00611c0 2024-09-13T17:03:04.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7d7fff640 1 -- 172.21.15.39:0/1342718940 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d80ac7a0 con 0x7ff7e00611c0 2024-09-13T17:03:04.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7d7fff640 1 -- 172.21.15.39:0/1342718940 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80acb30 con 0x7ff7e00611c0 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 -- 172.21.15.39:0/1342718940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e00611c0 msgr2=0x7ff7e00bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 --2- 172.21.15.39:0/1342718940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e00611c0 0x7ff7e00bfad0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80097c0 tx=0x7ff7d801c3e0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 -- 172.21.15.39:0/1342718940 shutdown_connections 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 --2- 172.21.15.39:0/1342718940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e00611c0 0x7ff7e00bfad0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 --2- 172.21.15.39:0/1342718940 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 0x7ff7e0153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 -- 172.21.15.39:0/1342718940 >> 172.21.15.39:0/1342718940 conn(0x7ff7e0069e10 msgr2=0x7ff7e006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 -- 172.21.15.39:0/1342718940 shutdown_connections 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 -- 172.21.15.39:0/1342718940 wait complete. 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 Processor -- start 2024-09-13T17:03:04.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 -- start start 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7e00611c0 0x7ff7e0143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 0x7ff7e01efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e0151860 0x7ff7e01f0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e0144560 con 0x7ff7e0151860 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e01ec750 con 0x7ff7e00611c0 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e771c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e01ecfb0 con 0x7ff7e00bf4f0 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e0151860 0x7ff7e01f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e0151860 0x7ff7e01f0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59108/0 (socket says 172.21.15.39:59108) 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 -- 172.21.15.39:0/4266698331 learned_addr learned my addr 172.21.15.39:0/4266698331 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:04.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e4cb7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 0x7ff7e01efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e54b8640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7e00611c0 0x7ff7e0143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 -- 172.21.15.39:0/4266698331 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 msgr2=0x7ff7e01efcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 0x7ff7e01efcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 -- 172.21.15.39:0/4266698331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7e00611c0 msgr2=0x7ff7e0143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7e00611c0 0x7ff7e0143ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 -- 172.21.15.39:0/4266698331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d8009420 con 0x7ff7e0151860 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.432+0000 7ff7e5cb9640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e0151860 0x7ff7e01f0200 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff7d801a010 tx=0x7ff7d801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7d5ffb640 1 -- 172.21.15.39:0/4266698331 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80b7650 con 0x7ff7e0151860 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e0144280 con 0x7ff7e0151860 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7d5ffb640 1 -- 172.21.15.39:0/4266698331 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d80b77d0 con 0x7ff7e0151860 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7d5ffb640 1 -- 172.21.15.39:0/4266698331 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d80b7b80 con 0x7ff7e0151860 2024-09-13T17:03:04.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e01ecc40 con 0x7ff7e0151860 2024-09-13T17:03:04.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e01ed120 con 0x7ff7e0151860 2024-09-13T17:03:04.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7d5ffb640 1 -- 172.21.15.39:0/4266698331 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff7d80b3070 con 0x7ff7e0151860 2024-09-13T17:03:04.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7d5ffb640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7c40666c0 0x7ff7c4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:04.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7d5ffb640 1 -- 172.21.15.39:0/4266698331 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7ff7d811f420 con 0x7ff7e0151860 2024-09-13T17:03:04.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7e54b8640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7c40666c0 0x7ff7c4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:04.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.436+0000 7ff7e54b8640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7c40666c0 0x7ff7c4068b80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc005c10 tx=0x7ff7dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:04.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.444+0000 7ff7d5ffb640 1 -- 172.21.15.39:0/4266698331 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff7d80ebf70 con 0x7ff7e0151860 2024-09-13T17:03:04.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.660+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7e00bfad0 con 0x7ff7e0151860 2024-09-13T17:03:04.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.660+0000 7ff7d5ffb640 1 -- 172.21.15.39:0/4266698331 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+5173 (secure 0 0 0) 0x7ff7d80b1090 con 0x7ff7e0151860 2024-09-13T17:03:04.660 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:04.660 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:04.660 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 100 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7c40666c0 msgr2=0x7ff7c4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7c40666c0 0x7ff7c4068b80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc005c10 tx=0x7ff7dc009040 comp rx=0 tx=0).stop 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e0151860 msgr2=0x7ff7e01f0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e0151860 0x7ff7e01f0200 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff7d801a010 tx=0x7ff7d801ac20 comp rx=0 tx=0).stop 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 shutdown_connections 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7c40666c0 0x7ff7c4068b80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7e0151860 0x7ff7e01f0200 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff7e00bf4f0 0x7ff7e01efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 --2- 172.21.15.39:0/4266698331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7e00611c0 0x7ff7e0143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 >> 172.21.15.39:0/4266698331 conn(0x7ff7e0069e10 msgr2=0x7ff7e014fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 shutdown_connections 2024-09-13T17:03:04.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:04.664+0000 7ff7e771c640 1 -- 172.21.15.39:0/4266698331 wait complete. 2024-09-13T17:03:04.680 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:03:06.681 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:06.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a94ce640 1 Processor -- start 2024-09-13T17:03:06.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a94ce640 1 -- start start 2024-09-13T17:03:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a94ce640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3a4153ef0 0x7fc3a4070f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a94ce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40c6830 con 0x7fc3a414a940 2024-09-13T17:03:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a94ce640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a40c6b10 con 0x7fc3a40c6320 2024-09-13T17:03:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a94ce640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a41542f0 con 0x7fc3a4153ef0 2024-09-13T17:03:06.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a2ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc3a414a940 0x7fc3a40c4290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53372/0 (socket says 172.21.15.39:53372) 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a2ffd640 1 -- 172.21.15.39:0/2567668890 learned_addr learned my addr 172.21.15.39:0/2567668890 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939956908 0 0) 0x7fc3a40c6830 con 0x7fc3a414a940 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a37fe640 1 --2- 172.21.15.39:0/2567668890 >> v2:172.21.15.84:3300/0 conn(0x7fc3a4153ef0 0x7fc3a4070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc38c003530 con 0x7fc3a414a940 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518445919 0 0) 0x7fc38c003530 con 0x7fc3a414a940 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 >> v2:172.21.15.84:3300/0 conn(0x7fc3a4153ef0 msgr2=0x7fc3a4070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 --2- 172.21.15.39:0/2567668890 >> v2:172.21.15.84:3300/0 conn(0x7fc3a4153ef0 0x7fc3a4070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 >> v1:172.21.15.84:6789/0 conn(0x7fc3a40c6320 legacy=0x7fc3a4070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a40c6df0 con 0x7fc3a414a940 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fc394001680 con 0x7fc3a414a940 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 >> v1:172.21.15.39:6789/0 conn(0x7fc3a414a940 legacy=0x7fc3a40c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc38c003980 0x7fc38c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:06.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c0047d0 0x7fc38c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc38c005390 0x7fc38c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c00c1f0 con 0x7fc38c0047d0 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c00c360 con 0x7fc38c005390 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c00c520 con 0x7fc38c003980 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a27fc640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c0047d0 0x7fc38c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a37fe640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc38c005390 0x7fc38c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a2ffd640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc38c003980 0x7fc38c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a27fc640 1 -- 172.21.15.39:0/2567668890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc38c003980 msgr2=0x7fc38c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a27fc640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc38c003980 0x7fc38c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a27fc640 1 -- 172.21.15.39:0/2567668890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc38c005390 msgr2=0x7fc38c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:06.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a27fc640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc38c005390 0x7fc38c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a37fe640 1 -- 172.21.15.39:0/2567668890 reap_dead start 2024-09-13T17:03:06.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.812+0000 7fc3a27fc640 1 -- 172.21.15.39:0/2567668890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c003530 con 0x7fc38c0047d0 2024-09-13T17:03:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a27fc640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c0047d0 0x7fc38c004c50 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc398001780 tx=0x7fc39801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c765f8d451d1c64d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3980050d0 con 0x7fc38c0047d0 2024-09-13T17:03:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980a7050 con 0x7fc38c0047d0 2024-09-13T17:03:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a1ffb640 1 -- 172.21.15.39:0/2567668890 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398005490 con 0x7fc38c0047d0 2024-09-13T17:03:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2567668890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c0047d0 msgr2=0x7fc38c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:06.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a94ce640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c0047d0 0x7fc38c004c50 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc398001780 tx=0x7fc39801c400 comp rx=0 tx=0).stop 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2567668890 shutdown_connections 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a94ce640 1 --2- 172.21.15.39:0/2567668890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c0047d0 0x7fc38c004c50 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2567668890 >> 172.21.15.39:0/2567668890 conn(0x7fc3a40ab460 msgr2=0x7fc3a40ab870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2567668890 shutdown_connections 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2567668890 wait complete. 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a94ce640 1 Processor -- start 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.816+0000 7fc3a94ce640 1 -- start start 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a94ce640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc38c0047d0 0x7fc3a41ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a94ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a41efce0 0x7fc3a40bea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a94ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a40bef60 0x7fc3a41552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:06.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a94ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40bd610 con 0x7fc3a40bef60 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a94ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a414a510 con 0x7fc3a41efce0 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a94ce640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a41f02a0 con 0x7fc38c0047d0 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a40bef60 0x7fc3a41552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a40bef60 0x7fc3a41552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59116/0 (socket says 172.21.15.39:59116) 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a37fe640 1 -- 172.21.15.39:0/2411543508 learned_addr learned my addr 172.21.15.39:0/2411543508 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a2ffd640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc38c0047d0 0x7fc3a41ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a27fc640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a41efce0 0x7fc3a40bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a37fe640 1 -- 172.21.15.39:0/2411543508 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc38c0047d0 msgr2=0x7fc3a41ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a37fe640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc38c0047d0 0x7fc3a41ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a37fe640 1 -- 172.21.15.39:0/2411543508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a41efce0 msgr2=0x7fc3a40bea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a37fe640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a41efce0 0x7fc3a40bea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a37fe640 1 -- 172.21.15.39:0/2411543508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a40bd820 con 0x7fc3a40bef60 2024-09-13T17:03:06.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a37fe640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a40bef60 0x7fc3a41552e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4151ac0 tx=0x7fc39c009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:06.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc38b7fe640 1 -- 172.21.15.39:0/2411543508 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c009ea0 con 0x7fc3a40bef60 2024-09-13T17:03:06.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc38b7fe640 1 -- 172.21.15.39:0/2411543508 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c00f050 con 0x7fc3a40bef60 2024-09-13T17:03:06.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a40bdb00 con 0x7fc3a40bef60 2024-09-13T17:03:06.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc38b7fe640 1 -- 172.21.15.39:0/2411543508 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c014410 con 0x7fc3a40bef60 2024-09-13T17:03:06.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a40bf3e0 con 0x7fc3a40bef60 2024-09-13T17:03:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a40be740 con 0x7fc3a40bef60 2024-09-13T17:03:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc38b7fe640 1 -- 172.21.15.39:0/2411543508 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc39c014570 con 0x7fc3a40bef60 2024-09-13T17:03:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc38b7fe640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc390066710 0x7fc390068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc38b7fe640 1 -- 172.21.15.39:0/2411543508 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7fc39c084e40 con 0x7fc3a40bef60 2024-09-13T17:03:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.820+0000 7fc3a2ffd640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc390066710 0x7fc390068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.824+0000 7fc3a2ffd640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc390066710 0x7fc390068bd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc394001ca0 tx=0x7fc394007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:06.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:06.828+0000 7fc38b7fe640 1 -- 172.21.15.39:0/2411543508 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc3a40be740 con 0x7fc3a40bef60 2024-09-13T17:03:07.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.068+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3a40be740 con 0x7fc3a40bef60 2024-09-13T17:03:07.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.068+0000 7fc38b7fe640 1 -- 172.21.15.39:0/2411543508 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+5173 (secure 0 0 0) 0x7fc3a40be740 con 0x7fc3a40bef60 2024-09-13T17:03:07.068 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:07.068 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:07.069 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 100 2024-09-13T17:03:07.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc390066710 msgr2=0x7fc390068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:07.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc390066710 0x7fc390068bd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc394001ca0 tx=0x7fc394007040 comp rx=0 tx=0).stop 2024-09-13T17:03:07.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a40bef60 msgr2=0x7fc3a41552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:07.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a40bef60 0x7fc3a41552e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4151ac0 tx=0x7fc39c009cb0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 shutdown_connections 2024-09-13T17:03:07.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc390066710 0x7fc390068bd0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3a41efce0 0x7fc3a40bea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3a40bef60 0x7fc3a41552e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 --2- 172.21.15.39:0/2411543508 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc38c0047d0 0x7fc3a41ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 >> 172.21.15.39:0/2411543508 conn(0x7fc3a40ab460 msgr2=0x7fc3a40ab840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:07.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 shutdown_connections 2024-09-13T17:03:07.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.072+0000 7fc3a94ce640 1 -- 172.21.15.39:0/2411543508 wait complete. 2024-09-13T17:03:07.088 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:03:07.125 INFO:tasks.mds_thrash.fs.[a]:reviving mds.c 2024-09-13T17:03:07.126 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:03:07.126 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:03:07.128 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:03:07.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:07.193 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T17:03:07.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.248+0000 7fb8cd28c640 1 Processor -- start 2024-09-13T17:03:07.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.248+0000 7fb8cd28c640 1 -- start start 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c806b080 0x7fb8c806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb8c81576e0 0x7fb8c8157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c80c15d0 con 0x7fb8c80c1810 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c806d0b0 con 0x7fb8c81576e0 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80c1f00 con 0x7fb8c806b080 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb8c81576e0 0x7fb8c8157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb8c81576e0 0x7fb8c8157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59160/0 (socket says 172.21.15.39:59160) 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c77fe640 1 -- 172.21.15.39:0/2667173539 learned_addr learned my addr 172.21.15.39:0/2667173539 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c6ffd640 1 --2- 172.21.15.39:0/2667173539 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c806b080 0x7fb8c806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c77fe640 1 -- 172.21.15.39:0/2667173539 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c806b080 msgr2=0x7fb8c806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c77fe640 1 --2- 172.21.15.39:0/2667173539 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c806b080 0x7fb8c806e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c77fe640 1 -- 172.21.15.39:0/2667173539 >> v1:172.21.15.84:6789/0 conn(0x7fb8c80c1810 legacy=0x7fb8c8151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:07.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c77fe640 1 -- 172.21.15.39:0/2667173539 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c80c21a0 con 0x7fb8c81576e0 2024-09-13T17:03:07.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c6ffd640 1 --2- 172.21.15.39:0/2667173539 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c806b080 0x7fb8c806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:03:07.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c77fe640 1 --2- 172.21.15.39:0/2667173539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c81576e0 0x7fb8c8157af0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80a6470 tx=0x7fb8b801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed4cee24f621640 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:07.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c5ffb640 1 -- 172.21.15.39:0/2667173539 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80ac600 con 0x7fb8c81576e0 2024-09-13T17:03:07.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c5ffb640 1 -- 172.21.15.39:0/2667173539 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b80a8050 con 0x7fb8c81576e0 2024-09-13T17:03:07.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8c5ffb640 1 -- 172.21.15.39:0/2667173539 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80ac980 con 0x7fb8c81576e0 2024-09-13T17:03:07.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2667173539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c81576e0 msgr2=0x7fb8c8157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:07.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 --2- 172.21.15.39:0/2667173539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c81576e0 0x7fb8c8157af0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80a6470 tx=0x7fb8b801c4d0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2667173539 shutdown_connections 2024-09-13T17:03:07.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 --2- 172.21.15.39:0/2667173539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c81576e0 0x7fb8c8157af0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 --2- 172.21.15.39:0/2667173539 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c806b080 0x7fb8c806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2667173539 >> 172.21.15.39:0/2667173539 conn(0x7fb8c80c32a0 msgr2=0x7fb8c80c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:07.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2667173539 shutdown_connections 2024-09-13T17:03:07.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.252+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2667173539 wait complete. 2024-09-13T17:03:07.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 Processor -- start 2024-09-13T17:03:07.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 -- start start 2024-09-13T17:03:07.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c806b080 0x7fb8c80a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c80c1810 0x7fb8c80a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c81576e0 0x7fb8c81efa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80a64c0 con 0x7fb8c806b080 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80a5f00 con 0x7fb8c80c1810 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81ed010 con 0x7fb8c81576e0 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c806b080 0x7fb8c80a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c806b080 0x7fb8c80a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59162/0 (socket says 172.21.15.39:59162) 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c6ffd640 1 -- 172.21.15.39:0/2320055488 learned_addr learned my addr 172.21.15.39:0/2320055488 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c77fe640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c81576e0 0x7fb8c81efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c6ffd640 1 -- 172.21.15.39:0/2320055488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c81576e0 msgr2=0x7fb8c81efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c6ffd640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c81576e0 0x7fb8c81efa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c67fc640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c80c1810 0x7fb8c80a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:07.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c6ffd640 1 -- 172.21.15.39:0/2320055488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c80c1810 msgr2=0x7fb8c80a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:07.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c6ffd640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c80c1810 0x7fb8c80a6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c6ffd640 1 -- 172.21.15.39:0/2320055488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80094d0 con 0x7fb8c806b080 2024-09-13T17:03:07.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8c6ffd640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c806b080 0x7fb8c80a6950 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c7f0 tx=0x7fb8c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:07.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8af7fe640 1 -- 172.21.15.39:0/2320055488 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0007820 con 0x7fb8c806b080 2024-09-13T17:03:07.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8af7fe640 1 -- 172.21.15.39:0/2320055488 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0007980 con 0x7fb8c806b080 2024-09-13T17:03:07.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c81ed2b0 con 0x7fb8c806b080 2024-09-13T17:03:07.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8af7fe640 1 -- 172.21.15.39:0/2320055488 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0007d30 con 0x7fb8c806b080 2024-09-13T17:03:07.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.256+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c815d8c0 con 0x7fb8c806b080 2024-09-13T17:03:07.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.260+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c80619c0 con 0x7fb8c806b080 2024-09-13T17:03:07.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.260+0000 7fb8af7fe640 1 -- 172.21.15.39:0/2320055488 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb8c0024480 con 0x7fb8c806b080 2024-09-13T17:03:07.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.260+0000 7fb8af7fe640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8b40666c0 0x7fb8b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:07.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.260+0000 7fb8c67fc640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8b40666c0 0x7fb8b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:07.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.260+0000 7fb8af7fe640 1 -- 172.21.15.39:0/2320055488 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7fb8c0085270 con 0x7fb8c806b080 2024-09-13T17:03:07.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.260+0000 7fb8c67fc640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8b40666c0 0x7fb8b4068b80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8150760 tx=0x7fb8bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:07.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.264+0000 7fb8af7fe640 1 -- 172.21.15.39:0/2320055488 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb8c0051cf0 con 0x7fb8c806b080 2024-09-13T17:03:07.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.464+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8c806e680 con 0x7fb8c806b080 2024-09-13T17:03:07.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8af7fe640 1 -- 172.21.15.39:0/2320055488 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+5173 (secure 0 0 0) 0x7fb8c001f090 con 0x7fb8c806b080 2024-09-13T17:03:07.466 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:07.466 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:07.466 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 100 2024-09-13T17:03:07.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8b40666c0 msgr2=0x7fb8b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:07.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8b40666c0 0x7fb8b4068b80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8150760 tx=0x7fb8bc009040 comp rx=0 tx=0).stop 2024-09-13T17:03:07.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c806b080 msgr2=0x7fb8c80a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:07.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c806b080 0x7fb8c80a6950 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c7f0 tx=0x7fb8c000ccb0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 shutdown_connections 2024-09-13T17:03:07.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb8b40666c0 0x7fb8b4068b80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8c80c1810 0x7fb8c80a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb8c81576e0 0x7fb8c81efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 --2- 172.21.15.39:0/2320055488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c806b080 0x7fb8c80a6950 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:07.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 >> 172.21.15.39:0/2320055488 conn(0x7fb8c80c32a0 msgr2=0x7fb8c8151370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:07.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 shutdown_connections 2024-09-13T17:03:07.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:07.468+0000 7fb8cd28c640 1 -- 172.21.15.39:0/2320055488 wait complete. 2024-09-13T17:03:07.484 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-09-13T17:03:07.969 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T17:03:07.968+0000 7ff857292640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:03:09.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:09.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607f3d1640 1 Processor -- start 2024-09-13T17:03:09.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607f3d1640 1 -- start start 2024-09-13T17:03:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607f3d1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f60780c1810 0x7f607806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607f3d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f60781576e0 0x7f6078157b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607f3d1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60780c15d0 con 0x7f607806b080 2024-09-13T17:03:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607f3d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607806d0b0 con 0x7f60781576e0 2024-09-13T17:03:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607f3d1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60780c1f00 con 0x7f60780c1810 2024-09-13T17:03:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607d96e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f60781576e0 0x7f6078157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607d96e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f60781576e0 0x7f6078157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59170/0 (socket says 172.21.15.39:59170) 2024-09-13T17:03:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607d96e640 1 -- 172.21.15.39:0/3187986301 learned_addr learned my addr 172.21.15.39:0/3187986301 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607c96c640 1 --2- 172.21.15.39:0/3187986301 >> v2:172.21.15.84:3300/0 conn(0x7f60780c1810 0x7f607806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607d96e640 1 -- 172.21.15.39:0/3187986301 >> v2:172.21.15.84:3300/0 conn(0x7f60780c1810 msgr2=0x7f607806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607d96e640 1 --2- 172.21.15.39:0/3187986301 >> v2:172.21.15.84:3300/0 conn(0x7f60780c1810 0x7f607806e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607d96e640 1 -- 172.21.15.39:0/3187986301 >> v1:172.21.15.84:6789/0 conn(0x7f607806b080 legacy=0x7f6078151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:09.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.240+0000 7f607d96e640 1 -- 172.21.15.39:0/3187986301 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60780c21a0 con 0x7f60781576e0 2024-09-13T17:03:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607d96e640 1 --2- 172.21.15.39:0/3187986301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60781576e0 0x7f6078157b30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f607806be40 tx=0x7f607001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b54a82178ac4b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f606ffff640 1 -- 172.21.15.39:0/3187986301 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700a5750 con 0x7f60781576e0 2024-09-13T17:03:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f606ffff640 1 -- 172.21.15.39:0/3187986301 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60700a58b0 con 0x7f60781576e0 2024-09-13T17:03:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f606ffff640 1 -- 172.21.15.39:0/3187986301 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700a5c40 con 0x7f60781576e0 2024-09-13T17:03:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 -- 172.21.15.39:0/3187986301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60781576e0 msgr2=0x7f6078157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 --2- 172.21.15.39:0/3187986301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60781576e0 0x7f6078157b30 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f607806be40 tx=0x7f607001c980 comp rx=0 tx=0).stop 2024-09-13T17:03:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 -- 172.21.15.39:0/3187986301 shutdown_connections 2024-09-13T17:03:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 --2- 172.21.15.39:0/3187986301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60781576e0 0x7f6078157b30 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 --2- 172.21.15.39:0/3187986301 >> v2:172.21.15.84:3300/0 conn(0x7f60780c1810 0x7f607806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 -- 172.21.15.39:0/3187986301 >> 172.21.15.39:0/3187986301 conn(0x7f60780c32a0 msgr2=0x7f60780c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 -- 172.21.15.39:0/3187986301 shutdown_connections 2024-09-13T17:03:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 -- 172.21.15.39:0/3187986301 wait complete. 2024-09-13T17:03:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 Processor -- start 2024-09-13T17:03:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.244+0000 7f607f3d1640 1 -- start start 2024-09-13T17:03:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607f3d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607806b080 0x7f60781ed590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607f3d1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60780c1810 0x7f60781ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607f3d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60781576e0 0x7f607814c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607f3d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60781f0590 con 0x7f607806b080 2024-09-13T17:03:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607f3d1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6078154ba0 con 0x7f60781576e0 2024-09-13T17:03:09.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607f3d1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60781f02b0 con 0x7f60780c1810 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d16d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607806b080 0x7f60781ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d16d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607806b080 0x7f60781ed590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59176/0 (socket says 172.21.15.39:59176) 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d16d640 1 -- 172.21.15.39:0/1260197619 learned_addr learned my addr 172.21.15.39:0/1260197619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607c96c640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60780c1810 0x7f60781ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d96e640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60781576e0 0x7f607814c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d16d640 1 -- 172.21.15.39:0/1260197619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60780c1810 msgr2=0x7f60781ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d16d640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60780c1810 0x7f60781ef8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d16d640 1 -- 172.21.15.39:0/1260197619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60781576e0 msgr2=0x7f607814c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d16d640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60781576e0 0x7f607814c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d16d640 1 -- 172.21.15.39:0/1260197619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070009b30 con 0x7f607806b080 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607d16d640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607806b080 0x7f60781ed590 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f607400c450 tx=0x7f607400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f606dffb640 1 -- 172.21.15.39:0/1260197619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074012830 con 0x7f607806b080 2024-09-13T17:03:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f606dffb640 1 -- 172.21.15.39:0/1260197619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60740129b0 con 0x7f607806b080 2024-09-13T17:03:09.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f606dffb640 1 -- 172.21.15.39:0/1260197619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074012d60 con 0x7f607806b080 2024-09-13T17:03:09.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607814c3e0 con 0x7f607806b080 2024-09-13T17:03:09.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60780ac950 con 0x7f607806b080 2024-09-13T17:03:09.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60780619c0 con 0x7f607806b080 2024-09-13T17:03:09.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f606dffb640 1 -- 172.21.15.39:0/1260197619 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f607401c460 con 0x7f607806b080 2024-09-13T17:03:09.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f606dffb640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f605c066690 0x7f605c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f606dffb640 1 -- 172.21.15.39:0/1260197619 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7f6074084990 con 0x7f607806b080 2024-09-13T17:03:09.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.248+0000 7f607c96c640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f605c066690 0x7f605c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.252+0000 7f607c96c640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f605c066690 0x7f605c068b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f60780c1280 tx=0x7f6068008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:09.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.256+0000 7f606dffb640 1 -- 172.21.15.39:0/1260197619 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6074051390 con 0x7f607806b080 2024-09-13T17:03:09.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.480+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6078067130 con 0x7f607806b080 2024-09-13T17:03:09.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.480+0000 7f606dffb640 1 -- 172.21.15.39:0/1260197619 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f60740549b0 con 0x7f607806b080 2024-09-13T17:03:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:09.480 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:09.480 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 101 2024-09-13T17:03:09.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f605c066690 msgr2=0x7f605c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f605c066690 0x7f605c068b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f60780c1280 tx=0x7f6068008040 comp rx=0 tx=0).stop 2024-09-13T17:03:09.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607806b080 msgr2=0x7f60781ed590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607806b080 0x7f60781ed590 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f607400c450 tx=0x7f607400c910 comp rx=0 tx=0).stop 2024-09-13T17:03:09.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 shutdown_connections 2024-09-13T17:03:09.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f605c066690 0x7f605c068b50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60781576e0 0x7f607814c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60780c1810 0x7f60781ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 --2- 172.21.15.39:0/1260197619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607806b080 0x7f60781ed590 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 >> 172.21.15.39:0/1260197619 conn(0x7f60780c32a0 msgr2=0x7f60780664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:09.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 shutdown_connections 2024-09-13T17:03:09.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.484+0000 7f607f3d1640 1 -- 172.21.15.39:0/1260197619 wait complete. 2024-09-13T17:03:09.485 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:09.504 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:03:09.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.616+0000 7f00b3c35640 1 Processor -- start 2024-09-13T17:03:09.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.616+0000 7f00b3c35640 1 -- start start 2024-09-13T17:03:09.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b3c35640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f00ac151500 0x7f00ac0c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b3c35640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00ac071dd0 0x7f00ac154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b3c35640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac0c5a30 con 0x7f00ac0c47c0 2024-09-13T17:03:09.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b3c35640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac1576a0 con 0x7f00ac071dd0 2024-09-13T17:03:09.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b3c35640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac157980 con 0x7f00ac151500 2024-09-13T17:03:09.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b11d0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00ac071dd0 0x7f00ac154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b11d0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00ac071dd0 0x7f00ac154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59178/0 (socket says 172.21.15.39:59178) 2024-09-13T17:03:09.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b11d0640 1 -- 172.21.15.39:0/1330304900 learned_addr learned my addr 172.21.15.39:0/1330304900 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:09.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b19d1640 1 --2- 172.21.15.39:0/1330304900 >> v2:172.21.15.84:3300/0 conn(0x7f00ac151500 0x7f00ac0c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b11d0640 1 -- 172.21.15.39:0/1330304900 >> v2:172.21.15.84:3300/0 conn(0x7f00ac151500 msgr2=0x7f00ac0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:09.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b11d0640 1 --2- 172.21.15.39:0/1330304900 >> v2:172.21.15.84:3300/0 conn(0x7f00ac151500 0x7f00ac0c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b11d0640 1 -- 172.21.15.39:0/1330304900 >> v1:172.21.15.39:6789/0 conn(0x7f00ac0c47c0 legacy=0x7f00ac1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b11d0640 1 -- 172.21.15.39:0/1330304900 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac157c60 con 0x7f00ac071dd0 2024-09-13T17:03:09.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b11d0640 1 --2- 172.21.15.39:0/1330304900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac071dd0 0x7f00ac154070 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f009c009870 tx=0x7f009c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db748df4145897c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:09.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b09cf640 1 -- 172.21.15.39:0/1330304900 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c0ab680 con 0x7f00ac071dd0 2024-09-13T17:03:09.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b09cf640 1 -- 172.21.15.39:0/1330304900 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c0ab7e0 con 0x7f00ac071dd0 2024-09-13T17:03:09.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.620+0000 7f00b09cf640 1 -- 172.21.15.39:0/1330304900 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c0abb70 con 0x7f00ac071dd0 2024-09-13T17:03:09.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- 172.21.15.39:0/1330304900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac071dd0 msgr2=0x7f00ac154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 --2- 172.21.15.39:0/1330304900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac071dd0 0x7f00ac154070 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f009c009870 tx=0x7f009c01c460 comp rx=0 tx=0).stop 2024-09-13T17:03:09.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- 172.21.15.39:0/1330304900 shutdown_connections 2024-09-13T17:03:09.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 --2- 172.21.15.39:0/1330304900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac071dd0 0x7f00ac154070 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 --2- 172.21.15.39:0/1330304900 >> v2:172.21.15.84:3300/0 conn(0x7f00ac151500 0x7f00ac0c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- 172.21.15.39:0/1330304900 >> 172.21.15.39:0/1330304900 conn(0x7f00ac0ab530 msgr2=0x7f00ac0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:09.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- 172.21.15.39:0/1330304900 shutdown_connections 2024-09-13T17:03:09.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- 172.21.15.39:0/1330304900 wait complete. 2024-09-13T17:03:09.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 Processor -- start 2024-09-13T17:03:09.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- start start 2024-09-13T17:03:09.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00ac071dd0 0x7f00ac156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00ac0c47c0 0x7f00ac150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac151500 0x7f00ac0a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac1ed660 con 0x7f00ac151500 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac157180 con 0x7f00ac071dd0 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0aad00 con 0x7f00ac0c47c0 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b21d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac151500 0x7f00ac0a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b21d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac151500 0x7f00ac0a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59190/0 (socket says 172.21.15.39:59190) 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b21d2640 1 -- 172.21.15.39:0/3356297786 learned_addr learned my addr 172.21.15.39:0/3356297786 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b11d0640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00ac0c47c0 0x7f00ac150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b19d1640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00ac071dd0 0x7f00ac156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b21d2640 1 -- 172.21.15.39:0/3356297786 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00ac0c47c0 msgr2=0x7f00ac150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b21d2640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00ac0c47c0 0x7f00ac150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b21d2640 1 -- 172.21.15.39:0/3356297786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00ac071dd0 msgr2=0x7f00ac156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b21d2640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00ac071dd0 0x7f00ac156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b21d2640 1 -- 172.21.15.39:0/3356297786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c0094d0 con 0x7f00ac151500 2024-09-13T17:03:09.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b19d1640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00ac071dd0 0x7f00ac156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:03:09.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b21d2640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac151500 0x7f00ac0a93a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c7d0 tx=0x7f00a400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:09.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00a27fc640 1 -- 172.21.15.39:0/3356297786 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4003ba0 con 0x7f00ac151500 2024-09-13T17:03:09.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00a27fc640 1 -- 172.21.15.39:0/3356297786 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a4003d00 con 0x7f00ac151500 2024-09-13T17:03:09.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.624+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac1ed0c0 con 0x7f00ac151500 2024-09-13T17:03:09.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.628+0000 7f00a27fc640 1 -- 172.21.15.39:0/3356297786 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a400aa30 con 0x7f00ac151500 2024-09-13T17:03:09.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.628+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac1ed380 con 0x7f00ac151500 2024-09-13T17:03:09.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.628+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac155200 con 0x7f00ac151500 2024-09-13T17:03:09.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.628+0000 7f00a27fc640 1 -- 172.21.15.39:0/3356297786 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00a400ab90 con 0x7f00ac151500 2024-09-13T17:03:09.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.628+0000 7f00a27fc640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0090066690 0x7f0090068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:09.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.628+0000 7f00a27fc640 1 -- 172.21.15.39:0/3356297786 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7f00a40848f0 con 0x7f00ac151500 2024-09-13T17:03:09.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.628+0000 7f00b19d1640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0090066690 0x7f0090068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:09.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.628+0000 7f00b19d1640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0090066690 0x7f0090068b50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f00a80097a0 tx=0x7f00a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:09.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.632+0000 7f00a27fc640 1 -- 172.21.15.39:0/3356297786 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f00a40512f0 con 0x7f00ac151500 2024-09-13T17:03:09.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.852+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00ac0c4280 con 0x7f00ac151500 2024-09-13T17:03:09.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00a27fc640 1 -- 172.21.15.39:0/3356297786 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f00a4054910 con 0x7f00ac151500 2024-09-13T17:03:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:09.854 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:09.854 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 101 2024-09-13T17:03:09.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0090066690 msgr2=0x7f0090068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0090066690 0x7f0090068b50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f00a80097a0 tx=0x7f00a8008040 comp rx=0 tx=0).stop 2024-09-13T17:03:09.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac151500 msgr2=0x7f00ac0a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:09.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac151500 0x7f00ac0a93a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c7d0 tx=0x7f00a400cc90 comp rx=0 tx=0).stop 2024-09-13T17:03:09.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 shutdown_connections 2024-09-13T17:03:09.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0090066690 0x7f0090068b50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac151500 0x7f00ac0a93a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f00ac0c47c0 0x7f00ac150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 --2- 172.21.15.39:0/3356297786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00ac071dd0 0x7f00ac156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:09.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 >> 172.21.15.39:0/3356297786 conn(0x7f00ac0ab530 msgr2=0x7f00ac14c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:09.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 shutdown_connections 2024-09-13T17:03:09.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:09.856+0000 7f00b3c35640 1 -- 172.21.15.39:0/3356297786 wait complete. 2024-09-13T17:03:09.873 INFO:tasks.mds_thrash.fs.[a]:mds.c reported in up:standby state 2024-09-13T17:03:09.874 INFO:tasks.mds_thrash.fs.[a]:waiting for 56 secs before thrashing 2024-09-13T17:03:11.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf654a0640 1 Processor -- start 2024-09-13T17:03:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf654a0640 1 -- start start 2024-09-13T17:03:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf654a0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf600c0a90 0x7fdf600c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf654a0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf600c2ef0 0x7fdf600c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf654a0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6006ddf0 con 0x7fdf60065ee0 2024-09-13T17:03:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf654a0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf600c53d0 con 0x7fdf600c2ef0 2024-09-13T17:03:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf654a0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c5950 con 0x7fdf600c0a90 2024-09-13T17:03:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5effd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdf60065ee0 0x7fdf6014f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40388/0 (socket says 172.21.15.39:40388) 2024-09-13T17:03:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5effd640 1 -- 172.21.15.39:0/4261891925 learned_addr learned my addr 172.21.15.39:0/4261891925 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:11.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5f7fe640 1 --2- 172.21.15.39:0/4261891925 >> v2:172.21.15.84:3300/0 conn(0x7fdf600c2ef0 0x7fdf600c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:11.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5e7fc640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf600c0a90 0x7fdf600c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:11.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956328258 0 0) 0x7fdf6006ddf0 con 0x7fdf60065ee0 2024-09-13T17:03:11.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf48003530 con 0x7fdf60065ee0 2024-09-13T17:03:11.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2281856181 0 0) 0x7fdf48003530 con 0x7fdf60065ee0 2024-09-13T17:03:11.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf600c0a90 msgr2=0x7fdf600c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf600c0a90 0x7fdf600c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 >> v2:172.21.15.84:3300/0 conn(0x7fdf600c2ef0 msgr2=0x7fdf600c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 --2- 172.21.15.39:0/4261891925 >> v2:172.21.15.84:3300/0 conn(0x7fdf600c2ef0 0x7fdf600c22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600c2820 con 0x7fdf60065ee0 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fdf50001680 con 0x7fdf60065ee0 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 >> v1:172.21.15.39:6789/0 conn(0x7fdf60065ee0 legacy=0x7fdf6014f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf48003980 0x7fdf48003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf600c0a90 0x7fdf48005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:11.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c53d0 con 0x7fdf48003980 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c5950 con 0x7fdf480047d0 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4800b990 con 0x7fdf600c0a90 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5effd640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf48003980 0x7fdf48003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5f7fe640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf600c0a90 0x7fdf48005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5e7fc640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5effd640 1 -- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf600c0a90 msgr2=0x7fdf48005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5effd640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf600c0a90 0x7fdf48005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5effd640 1 -- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480047d0 msgr2=0x7fdf48004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5effd640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf480047d0 0x7fdf48004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.652+0000 7fdf5effd640 1 -- 172.21.15.39:0/4261891925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48003530 con 0x7fdf48003980 2024-09-13T17:03:11.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf5effd640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf48003980 0x7fdf48003da0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000c0e0 tx=0x7fdf5001cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7b261aaa1b524c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:11.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf500058e0 con 0x7fdf48003980 2024-09-13T17:03:11.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf50005a40 con 0x7fdf48003980 2024-09-13T17:03:11.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf5dffb640 1 -- 172.21.15.39:0/4261891925 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50005df0 con 0x7fdf48003980 2024-09-13T17:03:11.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf654a0640 1 -- 172.21.15.39:0/4261891925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf48003980 msgr2=0x7fdf48003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:11.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf654a0640 1 --2- 172.21.15.39:0/4261891925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf48003980 0x7fdf48003da0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000c0e0 tx=0x7fdf5001cb50 comp rx=0 tx=0).stop 2024-09-13T17:03:11.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf5f7fe640 1 -- 172.21.15.39:0/4261891925 reap_dead start 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf654a0640 1 -- 172.21.15.39:0/4261891925 shutdown_connections 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf654a0640 1 -- 172.21.15.39:0/4261891925 >> 172.21.15.39:0/4261891925 conn(0x7fdf600c3310 msgr2=0x7fdf600c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf654a0640 1 -- 172.21.15.39:0/4261891925 shutdown_connections 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.656+0000 7fdf654a0640 1 -- 172.21.15.39:0/4261891925 wait complete. 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 Processor -- start 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 -- start start 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf601550d0 0x7fdf601eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf601554b0 0x7fdf60154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf60154aa0 0x7fdf601487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60149470 con 0x7fdf601550d0 2024-09-13T17:03:11.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf601540d0 con 0x7fdf60154aa0 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf601eb0f0 con 0x7fdf601554b0 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf601550d0 0x7fdf601eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf601550d0 0x7fdf601eabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54076/0 (socket says 172.21.15.39:54076) 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5effd640 1 -- 172.21.15.39:0/4044182914 learned_addr learned my addr 172.21.15.39:0/4044182914 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5e7fc640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf601554b0 0x7fdf60154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5f7fe640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf60154aa0 0x7fdf601487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5effd640 1 -- 172.21.15.39:0/4044182914 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf601554b0 msgr2=0x7fdf60154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5effd640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf601554b0 0x7fdf60154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5effd640 1 -- 172.21.15.39:0/4044182914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf60154aa0 msgr2=0x7fdf601487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5effd640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf60154aa0 0x7fdf601487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5effd640 1 -- 172.21.15.39:0/4044182914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6014b990 con 0x7fdf601550d0 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf5effd640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf601550d0 0x7fdf601eabb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fdf50000c10 tx=0x7fdf48005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf477fe640 1 -- 172.21.15.39:0/4044182914 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50006ec0 con 0x7fdf601550d0 2024-09-13T17:03:11.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf477fe640 1 -- 172.21.15.39:0/4044182914 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf500048b0 con 0x7fdf601550d0 2024-09-13T17:03:11.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6014bc70 con 0x7fdf601550d0 2024-09-13T17:03:11.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf477fe640 1 -- 172.21.15.39:0/4044182914 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50004c60 con 0x7fdf601550d0 2024-09-13T17:03:11.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf60152f10 con 0x7fdf601550d0 2024-09-13T17:03:11.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf60153bc0 con 0x7fdf601550d0 2024-09-13T17:03:11.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.660+0000 7fdf477fe640 1 -- 172.21.15.39:0/4044182914 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf500070d0 con 0x7fdf601550d0 2024-09-13T17:03:11.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.664+0000 7fdf477fe640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf4c0666c0 0x7fdf4c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:11.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.664+0000 7fdf477fe640 1 -- 172.21.15.39:0/4044182914 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7fdf5011e310 con 0x7fdf601550d0 2024-09-13T17:03:11.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.664+0000 7fdf5e7fc640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf4c0666c0 0x7fdf4c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:11.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.664+0000 7fdf5e7fc640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf4c0666c0 0x7fdf4c068b80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdf60066710 tx=0x7fdf54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:11.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.668+0000 7fdf477fe640 1 -- 172.21.15.39:0/4044182914 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf60153bc0 con 0x7fdf601550d0 2024-09-13T17:03:11.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.896+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf60153bc0 con 0x7fdf601550d0 2024-09-13T17:03:11.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.896+0000 7fdf477fe640 1 -- 172.21.15.39:0/4044182914 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5931 (secure 0 0 0) 0x7fdf60153bc0 con 0x7fdf601550d0 2024-09-13T17:03:11.895 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:11.896 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:11.896 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 101 2024-09-13T17:03:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.896+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf4c0666c0 msgr2=0x7fdf4c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.896+0000 7fdf654a0640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf4c0666c0 0x7fdf4c068b80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdf60066710 tx=0x7fdf54008040 comp rx=0 tx=0).stop 2024-09-13T17:03:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.896+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf601550d0 msgr2=0x7fdf601eabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.896+0000 7fdf654a0640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf601550d0 0x7fdf601eabb0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fdf50000c10 tx=0x7fdf48005800 comp rx=0 tx=0).stop 2024-09-13T17:03:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.900+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 shutdown_connections 2024-09-13T17:03:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.900+0000 7fdf654a0640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf4c0666c0 0x7fdf4c068b80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.900+0000 7fdf654a0640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf60154aa0 0x7fdf601487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.900+0000 7fdf654a0640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf601554b0 0x7fdf60154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.900+0000 7fdf654a0640 1 --2- 172.21.15.39:0/4044182914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf601550d0 0x7fdf601eabb0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:11.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.900+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 >> 172.21.15.39:0/4044182914 conn(0x7fdf600c3310 msgr2=0x7fdf6015b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:11.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.900+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 shutdown_connections 2024-09-13T17:03:11.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:11.900+0000 7fdf654a0640 1 -- 172.21.15.39:0/4044182914 wait complete. 2024-09-13T17:03:11.916 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:03:13.917 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:14.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faab62e8640 1 Processor -- start 2024-09-13T17:03:14.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faab62e8640 1 -- start start 2024-09-13T17:03:14.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faab62e8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab00c46a0 0x7faab006ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:14.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faab62e8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faab006b360 0x7faab0157990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:14.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faab62e8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab00c0df0 con 0x7faab014dde0 2024-09-13T17:03:14.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faab62e8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab00c1240 con 0x7faab006b360 2024-09-13T17:03:14.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faab62e8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab00c1520 con 0x7faab00c46a0 2024-09-13T17:03:14.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7faab014dde0 0x7faab01547c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40402/0 (socket says 172.21.15.39:40402) 2024-09-13T17:03:14.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaffff640 1 -- 172.21.15.39:0/3603568183 learned_addr learned my addr 172.21.15.39:0/3603568183 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:14.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faab4885640 1 --2- 172.21.15.39:0/3603568183 >> v2:172.21.15.84:3300/0 conn(0x7faab006b360 0x7faab0157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:14.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaf7fe640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab00c46a0 0x7faab006ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:14.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026571647 0 0) 0x7faab00c0df0 con 0x7faab014dde0 2024-09-13T17:03:14.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa98003530 con 0x7faab014dde0 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 730068244 0 0) 0x7faa98003530 con 0x7faab014dde0 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab00c46a0 msgr2=0x7faab006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab00c46a0 0x7faab006ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 >> v2:172.21.15.84:3300/0 conn(0x7faab006b360 msgr2=0x7faab0157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 --2- 172.21.15.39:0/3603568183 >> v2:172.21.15.84:3300/0 conn(0x7faab006b360 0x7faab0157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab00c1800 con 0x7faab014dde0 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7faaa0001a00 con 0x7faab014dde0 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 >> v1:172.21.15.39:6789/0 conn(0x7faab014dde0 legacy=0x7faab01547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa98003980 0x7faa98003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:14.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faa98004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab00c46a0 0x7faa98005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab00c1240 con 0x7faa980047d0 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab00c1520 con 0x7faa98003980 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9800b990 con 0x7faab00c46a0 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaf7fe640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faa98004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaffff640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa98003980 0x7faa98003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faab4885640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab00c46a0 0x7faa98005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaf7fe640 1 -- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab00c46a0 msgr2=0x7faa98005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaf7fe640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab00c46a0 0x7faa98005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaf7fe640 1 -- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa98003980 msgr2=0x7faa98003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:14.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaf7fe640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa98003980 0x7faa98003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.068+0000 7faaaf7fe640 1 -- 172.21.15.39:0/3603568183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa98003530 con 0x7faa980047d0 2024-09-13T17:03:14.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faaaf7fe640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faa98004c50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009af0 tx=0x7faaa401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebace2d698111962 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:14.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40059a0 con 0x7faa980047d0 2024-09-13T17:03:14.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa4005b00 con 0x7faa980047d0 2024-09-13T17:03:14.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faaaeffd640 1 -- 172.21.15.39:0/3603568183 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4006030 con 0x7faa980047d0 2024-09-13T17:03:14.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faab62e8640 1 -- 172.21.15.39:0/3603568183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 msgr2=0x7faa98004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:14.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faab62e8640 1 --2- 172.21.15.39:0/3603568183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faa98004c50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009af0 tx=0x7faaa401ca00 comp rx=0 tx=0).stop 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faab4885640 1 -- 172.21.15.39:0/3603568183 reap_dead start 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faab62e8640 1 -- 172.21.15.39:0/3603568183 shutdown_connections 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faab62e8640 1 -- 172.21.15.39:0/3603568183 >> 172.21.15.39:0/3603568183 conn(0x7faab00c32a0 msgr2=0x7faab00c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faab62e8640 1 -- 172.21.15.39:0/3603568183 shutdown_connections 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faab62e8640 1 -- 172.21.15.39:0/3603568183 wait complete. 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faab62e8640 1 Processor -- start 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.072+0000 7faab62e8640 1 -- start start 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab62e8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faab01f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab62e8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab0155160 0x7faab0143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:14.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab62e8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab0143e70 0x7faab01ed130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab62e8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab01f06a0 con 0x7faa980047d0 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab62e8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab01f0c60 con 0x7faab0143e70 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab62e8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab0143c70 con 0x7faab0155160 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faab01f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faab01f10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54092/0 (socket says 172.21.15.39:54092) 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaffff640 1 -- 172.21.15.39:0/835394658 learned_addr learned my addr 172.21.15.39:0/835394658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaf7fe640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab0155160 0x7faab0143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab4885640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab0143e70 0x7faab01ed130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaffff640 1 -- 172.21.15.39:0/835394658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab0155160 msgr2=0x7faab0143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:14.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaffff640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab0155160 0x7faab0143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaffff640 1 -- 172.21.15.39:0/835394658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab0143e70 msgr2=0x7faab01ed130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaffff640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab0143e70 0x7faab01ed130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaffff640 1 -- 172.21.15.39:0/835394658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0153a30 con 0x7faa980047d0 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaffff640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faab01f10f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7faaa0001590 tx=0x7faaa000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaacff9640 1 -- 172.21.15.39:0/835394658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa000ee50 con 0x7faa980047d0 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaacff9640 1 -- 172.21.15.39:0/835394658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa0013420 con 0x7faa980047d0 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab0153d10 con 0x7faa980047d0 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaacff9640 1 -- 172.21.15.39:0/835394658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa00137d0 con 0x7faa980047d0 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab006ea40 con 0x7faa980047d0 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab00c0df0 con 0x7faa980047d0 2024-09-13T17:03:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaacff9640 1 -- 172.21.15.39:0/835394658 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faaa0013930 con 0x7faa980047d0 2024-09-13T17:03:14.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaacff9640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa9c0666c0 0x7faa9c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:14.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaacff9640 1 -- 172.21.15.39:0/835394658 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7faaa00852b0 con 0x7faa980047d0 2024-09-13T17:03:14.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.076+0000 7faaaf7fe640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa9c0666c0 0x7faa9c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:14.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.080+0000 7faaaf7fe640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa9c0666c0 0x7faa9c068b80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faaa40ab000 tx=0x7faaa40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:14.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.084+0000 7faaacff9640 1 -- 172.21.15.39:0/835394658 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faaa0051cb0 con 0x7faa980047d0 2024-09-13T17:03:14.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.328+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faab00a5ff0 con 0x7faa980047d0 2024-09-13T17:03:14.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faaacff9640 1 -- 172.21.15.39:0/835394658 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5931 (secure 0 0 0) 0x7faaa00552d0 con 0x7faa980047d0 2024-09-13T17:03:14.330 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:14.330 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:14.330 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 101 2024-09-13T17:03:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa9c0666c0 msgr2=0x7faa9c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa9c0666c0 0x7faa9c068b80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faaa40ab000 tx=0x7faaa40046d0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 msgr2=0x7faab01f10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faab01f10f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7faaa0001590 tx=0x7faaa000e530 comp rx=0 tx=0).stop 2024-09-13T17:03:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 shutdown_connections 2024-09-13T17:03:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa9c0666c0 0x7faa9c068b80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab0143e70 0x7faab01ed130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faab0155160 0x7faab0143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 --2- 172.21.15.39:0/835394658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa980047d0 0x7faab01f10f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 >> 172.21.15.39:0/835394658 conn(0x7faab00c32a0 msgr2=0x7faab00c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 shutdown_connections 2024-09-13T17:03:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:14.332+0000 7faab62e8640 1 -- 172.21.15.39:0/835394658 wait complete. 2024-09-13T17:03:14.350 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:03:16.351 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:16.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f3551e99640 1 Processor -- start 2024-09-13T17:03:16.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f3551e99640 1 -- start start 2024-09-13T17:03:16.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f3551e99640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f354c06b080 0x7f354c06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:16.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f3551e99640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 0x7f354c151390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:16.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f3551e99640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c0c15d0 con 0x7f354c1518d0 2024-09-13T17:03:16.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f3551e99640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c06d0b0 con 0x7f354c06b080 2024-09-13T17:03:16.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f3551e99640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0c1f00 con 0x7f354c0c1810 2024-09-13T17:03:16.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f354b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f354c06b080 0x7f354c06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:16.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f354b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f354c06b080 0x7f354c06e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54104/0 (socket says 172.21.15.39:54104) 2024-09-13T17:03:16.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f354b7fe640 1 -- 172.21.15.39:0/200910387 learned_addr learned my addr 172.21.15.39:0/200910387 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f354affd640 1 --2- 172.21.15.39:0/200910387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 0x7f354c151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f354b7fe640 1 -- 172.21.15.39:0/200910387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 msgr2=0x7f354c151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f354b7fe640 1 --2- 172.21.15.39:0/200910387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 0x7f354c151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f354b7fe640 1 -- 172.21.15.39:0/200910387 >> v1:172.21.15.84:6789/0 conn(0x7f354c1518d0 legacy=0x7f354c1576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f354b7fe640 1 -- 172.21.15.39:0/200910387 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0c21a0 con 0x7f354c06b080 2024-09-13T17:03:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.511+0000 7f354affd640 1 --2- 172.21.15.39:0/200910387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 0x7f354c151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:03:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f354b7fe640 1 --2- 172.21.15.39:0/200910387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c06b080 0x7f354c06e680 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f353c009870 tx=0x7f353c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae93d59a1d0c4cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f354a7fc640 1 -- 172.21.15.39:0/200910387 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c0ab620 con 0x7f354c06b080 2024-09-13T17:03:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f354a7fc640 1 -- 172.21.15.39:0/200910387 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c0ab780 con 0x7f354c06b080 2024-09-13T17:03:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f354a7fc640 1 -- 172.21.15.39:0/200910387 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353c0abb10 con 0x7f354c06b080 2024-09-13T17:03:16.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 -- 172.21.15.39:0/200910387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c06b080 msgr2=0x7f354c06e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:16.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 --2- 172.21.15.39:0/200910387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c06b080 0x7f354c06e680 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f353c009870 tx=0x7f353c01c460 comp rx=0 tx=0).stop 2024-09-13T17:03:16.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 -- 172.21.15.39:0/200910387 shutdown_connections 2024-09-13T17:03:16.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 --2- 172.21.15.39:0/200910387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 0x7f354c151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:16.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 --2- 172.21.15.39:0/200910387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c06b080 0x7f354c06e680 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:16.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 -- 172.21.15.39:0/200910387 >> 172.21.15.39:0/200910387 conn(0x7f354c0c32a0 msgr2=0x7f354c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:16.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 -- 172.21.15.39:0/200910387 shutdown_connections 2024-09-13T17:03:16.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 -- 172.21.15.39:0/200910387 wait complete. 2024-09-13T17:03:16.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 Processor -- start 2024-09-13T17:03:16.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 -- start start 2024-09-13T17:03:16.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f354c06b080 0x7f354c1ec370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:16.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 0x7f354c153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:16.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c1518d0 0x7f354c1f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:16.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c1ec8b0 con 0x7f354c1518d0 2024-09-13T17:03:16.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c155640 con 0x7f354c06b080 2024-09-13T17:03:16.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f3551e99640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c155920 con 0x7f354c0c1810 2024-09-13T17:03:16.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f354bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c1518d0 0x7f354c1f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:16.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f354bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c1518d0 0x7f354c1f0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54110/0 (socket says 172.21.15.39:54110) 2024-09-13T17:03:16.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f354bfff640 1 -- 172.21.15.39:0/312266881 learned_addr learned my addr 172.21.15.39:0/312266881 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:16.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f354affd640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 0x7f354c153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:16.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.515+0000 7f354b7fe640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f354c06b080 0x7f354c1ec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:16.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f354bfff640 1 -- 172.21.15.39:0/312266881 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 msgr2=0x7f354c153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:16.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f354bfff640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 0x7f354c153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:16.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f354bfff640 1 -- 172.21.15.39:0/312266881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f354c06b080 msgr2=0x7f354c1ec370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:16.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f354bfff640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f354c06b080 0x7f354c1ec370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:16.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f354bfff640 1 -- 172.21.15.39:0/312266881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c0094d0 con 0x7f354c1518d0 2024-09-13T17:03:16.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f354bfff640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c1518d0 0x7f354c1f0180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f354400b970 tx=0x7f354400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:16.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f3533fff640 1 -- 172.21.15.39:0/312266881 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400cb40 con 0x7f354c1518d0 2024-09-13T17:03:16.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f3533fff640 1 -- 172.21.15.39:0/312266881 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354400cca0 con 0x7f354c1518d0 2024-09-13T17:03:16.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c155c00 con 0x7f354c1518d0 2024-09-13T17:03:16.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f3533fff640 1 -- 172.21.15.39:0/312266881 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544014680 con 0x7f354c1518d0 2024-09-13T17:03:16.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c1efc60 con 0x7f354c1518d0 2024-09-13T17:03:16.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c0619c0 con 0x7f354c1518d0 2024-09-13T17:03:16.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f3533fff640 1 -- 172.21.15.39:0/312266881 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f354401d3b0 con 0x7f354c1518d0 2024-09-13T17:03:16.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f3533fff640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3538066690 0x7f3538068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:16.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f3533fff640 1 -- 172.21.15.39:0/312266881 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7f3544084920 con 0x7f354c1518d0 2024-09-13T17:03:16.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f354b7fe640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3538066690 0x7f3538068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:16.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.519+0000 7f354b7fe640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3538066690 0x7f3538068b50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f353c0af000 tx=0x7f353c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:16.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.523+0000 7f3533fff640 1 -- 172.21.15.39:0/312266881 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3544051320 con 0x7f354c1518d0 2024-09-13T17:03:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.751+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f354c1efe70 con 0x7f354c1518d0 2024-09-13T17:03:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3533fff640 1 -- 172.21.15.39:0/312266881 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f3544054940 con 0x7f354c1518d0 2024-09-13T17:03:16.754 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:16.754 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 101 2024-09-13T17:03:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3538066690 msgr2=0x7f3538068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3538066690 0x7f3538068b50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f353c0af000 tx=0x7f353c0a7000 comp rx=0 tx=0).stop 2024-09-13T17:03:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c1518d0 msgr2=0x7f354c1f0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c1518d0 0x7f354c1f0180 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f354400b970 tx=0x7f354400be30 comp rx=0 tx=0).stop 2024-09-13T17:03:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 shutdown_connections 2024-09-13T17:03:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3538066690 0x7f3538068b50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f354c1518d0 0x7f354c1f0180 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f354c0c1810 0x7f354c153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 --2- 172.21.15.39:0/312266881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f354c06b080 0x7f354c1ec370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:16.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 >> 172.21.15.39:0/312266881 conn(0x7f354c0c32a0 msgr2=0x7f354c0c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:16.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 shutdown_connections 2024-09-13T17:03:16.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:16.755+0000 7f3551e99640 1 -- 172.21.15.39:0/312266881 wait complete. 2024-09-13T17:03:16.774 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:03:18.775 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:18.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f998add4640 1 Processor -- start 2024-09-13T17:03:18.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f998add4640 1 -- start start 2024-09-13T17:03:18.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f998add4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 0x7f998415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:18.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f998add4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99840c3ae0 con 0x7f99840c21d0 2024-09-13T17:03:18.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f998add4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99840ab290 con 0x7f998406aac0 2024-09-13T17:03:18.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f998add4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99840abb30 con 0x7f99840aae00 2024-09-13T17:03:18.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f9988b70640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f99840c21d0 0x7f9984151c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40404/0 (socket says 172.21.15.39:40404) 2024-09-13T17:03:18.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f9988b70640 1 -- 172.21.15.39:0/2795704913 learned_addr learned my addr 172.21.15.39:0/2795704913 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:18.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f9989371640 1 --2- 172.21.15.39:0/2795704913 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 0x7f998415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:18.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f99837fe640 1 -- 172.21.15.39:0/2795704913 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731639051 0 0) 0x7f99840c3ae0 con 0x7f99840c21d0 2024-09-13T17:03:18.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f99837fe640 1 -- 172.21.15.39:0/2795704913 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9974003530 con 0x7f99840c21d0 2024-09-13T17:03:18.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f9989371640 1 -- 172.21.15.39:0/2795704913 >> v1:172.21.15.84:6789/0 conn(0x7f998406aac0 legacy=0x7f998414b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:18.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f9989371640 1 -- 172.21.15.39:0/2795704913 >> v1:172.21.15.39:6789/0 conn(0x7f99840c21d0 legacy=0x7f9984151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:18.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.911+0000 7f9989371640 1 -- 172.21.15.39:0/2795704913 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99840abe10 con 0x7f99840aae00 2024-09-13T17:03:18.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f9989371640 1 --2- 172.21.15.39:0/2795704913 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 0x7f998415d5d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f997c009dc0 tx=0x7f997c01c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b772490d026d8e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:18.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f99837fe640 1 -- 172.21.15.39:0/2795704913 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c0a5990 con 0x7f99840aae00 2024-09-13T17:03:18.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f99837fe640 1 -- 172.21.15.39:0/2795704913 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997c0a5af0 con 0x7f99840aae00 2024-09-13T17:03:18.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f99837fe640 1 -- 172.21.15.39:0/2795704913 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c0a5e60 con 0x7f99840aae00 2024-09-13T17:03:18.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f998add4640 1 -- 172.21.15.39:0/2795704913 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 msgr2=0x7f998415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:18.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f998add4640 1 --2- 172.21.15.39:0/2795704913 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 0x7f998415d5d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f997c009dc0 tx=0x7f997c01c8d0 comp rx=0 tx=0).stop 2024-09-13T17:03:18.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f998add4640 1 -- 172.21.15.39:0/2795704913 shutdown_connections 2024-09-13T17:03:18.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f998add4640 1 --2- 172.21.15.39:0/2795704913 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 0x7f998415d5d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:18.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f998add4640 1 -- 172.21.15.39:0/2795704913 >> 172.21.15.39:0/2795704913 conn(0x7f9984069e10 msgr2=0x7f998406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:18.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f998add4640 1 -- 172.21.15.39:0/2795704913 shutdown_connections 2024-09-13T17:03:18.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.915+0000 7f998add4640 1 -- 172.21.15.39:0/2795704913 wait complete. 2024-09-13T17:03:18.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 Processor -- start 2024-09-13T17:03:18.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 -- start start 2024-09-13T17:03:18.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f998406aac0 0x7f99841f0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 0x7f9984143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99840c21d0 0x7f9984070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9984071610 con 0x7f998406aac0 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9984144830 con 0x7f99840c21d0 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9984070d70 con 0x7f99840aae00 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9988b70640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f998406aac0 0x7f99841f0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9988b70640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f998406aac0 0x7f99841f0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54118/0 (socket says 172.21.15.39:54118) 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9988b70640 1 -- 172.21.15.39:0/2754762756 learned_addr learned my addr 172.21.15.39:0/2754762756 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9983fff640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 0x7f9984143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9989371640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99840c21d0 0x7f9984070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:18.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9988b70640 1 -- 172.21.15.39:0/2754762756 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 msgr2=0x7f9984143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:18.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9988b70640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 0x7f9984143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:18.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9988b70640 1 -- 172.21.15.39:0/2754762756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99840c21d0 msgr2=0x7f9984070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:18.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9988b70640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99840c21d0 0x7f9984070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:18.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9988b70640 1 -- 172.21.15.39:0/2754762756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c009a70 con 0x7f998406aac0 2024-09-13T17:03:18.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9989371640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99840c21d0 0x7f9984070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:18.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f9988b70640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f998406aac0 0x7f99841f0830 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f997800c4d0 tx=0x7f997800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:18.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f99817fa640 1 -- 172.21.15.39:0/2754762756 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978005cf0 con 0x7f998406aac0 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f99817fa640 1 -- 172.21.15.39:0/2754762756 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9978005e50 con 0x7f998406aac0 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f99817fa640 1 -- 172.21.15.39:0/2754762756 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978007270 con 0x7f998406aac0 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9984071050 con 0x7f998406aac0 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.919+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998406fd20 con 0x7f998406aac0 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.923+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998414c990 con 0x7f998406aac0 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.923+0000 7f99817fa640 1 -- 172.21.15.39:0/2754762756 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f99780073d0 con 0x7f998406aac0 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.923+0000 7f99817fa640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99680666c0 0x7f9968068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.923+0000 7f99817fa640 1 -- 172.21.15.39:0/2754762756 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 8185+0+0 (secure 0 0 0) 0x7f9978084090 con 0x7f998406aac0 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.923+0000 7f9983fff640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99680666c0 0x7f9968068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.923+0000 7f9983fff640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99680666c0 0x7f9968068b80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9984143910 tx=0x7f9970009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:18.927+0000 7f99817fa640 1 -- 172.21.15.39:0/2754762756 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f998414c990 con 0x7f998406aac0 2024-09-13T17:03:19.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.167+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f998414c990 con 0x7f998406aac0 2024-09-13T17:03:19.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f99817fa640 1 -- 172.21.15.39:0/2754762756 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f998414c990 con 0x7f998406aac0 2024-09-13T17:03:19.171 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:19.171 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6711,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":101}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:01:27.989091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5938":{"gid":5938,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":22,"addr":"172.21.15.84:6835/828722052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":828722052},{"type":"v1","addr":"172.21.15.84:6835","nonce":828722052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:19.171 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 101 2024-09-13T17:03:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99680666c0 msgr2=0x7f9968068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99680666c0 0x7f9968068b80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9984143910 tx=0x7f9970009040 comp rx=0 tx=0).stop 2024-09-13T17:03:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f998406aac0 msgr2=0x7f99841f0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f998406aac0 0x7f99841f0830 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f997800c4d0 tx=0x7f997800c990 comp rx=0 tx=0).stop 2024-09-13T17:03:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 shutdown_connections 2024-09-13T17:03:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99680666c0 0x7f9968068b80 secure :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9984143910 tx=0x7f9970009040 comp rx=0 tx=0).stop 2024-09-13T17:03:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99840c21d0 0x7f9984070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99840aae00 0x7f9984143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 --2- 172.21.15.39:0/2754762756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f998406aac0 0x7f99841f0830 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 >> 172.21.15.39:0/2754762756 conn(0x7f9984069e10 msgr2=0x7f998414b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 shutdown_connections 2024-09-13T17:03:19.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:19.171+0000 7f998add4640 1 -- 172.21.15.39:0/2754762756 wait complete. 2024-09-13T17:03:19.190 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:03:21.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c79353640 1 Processor -- start 2024-09-13T17:03:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c79353640 1 -- start start 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c79353640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3c740c28d0 0x7f3c74154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c79353640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 0x7f3c740c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c79353640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3c740c1280 0x7f3c740c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c79353640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c740611c0 con 0x7f3c740c1280 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c79353640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c740c0ac0 con 0x7f3c740c28d0 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c79353640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740613d0 con 0x7f3c7406fba0 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c737fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3c740c1280 0x7f3c740c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c737fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3c740c1280 0x7f3c740c16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59256/0 (socket says 172.21.15.39:59256) 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.315+0000 7f3c737fe640 1 -- 172.21.15.39:0/1561068143 learned_addr learned my addr 172.21.15.39:0/1561068143 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c727fc640 1 --2- 172.21.15.39:0/1561068143 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 0x7f3c740c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c72ffd640 1 --2- 172.21.15.39:0/1561068143 >> v2:172.21.15.84:3300/0 conn(0x7f3c740c28d0 0x7f3c74154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c737fe640 1 -- 172.21.15.39:0/1561068143 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 msgr2=0x7f3c740c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:21.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c737fe640 1 --2- 172.21.15.39:0/1561068143 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 0x7f3c740c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c737fe640 1 -- 172.21.15.39:0/1561068143 >> v2:172.21.15.84:3300/0 conn(0x7f3c740c28d0 msgr2=0x7f3c74154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:21.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c737fe640 1 --2- 172.21.15.39:0/1561068143 >> v2:172.21.15.84:3300/0 conn(0x7f3c740c28d0 0x7f3c74154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c737fe640 1 -- 172.21.15.39:0/1561068143 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c740616b0 con 0x7f3c740c1280 2024-09-13T17:03:21.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c737fe640 1 --2- 172.21.15.39:0/1561068143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 0x7f3c740c16f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3c7406f530 tx=0x7f3c6c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebc25cc89bcf091 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:21.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c71ffb640 1 -- 172.21.15.39:0/1561068143 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0a8460 con 0x7f3c740c1280 2024-09-13T17:03:21.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c71ffb640 1 -- 172.21.15.39:0/1561068143 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0a85c0 con 0x7f3c740c1280 2024-09-13T17:03:21.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c71ffb640 1 -- 172.21.15.39:0/1561068143 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0a8950 con 0x7f3c740c1280 2024-09-13T17:03:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c79353640 1 -- 172.21.15.39:0/1561068143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 msgr2=0x7f3c740c16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c79353640 1 --2- 172.21.15.39:0/1561068143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 0x7f3c740c16f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3c7406f530 tx=0x7f3c6c01c6a0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c79353640 1 -- 172.21.15.39:0/1561068143 shutdown_connections 2024-09-13T17:03:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c79353640 1 --2- 172.21.15.39:0/1561068143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 0x7f3c740c16f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c79353640 1 --2- 172.21.15.39:0/1561068143 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 0x7f3c740c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c79353640 1 --2- 172.21.15.39:0/1561068143 >> v2:172.21.15.84:3300/0 conn(0x7f3c740c28d0 0x7f3c74154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c79353640 1 -- 172.21.15.39:0/1561068143 >> 172.21.15.39:0/1561068143 conn(0x7f3c74069e10 msgr2=0x7f3c7406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c79353640 1 -- 172.21.15.39:0/1561068143 shutdown_connections 2024-09-13T17:03:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.319+0000 7f3c79353640 1 -- 172.21.15.39:0/1561068143 wait complete. 2024-09-13T17:03:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 Processor -- start 2024-09-13T17:03:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 -- start start 2024-09-13T17:03:21.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 0x7f3c741ec870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 0x7f3c740c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c740c28d0 0x7f3c74147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7414d600 con 0x7f3c740c1280 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7414f900 con 0x7f3c740c28d0 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7414c850 con 0x7f3c7406fba0 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c727fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 0x7f3c740c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c727fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 0x7f3c740c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59260/0 (socket says 172.21.15.39:59260) 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c727fc640 1 -- 172.21.15.39:0/1296788028 learned_addr learned my addr 172.21.15.39:0/1296788028 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c72ffd640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 0x7f3c741ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c737fe640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c740c28d0 0x7f3c74147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c727fc640 1 -- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 msgr2=0x7f3c741ec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c727fc640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 0x7f3c741ec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c727fc640 1 -- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c740c28d0 msgr2=0x7f3c74147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:21.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c727fc640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c740c28d0 0x7f3c74147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c727fc640 1 -- 172.21.15.39:0/1296788028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c009b30 con 0x7f3c740c1280 2024-09-13T17:03:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c72ffd640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 0x7f3c741ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:03:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c737fe640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c740c28d0 0x7f3c74147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:03:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c727fc640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 0x7f3c740c5840 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800d7d0 tx=0x7f3c6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c5b7fe640 1 -- 172.21.15.39:0/1296788028 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680096a0 con 0x7f3c740c1280 2024-09-13T17:03:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c5b7fe640 1 -- 172.21.15.39:0/1296788028 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c68009800 con 0x7f3c740c1280 2024-09-13T17:03:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c741ecdb0 con 0x7f3c740c1280 2024-09-13T17:03:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c5b7fe640 1 -- 172.21.15.39:0/1296788028 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68009bb0 con 0x7f3c740c1280 2024-09-13T17:03:21.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7414ca80 con 0x7f3c740c1280 2024-09-13T17:03:21.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.323+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c740c2cb0 con 0x7f3c740c1280 2024-09-13T17:03:21.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.327+0000 7f3c5b7fe640 1 -- 172.21.15.39:0/1296788028 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c68009d10 con 0x7f3c740c1280 2024-09-13T17:03:21.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.327+0000 7f3c5b7fe640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c60066690 0x7f3c60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.327+0000 7f3c5b7fe640 1 -- 172.21.15.39:0/1296788028 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7f3c68085100 con 0x7f3c740c1280 2024-09-13T17:03:21.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.327+0000 7f3c72ffd640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c60066690 0x7f3c60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.327+0000 7f3c72ffd640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c60066690 0x7f3c60068b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3c7414d760 tx=0x7f3c64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:21.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.331+0000 7f3c5b7fe640 1 -- 172.21.15.39:0/1296788028 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c68051a20 con 0x7f3c740c1280 2024-09-13T17:03:21.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c740bf690 con 0x7f3c740c1280 2024-09-13T17:03:21.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c5b7fe640 1 -- 172.21.15.39:0/1296788028 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+5180 (secure 0 0 0) 0x7f3c68055040 con 0x7f3c740c1280 2024-09-13T17:03:21.579 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:21.579 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:21.014422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:reconnect","state_seq":5,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:21.579 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 103 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c60066690 msgr2=0x7f3c60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c60066690 0x7f3c60068b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3c7414d760 tx=0x7f3c64008040 comp rx=0 tx=0).stop 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 msgr2=0x7f3c740c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 0x7f3c740c5840 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800d7d0 tx=0x7f3c6800dc90 comp rx=0 tx=0).stop 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 shutdown_connections 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c60066690 0x7f3c60068b50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c740c28d0 0x7f3c74147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c740c1280 0x7f3c740c5840 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 --2- 172.21.15.39:0/1296788028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3c7406fba0 0x7f3c741ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 >> 172.21.15.39:0/1296788028 conn(0x7f3c74069e10 msgr2=0x7f3c74154710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:21.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 shutdown_connections 2024-09-13T17:03:21.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.579+0000 7f3c79353640 1 -- 172.21.15.39:0/1296788028 wait complete. 2024-09-13T17:03:21.598 INFO:tasks.mds_thrash.fs.[b]:mds.d down, removed from mdsmap 2024-09-13T17:03:21.598 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:03:21.598 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:21.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.735+0000 7f2705cba640 1 Processor -- start 2024-09-13T17:03:21.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f2705cba640 1 -- start start 2024-09-13T17:03:21.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f2705cba640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2700065ee0 0x7f27000662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f2705cba640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f27000c6270 0x7f27000bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f2705cba640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270006ddf0 con 0x7f27000c2ee0 2024-09-13T17:03:21.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f2705cba640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27000c4710 con 0x7f2700065ee0 2024-09-13T17:03:21.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f2705cba640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27000c49d0 con 0x7f27000c6270 2024-09-13T17:03:21.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26ff7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2700065ee0 0x7f27000662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26ff7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2700065ee0 0x7f27000662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59274/0 (socket says 172.21.15.39:59274) 2024-09-13T17:03:21.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26ff7fe640 1 -- 172.21.15.39:0/1700782243 learned_addr learned my addr 172.21.15.39:0/1700782243 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:21.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26fffff640 1 --2- 172.21.15.39:0/1700782243 >> v2:172.21.15.84:3300/0 conn(0x7f27000c6270 0x7f27000bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26ff7fe640 1 -- 172.21.15.39:0/1700782243 >> v2:172.21.15.84:3300/0 conn(0x7f27000c6270 msgr2=0x7f27000bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:21.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26ff7fe640 1 --2- 172.21.15.39:0/1700782243 >> v2:172.21.15.84:3300/0 conn(0x7f27000c6270 0x7f27000bfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26ff7fe640 1 -- 172.21.15.39:0/1700782243 >> v1:172.21.15.84:6789/0 conn(0x7f27000c2ee0 legacy=0x7f27000c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:21.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26ff7fe640 1 -- 172.21.15.39:0/1700782243 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2700154880 con 0x7f2700065ee0 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26ff7fe640 1 --2- 172.21.15.39:0/1700782243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2700065ee0 0x7f27000662c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f27000c78d0 tx=0x7f26f001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0fc88b1c2fb736 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26feffd640 1 -- 172.21.15.39:0/1700782243 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00ad660 con 0x7f2700065ee0 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26feffd640 1 -- 172.21.15.39:0/1700782243 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f00ad7c0 con 0x7f2700065ee0 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.739+0000 7f26feffd640 1 -- 172.21.15.39:0/1700782243 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00adb50 con 0x7f2700065ee0 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- 172.21.15.39:0/1700782243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2700065ee0 msgr2=0x7f27000662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 --2- 172.21.15.39:0/1700782243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2700065ee0 0x7f27000662c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f27000c78d0 tx=0x7f26f001c450 comp rx=0 tx=0).stop 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- 172.21.15.39:0/1700782243 shutdown_connections 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 --2- 172.21.15.39:0/1700782243 >> v2:172.21.15.84:3300/0 conn(0x7f27000c6270 0x7f27000bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 --2- 172.21.15.39:0/1700782243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2700065ee0 0x7f27000662c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- 172.21.15.39:0/1700782243 >> 172.21.15.39:0/1700782243 conn(0x7f27000c3310 msgr2=0x7f27000c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:21.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- 172.21.15.39:0/1700782243 shutdown_connections 2024-09-13T17:03:21.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- 172.21.15.39:0/1700782243 wait complete. 2024-09-13T17:03:21.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 Processor -- start 2024-09-13T17:03:21.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- start start 2024-09-13T17:03:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2700065ee0 0x7f27000a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000c2ee0 0x7f27001452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27000c6270 0x7f2700145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2700158090 con 0x7f27000c2ee0 2024-09-13T17:03:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2700157db0 con 0x7f2700065ee0 2024-09-13T17:03:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2700147a30 con 0x7f27000c6270 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000c2ee0 0x7f27001452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000c2ee0 0x7f27001452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59286/0 (socket says 172.21.15.39:59286) 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26f7fff640 1 -- 172.21.15.39:0/273013443 learned_addr learned my addr 172.21.15.39:0/273013443 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26ff7fe640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2700065ee0 0x7f27000a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26f7fff640 1 -- 172.21.15.39:0/273013443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27000c6270 msgr2=0x7f2700145810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26f7fff640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27000c6270 0x7f2700145810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26f7fff640 1 -- 172.21.15.39:0/273013443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2700065ee0 msgr2=0x7f27000a6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26f7fff640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2700065ee0 0x7f27000a6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26f7fff640 1 -- 172.21.15.39:0/273013443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f00094d0 con 0x7f27000c2ee0 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26f7fff640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000c2ee0 0x7f27001452d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c450 tx=0x7f26e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26fcff9640 1 -- 172.21.15.39:0/273013443 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8012830 con 0x7f27000c2ee0 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26fcff9640 1 -- 172.21.15.39:0/273013443 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e80129b0 con 0x7f27000c2ee0 2024-09-13T17:03:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f26fcff9640 1 -- 172.21.15.39:0/273013443 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8012d60 con 0x7f27000c2ee0 2024-09-13T17:03:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2700147d10 con 0x7f27000c2ee0 2024-09-13T17:03:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.743+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27000a7140 con 0x7f27000c2ee0 2024-09-13T17:03:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.747+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27000a7420 con 0x7f27000c2ee0 2024-09-13T17:03:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.747+0000 7f26fcff9640 1 -- 172.21.15.39:0/273013443 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f26e801c460 con 0x7f27000c2ee0 2024-09-13T17:03:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.747+0000 7f26fcff9640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f26e00666c0 0x7f26e0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.747+0000 7f26fcff9640 1 -- 172.21.15.39:0/273013443 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7f26e8084a20 con 0x7f27000c2ee0 2024-09-13T17:03:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.747+0000 7f26ff7fe640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f26e00666c0 0x7f26e0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.747+0000 7f26ff7fe640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f26e00666c0 0x7f26e0068b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f26f00af000 tx=0x7f26f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:21.751+0000 7f26fcff9640 1 -- 172.21.15.39:0/273013443 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f26e80513d0 con 0x7f27000c2ee0 2024-09-13T17:03:22.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.007+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2700147ea0 con 0x7f27000c2ee0 2024-09-13T17:03:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.007+0000 7f26fcff9640 1 -- 172.21.15.39:0/273013443 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+5180 (secure 0 0 0) 0x7f26e80549f0 con 0x7f27000c2ee0 2024-09-13T17:03:22.007 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:22.007 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:21.014422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:reconnect","state_seq":5,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:22.008 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 103 2024-09-13T17:03:22.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f26e00666c0 msgr2=0x7f26e0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:22.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f26e00666c0 0x7f26e0068b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f26f00af000 tx=0x7f26f0008040 comp rx=0 tx=0).stop 2024-09-13T17:03:22.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000c2ee0 msgr2=0x7f27001452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:22.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000c2ee0 0x7f27001452d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c450 tx=0x7f26e800c910 comp rx=0 tx=0).stop 2024-09-13T17:03:22.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 shutdown_connections 2024-09-13T17:03:22.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f26e00666c0 0x7f26e0068b80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:22.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27000c6270 0x7f2700145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:22.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000c2ee0 0x7f27001452d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:22.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 --2- 172.21.15.39:0/273013443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2700065ee0 0x7f27000a6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:22.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 >> 172.21.15.39:0/273013443 conn(0x7f27000c3310 msgr2=0x7f270015ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:22.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 shutdown_connections 2024-09-13T17:03:22.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:22.011+0000 7f2705cba640 1 -- 172.21.15.39:0/273013443 wait complete. 2024-09-13T17:03:22.030 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 103, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 100, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6585": { "addr": "172.21.15.39:6835/611669592", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 611669592, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 611669592, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6585, "incarnation": 97, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 43, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:03:03.021063+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6585 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 103, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6711": { "addr": "172.21.15.39:6837/3105501456", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 3105501456, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 3105501456, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6711, "incarnation": 102, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 44, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:03:21.014422+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6711 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/440028459", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 440028459, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 440028459, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 96, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5561, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:03:23.140 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-13T17:03:23.139+0000 7f4081384640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:03:24.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:24.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f15a91c9640 1 Processor -- start 2024-09-13T17:03:24.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f15a91c9640 1 -- start start 2024-09-13T17:03:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f15a91c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15a40c2370 0x7f15a40c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f15a91c9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f15a4154bb0 0x7f15a40c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f15a91c9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a4153e30 con 0x7f15a40c5870 2024-09-13T17:03:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f15a91c9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a40720f0 con 0x7f15a4154bb0 2024-09-13T17:03:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f15a91c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a4151b30 con 0x7f15a40c2370 2024-09-13T17:03:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f159a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f15a4154bb0 0x7f15a40c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f159a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f15a4154bb0 0x7f15a40c0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59290/0 (socket says 172.21.15.39:59290) 2024-09-13T17:03:24.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f159a59c640 1 -- 172.21.15.39:0/473581590 learned_addr learned my addr 172.21.15.39:0/473581590 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f15a2d9d640 1 --2- 172.21.15.39:0/473581590 >> v2:172.21.15.84:3300/0 conn(0x7f15a40c2370 0x7f15a40c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f159a59c640 1 -- 172.21.15.39:0/473581590 >> v2:172.21.15.84:3300/0 conn(0x7f15a40c2370 msgr2=0x7f15a40c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f159a59c640 1 --2- 172.21.15.39:0/473581590 >> v2:172.21.15.84:3300/0 conn(0x7f15a40c2370 0x7f15a40c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f159a59c640 1 -- 172.21.15.39:0/473581590 >> v1:172.21.15.84:6789/0 conn(0x7f15a40c5870 legacy=0x7f15a4157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f159a59c640 1 -- 172.21.15.39:0/473581590 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a40a7000 con 0x7f15a4154bb0 2024-09-13T17:03:24.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.167+0000 7f15a2d9d640 1 --2- 172.21.15.39:0/473581590 >> v2:172.21.15.84:3300/0 conn(0x7f15a40c2370 0x7f15a40c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f159a59c640 1 --2- 172.21.15.39:0/473581590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 0x7f15a40c0da0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f15a4071140 tx=0x7f158c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d989a0604edd2d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a259c640 1 -- 172.21.15.39:0/473581590 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c0ac600 con 0x7f15a4154bb0 2024-09-13T17:03:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a259c640 1 -- 172.21.15.39:0/473581590 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c0a8050 con 0x7f15a4154bb0 2024-09-13T17:03:24.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a259c640 1 -- 172.21.15.39:0/473581590 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c0ac980 con 0x7f15a4154bb0 2024-09-13T17:03:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a91c9640 1 -- 172.21.15.39:0/473581590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 msgr2=0x7f15a40c0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a91c9640 1 --2- 172.21.15.39:0/473581590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 0x7f15a40c0da0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f15a4071140 tx=0x7f158c01c490 comp rx=0 tx=0).stop 2024-09-13T17:03:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a91c9640 1 -- 172.21.15.39:0/473581590 shutdown_connections 2024-09-13T17:03:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a91c9640 1 --2- 172.21.15.39:0/473581590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 0x7f15a40c0da0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a91c9640 1 --2- 172.21.15.39:0/473581590 >> v2:172.21.15.84:3300/0 conn(0x7f15a40c2370 0x7f15a40c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a91c9640 1 -- 172.21.15.39:0/473581590 >> 172.21.15.39:0/473581590 conn(0x7f15a40a9720 msgr2=0x7f15a40a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a91c9640 1 -- 172.21.15.39:0/473581590 shutdown_connections 2024-09-13T17:03:24.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.171+0000 7f15a91c9640 1 -- 172.21.15.39:0/473581590 wait complete. 2024-09-13T17:03:24.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 Processor -- start 2024-09-13T17:03:24.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 -- start start 2024-09-13T17:03:24.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15a40c2370 0x7f15a414dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:24.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15a40c5870 0x7f15a41ed070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:24.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 0x7f15a41ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:24.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40a7550 con 0x7f15a4154bb0 2024-09-13T17:03:24.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a40a8e10 con 0x7f15a40c5870 2024-09-13T17:03:24.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4156890 con 0x7f15a40c2370 2024-09-13T17:03:24.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 0x7f15a41ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:24.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 0x7f15a41ed5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59306/0 (socket says 172.21.15.39:59306) 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a359e640 1 -- 172.21.15.39:0/2880919979 learned_addr learned my addr 172.21.15.39:0/2880919979 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a2d9d640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15a40c2370 0x7f15a414dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f159a59c640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15a40c5870 0x7f15a41ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a359e640 1 -- 172.21.15.39:0/2880919979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15a40c2370 msgr2=0x7f15a414dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a359e640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15a40c2370 0x7f15a414dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a359e640 1 -- 172.21.15.39:0/2880919979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15a40c5870 msgr2=0x7f15a41ed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a359e640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15a40c5870 0x7f15a41ed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a359e640 1 -- 172.21.15.39:0/2880919979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0094d0 con 0x7f15a4154bb0 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a359e640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 0x7f15a41ed5b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c770 tx=0x7f159c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f159bfff640 1 -- 172.21.15.39:0/2880919979 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c003b80 con 0x7f15a4154bb0 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f159bfff640 1 -- 172.21.15.39:0/2880919979 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c003ce0 con 0x7f15a4154bb0 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a40a9060 con 0x7f15a4154bb0 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f159bfff640 1 -- 172.21.15.39:0/2880919979 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c00aa90 con 0x7f15a4154bb0 2024-09-13T17:03:24.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a4156240 con 0x7f15a4154bb0 2024-09-13T17:03:24.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a41ecd90 con 0x7f15a4154bb0 2024-09-13T17:03:24.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.175+0000 7f159bfff640 1 -- 172.21.15.39:0/2880919979 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f159c00abf0 con 0x7f15a4154bb0 2024-09-13T17:03:24.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.179+0000 7f159bfff640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1584066690 0x7f1584068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:24.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.179+0000 7f15a2d9d640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1584066690 0x7f1584068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:24.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.179+0000 7f159bfff640 1 -- 172.21.15.39:0/2880919979 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7f159c0849c0 con 0x7f15a4154bb0 2024-09-13T17:03:24.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.179+0000 7f15a2d9d640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1584066690 0x7f1584068b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f159400c900 tx=0x7f1594008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:24.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.183+0000 7f159bfff640 1 -- 172.21.15.39:0/2880919979 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15a41ecd90 con 0x7f15a4154bb0 2024-09-13T17:03:24.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.407+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15a41ecd90 con 0x7f15a4154bb0 2024-09-13T17:03:24.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f159bfff640 1 -- 172.21.15.39:0/2880919979 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+5177 (secure 0 0 0) 0x7f15a41ecd90 con 0x7f15a4154bb0 2024-09-13T17:03:24.410 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:24.410 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:23.025135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:24.410 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 105 2024-09-13T17:03:24.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1584066690 msgr2=0x7f1584068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:24.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1584066690 0x7f1584068b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f159400c900 tx=0x7f1594008040 comp rx=0 tx=0).stop 2024-09-13T17:03:24.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 msgr2=0x7f15a41ed5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:24.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 0x7f15a41ed5b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c770 tx=0x7f159c00cc30 comp rx=0 tx=0).stop 2024-09-13T17:03:24.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 shutdown_connections 2024-09-13T17:03:24.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1584066690 0x7f1584068b50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:24.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15a40c5870 0x7f15a41ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:24.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15a4154bb0 0x7f15a41ed5b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:24.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 --2- 172.21.15.39:0/2880919979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f15a40c2370 0x7f15a414dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:24.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 >> 172.21.15.39:0/2880919979 conn(0x7f15a40a9720 msgr2=0x7f15a40c1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:24.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 shutdown_connections 2024-09-13T17:03:24.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:24.411+0000 7f15a91c9640 1 -- 172.21.15.39:0/2880919979 wait complete. 2024-09-13T17:03:24.432 INFO:tasks.mds_thrash.fs.[b]:mds.c has gained rank=0, replacing gid=5938 2024-09-13T17:03:24.432 INFO:tasks.mds_thrash.fs.[b]:waiting for 9 secs before reviving mds.d 2024-09-13T17:03:28.668 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:03:28.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:03:33.433 INFO:tasks.mds_thrash.fs.[b]:reviving mds.d 2024-09-13T17:03:33.434 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T17:03:33.434 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T17:03:33.436 INFO:tasks.ceph.mds.d:Started 2024-09-13T17:03:33.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:33.511 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T17:03:33.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.575+0000 7f21a5e70640 1 Processor -- start 2024-09-13T17:03:33.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.575+0000 7f21a5e70640 1 -- start start 2024-09-13T17:03:33.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.575+0000 7f21a5e70640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f21a0071af0 0x7f21a0150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:33.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.575+0000 7f21a5e70640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a00c4760 con 0x7f21a014a770 2024-09-13T17:03:33.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.575+0000 7f21a5e70640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a0151120 con 0x7f21a00c6060 2024-09-13T17:03:33.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.575+0000 7f21a5e70640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a0071f40 con 0x7f21a0071af0 2024-09-13T17:03:33.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.575+0000 7f219ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f21a0071af0 0x7f21a0150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.575+0000 7f219f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f21a014a770 0x7f21a006ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34842/0 (socket says 172.21.15.39:34842) 2024-09-13T17:03:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.575+0000 7f219f7fe640 1 -- 172.21.15.39:0/1124734313 learned_addr learned my addr 172.21.15.39:0/1124734313 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.579+0000 7f219ffff640 1 -- 172.21.15.39:0/1124734313 >> v1:172.21.15.84:6789/0 conn(0x7f21a00c6060 legacy=0x7f21a00713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.579+0000 7f219ffff640 1 -- 172.21.15.39:0/1124734313 >> v1:172.21.15.39:6789/0 conn(0x7f21a014a770 legacy=0x7f21a006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:33.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.579+0000 7f219ffff640 1 -- 172.21.15.39:0/1124734313 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a0151400 con 0x7f21a0071af0 2024-09-13T17:03:33.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.579+0000 7f219ffff640 1 --2- 172.21.15.39:0/1124734313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a0071af0 0x7f21a0150ad0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f219800bfa0 tx=0x7f219801e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadcf5cc95aac7b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:33.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.579+0000 7f219e7fc640 1 -- 172.21.15.39:0/1124734313 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800b2d0 con 0x7f21a0071af0 2024-09-13T17:03:33.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.579+0000 7f219e7fc640 1 -- 172.21.15.39:0/1124734313 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21980a75c0 con 0x7f21a0071af0 2024-09-13T17:03:33.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.579+0000 7f219e7fc640 1 -- 172.21.15.39:0/1124734313 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21980a7950 con 0x7f21a0071af0 2024-09-13T17:03:33.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.579+0000 7f21a5e70640 1 -- 172.21.15.39:0/1124734313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a0071af0 msgr2=0x7f21a0150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:33.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.579+0000 7f21a5e70640 1 --2- 172.21.15.39:0/1124734313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a0071af0 0x7f21a0150ad0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f219800bfa0 tx=0x7f219801e970 comp rx=0 tx=0).stop 2024-09-13T17:03:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- 172.21.15.39:0/1124734313 shutdown_connections 2024-09-13T17:03:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 --2- 172.21.15.39:0/1124734313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a0071af0 0x7f21a0150ad0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- 172.21.15.39:0/1124734313 >> 172.21.15.39:0/1124734313 conn(0x7f21a00c32a0 msgr2=0x7f21a00c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- 172.21.15.39:0/1124734313 shutdown_connections 2024-09-13T17:03:33.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- 172.21.15.39:0/1124734313 wait complete. 2024-09-13T17:03:33.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 Processor -- start 2024-09-13T17:03:33.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- start start 2024-09-13T17:03:33.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0071af0 0x7f21a0152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:33.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f21a00c6060 0x7f21a01f0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:33.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a014a770 0x7f21a00c0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:33.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00bf150 con 0x7f21a014a770 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00a6130 con 0x7f21a0071af0 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a00a5e50 con 0x7f21a00c6060 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a014a770 0x7f21a00c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a014a770 0x7f21a00c0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38822/0 (socket says 172.21.15.39:38822) 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219ffff640 1 -- 172.21.15.39:0/386864335 learned_addr learned my addr 172.21.15.39:0/386864335 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219f7fe640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0071af0 0x7f21a0152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219effd640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f21a00c6060 0x7f21a01f0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219ffff640 1 -- 172.21.15.39:0/386864335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f21a00c6060 msgr2=0x7f21a01f0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219ffff640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f21a00c6060 0x7f21a01f0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219ffff640 1 -- 172.21.15.39:0/386864335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0071af0 msgr2=0x7f21a0152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219ffff640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0071af0 0x7f21a0152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219ffff640 1 -- 172.21.15.39:0/386864335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219800bc80 con 0x7f21a014a770 2024-09-13T17:03:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f219ffff640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a014a770 0x7f21a00c0590 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f21980b1040 tx=0x7f219801b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:33.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f2187fff640 1 -- 172.21.15.39:0/386864335 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21980bd070 con 0x7f21a014a770 2024-09-13T17:03:33.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f2187fff640 1 -- 172.21.15.39:0/386864335 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21980a7c30 con 0x7f21a014a770 2024-09-13T17:03:33.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a01f0af0 con 0x7f21a014a770 2024-09-13T17:03:33.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f2187fff640 1 -- 172.21.15.39:0/386864335 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198009540 con 0x7f21a014a770 2024-09-13T17:03:33.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.583+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a00c0070 con 0x7f21a014a770 2024-09-13T17:03:33.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.587+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a00619c0 con 0x7f21a014a770 2024-09-13T17:03:33.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.587+0000 7f2187fff640 1 -- 172.21.15.39:0/386864335 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f21980096a0 con 0x7f21a014a770 2024-09-13T17:03:33.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.587+0000 7f2187fff640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f218c0666c0 0x7f218c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:33.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.587+0000 7f2187fff640 1 -- 172.21.15.39:0/386864335 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7f219811f1a0 con 0x7f21a014a770 2024-09-13T17:03:33.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.587+0000 7f219f7fe640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f218c0666c0 0x7f218c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:33.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.587+0000 7f219f7fe640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f218c0666c0 0x7f218c068b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2190000f70 tx=0x7f2190007e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:33.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.591+0000 7f2187fff640 1 -- 172.21.15.39:0/386864335 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f21980ebc00 con 0x7f21a014a770 2024-09-13T17:03:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.883+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21a00c0260 con 0x7f21a014a770 2024-09-13T17:03:33.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.883+0000 7f2187fff640 1 -- 172.21.15.39:0/386864335 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+5177 (secure 0 0 0) 0x7f21980ef220 con 0x7f21a014a770 2024-09-13T17:03:33.885 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:33.885 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:23.025135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:33.886 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 105 2024-09-13T17:03:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f218c0666c0 msgr2=0x7f218c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f218c0666c0 0x7f218c068b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2190000f70 tx=0x7f2190007e10 comp rx=0 tx=0).stop 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a014a770 msgr2=0x7f21a00c0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a014a770 0x7f21a00c0590 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f21980b1040 tx=0x7f219801b710 comp rx=0 tx=0).stop 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 shutdown_connections 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f218c0666c0 0x7f218c068b80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21a014a770 0x7f21a00c0590 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f21a00c6060 0x7f21a01f0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 --2- 172.21.15.39:0/386864335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0071af0 0x7f21a0152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 >> 172.21.15.39:0/386864335 conn(0x7f21a00c32a0 msgr2=0x7f21a014f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 shutdown_connections 2024-09-13T17:03:33.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:33.887+0000 7f21a5e70640 1 -- 172.21.15.39:0/386864335 wait complete. 2024-09-13T17:03:33.908 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-09-13T17:03:35.910 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.047+0000 7f0af5343640 1 Processor -- start 2024-09-13T17:03:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.047+0000 7f0af5343640 1 -- start start 2024-09-13T17:03:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0af5343640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af006fa50 0x7f0af0156130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0af5343640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0af00c6000 0x7f0af00c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0af5343640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af00c5e50 con 0x7f0af00c7960 2024-09-13T17:03:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0af5343640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af006ff50 con 0x7f0af00c6000 2024-09-13T17:03:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0af5343640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af0072000 con 0x7f0af006fa50 2024-09-13T17:03:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aeeffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0af00c7960 0x7f0af0155a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34848/0 (socket says 172.21.15.39:34848) 2024-09-13T17:03:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aeeffd640 1 -- 172.21.15.39:0/579056308 learned_addr learned my addr 172.21.15.39:0/579056308 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aee7fc640 1 --2- 172.21.15.39:0/579056308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af006fa50 0x7f0af0156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aef7fe640 1 --2- 172.21.15.39:0/579056308 >> v2:172.21.15.39:3300/0 conn(0x7f0af00c6000 0x7f0af00c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aedffb640 1 -- 172.21.15.39:0/579056308 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571296244 0 0) 0x7f0af00c5e50 con 0x7f0af00c7960 2024-09-13T17:03:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aedffb640 1 -- 172.21.15.39:0/579056308 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad8003530 con 0x7f0af00c7960 2024-09-13T17:03:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aee7fc640 1 -- 172.21.15.39:0/579056308 >> v2:172.21.15.39:3300/0 conn(0x7f0af00c6000 msgr2=0x7f0af00c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aedffb640 1 -- 172.21.15.39:0/579056308 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2574413904 0 0) 0x7f0ad8003530 con 0x7f0af00c7960 2024-09-13T17:03:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aee7fc640 1 --2- 172.21.15.39:0/579056308 >> v2:172.21.15.39:3300/0 conn(0x7f0af00c6000 0x7f0af00c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aee7fc640 1 -- 172.21.15.39:0/579056308 >> v1:172.21.15.39:6789/0 conn(0x7f0af00c7960 legacy=0x7f0af0155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aee7fc640 1 -- 172.21.15.39:0/579056308 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af00722e0 con 0x7f0af006fa50 2024-09-13T17:03:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aef7fe640 1 --2- 172.21.15.39:0/579056308 >> v2:172.21.15.39:3300/0 conn(0x7f0af00c6000 0x7f0af00c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:03:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.051+0000 7f0aee7fc640 1 --2- 172.21.15.39:0/579056308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af006fa50 0x7f0af0156130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40097c0 tx=0x7f0ae401c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a92738f3f78434d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aedffb640 1 -- 172.21.15.39:0/579056308 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0001680 con 0x7f0af006fa50 2024-09-13T17:03:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aedffb640 1 -- 172.21.15.39:0/579056308 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae40a7050 con 0x7f0af006fa50 2024-09-13T17:03:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aedffb640 1 -- 172.21.15.39:0/579056308 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae40ab600 con 0x7f0af006fa50 2024-09-13T17:03:36.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- 172.21.15.39:0/579056308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af006fa50 msgr2=0x7f0af0156130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:36.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 --2- 172.21.15.39:0/579056308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af006fa50 0x7f0af0156130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40097c0 tx=0x7f0ae401c300 comp rx=0 tx=0).stop 2024-09-13T17:03:36.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- 172.21.15.39:0/579056308 shutdown_connections 2024-09-13T17:03:36.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 --2- 172.21.15.39:0/579056308 >> v2:172.21.15.39:3300/0 conn(0x7f0af00c6000 0x7f0af00c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:36.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 --2- 172.21.15.39:0/579056308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af006fa50 0x7f0af0156130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:36.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- 172.21.15.39:0/579056308 >> 172.21.15.39:0/579056308 conn(0x7f0af006c020 msgr2=0x7f0af006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:36.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- 172.21.15.39:0/579056308 shutdown_connections 2024-09-13T17:03:36.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- 172.21.15.39:0/579056308 wait complete. 2024-09-13T17:03:36.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 Processor -- start 2024-09-13T17:03:36.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- start start 2024-09-13T17:03:36.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0af006fa50 0x7f0af01eab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:36.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af00c6000 0x7f0af0156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:36.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af00c7960 0x7f0af0150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:36.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01eb110 con 0x7f0af006fa50 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af015abc0 con 0x7f0af00c7960 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af015adf0 con 0x7f0af00c6000 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aeeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0af006fa50 0x7f0af01eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aeeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0af006fa50 0x7f0af01eab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38840/0 (socket says 172.21.15.39:38840) 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aeeffd640 1 -- 172.21.15.39:0/3000850249 learned_addr learned my addr 172.21.15.39:0/3000850249 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aee7fc640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af00c6000 0x7f0af0156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aef7fe640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af00c7960 0x7f0af0150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aeeffd640 1 -- 172.21.15.39:0/3000850249 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af00c6000 msgr2=0x7f0af0156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aeeffd640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af00c6000 0x7f0af0156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aeeffd640 1 -- 172.21.15.39:0/3000850249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af00c7960 msgr2=0x7f0af0150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aeeffd640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af00c7960 0x7f0af0150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aeeffd640 1 -- 172.21.15.39:0/3000850249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8009a70 con 0x7f0af006fa50 2024-09-13T17:03:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0aeeffd640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0af006fa50 0x7f0af01eab20 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c780 tx=0x7f0ae000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:36.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0ad77fe640 1 -- 172.21.15.39:0/3000850249 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0005ae0 con 0x7f0af006fa50 2024-09-13T17:03:36.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0ad77fe640 1 -- 172.21.15.39:0/3000850249 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae0005c40 con 0x7f0af006fa50 2024-09-13T17:03:36.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae4009420 con 0x7f0af006fa50 2024-09-13T17:03:36.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0ad77fe640 1 -- 172.21.15.39:0/3000850249 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0007070 con 0x7f0af006fa50 2024-09-13T17:03:36.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.055+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af015b3b0 con 0x7f0af006fa50 2024-09-13T17:03:36.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.059+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af0151960 con 0x7f0af006fa50 2024-09-13T17:03:36.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.059+0000 7f0ad77fe640 1 -- 172.21.15.39:0/3000850249 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ae00071f0 con 0x7f0af006fa50 2024-09-13T17:03:36.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.059+0000 7f0ad77fe640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0adc0666c0 0x7f0adc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:36.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.059+0000 7f0ad77fe640 1 -- 172.21.15.39:0/3000850249 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7f0ae0084730 con 0x7f0af006fa50 2024-09-13T17:03:36.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.059+0000 7f0aee7fc640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0adc0666c0 0x7f0adc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:36.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.059+0000 7f0aee7fc640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0adc0666c0 0x7f0adc068b80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40ad000 tx=0x7f0ae4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:36.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.063+0000 7f0ad77fe640 1 -- 172.21.15.39:0/3000850249 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0ae0051110 con 0x7f0af006fa50 2024-09-13T17:03:36.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0af00beee0 con 0x7f0af006fa50 2024-09-13T17:03:36.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0ad77fe640 1 -- 172.21.15.39:0/3000850249 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f0ae0054730 con 0x7f0af006fa50 2024-09-13T17:03:36.327 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:36.327 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:23.025135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:36.327 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 106 2024-09-13T17:03:36.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0adc0666c0 msgr2=0x7f0adc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:36.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0adc0666c0 0x7f0adc068b80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40ad000 tx=0x7f0ae4004740 comp rx=0 tx=0).stop 2024-09-13T17:03:36.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0af006fa50 msgr2=0x7f0af01eab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:36.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0af006fa50 0x7f0af01eab20 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c780 tx=0x7f0ae000cc40 comp rx=0 tx=0).stop 2024-09-13T17:03:36.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 shutdown_connections 2024-09-13T17:03:36.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0adc0666c0 0x7f0adc068b80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:36.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0af00c7960 0x7f0af0150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:36.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0af00c6000 0x7f0af0156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:36.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 --2- 172.21.15.39:0/3000850249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0af006fa50 0x7f0af01eab20 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:36.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 >> 172.21.15.39:0/3000850249 conn(0x7f0af006c020 msgr2=0x7f0af015bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:36.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 shutdown_connections 2024-09-13T17:03:36.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:36.327+0000 7f0af5343640 1 -- 172.21.15.39:0/3000850249 wait complete. 2024-09-13T17:03:36.348 INFO:tasks.mds_thrash.fs.[b]:mds.d reported in up:standby state 2024-09-13T17:03:36.348 INFO:tasks.mds_thrash.fs.[b]:waiting for 1 secs before thrashing 2024-09-13T17:03:37.349 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:37.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.467+0000 7f9273e44640 1 Processor -- start 2024-09-13T17:03:37.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.467+0000 7f9273e44640 1 -- start start 2024-09-13T17:03:37.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.467+0000 7f9273e44640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f926c154bb0 0x7f926c0c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.467+0000 7f9273e44640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c0c5870 0x7f926c15e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.467+0000 7f9273e44640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c153e30 con 0x7f926c0c2370 2024-09-13T17:03:37.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.467+0000 7f9273e44640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c0720f0 con 0x7f926c154bb0 2024-09-13T17:03:37.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.467+0000 7f9273e44640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c151b30 con 0x7f926c0c5870 2024-09-13T17:03:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92723e1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c0c5870 0x7f926c15e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92723e1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c0c5870 0x7f926c15e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:46010/0 (socket says 172.21.15.39:46010) 2024-09-13T17:03:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92723e1640 1 -- 172.21.15.39:0/2981131343 learned_addr learned my addr 172.21.15.39:0/2981131343 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92713df640 1 --2- 172.21.15.39:0/2981131343 >> v2:172.21.15.84:3300/0 conn(0x7f926c154bb0 0x7f926c0c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92723e1640 1 -- 172.21.15.39:0/2981131343 >> v2:172.21.15.84:3300/0 conn(0x7f926c154bb0 msgr2=0x7f926c0c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92723e1640 1 --2- 172.21.15.39:0/2981131343 >> v2:172.21.15.84:3300/0 conn(0x7f926c154bb0 0x7f926c0c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92723e1640 1 -- 172.21.15.39:0/2981131343 >> v1:172.21.15.84:6789/0 conn(0x7f926c0c2370 legacy=0x7f926c0c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92723e1640 1 -- 172.21.15.39:0/2981131343 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0a7000 con 0x7f926c0c5870 2024-09-13T17:03:37.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92713df640 1 --2- 172.21.15.39:0/2981131343 >> v2:172.21.15.84:3300/0 conn(0x7f926c154bb0 0x7f926c0c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:03:37.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f92723e1640 1 --2- 172.21.15.39:0/2981131343 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c0c5870 0x7f926c15e7b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9264009e50 tx=0x7f926401c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5bd18f7ec737b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:37.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9270bde640 1 -- 172.21.15.39:0/2981131343 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640a5750 con 0x7f926c0c5870 2024-09-13T17:03:37.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9270bde640 1 -- 172.21.15.39:0/2981131343 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92640a58b0 con 0x7f926c0c5870 2024-09-13T17:03:37.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9270bde640 1 -- 172.21.15.39:0/2981131343 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92640a5c40 con 0x7f926c0c5870 2024-09-13T17:03:37.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9273e44640 1 -- 172.21.15.39:0/2981131343 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c0c5870 msgr2=0x7f926c15e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9273e44640 1 --2- 172.21.15.39:0/2981131343 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c0c5870 0x7f926c15e7b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9264009e50 tx=0x7f926401c9a0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9273e44640 1 -- 172.21.15.39:0/2981131343 shutdown_connections 2024-09-13T17:03:37.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9273e44640 1 --2- 172.21.15.39:0/2981131343 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c0c5870 0x7f926c15e7b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9273e44640 1 --2- 172.21.15.39:0/2981131343 >> v2:172.21.15.84:3300/0 conn(0x7f926c154bb0 0x7f926c0c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9273e44640 1 -- 172.21.15.39:0/2981131343 >> 172.21.15.39:0/2981131343 conn(0x7f926c0a9720 msgr2=0x7f926c0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:37.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9273e44640 1 -- 172.21.15.39:0/2981131343 shutdown_connections 2024-09-13T17:03:37.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.471+0000 7f9273e44640 1 -- 172.21.15.39:0/2981131343 wait complete. 2024-09-13T17:03:37.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 Processor -- start 2024-09-13T17:03:37.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 -- start start 2024-09-13T17:03:37.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c0c2370 0x7f926c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c0c5870 0x7f926c1439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c154bb0 0x7f926c143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c14eac0 con 0x7f926c0c5870 2024-09-13T17:03:37.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c144460 con 0x7f926c0c2370 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c1500a0 con 0x7f926c154bb0 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92713df640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c0c5870 0x7f926c1439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92713df640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c0c5870 0x7f926c1439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38856/0 (socket says 172.21.15.39:38856) 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92713df640 1 -- 172.21.15.39:0/3578590689 learned_addr learned my addr 172.21.15.39:0/3578590689 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92723e1640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c154bb0 0x7f926c143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9271be0640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c0c2370 0x7f926c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92713df640 1 -- 172.21.15.39:0/3578590689 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c154bb0 msgr2=0x7f926c143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92713df640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c154bb0 0x7f926c143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92713df640 1 -- 172.21.15.39:0/3578590689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c0c2370 msgr2=0x7f926c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92713df640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c0c2370 0x7f926c06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92713df640 1 -- 172.21.15.39:0/3578590689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c009420 con 0x7f926c0c5870 2024-09-13T17:03:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9271be0640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c0c2370 0x7f926c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92713df640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c0c5870 0x7f926c1439e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f925c00e9e0 tx=0x7f925c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92627fc640 1 -- 172.21.15.39:0/3578590689 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00c9d0 con 0x7f926c0c5870 2024-09-13T17:03:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92627fc640 1 -- 172.21.15.39:0/3578590689 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c00cb30 con 0x7f926c0c5870 2024-09-13T17:03:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9264009b30 con 0x7f926c0c5870 2024-09-13T17:03:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f92627fc640 1 -- 172.21.15.39:0/3578590689 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00cee0 con 0x7f926c0c5870 2024-09-13T17:03:37.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c15e7f0 con 0x7f926c0c5870 2024-09-13T17:03:37.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.475+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c0c2750 con 0x7f926c0c5870 2024-09-13T17:03:37.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.479+0000 7f92627fc640 1 -- 172.21.15.39:0/3578590689 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f925c013470 con 0x7f926c0c5870 2024-09-13T17:03:37.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.479+0000 7f92627fc640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f92500666c0 0x7f9250068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.479+0000 7f92627fc640 1 -- 172.21.15.39:0/3578590689 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7f925c084910 con 0x7f926c0c5870 2024-09-13T17:03:37.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.479+0000 7f9271be0640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f92500666c0 0x7f9250068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.479+0000 7f9271be0640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f92500666c0 0x7f9250068b80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9268000f70 tx=0x7f9268009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:37.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.483+0000 7f92627fc640 1 -- 172.21.15.39:0/3578590689 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f925c051370 con 0x7f926c0c5870 2024-09-13T17:03:37.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f926c06be00 con 0x7f926c0c5870 2024-09-13T17:03:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f92627fc640 1 -- 172.21.15.39:0/3578590689 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f925c054990 con 0x7f926c0c5870 2024-09-13T17:03:37.686 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:37.686 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:23.025135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:37.686 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 106 2024-09-13T17:03:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f92500666c0 msgr2=0x7f9250068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f92500666c0 0x7f9250068b80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9268000f70 tx=0x7f9268009040 comp rx=0 tx=0).stop 2024-09-13T17:03:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c0c5870 msgr2=0x7f926c1439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c0c5870 0x7f926c1439e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f925c00e9e0 tx=0x7f925c00eea0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 shutdown_connections 2024-09-13T17:03:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f92500666c0 0x7f9250068b80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f926c154bb0 0x7f926c143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f926c0c5870 0x7f926c1439e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 --2- 172.21.15.39:0/3578590689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f926c0c2370 0x7f926c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 >> 172.21.15.39:0/3578590689 conn(0x7f926c0a9720 msgr2=0x7f926c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 shutdown_connections 2024-09-13T17:03:37.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.687+0000 7f9273e44640 1 -- 172.21.15.39:0/3578590689 wait complete. 2024-09-13T17:03:37.707 INFO:tasks.mds_thrash.fs.[b]:kill mds.c (rank=0) 2024-09-13T17:03:37.708 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:03:37.708 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:03:37.719 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:03:37.719 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:37.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c8fd7640 1 Processor -- start 2024-09-13T17:03:37.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c8fd7640 1 -- start start 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c8fd7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29c414f1b0 0x7f29c4151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c8fd7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c40c7470 0x7f29c40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c8fd7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c414f590 con 0x7f29c406e5e0 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c8fd7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c406f660 con 0x7f29c414f1b0 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c8fd7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c406eae0 con 0x7f29c40c7470 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29c414f1b0 0x7f29c4151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29c414f1b0 0x7f29c4151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38864/0 (socket says 172.21.15.39:38864) 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c259c640 1 -- 172.21.15.39:0/1077924169 learned_addr learned my addr 172.21.15.39:0/1077924169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.875+0000 7f29c2d9d640 1 --2- 172.21.15.39:0/1077924169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c40c7470 0x7f29c40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c259c640 1 -- 172.21.15.39:0/1077924169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c40c7470 msgr2=0x7f29c40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c259c640 1 --2- 172.21.15.39:0/1077924169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c40c7470 0x7f29c40c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c259c640 1 -- 172.21.15.39:0/1077924169 >> v1:172.21.15.84:6789/0 conn(0x7f29c406e5e0 legacy=0x7f29c40714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c259c640 1 -- 172.21.15.39:0/1077924169 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c406edc0 con 0x7f29c414f1b0 2024-09-13T17:03:37.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c259c640 1 --2- 172.21.15.39:0/1077924169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 0x7f29c4151bb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009870 tx=0x7f29b401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893be6015207233 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:37.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c1d9b640 1 -- 172.21.15.39:0/1077924169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c40c69a0 con 0x7f29c414f1b0 2024-09-13T17:03:37.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c1d9b640 1 -- 172.21.15.39:0/1077924169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b40a95f0 con 0x7f29c414f1b0 2024-09-13T17:03:37.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c1d9b640 1 -- 172.21.15.39:0/1077924169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40a9980 con 0x7f29c414f1b0 2024-09-13T17:03:37.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c8fd7640 1 -- 172.21.15.39:0/1077924169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 msgr2=0x7f29c4151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c8fd7640 1 --2- 172.21.15.39:0/1077924169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 0x7f29c4151bb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009870 tx=0x7f29b401c530 comp rx=0 tx=0).stop 2024-09-13T17:03:37.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c8fd7640 1 -- 172.21.15.39:0/1077924169 shutdown_connections 2024-09-13T17:03:37.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c8fd7640 1 --2- 172.21.15.39:0/1077924169 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c40c7470 0x7f29c40c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c8fd7640 1 --2- 172.21.15.39:0/1077924169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 0x7f29c4151bb0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c8fd7640 1 -- 172.21.15.39:0/1077924169 >> 172.21.15.39:0/1077924169 conn(0x7f29c40a9720 msgr2=0x7f29c40a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:37.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c8fd7640 1 -- 172.21.15.39:0/1077924169 shutdown_connections 2024-09-13T17:03:37.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.879+0000 7f29c8fd7640 1 -- 172.21.15.39:0/1077924169 wait complete. 2024-09-13T17:03:37.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 Processor -- start 2024-09-13T17:03:37.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 -- start start 2024-09-13T17:03:37.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c406e5e0 0x7f29c41ecb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29c40c7470 0x7f29c415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 0x7f29c415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c41ed040 con 0x7f29c414f1b0 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40ac8d0 con 0x7f29c40c7470 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40ad0e0 con 0x7f29c406e5e0 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 0x7f29c415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 0x7f29c415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38866/0 (socket says 172.21.15.39:38866) 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c2d9d640 1 -- 172.21.15.39:0/709240501 learned_addr learned my addr 172.21.15.39:0/709240501 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29bbfff640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29c40c7470 0x7f29c415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c259c640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c406e5e0 0x7f29c41ecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c2d9d640 1 -- 172.21.15.39:0/709240501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c406e5e0 msgr2=0x7f29c41ecb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c2d9d640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c406e5e0 0x7f29c41ecb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c2d9d640 1 -- 172.21.15.39:0/709240501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29c40c7470 msgr2=0x7f29c415a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:37.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c2d9d640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29c40c7470 0x7f29c415a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:37.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c2d9d640 1 -- 172.21.15.39:0/709240501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b40094d0 con 0x7f29c414f1b0 2024-09-13T17:03:37.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c2d9d640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 0x7f29c415ab40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f29bc007e40 tx=0x7f29bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:37.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29baffd640 1 -- 172.21.15.39:0/709240501 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc014620 con 0x7f29c414f1b0 2024-09-13T17:03:37.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29baffd640 1 -- 172.21.15.39:0/709240501 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc00ee90 con 0x7f29c414f1b0 2024-09-13T17:03:37.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c40c16b0 con 0x7f29c414f1b0 2024-09-13T17:03:37.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29baffd640 1 -- 172.21.15.39:0/709240501 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0149e0 con 0x7f29c414f1b0 2024-09-13T17:03:37.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c40acb20 con 0x7f29c414f1b0 2024-09-13T17:03:37.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c40ab430 con 0x7f29c414f1b0 2024-09-13T17:03:37.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.883+0000 7f29baffd640 1 -- 172.21.15.39:0/709240501 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29bc014b40 con 0x7f29c414f1b0 2024-09-13T17:03:37.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.887+0000 7f29baffd640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29a4066690 0x7f29a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:37.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.887+0000 7f29baffd640 1 -- 172.21.15.39:0/709240501 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7f29bc085460 con 0x7f29c414f1b0 2024-09-13T17:03:37.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.887+0000 7f29c259c640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29a4066690 0x7f29a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:37.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.887+0000 7f29c259c640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29a4066690 0x7f29a4068b50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f29b40ae000 tx=0x7f29b40b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:37.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:37.891+0000 7f29baffd640 1 -- 172.21.15.39:0/709240501 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f29c40ab430 con 0x7f29c414f1b0 2024-09-13T17:03:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.115+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29c40ab430 con 0x7f29c414f1b0 2024-09-13T17:03:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29baffd640 1 -- 172.21.15.39:0/709240501 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f29c40ab430 con 0x7f29c414f1b0 2024-09-13T17:03:38.118 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:38.118 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:23.025135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:38.118 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 106 2024-09-13T17:03:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29a4066690 msgr2=0x7f29a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29a4066690 0x7f29a4068b50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f29b40ae000 tx=0x7f29b40b3040 comp rx=0 tx=0).stop 2024-09-13T17:03:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 msgr2=0x7f29c415ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 0x7f29c415ab40 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f29bc007e40 tx=0x7f29bc00e4b0 comp rx=0 tx=0).stop 2024-09-13T17:03:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 shutdown_connections 2024-09-13T17:03:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29a4066690 0x7f29a4068b50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29c40c7470 0x7f29c415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29c414f1b0 0x7f29c415ab40 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 --2- 172.21.15.39:0/709240501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29c406e5e0 0x7f29c41ecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 >> 172.21.15.39:0/709240501 conn(0x7f29c40a9720 msgr2=0x7f29c406a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 shutdown_connections 2024-09-13T17:03:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:38.119+0000 7f29c8fd7640 1 -- 172.21.15.39:0/709240501 wait complete. 2024-09-13T17:03:38.137 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:03:40.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:40.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f6b32640 1 Processor -- start 2024-09-13T17:03:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f6b32640 1 -- start start 2024-09-13T17:03:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f6b32640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f014bd50 0x7ff8f00611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f6b32640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8f01551b0 0x7ff8f0155660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f6b32640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f006d5e0 con 0x7ff8f006fd50 2024-09-13T17:03:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f6b32640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f014c130 con 0x7ff8f01551b0 2024-09-13T17:03:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f6b32640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f006d3f0 con 0x7ff8f014bd50 2024-09-13T17:03:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f50cf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8f01551b0 0x7ff8f0155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f50cf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8f01551b0 0x7ff8f0155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57378/0 (socket says 172.21.15.39:57378) 2024-09-13T17:03:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f50cf640 1 -- 172.21.15.39:0/133971652 learned_addr learned my addr 172.21.15.39:0/133971652 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f48ce640 1 --2- 172.21.15.39:0/133971652 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f014bd50 0x7ff8f00611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:40.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f50cf640 1 -- 172.21.15.39:0/133971652 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f014bd50 msgr2=0x7ff8f00611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:40.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f50cf640 1 --2- 172.21.15.39:0/133971652 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f014bd50 0x7ff8f00611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f50cf640 1 -- 172.21.15.39:0/133971652 >> v1:172.21.15.84:6789/0 conn(0x7ff8f006fd50 legacy=0x7ff8f00701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:40.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.279+0000 7ff8f50cf640 1 -- 172.21.15.39:0/133971652 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f006d880 con 0x7ff8f01551b0 2024-09-13T17:03:40.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8f50cf640 1 --2- 172.21.15.39:0/133971652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 0x7ff8f0155660 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009e50 tx=0x7ff8e801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c91d87303ec1b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8ef7fe640 1 -- 172.21.15.39:0/133971652 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80a5660 con 0x7ff8f01551b0 2024-09-13T17:03:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8ef7fe640 1 -- 172.21.15.39:0/133971652 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e80a57c0 con 0x7ff8f01551b0 2024-09-13T17:03:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8ef7fe640 1 -- 172.21.15.39:0/133971652 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80a5b50 con 0x7ff8f01551b0 2024-09-13T17:03:40.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8f6b32640 1 -- 172.21.15.39:0/133971652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 msgr2=0x7ff8f0155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:40.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8f6b32640 1 --2- 172.21.15.39:0/133971652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 0x7ff8f0155660 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009e50 tx=0x7ff8e801c9a0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8f6b32640 1 -- 172.21.15.39:0/133971652 shutdown_connections 2024-09-13T17:03:40.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8f6b32640 1 --2- 172.21.15.39:0/133971652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 0x7ff8f0155660 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8f6b32640 1 --2- 172.21.15.39:0/133971652 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f014bd50 0x7ff8f00611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8f6b32640 1 -- 172.21.15.39:0/133971652 >> 172.21.15.39:0/133971652 conn(0x7ff8f0069e10 msgr2=0x7ff8f006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:40.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8f6b32640 1 -- 172.21.15.39:0/133971652 shutdown_connections 2024-09-13T17:03:40.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.283+0000 7ff8f6b32640 1 -- 172.21.15.39:0/133971652 wait complete. 2024-09-13T17:03:40.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 Processor -- start 2024-09-13T17:03:40.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 -- start start 2024-09-13T17:03:40.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f006fd50 0x7ff8f01462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:40.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f014bd50 0x7ff8f01ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:40.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 0x7ff8f00c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:40.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00aa820 con 0x7ff8f01551b0 2024-09-13T17:03:40.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01ec820 con 0x7ff8f014bd50 2024-09-13T17:03:40.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00ab2d0 con 0x7ff8f006fd50 2024-09-13T17:03:40.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f50cf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 0x7ff8f00c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:40.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f50cf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 0x7ff8f00c4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57380/0 (socket says 172.21.15.39:57380) 2024-09-13T17:03:40.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f50cf640 1 -- 172.21.15.39:0/847863658 learned_addr learned my addr 172.21.15.39:0/847863658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:40.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f48ce640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f006fd50 0x7ff8f01462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:40.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8effff640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f014bd50 0x7ff8f01ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:40.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f50cf640 1 -- 172.21.15.39:0/847863658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f006fd50 msgr2=0x7ff8f01462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:40.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f50cf640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f006fd50 0x7ff8f01462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f50cf640 1 -- 172.21.15.39:0/847863658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f014bd50 msgr2=0x7ff8f01ecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:40.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f50cf640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f014bd50 0x7ff8f01ecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f50cf640 1 -- 172.21.15.39:0/847863658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8009b30 con 0x7ff8f01551b0 2024-09-13T17:03:40.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f50cf640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 0x7ff8f00c4430 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80af040 tx=0x7ff8e8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:40.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8ed7fa640 1 -- 172.21.15.39:0/847863658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80b8670 con 0x7ff8f01551b0 2024-09-13T17:03:40.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8ed7fa640 1 -- 172.21.15.39:0/847863658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e80b87d0 con 0x7ff8f01551b0 2024-09-13T17:03:40.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f01eca30 con 0x7ff8f01551b0 2024-09-13T17:03:40.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8ed7fa640 1 -- 172.21.15.39:0/847863658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80b8b80 con 0x7ff8f01551b0 2024-09-13T17:03:40.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f00aad10 con 0x7ff8f01551b0 2024-09-13T17:03:40.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.287+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f00611c0 con 0x7ff8f01551b0 2024-09-13T17:03:40.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.291+0000 7ff8ed7fa640 1 -- 172.21.15.39:0/847863658 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8e80b8ce0 con 0x7ff8f01551b0 2024-09-13T17:03:40.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.291+0000 7ff8ed7fa640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8d4066690 0x7ff8d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:40.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.291+0000 7ff8ed7fa640 1 -- 172.21.15.39:0/847863658 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7ff8e8120000 con 0x7ff8f01551b0 2024-09-13T17:03:40.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.291+0000 7ff8f48ce640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8d4066690 0x7ff8d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:40.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.291+0000 7ff8f48ce640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8d4066690 0x7ff8d4068b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009a50 tx=0x7ff8e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:40.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.295+0000 7ff8ed7fa640 1 -- 172.21.15.39:0/847863658 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff8e80ec9d0 con 0x7ff8f01551b0 2024-09-13T17:03:40.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.595+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8f01ecbc0 con 0x7ff8f01551b0 2024-09-13T17:03:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.595+0000 7ff8ed7fa640 1 -- 172.21.15.39:0/847863658 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+5932 (secure 0 0 0) 0x7ff8e8009510 con 0x7ff8f01551b0 2024-09-13T17:03:40.597 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:40.597 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:23.025135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 106 2024-09-13T17:03:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8d4066690 msgr2=0x7ff8d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8d4066690 0x7ff8d4068b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009a50 tx=0x7ff8e4008040 comp rx=0 tx=0).stop 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 msgr2=0x7ff8f00c4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 0x7ff8f00c4430 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80af040 tx=0x7ff8e8019740 comp rx=0 tx=0).stop 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 shutdown_connections 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8d4066690 0x7ff8d4068b50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8f014bd50 0x7ff8f01ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8f01551b0 0x7ff8f00c4430 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 --2- 172.21.15.39:0/847863658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8f006fd50 0x7ff8f01462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 >> 172.21.15.39:0/847863658 conn(0x7ff8f0069e10 msgr2=0x7ff8f0153be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 shutdown_connections 2024-09-13T17:03:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:40.599+0000 7ff8f6b32640 1 -- 172.21.15.39:0/847863658 wait complete. 2024-09-13T17:03:40.618 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:03:42.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:42.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc219176640 1 Processor -- start 2024-09-13T17:03:42.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc219176640 1 -- start start 2024-09-13T17:03:42.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc219176640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc2140c7610 0x7fc214151b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc219176640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc2140a72b0 0x7fc214071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc219176640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21406feb0 con 0x7fc2141556d0 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc219176640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2140aae50 con 0x7fc2140c7610 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc219176640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc214155ab0 con 0x7fc2140a72b0 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc21259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc2140c7610 0x7fc214151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc212d9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc2141556d0 0x7fc2140c36e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:47578/0 (socket says 172.21.15.39:47578) 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc212d9d640 1 -- 172.21.15.39:0/1603923379 learned_addr learned my addr 172.21.15.39:0/1603923379 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc21259c640 1 -- 172.21.15.39:0/1603923379 >> v2:172.21.15.84:3300/0 conn(0x7fc2140a72b0 msgr2=0x7fc214071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc21259c640 1 --2- 172.21.15.39:0/1603923379 >> v2:172.21.15.84:3300/0 conn(0x7fc2140a72b0 0x7fc214071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc21259c640 1 -- 172.21.15.39:0/1603923379 >> v1:172.21.15.39:6789/0 conn(0x7fc2141556d0 legacy=0x7fc2140c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc21259c640 1 -- 172.21.15.39:0/1603923379 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2140c6c30 con 0x7fc2140c7610 2024-09-13T17:03:42.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.767+0000 7fc211d9b640 1 -- 172.21.15.39:0/1603923379 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058460492 0 0) 0x7fc21406feb0 con 0x7fc2141556d0 2024-09-13T17:03:42.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc21259c640 1 --2- 172.21.15.39:0/1603923379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2140c7610 0x7fc214151b80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc2140714c0 tx=0x7fc20801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eef9419ea0b7515 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:42.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc211d9b640 1 -- 172.21.15.39:0/1603923379 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080ab600 con 0x7fc2140c7610 2024-09-13T17:03:42.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc211d9b640 1 -- 172.21.15.39:0/1603923379 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080a7050 con 0x7fc2140c7610 2024-09-13T17:03:42.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc211d9b640 1 -- 172.21.15.39:0/1603923379 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080ab980 con 0x7fc2140c7610 2024-09-13T17:03:42.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc219176640 1 -- 172.21.15.39:0/1603923379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2140c7610 msgr2=0x7fc214151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:42.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc219176640 1 --2- 172.21.15.39:0/1603923379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2140c7610 0x7fc214151b80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fc2140714c0 tx=0x7fc20801c330 comp rx=0 tx=0).stop 2024-09-13T17:03:42.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc219176640 1 -- 172.21.15.39:0/1603923379 shutdown_connections 2024-09-13T17:03:42.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc219176640 1 --2- 172.21.15.39:0/1603923379 >> v2:172.21.15.84:3300/0 conn(0x7fc2140a72b0 0x7fc214071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:42.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc219176640 1 --2- 172.21.15.39:0/1603923379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2140c7610 0x7fc214151b80 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:42.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc219176640 1 -- 172.21.15.39:0/1603923379 >> 172.21.15.39:0/1603923379 conn(0x7fc2140a97b0 msgr2=0x7fc2140a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:42.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc219176640 1 -- 172.21.15.39:0/1603923379 shutdown_connections 2024-09-13T17:03:42.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.771+0000 7fc219176640 1 -- 172.21.15.39:0/1603923379 wait complete. 2024-09-13T17:03:42.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 Processor -- start 2024-09-13T17:03:42.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 -- start start 2024-09-13T17:03:42.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140a72b0 0x7fc21406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:42.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2141556d0 0x7fc21414a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:42.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc214151670 0x7fc214148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:42.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140c4620 con 0x7fc2141556d0 2024-09-13T17:03:42.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21415ae60 con 0x7fc2140a72b0 2024-09-13T17:03:42.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21406bdf0 con 0x7fc214151670 2024-09-13T17:03:42.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2141556d0 0x7fc21414a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:42.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2141556d0 0x7fc21414a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57404/0 (socket says 172.21.15.39:57404) 2024-09-13T17:03:42.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21259c640 1 -- 172.21.15.39:0/2323286637 learned_addr learned my addr 172.21.15.39:0/2323286637 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc212d9d640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140a72b0 0x7fc21406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21359e640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc214151670 0x7fc214148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21259c640 1 -- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc214151670 msgr2=0x7fc214148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21259c640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc214151670 0x7fc214148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21259c640 1 -- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140a72b0 msgr2=0x7fc21406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21259c640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140a72b0 0x7fc21406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21259c640 1 -- 172.21.15.39:0/2323286637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc208009420 con 0x7fc2141556d0 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21359e640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc214151670 0x7fc214148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc212d9d640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140a72b0 0x7fc21406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc21259c640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2141556d0 0x7fc21414a190 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc208018f40 tx=0x7fc20801ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc1fb7fe640 1 -- 172.21.15.39:0/2323286637 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b7650 con 0x7fc2141556d0 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc1fb7fe640 1 -- 172.21.15.39:0/2323286637 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080b77b0 con 0x7fc2141556d0 2024-09-13T17:03:42.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21415b280 con 0x7fc2141556d0 2024-09-13T17:03:42.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc1fb7fe640 1 -- 172.21.15.39:0/2323286637 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b7b40 con 0x7fc2141556d0 2024-09-13T17:03:42.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.775+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21406b210 con 0x7fc2141556d0 2024-09-13T17:03:42.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.779+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2140c2170 con 0x7fc2141556d0 2024-09-13T17:03:42.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.779+0000 7fc1fb7fe640 1 -- 172.21.15.39:0/2323286637 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2080b7ca0 con 0x7fc2141556d0 2024-09-13T17:03:42.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.779+0000 7fc1fb7fe640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2000666c0 0x7fc200068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:42.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.779+0000 7fc1fb7fe640 1 -- 172.21.15.39:0/2323286637 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7fc20811f5d0 con 0x7fc2141556d0 2024-09-13T17:03:42.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.779+0000 7fc212d9d640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2000666c0 0x7fc200068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:42.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.779+0000 7fc212d9d640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2000666c0 0x7fc200068b80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc204007fd0 tx=0x7fc204009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:42.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:42.783+0000 7fc1fb7fe640 1 -- 172.21.15.39:0/2323286637 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc2080ec0a0 con 0x7fc2141556d0 2024-09-13T17:03:43.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.063+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc21415afa0 con 0x7fc2141556d0 2024-09-13T17:03:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc1fb7fe640 1 -- 172.21.15.39:0/2323286637 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+5932 (secure 0 0 0) 0x7fc21415afa0 con 0x7fc2141556d0 2024-09-13T17:03:43.066 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:43.066 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:23.025135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:43.066 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 106 2024-09-13T17:03:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2000666c0 msgr2=0x7fc200068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:43.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2000666c0 0x7fc200068b80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc204007fd0 tx=0x7fc204009040 comp rx=0 tx=0).stop 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2141556d0 msgr2=0x7fc21414a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2141556d0 0x7fc21414a190 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc208018f40 tx=0x7fc20801ab70 comp rx=0 tx=0).stop 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 shutdown_connections 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2000666c0 0x7fc200068b80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc214151670 0x7fc214148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2141556d0 0x7fc21414a190 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 --2- 172.21.15.39:0/2323286637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2140a72b0 0x7fc21406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 >> 172.21.15.39:0/2323286637 conn(0x7fc2140a97b0 msgr2=0x7fc2140abb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 shutdown_connections 2024-09-13T17:03:43.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:43.067+0000 7fc219176640 1 -- 172.21.15.39:0/2323286637 wait complete. 2024-09-13T17:03:43.092 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:03:45.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:45.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f199197f640 1 Processor -- start 2024-09-13T17:03:45.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f199197f640 1 -- start start 2024-09-13T17:03:45.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f199197f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f198c0c21d0 0x7f198c0aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:45.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f199197f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f198c14b850 0x7f198c14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:45.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f199197f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c0c3ae0 con 0x7f198c06aac0 2024-09-13T17:03:45.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f199197f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c0ab340 con 0x7f198c0c21d0 2024-09-13T17:03:45.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f199197f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c0abb30 con 0x7f198c14b850 2024-09-13T17:03:45.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f198affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f198c0c21d0 0x7f198c0aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:45.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f198affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f198c0c21d0 0x7f198c0aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57408/0 (socket says 172.21.15.39:57408) 2024-09-13T17:03:45.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f198affd640 1 -- 172.21.15.39:0/725252609 learned_addr learned my addr 172.21.15.39:0/725252609 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:45.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f198b7fe640 1 --2- 172.21.15.39:0/725252609 >> v2:172.21.15.84:3300/0 conn(0x7f198c14b850 0x7f198c14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:45.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f198affd640 1 -- 172.21.15.39:0/725252609 >> v2:172.21.15.84:3300/0 conn(0x7f198c14b850 msgr2=0x7f198c14bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:45.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f198affd640 1 --2- 172.21.15.39:0/725252609 >> v2:172.21.15.84:3300/0 conn(0x7f198c14b850 0x7f198c14bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f198affd640 1 -- 172.21.15.39:0/725252609 >> v1:172.21.15.39:6789/0 conn(0x7f198c06aac0 legacy=0x7f198c151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:45.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f198affd640 1 -- 172.21.15.39:0/725252609 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c0abe10 con 0x7f198c0c21d0 2024-09-13T17:03:45.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f1989ffb640 1 -- 172.21.15.39:0/725252609 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238484511 0 0) 0x7f198c0c3ae0 con 0x7f198c06aac0 2024-09-13T17:03:45.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.247+0000 7f198b7fe640 1 --2- 172.21.15.39:0/725252609 >> v2:172.21.15.84:3300/0 conn(0x7f198c14b850 0x7f198c14bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:03:45.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f198affd640 1 --2- 172.21.15.39:0/725252609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 0x7f198c0aae00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f198c06de50 tx=0x7f197c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ba76c6faafdf38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:45.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f1989ffb640 1 -- 172.21.15.39:0/725252609 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0ab660 con 0x7f198c0c21d0 2024-09-13T17:03:45.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f1989ffb640 1 -- 172.21.15.39:0/725252609 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c0ab7c0 con 0x7f198c0c21d0 2024-09-13T17:03:45.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f1989ffb640 1 -- 172.21.15.39:0/725252609 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0abb50 con 0x7f198c0c21d0 2024-09-13T17:03:45.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f199197f640 1 -- 172.21.15.39:0/725252609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 msgr2=0x7f198c0aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:45.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f199197f640 1 --2- 172.21.15.39:0/725252609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 0x7f198c0aae00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f198c06de50 tx=0x7f197c01c460 comp rx=0 tx=0).stop 2024-09-13T17:03:45.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f199197f640 1 -- 172.21.15.39:0/725252609 shutdown_connections 2024-09-13T17:03:45.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f199197f640 1 --2- 172.21.15.39:0/725252609 >> v2:172.21.15.84:3300/0 conn(0x7f198c14b850 0x7f198c14bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f199197f640 1 --2- 172.21.15.39:0/725252609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 0x7f198c0aae00 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f199197f640 1 -- 172.21.15.39:0/725252609 >> 172.21.15.39:0/725252609 conn(0x7f198c069e10 msgr2=0x7f198c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:45.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f199197f640 1 -- 172.21.15.39:0/725252609 shutdown_connections 2024-09-13T17:03:45.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.251+0000 7f199197f640 1 -- 172.21.15.39:0/725252609 wait complete. 2024-09-13T17:03:45.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 Processor -- start 2024-09-13T17:03:45.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 -- start start 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f198c06aac0 0x7f198c155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 0x7f198c1504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f198c14b850 0x7f198c150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c142830 con 0x7f198c0c21d0 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c153fc0 con 0x7f198c14b850 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1560d0 con 0x7f198c06aac0 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 0x7f198c1504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 0x7f198c1504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57416/0 (socket says 172.21.15.39:57416) 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198a7fc640 1 -- 172.21.15.39:0/1160990080 learned_addr learned my addr 172.21.15.39:0/1160990080 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:45.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198affd640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f198c06aac0 0x7f198c155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198b7fe640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f198c14b850 0x7f198c150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198a7fc640 1 -- 172.21.15.39:0/1160990080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f198c06aac0 msgr2=0x7f198c155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198a7fc640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f198c06aac0 0x7f198c155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198a7fc640 1 -- 172.21.15.39:0/1160990080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f198c14b850 msgr2=0x7f198c150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198a7fc640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f198c14b850 0x7f198c150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198a7fc640 1 -- 172.21.15.39:0/1160990080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0094d0 con 0x7f198c0c21d0 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f198a7fc640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 0x7f198c1504b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1980001ec0 tx=0x7f198000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f1973fff640 1 -- 172.21.15.39:0/1160990080 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800054e0 con 0x7f198c0c21d0 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f1973fff640 1 -- 172.21.15.39:0/1160990080 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980003600 con 0x7f198c0c21d0 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c155df0 con 0x7f198c0c21d0 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f1973fff640 1 -- 172.21.15.39:0/1160990080 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800039b0 con 0x7f198c0c21d0 2024-09-13T17:03:45.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c1eab50 con 0x7f198c0c21d0 2024-09-13T17:03:45.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.255+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c1ea6d0 con 0x7f198c0c21d0 2024-09-13T17:03:45.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.259+0000 7f1973fff640 1 -- 172.21.15.39:0/1160990080 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1980003b10 con 0x7f198c0c21d0 2024-09-13T17:03:45.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.259+0000 7f1973fff640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1978066690 0x7f1978068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:45.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.259+0000 7f1973fff640 1 -- 172.21.15.39:0/1160990080 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7f1980084150 con 0x7f198c0c21d0 2024-09-13T17:03:45.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.259+0000 7f198affd640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1978066690 0x7f1978068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:45.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.259+0000 7f198affd640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1978066690 0x7f1978068b50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f197c0af000 tx=0x7f197c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:45.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.263+0000 7f1973fff640 1 -- 172.21.15.39:0/1160990080 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f198c1ea6d0 con 0x7f198c0c21d0 2024-09-13T17:03:45.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f198c1ea6d0 con 0x7f198c0c21d0 2024-09-13T17:03:45.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f1973fff640 1 -- 172.21.15.39:0/1160990080 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f198c1ea6d0 con 0x7f198c0c21d0 2024-09-13T17:03:45.503 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:45.503 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:23.025135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:45.503 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 106 2024-09-13T17:03:45.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1978066690 msgr2=0x7f1978068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:45.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1978066690 0x7f1978068b50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f197c0af000 tx=0x7f197c0a7000 comp rx=0 tx=0).stop 2024-09-13T17:03:45.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 msgr2=0x7f198c1504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:45.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 0x7f198c1504b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1980001ec0 tx=0x7f198000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 shutdown_connections 2024-09-13T17:03:45.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1978066690 0x7f1978068b50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f198c14b850 0x7f198c150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f198c0c21d0 0x7f198c1504b0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 --2- 172.21.15.39:0/1160990080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f198c06aac0 0x7f198c155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:45.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 >> 172.21.15.39:0/1160990080 conn(0x7f198c069e10 msgr2=0x7f198c15b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:45.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 shutdown_connections 2024-09-13T17:03:45.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:45.503+0000 7f199197f640 1 -- 172.21.15.39:0/1160990080 wait complete. 2024-09-13T17:03:45.526 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:03:47.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.675+0000 7fa688f38640 1 Processor -- start 2024-09-13T17:03:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.675+0000 7fa688f38640 1 -- start start 2024-09-13T17:03:47.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa68414f170 0x7fa6840a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:47.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa6840a5090 0x7fa684154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:47.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa684155c90 con 0x7fa6840a5670 2024-09-13T17:03:47.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6840611c0 con 0x7fa6840a5090 2024-09-13T17:03:47.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6840613d0 con 0x7fa68414f170 2024-09-13T17:03:47.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa683fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa6840a5090 0x7fa684154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa683fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa6840a5090 0x7fa684154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57424/0 (socket says 172.21.15.39:57424) 2024-09-13T17:03:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa683fff640 1 -- 172.21.15.39:0/3347065754 learned_addr learned my addr 172.21.15.39:0/3347065754 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa6837fe640 1 --2- 172.21.15.39:0/3347065754 >> v2:172.21.15.84:3300/0 conn(0x7fa68414f170 0x7fa6840a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa683fff640 1 -- 172.21.15.39:0/3347065754 >> v2:172.21.15.84:3300/0 conn(0x7fa68414f170 msgr2=0x7fa6840a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa683fff640 1 --2- 172.21.15.39:0/3347065754 >> v2:172.21.15.84:3300/0 conn(0x7fa68414f170 0x7fa6840a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa683fff640 1 -- 172.21.15.39:0/3347065754 >> v1:172.21.15.84:6789/0 conn(0x7fa6840a5670 legacy=0x7fa684071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:47.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa683fff640 1 -- 172.21.15.39:0/3347065754 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa684072470 con 0x7fa6840a5090 2024-09-13T17:03:47.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa683fff640 1 --2- 172.21.15.39:0/3347065754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6840a5090 0x7fa684154fa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa6840a8af0 tx=0x7fa67c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22fd7451cffe09f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:47.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa682ffd640 1 -- 172.21.15.39:0/3347065754 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0a5660 con 0x7fa6840a5090 2024-09-13T17:03:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa682ffd640 1 -- 172.21.15.39:0/3347065754 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c0a57c0 con 0x7fa6840a5090 2024-09-13T17:03:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa682ffd640 1 -- 172.21.15.39:0/3347065754 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0a5b50 con 0x7fa6840a5090 2024-09-13T17:03:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 -- 172.21.15.39:0/3347065754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6840a5090 msgr2=0x7fa684154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 --2- 172.21.15.39:0/3347065754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6840a5090 0x7fa684154fa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa6840a8af0 tx=0x7fa67c01c890 comp rx=0 tx=0).stop 2024-09-13T17:03:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 -- 172.21.15.39:0/3347065754 shutdown_connections 2024-09-13T17:03:47.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 --2- 172.21.15.39:0/3347065754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6840a5090 0x7fa684154fa0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:47.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 --2- 172.21.15.39:0/3347065754 >> v2:172.21.15.84:3300/0 conn(0x7fa68414f170 0x7fa6840a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:47.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 -- 172.21.15.39:0/3347065754 >> 172.21.15.39:0/3347065754 conn(0x7fa6840a7af0 msgr2=0x7fa6840a7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:47.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 -- 172.21.15.39:0/3347065754 shutdown_connections 2024-09-13T17:03:47.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.679+0000 7fa688f38640 1 -- 172.21.15.39:0/3347065754 wait complete. 2024-09-13T17:03:47.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 Processor -- start 2024-09-13T17:03:47.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 -- start start 2024-09-13T17:03:47.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6840a5090 0x7fa6841ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:47.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa68414f170 0x7fa684146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:47.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa684146d40 0x7fa68414f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68414b5f0 con 0x7fa684146d40 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa684152400 con 0x7fa68414f170 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68414bf70 con 0x7fa6840a5090 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa683fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa684146d40 0x7fa68414f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa683fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa684146d40 0x7fa68414f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57432/0 (socket says 172.21.15.39:57432) 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa683fff640 1 -- 172.21.15.39:0/841530032 learned_addr learned my addr 172.21.15.39:0/841530032 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa6837fe640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6840a5090 0x7fa6841ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa67bfff640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa68414f170 0x7fa684146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa683fff640 1 -- 172.21.15.39:0/841530032 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6840a5090 msgr2=0x7fa6841ee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:47.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa683fff640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6840a5090 0x7fa6841ee3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa683fff640 1 -- 172.21.15.39:0/841530032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa68414f170 msgr2=0x7fa684146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa683fff640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa68414f170 0x7fa684146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa683fff640 1 -- 172.21.15.39:0/841530032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c009b30 con 0x7fa684146d40 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa683fff640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa684146d40 0x7fa68414f880 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa67c01a4c0 tx=0x7fa67c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa680ff9640 1 -- 172.21.15.39:0/841530032 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0a5660 con 0x7fa684146d40 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa680ff9640 1 -- 172.21.15.39:0/841530032 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67c0b88a0 con 0x7fa684146d40 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6841eed40 con 0x7fa684146d40 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa680ff9640 1 -- 172.21.15.39:0/841530032 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c0b8c50 con 0x7fa684146d40 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.683+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68414bc90 con 0x7fa684146d40 2024-09-13T17:03:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.687+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa684070940 con 0x7fa684146d40 2024-09-13T17:03:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.687+0000 7fa680ff9640 1 -- 172.21.15.39:0/841530032 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa67c0b6440 con 0x7fa684146d40 2024-09-13T17:03:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.687+0000 7fa680ff9640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa664066690 0x7fa664068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.687+0000 7fa680ff9640 1 -- 172.21.15.39:0/841530032 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 8271+0+0 (secure 0 0 0) 0x7fa67c120000 con 0x7fa684146d40 2024-09-13T17:03:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.687+0000 7fa6837fe640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa664066690 0x7fa664068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.687+0000 7fa6837fe640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa664066690 0x7fa664068b50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa67400b9d0 tx=0x7fa674008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:47.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.691+0000 7fa680ff9640 1 -- 172.21.15.39:0/841530032 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa67c0ec9d0 con 0x7fa684146d40 2024-09-13T17:03:47.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.979+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa684154fa0 con 0x7fa684146d40 2024-09-13T17:03:47.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa680ff9640 1 -- 172.21.15.39:0/841530032 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+5932 (secure 0 0 0) 0x7fa67c009510 con 0x7fa684146d40 2024-09-13T17:03:47.981 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:47.982 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:23.025135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6711},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6711":{"gid":6711,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6837/3105501456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3105501456},{"type":"v1","addr":"172.21.15.39:6837","nonce":3105501456}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:47.982 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 106 2024-09-13T17:03:47.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa664066690 msgr2=0x7fa664068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:47.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa664066690 0x7fa664068b50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa67400b9d0 tx=0x7fa674008040 comp rx=0 tx=0).stop 2024-09-13T17:03:47.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa684146d40 msgr2=0x7fa68414f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:47.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa684146d40 0x7fa68414f880 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa67c01a4c0 tx=0x7fa67c019630 comp rx=0 tx=0).stop 2024-09-13T17:03:47.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 shutdown_connections 2024-09-13T17:03:47.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa664066690 0x7fa664068b50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:47.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa68414f170 0x7fa684146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:47.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa684146d40 0x7fa68414f880 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:47.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 --2- 172.21.15.39:0/841530032 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6840a5090 0x7fa6841ee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:47.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 >> 172.21.15.39:0/841530032 conn(0x7fa6840a7af0 msgr2=0x7fa684148060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:47.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 shutdown_connections 2024-09-13T17:03:47.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:47.983+0000 7fa688f38640 1 -- 172.21.15.39:0/841530032 wait complete. 2024-09-13T17:03:48.002 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:03:50.004 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:50.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.143+0000 7ffa71054640 1 Processor -- start 2024-09-13T17:03:50.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.143+0000 7ffa71054640 1 -- start start 2024-09-13T17:03:50.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.143+0000 7ffa71054640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffa6c071bb0 0x7ffa6c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.143+0000 7ffa71054640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 0x7ffa6c06d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.143+0000 7ffa71054640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6c150e00 con 0x7ffa6c0ab9e0 2024-09-13T17:03:50.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.143+0000 7ffa71054640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6c0abdc0 con 0x7ffa6c071bb0 2024-09-13T17:03:50.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.143+0000 7ffa71054640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c150250 con 0x7ffa6c0c0ac0 2024-09-13T17:03:50.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.143+0000 7ffa6a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffa6c071bb0 0x7ffa6c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa6a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffa6c071bb0 0x7ffa6c06cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39040/0 (socket says 172.21.15.39:39040) 2024-09-13T17:03:50.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa6a59c640 1 -- 172.21.15.39:0/2224377290 learned_addr learned my addr 172.21.15.39:0/2224377290 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:50.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa6b59e640 1 --2- 172.21.15.39:0/2224377290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 0x7ffa6c06d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa6a59c640 1 -- 172.21.15.39:0/2224377290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 msgr2=0x7ffa6c06d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:50.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa6a59c640 1 --2- 172.21.15.39:0/2224377290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 0x7ffa6c06d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa6a59c640 1 -- 172.21.15.39:0/2224377290 >> v1:172.21.15.84:6789/0 conn(0x7ffa6c0ab9e0 legacy=0x7ffa6c0c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa6a59c640 1 -- 172.21.15.39:0/2224377290 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c14a8f0 con 0x7ffa6c071bb0 2024-09-13T17:03:50.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa6a59c640 1 --2- 172.21.15.39:0/2224377290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c071bb0 0x7ffa6c06cbe0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ffa60006fd0 tx=0x7ffa6001e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc19f13025fbad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:50.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa69d9b640 1 -- 172.21.15.39:0/2224377290 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60004230 con 0x7ffa6c071bb0 2024-09-13T17:03:50.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa69d9b640 1 -- 172.21.15.39:0/2224377290 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa600ab070 con 0x7ffa6c071bb0 2024-09-13T17:03:50.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa69d9b640 1 -- 172.21.15.39:0/2224377290 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600a7730 con 0x7ffa6c071bb0 2024-09-13T17:03:50.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa71054640 1 -- 172.21.15.39:0/2224377290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c071bb0 msgr2=0x7ffa6c06cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa71054640 1 --2- 172.21.15.39:0/2224377290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c071bb0 0x7ffa6c06cbe0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ffa60006fd0 tx=0x7ffa6001e9f0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa71054640 1 -- 172.21.15.39:0/2224377290 shutdown_connections 2024-09-13T17:03:50.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa71054640 1 --2- 172.21.15.39:0/2224377290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 0x7ffa6c06d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa71054640 1 --2- 172.21.15.39:0/2224377290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c071bb0 0x7ffa6c06cbe0 secure :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ffa60006fd0 tx=0x7ffa6001e9f0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa71054640 1 -- 172.21.15.39:0/2224377290 >> 172.21.15.39:0/2224377290 conn(0x7ffa6c069e10 msgr2=0x7ffa6c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:50.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa71054640 1 -- 172.21.15.39:0/2224377290 shutdown_connections 2024-09-13T17:03:50.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.147+0000 7ffa71054640 1 -- 172.21.15.39:0/2224377290 wait complete. 2024-09-13T17:03:50.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 Processor -- start 2024-09-13T17:03:50.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 -- start start 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6c0ab9e0 0x7ffa6c0bf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 0x7ffa6c0aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c0aab90 0x7ffa6c140cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c14a050 con 0x7ffa6c0aab90 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c1496b0 con 0x7ffa6c0ab9e0 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c149990 con 0x7ffa6c0c0ac0 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c0aab90 0x7ffa6c140cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c0aab90 0x7ffa6c140cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39042/0 (socket says 172.21.15.39:39042) 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6b59e640 1 -- 172.21.15.39:0/2969012580 learned_addr learned my addr 172.21.15.39:0/2969012580 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:50.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6ad9d640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6c0ab9e0 0x7ffa6c0bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6a59c640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 0x7ffa6c0aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6b59e640 1 -- 172.21.15.39:0/2969012580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 msgr2=0x7ffa6c0aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6b59e640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 0x7ffa6c0aa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6b59e640 1 -- 172.21.15.39:0/2969012580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6c0ab9e0 msgr2=0x7ffa6c0bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6b59e640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6c0ab9e0 0x7ffa6c0bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6b59e640 1 -- 172.21.15.39:0/2969012580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6000b780 con 0x7ffa6c0aab90 2024-09-13T17:03:50.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa6b59e640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c0aab90 0x7ffa6c140cf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009fd0 tx=0x7ffa6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:50.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa537fe640 1 -- 172.21.15.39:0/2969012580 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64014420 con 0x7ffa6c0aab90 2024-09-13T17:03:50.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa537fe640 1 -- 172.21.15.39:0/2969012580 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64014580 con 0x7ffa6c0aab90 2024-09-13T17:03:50.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c149c70 con 0x7ffa6c0aab90 2024-09-13T17:03:50.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa537fe640 1 -- 172.21.15.39:0/2969012580 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64014930 con 0x7ffa6c0aab90 2024-09-13T17:03:50.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6c0bfc90 con 0x7ffa6c0aab90 2024-09-13T17:03:50.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.151+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6c14a660 con 0x7ffa6c0aab90 2024-09-13T17:03:50.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.155+0000 7ffa537fe640 1 -- 172.21.15.39:0/2969012580 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa64014a90 con 0x7ffa6c0aab90 2024-09-13T17:03:50.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.155+0000 7ffa537fe640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffa58066690 0x7ffa58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.155+0000 7ffa537fe640 1 -- 172.21.15.39:0/2969012580 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7ffa64085380 con 0x7ffa6c0aab90 2024-09-13T17:03:50.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.155+0000 7ffa6ad9d640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffa58066690 0x7ffa58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.155+0000 7ffa6ad9d640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffa58066690 0x7ffa58068b50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c007fd0 tx=0x7ffa5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:50.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.159+0000 7ffa537fe640 1 -- 172.21.15.39:0/2969012580 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffa64051d80 con 0x7ffa6c0aab90 2024-09-13T17:03:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.359+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa6c0c3950 con 0x7ffa6c0aab90 2024-09-13T17:03:50.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.359+0000 7ffa537fe640 1 -- 172.21.15.39:0/2969012580 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5174 (secure 0 0 0) 0x7ffa640553a0 con 0x7ffa6c0aab90 2024-09-13T17:03:50.361 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:50.361 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:49.744244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:50.361 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 107 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffa58066690 msgr2=0x7ffa58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffa58066690 0x7ffa58068b50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c007fd0 tx=0x7ffa5c009040 comp rx=0 tx=0).stop 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c0aab90 msgr2=0x7ffa6c140cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c0aab90 0x7ffa6c140cf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009fd0 tx=0x7ffa6400e4b0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 shutdown_connections 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ffa58066690 0x7ffa58068b50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa6c0aab90 0x7ffa6c140cf0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ffa6c0c0ac0 0x7ffa6c0aa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 --2- 172.21.15.39:0/2969012580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa6c0ab9e0 0x7ffa6c0bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 >> 172.21.15.39:0/2969012580 conn(0x7ffa6c069e10 msgr2=0x7ffa6c0c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 shutdown_connections 2024-09-13T17:03:50.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.363+0000 7ffa71054640 1 -- 172.21.15.39:0/2969012580 wait complete. 2024-09-13T17:03:50.381 INFO:tasks.mds_thrash.fs.[b]:mds.c down, removed from mdsmap 2024-09-13T17:03:50.381 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:03:50.382 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:50.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe744000640 1 Processor -- start 2024-09-13T17:03:50.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe744000640 1 -- start start 2024-09-13T17:03:50.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe744000640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe73c14a770 0x7fe73c150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe744000640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe73c0c6060 0x7fe73c14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe744000640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0c4760 con 0x7fe73c06ae20 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe744000640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c06df80 con 0x7fe73c0c6060 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe744000640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c0c1950 con 0x7fe73c14a770 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe74159b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe73c0c6060 0x7fe73c14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe74159b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe73c0c6060 0x7fe73c14ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39046/0 (socket says 172.21.15.39:39046) 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe74159b640 1 -- 172.21.15.39:0/3840476515 learned_addr learned my addr 172.21.15.39:0/3840476515 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe741d9c640 1 --2- 172.21.15.39:0/3840476515 >> v2:172.21.15.84:3300/0 conn(0x7fe73c14a770 0x7fe73c150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe74159b640 1 -- 172.21.15.39:0/3840476515 >> v2:172.21.15.84:3300/0 conn(0x7fe73c14a770 msgr2=0x7fe73c150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe74159b640 1 --2- 172.21.15.39:0/3840476515 >> v2:172.21.15.84:3300/0 conn(0x7fe73c14a770 0x7fe73c150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe74159b640 1 -- 172.21.15.39:0/3840476515 >> v1:172.21.15.84:6789/0 conn(0x7fe73c06ae20 legacy=0x7fe73c0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.531+0000 7fe74159b640 1 -- 172.21.15.39:0/3840476515 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c151120 con 0x7fe73c0c6060 2024-09-13T17:03:50.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe74159b640 1 --2- 172.21.15.39:0/3840476515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 0x7fe73c14ab50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0c3270 tx=0x7fe72c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96375164baa69bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:50.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe740d9a640 1 -- 172.21.15.39:0/3840476515 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0ab600 con 0x7fe73c0c6060 2024-09-13T17:03:50.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe740d9a640 1 -- 172.21.15.39:0/3840476515 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c0a7050 con 0x7fe73c0c6060 2024-09-13T17:03:50.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe740d9a640 1 -- 172.21.15.39:0/3840476515 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0ab980 con 0x7fe73c0c6060 2024-09-13T17:03:50.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 -- 172.21.15.39:0/3840476515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 msgr2=0x7fe73c14ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 --2- 172.21.15.39:0/3840476515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 0x7fe73c14ab50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0c3270 tx=0x7fe72c01c460 comp rx=0 tx=0).stop 2024-09-13T17:03:50.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 -- 172.21.15.39:0/3840476515 shutdown_connections 2024-09-13T17:03:50.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 --2- 172.21.15.39:0/3840476515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 0x7fe73c14ab50 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 --2- 172.21.15.39:0/3840476515 >> v2:172.21.15.84:3300/0 conn(0x7fe73c14a770 0x7fe73c150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 -- 172.21.15.39:0/3840476515 >> 172.21.15.39:0/3840476515 conn(0x7fe73c0c32a0 msgr2=0x7fe73c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:50.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 -- 172.21.15.39:0/3840476515 shutdown_connections 2024-09-13T17:03:50.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 -- 172.21.15.39:0/3840476515 wait complete. 2024-09-13T17:03:50.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 Processor -- start 2024-09-13T17:03:50.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.535+0000 7fe744000640 1 -- start start 2024-09-13T17:03:50.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe744000640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe73c06ae20 0x7fe73c0bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe744000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 0x7fe73c144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe744000640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe73c14a770 0x7fe73c0bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe744000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0bdc40 con 0x7fe73c0c6060 2024-09-13T17:03:50.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe744000640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c1ea7d0 con 0x7fe73c14a770 2024-09-13T17:03:50.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe744000640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c1ee1d0 con 0x7fe73c06ae20 2024-09-13T17:03:50.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74159b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 0x7fe73c144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74159b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 0x7fe73c144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39056/0 (socket says 172.21.15.39:39056) 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74159b640 1 -- 172.21.15.39:0/3122850930 learned_addr learned my addr 172.21.15.39:0/3122850930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe741d9c640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe73c06ae20 0x7fe73c0bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74259d640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe73c14a770 0x7fe73c0bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74159b640 1 -- 172.21.15.39:0/3122850930 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe73c06ae20 msgr2=0x7fe73c0bcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74159b640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe73c06ae20 0x7fe73c0bcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74159b640 1 -- 172.21.15.39:0/3122850930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe73c14a770 msgr2=0x7fe73c0bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74159b640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe73c14a770 0x7fe73c0bfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74159b640 1 -- 172.21.15.39:0/3122850930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c0094d0 con 0x7fe73c0c6060 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe74159b640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 0x7fe73c144850 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe72c019070 tx=0x7fe72c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe7327fc640 1 -- 172.21.15.39:0/3122850930 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0b37d0 con 0x7fe73c0c6060 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe7327fc640 1 -- 172.21.15.39:0/3122850930 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c0b3930 con 0x7fe73c0c6060 2024-09-13T17:03:50.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe7327fc640 1 -- 172.21.15.39:0/3122850930 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0b3ca0 con 0x7fe73c0c6060 2024-09-13T17:03:50.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c1ee4b0 con 0x7fe73c0c6060 2024-09-13T17:03:50.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c1eea70 con 0x7fe73c0c6060 2024-09-13T17:03:50.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c0619c0 con 0x7fe73c0c6060 2024-09-13T17:03:50.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.539+0000 7fe7327fc640 1 -- 172.21.15.39:0/3122850930 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe72c0b2070 con 0x7fe73c0c6060 2024-09-13T17:03:50.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.543+0000 7fe7327fc640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe720066690 0x7fe720068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:50.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.543+0000 7fe7327fc640 1 -- 172.21.15.39:0/3122850930 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7fe72c11f960 con 0x7fe73c0c6060 2024-09-13T17:03:50.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.543+0000 7fe741d9c640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe720066690 0x7fe720068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:50.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.543+0000 7fe741d9c640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe720066690 0x7fe720068b50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b9d0 tx=0x7fe738008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:50.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.547+0000 7fe7327fc640 1 -- 172.21.15.39:0/3122850930 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe72c0ec330 con 0x7fe73c0c6060 2024-09-13T17:03:50.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.795+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe73c1ea990 con 0x7fe73c0c6060 2024-09-13T17:03:50.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.795+0000 7fe7327fc640 1 -- 172.21.15.39:0/3122850930 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5174 (secure 0 0 0) 0x7fe72c0b0090 con 0x7fe73c0c6060 2024-09-13T17:03:50.795 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:50.795 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:49.744244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:50.796 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 107 2024-09-13T17:03:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe720066690 msgr2=0x7fe720068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe720066690 0x7fe720068b50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe73800b9d0 tx=0x7fe738008040 comp rx=0 tx=0).stop 2024-09-13T17:03:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 msgr2=0x7fe73c144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 0x7fe73c144850 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe72c019070 tx=0x7fe72c019200 comp rx=0 tx=0).stop 2024-09-13T17:03:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 shutdown_connections 2024-09-13T17:03:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe720066690 0x7fe720068b50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe73c14a770 0x7fe73c0bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe73c0c6060 0x7fe73c144850 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 --2- 172.21.15.39:0/3122850930 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe73c06ae20 0x7fe73c0bcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 >> 172.21.15.39:0/3122850930 conn(0x7fe73c0c32a0 msgr2=0x7fe73c0c24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 shutdown_connections 2024-09-13T17:03:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:50.799+0000 7fe744000640 1 -- 172.21.15.39:0/3122850930 wait complete. 2024-09-13T17:03:50.816 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 107, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 100, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6585": { "addr": "172.21.15.39:6835/611669592", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 611669592, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 611669592, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6585, "incarnation": 97, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 43, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:03:03.021063+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6585 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 107, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6235": { "addr": "172.21.15.84:6835/263561599", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 263561599, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 263561599, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6235, "incarnation": 107, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 45, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:03:49.744244+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6235 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/440028459", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 440028459, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 440028459, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 96, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5561, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:03:52.817 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:52.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e10ff4640 1 Processor -- start 2024-09-13T17:03:52.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e10ff4640 1 -- start start 2024-09-13T17:03:52.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e10ff4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0e0c065ee0 0x7f0e0c0662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:52.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e10ff4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c0c2ee0 0x7f0e0c0c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:52.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e10ff4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0c06ddf0 con 0x7f0e0c0c6270 2024-09-13T17:03:52.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e10ff4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e0c0c4710 con 0x7f0e0c065ee0 2024-09-13T17:03:52.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e10ff4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0c49d0 con 0x7f0e0c0c2ee0 2024-09-13T17:03:52.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0ad9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0e0c0c6270 0x7f0e0c0bfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:39702/0 (socket says 172.21.15.39:39702) 2024-09-13T17:03:52.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0ad9d640 1 -- 172.21.15.39:0/3687401611 learned_addr learned my addr 172.21.15.39:0/3687401611 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:52.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0a59c640 1 --2- 172.21.15.39:0/3687401611 >> v2:172.21.15.84:3300/0 conn(0x7f0e0c065ee0 0x7f0e0c0662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:52.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e09d9b640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c0c2ee0 0x7f0e0c0c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:52.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142159096 0 0) 0x7f0e0c06ddf0 con 0x7f0e0c0c6270 2024-09-13T17:03:52.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df4003530 con 0x7f0e0c0c6270 2024-09-13T17:03:52.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 768925954 0 0) 0x7f0df4003530 con 0x7f0e0c0c6270 2024-09-13T17:03:52.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c0c2ee0 msgr2=0x7f0e0c0c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:52.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c0c2ee0 0x7f0e0c0c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:52.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 >> v2:172.21.15.84:3300/0 conn(0x7f0e0c065ee0 msgr2=0x7f0e0c0662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:52.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 --2- 172.21.15.39:0/3687401611 >> v2:172.21.15.84:3300/0 conn(0x7f0e0c065ee0 0x7f0e0c0662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:52.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c154880 con 0x7f0e0c0c6270 2024-09-13T17:03:52.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f0e04001a00 con 0x7f0e0c0c6270 2024-09-13T17:03:52.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 >> v1:172.21.15.39:6789/0 conn(0x7f0e0c0c6270 legacy=0x7f0e0c0bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:52.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4003980 0x7f0df4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c0c2ee0 0x7f0df40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df4004d10 0x7f0df4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0c4710 con 0x7f0df4004d10 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0c49d0 con 0x7f0df4003980 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df400b9c0 con 0x7f0e0c0c2ee0 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0ad9d640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df4004d10 0x7f0df4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0a59c640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4003980 0x7f0df4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e09d9b640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c0c2ee0 0x7f0df40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0ad9d640 1 -- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c0c2ee0 msgr2=0x7f0df40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0ad9d640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c0c2ee0 0x7f0df40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0ad9d640 1 -- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4003980 msgr2=0x7f0df4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0ad9d640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4003980 0x7f0df4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:52.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0ad9d640 1 -- 172.21.15.39:0/3687401611 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df4003530 con 0x7f0df4004d10 2024-09-13T17:03:52.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.971+0000 7f0e0a59c640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4003980 0x7f0df4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e0ad9d640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df4004d10 0x7f0df4005190 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400a830 tx=0x7f0e0401fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91058b6a41f42c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040aa6f0 con 0x7f0df4004d10 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e040aa850 con 0x7f0df4004d10 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e0959a640 1 -- 172.21.15.39:0/3687401611 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040aac00 con 0x7f0df4004d10 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e10ff4640 1 -- 172.21.15.39:0/3687401611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df4004d10 msgr2=0x7f0df4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e10ff4640 1 --2- 172.21.15.39:0/3687401611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0df4004d10 0x7f0df4005190 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400a830 tx=0x7f0e0401fac0 comp rx=0 tx=0).stop 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e0ad9d640 1 -- 172.21.15.39:0/3687401611 reap_dead start 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e10ff4640 1 -- 172.21.15.39:0/3687401611 shutdown_connections 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e10ff4640 1 -- 172.21.15.39:0/3687401611 >> 172.21.15.39:0/3687401611 conn(0x7f0e0c0c3310 msgr2=0x7f0e0c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e10ff4640 1 -- 172.21.15.39:0/3687401611 shutdown_connections 2024-09-13T17:03:52.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.975+0000 7f0e10ff4640 1 -- 172.21.15.39:0/3687401611 wait complete. 2024-09-13T17:03:52.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 Processor -- start 2024-09-13T17:03:52.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 -- start start 2024-09-13T17:03:52.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4004d10 0x7f0e0c0ac670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c155150 0x7f0e0c1e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0c1ec490 0x7f0e0c0c1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c157920 con 0x7f0e0c1ec490 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c06edb0 con 0x7f0df4004d10 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c1e9df0 con 0x7f0e0c155150 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0c1ec490 0x7f0e0c0c1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0ad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0c1ec490 0x7f0e0c0c1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39076/0 (socket says 172.21.15.39:39076) 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0ad9d640 1 -- 172.21.15.39:0/686589800 learned_addr learned my addr 172.21.15.39:0/686589800 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e09d9b640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c155150 0x7f0e0c1e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0a59c640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4004d10 0x7f0e0c0ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0ad9d640 1 -- 172.21.15.39:0/686589800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c155150 msgr2=0x7f0e0c1e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:52.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0ad9d640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c155150 0x7f0e0c1e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0ad9d640 1 -- 172.21.15.39:0/686589800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4004d10 msgr2=0x7f0e0c0ac670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0ad9d640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4004d10 0x7f0e0c0ac670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0ad9d640 1 -- 172.21.15.39:0/686589800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c152380 con 0x7f0e0c1ec490 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e09d9b640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c155150 0x7f0e0c1e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e0ad9d640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0c1ec490 0x7f0e0c0c1920 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0e0401f8f0 tx=0x7f0df4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0dfaffd640 1 -- 172.21.15.39:0/686589800 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040aa6f0 con 0x7f0e0c1ec490 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0dfaffd640 1 -- 172.21.15.39:0/686589800 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e04002000 con 0x7f0e0c1ec490 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c157380 con 0x7f0e0c1ec490 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0dfaffd640 1 -- 172.21.15.39:0/686589800 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040019c0 con 0x7f0e0c1ec490 2024-09-13T17:03:52.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c151fa0 con 0x7f0e0c1ec490 2024-09-13T17:03:52.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.979+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c0acbb0 con 0x7f0e0c1ec490 2024-09-13T17:03:52.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.983+0000 7f0dfaffd640 1 -- 172.21.15.39:0/686589800 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e040be480 con 0x7f0e0c1ec490 2024-09-13T17:03:52.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.983+0000 7f0dfaffd640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0dec0666c0 0x7f0dec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:52.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.983+0000 7f0dfaffd640 1 -- 172.21.15.39:0/686589800 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7f0e040b9070 con 0x7f0e0c1ec490 2024-09-13T17:03:52.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.983+0000 7f0e0a59c640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0dec0666c0 0x7f0dec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:52.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.983+0000 7f0e0a59c640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0dec0666c0 0x7f0dec068b80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009990 tx=0x7f0dfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:52.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:52.987+0000 7f0dfaffd640 1 -- 172.21.15.39:0/686589800 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0e040eefd0 con 0x7f0e0c1ec490 2024-09-13T17:03:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.211+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e0c151e00 con 0x7f0e0c1ec490 2024-09-13T17:03:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.211+0000 7f0dfaffd640 1 -- 172.21.15.39:0/686589800 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5174 (secure 0 0 0) 0x7f0e0c151e00 con 0x7f0e0c1ec490 2024-09-13T17:03:53.212 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:53.212 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 110 2024-09-13T17:03:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0dec0666c0 msgr2=0x7f0dec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0dec0666c0 0x7f0dec068b80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009990 tx=0x7f0dfc008040 comp rx=0 tx=0).stop 2024-09-13T17:03:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0c1ec490 msgr2=0x7f0e0c0c1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0c1ec490 0x7f0e0c0c1920 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f0e0401f8f0 tx=0x7f0df4005800 comp rx=0 tx=0).stop 2024-09-13T17:03:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 shutdown_connections 2024-09-13T17:03:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0dec0666c0 0x7f0dec068b80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0c1ec490 0x7f0e0c0c1920 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e0c155150 0x7f0e0c1e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 --2- 172.21.15.39:0/686589800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0df4004d10 0x7f0e0c0ac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 >> 172.21.15.39:0/686589800 conn(0x7f0e0c0c3310 msgr2=0x7f0e0c14ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 shutdown_connections 2024-09-13T17:03:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:53.215+0000 7f0e10ff4640 1 -- 172.21.15.39:0/686589800 wait complete. 2024-09-13T17:03:53.232 INFO:tasks.mds_thrash.fs.[b]:mds.d has gained rank=0, replacing gid=6711 2024-09-13T17:03:53.232 INFO:tasks.mds_thrash.fs.[b]:waiting for 6 secs before reviving mds.c 2024-09-13T17:03:54.025 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T17:03:54.024+0000 7f6a2d999640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:03:59.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:03:59.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:03:59.233 INFO:tasks.mds_thrash.fs.[b]:reviving mds.c 2024-09-13T17:03:59.233 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:03:59.234 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:03:59.237 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:03:59.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:03:59.310 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T17:03:59.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22f0192640 1 Processor -- start 2024-09-13T17:03:59.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22f0192640 1 -- start start 2024-09-13T17:03:59.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22f0192640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f22e8153ff0 0x7f22e8154430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:59.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22f0192640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e81519e0 con 0x7f22e814acf0 2024-09-13T17:03:59.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22f0192640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e80c4e20 con 0x7f22e8151550 2024-09-13T17:03:59.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22f0192640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e8154ab0 con 0x7f22e8153ff0 2024-09-13T17:03:59.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22edf2e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f22e814acf0 0x7f22e80c4710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:39718/0 (socket says 172.21.15.39:39718) 2024-09-13T17:03:59.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22edf2e640 1 -- 172.21.15.39:0/20758080 learned_addr learned my addr 172.21.15.39:0/20758080 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ee72f640 1 --2- 172.21.15.39:0/20758080 >> v2:172.21.15.84:3300/0 conn(0x7f22e8153ff0 0x7f22e8154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205586468 0 0) 0x7f22e81519e0 con 0x7f22e814acf0 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d4003530 con 0x7f22e814acf0 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784835834 0 0) 0x7f22d8001120 con 0x7f22e8151550 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e81519e0 con 0x7f22e8151550 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2398986681 0 0) 0x7f22d4003530 con 0x7f22e814acf0 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 >> v2:172.21.15.84:3300/0 conn(0x7f22e8153ff0 msgr2=0x7f22e8154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 --2- 172.21.15.39:0/20758080 >> v2:172.21.15.84:3300/0 conn(0x7f22e8153ff0 0x7f22e8154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 >> v1:172.21.15.84:6789/0 conn(0x7f22e8151550 legacy=0x7f22e806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e8154d90 con 0x7f22e814acf0 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f22e4001680 con 0x7f22e814acf0 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 >> v1:172.21.15.39:6789/0 conn(0x7f22e814acf0 legacy=0x7f22e80c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:59.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d4003b10 0x7f22d4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d40047d0 0x7f22d4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d4005190 0x7f22d400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8154ab0 con 0x7f22d4003b10 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d400c0c0 con 0x7f22d4005190 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d400c280 con 0x7f22d40047d0 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22edf2e640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d4003b10 0x7f22d4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ee72f640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d4005190 0x7f22d400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ed72d640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d40047d0 0x7f22d4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22edf2e640 1 -- 172.21.15.39:0/20758080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d40047d0 msgr2=0x7f22d4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22edf2e640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d40047d0 0x7f22d4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22edf2e640 1 -- 172.21.15.39:0/20758080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d4005190 msgr2=0x7f22d400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22edf2e640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d4005190 0x7f22d400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ee72f640 1 -- 172.21.15.39:0/20758080 reap_dead start 2024-09-13T17:03:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22edf2e640 1 -- 172.21.15.39:0/20758080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4003530 con 0x7f22d4003b10 2024-09-13T17:03:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.383+0000 7f22ed72d640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d40047d0 0x7f22d4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:03:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22edf2e640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d4003b10 0x7f22d4003f10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c160 tx=0x7f22e401cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183f4c6b66677419 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e40058e0 con 0x7f22d4003b10 2024-09-13T17:03:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e4005a60 con 0x7f22d4003b10 2024-09-13T17:03:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22ecf2c640 1 -- 172.21.15.39:0/20758080 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4005e50 con 0x7f22d4003b10 2024-09-13T17:03:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 -- 172.21.15.39:0/20758080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d4003b10 msgr2=0x7f22d4003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d4003b10 0x7f22d4003f10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f22e400c160 tx=0x7f22e401cbb0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 -- 172.21.15.39:0/20758080 shutdown_connections 2024-09-13T17:03:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 --2- 172.21.15.39:0/20758080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22d4003b10 0x7f22d4003f10 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 -- 172.21.15.39:0/20758080 >> 172.21.15.39:0/20758080 conn(0x7f22e80c32a0 msgr2=0x7f22e80c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:59.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 -- 172.21.15.39:0/20758080 shutdown_connections 2024-09-13T17:03:59.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 -- 172.21.15.39:0/20758080 wait complete. 2024-09-13T17:03:59.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 Processor -- start 2024-09-13T17:03:59.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 -- start start 2024-09-13T17:03:59.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d4003b10 0x7f22e8147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22e81ed400 0x7f22e81de2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22e81de830 0x7f22e81f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80a8350 con 0x7f22e81de830 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8072360 con 0x7f22e81ed400 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.387+0000 7f22f0192640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8148070 con 0x7f22d4003b10 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ee72f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22e81de830 0x7f22e81f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22edf2e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d4003b10 0x7f22e8147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ee72f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22e81de830 0x7f22e81f0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39108/0 (socket says 172.21.15.39:39108) 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ee72f640 1 -- 172.21.15.39:0/2085088420 learned_addr learned my addr 172.21.15.39:0/2085088420 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ed72d640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22e81ed400 0x7f22e81de2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ee72f640 1 -- 172.21.15.39:0/2085088420 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d4003b10 msgr2=0x7f22e8147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ee72f640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d4003b10 0x7f22e8147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ee72f640 1 -- 172.21.15.39:0/2085088420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22e81ed400 msgr2=0x7f22e81de2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:59.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ee72f640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22e81ed400 0x7f22e81de2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ee72f640 1 -- 172.21.15.39:0/2085088420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e81ecc20 con 0x7f22e81de830 2024-09-13T17:03:59.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22ee72f640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22e81de830 0x7f22e81f0570 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f22e80c1e10 tx=0x7f22e0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:59.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22de7fc640 1 -- 172.21.15.39:0/2085088420 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000e040 con 0x7f22e81de830 2024-09-13T17:03:59.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22de7fc640 1 -- 172.21.15.39:0/2085088420 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e000f040 con 0x7f22e81de830 2024-09-13T17:03:59.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e80a6350 con 0x7f22e81de830 2024-09-13T17:03:59.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22de7fc640 1 -- 172.21.15.39:0/2085088420 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000fe60 con 0x7f22e81de830 2024-09-13T17:03:59.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e80a8070 con 0x7f22e81de830 2024-09-13T17:03:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e80a60a0 con 0x7f22e81de830 2024-09-13T17:03:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22de7fc640 1 -- 172.21.15.39:0/2085088420 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f22e0015430 con 0x7f22e81de830 2024-09-13T17:03:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22de7fc640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22cc0666c0 0x7f22cc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:03:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22de7fc640 1 -- 172.21.15.39:0/2085088420 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7f22e00855a0 con 0x7f22e81de830 2024-09-13T17:03:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22edf2e640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22cc0666c0 0x7f22cc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:03:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.391+0000 7f22edf2e640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22cc0666c0 0x7f22cc068b80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f22e40aa000 tx=0x7f22e4001a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:03:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.399+0000 7f22de7fc640 1 -- 172.21.15.39:0/2085088420 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f22e0051ef0 con 0x7f22e81de830 2024-09-13T17:03:59.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.643+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22e80a5ac0 con 0x7f22e81de830 2024-09-13T17:03:59.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22de7fc640 1 -- 172.21.15.39:0/2085088420 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5174 (secure 0 0 0) 0x7f22e0055510 con 0x7f22e81de830 2024-09-13T17:03:59.645 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:03:59.645 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:03:59.646 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 110 2024-09-13T17:03:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22cc0666c0 msgr2=0x7f22cc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22cc0666c0 0x7f22cc068b80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f22e40aa000 tx=0x7f22e4001a00 comp rx=0 tx=0).stop 2024-09-13T17:03:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22e81de830 msgr2=0x7f22e81f0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:03:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22e81de830 0x7f22e81f0570 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f22e80c1e10 tx=0x7f22e0011f10 comp rx=0 tx=0).stop 2024-09-13T17:03:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 shutdown_connections 2024-09-13T17:03:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22cc0666c0 0x7f22cc068b80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22e81ed400 0x7f22e81de2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22e81de830 0x7f22e81f0570 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 --2- 172.21.15.39:0/2085088420 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22d4003b10 0x7f22e8147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:03:59.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 >> 172.21.15.39:0/2085088420 conn(0x7f22e80c32a0 msgr2=0x7f22e8148900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:03:59.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 shutdown_connections 2024-09-13T17:03:59.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:03:59.647+0000 7f22f0192640 1 -- 172.21.15.39:0/2085088420 wait complete. 2024-09-13T17:03:59.668 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-09-13T17:04:01.669 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:01.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b5ddc0640 1 Processor -- start 2024-09-13T17:04:01.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b5ddc0640 1 -- start start 2024-09-13T17:04:01.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b5ddc0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b580611c0 0x7f2b580bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:01.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b5ddc0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b580c1340 con 0x7f2b58151860 2024-09-13T17:04:01.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b5ddc0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b581578f0 con 0x7f2b580bf4f0 2024-09-13T17:04:01.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b5ddc0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b580c1590 con 0x7f2b580611c0 2024-09-13T17:04:01.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b577fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2b58151860 0x7f2b58157d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36394/0 (socket says 172.21.15.39:36394) 2024-09-13T17:04:01.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b577fe640 1 -- 172.21.15.39:0/3361243105 learned_addr learned my addr 172.21.15.39:0/3361243105 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:04:01.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b57fff640 1 --2- 172.21.15.39:0/3361243105 >> v2:172.21.15.84:3300/0 conn(0x7f2b580611c0 0x7f2b580bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:01.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389802030 0 0) 0x7f2b580c1340 con 0x7f2b58151860 2024-09-13T17:04:01.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b40003530 con 0x7f2b58151860 2024-09-13T17:04:01.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 111916850 0 0) 0x7f2b40003530 con 0x7f2b58151860 2024-09-13T17:04:01.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 >> v2:172.21.15.84:3300/0 conn(0x7f2b580611c0 msgr2=0x7f2b580bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:01.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 --2- 172.21.15.39:0/3361243105 >> v2:172.21.15.84:3300/0 conn(0x7f2b580611c0 0x7f2b580bfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:01.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 >> v1:172.21.15.84:6789/0 conn(0x7f2b580bf4f0 legacy=0x7f2b58153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b580c1870 con 0x7f2b58151860 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f2b48001680 con 0x7f2b58151860 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 >> v1:172.21.15.39:6789/0 conn(0x7f2b58151860 legacy=0x7f2b58157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b40003980 0x7f2b40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b400047d0 0x7f2b40004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b40005190 0x7f2b4000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580c1590 con 0x7f2b40003980 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4000c130 con 0x7f2b40005190 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4000c2f0 con 0x7f2b400047d0 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b577fe640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b40003980 0x7f2b40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b56ffd640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b400047d0 0x7f2b40004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b57fff640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b40005190 0x7f2b4000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b577fe640 1 -- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b400047d0 msgr2=0x7f2b40004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:01.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b577fe640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b400047d0 0x7f2b40004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:01.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b577fe640 1 -- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b40005190 msgr2=0x7f2b4000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:01.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b577fe640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b40005190 0x7f2b4000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:01.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b577fe640 1 -- 172.21.15.39:0/3361243105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40003530 con 0x7f2b40003980 2024-09-13T17:04:01.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b57fff640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b40005190 0x7f2b4000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:04:01.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b57fff640 1 -- 172.21.15.39:0/3361243105 reap_dead start 2024-09-13T17:04:01.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.823+0000 7f2b56ffd640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b400047d0 0x7f2b40004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b577fe640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b40003980 0x7f2b40003da0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800c240 tx=0x7f2b4801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32675eb7779641f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48005ae0 con 0x7f2b40003980 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48005c40 con 0x7f2b40003980 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b567fc640 1 -- 172.21.15.39:0/3361243105 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480a5ab0 con 0x7f2b40003980 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/3361243105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b40003980 msgr2=0x7f2b40003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b5ddc0640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b40003980 0x7f2b40003da0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800c240 tx=0x7f2b4801cbf0 comp rx=0 tx=0).stop 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/3361243105 shutdown_connections 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b5ddc0640 1 --2- 172.21.15.39:0/3361243105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b40003980 0x7f2b40003da0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/3361243105 >> 172.21.15.39:0/3361243105 conn(0x7f2b58069e10 msgr2=0x7f2b5806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:01.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/3361243105 shutdown_connections 2024-09-13T17:04:01.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/3361243105 wait complete. 2024-09-13T17:04:01.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.827+0000 7f2b5ddc0640 1 Processor -- start 2024-09-13T17:04:01.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 -- start start 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b40003980 0x7f2b581e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b580c4ab0 0x7f2b581ed8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b58071960 0x7f2b580709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58145860 con 0x7f2b58071960 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b581eced0 con 0x7f2b580c4ab0 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580c5850 con 0x7f2b40003980 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b58071960 0x7f2b580709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b58071960 0x7f2b580709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46864/0 (socket says 172.21.15.39:46864) 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b57fff640 1 -- 172.21.15.39:0/2281793581 learned_addr learned my addr 172.21.15.39:0/2281793581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b56ffd640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b580c4ab0 0x7f2b581ed8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b577fe640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b40003980 0x7f2b581e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:01.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b57fff640 1 -- 172.21.15.39:0/2281793581 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b40003980 msgr2=0x7f2b581e1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b57fff640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b40003980 0x7f2b581e1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b57fff640 1 -- 172.21.15.39:0/2281793581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b580c4ab0 msgr2=0x7f2b581ed8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b57fff640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b580c4ab0 0x7f2b581ed8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b57fff640 1 -- 172.21.15.39:0/2281793581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b581ede20 con 0x7f2b58071960 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b577fe640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b40003980 0x7f2b581e1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b57fff640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b58071960 0x7f2b580709c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2b5806ac40 tx=0x7f2b5000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b3ffff640 1 -- 172.21.15.39:0/2281793581 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50016420 con 0x7f2b58071960 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b3ffff640 1 -- 172.21.15.39:0/2281793581 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b50016580 con 0x7f2b58071960 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b3ffff640 1 -- 172.21.15.39:0/2281793581 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b500168f0 con 0x7f2b58071960 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b58070f00 con 0x7f2b58071960 2024-09-13T17:04:01.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b580c5570 con 0x7f2b58071960 2024-09-13T17:04:01.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b580c0f10 con 0x7f2b58071960 2024-09-13T17:04:01.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.831+0000 7f2b3ffff640 1 -- 172.21.15.39:0/2281793581 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b50016a50 con 0x7f2b58071960 2024-09-13T17:04:01.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.835+0000 7f2b3ffff640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2b440666c0 0x7f2b44068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:01.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.835+0000 7f2b3ffff640 1 -- 172.21.15.39:0/2281793581 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7f2b50085900 con 0x7f2b58071960 2024-09-13T17:04:01.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.835+0000 7f2b577fe640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2b440666c0 0x7f2b44068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:01.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.835+0000 7f2b577fe640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2b440666c0 0x7f2b44068b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2b480ab000 tx=0x7f2b480a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:01.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:01.839+0000 7f2b3ffff640 1 -- 172.21.15.39:0/2281793581 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2b50052250 con 0x7f2b58071960 2024-09-13T17:04:02.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.055+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b580c3950 con 0x7f2b58071960 2024-09-13T17:04:02.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.055+0000 7f2b3ffff640 1 -- 172.21.15.39:0/2281793581 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f2b50055870 con 0x7f2b58071960 2024-09-13T17:04:02.054 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:02.055 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:02.055 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:02.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2b440666c0 msgr2=0x7f2b44068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:02.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2b440666c0 0x7f2b44068b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2b480ab000 tx=0x7f2b480a7070 comp rx=0 tx=0).stop 2024-09-13T17:04:02.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b58071960 msgr2=0x7f2b580709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:02.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b58071960 0x7f2b580709c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2b5806ac40 tx=0x7f2b5000f520 comp rx=0 tx=0).stop 2024-09-13T17:04:02.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 shutdown_connections 2024-09-13T17:04:02.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2b440666c0 0x7f2b44068b80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:02.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b580c4ab0 0x7f2b581ed8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:02.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b58071960 0x7f2b580709c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:02.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 --2- 172.21.15.39:0/2281793581 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2b40003980 0x7f2b581e1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:02.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 >> 172.21.15.39:0/2281793581 conn(0x7f2b58069e10 msgr2=0x7f2b581e3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:02.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 shutdown_connections 2024-09-13T17:04:02.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:02.059+0000 7f2b5ddc0640 1 -- 172.21.15.39:0/2281793581 wait complete. 2024-09-13T17:04:02.079 INFO:tasks.mds_thrash.fs.[b]:mds.c reported in up:standby state 2024-09-13T17:04:02.080 INFO:tasks.mds_thrash.fs.[b]:waiting for 94 secs before thrashing 2024-09-13T17:04:05.875 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:06.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.015+0000 7fdcdaa88640 1 Processor -- start 2024-09-13T17:04:06.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.015+0000 7fdcdaa88640 1 -- start start 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcdaa88640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdcd414f1b0 0x7fdcd4151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcdaa88640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdcd406e5e0 0x7fdcd40714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcdaa88640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd40c7470 0x7fdcd40c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcdaa88640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd406f660 con 0x7fdcd414f1b0 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcdaa88640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd414f590 con 0x7fdcd406e5e0 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcdaa88640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd406eae0 con 0x7fdcd40c7470 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd8824640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdcd414f1b0 0x7fdcd4151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd8824640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdcd414f1b0 0x7fdcd4151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46868/0 (socket says 172.21.15.39:46868) 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd8824640 1 -- 172.21.15.39:0/2738751597 learned_addr learned my addr 172.21.15.39:0/2738751597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd9025640 1 --2- 172.21.15.39:0/2738751597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd40c7470 0x7fdcd40c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd3fff640 1 --2- 172.21.15.39:0/2738751597 >> v2:172.21.15.84:3300/0 conn(0x7fdcd406e5e0 0x7fdcd40714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd8824640 1 -- 172.21.15.39:0/2738751597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd40c7470 msgr2=0x7fdcd40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:06.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd8824640 1 --2- 172.21.15.39:0/2738751597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd40c7470 0x7fdcd40c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd8824640 1 -- 172.21.15.39:0/2738751597 >> v2:172.21.15.84:3300/0 conn(0x7fdcd406e5e0 msgr2=0x7fdcd40714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:06.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd8824640 1 --2- 172.21.15.39:0/2738751597 >> v2:172.21.15.84:3300/0 conn(0x7fdcd406e5e0 0x7fdcd40714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd8824640 1 -- 172.21.15.39:0/2738751597 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd406edc0 con 0x7fdcd414f1b0 2024-09-13T17:04:06.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.019+0000 7fdcd3fff640 1 --2- 172.21.15.39:0/2738751597 >> v2:172.21.15.84:3300/0 conn(0x7fdcd406e5e0 0x7fdcd40714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:04:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcd8824640 1 --2- 172.21.15.39:0/2738751597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd414f1b0 0x7fdcd4151bb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40c1fe0 tx=0x7fdcc801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad26ceaae939059 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcd37fe640 1 -- 172.21.15.39:0/2738751597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40c69a0 con 0x7fdcd414f1b0 2024-09-13T17:04:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcd37fe640 1 -- 172.21.15.39:0/2738751597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc80a6040 con 0x7fdcd414f1b0 2024-09-13T17:04:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcd37fe640 1 -- 172.21.15.39:0/2738751597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80aa830 con 0x7fdcd414f1b0 2024-09-13T17:04:06.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 -- 172.21.15.39:0/2738751597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd414f1b0 msgr2=0x7fdcd4151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/2738751597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd414f1b0 0x7fdcd4151bb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40c1fe0 tx=0x7fdcc801c4f0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 -- 172.21.15.39:0/2738751597 shutdown_connections 2024-09-13T17:04:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/2738751597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd40c7470 0x7fdcd40c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/2738751597 >> v2:172.21.15.84:3300/0 conn(0x7fdcd406e5e0 0x7fdcd40714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/2738751597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd414f1b0 0x7fdcd4151bb0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 -- 172.21.15.39:0/2738751597 >> 172.21.15.39:0/2738751597 conn(0x7fdcd40a9720 msgr2=0x7fdcd40a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 -- 172.21.15.39:0/2738751597 shutdown_connections 2024-09-13T17:04:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 -- 172.21.15.39:0/2738751597 wait complete. 2024-09-13T17:04:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.023+0000 7fdcdaa88640 1 Processor -- start 2024-09-13T17:04:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 -- start start 2024-09-13T17:04:06.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd406e5e0 0x7fdcd406b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd40c7470 0x7fdcd40c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd414f1b0 0x7fdcd4149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd41f07f0 con 0x7fdcd40c7470 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd41f17c0 con 0x7fdcd406e5e0 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd406b980 con 0x7fdcd414f1b0 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd40c7470 0x7fdcd40c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd40c7470 0x7fdcd40c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46882/0 (socket says 172.21.15.39:46882) 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd3fff640 1 -- 172.21.15.39:0/1709575085 learned_addr learned my addr 172.21.15.39:0/1709575085 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd9025640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd414f1b0 0x7fdcd4149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd8824640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd406e5e0 0x7fdcd406b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd3fff640 1 -- 172.21.15.39:0/1709575085 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd414f1b0 msgr2=0x7fdcd4149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd3fff640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd414f1b0 0x7fdcd4149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd3fff640 1 -- 172.21.15.39:0/1709575085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd406e5e0 msgr2=0x7fdcd406b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd3fff640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd406e5e0 0x7fdcd406b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd3fff640 1 -- 172.21.15.39:0/1709575085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc80094d0 con 0x7fdcd40c7470 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd3fff640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd40c7470 0x7fdcd40c3cd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000c7f0 tx=0x7fdcc000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd17fa640 1 -- 172.21.15.39:0/1709575085 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000cea0 con 0x7fdcd40c7470 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd17fa640 1 -- 172.21.15.39:0/1709575085 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc00079d0 con 0x7fdcd40c7470 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd406b6e0 con 0x7fdcd40c7470 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd17fa640 1 -- 172.21.15.39:0/1709575085 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0007d60 con 0x7fdcd40c7470 2024-09-13T17:04:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd41f0a00 con 0x7fdcd40c7470 2024-09-13T17:04:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd40714e0 con 0x7fdcd40c7470 2024-09-13T17:04:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.027+0000 7fdcd17fa640 1 -- 172.21.15.39:0/1709575085 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdcc00113f0 con 0x7fdcd40c7470 2024-09-13T17:04:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.031+0000 7fdcd17fa640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcb8066690 0x7fdcb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.031+0000 7fdcd17fa640 1 -- 172.21.15.39:0/1709575085 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7fdcc0084cb0 con 0x7fdcd40c7470 2024-09-13T17:04:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.031+0000 7fdcd8824640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcb8066690 0x7fdcb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.031+0000 7fdcd8824640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcb8066690 0x7fdcb8068b50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ae000 tx=0x7fdcc80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:06.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.035+0000 7fdcd17fa640 1 -- 172.21.15.39:0/1709575085 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdcc008a020 con 0x7fdcd40c7470 2024-09-13T17:04:06.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.259+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcd40c2050 con 0x7fdcd40c7470 2024-09-13T17:04:06.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcd17fa640 1 -- 172.21.15.39:0/1709575085 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7fdcc0051600 con 0x7fdcd40c7470 2024-09-13T17:04:06.258 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:06.258 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:06.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcb8066690 msgr2=0x7fdcb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcb8066690 0x7fdcb8068b50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80ae000 tx=0x7fdcc80a7000 comp rx=0 tx=0).stop 2024-09-13T17:04:06.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd40c7470 msgr2=0x7fdcd40c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd40c7470 0x7fdcd40c3cd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000c7f0 tx=0x7fdcc000ccb0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 shutdown_connections 2024-09-13T17:04:06.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdcb8066690 0x7fdcb8068b50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcd414f1b0 0x7fdcd4149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcd40c7470 0x7fdcd40c3cd0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 --2- 172.21.15.39:0/1709575085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcd406e5e0 0x7fdcd406b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 >> 172.21.15.39:0/1709575085 conn(0x7fdcd40a9720 msgr2=0x7fdcd40609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:06.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 shutdown_connections 2024-09-13T17:04:06.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.263+0000 7fdcdaa88640 1 -- 172.21.15.39:0/1709575085 wait complete. 2024-09-13T17:04:06.278 INFO:tasks.mds_thrash.fs.[a]:kill mds.a (rank=0) 2024-09-13T17:04:06.278 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:04:06.278 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:04:06.291 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:04:06.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.459+0000 7f537dfe7640 1 Processor -- start 2024-09-13T17:04:06.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.459+0000 7f537dfe7640 1 -- start start 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f537dfe7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 0x7f5378150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f537dfe7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f537814ab50 0x7f53780713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f537dfe7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537814b000 con 0x7f53780c6060 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f537dfe7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537806df80 con 0x7f537814ab50 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f537dfe7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53780c1950 con 0x7f537814a770 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f536effd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f53780c6060 0x7f537806ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36408/0 (socket says 172.21.15.39:36408) 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f536effd640 1 -- 172.21.15.39:0/3554394748 learned_addr learned my addr 172.21.15.39:0/3554394748 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f53777fe640 1 --2- 172.21.15.39:0/3554394748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 0x7f5378150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f5377fff640 1 --2- 172.21.15.39:0/3554394748 >> v2:172.21.15.84:3300/0 conn(0x7f537814ab50 0x7f53780713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f5376ffd640 1 -- 172.21.15.39:0/3554394748 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442830411 0 0) 0x7f537814b000 con 0x7f53780c6060 2024-09-13T17:04:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f5376ffd640 1 -- 172.21.15.39:0/3554394748 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5364003530 con 0x7f53780c6060 2024-09-13T17:04:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f53777fe640 1 -- 172.21.15.39:0/3554394748 >> v2:172.21.15.84:3300/0 conn(0x7f537814ab50 msgr2=0x7f53780713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f53777fe640 1 --2- 172.21.15.39:0/3554394748 >> v2:172.21.15.84:3300/0 conn(0x7f537814ab50 0x7f53780713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f53777fe640 1 -- 172.21.15.39:0/3554394748 >> v1:172.21.15.39:6789/0 conn(0x7f53780c6060 legacy=0x7f537806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f53777fe640 1 -- 172.21.15.39:0/3554394748 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53780c4760 con 0x7f537814a770 2024-09-13T17:04:06.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.463+0000 7f5377fff640 1 --2- 172.21.15.39:0/3554394748 >> v2:172.21.15.84:3300/0 conn(0x7f537814ab50 0x7f53780713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:04:06.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f53777fe640 1 --2- 172.21.15.39:0/3554394748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 0x7f5378150ad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f536800ba60 tx=0x7f536801eab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec819fd232446d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5376ffd640 1 -- 172.21.15.39:0/3554394748 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53680a7b80 con 0x7f537814a770 2024-09-13T17:04:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5376ffd640 1 -- 172.21.15.39:0/3554394748 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53680a7ce0 con 0x7f537814a770 2024-09-13T17:04:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5376ffd640 1 -- 172.21.15.39:0/3554394748 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53680af660 con 0x7f537814a770 2024-09-13T17:04:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 -- 172.21.15.39:0/3554394748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 msgr2=0x7f5378150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 --2- 172.21.15.39:0/3554394748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 0x7f5378150ad0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f536800ba60 tx=0x7f536801eab0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 -- 172.21.15.39:0/3554394748 shutdown_connections 2024-09-13T17:04:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 --2- 172.21.15.39:0/3554394748 >> v2:172.21.15.84:3300/0 conn(0x7f537814ab50 0x7f53780713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 --2- 172.21.15.39:0/3554394748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 0x7f5378150ad0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 -- 172.21.15.39:0/3554394748 >> 172.21.15.39:0/3554394748 conn(0x7f53780c32a0 msgr2=0x7f53780c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 -- 172.21.15.39:0/3554394748 shutdown_connections 2024-09-13T17:04:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 -- 172.21.15.39:0/3554394748 wait complete. 2024-09-13T17:04:06.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 Processor -- start 2024-09-13T17:04:06.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 -- start start 2024-09-13T17:04:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53780c6060 0x7f537814d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 0x7f5378152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f537814ab50 0x7f5378149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537814de50 con 0x7f537814ab50 2024-09-13T17:04:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537814f990 con 0x7f53780c6060 2024-09-13T17:04:06.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f537dfe7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537814fc70 con 0x7f537814a770 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5377fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f537814ab50 0x7f5378149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5377fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f537814ab50 0x7f5378149b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46890/0 (socket says 172.21.15.39:46890) 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5377fff640 1 -- 172.21.15.39:0/1559140468 learned_addr learned my addr 172.21.15.39:0/1559140468 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f53777fe640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53780c6060 0x7f537814d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f536effd640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 0x7f5378152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5377fff640 1 -- 172.21.15.39:0/1559140468 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 msgr2=0x7f5378152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5377fff640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 0x7f5378152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5377fff640 1 -- 172.21.15.39:0/1559140468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53780c6060 msgr2=0x7f537814d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5377fff640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53780c6060 0x7f537814d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5377fff640 1 -- 172.21.15.39:0/1559140468 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5370009a70 con 0x7f537814ab50 2024-09-13T17:04:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f53777fe640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53780c6060 0x7f537814d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:04:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.467+0000 7f5377fff640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f537814ab50 0x7f5378149b40 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f537000efc0 tx=0x7f5370009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f5374ff9640 1 -- 172.21.15.39:0/1559140468 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370009d70 con 0x7f537814ab50 2024-09-13T17:04:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f5374ff9640 1 -- 172.21.15.39:0/1559140468 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537000f040 con 0x7f537814ab50 2024-09-13T17:04:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536800b6c0 con 0x7f537814ab50 2024-09-13T17:04:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f5374ff9640 1 -- 172.21.15.39:0/1559140468 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53700039a0 con 0x7f537814ab50 2024-09-13T17:04:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5378150230 con 0x7f537814ab50 2024-09-13T17:04:06.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53780619c0 con 0x7f537814ab50 2024-09-13T17:04:06.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f5374ff9640 1 -- 172.21.15.39:0/1559140468 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5370014480 con 0x7f537814ab50 2024-09-13T17:04:06.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f5374ff9640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5358066690 0x7f5358068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:06.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f5374ff9640 1 -- 172.21.15.39:0/1559140468 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7f5370085740 con 0x7f537814ab50 2024-09-13T17:04:06.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f53777fe640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5358066690 0x7f5358068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:06.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.471+0000 7f53777fe640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5358066690 0x7f5358068b50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f53680b1000 tx=0x7f5368008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:06.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.475+0000 7f5374ff9640 1 -- 172.21.15.39:0/1559140468 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5370052090 con 0x7f537814ab50 2024-09-13T17:04:06.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.703+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5378071d30 con 0x7f537814ab50 2024-09-13T17:04:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f5374ff9640 1 -- 172.21.15.39:0/1559140468 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f5370018090 con 0x7f537814ab50 2024-09-13T17:04:06.701 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:06.701 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:06.702 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5358066690 msgr2=0x7f5358068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5358066690 0x7f5358068b50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f53680b1000 tx=0x7f5368008040 comp rx=0 tx=0).stop 2024-09-13T17:04:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f537814ab50 msgr2=0x7f5378149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f537814ab50 0x7f5378149b40 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f537000efc0 tx=0x7f5370009f90 comp rx=0 tx=0).stop 2024-09-13T17:04:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 shutdown_connections 2024-09-13T17:04:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5358066690 0x7f5358068b50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f537814ab50 0x7f5378149b40 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f537814a770 0x7f5378152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 --2- 172.21.15.39:0/1559140468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53780c6060 0x7f537814d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:06.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 >> 172.21.15.39:0/1559140468 conn(0x7f53780c32a0 msgr2=0x7f53780bc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:06.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 shutdown_connections 2024-09-13T17:04:06.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:06.707+0000 7f537dfe7640 1 -- 172.21.15.39:0/1559140468 wait complete. 2024-09-13T17:04:06.721 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:04:08.722 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:08.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc583425640 1 Processor -- start 2024-09-13T17:04:08.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc583425640 1 -- start start 2024-09-13T17:04:08.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc583425640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc57c0c6060 0x7fc57c150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:08.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc583425640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14ab50 0x7fc57c0713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:08.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc583425640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57c0c4760 con 0x7fc57c14a770 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc583425640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57c06df80 con 0x7fc57c0c6060 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc583425640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0c1950 con 0x7fc57c14ab50 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc5809c0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc57c0c6060 0x7fc57c150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc5809c0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc57c0c6060 0x7fc57c150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46902/0 (socket says 172.21.15.39:46902) 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc5809c0640 1 -- 172.21.15.39:0/341502792 learned_addr learned my addr 172.21.15.39:0/341502792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc5811c1640 1 --1- 172.21.15.39:0/341502792 >> v1:172.21.15.39:6789/0 conn(0x7fc57c14a770 0x7fc57c06ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36414/0 (socket says 172.21.15.39:36414) 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.859+0000 7fc5819c2640 1 --2- 172.21.15.39:0/341502792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14ab50 0x7fc57c0713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc5809c0640 1 -- 172.21.15.39:0/341502792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14ab50 msgr2=0x7fc57c0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc5809c0640 1 --2- 172.21.15.39:0/341502792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14ab50 0x7fc57c0713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc5809c0640 1 -- 172.21.15.39:0/341502792 >> v1:172.21.15.39:6789/0 conn(0x7fc57c14a770 legacy=0x7fc57c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc5809c0640 1 -- 172.21.15.39:0/341502792 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c14afe0 con 0x7fc57c0c6060 2024-09-13T17:04:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc5809c0640 1 --2- 172.21.15.39:0/341502792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0c6060 0x7fc57c150ad0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc578009870 tx=0x7fc57801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355e06cc8aa956a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc573fff640 1 -- 172.21.15.39:0/341502792 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5780a6040 con 0x7fc57c0c6060 2024-09-13T17:04:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc573fff640 1 -- 172.21.15.39:0/341502792 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5780aa600 con 0x7fc57c0c6060 2024-09-13T17:04:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc573fff640 1 -- 172.21.15.39:0/341502792 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5780aa990 con 0x7fc57c0c6060 2024-09-13T17:04:08.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc583425640 1 -- 172.21.15.39:0/341502792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0c6060 msgr2=0x7fc57c150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:08.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc583425640 1 --2- 172.21.15.39:0/341502792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0c6060 0x7fc57c150ad0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc578009870 tx=0x7fc57801c4a0 comp rx=0 tx=0).stop 2024-09-13T17:04:08.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc583425640 1 -- 172.21.15.39:0/341502792 shutdown_connections 2024-09-13T17:04:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc583425640 1 --2- 172.21.15.39:0/341502792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14ab50 0x7fc57c0713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc583425640 1 --2- 172.21.15.39:0/341502792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0c6060 0x7fc57c150ad0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc583425640 1 -- 172.21.15.39:0/341502792 >> 172.21.15.39:0/341502792 conn(0x7fc57c0c32a0 msgr2=0x7fc57c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc583425640 1 -- 172.21.15.39:0/341502792 shutdown_connections 2024-09-13T17:04:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.863+0000 7fc583425640 1 -- 172.21.15.39:0/341502792 wait complete. 2024-09-13T17:04:08.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 Processor -- start 2024-09-13T17:04:08.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 -- start start 2024-09-13T17:04:08.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c0c6060 0x7fc57c0c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14a770 0x7fc57c0abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c14ab50 0x7fc57c0ac390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c143b10 con 0x7fc57c14ab50 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c154740 con 0x7fc57c0c6060 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c14c700 con 0x7fc57c14a770 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5819c2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c14ab50 0x7fc57c0ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5819c2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c14ab50 0x7fc57c0ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46912/0 (socket says 172.21.15.39:46912) 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5819c2640 1 -- 172.21.15.39:0/3582975057 learned_addr learned my addr 172.21.15.39:0/3582975057 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5809c0640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14a770 0x7fc57c0abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:08.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5811c1640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c0c6060 0x7fc57c0c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5819c2640 1 -- 172.21.15.39:0/3582975057 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14a770 msgr2=0x7fc57c0abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5819c2640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14a770 0x7fc57c0abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5819c2640 1 -- 172.21.15.39:0/3582975057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c0c6060 msgr2=0x7fc57c0c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5819c2640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c0c6060 0x7fc57c0c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5819c2640 1 -- 172.21.15.39:0/3582975057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5780094d0 con 0x7fc57c14ab50 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5811c1640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c0c6060 0x7fc57c0c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc5819c2640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c14ab50 0x7fc57c0ac390 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc574009dd0 tx=0x7fc57400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc571ffb640 1 -- 172.21.15.39:0/3582975057 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574014480 con 0x7fc57c14ab50 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc571ffb640 1 -- 172.21.15.39:0/3582975057 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57400f040 con 0x7fc57c14ab50 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c1eb000 con 0x7fc57c14ab50 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc571ffb640 1 -- 172.21.15.39:0/3582975057 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574014840 con 0x7fc57c14ab50 2024-09-13T17:04:08.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.867+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c1eb2a0 con 0x7fc57c14ab50 2024-09-13T17:04:08.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.871+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c0619c0 con 0x7fc57c14ab50 2024-09-13T17:04:08.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.871+0000 7fc571ffb640 1 -- 172.21.15.39:0/3582975057 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc5740149a0 con 0x7fc57c14ab50 2024-09-13T17:04:08.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.871+0000 7fc571ffb640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc560066690 0x7fc560068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:08.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.871+0000 7fc571ffb640 1 -- 172.21.15.39:0/3582975057 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7fc5740856b0 con 0x7fc57c14ab50 2024-09-13T17:04:08.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.871+0000 7fc5811c1640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc560066690 0x7fc560068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:08.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.871+0000 7fc5811c1640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc560066690 0x7fc560068b50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0c2de0 tx=0x7fc56c007d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:08.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:08.875+0000 7fc571ffb640 1 -- 172.21.15.39:0/3582975057 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc574052000 con 0x7fc57c14ab50 2024-09-13T17:04:09.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.067+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc57c0a9610 con 0x7fc57c14ab50 2024-09-13T17:04:09.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc571ffb640 1 -- 172.21.15.39:0/3582975057 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7fc57c0a9610 con 0x7fc57c14ab50 2024-09-13T17:04:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:09.066 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:09.067 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:09.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc560066690 msgr2=0x7fc560068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:09.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc560066690 0x7fc560068b50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0c2de0 tx=0x7fc56c007d90 comp rx=0 tx=0).stop 2024-09-13T17:04:09.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c14ab50 msgr2=0x7fc57c0ac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:09.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c14ab50 0x7fc57c0ac390 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc574009dd0 tx=0x7fc57400c490 comp rx=0 tx=0).stop 2024-09-13T17:04:09.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 shutdown_connections 2024-09-13T17:04:09.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc560066690 0x7fc560068b50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:09.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c14ab50 0x7fc57c0ac390 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:09.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc57c14a770 0x7fc57c0abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 --2- 172.21.15.39:0/3582975057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc57c0c6060 0x7fc57c0c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 >> 172.21.15.39:0/3582975057 conn(0x7fc57c0c32a0 msgr2=0x7fc57c150950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 shutdown_connections 2024-09-13T17:04:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:09.071+0000 7fc583425640 1 -- 172.21.15.39:0/3582975057 wait complete. 2024-09-13T17:04:09.088 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:04:11.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:11.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.223+0000 7f8730c32640 1 Processor -- start 2024-09-13T17:04:11.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8730c32640 1 -- start start 2024-09-13T17:04:11.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8730c32640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c151750 0x7f872c0c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8730c32640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f872c0bd010 0x7f872c070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8730c32640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872c071080 con 0x7f872c0c7860 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8730c32640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872c151b30 con 0x7f872c0bd010 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8730c32640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c157e80 con 0x7f872c151750 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8729d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f872c0bd010 0x7f872c070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8729d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f872c0bd010 0x7f872c070430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50654/0 (socket says 172.21.15.39:50654) 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8729d9b640 1 -- 172.21.15.39:0/3737049957 learned_addr learned my addr 172.21.15.39:0/3737049957 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f872a59c640 1 --2- 172.21.15.39:0/3737049957 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c151750 0x7f872c0c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8729d9b640 1 -- 172.21.15.39:0/3737049957 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c151750 msgr2=0x7f872c0c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8729d9b640 1 --2- 172.21.15.39:0/3737049957 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c151750 0x7f872c0c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8729d9b640 1 -- 172.21.15.39:0/3737049957 >> v1:172.21.15.39:6789/0 conn(0x7f872c0c7860 legacy=0x7f872c070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:11.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8729d9b640 1 -- 172.21.15.39:0/3737049957 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c158160 con 0x7f872c0bd010 2024-09-13T17:04:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f8729d9b640 1 --2- 172.21.15.39:0/3737049957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 0x7f872c070430 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f8720009870 tx=0x7f872001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf5754feffda085 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f872959a640 1 -- 172.21.15.39:0/3737049957 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200a95f0 con 0x7f872c0bd010 2024-09-13T17:04:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f872959a640 1 -- 172.21.15.39:0/3737049957 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87200a9750 con 0x7f872c0bd010 2024-09-13T17:04:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.227+0000 7f872959a640 1 -- 172.21.15.39:0/3737049957 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200a9ae0 con 0x7f872c0bd010 2024-09-13T17:04:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- 172.21.15.39:0/3737049957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 msgr2=0x7f872c070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 --2- 172.21.15.39:0/3737049957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 0x7f872c070430 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f8720009870 tx=0x7f872001c530 comp rx=0 tx=0).stop 2024-09-13T17:04:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- 172.21.15.39:0/3737049957 shutdown_connections 2024-09-13T17:04:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 --2- 172.21.15.39:0/3737049957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 0x7f872c070430 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 --2- 172.21.15.39:0/3737049957 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c151750 0x7f872c0c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- 172.21.15.39:0/3737049957 >> 172.21.15.39:0/3737049957 conn(0x7f872c069e10 msgr2=0x7f872c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- 172.21.15.39:0/3737049957 shutdown_connections 2024-09-13T17:04:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- 172.21.15.39:0/3737049957 wait complete. 2024-09-13T17:04:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 Processor -- start 2024-09-13T17:04:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- start start 2024-09-13T17:04:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 0x7f872c152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c0c7860 0x7f872c14aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f872c151750 0x7f872c1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c1f0510 con 0x7f872c0bd010 2024-09-13T17:04:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c154570 con 0x7f872c151750 2024-09-13T17:04:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c154810 con 0x7f872c0c7860 2024-09-13T17:04:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 0x7f872c152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 0x7f872c152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50662/0 (socket says 172.21.15.39:50662) 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872a59c640 1 -- 172.21.15.39:0/3869156006 learned_addr learned my addr 172.21.15.39:0/3869156006 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8729d9b640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c0c7860 0x7f872c14aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872ad9d640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f872c151750 0x7f872c1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872a59c640 1 -- 172.21.15.39:0/3869156006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c0c7860 msgr2=0x7f872c14aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872a59c640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c0c7860 0x7f872c14aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872a59c640 1 -- 172.21.15.39:0/3869156006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f872c151750 msgr2=0x7f872c1508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872a59c640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f872c151750 0x7f872c1508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872a59c640 1 -- 172.21.15.39:0/3869156006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87200094d0 con 0x7f872c0bd010 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f872a59c640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 0x7f872c152a20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f871c00d800 tx=0x7f871c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f871affd640 1 -- 172.21.15.39:0/3869156006 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00b6c0 con 0x7f872c0bd010 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f871affd640 1 -- 172.21.15.39:0/3869156006 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c00b820 con 0x7f872c0bd010 2024-09-13T17:04:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f871affd640 1 -- 172.21.15.39:0/3869156006 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00bb90 con 0x7f872c0bd010 2024-09-13T17:04:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c154af0 con 0x7f872c0bd010 2024-09-13T17:04:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.231+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872c0ad4c0 con 0x7f872c0bd010 2024-09-13T17:04:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.235+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872c0c5cc0 con 0x7f872c0bd010 2024-09-13T17:04:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.235+0000 7f871affd640 1 -- 172.21.15.39:0/3869156006 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f871c00bcf0 con 0x7f872c0bd010 2024-09-13T17:04:11.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.235+0000 7f871affd640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f870c066690 0x7f870c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:11.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.235+0000 7f8729d9b640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f870c066690 0x7f870c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:11.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.235+0000 7f871affd640 1 -- 172.21.15.39:0/3869156006 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7f871c085140 con 0x7f872c0bd010 2024-09-13T17:04:11.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.235+0000 7f8729d9b640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f870c066690 0x7f870c068b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f87200ae000 tx=0x7f87200b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:11.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.239+0000 7f871affd640 1 -- 172.21.15.39:0/3869156006 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f871c051a90 con 0x7f872c0bd010 2024-09-13T17:04:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.435+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f872c0c42e0 con 0x7f872c0bd010 2024-09-13T17:04:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.435+0000 7f871affd640 1 -- 172.21.15.39:0/3869156006 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f871c011340 con 0x7f872c0bd010 2024-09-13T17:04:11.431 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:11.431 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:11.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f870c066690 msgr2=0x7f870c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f870c066690 0x7f870c068b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f87200ae000 tx=0x7f87200b3040 comp rx=0 tx=0).stop 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 msgr2=0x7f872c152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 0x7f872c152a20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f871c00d800 tx=0x7f871c00dcc0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 shutdown_connections 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f870c066690 0x7f870c068b50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f872c151750 0x7f872c1508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f872c0c7860 0x7f872c14aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 --2- 172.21.15.39:0/3869156006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f872c0bd010 0x7f872c152a20 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 >> 172.21.15.39:0/3869156006 conn(0x7f872c069e10 msgr2=0x7f872c14d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 shutdown_connections 2024-09-13T17:04:11.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:11.439+0000 7f8730c32640 1 -- 172.21.15.39:0/3869156006 wait complete. 2024-09-13T17:04:11.450 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:04:13.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.587+0000 7f5d076d0640 1 Processor -- start 2024-09-13T17:04:13.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d076d0640 1 -- start start 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d076d0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d0006c870 0x7f5d000aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d076d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d000c40b0 0x7f5d0015c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d076d0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d00152a80 con 0x7f5d000aabc0 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d076d0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d000c30f0 con 0x7f5d0006c870 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d076d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00151e00 con 0x7f5d000c40b0 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d04c6b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d0006c870 0x7f5d000aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d04c6b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d0006c870 0x7f5d000aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50670/0 (socket says 172.21.15.39:50670) 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d04c6b640 1 -- 172.21.15.39:0/236690233 learned_addr learned my addr 172.21.15.39:0/236690233 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d05c6d640 1 --2- 172.21.15.39:0/236690233 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d000c40b0 0x7f5d0015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d04c6b640 1 -- 172.21.15.39:0/236690233 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d000c40b0 msgr2=0x7f5d0015c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:13.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d04c6b640 1 --2- 172.21.15.39:0/236690233 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d000c40b0 0x7f5d0015c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d04c6b640 1 -- 172.21.15.39:0/236690233 >> v1:172.21.15.39:6789/0 conn(0x7f5d000aabc0 legacy=0x7f5d000c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:13.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d04c6b640 1 -- 172.21.15.39:0/236690233 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d001520e0 con 0x7f5d0006c870 2024-09-13T17:04:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5d04c6b640 1 --2- 172.21.15.39:0/236690233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d0006c870 0x7f5d000aafa0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009870 tx=0x7f5cf801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a65cf5b3a8a1fae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.591+0000 7f5cfffff640 1 -- 172.21.15.39:0/236690233 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80a6040 con 0x7f5d0006c870 2024-09-13T17:04:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5cfffff640 1 -- 172.21.15.39:0/236690233 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf80aa600 con 0x7f5d0006c870 2024-09-13T17:04:13.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5cfffff640 1 -- 172.21.15.39:0/236690233 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80aa990 con 0x7f5d0006c870 2024-09-13T17:04:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 -- 172.21.15.39:0/236690233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d0006c870 msgr2=0x7f5d000aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 --2- 172.21.15.39:0/236690233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d0006c870 0x7f5d000aafa0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009870 tx=0x7f5cf801c4a0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 -- 172.21.15.39:0/236690233 shutdown_connections 2024-09-13T17:04:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 --2- 172.21.15.39:0/236690233 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d000c40b0 0x7f5d0015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 --2- 172.21.15.39:0/236690233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d0006c870 0x7f5d000aafa0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 -- 172.21.15.39:0/236690233 >> 172.21.15.39:0/236690233 conn(0x7f5d00069e10 msgr2=0x7f5d0006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:13.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 -- 172.21.15.39:0/236690233 shutdown_connections 2024-09-13T17:04:13.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 -- 172.21.15.39:0/236690233 wait complete. 2024-09-13T17:04:13.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 Processor -- start 2024-09-13T17:04:13.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.595+0000 7f5d076d0640 1 -- start start 2024-09-13T17:04:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d076d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d0006c870 0x7f5d00071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d076d0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000aabc0 0x7f5d000a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d076d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d000c40b0 0x7f5d000a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d076d0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0014b490 con 0x7f5d000aabc0 2024-09-13T17:04:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d076d0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00150280 con 0x7f5d0006c870 2024-09-13T17:04:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d076d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d001eb620 con 0x7f5d000c40b0 2024-09-13T17:04:13.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d04c6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000aabc0 0x7f5d000a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d04c6b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000aabc0 0x7f5d000a6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50676/0 (socket says 172.21.15.39:50676) 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d04c6b640 1 -- 172.21.15.39:0/2695461015 learned_addr learned my addr 172.21.15.39:0/2695461015 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d04c6b640 1 -- 172.21.15.39:0/2695461015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d000c40b0 msgr2=0x7f5d000a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d04c6b640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d000c40b0 0x7f5d000a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d04c6b640 1 -- 172.21.15.39:0/2695461015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d0006c870 msgr2=0x7f5d00071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d04c6b640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d0006c870 0x7f5d00071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d04c6b640 1 -- 172.21.15.39:0/2695461015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf80094d0 con 0x7f5d000aabc0 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d04c6b640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000aabc0 0x7f5d000a6850 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5cf801a0d0 tx=0x7f5cf801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5cfdffb640 1 -- 172.21.15.39:0/2695461015 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80b3800 con 0x7f5d000aabc0 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5cfdffb640 1 -- 172.21.15.39:0/2695461015 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf80b39a0 con 0x7f5d000aabc0 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0014ae40 con 0x7f5d000aabc0 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5cfdffb640 1 -- 172.21.15.39:0/2695461015 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80b3d50 con 0x7f5d000aabc0 2024-09-13T17:04:13.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d000c0820 con 0x7f5d000aabc0 2024-09-13T17:04:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d000aafa0 con 0x7f5d000aabc0 2024-09-13T17:04:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5cfdffb640 1 -- 172.21.15.39:0/2695461015 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5cf80c6430 con 0x7f5d000aabc0 2024-09-13T17:04:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5cfdffb640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5cec0666c0 0x7f5cec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5cfdffb640 1 -- 172.21.15.39:0/2695461015 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7f5cf811f670 con 0x7f5d000aabc0 2024-09-13T17:04:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.599+0000 7f5d0546c640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5cec0666c0 0x7f5cec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:13.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.603+0000 7f5d0546c640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5cec0666c0 0x7f5cec068b80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007fd0 tx=0x7f5cf0007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:13.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.607+0000 7f5cfdffb640 1 -- 172.21.15.39:0/2695461015 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5cf80ebfc0 con 0x7f5d000aabc0 2024-09-13T17:04:13.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d000a6470 con 0x7f5d000aabc0 2024-09-13T17:04:13.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5cfdffb640 1 -- 172.21.15.39:0/2695461015 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f5cf80ef5e0 con 0x7f5d000aabc0 2024-09-13T17:04:13.801 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:13.801 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5cec0666c0 msgr2=0x7f5cec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5cec0666c0 0x7f5cec068b80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007fd0 tx=0x7f5cf0007d40 comp rx=0 tx=0).stop 2024-09-13T17:04:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000aabc0 msgr2=0x7f5d000a6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000aabc0 0x7f5d000a6850 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5cf801a0d0 tx=0x7f5cf801ace0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 shutdown_connections 2024-09-13T17:04:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5cec0666c0 0x7f5cec068b80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d000c40b0 0x7f5d000a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d000aabc0 0x7f5d000a6850 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 --2- 172.21.15.39:0/2695461015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d0006c870 0x7f5d00071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 >> 172.21.15.39:0/2695461015 conn(0x7f5d00069e10 msgr2=0x7f5d001559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 shutdown_connections 2024-09-13T17:04:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:13.803+0000 7f5d076d0640 1 -- 172.21.15.39:0/2695461015 wait complete. 2024-09-13T17:04:13.818 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:04:15.820 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:15.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.959+0000 7f708fd89640 1 Processor -- start 2024-09-13T17:04:15.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.959+0000 7f708fd89640 1 -- start start 2024-09-13T17:04:15.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708fd89640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f70880aabc0 0x7f70880aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:15.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708fd89640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f708806c870 0x7f70880c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:15.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708fd89640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7088152a80 con 0x7f70880c3ee0 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708fd89640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70880c30f0 con 0x7f70880aabc0 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708fd89640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7088151e00 con 0x7f708806c870 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708db25640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f70880aabc0 0x7f70880aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708db25640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f70880aabc0 0x7f70880aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50682/0 (socket says 172.21.15.39:50682) 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708db25640 1 -- 172.21.15.39:0/3304979784 learned_addr learned my addr 172.21.15.39:0/3304979784 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708d324640 1 --2- 172.21.15.39:0/3304979784 >> v2:172.21.15.84:3300/0 conn(0x7f708806c870 0x7f70880c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708db25640 1 -- 172.21.15.39:0/3304979784 >> v2:172.21.15.84:3300/0 conn(0x7f708806c870 msgr2=0x7f70880c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708db25640 1 --2- 172.21.15.39:0/3304979784 >> v2:172.21.15.84:3300/0 conn(0x7f708806c870 0x7f70880c39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708db25640 1 -- 172.21.15.39:0/3304979784 >> v1:172.21.15.84:6789/0 conn(0x7f70880c3ee0 legacy=0x7f70881550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:15.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708db25640 1 -- 172.21.15.39:0/3304979784 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70881520e0 con 0x7f70880aabc0 2024-09-13T17:04:15.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708db25640 1 --2- 172.21.15.39:0/3304979784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70880aabc0 0x7f70880aafa0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f70880bfe70 tx=0x7f708401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4703c4e574d9aa48 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:15.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708cb23640 1 -- 172.21.15.39:0/3304979784 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70840ab620 con 0x7f70880aabc0 2024-09-13T17:04:15.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708cb23640 1 -- 172.21.15.39:0/3304979784 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70840ab780 con 0x7f70880aabc0 2024-09-13T17:04:15.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708cb23640 1 -- 172.21.15.39:0/3304979784 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70840abb10 con 0x7f70880aabc0 2024-09-13T17:04:15.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708fd89640 1 -- 172.21.15.39:0/3304979784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70880aabc0 msgr2=0x7f70880aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:15.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.963+0000 7f708fd89640 1 --2- 172.21.15.39:0/3304979784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70880aabc0 0x7f70880aafa0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f70880bfe70 tx=0x7f708401c420 comp rx=0 tx=0).stop 2024-09-13T17:04:15.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- 172.21.15.39:0/3304979784 shutdown_connections 2024-09-13T17:04:15.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 --2- 172.21.15.39:0/3304979784 >> v2:172.21.15.84:3300/0 conn(0x7f708806c870 0x7f70880c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:15.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 --2- 172.21.15.39:0/3304979784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70880aabc0 0x7f70880aafa0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:15.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- 172.21.15.39:0/3304979784 >> 172.21.15.39:0/3304979784 conn(0x7f7088069e10 msgr2=0x7f708806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:15.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- 172.21.15.39:0/3304979784 shutdown_connections 2024-09-13T17:04:15.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- 172.21.15.39:0/3304979784 wait complete. 2024-09-13T17:04:15.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 Processor -- start 2024-09-13T17:04:15.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- start start 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f708806c870 0x7f7088070c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70880aabc0 0x7f7088150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f70880c3ee0 0x7f70881ea850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088071440 con 0x7f708806c870 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70881eddc0 con 0x7f70880aabc0 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70881ee0a0 con 0x7f70880c3ee0 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708db25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f708806c870 0x7f7088070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708db25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f708806c870 0x7f7088070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50684/0 (socket says 172.21.15.39:50684) 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708db25640 1 -- 172.21.15.39:0/3211297846 learned_addr learned my addr 172.21.15.39:0/3211297846 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708d324640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70880aabc0 0x7f7088150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:15.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708e326640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f70880c3ee0 0x7f70881ea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708db25640 1 -- 172.21.15.39:0/3211297846 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f70880c3ee0 msgr2=0x7f70881ea850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708db25640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f70880c3ee0 0x7f70881ea850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708db25640 1 -- 172.21.15.39:0/3211297846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70880aabc0 msgr2=0x7f7088150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708db25640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70880aabc0 0x7f7088150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708db25640 1 -- 172.21.15.39:0/3211297846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70840094d0 con 0x7f708806c870 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708d324640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70880aabc0 0x7f7088150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708db25640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f708806c870 0x7f7088070c00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f708401ccc0 tx=0x7f708401ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f707e7fc640 1 -- 172.21.15.39:0/3211297846 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70840be3f0 con 0x7f708806c870 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f707e7fc640 1 -- 172.21.15.39:0/3211297846 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70840be550 con 0x7f708806c870 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f707e7fc640 1 -- 172.21.15.39:0/3211297846 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70840be8c0 con 0x7f708806c870 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70881ee380 con 0x7f708806c870 2024-09-13T17:04:15.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.967+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70880a6600 con 0x7f708806c870 2024-09-13T17:04:15.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.971+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70880c39a0 con 0x7f708806c870 2024-09-13T17:04:15.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.971+0000 7f707e7fc640 1 -- 172.21.15.39:0/3211297846 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f70840c7440 con 0x7f708806c870 2024-09-13T17:04:15.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.971+0000 7f707e7fc640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f706c066690 0x7f706c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:15.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.971+0000 7f707e7fc640 1 -- 172.21.15.39:0/3211297846 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7f708411f3a0 con 0x7f708806c870 2024-09-13T17:04:15.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.971+0000 7f708d324640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f706c066690 0x7f706c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:15.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.971+0000 7f708d324640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f706c066690 0x7f706c068b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f70880a6d30 tx=0x7f7078008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:15.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:15.975+0000 7f707e7fc640 1 -- 172.21.15.39:0/3211297846 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f70840ebcf0 con 0x7f708806c870 2024-09-13T17:04:16.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.175+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7088152a80 con 0x7f708806c870 2024-09-13T17:04:16.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.175+0000 7f707e7fc640 1 -- 172.21.15.39:0/3211297846 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f7088152a80 con 0x7f708806c870 2024-09-13T17:04:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:16.171 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:16.172 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f706c066690 msgr2=0x7f706c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f706c066690 0x7f706c068b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f70880a6d30 tx=0x7f7078008040 comp rx=0 tx=0).stop 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f708806c870 msgr2=0x7f7088070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f708806c870 0x7f7088070c00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f708401ccc0 tx=0x7f708401ac60 comp rx=0 tx=0).stop 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 shutdown_connections 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f706c066690 0x7f706c068b50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70880aabc0 0x7f7088150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f70880c3ee0 0x7f70881ea850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 --2- 172.21.15.39:0/3211297846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f708806c870 0x7f7088070c00 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 >> 172.21.15.39:0/3211297846 conn(0x7f7088069e10 msgr2=0x7f708806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 shutdown_connections 2024-09-13T17:04:16.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:16.179+0000 7f708fd89640 1 -- 172.21.15.39:0/3211297846 wait complete. 2024-09-13T17:04:16.191 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:04:18.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:18.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.331+0000 7f50b9886640 1 Processor -- start 2024-09-13T17:04:18.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.331+0000 7f50b9886640 1 -- start start 2024-09-13T17:04:18.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b9886640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b40c24c0 0x7f50b4157d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:18.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b9886640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b40c6510 con 0x7f50b40c0240 2024-09-13T17:04:18.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b9886640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b40c2950 con 0x7f50b40c6130 2024-09-13T17:04:18.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b9886640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b40c1c80 con 0x7f50b40c24c0 2024-09-13T17:04:18.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b27fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f50b40c0240 0x7f50b406ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:60908/0 (socket says 172.21.15.39:60908) 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b27fc640 1 -- 172.21.15.39:0/1925058906 learned_addr learned my addr 172.21.15.39:0/1925058906 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b37fe640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b40c24c0 0x7f50b4157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218311320 0 0) 0x7f50a8001120 con 0x7f50b40c0240 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f509c003530 con 0x7f50b40c0240 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006902016 0 0) 0x7f509c003530 con 0x7f50b40c0240 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b40c24c0 msgr2=0x7f50b4157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b40c24c0 0x7f50b4157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 >> v1:172.21.15.84:6789/0 conn(0x7f50b40c6130 legacy=0x7f50b4153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b40c2bc0 con 0x7f50b40c0240 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f50a8001810 con 0x7f50b40c0240 2024-09-13T17:04:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 >> v1:172.21.15.39:6789/0 conn(0x7f50b40c0240 legacy=0x7f50b406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f509c0039b0 0x7f509c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f509c0047d0 0x7f509c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b40c24c0 0x7f509c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b40c1c80 con 0x7f509c0039b0 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c00ba70 con 0x7f509c0047d0 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c00bc30 con 0x7f50b40c24c0 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b2ffd640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f509c0039b0 0x7f509c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b27fc640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f509c0047d0 0x7f509c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b2ffd640 1 -- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b40c24c0 msgr2=0x7f509c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b2ffd640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b40c24c0 0x7f509c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b2ffd640 1 -- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f509c0047d0 msgr2=0x7f509c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b2ffd640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f509c0047d0 0x7f509c004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.335+0000 7f50b2ffd640 1 -- 172.21.15.39:0/1925058906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c003530 con 0x7f509c0039b0 2024-09-13T17:04:18.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b2ffd640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f509c0039b0 0x7f509c003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f50a4001750 tx=0x7f50a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65172b3313cf45ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:18.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a40052d0 con 0x7f509c0039b0 2024-09-13T17:04:18.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a40a7050 con 0x7f509c0039b0 2024-09-13T17:04:18.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b1ffb640 1 -- 172.21.15.39:0/1925058906 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4003600 con 0x7f509c0039b0 2024-09-13T17:04:18.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 -- 172.21.15.39:0/1925058906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f509c0039b0 msgr2=0x7f509c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 --2- 172.21.15.39:0/1925058906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f509c0039b0 0x7f509c003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f50a4001750 tx=0x7f50a401c3c0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b37fe640 1 -- 172.21.15.39:0/1925058906 reap_dead start 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 -- 172.21.15.39:0/1925058906 shutdown_connections 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 -- 172.21.15.39:0/1925058906 >> 172.21.15.39:0/1925058906 conn(0x7f50b40c32a0 msgr2=0x7f50b40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 -- 172.21.15.39:0/1925058906 shutdown_connections 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 -- 172.21.15.39:0/1925058906 wait complete. 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 Processor -- start 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 -- start start 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f509c0039b0 0x7f50b4152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b4149a80 0x7f50b4149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b4070bd0 0x7f50b4071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b414a420 con 0x7f50b4149a80 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4070740 con 0x7f509c0039b0 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b9886640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41ef900 con 0x7f50b4070bd0 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b4149a80 0x7f50b4149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b4149a80 0x7f50b4149ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50706/0 (socket says 172.21.15.39:50706) 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b27fc640 1 -- 172.21.15.39:0/3486033685 learned_addr learned my addr 172.21.15.39:0/3486033685 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b2ffd640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f509c0039b0 0x7f50b4152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b37fe640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b4070bd0 0x7f50b4071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b27fc640 1 -- 172.21.15.39:0/3486033685 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b4070bd0 msgr2=0x7f50b4071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b27fc640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b4070bd0 0x7f50b4071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b27fc640 1 -- 172.21.15.39:0/3486033685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f509c0039b0 msgr2=0x7f50b4152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b27fc640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f509c0039b0 0x7f50b4152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b27fc640 1 -- 172.21.15.39:0/3486033685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b41efbc0 con 0x7f50b4149a80 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b37fe640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b4070bd0 0x7f50b4071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.339+0000 7f50b27fc640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b4149a80 0x7f50b4149ee0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f50a800d820 tx=0x7f50a800dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f509bfff640 1 -- 172.21.15.39:0/3486033685 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80074e0 con 0x7f50b4149a80 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f509bfff640 1 -- 172.21.15.39:0/3486033685 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a8007640 con 0x7f50b4149a80 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b41536f0 con 0x7f50b4149a80 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f509bfff640 1 -- 172.21.15.39:0/3486033685 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8004a10 con 0x7f50b4149a80 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b40664a0 con 0x7f50b4149a80 2024-09-13T17:04:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b406ae20 con 0x7f50b4149a80 2024-09-13T17:04:18.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f509bfff640 1 -- 172.21.15.39:0/3486033685 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f50a8004b70 con 0x7f50b4149a80 2024-09-13T17:04:18.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f509bfff640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50a0066690 0x7f50a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:18.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f509bfff640 1 -- 172.21.15.39:0/3486033685 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7f50a80846e0 con 0x7f50b4149a80 2024-09-13T17:04:18.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f50b2ffd640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50a0066690 0x7f50a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:18.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.343+0000 7f50b2ffd640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50a0066690 0x7f50a0068b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f509c0057b0 tx=0x7f50a40011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:18.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.347+0000 7f509bfff640 1 -- 172.21.15.39:0/3486033685 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f50a8051030 con 0x7f50b4149a80 2024-09-13T17:04:18.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.547+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50b40c0020 con 0x7f50b4149a80 2024-09-13T17:04:18.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.547+0000 7f509bfff640 1 -- 172.21.15.39:0/3486033685 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f50a8019020 con 0x7f50b4149a80 2024-09-13T17:04:18.544 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:18.544 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:18.545 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50a0066690 msgr2=0x7f50a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50a0066690 0x7f50a0068b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f509c0057b0 tx=0x7f50a40011a0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b4149a80 msgr2=0x7f50b4149ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b4149a80 0x7f50b4149ee0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f50a800d820 tx=0x7f50a800dc20 comp rx=0 tx=0).stop 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 shutdown_connections 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f50a0066690 0x7f50a0068b50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f50b4070bd0 0x7f50b4071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b4149a80 0x7f50b4149ee0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 --2- 172.21.15.39:0/3486033685 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f509c0039b0 0x7f50b4152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 >> 172.21.15.39:0/3486033685 conn(0x7f50b40c32a0 msgr2=0x7f50b40a7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 shutdown_connections 2024-09-13T17:04:18.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:18.551+0000 7f50b9886640 1 -- 172.21.15.39:0/3486033685 wait complete. 2024-09-13T17:04:18.562 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:04:20.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d2b9a640 1 Processor -- start 2024-09-13T17:04:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d2b9a640 1 -- start start 2024-09-13T17:04:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d2b9a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f56cc06c870 0x7f56cc0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d2b9a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc152a80 con 0x7f56cc0c40b0 2024-09-13T17:04:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d2b9a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc0c30f0 con 0x7f56cc0aabc0 2024-09-13T17:04:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d2b9a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc151e00 con 0x7f56cc06c870 2024-09-13T17:04:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d1137640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f56cc0c40b0 0x7f56cc15c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48212/0 (socket says 172.21.15.39:48212) 2024-09-13T17:04:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d1137640 1 -- 172.21.15.39:0/830420336 learned_addr learned my addr 172.21.15.39:0/830420336 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:04:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cbfff640 1 --2- 172.21.15.39:0/830420336 >> v2:172.21.15.84:3300/0 conn(0x7f56cc06c870 0x7f56cc0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761101740 0 0) 0x7f56c4001120 con 0x7f56cc0c40b0 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b8003530 con 0x7f56cc0c40b0 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050647975 0 0) 0x7f56cc0c30f0 con 0x7f56cc0aabc0 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56c4001120 con 0x7f56cc0aabc0 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3153111524 0 0) 0x7f56b8003530 con 0x7f56cc0c40b0 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 >> v2:172.21.15.84:3300/0 conn(0x7f56cc06c870 msgr2=0x7f56cc0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 --2- 172.21.15.39:0/830420336 >> v2:172.21.15.84:3300/0 conn(0x7f56cc06c870 0x7f56cc0aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 >> v1:172.21.15.84:6789/0 conn(0x7f56cc0aabc0 legacy=0x7f56cc0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc1520e0 con 0x7f56cc0c40b0 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f56c4001810 con 0x7f56cc0c40b0 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 >> v1:172.21.15.39:6789/0 conn(0x7f56cc0c40b0 legacy=0x7f56cc15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56b8003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f56b80047d0 0x7f56b8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56b80051e0 0x7f56b800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc151e00 con 0x7f56b8003b60 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b800c110 con 0x7f56b80051e0 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b800c2d0 con 0x7f56b80047d0 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d0936640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56b8003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d1137640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56b80051e0 0x7f56b800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56cbfff640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f56b80047d0 0x7f56b8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d0936640 1 -- 172.21.15.39:0/830420336 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f56b80047d0 msgr2=0x7f56b8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d0936640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f56b80047d0 0x7f56b8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d0936640 1 -- 172.21.15.39:0/830420336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56b80051e0 msgr2=0x7f56b800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d0936640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56b80051e0 0x7f56b800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d0936640 1 -- 172.21.15.39:0/830420336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b8003530 con 0x7f56b8003b60 2024-09-13T17:04:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d1137640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56b80051e0 0x7f56b800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:04:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.702+0000 7f56d1137640 1 -- 172.21.15.39:0/830420336 reap_dead start 2024-09-13T17:04:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d0936640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56b8003f60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0018d0 tx=0x7f56bc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62daa0ad736ca6a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0052d0 con 0x7f56b8003b60 2024-09-13T17:04:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc005450 con 0x7f56b8003b60 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56cb7fe640 1 -- 172.21.15.39:0/830420336 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc003840 con 0x7f56b8003b60 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d2b9a640 1 -- 172.21.15.39:0/830420336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 msgr2=0x7f56b8003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d2b9a640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56b8003f60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0018d0 tx=0x7f56bc01c530 comp rx=0 tx=0).stop 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d2b9a640 1 -- 172.21.15.39:0/830420336 shutdown_connections 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d2b9a640 1 --2- 172.21.15.39:0/830420336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56b8003f60 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d2b9a640 1 -- 172.21.15.39:0/830420336 >> 172.21.15.39:0/830420336 conn(0x7f56cc069e10 msgr2=0x7f56cc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d2b9a640 1 -- 172.21.15.39:0/830420336 shutdown_connections 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d2b9a640 1 -- 172.21.15.39:0/830420336 wait complete. 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d2b9a640 1 Processor -- start 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.706+0000 7f56d2b9a640 1 -- start start 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d2b9a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56cc1dd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d2b9a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc1de3f0 0x7f56cc06f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d2b9a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f56cc1dfc20 0x7f56cc1e0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d2b9a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc070970 con 0x7f56b8003b60 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d2b9a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0704e0 con 0x7f56cc1de3f0 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d2b9a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc071fe0 con 0x7f56cc1dfc20 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d0936640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56cc1dd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d0936640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56cc1dd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37848/0 (socket says 172.21.15.39:37848) 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d0936640 1 -- 172.21.15.39:0/443472389 learned_addr learned my addr 172.21.15.39:0/443472389 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56cbfff640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc1de3f0 0x7f56cc06f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d1137640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f56cc1dfc20 0x7f56cc1e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d0936640 1 -- 172.21.15.39:0/443472389 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f56cc1dfc20 msgr2=0x7f56cc1e0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d0936640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f56cc1dfc20 0x7f56cc1e0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d0936640 1 -- 172.21.15.39:0/443472389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc1de3f0 msgr2=0x7f56cc06f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d0936640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc1de3f0 0x7f56cc06f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d0936640 1 -- 172.21.15.39:0/443472389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc072560 con 0x7f56b8003b60 2024-09-13T17:04:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d0936640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56cc1dd960 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0ae040 tx=0x7f56b8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56c97fa640 1 -- 172.21.15.39:0/443472389 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0052d0 con 0x7f56b8003b60 2024-09-13T17:04:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56c97fa640 1 -- 172.21.15.39:0/443472389 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc0067e0 con 0x7f56b8003b60 2024-09-13T17:04:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc072280 con 0x7f56b8003b60 2024-09-13T17:04:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56c97fa640 1 -- 172.21.15.39:0/443472389 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc006b50 con 0x7f56b8003b60 2024-09-13T17:04:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc1ddea0 con 0x7f56b8003b60 2024-09-13T17:04:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc0610f0 con 0x7f56b8003b60 2024-09-13T17:04:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56c97fa640 1 -- 172.21.15.39:0/443472389 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f56bc0b2040 con 0x7f56b8003b60 2024-09-13T17:04:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56c97fa640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f56ac066690 0x7f56ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56cbfff640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f56ac066690 0x7f56ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56c97fa640 1 -- 172.21.15.39:0/443472389 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7f56bc11ea30 con 0x7f56b8003b60 2024-09-13T17:04:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.710+0000 7f56cbfff640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f56ac066690 0x7f56ac068b50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0c01c0 tx=0x7f56b400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.714+0000 7f56c97fa640 1 -- 172.21.15.39:0/443472389 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f56bc0eb430 con 0x7f56b8003b60 2024-09-13T17:04:20.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.918+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56cc0c3d90 con 0x7f56b8003b60 2024-09-13T17:04:20.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.918+0000 7f56c97fa640 1 -- 172.21.15.39:0/443472389 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f56bc0c2090 con 0x7f56b8003b60 2024-09-13T17:04:20.916 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:20.916 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:20.916 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:20.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f56ac066690 msgr2=0x7f56ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:20.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f56ac066690 0x7f56ac068b50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0c01c0 tx=0x7f56b400b040 comp rx=0 tx=0).stop 2024-09-13T17:04:20.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 msgr2=0x7f56cc1dd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:20.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56cc1dd960 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0ae040 tx=0x7f56b8005800 comp rx=0 tx=0).stop 2024-09-13T17:04:20.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 shutdown_connections 2024-09-13T17:04:20.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f56ac066690 0x7f56ac068b50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56cc1de3f0 0x7f56cc06f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f56cc1dfc20 0x7f56cc1e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 --2- 172.21.15.39:0/443472389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b8003b60 0x7f56cc1dd960 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:20.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 >> 172.21.15.39:0/443472389 conn(0x7f56cc069e10 msgr2=0x7f56cc0c4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:20.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 shutdown_connections 2024-09-13T17:04:20.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:20.922+0000 7f56d2b9a640 1 -- 172.21.15.39:0/443472389 wait complete. 2024-09-13T17:04:20.934 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:04:22.935 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7c5384640 1 Processor -- start 2024-09-13T17:04:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7c5384640 1 -- start start 2024-09-13T17:04:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7c5384640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7c014e0c0 0x7fc7c00a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7c5384640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c4740 0x7fc7c00c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7c5384640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c00c2780 con 0x7fc7c00c1f50 2024-09-13T17:04:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7c5384640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c00c2a60 con 0x7fc7c014e0c0 2024-09-13T17:04:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7c5384640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c00c2d40 con 0x7fc7c00c4740 2024-09-13T17:04:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7beffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7c014e0c0 0x7fc7c00a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7beffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7c014e0c0 0x7fc7c00a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37860/0 (socket says 172.21.15.39:37860) 2024-09-13T17:04:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7beffd640 1 -- 172.21.15.39:0/4095675306 learned_addr learned my addr 172.21.15.39:0/4095675306 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:23.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7bf7fe640 1 --2- 172.21.15.39:0/4095675306 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c4740 0x7fc7c00c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:23.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7beffd640 1 -- 172.21.15.39:0/4095675306 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c4740 msgr2=0x7fc7c00c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:23.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7beffd640 1 --2- 172.21.15.39:0/4095675306 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c4740 0x7fc7c00c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:23.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7beffd640 1 -- 172.21.15.39:0/4095675306 >> v1:172.21.15.84:6789/0 conn(0x7fc7c00c1f50 legacy=0x7fc7c006cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:23.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.074+0000 7fc7beffd640 1 -- 172.21.15.39:0/4095675306 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c00c3020 con 0x7fc7c014e0c0 2024-09-13T17:04:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7beffd640 1 --2- 172.21.15.39:0/4095675306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 0x7fc7c00a4e70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000bb10 tx=0x7fc7b001eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85654930e3bd4de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7be7fc640 1 -- 172.21.15.39:0/4095675306 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00a7a70 con 0x7fc7c014e0c0 2024-09-13T17:04:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7be7fc640 1 -- 172.21.15.39:0/4095675306 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b00a7bd0 con 0x7fc7c014e0c0 2024-09-13T17:04:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7be7fc640 1 -- 172.21.15.39:0/4095675306 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00af440 con 0x7fc7c014e0c0 2024-09-13T17:04:23.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 -- 172.21.15.39:0/4095675306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 msgr2=0x7fc7c00a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:23.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 --2- 172.21.15.39:0/4095675306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 0x7fc7c00a4e70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000bb10 tx=0x7fc7b001eb70 comp rx=0 tx=0).stop 2024-09-13T17:04:23.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 -- 172.21.15.39:0/4095675306 shutdown_connections 2024-09-13T17:04:23.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 --2- 172.21.15.39:0/4095675306 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c4740 0x7fc7c00c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:23.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 --2- 172.21.15.39:0/4095675306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 0x7fc7c00a4e70 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 -- 172.21.15.39:0/4095675306 >> 172.21.15.39:0/4095675306 conn(0x7fc7c0069e10 msgr2=0x7fc7c006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 -- 172.21.15.39:0/4095675306 shutdown_connections 2024-09-13T17:04:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 -- 172.21.15.39:0/4095675306 wait complete. 2024-09-13T17:04:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 Processor -- start 2024-09-13T17:04:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.078+0000 7fc7c5384640 1 -- start start 2024-09-13T17:04:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7c5384640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c1f50 0x7fc7c0153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7c5384640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c00c4740 0x7fc7c0147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7c5384640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 0x7fc7c01ed950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7c5384640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c0153b70 con 0x7fc7c014e0c0 2024-09-13T17:04:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7c5384640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c01526f0 con 0x7fc7c00c4740 2024-09-13T17:04:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7c5384640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c0152c50 con 0x7fc7c00c1f50 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7bf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 0x7fc7c01ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7bf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 0x7fc7c01ed950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37864/0 (socket says 172.21.15.39:37864) 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7bf7fe640 1 -- 172.21.15.39:0/2472889113 learned_addr learned my addr 172.21.15.39:0/2472889113 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7beffd640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c1f50 0x7fc7c0153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7b7fff640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c00c4740 0x7fc7c0147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7bf7fe640 1 -- 172.21.15.39:0/2472889113 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c1f50 msgr2=0x7fc7c0153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7bf7fe640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c1f50 0x7fc7c0153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7bf7fe640 1 -- 172.21.15.39:0/2472889113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c00c4740 msgr2=0x7fc7c0147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7bf7fe640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c00c4740 0x7fc7c0147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7bf7fe640 1 -- 172.21.15.39:0/2472889113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b000b770 con 0x7fc7c014e0c0 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7bf7fe640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 0x7fc7c01ed950 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800efc0 tx=0x7fc7b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7b77fe640 1 -- 172.21.15.39:0/2472889113 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8014480 con 0x7fc7c014e0c0 2024-09-13T17:04:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7b77fe640 1 -- 172.21.15.39:0/2472889113 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b800f040 con 0x7fc7c014e0c0 2024-09-13T17:04:23.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7b77fe640 1 -- 172.21.15.39:0/2472889113 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8014820 con 0x7fc7c014e0c0 2024-09-13T17:04:23.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c0153680 con 0x7fc7c014e0c0 2024-09-13T17:04:23.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c00a4f80 con 0x7fc7c014e0c0 2024-09-13T17:04:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c00a51a0 con 0x7fc7c014e0c0 2024-09-13T17:04:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7b77fe640 1 -- 172.21.15.39:0/2472889113 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7b8014980 con 0x7fc7c014e0c0 2024-09-13T17:04:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7b77fe640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7a0066690 0x7fc7a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7b77fe640 1 -- 172.21.15.39:0/2472889113 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 8357+0+0 (secure 0 0 0) 0x7fc7b80859a0 con 0x7fc7c014e0c0 2024-09-13T17:04:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.082+0000 7fc7beffd640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7a0066690 0x7fc7a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.086+0000 7fc7beffd640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7a0066690 0x7fc7a0068b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00b1000 tx=0x7fc7b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:23.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.090+0000 7fc7b77fe640 1 -- 172.21.15.39:0/2472889113 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc7b8052260 con 0x7fc7c014e0c0 2024-09-13T17:04:23.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.286+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7c006c8d0 con 0x7fc7c014e0c0 2024-09-13T17:04:23.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7b77fe640 1 -- 172.21.15.39:0/2472889113 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5932 (secure 0 0 0) 0x7fc7b8018020 con 0x7fc7c014e0c0 2024-09-13T17:04:23.286 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:23.286 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:03:03.021063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6585":{"gid":6585,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":24,"addr":"172.21.15.39:6835/611669592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":611669592},{"type":"v1","addr":"172.21.15.39:6835","nonce":611669592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:23.286 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 111 2024-09-13T17:04:23.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7a0066690 msgr2=0x7fc7a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:23.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7a0066690 0x7fc7a0068b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00b1000 tx=0x7fc7b0008040 comp rx=0 tx=0).stop 2024-09-13T17:04:23.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 msgr2=0x7fc7c01ed950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:23.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 0x7fc7c01ed950 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800efc0 tx=0x7fc7b800c490 comp rx=0 tx=0).stop 2024-09-13T17:04:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 shutdown_connections 2024-09-13T17:04:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7a0066690 0x7fc7a0068b50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7c00c4740 0x7fc7c0147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7c014e0c0 0x7fc7c01ed950 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 --2- 172.21.15.39:0/2472889113 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7c00c1f50 0x7fc7c0153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 >> 172.21.15.39:0/2472889113 conn(0x7fc7c0069e10 msgr2=0x7fc7c00c1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 shutdown_connections 2024-09-13T17:04:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:23.290+0000 7fc7c5384640 1 -- 172.21.15.39:0/2472889113 wait complete. 2024-09-13T17:04:23.305 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:04:25.306 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:25.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91af8c0640 1 Processor -- start 2024-09-13T17:04:25.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91af8c0640 1 -- start start 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91af8c0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f91a8071dd0 0x7f91a8154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91af8c0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a80c5a30 con 0x7f91a8151500 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91af8c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a81576a0 con 0x7f91a80c7800 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91af8c0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a8157980 con 0x7f91a8071dd0 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91ad65c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f91a8151500 0x7f91a80c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48228/0 (socket says 172.21.15.39:48228) 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91ad65c640 1 -- 172.21.15.39:0/1769012011 learned_addr learned my addr 172.21.15.39:0/1769012011 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91ace5b640 1 --2- 172.21.15.39:0/1769012011 >> v2:172.21.15.84:3300/0 conn(0x7f91a8071dd0 0x7f91a8154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f919bfff640 1 -- 172.21.15.39:0/1769012011 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292635696 0 0) 0x7f91a80c5a30 con 0x7f91a8151500 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f919bfff640 1 -- 172.21.15.39:0/1769012011 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9194003530 con 0x7f91a8151500 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91ace5b640 1 -- 172.21.15.39:0/1769012011 >> v1:172.21.15.84:6789/0 conn(0x7f91a80c7800 legacy=0x7f91a81545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91ace5b640 1 -- 172.21.15.39:0/1769012011 >> v1:172.21.15.39:6789/0 conn(0x7f91a8151500 legacy=0x7f91a80c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.438+0000 7f91ace5b640 1 -- 172.21.15.39:0/1769012011 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a8157c60 con 0x7f91a8071dd0 2024-09-13T17:04:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91ace5b640 1 --2- 172.21.15.39:0/1769012011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91a8071dd0 0x7f91a8154070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f91a40097f0 tx=0x7f91a401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4942581c207c81f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f919bfff640 1 -- 172.21.15.39:0/1769012011 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a40ab600 con 0x7f91a8071dd0 2024-09-13T17:04:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f919bfff640 1 -- 172.21.15.39:0/1769012011 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a40ab760 con 0x7f91a8071dd0 2024-09-13T17:04:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f919bfff640 1 -- 172.21.15.39:0/1769012011 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a40abad0 con 0x7f91a8071dd0 2024-09-13T17:04:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91af8c0640 1 -- 172.21.15.39:0/1769012011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91a8071dd0 msgr2=0x7f91a8154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91af8c0640 1 --2- 172.21.15.39:0/1769012011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91a8071dd0 0x7f91a8154070 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f91a40097f0 tx=0x7f91a401c3e0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91af8c0640 1 -- 172.21.15.39:0/1769012011 shutdown_connections 2024-09-13T17:04:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91af8c0640 1 --2- 172.21.15.39:0/1769012011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91a8071dd0 0x7f91a8154070 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91af8c0640 1 -- 172.21.15.39:0/1769012011 >> 172.21.15.39:0/1769012011 conn(0x7f91a80ab530 msgr2=0x7f91a80ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91af8c0640 1 -- 172.21.15.39:0/1769012011 shutdown_connections 2024-09-13T17:04:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91af8c0640 1 -- 172.21.15.39:0/1769012011 wait complete. 2024-09-13T17:04:25.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91af8c0640 1 Processor -- start 2024-09-13T17:04:25.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.442+0000 7f91af8c0640 1 -- start start 2024-09-13T17:04:25.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91af8c0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91a8071dd0 0x7f91a81ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91af8c0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91a80c7800 0x7f91a8145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91af8c0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91a8151500 0x7f91a81eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91af8c0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a8141080 con 0x7f91a80c7800 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91af8c0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a814adc0 con 0x7f91a8071dd0 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91af8c0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81531d0 con 0x7f91a8151500 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ace5b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91a80c7800 0x7f91a8145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ace5b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91a80c7800 0x7f91a8145ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37870/0 (socket says 172.21.15.39:37870) 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ace5b640 1 -- 172.21.15.39:0/2731003702 learned_addr learned my addr 172.21.15.39:0/2731003702 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ad65c640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91a8071dd0 0x7f91a81ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ade5d640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91a8151500 0x7f91a81eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ace5b640 1 -- 172.21.15.39:0/2731003702 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91a8151500 msgr2=0x7f91a81eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ace5b640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91a8151500 0x7f91a81eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ace5b640 1 -- 172.21.15.39:0/2731003702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91a8071dd0 msgr2=0x7f91a81ec7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ace5b640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91a8071dd0 0x7f91a81ec7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ace5b640 1 -- 172.21.15.39:0/2731003702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a4009420 con 0x7f91a80c7800 2024-09-13T17:04:25.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91ace5b640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91a80c7800 0x7f91a8145ea0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f91a40ad040 tx=0x7f91a401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:25.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f9199ffb640 1 -- 172.21.15.39:0/2731003702 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4004220 con 0x7f91a80c7800 2024-09-13T17:04:25.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f9199ffb640 1 -- 172.21.15.39:0/2731003702 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a40b9070 con 0x7f91a80c7800 2024-09-13T17:04:25.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f9199ffb640 1 -- 172.21.15.39:0/2731003702 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4003620 con 0x7f91a80c7800 2024-09-13T17:04:25.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a8153490 con 0x7f91a80c7800 2024-09-13T17:04:25.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a8153a50 con 0x7f91a80c7800 2024-09-13T17:04:25.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a80c4280 con 0x7f91a80c7800 2024-09-13T17:04:25.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f9199ffb640 1 -- 172.21.15.39:0/2731003702 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f91a4003780 con 0x7f91a80c7800 2024-09-13T17:04:25.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f9199ffb640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f918c0666c0 0x7f918c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.446+0000 7f9199ffb640 1 -- 172.21.15.39:0/2731003702 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f91a40bb070 con 0x7f91a80c7800 2024-09-13T17:04:25.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.450+0000 7f91ad65c640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f918c0666c0 0x7f918c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.450+0000 7f91ad65c640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f918c0666c0 0x7f918c068b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f919c005890 tx=0x7f919c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:25.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.454+0000 7f9199ffb640 1 -- 172.21.15.39:0/2731003702 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f91a40ebcb0 con 0x7f91a80c7800 2024-09-13T17:04:25.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.650+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91a8071bb0 con 0x7f91a80c7800 2024-09-13T17:04:25.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.650+0000 7f9199ffb640 1 -- 172.21.15.39:0/2731003702 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+5177 (secure 0 0 0) 0x7f91a40c4070 con 0x7f91a80c7800 2024-09-13T17:04:25.649 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:25.649 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:24.752830+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:25.650 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 112 2024-09-13T17:04:25.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f918c0666c0 msgr2=0x7f918c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f918c0666c0 0x7f918c068b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f919c005890 tx=0x7f919c009040 comp rx=0 tx=0).stop 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91a80c7800 msgr2=0x7f91a8145ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91a80c7800 0x7f91a8145ea0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f91a40ad040 tx=0x7f91a401ac20 comp rx=0 tx=0).stop 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 shutdown_connections 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f918c0666c0 0x7f918c068b80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91a8151500 0x7f91a81eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91a80c7800 0x7f91a8145ea0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 --2- 172.21.15.39:0/2731003702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91a8071dd0 0x7f91a81ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 >> 172.21.15.39:0/2731003702 conn(0x7f91a80ab530 msgr2=0x7f91a8070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 shutdown_connections 2024-09-13T17:04:25.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.654+0000 7f91af8c0640 1 -- 172.21.15.39:0/2731003702 wait complete. 2024-09-13T17:04:25.669 INFO:tasks.mds_thrash.fs.[a]:mds.a down, removed from mdsmap 2024-09-13T17:04:25.669 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:04:25.672 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:25.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f493e4a6640 1 Processor -- start 2024-09-13T17:04:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f493e4a6640 1 -- start start 2024-09-13T17:04:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f493e4a6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4938071b70 0x7f4938070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f493e4a6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493806b600 con 0x7f493814d8d0 2024-09-13T17:04:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f493e4a6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4938150520 con 0x7f49380714a0 2024-09-13T17:04:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f493e4a6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493806e4b0 con 0x7f4938071b70 2024-09-13T17:04:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4937fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f493814d8d0 0x7f49380c18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48232/0 (socket says 172.21.15.39:48232) 2024-09-13T17:04:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4937fff640 1 -- 172.21.15.39:0/2626650374 learned_addr learned my addr 172.21.15.39:0/2626650374 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:04:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f49377fe640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4938071b70 0x7f4938070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405563567 0 0) 0x7f493806b600 con 0x7f493814d8d0 2024-09-13T17:04:25.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4920003530 con 0x7f493814d8d0 2024-09-13T17:04:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4150496820 0 0) 0x7f4920003530 con 0x7f493814d8d0 2024-09-13T17:04:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4938071b70 msgr2=0x7f4938070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4938071b70 0x7f4938070590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 >> v1:172.21.15.84:6789/0 conn(0x7f49380714a0 legacy=0x7f4938070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493806fab0 con 0x7f493814d8d0 2024-09-13T17:04:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4928001680 con 0x7f493814d8d0 2024-09-13T17:04:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 >> v1:172.21.15.39:6789/0 conn(0x7f493814d8d0 legacy=0x7f49380c18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200039b0 0x7f4920003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49200047d0 0x7f4920004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4920005150 0x7f492000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493806e4b0 con 0x7f49200039b0 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492000bf80 con 0x7f4920005150 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492000c140 con 0x7f49200047d0 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4937fff640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200039b0 0x7f4920003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f49377fe640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49200047d0 0x7f4920004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f493ca43640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4920005150 0x7f492000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4937fff640 1 -- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49200047d0 msgr2=0x7f4920004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4937fff640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49200047d0 0x7f4920004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4937fff640 1 -- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4920005150 msgr2=0x7f492000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4937fff640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4920005150 0x7f492000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.794+0000 7f4937fff640 1 -- 172.21.15.39:0/2626650374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4920003530 con 0x7f49200039b0 2024-09-13T17:04:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f4937fff640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200039b0 0x7f4920003dd0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f492800c1c0 tx=0x7f492801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04ea9a1978e6567 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928005ae0 con 0x7f49200039b0 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928005c40 con 0x7f49200039b0 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f4936ffd640 1 -- 172.21.15.39:0/2626650374 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49280a5780 con 0x7f49200039b0 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493e4a6640 1 -- 172.21.15.39:0/2626650374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200039b0 msgr2=0x7f4920003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493e4a6640 1 --2- 172.21.15.39:0/2626650374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200039b0 0x7f4920003dd0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f492800c1c0 tx=0x7f492801cc10 comp rx=0 tx=0).stop 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493ca43640 1 -- 172.21.15.39:0/2626650374 reap_dead start 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493e4a6640 1 -- 172.21.15.39:0/2626650374 shutdown_connections 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493e4a6640 1 -- 172.21.15.39:0/2626650374 >> 172.21.15.39:0/2626650374 conn(0x7f49380c29e0 msgr2=0x7f49380c2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493e4a6640 1 -- 172.21.15.39:0/2626650374 shutdown_connections 2024-09-13T17:04:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493e4a6640 1 -- 172.21.15.39:0/2626650374 wait complete. 2024-09-13T17:04:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493e4a6640 1 Processor -- start 2024-09-13T17:04:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493e4a6640 1 -- start start 2024-09-13T17:04:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.798+0000 7f493e4a6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49200039b0 0x7f49380c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f493e4a6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381dda60 0x7f49381ddee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f493e4a6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49381436f0 0x7f4938143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f493e4a6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4938070170 con 0x7f49381dda60 2024-09-13T17:04:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f493e4a6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4938149920 con 0x7f49381436f0 2024-09-13T17:04:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f493e4a6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4938148440 con 0x7f49200039b0 2024-09-13T17:04:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f49377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381dda60 0x7f49381ddee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f49377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381dda60 0x7f49381ddee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37902/0 (socket says 172.21.15.39:37902) 2024-09-13T17:04:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f49377fe640 1 -- 172.21.15.39:0/1350260158 learned_addr learned my addr 172.21.15.39:0/1350260158 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f4937fff640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49200039b0 0x7f49380c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f493ca43640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49381436f0 0x7f4938143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f49377fe640 1 -- 172.21.15.39:0/1350260158 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49200039b0 msgr2=0x7f49380c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f49377fe640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49200039b0 0x7f49380c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f49377fe640 1 -- 172.21.15.39:0/1350260158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49381436f0 msgr2=0x7f4938143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f49377fe640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49381436f0 0x7f4938143ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f49377fe640 1 -- 172.21.15.39:0/1350260158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938147c10 con 0x7f49381dda60 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f49377fe640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381dda60 0x7f49381ddee0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f492c010770 tx=0x7f492c010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f4934ff9640 1 -- 172.21.15.39:0/1350260158 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00ecc0 con 0x7f49381dda60 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f4934ff9640 1 -- 172.21.15.39:0/1350260158 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00ee20 con 0x7f49381dda60 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4938149450 con 0x7f49381dda60 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f4934ff9640 1 -- 172.21.15.39:0/1350260158 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c004220 con 0x7f49381dda60 2024-09-13T17:04:25.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49381471a0 con 0x7f49381dda60 2024-09-13T17:04:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4938147da0 con 0x7f49381dda60 2024-09-13T17:04:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f4934ff9640 1 -- 172.21.15.39:0/1350260158 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f492c004380 con 0x7f49381dda60 2024-09-13T17:04:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f4934ff9640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f49240666c0 0x7f4924068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f4934ff9640 1 -- 172.21.15.39:0/1350260158 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f492c016070 con 0x7f49381dda60 2024-09-13T17:04:25.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f4937fff640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f49240666c0 0x7f4924068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:25.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.802+0000 7f4937fff640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f49240666c0 0x7f4924068b80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f492800c1c0 tx=0x7f49280a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:25.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:25.806+0000 7f4934ff9640 1 -- 172.21.15.39:0/1350260158 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f492c051ac0 con 0x7f49381dda60 2024-09-13T17:04:25.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.002+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49381de420 con 0x7f49381dda60 2024-09-13T17:04:25.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.002+0000 7f4934ff9640 1 -- 172.21.15.39:0/1350260158 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+5177 (secure 0 0 0) 0x7f492c0550e0 con 0x7f49381dda60 2024-09-13T17:04:26.000 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:26.000 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:24.752830+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:26.000 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 112 2024-09-13T17:04:26.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f49240666c0 msgr2=0x7f4924068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:26.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f49240666c0 0x7f4924068b80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f492800c1c0 tx=0x7f49280a7070 comp rx=0 tx=0).stop 2024-09-13T17:04:26.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381dda60 msgr2=0x7f49381ddee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:26.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381dda60 0x7f49381ddee0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f492c010770 tx=0x7f492c010c30 comp rx=0 tx=0).stop 2024-09-13T17:04:26.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 shutdown_connections 2024-09-13T17:04:26.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f49240666c0 0x7f4924068b80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:26.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49381436f0 0x7f4938143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:26.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49381dda60 0x7f49381ddee0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:26.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 --2- 172.21.15.39:0/1350260158 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f49200039b0 0x7f49380c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:26.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 >> 172.21.15.39:0/1350260158 conn(0x7f49380c29e0 msgr2=0x7f4938070d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:26.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 shutdown_connections 2024-09-13T17:04:26.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:26.006+0000 7f493e4a6640 1 -- 172.21.15.39:0/1350260158 wait complete. 2024-09-13T17:04:26.018 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 112, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 112, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6858": { "addr": "172.21.15.39:6837/1590039204", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 1590039204, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 1590039204, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6858, "incarnation": 112, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 46, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:04:24.752830+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6858 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 110, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6235": { "addr": "172.21.15.84:6835/263561599", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 263561599, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 263561599, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6235, "incarnation": 107, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 45, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:03:53.068981+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6235 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/440028459", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 440028459, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 440028459, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 96, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5561, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:04:28.019 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:28.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc8a1e3a640 1 Processor -- start 2024-09-13T17:04:28.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc8a1e3a640 1 -- start start 2024-09-13T17:04:28.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc8a1e3a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14a770 0x7fc89c150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:28.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc8a1e3a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc89c14ab50 0x7fc89c0713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:28.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc8a1e3a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c14b000 con 0x7fc89c0c6060 2024-09-13T17:04:28.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc8a1e3a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c06df80 con 0x7fc89c14ab50 2024-09-13T17:04:28.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc8a1e3a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0c1950 con 0x7fc89c14a770 2024-09-13T17:04:28.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc89bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc89c14ab50 0x7fc89c0713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:28.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14a770 0x7fc89c150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:28.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14a770 0x7fc89c150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:60404/0 (socket says 172.21.15.39:60404) 2024-09-13T17:04:28.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc89b7fe640 1 -- 172.21.15.39:0/2668811295 learned_addr learned my addr 172.21.15.39:0/2668811295 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:28.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc89affd640 1 --1- 172.21.15.39:0/2668811295 >> v1:172.21.15.84:6789/0 conn(0x7fc89c0c6060 0x7fc89c06ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:49898/0 (socket says 172.21.15.39:49898) 2024-09-13T17:04:28.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc89b7fe640 1 -- 172.21.15.39:0/2668811295 >> v2:172.21.15.84:3300/0 conn(0x7fc89c14ab50 msgr2=0x7fc89c0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:28.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc89b7fe640 1 --2- 172.21.15.39:0/2668811295 >> v2:172.21.15.84:3300/0 conn(0x7fc89c14ab50 0x7fc89c0713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc89b7fe640 1 -- 172.21.15.39:0/2668811295 >> v1:172.21.15.84:6789/0 conn(0x7fc89c0c6060 legacy=0x7fc89c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:28.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.162+0000 7fc89b7fe640 1 -- 172.21.15.39:0/2668811295 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0c4760 con 0x7fc89c14a770 2024-09-13T17:04:28.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.166+0000 7fc89b7fe640 1 --2- 172.21.15.39:0/2668811295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14a770 0x7fc89c150ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc890009870 tx=0x7fc89001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d76befcc80f0c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:28.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.166+0000 7fc89a7fc640 1 -- 172.21.15.39:0/2668811295 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900a6040 con 0x7fc89c14a770 2024-09-13T17:04:28.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.166+0000 7fc89a7fc640 1 -- 172.21.15.39:0/2668811295 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8900aa600 con 0x7fc89c14a770 2024-09-13T17:04:28.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.166+0000 7fc89a7fc640 1 -- 172.21.15.39:0/2668811295 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900aa990 con 0x7fc89c14a770 2024-09-13T17:04:28.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.166+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/2668811295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14a770 msgr2=0x7fc89c150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:28.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.166+0000 7fc8a1e3a640 1 --2- 172.21.15.39:0/2668811295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14a770 0x7fc89c150ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc890009870 tx=0x7fc89001c4a0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/2668811295 shutdown_connections 2024-09-13T17:04:28.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 --2- 172.21.15.39:0/2668811295 >> v2:172.21.15.84:3300/0 conn(0x7fc89c14ab50 0x7fc89c0713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 --2- 172.21.15.39:0/2668811295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14a770 0x7fc89c150ad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/2668811295 >> 172.21.15.39:0/2668811295 conn(0x7fc89c0c32a0 msgr2=0x7fc89c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:28.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/2668811295 shutdown_connections 2024-09-13T17:04:28.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/2668811295 wait complete. 2024-09-13T17:04:28.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 Processor -- start 2024-09-13T17:04:28.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- start start 2024-09-13T17:04:28.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0c6060 0x7fc89c1eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:28.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14ab50 0x7fc89c152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:28.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c14a370 0x7fc89c154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:28.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0c0ca0 con 0x7fc89c14a370 2024-09-13T17:04:28.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1ea1f0 con 0x7fc89c0c6060 2024-09-13T17:04:28.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1480d0 con 0x7fc89c14ab50 2024-09-13T17:04:28.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c14a370 0x7fc89c154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:28.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c14a370 0x7fc89c154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37904/0 (socket says 172.21.15.39:37904) 2024-09-13T17:04:28.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89bfff640 1 -- 172.21.15.39:0/3989475646 learned_addr learned my addr 172.21.15.39:0/3989475646 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89affd640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14ab50 0x7fc89c152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89b7fe640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0c6060 0x7fc89c1eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89bfff640 1 -- 172.21.15.39:0/3989475646 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14ab50 msgr2=0x7fc89c152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89bfff640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14ab50 0x7fc89c152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89bfff640 1 -- 172.21.15.39:0/3989475646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0c6060 msgr2=0x7fc89c1eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89bfff640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0c6060 0x7fc89c1eac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89bfff640 1 -- 172.21.15.39:0/3989475646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8900094d0 con 0x7fc89c14a370 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc89bfff640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c14a370 0x7fc89c154250 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc89400efc0 tx=0x7fc89400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc883fff640 1 -- 172.21.15.39:0/3989475646 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894014480 con 0x7fc89c14a370 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc883fff640 1 -- 172.21.15.39:0/3989475646 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89400f040 con 0x7fc89c14a370 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc883fff640 1 -- 172.21.15.39:0/3989475646 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894014820 con 0x7fc89c14a370 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c1eb140 con 0x7fc89c14a370 2024-09-13T17:04:28.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.170+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c0c09c0 con 0x7fc89c14a370 2024-09-13T17:04:28.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.174+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c0619c0 con 0x7fc89c14a370 2024-09-13T17:04:28.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.174+0000 7fc883fff640 1 -- 172.21.15.39:0/3989475646 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc894014980 con 0x7fc89c14a370 2024-09-13T17:04:28.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.174+0000 7fc883fff640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc888066690 0x7fc888068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:28.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.174+0000 7fc883fff640 1 -- 172.21.15.39:0/3989475646 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7fc894085a50 con 0x7fc89c14a370 2024-09-13T17:04:28.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.174+0000 7fc89b7fe640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc888066690 0x7fc888068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:28.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.174+0000 7fc89b7fe640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc888066690 0x7fc888068b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc8900ae000 tx=0x7fc8900a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:28.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.178+0000 7fc883fff640 1 -- 172.21.15.39:0/3989475646 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc894052350 con 0x7fc89c14a370 2024-09-13T17:04:28.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.410+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc89c071d30 con 0x7fc89c14a370 2024-09-13T17:04:28.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.410+0000 7fc883fff640 1 -- 172.21.15.39:0/3989475646 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+5178 (secure 0 0 0) 0x7fc894018020 con 0x7fc89c14a370 2024-09-13T17:04:28.409 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:28.409 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:28.409 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 115 2024-09-13T17:04:28.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc888066690 msgr2=0x7fc888068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:28.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc888066690 0x7fc888068b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc8900ae000 tx=0x7fc8900a7000 comp rx=0 tx=0).stop 2024-09-13T17:04:28.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c14a370 msgr2=0x7fc89c154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:28.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c14a370 0x7fc89c154250 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc89400efc0 tx=0x7fc89400c490 comp rx=0 tx=0).stop 2024-09-13T17:04:28.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 shutdown_connections 2024-09-13T17:04:28.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc888066690 0x7fc888068b50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc89c14a370 0x7fc89c154250 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc89c14ab50 0x7fc89c152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 --2- 172.21.15.39:0/3989475646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc89c0c6060 0x7fc89c1eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:28.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 >> 172.21.15.39:0/3989475646 conn(0x7fc89c0c32a0 msgr2=0x7fc89c065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:28.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 shutdown_connections 2024-09-13T17:04:28.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:28.414+0000 7fc8a1e3a640 1 -- 172.21.15.39:0/3989475646 wait complete. 2024-09-13T17:04:28.430 INFO:tasks.mds_thrash.fs.[a]:mds.c has gained rank=0, replacing gid=6585 2024-09-13T17:04:28.431 INFO:tasks.mds_thrash.fs.[a]:waiting for 8 secs before reviving mds.a 2024-09-13T17:04:29.059 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:04:29.062 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:04:30.033 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-13T17:04:30.034+0000 7f1c8fc19640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:04:36.431 INFO:tasks.mds_thrash.fs.[a]:reviving mds.a 2024-09-13T17:04:36.432 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:04:36.432 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:04:36.434 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:04:36.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:36.523 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T17:04:36.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e83f16640 1 Processor -- start 2024-09-13T17:04:36.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e83f16640 1 -- start start 2024-09-13T17:04:36.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e83f16640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0e7c0c2560 0x7f0e7c14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:36.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e83f16640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 0x7f0e7c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:36.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e83f16640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c06fb40 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e83f16640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c071730 con 0x7f0e7c0c2560 2024-09-13T17:04:36.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e83f16640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c150da0 con 0x7f0e7c14e630 2024-09-13T17:04:36.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e814b1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0e7c0c2560 0x7f0e7c14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:36.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e814b1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0e7c0c2560 0x7f0e7c14e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43160/0 (socket says 172.21.15.39:43160) 2024-09-13T17:04:36.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e814b1640 1 -- 172.21.15.39:0/3368382705 learned_addr learned my addr 172.21.15.39:0/3368382705 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:36.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e824b3640 1 --2- 172.21.15.39:0/3368382705 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 0x7f0e7c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:36.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e814b1640 1 -- 172.21.15.39:0/3368382705 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 msgr2=0x7f0e7c1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:36.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e814b1640 1 --2- 172.21.15.39:0/3368382705 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 0x7f0e7c1545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e814b1640 1 -- 172.21.15.39:0/3368382705 >> v1:172.21.15.84:6789/0 conn(0x7f0e7c0c2db0 legacy=0x7f0e7c061200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:36.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.590+0000 7f0e814b1640 1 -- 172.21.15.39:0/3368382705 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c0706c0 con 0x7f0e7c0c2560 2024-09-13T17:04:36.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e814b1640 1 --2- 172.21.15.39:0/3368382705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2560 0x7f0e7c14e0f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00bb10 tx=0x7f0e6c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ed81703de4573e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:36.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e80cb0640 1 -- 172.21.15.39:0/3368382705 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0a7b70 con 0x7f0e7c0c2560 2024-09-13T17:04:36.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e80cb0640 1 -- 172.21.15.39:0/3368382705 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c0a7cd0 con 0x7f0e7c0c2560 2024-09-13T17:04:36.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e80cb0640 1 -- 172.21.15.39:0/3368382705 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0af660 con 0x7f0e7c0c2560 2024-09-13T17:04:36.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e83f16640 1 -- 172.21.15.39:0/3368382705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2560 msgr2=0x7f0e7c14e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:36.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e83f16640 1 --2- 172.21.15.39:0/3368382705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2560 0x7f0e7c14e0f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00bb10 tx=0x7f0e6c01ec30 comp rx=0 tx=0).stop 2024-09-13T17:04:36.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e83f16640 1 -- 172.21.15.39:0/3368382705 shutdown_connections 2024-09-13T17:04:36.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e83f16640 1 --2- 172.21.15.39:0/3368382705 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 0x7f0e7c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e83f16640 1 --2- 172.21.15.39:0/3368382705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2560 0x7f0e7c14e0f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e83f16640 1 -- 172.21.15.39:0/3368382705 >> 172.21.15.39:0/3368382705 conn(0x7f0e7c069e10 msgr2=0x7f0e7c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:36.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e83f16640 1 -- 172.21.15.39:0/3368382705 shutdown_connections 2024-09-13T17:04:36.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.594+0000 7f0e83f16640 1 -- 172.21.15.39:0/3368382705 wait complete. 2024-09-13T17:04:36.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 Processor -- start 2024-09-13T17:04:36.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 -- start start 2024-09-13T17:04:36.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e7c0c2560 0x7f0e7c14d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:36.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2db0 0x7f0e7c14c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:36.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 0x7f0e7c14c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c5840 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c55d0 con 0x7f0e7c0c2560 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c52b0 con 0x7f0e7c14e630 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e814b1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2db0 0x7f0e7c14c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e814b1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2db0 0x7f0e7c14c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43172/0 (socket says 172.21.15.39:43172) 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e814b1640 1 -- 172.21.15.39:0/1666541842 learned_addr learned my addr 172.21.15.39:0/1666541842 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e81cb2640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e7c0c2560 0x7f0e7c14d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e824b3640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 0x7f0e7c14c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e814b1640 1 -- 172.21.15.39:0/1666541842 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 msgr2=0x7f0e7c14c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e814b1640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 0x7f0e7c14c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e814b1640 1 -- 172.21.15.39:0/1666541842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e7c0c2560 msgr2=0x7f0e7c14d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e814b1640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e7c0c2560 0x7f0e7c14d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e814b1640 1 -- 172.21.15.39:0/1666541842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c00b770 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e814b1640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2db0 0x7f0e7c14c220 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c01d2e0 tx=0x7f0e6c01b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e727fc640 1 -- 172.21.15.39:0/1666541842 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c006ee0 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e727fc640 1 -- 172.21.15.39:0/1666541842 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c0ba640 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c14ba20 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e727fc640 1 -- 172.21.15.39:0/1666541842 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0ba9f0 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.598+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c15a790 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.602+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c06cb40 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.602+0000 7f0e727fc640 1 -- 172.21.15.39:0/1666541842 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e6c0b6040 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.602+0000 7f0e727fc640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e60066690 0x7f0e60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.602+0000 7f0e727fc640 1 -- 172.21.15.39:0/1666541842 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f0e6c0a7e40 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.602+0000 7f0e81cb2640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e60066690 0x7f0e60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.602+0000 7f0e81cb2640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e60066690 0x7f0e60068b50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0e78000f70 tx=0x7f0e78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:36.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.606+0000 7f0e727fc640 1 -- 172.21.15.39:0/1666541842 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0e6c0eeaa0 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.842+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e7c14e0f0 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.842+0000 7f0e727fc640 1 -- 172.21.15.39:0/1666541842 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+5178 (secure 0 0 0) 0x7f0e6c0f20c0 con 0x7f0e7c0c2db0 2024-09-13T17:04:36.840 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:36.840 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:36.840 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 115 2024-09-13T17:04:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e60066690 msgr2=0x7f0e60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e60066690 0x7f0e60068b50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0e78000f70 tx=0x7f0e78009040 comp rx=0 tx=0).stop 2024-09-13T17:04:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2db0 msgr2=0x7f0e7c14c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2db0 0x7f0e7c14c220 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c01d2e0 tx=0x7f0e6c01b9d0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 shutdown_connections 2024-09-13T17:04:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e60066690 0x7f0e60068b50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e7c14e630 0x7f0e7c14c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7c0c2db0 0x7f0e7c14c220 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 --2- 172.21.15.39:0/1666541842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e7c0c2560 0x7f0e7c14d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 >> 172.21.15.39:0/1666541842 conn(0x7f0e7c069e10 msgr2=0x7f0e7c070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 shutdown_connections 2024-09-13T17:04:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:36.846+0000 7f0e83f16640 1 -- 172.21.15.39:0/1666541842 wait complete. 2024-09-13T17:04:36.861 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T17:04:38.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:39.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4e3db640 1 Processor -- start 2024-09-13T17:04:39.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4e3db640 1 -- start start 2024-09-13T17:04:39.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4e3db640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b48151500 0x7f5b480c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:39.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4e3db640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b48071dd0 0x7f5b48154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:39.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4e3db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5b480c47c0 0x7f5b481545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:39.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4e3db640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b480c5a30 con 0x7f5b480c47c0 2024-09-13T17:04:39.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4e3db640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b481576a0 con 0x7f5b48151500 2024-09-13T17:04:39.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4e3db640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b48157980 con 0x7f5b48071dd0 2024-09-13T17:04:39.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4c978640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5b480c47c0 0x7f5b481545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:39.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4c978640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5b480c47c0 0x7f5b481545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43188/0 (socket says 172.21.15.39:43188) 2024-09-13T17:04:39.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b4c978640 1 -- 172.21.15.39:0/1187504527 learned_addr learned my addr 172.21.15.39:0/1187504527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:39.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b477fe640 1 --2- 172.21.15.39:0/1187504527 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b48071dd0 0x7f5b48154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:39.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.010+0000 7f5b47fff640 1 --2- 172.21.15.39:0/1187504527 >> v2:172.21.15.84:3300/0 conn(0x7f5b48151500 0x7f5b480c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:39.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4c978640 1 -- 172.21.15.39:0/1187504527 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b48071dd0 msgr2=0x7f5b48154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:39.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4c978640 1 --2- 172.21.15.39:0/1187504527 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b48071dd0 0x7f5b48154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4c978640 1 -- 172.21.15.39:0/1187504527 >> v2:172.21.15.84:3300/0 conn(0x7f5b48151500 msgr2=0x7f5b480c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:39.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4c978640 1 --2- 172.21.15.39:0/1187504527 >> v2:172.21.15.84:3300/0 conn(0x7f5b48151500 0x7f5b480c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4c978640 1 -- 172.21.15.39:0/1187504527 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b48157c60 con 0x7f5b480c47c0 2024-09-13T17:04:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4c978640 1 --2- 172.21.15.39:0/1187504527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b480c47c0 0x7f5b481545b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5b480a9470 tx=0x7f5b4001d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f85d505b8b31eba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b46ffd640 1 -- 172.21.15.39:0/1187504527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b400a57d0 con 0x7f5b480c47c0 2024-09-13T17:04:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b46ffd640 1 -- 172.21.15.39:0/1187504527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b400a5930 con 0x7f5b480c47c0 2024-09-13T17:04:39.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b46ffd640 1 -- 172.21.15.39:0/1187504527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b400a5cc0 con 0x7f5b480c47c0 2024-09-13T17:04:39.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4e3db640 1 -- 172.21.15.39:0/1187504527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b480c47c0 msgr2=0x7f5b481545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:39.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/1187504527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b480c47c0 0x7f5b481545b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5b480a9470 tx=0x7f5b4001d390 comp rx=0 tx=0).stop 2024-09-13T17:04:39.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4e3db640 1 -- 172.21.15.39:0/1187504527 shutdown_connections 2024-09-13T17:04:39.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/1187504527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b480c47c0 0x7f5b481545b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/1187504527 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b48071dd0 0x7f5b48154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/1187504527 >> v2:172.21.15.84:3300/0 conn(0x7f5b48151500 0x7f5b480c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4e3db640 1 -- 172.21.15.39:0/1187504527 >> 172.21.15.39:0/1187504527 conn(0x7f5b480ab530 msgr2=0x7f5b480ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:39.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4e3db640 1 -- 172.21.15.39:0/1187504527 shutdown_connections 2024-09-13T17:04:39.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.014+0000 7f5b4e3db640 1 -- 172.21.15.39:0/1187504527 wait complete. 2024-09-13T17:04:39.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 Processor -- start 2024-09-13T17:04:39.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 -- start start 2024-09-13T17:04:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b48071dd0 0x7f5b4814a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b480c47c0 0x7f5b48147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b48151500 0x7f5b48147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4814b7e0 con 0x7f5b48071dd0 2024-09-13T17:04:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b48151fb0 con 0x7f5b48151500 2024-09-13T17:04:39.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b481ed190 con 0x7f5b480c47c0 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b48071dd0 0x7f5b4814a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b48071dd0 0x7f5b4814a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43194/0 (socket says 172.21.15.39:43194) 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b47fff640 1 -- 172.21.15.39:0/425767855 learned_addr learned my addr 172.21.15.39:0/425767855 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b477fe640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b480c47c0 0x7f5b48147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4c978640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b48151500 0x7f5b48147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b47fff640 1 -- 172.21.15.39:0/425767855 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b480c47c0 msgr2=0x7f5b48147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b47fff640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b480c47c0 0x7f5b48147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b47fff640 1 -- 172.21.15.39:0/425767855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b48151500 msgr2=0x7f5b48147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b47fff640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b48151500 0x7f5b48147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b47fff640 1 -- 172.21.15.39:0/425767855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b40009b30 con 0x7f5b48071dd0 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b47fff640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b48071dd0 0x7f5b4814a9d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800e870 tx=0x7f5b3800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b44ff9640 1 -- 172.21.15.39:0/425767855 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38009510 con 0x7f5b48071dd0 2024-09-13T17:04:39.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b44ff9640 1 -- 172.21.15.39:0/425767855 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3800eec0 con 0x7f5b48071dd0 2024-09-13T17:04:39.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b48152250 con 0x7f5b48071dd0 2024-09-13T17:04:39.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b44ff9640 1 -- 172.21.15.39:0/425767855 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3800cb40 con 0x7f5b48071dd0 2024-09-13T17:04:39.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4814af10 con 0x7f5b48071dd0 2024-09-13T17:04:39.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.018+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b480711d0 con 0x7f5b48071dd0 2024-09-13T17:04:39.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.022+0000 7f5b44ff9640 1 -- 172.21.15.39:0/425767855 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b3800ccc0 con 0x7f5b48071dd0 2024-09-13T17:04:39.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.022+0000 7f5b44ff9640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b34066690 0x7f5b34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:39.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.022+0000 7f5b44ff9640 1 -- 172.21.15.39:0/425767855 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f5b38084d20 con 0x7f5b48071dd0 2024-09-13T17:04:39.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.022+0000 7f5b477fe640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b34066690 0x7f5b34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:39.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.022+0000 7f5b477fe640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b34066690 0x7f5b34068b50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5b480c4360 tx=0x7f5b3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:39.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.026+0000 7f5b44ff9640 1 -- 172.21.15.39:0/425767855 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b38051620 con 0x7f5b48071dd0 2024-09-13T17:04:39.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.274+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b48071bb0 con 0x7f5b48071dd0 2024-09-13T17:04:39.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.274+0000 7f5b44ff9640 1 -- 172.21.15.39:0/425767855 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f5b38021090 con 0x7f5b48071dd0 2024-09-13T17:04:39.271 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:39.272 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:39.272 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-13T17:04:39.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b34066690 msgr2=0x7f5b34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:39.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b34066690 0x7f5b34068b50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5b480c4360 tx=0x7f5b3c008040 comp rx=0 tx=0).stop 2024-09-13T17:04:39.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b48071dd0 msgr2=0x7f5b4814a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:39.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b48071dd0 0x7f5b4814a9d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800e870 tx=0x7f5b3800ec70 comp rx=0 tx=0).stop 2024-09-13T17:04:39.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 shutdown_connections 2024-09-13T17:04:39.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b34066690 0x7f5b34068b50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b48151500 0x7f5b48147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b480c47c0 0x7f5b48147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 --2- 172.21.15.39:0/425767855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b48071dd0 0x7f5b4814a9d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:39.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 >> 172.21.15.39:0/425767855 conn(0x7f5b480ab530 msgr2=0x7f5b4814f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:39.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 shutdown_connections 2024-09-13T17:04:39.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:39.278+0000 7f5b4e3db640 1 -- 172.21.15.39:0/425767855 wait complete. 2024-09-13T17:04:39.292 INFO:tasks.mds_thrash.fs.[a]:mds.a reported in up:standby state 2024-09-13T17:04:39.293 INFO:tasks.mds_thrash.fs.[a]:waiting for 14 secs before thrashing 2024-09-13T17:04:53.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:53.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdf04ef4640 1 Processor -- start 2024-09-13T17:04:53.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdf04ef4640 1 -- start start 2024-09-13T17:04:53.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdf04ef4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf00065ee0 0x7fdf0014f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdf04ef4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf000c0a90 0x7fdf000c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdf04ef4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0006ddf0 con 0x7fdf000c2ef0 2024-09-13T17:04:53.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdf04ef4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf000c53d0 con 0x7fdf00065ee0 2024-09-13T17:04:53.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdf04ef4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000c5950 con 0x7fdf000c0a90 2024-09-13T17:04:53.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefed9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdf000c2ef0 0x7fdf000c22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50428/0 (socket says 172.21.15.39:50428) 2024-09-13T17:04:53.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefed9d640 1 -- 172.21.15.39:0/1665333530 learned_addr learned my addr 172.21.15.39:0/1665333530 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:04:53.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefdd9b640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf000c0a90 0x7fdf000c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefe59c640 1 --2- 172.21.15.39:0/1665333530 >> v2:172.21.15.84:3300/0 conn(0x7fdf00065ee0 0x7fdf0014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381542334 0 0) 0x7fdf0006ddf0 con 0x7fdf000c2ef0 2024-09-13T17:04:53.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdee8007940 con 0x7fdf000c2ef0 2024-09-13T17:04:53.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445667049 0 0) 0x7fdee8007940 con 0x7fdf000c2ef0 2024-09-13T17:04:53.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf000c0a90 msgr2=0x7fdf000c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:53.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf000c0a90 0x7fdf000c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 >> v2:172.21.15.84:3300/0 conn(0x7fdf00065ee0 msgr2=0x7fdf0014f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:53.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 --2- 172.21.15.39:0/1665333530 >> v2:172.21.15.84:3300/0 conn(0x7fdf00065ee0 0x7fdf0014f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0006e040 con 0x7fdf000c2ef0 2024-09-13T17:04:53.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fdef8001a00 con 0x7fdf000c2ef0 2024-09-13T17:04:53.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 >> v1:172.21.15.39:6789/0 conn(0x7fdf000c2ef0 legacy=0x7fdf000c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf000c0a90 0x7fdee8007d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdee8008310 0x7fdee8008be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdee8009120 0x7fdee80095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000c53d0 con 0x7fdee8008310 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000c5950 con 0x7fdee8009120 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee800fda0 con 0x7fdf000c0a90 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefdd9b640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdee8008310 0x7fdee8008be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefe59c640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf000c0a90 0x7fdee8007d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefed9d640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdee8009120 0x7fdee80095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefdd9b640 1 -- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf000c0a90 msgr2=0x7fdee8007d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefdd9b640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf000c0a90 0x7fdee8007d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefdd9b640 1 -- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdee8009120 msgr2=0x7fdee80095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:53.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefdd9b640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdee8009120 0x7fdee80095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefdd9b640 1 -- 172.21.15.39:0/1665333530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee8007940 con 0x7fdee8008310 2024-09-13T17:04:53.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.441+0000 7fdefed9d640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdee8009120 0x7fdee80095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:04:53.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdefdd9b640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdee8008310 0x7fdee8008be0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009ab0 tx=0x7fdef401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a01cf31118dac25 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:53.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40a5640 con 0x7fdee8008310 2024-09-13T17:04:53.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef40a57a0 con 0x7fdee8008310 2024-09-13T17:04:53.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdefd59a640 1 -- 172.21.15.39:0/1665333530 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40a5b50 con 0x7fdee8008310 2024-09-13T17:04:53.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdf04ef4640 1 -- 172.21.15.39:0/1665333530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdee8008310 msgr2=0x7fdee8008be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdf04ef4640 1 --2- 172.21.15.39:0/1665333530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdee8008310 0x7fdee8008be0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009ab0 tx=0x7fdef401c9f0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdefed9d640 1 -- 172.21.15.39:0/1665333530 reap_dead start 2024-09-13T17:04:53.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdf04ef4640 1 -- 172.21.15.39:0/1665333530 shutdown_connections 2024-09-13T17:04:53.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdf04ef4640 1 -- 172.21.15.39:0/1665333530 >> 172.21.15.39:0/1665333530 conn(0x7fdf000c3310 msgr2=0x7fdf000c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:53.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdf04ef4640 1 -- 172.21.15.39:0/1665333530 shutdown_connections 2024-09-13T17:04:53.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.445+0000 7fdf04ef4640 1 -- 172.21.15.39:0/1665333530 wait complete. 2024-09-13T17:04:53.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 Processor -- start 2024-09-13T17:04:53.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 -- start start 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdee8008310 0x7fdf001e5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf000ac3c0 0x7fdf00152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf00156010 0x7fdf000ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf00154f30 con 0x7fdf000ac3c0 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001e54a0 con 0x7fdf00156010 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf000ac840 con 0x7fdee8008310 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefdd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf000ac3c0 0x7fdf00152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefdd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf000ac3c0 0x7fdf00152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46640/0 (socket says 172.21.15.39:46640) 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefdd9b640 1 -- 172.21.15.39:0/4121810369 learned_addr learned my addr 172.21.15.39:0/4121810369 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefe59c640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdee8008310 0x7fdf001e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefed9d640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf00156010 0x7fdf000ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefdd9b640 1 -- 172.21.15.39:0/4121810369 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdee8008310 msgr2=0x7fdf001e5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefdd9b640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdee8008310 0x7fdf001e5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefdd9b640 1 -- 172.21.15.39:0/4121810369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf00156010 msgr2=0x7fdf000ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefdd9b640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf00156010 0x7fdf000ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefdd9b640 1 -- 172.21.15.39:0/4121810369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf001e4e50 con 0x7fdf000ac3c0 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdefdd9b640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf000ac3c0 0x7fdf00152f60 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009ab0 tx=0x7fdee8009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdeeeffd640 1 -- 172.21.15.39:0/4121810369 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4004230 con 0x7fdf000ac3c0 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdeeeffd640 1 -- 172.21.15.39:0/4121810369 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef4004390 con 0x7fdf000ac3c0 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdeeeffd640 1 -- 172.21.15.39:0/4121810369 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40ad850 con 0x7fdf000ac3c0 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf001e5130 con 0x7fdf000ac3c0 2024-09-13T17:04:53.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf000acb20 con 0x7fdf000ac3c0 2024-09-13T17:04:53.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.449+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf001e5630 con 0x7fdf000ac3c0 2024-09-13T17:04:53.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.453+0000 7fdeeeffd640 1 -- 172.21.15.39:0/4121810369 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdef40b4040 con 0x7fdf000ac3c0 2024-09-13T17:04:53.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.453+0000 7fdeeeffd640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdee0066690 0x7fdee0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.453+0000 7fdeeeffd640 1 -- 172.21.15.39:0/4121810369 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7fdef411f2a0 con 0x7fdf000ac3c0 2024-09-13T17:04:53.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.453+0000 7fdefe59c640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdee0066690 0x7fdee0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.453+0000 7fdefe59c640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdee0066690 0x7fdee0068b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009c60 tx=0x7fdef0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:53.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.457+0000 7fdeeeffd640 1 -- 172.21.15.39:0/4121810369 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdef40ebbc0 con 0x7fdf000ac3c0 2024-09-13T17:04:53.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.681+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf000c5f00 con 0x7fdf000ac3c0 2024-09-13T17:04:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdeeeffd640 1 -- 172.21.15.39:0/4121810369 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5936 (secure 0 0 0) 0x7fdef40ef1e0 con 0x7fdf000ac3c0 2024-09-13T17:04:53.682 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:53.682 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-13T17:04:53.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdee0066690 msgr2=0x7fdee0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdee0066690 0x7fdee0068b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009c60 tx=0x7fdef0009190 comp rx=0 tx=0).stop 2024-09-13T17:04:53.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf000ac3c0 msgr2=0x7fdf00152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf000ac3c0 0x7fdf00152f60 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009ab0 tx=0x7fdee8009c10 comp rx=0 tx=0).stop 2024-09-13T17:04:53.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 shutdown_connections 2024-09-13T17:04:53.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdee0066690 0x7fdee0068b50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf00156010 0x7fdf000ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf000ac3c0 0x7fdf00152f60 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 --2- 172.21.15.39:0/4121810369 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdee8008310 0x7fdf001e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 >> 172.21.15.39:0/4121810369 conn(0x7fdf000c3310 msgr2=0x7fdf000c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:53.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 shutdown_connections 2024-09-13T17:04:53.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.685+0000 7fdf04ef4640 1 -- 172.21.15.39:0/4121810369 wait complete. 2024-09-13T17:04:53.702 INFO:tasks.mds_thrash.fs.[a]:kill mds.c (rank=0) 2024-09-13T17:04:53.702 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:04:53.703 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:04:53.713 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:04:53.713 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:53.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.893+0000 7f4a95435640 1 Processor -- start 2024-09-13T17:04:53.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.893+0000 7f4a95435640 1 -- start start 2024-09-13T17:04:53.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.893+0000 7f4a95435640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a900c21d0 0x7f4a900aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.893+0000 7f4a95435640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4a9014b850 0x7f4a9014bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.893+0000 7f4a95435640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a900c3ae0 con 0x7f4a9006aac0 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.893+0000 7f4a95435640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a900ab340 con 0x7f4a9014b850 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.893+0000 7f4a95435640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900abb30 con 0x7f4a900c21d0 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8e7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4a9006aac0 0x7f4a90151c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50430/0 (socket says 172.21.15.39:50430) 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8e7fc640 1 -- 172.21.15.39:0/438751917 learned_addr learned my addr 172.21.15.39:0/438751917 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8effd640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a900c21d0 0x7f4a900aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8f7fe640 1 --2- 172.21.15.39:0/438751917 >> v2:172.21.15.84:3300/0 conn(0x7f4a9014b850 0x7f4a9014bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631461624 0 0) 0x7f4a900c3ae0 con 0x7f4a9006aac0 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a78003530 con 0x7f4a9006aac0 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3667667257 0 0) 0x7f4a78003530 con 0x7f4a9006aac0 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a900c21d0 msgr2=0x7f4a900aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a900c21d0 0x7f4a900aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 >> v2:172.21.15.84:3300/0 conn(0x7f4a9014b850 msgr2=0x7f4a9014bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 --2- 172.21.15.39:0/438751917 >> v2:172.21.15.84:3300/0 conn(0x7f4a9014b850 0x7f4a9014bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a900abe10 con 0x7f4a9006aac0 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4a84001680 con 0x7f4a9006aac0 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 >> v1:172.21.15.39:6789/0 conn(0x7f4a9006aac0 legacy=0x7f4a90151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a78003980 0x7f4a78003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a900c21d0 0x7f4a780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a78004d10 0x7f4a78005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900ab340 con 0x7f4a78004d10 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900abb30 con 0x7f4a78003980 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7800b990 con 0x7f4a900c21d0 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8f7fe640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a78004d10 0x7f4a78005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8effd640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a78003980 0x7f4a78003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8e7fc640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a900c21d0 0x7f4a780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8f7fe640 1 -- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a900c21d0 msgr2=0x7f4a780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:53.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8f7fe640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a900c21d0 0x7f4a780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8f7fe640 1 -- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a78003980 msgr2=0x7f4a78003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:53.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8f7fe640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a78003980 0x7f4a78003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8f7fe640 1 -- 172.21.15.39:0/438751917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a78003530 con 0x7f4a78004d10 2024-09-13T17:04:53.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.897+0000 7f4a8effd640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a78003980 0x7f4a78003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:04:53.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a78004d10 0x7f4a78005190 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009fc0 tx=0x7f4a8801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520e8faf2bbd705a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:53.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a880a5bc0 con 0x7f4a78004d10 2024-09-13T17:04:53.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8800f040 con 0x7f4a78004d10 2024-09-13T17:04:53.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8dffb640 1 -- 172.21.15.39:0/438751917 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a880ad440 con 0x7f4a78004d10 2024-09-13T17:04:53.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 -- 172.21.15.39:0/438751917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a78004d10 msgr2=0x7f4a78005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 --2- 172.21.15.39:0/438751917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a78004d10 0x7f4a78005190 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009fc0 tx=0x7f4a8801ccc0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 -- 172.21.15.39:0/438751917 reap_dead start 2024-09-13T17:04:53.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 -- 172.21.15.39:0/438751917 shutdown_connections 2024-09-13T17:04:53.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 -- 172.21.15.39:0/438751917 >> 172.21.15.39:0/438751917 conn(0x7f4a90069e10 msgr2=0x7f4a9006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:53.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 -- 172.21.15.39:0/438751917 shutdown_connections 2024-09-13T17:04:53.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 -- 172.21.15.39:0/438751917 wait complete. 2024-09-13T17:04:53.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 Processor -- start 2024-09-13T17:04:53.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 -- start start 2024-09-13T17:04:53.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a90143480 0x7f4a90143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a901543a0 0x7f4a90143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a90144360 0x7f4a90151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900c05f0 con 0x7f4a90144360 2024-09-13T17:04:53.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a901465a0 con 0x7f4a901543a0 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a95435640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a901e6dd0 con 0x7f4a90143480 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a90144360 0x7f4a90151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a90144360 0x7f4a90151670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46666/0 (socket says 172.21.15.39:46666) 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 -- 172.21.15.39:0/160720991 learned_addr learned my addr 172.21.15.39:0/160720991 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8effd640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a90143480 0x7f4a90143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8e7fc640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a901543a0 0x7f4a90143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 -- 172.21.15.39:0/160720991 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a90143480 msgr2=0x7f4a90143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a90143480 0x7f4a90143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 -- 172.21.15.39:0/160720991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a901543a0 msgr2=0x7f4a90143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a901543a0 0x7f4a90143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.901+0000 7f4a8f7fe640 1 -- 172.21.15.39:0/160720991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a90145d00 con 0x7f4a90144360 2024-09-13T17:04:53.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a8f7fe640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a90144360 0x7f4a90151670 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4a88002240 tx=0x7f4a88009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:53.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a777fe640 1 -- 172.21.15.39:0/160720991 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a880a5ea0 con 0x7f4a90144360 2024-09-13T17:04:53.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a777fe640 1 -- 172.21.15.39:0/160720991 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a880039a0 con 0x7f4a90144360 2024-09-13T17:04:53.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a90145fe0 con 0x7f4a90144360 2024-09-13T17:04:53.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a777fe640 1 -- 172.21.15.39:0/160720991 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88003d10 con 0x7f4a90144360 2024-09-13T17:04:53.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a90151bb0 con 0x7f4a90144360 2024-09-13T17:04:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a900c3560 con 0x7f4a90144360 2024-09-13T17:04:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a777fe640 1 -- 172.21.15.39:0/160720991 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4a880b4040 con 0x7f4a90144360 2024-09-13T17:04:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a777fe640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a7c0666c0 0x7f4a7c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a777fe640 1 -- 172.21.15.39:0/160720991 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f4a8811e900 con 0x7f4a90144360 2024-09-13T17:04:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a8effd640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a7c0666c0 0x7f4a7c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.905+0000 7f4a8effd640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a7c0666c0 0x7f4a7c068b80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4a800099b0 tx=0x7f4a80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:53.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:53.913+0000 7f4a777fe640 1 -- 172.21.15.39:0/160720991 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4a880eb2b0 con 0x7f4a90144360 2024-09-13T17:04:54.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a901e6f10 con 0x7f4a90144360 2024-09-13T17:04:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a777fe640 1 -- 172.21.15.39:0/160720991 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f4a880b2020 con 0x7f4a90144360 2024-09-13T17:04:54.143 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:54.143 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:54.144 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-13T17:04:54.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a7c0666c0 msgr2=0x7f4a7c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:54.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a7c0666c0 0x7f4a7c068b80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4a800099b0 tx=0x7f4a80008040 comp rx=0 tx=0).stop 2024-09-13T17:04:54.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a90144360 msgr2=0x7f4a90151670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:54.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a90144360 0x7f4a90151670 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4a88002240 tx=0x7f4a88009750 comp rx=0 tx=0).stop 2024-09-13T17:04:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 shutdown_connections 2024-09-13T17:04:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a7c0666c0 0x7f4a7c068b80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a901543a0 0x7f4a90143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a90144360 0x7f4a90151670 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 --2- 172.21.15.39:0/160720991 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a90143480 0x7f4a90143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 >> 172.21.15.39:0/160720991 conn(0x7f4a90069e10 msgr2=0x7f4a900c60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 shutdown_connections 2024-09-13T17:04:54.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:54.145+0000 7f4a95435640 1 -- 172.21.15.39:0/160720991 wait complete. 2024-09-13T17:04:54.164 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:04:56.165 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.305+0000 7f80e9fda640 1 Processor -- start 2024-09-13T17:04:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.305+0000 7f80e9fda640 1 -- start start 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.305+0000 7f80e9fda640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f80e414e0c0 0x7f80e40a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.305+0000 7f80e9fda640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f80e40c1f50 0x7f80e406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.305+0000 7f80e9fda640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 0x7f80e40c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.305+0000 7f80e9fda640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e40c2780 con 0x7f80e414e0c0 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.305+0000 7f80e9fda640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e40c2a60 con 0x7f80e40c1f50 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.305+0000 7f80e9fda640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e40c2d40 con 0x7f80e406d300 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f80e414e0c0 0x7f80e40a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f80e414e0c0 0x7f80e40a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46670/0 (socket says 172.21.15.39:46670) 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e37fe640 1 -- 172.21.15.39:0/367671216 learned_addr learned my addr 172.21.15.39:0/367671216 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e3fff640 1 --2- 172.21.15.39:0/367671216 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 0x7f80e40c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80dbfff640 1 --2- 172.21.15.39:0/367671216 >> v2:172.21.15.84:3300/0 conn(0x7f80e40c1f50 0x7f80e406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e37fe640 1 -- 172.21.15.39:0/367671216 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 msgr2=0x7f80e40c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e37fe640 1 --2- 172.21.15.39:0/367671216 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 0x7f80e40c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e37fe640 1 -- 172.21.15.39:0/367671216 >> v2:172.21.15.84:3300/0 conn(0x7f80e40c1f50 msgr2=0x7f80e406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:04:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e37fe640 1 --2- 172.21.15.39:0/367671216 >> v2:172.21.15.84:3300/0 conn(0x7f80e40c1f50 0x7f80e406cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e37fe640 1 -- 172.21.15.39:0/367671216 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40c3020 con 0x7f80e414e0c0 2024-09-13T17:04:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e3fff640 1 --2- 172.21.15.39:0/367671216 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 0x7f80e40c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:04:56.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e37fe640 1 --2- 172.21.15.39:0/367671216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e414e0c0 0x7f80e40a4e70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f80e40c57d0 tx=0x7f80d401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd23e7290baee44b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:56.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e2ffd640 1 -- 172.21.15.39:0/367671216 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40c5490 con 0x7f80e414e0c0 2024-09-13T17:04:56.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e2ffd640 1 -- 172.21.15.39:0/367671216 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d40a7050 con 0x7f80e414e0c0 2024-09-13T17:04:56.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.309+0000 7f80e2ffd640 1 -- 172.21.15.39:0/367671216 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d40ab820 con 0x7f80e414e0c0 2024-09-13T17:04:56.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- 172.21.15.39:0/367671216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e414e0c0 msgr2=0x7f80e40a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:56.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 --2- 172.21.15.39:0/367671216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e414e0c0 0x7f80e40a4e70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f80e40c57d0 tx=0x7f80d401c460 comp rx=0 tx=0).stop 2024-09-13T17:04:56.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- 172.21.15.39:0/367671216 shutdown_connections 2024-09-13T17:04:56.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 --2- 172.21.15.39:0/367671216 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 0x7f80e40c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 --2- 172.21.15.39:0/367671216 >> v2:172.21.15.84:3300/0 conn(0x7f80e40c1f50 0x7f80e406cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 --2- 172.21.15.39:0/367671216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e414e0c0 0x7f80e40a4e70 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- 172.21.15.39:0/367671216 >> 172.21.15.39:0/367671216 conn(0x7f80e4069e10 msgr2=0x7f80e406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:56.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- 172.21.15.39:0/367671216 shutdown_connections 2024-09-13T17:04:56.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- 172.21.15.39:0/367671216 wait complete. 2024-09-13T17:04:56.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 Processor -- start 2024-09-13T17:04:56.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- start start 2024-09-13T17:04:56.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 0x7f80e4144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:56.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e40c1f50 0x7f80e414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:56.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e414e0c0 0x7f80e41ef900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:56.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4145760 con 0x7f80e40c1f50 2024-09-13T17:04:56.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e414a220 con 0x7f80e414e0c0 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4143aa0 con 0x7f80e406d300 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80dbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e40c1f50 0x7f80e414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80dbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e40c1f50 0x7f80e414afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46674/0 (socket says 172.21.15.39:46674) 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80dbfff640 1 -- 172.21.15.39:0/2397255362 learned_addr learned my addr 172.21.15.39:0/2397255362 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e37fe640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 0x7f80e4144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e3fff640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e414e0c0 0x7f80e41ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80dbfff640 1 -- 172.21.15.39:0/2397255362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 msgr2=0x7f80e4144ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80dbfff640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 0x7f80e4144ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80dbfff640 1 -- 172.21.15.39:0/2397255362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e414e0c0 msgr2=0x7f80e41ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80dbfff640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e414e0c0 0x7f80e41ef900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80dbfff640 1 -- 172.21.15.39:0/2397255362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d40094d0 con 0x7f80e40c1f50 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e3fff640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e414e0c0 0x7f80e41ef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80dbfff640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e40c1f50 0x7f80e414afa0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00d7d0 tx=0x7f80cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e0ff9640 1 -- 172.21.15.39:0/2397255362 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc018070 con 0x7f80e40c1f50 2024-09-13T17:04:56.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e0ff9640 1 -- 172.21.15.39:0/2397255362 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc004020 con 0x7f80e40c1f50 2024-09-13T17:04:56.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e4149eb0 con 0x7f80e40c1f50 2024-09-13T17:04:56.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e0ff9640 1 -- 172.21.15.39:0/2397255362 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0041d0 con 0x7f80e40c1f50 2024-09-13T17:04:56.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.313+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e414b4e0 con 0x7f80e40c1f50 2024-09-13T17:04:56.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.317+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e4144ba0 con 0x7f80e40c1f50 2024-09-13T17:04:56.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.317+0000 7f80e0ff9640 1 -- 172.21.15.39:0/2397255362 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80cc004330 con 0x7f80e40c1f50 2024-09-13T17:04:56.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.317+0000 7f80e0ff9640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c4066690 0x7f80c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:56.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.317+0000 7f80e0ff9640 1 -- 172.21.15.39:0/2397255362 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f80cc084ac0 con 0x7f80e40c1f50 2024-09-13T17:04:56.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.317+0000 7f80e37fe640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c4066690 0x7f80c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:56.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.317+0000 7f80e37fe640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c4066690 0x7f80c4068b50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f80e40c3370 tx=0x7f80d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:56.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.321+0000 7f80e0ff9640 1 -- 172.21.15.39:0/2397255362 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f80cc0513c0 con 0x7f80e40c1f50 2024-09-13T17:04:56.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.517+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80e414a040 con 0x7f80e40c1f50 2024-09-13T17:04:56.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.517+0000 7f80e0ff9640 1 -- 172.21.15.39:0/2397255362 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f80cc0549e0 con 0x7f80e40c1f50 2024-09-13T17:04:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:56.516 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:56.516 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-13T17:04:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c4066690 msgr2=0x7f80c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c4066690 0x7f80c4068b50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f80e40c3370 tx=0x7f80d4008040 comp rx=0 tx=0).stop 2024-09-13T17:04:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e40c1f50 msgr2=0x7f80e414afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e40c1f50 0x7f80e414afa0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00d7d0 tx=0x7f80cc00dc90 comp rx=0 tx=0).stop 2024-09-13T17:04:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 shutdown_connections 2024-09-13T17:04:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c4066690 0x7f80c4068b50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80e414e0c0 0x7f80e41ef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80e40c1f50 0x7f80e414afa0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 --2- 172.21.15.39:0/2397255362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80e406d300 0x7f80e4144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 >> 172.21.15.39:0/2397255362 conn(0x7f80e4069e10 msgr2=0x7f80e40c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 shutdown_connections 2024-09-13T17:04:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:56.521+0000 7f80e9fda640 1 -- 172.21.15.39:0/2397255362 wait complete. 2024-09-13T17:04:56.535 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:04:58.536 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:04:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4ac085640 1 Processor -- start 2024-09-13T17:04:58.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4ac085640 1 -- start start 2024-09-13T17:04:58.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4ac085640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe4a4155160 0x7fe4a41520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:58.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4ac085640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a4151bf0 con 0x7fe4a414bf30 2024-09-13T17:04:58.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4ac085640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a4155660 con 0x7fe4a40a5ef0 2024-09-13T17:04:58.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4ac085640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a4155940 con 0x7fe4a4155160 2024-09-13T17:04:58.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4a9620640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe4a4155160 0x7fe4a41520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:58.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4a9620640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe4a4155160 0x7fe4a41520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46684/0 (socket says 172.21.15.39:46684) 2024-09-13T17:04:58.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4a9620640 1 -- 172.21.15.39:0/2080643371 learned_addr learned my addr 172.21.15.39:0/2080643371 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:58.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4a9620640 1 -- 172.21.15.39:0/2080643371 >> v1:172.21.15.84:6789/0 conn(0x7fe4a40a5ef0 legacy=0x7fe4a4152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:58.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4a9620640 1 -- 172.21.15.39:0/2080643371 >> v1:172.21.15.39:6789/0 conn(0x7fe4a414bf30 legacy=0x7fe4a406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:58.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.673+0000 7fe4a9620640 1 -- 172.21.15.39:0/2080643371 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a4155c20 con 0x7fe4a4155160 2024-09-13T17:04:58.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4a9620640 1 --2- 172.21.15.39:0/2080643371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 0x7fe4a41520f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe494009870 tx=0x7fe49401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c28ce6145695449 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:58.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4a8e1f640 1 -- 172.21.15.39:0/2080643371 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940aa670 con 0x7fe4a4155160 2024-09-13T17:04:58.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4a8e1f640 1 -- 172.21.15.39:0/2080643371 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4940aa7d0 con 0x7fe4a4155160 2024-09-13T17:04:58.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4a8e1f640 1 -- 172.21.15.39:0/2080643371 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940aab60 con 0x7fe4a4155160 2024-09-13T17:04:58.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 -- 172.21.15.39:0/2080643371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 msgr2=0x7fe4a41520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:58.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 --2- 172.21.15.39:0/2080643371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 0x7fe4a41520f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fe494009870 tx=0x7fe49401c530 comp rx=0 tx=0).stop 2024-09-13T17:04:58.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 -- 172.21.15.39:0/2080643371 shutdown_connections 2024-09-13T17:04:58.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 --2- 172.21.15.39:0/2080643371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 0x7fe4a41520f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:58.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 -- 172.21.15.39:0/2080643371 >> 172.21.15.39:0/2080643371 conn(0x7fe4a40c32a0 msgr2=0x7fe4a40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:58.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 -- 172.21.15.39:0/2080643371 shutdown_connections 2024-09-13T17:04:58.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 -- 172.21.15.39:0/2080643371 wait complete. 2024-09-13T17:04:58.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 Processor -- start 2024-09-13T17:04:58.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 -- start start 2024-09-13T17:04:58.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4a40a5ef0 0x7fe4a414ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:58.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4a414bf30 0x7fe4a4154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:58.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 0x7fe4a414c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:58.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a4154470 con 0x7fe4a4155160 2024-09-13T17:04:58.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a414e260 con 0x7fe4a40a5ef0 2024-09-13T17:04:58.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4ac085640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a4153a70 con 0x7fe4a414bf30 2024-09-13T17:04:58.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4aa622640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 0x7fe4a414c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4aa622640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 0x7fe4a414c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46696/0 (socket says 172.21.15.39:46696) 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4aa622640 1 -- 172.21.15.39:0/2542154899 learned_addr learned my addr 172.21.15.39:0/2542154899 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4a9e21640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4a40a5ef0 0x7fe4a414ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.677+0000 7fe4a9620640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4a414bf30 0x7fe4a4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4aa622640 1 -- 172.21.15.39:0/2542154899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4a414bf30 msgr2=0x7fe4a4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4aa622640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4a414bf30 0x7fe4a4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4aa622640 1 -- 172.21.15.39:0/2542154899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4a40a5ef0 msgr2=0x7fe4a414ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4aa622640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4a40a5ef0 0x7fe4a414ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4aa622640 1 -- 172.21.15.39:0/2542154899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4940094d0 con 0x7fe4a4155160 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4a9620640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4a414bf30 0x7fe4a4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4aa622640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 0x7fe4a414c560 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c730 tx=0x7fe49c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe49a7fc640 1 -- 172.21.15.39:0/2542154899 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c006d60 con 0x7fe4a4155160 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe49a7fc640 1 -- 172.21.15.39:0/2542154899 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c005020 con 0x7fe4a4155160 2024-09-13T17:04:58.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a4154dd0 con 0x7fe4a4155160 2024-09-13T17:04:58.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe49a7fc640 1 -- 172.21.15.39:0/2542154899 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c005410 con 0x7fe4a4155160 2024-09-13T17:04:58.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a41edd70 con 0x7fe4a4155160 2024-09-13T17:04:58.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a40a5d60 con 0x7fe4a4155160 2024-09-13T17:04:58.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe49a7fc640 1 -- 172.21.15.39:0/2542154899 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe49c005570 con 0x7fe4a4155160 2024-09-13T17:04:58.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe49a7fc640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe488066690 0x7fe488068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:04:58.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4a9e21640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe488066690 0x7fe488068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:04:58.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe49a7fc640 1 -- 172.21.15.39:0/2542154899 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7fe49c084540 con 0x7fe4a4155160 2024-09-13T17:04:58.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.681+0000 7fe4a9e21640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe488066690 0x7fe488068b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0007fd0 tx=0x7fe4a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:04:58.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.685+0000 7fe49a7fc640 1 -- 172.21.15.39:0/2542154899 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe49c050e40 con 0x7fe4a4155160 2024-09-13T17:04:58.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.877+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4a415c560 con 0x7fe4a4155160 2024-09-13T17:04:58.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe49a7fc640 1 -- 172.21.15.39:0/2542154899 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5936 (secure 0 0 0) 0x7fe49c054460 con 0x7fe4a4155160 2024-09-13T17:04:58.879 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:04:58.879 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:04:58.879 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe488066690 msgr2=0x7fe488068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe488066690 0x7fe488068b50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0007fd0 tx=0x7fe4a0009040 comp rx=0 tx=0).stop 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 msgr2=0x7fe4a414c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 0x7fe4a414c560 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c730 tx=0x7fe49c00cbf0 comp rx=0 tx=0).stop 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 shutdown_connections 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe488066690 0x7fe488068b50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4a4155160 0x7fe4a414c560 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4a414bf30 0x7fe4a4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 --2- 172.21.15.39:0/2542154899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4a40a5ef0 0x7fe4a414ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 >> 172.21.15.39:0/2542154899 conn(0x7fe4a40c32a0 msgr2=0x7fe4a41520f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:04:58.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 shutdown_connections 2024-09-13T17:04:58.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:04:58.881+0000 7fe4ac085640 1 -- 172.21.15.39:0/2542154899 wait complete. 2024-09-13T17:04:58.898 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:04:59.121 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:04:59.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:05:00.900 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:01.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.037+0000 7f865a1d5640 1 Processor -- start 2024-09-13T17:05:01.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.037+0000 7f865a1d5640 1 -- start start 2024-09-13T17:05:01.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f865a1d5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8654151500 0x7f86540c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:01.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f865a1d5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 0x7f8654154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:01.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f865a1d5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86540c5a30 con 0x7f86540c47c0 2024-09-13T17:05:01.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f865a1d5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86541576a0 con 0x7f8654151500 2024-09-13T17:05:01.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f865a1d5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8654157980 con 0x7f8654071dd0 2024-09-13T17:05:01.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f86537fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8654151500 0x7f86540c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:01.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f86537fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8654151500 0x7f86540c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41722/0 (socket says 172.21.15.39:41722) 2024-09-13T17:05:01.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f86537fe640 1 -- 172.21.15.39:0/3554419520 learned_addr learned my addr 172.21.15.39:0/3554419520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:01.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f8652ffd640 1 --2- 172.21.15.39:0/3554419520 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 0x7f8654154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:01.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f86537fe640 1 -- 172.21.15.39:0/3554419520 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 msgr2=0x7f8654154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:01.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f86537fe640 1 --2- 172.21.15.39:0/3554419520 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 0x7f8654154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f86537fe640 1 -- 172.21.15.39:0/3554419520 >> v1:172.21.15.84:6789/0 conn(0x7f86540c47c0 legacy=0x7f86541545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:01.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f86537fe640 1 -- 172.21.15.39:0/3554419520 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654157c60 con 0x7f8654151500 2024-09-13T17:05:01.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.041+0000 7f86537fe640 1 --2- 172.21.15.39:0/3554419520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8654151500 0x7f86540c4280 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8644009870 tx=0x7f864401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b89e170c439c4a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:01.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f86527fc640 1 -- 172.21.15.39:0/3554419520 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440aa620 con 0x7f8654151500 2024-09-13T17:05:01.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f86527fc640 1 -- 172.21.15.39:0/3554419520 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86440aa780 con 0x7f8654151500 2024-09-13T17:05:01.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f86527fc640 1 -- 172.21.15.39:0/3554419520 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440aab10 con 0x7f8654151500 2024-09-13T17:05:01.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- 172.21.15.39:0/3554419520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8654151500 msgr2=0x7f86540c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:01.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 --2- 172.21.15.39:0/3554419520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8654151500 0x7f86540c4280 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8644009870 tx=0x7f864401c4a0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- 172.21.15.39:0/3554419520 shutdown_connections 2024-09-13T17:05:01.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 --2- 172.21.15.39:0/3554419520 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 0x7f8654154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 --2- 172.21.15.39:0/3554419520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8654151500 0x7f86540c4280 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- 172.21.15.39:0/3554419520 >> 172.21.15.39:0/3554419520 conn(0x7f86540ab530 msgr2=0x7f86540ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:01.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- 172.21.15.39:0/3554419520 shutdown_connections 2024-09-13T17:05:01.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- 172.21.15.39:0/3554419520 wait complete. 2024-09-13T17:05:01.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 Processor -- start 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- start start 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 0x7f8654070430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86540c47c0 0x7f86540c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865414bbb0 0x7f86540c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541ed360 con 0x7f865414bbb0 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541532b0 con 0x7f86540c47c0 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86541ec790 con 0x7f8654071dd0 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8653fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865414bbb0 0x7f86540c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8653fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865414bbb0 0x7f86540c2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41728/0 (socket says 172.21.15.39:41728) 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8653fff640 1 -- 172.21.15.39:0/2822345004 learned_addr learned my addr 172.21.15.39:0/2822345004 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f86537fe640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 0x7f8654070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:01.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8652ffd640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86540c47c0 0x7f86540c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8653fff640 1 -- 172.21.15.39:0/2822345004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 msgr2=0x7f8654070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8653fff640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 0x7f8654070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8653fff640 1 -- 172.21.15.39:0/2822345004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86540c47c0 msgr2=0x7f86540c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8653fff640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86540c47c0 0x7f86540c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8653fff640 1 -- 172.21.15.39:0/2822345004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440094d0 con 0x7f865414bbb0 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f86537fe640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 0x7f8654070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f8653fff640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865414bbb0 0x7f86540c2f30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c760 tx=0x7f864c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f864bfff640 1 -- 172.21.15.39:0/2822345004 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c003b80 con 0x7f865414bbb0 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f864bfff640 1 -- 172.21.15.39:0/2822345004 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c003ce0 con 0x7f865414bbb0 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f864bfff640 1 -- 172.21.15.39:0/2822345004 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00aa30 con 0x7f865414bbb0 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86541eca30 con 0x7f865414bbb0 2024-09-13T17:05:01.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.045+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86541ecff0 con 0x7f865414bbb0 2024-09-13T17:05:01.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.049+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86541ed510 con 0x7f865414bbb0 2024-09-13T17:05:01.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.049+0000 7f864bfff640 1 -- 172.21.15.39:0/2822345004 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f864c00ab90 con 0x7f865414bbb0 2024-09-13T17:05:01.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.049+0000 7f864bfff640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f86340666c0 0x7f8634068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:01.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.049+0000 7f864bfff640 1 -- 172.21.15.39:0/2822345004 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f864c084990 con 0x7f865414bbb0 2024-09-13T17:05:01.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.049+0000 7f86537fe640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f86340666c0 0x7f8634068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:01.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.049+0000 7f86537fe640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f86340666c0 0x7f8634068b80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f86440ae000 tx=0x7f86440a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:01.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.053+0000 7f864bfff640 1 -- 172.21.15.39:0/2822345004 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f86541ed510 con 0x7f865414bbb0 2024-09-13T17:05:01.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.253+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86541ed510 con 0x7f865414bbb0 2024-09-13T17:05:01.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.253+0000 7f864bfff640 1 -- 172.21.15.39:0/2822345004 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f86541ed510 con 0x7f865414bbb0 2024-09-13T17:05:01.253 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:01.253 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:01.253 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-13T17:05:01.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f86340666c0 msgr2=0x7f8634068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:01.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f86340666c0 0x7f8634068b80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f86440ae000 tx=0x7f86440a7000 comp rx=0 tx=0).stop 2024-09-13T17:05:01.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865414bbb0 msgr2=0x7f86540c2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:01.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865414bbb0 0x7f86540c2f30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c760 tx=0x7f864c00cc20 comp rx=0 tx=0).stop 2024-09-13T17:05:01.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 shutdown_connections 2024-09-13T17:05:01.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f86340666c0 0x7f8634068b80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86540c47c0 0x7f86540c29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f865414bbb0 0x7f86540c2f30 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 --2- 172.21.15.39:0/2822345004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8654071dd0 0x7f8654070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:01.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 >> 172.21.15.39:0/2822345004 conn(0x7f86540ab530 msgr2=0x7f865415df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:01.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 shutdown_connections 2024-09-13T17:05:01.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:01.257+0000 7f865a1d5640 1 -- 172.21.15.39:0/2822345004 wait complete. 2024-09-13T17:05:01.272 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:05:03.273 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:03.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75cb0a9640 1 Processor -- start 2024-09-13T17:05:03.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75cb0a9640 1 -- start start 2024-09-13T17:05:03.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75cb0a9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c40ca080 0x7f75c4151000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75cb0a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f75c40716c0 0x7f75c4153d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75cb0a9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c40c9710 con 0x7f75c4151540 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75cb0a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c4157fc0 con 0x7f75c40716c0 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75cb0a9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c4157340 con 0x7f75c40ca080 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c3fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f75c4151540 0x7f75c4070fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54996/0 (socket says 172.21.15.39:54996) 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c3fff640 1 -- 172.21.15.39:0/3372782373 learned_addr learned my addr 172.21.15.39:0/3372782373 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c8e45640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c40ca080 0x7f75c4151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c9646640 1 --2- 172.21.15.39:0/3372782373 >> v2:172.21.15.84:3300/0 conn(0x7f75c40716c0 0x7f75c4153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178557113 0 0) 0x7f75c40c9710 con 0x7f75c4151540 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75ac003530 con 0x7f75c4151540 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 987925127 0 0) 0x7f75ac003530 con 0x7f75c4151540 2024-09-13T17:05:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c40ca080 msgr2=0x7f75c4151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c40ca080 0x7f75c4151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 >> v2:172.21.15.84:3300/0 conn(0x7f75c40716c0 msgr2=0x7f75c4153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 --2- 172.21.15.39:0/3372782373 >> v2:172.21.15.84:3300/0 conn(0x7f75c40716c0 0x7f75c4153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c4157620 con 0x7f75c4151540 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f75b8001680 con 0x7f75c4151540 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 >> v1:172.21.15.39:6789/0 conn(0x7f75c4151540 legacy=0x7f75c4070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75ac003980 0x7f75ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c40ca080 0x7f75ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c4157fc0 con 0x7f75ac0047d0 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c4157340 con 0x7f75ac003980 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac00ba10 con 0x7f75c40ca080 2024-09-13T17:05:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c3fff640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c8e45640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75ac003980 0x7f75ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c9646640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c40ca080 0x7f75ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c3fff640 1 -- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c40ca080 msgr2=0x7f75ac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c3fff640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c40ca080 0x7f75ac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c3fff640 1 -- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75ac003980 msgr2=0x7f75ac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c3fff640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75ac003980 0x7f75ac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c3fff640 1 -- 172.21.15.39:0/3372782373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac003530 con 0x7f75ac0047d0 2024-09-13T17:05:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.413+0000 7f75c9646640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c40ca080 0x7f75ac005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:03.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75c3fff640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75ac004c50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c240 tx=0x7f75b801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ba9b1b5b51fa7e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:03.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4154a70 con 0x7f75ac0047d0 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b8005b00 con 0x7f75ac0047d0 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75c37fe640 1 -- 172.21.15.39:0/3372782373 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8007070 con 0x7f75ac0047d0 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75cb0a9640 1 -- 172.21.15.39:0/3372782373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 msgr2=0x7f75ac004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75cb0a9640 1 --2- 172.21.15.39:0/3372782373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75ac004c50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c240 tx=0x7f75b801cc60 comp rx=0 tx=0).stop 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75c9646640 1 -- 172.21.15.39:0/3372782373 reap_dead start 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75cb0a9640 1 -- 172.21.15.39:0/3372782373 shutdown_connections 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75cb0a9640 1 -- 172.21.15.39:0/3372782373 >> 172.21.15.39:0/3372782373 conn(0x7f75c406a210 msgr2=0x7f75c406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75cb0a9640 1 -- 172.21.15.39:0/3372782373 shutdown_connections 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75cb0a9640 1 -- 172.21.15.39:0/3372782373 wait complete. 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75cb0a9640 1 Processor -- start 2024-09-13T17:05:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.417+0000 7f75cb0a9640 1 -- start start 2024-09-13T17:05:03.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75cb0a9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75c40a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:03.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75cb0a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75c40abfd0 0x7f75c40ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:03.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75cb0a9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c414d510 0x7f75c41ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:03.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75cb0a9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c414e080 con 0x7f75ac0047d0 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75cb0a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c414e320 con 0x7f75c40abfd0 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75cb0a9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c414e600 con 0x7f75c414d510 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c8e45640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75c40a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c8e45640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75c40a9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41738/0 (socket says 172.21.15.39:41738) 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c8e45640 1 -- 172.21.15.39:0/2958325741 learned_addr learned my addr 172.21.15.39:0/2958325741 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c3fff640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75c40abfd0 0x7f75c40ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c9646640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c414d510 0x7f75c41ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c8e45640 1 -- 172.21.15.39:0/2958325741 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c414d510 msgr2=0x7f75c41ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c8e45640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c414d510 0x7f75c41ecae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c8e45640 1 -- 172.21.15.39:0/2958325741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75c40abfd0 msgr2=0x7f75c40ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c8e45640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75c40abfd0 0x7f75c40ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c8e45640 1 -- 172.21.15.39:0/2958325741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c41ec2e0 con 0x7f75ac0047d0 2024-09-13T17:05:03.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c8e45640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75c40a9390 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009fb0 tx=0x7f75b40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:03.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c17fa640 1 -- 172.21.15.39:0/2958325741 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b400e030 con 0x7f75ac0047d0 2024-09-13T17:05:03.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c17fa640 1 -- 172.21.15.39:0/2958325741 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b4016470 con 0x7f75ac0047d0 2024-09-13T17:05:03.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c17fa640 1 -- 172.21.15.39:0/2958325741 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b4016820 con 0x7f75ac0047d0 2024-09-13T17:05:03.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c41ec5c0 con 0x7f75ac0047d0 2024-09-13T17:05:03.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c40ae140 con 0x7f75ac0047d0 2024-09-13T17:05:03.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c4153d00 con 0x7f75ac0047d0 2024-09-13T17:05:03.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c17fa640 1 -- 172.21.15.39:0/2958325741 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75b4016980 con 0x7f75ac0047d0 2024-09-13T17:05:03.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c17fa640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75b0066690 0x7f75b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:03.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c17fa640 1 -- 172.21.15.39:0/2958325741 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f75b400f070 con 0x7f75ac0047d0 2024-09-13T17:05:03.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.421+0000 7f75c3fff640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75b0066690 0x7f75b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:03.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.425+0000 7f75c3fff640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75b0066690 0x7f75b0068b50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f75b80ab000 tx=0x7f75b80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:03.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.429+0000 7f75c17fa640 1 -- 172.21.15.39:0/2958325741 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f75b4052350 con 0x7f75ac0047d0 2024-09-13T17:05:03.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.621+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75c40ab770 con 0x7f75ac0047d0 2024-09-13T17:05:03.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75c17fa640 1 -- 172.21.15.39:0/2958325741 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f75b4055970 con 0x7f75ac0047d0 2024-09-13T17:05:03.622 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:03.622 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:03.622 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75b0066690 msgr2=0x7f75b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75b0066690 0x7f75b0068b50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f75b80ab000 tx=0x7f75b80a7070 comp rx=0 tx=0).stop 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 msgr2=0x7f75c40a9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75c40a9390 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009fb0 tx=0x7f75b40115a0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 shutdown_connections 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75b0066690 0x7f75b0068b50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75c40abfd0 0x7f75c40ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75c414d510 0x7f75c41ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 --2- 172.21.15.39:0/2958325741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75ac0047d0 0x7f75c40a9390 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 >> 172.21.15.39:0/2958325741 conn(0x7f75c406a210 msgr2=0x7f75c414cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 shutdown_connections 2024-09-13T17:05:03.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:03.625+0000 7f75cb0a9640 1 -- 172.21.15.39:0/2958325741 wait complete. 2024-09-13T17:05:03.643 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:05:05.643 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:05.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.777+0000 7f5b9c0c4640 1 Processor -- start 2024-09-13T17:05:05.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.777+0000 7f5b9c0c4640 1 -- start start 2024-09-13T17:05:05.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9c0c4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b940c63e0 0x7f5b941525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:05.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9c0c4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5b940c2640 0x7f5b940c0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:05.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9c0c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9406c5d0 con 0x7f5b940c2260 2024-09-13T17:05:05.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9c0c4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9406d1f0 con 0x7f5b940c2640 2024-09-13T17:05:05.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9c0c4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9406c850 con 0x7f5b940c63e0 2024-09-13T17:05:05.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9a661640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5b940c2640 0x7f5b940c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:05.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b99e60640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5b940c2260 0x7f5b9414ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:55002/0 (socket says 172.21.15.39:55002) 2024-09-13T17:05:05.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b99e60640 1 -- 172.21.15.39:0/2109433062 learned_addr learned my addr 172.21.15.39:0/2109433062 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:05.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9965f640 1 --2- 172.21.15.39:0/2109433062 >> v2:172.21.15.84:3300/0 conn(0x7f5b940c63e0 0x7f5b941525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:05.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9a661640 1 -- 172.21.15.39:0/2109433062 >> v2:172.21.15.84:3300/0 conn(0x7f5b940c63e0 msgr2=0x7f5b941525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:05.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9a661640 1 --2- 172.21.15.39:0/2109433062 >> v2:172.21.15.84:3300/0 conn(0x7f5b940c63e0 0x7f5b941525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9a661640 1 -- 172.21.15.39:0/2109433062 >> v1:172.21.15.39:6789/0 conn(0x7f5b940c2260 legacy=0x7f5b9414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:05.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9a661640 1 -- 172.21.15.39:0/2109433062 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9406cb30 con 0x7f5b940c2640 2024-09-13T17:05:05.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b98e5e640 1 -- 172.21.15.39:0/2109433062 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482091237 0 0) 0x7f5b9406c5d0 con 0x7f5b940c2260 2024-09-13T17:05:05.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b9a661640 1 --2- 172.21.15.39:0/2109433062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c2640 0x7f5b940c0290 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5b9406f030 tx=0x7f5b8c01c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6e1c82c46dc555 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:05.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b98e5e640 1 -- 172.21.15.39:0/2109433062 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940a5320 con 0x7f5b940c2640 2024-09-13T17:05:05.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b98e5e640 1 -- 172.21.15.39:0/2109433062 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c0a58c0 con 0x7f5b940c2640 2024-09-13T17:05:05.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.781+0000 7f5b98e5e640 1 -- 172.21.15.39:0/2109433062 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940c0fe0 con 0x7f5b940c2640 2024-09-13T17:05:05.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/2109433062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c2640 msgr2=0x7f5b940c0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:05.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 --2- 172.21.15.39:0/2109433062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c2640 0x7f5b940c0290 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5b9406f030 tx=0x7f5b8c01c7e0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/2109433062 shutdown_connections 2024-09-13T17:05:05.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 --2- 172.21.15.39:0/2109433062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c2640 0x7f5b940c0290 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 --2- 172.21.15.39:0/2109433062 >> v2:172.21.15.84:3300/0 conn(0x7f5b940c63e0 0x7f5b941525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/2109433062 >> 172.21.15.39:0/2109433062 conn(0x7f5b9406a6a0 msgr2=0x7f5b940c5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:05.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/2109433062 shutdown_connections 2024-09-13T17:05:05.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/2109433062 wait complete. 2024-09-13T17:05:05.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 Processor -- start 2024-09-13T17:05:05.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- start start 2024-09-13T17:05:05.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b940c2260 0x7f5b94146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b940c2640 0x7f5b941447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c63e0 0x7f5b94144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b94145220 con 0x7f5b940c63e0 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9415b5d0 con 0x7f5b940c2640 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b941e9e20 con 0x7f5b940c2260 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9a661640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c63e0 0x7f5b94144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9a661640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c63e0 0x7f5b94144ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41754/0 (socket says 172.21.15.39:41754) 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9a661640 1 -- 172.21.15.39:0/4091626975 learned_addr learned my addr 172.21.15.39:0/4091626975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9965f640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b940c2640 0x7f5b941447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b99e60640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b940c2260 0x7f5b94146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:05.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9a661640 1 -- 172.21.15.39:0/4091626975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b940c2260 msgr2=0x7f5b94146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9a661640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b940c2260 0x7f5b94146d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9a661640 1 -- 172.21.15.39:0/4091626975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b940c2640 msgr2=0x7f5b941447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9a661640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b940c2640 0x7f5b941447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9a661640 1 -- 172.21.15.39:0/4091626975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c009a70 con 0x7f5b940c63e0 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9a661640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c63e0 0x7f5b94144ce0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0b0040 tx=0x7f5b8c019580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b8a7fc640 1 -- 172.21.15.39:0/4091626975 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0b9860 con 0x7f5b940c63e0 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b8a7fc640 1 -- 172.21.15.39:0/4091626975 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c0b99c0 con 0x7f5b940c63e0 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b941ea0e0 con 0x7f5b940c63e0 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b8a7fc640 1 -- 172.21.15.39:0/4091626975 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0b9db0 con 0x7f5b940c63e0 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.785+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b941ea6a0 con 0x7f5b940c63e0 2024-09-13T17:05:05.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.789+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b940c0290 con 0x7f5b940c63e0 2024-09-13T17:05:05.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.789+0000 7f5b8a7fc640 1 -- 172.21.15.39:0/4091626975 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b8c0ad440 con 0x7f5b940c63e0 2024-09-13T17:05:05.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.789+0000 7f5b8a7fc640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b78066690 0x7f5b78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:05.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.789+0000 7f5b8a7fc640 1 -- 172.21.15.39:0/4091626975 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f5b8c120010 con 0x7f5b940c63e0 2024-09-13T17:05:05.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.789+0000 7f5b99e60640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b78066690 0x7f5b78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:05.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.789+0000 7f5b99e60640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b78066690 0x7f5b78068b50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5b90000f70 tx=0x7f5b90007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:05.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.793+0000 7f5b8a7fc640 1 -- 172.21.15.39:0/4091626975 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b8c0eca30 con 0x7f5b940c63e0 2024-09-13T17:05:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.989+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b941528c0 con 0x7f5b940c63e0 2024-09-13T17:05:05.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.989+0000 7f5b8a7fc640 1 -- 172.21.15.39:0/4091626975 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f5b8c0f0050 con 0x7f5b940c63e0 2024-09-13T17:05:05.988 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:05.988 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:05.989 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-13T17:05:05.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b78066690 msgr2=0x7f5b78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b78066690 0x7f5b78068b50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5b90000f70 tx=0x7f5b90007c10 comp rx=0 tx=0).stop 2024-09-13T17:05:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c63e0 msgr2=0x7f5b94144ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c63e0 0x7f5b94144ce0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0b0040 tx=0x7f5b8c019580 comp rx=0 tx=0).stop 2024-09-13T17:05:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 shutdown_connections 2024-09-13T17:05:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5b78066690 0x7f5b78068b50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b940c2640 0x7f5b941447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b940c63e0 0x7f5b94144ce0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 --2- 172.21.15.39:0/4091626975 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5b940c2260 0x7f5b94146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 >> 172.21.15.39:0/4091626975 conn(0x7f5b9406a6a0 msgr2=0x7f5b94147920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 shutdown_connections 2024-09-13T17:05:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:05.993+0000 7f5b9c0c4640 1 -- 172.21.15.39:0/4091626975 wait complete. 2024-09-13T17:05:06.007 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:05:08.008 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:08.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74d58e9640 1 Processor -- start 2024-09-13T17:05:08.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74d58e9640 1 -- start start 2024-09-13T17:05:08.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74d58e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f74d00aabc0 0x7f74d00aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:08.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74d58e9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f74d006c870 0x7f74d00c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:08.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74d58e9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d0152a80 con 0x7f74d00c3ee0 2024-09-13T17:05:08.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74d58e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d00c30f0 con 0x7f74d00aabc0 2024-09-13T17:05:08.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74d58e9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d0151e00 con 0x7f74d006c870 2024-09-13T17:05:08.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74ceffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f74d00aabc0 0x7f74d00aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:08.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74cf7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f74d00c3ee0 0x7f74d01550e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:55008/0 (socket says 172.21.15.39:55008) 2024-09-13T17:05:08.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74cf7fe640 1 -- 172.21.15.39:0/335486439 learned_addr learned my addr 172.21.15.39:0/335486439 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:08.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74ce7fc640 1 --2- 172.21.15.39:0/335486439 >> v2:172.21.15.84:3300/0 conn(0x7f74d006c870 0x7f74d00c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:08.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74ceffd640 1 -- 172.21.15.39:0/335486439 >> v2:172.21.15.84:3300/0 conn(0x7f74d006c870 msgr2=0x7f74d00c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:08.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74ceffd640 1 --2- 172.21.15.39:0/335486439 >> v2:172.21.15.84:3300/0 conn(0x7f74d006c870 0x7f74d00c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74ceffd640 1 -- 172.21.15.39:0/335486439 >> v1:172.21.15.39:6789/0 conn(0x7f74d00c3ee0 legacy=0x7f74d01550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:08.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74ceffd640 1 -- 172.21.15.39:0/335486439 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d01520e0 con 0x7f74d00aabc0 2024-09-13T17:05:08.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.137+0000 7f74cdffb640 1 -- 172.21.15.39:0/335486439 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104915543 0 0) 0x7f74c80014a0 con 0x7f74d00c3ee0 2024-09-13T17:05:08.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ceffd640 1 --2- 172.21.15.39:0/335486439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00aabc0 0x7f74d00aafa0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f74d00bfe70 tx=0x7f74c001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1217ed05f934e9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:08.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74cdffb640 1 -- 172.21.15.39:0/335486439 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00ab600 con 0x7f74d00aabc0 2024-09-13T17:05:08.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74cdffb640 1 -- 172.21.15.39:0/335486439 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c00a7050 con 0x7f74d00aabc0 2024-09-13T17:05:08.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74cdffb640 1 -- 172.21.15.39:0/335486439 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00ab980 con 0x7f74d00aabc0 2024-09-13T17:05:08.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 -- 172.21.15.39:0/335486439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00aabc0 msgr2=0x7f74d00aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:08.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 --2- 172.21.15.39:0/335486439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00aabc0 0x7f74d00aafa0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f74d00bfe70 tx=0x7f74c001c3c0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 -- 172.21.15.39:0/335486439 shutdown_connections 2024-09-13T17:05:08.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 --2- 172.21.15.39:0/335486439 >> v2:172.21.15.84:3300/0 conn(0x7f74d006c870 0x7f74d00c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 --2- 172.21.15.39:0/335486439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00aabc0 0x7f74d00aafa0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 -- 172.21.15.39:0/335486439 >> 172.21.15.39:0/335486439 conn(0x7f74d0069e10 msgr2=0x7f74d006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:08.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 -- 172.21.15.39:0/335486439 shutdown_connections 2024-09-13T17:05:08.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 -- 172.21.15.39:0/335486439 wait complete. 2024-09-13T17:05:08.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 Processor -- start 2024-09-13T17:05:08.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 -- start start 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74d006c870 0x7f74d01ee760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00c3ee0 0x7f74d01eb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d006f6f0 0x7f74d015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01ea7e0 con 0x7f74d00c3ee0 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01eeca0 con 0x7f74d006f6f0 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74d58e9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01eb730 con 0x7f74d006c870 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ce7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00c3ee0 0x7f74d01eb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ce7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00c3ee0 0x7f74d01eb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41766/0 (socket says 172.21.15.39:41766) 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ce7fc640 1 -- 172.21.15.39:0/21024241 learned_addr learned my addr 172.21.15.39:0/21024241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ceffd640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74d006c870 0x7f74d01ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74cf7fe640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d006f6f0 0x7f74d015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ce7fc640 1 -- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74d006c870 msgr2=0x7f74d01ee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ce7fc640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74d006c870 0x7f74d01ee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ce7fc640 1 -- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d006f6f0 msgr2=0x7f74d015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ce7fc640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d006f6f0 0x7f74d015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.141+0000 7f74ce7fc640 1 -- 172.21.15.39:0/21024241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0009420 con 0x7f74d00c3ee0 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74cf7fe640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d006f6f0 0x7f74d015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74ceffd640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74d006c870 0x7f74d01ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74ce7fc640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00c3ee0 0x7f74d01eb1f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f74c400e930 tx=0x7f74c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74b7fff640 1 -- 172.21.15.39:0/21024241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4009690 con 0x7f74d00c3ee0 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74b7fff640 1 -- 172.21.15.39:0/21024241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c400c9a0 con 0x7f74d00c3ee0 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74b7fff640 1 -- 172.21.15.39:0/21024241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c400cd10 con 0x7f74d00c3ee0 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d00a6900 con 0x7f74d00c3ee0 2024-09-13T17:05:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d00a6ec0 con 0x7f74d00c3ee0 2024-09-13T17:05:08.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d00a7080 con 0x7f74d00c3ee0 2024-09-13T17:05:08.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74b7fff640 1 -- 172.21.15.39:0/21024241 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74c4013470 con 0x7f74d00c3ee0 2024-09-13T17:05:08.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74b7fff640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74bc0666c0 0x7f74bc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:08.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74b7fff640 1 -- 172.21.15.39:0/21024241 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 8443+0+0 (secure 0 0 0) 0x7f74c4084d90 con 0x7f74d00c3ee0 2024-09-13T17:05:08.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74ceffd640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74bc0666c0 0x7f74bc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:08.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.145+0000 7f74ceffd640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74bc0666c0 0x7f74bc068b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f74c00ad000 tx=0x7f74c00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:08.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.149+0000 7f74b7fff640 1 -- 172.21.15.39:0/21024241 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74c4051750 con 0x7f74d00c3ee0 2024-09-13T17:05:08.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.349+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74d006c650 con 0x7f74d00c3ee0 2024-09-13T17:05:08.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.349+0000 7f74b7fff640 1 -- 172.21.15.39:0/21024241 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f74c4021090 con 0x7f74d00c3ee0 2024-09-13T17:05:08.349 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:08.349 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":6948,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:04:28.058879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6858},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6858":{"gid":6858,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":10,"addr":"172.21.15.39:6837/1590039204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1590039204},{"type":"v1","addr":"172.21.15.39:6837","nonce":1590039204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:08.349 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-13T17:05:08.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74bc0666c0 msgr2=0x7f74bc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74bc0666c0 0x7f74bc068b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f74c00ad000 tx=0x7f74c00a6000 comp rx=0 tx=0).stop 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00c3ee0 msgr2=0x7f74d01eb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00c3ee0 0x7f74d01eb1f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f74c400e930 tx=0x7f74c400edf0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 shutdown_connections 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f74bc0666c0 0x7f74bc068b80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74d006f6f0 0x7f74d015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d00c3ee0 0x7f74d01eb1f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 --2- 172.21.15.39:0/21024241 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f74d006c870 0x7f74d01ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 >> 172.21.15.39:0/21024241 conn(0x7f74d0069e10 msgr2=0x7f74d0155030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 shutdown_connections 2024-09-13T17:05:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:08.353+0000 7f74d58e9640 1 -- 172.21.15.39:0/21024241 wait complete. 2024-09-13T17:05:08.366 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:05:10.367 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:10.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d936a640 1 Processor -- start 2024-09-13T17:05:10.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d936a640 1 -- start start 2024-09-13T17:05:10.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d936a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4d406d020 0x7fb4d40a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d936a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4d4151460 0x7fb4d414db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d936a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d41518f0 con 0x7fb4d414e070 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d936a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d40a8c50 con 0x7fb4d406d020 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d936a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d4154100 con 0x7fb4d4151460 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4d406d020 0x7fb4d40a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4d406d020 0x7fb4d40a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38542/0 (socket says 172.21.15.39:38542) 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d2ffd640 1 -- 172.21.15.39:0/2469374645 learned_addr learned my addr 172.21.15.39:0/2469374645 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d27fc640 1 --2- 172.21.15.39:0/2469374645 >> v2:172.21.15.84:3300/0 conn(0x7fb4d4151460 0x7fb4d414db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d2ffd640 1 -- 172.21.15.39:0/2469374645 >> v2:172.21.15.84:3300/0 conn(0x7fb4d4151460 msgr2=0x7fb4d414db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d2ffd640 1 --2- 172.21.15.39:0/2469374645 >> v2:172.21.15.84:3300/0 conn(0x7fb4d4151460 0x7fb4d414db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d2ffd640 1 -- 172.21.15.39:0/2469374645 >> v1:172.21.15.39:6789/0 conn(0x7fb4d414e070 legacy=0x7fb4d40c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d2ffd640 1 -- 172.21.15.39:0/2469374645 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d41543e0 con 0x7fb4d406d020 2024-09-13T17:05:10.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d2ffd640 1 --2- 172.21.15.39:0/2469374645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d406d020 0x7fb4d40a4e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40c3920 tx=0x7fb4c401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3368243b26cf707 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:10.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.509+0000 7fb4d1ffb640 1 -- 172.21.15.39:0/2469374645 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40ad660 con 0x7fb4d406d020 2024-09-13T17:05:10.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d1ffb640 1 -- 172.21.15.39:0/2469374645 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c40ad7c0 con 0x7fb4d406d020 2024-09-13T17:05:10.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d1ffb640 1 -- 172.21.15.39:0/2469374645 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40adb50 con 0x7fb4d406d020 2024-09-13T17:05:10.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- 172.21.15.39:0/2469374645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d406d020 msgr2=0x7fb4d40a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 --2- 172.21.15.39:0/2469374645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d406d020 0x7fb4d40a4e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40c3920 tx=0x7fb4c401c460 comp rx=0 tx=0).stop 2024-09-13T17:05:10.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- 172.21.15.39:0/2469374645 shutdown_connections 2024-09-13T17:05:10.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 --2- 172.21.15.39:0/2469374645 >> v2:172.21.15.84:3300/0 conn(0x7fb4d4151460 0x7fb4d414db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 --2- 172.21.15.39:0/2469374645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d406d020 0x7fb4d40a4e70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- 172.21.15.39:0/2469374645 >> 172.21.15.39:0/2469374645 conn(0x7fb4d4069e10 msgr2=0x7fb4d406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:10.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- 172.21.15.39:0/2469374645 shutdown_connections 2024-09-13T17:05:10.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- 172.21.15.39:0/2469374645 wait complete. 2024-09-13T17:05:10.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 Processor -- start 2024-09-13T17:05:10.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- start start 2024-09-13T17:05:10.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4d406d020 0x7fb4d40a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d414e070 0x7fb4d41ed590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4d4151460 0x7fb4d40a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d414b7e0 con 0x7fb4d414e070 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40a5940 con 0x7fb4d406d020 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d4072520 con 0x7fb4d4151460 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d414e070 0x7fb4d41ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d414e070 0x7fb4d41ed590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38544/0 (socket says 172.21.15.39:38544) 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d27fc640 1 -- 172.21.15.39:0/4222454387 learned_addr learned my addr 172.21.15.39:0/4222454387 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d2ffd640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4d406d020 0x7fb4d40a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d37fe640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4d4151460 0x7fb4d40a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d27fc640 1 -- 172.21.15.39:0/4222454387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4d4151460 msgr2=0x7fb4d40a6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d27fc640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4d4151460 0x7fb4d40a6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d27fc640 1 -- 172.21.15.39:0/4222454387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4d406d020 msgr2=0x7fb4d40a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d27fc640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4d406d020 0x7fb4d40a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d27fc640 1 -- 172.21.15.39:0/4222454387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c40094d0 con 0x7fb4d414e070 2024-09-13T17:05:10.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d27fc640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d414e070 0x7fb4d41ed590 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800d5f0 tx=0x7fb4c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:10.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4bb7fe640 1 -- 172.21.15.39:0/4222454387 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80125f0 con 0x7fb4d414e070 2024-09-13T17:05:10.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4bb7fe640 1 -- 172.21.15.39:0/4222454387 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c8012750 con 0x7fb4d414e070 2024-09-13T17:05:10.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4bb7fe640 1 -- 172.21.15.39:0/4222454387 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8012ae0 con 0x7fb4d414e070 2024-09-13T17:05:10.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d41ecdd0 con 0x7fb4d414e070 2024-09-13T17:05:10.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.513+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d4072240 con 0x7fb4d414e070 2024-09-13T17:05:10.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.517+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d41ecc30 con 0x7fb4d414e070 2024-09-13T17:05:10.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.517+0000 7fb4bb7fe640 1 -- 172.21.15.39:0/4222454387 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4c8012c40 con 0x7fb4d414e070 2024-09-13T17:05:10.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.517+0000 7fb4bb7fe640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4c0066690 0x7fb4c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.517+0000 7fb4bb7fe640 1 -- 172.21.15.39:0/4222454387 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7fb4c80850f0 con 0x7fb4d414e070 2024-09-13T17:05:10.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.517+0000 7fb4d2ffd640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4c0066690 0x7fb4c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.517+0000 7fb4d2ffd640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4c0066690 0x7fb4c0068b50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40af000 tx=0x7fb4c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.521+0000 7fb4bb7fe640 1 -- 172.21.15.39:0/4222454387 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb4d41ecc30 con 0x7fb4d414e070 2024-09-13T17:05:10.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.717+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4d41ecc30 con 0x7fb4d414e070 2024-09-13T17:05:10.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4bb7fe640 1 -- 172.21.15.39:0/4222454387 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+5177 (secure 0 0 0) 0x7fb4d41ecc30 con 0x7fb4d414e070 2024-09-13T17:05:10.719 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:10.719 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":117,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:09.766272+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:10.719 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 117 2024-09-13T17:05:10.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4c0066690 msgr2=0x7fb4c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4c0066690 0x7fb4c0068b50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40af000 tx=0x7fb4c4008040 comp rx=0 tx=0).stop 2024-09-13T17:05:10.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d414e070 msgr2=0x7fb4d41ed590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d414e070 0x7fb4d41ed590 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800d5f0 tx=0x7fb4c800dab0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 shutdown_connections 2024-09-13T17:05:10.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb4c0066690 0x7fb4c0068b50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4d4151460 0x7fb4d40a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4d414e070 0x7fb4d41ed590 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 --2- 172.21.15.39:0/4222454387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4d406d020 0x7fb4d40a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 >> 172.21.15.39:0/4222454387 conn(0x7fb4d4069e10 msgr2=0x7fb4d414c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:10.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 shutdown_connections 2024-09-13T17:05:10.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.721+0000 7fb4d936a640 1 -- 172.21.15.39:0/4222454387 wait complete. 2024-09-13T17:05:10.738 INFO:tasks.mds_thrash.fs.[a]:mds.c down, removed from mdsmap 2024-09-13T17:05:10.738 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:05:10.739 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:10.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f604a23b640 1 Processor -- start 2024-09-13T17:05:10.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f604a23b640 1 -- start start 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f604a23b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f60440aabc0 0x7f60440aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f604a23b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f60440c40b0 0x7f604415c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f604a23b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6044152a80 con 0x7f604406c870 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f604a23b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60440c30f0 con 0x7f60440c40b0 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f604a23b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6044151e00 con 0x7f60440aabc0 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f6043fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f60440c40b0 0x7f604415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f6042ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f604406c870 0x7f60440c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50490/0 (socket says 172.21.15.39:50490) 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.865+0000 7f6042ffd640 1 -- 172.21.15.39:0/2793776519 learned_addr learned my addr 172.21.15.39:0/2793776519 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f60437fe640 1 --2- 172.21.15.39:0/2793776519 >> v2:172.21.15.84:3300/0 conn(0x7f60440aabc0 0x7f60440aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f6043fff640 1 -- 172.21.15.39:0/2793776519 >> v2:172.21.15.84:3300/0 conn(0x7f60440aabc0 msgr2=0x7f60440aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f6043fff640 1 --2- 172.21.15.39:0/2793776519 >> v2:172.21.15.84:3300/0 conn(0x7f60440aabc0 0x7f60440aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f6043fff640 1 -- 172.21.15.39:0/2793776519 >> v1:172.21.15.39:6789/0 conn(0x7f604406c870 legacy=0x7f60440c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f6043fff640 1 -- 172.21.15.39:0/2793776519 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60441520e0 con 0x7f60440c40b0 2024-09-13T17:05:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f6043fff640 1 --2- 172.21.15.39:0/2793776519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60440c40b0 0x7f604415c470 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f60440c33a0 tx=0x7f603c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77461dd95bec16 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f60427fc640 1 -- 172.21.15.39:0/2793776519 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0a5930 con 0x7f60440c40b0 2024-09-13T17:05:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f60427fc640 1 -- 172.21.15.39:0/2793776519 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c0a5a90 con 0x7f60440c40b0 2024-09-13T17:05:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f60427fc640 1 -- 172.21.15.39:0/2793776519 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0a5e20 con 0x7f60440c40b0 2024-09-13T17:05:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f604a23b640 1 -- 172.21.15.39:0/2793776519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60440c40b0 msgr2=0x7f604415c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f604a23b640 1 --2- 172.21.15.39:0/2793776519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60440c40b0 0x7f604415c470 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f60440c33a0 tx=0x7f603c01c870 comp rx=0 tx=0).stop 2024-09-13T17:05:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f604a23b640 1 -- 172.21.15.39:0/2793776519 shutdown_connections 2024-09-13T17:05:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f604a23b640 1 --2- 172.21.15.39:0/2793776519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60440c40b0 0x7f604415c470 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f604a23b640 1 --2- 172.21.15.39:0/2793776519 >> v2:172.21.15.84:3300/0 conn(0x7f60440aabc0 0x7f60440aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f604a23b640 1 -- 172.21.15.39:0/2793776519 >> 172.21.15.39:0/2793776519 conn(0x7f6044069e10 msgr2=0x7f604406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f604a23b640 1 -- 172.21.15.39:0/2793776519 shutdown_connections 2024-09-13T17:05:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.869+0000 7f604a23b640 1 -- 172.21.15.39:0/2793776519 wait complete. 2024-09-13T17:05:10.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 Processor -- start 2024-09-13T17:05:10.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 -- start start 2024-09-13T17:05:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f604406c870 0x7f6044150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60440aabc0 0x7f6044145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60440c40b0 0x7f6044145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60440a6790 con 0x7f604406c870 2024-09-13T17:05:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60440aaa30 con 0x7f60440c40b0 2024-09-13T17:05:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60441eccb0 con 0x7f60440aabc0 2024-09-13T17:05:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f60437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f604406c870 0x7f6044150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f60437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f604406c870 0x7f6044150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38566/0 (socket says 172.21.15.39:38566) 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f60437fe640 1 -- 172.21.15.39:0/3536786004 learned_addr learned my addr 172.21.15.39:0/3536786004 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f6043fff640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60440c40b0 0x7f6044145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f6042ffd640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60440aabc0 0x7f6044145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f60437fe640 1 -- 172.21.15.39:0/3536786004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60440aabc0 msgr2=0x7f6044145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f60437fe640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60440aabc0 0x7f6044145560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f60437fe640 1 -- 172.21.15.39:0/3536786004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60440c40b0 msgr2=0x7f6044145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f60437fe640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60440c40b0 0x7f6044145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f60437fe640 1 -- 172.21.15.39:0/3536786004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c009a70 con 0x7f604406c870 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f6043fff640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60440c40b0 0x7f6044145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f60437fe640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f604406c870 0x7f6044150d40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f603400d830 tx=0x7f603400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f602bfff640 1 -- 172.21.15.39:0/3536786004 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400b6c0 con 0x7f604406c870 2024-09-13T17:05:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f602bfff640 1 -- 172.21.15.39:0/3536786004 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603400b820 con 0x7f604406c870 2024-09-13T17:05:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f602bfff640 1 -- 172.21.15.39:0/3536786004 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400bb90 con 0x7f604406c870 2024-09-13T17:05:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60441ec380 con 0x7f604406c870 2024-09-13T17:05:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60441ec940 con 0x7f604406c870 2024-09-13T17:05:10.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.873+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60440aafa0 con 0x7f604406c870 2024-09-13T17:05:10.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.877+0000 7f602bfff640 1 -- 172.21.15.39:0/3536786004 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f603401e460 con 0x7f604406c870 2024-09-13T17:05:10.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.877+0000 7f602bfff640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f60300666c0 0x7f6030068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:10.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.877+0000 7f602bfff640 1 -- 172.21.15.39:0/3536786004 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f60340853e0 con 0x7f604406c870 2024-09-13T17:05:10.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.877+0000 7f6042ffd640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f60300666c0 0x7f6030068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.877+0000 7f6042ffd640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f60300666c0 0x7f6030068b80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f60441540a0 tx=0x7f6038009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:10.881+0000 7f602bfff640 1 -- 172.21.15.39:0/3536786004 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6034051c80 con 0x7f604406c870 2024-09-13T17:05:11.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.085+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f604406c650 con 0x7f604406c870 2024-09-13T17:05:11.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.085+0000 7f602bfff640 1 -- 172.21.15.39:0/3536786004 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+5177 (secure 0 0 0) 0x7f60340552a0 con 0x7f604406c870 2024-09-13T17:05:11.084 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:11.084 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":117,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:09.766272+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:11.084 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 117 2024-09-13T17:05:11.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f60300666c0 msgr2=0x7f6030068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:11.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f60300666c0 0x7f6030068b80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f60441540a0 tx=0x7f6038009040 comp rx=0 tx=0).stop 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f604406c870 msgr2=0x7f6044150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f604406c870 0x7f6044150d40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f603400d830 tx=0x7f603400dcf0 comp rx=0 tx=0).stop 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 shutdown_connections 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f60300666c0 0x7f6030068b80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60440c40b0 0x7f6044145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f60440aabc0 0x7f6044145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 --2- 172.21.15.39:0/3536786004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f604406c870 0x7f6044150d40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 >> 172.21.15.39:0/3536786004 conn(0x7f6044069e10 msgr2=0x7f604406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 shutdown_connections 2024-09-13T17:05:11.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:11.089+0000 7f604a23b640 1 -- 172.21.15.39:0/3536786004 wait complete. 2024-09-13T17:05:11.104 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 117, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 117, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6948": { "addr": "172.21.15.39:6835/4200926406", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 4200926406, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 4200926406, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6948, "incarnation": 117, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 47, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:05:09.766272+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6948 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 110, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6235": { "addr": "172.21.15.84:6835/263561599", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 263561599, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 263561599, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6235, "incarnation": 107, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 45, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:03:53.068981+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6235 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/440028459", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 440028459, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 440028459, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 96, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5561, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:05:13.105 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:13.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.265+0000 7f048a5ea640 1 Processor -- start 2024-09-13T17:05:13.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.265+0000 7f048a5ea640 1 -- start start 2024-09-13T17:05:13.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f048a5ea640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0484154bb0 0x7f04840c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:13.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f048a5ea640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f04840c5870 0x7f048415e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:13.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f048a5ea640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0484153e30 con 0x7f04840c2370 2024-09-13T17:05:13.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f048a5ea640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04840720f0 con 0x7f04840c5870 2024-09-13T17:05:13.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f048a5ea640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0484151b30 con 0x7f0484154bb0 2024-09-13T17:05:13.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f0488b87640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f04840c5870 0x7f048415e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:13.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f0483fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f04840c2370 0x7f04840c0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50494/0 (socket says 172.21.15.39:50494) 2024-09-13T17:05:13.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f0483fff640 1 -- 172.21.15.39:0/3398430322 learned_addr learned my addr 172.21.15.39:0/3398430322 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:13.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f047bfff640 1 --2- 172.21.15.39:0/3398430322 >> v2:172.21.15.84:3300/0 conn(0x7f0484154bb0 0x7f04840c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:13.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f0488b87640 1 -- 172.21.15.39:0/3398430322 >> v2:172.21.15.84:3300/0 conn(0x7f0484154bb0 msgr2=0x7f04840c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:13.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f0488b87640 1 --2- 172.21.15.39:0/3398430322 >> v2:172.21.15.84:3300/0 conn(0x7f0484154bb0 0x7f04840c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f0488b87640 1 -- 172.21.15.39:0/3398430322 >> v1:172.21.15.39:6789/0 conn(0x7f04840c2370 legacy=0x7f04840c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:13.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f0488b87640 1 -- 172.21.15.39:0/3398430322 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04840a7000 con 0x7f04840c5870 2024-09-13T17:05:13.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f04837fe640 1 -- 172.21.15.39:0/3398430322 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932585154 0 0) 0x7f0484153e30 con 0x7f04840c2370 2024-09-13T17:05:13.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f0488b87640 1 --2- 172.21.15.39:0/3398430322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c5870 0x7f048415e7b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0484066de0 tx=0x7f047c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca642263be8df9c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:13.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f04837fe640 1 -- 172.21.15.39:0/3398430322 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0a5850 con 0x7f04840c5870 2024-09-13T17:05:13.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f04837fe640 1 -- 172.21.15.39:0/3398430322 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c0a59b0 con 0x7f04840c5870 2024-09-13T17:05:13.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.269+0000 7f04837fe640 1 -- 172.21.15.39:0/3398430322 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0a5d40 con 0x7f04840c5870 2024-09-13T17:05:13.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- 172.21.15.39:0/3398430322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c5870 msgr2=0x7f048415e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:13.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 --2- 172.21.15.39:0/3398430322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c5870 0x7f048415e7b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0484066de0 tx=0x7f047c01c8a0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- 172.21.15.39:0/3398430322 shutdown_connections 2024-09-13T17:05:13.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 --2- 172.21.15.39:0/3398430322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c5870 0x7f048415e7b0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 --2- 172.21.15.39:0/3398430322 >> v2:172.21.15.84:3300/0 conn(0x7f0484154bb0 0x7f04840c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- 172.21.15.39:0/3398430322 >> 172.21.15.39:0/3398430322 conn(0x7f04840a9720 msgr2=0x7f04840a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:13.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- 172.21.15.39:0/3398430322 shutdown_connections 2024-09-13T17:05:13.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- 172.21.15.39:0/3398430322 wait complete. 2024-09-13T17:05:13.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 Processor -- start 2024-09-13T17:05:13.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- start start 2024-09-13T17:05:13.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c2370 0x7f048415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:13.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04840c5870 0x7f0484153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:13.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0484154bb0 0x7f0484143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:13.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04841ef960 con 0x7f04840c2370 2024-09-13T17:05:13.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048415ea00 con 0x7f04840c5870 2024-09-13T17:05:13.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04841efbb0 con 0x7f0484154bb0 2024-09-13T17:05:13.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0483fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c2370 0x7f048415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:13.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0483fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c2370 0x7f048415e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38576/0 (socket says 172.21.15.39:38576) 2024-09-13T17:05:13.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0483fff640 1 -- 172.21.15.39:0/125884949 learned_addr learned my addr 172.21.15.39:0/125884949 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:13.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0488b87640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0484154bb0 0x7f0484143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:13.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f047bfff640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04840c5870 0x7f0484153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:13.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0483fff640 1 -- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0484154bb0 msgr2=0x7f0484143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:13.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0483fff640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0484154bb0 0x7f0484143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0483fff640 1 -- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04840c5870 msgr2=0x7f0484153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:13.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0483fff640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04840c5870 0x7f0484153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0483fff640 1 -- 172.21.15.39:0/125884949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047c009a70 con 0x7f04840c2370 2024-09-13T17:05:13.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0488b87640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0484154bb0 0x7f0484143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:13.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f047bfff640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04840c5870 0x7f0484153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:13.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f0483fff640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c2370 0x7f048415e4c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f047400c520 tx=0x7f047400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:13.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f04817fa640 1 -- 172.21.15.39:0/125884949 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04740074e0 con 0x7f04840c2370 2024-09-13T17:05:13.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f04817fa640 1 -- 172.21.15.39:0/125884949 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0474007640 con 0x7f04840c2370 2024-09-13T17:05:13.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.273+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04841efe90 con 0x7f04840c2370 2024-09-13T17:05:13.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.277+0000 7f04817fa640 1 -- 172.21.15.39:0/125884949 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04740049f0 con 0x7f04840c2370 2024-09-13T17:05:13.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.277+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04841f0450 con 0x7f04840c2370 2024-09-13T17:05:13.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.277+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04840c2750 con 0x7f04840c2370 2024-09-13T17:05:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.277+0000 7f04817fa640 1 -- 172.21.15.39:0/125884949 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0474004b50 con 0x7f04840c2370 2024-09-13T17:05:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.277+0000 7f04817fa640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0464066690 0x7f0464068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.277+0000 7f04817fa640 1 -- 172.21.15.39:0/125884949 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f04740842d0 con 0x7f04840c2370 2024-09-13T17:05:13.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.277+0000 7f047bfff640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0464066690 0x7f0464068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:13.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.277+0000 7f047bfff640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0464066690 0x7f0464068b50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f048414b040 tx=0x7f046c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:13.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.281+0000 7f04817fa640 1 -- 172.21.15.39:0/125884949 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0474050bb0 con 0x7f04840c2370 2024-09-13T17:05:13.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.525+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04840ad840 con 0x7f04840c2370 2024-09-13T17:05:13.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.525+0000 7f04817fa640 1 -- 172.21.15.39:0/125884949 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+5178 (secure 0 0 0) 0x7f0474018070 con 0x7f04840c2370 2024-09-13T17:05:13.525 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:13.525 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":120,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:13.525 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 120 2024-09-13T17:05:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0464066690 msgr2=0x7f0464068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0464066690 0x7f0464068b50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f048414b040 tx=0x7f046c008040 comp rx=0 tx=0).stop 2024-09-13T17:05:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c2370 msgr2=0x7f048415e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c2370 0x7f048415e4c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f047400c520 tx=0x7f047400c9e0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 shutdown_connections 2024-09-13T17:05:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0464066690 0x7f0464068b50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f04840c5870 0x7f0484153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0484154bb0 0x7f0484143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 --2- 172.21.15.39:0/125884949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04840c2370 0x7f048415e4c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 >> 172.21.15.39:0/125884949 conn(0x7f04840a9720 msgr2=0x7f04840a8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:13.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 shutdown_connections 2024-09-13T17:05:13.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:13.529+0000 7f048a5ea640 1 -- 172.21.15.39:0/125884949 wait complete. 2024-09-13T17:05:13.549 INFO:tasks.mds_thrash.fs.[a]:mds.a has gained rank=0, replacing gid=6858 2024-09-13T17:05:13.550 INFO:tasks.mds_thrash.fs.[a]:waiting for 1 secs before reviving mds.c 2024-09-13T17:05:14.551 INFO:tasks.mds_thrash.fs.[a]:reviving mds.c 2024-09-13T17:05:14.551 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:05:14.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:05:14.554 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:05:14.554 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:14.642 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T17:05:14.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.693+0000 7ff32848f640 1 Processor -- start 2024-09-13T17:05:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.693+0000 7ff32848f640 1 -- start start 2024-09-13T17:05:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff32848f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff320151500 0x7ff3200c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff32848f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff320071dd0 0x7ff320154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff32848f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3200c5a30 con 0x7ff3200c47c0 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff32848f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3201576a0 con 0x7ff320071dd0 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff32848f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff320157980 con 0x7ff320151500 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325a2a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff320071dd0 0x7ff320154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff326a2c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff3200c47c0 0x7ff3201545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50510/0 (socket says 172.21.15.39:50510) 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325a2a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff320071dd0 0x7ff320154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38598/0 (socket says 172.21.15.39:38598) 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff326a2c640 1 -- 172.21.15.39:0/8861785 learned_addr learned my addr 172.21.15.39:0/8861785 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff32622b640 1 --2- 172.21.15.39:0/8861785 >> v2:172.21.15.84:3300/0 conn(0x7ff320151500 0x7ff3200c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325a2a640 1 -- 172.21.15.39:0/8861785 >> v2:172.21.15.84:3300/0 conn(0x7ff320151500 msgr2=0x7ff3200c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325a2a640 1 --2- 172.21.15.39:0/8861785 >> v2:172.21.15.84:3300/0 conn(0x7ff320151500 0x7ff3200c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325a2a640 1 -- 172.21.15.39:0/8861785 >> v1:172.21.15.39:6789/0 conn(0x7ff3200c47c0 legacy=0x7ff3201545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325a2a640 1 -- 172.21.15.39:0/8861785 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff320157c60 con 0x7ff320071dd0 2024-09-13T17:05:14.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325229640 1 -- 172.21.15.39:0/8861785 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228884116 0 0) 0x7ff3200c5a30 con 0x7ff3200c47c0 2024-09-13T17:05:14.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325a2a640 1 --2- 172.21.15.39:0/8861785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff320071dd0 0x7ff320154070 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff3100097c0 tx=0x7ff31001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73dddb6e05a292c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:14.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325229640 1 -- 172.21.15.39:0/8861785 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100ac620 con 0x7ff320071dd0 2024-09-13T17:05:14.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325229640 1 -- 172.21.15.39:0/8861785 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3100ac780 con 0x7ff320071dd0 2024-09-13T17:05:14.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.697+0000 7ff325229640 1 -- 172.21.15.39:0/8861785 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100acb10 con 0x7ff320071dd0 2024-09-13T17:05:14.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 -- 172.21.15.39:0/8861785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff320071dd0 msgr2=0x7ff320154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:14.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 --2- 172.21.15.39:0/8861785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff320071dd0 0x7ff320154070 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff3100097c0 tx=0x7ff31001c3a0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 -- 172.21.15.39:0/8861785 shutdown_connections 2024-09-13T17:05:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 --2- 172.21.15.39:0/8861785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff320071dd0 0x7ff320154070 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 --2- 172.21.15.39:0/8861785 >> v2:172.21.15.84:3300/0 conn(0x7ff320151500 0x7ff3200c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 -- 172.21.15.39:0/8861785 >> 172.21.15.39:0/8861785 conn(0x7ff3200ab530 msgr2=0x7ff3200ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 -- 172.21.15.39:0/8861785 shutdown_connections 2024-09-13T17:05:14.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 -- 172.21.15.39:0/8861785 wait complete. 2024-09-13T17:05:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 Processor -- start 2024-09-13T17:05:14.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 -- start start 2024-09-13T17:05:14.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff320071dd0 0x7ff3200bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:14.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3200c47c0 0x7ff320151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:14.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff320151500 0x7ff32013dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:14.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320153140 con 0x7ff3200c47c0 2024-09-13T17:05:14.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200ab210 con 0x7ff320071dd0 2024-09-13T17:05:14.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32848f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff320152560 con 0x7ff320151500 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff325a2a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3200c47c0 0x7ff320151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff325a2a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3200c47c0 0x7ff320151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38612/0 (socket says 172.21.15.39:38612) 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff325a2a640 1 -- 172.21.15.39:0/2383063071 learned_addr learned my addr 172.21.15.39:0/2383063071 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff326a2c640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff320151500 0x7ff32013dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff32622b640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff320071dd0 0x7ff3200bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff325a2a640 1 -- 172.21.15.39:0/2383063071 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff320151500 msgr2=0x7ff32013dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff325a2a640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff320151500 0x7ff32013dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff325a2a640 1 -- 172.21.15.39:0/2383063071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff320071dd0 msgr2=0x7ff3200bf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff325a2a640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff320071dd0 0x7ff3200bf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.701+0000 7ff325a2a640 1 -- 172.21.15.39:0/2383063071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff310009420 con 0x7ff3200c47c0 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff325a2a640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3200c47c0 0x7ff320151c20 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff3100af040 tx=0x7ff310019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff316ffd640 1 -- 172.21.15.39:0/2383063071 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100b5410 con 0x7ff3200c47c0 2024-09-13T17:05:14.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff316ffd640 1 -- 172.21.15.39:0/2383063071 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3100b5570 con 0x7ff3200c47c0 2024-09-13T17:05:14.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff316ffd640 1 -- 172.21.15.39:0/2383063071 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100b5920 con 0x7ff3200c47c0 2024-09-13T17:05:14.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3201f0a30 con 0x7ff3200c47c0 2024-09-13T17:05:14.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff320152d60 con 0x7ff3200c47c0 2024-09-13T17:05:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff320153280 con 0x7ff3200c47c0 2024-09-13T17:05:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff316ffd640 1 -- 172.21.15.39:0/2383063071 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3100b3040 con 0x7ff3200c47c0 2024-09-13T17:05:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff316ffd640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3040666c0 0x7ff304068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff316ffd640 1 -- 172.21.15.39:0/2383063071 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7ff31011edd0 con 0x7ff3200c47c0 2024-09-13T17:05:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff32622b640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3040666c0 0x7ff304068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:14.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.705+0000 7ff32622b640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3040666c0 0x7ff304068b80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff32013f8b0 tx=0x7ff31c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:14.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.709+0000 7ff316ffd640 1 -- 172.21.15.39:0/2383063071 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff320153280 con 0x7ff3200c47c0 2024-09-13T17:05:14.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.965+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff320153280 con 0x7ff3200c47c0 2024-09-13T17:05:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.965+0000 7ff316ffd640 1 -- 172.21.15.39:0/2383063071 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+5178 (secure 0 0 0) 0x7ff320153280 con 0x7ff3200c47c0 2024-09-13T17:05:14.965 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:14.965 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":120,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 120 2024-09-13T17:05:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3040666c0 msgr2=0x7ff304068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3040666c0 0x7ff304068b80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff32013f8b0 tx=0x7ff31c008040 comp rx=0 tx=0).stop 2024-09-13T17:05:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3200c47c0 msgr2=0x7ff320151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3200c47c0 0x7ff320151c20 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff3100af040 tx=0x7ff310019140 comp rx=0 tx=0).stop 2024-09-13T17:05:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 shutdown_connections 2024-09-13T17:05:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3040666c0 0x7ff304068b80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff320151500 0x7ff32013dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3200c47c0 0x7ff320151c20 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 --2- 172.21.15.39:0/2383063071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff320071dd0 0x7ff3200bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 >> 172.21.15.39:0/2383063071 conn(0x7ff3200ab530 msgr2=0x7ff3200ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 shutdown_connections 2024-09-13T17:05:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:14.969+0000 7ff32848f640 1 -- 172.21.15.39:0/2383063071 wait complete. 2024-09-13T17:05:14.985 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-09-13T17:05:16.986 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:17.049 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T17:05:17.049+0000 7f94976c0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:05:17.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f7000dbd640 1 Processor -- start 2024-09-13T17:05:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f7000dbd640 1 -- start start 2024-09-13T17:05:17.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f7000dbd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6ffc14aa40 0x7f6ffc0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f7000dbd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6ffc0c4710 0x7f6ffc15b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f7000dbd640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc0c76a0 con 0x7f6ffc0c1750 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f7000dbd640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ffc151b30 con 0x7f6ffc14aa40 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f7000dbd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ffc151570 con 0x7f6ffc0c4710 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f6ffa59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6ffc14aa40 0x7f6ffc0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f6ffa59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6ffc14aa40 0x7f6ffc0c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38620/0 (socket says 172.21.15.39:38620) 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f6ffa59c640 1 -- 172.21.15.39:0/1516665191 learned_addr learned my addr 172.21.15.39:0/1516665191 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f6ffad9d640 1 --2- 172.21.15.39:0/1516665191 >> v2:172.21.15.84:3300/0 conn(0x7f6ffc0c4710 0x7f6ffc15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f6ffa59c640 1 -- 172.21.15.39:0/1516665191 >> v2:172.21.15.84:3300/0 conn(0x7f6ffc0c4710 msgr2=0x7f6ffc15b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f6ffa59c640 1 --2- 172.21.15.39:0/1516665191 >> v2:172.21.15.84:3300/0 conn(0x7f6ffc0c4710 0x7f6ffc15b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f6ffa59c640 1 -- 172.21.15.39:0/1516665191 >> v1:172.21.15.84:6789/0 conn(0x7f6ffc0c1750 legacy=0x7f6ffc0c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f6ffa59c640 1 -- 172.21.15.39:0/1516665191 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc151850 con 0x7f6ffc14aa40 2024-09-13T17:05:17.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.141+0000 7f6ffad9d640 1 --2- 172.21.15.39:0/1516665191 >> v2:172.21.15.84:3300/0 conn(0x7f6ffc0c4710 0x7f6ffc15b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f6ffa59c640 1 --2- 172.21.15.39:0/1516665191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 0x7f6ffc0c1210 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0c24a0 tx=0x7f6fec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424fb618753a358d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f6ff9d9b640 1 -- 172.21.15.39:0/1516665191 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0ab660 con 0x7f6ffc14aa40 2024-09-13T17:05:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f6ff9d9b640 1 -- 172.21.15.39:0/1516665191 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec0ab7c0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f6ff9d9b640 1 -- 172.21.15.39:0/1516665191 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0abb50 con 0x7f6ffc14aa40 2024-09-13T17:05:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f7000dbd640 1 -- 172.21.15.39:0/1516665191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 msgr2=0x7f6ffc0c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:17.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f7000dbd640 1 --2- 172.21.15.39:0/1516665191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 0x7f6ffc0c1210 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0c24a0 tx=0x7f6fec01c460 comp rx=0 tx=0).stop 2024-09-13T17:05:17.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f7000dbd640 1 -- 172.21.15.39:0/1516665191 shutdown_connections 2024-09-13T17:05:17.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f7000dbd640 1 --2- 172.21.15.39:0/1516665191 >> v2:172.21.15.84:3300/0 conn(0x7f6ffc0c4710 0x7f6ffc15b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f7000dbd640 1 --2- 172.21.15.39:0/1516665191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 0x7f6ffc0c1210 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f7000dbd640 1 -- 172.21.15.39:0/1516665191 >> 172.21.15.39:0/1516665191 conn(0x7f6ffc0c3310 msgr2=0x7f6ffc0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:17.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f7000dbd640 1 -- 172.21.15.39:0/1516665191 shutdown_connections 2024-09-13T17:05:17.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.145+0000 7f7000dbd640 1 -- 172.21.15.39:0/1516665191 wait complete. 2024-09-13T17:05:17.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 Processor -- start 2024-09-13T17:05:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 -- start start 2024-09-13T17:05:17.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ffc0c1750 0x7f6ffc153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ffc0c4710 0x7f6ffc144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 0x7f6ffc070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc152990 con 0x7f6ffc14aa40 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc154f50 con 0x7f6ffc0c4710 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc151ff0 con 0x7f6ffc0c1750 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 0x7f6ffc070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 0x7f6ffc070340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38624/0 (socket says 172.21.15.39:38624) 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffad9d640 1 -- 172.21.15.39:0/4237987636 learned_addr learned my addr 172.21.15.39:0/4237987636 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ff1d9b640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ffc0c4710 0x7f6ffc144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffa59c640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ffc0c1750 0x7f6ffc153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffad9d640 1 -- 172.21.15.39:0/4237987636 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ffc0c1750 msgr2=0x7f6ffc153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffad9d640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ffc0c1750 0x7f6ffc153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffad9d640 1 -- 172.21.15.39:0/4237987636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ffc0c4710 msgr2=0x7f6ffc144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffad9d640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ffc0c4710 0x7f6ffc144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffad9d640 1 -- 172.21.15.39:0/4237987636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0094d0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ffad9d640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 0x7f6ffc070340 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4007d70 tx=0x7f6ff400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ff37fe640 1 -- 172.21.15.39:0/4237987636 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff40039a0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ff37fe640 1 -- 172.21.15.39:0/4237987636 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff400f040 con 0x7f6ffc14aa40 2024-09-13T17:05:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc1522d0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f6ff37fe640 1 -- 172.21.15.39:0/4237987636 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4003d40 con 0x7f6ffc14aa40 2024-09-13T17:05:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc1446b0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.149+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc152ad0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.153+0000 7f6ff37fe640 1 -- 172.21.15.39:0/4237987636 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6ff4004030 con 0x7f6ffc14aa40 2024-09-13T17:05:17.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.153+0000 7f6ff37fe640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fdc066690 0x7f6fdc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:17.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.153+0000 7f6ff37fe640 1 -- 172.21.15.39:0/4237987636 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f6ff4084dd0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.153+0000 7f6ffa59c640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fdc066690 0x7f6fdc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:17.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.153+0000 7f6ffa59c640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fdc066690 0x7f6fdc068b50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0af000 tx=0x7f6fec0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:17.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.157+0000 7f6ff37fe640 1 -- 172.21.15.39:0/4237987636 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6ffc152ad0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.389+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ffc152ad0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f6ff37fe640 1 -- 172.21.15.39:0/4237987636 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f6ffc152ad0 con 0x7f6ffc14aa40 2024-09-13T17:05:17.390 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:17.390 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:17.390 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:17.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fdc066690 msgr2=0x7f6fdc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:17.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fdc066690 0x7f6fdc068b50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0af000 tx=0x7f6fec0a7000 comp rx=0 tx=0).stop 2024-09-13T17:05:17.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 msgr2=0x7f6ffc070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:17.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 0x7f6ffc070340 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4007d70 tx=0x7f6ff400d4a0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 shutdown_connections 2024-09-13T17:05:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6fdc066690 0x7f6fdc068b50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ffc0c4710 0x7f6ffc144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ffc14aa40 0x7f6ffc070340 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 --2- 172.21.15.39:0/4237987636 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ffc0c1750 0x7f6ffc153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 >> 172.21.15.39:0/4237987636 conn(0x7f6ffc0c3310 msgr2=0x7f6ffc144cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 shutdown_connections 2024-09-13T17:05:17.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:17.393+0000 7f7000dbd640 1 -- 172.21.15.39:0/4237987636 wait complete. 2024-09-13T17:05:17.412 INFO:tasks.mds_thrash.fs.[a]:mds.c reported in up:standby state 2024-09-13T17:05:17.412 INFO:tasks.mds_thrash.fs.[a]:waiting for 27 secs before thrashing 2024-09-13T17:05:29.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:05:29.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:05:36.080 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:36.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.228+0000 7f57f7686640 1 Processor -- start 2024-09-13T17:05:36.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.228+0000 7f57f7686640 1 -- start start 2024-09-13T17:05:36.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f7686640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f00c21d0 0x7f57f00aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f7686640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f00c3ae0 con 0x7f57f006aac0 2024-09-13T17:05:36.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f7686640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f00abb30 con 0x7f57f014b850 2024-09-13T17:05:36.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f7686640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00ab340 con 0x7f57f00c21d0 2024-09-13T17:05:36.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f4c21640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f57f006aac0 0x7f57f0151c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57346/0 (socket says 172.21.15.39:57346) 2024-09-13T17:05:36.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f4c21640 1 -- 172.21.15.39:0/35016522 learned_addr learned my addr 172.21.15.39:0/35016522 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:36.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f5422640 1 --2- 172.21.15.39:0/35016522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f00c21d0 0x7f57f00aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57e7fff640 1 -- 172.21.15.39:0/35016522 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796133500 0 0) 0x7f57f00c3ae0 con 0x7f57f006aac0 2024-09-13T17:05:36.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57e7fff640 1 -- 172.21.15.39:0/35016522 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57dc003530 con 0x7f57f006aac0 2024-09-13T17:05:36.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f5422640 1 -- 172.21.15.39:0/35016522 >> v1:172.21.15.84:6789/0 conn(0x7f57f014b850 legacy=0x7f57f015c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f5422640 1 -- 172.21.15.39:0/35016522 >> v1:172.21.15.39:6789/0 conn(0x7f57f006aac0 legacy=0x7f57f0151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57e7fff640 1 -- 172.21.15.39:0/35016522 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273315954 0 0) 0x7f57f00abb30 con 0x7f57f014b850 2024-09-13T17:05:36.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f5422640 1 -- 172.21.15.39:0/35016522 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00abe10 con 0x7f57f00c21d0 2024-09-13T17:05:36.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.232+0000 7f57f5422640 1 --2- 172.21.15.39:0/35016522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f00c21d0 0x7f57f00aae00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0097f0 tx=0x7f57ec01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f0ef9c5c4263f66 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:36.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57e7fff640 1 -- 172.21.15.39:0/35016522 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0ab620 con 0x7f57f00c21d0 2024-09-13T17:05:36.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57e7fff640 1 -- 172.21.15.39:0/35016522 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec0ab780 con 0x7f57f00c21d0 2024-09-13T17:05:36.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57e7fff640 1 -- 172.21.15.39:0/35016522 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0abb10 con 0x7f57f00c21d0 2024-09-13T17:05:36.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 -- 172.21.15.39:0/35016522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f00c21d0 msgr2=0x7f57f00aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 --2- 172.21.15.39:0/35016522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f00c21d0 0x7f57f00aae00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0097f0 tx=0x7f57ec01c420 comp rx=0 tx=0).stop 2024-09-13T17:05:36.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 -- 172.21.15.39:0/35016522 shutdown_connections 2024-09-13T17:05:36.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 --2- 172.21.15.39:0/35016522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f00c21d0 0x7f57f00aae00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 -- 172.21.15.39:0/35016522 >> 172.21.15.39:0/35016522 conn(0x7f57f0069e10 msgr2=0x7f57f006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:36.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 -- 172.21.15.39:0/35016522 shutdown_connections 2024-09-13T17:05:36.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 -- 172.21.15.39:0/35016522 wait complete. 2024-09-13T17:05:36.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 Processor -- start 2024-09-13T17:05:36.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 -- start start 2024-09-13T17:05:36.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f006aac0 0x7f57f00bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57f00c21d0 0x7f57f00bd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f014b850 0x7f57f0149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00bfe80 con 0x7f57f00c21d0 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00bcb40 con 0x7f57f006aac0 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f7686640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0154ff0 con 0x7f57f014b850 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f4c21640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57f00c21d0 0x7f57f00bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f4c21640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57f00c21d0 0x7f57f00bd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55780/0 (socket says 172.21.15.39:55780) 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f5422640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f006aac0 0x7f57f00bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f4c21640 1 -- 172.21.15.39:0/3457304501 learned_addr learned my addr 172.21.15.39:0/3457304501 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.236+0000 7f57f5c23640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f014b850 0x7f57f0149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f4c21640 1 -- 172.21.15.39:0/3457304501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f014b850 msgr2=0x7f57f0149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f4c21640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f014b850 0x7f57f0149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f4c21640 1 -- 172.21.15.39:0/3457304501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f006aac0 msgr2=0x7f57f00bcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f4c21640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f006aac0 0x7f57f00bcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f4c21640 1 -- 172.21.15.39:0/3457304501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009420 con 0x7f57f00c21d0 2024-09-13T17:05:36.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f4c21640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57f00c21d0 0x7f57f00bd510 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c460 tx=0x7f57e000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:36.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57e5ffb640 1 -- 172.21.15.39:0/3457304501 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0005cf0 con 0x7f57f00c21d0 2024-09-13T17:05:36.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57e5ffb640 1 -- 172.21.15.39:0/3457304501 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0005e50 con 0x7f57f00c21d0 2024-09-13T17:05:36.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57e5ffb640 1 -- 172.21.15.39:0/3457304501 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0007290 con 0x7f57f00c21d0 2024-09-13T17:05:36.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f014ab30 con 0x7f57f00c21d0 2024-09-13T17:05:36.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f00c03b0 con 0x7f57f00c21d0 2024-09-13T17:05:36.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f00c3560 con 0x7f57f00c21d0 2024-09-13T17:05:36.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57e5ffb640 1 -- 172.21.15.39:0/3457304501 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57e00073f0 con 0x7f57f00c21d0 2024-09-13T17:05:36.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57e5ffb640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57d40666c0 0x7f57d4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57e5ffb640 1 -- 172.21.15.39:0/3457304501 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f57e0084080 con 0x7f57f00c21d0 2024-09-13T17:05:36.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.240+0000 7f57f5422640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57d40666c0 0x7f57d4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.244+0000 7f57f5422640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57d40666c0 0x7f57d4068b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0af000 tx=0x7f57ec0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:36.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.248+0000 7f57e5ffb640 1 -- 172.21.15.39:0/3457304501 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f57e00509d0 con 0x7f57f00c21d0 2024-09-13T17:05:36.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.468+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57f00bcc80 con 0x7f57f00c21d0 2024-09-13T17:05:36.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.468+0000 7f57e5ffb640 1 -- 172.21.15.39:0/3457304501 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f57e0017070 con 0x7f57f00c21d0 2024-09-13T17:05:36.467 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:36.467 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:36.468 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.468+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57d40666c0 msgr2=0x7f57d4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.468+0000 7f57f7686640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57d40666c0 0x7f57d4068b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0af000 tx=0x7f57ec0a7000 comp rx=0 tx=0).stop 2024-09-13T17:05:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.468+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57f00c21d0 msgr2=0x7f57f00bd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.468+0000 7f57f7686640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57f00c21d0 0x7f57f00bd510 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c460 tx=0x7f57e000c920 comp rx=0 tx=0).stop 2024-09-13T17:05:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.468+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 shutdown_connections 2024-09-13T17:05:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.468+0000 7f57f7686640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57d40666c0 0x7f57d4068b80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.472+0000 7f57f7686640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57f014b850 0x7f57f0149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.472+0000 7f57f7686640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57f00c21d0 0x7f57f00bd510 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.472+0000 7f57f7686640 1 --2- 172.21.15.39:0/3457304501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f006aac0 0x7f57f00bcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.472+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 >> 172.21.15.39:0/3457304501 conn(0x7f57f0069e10 msgr2=0x7f57f00c3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:36.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.472+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 shutdown_connections 2024-09-13T17:05:36.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.472+0000 7f57f7686640 1 -- 172.21.15.39:0/3457304501 wait complete. 2024-09-13T17:05:36.489 INFO:tasks.mds_thrash.fs.[b]:kill mds.d (rank=0) 2024-09-13T17:05:36.489 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T17:05:36.489 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:05:36.506 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T17:05:36.507 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:36.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.644+0000 7f2f1a325640 1 Processor -- start 2024-09-13T17:05:36.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.644+0000 7f2f1a325640 1 -- start start 2024-09-13T17:05:36.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f1a325640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2f140c4260 0x7f2f140c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f1a325640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140bcfb0 0x7f2f14070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f1a325640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f14157360 con 0x7f2f140c77e0 2024-09-13T17:05:36.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f188c2640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2f140c77e0 0x7f2f14153df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57352/0 (socket says 172.21.15.39:57352) 2024-09-13T17:05:36.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f13fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2f140c4260 0x7f2f140c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f188c2640 1 -- 172.21.15.39:0/2912094143 learned_addr learned my addr 172.21.15.39:0/2912094143 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:36.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f1a325640 1 -- 172.21.15.39:0/2912094143 --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f140c80b0 con 0x7f2f140c4260 2024-09-13T17:05:36.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f1a325640 1 -- 172.21.15.39:0/2912094143 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f141575b0 con 0x7f2f140bcfb0 2024-09-13T17:05:36.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f137fe640 1 --2- 172.21.15.39:0/2912094143 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140bcfb0 0x7f2f14070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f12ffd640 1 -- 172.21.15.39:0/2912094143 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736142998 0 0) 0x7f2f14157360 con 0x7f2f140c77e0 2024-09-13T17:05:36.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f12ffd640 1 -- 172.21.15.39:0/2912094143 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2efc003530 con 0x7f2f140c77e0 2024-09-13T17:05:36.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f13fff640 1 -- 172.21.15.39:0/2912094143 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140bcfb0 msgr2=0x7f2f14070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:36.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f13fff640 1 --2- 172.21.15.39:0/2912094143 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140bcfb0 0x7f2f14070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f13fff640 1 -- 172.21.15.39:0/2912094143 >> v1:172.21.15.39:6789/0 conn(0x7f2f140c77e0 legacy=0x7f2f14153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f13fff640 1 -- 172.21.15.39:0/2912094143 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f14157890 con 0x7f2f140c4260 2024-09-13T17:05:36.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f13fff640 1 --2- 172.21.15.39:0/2912094143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140c4260 0x7f2f140c4640 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2f040097c0 tx=0x7f2f0401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff590eebe264ec47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:36.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f12ffd640 1 -- 172.21.15.39:0/2912094143 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14154d10 con 0x7f2f140c4260 2024-09-13T17:05:36.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f12ffd640 1 -- 172.21.15.39:0/2912094143 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f040a7050 con 0x7f2f140c4260 2024-09-13T17:05:36.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f12ffd640 1 -- 172.21.15.39:0/2912094143 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040ab820 con 0x7f2f140c4260 2024-09-13T17:05:36.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f1a325640 1 -- 172.21.15.39:0/2912094143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140c4260 msgr2=0x7f2f140c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.648+0000 7f2f1a325640 1 --2- 172.21.15.39:0/2912094143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140c4260 0x7f2f140c4640 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2f040097c0 tx=0x7f2f0401c380 comp rx=0 tx=0).stop 2024-09-13T17:05:36.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- 172.21.15.39:0/2912094143 shutdown_connections 2024-09-13T17:05:36.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 --2- 172.21.15.39:0/2912094143 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140bcfb0 0x7f2f14070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 --2- 172.21.15.39:0/2912094143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140c4260 0x7f2f140c4640 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- 172.21.15.39:0/2912094143 >> 172.21.15.39:0/2912094143 conn(0x7f2f14069e10 msgr2=0x7f2f1406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:36.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- 172.21.15.39:0/2912094143 shutdown_connections 2024-09-13T17:05:36.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- 172.21.15.39:0/2912094143 wait complete. 2024-09-13T17:05:36.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 Processor -- start 2024-09-13T17:05:36.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- start start 2024-09-13T17:05:36.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140bcfb0 0x7f2f141501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f140c4260 0x7f2f141edd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140c77e0 0x7f2f141ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f141ee800 con 0x7f2f140bcfb0 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f14148010 con 0x7f2f140c4260 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f14147d70 con 0x7f2f140c77e0 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f13fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140bcfb0 0x7f2f141501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f13fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140bcfb0 0x7f2f141501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55810/0 (socket says 172.21.15.39:55810) 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f13fff640 1 -- 172.21.15.39:0/974238260 learned_addr learned my addr 172.21.15.39:0/974238260 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f188c2640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140c77e0 0x7f2f141ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f137fe640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f140c4260 0x7f2f141edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f13fff640 1 -- 172.21.15.39:0/974238260 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140c77e0 msgr2=0x7f2f141ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f13fff640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140c77e0 0x7f2f141ee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f13fff640 1 -- 172.21.15.39:0/974238260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f140c4260 msgr2=0x7f2f141edd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f13fff640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f140c4260 0x7f2f141edd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f13fff640 1 -- 172.21.15.39:0/974238260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f04009420 con 0x7f2f140bcfb0 2024-09-13T17:05:36.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f13fff640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140bcfb0 0x7f2f141501d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2f040ae040 tx=0x7f2f0401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:36.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f10ff9640 1 -- 172.21.15.39:0/974238260 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040b7450 con 0x7f2f140bcfb0 2024-09-13T17:05:36.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f10ff9640 1 -- 172.21.15.39:0/974238260 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f040b75d0 con 0x7f2f140bcfb0 2024-09-13T17:05:36.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f10ff9640 1 -- 172.21.15.39:0/974238260 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040b7980 con 0x7f2f140bcfb0 2024-09-13T17:05:36.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f141f04a0 con 0x7f2f140bcfb0 2024-09-13T17:05:36.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.652+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f141ed880 con 0x7f2f140bcfb0 2024-09-13T17:05:36.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.656+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f141eda10 con 0x7f2f140bcfb0 2024-09-13T17:05:36.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.656+0000 7f2f10ff9640 1 -- 172.21.15.39:0/974238260 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f040b7ae0 con 0x7f2f140bcfb0 2024-09-13T17:05:36.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.656+0000 7f2f10ff9640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f00066690 0x7f2f00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:36.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.656+0000 7f2f10ff9640 1 -- 172.21.15.39:0/974238260 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f2f0411f380 con 0x7f2f140bcfb0 2024-09-13T17:05:36.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.656+0000 7f2f137fe640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f00066690 0x7f2f00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:36.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.656+0000 7f2f137fe640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f00066690 0x7f2f00068b50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2f14150c10 tx=0x7f2f08009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:36.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.660+0000 7f2f10ff9640 1 -- 172.21.15.39:0/974238260 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2f040ebc40 con 0x7f2f140bcfb0 2024-09-13T17:05:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.912+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f1414d440 con 0x7f2f140bcfb0 2024-09-13T17:05:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f10ff9640 1 -- 172.21.15.39:0/974238260 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f2f040b0090 con 0x7f2f140bcfb0 2024-09-13T17:05:36.915 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:36.915 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f00066690 msgr2=0x7f2f00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f00066690 0x7f2f00068b50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2f14150c10 tx=0x7f2f08009190 comp rx=0 tx=0).stop 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140bcfb0 msgr2=0x7f2f141501d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140bcfb0 0x7f2f141501d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2f040ae040 tx=0x7f2f0401abc0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 shutdown_connections 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2f00066690 0x7f2f00068b50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f140c4260 0x7f2f141edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2f140c77e0 0x7f2f141ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 --2- 172.21.15.39:0/974238260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f140bcfb0 0x7f2f141501d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 >> 172.21.15.39:0/974238260 conn(0x7f2f14069e10 msgr2=0x7f2f1415eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 shutdown_connections 2024-09-13T17:05:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:36.916+0000 7f2f1a325640 1 -- 172.21.15.39:0/974238260 wait complete. 2024-09-13T17:05:36.934 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:05:38.935 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:39.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.084+0000 7f6e5c805640 1 Processor -- start 2024-09-13T17:05:39.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.084+0000 7f6e5c805640 1 -- start start 2024-09-13T17:05:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e5c805640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 0x7f6e5806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e5c805640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6e581576e0 0x7f6e58157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e5c805640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e580c15d0 con 0x7f6e580c1810 2024-09-13T17:05:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e5c805640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5806d0b0 con 0x7f6e581576e0 2024-09-13T17:05:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e5c805640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580c1f00 con 0x7f6e5806b080 2024-09-13T17:05:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e56d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6e581576e0 0x7f6e58157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e56d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6e581576e0 0x7f6e58157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55820/0 (socket says 172.21.15.39:55820) 2024-09-13T17:05:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e56d9d640 1 -- 172.21.15.39:0/3146581273 learned_addr learned my addr 172.21.15.39:0/3146581273 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e5659c640 1 --2- 172.21.15.39:0/3146581273 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 0x7f6e5806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:39.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e56d9d640 1 -- 172.21.15.39:0/3146581273 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 msgr2=0x7f6e5806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:39.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e56d9d640 1 --2- 172.21.15.39:0/3146581273 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 0x7f6e5806e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:39.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e56d9d640 1 -- 172.21.15.39:0/3146581273 >> v1:172.21.15.84:6789/0 conn(0x7f6e580c1810 legacy=0x7f6e58151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:39.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e56d9d640 1 -- 172.21.15.39:0/3146581273 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580c21a0 con 0x7f6e581576e0 2024-09-13T17:05:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e56d9d640 1 --2- 172.21.15.39:0/3146581273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e581576e0 0x7f6e58157af0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009e50 tx=0x7f6e5001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c9baf56efd70b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.088+0000 7f6e5559a640 1 -- 172.21.15.39:0/3146581273 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e500a57e0 con 0x7f6e581576e0 2024-09-13T17:05:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5559a640 1 -- 172.21.15.39:0/3146581273 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e500a5940 con 0x7f6e581576e0 2024-09-13T17:05:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5559a640 1 -- 172.21.15.39:0/3146581273 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e500a5cd0 con 0x7f6e581576e0 2024-09-13T17:05:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- 172.21.15.39:0/3146581273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e581576e0 msgr2=0x7f6e58157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 --2- 172.21.15.39:0/3146581273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e581576e0 0x7f6e58157af0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009e50 tx=0x7f6e5001ca00 comp rx=0 tx=0).stop 2024-09-13T17:05:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- 172.21.15.39:0/3146581273 shutdown_connections 2024-09-13T17:05:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 --2- 172.21.15.39:0/3146581273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e581576e0 0x7f6e58157af0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 --2- 172.21.15.39:0/3146581273 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 0x7f6e5806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- 172.21.15.39:0/3146581273 >> 172.21.15.39:0/3146581273 conn(0x7f6e580c32a0 msgr2=0x7f6e580c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- 172.21.15.39:0/3146581273 shutdown_connections 2024-09-13T17:05:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- 172.21.15.39:0/3146581273 wait complete. 2024-09-13T17:05:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 Processor -- start 2024-09-13T17:05:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- start start 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 0x7f6e581531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e580c1810 0x7f6e580705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e581576e0 0x7f6e58070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581ef950 con 0x7f6e580c1810 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580722f0 con 0x7f6e581576e0 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58156c70 con 0x7f6e5806b080 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e580c1810 0x7f6e580705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e55d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e580c1810 0x7f6e580705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55836/0 (socket says 172.21.15.39:55836) 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e55d9b640 1 -- 172.21.15.39:0/614962580 learned_addr learned my addr 172.21.15.39:0/614962580 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5659c640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 0x7f6e581531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e56d9d640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e581576e0 0x7f6e58070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e55d9b640 1 -- 172.21.15.39:0/614962580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 msgr2=0x7f6e581531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e55d9b640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 0x7f6e581531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e55d9b640 1 -- 172.21.15.39:0/614962580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e581576e0 msgr2=0x7f6e58070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:39.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e55d9b640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e581576e0 0x7f6e58070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:39.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e55d9b640 1 -- 172.21.15.39:0/614962580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50009b30 con 0x7f6e580c1810 2024-09-13T17:05:39.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e55d9b640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e580c1810 0x7f6e580705d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c450 tx=0x7f6e4c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:39.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e46ffd640 1 -- 172.21.15.39:0/614962580 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0052d0 con 0x7f6e580c1810 2024-09-13T17:05:39.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e46ffd640 1 -- 172.21.15.39:0/614962580 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c005450 con 0x7f6e580c1810 2024-09-13T17:05:39.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e58155dc0 con 0x7f6e580c1810 2024-09-13T17:05:39.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.092+0000 7f6e46ffd640 1 -- 172.21.15.39:0/614962580 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c012870 con 0x7f6e580c1810 2024-09-13T17:05:39.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.096+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e580719c0 con 0x7f6e580c1810 2024-09-13T17:05:39.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.096+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e580619c0 con 0x7f6e580c1810 2024-09-13T17:05:39.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.096+0000 7f6e46ffd640 1 -- 172.21.15.39:0/614962580 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e4c0129d0 con 0x7f6e580c1810 2024-09-13T17:05:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.096+0000 7f6e46ffd640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e38066690 0x7f6e38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.096+0000 7f6e46ffd640 1 -- 172.21.15.39:0/614962580 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f6e4c0846b0 con 0x7f6e580c1810 2024-09-13T17:05:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.096+0000 7f6e5659c640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e38066690 0x7f6e38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.096+0000 7f6e5659c640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e38066690 0x7f6e38068b50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007a60 tx=0x7f6e48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.100+0000 7f6e46ffd640 1 -- 172.21.15.39:0/614962580 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6e4c051000 con 0x7f6e580c1810 2024-09-13T17:05:39.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.328+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e58067130 con 0x7f6e580c1810 2024-09-13T17:05:39.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.328+0000 7f6e46ffd640 1 -- 172.21.15.39:0/614962580 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f6e4c054620 con 0x7f6e580c1810 2024-09-13T17:05:39.328 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:39.329 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:39.329 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e38066690 msgr2=0x7f6e38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e38066690 0x7f6e38068b50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007a60 tx=0x7f6e48008040 comp rx=0 tx=0).stop 2024-09-13T17:05:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e580c1810 msgr2=0x7f6e580705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e580c1810 0x7f6e580705d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c450 tx=0x7f6e4c00c910 comp rx=0 tx=0).stop 2024-09-13T17:05:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 shutdown_connections 2024-09-13T17:05:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6e38066690 0x7f6e38068b50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e581576e0 0x7f6e58070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e580c1810 0x7f6e580705d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 --2- 172.21.15.39:0/614962580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6e5806b080 0x7f6e581531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 >> 172.21.15.39:0/614962580 conn(0x7f6e580c32a0 msgr2=0x7f6e580c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 shutdown_connections 2024-09-13T17:05:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:39.332+0000 7f6e5c805640 1 -- 172.21.15.39:0/614962580 wait complete. 2024-09-13T17:05:39.349 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:05:41.350 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:41.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d6f9a640 1 Processor -- start 2024-09-13T17:05:41.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d6f9a640 1 -- start start 2024-09-13T17:05:41.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d6f9a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f51d00c0be0 0x7f51d015f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:41.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d6f9a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d0157330 con 0x7f51d00c6290 2024-09-13T17:05:41.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d6f9a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d00c4740 con 0x7f51d00c2690 2024-09-13T17:05:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d6f9a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d00c4a20 con 0x7f51d00c0be0 2024-09-13T17:05:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51cffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f51d00c6290 0x7f51d01577c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36558/0 (socket says 172.21.15.39:36558) 2024-09-13T17:05:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51cffff640 1 -- 172.21.15.39:0/2922241992 learned_addr learned my addr 172.21.15.39:0/2922241992 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d5537640 1 --2- 172.21.15.39:0/2922241992 >> v2:172.21.15.39:3300/0 conn(0x7f51d00c0be0 0x7f51d015f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d5537640 1 -- 172.21.15.39:0/2922241992 >> v1:172.21.15.84:6789/0 conn(0x7f51d00c2690 legacy=0x7f51d006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d5537640 1 -- 172.21.15.39:0/2922241992 >> v1:172.21.15.39:6789/0 conn(0x7f51d00c6290 legacy=0x7f51d01577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.492+0000 7f51d5537640 1 -- 172.21.15.39:0/2922241992 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0154880 con 0x7f51d00c0be0 2024-09-13T17:05:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51d5537640 1 --2- 172.21.15.39:0/2922241992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c0be0 0x7f51d015f640 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009d90 tx=0x7f51c801d2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afeb4dda42de124d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:41.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51cf7fe640 1 -- 172.21.15.39:0/2922241992 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80a5950 con 0x7f51d00c0be0 2024-09-13T17:05:41.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51cf7fe640 1 -- 172.21.15.39:0/2922241992 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c80a5ab0 con 0x7f51d00c0be0 2024-09-13T17:05:41.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51cf7fe640 1 -- 172.21.15.39:0/2922241992 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80a5e40 con 0x7f51d00c0be0 2024-09-13T17:05:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51d6f9a640 1 -- 172.21.15.39:0/2922241992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c0be0 msgr2=0x7f51d015f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51d6f9a640 1 --2- 172.21.15.39:0/2922241992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c0be0 0x7f51d015f640 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009d90 tx=0x7f51c801d2d0 comp rx=0 tx=0).stop 2024-09-13T17:05:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51d6f9a640 1 -- 172.21.15.39:0/2922241992 shutdown_connections 2024-09-13T17:05:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51d6f9a640 1 --2- 172.21.15.39:0/2922241992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c0be0 0x7f51d015f640 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51d6f9a640 1 -- 172.21.15.39:0/2922241992 >> 172.21.15.39:0/2922241992 conn(0x7f51d00c32a0 msgr2=0x7f51d00c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51d6f9a640 1 -- 172.21.15.39:0/2922241992 shutdown_connections 2024-09-13T17:05:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.496+0000 7f51d6f9a640 1 -- 172.21.15.39:0/2922241992 wait complete. 2024-09-13T17:05:41.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 Processor -- start 2024-09-13T17:05:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 -- start start 2024-09-13T17:05:41.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d00c0be0 0x7f51d006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:41.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c2690 0x7f51d01f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:41.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f51d00c6290 0x7f51d01489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:41.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d00a9a70 con 0x7f51d00c2690 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d00a94f0 con 0x7f51d00c0be0 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01f09a0 con 0x7f51d00c6290 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c2690 0x7f51d01f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c2690 0x7f51d01f0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40310/0 (socket says 172.21.15.39:40310) 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cffff640 1 -- 172.21.15.39:0/855350573 learned_addr learned my addr 172.21.15.39:0/855350573 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d5537640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f51d00c6290 0x7f51d01489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d4d36640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d00c0be0 0x7f51d006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cffff640 1 -- 172.21.15.39:0/855350573 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f51d00c6290 msgr2=0x7f51d01489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cffff640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f51d00c6290 0x7f51d01489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cffff640 1 -- 172.21.15.39:0/855350573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d00c0be0 msgr2=0x7f51d006dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cffff640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d00c0be0 0x7f51d006dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cffff640 1 -- 172.21.15.39:0/855350573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8009a70 con 0x7f51d00c2690 2024-09-13T17:05:41.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cffff640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c2690 0x7f51d01f0e30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00c530 tx=0x7f51bc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:41.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cd7fa640 1 -- 172.21.15.39:0/855350573 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc005cf0 con 0x7f51d00c2690 2024-09-13T17:05:41.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cd7fa640 1 -- 172.21.15.39:0/855350573 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc005e50 con 0x7f51d00c2690 2024-09-13T17:05:41.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d0152d70 con 0x7f51d00c2690 2024-09-13T17:05:41.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51cd7fa640 1 -- 172.21.15.39:0/855350573 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc007270 con 0x7f51d00c2690 2024-09-13T17:05:41.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.500+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d00ac320 con 0x7f51d00c2690 2024-09-13T17:05:41.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.504+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d00a6330 con 0x7f51d00c2690 2024-09-13T17:05:41.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.504+0000 7f51cd7fa640 1 -- 172.21.15.39:0/855350573 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f51bc0073d0 con 0x7f51d00c2690 2024-09-13T17:05:41.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.504+0000 7f51cd7fa640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51b4066690 0x7f51b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:41.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.504+0000 7f51cd7fa640 1 -- 172.21.15.39:0/855350573 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f51bc0840e0 con 0x7f51d00c2690 2024-09-13T17:05:41.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.504+0000 7f51d4d36640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51b4066690 0x7f51b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:41.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.504+0000 7f51d4d36640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51b4066690 0x7f51b4068b50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f51c4004e30 tx=0x7f51c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:41.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.508+0000 7f51cd7fa640 1 -- 172.21.15.39:0/855350573 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f51bc050a30 con 0x7f51d00c2690 2024-09-13T17:05:41.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.760+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51d01f0790 con 0x7f51d00c2690 2024-09-13T17:05:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51cd7fa640 1 -- 172.21.15.39:0/855350573 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f51bc017020 con 0x7f51d00c2690 2024-09-13T17:05:41.762 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:41.762 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:41.763 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51b4066690 msgr2=0x7f51b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51b4066690 0x7f51b4068b50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f51c4004e30 tx=0x7f51c4009040 comp rx=0 tx=0).stop 2024-09-13T17:05:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c2690 msgr2=0x7f51d01f0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c2690 0x7f51d01f0e30 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00c530 tx=0x7f51bc00c9f0 comp rx=0 tx=0).stop 2024-09-13T17:05:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 shutdown_connections 2024-09-13T17:05:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f51b4066690 0x7f51b4068b50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f51d00c6290 0x7f51d01489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d00c2690 0x7f51d01f0e30 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 --2- 172.21.15.39:0/855350573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d00c0be0 0x7f51d006dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 >> 172.21.15.39:0/855350573 conn(0x7f51d00c32a0 msgr2=0x7f51d00a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 shutdown_connections 2024-09-13T17:05:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:41.764+0000 7f51d6f9a640 1 -- 172.21.15.39:0/855350573 wait complete. 2024-09-13T17:05:41.782 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:05:43.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:43.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.936+0000 7fafde050640 1 Processor -- start 2024-09-13T17:05:43.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.936+0000 7fafde050640 1 -- start start 2024-09-13T17:05:43.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafde050640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd8151860 0x7fafd8157d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:43.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafde050640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fafd80bf4f0 0x7fafd8153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:43.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafde050640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd80c1340 con 0x7fafd80611c0 2024-09-13T17:05:43.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafde050640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd81578f0 con 0x7fafd80bf4f0 2024-09-13T17:05:43.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafde050640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd81582c0 con 0x7fafd8151860 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd77fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd8151860 0x7fafd8157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fafd80bf4f0 0x7fafd8153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fafd80bf4f0 0x7fafd8153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:43490/0 (socket says 172.21.15.39:43490) 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd6ffd640 1 -- 172.21.15.39:0/989131526 learned_addr learned my addr 172.21.15.39:0/989131526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd7fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fafd80611c0 0x7fafd80bfad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:54418/0 (socket says 172.21.15.39:54418) 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd6ffd640 1 -- 172.21.15.39:0/989131526 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd8151860 msgr2=0x7fafd8157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd6ffd640 1 --2- 172.21.15.39:0/989131526 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd8151860 0x7fafd8157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd6ffd640 1 -- 172.21.15.39:0/989131526 >> v1:172.21.15.84:6789/0 conn(0x7fafd80611c0 legacy=0x7fafd80bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd6ffd640 1 -- 172.21.15.39:0/989131526 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd80c1590 con 0x7fafd80bf4f0 2024-09-13T17:05:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.940+0000 7fafd77fe640 1 --2- 172.21.15.39:0/989131526 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd8151860 0x7fafd8157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:05:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafd6ffd640 1 --2- 172.21.15.39:0/989131526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd80bf4f0 0x7fafd8153df0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009870 tx=0x7fafcc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3cb8db0c25b9fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafd67fc640 1 -- 172.21.15.39:0/989131526 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0a6040 con 0x7fafd80bf4f0 2024-09-13T17:05:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafd67fc640 1 -- 172.21.15.39:0/989131526 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc0aa600 con 0x7fafd80bf4f0 2024-09-13T17:05:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafd67fc640 1 -- 172.21.15.39:0/989131526 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0aa990 con 0x7fafd80bf4f0 2024-09-13T17:05:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 -- 172.21.15.39:0/989131526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd80bf4f0 msgr2=0x7fafd8153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 --2- 172.21.15.39:0/989131526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd80bf4f0 0x7fafd8153df0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009870 tx=0x7fafcc01c4a0 comp rx=0 tx=0).stop 2024-09-13T17:05:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 -- 172.21.15.39:0/989131526 shutdown_connections 2024-09-13T17:05:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 --2- 172.21.15.39:0/989131526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd80bf4f0 0x7fafd8153df0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 --2- 172.21.15.39:0/989131526 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd8151860 0x7fafd8157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 -- 172.21.15.39:0/989131526 >> 172.21.15.39:0/989131526 conn(0x7fafd8069e10 msgr2=0x7fafd806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 -- 172.21.15.39:0/989131526 shutdown_connections 2024-09-13T17:05:43.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 -- 172.21.15.39:0/989131526 wait complete. 2024-09-13T17:05:43.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 Processor -- start 2024-09-13T17:05:43.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 -- start start 2024-09-13T17:05:43.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd80611c0 0x7fafd8141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:43.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd80bf4f0 0x7fafd8140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:43.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd8151860 0x7fafd80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:43.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8146310 con 0x7fafd8151860 2024-09-13T17:05:43.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.944+0000 7fafde050640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8145fe0 con 0x7fafd80bf4f0 2024-09-13T17:05:43.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafde050640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd8147660 con 0x7fafd80611c0 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd8151860 0x7fafd80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd8151860 0x7fafd80c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40318/0 (socket says 172.21.15.39:40318) 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd7fff640 1 -- 172.21.15.39:0/1756685388 learned_addr learned my addr 172.21.15.39:0/1756685388 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd77fe640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd80611c0 0x7fafd8141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd6ffd640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd80bf4f0 0x7fafd8140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd7fff640 1 -- 172.21.15.39:0/1756685388 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd80611c0 msgr2=0x7fafd8141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd7fff640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd80611c0 0x7fafd8141930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd7fff640 1 -- 172.21.15.39:0/1756685388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd80bf4f0 msgr2=0x7fafd8140450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd7fff640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd80bf4f0 0x7fafd8140450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd7fff640 1 -- 172.21.15.39:0/1756685388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc8009420 con 0x7fafd8151860 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd7fff640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd8151860 0x7fafd80c5710 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fafd000c700 tx=0x7fafd000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafbffff640 1 -- 172.21.15.39:0/1756685388 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd000ce10 con 0x7fafd8151860 2024-09-13T17:05:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafbffff640 1 -- 172.21.15.39:0/1756685388 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd000a640 con 0x7fafd8151860 2024-09-13T17:05:43.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc0094d0 con 0x7fafd8151860 2024-09-13T17:05:43.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafbffff640 1 -- 172.21.15.39:0/1756685388 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd000a9b0 con 0x7fafd8151860 2024-09-13T17:05:43.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd8147c20 con 0x7fafd8151860 2024-09-13T17:05:43.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd8153df0 con 0x7fafd8151860 2024-09-13T17:05:43.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafbffff640 1 -- 172.21.15.39:0/1756685388 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fafd000ab10 con 0x7fafd8151860 2024-09-13T17:05:43.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafbffff640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fafc40666c0 0x7fafc4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:43.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafbffff640 1 -- 172.21.15.39:0/1756685388 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7fafd0084400 con 0x7fafd8151860 2024-09-13T17:05:43.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.948+0000 7fafd77fe640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fafc40666c0 0x7fafc4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:43.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.952+0000 7fafd77fe640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fafc40666c0 0x7fafc4068b80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009cf0 tx=0x7fafc8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:43.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:43.956+0000 7fafbffff640 1 -- 172.21.15.39:0/1756685388 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fafd0050cc0 con 0x7fafd8151860 2024-09-13T17:05:44.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.220+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafd80c0f10 con 0x7fafd8151860 2024-09-13T17:05:44.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.220+0000 7fafbffff640 1 -- 172.21.15.39:0/1756685388 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7fafd0017090 con 0x7fafd8151860 2024-09-13T17:05:44.222 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:44.222 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:44.222 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:44.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fafc40666c0 msgr2=0x7fafc4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fafc40666c0 0x7fafc4068b80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009cf0 tx=0x7fafc8009190 comp rx=0 tx=0).stop 2024-09-13T17:05:44.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd8151860 msgr2=0x7fafd80c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd8151860 0x7fafd80c5710 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fafd000c700 tx=0x7fafd000cbc0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 shutdown_connections 2024-09-13T17:05:44.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fafc40666c0 0x7fafc4068b80 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009cf0 tx=0x7fafc8009190 comp rx=0 tx=0).stop 2024-09-13T17:05:44.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd80bf4f0 0x7fafd8140450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd8151860 0x7fafd80c5710 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 --2- 172.21.15.39:0/1756685388 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fafd80611c0 0x7fafd8141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 >> 172.21.15.39:0/1756685388 conn(0x7fafd8069e10 msgr2=0x7fafd8154570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:44.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 shutdown_connections 2024-09-13T17:05:44.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.224+0000 7fafde050640 1 -- 172.21.15.39:0/1756685388 wait complete. 2024-09-13T17:05:44.245 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:05:44.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.544+0000 7f0139a7a640 1 Processor -- start 2024-09-13T17:05:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.544+0000 7f0139a7a640 1 -- start start 2024-09-13T17:05:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0139a7a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 0x7f0134157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0139a7a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f013406f120 0x7f0134153e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0139a7a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0134157940 con 0x7f01340c5720 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0139a7a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01340c7690 con 0x7f013406f120 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0139a7a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0134158310 con 0x7f01340c3a40 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0132ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f013406f120 0x7f0134153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0132ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f013406f120 0x7f0134153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40328/0 (socket says 172.21.15.39:40328) 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0132ffd640 1 -- 172.21.15.39:0/760525415 learned_addr learned my addr 172.21.15.39:0/760525415 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f01337fe640 1 --2- 172.21.15.39:0/760525415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 0x7f0134157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0132ffd640 1 -- 172.21.15.39:0/760525415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 msgr2=0x7f0134157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0132ffd640 1 --2- 172.21.15.39:0/760525415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 0x7f0134157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0132ffd640 1 -- 172.21.15.39:0/760525415 >> v1:172.21.15.84:6789/0 conn(0x7f01340c5720 legacy=0x7f01340c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0132ffd640 1 -- 172.21.15.39:0/760525415 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01340c73b0 con 0x7f013406f120 2024-09-13T17:05:44.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f0132ffd640 1 --2- 172.21.15.39:0/760525415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f013406f120 0x7f0134153e40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0128009870 tx=0x7f012801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bd419561453023 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:44.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f01327fc640 1 -- 172.21.15.39:0/760525415 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01280ac640 con 0x7f013406f120 2024-09-13T17:05:44.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f01327fc640 1 -- 172.21.15.39:0/760525415 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01280ac7a0 con 0x7f013406f120 2024-09-13T17:05:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.548+0000 7f01327fc640 1 -- 172.21.15.39:0/760525415 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01280acb30 con 0x7f013406f120 2024-09-13T17:05:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- 172.21.15.39:0/760525415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f013406f120 msgr2=0x7f0134153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 --2- 172.21.15.39:0/760525415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f013406f120 0x7f0134153e40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0128009870 tx=0x7f012801c4d0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- 172.21.15.39:0/760525415 shutdown_connections 2024-09-13T17:05:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 --2- 172.21.15.39:0/760525415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f013406f120 0x7f0134153e40 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 --2- 172.21.15.39:0/760525415 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 0x7f0134157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- 172.21.15.39:0/760525415 >> 172.21.15.39:0/760525415 conn(0x7f01340c7820 msgr2=0x7f01340c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- 172.21.15.39:0/760525415 shutdown_connections 2024-09-13T17:05:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- 172.21.15.39:0/760525415 wait complete. 2024-09-13T17:05:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 Processor -- start 2024-09-13T17:05:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- start start 2024-09-13T17:05:44.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013406f120 0x7f013414bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 0x7f013414f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c5720 0x7f01341efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01341ec610 con 0x7f01340c5720 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013414c450 con 0x7f013406f120 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013415aa80 con 0x7f01340c3a40 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0133fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c5720 0x7f01341efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0133fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c5720 0x7f01341efb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40340/0 (socket says 172.21.15.39:40340) 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0133fff640 1 -- 172.21.15.39:0/3136528183 learned_addr learned my addr 172.21.15.39:0/3136528183 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f01337fe640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013406f120 0x7f013414bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0132ffd640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 0x7f013414f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0133fff640 1 -- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 msgr2=0x7f013414f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0133fff640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 0x7f013414f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0133fff640 1 -- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013406f120 msgr2=0x7f013414bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0133fff640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013406f120 0x7f013414bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0133fff640 1 -- 172.21.15.39:0/3136528183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01280094d0 con 0x7f01340c5720 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f01337fe640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013406f120 0x7f013414bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0132ffd640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 0x7f013414f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0133fff640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c5720 0x7f01341efb80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c730 tx=0x7f012c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f011bfff640 1 -- 172.21.15.39:0/3136528183 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c003b80 con 0x7f01340c5720 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f011bfff640 1 -- 172.21.15.39:0/3136528183 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c003ce0 con 0x7f01340c5720 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0134158b70 con 0x7f01340c5720 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f011bfff640 1 -- 172.21.15.39:0/3136528183 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c00aa30 con 0x7f01340c5720 2024-09-13T17:05:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.552+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01341ecb00 con 0x7f01340c5720 2024-09-13T17:05:44.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.556+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01340c3e20 con 0x7f01340c5720 2024-09-13T17:05:44.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.556+0000 7f011bfff640 1 -- 172.21.15.39:0/3136528183 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f012c00ab90 con 0x7f01340c5720 2024-09-13T17:05:44.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.556+0000 7f011bfff640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f01200666c0 0x7f0120068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.556+0000 7f011bfff640 1 -- 172.21.15.39:0/3136528183 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f012c0849e0 con 0x7f01340c5720 2024-09-13T17:05:44.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.556+0000 7f01337fe640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f01200666c0 0x7f0120068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.556+0000 7f01337fe640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f01200666c0 0x7f0120068b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0124007a70 tx=0x7f0124008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:44.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.564+0000 7f011bfff640 1 -- 172.21.15.39:0/3136528183 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f012c051280 con 0x7f01340c5720 2024-09-13T17:05:44.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.764+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01340c5d70 con 0x7f01340c5720 2024-09-13T17:05:44.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f011bfff640 1 -- 172.21.15.39:0/3136528183 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f012c0548a0 con 0x7f01340c5720 2024-09-13T17:05:44.767 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:44.767 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:44.768 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f01200666c0 msgr2=0x7f0120068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f01200666c0 0x7f0120068b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0124007a70 tx=0x7f0124008040 comp rx=0 tx=0).stop 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c5720 msgr2=0x7f01341efb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c5720 0x7f01341efb80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c730 tx=0x7f012c00cbf0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 shutdown_connections 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f01200666c0 0x7f0120068b80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c5720 0x7f01341efb80 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f01340c3a40 0x7f013414f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 --2- 172.21.15.39:0/3136528183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f013406f120 0x7f013414bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 >> 172.21.15.39:0/3136528183 conn(0x7f01340c7820 msgr2=0x7f01340c7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:44.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 shutdown_connections 2024-09-13T17:05:44.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.768+0000 7f0139a7a640 1 -- 172.21.15.39:0/3136528183 wait complete. 2024-09-13T17:05:44.784 INFO:tasks.mds_thrash.fs.[a]:kill mds.a (rank=0) 2024-09-13T17:05:44.785 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:05:44.785 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:05:44.794 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:05:44.795 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:44.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c3874640 1 Processor -- start 2024-09-13T17:05:44.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c3874640 1 -- start start 2024-09-13T17:05:44.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c3874640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64bc0c6000 0x7f64bc156130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c3874640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64bc0c5e50 con 0x7f64bc06fa50 2024-09-13T17:05:44.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c3874640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64bc06ff50 con 0x7f64bc0c7960 2024-09-13T17:05:44.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c3874640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc072000 con 0x7f64bc0c6000 2024-09-13T17:05:44.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c0e0f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f64bc06fa50 0x7f64bc0c2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36562/0 (socket says 172.21.15.39:36562) 2024-09-13T17:05:44.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c0e0f640 1 -- 172.21.15.39:0/1533234820 learned_addr learned my addr 172.21.15.39:0/1533234820 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:44.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c1e11640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64bc0c6000 0x7f64bc156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763450019 0 0) 0x7f64bc0c5e50 con 0x7f64bc06fa50 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a8003530 con 0x7f64bc06fa50 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 835154541 0 0) 0x7f64a8003530 con 0x7f64bc06fa50 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64bc0c6000 msgr2=0x7f64bc156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64bc0c6000 0x7f64bc156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 >> v1:172.21.15.84:6789/0 conn(0x7f64bc0c7960 legacy=0x7f64bc155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc0722e0 con 0x7f64bc06fa50 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f64ac001680 con 0x7f64bc06fa50 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 >> v1:172.21.15.39:6789/0 conn(0x7f64bc06fa50 legacy=0x7f64bc0c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a80039b0 0x7f64a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64a80047d0 0x7f64a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64bc0c6000 0x7f64a8005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc072000 con 0x7f64a80039b0 2024-09-13T17:05:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a800bb20 con 0x7f64a80047d0 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a800bce0 con 0x7f64bc0c6000 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c1610640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a80039b0 0x7f64a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c0e0f640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64a80047d0 0x7f64a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c1e11640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64bc0c6000 0x7f64a8005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c1610640 1 -- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64bc0c6000 msgr2=0x7f64a8005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c1610640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64bc0c6000 0x7f64a8005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c1610640 1 -- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64a80047d0 msgr2=0x7f64a8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c1610640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64a80047d0 0x7f64a8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c1610640 1 -- 172.21.15.39:0/1533234820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8003530 con 0x7f64a80039b0 2024-09-13T17:05:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.956+0000 7f64c0e0f640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64a80047d0 0x7f64a8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64c1610640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a80039b0 0x7f64a8003dd0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f64b8001780 tx=0x7f64b801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a91c7f7a77de82 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b80052d0 con 0x7f64a80039b0 2024-09-13T17:05:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b80a7050 con 0x7f64a80039b0 2024-09-13T17:05:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64b3fff640 1 -- 172.21.15.39:0/1533234820 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8003600 con 0x7f64a80039b0 2024-09-13T17:05:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64c3874640 1 -- 172.21.15.39:0/1533234820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a80039b0 msgr2=0x7f64a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64c3874640 1 --2- 172.21.15.39:0/1533234820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64a80039b0 0x7f64a8003dd0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f64b8001780 tx=0x7f64b801c3f0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64c1e11640 1 -- 172.21.15.39:0/1533234820 reap_dead start 2024-09-13T17:05:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64c3874640 1 -- 172.21.15.39:0/1533234820 shutdown_connections 2024-09-13T17:05:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64c3874640 1 -- 172.21.15.39:0/1533234820 >> 172.21.15.39:0/1533234820 conn(0x7f64bc06c020 msgr2=0x7f64bc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64c3874640 1 -- 172.21.15.39:0/1533234820 shutdown_connections 2024-09-13T17:05:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.960+0000 7f64c3874640 1 -- 172.21.15.39:0/1533234820 wait complete. 2024-09-13T17:05:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 Processor -- start 2024-09-13T17:05:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 -- start start 2024-09-13T17:05:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64a80039b0 0x7f64bc0c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64bc0a6590 0x7f64bc06b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64bc06b670 0x7f64bc1ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0bf1d0 con 0x7f64bc0a6590 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0c1090 con 0x7f64bc06b670 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc06bb00 con 0x7f64a80039b0 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c0e0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64bc0a6590 0x7f64bc06b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c0e0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64bc0a6590 0x7f64bc06b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40362/0 (socket says 172.21.15.39:40362) 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c0e0f640 1 -- 172.21.15.39:0/3507080985 learned_addr learned my addr 172.21.15.39:0/3507080985 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c1610640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64a80039b0 0x7f64bc0c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c1e11640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64bc06b670 0x7f64bc1ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c0e0f640 1 -- 172.21.15.39:0/3507080985 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64a80039b0 msgr2=0x7f64bc0c16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c0e0f640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64a80039b0 0x7f64bc0c16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c0e0f640 1 -- 172.21.15.39:0/3507080985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64bc06b670 msgr2=0x7f64bc1ecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c0e0f640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64bc06b670 0x7f64bc1ecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c0e0f640 1 -- 172.21.15.39:0/3507080985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc0a8060 con 0x7f64bc0a6590 2024-09-13T17:05:44.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c1610640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64a80039b0 0x7f64bc0c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:44.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c0e0f640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64bc0a6590 0x7f64bc06b130 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00e780 tx=0x7f64ac00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:44.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64b1ffb640 1 -- 172.21.15.39:0/3507080985 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac00c6c0 con 0x7f64bc0a6590 2024-09-13T17:05:44.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64b1ffb640 1 -- 172.21.15.39:0/3507080985 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ac00c820 con 0x7f64bc0a6590 2024-09-13T17:05:44.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64bc0c0db0 con 0x7f64bc0a6590 2024-09-13T17:05:44.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64b1ffb640 1 -- 172.21.15.39:0/3507080985 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac00cb90 con 0x7f64bc0a6590 2024-09-13T17:05:44.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64bc0a7aa0 con 0x7f64bc0a6590 2024-09-13T17:05:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64bc1ec8d0 con 0x7f64bc0a6590 2024-09-13T17:05:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.964+0000 7f64b1ffb640 1 -- 172.21.15.39:0/3507080985 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64ac00ccf0 con 0x7f64bc0a6590 2024-09-13T17:05:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.968+0000 7f64b1ffb640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64a0066690 0x7f64a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.968+0000 7f64b1ffb640 1 -- 172.21.15.39:0/3507080985 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f64ac084b90 con 0x7f64bc0a6590 2024-09-13T17:05:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.968+0000 7f64c1610640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64a0066690 0x7f64a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:44.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.968+0000 7f64c1610640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64a0066690 0x7f64a0068b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f64b80ac000 tx=0x7f64b80011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:44.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:44.972+0000 7f64b1ffb640 1 -- 172.21.15.39:0/3507080985 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f64ac051430 con 0x7f64bc0a6590 2024-09-13T17:05:45.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.180+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64bc0c2570 con 0x7f64bc0a6590 2024-09-13T17:05:45.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.180+0000 7f64b1ffb640 1 -- 172.21.15.39:0/3507080985 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f64ac054a50 con 0x7f64bc0a6590 2024-09-13T17:05:45.180 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:45.180 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:45.180 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:45.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64a0066690 msgr2=0x7f64a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64a0066690 0x7f64a0068b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f64b80ac000 tx=0x7f64b80011a0 comp rx=0 tx=0).stop 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64bc0a6590 msgr2=0x7f64bc06b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64bc0a6590 0x7f64bc06b130 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00e780 tx=0x7f64ac00ec40 comp rx=0 tx=0).stop 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 shutdown_connections 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f64a0066690 0x7f64a0068b50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f64bc06b670 0x7f64bc1ecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64bc0a6590 0x7f64bc06b130 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 --2- 172.21.15.39:0/3507080985 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f64a80039b0 0x7f64bc0c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 >> 172.21.15.39:0/3507080985 conn(0x7f64bc06c020 msgr2=0x7f64bc155040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 shutdown_connections 2024-09-13T17:05:45.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:45.184+0000 7f64c3874640 1 -- 172.21.15.39:0/3507080985 wait complete. 2024-09-13T17:05:45.198 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:05:46.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:46.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f738b24a640 1 Processor -- start 2024-09-13T17:05:46.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f738b24a640 1 -- start start 2024-09-13T17:05:46.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f738b24a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73841515c0 0x7f7384154a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f738b24a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7384157f80 0x7f73840c2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f738b24a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7384154590 con 0x7f73840bf580 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f738b24a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738406d7a0 con 0x7f73841515c0 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f738b24a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73841583b0 con 0x7f7384157f80 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f7388fe6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73841515c0 0x7f7384154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f7388fe6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73841515c0 0x7f7384154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40378/0 (socket says 172.21.15.39:40378) 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f7383fff640 1 --1- 172.21.15.39:0/932728406 >> v1:172.21.15.39:6789/0 conn(0x7f73840bf580 0x7f738406baa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36572/0 (socket says 172.21.15.39:36572) 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f7388fe6640 1 -- 172.21.15.39:0/932728406 learned_addr learned my addr 172.21.15.39:0/932728406 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f73897e7640 1 --2- 172.21.15.39:0/932728406 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7384157f80 0x7f73840c2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f7388fe6640 1 -- 172.21.15.39:0/932728406 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7384157f80 msgr2=0x7f73840c2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f7388fe6640 1 --2- 172.21.15.39:0/932728406 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7384157f80 0x7f73840c2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f7388fe6640 1 -- 172.21.15.39:0/932728406 >> v1:172.21.15.39:6789/0 conn(0x7f73840bf580 legacy=0x7f738406baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.380+0000 7f7388fe6640 1 -- 172.21.15.39:0/932728406 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738406c1b0 con 0x7f73841515c0 2024-09-13T17:05:46.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f7388fe6640 1 --2- 172.21.15.39:0/932728406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73841515c0 0x7f7384154a20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7370009870 tx=0x7f737001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80afbc5b203152d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:46.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f73837fe640 1 -- 172.21.15.39:0/932728406 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73700ab620 con 0x7f73841515c0 2024-09-13T17:05:46.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f73837fe640 1 -- 172.21.15.39:0/932728406 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73700ab780 con 0x7f73841515c0 2024-09-13T17:05:46.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f73837fe640 1 -- 172.21.15.39:0/932728406 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73700abb10 con 0x7f73841515c0 2024-09-13T17:05:46.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 -- 172.21.15.39:0/932728406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73841515c0 msgr2=0x7f7384154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:46.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 --2- 172.21.15.39:0/932728406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73841515c0 0x7f7384154a20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7370009870 tx=0x7f737001c420 comp rx=0 tx=0).stop 2024-09-13T17:05:46.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 -- 172.21.15.39:0/932728406 shutdown_connections 2024-09-13T17:05:46.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 --2- 172.21.15.39:0/932728406 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7384157f80 0x7f73840c2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 --2- 172.21.15.39:0/932728406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73841515c0 0x7f7384154a20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 -- 172.21.15.39:0/932728406 >> 172.21.15.39:0/932728406 conn(0x7f73840c6050 msgr2=0x7f73840c6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:46.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 -- 172.21.15.39:0/932728406 shutdown_connections 2024-09-13T17:05:46.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 -- 172.21.15.39:0/932728406 wait complete. 2024-09-13T17:05:46.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 Processor -- start 2024-09-13T17:05:46.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.384+0000 7f738b24a640 1 -- start start 2024-09-13T17:05:46.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f738b24a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73840bf580 0x7f738414cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:46.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f738b24a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73841515c0 0x7f73840ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f738b24a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7384157f80 0x7f738414ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f738b24a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840abf90 con 0x7f7384157f80 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f738b24a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7384070be0 con 0x7f73841515c0 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f738b24a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73841ed110 con 0x7f73840bf580 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73897e7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7384157f80 0x7f738414ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73897e7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7384157f80 0x7f738414ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40394/0 (socket says 172.21.15.39:40394) 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73897e7640 1 -- 172.21.15.39:0/2485959827 learned_addr learned my addr 172.21.15.39:0/2485959827 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f7388fe6640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73840bf580 0x7f738414cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f7383fff640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73841515c0 0x7f73840ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73897e7640 1 -- 172.21.15.39:0/2485959827 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73840bf580 msgr2=0x7f738414cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:46.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73897e7640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73840bf580 0x7f738414cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73897e7640 1 -- 172.21.15.39:0/2485959827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73841515c0 msgr2=0x7f73840ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73897e7640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73841515c0 0x7f73840ab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73897e7640 1 -- 172.21.15.39:0/2485959827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700094d0 con 0x7f7384157f80 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f7388fe6640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73840bf580 0x7f738414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73897e7640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7384157f80 0x7f738414ac00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f737c007e60 tx=0x7f737c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73817fa640 1 -- 172.21.15.39:0/2485959827 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c014420 con 0x7f7384157f80 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73817fa640 1 -- 172.21.15.39:0/2485959827 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c0145a0 con 0x7f7384157f80 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73841ed3f0 con 0x7f7384157f80 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73817fa640 1 -- 172.21.15.39:0/2485959827 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c014950 con 0x7f7384157f80 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7384070890 con 0x7f7384157f80 2024-09-13T17:05:46.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73840c03a0 con 0x7f7384157f80 2024-09-13T17:05:46.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73817fa640 1 -- 172.21.15.39:0/2485959827 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f737c014ab0 con 0x7f7384157f80 2024-09-13T17:05:46.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73817fa640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73680666c0 0x7f7368068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:46.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f73817fa640 1 -- 172.21.15.39:0/2485959827 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f737c085570 con 0x7f7384157f80 2024-09-13T17:05:46.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f7388fe6640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73680666c0 0x7f7368068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:46.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.388+0000 7f7388fe6640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73680666c0 0x7f7368068b80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f73700af000 tx=0x7f73700a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:46.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.392+0000 7f73817fa640 1 -- 172.21.15.39:0/2485959827 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f737c051e10 con 0x7f7384157f80 2024-09-13T17:05:46.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73840ac0d0 con 0x7f7384157f80 2024-09-13T17:05:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f73817fa640 1 -- 172.21.15.39:0/2485959827 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f737c055430 con 0x7f7384157f80 2024-09-13T17:05:46.587 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:46.587 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:46.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73680666c0 msgr2=0x7f7368068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:46.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73680666c0 0x7f7368068b80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f73700af000 tx=0x7f73700a7000 comp rx=0 tx=0).stop 2024-09-13T17:05:46.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7384157f80 msgr2=0x7f738414ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7384157f80 0x7f738414ac00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f737c007e60 tx=0x7f737c00e4b0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 shutdown_connections 2024-09-13T17:05:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73680666c0 0x7f7368068b80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73841515c0 0x7f73840ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7384157f80 0x7f738414ac00 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 --2- 172.21.15.39:0/2485959827 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73840bf580 0x7f738414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 >> 172.21.15.39:0/2485959827 conn(0x7f73840c6050 msgr2=0x7f73840c1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 shutdown_connections 2024-09-13T17:05:46.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:46.588+0000 7f738b24a640 1 -- 172.21.15.39:0/2485959827 wait complete. 2024-09-13T17:05:46.604 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:05:47.199 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc4214640 1 Processor -- start 2024-09-13T17:05:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc4214640 1 -- start start 2024-09-13T17:05:47.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc4214640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fedbc1526b0 0x7fedbc0c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:47.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc4214640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedbc14f790 con 0x7fedbc06c940 2024-09-13T17:05:47.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc4214640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedbc152a90 con 0x7fedbc1555f0 2024-09-13T17:05:47.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc4214640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedbc06ce40 con 0x7fedbc1526b0 2024-09-13T17:05:47.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc17af640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fedbc06c940 0x7fedbc0aabc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36588/0 (socket says 172.21.15.39:36588) 2024-09-13T17:05:47.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc17af640 1 -- 172.21.15.39:0/81433593 learned_addr learned my addr 172.21.15.39:0/81433593 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc1fb0640 1 --2- 172.21.15.39:0/81433593 >> v2:172.21.15.84:3300/0 conn(0x7fedbc1526b0 0x7fedbc0c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062985036 0 0) 0x7fedbc14f790 con 0x7fedbc06c940 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feda8003530 con 0x7fedbc06c940 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1011096037 0 0) 0x7feda8003530 con 0x7fedbc06c940 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 >> v2:172.21.15.84:3300/0 conn(0x7fedbc1526b0 msgr2=0x7fedbc0c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 --2- 172.21.15.39:0/81433593 >> v2:172.21.15.84:3300/0 conn(0x7fedbc1526b0 0x7fedbc0c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 >> v1:172.21.15.84:6789/0 conn(0x7fedbc1555f0 legacy=0x7fedbc155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc06d120 con 0x7fedbc06c940 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fedac0016a0 con 0x7fedbc06c940 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 >> v1:172.21.15.39:6789/0 conn(0x7fedbc06c940 legacy=0x7fedbc0aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:47.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7feda8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feda80047d0 0x7feda8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda8005190 0x7feda800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc06ce40 con 0x7feda80039d0 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda800c090 con 0x7feda8005190 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda800c250 con 0x7feda80047d0 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc1fb0640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7feda8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc17af640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feda80047d0 0x7feda8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc1fb0640 1 -- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feda80047d0 msgr2=0x7feda8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc1fb0640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feda80047d0 0x7feda8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc1fb0640 1 -- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda8005190 msgr2=0x7feda800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:47.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc27b1640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda8005190 0x7feda800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:47.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc1fb0640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda8005190 0x7feda800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc1fb0640 1 -- 172.21.15.39:0/81433593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda8003530 con 0x7feda80039d0 2024-09-13T17:05:47.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc17af640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feda80047d0 0x7feda8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:47.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc27b1640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feda8005190 0x7feda800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:47.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.336+0000 7fedc27b1640 1 -- 172.21.15.39:0/81433593 reap_dead start 2024-09-13T17:05:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc1fb0640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7feda8003df0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009ae0 tx=0x7fedb801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5942650ac9ac88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80a6450 con 0x7feda80039d0 2024-09-13T17:05:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb80a65b0 con 0x7feda80039d0 2024-09-13T17:05:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc0fae640 1 -- 172.21.15.39:0/81433593 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80a6960 con 0x7feda80039d0 2024-09-13T17:05:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 -- 172.21.15.39:0/81433593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 msgr2=0x7feda8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7feda8003df0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009ae0 tx=0x7fedb801ca70 comp rx=0 tx=0).stop 2024-09-13T17:05:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 -- 172.21.15.39:0/81433593 shutdown_connections 2024-09-13T17:05:47.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 --2- 172.21.15.39:0/81433593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7feda8003df0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 -- 172.21.15.39:0/81433593 >> 172.21.15.39:0/81433593 conn(0x7fedbc069e10 msgr2=0x7fedbc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 -- 172.21.15.39:0/81433593 shutdown_connections 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 -- 172.21.15.39:0/81433593 wait complete. 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 Processor -- start 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 -- start start 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.340+0000 7fedc4214640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7fedbc06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc4214640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fedbc1efa90 0x7fedbc150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc4214640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc1e0b00 0x7fedbc14eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc4214640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc1efef0 con 0x7feda80039d0 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc4214640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc070b60 con 0x7fedbc1e0b00 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc4214640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc06f750 con 0x7fedbc1efa90 2024-09-13T17:05:47.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc1fb0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7fedbc06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc1fb0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7fedbc06f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40410/0 (socket says 172.21.15.39:40410) 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc1fb0640 1 -- 172.21.15.39:0/1642940018 learned_addr learned my addr 172.21.15.39:0/1642940018 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc17af640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fedbc1efa90 0x7fedbc150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc27b1640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc1e0b00 0x7fedbc14eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc1fb0640 1 -- 172.21.15.39:0/1642940018 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fedbc1efa90 msgr2=0x7fedbc150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc1fb0640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fedbc1efa90 0x7fedbc150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc1fb0640 1 -- 172.21.15.39:0/1642940018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc1e0b00 msgr2=0x7fedbc14eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc1fb0640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc1e0b00 0x7fedbc14eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc1fb0640 1 -- 172.21.15.39:0/1642940018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc0bf610 con 0x7feda80039d0 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc17af640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fedbc1efa90 0x7fedbc150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:47.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc1fb0640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7fedbc06f9c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009a00 tx=0x7fedb8019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedb27fc640 1 -- 172.21.15.39:0/1642940018 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80a6dc0 con 0x7feda80039d0 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedb27fc640 1 -- 172.21.15.39:0/1642940018 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb80b9440 con 0x7feda80039d0 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedbc06f190 con 0x7feda80039d0 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedb27fc640 1 -- 172.21.15.39:0/1642940018 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80b97f0 con 0x7feda80039d0 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedbc1e05e0 con 0x7feda80039d0 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedbc1efe70 con 0x7feda80039d0 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedb27fc640 1 -- 172.21.15.39:0/1642940018 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fedb80a7430 con 0x7feda80039d0 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedb27fc640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feda00666c0 0x7feda0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedb27fc640 1 -- 172.21.15.39:0/1642940018 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7fedb811e670 con 0x7feda80039d0 2024-09-13T17:05:47.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc17af640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feda00666c0 0x7feda0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:47.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.344+0000 7fedc17af640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feda00666c0 0x7feda0068b80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fedac0017d0 tx=0x7fedac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:47.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.348+0000 7fedb27fc640 1 -- 172.21.15.39:0/1642940018 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fedb80eadf0 con 0x7feda80039d0 2024-09-13T17:05:47.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.540+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedbc0aabc0 con 0x7feda80039d0 2024-09-13T17:05:47.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.540+0000 7fedb27fc640 1 -- 172.21.15.39:0/1642940018 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7fedb80bf030 con 0x7feda80039d0 2024-09-13T17:05:47.540 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:47.541 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:47.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feda00666c0 msgr2=0x7feda0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:47.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feda00666c0 0x7feda0068b80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fedac0017d0 tx=0x7fedac009040 comp rx=0 tx=0).stop 2024-09-13T17:05:47.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 msgr2=0x7fedbc06f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:47.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7fedbc06f9c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009a00 tx=0x7fedb8019810 comp rx=0 tx=0).stop 2024-09-13T17:05:47.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 shutdown_connections 2024-09-13T17:05:47.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feda00666c0 0x7feda0068b80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc1e0b00 0x7fedbc14eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fedbc1efa90 0x7fedbc150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 --2- 172.21.15.39:0/1642940018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feda80039d0 0x7fedbc06f9c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:47.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 >> 172.21.15.39:0/1642940018 conn(0x7fedbc069e10 msgr2=0x7fedbc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:47.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 shutdown_connections 2024-09-13T17:05:47.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:47.544+0000 7fedc4214640 1 -- 172.21.15.39:0/1642940018 wait complete. 2024-09-13T17:05:47.558 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:05:48.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:48.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f8b44640 1 Processor -- start 2024-09-13T17:05:48.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f8b44640 1 -- start start 2024-09-13T17:05:48.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f8b44640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42f4155d00 0x7f42f40c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:48.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f8b44640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f414f790 con 0x7f42f41526b0 2024-09-13T17:05:48.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f8b44640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f4152a90 con 0x7f42f406c940 2024-09-13T17:05:48.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f8b44640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f406ce40 con 0x7f42f4155d00 2024-09-13T17:05:48.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f259c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f42f41526b0 0x7f42f40aabc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36598/0 (socket says 172.21.15.39:36598) 2024-09-13T17:05:48.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f259c640 1 -- 172.21.15.39:0/2280378492 learned_addr learned my addr 172.21.15.39:0/2280378492 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:48.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f2d9d640 1 --2- 172.21.15.39:0/2280378492 >> v2:172.21.15.39:3300/0 conn(0x7f42f4155d00 0x7f42f40c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:48.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f2d9d640 1 -- 172.21.15.39:0/2280378492 >> v1:172.21.15.84:6789/0 conn(0x7f42f406c940 legacy=0x7f42f41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:48.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f2d9d640 1 -- 172.21.15.39:0/2280378492 >> v1:172.21.15.39:6789/0 conn(0x7f42f41526b0 legacy=0x7f42f40aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:48.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.740+0000 7f42f2d9d640 1 -- 172.21.15.39:0/2280378492 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f406d120 con 0x7f42f4155d00 2024-09-13T17:05:48.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f2d9d640 1 --2- 172.21.15.39:0/2280378492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f4155d00 0x7f42f40c12b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009d90 tx=0x7f42ec01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259ff1bf38f1481a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:48.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f159a640 1 -- 172.21.15.39:0/2280378492 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0a5890 con 0x7f42f4155d00 2024-09-13T17:05:48.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f159a640 1 -- 172.21.15.39:0/2280378492 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec0a59f0 con 0x7f42f4155d00 2024-09-13T17:05:48.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f159a640 1 -- 172.21.15.39:0/2280378492 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0a5d80 con 0x7f42f4155d00 2024-09-13T17:05:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- 172.21.15.39:0/2280378492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f4155d00 msgr2=0x7f42f40c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 --2- 172.21.15.39:0/2280378492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f4155d00 0x7f42f40c12b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009d90 tx=0x7f42ec01c8e0 comp rx=0 tx=0).stop 2024-09-13T17:05:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- 172.21.15.39:0/2280378492 shutdown_connections 2024-09-13T17:05:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 --2- 172.21.15.39:0/2280378492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f4155d00 0x7f42f40c12b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:48.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- 172.21.15.39:0/2280378492 >> 172.21.15.39:0/2280378492 conn(0x7f42f4069e10 msgr2=0x7f42f406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:48.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- 172.21.15.39:0/2280378492 shutdown_connections 2024-09-13T17:05:48.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- 172.21.15.39:0/2280378492 wait complete. 2024-09-13T17:05:48.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 Processor -- start 2024-09-13T17:05:48.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- start start 2024-09-13T17:05:48.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f406c940 0x7f42f414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:48.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f41526b0 0x7f42f4071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:48.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42f4155d00 0x7f42f41ea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:48.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f4071300 con 0x7f42f406c940 2024-09-13T17:05:48.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40717c0 con 0x7f42f41526b0 2024-09-13T17:05:48.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40a7b10 con 0x7f42f4155d00 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f406c940 0x7f42f414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f406c940 0x7f42f414a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40426/0 (socket says 172.21.15.39:40426) 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f259c640 1 -- 172.21.15.39:0/3999957716 learned_addr learned my addr 172.21.15.39:0/3999957716 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f259c640 1 -- 172.21.15.39:0/3999957716 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42f4155d00 msgr2=0x7f42f41ea2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f259c640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42f4155d00 0x7f42f41ea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f259c640 1 -- 172.21.15.39:0/3999957716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f41526b0 msgr2=0x7f42f4071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f259c640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f41526b0 0x7f42f4071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f259c640 1 -- 172.21.15.39:0/3999957716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec009a70 con 0x7f42f406c940 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f259c640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f406c940 0x7f42f414a3a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f42e4005890 tx=0x7f42e400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42eaffd640 1 -- 172.21.15.39:0/3999957716 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4005cf0 con 0x7f42f406c940 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42eaffd640 1 -- 172.21.15.39:0/3999957716 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e4005e50 con 0x7f42f406c940 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f40a7df0 con 0x7f42f406c940 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.744+0000 7f42eaffd640 1 -- 172.21.15.39:0/3999957716 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4007290 con 0x7f42f406c940 2024-09-13T17:05:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.748+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f414a8e0 con 0x7f42f406c940 2024-09-13T17:05:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.748+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f4071980 con 0x7f42f406c940 2024-09-13T17:05:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.748+0000 7f42eaffd640 1 -- 172.21.15.39:0/3999957716 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f42e40073f0 con 0x7f42f406c940 2024-09-13T17:05:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.748+0000 7f42eaffd640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42d4066690 0x7f42d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:48.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.748+0000 7f42eaffd640 1 -- 172.21.15.39:0/3999957716 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f42e40840e0 con 0x7f42f406c940 2024-09-13T17:05:48.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.748+0000 7f42f1d9b640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42d4066690 0x7f42d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:48.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.748+0000 7f42f1d9b640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42d4066690 0x7f42d4068b50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000f70 tx=0x7f42dc0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:48.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.752+0000 7f42eaffd640 1 -- 172.21.15.39:0/3999957716 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f42e4050a20 con 0x7f42f406c940 2024-09-13T17:05:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42f40a5160 con 0x7f42f406c940 2024-09-13T17:05:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42eaffd640 1 -- 172.21.15.39:0/3999957716 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f42e4017020 con 0x7f42f406c940 2024-09-13T17:05:48.943 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:48.943 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:48.944 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42d4066690 msgr2=0x7f42d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42d4066690 0x7f42d4068b50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f42dc000f70 tx=0x7f42dc0079c0 comp rx=0 tx=0).stop 2024-09-13T17:05:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f406c940 msgr2=0x7f42f414a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f406c940 0x7f42f414a3a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f42e4005890 tx=0x7f42e400c960 comp rx=0 tx=0).stop 2024-09-13T17:05:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 shutdown_connections 2024-09-13T17:05:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42d4066690 0x7f42d4068b50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42f41526b0 0x7f42f4071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42f4155d00 0x7f42f41ea2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 --2- 172.21.15.39:0/3999957716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42f406c940 0x7f42f414a3a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 >> 172.21.15.39:0/3999957716 conn(0x7f42f4069e10 msgr2=0x7f42f406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 shutdown_connections 2024-09-13T17:05:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:48.944+0000 7f42f8b44640 1 -- 172.21.15.39:0/3999957716 wait complete. 2024-09-13T17:05:48.961 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:05:49.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:49.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda0990b640 1 Processor -- start 2024-09-13T17:05:49.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda0990b640 1 -- start start 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda0990b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fda0414e0c0 0x7fda040a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda0990b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 0x7fda0406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda0990b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fda0406d300 0x7fda040c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda0990b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040c2780 con 0x7fda0414e0c0 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda0990b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040c2a60 con 0x7fda0406d300 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda0990b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040c2d40 con 0x7fda040c1f50 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda02ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fda0414e0c0 0x7fda040a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda02ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fda0414e0c0 0x7fda040a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40434/0 (socket says 172.21.15.39:40434) 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda02ffd640 1 -- 172.21.15.39:0/2130207201 learned_addr learned my addr 172.21.15.39:0/2130207201 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda027fc640 1 --2- 172.21.15.39:0/2130207201 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 0x7fda0406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda037fe640 1 --2- 172.21.15.39:0/2130207201 >> v2:172.21.15.84:3300/0 conn(0x7fda0406d300 0x7fda040c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda02ffd640 1 -- 172.21.15.39:0/2130207201 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 msgr2=0x7fda0406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:49.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda02ffd640 1 --2- 172.21.15.39:0/2130207201 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 0x7fda0406cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda02ffd640 1 -- 172.21.15.39:0/2130207201 >> v2:172.21.15.84:3300/0 conn(0x7fda0406d300 msgr2=0x7fda040c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:49.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda02ffd640 1 --2- 172.21.15.39:0/2130207201 >> v2:172.21.15.84:3300/0 conn(0x7fda0406d300 0x7fda040c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda02ffd640 1 -- 172.21.15.39:0/2130207201 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040c3020 con 0x7fda0414e0c0 2024-09-13T17:05:49.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda027fc640 1 --2- 172.21.15.39:0/2130207201 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 0x7fda0406cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:49.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.696+0000 7fda037fe640 1 --2- 172.21.15.39:0/2130207201 >> v2:172.21.15.84:3300/0 conn(0x7fda0406d300 0x7fda040c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:05:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 --2- 172.21.15.39:0/2130207201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0414e0c0 0x7fda040a4e70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fda040c3f40 tx=0x7fd9f401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a252596f1057c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda01ffb640 1 -- 172.21.15.39:0/2130207201 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040c5490 con 0x7fda0414e0c0 2024-09-13T17:05:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda01ffb640 1 -- 172.21.15.39:0/2130207201 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f40a7050 con 0x7fda0414e0c0 2024-09-13T17:05:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda01ffb640 1 -- 172.21.15.39:0/2130207201 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40ab820 con 0x7fda0414e0c0 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- 172.21.15.39:0/2130207201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0414e0c0 msgr2=0x7fda040a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 --2- 172.21.15.39:0/2130207201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0414e0c0 0x7fda040a4e70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fda040c3f40 tx=0x7fd9f401c460 comp rx=0 tx=0).stop 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- 172.21.15.39:0/2130207201 shutdown_connections 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 --2- 172.21.15.39:0/2130207201 >> v2:172.21.15.84:3300/0 conn(0x7fda0406d300 0x7fda040c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 --2- 172.21.15.39:0/2130207201 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 0x7fda0406cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 --2- 172.21.15.39:0/2130207201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0414e0c0 0x7fda040a4e70 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- 172.21.15.39:0/2130207201 >> 172.21.15.39:0/2130207201 conn(0x7fda04069e10 msgr2=0x7fda0406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- 172.21.15.39:0/2130207201 shutdown_connections 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- 172.21.15.39:0/2130207201 wait complete. 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 Processor -- start 2024-09-13T17:05:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- start start 2024-09-13T17:05:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0406d300 0x7fda040717d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 0x7fda04140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0414e0c0 0x7fda04140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda041eaed0 con 0x7fda0406d300 2024-09-13T17:05:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda041eb400 con 0x7fda0414e0c0 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0406f630 con 0x7fda040c1f50 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0406d300 0x7fda040717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0406d300 0x7fda040717d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40446/0 (socket says 172.21.15.39:40446) 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 -- 172.21.15.39:0/353363393 learned_addr learned my addr 172.21.15.39:0/353363393 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda037fe640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0414e0c0 0x7fda04140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda027fc640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 0x7fda04140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 -- 172.21.15.39:0/353363393 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 msgr2=0x7fda04140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 0x7fda04140510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 -- 172.21.15.39:0/353363393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0414e0c0 msgr2=0x7fda04140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0414e0c0 0x7fda04140a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 -- 172.21.15.39:0/353363393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f40094d0 con 0x7fda0406d300 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda02ffd640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0406d300 0x7fda040717d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd9f401c940 tx=0x7fd9f4019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fd9ebfff640 1 -- 172.21.15.39:0/353363393 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40b6640 con 0x7fda0406d300 2024-09-13T17:05:49.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fd9ebfff640 1 -- 172.21.15.39:0/353363393 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f40b67a0 con 0x7fda0406d300 2024-09-13T17:05:49.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda04144e80 con 0x7fda0406d300 2024-09-13T17:05:49.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.700+0000 7fd9ebfff640 1 -- 172.21.15.39:0/353363393 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40b6b30 con 0x7fda0406d300 2024-09-13T17:05:49.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.704+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0406f2c0 con 0x7fda0406d300 2024-09-13T17:05:49.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.704+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda040c0cb0 con 0x7fda0406d300 2024-09-13T17:05:49.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.704+0000 7fd9ebfff640 1 -- 172.21.15.39:0/353363393 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9f40b2040 con 0x7fda0406d300 2024-09-13T17:05:49.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.704+0000 7fd9ebfff640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f0066690 0x7fd9f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:49.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.704+0000 7fd9ebfff640 1 -- 172.21.15.39:0/353363393 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7fd9f411ec80 con 0x7fda0406d300 2024-09-13T17:05:49.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.704+0000 7fda027fc640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f0066690 0x7fd9f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:49.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.704+0000 7fda027fc640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f0066690 0x7fd9f0068b50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fda04147dd0 tx=0x7fd9f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.708+0000 7fd9ebfff640 1 -- 172.21.15.39:0/353363393 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd9f40eb520 con 0x7fda0406d300 2024-09-13T17:05:49.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.900+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda0406a550 con 0x7fda0406d300 2024-09-13T17:05:49.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.900+0000 7fd9ebfff640 1 -- 172.21.15.39:0/353363393 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7fd9f40eeb40 con 0x7fda0406d300 2024-09-13T17:05:49.900 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:49.901 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:49.901 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:49.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f0066690 msgr2=0x7fd9f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f0066690 0x7fd9f0068b50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fda04147dd0 tx=0x7fd9f8008040 comp rx=0 tx=0).stop 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0406d300 msgr2=0x7fda040717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0406d300 0x7fda040717d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd9f401c940 tx=0x7fd9f4019200 comp rx=0 tx=0).stop 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 shutdown_connections 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f0066690 0x7fd9f0068b50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda0414e0c0 0x7fda04140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda040c1f50 0x7fda04140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 --2- 172.21.15.39:0/353363393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda0406d300 0x7fda040717d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 >> 172.21.15.39:0/353363393 conn(0x7fda04069e10 msgr2=0x7fda0406bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 shutdown_connections 2024-09-13T17:05:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:49.904+0000 7fda0990b640 1 -- 172.21.15.39:0/353363393 wait complete. 2024-09-13T17:05:49.919 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:05:50.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.096+0000 7ff1344e6640 1 Processor -- start 2024-09-13T17:05:51.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.096+0000 7ff1344e6640 1 -- start start 2024-09-13T17:05:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff1344e6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c06d020 0x7ff12c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff1344e6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff12c0c4530 0x7ff12c0c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff1344e6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12c154100 con 0x7ff12c151460 2024-09-13T17:05:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff1344e6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12c0a8c50 con 0x7ff12c0c4530 2024-09-13T17:05:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff1344e6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1543e0 con 0x7ff12c06d020 2024-09-13T17:05:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132a83640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff12c0c4530 0x7ff12c0c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132a83640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff12c0c4530 0x7ff12c0c49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50438/0 (socket says 172.21.15.39:50438) 2024-09-13T17:05:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132a83640 1 -- 172.21.15.39:0/3741755028 learned_addr learned my addr 172.21.15.39:0/3741755028 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132282640 1 --2- 172.21.15.39:0/3741755028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c06d020 0x7ff12c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132a83640 1 -- 172.21.15.39:0/3741755028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c06d020 msgr2=0x7ff12c0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132a83640 1 --2- 172.21.15.39:0/3741755028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c06d020 0x7ff12c0a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132a83640 1 -- 172.21.15.39:0/3741755028 >> v1:172.21.15.84:6789/0 conn(0x7ff12c151460 legacy=0x7ff12c14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132a83640 1 -- 172.21.15.39:0/3741755028 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c1546c0 con 0x7ff12c0c4530 2024-09-13T17:05:51.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132282640 1 --2- 172.21.15.39:0/3741755028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c06d020 0x7ff12c0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:51.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff132a83640 1 --2- 172.21.15.39:0/3741755028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c0c4530 0x7ff12c0c49c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff124009e50 tx=0x7ff12401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2261c9357a3b3088 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:51.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff131280640 1 -- 172.21.15.39:0/3741755028 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1240a57b0 con 0x7ff12c0c4530 2024-09-13T17:05:51.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff131280640 1 -- 172.21.15.39:0/3741755028 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1240a5910 con 0x7ff12c0c4530 2024-09-13T17:05:51.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.100+0000 7ff131280640 1 -- 172.21.15.39:0/3741755028 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1240a5ca0 con 0x7ff12c0c4530 2024-09-13T17:05:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- 172.21.15.39:0/3741755028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c0c4530 msgr2=0x7ff12c0c49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 --2- 172.21.15.39:0/3741755028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c0c4530 0x7ff12c0c49c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff124009e50 tx=0x7ff12401c930 comp rx=0 tx=0).stop 2024-09-13T17:05:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- 172.21.15.39:0/3741755028 shutdown_connections 2024-09-13T17:05:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 --2- 172.21.15.39:0/3741755028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c0c4530 0x7ff12c0c49c0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 --2- 172.21.15.39:0/3741755028 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c06d020 0x7ff12c0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- 172.21.15.39:0/3741755028 >> 172.21.15.39:0/3741755028 conn(0x7ff12c069e10 msgr2=0x7ff12c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- 172.21.15.39:0/3741755028 shutdown_connections 2024-09-13T17:05:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- 172.21.15.39:0/3741755028 wait complete. 2024-09-13T17:05:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 Processor -- start 2024-09-13T17:05:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- start start 2024-09-13T17:05:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c06d020 0x7ff12c0a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff12c0c4530 0x7ff12c148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c151460 0x7ff12c1500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0a58d0 con 0x7ff12c06d020 2024-09-13T17:05:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1ece80 con 0x7ff12c0c4530 2024-09-13T17:05:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0ab4b0 con 0x7ff12c151460 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132282640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c06d020 0x7ff12c0a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132282640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c06d020 0x7ff12c0a9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50452/0 (socket says 172.21.15.39:50452) 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132282640 1 -- 172.21.15.39:0/2662900366 learned_addr learned my addr 172.21.15.39:0/2662900366 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132a83640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c151460 0x7ff12c1500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff131a81640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff12c0c4530 0x7ff12c148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132282640 1 -- 172.21.15.39:0/2662900366 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c151460 msgr2=0x7ff12c1500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132282640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c151460 0x7ff12c1500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132282640 1 -- 172.21.15.39:0/2662900366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff12c0c4530 msgr2=0x7ff12c148b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132282640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff12c0c4530 0x7ff12c148b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132282640 1 -- 172.21.15.39:0/2662900366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124009b30 con 0x7ff12c06d020 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff131a81640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff12c0c4530 0x7ff12c148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff132282640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c06d020 0x7ff12c0a9a90 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c7f0 tx=0x7ff11c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff122ffd640 1 -- 172.21.15.39:0/2662900366 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c007820 con 0x7ff12c06d020 2024-09-13T17:05:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff122ffd640 1 -- 172.21.15.39:0/2662900366 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c007980 con 0x7ff12c06d020 2024-09-13T17:05:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c0ab260 con 0x7ff12c06d020 2024-09-13T17:05:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff122ffd640 1 -- 172.21.15.39:0/2662900366 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c007d30 con 0x7ff12c06d020 2024-09-13T17:05:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c14c500 con 0x7ff12c06d020 2024-09-13T17:05:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.104+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c1ed0f0 con 0x7ff12c06d020 2024-09-13T17:05:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.108+0000 7ff122ffd640 1 -- 172.21.15.39:0/2662900366 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff11c0113f0 con 0x7ff12c06d020 2024-09-13T17:05:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.108+0000 7ff122ffd640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff110066690 0x7ff110068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.108+0000 7ff122ffd640 1 -- 172.21.15.39:0/2662900366 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7ff11c084f70 con 0x7ff12c06d020 2024-09-13T17:05:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.108+0000 7ff131a81640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff110066690 0x7ff110068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.108+0000 7ff131a81640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff110066690 0x7ff110068b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0c4e60 tx=0x7ff128009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:51.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.112+0000 7ff122ffd640 1 -- 172.21.15.39:0/2662900366 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff11c0b6db0 con 0x7ff12c06d020 2024-09-13T17:05:51.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff12c0a4e70 con 0x7ff12c06d020 2024-09-13T17:05:51.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff122ffd640 1 -- 172.21.15.39:0/2662900366 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7ff11c01f020 con 0x7ff12c06d020 2024-09-13T17:05:51.309 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:51.309 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:51.311 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:51.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff110066690 msgr2=0x7ff110068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:51.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff110066690 0x7ff110068b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0c4e60 tx=0x7ff128009040 comp rx=0 tx=0).stop 2024-09-13T17:05:51.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c06d020 msgr2=0x7ff12c0a9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:51.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c06d020 0x7ff12c0a9a90 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c7f0 tx=0x7ff11c00ccb0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 shutdown_connections 2024-09-13T17:05:51.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff110066690 0x7ff110068b50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff12c0c4530 0x7ff12c148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff12c151460 0x7ff12c1500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 --2- 172.21.15.39:0/2662900366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff12c06d020 0x7ff12c0a9a90 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:51.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.308+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 >> 172.21.15.39:0/2662900366 conn(0x7ff12c069e10 msgr2=0x7ff12c14d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:51.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.312+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 shutdown_connections 2024-09-13T17:05:51.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:51.312+0000 7ff1344e6640 1 -- 172.21.15.39:0/2662900366 wait complete. 2024-09-13T17:05:51.327 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:05:51.920 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:52.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd72dad640 1 Processor -- start 2024-09-13T17:05:52.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd72dad640 1 -- start start 2024-09-13T17:05:52.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd72dad640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd6c151750 0x7fcd6c0c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:52.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd72dad640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcd6c0c7860 0x7fcd6c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:52.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd72dad640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6c071080 con 0x7fcd6c0bd010 2024-09-13T17:05:52.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd72dad640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c151b30 con 0x7fcd6c0c7860 2024-09-13T17:05:52.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd72dad640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c157e80 con 0x7fcd6c151750 2024-09-13T17:05:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd7134a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcd6c0c7860 0x7fcd6c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd7134a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcd6c0c7860 0x7fcd6c070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50464/0 (socket says 172.21.15.39:50464) 2024-09-13T17:05:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd7134a640 1 -- 172.21.15.39:0/2637039273 learned_addr learned my addr 172.21.15.39:0/2637039273 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.056+0000 7fcd7134a640 1 -- 172.21.15.39:0/2637039273 >> v2:172.21.15.84:3300/0 conn(0x7fcd6c151750 msgr2=0x7fcd6c0c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd70b49640 1 --2- 172.21.15.39:0/2637039273 >> v2:172.21.15.84:3300/0 conn(0x7fcd6c151750 0x7fcd6c0c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd7134a640 1 --2- 172.21.15.39:0/2637039273 >> v2:172.21.15.84:3300/0 conn(0x7fcd6c151750 0x7fcd6c0c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd7134a640 1 -- 172.21.15.39:0/2637039273 >> v1:172.21.15.39:6789/0 conn(0x7fcd6c0bd010 legacy=0x7fcd6c070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd7134a640 1 -- 172.21.15.39:0/2637039273 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c158160 con 0x7fcd6c0c7860 2024-09-13T17:05:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd70b49640 1 --2- 172.21.15.39:0/2637039273 >> v2:172.21.15.84:3300/0 conn(0x7fcd6c151750 0x7fcd6c0c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd7134a640 1 --2- 172.21.15.39:0/2637039273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0c7860 0x7fcd6c070b40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0afaf0 tx=0x7fcd6401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6454ff00ff10e873 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd6b7fe640 1 -- 172.21.15.39:0/2637039273 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640a5880 con 0x7fcd6c0c7860 2024-09-13T17:05:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd6b7fe640 1 -- 172.21.15.39:0/2637039273 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd640a59e0 con 0x7fcd6c0c7860 2024-09-13T17:05:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd6b7fe640 1 -- 172.21.15.39:0/2637039273 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640a5d70 con 0x7fcd6c0c7860 2024-09-13T17:05:52.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd72dad640 1 -- 172.21.15.39:0/2637039273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0c7860 msgr2=0x7fcd6c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:52.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd72dad640 1 --2- 172.21.15.39:0/2637039273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0c7860 0x7fcd6c070b40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0afaf0 tx=0x7fcd6401c9c0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd72dad640 1 -- 172.21.15.39:0/2637039273 shutdown_connections 2024-09-13T17:05:52.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd72dad640 1 --2- 172.21.15.39:0/2637039273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0c7860 0x7fcd6c070b40 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd72dad640 1 --2- 172.21.15.39:0/2637039273 >> v2:172.21.15.84:3300/0 conn(0x7fcd6c151750 0x7fcd6c0c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd72dad640 1 -- 172.21.15.39:0/2637039273 >> 172.21.15.39:0/2637039273 conn(0x7fcd6c069e10 msgr2=0x7fcd6c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:52.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd72dad640 1 -- 172.21.15.39:0/2637039273 shutdown_connections 2024-09-13T17:05:52.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.060+0000 7fcd72dad640 1 -- 172.21.15.39:0/2637039273 wait complete. 2024-09-13T17:05:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 Processor -- start 2024-09-13T17:05:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 -- start start 2024-09-13T17:05:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0bd010 0x7fcd6c14aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6c0c7860 0x7fcd6c156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd6c151750 0x7fcd6c0c0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c14b420 con 0x7fcd6c0bd010 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1efa20 con 0x7fcd6c0c7860 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1efc70 con 0x7fcd6c151750 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd70b49640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0bd010 0x7fcd6c14aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd70b49640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0bd010 0x7fcd6c14aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50470/0 (socket says 172.21.15.39:50470) 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd70b49640 1 -- 172.21.15.39:0/439080192 learned_addr learned my addr 172.21.15.39:0/439080192 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd7134a640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd6c151750 0x7fcd6c0c0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd6bfff640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6c0c7860 0x7fcd6c156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd70b49640 1 -- 172.21.15.39:0/439080192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd6c151750 msgr2=0x7fcd6c0c0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd70b49640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd6c151750 0x7fcd6c0c0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd70b49640 1 -- 172.21.15.39:0/439080192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6c0c7860 msgr2=0x7fcd6c156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd70b49640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6c0c7860 0x7fcd6c156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd70b49640 1 -- 172.21.15.39:0/439080192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64009b30 con 0x7fcd6c0bd010 2024-09-13T17:05:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd6bfff640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6c0c7860 0x7fcd6c156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd70b49640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0bd010 0x7fcd6c14aee0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c7c0 tx=0x7fcd6000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd697fa640 1 -- 172.21.15.39:0/439080192 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60007a30 con 0x7fcd6c0bd010 2024-09-13T17:05:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd697fa640 1 -- 172.21.15.39:0/439080192 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd60007b90 con 0x7fcd6c0bd010 2024-09-13T17:05:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6c1eff50 con 0x7fcd6c0bd010 2024-09-13T17:05:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd697fa640 1 -- 172.21.15.39:0/439080192 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000ce70 con 0x7fcd6c0bd010 2024-09-13T17:05:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c1f0510 con 0x7fcd6c0bd010 2024-09-13T17:05:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c0c3cf0 con 0x7fcd6c0bd010 2024-09-13T17:05:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd697fa640 1 -- 172.21.15.39:0/439080192 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcd600113f0 con 0x7fcd6c0bd010 2024-09-13T17:05:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.064+0000 7fcd697fa640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd50066690 0x7fcd50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.068+0000 7fcd6bfff640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd50066690 0x7fcd50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.068+0000 7fcd697fa640 1 -- 172.21.15.39:0/439080192 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7fcd60085280 con 0x7fcd6c0bd010 2024-09-13T17:05:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.068+0000 7fcd6bfff640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd50066690 0x7fcd50068b50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c1570d0 tx=0x7fcd58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.072+0000 7fcd697fa640 1 -- 172.21.15.39:0/439080192 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcd60051a00 con 0x7fcd6c0bd010 2024-09-13T17:05:52.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.260+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd6c1f0700 con 0x7fcd6c0bd010 2024-09-13T17:05:52.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd697fa640 1 -- 172.21.15.39:0/439080192 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7fcd6001e020 con 0x7fcd6c0bd010 2024-09-13T17:05:52.262 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:52.262 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:52.263 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:52.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd50066690 msgr2=0x7fcd50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:52.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd50066690 0x7fcd50068b50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c1570d0 tx=0x7fcd58009040 comp rx=0 tx=0).stop 2024-09-13T17:05:52.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0bd010 msgr2=0x7fcd6c14aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:52.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0bd010 0x7fcd6c14aee0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c7c0 tx=0x7fcd6000cc80 comp rx=0 tx=0).stop 2024-09-13T17:05:52.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 shutdown_connections 2024-09-13T17:05:52.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcd50066690 0x7fcd50068b50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd6c0c7860 0x7fcd6c156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd6c151750 0x7fcd6c0c0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 --2- 172.21.15.39:0/439080192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd6c0bd010 0x7fcd6c14aee0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:52.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 >> 172.21.15.39:0/439080192 conn(0x7fcd6c069e10 msgr2=0x7fcd6c0ca210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:52.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 shutdown_connections 2024-09-13T17:05:52.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:52.264+0000 7fcd72dad640 1 -- 172.21.15.39:0/439080192 wait complete. 2024-09-13T17:05:52.280 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:05:53.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:53.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa6a60ae640 1 Processor -- start 2024-09-13T17:05:53.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa6a60ae640 1 -- start start 2024-09-13T17:05:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa6a60ae640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a00c26c0 0x7fa6a006f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa6a60ae640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a006d750 con 0x7fa6a00c7180 2024-09-13T17:05:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa6a60ae640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a0155ce0 con 0x7fa6a014f340 2024-09-13T17:05:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa6a60ae640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a006e410 con 0x7fa6a00c26c0 2024-09-13T17:05:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69effd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa6a00c7180 0x7fa6a006f050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:59646/0 (socket says 172.21.15.39:59646) 2024-09-13T17:05:53.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69effd640 1 -- 172.21.15.39:0/452922755 learned_addr learned my addr 172.21.15.39:0/452922755 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:53.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69ffff640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a00c26c0 0x7fa6a006f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:53.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103371596 0 0) 0x7fa6a006d750 con 0x7fa6a00c7180 2024-09-13T17:05:53.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa688003530 con 0x7fa6a00c7180 2024-09-13T17:05:53.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2916803453 0 0) 0x7fa688003530 con 0x7fa6a00c7180 2024-09-13T17:05:53.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a00c26c0 msgr2=0x7fa6a006f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:53.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a00c26c0 0x7fa6a006f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 >> v1:172.21.15.84:6789/0 conn(0x7fa6a014f340 legacy=0x7fa6a0071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a006d3e0 con 0x7fa6a00c7180 2024-09-13T17:05:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fa694001a00 con 0x7fa6a00c7180 2024-09-13T17:05:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 >> v1:172.21.15.39:6789/0 conn(0x7fa6a00c7180 legacy=0x7fa6a006f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6880039b0 0x7fa688003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6880047d0 0x7fa688004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a00c26c0 0x7fa6880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a006e410 con 0x7fa6880039b0 2024-09-13T17:05:53.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68800baf0 con 0x7fa6880047d0 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68800bcb0 con 0x7fa6a00c26c0 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69f7fe640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6880039b0 0x7fa688003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69effd640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6880047d0 0x7fa688004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69ffff640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a00c26c0 0x7fa6880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69f7fe640 1 -- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a00c26c0 msgr2=0x7fa6880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69f7fe640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a00c26c0 0x7fa6880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69f7fe640 1 -- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6880047d0 msgr2=0x7fa688004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69f7fe640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6880047d0 0x7fa688004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69f7fe640 1 -- 172.21.15.39:0/452922755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688003530 con 0x7fa6880039b0 2024-09-13T17:05:53.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.464+0000 7fa69ffff640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a00c26c0 0x7fa6880051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa69f7fe640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6880039b0 0x7fa688003dd0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa698001780 tx=0x7fa69801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e5d9d35ee887b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6980052d0 con 0x7fa6880039b0 2024-09-13T17:05:53.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6980a7050 con 0x7fa6880039b0 2024-09-13T17:05:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa69e7fc640 1 -- 172.21.15.39:0/452922755 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698003600 con 0x7fa6880039b0 2024-09-13T17:05:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 -- 172.21.15.39:0/452922755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6880039b0 msgr2=0x7fa688003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 --2- 172.21.15.39:0/452922755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6880039b0 0x7fa688003dd0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa698001780 tx=0x7fa69801c420 comp rx=0 tx=0).stop 2024-09-13T17:05:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa69ffff640 1 -- 172.21.15.39:0/452922755 reap_dead start 2024-09-13T17:05:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 -- 172.21.15.39:0/452922755 shutdown_connections 2024-09-13T17:05:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 -- 172.21.15.39:0/452922755 >> 172.21.15.39:0/452922755 conn(0x7fa6a00a96b0 msgr2=0x7fa6a00a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 -- 172.21.15.39:0/452922755 shutdown_connections 2024-09-13T17:05:53.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 -- 172.21.15.39:0/452922755 wait complete. 2024-09-13T17:05:53.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 Processor -- start 2024-09-13T17:05:53.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 -- start start 2024-09-13T17:05:53.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a015a8f0 0x7fa6a00ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:53.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6a00ad040 0x7fa6a006a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:53.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6a006a940 0x7fa6a0153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00a7000 con 0x7fa6a006a940 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a014e0b0 con 0x7fa6a00ad040 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.468+0000 7fa6a60ae640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a006b1b0 con 0x7fa6a015a8f0 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6a006a940 0x7fa6a0153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6a006a940 0x7fa6a0153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50484/0 (socket says 172.21.15.39:50484) 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69ffff640 1 -- 172.21.15.39:0/4030664207 learned_addr learned my addr 172.21.15.39:0/4030664207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69effd640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6a00ad040 0x7fa6a006a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69f7fe640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a015a8f0 0x7fa6a00ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69ffff640 1 -- 172.21.15.39:0/4030664207 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a015a8f0 msgr2=0x7fa6a00ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:53.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69ffff640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a015a8f0 0x7fa6a00ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69ffff640 1 -- 172.21.15.39:0/4030664207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6a00ad040 msgr2=0x7fa6a006a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:53.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69ffff640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6a00ad040 0x7fa6a006a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69ffff640 1 -- 172.21.15.39:0/4030664207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a01f0380 con 0x7fa6a006a940 2024-09-13T17:05:53.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69ffff640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6a006a940 0x7fa6a0153f80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00c6a90 tx=0x7fa690010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:53.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa687fff640 1 -- 172.21.15.39:0/4030664207 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000e840 con 0x7fa6a006a940 2024-09-13T17:05:53.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa687fff640 1 -- 172.21.15.39:0/4030664207 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69000e9a0 con 0x7fa6a006a940 2024-09-13T17:05:53.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a014cd00 con 0x7fa6a006a940 2024-09-13T17:05:53.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa687fff640 1 -- 172.21.15.39:0/4030664207 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000ed30 con 0x7fa6a006a940 2024-09-13T17:05:53.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a014c9d0 con 0x7fa6a006a940 2024-09-13T17:05:53.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a014c830 con 0x7fa6a006a940 2024-09-13T17:05:53.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa687fff640 1 -- 172.21.15.39:0/4030664207 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa69001c400 con 0x7fa6a006a940 2024-09-13T17:05:53.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa687fff640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa68c0666c0 0x7fa68c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:53.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69f7fe640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa68c0666c0 0x7fa68c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:53.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa687fff640 1 -- 172.21.15.39:0/4030664207 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7fa690084940 con 0x7fa6a006a940 2024-09-13T17:05:53.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.472+0000 7fa69f7fe640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa68c0666c0 0x7fa68c068b80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa698001750 tx=0x7fa6980011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:53.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.476+0000 7fa687fff640 1 -- 172.21.15.39:0/4030664207 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa6a014c830 con 0x7fa6a006a940 2024-09-13T17:05:53.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.672+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6a014d170 con 0x7fa6a006a940 2024-09-13T17:05:53.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.672+0000 7fa687fff640 1 -- 172.21.15.39:0/4030664207 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7fa6a014d170 con 0x7fa6a006a940 2024-09-13T17:05:53.674 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:53.674 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:53.674 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa68c0666c0 msgr2=0x7fa68c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa68c0666c0 0x7fa68c068b80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa698001750 tx=0x7fa6980011a0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6a006a940 msgr2=0x7fa6a0153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6a006a940 0x7fa6a0153f80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00c6a90 tx=0x7fa690010b60 comp rx=0 tx=0).stop 2024-09-13T17:05:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 shutdown_connections 2024-09-13T17:05:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa68c0666c0 0x7fa68c068b80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6a00ad040 0x7fa6a006a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6a006a940 0x7fa6a0153f80 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 --2- 172.21.15.39:0/4030664207 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa6a015a8f0 0x7fa6a00ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 >> 172.21.15.39:0/4030664207 conn(0x7fa6a00a96b0 msgr2=0x7fa6a014fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 shutdown_connections 2024-09-13T17:05:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:53.676+0000 7fa6a60ae640 1 -- 172.21.15.39:0/4030664207 wait complete. 2024-09-13T17:05:53.692 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:05:54.281 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.416+0000 7f037224f640 1 Processor -- start 2024-09-13T17:05:54.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f037224f640 1 -- start start 2024-09-13T17:05:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f037224f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c150f80 0x7f036c153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:54.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f037224f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f036c153e70 0x7f036c0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f037224f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c1546c0 con 0x7f036c0c5330 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f037224f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c0c19a0 con 0x7f036c153e70 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f037224f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0c49f0 con 0x7f036c150f80 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f036bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f036c153e70 0x7f036c0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f036bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f036c153e70 0x7f036c0c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50498/0 (socket says 172.21.15.39:50498) 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f036bfff640 1 -- 172.21.15.39:0/3229137287 learned_addr learned my addr 172.21.15.39:0/3229137287 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f036b7fe640 1 --2- 172.21.15.39:0/3229137287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c150f80 0x7f036c153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f036bfff640 1 -- 172.21.15.39:0/3229137287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c150f80 msgr2=0x7f036c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f036bfff640 1 --2- 172.21.15.39:0/3229137287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c150f80 0x7f036c153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f036bfff640 1 -- 172.21.15.39:0/3229137287 >> v1:172.21.15.84:6789/0 conn(0x7f036c0c5330 legacy=0x7f036c06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:54.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.420+0000 7f036bfff640 1 -- 172.21.15.39:0/3229137287 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c1543e0 con 0x7f036c153e70 2024-09-13T17:05:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036bfff640 1 --2- 172.21.15.39:0/3229137287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c153e70 0x7f036c0c1210 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0364009e50 tx=0x7f036401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967eb914d959a4fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036a7fc640 1 -- 172.21.15.39:0/3229137287 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640a57b0 con 0x7f036c153e70 2024-09-13T17:05:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036a7fc640 1 -- 172.21.15.39:0/3229137287 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03640a5910 con 0x7f036c153e70 2024-09-13T17:05:54.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036a7fc640 1 -- 172.21.15.39:0/3229137287 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640a5ca0 con 0x7f036c153e70 2024-09-13T17:05:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 -- 172.21.15.39:0/3229137287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c153e70 msgr2=0x7f036c0c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 --2- 172.21.15.39:0/3229137287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c153e70 0x7f036c0c1210 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0364009e50 tx=0x7f036401c970 comp rx=0 tx=0).stop 2024-09-13T17:05:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 -- 172.21.15.39:0/3229137287 shutdown_connections 2024-09-13T17:05:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 --2- 172.21.15.39:0/3229137287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c153e70 0x7f036c0c1210 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 --2- 172.21.15.39:0/3229137287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c150f80 0x7f036c153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 -- 172.21.15.39:0/3229137287 >> 172.21.15.39:0/3229137287 conn(0x7f036c069e10 msgr2=0x7f036c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 -- 172.21.15.39:0/3229137287 shutdown_connections 2024-09-13T17:05:54.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 -- 172.21.15.39:0/3229137287 wait complete. 2024-09-13T17:05:54.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 Processor -- start 2024-09-13T17:05:54.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 -- start start 2024-09-13T17:05:54.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c0c5330 0x7f036c14dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:54.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c150f80 0x7f036c14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:54.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f036c153e70 0x7f036c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:54.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c158690 con 0x7f036c150f80 2024-09-13T17:05:54.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c154fa0 con 0x7f036c153e70 2024-09-13T17:05:54.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f037224f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c14e750 con 0x7f036c0c5330 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c150f80 0x7f036c14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c150f80 0x7f036c14e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50500/0 (socket says 172.21.15.39:50500) 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036affd640 1 -- 172.21.15.39:0/1189883203 learned_addr learned my addr 172.21.15.39:0/1189883203 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036b7fe640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c0c5330 0x7f036c14dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036bfff640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f036c153e70 0x7f036c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036affd640 1 -- 172.21.15.39:0/1189883203 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c0c5330 msgr2=0x7f036c14dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036affd640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c0c5330 0x7f036c14dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036affd640 1 -- 172.21.15.39:0/1189883203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f036c153e70 msgr2=0x7f036c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036affd640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f036c153e70 0x7f036c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036affd640 1 -- 172.21.15.39:0/1189883203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364009b30 con 0x7f036c150f80 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f036affd640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c150f80 0x7f036c14e210 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f036000c560 tx=0x7f036000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.424+0000 7f0353fff640 1 -- 172.21.15.39:0/1189883203 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360005220 con 0x7f036c150f80 2024-09-13T17:05:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f0353fff640 1 -- 172.21.15.39:0/1189883203 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360005380 con 0x7f036c150f80 2024-09-13T17:05:54.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c154cc0 con 0x7f036c150f80 2024-09-13T17:05:54.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f0353fff640 1 -- 172.21.15.39:0/1189883203 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360007270 con 0x7f036c150f80 2024-09-13T17:05:54.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c1eb150 con 0x7f036c150f80 2024-09-13T17:05:54.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036c14f9e0 con 0x7f036c150f80 2024-09-13T17:05:54.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f0353fff640 1 -- 172.21.15.39:0/1189883203 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03600073d0 con 0x7f036c150f80 2024-09-13T17:05:54.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f0353fff640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0358066690 0x7f0358068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:54.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f0353fff640 1 -- 172.21.15.39:0/1189883203 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 8529+0+0 (secure 0 0 0) 0x7f0360084500 con 0x7f036c150f80 2024-09-13T17:05:54.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f036b7fe640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0358066690 0x7f0358068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:54.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.428+0000 7f036b7fe640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0358066690 0x7f0358068b50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f035c00b9d0 tx=0x7f035c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.432+0000 7f0353fff640 1 -- 172.21.15.39:0/1189883203 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0360050da0 con 0x7f036c150f80 2024-09-13T17:05:54.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.624+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f036c153930 con 0x7f036c150f80 2024-09-13T17:05:54.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.624+0000 7f0353fff640 1 -- 172.21.15.39:0/1189883203 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f03600543c0 con 0x7f036c150f80 2024-09-13T17:05:54.625 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:54.625 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96},{"gid":7050,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:03:53.068981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6235":{"gid":6235,"name":"d","rank":0,"incarnation":107,"state":"up:active","state_seq":8,"addr":"172.21.15.84:6835/263561599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":263561599},{"type":"v1","addr":"172.21.15.84:6835","nonce":263561599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:54.625 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 121 2024-09-13T17:05:54.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0358066690 msgr2=0x7f0358068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:54.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0358066690 0x7f0358068b50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f035c00b9d0 tx=0x7f035c008040 comp rx=0 tx=0).stop 2024-09-13T17:05:54.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c150f80 msgr2=0x7f036c14e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:54.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c150f80 0x7f036c14e210 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f036000c560 tx=0x7f036000ca20 comp rx=0 tx=0).stop 2024-09-13T17:05:54.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 shutdown_connections 2024-09-13T17:05:54.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0358066690 0x7f0358068b50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:54.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f036c153e70 0x7f036c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:54.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c150f80 0x7f036c14e210 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:54.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 --2- 172.21.15.39:0/1189883203 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f036c0c5330 0x7f036c14dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:54.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 >> 172.21.15.39:0/1189883203 conn(0x7f036c069e10 msgr2=0x7f036c14fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:54.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 shutdown_connections 2024-09-13T17:05:54.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:54.628+0000 7f037224f640 1 -- 172.21.15.39:0/1189883203 wait complete. 2024-09-13T17:05:54.643 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:05:55.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.828+0000 7fa3be133640 1 Processor -- start 2024-09-13T17:05:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.828+0000 7fa3be133640 1 -- start start 2024-09-13T17:05:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3be133640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3b80a4e70 0x7fa3b8151460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3be133640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80c4530 0x7fa3b80c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3be133640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b8154100 con 0x7fa3b806d020 2024-09-13T17:05:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3be133640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b80a8c50 con 0x7fa3b80a4e70 2024-09-13T17:05:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3be133640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b81519a0 con 0x7fa3b80c4530 2024-09-13T17:05:55.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b7fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80c4530 0x7fa3b80c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:55.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b6ffd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3b80a4e70 0x7fa3b8151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:55.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b7fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80c4530 0x7fa3b80c4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:41448/0 (socket says 172.21.15.39:41448) 2024-09-13T17:05:55.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b7fff640 1 -- 172.21.15.39:0/1940115217 learned_addr learned my addr 172.21.15.39:0/1940115217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:55.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b77fe640 1 --1- 172.21.15.39:0/1940115217 >> v1:172.21.15.84:6789/0 conn(0x7fa3b806d020 0x7fa3b814db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:46444/0 (socket says 172.21.15.39:46444) 2024-09-13T17:05:55.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b67fc640 1 -- 172.21.15.39:0/1940115217 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260409095 0 0) 0x7fa3b8154100 con 0x7fa3b806d020 2024-09-13T17:05:55.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b67fc640 1 -- 172.21.15.39:0/1940115217 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3a0003530 con 0x7fa3b806d020 2024-09-13T17:05:55.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b7fff640 1 -- 172.21.15.39:0/1940115217 >> v2:172.21.15.84:3300/0 conn(0x7fa3b80a4e70 msgr2=0x7fa3b8151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:55.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b7fff640 1 --2- 172.21.15.39:0/1940115217 >> v2:172.21.15.84:3300/0 conn(0x7fa3b80a4e70 0x7fa3b8151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:55.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b7fff640 1 -- 172.21.15.39:0/1940115217 >> v1:172.21.15.84:6789/0 conn(0x7fa3b806d020 legacy=0x7fa3b814db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:55.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b7fff640 1 -- 172.21.15.39:0/1940115217 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b81543e0 con 0x7fa3b80c4530 2024-09-13T17:05:55.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b7fff640 1 --2- 172.21.15.39:0/1940115217 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80c4530 0x7fa3b80c4980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000d150 tx=0x7fa3b001eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30dfde32af6ac787 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:55.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b67fc640 1 -- 172.21.15.39:0/1940115217 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000b2d0 con 0x7fa3b80c4530 2024-09-13T17:05:55.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b67fc640 1 -- 172.21.15.39:0/1940115217 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000b430 con 0x7fa3b80c4530 2024-09-13T17:05:55.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.832+0000 7fa3b67fc640 1 -- 172.21.15.39:0/1940115217 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00a7a50 con 0x7fa3b80c4530 2024-09-13T17:05:55.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 -- 172.21.15.39:0/1940115217 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80c4530 msgr2=0x7fa3b80c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:55.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 --2- 172.21.15.39:0/1940115217 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80c4530 0x7fa3b80c4980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000d150 tx=0x7fa3b001eb70 comp rx=0 tx=0).stop 2024-09-13T17:05:55.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 -- 172.21.15.39:0/1940115217 shutdown_connections 2024-09-13T17:05:55.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 --2- 172.21.15.39:0/1940115217 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80c4530 0x7fa3b80c4980 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 --2- 172.21.15.39:0/1940115217 >> v2:172.21.15.84:3300/0 conn(0x7fa3b80a4e70 0x7fa3b8151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 -- 172.21.15.39:0/1940115217 >> 172.21.15.39:0/1940115217 conn(0x7fa3b8069e10 msgr2=0x7fa3b806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 -- 172.21.15.39:0/1940115217 shutdown_connections 2024-09-13T17:05:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 -- 172.21.15.39:0/1940115217 wait complete. 2024-09-13T17:05:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 Processor -- start 2024-09-13T17:05:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 -- start start 2024-09-13T17:05:55.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3b806d020 0x7fa3b81e8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:55.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80a4e70 0x7fa3b81440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:55.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3b80c4530 0x7fa3b80aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:55.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80a9ce0 con 0x7fa3b80c4530 2024-09-13T17:05:55.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b81e92b0 con 0x7fa3b806d020 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3be133640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80aaee0 con 0x7fa3b80a4e70 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3b80c4530 0x7fa3b80aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3b80c4530 0x7fa3b80aa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50512/0 (socket says 172.21.15.39:50512) 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b7fff640 1 -- 172.21.15.39:0/650026067 learned_addr learned my addr 172.21.15.39:0/650026067 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b6ffd640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80a4e70 0x7fa3b81440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b77fe640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3b806d020 0x7fa3b81e8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b7fff640 1 -- 172.21.15.39:0/650026067 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80a4e70 msgr2=0x7fa3b81440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b7fff640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80a4e70 0x7fa3b81440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b7fff640 1 -- 172.21.15.39:0/650026067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3b806d020 msgr2=0x7fa3b81e8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b7fff640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3b806d020 0x7fa3b81e8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b7fff640 1 -- 172.21.15.39:0/650026067 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b000bd40 con 0x7fa3b80c4530 2024-09-13T17:05:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.836+0000 7fa3b7fff640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3b80c4530 0x7fa3b80aa170 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa3b001d220 tx=0x7fa3b001b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:55.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa39ffff640 1 -- 172.21.15.39:0/650026067 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00a7a50 con 0x7fa3b80c4530 2024-09-13T17:05:55.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa39ffff640 1 -- 172.21.15.39:0/650026067 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b00a7bb0 con 0x7fa3b80c4530 2024-09-13T17:05:55.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa39ffff640 1 -- 172.21.15.39:0/650026067 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0004040 con 0x7fa3b80c4530 2024-09-13T17:05:55.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b80a99b0 con 0x7fa3b80c4530 2024-09-13T17:05:55.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b80ab460 con 0x7fa3b80c4530 2024-09-13T17:05:55.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b80a9e90 con 0x7fa3b80c4530 2024-09-13T17:05:55.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa39ffff640 1 -- 172.21.15.39:0/650026067 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3b00041a0 con 0x7fa3b80c4530 2024-09-13T17:05:55.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa39ffff640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa3a4066690 0x7fa3a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:55.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa3b77fe640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa3a4066690 0x7fa3a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:55.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa39ffff640 1 -- 172.21.15.39:0/650026067 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 8615+0+0 (secure 0 0 0) 0x7fa3b01221b0 con 0x7fa3b80c4530 2024-09-13T17:05:55.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.840+0000 7fa3b77fe640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa3a4066690 0x7fa3a4068b50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8000f70 tx=0x7fa3a8000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:55.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:55.848+0000 7fa39ffff640 1 -- 172.21.15.39:0/650026067 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa3b80a9e90 con 0x7fa3b80c4530 2024-09-13T17:05:56.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3b80a9e90 con 0x7fa3b80c4530 2024-09-13T17:05:56.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa39ffff640 1 -- 172.21.15.39:0/650026067 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+5185 (secure 0 0 0) 0x7fa3b80a9e90 con 0x7fa3b80c4530 2024-09-13T17:05:56.039 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:56.039 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":123,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:55.788297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:56.039 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 123 2024-09-13T17:05:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa3a4066690 msgr2=0x7fa3a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa3a4066690 0x7fa3a4068b50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8000f70 tx=0x7fa3a8000ee0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3b80c4530 msgr2=0x7fa3b80aa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3b80c4530 0x7fa3b80aa170 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa3b001d220 tx=0x7fa3b001b910 comp rx=0 tx=0).stop 2024-09-13T17:05:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 shutdown_connections 2024-09-13T17:05:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa3a4066690 0x7fa3a4068b50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3b80c4530 0x7fa3b80aa170 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa3b80a4e70 0x7fa3b81440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 --2- 172.21.15.39:0/650026067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3b806d020 0x7fa3b81e8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 >> 172.21.15.39:0/650026067 conn(0x7fa3b8069e10 msgr2=0x7fa3b81533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:56.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 shutdown_connections 2024-09-13T17:05:56.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.040+0000 7fa3be133640 1 -- 172.21.15.39:0/650026067 wait complete. 2024-09-13T17:05:56.056 INFO:tasks.mds_thrash.fs.[b]:mds.d down, removed from mdsmap 2024-09-13T17:05:56.057 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:05:56.057 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:56.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.168+0000 7f7a9bf0a640 1 Processor -- start 2024-09-13T17:05:56.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.168+0000 7f7a9bf0a640 1 -- start start 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a9bf0a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9414e0c0 0x7f7a940a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a9bf0a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7a940c4740 0x7f7a940c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a9bf0a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940c2780 con 0x7f7a940c1f50 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a9bf0a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a940c2a60 con 0x7f7a940c4740 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a9bf0a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940c2d40 con 0x7f7a9414e0c0 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a994a5640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7a940c1f50 0x7f7a9406cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:59652/0 (socket says 172.21.15.39:59652) 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a994a5640 1 -- 172.21.15.39:0/2830830326 learned_addr learned my addr 172.21.15.39:0/2830830326 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a99ca6640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9414e0c0 0x7f7a940a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226043812 0 0) 0x7f7a940c2780 con 0x7f7a940c1f50 2024-09-13T17:05:56.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a9a4a7640 1 --2- 172.21.15.39:0/2830830326 >> v2:172.21.15.84:3300/0 conn(0x7f7a940c4740 0x7f7a940c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a80003530 con 0x7f7a940c1f50 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1724229740 0 0) 0x7f7a80003530 con 0x7f7a940c1f50 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9414e0c0 msgr2=0x7f7a940a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9414e0c0 0x7f7a940a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 >> v2:172.21.15.84:3300/0 conn(0x7f7a940c4740 msgr2=0x7f7a940c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 --2- 172.21.15.39:0/2830830326 >> v2:172.21.15.84:3300/0 conn(0x7f7a940c4740 0x7f7a940c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a940c3020 con 0x7f7a940c1f50 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f7a90001680 con 0x7f7a940c1f50 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 >> v1:172.21.15.39:6789/0 conn(0x7f7a940c1f50 legacy=0x7f7a9406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9414e0c0 0x7f7a80003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a800047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a80004d10 0x7f7a80005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940c2a60 con 0x7f7a80003f00 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940c2d40 con 0x7f7a80004d10 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000dca0 con 0x7f7a9414e0c0 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a994a5640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a800047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a99ca6640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9414e0c0 0x7f7a80003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a9a4a7640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a80004d10 0x7f7a80005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a994a5640 1 -- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9414e0c0 msgr2=0x7f7a80003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a994a5640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9414e0c0 0x7f7a80003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a994a5640 1 -- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a80004d10 msgr2=0x7f7a80005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a994a5640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a80004d10 0x7f7a80005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a994a5640 1 -- 172.21.15.39:0/2830830326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80003530 con 0x7f7a80003f00 2024-09-13T17:05:56.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a9a4a7640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a80004d10 0x7f7a80005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a994a5640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a800047d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c290 tx=0x7f7a9001cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e20d5b882db9163 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900058e0 con 0x7f7a80003f00 2024-09-13T17:05:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a90005a40 con 0x7f7a80003f00 2024-09-13T17:05:56.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.172+0000 7f7a98ca4640 1 -- 172.21.15.39:0/2830830326 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90005df0 con 0x7f7a80003f00 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/2830830326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 msgr2=0x7f7a800047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 --2- 172.21.15.39:0/2830830326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a800047d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c290 tx=0x7f7a9001cc90 comp rx=0 tx=0).stop 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9a4a7640 1 -- 172.21.15.39:0/2830830326 reap_dead start 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/2830830326 shutdown_connections 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/2830830326 >> 172.21.15.39:0/2830830326 conn(0x7f7a94069e10 msgr2=0x7f7a9406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/2830830326 shutdown_connections 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/2830830326 wait complete. 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 Processor -- start 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- start start 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a94146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a941ecf00 0x7f7a941f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a9414fd20 0x7f7a941f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94146fa0 con 0x7f7a80003f00 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9414ee10 con 0x7f7a9414fd20 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9414e850 con 0x7f7a941ecf00 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a99ca6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a94146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a99ca6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a94146570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50522/0 (socket says 172.21.15.39:50522) 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a99ca6640 1 -- 172.21.15.39:0/526224561 learned_addr learned my addr 172.21.15.39:0/526224561 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9a4a7640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a9414fd20 0x7f7a941f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a994a5640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a941ecf00 0x7f7a941f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a99ca6640 1 -- 172.21.15.39:0/526224561 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a941ecf00 msgr2=0x7f7a941f0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a99ca6640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a941ecf00 0x7f7a941f0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a99ca6640 1 -- 172.21.15.39:0/526224561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a9414fd20 msgr2=0x7f7a941f0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a99ca6640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a9414fd20 0x7f7a941f0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a99ca6640 1 -- 172.21.15.39:0/526224561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9414eb30 con 0x7f7a80003f00 2024-09-13T17:05:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9a4a7640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a9414fd20 0x7f7a941f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a99ca6640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a94146570 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f7a840115e0 tx=0x7f7a84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a8a7fc640 1 -- 172.21.15.39:0/526224561 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400f8f0 con 0x7f7a80003f00 2024-09-13T17:05:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a8a7fc640 1 -- 172.21.15.39:0/526224561 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8400fa50 con 0x7f7a80003f00 2024-09-13T17:05:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a8a7fc640 1 -- 172.21.15.39:0/526224561 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400fdc0 con 0x7f7a80003f00 2024-09-13T17:05:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a941f0510 con 0x7f7a80003f00 2024-09-13T17:05:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a94146ab0 con 0x7f7a80003f00 2024-09-13T17:05:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.176+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a940c2780 con 0x7f7a80003f00 2024-09-13T17:05:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.180+0000 7f7a8a7fc640 1 -- 172.21.15.39:0/526224561 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a84015460 con 0x7f7a80003f00 2024-09-13T17:05:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.180+0000 7f7a8a7fc640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a78066690 0x7f7a78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.180+0000 7f7a8a7fc640 1 -- 172.21.15.39:0/526224561 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 8615+0+0 (secure 0 0 0) 0x7f7a840850d0 con 0x7f7a80003f00 2024-09-13T17:05:56.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.180+0000 7f7a994a5640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a78066690 0x7f7a78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.180+0000 7f7a994a5640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a78066690 0x7f7a78068b50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7a900aa000 tx=0x7f7a90001a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.184+0000 7f7a8a7fc640 1 -- 172.21.15.39:0/526224561 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7a84051920 con 0x7f7a80003f00 2024-09-13T17:05:56.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a94147fd0 con 0x7f7a80003f00 2024-09-13T17:05:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a8a7fc640 1 -- 172.21.15.39:0/526224561 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+5185 (secure 0 0 0) 0x7f7a84022090 con 0x7f7a80003f00 2024-09-13T17:05:56.371 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:56.371 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":123,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:55.788297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:56.372 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 123 2024-09-13T17:05:56.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a78066690 msgr2=0x7f7a78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a78066690 0x7f7a78068b50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7a900aa000 tx=0x7f7a90001a10 comp rx=0 tx=0).stop 2024-09-13T17:05:56.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 msgr2=0x7f7a94146570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a94146570 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f7a840115e0 tx=0x7f7a84011aa0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 shutdown_connections 2024-09-13T17:05:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a78066690 0x7f7a78068b50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a9414fd20 0x7f7a941f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a941ecf00 0x7f7a941f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 --2- 172.21.15.39:0/526224561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a80003f00 0x7f7a94146570 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 >> 172.21.15.39:0/526224561 conn(0x7f7a94069e10 msgr2=0x7f7a9414ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 shutdown_connections 2024-09-13T17:05:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.372+0000 7f7a9bf0a640 1 -- 172.21.15.39:0/526224561 wait complete. 2024-09-13T17:05:56.390 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 123, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 120, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6948": { "addr": "172.21.15.39:6835/4200926406", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 4200926406, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 4200926406, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6948, "incarnation": 117, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 12 } }, "last_failure": 0, "last_failure_osd_epoch": 47, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:05:13.123091+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6948 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 123, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_7050": { "addr": "172.21.15.39:6837/1392182763", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 1392182763, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 1392182763, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7050, "incarnation": 122, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 12 } }, "last_failure": 0, "last_failure_osd_epoch": 48, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:05:55.788297+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7050 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/440028459", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 440028459, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 440028459, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 96, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5561, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:05:56.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:56.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.772+0000 7fbea1ee1640 1 Processor -- start 2024-09-13T17:05:56.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.772+0000 7fbea1ee1640 1 -- start start 2024-09-13T17:05:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbe9c0c5330 0x7fbe9c0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbe9c06c090 0x7fbe9c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c1546c0 con 0x7fbe9c150f80 2024-09-13T17:05:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c0c19a0 con 0x7fbe9c0c5330 2024-09-13T17:05:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c0c49f0 con 0x7fbe9c06c090 2024-09-13T17:05:56.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbe9c0c5330 0x7fbe9c0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbe9c0c5330 0x7fbe9c0c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50534/0 (socket says 172.21.15.39:50534) 2024-09-13T17:05:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9affd640 1 -- 172.21.15.39:0/3232599353 learned_addr learned my addr 172.21.15.39:0/3232599353 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9bfff640 1 --2- 172.21.15.39:0/3232599353 >> v2:172.21.15.84:3300/0 conn(0x7fbe9c06c090 0x7fbe9c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9affd640 1 -- 172.21.15.39:0/3232599353 >> v2:172.21.15.84:3300/0 conn(0x7fbe9c06c090 msgr2=0x7fbe9c06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9affd640 1 --2- 172.21.15.39:0/3232599353 >> v2:172.21.15.84:3300/0 conn(0x7fbe9c06c090 0x7fbe9c06c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9affd640 1 -- 172.21.15.39:0/3232599353 >> v1:172.21.15.84:6789/0 conn(0x7fbe9c150f80 legacy=0x7fbe9c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9affd640 1 -- 172.21.15.39:0/3232599353 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c1543e0 con 0x7fbe9c0c5330 2024-09-13T17:05:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9affd640 1 --2- 172.21.15.39:0/3232599353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 0x7fbe9c0c1210 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009870 tx=0x7fbe9001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207d22486cb7656f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9a7fc640 1 -- 172.21.15.39:0/3232599353 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900ab620 con 0x7fbe9c0c5330 2024-09-13T17:05:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9a7fc640 1 -- 172.21.15.39:0/3232599353 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe900ab780 con 0x7fbe9c0c5330 2024-09-13T17:05:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbe9a7fc640 1 -- 172.21.15.39:0/3232599353 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900abb10 con 0x7fbe9c0c5330 2024-09-13T17:05:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 -- 172.21.15.39:0/3232599353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 msgr2=0x7fbe9c0c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 --2- 172.21.15.39:0/3232599353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 0x7fbe9c0c1210 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009870 tx=0x7fbe9001c420 comp rx=0 tx=0).stop 2024-09-13T17:05:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 -- 172.21.15.39:0/3232599353 shutdown_connections 2024-09-13T17:05:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 --2- 172.21.15.39:0/3232599353 >> v2:172.21.15.84:3300/0 conn(0x7fbe9c06c090 0x7fbe9c06c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 --2- 172.21.15.39:0/3232599353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 0x7fbe9c0c1210 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.776+0000 7fbea1ee1640 1 -- 172.21.15.39:0/3232599353 >> 172.21.15.39:0/3232599353 conn(0x7fbe9c069e10 msgr2=0x7fbe9c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:56.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 -- 172.21.15.39:0/3232599353 shutdown_connections 2024-09-13T17:05:56.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 -- 172.21.15.39:0/3232599353 wait complete. 2024-09-13T17:05:56.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 Processor -- start 2024-09-13T17:05:56.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 -- start start 2024-09-13T17:05:56.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbe9c06c090 0x7fbe9c0a8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 0x7fbe9c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c150f80 0x7fbe9c1ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c072600 con 0x7fbe9c0c5330 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c06fcc0 con 0x7fbe9c150f80 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c070150 con 0x7fbe9c06c090 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 0x7fbe9c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 0x7fbe9c152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50540/0 (socket says 172.21.15.39:50540) 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9affd640 1 -- 172.21.15.39:0/2399574651 learned_addr learned my addr 172.21.15.39:0/2399574651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9b7fe640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbe9c06c090 0x7fbe9c0a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9bfff640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c150f80 0x7fbe9c1ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9affd640 1 -- 172.21.15.39:0/2399574651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbe9c06c090 msgr2=0x7fbe9c0a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9affd640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbe9c06c090 0x7fbe9c0a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9affd640 1 -- 172.21.15.39:0/2399574651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c150f80 msgr2=0x7fbe9c1ecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9affd640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c150f80 0x7fbe9c1ecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9affd640 1 -- 172.21.15.39:0/2399574651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe900094d0 con 0x7fbe9c0c5330 2024-09-13T17:05:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe9affd640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 0x7fbe9c152c60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fbe900ad040 tx=0x7fbe9001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe83fff640 1 -- 172.21.15.39:0/2399574651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900be3f0 con 0x7fbe9c0c5330 2024-09-13T17:05:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe83fff640 1 -- 172.21.15.39:0/2399574651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe900be570 con 0x7fbe9c0c5330 2024-09-13T17:05:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbe83fff640 1 -- 172.21.15.39:0/2399574651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900be920 con 0x7fbe9c0c5330 2024-09-13T17:05:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c06f310 con 0x7fbe9c0c5330 2024-09-13T17:05:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.780+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c1536d0 con 0x7fbe9c0c5330 2024-09-13T17:05:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.784+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c06fe30 con 0x7fbe9c0c5330 2024-09-13T17:05:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.784+0000 7fbe83fff640 1 -- 172.21.15.39:0/2399574651 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbe900c7440 con 0x7fbe9c0c5330 2024-09-13T17:05:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.784+0000 7fbe83fff640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbe880666c0 0x7fbe88068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.784+0000 7fbe83fff640 1 -- 172.21.15.39:0/2399574651 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 8615+0+0 (secure 0 0 0) 0x7fbe9011f470 con 0x7fbe9c0c5330 2024-09-13T17:05:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.784+0000 7fbe9b7fe640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbe880666c0 0x7fbe88068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.784+0000 7fbe9b7fe640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbe880666c0 0x7fbe88068b80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c004e30 tx=0x7fbe8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:56.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.788+0000 7fbe83fff640 1 -- 172.21.15.39:0/2399574651 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbe900ebcc0 con 0x7fbe9c0c5330 2024-09-13T17:05:56.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.976+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe9c06fab0 con 0x7fbe9c0c5330 2024-09-13T17:05:56.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbe83fff640 1 -- 172.21.15.39:0/2399574651 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+5182 (secure 0 0 0) 0x7fbe900ef2e0 con 0x7fbe9c0c5330 2024-09-13T17:05:56.979 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:56.979 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":124,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:56.793545+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:56.979 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 124 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbe880666c0 msgr2=0x7fbe88068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbe880666c0 0x7fbe88068b80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c004e30 tx=0x7fbe8c009040 comp rx=0 tx=0).stop 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 msgr2=0x7fbe9c152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 0x7fbe9c152c60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fbe900ad040 tx=0x7fbe9001ac60 comp rx=0 tx=0).stop 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 shutdown_connections 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbe880666c0 0x7fbe88068b80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe9c150f80 0x7fbe9c1ecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe9c0c5330 0x7fbe9c152c60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 --2- 172.21.15.39:0/2399574651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbe9c06c090 0x7fbe9c0a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 >> 172.21.15.39:0/2399574651 conn(0x7fbe9c069e10 msgr2=0x7fbe9c0c2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:56.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 shutdown_connections 2024-09-13T17:05:56.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:56.980+0000 7fbea1ee1640 1 -- 172.21.15.39:0/2399574651 wait complete. 2024-09-13T17:05:56.996 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:05:58.391 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:58.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb611c3e640 1 Processor -- start 2024-09-13T17:05:58.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb611c3e640 1 -- start start 2024-09-13T17:05:58.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb611c3e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb60c071dd0 0x7fb60c154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:58.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb611c3e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c0c5a30 con 0x7fb60c0c7800 2024-09-13T17:05:58.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb611c3e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c1576a0 con 0x7fb60c151500 2024-09-13T17:05:58.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb611c3e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c157980 con 0x7fb60c071dd0 2024-09-13T17:05:58.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb60affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb60c071dd0 0x7fb60c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:58.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb60affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb60c071dd0 0x7fb60c154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50542/0 (socket says 172.21.15.39:50542) 2024-09-13T17:05:58.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb60affd640 1 -- 172.21.15.39:0/2627618908 learned_addr learned my addr 172.21.15.39:0/2627618908 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:58.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb60affd640 1 -- 172.21.15.39:0/2627618908 >> v1:172.21.15.84:6789/0 conn(0x7fb60c151500 legacy=0x7fb60c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:58.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb60affd640 1 -- 172.21.15.39:0/2627618908 >> v1:172.21.15.39:6789/0 conn(0x7fb60c0c7800 legacy=0x7fb60c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:58.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb60affd640 1 -- 172.21.15.39:0/2627618908 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c157c60 con 0x7fb60c071dd0 2024-09-13T17:05:58.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.528+0000 7fb60affd640 1 --2- 172.21.15.39:0/2627618908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c071dd0 0x7fb60c154070 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0098a0 tx=0x7fb5fc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62df854d75d4de91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:58.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60a7fc640 1 -- 172.21.15.39:0/2627618908 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0a6040 con 0x7fb60c071dd0 2024-09-13T17:05:58.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60a7fc640 1 -- 172.21.15.39:0/2627618908 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0aa600 con 0x7fb60c071dd0 2024-09-13T17:05:58.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60a7fc640 1 -- 172.21.15.39:0/2627618908 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0aa990 con 0x7fb60c071dd0 2024-09-13T17:05:58.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 -- 172.21.15.39:0/2627618908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c071dd0 msgr2=0x7fb60c154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:58.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 --2- 172.21.15.39:0/2627618908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c071dd0 0x7fb60c154070 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0098a0 tx=0x7fb5fc01c480 comp rx=0 tx=0).stop 2024-09-13T17:05:58.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 -- 172.21.15.39:0/2627618908 shutdown_connections 2024-09-13T17:05:58.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 --2- 172.21.15.39:0/2627618908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c071dd0 0x7fb60c154070 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:58.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 -- 172.21.15.39:0/2627618908 >> 172.21.15.39:0/2627618908 conn(0x7fb60c0ab530 msgr2=0x7fb60c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:58.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 -- 172.21.15.39:0/2627618908 shutdown_connections 2024-09-13T17:05:58.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 -- 172.21.15.39:0/2627618908 wait complete. 2024-09-13T17:05:58.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 Processor -- start 2024-09-13T17:05:58.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 -- start start 2024-09-13T17:05:58.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb60c071dd0 0x7fb60c0aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:58.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c0c7800 0x7fb60c147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:58.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60c151500 0x7fb60c1475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:58.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0c1610 con 0x7fb60c0c7800 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c1effe0 con 0x7fb60c151500 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb611c3e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c1f0510 con 0x7fb60c071dd0 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c0c7800 0x7fb60c147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c0c7800 0x7fb60c147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50556/0 (socket says 172.21.15.39:50556) 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60affd640 1 -- 172.21.15.39:0/2734697658 learned_addr learned my addr 172.21.15.39:0/2734697658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60b7fe640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb60c071dd0 0x7fb60c0aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60bfff640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60c151500 0x7fb60c1475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60affd640 1 -- 172.21.15.39:0/2734697658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb60c071dd0 msgr2=0x7fb60c0aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60affd640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb60c071dd0 0x7fb60c0aa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60affd640 1 -- 172.21.15.39:0/2734697658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60c151500 msgr2=0x7fb60c1475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60affd640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60c151500 0x7fb60c1475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60affd640 1 -- 172.21.15.39:0/2734697658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc0094d0 con 0x7fb60c0c7800 2024-09-13T17:05:58.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb60affd640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c0c7800 0x7fb60c147070 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc019090 tx=0x7fb5fc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:58.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.532+0000 7fb5f3fff640 1 -- 172.21.15.39:0/2734697658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0b3800 con 0x7fb60c0c7800 2024-09-13T17:05:58.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb5f3fff640 1 -- 172.21.15.39:0/2734697658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0b39a0 con 0x7fb60c0c7800 2024-09-13T17:05:58.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c148c70 con 0x7fb60c0c7800 2024-09-13T17:05:58.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb5f3fff640 1 -- 172.21.15.39:0/2734697658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0b3d50 con 0x7fb60c0c7800 2024-09-13T17:05:58.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c148620 con 0x7fb60c0c7800 2024-09-13T17:05:58.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c0c4280 con 0x7fb60c0c7800 2024-09-13T17:05:58.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb5f3fff640 1 -- 172.21.15.39:0/2734697658 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5fc0c6430 con 0x7fb60c0c7800 2024-09-13T17:05:58.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb5f3fff640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f80666c0 0x7fb5f8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:58.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb5f3fff640 1 -- 172.21.15.39:0/2734697658 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 8615+0+0 (secure 0 0 0) 0x7fb5fc11f9d0 con 0x7fb60c0c7800 2024-09-13T17:05:58.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb60b7fe640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f80666c0 0x7fb5f8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:58.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.536+0000 7fb60b7fe640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f80666c0 0x7fb5f8068b80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb6000015b0 tx=0x7fb600009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:58.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.540+0000 7fb5f3fff640 1 -- 172.21.15.39:0/2734697658 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb5fc0ec220 con 0x7fb60c0c7800 2024-09-13T17:05:58.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.744+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb60c071bb0 con 0x7fb60c0c7800 2024-09-13T17:05:58.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.744+0000 7fb5f3fff640 1 -- 172.21.15.39:0/2734697658 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+5182 (secure 0 0 0) 0x7fb5fc0ef840 con 0x7fb60c0c7800 2024-09-13T17:05:58.743 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:58.743 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:58.744 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 125 2024-09-13T17:05:58.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f80666c0 msgr2=0x7fb5f8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:58.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f80666c0 0x7fb5f8068b80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb6000015b0 tx=0x7fb600009040 comp rx=0 tx=0).stop 2024-09-13T17:05:58.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c0c7800 msgr2=0x7fb60c147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:58.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c0c7800 0x7fb60c147070 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc019090 tx=0x7fb5fc01acc0 comp rx=0 tx=0).stop 2024-09-13T17:05:58.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 shutdown_connections 2024-09-13T17:05:58.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5f80666c0 0x7fb5f8068b80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:58.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb60c151500 0x7fb60c1475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:58.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb60c0c7800 0x7fb60c147070 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:58.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 --2- 172.21.15.39:0/2734697658 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb60c071dd0 0x7fb60c0aa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:58.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 >> 172.21.15.39:0/2734697658 conn(0x7fb60c0ab530 msgr2=0x7fb60c15ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:58.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 shutdown_connections 2024-09-13T17:05:58.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:58.748+0000 7fb611c3e640 1 -- 172.21.15.39:0/2734697658 wait complete. 2024-09-13T17:05:58.763 INFO:tasks.mds_thrash.fs.[b]:mds.c has gained rank=0, replacing gid=6235 2024-09-13T17:05:58.763 INFO:tasks.mds_thrash.fs.[b]:waiting for 9 secs before reviving mds.d 2024-09-13T17:05:58.997 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a5c8e2640 1 Processor -- start 2024-09-13T17:05:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a5c8e2640 1 -- start start 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a5c8e2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0a580c20d0 0x7f0a580c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a5c8e2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5806d370 0x7f0a58157f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a5c8e2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a58157370 con 0x7f0a580c1790 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a5c8e2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a580c04a0 con 0x7f0a580c20d0 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a5c8e2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a58153df0 con 0x7f0a5806d370 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a55d9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0a580c1790 0x7f0a58157800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:59664/0 (socket says 172.21.15.39:59664) 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a55d9b640 1 -- 172.21.15.39:0/1169645177 learned_addr learned my addr 172.21.15.39:0/1169645177 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a5659c640 1 --2- 172.21.15.39:0/1169645177 >> v2:172.21.15.39:3300/0 conn(0x7f0a580c20d0 0x7f0a580c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.136+0000 7f0a56d9d640 1 --2- 172.21.15.39:0/1169645177 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5806d370 0x7f0a58157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5659c640 1 -- 172.21.15.39:0/1169645177 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5806d370 msgr2=0x7f0a58157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5659c640 1 --2- 172.21.15.39:0/1169645177 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5806d370 0x7f0a58157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5659c640 1 -- 172.21.15.39:0/1169645177 >> v1:172.21.15.39:6789/0 conn(0x7f0a580c1790 legacy=0x7f0a58157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5659c640 1 -- 172.21.15.39:0/1169645177 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5806c700 con 0x7f0a580c20d0 2024-09-13T17:05:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5659c640 1 --2- 172.21.15.39:0/1169645177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a580c20d0 0x7f0a580c24b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0a480097c0 tx=0x7f0a4801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e3a2bd50145b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5559a640 1 -- 172.21.15.39:0/1169645177 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a480ab670 con 0x7f0a580c20d0 2024-09-13T17:05:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5559a640 1 -- 172.21.15.39:0/1169645177 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a480ab7d0 con 0x7f0a580c20d0 2024-09-13T17:05:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5559a640 1 -- 172.21.15.39:0/1169645177 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a480abb60 con 0x7f0a580c20d0 2024-09-13T17:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/1169645177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a580c20d0 msgr2=0x7f0a580c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5c8e2640 1 --2- 172.21.15.39:0/1169645177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a580c20d0 0x7f0a580c24b0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0a480097c0 tx=0x7f0a4801c470 comp rx=0 tx=0).stop 2024-09-13T17:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/1169645177 shutdown_connections 2024-09-13T17:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5c8e2640 1 --2- 172.21.15.39:0/1169645177 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a5806d370 0x7f0a58157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5c8e2640 1 --2- 172.21.15.39:0/1169645177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a580c20d0 0x7f0a580c24b0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/1169645177 >> 172.21.15.39:0/1169645177 conn(0x7f0a58069e10 msgr2=0x7f0a5806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/1169645177 shutdown_connections 2024-09-13T17:05:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.140+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/1169645177 wait complete. 2024-09-13T17:05:59.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 Processor -- start 2024-09-13T17:05:59.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 -- start start 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5806d370 0x7f0a5815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a580c1790 0x7f0a58150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a580c20d0 0x7f0a581f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a58070510 con 0x7f0a5806d370 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5806fb70 con 0x7f0a580c1790 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5806fe50 con 0x7f0a580c20d0 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5806d370 0x7f0a5815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5806d370 0x7f0a5815cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50586/0 (socket says 172.21.15.39:50586) 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5659c640 1 -- 172.21.15.39:0/617014441 learned_addr learned my addr 172.21.15.39:0/617014441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:05:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a55d9b640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a580c1790 0x7f0a58150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a56d9d640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a580c20d0 0x7f0a581f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5659c640 1 -- 172.21.15.39:0/617014441 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a580c20d0 msgr2=0x7f0a581f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5659c640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a580c20d0 0x7f0a581f03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5659c640 1 -- 172.21.15.39:0/617014441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a580c1790 msgr2=0x7f0a58150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5659c640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a580c1790 0x7f0a58150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5659c640 1 -- 172.21.15.39:0/617014441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48009420 con 0x7f0a5806d370 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a55d9b640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a580c1790 0x7f0a58150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5659c640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5806d370 0x7f0a5815cca0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0a480ad040 tx=0x7f0a4801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a46ffd640 1 -- 172.21.15.39:0/617014441 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a480b5810 con 0x7f0a5806d370 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a46ffd640 1 -- 172.21.15.39:0/617014441 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a480b5990 con 0x7f0a5806d370 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a58070130 con 0x7f0a5806d370 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a46ffd640 1 -- 172.21.15.39:0/617014441 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a480b5d40 con 0x7f0a5806d370 2024-09-13T17:05:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a581505d0 con 0x7f0a5806d370 2024-09-13T17:05:59.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a580c24b0 con 0x7f0a5806d370 2024-09-13T17:05:59.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a46ffd640 1 -- 172.21.15.39:0/617014441 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a480c93f0 con 0x7f0a5806d370 2024-09-13T17:05:59.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a46ffd640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a380666c0 0x7f0a38068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:05:59.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a46ffd640 1 -- 172.21.15.39:0/617014441 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 8615+0+0 (secure 0 0 0) 0x7f0a4811fc00 con 0x7f0a5806d370 2024-09-13T17:05:59.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.144+0000 7f0a55d9b640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a380666c0 0x7f0a38068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:05:59.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.148+0000 7f0a55d9b640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a380666c0 0x7f0a38068b80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0a580a85e0 tx=0x7f0a4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:05:59.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.152+0000 7f0a46ffd640 1 -- 172.21.15.39:0/617014441 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0a48126080 con 0x7f0a5806d370 2024-09-13T17:05:59.321 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:05:59.324 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:05:59.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.344+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a5814e990 con 0x7f0a5806d370 2024-09-13T17:05:59.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.344+0000 7f0a46ffd640 1 -- 172.21.15.39:0/617014441 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+5182 (secure 0 0 0) 0x7f0a480c4020 con 0x7f0a5806d370 2024-09-13T17:05:59.346 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:05:59.346 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:05:13.123091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6948":{"gid":6948,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/4200926406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4200926406},{"type":"v1","addr":"172.21.15.39:6835","nonce":4200926406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:05:59.346 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 125 2024-09-13T17:05:59.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a380666c0 msgr2=0x7f0a38068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:59.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a380666c0 0x7f0a38068b80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0a580a85e0 tx=0x7f0a4c009040 comp rx=0 tx=0).stop 2024-09-13T17:05:59.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5806d370 msgr2=0x7f0a5815cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:05:59.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5806d370 0x7f0a5815cca0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0a480ad040 tx=0x7f0a4801acb0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 shutdown_connections 2024-09-13T17:05:59.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0a380666c0 0x7f0a38068b80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a580c1790 0x7f0a58150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0a580c20d0 0x7f0a581f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 --2- 172.21.15.39:0/617014441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a5806d370 0x7f0a5815cca0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:05:59.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 >> 172.21.15.39:0/617014441 conn(0x7f0a58069e10 msgr2=0x7f0a5815dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:05:59.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 shutdown_connections 2024-09-13T17:05:59.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:05:59.348+0000 7f0a5c8e2640 1 -- 172.21.15.39:0/617014441 wait complete. 2024-09-13T17:05:59.365 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:06:00.148 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-13T17:06:00.152+0000 7f7055257640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:06:01.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:06:01.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f16a6e640 1 Processor -- start 2024-09-13T17:06:01.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f16a6e640 1 -- start start 2024-09-13T17:06:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f16a6e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f100c7760 0x7f0f1015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f16a6e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f10158900 0x7f0f10158db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f16a6e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f10156070 con 0x7f0f10155c90 2024-09-13T17:06:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f16a6e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f101584b0 con 0x7f0f10158900 2024-09-13T17:06:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f16a6e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f101592f0 con 0x7f0f100c7760 2024-09-13T17:06:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f07fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f100c7760 0x7f0f1015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f1500b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0f10158900 0x7f0f10158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f07fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f100c7760 0x7f0f1015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:36898/0 (socket says 172.21.15.39:36898) 2024-09-13T17:06:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f07fff640 1 -- 172.21.15.39:0/292060924 learned_addr learned my addr 172.21.15.39:0/292060924 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:06:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f1500b640 1 -- 172.21.15.39:0/292060924 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f100c7760 msgr2=0x7f0f1015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:06:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f1500b640 1 --2- 172.21.15.39:0/292060924 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f100c7760 0x7f0f1015c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f1500b640 1 -- 172.21.15.39:0/292060924 >> v1:172.21.15.84:6789/0 conn(0x7f0f10155c90 legacy=0x7f0f100c41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f1500b640 1 -- 172.21.15.39:0/292060924 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100c6270 con 0x7f0f10158900 2024-09-13T17:06:01.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.504+0000 7f0f07fff640 1 --2- 172.21.15.39:0/292060924 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f100c7760 0x7f0f1015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:06:01.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f1500b640 1 --2- 172.21.15.39:0/292060924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f10158900 0x7f0f10158db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009d90 tx=0x7f0f0801c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61b47fd99b9aedd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:01.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f077fe640 1 -- 172.21.15.39:0/292060924 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f080a58e0 con 0x7f0f10158900 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f077fe640 1 -- 172.21.15.39:0/292060924 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f080a5a40 con 0x7f0f10158900 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f077fe640 1 -- 172.21.15.39:0/292060924 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f080a5dd0 con 0x7f0f10158900 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f16a6e640 1 -- 172.21.15.39:0/292060924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f10158900 msgr2=0x7f0f10158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f16a6e640 1 --2- 172.21.15.39:0/292060924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f10158900 0x7f0f10158db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009d90 tx=0x7f0f0801c820 comp rx=0 tx=0).stop 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f16a6e640 1 -- 172.21.15.39:0/292060924 shutdown_connections 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f16a6e640 1 --2- 172.21.15.39:0/292060924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f10158900 0x7f0f10158db0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f16a6e640 1 --2- 172.21.15.39:0/292060924 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f100c7760 0x7f0f1015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f16a6e640 1 -- 172.21.15.39:0/292060924 >> 172.21.15.39:0/292060924 conn(0x7f0f100ab430 msgr2=0x7f0f100ab840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f16a6e640 1 -- 172.21.15.39:0/292060924 shutdown_connections 2024-09-13T17:06:01.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.508+0000 7f0f16a6e640 1 -- 172.21.15.39:0/292060924 wait complete. 2024-09-13T17:06:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 Processor -- start 2024-09-13T17:06:01.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 -- start start 2024-09-13T17:06:01.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f100c7760 0x7f0f10154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f10155c90 0x7f0f10154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f10158900 0x7f0f100ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100af890 con 0x7f0f10155c90 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100bfea0 con 0x7f0f100c7760 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100acb80 con 0x7f0f10158900 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f07fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f10155c90 0x7f0f10154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f07fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f10155c90 0x7f0f10154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58366/0 (socket says 172.21.15.39:58366) 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f07fff640 1 -- 172.21.15.39:0/1769513191 learned_addr learned my addr 172.21.15.39:0/1769513191 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f1480a640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f100c7760 0x7f0f10154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f1500b640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f10158900 0x7f0f100ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f07fff640 1 -- 172.21.15.39:0/1769513191 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f10158900 msgr2=0x7f0f100ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f07fff640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f10158900 0x7f0f100ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f07fff640 1 -- 172.21.15.39:0/1769513191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f100c7760 msgr2=0x7f0f10154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f07fff640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f100c7760 0x7f0f10154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f07fff640 1 -- 172.21.15.39:0/1769513191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f000094d0 con 0x7f0f10155c90 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f1480a640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f100c7760 0x7f0f10154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f07fff640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f10155c90 0x7f0f10154e40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9b0 tx=0x7f0f0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f057fa640 1 -- 172.21.15.39:0/1769513191 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000c9a0 con 0x7f0f10155c90 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f08009a70 con 0x7f0f10155c90 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f057fa640 1 -- 172.21.15.39:0/1769513191 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0000cb00 con 0x7f0f10155c90 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f057fa640 1 -- 172.21.15.39:0/1769513191 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00013470 con 0x7f0f10155c90 2024-09-13T17:06:01.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f100af5b0 con 0x7f0f10155c90 2024-09-13T17:06:01.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f100c83b0 con 0x7f0f10155c90 2024-09-13T17:06:01.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f057fa640 1 -- 172.21.15.39:0/1769513191 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f000135d0 con 0x7f0f10155c90 2024-09-13T17:06:01.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f057fa640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0efc066690 0x7f0efc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f057fa640 1 -- 172.21.15.39:0/1769513191 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f0f00085000 con 0x7f0f10155c90 2024-09-13T17:06:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.512+0000 7f0f1480a640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0efc066690 0x7f0efc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.516+0000 7f0f1480a640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0efc066690 0x7f0efc068b50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f100c0210 tx=0x7f0f0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:01.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.520+0000 7f0f057fa640 1 -- 172.21.15.39:0/1769513191 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0f00051780 con 0x7f0f10155c90 2024-09-13T17:06:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.720+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f10158db0 con 0x7f0f10155c90 2024-09-13T17:06:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.720+0000 7f0f057fa640 1 -- 172.21.15.39:0/1769513191 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) v1 ==== 95+0+4429 (secure 0 0 0) 0x7f0f00054da0 con 0x7f0f10155c90 2024-09-13T17:06:01.721 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:06:01.721 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":127,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:00.833851+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:reconnect","state_seq":67,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:06:01.721 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 127 2024-09-13T17:06:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0efc066690 msgr2=0x7f0efc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0efc066690 0x7f0efc068b50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0f100c0210 tx=0x7f0f0c009040 comp rx=0 tx=0).stop 2024-09-13T17:06:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f10155c90 msgr2=0x7f0f10154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f10155c90 0x7f0f10154e40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9b0 tx=0x7f0f0000ee70 comp rx=0 tx=0).stop 2024-09-13T17:06:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 shutdown_connections 2024-09-13T17:06:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0efc066690 0x7f0efc068b50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0f10158900 0x7f0f100ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f10155c90 0x7f0f10154e40 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 --2- 172.21.15.39:0/1769513191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f100c7760 0x7f0f10154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 >> 172.21.15.39:0/1769513191 conn(0x7f0f100ab430 msgr2=0x7f0f101576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 shutdown_connections 2024-09-13T17:06:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.724+0000 7f0f16a6e640 1 -- 172.21.15.39:0/1769513191 wait complete. 2024-09-13T17:06:01.741 INFO:tasks.mds_thrash.fs.[a]:mds.a down, removed from mdsmap 2024-09-13T17:06:01.741 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:06:01.741 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:06:01.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.864+0000 7f88c1664640 1 Processor -- start 2024-09-13T17:06:01.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88c1664640 1 -- start start 2024-09-13T17:06:01.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88c1664640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f88bc150f80 0x7f88bc153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88c1664640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc1546c0 con 0x7f88bc153e70 2024-09-13T17:06:01.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88c1664640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc0c49f0 con 0x7f88bc0c5330 2024-09-13T17:06:01.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88c1664640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc0c19a0 con 0x7f88bc150f80 2024-09-13T17:06:01.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88baffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f88bc150f80 0x7f88bc153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88baffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f88bc150f80 0x7f88bc153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58380/0 (socket says 172.21.15.39:58380) 2024-09-13T17:06:01.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88baffd640 1 -- 172.21.15.39:0/3225195708 learned_addr learned my addr 172.21.15.39:0/3225195708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:06:01.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88baffd640 1 -- 172.21.15.39:0/3225195708 >> v1:172.21.15.84:6789/0 conn(0x7f88bc0c5330 legacy=0x7f88bc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88baffd640 1 -- 172.21.15.39:0/3225195708 >> v1:172.21.15.39:6789/0 conn(0x7f88bc153e70 legacy=0x7f88bc06c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.868+0000 7f88baffd640 1 -- 172.21.15.39:0/3225195708 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc1543e0 con 0x7f88bc150f80 2024-09-13T17:06:01.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88baffd640 1 --2- 172.21.15.39:0/3225195708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc150f80 0x7f88bc153930 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009e80 tx=0x7f88ac01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e677f91c035043b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:01.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88b9ffb640 1 -- 172.21.15.39:0/3225195708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0a5610 con 0x7f88bc150f80 2024-09-13T17:06:01.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88b9ffb640 1 -- 172.21.15.39:0/3225195708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac0a5770 con 0x7f88bc150f80 2024-09-13T17:06:01.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88b9ffb640 1 -- 172.21.15.39:0/3225195708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0a5b00 con 0x7f88bc150f80 2024-09-13T17:06:01.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88c1664640 1 -- 172.21.15.39:0/3225195708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc150f80 msgr2=0x7f88bc153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88c1664640 1 --2- 172.21.15.39:0/3225195708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc150f80 0x7f88bc153930 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009e80 tx=0x7f88ac01c8c0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88c1664640 1 -- 172.21.15.39:0/3225195708 shutdown_connections 2024-09-13T17:06:01.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88c1664640 1 --2- 172.21.15.39:0/3225195708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc150f80 0x7f88bc153930 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88c1664640 1 -- 172.21.15.39:0/3225195708 >> 172.21.15.39:0/3225195708 conn(0x7f88bc069e10 msgr2=0x7f88bc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:01.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88c1664640 1 -- 172.21.15.39:0/3225195708 shutdown_connections 2024-09-13T17:06:01.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88c1664640 1 -- 172.21.15.39:0/3225195708 wait complete. 2024-09-13T17:06:01.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88c1664640 1 Processor -- start 2024-09-13T17:06:01.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.872+0000 7f88c1664640 1 -- start start 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88c1664640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f88bc0c5330 0x7f88bc14d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88c1664640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88bc153e70 0x7f88bc06fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88c1664640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc0a7d30 0x7f88bc0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88c1664640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc071a90 con 0x7f88bc0a7d30 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88c1664640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc14cf00 con 0x7f88bc153e70 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88c1664640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc071f70 con 0x7f88bc0c5330 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc0a7d30 0x7f88bc0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc0a7d30 0x7f88bc0701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58388/0 (socket says 172.21.15.39:58388) 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88bb7fe640 1 -- 172.21.15.39:0/3857183530 learned_addr learned my addr 172.21.15.39:0/3857183530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:06:01.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88baffd640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f88bc0c5330 0x7f88bc14d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88ba7fc640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88bc153e70 0x7f88bc06fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88bb7fe640 1 -- 172.21.15.39:0/3857183530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f88bc0c5330 msgr2=0x7f88bc14d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88bb7fe640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f88bc0c5330 0x7f88bc14d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88bb7fe640 1 -- 172.21.15.39:0/3857183530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88bc153e70 msgr2=0x7f88bc06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88bb7fe640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88bc153e70 0x7f88bc06fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88bb7fe640 1 -- 172.21.15.39:0/3857183530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac009b30 con 0x7f88bc0a7d30 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88ba7fc640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88bc153e70 0x7f88bc06fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88bb7fe640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc0a7d30 0x7f88bc0701c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c340 tx=0x7f88b400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88b37fe640 1 -- 172.21.15.39:0/3857183530 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4005220 con 0x7f88bc0a7d30 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88b37fe640 1 -- 172.21.15.39:0/3857183530 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4005380 con 0x7f88bc0a7d30 2024-09-13T17:06:01.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc072210 con 0x7f88bc0a7d30 2024-09-13T17:06:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88b37fe640 1 -- 172.21.15.39:0/3857183530 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4007270 con 0x7f88bc0a7d30 2024-09-13T17:06:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc14cb90 con 0x7f88bc0a7d30 2024-09-13T17:06:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc153930 con 0x7f88bc0a7d30 2024-09-13T17:06:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88b37fe640 1 -- 172.21.15.39:0/3857183530 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f88b40073d0 con 0x7f88bc0a7d30 2024-09-13T17:06:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88b37fe640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f889c066690 0x7f889c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88b37fe640 1 -- 172.21.15.39:0/3857183530 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f88b4084190 con 0x7f88bc0a7d30 2024-09-13T17:06:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88baffd640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f889c066690 0x7f889c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.876+0000 7f88baffd640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f889c066690 0x7f889c068b50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0ae000 tx=0x7f88ac0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:01.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:01.880+0000 7f88b37fe640 1 -- 172.21.15.39:0/3857183530 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f88b4050990 con 0x7f88bc0a7d30 2024-09-13T17:06:02.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88bc0c3c00 con 0x7f88bc0a7d30 2024-09-13T17:06:02.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88b37fe640 1 -- 172.21.15.39:0/3857183530 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4426 (secure 0 0 0) 0x7f88b4053fb0 con 0x7f88bc0a7d30 2024-09-13T17:06:02.083 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:06:02.083 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":128,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:01.842553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:rejoin","state_seq":68,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:06:02.083 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 128 2024-09-13T17:06:02.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f889c066690 msgr2=0x7f889c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:02.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f889c066690 0x7f889c068b50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0ae000 tx=0x7f88ac0a7070 comp rx=0 tx=0).stop 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc0a7d30 msgr2=0x7f88bc0701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc0a7d30 0x7f88bc0701c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c340 tx=0x7f88b400c800 comp rx=0 tx=0).stop 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 shutdown_connections 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f889c066690 0x7f889c068b50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88bc153e70 0x7f88bc06fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88bc0a7d30 0x7f88bc0701c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 --2- 172.21.15.39:0/3857183530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f88bc0c5330 0x7f88bc14d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 >> 172.21.15.39:0/3857183530 conn(0x7f88bc069e10 msgr2=0x7f88bc152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 shutdown_connections 2024-09-13T17:06:02.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:02.084+0000 7f88c1664640 1 -- 172.21.15.39:0/3857183530 wait complete. 2024-09-13T17:06:02.103 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 128, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 128, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5561": { "addr": "172.21.15.84:6833/440028459", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 440028459, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 440028459, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5561, "incarnation": 126, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:rejoin", "state_seq": 68 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:06:01.842553+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5561 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 125, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_7050": { "addr": "172.21.15.39:6837/1392182763", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 1392182763, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 1392182763, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7050, "incarnation": 122, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 48, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:05:57.807169+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7050 } } } ], "standbys": [] } 2024-09-13T17:06:04.003 INFO:tasks.ceph.mds.b.smithi084.stderr:2024-09-13T17:06:04.002+0000 7f8a9b7ad640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:06:04.104 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:06:04.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c76f3640 1 Processor -- start 2024-09-13T17:06:04.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c76f3640 1 -- start start 2024-09-13T17:06:04.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c76f3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f25c014f1b0 0x7f25c0151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:04.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c76f3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 0x7f25c00714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:04.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c76f3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c014f590 con 0x7f25c00c7470 2024-09-13T17:06:04.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c76f3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c006f660 con 0x7f25c014f1b0 2024-09-13T17:06:04.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c76f3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c006eae0 con 0x7f25c006e5e0 2024-09-13T17:06:04.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c5c90640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f25c00c7470 0x7f25c00c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:59550/0 (socket says 172.21.15.39:59550) 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c5c90640 1 -- 172.21.15.39:0/231295225 learned_addr learned my addr 172.21.15.39:0/231295225 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c4c8e640 1 --2- 172.21.15.39:0/231295225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 0x7f25c00714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c548f640 1 --2- 172.21.15.39:0/231295225 >> v2:172.21.15.84:3300/0 conn(0x7f25c014f1b0 0x7f25c0151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25b7fff640 1 -- 172.21.15.39:0/231295225 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101799945 0 0) 0x7f25c014f590 con 0x7f25c00c7470 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25b7fff640 1 -- 172.21.15.39:0/231295225 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ac003530 con 0x7f25c00c7470 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c4c8e640 1 -- 172.21.15.39:0/231295225 >> v2:172.21.15.84:3300/0 conn(0x7f25c014f1b0 msgr2=0x7f25c0151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c4c8e640 1 --2- 172.21.15.39:0/231295225 >> v2:172.21.15.84:3300/0 conn(0x7f25c014f1b0 0x7f25c0151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25b7fff640 1 -- 172.21.15.39:0/231295225 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3352912473 0 0) 0x7f25ac003530 con 0x7f25c00c7470 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c4c8e640 1 -- 172.21.15.39:0/231295225 >> v1:172.21.15.39:6789/0 conn(0x7f25c00c7470 legacy=0x7f25c00c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:04.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.244+0000 7f25c4c8e640 1 -- 172.21.15.39:0/231295225 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c006edc0 con 0x7f25c006e5e0 2024-09-13T17:06:04.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c4c8e640 1 --2- 172.21.15.39:0/231295225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 0x7f25c00714e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25b00097c0 tx=0x7f25b001c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acff8e24479e410c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:04.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25b7fff640 1 -- 172.21.15.39:0/231295225 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8001a00 con 0x7f25c006e5e0 2024-09-13T17:06:04.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25b7fff640 1 -- 172.21.15.39:0/231295225 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b00a7050 con 0x7f25c006e5e0 2024-09-13T17:06:04.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25b7fff640 1 -- 172.21.15.39:0/231295225 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b00ab600 con 0x7f25c006e5e0 2024-09-13T17:06:04.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c76f3640 1 -- 172.21.15.39:0/231295225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 msgr2=0x7f25c00714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:04.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c76f3640 1 --2- 172.21.15.39:0/231295225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 0x7f25c00714e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25b00097c0 tx=0x7f25b001c390 comp rx=0 tx=0).stop 2024-09-13T17:06:04.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c76f3640 1 -- 172.21.15.39:0/231295225 shutdown_connections 2024-09-13T17:06:04.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c76f3640 1 --2- 172.21.15.39:0/231295225 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 0x7f25c00714e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:04.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c76f3640 1 --2- 172.21.15.39:0/231295225 >> v2:172.21.15.84:3300/0 conn(0x7f25c014f1b0 0x7f25c0151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:04.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c76f3640 1 -- 172.21.15.39:0/231295225 >> 172.21.15.39:0/231295225 conn(0x7f25c00a9720 msgr2=0x7f25c00a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:04.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c76f3640 1 -- 172.21.15.39:0/231295225 shutdown_connections 2024-09-13T17:06:04.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c76f3640 1 -- 172.21.15.39:0/231295225 wait complete. 2024-09-13T17:06:04.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.248+0000 7f25c76f3640 1 Processor -- start 2024-09-13T17:06:04.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 -- start start 2024-09-13T17:06:04.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 0x7f25c015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:04.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c00c7470 0x7f25c01f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:04.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25c014f1b0 0x7f25c01f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:04.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c00c34b0 con 0x7f25c014f1b0 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0146020 con 0x7f25c00c7470 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0146300 con 0x7f25c006e5e0 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c5c90640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25c014f1b0 0x7f25c01f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c5c90640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25c014f1b0 0x7f25c01f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58402/0 (socket says 172.21.15.39:58402) 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c5c90640 1 -- 172.21.15.39:0/2593837713 learned_addr learned my addr 172.21.15.39:0/2593837713 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c548f640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 0x7f25c015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c4c8e640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c00c7470 0x7f25c01f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c5c90640 1 -- 172.21.15.39:0/2593837713 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 msgr2=0x7f25c015a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c5c90640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 0x7f25c015a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c5c90640 1 -- 172.21.15.39:0/2593837713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c00c7470 msgr2=0x7f25c01f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c5c90640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c00c7470 0x7f25c01f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c5c90640 1 -- 172.21.15.39:0/2593837713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0009420 con 0x7f25c014f1b0 2024-09-13T17:06:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c5c90640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25c014f1b0 0x7f25c01f08c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f25b8001f80 tx=0x7f25b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:04.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25b5ffb640 1 -- 172.21.15.39:0/2593837713 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8012400 con 0x7f25c014f1b0 2024-09-13T17:06:04.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25b5ffb640 1 -- 172.21.15.39:0/2593837713 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b8012560 con 0x7f25c014f1b0 2024-09-13T17:06:04.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c01465e0 con 0x7f25c014f1b0 2024-09-13T17:06:04.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25b5ffb640 1 -- 172.21.15.39:0/2593837713 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8012910 con 0x7f25c014f1b0 2024-09-13T17:06:04.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c006bdd0 con 0x7f25c014f1b0 2024-09-13T17:06:04.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c00714e0 con 0x7f25c014f1b0 2024-09-13T17:06:04.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.252+0000 7f25b5ffb640 1 -- 172.21.15.39:0/2593837713 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f25b8012a70 con 0x7f25c014f1b0 2024-09-13T17:06:04.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.256+0000 7f25b5ffb640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25a4066690 0x7f25a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:04.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.256+0000 7f25b5ffb640 1 -- 172.21.15.39:0/2593837713 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f25b8085500 con 0x7f25c014f1b0 2024-09-13T17:06:04.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.256+0000 7f25c548f640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25a4066690 0x7f25a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:04.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.256+0000 7f25c548f640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25a4066690 0x7f25a4068b50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009ac0 tx=0x7f25bc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:04.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.260+0000 7f25b5ffb640 1 -- 172.21.15.39:0/2593837713 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f25b8051c10 con 0x7f25c014f1b0 2024-09-13T17:06:04.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.476+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25c01f0030 con 0x7f25c014f1b0 2024-09-13T17:06:04.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.476+0000 7f25b5ffb640 1 -- 172.21.15.39:0/2593837713 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+4426 (secure 0 0 0) 0x7f25b8055230 con 0x7f25c014f1b0 2024-09-13T17:06:04.475 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:06:04.475 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":129,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:06:04.476 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 129 2024-09-13T17:06:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25a4066690 msgr2=0x7f25a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25a4066690 0x7f25a4068b50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009ac0 tx=0x7f25bc009190 comp rx=0 tx=0).stop 2024-09-13T17:06:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25c014f1b0 msgr2=0x7f25c01f08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25c014f1b0 0x7f25c01f08c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f25b8001f80 tx=0x7f25b800a430 comp rx=0 tx=0).stop 2024-09-13T17:06:04.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 shutdown_connections 2024-09-13T17:06:04.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25a4066690 0x7f25a4068b50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:04.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25c00c7470 0x7f25c01f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:04.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25c014f1b0 0x7f25c01f08c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:04.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 --2- 172.21.15.39:0/2593837713 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25c006e5e0 0x7f25c015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:04.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 >> 172.21.15.39:0/2593837713 conn(0x7f25c00a9720 msgr2=0x7f25c0151480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:04.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 shutdown_connections 2024-09-13T17:06:04.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:04.480+0000 7f25c76f3640 1 -- 172.21.15.39:0/2593837713 wait complete. 2024-09-13T17:06:04.496 INFO:tasks.mds_thrash.fs.[a]:mds.b has gained rank=0, replacing gid=6948 2024-09-13T17:06:04.496 INFO:tasks.mds_thrash.fs.[a]:waiting for 7 secs before reviving mds.a 2024-09-13T17:06:07.764 INFO:tasks.mds_thrash.fs.[b]:reviving mds.d 2024-09-13T17:06:07.764 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T17:06:07.764 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T17:06:07.767 INFO:tasks.ceph.mds.d:Started 2024-09-13T17:06:07.767 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:06:07.851 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T17:06:07.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e4a749640 1 Processor -- start 2024-09-13T17:06:07.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e4a749640 1 -- start start 2024-09-13T17:06:07.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e4a749640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7e440c21d0 0x7f7e440aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e4a749640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4406aac0 0x7f7e44151c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e4a749640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e4414b850 0x7f7e4414bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e4a749640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e440ab340 con 0x7f7e440c21d0 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e4a749640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e440c3ae0 con 0x7f7e4414b850 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e4a749640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440abb30 con 0x7f7e4406aac0 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e43fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7e440c21d0 0x7f7e440aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e43fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7e440c21d0 0x7f7e440aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58418/0 (socket says 172.21.15.39:58418) 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e43fff640 1 -- 172.21.15.39:0/663389007 learned_addr learned my addr 172.21.15.39:0/663389007 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e48ce6640 1 --2- 172.21.15.39:0/663389007 >> v2:172.21.15.84:3300/0 conn(0x7f7e4414b850 0x7f7e4414bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:07.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.916+0000 7f7e437fe640 1 --2- 172.21.15.39:0/663389007 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4406aac0 0x7f7e44151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:07.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e48ce6640 1 -- 172.21.15.39:0/663389007 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4406aac0 msgr2=0x7f7e44151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:06:07.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e48ce6640 1 --2- 172.21.15.39:0/663389007 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4406aac0 0x7f7e44151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:07.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e48ce6640 1 -- 172.21.15.39:0/663389007 >> v2:172.21.15.39:3300/0 conn(0x7f7e440c21d0 msgr2=0x7f7e440aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:06:07.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e48ce6640 1 --2- 172.21.15.39:0/663389007 >> v2:172.21.15.39:3300/0 conn(0x7f7e440c21d0 0x7f7e440aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:07.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e48ce6640 1 -- 172.21.15.39:0/663389007 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e440abe10 con 0x7f7e4414b850 2024-09-13T17:06:07.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e43fff640 1 --2- 172.21.15.39:0/663389007 >> v2:172.21.15.39:3300/0 conn(0x7f7e440c21d0 0x7f7e440aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:06:07.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e437fe640 1 --2- 172.21.15.39:0/663389007 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4406aac0 0x7f7e44151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:06:07.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e48ce6640 1 --2- 172.21.15.39:0/663389007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e4414b850 0x7f7e4414bce0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e4406f130 tx=0x7f7e2c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caad8c77650f216e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:07.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e42ffd640 1 -- 172.21.15.39:0/663389007 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0ad680 con 0x7f7e4414b850 2024-09-13T17:06:07.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e42ffd640 1 -- 172.21.15.39:0/663389007 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c0ad7e0 con 0x7f7e4414b850 2024-09-13T17:06:07.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e42ffd640 1 -- 172.21.15.39:0/663389007 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0adb70 con 0x7f7e4414b850 2024-09-13T17:06:07.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e4a749640 1 -- 172.21.15.39:0/663389007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e4414b850 msgr2=0x7f7e4414bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:07.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.920+0000 7f7e4a749640 1 --2- 172.21.15.39:0/663389007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e4414b850 0x7f7e4414bce0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7e4406f130 tx=0x7f7e2c01c3c0 comp rx=0 tx=0).stop 2024-09-13T17:06:07.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 -- 172.21.15.39:0/663389007 shutdown_connections 2024-09-13T17:06:07.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 --2- 172.21.15.39:0/663389007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e4414b850 0x7f7e4414bce0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:07.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 --2- 172.21.15.39:0/663389007 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4406aac0 0x7f7e44151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:07.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 --2- 172.21.15.39:0/663389007 >> v2:172.21.15.39:3300/0 conn(0x7f7e440c21d0 0x7f7e440aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:07.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 -- 172.21.15.39:0/663389007 >> 172.21.15.39:0/663389007 conn(0x7f7e44069e10 msgr2=0x7f7e4406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:07.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 -- 172.21.15.39:0/663389007 shutdown_connections 2024-09-13T17:06:07.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 -- 172.21.15.39:0/663389007 wait complete. 2024-09-13T17:06:07.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 Processor -- start 2024-09-13T17:06:07.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 -- start start 2024-09-13T17:06:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4406aac0 0x7f7e440c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e440c21d0 0x7f7e440bbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:07.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4414b850 0x7f7e440bc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440bff30 con 0x7f7e4406aac0 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440bfb10 con 0x7f7e440c21d0 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e4a749640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440c0160 con 0x7f7e4414b850 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4406aac0 0x7f7e440c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4406aac0 0x7f7e440c11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58432/0 (socket says 172.21.15.39:58432) 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e43fff640 1 -- 172.21.15.39:0/1758413164 learned_addr learned my addr 172.21.15.39:0/1758413164 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e437fe640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e440c21d0 0x7f7e440bbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e48ce6640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4414b850 0x7f7e440bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e43fff640 1 -- 172.21.15.39:0/1758413164 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4414b850 msgr2=0x7f7e440bc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e43fff640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4414b850 0x7f7e440bc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e43fff640 1 -- 172.21.15.39:0/1758413164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e440c21d0 msgr2=0x7f7e440bbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e43fff640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e440c21d0 0x7f7e440bbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e43fff640 1 -- 172.21.15.39:0/1758413164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34009420 con 0x7f7e4406aac0 2024-09-13T17:06:07.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.924+0000 7f7e43fff640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4406aac0 0x7f7e440c11b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009e80 tx=0x7f7e3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:07.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e40ff9640 1 -- 172.21.15.39:0/1758413164 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c00ce60 con 0x7f7e4406aac0 2024-09-13T17:06:07.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e40ff9640 1 -- 172.21.15.39:0/1758413164 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c00f040 con 0x7f7e4406aac0 2024-09-13T17:06:07.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c009b30 con 0x7f7e4406aac0 2024-09-13T17:06:07.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e40ff9640 1 -- 172.21.15.39:0/1758413164 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c014680 con 0x7f7e4406aac0 2024-09-13T17:06:07.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c009420 con 0x7f7e4406aac0 2024-09-13T17:06:07.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e440c3560 con 0x7f7e4406aac0 2024-09-13T17:06:07.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e40ff9640 1 -- 172.21.15.39:0/1758413164 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e3c0147e0 con 0x7f7e4406aac0 2024-09-13T17:06:07.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e40ff9640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7e24066690 0x7f7e24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:07.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e40ff9640 1 -- 172.21.15.39:0/1758413164 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f7e3c085430 con 0x7f7e4406aac0 2024-09-13T17:06:07.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.928+0000 7f7e437fe640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7e24066690 0x7f7e24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:07.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.932+0000 7f7e437fe640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7e24066690 0x7f7e24068b50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7e44061100 tx=0x7f7e34009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:07.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:07.936+0000 7f7e40ff9640 1 -- 172.21.15.39:0/1758413164 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7e3c051ce0 con 0x7f7e4406aac0 2024-09-13T17:06:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.164+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e4415bd60 con 0x7f7e4406aac0 2024-09-13T17:06:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.164+0000 7f7e40ff9640 1 -- 172.21.15.39:0/1758413164 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+5183 (secure 0 0 0) 0x7f7e3c055300 con 0x7f7e4406aac0 2024-09-13T17:06:08.164 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:06:08.164 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":130,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:06:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 130 2024-09-13T17:06:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7e24066690 msgr2=0x7f7e24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7e24066690 0x7f7e24068b50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7e44061100 tx=0x7f7e34009190 comp rx=0 tx=0).stop 2024-09-13T17:06:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4406aac0 msgr2=0x7f7e440c11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4406aac0 0x7f7e440c11b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009e80 tx=0x7f7e3c00c490 comp rx=0 tx=0).stop 2024-09-13T17:06:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 shutdown_connections 2024-09-13T17:06:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7e24066690 0x7f7e24068b50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:08.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e440c21d0 0x7f7e440bbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:08.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7e4414b850 0x7f7e440bc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:08.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 --2- 172.21.15.39:0/1758413164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e4406aac0 0x7f7e440c11b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:08.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 >> 172.21.15.39:0/1758413164 conn(0x7f7e44069e10 msgr2=0x7f7e440c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:08.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 shutdown_connections 2024-09-13T17:06:08.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:08.168+0000 7f7e4a749640 1 -- 172.21.15.39:0/1758413164 wait complete. 2024-09-13T17:06:08.183 INFO:tasks.mds_thrash.fs.[b]:mds.d reported in up:standby state 2024-09-13T17:06:08.184 INFO:tasks.mds_thrash.fs.[b]:waiting for 53 secs before thrashing 2024-09-13T17:06:11.497 INFO:tasks.mds_thrash.fs.[a]:reviving mds.a 2024-09-13T17:06:11.498 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:06:11.498 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:06:11.501 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:06:11.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:06:11.589 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T17:06:11.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2ff2eb640 1 Processor -- start 2024-09-13T17:06:11.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2ff2eb640 1 -- start start 2024-09-13T17:06:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2ff2eb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2f8151500 0x7fd2f80c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2ff2eb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80c47c0 0x7fd2f8154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2ff2eb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f80c5a30 con 0x7fd2f8071dd0 2024-09-13T17:06:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2ff2eb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f81576a0 con 0x7fd2f8151500 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2ff2eb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8157980 con 0x7fd2f80c47c0 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fc886640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd2f8071dd0 0x7fd2f8154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40026/0 (socket says 172.21.15.39:40026) 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fc886640 1 -- 172.21.15.39:0/3405982711 learned_addr learned my addr 172.21.15.39:0/3405982711 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd087640 1 --2- 172.21.15.39:0/3405982711 >> v2:172.21.15.84:3300/0 conn(0x7fd2f8151500 0x7fd2f80c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd888640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80c47c0 0x7fd2f8154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299143280 0 0) 0x7fd2f80c5a30 con 0x7fd2f8071dd0 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2e0003530 con 0x7fd2f8071dd0 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 575226716 0 0) 0x7fd2e0003530 con 0x7fd2f8071dd0 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80c47c0 msgr2=0x7fd2f8154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80c47c0 0x7fd2f8154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 >> v2:172.21.15.84:3300/0 conn(0x7fd2f8151500 msgr2=0x7fd2f80c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:06:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 --2- 172.21.15.39:0/3405982711 >> v2:172.21.15.84:3300/0 conn(0x7fd2f8151500 0x7fd2f80c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f8157c60 con 0x7fd2f8071dd0 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fd2ec0016a0 con 0x7fd2f8071dd0 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 >> v1:172.21.15.39:6789/0 conn(0x7fd2f8071dd0 legacy=0x7fd2f8154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2e0003980 0x7fd2e0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e00047d0 0x7fd2e0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80c47c0 0x7fd2e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f81576a0 con 0x7fd2e0003980 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8157980 con 0x7fd2e00047d0 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e000b990 con 0x7fd2f80c47c0 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd087640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2e0003980 0x7fd2e0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd888640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80c47c0 0x7fd2e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:11.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd087640 1 -- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80c47c0 msgr2=0x7fd2e0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:06:11.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd087640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80c47c0 0x7fd2e0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd087640 1 -- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e00047d0 msgr2=0x7fd2e0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:06:11.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fc886640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e00047d0 0x7fd2e0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:11.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd087640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e00047d0 0x7fd2e0004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd087640 1 -- 172.21.15.39:0/3405982711 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e0003530 con 0x7fd2e0003980 2024-09-13T17:06:11.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.639+0000 7fd2fd888640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80c47c0 0x7fd2e0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:06:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2fd087640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2e0003980 0x7fd2e0003da0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800e170 tx=0x7fd2e801d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96900939ba5dbec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80a5ba0 con 0x7fd2e0003980 2024-09-13T17:06:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e800f040 con 0x7fd2e0003980 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2f7fff640 1 -- 172.21.15.39:0/3405982711 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80ad3f0 con 0x7fd2e0003980 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2ff2eb640 1 -- 172.21.15.39:0/3405982711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2e0003980 msgr2=0x7fd2e0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2ff2eb640 1 --2- 172.21.15.39:0/3405982711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2e0003980 0x7fd2e0003da0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800e170 tx=0x7fd2e801d6d0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2fd888640 1 -- 172.21.15.39:0/3405982711 reap_dead start 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2ff2eb640 1 -- 172.21.15.39:0/3405982711 shutdown_connections 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2ff2eb640 1 -- 172.21.15.39:0/3405982711 >> 172.21.15.39:0/3405982711 conn(0x7fd2f80ab530 msgr2=0x7fd2f80ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2ff2eb640 1 -- 172.21.15.39:0/3405982711 shutdown_connections 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.643+0000 7fd2ff2eb640 1 -- 172.21.15.39:0/3405982711 wait complete. 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 Processor -- start 2024-09-13T17:06:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 -- start start 2024-09-13T17:06:11.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e0003980 0x7fd2f81f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2f8153900 0x7fd2f8151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80aa7d0 0x7fd2f80aac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f81f0540 con 0x7fd2f8153900 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f81eff10 con 0x7fd2e0003980 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f8153110 con 0x7fd2f80aa7d0 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fc886640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2f8153900 0x7fd2f8151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fc886640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2f8153900 0x7fd2f8151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34208/0 (socket says 172.21.15.39:34208) 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fc886640 1 -- 172.21.15.39:0/959131680 learned_addr learned my addr 172.21.15.39:0/959131680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fd087640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e0003980 0x7fd2f81f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fd888640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80aa7d0 0x7fd2f80aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fc886640 1 -- 172.21.15.39:0/959131680 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80aa7d0 msgr2=0x7fd2f80aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:11.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fc886640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80aa7d0 0x7fd2f80aac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fc886640 1 -- 172.21.15.39:0/959131680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e0003980 msgr2=0x7fd2f81f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fc886640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e0003980 0x7fd2f81f09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fc886640 1 -- 172.21.15.39:0/959131680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f81533f0 con 0x7fd2f8153900 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2fc886640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2f8153900 0x7fd2f8151bd0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec007fd0 tx=0x7fd2ec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2f5ffb640 1 -- 172.21.15.39:0/959131680 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0072d0 con 0x7fd2f8153900 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2f5ffb640 1 -- 172.21.15.39:0/959131680 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec007430 con 0x7fd2f8153900 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 -- 172.21.15.39:0/959131680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f81f01d0 con 0x7fd2f8153900 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2f5ffb640 1 -- 172.21.15.39:0/959131680 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0047f0 con 0x7fd2f8153900 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2ff2eb640 1 -- 172.21.15.39:0/959131680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f8152d70 con 0x7fd2f8153900 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.647+0000 7fd2df7fe640 1 -- 172.21.15.39:0/959131680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d0005260 con 0x7fd2f8153900 2024-09-13T17:06:11.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.651+0000 7fd2f5ffb640 1 -- 172.21.15.39:0/959131680 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd2ec004950 con 0x7fd2f8153900 2024-09-13T17:06:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.651+0000 7fd2f5ffb640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd2e40666c0 0x7fd2e4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:06:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.651+0000 7fd2f5ffb640 1 -- 172.21.15.39:0/959131680 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7fd2ec084590 con 0x7fd2f8153900 2024-09-13T17:06:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.651+0000 7fd2fd087640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd2e40666c0 0x7fd2e4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:06:11.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.651+0000 7fd2fd087640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd2e40666c0 0x7fd2e4068b80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80ae000 tx=0x7fd2e801cc70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:06:11.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.655+0000 7fd2f5ffb640 1 -- 172.21.15.39:0/959131680 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd2ec050d00 con 0x7fd2f8153900 2024-09-13T17:06:11.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.887+0000 7fd2df7fe640 1 -- 172.21.15.39:0/959131680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2d0005a90 con 0x7fd2f8153900 2024-09-13T17:06:11.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.887+0000 7fd2f5ffb640 1 -- 172.21.15.39:0/959131680 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7fd2ec019090 con 0x7fd2f8153900 2024-09-13T17:06:11.889 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:06:11.889 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:06:11.889 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:06:11.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 -- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd2e40666c0 msgr2=0x7fd2e4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd2e40666c0 0x7fd2e4068b80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80ae000 tx=0x7fd2e801cc70 comp rx=0 tx=0).stop 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 -- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2f8153900 msgr2=0x7fd2f8151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2f8153900 0x7fd2f8151bd0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec007fd0 tx=0x7fd2ec00dc00 comp rx=0 tx=0).stop 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 -- 172.21.15.39:0/959131680 shutdown_connections 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd2e40666c0 0x7fd2e4068b80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd2f80aa7d0 0x7fd2f80aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2f8153900 0x7fd2f8151bd0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 --2- 172.21.15.39:0/959131680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e0003980 0x7fd2f81f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 -- 172.21.15.39:0/959131680 >> 172.21.15.39:0/959131680 conn(0x7fd2f80ab530 msgr2=0x7fd2f80c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 -- 172.21.15.39:0/959131680 shutdown_connections 2024-09-13T17:06:11.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:06:11.891+0000 7fd2df7fe640 1 -- 172.21.15.39:0/959131680 wait complete. 2024-09-13T17:06:11.909 INFO:tasks.mds_thrash.fs.[a]:mds.a reported in up:standby state 2024-09-13T17:06:11.909 INFO:tasks.mds_thrash.fs.[a]:waiting for 68 secs before thrashing 2024-09-13T17:06:29.382 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:06:29.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:06:59.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:06:59.439 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:07:01.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:01.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68cd7eb640 1 Processor -- start 2024-09-13T17:07:01.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68cd7eb640 1 -- start start 2024-09-13T17:07:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68cd7eb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68c814f1b0 0x7f68c8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68cd7eb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f68c806e5e0 0x7f68c80714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68cd7eb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c814f590 con 0x7f68c80c7470 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68cd7eb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c806f660 con 0x7f68c806e5e0 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68cd7eb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c806eae0 con 0x7f68c814f1b0 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68c67fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f68c806e5e0 0x7f68c80714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68c67fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f68c806e5e0 0x7f68c80714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45534/0 (socket says 172.21.15.39:45534) 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68c67fc640 1 -- 172.21.15.39:0/2849710678 learned_addr learned my addr 172.21.15.39:0/2849710678 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68c6ffd640 1 --2- 172.21.15.39:0/2849710678 >> v2:172.21.15.84:3300/0 conn(0x7f68c814f1b0 0x7f68c8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68c67fc640 1 -- 172.21.15.39:0/2849710678 >> v2:172.21.15.84:3300/0 conn(0x7f68c814f1b0 msgr2=0x7f68c8151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68c67fc640 1 --2- 172.21.15.39:0/2849710678 >> v2:172.21.15.84:3300/0 conn(0x7f68c814f1b0 0x7f68c8151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68c67fc640 1 -- 172.21.15.39:0/2849710678 >> v1:172.21.15.84:6789/0 conn(0x7f68c80c7470 legacy=0x7f68c80c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68c67fc640 1 -- 172.21.15.39:0/2849710678 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c806edc0 con 0x7f68c806e5e0 2024-09-13T17:07:01.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.330+0000 7f68c6ffd640 1 --2- 172.21.15.39:0/2849710678 >> v2:172.21.15.84:3300/0 conn(0x7f68c814f1b0 0x7f68c8151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:07:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68c67fc640 1 --2- 172.21.15.39:0/2849710678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 0x7f68c80714e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f68c8071140 tx=0x7f68bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84499d11e8a9aecb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:01.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68c5ffb640 1 -- 172.21.15.39:0/2849710678 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0aa650 con 0x7f68c806e5e0 2024-09-13T17:07:01.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68c5ffb640 1 -- 172.21.15.39:0/2849710678 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc0aa7b0 con 0x7f68c806e5e0 2024-09-13T17:07:01.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68c5ffb640 1 -- 172.21.15.39:0/2849710678 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0aab40 con 0x7f68c806e5e0 2024-09-13T17:07:01.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68cd7eb640 1 -- 172.21.15.39:0/2849710678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 msgr2=0x7f68c80714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:01.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68cd7eb640 1 --2- 172.21.15.39:0/2849710678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 0x7f68c80714e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f68c8071140 tx=0x7f68bc01c4f0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68cd7eb640 1 -- 172.21.15.39:0/2849710678 shutdown_connections 2024-09-13T17:07:01.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68cd7eb640 1 --2- 172.21.15.39:0/2849710678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 0x7f68c80714e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68cd7eb640 1 --2- 172.21.15.39:0/2849710678 >> v2:172.21.15.84:3300/0 conn(0x7f68c814f1b0 0x7f68c8151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68cd7eb640 1 -- 172.21.15.39:0/2849710678 >> 172.21.15.39:0/2849710678 conn(0x7f68c80a9720 msgr2=0x7f68c80a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:01.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68cd7eb640 1 -- 172.21.15.39:0/2849710678 shutdown_connections 2024-09-13T17:07:01.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.338+0000 7f68cd7eb640 1 -- 172.21.15.39:0/2849710678 wait complete. 2024-09-13T17:07:01.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68cd7eb640 1 Processor -- start 2024-09-13T17:07:01.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68cd7eb640 1 -- start start 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68cd7eb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 0x7f68c81ee590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68cd7eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c80c7470 0x7f68c80c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68cd7eb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f68c814f1b0 0x7f68c80c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68cd7eb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c81eead0 con 0x7f68c806e5e0 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68cd7eb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8154630 con 0x7f68c80c7470 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68cd7eb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8153b90 con 0x7f68c814f1b0 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 0x7f68c81ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68c6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 0x7f68c81ee590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45536/0 (socket says 172.21.15.39:45536) 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68c6ffd640 1 -- 172.21.15.39:0/1553874786 learned_addr learned my addr 172.21.15.39:0/1553874786 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68c6ffd640 1 -- 172.21.15.39:0/1553874786 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f68c814f1b0 msgr2=0x7f68c80c4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68c6ffd640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f68c814f1b0 0x7f68c80c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68c6ffd640 1 -- 172.21.15.39:0/1553874786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c80c7470 msgr2=0x7f68c80c4750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:07:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68c6ffd640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c80c7470 0x7f68c80c4750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68c6ffd640 1 -- 172.21.15.39:0/1553874786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc0094d0 con 0x7f68c806e5e0 2024-09-13T17:07:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.342+0000 7f68c6ffd640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 0x7f68c81ee590 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f68b800d830 tx=0x7f68b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:01.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.346+0000 7f68af7fe640 1 -- 172.21.15.39:0/1553874786 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b800b6c0 con 0x7f68c806e5e0 2024-09-13T17:07:01.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.346+0000 7f68af7fe640 1 -- 172.21.15.39:0/1553874786 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b800b840 con 0x7f68c806e5e0 2024-09-13T17:07:01.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.346+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c8154300 con 0x7f68c806e5e0 2024-09-13T17:07:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.346+0000 7f68af7fe640 1 -- 172.21.15.39:0/1553874786 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b800bbf0 con 0x7f68c806e5e0 2024-09-13T17:07:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.346+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c8150850 con 0x7f68c806e5e0 2024-09-13T17:07:01.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.350+0000 7f68af7fe640 1 -- 172.21.15.39:0/1553874786 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f68b800bd50 con 0x7f68c806e5e0 2024-09-13T17:07:01.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.350+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c80714e0 con 0x7f68c806e5e0 2024-09-13T17:07:01.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.350+0000 7f68af7fe640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68b4066710 0x7f68b4068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.350+0000 7f68af7fe640 1 -- 172.21.15.39:0/1553874786 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f68b8088170 con 0x7f68c806e5e0 2024-09-13T17:07:01.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.350+0000 7f68c67fc640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68b4066710 0x7f68b4068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.350+0000 7f68c67fc640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68b4066710 0x7f68b4068bd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0ae000 tx=0x7f68bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:01.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.358+0000 7f68af7fe640 1 -- 172.21.15.39:0/1553874786 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f68b8089060 con 0x7f68c806e5e0 2024-09-13T17:07:01.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.586+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68c80c2050 con 0x7f68c806e5e0 2024-09-13T17:07:01.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.586+0000 7f68af7fe640 1 -- 172.21.15.39:0/1553874786 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f68b80511b0 con 0x7f68c806e5e0 2024-09-13T17:07:01.586 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:01.586 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:01.587 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:07:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68b4066710 msgr2=0x7f68b4068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68b4066710 0x7f68b4068bd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0ae000 tx=0x7f68bc008040 comp rx=0 tx=0).stop 2024-09-13T17:07:01.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 msgr2=0x7f68c81ee590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 0x7f68c81ee590 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f68b800d830 tx=0x7f68b800dcf0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 shutdown_connections 2024-09-13T17:07:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68b4066710 0x7f68b4068bd0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c80c7470 0x7f68c80c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f68c814f1b0 0x7f68c80c4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 --2- 172.21.15.39:0/1553874786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c806e5e0 0x7f68c81ee590 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 >> 172.21.15.39:0/1553874786 conn(0x7f68c80a9720 msgr2=0x7f68c81512e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 shutdown_connections 2024-09-13T17:07:01.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.590+0000 7f68cd7eb640 1 -- 172.21.15.39:0/1553874786 wait complete. 2024-09-13T17:07:01.611 INFO:tasks.mds_thrash.fs.[b]:kill mds.c (rank=0) 2024-09-13T17:07:01.611 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:07:01.611 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:07:01.622 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:07:01.623 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e7b920640 1 Processor -- start 2024-09-13T17:07:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e7b920640 1 -- start start 2024-09-13T17:07:01.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e7b920640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5e7414bf30 0x7f5e7406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e7b920640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e7b920640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e74151bf0 con 0x7f5e7406b360 2024-09-13T17:07:01.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e7b920640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e74155660 con 0x7f5e7414bf30 2024-09-13T17:07:01.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e7b920640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74155940 con 0x7f5e74155160 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e79ebd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5e7406b360 0x7f5e74152630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51698/0 (socket says 172.21.15.39:51698) 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e79ebd640 1 -- 172.21.15.39:0/4057479572 learned_addr learned my addr 172.21.15.39:0/4057479572 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e78ebb640 1 --2- 172.21.15.39:0/4057479572 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e796bc640 1 --2- 172.21.15.39:0/4057479572 >> v2:172.21.15.84:3300/0 conn(0x7f5e7414bf30 0x7f5e7406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e6bfff640 1 -- 172.21.15.39:0/4057479572 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063042839 0 0) 0x7f5e74151bf0 con 0x7f5e7406b360 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e6bfff640 1 -- 172.21.15.39:0/4057479572 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e60003530 con 0x7f5e7406b360 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e78ebb640 1 -- 172.21.15.39:0/4057479572 >> v2:172.21.15.84:3300/0 conn(0x7f5e7414bf30 msgr2=0x7f5e7406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e78ebb640 1 --2- 172.21.15.39:0/4057479572 >> v2:172.21.15.84:3300/0 conn(0x7f5e7414bf30 0x7f5e7406ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e78ebb640 1 -- 172.21.15.39:0/4057479572 >> v1:172.21.15.39:6789/0 conn(0x7f5e7406b360 legacy=0x7f5e74152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e6bfff640 1 -- 172.21.15.39:0/4057479572 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635915489 0 0) 0x7f5e60003530 con 0x7f5e7406b360 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e78ebb640 1 -- 172.21.15.39:0/4057479572 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74155c20 con 0x7f5e74155160 2024-09-13T17:07:01.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.790+0000 7f5e796bc640 1 --2- 172.21.15.39:0/4057479572 >> v2:172.21.15.84:3300/0 conn(0x7f5e7414bf30 0x7f5e7406ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:07:01.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e78ebb640 1 --2- 172.21.15.39:0/4057479572 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741520f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e640097f0 tx=0x7f5e6401c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c232e70c2f0b52f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:01.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e6bfff640 1 -- 172.21.15.39:0/4057479572 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c001a00 con 0x7f5e74155160 2024-09-13T17:07:01.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e6bfff640 1 -- 172.21.15.39:0/4057479572 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e640a7050 con 0x7f5e74155160 2024-09-13T17:07:01.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e6bfff640 1 -- 172.21.15.39:0/4057479572 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e640ab600 con 0x7f5e74155160 2024-09-13T17:07:01.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 -- 172.21.15.39:0/4057479572 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 msgr2=0x7f5e741520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:01.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 --2- 172.21.15.39:0/4057479572 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741520f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e640097f0 tx=0x7f5e6401c390 comp rx=0 tx=0).stop 2024-09-13T17:07:01.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 -- 172.21.15.39:0/4057479572 shutdown_connections 2024-09-13T17:07:01.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 --2- 172.21.15.39:0/4057479572 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741520f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 --2- 172.21.15.39:0/4057479572 >> v2:172.21.15.84:3300/0 conn(0x7f5e7414bf30 0x7f5e7406ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 -- 172.21.15.39:0/4057479572 >> 172.21.15.39:0/4057479572 conn(0x7f5e740c32a0 msgr2=0x7f5e740c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:01.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 -- 172.21.15.39:0/4057479572 shutdown_connections 2024-09-13T17:07:01.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 -- 172.21.15.39:0/4057479572 wait complete. 2024-09-13T17:07:01.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 Processor -- start 2024-09-13T17:07:01.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 -- start start 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e7406b360 0x7f5e74143be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e7414bf30 0x7f5e7414fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74151b70 con 0x7f5e7414bf30 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74151170 con 0x7f5e7406b360 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e7b920640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e74145830 con 0x7f5e74155160 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e78ebb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e7414bf30 0x7f5e7414fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e78ebb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e7414bf30 0x7f5e7414fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45542/0 (socket says 172.21.15.39:45542) 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e78ebb640 1 -- 172.21.15.39:0/453420488 learned_addr learned my addr 172.21.15.39:0/453420488 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:01.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e796bc640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e7406b360 0x7f5e74143be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.794+0000 7f5e79ebd640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e78ebb640 1 -- 172.21.15.39:0/453420488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 msgr2=0x7f5e741ebcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e78ebb640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741ebcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e78ebb640 1 -- 172.21.15.39:0/453420488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e7406b360 msgr2=0x7f5e74143be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e78ebb640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e7406b360 0x7f5e74143be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e78ebb640 1 -- 172.21.15.39:0/453420488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70009420 con 0x7f5e7414bf30 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e79ebd640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e78ebb640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e7414bf30 0x7f5e7414fcc0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5e64000c10 tx=0x7f5e64019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e69ffb640 1 -- 172.21.15.39:0/453420488 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64003af0 con 0x7f5e7414bf30 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e69ffb640 1 -- 172.21.15.39:0/453420488 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e64003c50 con 0x7f5e7414bf30 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e64009420 con 0x7f5e7414bf30 2024-09-13T17:07:01.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e69ffb640 1 -- 172.21.15.39:0/453420488 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e640b5410 con 0x7f5e7414bf30 2024-09-13T17:07:01.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e74151870 con 0x7f5e7414bf30 2024-09-13T17:07:01.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7406ae20 con 0x7f5e7414bf30 2024-09-13T17:07:01.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e69ffb640 1 -- 172.21.15.39:0/453420488 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e640b2070 con 0x7f5e7414bf30 2024-09-13T17:07:01.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e69ffb640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e580666c0 0x7f5e58068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:01.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e69ffb640 1 -- 172.21.15.39:0/453420488 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f5e6411efb0 con 0x7f5e7414bf30 2024-09-13T17:07:01.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e796bc640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e580666c0 0x7f5e58068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:01.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.798+0000 7f5e796bc640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e580666c0 0x7f5e58068b80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009c50 tx=0x7f5e70009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:01.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:01.802+0000 7f5e69ffb640 1 -- 172.21.15.39:0/453420488 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e64150db0 con 0x7f5e7414bf30 2024-09-13T17:07:02.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.034+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e740c3080 con 0x7f5e7414bf30 2024-09-13T17:07:02.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.034+0000 7f5e69ffb640 1 -- 172.21.15.39:0/453420488 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f5e640c3070 con 0x7f5e7414bf30 2024-09-13T17:07:02.034 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:02.034 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:02.034 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:07:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e580666c0 msgr2=0x7f5e58068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e580666c0 0x7f5e58068b80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009c50 tx=0x7f5e70009190 comp rx=0 tx=0).stop 2024-09-13T17:07:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e7414bf30 msgr2=0x7f5e7414fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e7414bf30 0x7f5e7414fcc0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5e64000c10 tx=0x7f5e64019130 comp rx=0 tx=0).stop 2024-09-13T17:07:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 shutdown_connections 2024-09-13T17:07:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5e580666c0 0x7f5e58068b80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5e74155160 0x7f5e741ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e7414bf30 0x7f5e7414fcc0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 --2- 172.21.15.39:0/453420488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e7406b360 0x7f5e74143be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 >> 172.21.15.39:0/453420488 conn(0x7f5e740c32a0 msgr2=0x7f5e740c5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 shutdown_connections 2024-09-13T17:07:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:02.038+0000 7f5e7b920640 1 -- 172.21.15.39:0/453420488 wait complete. 2024-09-13T17:07:02.055 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:07:04.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7cda26640 1 Processor -- start 2024-09-13T17:07:04.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7cda26640 1 -- start start 2024-09-13T17:07:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7cda26640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd7c8151850 0x7fd7c80c5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7cda26640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7c8154890 0x7fd7c80bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7cda26640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c806ca30 con 0x7fd7c80a4ff0 2024-09-13T17:07:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7cda26640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c806cca0 con 0x7fd7c8154890 2024-09-13T17:07:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7cda26640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c806cf80 con 0x7fd7c8151850 2024-09-13T17:07:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7c67fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7c8154890 0x7fd7c80bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7c77fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd7c80a4ff0 0x7fd7c80c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51714/0 (socket says 172.21.15.39:51714) 2024-09-13T17:07:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7c77fe640 1 -- 172.21.15.39:0/1295176717 learned_addr learned my addr 172.21.15.39:0/1295176717 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:07:04.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7c6ffd640 1 --2- 172.21.15.39:0/1295176717 >> v2:172.21.15.84:3300/0 conn(0x7fd7c8151850 0x7fd7c80c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7c67fc640 1 -- 172.21.15.39:0/1295176717 >> v2:172.21.15.84:3300/0 conn(0x7fd7c8151850 msgr2=0x7fd7c80c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7c67fc640 1 --2- 172.21.15.39:0/1295176717 >> v2:172.21.15.84:3300/0 conn(0x7fd7c8151850 0x7fd7c80c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7c67fc640 1 -- 172.21.15.39:0/1295176717 >> v1:172.21.15.39:6789/0 conn(0x7fd7c80a4ff0 legacy=0x7fd7c80c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:04.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.206+0000 7fd7c67fc640 1 -- 172.21.15.39:0/1295176717 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c806d260 con 0x7fd7c8154890 2024-09-13T17:07:04.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7c67fc640 1 --2- 172.21.15.39:0/1295176717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c8154890 0x7fd7c80bec30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd7c806adb0 tx=0x7fd7b801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a192ea43e25af4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:04.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7c5ffb640 1 -- 172.21.15.39:0/1295176717 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80ab610 con 0x7fd7c8154890 2024-09-13T17:07:04.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7c5ffb640 1 -- 172.21.15.39:0/1295176717 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c806d990 con 0x7fd7c8154890 2024-09-13T17:07:04.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7c5ffb640 1 -- 172.21.15.39:0/1295176717 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80ab990 con 0x7fd7c8154890 2024-09-13T17:07:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7cda26640 1 -- 172.21.15.39:0/1295176717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c8154890 msgr2=0x7fd7c80bec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7cda26640 1 --2- 172.21.15.39:0/1295176717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c8154890 0x7fd7c80bec30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd7c806adb0 tx=0x7fd7b801c3f0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7cda26640 1 -- 172.21.15.39:0/1295176717 shutdown_connections 2024-09-13T17:07:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7cda26640 1 --2- 172.21.15.39:0/1295176717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c8154890 0x7fd7c80bec30 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7cda26640 1 --2- 172.21.15.39:0/1295176717 >> v2:172.21.15.84:3300/0 conn(0x7fd7c8151850 0x7fd7c80c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7cda26640 1 -- 172.21.15.39:0/1295176717 >> 172.21.15.39:0/1295176717 conn(0x7fd7c80a75b0 msgr2=0x7fd7c80a79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7cda26640 1 -- 172.21.15.39:0/1295176717 shutdown_connections 2024-09-13T17:07:04.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.210+0000 7fd7cda26640 1 -- 172.21.15.39:0/1295176717 wait complete. 2024-09-13T17:07:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 Processor -- start 2024-09-13T17:07:04.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 -- start start 2024-09-13T17:07:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7c80a4ff0 0x7fd7c80aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:04.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7c8151850 0x7fd7c80a7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c81f0660 0x7fd7c81495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81ec400 con 0x7fd7c81f0660 2024-09-13T17:07:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81ef970 con 0x7fd7c80a4ff0 2024-09-13T17:07:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c80ab340 con 0x7fd7c8151850 2024-09-13T17:07:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c81f0660 0x7fd7c81495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c81f0660 0x7fd7c81495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45556/0 (socket says 172.21.15.39:45556) 2024-09-13T17:07:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c77fe640 1 -- 172.21.15.39:0/4230147358 learned_addr learned my addr 172.21.15.39:0/4230147358 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:04.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c6ffd640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7c80a4ff0 0x7fd7c80aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:04.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c67fc640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7c8151850 0x7fd7c80a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c77fe640 1 -- 172.21.15.39:0/4230147358 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7c8151850 msgr2=0x7fd7c80a7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c77fe640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7c8151850 0x7fd7c80a7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c77fe640 1 -- 172.21.15.39:0/4230147358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7c80a4ff0 msgr2=0x7fd7c80aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c77fe640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7c80a4ff0 0x7fd7c80aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c77fe640 1 -- 172.21.15.39:0/4230147358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b8009420 con 0x7fd7c81f0660 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c6ffd640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7c80a4ff0 0x7fd7c80aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7c77fe640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c81f0660 0x7fd7c81495a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c3c0 tx=0x7fd7bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7affff640 1 -- 172.21.15.39:0/4230147358 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc005cf0 con 0x7fd7c81f0660 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7affff640 1 -- 172.21.15.39:0/4230147358 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc005e50 con 0x7fd7c81f0660 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c81ec5f0 con 0x7fd7c81f0660 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7affff640 1 -- 172.21.15.39:0/4230147358 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0072d0 con 0x7fd7c81f0660 2024-09-13T17:07:04.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.214+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c81efe60 con 0x7fd7c81f0660 2024-09-13T17:07:04.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.218+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c80c5280 con 0x7fd7c81f0660 2024-09-13T17:07:04.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.218+0000 7fd7affff640 1 -- 172.21.15.39:0/4230147358 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd7bc007430 con 0x7fd7c81f0660 2024-09-13T17:07:04.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.218+0000 7fd7affff640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7b40666c0 0x7fd7b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:04.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.218+0000 7fd7affff640 1 -- 172.21.15.39:0/4230147358 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7fd7bc084000 con 0x7fd7c81f0660 2024-09-13T17:07:04.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.218+0000 7fd7c6ffd640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7b40666c0 0x7fd7b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:04.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.218+0000 7fd7c6ffd640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7b40666c0 0x7fd7b4068b80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0003fd0 tx=0x7fd7c00039e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:04.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.222+0000 7fd7affff640 1 -- 172.21.15.39:0/4230147358 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd7bc0508b0 con 0x7fd7c81f0660 2024-09-13T17:07:04.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7c8066a90 con 0x7fd7c81f0660 2024-09-13T17:07:04.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7affff640 1 -- 172.21.15.39:0/4230147358 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7fd7bc017090 con 0x7fd7c81f0660 2024-09-13T17:07:04.469 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:04.470 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:04.471 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:07:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7b40666c0 msgr2=0x7fd7b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7b40666c0 0x7fd7b4068b80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0003fd0 tx=0x7fd7c00039e0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c81f0660 msgr2=0x7fd7c81495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c81f0660 0x7fd7c81495a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00c3c0 tx=0x7fd7bc00c880 comp rx=0 tx=0).stop 2024-09-13T17:07:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 shutdown_connections 2024-09-13T17:07:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd7b40666c0 0x7fd7b4068b80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7c81f0660 0x7fd7c81495a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd7c8151850 0x7fd7c80a7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 --2- 172.21.15.39:0/4230147358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd7c80a4ff0 0x7fd7c80aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:04.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 >> 172.21.15.39:0/4230147358 conn(0x7fd7c80a75b0 msgr2=0x7fd7c8152ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:04.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 shutdown_connections 2024-09-13T17:07:04.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:04.470+0000 7fd7cda26640 1 -- 172.21.15.39:0/4230147358 wait complete. 2024-09-13T17:07:04.488 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:07:06.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:06.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.618+0000 7f6c70f1b640 1 Processor -- start 2024-09-13T17:07:06.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.618+0000 7f6c70f1b640 1 -- start start 2024-09-13T17:07:06.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c70f1b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c0c1810 0x7f6c6c06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:06.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c70f1b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c6c1576e0 0x7f6c6c157af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:06.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c70f1b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0c15d0 con 0x7f6c6c06b080 2024-09-13T17:07:06.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c70f1b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6c06d0b0 con 0x7f6c6c1576e0 2024-09-13T17:07:06.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c70f1b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0c1f00 con 0x7f6c6c0c1810 2024-09-13T17:07:06.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c6ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c6c1576e0 0x7f6c6c157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:06.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c69d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c0c1810 0x7f6c6c06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:06.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c6ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c6c1576e0 0x7f6c6c157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54250/0 (socket says 172.21.15.39:54250) 2024-09-13T17:07:06.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c6ad9d640 1 -- 172.21.15.39:0/160791317 learned_addr learned my addr 172.21.15.39:0/160791317 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:06.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c6a59c640 1 --1- 172.21.15.39:0/160791317 >> v1:172.21.15.84:6789/0 conn(0x7f6c6c06b080 0x7f6c6c151390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:54020/0 (socket says 172.21.15.39:54020) 2024-09-13T17:07:06.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c6ad9d640 1 -- 172.21.15.39:0/160791317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c0c1810 msgr2=0x7f6c6c06e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:06.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c6ad9d640 1 --2- 172.21.15.39:0/160791317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c0c1810 0x7f6c6c06e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:06.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c6ad9d640 1 -- 172.21.15.39:0/160791317 >> v1:172.21.15.84:6789/0 conn(0x7f6c6c06b080 legacy=0x7f6c6c151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:06.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c6ad9d640 1 -- 172.21.15.39:0/160791317 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0c21a0 con 0x7f6c6c1576e0 2024-09-13T17:07:06.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c69d9b640 1 --2- 172.21.15.39:0/160791317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c0c1810 0x7f6c6c06e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:07:06.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.622+0000 7f6c6ad9d640 1 --2- 172.21.15.39:0/160791317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c1576e0 0x7f6c6c157af0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009e50 tx=0x7f6c6401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=179049e62586a9f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:06.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c6959a640 1 -- 172.21.15.39:0/160791317 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640a5740 con 0x7f6c6c1576e0 2024-09-13T17:07:06.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c6959a640 1 -- 172.21.15.39:0/160791317 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c640a58a0 con 0x7f6c6c1576e0 2024-09-13T17:07:06.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c6959a640 1 -- 172.21.15.39:0/160791317 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640a5c30 con 0x7f6c6c1576e0 2024-09-13T17:07:06.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 -- 172.21.15.39:0/160791317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c1576e0 msgr2=0x7f6c6c157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:06.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 --2- 172.21.15.39:0/160791317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c1576e0 0x7f6c6c157af0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009e50 tx=0x7f6c6401c970 comp rx=0 tx=0).stop 2024-09-13T17:07:06.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 -- 172.21.15.39:0/160791317 shutdown_connections 2024-09-13T17:07:06.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 --2- 172.21.15.39:0/160791317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c1576e0 0x7f6c6c157af0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:06.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 --2- 172.21.15.39:0/160791317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c0c1810 0x7f6c6c06e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:06.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 -- 172.21.15.39:0/160791317 >> 172.21.15.39:0/160791317 conn(0x7f6c6c0c32a0 msgr2=0x7f6c6c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:06.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 -- 172.21.15.39:0/160791317 shutdown_connections 2024-09-13T17:07:06.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 -- 172.21.15.39:0/160791317 wait complete. 2024-09-13T17:07:06.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 Processor -- start 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 -- start start 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c06b080 0x7f6c6c070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c6c0c1810 0x7f6c6c1f0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c1576e0 0x7f6c6c1f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c14dbc0 con 0x7f6c6c0c1810 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c1efc00 con 0x7f6c6c06b080 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c70f1b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c14de00 con 0x7f6c6c1576e0 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c69d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c6c0c1810 0x7f6c6c1f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c69d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c6c0c1810 0x7f6c6c1f0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45566/0 (socket says 172.21.15.39:45566) 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c69d9b640 1 -- 172.21.15.39:0/1237573747 learned_addr learned my addr 172.21.15.39:0/1237573747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c6a59c640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c06b080 0x7f6c6c070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c6ad9d640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c1576e0 0x7f6c6c1f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c69d9b640 1 -- 172.21.15.39:0/1237573747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c1576e0 msgr2=0x7f6c6c1f05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c69d9b640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c1576e0 0x7f6c6c1f05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c69d9b640 1 -- 172.21.15.39:0/1237573747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c06b080 msgr2=0x7f6c6c070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c69d9b640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c06b080 0x7f6c6c070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c69d9b640 1 -- 172.21.15.39:0/1237573747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c60009420 con 0x7f6c6c0c1810 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c6a59c640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c06b080 0x7f6c6c070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.626+0000 7f6c69d9b640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c6c0c1810 0x7f6c6c1f0090 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000ebe0 tx=0x7f6c6000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c5affd640 1 -- 172.21.15.39:0/1237573747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000eee0 con 0x7f6c6c0c1810 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c5affd640 1 -- 172.21.15.39:0/1237573747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6000cc30 con 0x7f6c6c0c1810 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c64009b30 con 0x7f6c6c0c1810 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c5affd640 1 -- 172.21.15.39:0/1237573747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60013470 con 0x7f6c6c0c1810 2024-09-13T17:07:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c156170 con 0x7f6c6c0c1810 2024-09-13T17:07:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c0619c0 con 0x7f6c6c0c1810 2024-09-13T17:07:06.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c5affd640 1 -- 172.21.15.39:0/1237573747 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c600135d0 con 0x7f6c6c0c1810 2024-09-13T17:07:06.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c5affd640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c4c066690 0x7f6c4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:06.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c5affd640 1 -- 172.21.15.39:0/1237573747 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f6c60085040 con 0x7f6c6c0c1810 2024-09-13T17:07:06.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c6a59c640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c4c066690 0x7f6c4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:06.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.630+0000 7f6c6a59c640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c4c066690 0x7f6c4c068b50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c1522b0 tx=0x7f6c5c007d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:06.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.634+0000 7f6c5affd640 1 -- 172.21.15.39:0/1237573747 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6c60051840 con 0x7f6c6c0c1810 2024-09-13T17:07:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c6c1efd90 con 0x7f6c6c0c1810 2024-09-13T17:07:06.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c5affd640 1 -- 172.21.15.39:0/1237573747 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f6c60017090 con 0x7f6c6c0c1810 2024-09-13T17:07:06.863 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:06.863 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:06.864 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:07:06.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c4c066690 msgr2=0x7f6c4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c4c066690 0x7f6c4c068b50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c1522b0 tx=0x7f6c5c007d90 comp rx=0 tx=0).stop 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c6c0c1810 msgr2=0x7f6c6c1f0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c6c0c1810 0x7f6c6c1f0090 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000ebe0 tx=0x7f6c6000c520 comp rx=0 tx=0).stop 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 shutdown_connections 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c4c066690 0x7f6c4c068b50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c6c1576e0 0x7f6c6c1f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c6c0c1810 0x7f6c6c1f0090 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 --2- 172.21.15.39:0/1237573747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c6c06b080 0x7f6c6c070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 >> 172.21.15.39:0/1237573747 conn(0x7f6c6c0c32a0 msgr2=0x7f6c6c0c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 shutdown_connections 2024-09-13T17:07:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:06.866+0000 7f6c70f1b640 1 -- 172.21.15.39:0/1237573747 wait complete. 2024-09-13T17:07:06.883 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:07:08.884 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:09.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e753b640 1 Processor -- start 2024-09-13T17:07:09.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e753b640 1 -- start start 2024-09-13T17:07:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e753b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd4e01556d0 0x7fd4e00c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e753b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e00c7610 0x7fd4e0151b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e753b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e006feb0 con 0x7fd4e00a72b0 2024-09-13T17:07:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e753b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e00aae50 con 0x7fd4e01556d0 2024-09-13T17:07:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e753b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0155ab0 con 0x7fd4e00c7610 2024-09-13T17:07:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e52d7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd4e01556d0 0x7fd4e00c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e4ad6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e00c7610 0x7fd4e0151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e52d7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd4e01556d0 0x7fd4e00c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54256/0 (socket says 172.21.15.39:54256) 2024-09-13T17:07:09.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e4ad6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e00c7610 0x7fd4e0151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:34840/0 (socket says 172.21.15.39:34840) 2024-09-13T17:07:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e5ad8640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd4e00a72b0 0x7fd4e0071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:54026/0 (socket says 172.21.15.39:54026) 2024-09-13T17:07:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e52d7640 1 -- 172.21.15.39:0/2040057669 learned_addr learned my addr 172.21.15.39:0/2040057669 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e4ad6640 1 -- 172.21.15.39:0/2040057669 >> v2:172.21.15.84:3300/0 conn(0x7fd4e01556d0 msgr2=0x7fd4e00c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e4ad6640 1 --2- 172.21.15.39:0/2040057669 >> v2:172.21.15.84:3300/0 conn(0x7fd4e01556d0 0x7fd4e00c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e4ad6640 1 -- 172.21.15.39:0/2040057669 >> v1:172.21.15.84:6789/0 conn(0x7fd4e00a72b0 legacy=0x7fd4e0071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:09.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e4ad6640 1 -- 172.21.15.39:0/2040057669 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e00c3c20 con 0x7fd4e00c7610 2024-09-13T17:07:09.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.030+0000 7fd4e52d7640 1 --2- 172.21.15.39:0/2040057669 >> v2:172.21.15.84:3300/0 conn(0x7fd4e01556d0 0x7fd4e00c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:07:09.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4e4ad6640 1 --2- 172.21.15.39:0/2040057669 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e00c7610 0x7fd4e0151b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00097c0 tx=0x7fd4d001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b922e6a26040359 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:09.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4d7fff640 1 -- 172.21.15.39:0/2040057669 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00ab600 con 0x7fd4e00c7610 2024-09-13T17:07:09.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4d7fff640 1 -- 172.21.15.39:0/2040057669 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d00a7050 con 0x7fd4e00c7610 2024-09-13T17:07:09.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4d7fff640 1 -- 172.21.15.39:0/2040057669 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00ab980 con 0x7fd4e00c7610 2024-09-13T17:07:09.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4e753b640 1 -- 172.21.15.39:0/2040057669 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e00c7610 msgr2=0x7fd4e0151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:09.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4e753b640 1 --2- 172.21.15.39:0/2040057669 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e00c7610 0x7fd4e0151b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00097c0 tx=0x7fd4d001c380 comp rx=0 tx=0).stop 2024-09-13T17:07:09.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4e753b640 1 -- 172.21.15.39:0/2040057669 shutdown_connections 2024-09-13T17:07:09.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4e753b640 1 --2- 172.21.15.39:0/2040057669 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e00c7610 0x7fd4e0151b80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:09.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4e753b640 1 --2- 172.21.15.39:0/2040057669 >> v2:172.21.15.84:3300/0 conn(0x7fd4e01556d0 0x7fd4e00c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:09.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4e753b640 1 -- 172.21.15.39:0/2040057669 >> 172.21.15.39:0/2040057669 conn(0x7fd4e00a97b0 msgr2=0x7fd4e00a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:09.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.034+0000 7fd4e753b640 1 -- 172.21.15.39:0/2040057669 shutdown_connections 2024-09-13T17:07:09.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e753b640 1 -- 172.21.15.39:0/2040057669 wait complete. 2024-09-13T17:07:09.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e753b640 1 Processor -- start 2024-09-13T17:07:09.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e753b640 1 -- start start 2024-09-13T17:07:09.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e753b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4e00a72b0 0x7fd4e0149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:09.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e753b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e01556d0 0x7fd4e006d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:09.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e753b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4e0148c90 0x7fd4e0147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:09.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e753b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01ee6f0 con 0x7fd4e00a72b0 2024-09-13T17:07:09.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e753b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01ee9d0 con 0x7fd4e0148c90 2024-09-13T17:07:09.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e753b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e014c000 con 0x7fd4e01556d0 2024-09-13T17:07:09.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e52d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4e00a72b0 0x7fd4e0149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:09.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e52d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4e00a72b0 0x7fd4e0149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45582/0 (socket says 172.21.15.39:45582) 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e52d7640 1 -- 172.21.15.39:0/4229544015 learned_addr learned my addr 172.21.15.39:0/4229544015 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e4ad6640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e01556d0 0x7fd4e006d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e5ad8640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4e0148c90 0x7fd4e0147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e52d7640 1 -- 172.21.15.39:0/4229544015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e01556d0 msgr2=0x7fd4e006d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e52d7640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e01556d0 0x7fd4e006d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e52d7640 1 -- 172.21.15.39:0/4229544015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4e0148c90 msgr2=0x7fd4e0147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e52d7640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4e0148c90 0x7fd4e0147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e52d7640 1 -- 172.21.15.39:0/4229544015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0094d0 con 0x7fd4e00a72b0 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4e52d7640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4e00a72b0 0x7fd4e0149a90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00e9d0 tx=0x7fd4dc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.038+0000 7fd4d5ffb640 1 -- 172.21.15.39:0/4229544015 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00ca60 con 0x7fd4e00a72b0 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d0009420 con 0x7fd4e00a72b0 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4d5ffb640 1 -- 172.21.15.39:0/4229544015 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc00cbc0 con 0x7fd4e00a72b0 2024-09-13T17:07:09.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4d5ffb640 1 -- 172.21.15.39:0/4229544015 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc013470 con 0x7fd4e00a72b0 2024-09-13T17:07:09.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e014c5c0 con 0x7fd4e00a72b0 2024-09-13T17:07:09.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e0155f40 con 0x7fd4e00a72b0 2024-09-13T17:07:09.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4d5ffb640 1 -- 172.21.15.39:0/4229544015 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4dc0135d0 con 0x7fd4e00a72b0 2024-09-13T17:07:09.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4d5ffb640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4c4066690 0x7fd4c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:09.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4d5ffb640 1 -- 172.21.15.39:0/4229544015 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7fd4dc0850a0 con 0x7fd4e00a72b0 2024-09-13T17:07:09.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4e4ad6640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4c4066690 0x7fd4c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:09.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.042+0000 7fd4e4ad6640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4c4066690 0x7fd4c4068b50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00ad000 tx=0x7fd4d00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:09.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.046+0000 7fd4d5ffb640 1 -- 172.21.15.39:0/4229544015 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd4dc051820 con 0x7fd4e00a72b0 2024-09-13T17:07:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.278+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4e00722c0 con 0x7fd4e00a72b0 2024-09-13T17:07:09.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.278+0000 7fd4d5ffb640 1 -- 172.21.15.39:0/4229544015 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7fd4dc054e40 con 0x7fd4e00a72b0 2024-09-13T17:07:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:09.276 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:09.276 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4c4066690 msgr2=0x7fd4c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4c4066690 0x7fd4c4068b50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00ad000 tx=0x7fd4d00a6000 comp rx=0 tx=0).stop 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4e00a72b0 msgr2=0x7fd4e0149a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4e00a72b0 0x7fd4e0149a90 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00e9d0 tx=0x7fd4dc00ee90 comp rx=0 tx=0).stop 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 shutdown_connections 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4c4066690 0x7fd4c4068b50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4e0148c90 0x7fd4e0147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4e01556d0 0x7fd4e006d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 --2- 172.21.15.39:0/4229544015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4e00a72b0 0x7fd4e0149a90 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 >> 172.21.15.39:0/4229544015 conn(0x7fd4e00a97b0 msgr2=0x7fd4e00ac320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 shutdown_connections 2024-09-13T17:07:09.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:09.282+0000 7fd4e753b640 1 -- 172.21.15.39:0/4229544015 wait complete. 2024-09-13T17:07:09.298 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:07:11.299 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:11.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f43f43640 1 Processor -- start 2024-09-13T17:07:11.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f43f43640 1 -- start start 2024-09-13T17:07:11.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f43f43640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f3c065ee0 0x7f9f3c14f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f43f43640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9f3c0c0a90 0x7f9f3c0c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f43f43640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c0c2ef0 0x7f9f3c0c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f43f43640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3c0c53d0 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f43f43640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3c06ddf0 con 0x7f9f3c065ee0 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f43f43640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0c5950 con 0x7f9f3c0c2ef0 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f414de640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9f3c0c0a90 0x7f9f3c0c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f424e0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c0c2ef0 0x7f9f3c0c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f41cdf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f3c065ee0 0x7f9f3c14f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f414de640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9f3c0c0a90 0x7f9f3c0c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53166/0 (socket says 172.21.15.39:53166) 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f424e0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c0c2ef0 0x7f9f3c0c22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:48592/0 (socket says 172.21.15.39:48592) 2024-09-13T17:07:11.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f414de640 1 -- 172.21.15.39:0/708512805 learned_addr learned my addr 172.21.15.39:0/708512805 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:11.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f424e0640 1 -- 172.21.15.39:0/708512805 >> v2:172.21.15.84:3300/0 conn(0x7f9f3c065ee0 msgr2=0x7f9f3c14f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:11.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f424e0640 1 --2- 172.21.15.39:0/708512805 >> v2:172.21.15.84:3300/0 conn(0x7f9f3c065ee0 0x7f9f3c14f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f424e0640 1 -- 172.21.15.39:0/708512805 >> v2:172.21.15.39:3300/0 conn(0x7f9f3c0c0a90 msgr2=0x7f9f3c0c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f424e0640 1 --2- 172.21.15.39:0/708512805 >> v2:172.21.15.39:3300/0 conn(0x7f9f3c0c0a90 0x7f9f3c0c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f424e0640 1 -- 172.21.15.39:0/708512805 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c0c2820 con 0x7f9f3c0c2ef0 2024-09-13T17:07:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.446+0000 7f9f414de640 1 --2- 172.21.15.39:0/708512805 >> v2:172.21.15.39:3300/0 conn(0x7f9f3c0c0a90 0x7f9f3c0c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:07:11.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f424e0640 1 --2- 172.21.15.39:0/708512805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c0c2ef0 0x7f9f3c0c22e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0617d0 tx=0x7f9f3401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=583a87a574564770 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f40cdd640 1 -- 172.21.15.39:0/708512805 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340a5980 con 0x7f9f3c0c2ef0 2024-09-13T17:07:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f40cdd640 1 -- 172.21.15.39:0/708512805 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f340a5ae0 con 0x7f9f3c0c2ef0 2024-09-13T17:07:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f40cdd640 1 -- 172.21.15.39:0/708512805 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340a5e70 con 0x7f9f3c0c2ef0 2024-09-13T17:07:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f43f43640 1 -- 172.21.15.39:0/708512805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c0c2ef0 msgr2=0x7f9f3c0c22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f43f43640 1 --2- 172.21.15.39:0/708512805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c0c2ef0 0x7f9f3c0c22e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0617d0 tx=0x7f9f3401c900 comp rx=0 tx=0).stop 2024-09-13T17:07:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f43f43640 1 -- 172.21.15.39:0/708512805 shutdown_connections 2024-09-13T17:07:11.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f43f43640 1 --2- 172.21.15.39:0/708512805 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c0c2ef0 0x7f9f3c0c22e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f43f43640 1 --2- 172.21.15.39:0/708512805 >> v2:172.21.15.39:3300/0 conn(0x7f9f3c0c0a90 0x7f9f3c0c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f43f43640 1 --2- 172.21.15.39:0/708512805 >> v2:172.21.15.84:3300/0 conn(0x7f9f3c065ee0 0x7f9f3c14f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f43f43640 1 -- 172.21.15.39:0/708512805 >> 172.21.15.39:0/708512805 conn(0x7f9f3c0c3310 msgr2=0x7f9f3c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:11.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f43f43640 1 -- 172.21.15.39:0/708512805 shutdown_connections 2024-09-13T17:07:11.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.450+0000 7f9f43f43640 1 -- 172.21.15.39:0/708512805 wait complete. 2024-09-13T17:07:11.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 Processor -- start 2024-09-13T17:07:11.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 -- start start 2024-09-13T17:07:11.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c065ee0 0x7f9f3c154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:11.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f3c0c0a90 0x7f9f3c144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:11.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f3c0c2ef0 0x7f9f3c144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:11.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c072400 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c071bf0 con 0x7f9f3c0c2ef0 2024-09-13T17:07:11.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c071e40 con 0x7f9f3c065ee0 2024-09-13T17:07:11.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f414de640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f3c0c0a90 0x7f9f3c144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:11.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f414de640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f3c0c0a90 0x7f9f3c144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53172/0 (socket says 172.21.15.39:53172) 2024-09-13T17:07:11.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f414de640 1 -- 172.21.15.39:0/2077157679 learned_addr learned my addr 172.21.15.39:0/2077157679 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:11.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f41cdf640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c065ee0 0x7f9f3c154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:11.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f414de640 1 -- 172.21.15.39:0/2077157679 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c065ee0 msgr2=0x7f9f3c154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:11.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f414de640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c065ee0 0x7f9f3c154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f414de640 1 -- 172.21.15.39:0/2077157679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f3c0c2ef0 msgr2=0x7f9f3c144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:11.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f414de640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f3c0c2ef0 0x7f9f3c144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f414de640 1 -- 172.21.15.39:0/2077157679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c0094d0 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f414de640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f3c0c0a90 0x7f9f3c144780 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00e960 tx=0x7f9f2c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:11.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f327fc640 1 -- 172.21.15.39:0/2077157679 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00ca00 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f327fc640 1 -- 172.21.15.39:0/2077157679 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c00cb60 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f34009a70 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f327fc640 1 -- 172.21.15.39:0/2077157679 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00ced0 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3c144260 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.454+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3c1440c0 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.458+0000 7f9f327fc640 1 -- 172.21.15.39:0/2077157679 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9f2c013470 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.458+0000 7f9f327fc640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9f20066690 0x7f9f20068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:11.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.458+0000 7f9f327fc640 1 -- 172.21.15.39:0/2077157679 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f9f2c0855a0 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.458+0000 7f9f41cdf640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9f20066690 0x7f9f20068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:11.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.458+0000 7f9f41cdf640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9f20066690 0x7f9f20068b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c145f40 tx=0x7f9f38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:11.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.462+0000 7f9f327fc640 1 -- 172.21.15.39:0/2077157679 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9f2c051e20 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.726+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f3c1440c0 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.726+0000 7f9f327fc640 1 -- 172.21.15.39:0/2077157679 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f9f3c1440c0 con 0x7f9f3c0c0a90 2024-09-13T17:07:11.726 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:11.726 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:11.726 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9f20066690 msgr2=0x7f9f20068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9f20066690 0x7f9f20068b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c145f40 tx=0x7f9f38008040 comp rx=0 tx=0).stop 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f3c0c0a90 msgr2=0x7f9f3c144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f3c0c0a90 0x7f9f3c144780 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00e960 tx=0x7f9f2c00ee20 comp rx=0 tx=0).stop 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 shutdown_connections 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9f20066690 0x7f9f20068b50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f3c0c2ef0 0x7f9f3c144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f3c0c0a90 0x7f9f3c144780 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 --2- 172.21.15.39:0/2077157679 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9f3c065ee0 0x7f9f3c154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:11.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 >> 172.21.15.39:0/2077157679 conn(0x7f9f3c0c3310 msgr2=0x7f9f3c15b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:11.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 shutdown_connections 2024-09-13T17:07:11.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:11.730+0000 7f9f43f43640 1 -- 172.21.15.39:0/2077157679 wait complete. 2024-09-13T17:07:11.748 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:07:13.749 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:13.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.906+0000 7f38ebb60640 1 Processor -- start 2024-09-13T17:07:13.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.906+0000 7f38ebb60640 1 -- start start 2024-09-13T17:07:13.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.906+0000 7f38ebb60640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f38e40c7800 0x7f38e4154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:13.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.906+0000 7f38ebb60640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e40c5a30 con 0x7f38e4071dd0 2024-09-13T17:07:13.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.906+0000 7f38ebb60640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e41576a0 con 0x7f38e4151500 2024-09-13T17:07:13.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.906+0000 7f38ebb60640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e4157980 con 0x7f38e40c7800 2024-09-13T17:07:13.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.906+0000 7f38e90fb640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f38e4071dd0 0x7f38e4154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36204/0 (socket says 172.21.15.39:36204) 2024-09-13T17:07:13.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.906+0000 7f38e90fb640 1 -- 172.21.15.39:0/842670352 learned_addr learned my addr 172.21.15.39:0/842670352 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:07:13.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.906+0000 7f38ea0fd640 1 --2- 172.21.15.39:0/842670352 >> v2:172.21.15.84:3300/0 conn(0x7f38e40c7800 0x7f38e4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:13.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153030557 0 0) 0x7f38e40c5a30 con 0x7f38e4071dd0 2024-09-13T17:07:13.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d0003530 con 0x7f38e4071dd0 2024-09-13T17:07:13.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2195573378 0 0) 0x7f38d0003530 con 0x7f38e4071dd0 2024-09-13T17:07:13.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 >> v2:172.21.15.84:3300/0 conn(0x7f38e40c7800 msgr2=0x7f38e4154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:13.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 --2- 172.21.15.39:0/842670352 >> v2:172.21.15.84:3300/0 conn(0x7f38e40c7800 0x7f38e4154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:13.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 >> v1:172.21.15.84:6789/0 conn(0x7f38e4151500 legacy=0x7f38e40c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:13.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4157c60 con 0x7f38e4071dd0 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f38d40016a0 con 0x7f38e4071dd0 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38ea0fd640 1 --2- 172.21.15.39:0/842670352 >> v2:172.21.15.84:3300/0 conn(0x7f38e40c7800 0x7f38e4154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 >> v1:172.21.15.39:6789/0 conn(0x7f38e4071dd0 legacy=0x7f38e4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38d0003980 0x7f38d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d00047d0 0x7f38d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38d0005190 0x7f38d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4157980 con 0x7f38d0005190 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d000c130 con 0x7f38d00047d0 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d000c2f0 con 0x7f38d0003980 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38ea0fd640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38d0005190 0x7f38d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e90fb640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d00047d0 0x7f38d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38e98fc640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38d0003980 0x7f38d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:13.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38ea0fd640 1 -- 172.21.15.39:0/842670352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38d0003980 msgr2=0x7f38d0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:13.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38ea0fd640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38d0003980 0x7f38d0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:13.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38ea0fd640 1 -- 172.21.15.39:0/842670352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d00047d0 msgr2=0x7f38d0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:13.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38ea0fd640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d00047d0 0x7f38d0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:13.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38ea0fd640 1 -- 172.21.15.39:0/842670352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0003530 con 0x7f38d0005190 2024-09-13T17:07:13.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38ea0fd640 1 -- 172.21.15.39:0/842670352 reap_dead start 2024-09-13T17:07:13.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.910+0000 7f38ea0fd640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38d0005190 0x7f38d000b990 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00e320 tx=0x7f38dc01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc7bf1bf28b518a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:13.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0a5d40 con 0x7f38d0005190 2024-09-13T17:07:13.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc00f040 con 0x7f38d0005190 2024-09-13T17:07:13.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38e88fa640 1 -- 172.21.15.39:0/842670352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0a5ea0 con 0x7f38d0005190 2024-09-13T17:07:13.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 -- 172.21.15.39:0/842670352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38d0005190 msgr2=0x7f38d000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:13.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38d0005190 0x7f38d000b990 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00e320 tx=0x7f38dc01ce40 comp rx=0 tx=0).stop 2024-09-13T17:07:13.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 -- 172.21.15.39:0/842670352 shutdown_connections 2024-09-13T17:07:13.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 --2- 172.21.15.39:0/842670352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38d0005190 0x7f38d000b990 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:13.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 -- 172.21.15.39:0/842670352 >> 172.21.15.39:0/842670352 conn(0x7f38e40ab530 msgr2=0x7f38e40ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:13.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 -- 172.21.15.39:0/842670352 shutdown_connections 2024-09-13T17:07:13.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 -- 172.21.15.39:0/842670352 wait complete. 2024-09-13T17:07:13.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 Processor -- start 2024-09-13T17:07:13.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 -- start start 2024-09-13T17:07:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38d0005190 0x7f38e40a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e414c500 0x7f38e41561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e41566e0 0x7f38e4150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4156b70 con 0x7f38e41566e0 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40a9c30 con 0x7f38e414c500 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ebb60640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41e9560 con 0x7f38d0005190 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ea0fd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e41566e0 0x7f38e4150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ea0fd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e41566e0 0x7f38e4150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53188/0 (socket says 172.21.15.39:53188) 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ea0fd640 1 -- 172.21.15.39:0/3627690589 learned_addr learned my addr 172.21.15.39:0/3627690589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38e98fc640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38d0005190 0x7f38e40a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38e90fb640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e414c500 0x7f38e41561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ea0fd640 1 -- 172.21.15.39:0/3627690589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38d0005190 msgr2=0x7f38e40a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ea0fd640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38d0005190 0x7f38e40a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ea0fd640 1 -- 172.21.15.39:0/3627690589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e414c500 msgr2=0x7f38e41561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ea0fd640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e414c500 0x7f38e41561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ea0fd640 1 -- 172.21.15.39:0/3627690589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e41e9840 con 0x7f38e41566e0 2024-09-13T17:07:13.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.914+0000 7f38ea0fd640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e41566e0 0x7f38e4150330 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f38dc01d6e0 tx=0x7f38dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:13.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38da7fc640 1 -- 172.21.15.39:0/3627690589 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc004140 con 0x7f38e41566e0 2024-09-13T17:07:13.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38da7fc640 1 -- 172.21.15.39:0/3627690589 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc0042a0 con 0x7f38e41566e0 2024-09-13T17:07:13.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e41e9b20 con 0x7f38e41566e0 2024-09-13T17:07:13.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38da7fc640 1 -- 172.21.15.39:0/3627690589 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc004650 con 0x7f38e41566e0 2024-09-13T17:07:13.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e41e9210 con 0x7f38e41566e0 2024-09-13T17:07:13.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e41e9d30 con 0x7f38e41566e0 2024-09-13T17:07:13.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38da7fc640 1 -- 172.21.15.39:0/3627690589 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38dc0b6480 con 0x7f38e41566e0 2024-09-13T17:07:13.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38da7fc640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f38c8066690 0x7f38c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:13.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38e98fc640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f38c8066690 0x7f38c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:13.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38da7fc640 1 -- 172.21.15.39:0/3627690589 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f38dc11f560 con 0x7f38e41566e0 2024-09-13T17:07:13.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.918+0000 7f38e98fc640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f38c8066690 0x7f38c8068b50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000c10 tx=0x7f38e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:13.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:13.922+0000 7f38da7fc640 1 -- 172.21.15.39:0/3627690589 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f38dc0ebd00 con 0x7f38e41566e0 2024-09-13T17:07:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38e4157320 con 0x7f38e41566e0 2024-09-13T17:07:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38da7fc640 1 -- 172.21.15.39:0/3627690589 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f38dc009de0 con 0x7f38e41566e0 2024-09-13T17:07:14.156 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:14.156 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:07:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f38c8066690 msgr2=0x7f38c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f38c8066690 0x7f38c8068b50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000c10 tx=0x7f38e0009040 comp rx=0 tx=0).stop 2024-09-13T17:07:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e41566e0 msgr2=0x7f38e4150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e41566e0 0x7f38e4150330 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f38dc01d6e0 tx=0x7f38dc009750 comp rx=0 tx=0).stop 2024-09-13T17:07:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 shutdown_connections 2024-09-13T17:07:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f38c8066690 0x7f38c8068b50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38e414c500 0x7f38e41561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38e41566e0 0x7f38e4150330 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 --2- 172.21.15.39:0/3627690589 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f38d0005190 0x7f38e40a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 >> 172.21.15.39:0/3627690589 conn(0x7f38e40ab530 msgr2=0x7f38e40c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 shutdown_connections 2024-09-13T17:07:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:14.158+0000 7f38ebb60640 1 -- 172.21.15.39:0/3627690589 wait complete. 2024-09-13T17:07:14.175 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:07:16.177 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:16.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dde378640 1 Processor -- start 2024-09-13T17:07:16.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dde378640 1 -- start start 2024-09-13T17:07:16.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dde378640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1dd80bf4f0 0x7f1dd8153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dde378640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80611c0 0x7f1dd80bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dde378640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd80c1340 con 0x7f1dd8151860 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dde378640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd81578f0 con 0x7f1dd80bf4f0 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dde378640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd80c1590 con 0x7f1dd80611c0 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dd7fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1dd8151860 0x7f1dd8157d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36212/0 (socket says 172.21.15.39:36212) 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dd7fff640 1 -- 172.21.15.39:0/636557053 learned_addr learned my addr 172.21.15.39:0/636557053 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1ddc915640 1 --2- 172.21.15.39:0/636557053 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80611c0 0x7f1dd80bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dd77fe640 1 --2- 172.21.15.39:0/636557053 >> v2:172.21.15.84:3300/0 conn(0x7f1dd80bf4f0 0x7f1dd8153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dd6ffd640 1 -- 172.21.15.39:0/636557053 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952697336 0 0) 0x7f1dd80c1340 con 0x7f1dd8151860 2024-09-13T17:07:16.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dd6ffd640 1 -- 172.21.15.39:0/636557053 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc0003530 con 0x7f1dd8151860 2024-09-13T17:07:16.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1dd6ffd640 1 -- 172.21.15.39:0/636557053 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1870117576 0 0) 0x7f1dc0003530 con 0x7f1dd8151860 2024-09-13T17:07:16.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1ddc915640 1 -- 172.21.15.39:0/636557053 >> v2:172.21.15.84:3300/0 conn(0x7f1dd80bf4f0 msgr2=0x7f1dd8153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:16.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1ddc915640 1 --2- 172.21.15.39:0/636557053 >> v2:172.21.15.84:3300/0 conn(0x7f1dd80bf4f0 0x7f1dd8153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1ddc915640 1 -- 172.21.15.39:0/636557053 >> v1:172.21.15.39:6789/0 conn(0x7f1dd8151860 legacy=0x7f1dd8157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:16.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.322+0000 7f1ddc915640 1 -- 172.21.15.39:0/636557053 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd80c1870 con 0x7f1dd80611c0 2024-09-13T17:07:16.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1ddc915640 1 --2- 172.21.15.39:0/636557053 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80611c0 0x7f1dd80bfad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009d90 tx=0x7f1dd001c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edab7cb474662f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:16.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dd6ffd640 1 -- 172.21.15.39:0/636557053 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8001680 con 0x7f1dd80611c0 2024-09-13T17:07:16.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dd6ffd640 1 -- 172.21.15.39:0/636557053 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd00a55d0 con 0x7f1dd80611c0 2024-09-13T17:07:16.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dd6ffd640 1 -- 172.21.15.39:0/636557053 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd00a5940 con 0x7f1dd80611c0 2024-09-13T17:07:16.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dde378640 1 -- 172.21.15.39:0/636557053 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80611c0 msgr2=0x7f1dd80bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:16.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dde378640 1 --2- 172.21.15.39:0/636557053 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80611c0 0x7f1dd80bfad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009d90 tx=0x7f1dd001c7d0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dde378640 1 -- 172.21.15.39:0/636557053 shutdown_connections 2024-09-13T17:07:16.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dde378640 1 --2- 172.21.15.39:0/636557053 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80611c0 0x7f1dd80bfad0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dde378640 1 --2- 172.21.15.39:0/636557053 >> v2:172.21.15.84:3300/0 conn(0x7f1dd80bf4f0 0x7f1dd8153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dde378640 1 -- 172.21.15.39:0/636557053 >> 172.21.15.39:0/636557053 conn(0x7f1dd8069e10 msgr2=0x7f1dd806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:16.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dde378640 1 -- 172.21.15.39:0/636557053 shutdown_connections 2024-09-13T17:07:16.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.326+0000 7f1dde378640 1 -- 172.21.15.39:0/636557053 wait complete. 2024-09-13T17:07:16.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 Processor -- start 2024-09-13T17:07:16.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 -- start start 2024-09-13T17:07:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dd80611c0 0x7f1dd80c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80bf4f0 0x7f1dd81efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8151860 0x7f1dd81f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd80abad0 con 0x7f1dd8151860 2024-09-13T17:07:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd80a9410 con 0x7f1dd80611c0 2024-09-13T17:07:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd8151d90 con 0x7f1dd80bf4f0 2024-09-13T17:07:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1ddc915640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8151860 0x7f1dd81f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1ddc915640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8151860 0x7f1dd81f0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53194/0 (socket says 172.21.15.39:53194) 2024-09-13T17:07:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1ddc915640 1 -- 172.21.15.39:0/1275506321 learned_addr learned my addr 172.21.15.39:0/1275506321 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dd7fff640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dd80611c0 0x7f1dd80c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dd77fe640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80bf4f0 0x7f1dd81efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1ddc915640 1 -- 172.21.15.39:0/1275506321 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80bf4f0 msgr2=0x7f1dd81efbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1ddc915640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80bf4f0 0x7f1dd81efbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1ddc915640 1 -- 172.21.15.39:0/1275506321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dd80611c0 msgr2=0x7f1dd80c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1ddc915640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dd80611c0 0x7f1dd80c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1ddc915640 1 -- 172.21.15.39:0/1275506321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd0009a70 con 0x7f1dd8151860 2024-09-13T17:07:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1ddc915640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8151860 0x7f1dd81f0100 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1dd001ae80 tx=0x7f1dd0019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dd4ff9640 1 -- 172.21.15.39:0/1275506321 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0009450 con 0x7f1dd8151860 2024-09-13T17:07:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dd4ff9640 1 -- 172.21.15.39:0/1275506321 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd00a5950 con 0x7f1dd8151860 2024-09-13T17:07:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dd4ff9640 1 -- 172.21.15.39:0/1275506321 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd00a5ce0 con 0x7f1dd8151860 2024-09-13T17:07:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd80ab7f0 con 0x7f1dd8151860 2024-09-13T17:07:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd80abf40 con 0x7f1dd8151860 2024-09-13T17:07:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.330+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd80a9730 con 0x7f1dd8151860 2024-09-13T17:07:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.334+0000 7f1dd4ff9640 1 -- 172.21.15.39:0/1275506321 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1dd00b4070 con 0x7f1dd8151860 2024-09-13T17:07:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.334+0000 7f1dd4ff9640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1dc4066690 0x7f1dc4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.334+0000 7f1dd4ff9640 1 -- 172.21.15.39:0/1275506321 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f1dd011f460 con 0x7f1dd8151860 2024-09-13T17:07:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.334+0000 7f1dd7fff640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1dc4066690 0x7f1dc4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.334+0000 7f1dd7fff640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1dc4066690 0x7f1dc4068b50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80011f0 tx=0x7f1dc80016e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.338+0000 7f1dd4ff9640 1 -- 172.21.15.39:0/1275506321 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1dd00ebc60 con 0x7f1dd8151860 2024-09-13T17:07:16.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.558+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dd80ab650 con 0x7f1dd8151860 2024-09-13T17:07:16.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.558+0000 7f1dd4ff9640 1 -- 172.21.15.39:0/1275506321 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f1dd80ab650 con 0x7f1dd8151860 2024-09-13T17:07:16.558 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:16.558 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:16.558 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1dc4066690 msgr2=0x7f1dc4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1dc4066690 0x7f1dc4068b50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80011f0 tx=0x7f1dc80016e0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8151860 msgr2=0x7f1dd81f0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8151860 0x7f1dd81f0100 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1dd001ae80 tx=0x7f1dd0019570 comp rx=0 tx=0).stop 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 shutdown_connections 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1dc4066690 0x7f1dc4068b50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dd8151860 0x7f1dd81f0100 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1dd80bf4f0 0x7f1dd81efbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 --2- 172.21.15.39:0/1275506321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dd80611c0 0x7f1dd80c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:16.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 >> 172.21.15.39:0/1275506321 conn(0x7f1dd8069e10 msgr2=0x7f1dd8157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:16.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 shutdown_connections 2024-09-13T17:07:16.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:16.562+0000 7f1dde378640 1 -- 172.21.15.39:0/1275506321 wait complete. 2024-09-13T17:07:16.580 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:07:18.581 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:18.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f73a5c89640 1 Processor -- start 2024-09-13T17:07:18.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f73a5c89640 1 -- start start 2024-09-13T17:07:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f73a5c89640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73a006e5e0 0x7f73a0151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f73a5c89640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a014f590 con 0x7f73a014f1b0 2024-09-13T17:07:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f73a5c89640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a006eae0 con 0x7f73a00c7470 2024-09-13T17:07:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f73a5c89640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a006f660 con 0x7f73a006e5e0 2024-09-13T17:07:18.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f7397fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73a006e5e0 0x7f73a0151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:18.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f7397fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73a006e5e0 0x7f73a0151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53204/0 (socket says 172.21.15.39:53204) 2024-09-13T17:07:18.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f7397fff640 1 -- 172.21.15.39:0/3612558663 learned_addr learned my addr 172.21.15.39:0/3612558663 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:18.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f7397fff640 1 -- 172.21.15.39:0/3612558663 >> v1:172.21.15.84:6789/0 conn(0x7f73a00c7470 legacy=0x7f73a00c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:18.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f7397fff640 1 -- 172.21.15.39:0/3612558663 >> v1:172.21.15.39:6789/0 conn(0x7f73a014f1b0 legacy=0x7f73a00714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:18.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.742+0000 7f7397fff640 1 -- 172.21.15.39:0/3612558663 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a006edc0 con 0x7f73a006e5e0 2024-09-13T17:07:18.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f7397fff640 1 --2- 172.21.15.39:0/3612558663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 0x7f73a0151bb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f7388009870 tx=0x7f738801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba97cbed1d3e47be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:18.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f739effd640 1 -- 172.21.15.39:0/3612558663 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880aa650 con 0x7f73a006e5e0 2024-09-13T17:07:18.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f739effd640 1 -- 172.21.15.39:0/3612558663 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73880aa7b0 con 0x7f73a006e5e0 2024-09-13T17:07:18.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f739effd640 1 -- 172.21.15.39:0/3612558663 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880aab40 con 0x7f73a006e5e0 2024-09-13T17:07:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f73a5c89640 1 -- 172.21.15.39:0/3612558663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 msgr2=0x7f73a0151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f73a5c89640 1 --2- 172.21.15.39:0/3612558663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 0x7f73a0151bb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f7388009870 tx=0x7f738801c4f0 comp rx=0 tx=0).stop 2024-09-13T17:07:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f73a5c89640 1 -- 172.21.15.39:0/3612558663 shutdown_connections 2024-09-13T17:07:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f73a5c89640 1 --2- 172.21.15.39:0/3612558663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 0x7f73a0151bb0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f73a5c89640 1 -- 172.21.15.39:0/3612558663 >> 172.21.15.39:0/3612558663 conn(0x7f73a00a9720 msgr2=0x7f73a00a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f73a5c89640 1 -- 172.21.15.39:0/3612558663 shutdown_connections 2024-09-13T17:07:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f73a5c89640 1 -- 172.21.15.39:0/3612558663 wait complete. 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f73a5c89640 1 Processor -- start 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.746+0000 7f73a5c89640 1 -- start start 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f73a5c89640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 0x7f73a01463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f73a5c89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a00c7470 0x7f73a0145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f73a5c89640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73a014f1b0 0x7f73a00c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f73a5c89640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a014af30 con 0x7f73a006e5e0 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f73a5c89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a014a170 con 0x7f73a00c7470 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f73a5c89640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0146180 con 0x7f73a014f1b0 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 0x7f73a01463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 0x7f73a01463a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53218/0 (socket says 172.21.15.39:53218) 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739f7fe640 1 -- 172.21.15.39:0/414281630 learned_addr learned my addr 172.21.15.39:0/414281630 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f7397fff640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a00c7470 0x7f73a0145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739ffff640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73a014f1b0 0x7f73a00c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739f7fe640 1 -- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73a014f1b0 msgr2=0x7f73a00c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739f7fe640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73a014f1b0 0x7f73a00c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739f7fe640 1 -- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a00c7470 msgr2=0x7f73a0145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739f7fe640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a00c7470 0x7f73a0145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739f7fe640 1 -- 172.21.15.39:0/414281630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73880094d0 con 0x7f73a006e5e0 2024-09-13T17:07:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739ffff640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73a014f1b0 0x7f73a00c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:07:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f7397fff640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a00c7470 0x7f73a0145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739f7fe640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 0x7f73a01463a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f739000c3c0 tx=0x7f739000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739cff9640 1 -- 172.21.15.39:0/414281630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73900052d0 con 0x7f73a006e5e0 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739cff9640 1 -- 172.21.15.39:0/414281630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7390005430 con 0x7f73a006e5e0 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a00a9290 con 0x7f73a006e5e0 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739cff9640 1 -- 172.21.15.39:0/414281630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7390003840 con 0x7f73a006e5e0 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a01ed240 con 0x7f73a006e5e0 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a00a9520 con 0x7f73a006e5e0 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739cff9640 1 -- 172.21.15.39:0/414281630 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73900039a0 con 0x7f73a006e5e0 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739cff9640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7380066690 0x7f7380068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f739cff9640 1 -- 172.21.15.39:0/414281630 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 8701+0+0 (secure 0 0 0) 0x7f73900841a0 con 0x7f73a006e5e0 2024-09-13T17:07:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.750+0000 7f7397fff640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7380066690 0x7f7380068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:18.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.754+0000 7f7397fff640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7380066690 0x7f7380068b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f73880ae000 tx=0x7f7388008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:18.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.758+0000 7f739cff9640 1 -- 172.21.15.39:0/414281630 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7390050a40 con 0x7f73a006e5e0 2024-09-13T17:07:18.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.962+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73a0151bb0 con 0x7f73a006e5e0 2024-09-13T17:07:18.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.962+0000 7f739cff9640 1 -- 172.21.15.39:0/414281630 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f7390017090 con 0x7f73a006e5e0 2024-09-13T17:07:18.961 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:18.962 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:05:57.807169+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7050":{"gid":7050,"name":"c","rank":0,"incarnation":122,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/1392182763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1392182763},{"type":"v1","addr":"172.21.15.39:6837","nonce":1392182763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:18.962 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 131 2024-09-13T17:07:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7380066690 msgr2=0x7f7380068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7380066690 0x7f7380068b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f73880ae000 tx=0x7f7388008040 comp rx=0 tx=0).stop 2024-09-13T17:07:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 msgr2=0x7f73a01463a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 0x7f73a01463a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f739000c3c0 tx=0x7f739000c880 comp rx=0 tx=0).stop 2024-09-13T17:07:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 shutdown_connections 2024-09-13T17:07:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7380066690 0x7f7380068b50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73a00c7470 0x7f73a0145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73a014f1b0 0x7f73a00c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 --2- 172.21.15.39:0/414281630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73a006e5e0 0x7f73a01463a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 >> 172.21.15.39:0/414281630 conn(0x7f73a00a9720 msgr2=0x7f73a00a9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 shutdown_connections 2024-09-13T17:07:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:18.966+0000 7f73a5c89640 1 -- 172.21.15.39:0/414281630 wait complete. 2024-09-13T17:07:18.981 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:07:19.910 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.030+0000 7f59d8fe1640 1 Processor -- start 2024-09-13T17:07:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.030+0000 7f59d8fe1640 1 -- start start 2024-09-13T17:07:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d8fe1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59d414bf30 0x7f59d406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d8fe1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59d4155160 0x7f59d41520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d8fe1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d4151bf0 con 0x7f59d406b360 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d8fe1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d4155660 con 0x7f59d4155160 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d8fe1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d4155940 con 0x7f59d414bf30 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d1d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59d4155160 0x7f59d41520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d1d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59d4155160 0x7f59d41520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53224/0 (socket says 172.21.15.39:53224) 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d1d9b640 1 -- 172.21.15.39:0/1723103443 learned_addr learned my addr 172.21.15.39:0/1723103443 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d259c640 1 --2- 172.21.15.39:0/1723103443 >> v2:172.21.15.84:3300/0 conn(0x7f59d414bf30 0x7f59d406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d1d9b640 1 -- 172.21.15.39:0/1723103443 >> v2:172.21.15.84:3300/0 conn(0x7f59d414bf30 msgr2=0x7f59d406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d1d9b640 1 --2- 172.21.15.39:0/1723103443 >> v2:172.21.15.84:3300/0 conn(0x7f59d414bf30 0x7f59d406ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d1d9b640 1 -- 172.21.15.39:0/1723103443 >> v1:172.21.15.84:6789/0 conn(0x7f59d406b360 legacy=0x7f59d4152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d1d9b640 1 -- 172.21.15.39:0/1723103443 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d4155c20 con 0x7f59d4155160 2024-09-13T17:07:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d1d9b640 1 --2- 172.21.15.39:0/1723103443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59d4155160 0x7f59d41520f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f59d4066cf0 tx=0x7f59c801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975827eedbed3edf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d159a640 1 -- 172.21.15.39:0/1723103443 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80aa430 con 0x7f59d4155160 2024-09-13T17:07:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d159a640 1 -- 172.21.15.39:0/1723103443 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c80aa590 con 0x7f59d4155160 2024-09-13T17:07:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.034+0000 7f59d159a640 1 -- 172.21.15.39:0/1723103443 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80aa920 con 0x7f59d4155160 2024-09-13T17:07:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 -- 172.21.15.39:0/1723103443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59d4155160 msgr2=0x7f59d41520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 --2- 172.21.15.39:0/1723103443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59d4155160 0x7f59d41520f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f59d4066cf0 tx=0x7f59c801c4f0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 -- 172.21.15.39:0/1723103443 shutdown_connections 2024-09-13T17:07:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 --2- 172.21.15.39:0/1723103443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59d4155160 0x7f59d41520f0 secure :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f59d4066cf0 tx=0x7f59c801c4f0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 --2- 172.21.15.39:0/1723103443 >> v2:172.21.15.84:3300/0 conn(0x7f59d414bf30 0x7f59d406ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 -- 172.21.15.39:0/1723103443 >> 172.21.15.39:0/1723103443 conn(0x7f59d40c32a0 msgr2=0x7f59d40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:20.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 -- 172.21.15.39:0/1723103443 shutdown_connections 2024-09-13T17:07:20.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 -- 172.21.15.39:0/1723103443 wait complete. 2024-09-13T17:07:20.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 Processor -- start 2024-09-13T17:07:20.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 -- start start 2024-09-13T17:07:20.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59d406b360 0x7f59d40bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.038+0000 7f59d8fe1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59d414bf30 0x7f59d41489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d8fe1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59d4071960 0x7f59d4148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d8fe1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4149460 con 0x7f59d4071960 2024-09-13T17:07:20.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d8fe1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41ede50 con 0x7f59d414bf30 2024-09-13T17:07:20.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d8fe1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41f00f0 con 0x7f59d406b360 2024-09-13T17:07:20.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59d414bf30 0x7f59d41489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d259c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59d406b360 0x7f59d40bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59d414bf30 0x7f59d41489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:59956/0 (socket says 172.21.15.39:59956) 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d1d9b640 1 -- 172.21.15.39:0/2006069317 learned_addr learned my addr 172.21.15.39:0/2006069317 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d2d9d640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59d4071960 0x7f59d4148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d1d9b640 1 -- 172.21.15.39:0/2006069317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59d406b360 msgr2=0x7f59d40bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d1d9b640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59d406b360 0x7f59d40bfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d1d9b640 1 -- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59d4071960 msgr2=0x7f59d4148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d1d9b640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59d4071960 0x7f59d4148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d1d9b640 1 -- 172.21.15.39:0/2006069317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c80094d0 con 0x7f59d414bf30 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d1d9b640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59d414bf30 0x7f59d41489e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59c801c9d0 tx=0x7f59c801ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59c2ffd640 1 -- 172.21.15.39:0/2006069317 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80b3800 con 0x7f59d414bf30 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d41ee0a0 con 0x7f59d414bf30 2024-09-13T17:07:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d41ee660 con 0x7f59d414bf30 2024-09-13T17:07:20.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59c2ffd640 1 -- 172.21.15.39:0/2006069317 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c80b3960 con 0x7f59d414bf30 2024-09-13T17:07:20.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59c2ffd640 1 -- 172.21.15.39:0/2006069317 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c80b3d10 con 0x7f59d414bf30 2024-09-13T17:07:20.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.042+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b8005260 con 0x7f59d414bf30 2024-09-13T17:07:20.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.046+0000 7f59c2ffd640 1 -- 172.21.15.39:0/2006069317 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59c80c63c0 con 0x7f59d414bf30 2024-09-13T17:07:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.046+0000 7f59c2ffd640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59b4066690 0x7f59b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.046+0000 7f59c2ffd640 1 -- 172.21.15.39:0/2006069317 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7f59c811f6a0 con 0x7f59d414bf30 2024-09-13T17:07:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.046+0000 7f59d259c640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59b4066690 0x7f59b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.046+0000 7f59d259c640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59b4066690 0x7f59b4068b50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f59c400aae0 tx=0x7f59c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:20.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.058+0000 7f59c2ffd640 1 -- 172.21.15.39:0/2006069317 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59c80ebf10 con 0x7f59d414bf30 2024-09-13T17:07:20.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.290+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59b80057e0 con 0x7f59d414bf30 2024-09-13T17:07:20.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.290+0000 7f59c2ffd640 1 -- 172.21.15.39:0/2006069317 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+5182 (secure 0 0 0) 0x7f59c80ef530 con 0x7f59d414bf30 2024-09-13T17:07:20.290 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:20.290 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:19.798185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:20.290 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 132 2024-09-13T17:07:20.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59b4066690 msgr2=0x7f59b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59b4066690 0x7f59b4068b50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f59c400aae0 tx=0x7f59c4008040 comp rx=0 tx=0).stop 2024-09-13T17:07:20.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59d414bf30 msgr2=0x7f59d41489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59d414bf30 0x7f59d41489e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59c801c9d0 tx=0x7f59c801ad30 comp rx=0 tx=0).stop 2024-09-13T17:07:20.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 shutdown_connections 2024-09-13T17:07:20.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f59b4066690 0x7f59b4068b50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59d414bf30 0x7f59d41489e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59d4071960 0x7f59d4148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 --2- 172.21.15.39:0/2006069317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59d406b360 0x7f59d40bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 >> 172.21.15.39:0/2006069317 conn(0x7f59d40c32a0 msgr2=0x7f59d4150330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:20.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 shutdown_connections 2024-09-13T17:07:20.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.294+0000 7f59d8fe1640 1 -- 172.21.15.39:0/2006069317 wait complete. 2024-09-13T17:07:20.309 INFO:tasks.mds_thrash.fs.[a]:kill mds.b (rank=0) 2024-09-13T17:07:20.309 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:07:20.310 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:07:20.331 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:07:20.331 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:20.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.438+0000 7efddf736640 1 Processor -- start 2024-09-13T17:07:20.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.438+0000 7efddf736640 1 -- start start 2024-09-13T17:07:20.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efdd80c2db0 0x7efdd80bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efdd80611c0 0x7efdd81545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd80c00b0 con 0x7efdd806cbc0 2024-09-13T17:07:20.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd8153e70 con 0x7efdd80611c0 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd80c0300 con 0x7efdd80c2db0 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdddcd3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efdd80611c0 0x7efdd81545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddccd1640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7efdd806cbc0 0x7efdd81512b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33300/0 (socket says 172.21.15.39:33300) 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddccd1640 1 -- 172.21.15.39:0/2619695633 learned_addr learned my addr 172.21.15.39:0/2619695633 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddd4d2640 1 --2- 172.21.15.39:0/2619695633 >> v2:172.21.15.84:3300/0 conn(0x7efdd80c2db0 0x7efdd80bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdddcd3640 1 -- 172.21.15.39:0/2619695633 >> v2:172.21.15.84:3300/0 conn(0x7efdd80c2db0 msgr2=0x7efdd80bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdddcd3640 1 --2- 172.21.15.39:0/2619695633 >> v2:172.21.15.84:3300/0 conn(0x7efdd80c2db0 0x7efdd80bf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdddcd3640 1 -- 172.21.15.39:0/2619695633 >> v1:172.21.15.39:6789/0 conn(0x7efdd806cbc0 legacy=0x7efdd81512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdcffff640 1 -- 172.21.15.39:0/2619695633 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883182044 0 0) 0x7efdd80c00b0 con 0x7efdd806cbc0 2024-09-13T17:07:20.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdddcd3640 1 -- 172.21.15.39:0/2619695633 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd80c05e0 con 0x7efdd80611c0 2024-09-13T17:07:20.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdddcd3640 1 --2- 172.21.15.39:0/2619695633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 0x7efdd81545e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7efdd80619f0 tx=0x7efdd001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4448713cd1818e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:20.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdcffff640 1 -- 172.21.15.39:0/2619695633 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00a57a0 con 0x7efdd80611c0 2024-09-13T17:07:20.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdcffff640 1 -- 172.21.15.39:0/2619695633 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd00a5900 con 0x7efdd80611c0 2024-09-13T17:07:20.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efdcffff640 1 -- 172.21.15.39:0/2619695633 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00a5c90 con 0x7efdd80611c0 2024-09-13T17:07:20.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 -- 172.21.15.39:0/2619695633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 msgr2=0x7efdd81545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 --2- 172.21.15.39:0/2619695633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 0x7efdd81545e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7efdd80619f0 tx=0x7efdd001c810 comp rx=0 tx=0).stop 2024-09-13T17:07:20.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 -- 172.21.15.39:0/2619695633 shutdown_connections 2024-09-13T17:07:20.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 --2- 172.21.15.39:0/2619695633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 0x7efdd81545e0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 --2- 172.21.15.39:0/2619695633 >> v2:172.21.15.84:3300/0 conn(0x7efdd80c2db0 0x7efdd80bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 -- 172.21.15.39:0/2619695633 >> 172.21.15.39:0/2619695633 conn(0x7efdd8069e10 msgr2=0x7efdd806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:20.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 -- 172.21.15.39:0/2619695633 shutdown_connections 2024-09-13T17:07:20.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.442+0000 7efddf736640 1 -- 172.21.15.39:0/2619695633 wait complete. 2024-09-13T17:07:20.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 Processor -- start 2024-09-13T17:07:20.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 -- start start 2024-09-13T17:07:20.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 0x7efdd80c1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd806cbc0 0x7efdd8140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd80c2db0 0x7efdd81413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd8146550 con 0x7efdd80611c0 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd81ef780 con 0x7efdd80c2db0 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd8145f90 con 0x7efdd806cbc0 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddd4d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 0x7efdd80c1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddd4d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 0x7efdd80c1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45374/0 (socket says 172.21.15.39:45374) 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddd4d2640 1 -- 172.21.15.39:0/1056146824 learned_addr learned my addr 172.21.15.39:0/1056146824 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efdddcd3640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd80c2db0 0x7efdd81413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddccd1640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd806cbc0 0x7efdd8140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddd4d2640 1 -- 172.21.15.39:0/1056146824 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd806cbc0 msgr2=0x7efdd8140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddd4d2640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd806cbc0 0x7efdd8140e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddd4d2640 1 -- 172.21.15.39:0/1056146824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd80c2db0 msgr2=0x7efdd81413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddd4d2640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd80c2db0 0x7efdd81413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddd4d2640 1 -- 172.21.15.39:0/1056146824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0009a70 con 0x7efdd80611c0 2024-09-13T17:07:20.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efdddcd3640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd80c2db0 0x7efdd81413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:07:20.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddd4d2640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 0x7efdd80c1230 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7efdd400c7f0 tx=0x7efdd400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:20.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efdcdffb640 1 -- 172.21.15.39:0/1056146824 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4004220 con 0x7efdd80611c0 2024-09-13T17:07:20.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efdcdffb640 1 -- 172.21.15.39:0/1056146824 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd40043a0 con 0x7efdd80611c0 2024-09-13T17:07:20.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd8146270 con 0x7efdd80611c0 2024-09-13T17:07:20.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efdcdffb640 1 -- 172.21.15.39:0/1056146824 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4004510 con 0x7efdd80611c0 2024-09-13T17:07:20.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd81efcb0 con 0x7efdd80611c0 2024-09-13T17:07:20.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.446+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd81545e0 con 0x7efdd80611c0 2024-09-13T17:07:20.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.450+0000 7efdcdffb640 1 -- 172.21.15.39:0/1056146824 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efdd4003a70 con 0x7efdd80611c0 2024-09-13T17:07:20.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.450+0000 7efdcdffb640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdbc066690 0x7efdbc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:20.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.450+0000 7efdcdffb640 1 -- 172.21.15.39:0/1056146824 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7efdd4084d30 con 0x7efdd80611c0 2024-09-13T17:07:20.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.450+0000 7efddccd1640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdbc066690 0x7efdbc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:20.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.450+0000 7efddccd1640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdbc066690 0x7efdbc068b50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efdd80c0b30 tx=0x7efdc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:20.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.454+0000 7efdcdffb640 1 -- 172.21.15.39:0/1056146824 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efdd40514d0 con 0x7efdd80611c0 2024-09-13T17:07:20.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.646+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdd8145d70 con 0x7efdd80611c0 2024-09-13T17:07:20.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efdcdffb640 1 -- 172.21.15.39:0/1056146824 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+5182 (secure 0 0 0) 0x7efdd401e020 con 0x7efdd80611c0 2024-09-13T17:07:20.648 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:20.648 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:19.798185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:20.648 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 132 2024-09-13T17:07:20.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdbc066690 msgr2=0x7efdbc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdbc066690 0x7efdbc068b50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efdd80c0b30 tx=0x7efdc8009040 comp rx=0 tx=0).stop 2024-09-13T17:07:20.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 msgr2=0x7efdd80c1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:20.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 0x7efdd80c1230 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7efdd400c7f0 tx=0x7efdd400ccb0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 shutdown_connections 2024-09-13T17:07:20.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdbc066690 0x7efdbc068b50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efdd80c2db0 0x7efdd81413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efdd806cbc0 0x7efdd8140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 --2- 172.21.15.39:0/1056146824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd80611c0 0x7efdd80c1230 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:20.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 >> 172.21.15.39:0/1056146824 conn(0x7efdd8069e10 msgr2=0x7efdd8151ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:20.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 shutdown_connections 2024-09-13T17:07:20.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:20.650+0000 7efddf736640 1 -- 172.21.15.39:0/1056146824 wait complete. 2024-09-13T17:07:20.665 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:07:20.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efde1ac0640 1 Processor -- start 2024-09-13T17:07:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efde1ac0640 1 -- start start 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efde1ac0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efddc071dd0 0x7efddc154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efde1ac0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efddc0c7800 0x7efddc1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efde1ac0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc0c5a30 con 0x7efddc151500 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efde1ac0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc1576a0 con 0x7efddc071dd0 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efde1ac0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc157980 con 0x7efddc0c7800 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efddaffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efddc071dd0 0x7efddc154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efddaffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efddc071dd0 0x7efddc154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45378/0 (socket says 172.21.15.39:45378) 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efddaffd640 1 -- 172.21.15.39:0/2391697817 learned_addr learned my addr 172.21.15.39:0/2391697817 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efddbfff640 1 --2- 172.21.15.39:0/2391697817 >> v2:172.21.15.84:3300/0 conn(0x7efddc0c7800 0x7efddc1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efddaffd640 1 -- 172.21.15.39:0/2391697817 >> v2:172.21.15.84:3300/0 conn(0x7efddc0c7800 msgr2=0x7efddc1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:21.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efddaffd640 1 --2- 172.21.15.39:0/2391697817 >> v2:172.21.15.84:3300/0 conn(0x7efddc0c7800 0x7efddc1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efddaffd640 1 -- 172.21.15.39:0/2391697817 >> v1:172.21.15.84:6789/0 conn(0x7efddc151500 legacy=0x7efddc0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:21.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.118+0000 7efddaffd640 1 -- 172.21.15.39:0/2391697817 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc157c60 con 0x7efddc071dd0 2024-09-13T17:07:21.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddaffd640 1 --2- 172.21.15.39:0/2391697817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc071dd0 0x7efddc154070 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efddc0c4d20 tx=0x7efdd001ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb328c987a928162 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:21.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efdda7fc640 1 -- 172.21.15.39:0/2391697817 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00ab600 con 0x7efddc071dd0 2024-09-13T17:07:21.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efdda7fc640 1 -- 172.21.15.39:0/2391697817 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd00ab760 con 0x7efddc071dd0 2024-09-13T17:07:21.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efdda7fc640 1 -- 172.21.15.39:0/2391697817 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00abaf0 con 0x7efddc071dd0 2024-09-13T17:07:21.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 -- 172.21.15.39:0/2391697817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc071dd0 msgr2=0x7efddc154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 --2- 172.21.15.39:0/2391697817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc071dd0 0x7efddc154070 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efddc0c4d20 tx=0x7efdd001ce40 comp rx=0 tx=0).stop 2024-09-13T17:07:21.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 -- 172.21.15.39:0/2391697817 shutdown_connections 2024-09-13T17:07:21.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 --2- 172.21.15.39:0/2391697817 >> v2:172.21.15.84:3300/0 conn(0x7efddc0c7800 0x7efddc1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 --2- 172.21.15.39:0/2391697817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc071dd0 0x7efddc154070 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 -- 172.21.15.39:0/2391697817 >> 172.21.15.39:0/2391697817 conn(0x7efddc0ab530 msgr2=0x7efddc0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:21.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 -- 172.21.15.39:0/2391697817 shutdown_connections 2024-09-13T17:07:21.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 -- 172.21.15.39:0/2391697817 wait complete. 2024-09-13T17:07:21.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 Processor -- start 2024-09-13T17:07:21.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 -- start start 2024-09-13T17:07:21.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efddc071dd0 0x7efddc1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc0c7800 0x7efddc156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efddc151500 0x7efddc1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc155df0 con 0x7efddc0c7800 2024-09-13T17:07:21.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc1477a0 con 0x7efddc151500 2024-09-13T17:07:21.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efde1ac0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc1479f0 con 0x7efddc071dd0 2024-09-13T17:07:21.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc0c7800 0x7efddc156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc0c7800 0x7efddc156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45394/0 (socket says 172.21.15.39:45394) 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddaffd640 1 -- 172.21.15.39:0/206656599 learned_addr learned my addr 172.21.15.39:0/206656599 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddbfff640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efddc151500 0x7efddc1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddb7fe640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efddc071dd0 0x7efddc1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddaffd640 1 -- 172.21.15.39:0/206656599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efddc071dd0 msgr2=0x7efddc1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddaffd640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efddc071dd0 0x7efddc1494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddaffd640 1 -- 172.21.15.39:0/206656599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efddc151500 msgr2=0x7efddc1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddaffd640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efddc151500 0x7efddc1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.122+0000 7efddaffd640 1 -- 172.21.15.39:0/206656599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00094d0 con 0x7efddc0c7800 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efddaffd640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc0c7800 0x7efddc156260 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7efdd001a010 tx=0x7efdd001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efdc3fff640 1 -- 172.21.15.39:0/206656599 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00b4800 con 0x7efddc0c7800 2024-09-13T17:07:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efdc3fff640 1 -- 172.21.15.39:0/206656599 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd00b4960 con 0x7efddc0c7800 2024-09-13T17:07:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc147d20 con 0x7efddc0c7800 2024-09-13T17:07:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efdc3fff640 1 -- 172.21.15.39:0/206656599 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00b4cf0 con 0x7efddc0c7800 2024-09-13T17:07:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc1482e0 con 0x7efddc0c7800 2024-09-13T17:07:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc155f60 con 0x7efddc0c7800 2024-09-13T17:07:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efdc3fff640 1 -- 172.21.15.39:0/206656599 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efdd00c7440 con 0x7efddc0c7800 2024-09-13T17:07:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efdc3fff640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdc8066690 0x7efdc8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efdc3fff640 1 -- 172.21.15.39:0/206656599 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7efdd011f830 con 0x7efddc0c7800 2024-09-13T17:07:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efddb7fe640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdc8066690 0x7efdc8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.126+0000 7efddb7fe640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdc8066690 0x7efdc8068b50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efddc0c0090 tx=0x7efdcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:21.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.130+0000 7efdc3fff640 1 -- 172.21.15.39:0/206656599 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efdd00ebfd0 con 0x7efddc0c7800 2024-09-13T17:07:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.318+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efddc1484d0 con 0x7efddc0c7800 2024-09-13T17:07:21.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efdc3fff640 1 -- 172.21.15.39:0/206656599 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+5186 (secure 0 0 0) 0x7efdd00ef5f0 con 0x7efddc0c7800 2024-09-13T17:07:21.319 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:21.319 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":133,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:21.163233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:21.319 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 133 2024-09-13T17:07:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdc8066690 msgr2=0x7efdc8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdc8066690 0x7efdc8068b50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efddc0c0090 tx=0x7efdcc008040 comp rx=0 tx=0).stop 2024-09-13T17:07:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc0c7800 msgr2=0x7efddc156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc0c7800 0x7efddc156260 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7efdd001a010 tx=0x7efdd001ac20 comp rx=0 tx=0).stop 2024-09-13T17:07:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 shutdown_connections 2024-09-13T17:07:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efdc8066690 0x7efdc8068b50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efddc151500 0x7efddc1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efddc0c7800 0x7efddc156260 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 --2- 172.21.15.39:0/206656599 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efddc071dd0 0x7efddc1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 >> 172.21.15.39:0/206656599 conn(0x7efddc0ab530 msgr2=0x7efddc0c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 shutdown_connections 2024-09-13T17:07:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.322+0000 7efde1ac0640 1 -- 172.21.15.39:0/206656599 wait complete. 2024-09-13T17:07:21.336 INFO:tasks.mds_thrash.fs.[b]:mds.c down, removed from mdsmap 2024-09-13T17:07:21.336 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:07:21.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:21.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d2f3a640 1 Processor -- start 2024-09-13T17:07:21.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d2f3a640 1 -- start start 2024-09-13T17:07:21.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d2f3a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff8cc06d020 0x7ff8cc0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d2f3a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc151460 0x7ff8cc14db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d2f3a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8cc14e070 0x7ff8cc0c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d2f3a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8cc0a8c50 con 0x7ff8cc14e070 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d2f3a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8cc154100 con 0x7ff8cc06d020 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d2f3a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc1543e0 con 0x7ff8cc151460 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d14d7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8cc14e070 0x7ff8cc0c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d14d7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8cc14e070 0x7ff8cc0c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45406/0 (socket says 172.21.15.39:45406) 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d14d7640 1 -- 172.21.15.39:0/1534585959 learned_addr learned my addr 172.21.15.39:0/1534585959 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8cbfff640 1 --2- 172.21.15.39:0/1534585959 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc151460 0x7ff8cc14db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d14d7640 1 -- 172.21.15.39:0/1534585959 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc151460 msgr2=0x7ff8cc14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d14d7640 1 --2- 172.21.15.39:0/1534585959 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc151460 0x7ff8cc14db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d0cd6640 1 --2- 172.21.15.39:0/1534585959 >> v2:172.21.15.84:3300/0 conn(0x7ff8cc06d020 0x7ff8cc0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d14d7640 1 -- 172.21.15.39:0/1534585959 >> v2:172.21.15.84:3300/0 conn(0x7ff8cc06d020 msgr2=0x7ff8cc0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d14d7640 1 --2- 172.21.15.39:0/1534585959 >> v2:172.21.15.84:3300/0 conn(0x7ff8cc06d020 0x7ff8cc0a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.450+0000 7ff8d14d7640 1 -- 172.21.15.39:0/1534585959 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc1546c0 con 0x7ff8cc14e070 2024-09-13T17:07:21.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 --2- 172.21.15.39:0/1534585959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc14e070 0x7ff8cc0c4530 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc06d820 tx=0x7ff8bc01eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f0d82e6e3fa397 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:21.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8cb7fe640 1 -- 172.21.15.39:0/1534585959 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0c5280 con 0x7ff8cc14e070 2024-09-13T17:07:21.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8cb7fe640 1 -- 172.21.15.39:0/1534585959 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0a7b30 con 0x7ff8cc14e070 2024-09-13T17:07:21.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8cb7fe640 1 -- 172.21.15.39:0/1534585959 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0af440 con 0x7ff8cc14e070 2024-09-13T17:07:21.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1534585959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc14e070 msgr2=0x7ff8cc0c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1534585959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc14e070 0x7ff8cc0c4530 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc06d820 tx=0x7ff8bc01eaa0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1534585959 shutdown_connections 2024-09-13T17:07:21.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1534585959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc14e070 0x7ff8cc0c4530 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1534585959 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc151460 0x7ff8cc14db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1534585959 >> v2:172.21.15.84:3300/0 conn(0x7ff8cc06d020 0x7ff8cc0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1534585959 >> 172.21.15.39:0/1534585959 conn(0x7ff8cc069e10 msgr2=0x7ff8cc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:21.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1534585959 shutdown_connections 2024-09-13T17:07:21.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1534585959 wait complete. 2024-09-13T17:07:21.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 Processor -- start 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- start start 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc06d020 0x7ff8cc1ec290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8cc14e070 0x7ff8cc0bee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc151460 0x7ff8cc1eff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc1ef800 con 0x7ff8cc151460 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0aa460 con 0x7ff8cc14e070 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0aa740 con 0x7ff8cc06d020 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc151460 0x7ff8cc1eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc151460 0x7ff8cc1eff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45410/0 (socket says 172.21.15.39:45410) 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 -- 172.21.15.39:0/1448500288 learned_addr learned my addr 172.21.15.39:0/1448500288 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d0cd6640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc06d020 0x7ff8cc1ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8cbfff640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8cc14e070 0x7ff8cc0bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 -- 172.21.15.39:0/1448500288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc06d020 msgr2=0x7ff8cc1ec290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc06d020 0x7ff8cc1ec290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 -- 172.21.15.39:0/1448500288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8cc14e070 msgr2=0x7ff8cc0bee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8cc14e070 0x7ff8cc0bee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 -- 172.21.15.39:0/1448500288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc00b770 con 0x7ff8cc151460 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8cbfff640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8cc14e070 0x7ff8cc0bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d14d7640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc151460 0x7ff8cc1eff70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00b2c0 tx=0x7ff8bc01b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8c97fa640 1 -- 172.21.15.39:0/1448500288 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0ad750 con 0x7ff8cc151460 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8c97fa640 1 -- 172.21.15.39:0/1448500288 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0ad8d0 con 0x7ff8cc151460 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc1efa50 con 0x7ff8cc151460 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8c97fa640 1 -- 172.21.15.39:0/1448500288 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0adc80 con 0x7ff8cc151460 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.454+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc147500 con 0x7ff8cc151460 2024-09-13T17:07:21.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.458+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc1efc40 con 0x7ff8cc151460 2024-09-13T17:07:21.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.458+0000 7ff8c97fa640 1 -- 172.21.15.39:0/1448500288 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8bc0c33b0 con 0x7ff8cc151460 2024-09-13T17:07:21.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.458+0000 7ff8c97fa640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8ac066690 0x7ff8ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:21.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.458+0000 7ff8c97fa640 1 -- 172.21.15.39:0/1448500288 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7ff8bc11fff0 con 0x7ff8cc151460 2024-09-13T17:07:21.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.458+0000 7ff8d0cd6640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8ac066690 0x7ff8ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:21.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.458+0000 7ff8d0cd6640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8ac066690 0x7ff8ac068b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4007fd0 tx=0x7ff8c4007560 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:21.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.462+0000 7ff8c97fa640 1 -- 172.21.15.39:0/1448500288 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff8bc151db0 con 0x7ff8cc151460 2024-09-13T17:07:21.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.650+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8cc0c2210 con 0x7ff8cc151460 2024-09-13T17:07:21.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.650+0000 7ff8c97fa640 1 -- 172.21.15.39:0/1448500288 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+5186 (secure 0 0 0) 0x7ff8bc0bf090 con 0x7ff8cc151460 2024-09-13T17:07:21.648 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:21.648 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":133,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:21.163233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:reconnect","state_seq":19,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:21.649 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 133 2024-09-13T17:07:21.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.650+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8ac066690 msgr2=0x7ff8ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.650+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8ac066690 0x7ff8ac068b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4007fd0 tx=0x7ff8c4007560 comp rx=0 tx=0).stop 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc151460 msgr2=0x7ff8cc1eff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc151460 0x7ff8cc1eff70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00b2c0 tx=0x7ff8bc01b840 comp rx=0 tx=0).stop 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 shutdown_connections 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff8ac066690 0x7ff8ac068b50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8cc14e070 0x7ff8cc0bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc151460 0x7ff8cc1eff70 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 --2- 172.21.15.39:0/1448500288 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8cc06d020 0x7ff8cc1ec290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 >> 172.21.15.39:0/1448500288 conn(0x7ff8cc069e10 msgr2=0x7ff8cc147c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 shutdown_connections 2024-09-13T17:07:21.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:21.654+0000 7ff8d2f3a640 1 -- 172.21.15.39:0/1448500288 wait complete. 2024-09-13T17:07:21.666 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 133, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 129, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5561": { "addr": "172.21.15.84:6833/440028459", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 440028459, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 440028459, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5561, "incarnation": 126, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 69 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:06:02.851726+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5561 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 133, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_7230": { "addr": "172.21.15.39:6835/3941977343", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 3941977343, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 3941977343, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7230, "incarnation": 132, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 19 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:07:21.163233+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7230 } } } ], "standbys": [ { "addr": "172.21.15.84:6835/3923043380", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 3923043380, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 3923043380, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 130, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5963, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:07:22.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:22.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab6164b640 1 Processor -- start 2024-09-13T17:07:22.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab6164b640 1 -- start start 2024-09-13T17:07:22.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab6164b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c06c940 0x7fab5c0c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:22.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab6164b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fab5c1555f0 0x7fab5c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:22.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab6164b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c14f790 con 0x7fab5c1526b0 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab6164b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c152a90 con 0x7fab5c1555f0 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab6164b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c06ce40 con 0x7fab5c06c940 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab5b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fab5c1555f0 0x7fab5c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab5b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fab5c1555f0 0x7fab5c155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45422/0 (socket says 172.21.15.39:45422) 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab5b7fe640 1 -- 172.21.15.39:0/3167288134 learned_addr learned my addr 172.21.15.39:0/3167288134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab5b7fe640 1 -- 172.21.15.39:0/3167288134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c06c940 msgr2=0x7fab5c0c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab53fff640 1 --2- 172.21.15.39:0/3167288134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c06c940 0x7fab5c0c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab5a7fc640 1 -- 172.21.15.39:0/3167288134 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681134705 0 0) 0x7fab5c14f790 con 0x7fab5c1526b0 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab5b7fe640 1 --2- 172.21.15.39:0/3167288134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c06c940 0x7fab5c0c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab5b7fe640 1 -- 172.21.15.39:0/3167288134 >> v1:172.21.15.39:6789/0 conn(0x7fab5c1526b0 legacy=0x7fab5c0aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab5b7fe640 1 -- 172.21.15.39:0/3167288134 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c06d120 con 0x7fab5c1555f0 2024-09-13T17:07:22.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.802+0000 7fab53fff640 1 --2- 172.21.15.39:0/3167288134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c06c940 0x7fab5c0c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:07:22.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab5b7fe640 1 --2- 172.21.15.39:0/3167288134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 0x7fab5c155aa0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fab54009e50 tx=0x7fab5401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5168c19843db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:22.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab5a7fc640 1 -- 172.21.15.39:0/3167288134 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab540a57f0 con 0x7fab5c1555f0 2024-09-13T17:07:22.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab5a7fc640 1 -- 172.21.15.39:0/3167288134 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab540a5950 con 0x7fab5c1555f0 2024-09-13T17:07:22.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab5a7fc640 1 -- 172.21.15.39:0/3167288134 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab540a5ce0 con 0x7fab5c1555f0 2024-09-13T17:07:22.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 -- 172.21.15.39:0/3167288134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 msgr2=0x7fab5c155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:22.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 --2- 172.21.15.39:0/3167288134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 0x7fab5c155aa0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fab54009e50 tx=0x7fab5401ca30 comp rx=0 tx=0).stop 2024-09-13T17:07:22.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 -- 172.21.15.39:0/3167288134 shutdown_connections 2024-09-13T17:07:22.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 --2- 172.21.15.39:0/3167288134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 0x7fab5c155aa0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:22.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 --2- 172.21.15.39:0/3167288134 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c06c940 0x7fab5c0c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:22.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 -- 172.21.15.39:0/3167288134 >> 172.21.15.39:0/3167288134 conn(0x7fab5c069e10 msgr2=0x7fab5c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:22.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 -- 172.21.15.39:0/3167288134 shutdown_connections 2024-09-13T17:07:22.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 -- 172.21.15.39:0/3167288134 wait complete. 2024-09-13T17:07:22.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 Processor -- start 2024-09-13T17:07:22.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.806+0000 7fab6164b640 1 -- start start 2024-09-13T17:07:22.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab6164b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab5c06c940 0x7fab5c146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab6164b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c1526b0 0x7fab5c146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab6164b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 0x7fab5c14e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab6164b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0a7990 con 0x7fab5c1555f0 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab6164b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1ead80 con 0x7fab5c06c940 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab6164b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1eb060 con 0x7fab5c1526b0 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 0x7fab5c14e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 0x7fab5c14e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45428/0 (socket says 172.21.15.39:45428) 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5b7fe640 1 -- 172.21.15.39:0/2046883252 learned_addr learned my addr 172.21.15.39:0/2046883252 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5affd640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab5c06c940 0x7fab5c146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab53fff640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c1526b0 0x7fab5c146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5b7fe640 1 -- 172.21.15.39:0/2046883252 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c1526b0 msgr2=0x7fab5c146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:22.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5b7fe640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c1526b0 0x7fab5c146d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5b7fe640 1 -- 172.21.15.39:0/2046883252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab5c06c940 msgr2=0x7fab5c146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5b7fe640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab5c06c940 0x7fab5c146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5b7fe640 1 -- 172.21.15.39:0/2046883252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab54009b30 con 0x7fab5c1555f0 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5b7fe640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 0x7fab5c14e110 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fab54009e50 tx=0x7fab540197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab537fe640 1 -- 172.21.15.39:0/2046883252 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54009210 con 0x7fab5c1555f0 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab537fe640 1 -- 172.21.15.39:0/2046883252 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab540b8680 con 0x7fab5c1555f0 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab537fe640 1 -- 172.21.15.39:0/2046883252 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab540b89f0 con 0x7fab5c1555f0 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c0a55c0 con 0x7fab5c1555f0 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c0a5b80 con 0x7fab5c1555f0 2024-09-13T17:07:22.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c15c290 con 0x7fab5c1555f0 2024-09-13T17:07:22.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab537fe640 1 -- 172.21.15.39:0/2046883252 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab540b8b50 con 0x7fab5c1555f0 2024-09-13T17:07:22.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab537fe640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fab3c066690 0x7fab3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:22.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab537fe640 1 -- 172.21.15.39:0/2046883252 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7fab54120b60 con 0x7fab5c1555f0 2024-09-13T17:07:22.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.810+0000 7fab5affd640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fab3c066690 0x7fab3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:22.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.814+0000 7fab5affd640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fab3c066690 0x7fab3c068b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fab4c000f70 tx=0x7fab4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:22.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:22.814+0000 7fab537fe640 1 -- 172.21.15.39:0/2046883252 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fab540ed380 con 0x7fab5c1555f0 2024-09-13T17:07:23.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.006+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab5c06bcb0 con 0x7fab5c1555f0 2024-09-13T17:07:23.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab537fe640 1 -- 172.21.15.39:0/2046883252 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+5183 (secure 0 0 0) 0x7fab540f09a0 con 0x7fab5c1555f0 2024-09-13T17:07:23.007 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:23.007 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":134,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:22.182894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:23.007 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 134 2024-09-13T17:07:23.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fab3c066690 msgr2=0x7fab3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:23.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fab3c066690 0x7fab3c068b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fab4c000f70 tx=0x7fab4c009040 comp rx=0 tx=0).stop 2024-09-13T17:07:23.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 msgr2=0x7fab5c14e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:23.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 0x7fab5c14e110 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fab54009e50 tx=0x7fab540197d0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 shutdown_connections 2024-09-13T17:07:23.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fab3c066690 0x7fab3c068b50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab5c1555f0 0x7fab5c14e110 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fab5c1526b0 0x7fab5c146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 --2- 172.21.15.39:0/2046883252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab5c06c940 0x7fab5c146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 >> 172.21.15.39:0/2046883252 conn(0x7fab5c069e10 msgr2=0x7fab5c1513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:23.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 shutdown_connections 2024-09-13T17:07:23.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.010+0000 7fab6164b640 1 -- 172.21.15.39:0/2046883252 wait complete. 2024-09-13T17:07:23.024 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:07:23.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2ef3e640 1 Processor -- start 2024-09-13T17:07:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2ef3e640 1 -- start start 2024-09-13T17:07:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2ef3e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e280c1790 0x7f1e28157f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2ef3e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e2806d370 0x7f1e280c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2ef3e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e28157370 con 0x7f1e280c20d0 2024-09-13T17:07:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2ef3e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e280c04a0 con 0x7f1e2806d370 2024-09-13T17:07:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2ef3e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e28153df0 con 0x7f1e280c1790 2024-09-13T17:07:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2d4db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e2806d370 0x7f1e280c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2d4db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e2806d370 0x7f1e280c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45440/0 (socket says 172.21.15.39:45440) 2024-09-13T17:07:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e27fff640 1 --2- 172.21.15.39:0/1725356765 >> v2:172.21.15.84:3300/0 conn(0x7f1e280c1790 0x7f1e28157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2d4db640 1 -- 172.21.15.39:0/1725356765 learned_addr learned my addr 172.21.15.39:0/1725356765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2d4db640 1 -- 172.21.15.39:0/1725356765 >> v2:172.21.15.84:3300/0 conn(0x7f1e280c1790 msgr2=0x7f1e28157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2d4db640 1 --2- 172.21.15.39:0/1725356765 >> v2:172.21.15.84:3300/0 conn(0x7f1e280c1790 0x7f1e28157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2d4db640 1 -- 172.21.15.39:0/1725356765 >> v1:172.21.15.84:6789/0 conn(0x7f1e280c20d0 legacy=0x7f1e28157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.802+0000 7f1e2d4db640 1 -- 172.21.15.39:0/1725356765 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2806c700 con 0x7f1e2806d370 2024-09-13T17:07:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2d4db640 1 --2- 172.21.15.39:0/1725356765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e2806d370 0x7f1e280c24b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1e2806eca0 tx=0x7f1e2001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1aca19ab29c9031 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e277fe640 1 -- 172.21.15.39:0/1725356765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200a5880 con 0x7f1e2806d370 2024-09-13T17:07:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e277fe640 1 -- 172.21.15.39:0/1725356765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e200a59e0 con 0x7f1e2806d370 2024-09-13T17:07:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e277fe640 1 -- 172.21.15.39:0/1725356765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200a5d70 con 0x7f1e2806d370 2024-09-13T17:07:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/1725356765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e2806d370 msgr2=0x7f1e280c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 --2- 172.21.15.39:0/1725356765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e2806d370 0x7f1e280c24b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1e2806eca0 tx=0x7f1e2001c9c0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/1725356765 shutdown_connections 2024-09-13T17:07:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 --2- 172.21.15.39:0/1725356765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e2806d370 0x7f1e280c24b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 --2- 172.21.15.39:0/1725356765 >> v2:172.21.15.84:3300/0 conn(0x7f1e280c1790 0x7f1e28157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/1725356765 >> 172.21.15.39:0/1725356765 conn(0x7f1e28069e10 msgr2=0x7f1e2806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/1725356765 shutdown_connections 2024-09-13T17:07:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/1725356765 wait complete. 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 Processor -- start 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 -- start start 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.806+0000 7f1e2ef3e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e2806d370 0x7f1e280c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ef3e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e280c1790 0x7f1e2814a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ef3e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e280c20d0 0x7f1e2814b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ef3e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e281eccf0 con 0x7f1e280c20d0 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ef3e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e281596c0 con 0x7f1e280c1790 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ef3e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e281ec0b0 con 0x7f1e2806d370 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2d4db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e280c20d0 0x7f1e2814b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2d4db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e280c20d0 0x7f1e2814b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45454/0 (socket says 172.21.15.39:45454) 2024-09-13T17:07:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2d4db640 1 -- 172.21.15.39:0/3738190761 learned_addr learned my addr 172.21.15.39:0/3738190761 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e27fff640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e280c1790 0x7f1e2814a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ccda640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e2806d370 0x7f1e280c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2d4db640 1 -- 172.21.15.39:0/3738190761 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e2806d370 msgr2=0x7f1e280c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2d4db640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e2806d370 0x7f1e280c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2d4db640 1 -- 172.21.15.39:0/3738190761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e280c1790 msgr2=0x7f1e2814a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2d4db640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e280c1790 0x7f1e2814a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2d4db640 1 -- 172.21.15.39:0/3738190761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e20009b30 con 0x7f1e280c20d0 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e27fff640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e280c1790 0x7f1e2814a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2d4db640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e280c20d0 0x7f1e2814b7f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1e200af040 tx=0x7f1e20019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e257fa640 1 -- 172.21.15.39:0/3738190761 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200b8670 con 0x7f1e280c20d0 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e257fa640 1 -- 172.21.15.39:0/3738190761 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e200b87d0 con 0x7f1e280c20d0 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e281ec350 con 0x7f1e280c20d0 2024-09-13T17:07:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e257fa640 1 -- 172.21.15.39:0/3738190761 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200b8b80 con 0x7f1e280c20d0 2024-09-13T17:07:23.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e281ec910 con 0x7f1e280c20d0 2024-09-13T17:07:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e281ece30 con 0x7f1e280c20d0 2024-09-13T17:07:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e257fa640 1 -- 172.21.15.39:0/3738190761 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e200b8ce0 con 0x7f1e280c20d0 2024-09-13T17:07:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e257fa640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e14066690 0x7f1e14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e257fa640 1 -- 172.21.15.39:0/3738190761 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7f1e201205c0 con 0x7f1e280c20d0 2024-09-13T17:07:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ccda640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e14066690 0x7f1e14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.810+0000 7f1e2ccda640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e14066690 0x7f1e14068b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c000f70 tx=0x7f1e1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:23.814+0000 7f1e257fa640 1 -- 172.21.15.39:0/3738190761 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1e281ece30 con 0x7f1e280c20d0 2024-09-13T17:07:24.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.002+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e281ece30 con 0x7f1e280c20d0 2024-09-13T17:07:24.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e257fa640 1 -- 172.21.15.39:0/3738190761 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+5183 (secure 0 0 0) 0x7f1e281ece30 con 0x7f1e280c20d0 2024-09-13T17:07:24.003 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:24.003 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":135,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:24.003 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 135 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e14066690 msgr2=0x7f1e14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e14066690 0x7f1e14068b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c000f70 tx=0x7f1e1c009040 comp rx=0 tx=0).stop 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e280c20d0 msgr2=0x7f1e2814b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e280c20d0 0x7f1e2814b7f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1e200af040 tx=0x7f1e20019760 comp rx=0 tx=0).stop 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 shutdown_connections 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1e14066690 0x7f1e14068b50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e280c1790 0x7f1e2814a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e280c20d0 0x7f1e2814b7f0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 --2- 172.21.15.39:0/3738190761 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1e2806d370 0x7f1e280c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 >> 172.21.15.39:0/3738190761 conn(0x7f1e28069e10 msgr2=0x7f1e28151d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 shutdown_connections 2024-09-13T17:07:24.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.006+0000 7f1e2ef3e640 1 -- 172.21.15.39:0/3738190761 wait complete. 2024-09-13T17:07:24.020 INFO:tasks.mds_thrash.fs.[b]:mds.a has gained rank=0, replacing gid=7050 2024-09-13T17:07:24.020 INFO:tasks.mds_thrash.fs.[b]:waiting for 0 secs before reviving mds.c 2024-09-13T17:07:24.020 INFO:tasks.mds_thrash.fs.[b]:reviving mds.c 2024-09-13T17:07:24.020 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:07:24.021 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:07:24.023 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:07:24.023 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:24.081 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T17:07:24.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f94502d7640 1 Processor -- start 2024-09-13T17:07:24.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f94502d7640 1 -- start start 2024-09-13T17:07:24.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f94502d7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f944814e0c0 0x7f94480a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:24.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f94502d7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94480c4740 0x7f94480c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:24.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f94502d7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94480c2780 con 0x7f94480c1f50 2024-09-13T17:07:24.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f94502d7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94480c2a60 con 0x7f944814e0c0 2024-09-13T17:07:24.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f94502d7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94480c2d40 con 0x7f94480c4740 2024-09-13T17:07:24.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f944e073640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f944814e0c0 0x7f94480a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:24.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f944e073640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f944814e0c0 0x7f94480a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45492/0 (socket says 172.21.15.39:45492) 2024-09-13T17:07:24.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f944e073640 1 -- 172.21.15.39:0/1862228606 learned_addr learned my addr 172.21.15.39:0/1862228606 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:24.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f944e874640 1 --2- 172.21.15.39:0/1862228606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94480c4740 0x7f94480c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:24.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f944e073640 1 -- 172.21.15.39:0/1862228606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94480c4740 msgr2=0x7f94480c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:24.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f944e073640 1 --2- 172.21.15.39:0/1862228606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94480c4740 0x7f94480c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f944e073640 1 -- 172.21.15.39:0/1862228606 >> v1:172.21.15.84:6789/0 conn(0x7f94480c1f50 legacy=0x7f944806cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:24.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f944e073640 1 -- 172.21.15.39:0/1862228606 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94480c3020 con 0x7f944814e0c0 2024-09-13T17:07:24.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.134+0000 7f944e874640 1 --2- 172.21.15.39:0/1862228606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94480c4740 0x7f94480c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:07:24.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f944e073640 1 --2- 172.21.15.39:0/1862228606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f944814e0c0 0x7f94480a4e70 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9444009870 tx=0x7f944401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab01412d12cea1d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:24.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f944d071640 1 -- 172.21.15.39:0/1862228606 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444004230 con 0x7f944814e0c0 2024-09-13T17:07:24.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f944d071640 1 -- 172.21.15.39:0/1862228606 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94440a7050 con 0x7f944814e0c0 2024-09-13T17:07:24.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f944d071640 1 -- 172.21.15.39:0/1862228606 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444003620 con 0x7f944814e0c0 2024-09-13T17:07:24.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 -- 172.21.15.39:0/1862228606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f944814e0c0 msgr2=0x7f94480a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:24.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 --2- 172.21.15.39:0/1862228606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f944814e0c0 0x7f94480a4e70 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9444009870 tx=0x7f944401c460 comp rx=0 tx=0).stop 2024-09-13T17:07:24.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 -- 172.21.15.39:0/1862228606 shutdown_connections 2024-09-13T17:07:24.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 --2- 172.21.15.39:0/1862228606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f94480c4740 0x7f94480c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 --2- 172.21.15.39:0/1862228606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f944814e0c0 0x7f94480a4e70 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 -- 172.21.15.39:0/1862228606 >> 172.21.15.39:0/1862228606 conn(0x7f9448069e10 msgr2=0x7f944806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:24.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 -- 172.21.15.39:0/1862228606 shutdown_connections 2024-09-13T17:07:24.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 -- 172.21.15.39:0/1862228606 wait complete. 2024-09-13T17:07:24.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 Processor -- start 2024-09-13T17:07:24.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.138+0000 7f94502d7640 1 -- start start 2024-09-13T17:07:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f94502d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94480c1f50 0x7f944815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f94502d7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94480c4740 0x7f94481ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f94502d7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f944814e0c0 0x7f94481ecd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f94502d7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9448154700 con 0x7f94480c1f50 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f94502d7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944815b070 con 0x7f94480c4740 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f94502d7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9448153af0 con 0x7f944814e0c0 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944e073640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94480c1f50 0x7f944815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944e073640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94480c1f50 0x7f944815a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45500/0 (socket says 172.21.15.39:45500) 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944e073640 1 -- 172.21.15.39:0/1400012272 learned_addr learned my addr 172.21.15.39:0/1400012272 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944e073640 1 -- 172.21.15.39:0/1400012272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f944814e0c0 msgr2=0x7f94481ecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944e073640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f944814e0c0 0x7f94481ecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944e073640 1 -- 172.21.15.39:0/1400012272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94480c4740 msgr2=0x7f94481ec7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944d872640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94480c4740 0x7f94481ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944e073640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94480c4740 0x7f94481ec7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944e073640 1 -- 172.21.15.39:0/1400012272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94440094d0 con 0x7f94480c1f50 2024-09-13T17:07:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944e073640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94480c1f50 0x7f944815a750 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f94440ae040 tx=0x7f9444019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f943effd640 1 -- 172.21.15.39:0/1400012272 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444003c90 con 0x7f94480c1f50 2024-09-13T17:07:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f943effd640 1 -- 172.21.15.39:0/1400012272 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444003df0 con 0x7f94480c1f50 2024-09-13T17:07:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f943effd640 1 -- 172.21.15.39:0/1400012272 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94440bc680 con 0x7f94480c1f50 2024-09-13T17:07:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9448153dd0 con 0x7f94480c1f50 2024-09-13T17:07:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9448154390 con 0x7f94480c1f50 2024-09-13T17:07:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94480c4bd0 con 0x7f94480c1f50 2024-09-13T17:07:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f943effd640 1 -- 172.21.15.39:0/1400012272 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94440c5440 con 0x7f94480c1f50 2024-09-13T17:07:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f943effd640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f942c066690 0x7f942c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f943effd640 1 -- 172.21.15.39:0/1400012272 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7f9444120e30 con 0x7f94480c1f50 2024-09-13T17:07:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944d872640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f942c066690 0x7f942c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.142+0000 7f944d872640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f942c066690 0x7f942c068b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f944814d950 tx=0x7f9438009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:24.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.146+0000 7f943effd640 1 -- 172.21.15.39:0/1400012272 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f94440ed540 con 0x7f94480c1f50 2024-09-13T17:07:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.342+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9448061470 con 0x7f94480c1f50 2024-09-13T17:07:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.342+0000 7f943effd640 1 -- 172.21.15.39:0/1400012272 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f94440b0020 con 0x7f94480c1f50 2024-09-13T17:07:24.341 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:24.341 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":136,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":136}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:24.341 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 136 2024-09-13T17:07:24.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.342+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f942c066690 msgr2=0x7f942c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.342+0000 7f94502d7640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f942c066690 0x7f942c068b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f944814d950 tx=0x7f9438009040 comp rx=0 tx=0).stop 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.342+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94480c1f50 msgr2=0x7f944815a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.342+0000 7f94502d7640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94480c1f50 0x7f944815a750 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f94440ae040 tx=0x7f9444019200 comp rx=0 tx=0).stop 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.346+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 shutdown_connections 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.346+0000 7f94502d7640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f942c066690 0x7f942c068b50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.346+0000 7f94502d7640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94480c4740 0x7f94481ec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.346+0000 7f94502d7640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f944814e0c0 0x7f94481ecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.346+0000 7f94502d7640 1 --2- 172.21.15.39:0/1400012272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94480c1f50 0x7f944815a750 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.346+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 >> 172.21.15.39:0/1400012272 conn(0x7f9448069e10 msgr2=0x7f94480a7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.346+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 shutdown_connections 2024-09-13T17:07:24.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:24.346+0000 7f94502d7640 1 -- 172.21.15.39:0/1400012272 wait complete. 2024-09-13T17:07:24.357 INFO:tasks.mds_thrash.fs.[b]:mds.c reported in up:standby state 2024-09-13T17:07:24.358 INFO:tasks.mds_thrash.fs.[b]:waiting for 66 secs before thrashing 2024-09-13T17:07:25.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:25.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7592f98640 1 Processor -- start 2024-09-13T17:07:25.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7592f98640 1 -- start start 2024-09-13T17:07:25.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7592f98640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f758c14da00 0x7f758c150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:25.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7592f98640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 0x7f758c06b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:25.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7592f98640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758c0c1220 con 0x7f758c06b9f0 2024-09-13T17:07:25.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7592f98640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c1512d0 con 0x7f758c14da00 2024-09-13T17:07:25.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7592f98640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1510d0 con 0x7f758c0c2e60 2024-09-13T17:07:25.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7590d34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f758c14da00 0x7f758c150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:25.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f758bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 0x7f758c06b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7590d34640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f758c14da00 0x7f758c150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:60092/0 (socket says 172.21.15.39:60092) 2024-09-13T17:07:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7591535640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f758c06b9f0 0x7f758c0c16b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:48816/0 (socket says 172.21.15.39:48816) 2024-09-13T17:07:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f7590d34640 1 -- 172.21.15.39:0/3456284962 learned_addr learned my addr 172.21.15.39:0/3456284962 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f758bfff640 1 -- 172.21.15.39:0/3456284962 >> v2:172.21.15.84:3300/0 conn(0x7f758c14da00 msgr2=0x7f758c150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f758bfff640 1 --2- 172.21.15.39:0/3456284962 >> v2:172.21.15.84:3300/0 conn(0x7f758c14da00 0x7f758c150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f758bfff640 1 -- 172.21.15.39:0/3456284962 >> v1:172.21.15.84:6789/0 conn(0x7f758c06b9f0 legacy=0x7f758c0c16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.162+0000 7f758bfff640 1 -- 172.21.15.39:0/3456284962 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0c0330 con 0x7f758c0c2e60 2024-09-13T17:07:25.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f758bfff640 1 --2- 172.21.15.39:0/3456284962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 0x7f758c06b4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75780097c0 tx=0x7f757801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d002385d4392c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:25.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f758b7fe640 1 -- 172.21.15.39:0/3456284962 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780ab600 con 0x7f758c0c2e60 2024-09-13T17:07:25.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f758b7fe640 1 -- 172.21.15.39:0/3456284962 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75780a7050 con 0x7f758c0c2e60 2024-09-13T17:07:25.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f758b7fe640 1 -- 172.21.15.39:0/3456284962 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780ab980 con 0x7f758c0c2e60 2024-09-13T17:07:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 -- 172.21.15.39:0/3456284962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 msgr2=0x7f758c06b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 --2- 172.21.15.39:0/3456284962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 0x7f758c06b4b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75780097c0 tx=0x7f757801c410 comp rx=0 tx=0).stop 2024-09-13T17:07:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 -- 172.21.15.39:0/3456284962 shutdown_connections 2024-09-13T17:07:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 --2- 172.21.15.39:0/3456284962 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 0x7f758c06b4b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:25.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 --2- 172.21.15.39:0/3456284962 >> v2:172.21.15.84:3300/0 conn(0x7f758c14da00 0x7f758c150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:25.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 -- 172.21.15.39:0/3456284962 >> 172.21.15.39:0/3456284962 conn(0x7f758c06a6a0 msgr2=0x7f758c0c5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:25.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 -- 172.21.15.39:0/3456284962 shutdown_connections 2024-09-13T17:07:25.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 -- 172.21.15.39:0/3456284962 wait complete. 2024-09-13T17:07:25.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 Processor -- start 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 -- start start 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c06b9f0 0x7f758c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 0x7f758c1f0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c14da00 0x7f758c0a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c099fe0 con 0x7f758c14da00 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c09a2c0 con 0x7f758c06b9f0 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7592f98640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1efa40 con 0x7f758c0c2e60 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7591535640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c14da00 0x7f758c0a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7591535640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c14da00 0x7f758c0a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45516/0 (socket says 172.21.15.39:45516) 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7591535640 1 -- 172.21.15.39:0/2716600322 learned_addr learned my addr 172.21.15.39:0/2716600322 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:25.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7590d34640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c06b9f0 0x7f758c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f758bfff640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 0x7f758c1f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7591535640 1 -- 172.21.15.39:0/2716600322 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 msgr2=0x7f758c1f0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7591535640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 0x7f758c1f0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7591535640 1 -- 172.21.15.39:0/2716600322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c06b9f0 msgr2=0x7f758c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.166+0000 7f7591535640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c06b9f0 0x7f758c099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f7591535640 1 -- 172.21.15.39:0/2716600322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7578009420 con 0x7f758c14da00 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f7591535640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c14da00 0x7f758c0a9e50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f758400c7a0 tx=0x7f758400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f75897fa640 1 -- 172.21.15.39:0/2716600322 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400a640 con 0x7f758c14da00 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f75897fa640 1 -- 172.21.15.39:0/2716600322 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758400a7a0 con 0x7f758c14da00 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c1efd20 con 0x7f758c14da00 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f75897fa640 1 -- 172.21.15.39:0/2716600322 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400ab10 con 0x7f758c14da00 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c14e270 con 0x7f758c14da00 2024-09-13T17:07:25.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c1f0240 con 0x7f758c14da00 2024-09-13T17:07:25.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f75897fa640 1 -- 172.21.15.39:0/2716600322 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f758400ac70 con 0x7f758c14da00 2024-09-13T17:07:25.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f75897fa640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7570066690 0x7f7570068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:25.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f7590d34640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7570066690 0x7f7570068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:25.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f75897fa640 1 -- 172.21.15.39:0/2716600322 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7f75840845c0 con 0x7f758c14da00 2024-09-13T17:07:25.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.170+0000 7f7590d34640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7570066690 0x7f7570068b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7580009ab0 tx=0x7f7580009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:25.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.174+0000 7f75897fa640 1 -- 172.21.15.39:0/2716600322 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f758c1f0240 con 0x7f758c14da00 2024-09-13T17:07:25.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.374+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f758c1f0240 con 0x7f758c14da00 2024-09-13T17:07:25.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.374+0000 7f75897fa640 1 -- 172.21.15.39:0/2716600322 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f758c1f0240 con 0x7f758c14da00 2024-09-13T17:07:25.373 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:25.373 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":136,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":136}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:25.373 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 136 2024-09-13T17:07:25.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7570066690 msgr2=0x7f7570068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7570066690 0x7f7570068b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7580009ab0 tx=0x7f7580009240 comp rx=0 tx=0).stop 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c14da00 msgr2=0x7f758c0a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c14da00 0x7f758c0a9e50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f758400c7a0 tx=0x7f758400cc60 comp rx=0 tx=0).stop 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 shutdown_connections 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7570066690 0x7f7570068b50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f758c14da00 0x7f758c0a9e50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f758c0c2e60 0x7f758c1f0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 --2- 172.21.15.39:0/2716600322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f758c06b9f0 0x7f758c099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 >> 172.21.15.39:0/2716600322 conn(0x7f758c06a6a0 msgr2=0x7f758c14cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 shutdown_connections 2024-09-13T17:07:25.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:25.378+0000 7f7592f98640 1 -- 172.21.15.39:0/2716600322 wait complete. 2024-09-13T17:07:25.397 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:07:26.680 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T17:07:26.682+0000 7f7ce11d1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:07:27.397 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:27.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f533d3c6640 1 Processor -- start 2024-09-13T17:07:27.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f533d3c6640 1 -- start start 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f533d3c6640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f533814e0c0 0x7f53380a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f533d3c6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f53380c1f50 0x7f533806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f533d3c6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53380c2780 con 0x7f533806d300 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f533d3c6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53380c2a60 con 0x7f53380c1f50 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f533d3c6640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53380c2d40 con 0x7f533814e0c0 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f53367fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f53380c1f50 0x7f533806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f53367fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f53380c1f50 0x7f533806cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45526/0 (socket says 172.21.15.39:45526) 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f53367fc640 1 -- 172.21.15.39:0/3752225163 learned_addr learned my addr 172.21.15.39:0/3752225163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f5336ffd640 1 --2- 172.21.15.39:0/3752225163 >> v2:172.21.15.84:3300/0 conn(0x7f533814e0c0 0x7f53380a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f53367fc640 1 -- 172.21.15.39:0/3752225163 >> v2:172.21.15.84:3300/0 conn(0x7f533814e0c0 msgr2=0x7f53380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f53367fc640 1 --2- 172.21.15.39:0/3752225163 >> v2:172.21.15.84:3300/0 conn(0x7f533814e0c0 0x7f53380a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:27.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f53367fc640 1 -- 172.21.15.39:0/3752225163 >> v1:172.21.15.84:6789/0 conn(0x7f533806d300 legacy=0x7f53380c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:27.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.534+0000 7f53367fc640 1 -- 172.21.15.39:0/3752225163 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53380c3020 con 0x7f53380c1f50 2024-09-13T17:07:27.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f53367fc640 1 --2- 172.21.15.39:0/3752225163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53380c1f50 0x7f533806cdc0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f53380c3370 tx=0x7f532c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c456c5b92c3471 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:27.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f5335ffb640 1 -- 172.21.15.39:0/3752225163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c0ab600 con 0x7f53380c1f50 2024-09-13T17:07:27.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f5335ffb640 1 -- 172.21.15.39:0/3752225163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c0a7050 con 0x7f53380c1f50 2024-09-13T17:07:27.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f5335ffb640 1 -- 172.21.15.39:0/3752225163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c0ab980 con 0x7f53380c1f50 2024-09-13T17:07:27.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 -- 172.21.15.39:0/3752225163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53380c1f50 msgr2=0x7f533806cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:27.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 --2- 172.21.15.39:0/3752225163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53380c1f50 0x7f533806cdc0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f53380c3370 tx=0x7f532c01c420 comp rx=0 tx=0).stop 2024-09-13T17:07:27.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 -- 172.21.15.39:0/3752225163 shutdown_connections 2024-09-13T17:07:27.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 --2- 172.21.15.39:0/3752225163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53380c1f50 0x7f533806cdc0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:27.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 --2- 172.21.15.39:0/3752225163 >> v2:172.21.15.84:3300/0 conn(0x7f533814e0c0 0x7f53380a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:27.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 -- 172.21.15.39:0/3752225163 >> 172.21.15.39:0/3752225163 conn(0x7f5338069e10 msgr2=0x7f533806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:27.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 -- 172.21.15.39:0/3752225163 shutdown_connections 2024-09-13T17:07:27.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 -- 172.21.15.39:0/3752225163 wait complete. 2024-09-13T17:07:27.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 Processor -- start 2024-09-13T17:07:27.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 -- start start 2024-09-13T17:07:27.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f533806d300 0x7f533814ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:27.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53380c1f50 0x7f53381eb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:27.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f533814e0c0 0x7f53381ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:27.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533813f850 con 0x7f533814e0c0 2024-09-13T17:07:27.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533814b270 con 0x7f53380c1f50 2024-09-13T17:07:27.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f533d3c6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53381eb2a0 con 0x7f533806d300 2024-09-13T17:07:27.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f53377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f533814e0c0 0x7f53381ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:27.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f53377fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f533814e0c0 0x7f53381ebe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45540/0 (socket says 172.21.15.39:45540) 2024-09-13T17:07:27.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f53377fe640 1 -- 172.21.15.39:0/1938847542 learned_addr learned my addr 172.21.15.39:0/1938847542 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f53367fc640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53380c1f50 0x7f53381eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f5336ffd640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f533806d300 0x7f533814ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f53377fe640 1 -- 172.21.15.39:0/1938847542 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f533806d300 msgr2=0x7f533814ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.538+0000 7f53377fe640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f533806d300 0x7f533814ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f53377fe640 1 -- 172.21.15.39:0/1938847542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53380c1f50 msgr2=0x7f53381eb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f53377fe640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53380c1f50 0x7f53381eb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f53377fe640 1 -- 172.21.15.39:0/1938847542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c0094d0 con 0x7f533814e0c0 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f53367fc640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53380c1f50 0x7f53381eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f53377fe640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f533814e0c0 0x7f53381ebe80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f533000c7b0 tx=0x7f533000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f531f7fe640 1 -- 172.21.15.39:0/1938847542 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330004b70 con 0x7f533814e0c0 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f531f7fe640 1 -- 172.21.15.39:0/1938847542 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5330004cd0 con 0x7f533814e0c0 2024-09-13T17:07:27.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533813fa60 con 0x7f533814e0c0 2024-09-13T17:07:27.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f531f7fe640 1 -- 172.21.15.39:0/1938847542 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533000aa90 con 0x7f533814e0c0 2024-09-13T17:07:27.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5338140020 con 0x7f533814e0c0 2024-09-13T17:07:27.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5338140210 con 0x7f533814e0c0 2024-09-13T17:07:27.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f531f7fe640 1 -- 172.21.15.39:0/1938847542 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f533000abf0 con 0x7f533814e0c0 2024-09-13T17:07:27.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f531f7fe640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:27.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f531f7fe640 1 -- 172.21.15.39:0/1938847542 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7f5330085610 con 0x7f533814e0c0 2024-09-13T17:07:27.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f5336ffd640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:27.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.542+0000 7f5336ffd640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5338072430 tx=0x7f5328008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:27.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.546+0000 7f531f7fe640 1 -- 172.21.15.39:0/1938847542 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5330051e30 con 0x7f533814e0c0 2024-09-13T17:07:27.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.750+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f533815be30 con 0x7f533814e0c0 2024-09-13T17:07:27.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.750+0000 7f531f7fe640 1 -- 172.21.15.39:0/1938847542 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f5330055450 con 0x7f533814e0c0 2024-09-13T17:07:27.749 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:27.749 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":136,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":136}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:27.749 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 136 2024-09-13T17:07:27.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 msgr2=0x7f5324068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:27.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5338072430 tx=0x7f5328008040 comp rx=0 tx=0).stop 2024-09-13T17:07:27.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f533814e0c0 msgr2=0x7f53381ebe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:27.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f533814e0c0 0x7f53381ebe80 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f533000c7b0 tx=0x7f533000cc70 comp rx=0 tx=0).stop 2024-09-13T17:07:27.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 shutdown_connections 2024-09-13T17:07:27.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f53240666c0 0x7f5324068b80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:27.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f53380c1f50 0x7f53381eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:27.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f533814e0c0 0x7f53381ebe80 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:27.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 --2- 172.21.15.39:0/1938847542 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f533806d300 0x7f533814ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:27.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 >> 172.21.15.39:0/1938847542 conn(0x7f5338069e10 msgr2=0x7f53380a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:27.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 shutdown_connections 2024-09-13T17:07:27.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:27.754+0000 7f533d3c6640 1 -- 172.21.15.39:0/1938847542 wait complete. 2024-09-13T17:07:27.768 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:07:29.532 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:07:29.534 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:07:29.768 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd00b453640 1 Processor -- start 2024-09-13T17:07:29.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd00b453640 1 -- start start 2024-09-13T17:07:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd00b453640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd00414f540 0x7fd0040c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd00b453640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd0040c1ab0 0x7fd00406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd00b453640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00414f920 con 0x7fd00406b360 2024-09-13T17:07:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd00b453640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0040709c0 con 0x7fd0040c1ab0 2024-09-13T17:07:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd00b453640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0040c1780 con 0x7fd00414f540 2024-09-13T17:07:29.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd0099f0640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd00406b360 0x7fd004155200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33328/0 (socket says 172.21.15.39:33328) 2024-09-13T17:07:29.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd0089ee640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd0040c1ab0 0x7fd00406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:29.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd0099f0640 1 -- 172.21.15.39:0/2108039721 learned_addr learned my addr 172.21.15.39:0/2108039721 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:07:29.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd0091ef640 1 --2- 172.21.15.39:0/2108039721 >> v2:172.21.15.84:3300/0 conn(0x7fd00414f540 0x7fd0040c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:29.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd0089ee640 1 -- 172.21.15.39:0/2108039721 >> v2:172.21.15.84:3300/0 conn(0x7fd00414f540 msgr2=0x7fd0040c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:29.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd0089ee640 1 --2- 172.21.15.39:0/2108039721 >> v2:172.21.15.84:3300/0 conn(0x7fd00414f540 0x7fd0040c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:29.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd0089ee640 1 -- 172.21.15.39:0/2108039721 >> v1:172.21.15.39:6789/0 conn(0x7fd00406b360 legacy=0x7fd004155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:29.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.905+0000 7fd0089ee640 1 -- 172.21.15.39:0/2108039721 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004071580 con 0x7fd0040c1ab0 2024-09-13T17:07:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fd0089ee640 1 --2- 172.21.15.39:0/2108039721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0040c1ab0 0x7fd00406ae20 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd0040c4830 tx=0x7fcff401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5578171f288cea3a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fcffbfff640 1 -- 172.21.15.39:0/2108039721 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff40ab600 con 0x7fd0040c1ab0 2024-09-13T17:07:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fcffbfff640 1 -- 172.21.15.39:0/2108039721 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff40a7050 con 0x7fd0040c1ab0 2024-09-13T17:07:29.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fcffbfff640 1 -- 172.21.15.39:0/2108039721 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff40ab980 con 0x7fd0040c1ab0 2024-09-13T17:07:29.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fd00b453640 1 -- 172.21.15.39:0/2108039721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0040c1ab0 msgr2=0x7fd00406ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fd00b453640 1 --2- 172.21.15.39:0/2108039721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0040c1ab0 0x7fd00406ae20 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd0040c4830 tx=0x7fcff401c310 comp rx=0 tx=0).stop 2024-09-13T17:07:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fd00b453640 1 -- 172.21.15.39:0/2108039721 shutdown_connections 2024-09-13T17:07:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fd00b453640 1 --2- 172.21.15.39:0/2108039721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0040c1ab0 0x7fd00406ae20 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fd00b453640 1 --2- 172.21.15.39:0/2108039721 >> v2:172.21.15.84:3300/0 conn(0x7fd00414f540 0x7fd0040c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fd00b453640 1 -- 172.21.15.39:0/2108039721 >> 172.21.15.39:0/2108039721 conn(0x7fd0040c32a0 msgr2=0x7fd0040c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fd00b453640 1 -- 172.21.15.39:0/2108039721 shutdown_connections 2024-09-13T17:07:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.909+0000 7fd00b453640 1 -- 172.21.15.39:0/2108039721 wait complete. 2024-09-13T17:07:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 Processor -- start 2024-09-13T17:07:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 -- start start 2024-09-13T17:07:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00406b360 0x7fd004144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0040c1ab0 0x7fd00406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd00414f540 0x7fd0041dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004144a90 con 0x7fd00406b360 2024-09-13T17:07:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004145460 con 0x7fd0040c1ab0 2024-09-13T17:07:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00406e650 con 0x7fd00414f540 2024-09-13T17:07:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0091ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00406b360 0x7fd004144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0091ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00406b360 0x7fd004144f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45550/0 (socket says 172.21.15.39:45550) 2024-09-13T17:07:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0091ef640 1 -- 172.21.15.39:0/637578667 learned_addr learned my addr 172.21.15.39:0/637578667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0089ee640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0040c1ab0 0x7fd00406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0099f0640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd00414f540 0x7fd0041dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0091ef640 1 -- 172.21.15.39:0/637578667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd00414f540 msgr2=0x7fd0041dbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0091ef640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd00414f540 0x7fd0041dbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0091ef640 1 -- 172.21.15.39:0/637578667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0040c1ab0 msgr2=0x7fd00406d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0091ef640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0040c1ab0 0x7fd00406d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0091ef640 1 -- 172.21.15.39:0/637578667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff4009420 con 0x7fd00406b360 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0089ee640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0040c1ab0 0x7fd00406d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd0091ef640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00406b360 0x7fd004144f20 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e870 tx=0x7fd00000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fcff9ffb640 1 -- 172.21.15.39:0/637578667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000009510 con 0x7fd00406b360 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fcff9ffb640 1 -- 172.21.15.39:0/637578667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00000eec0 con 0x7fd00406b360 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fcff9ffb640 1 -- 172.21.15.39:0/637578667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000cb40 con 0x7fd00406b360 2024-09-13T17:07:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00406e090 con 0x7fd00406b360 2024-09-13T17:07:29.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd004144720 con 0x7fd00406b360 2024-09-13T17:07:29.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0040619c0 con 0x7fd00406b360 2024-09-13T17:07:29.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.913+0000 7fcff9ffb640 1 -- 172.21.15.39:0/637578667 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd00000cca0 con 0x7fd00406b360 2024-09-13T17:07:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.917+0000 7fcff9ffb640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfe8066690 0x7fcfe8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.917+0000 7fcff9ffb640 1 -- 172.21.15.39:0/637578667 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7fd000084d90 con 0x7fd00406b360 2024-09-13T17:07:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.917+0000 7fd0089ee640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfe8066690 0x7fcfe8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.917+0000 7fd0089ee640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfe8066690 0x7fcfe8068b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcff40ad000 tx=0x7fcff4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:29.921+0000 7fcff9ffb640 1 -- 172.21.15.39:0/637578667 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd0000515e0 con 0x7fd00406b360 2024-09-13T17:07:30.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.117+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0040c2aa0 con 0x7fd00406b360 2024-09-13T17:07:30.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.117+0000 7fcff9ffb640 1 -- 172.21.15.39:0/637578667 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+5941 (secure 0 0 0) 0x7fd000017090 con 0x7fd00406b360 2024-09-13T17:07:30.117 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:30.117 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":136,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":136}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:30.118 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 136 2024-09-13T17:07:30.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfe8066690 msgr2=0x7fcfe8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfe8066690 0x7fcfe8068b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcff40ad000 tx=0x7fcff4008040 comp rx=0 tx=0).stop 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00406b360 msgr2=0x7fd004144f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00406b360 0x7fd004144f20 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e870 tx=0x7fd00000ec70 comp rx=0 tx=0).stop 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 shutdown_connections 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfe8066690 0x7fcfe8068b50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0040c1ab0 0x7fd00406d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd00414f540 0x7fd0041dbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 --2- 172.21.15.39:0/637578667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00406b360 0x7fd004144f20 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 >> 172.21.15.39:0/637578667 conn(0x7fd0040c32a0 msgr2=0x7fd0040c53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 shutdown_connections 2024-09-13T17:07:30.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:30.121+0000 7fd00b453640 1 -- 172.21.15.39:0/637578667 wait complete. 2024-09-13T17:07:30.136 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:07:32.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:32.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa4a9cb640 1 Processor -- start 2024-09-13T17:07:32.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa4a9cb640 1 -- start start 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa4a9cb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faa4414bf30 0x7faa4406ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa4a9cb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faa4406b360 0x7faa44152800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa4a9cb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa44151bf0 con 0x7faa44155160 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa4a9cb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa44155660 con 0x7faa4414bf30 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa4a9cb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa44155940 con 0x7faa4406b360 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa43fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faa4414bf30 0x7faa4406ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa43fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faa4414bf30 0x7faa4406ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47692/0 (socket says 172.21.15.39:47692) 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa43fff640 1 -- 172.21.15.39:0/3216403248 learned_addr learned my addr 172.21.15.39:0/3216403248 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa3bfff640 1 --1- 172.21.15.39:0/3216403248 >> v1:172.21.15.39:6789/0 conn(0x7faa44155160 0x7faa441520f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56596/0 (socket says 172.21.15.39:56596) 2024-09-13T17:07:32.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa48f68640 1 --2- 172.21.15.39:0/3216403248 >> v2:172.21.15.84:3300/0 conn(0x7faa4406b360 0x7faa44152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:32.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa43fff640 1 -- 172.21.15.39:0/3216403248 >> v2:172.21.15.84:3300/0 conn(0x7faa4406b360 msgr2=0x7faa44152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:32.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa43fff640 1 --2- 172.21.15.39:0/3216403248 >> v2:172.21.15.84:3300/0 conn(0x7faa4406b360 0x7faa44152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa43fff640 1 -- 172.21.15.39:0/3216403248 >> v1:172.21.15.39:6789/0 conn(0x7faa44155160 legacy=0x7faa441520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:32.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa437fe640 1 -- 172.21.15.39:0/3216403248 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239887706 0 0) 0x7faa44151bf0 con 0x7faa44155160 2024-09-13T17:07:32.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.273+0000 7faa43fff640 1 -- 172.21.15.39:0/3216403248 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa44155c20 con 0x7faa4414bf30 2024-09-13T17:07:32.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa43fff640 1 --2- 172.21.15.39:0/3216403248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4414bf30 0x7faa4406ae20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7faa4406bfd0 tx=0x7faa3401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247547d68daa3033 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:32.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa437fe640 1 -- 172.21.15.39:0/3216403248 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa340a5860 con 0x7faa4414bf30 2024-09-13T17:07:32.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa437fe640 1 -- 172.21.15.39:0/3216403248 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa340a59c0 con 0x7faa4414bf30 2024-09-13T17:07:32.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa437fe640 1 -- 172.21.15.39:0/3216403248 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa340a5d50 con 0x7faa4414bf30 2024-09-13T17:07:32.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 -- 172.21.15.39:0/3216403248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4414bf30 msgr2=0x7faa4406ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:32.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 --2- 172.21.15.39:0/3216403248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4414bf30 0x7faa4406ae20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7faa4406bfd0 tx=0x7faa3401c9c0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 -- 172.21.15.39:0/3216403248 shutdown_connections 2024-09-13T17:07:32.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 --2- 172.21.15.39:0/3216403248 >> v2:172.21.15.84:3300/0 conn(0x7faa4406b360 0x7faa44152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 --2- 172.21.15.39:0/3216403248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4414bf30 0x7faa4406ae20 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 -- 172.21.15.39:0/3216403248 >> 172.21.15.39:0/3216403248 conn(0x7faa440c32a0 msgr2=0x7faa440c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:32.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 -- 172.21.15.39:0/3216403248 shutdown_connections 2024-09-13T17:07:32.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 -- 172.21.15.39:0/3216403248 wait complete. 2024-09-13T17:07:32.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 Processor -- start 2024-09-13T17:07:32.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 -- start start 2024-09-13T17:07:32.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4406b360 0x7faa4414b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa4414bf30 0x7faa44144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa44155160 0x7faa441f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440715c0 con 0x7faa4406b360 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44070d20 con 0x7faa4414bf30 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.277+0000 7faa4a9cb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa44071000 con 0x7faa44155160 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa43fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4406b360 0x7faa4414b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa43fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4406b360 0x7faa4414b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47706/0 (socket says 172.21.15.39:47706) 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa43fff640 1 -- 172.21.15.39:0/679060953 learned_addr learned my addr 172.21.15.39:0/679060953 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa3bfff640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa4414bf30 0x7faa44144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa48f68640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa44155160 0x7faa441f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa43fff640 1 -- 172.21.15.39:0/679060953 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa44155160 msgr2=0x7faa441f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:32.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa43fff640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa44155160 0x7faa441f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa43fff640 1 -- 172.21.15.39:0/679060953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa4414bf30 msgr2=0x7faa44144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa43fff640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa4414bf30 0x7faa44144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa43fff640 1 -- 172.21.15.39:0/679060953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa34009b30 con 0x7faa4406b360 2024-09-13T17:07:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa43fff640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4406b360 0x7faa4414b420 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7faa3401b070 tx=0x7faa34019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa417fa640 1 -- 172.21.15.39:0/679060953 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa340b8670 con 0x7faa4406b360 2024-09-13T17:07:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa417fa640 1 -- 172.21.15.39:0/679060953 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa340b87d0 con 0x7faa4406b360 2024-09-13T17:07:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa440712e0 con 0x7faa4406b360 2024-09-13T17:07:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa417fa640 1 -- 172.21.15.39:0/679060953 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa340b8b40 con 0x7faa4406b360 2024-09-13T17:07:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa440c0010 con 0x7faa4406b360 2024-09-13T17:07:32.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa441f0360 con 0x7faa4406b360 2024-09-13T17:07:32.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa417fa640 1 -- 172.21.15.39:0/679060953 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa340b8ca0 con 0x7faa4406b360 2024-09-13T17:07:32.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa417fa640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa24066690 0x7faa24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:32.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa417fa640 1 -- 172.21.15.39:0/679060953 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7faa34120440 con 0x7faa4406b360 2024-09-13T17:07:32.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa3bfff640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa24066690 0x7faa24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:32.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.281+0000 7faa3bfff640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa24066690 0x7faa24068b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7faa4406df40 tx=0x7faa2c007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:32.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.285+0000 7faa417fa640 1 -- 172.21.15.39:0/679060953 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faa441f0360 con 0x7faa4406b360 2024-09-13T17:07:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.481+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa441f0360 con 0x7faa4406b360 2024-09-13T17:07:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa417fa640 1 -- 172.21.15.39:0/679060953 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+5941 (secure 0 0 0) 0x7faa441f0360 con 0x7faa4406b360 2024-09-13T17:07:32.483 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:32.483 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":136,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":136}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:06:02.851726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5561},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5561":{"gid":5561,"name":"b","rank":0,"incarnation":126,"state":"up:active","state_seq":69,"addr":"172.21.15.84:6833/440028459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":440028459},{"type":"v1","addr":"172.21.15.84:6833","nonce":440028459}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 136 2024-09-13T17:07:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa24066690 msgr2=0x7faa24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa24066690 0x7faa24068b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7faa4406df40 tx=0x7faa2c007b60 comp rx=0 tx=0).stop 2024-09-13T17:07:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4406b360 msgr2=0x7faa4414b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4406b360 0x7faa4414b420 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7faa3401b070 tx=0x7faa34019760 comp rx=0 tx=0).stop 2024-09-13T17:07:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 shutdown_connections 2024-09-13T17:07:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7faa24066690 0x7faa24068b50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa4414bf30 0x7faa44144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7faa44155160 0x7faa441f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 --2- 172.21.15.39:0/679060953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa4406b360 0x7faa4414b420 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:32.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 >> 172.21.15.39:0/679060953 conn(0x7faa440c32a0 msgr2=0x7faa440a7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:32.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 shutdown_connections 2024-09-13T17:07:32.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:32.485+0000 7faa4a9cb640 1 -- 172.21.15.39:0/679060953 wait complete. 2024-09-13T17:07:32.502 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:07:34.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:34.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.641+0000 7f33ca885640 1 Processor -- start 2024-09-13T17:07:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.641+0000 7f33ca885640 1 -- start start 2024-09-13T17:07:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33ca885640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33c414f1b0 0x7f33c4151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33ca885640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c406e5e0 0x7f33c40714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33ca885640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c414f590 con 0x7f33c40c7470 2024-09-13T17:07:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33ca885640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c406f660 con 0x7f33c414f1b0 2024-09-13T17:07:34.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33ca885640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c406eae0 con 0x7f33c406e5e0 2024-09-13T17:07:34.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c406e5e0 0x7f33c40714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:34.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c406e5e0 0x7f33c40714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:33036/0 (socket says 172.21.15.39:33036) 2024-09-13T17:07:34.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33bbfff640 1 -- 172.21.15.39:0/63779135 learned_addr learned my addr 172.21.15.39:0/63779135 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:34.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33c3fff640 1 --2- 172.21.15.39:0/63779135 >> v2:172.21.15.84:3300/0 conn(0x7f33c414f1b0 0x7f33c4151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:34.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33bbfff640 1 -- 172.21.15.39:0/63779135 >> v2:172.21.15.84:3300/0 conn(0x7f33c414f1b0 msgr2=0x7f33c4151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:34.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33bbfff640 1 --2- 172.21.15.39:0/63779135 >> v2:172.21.15.84:3300/0 conn(0x7f33c414f1b0 0x7f33c4151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:34.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33bbfff640 1 -- 172.21.15.39:0/63779135 >> v1:172.21.15.84:6789/0 conn(0x7f33c40c7470 legacy=0x7f33c40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:34.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.645+0000 7f33bbfff640 1 -- 172.21.15.39:0/63779135 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c406edc0 con 0x7f33c406e5e0 2024-09-13T17:07:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33bbfff640 1 --2- 172.21.15.39:0/63779135 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c406e5e0 0x7f33c40714e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009870 tx=0x7f33ac01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4cce3e65be4b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33c37fe640 1 -- 172.21.15.39:0/63779135 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0a95f0 con 0x7f33c406e5e0 2024-09-13T17:07:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33c37fe640 1 -- 172.21.15.39:0/63779135 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac0a9750 con 0x7f33c406e5e0 2024-09-13T17:07:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33c37fe640 1 -- 172.21.15.39:0/63779135 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0a9ae0 con 0x7f33c406e5e0 2024-09-13T17:07:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 -- 172.21.15.39:0/63779135 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c406e5e0 msgr2=0x7f33c40714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 --2- 172.21.15.39:0/63779135 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c406e5e0 0x7f33c40714e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009870 tx=0x7f33ac01c530 comp rx=0 tx=0).stop 2024-09-13T17:07:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 -- 172.21.15.39:0/63779135 shutdown_connections 2024-09-13T17:07:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 --2- 172.21.15.39:0/63779135 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c406e5e0 0x7f33c40714e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 --2- 172.21.15.39:0/63779135 >> v2:172.21.15.84:3300/0 conn(0x7f33c414f1b0 0x7f33c4151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 -- 172.21.15.39:0/63779135 >> 172.21.15.39:0/63779135 conn(0x7f33c40a9720 msgr2=0x7f33c40a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 -- 172.21.15.39:0/63779135 shutdown_connections 2024-09-13T17:07:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 -- 172.21.15.39:0/63779135 wait complete. 2024-09-13T17:07:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 Processor -- start 2024-09-13T17:07:34.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 -- start start 2024-09-13T17:07:34.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33c406e5e0 0x7f33c415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:34.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c40c7470 0x7f33c40ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:34.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c414f1b0 0x7f33c41ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:34.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c41edf70 con 0x7f33c40c7470 2024-09-13T17:07:34.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4159e80 con 0x7f33c406e5e0 2024-09-13T17:07:34.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33ca885640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c41edc90 con 0x7f33c414f1b0 2024-09-13T17:07:34.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c40c7470 0x7f33c40ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c40c7470 0x7f33c40ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47708/0 (socket says 172.21.15.39:47708) 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33bbfff640 1 -- 172.21.15.39:0/585234419 learned_addr learned my addr 172.21.15.39:0/585234419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33c8e22640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c414f1b0 0x7f33c41ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.649+0000 7f33c3fff640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33c406e5e0 0x7f33c415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33bbfff640 1 -- 172.21.15.39:0/585234419 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c414f1b0 msgr2=0x7f33c41ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33bbfff640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c414f1b0 0x7f33c41ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33bbfff640 1 -- 172.21.15.39:0/585234419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33c406e5e0 msgr2=0x7f33c415b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33bbfff640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33c406e5e0 0x7f33c415b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33bbfff640 1 -- 172.21.15.39:0/585234419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0094d0 con 0x7f33c40c7470 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33bbfff640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c40c7470 0x7f33c40ace60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f33ac01a160 tx=0x7f33ac01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33c17fa640 1 -- 172.21.15.39:0/585234419 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0b1800 con 0x7f33c40c7470 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33c17fa640 1 -- 172.21.15.39:0/585234419 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac0b1980 con 0x7f33c40c7470 2024-09-13T17:07:34.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c415a690 con 0x7f33c40c7470 2024-09-13T17:07:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33c17fa640 1 -- 172.21.15.39:0/585234419 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0b1d30 con 0x7f33c40c7470 2024-09-13T17:07:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c415ac50 con 0x7f33c40c7470 2024-09-13T17:07:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c415a4d0 con 0x7f33c40c7470 2024-09-13T17:07:34.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33c17fa640 1 -- 172.21.15.39:0/585234419 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f33ac0bc450 con 0x7f33c40c7470 2024-09-13T17:07:34.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33c17fa640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33a4066690 0x7f33a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:34.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33c17fa640 1 -- 172.21.15.39:0/585234419 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 8787+0+0 (secure 0 0 0) 0x7f33ac11f770 con 0x7f33c40c7470 2024-09-13T17:07:34.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33c3fff640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33a4066690 0x7f33a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:34.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.653+0000 7f33c3fff640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33a4066690 0x7f33a4068b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009a50 tx=0x7f33b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:34.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.657+0000 7f33c17fa640 1 -- 172.21.15.39:0/585234419 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f33ac0ebd70 con 0x7f33c40c7470 2024-09-13T17:07:34.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.869+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33c415ae40 con 0x7f33c40c7470 2024-09-13T17:07:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.869+0000 7f33c17fa640 1 -- 172.21.15.39:0/585234419 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+5185 (secure 0 0 0) 0x7f33ac0ef390 con 0x7f33c40c7470 2024-09-13T17:07:34.870 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:34.870 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":137,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:34.802433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:34.870 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 137 2024-09-13T17:07:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33a4066690 msgr2=0x7f33a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33a4066690 0x7f33a4068b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009a50 tx=0x7f33b4008040 comp rx=0 tx=0).stop 2024-09-13T17:07:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c40c7470 msgr2=0x7f33c40ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c40c7470 0x7f33c40ace60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f33ac01a160 tx=0x7f33ac01ad70 comp rx=0 tx=0).stop 2024-09-13T17:07:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 shutdown_connections 2024-09-13T17:07:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f33a4066690 0x7f33a4068b50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:34.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f33c414f1b0 0x7f33c41ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33c40c7470 0x7f33c40ace60 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 --2- 172.21.15.39:0/585234419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33c406e5e0 0x7f33c415b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 >> 172.21.15.39:0/585234419 conn(0x7f33c40a9720 msgr2=0x7f33c414cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 shutdown_connections 2024-09-13T17:07:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:34.873+0000 7f33ca885640 1 -- 172.21.15.39:0/585234419 wait complete. 2024-09-13T17:07:34.893 INFO:tasks.mds_thrash.fs.[a]:mds.b down, removed from mdsmap 2024-09-13T17:07:34.894 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:07:34.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:34.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee755b1640 1 Processor -- start 2024-09-13T17:07:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee755b1640 1 -- start start 2024-09-13T17:07:35.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee755b1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fee70071dd0 0x7fee70154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:35.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee755b1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee700c7800 0x7fee701545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee755b1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee700c5a30 con 0x7fee70151500 2024-09-13T17:07:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee755b1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee701576a0 con 0x7fee70071dd0 2024-09-13T17:07:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee755b1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee70157980 con 0x7fee700c7800 2024-09-13T17:07:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee6f7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee700c7800 0x7fee701545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee6f7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee700c7800 0x7fee701545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:33050/0 (socket says 172.21.15.39:33050) 2024-09-13T17:07:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee6f7fe640 1 -- 172.21.15.39:0/4065557833 learned_addr learned my addr 172.21.15.39:0/4065557833 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee6e7fc640 1 --2- 172.21.15.39:0/4065557833 >> v2:172.21.15.84:3300/0 conn(0x7fee70071dd0 0x7fee70154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee6f7fe640 1 -- 172.21.15.39:0/4065557833 >> v2:172.21.15.84:3300/0 conn(0x7fee70071dd0 msgr2=0x7fee70154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:35.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee6f7fe640 1 --2- 172.21.15.39:0/4065557833 >> v2:172.21.15.84:3300/0 conn(0x7fee70071dd0 0x7fee70154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee6f7fe640 1 -- 172.21.15.39:0/4065557833 >> v1:172.21.15.84:6789/0 conn(0x7fee70151500 legacy=0x7fee700c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:35.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee6f7fe640 1 -- 172.21.15.39:0/4065557833 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee70157c60 con 0x7fee700c7800 2024-09-13T17:07:35.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.001+0000 7fee6e7fc640 1 --2- 172.21.15.39:0/4065557833 >> v2:172.21.15.84:3300/0 conn(0x7fee70071dd0 0x7fee70154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:07:35.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee6f7fe640 1 --2- 172.21.15.39:0/4065557833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee700c7800 0x7fee701545b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee68009e80 tx=0x7fee6801d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cdf49031aea51f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:35.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee6dffb640 1 -- 172.21.15.39:0/4065557833 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680a5830 con 0x7fee700c7800 2024-09-13T17:07:35.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee6dffb640 1 -- 172.21.15.39:0/4065557833 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee680a5990 con 0x7fee700c7800 2024-09-13T17:07:35.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee6dffb640 1 -- 172.21.15.39:0/4065557833 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680a5d20 con 0x7fee700c7800 2024-09-13T17:07:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 -- 172.21.15.39:0/4065557833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee700c7800 msgr2=0x7fee701545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 --2- 172.21.15.39:0/4065557833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee700c7800 0x7fee701545b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee68009e80 tx=0x7fee6801d3b0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 -- 172.21.15.39:0/4065557833 shutdown_connections 2024-09-13T17:07:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 --2- 172.21.15.39:0/4065557833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee700c7800 0x7fee701545b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 --2- 172.21.15.39:0/4065557833 >> v2:172.21.15.84:3300/0 conn(0x7fee70071dd0 0x7fee70154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 -- 172.21.15.39:0/4065557833 >> 172.21.15.39:0/4065557833 conn(0x7fee700ab530 msgr2=0x7fee700ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 -- 172.21.15.39:0/4065557833 shutdown_connections 2024-09-13T17:07:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 -- 172.21.15.39:0/4065557833 wait complete. 2024-09-13T17:07:35.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 Processor -- start 2024-09-13T17:07:35.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.005+0000 7fee755b1640 1 -- start start 2024-09-13T17:07:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee755b1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee70071dd0 0x7fee7014e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee755b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee700c7800 0x7fee701532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee755b1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee70151500 0x7fee701efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee755b1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70151f40 con 0x7fee70071dd0 2024-09-13T17:07:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee755b1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70152b30 con 0x7fee700c7800 2024-09-13T17:07:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee755b1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee70156980 con 0x7fee70151500 2024-09-13T17:07:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee70071dd0 0x7fee7014e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee70071dd0 0x7fee7014e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47710/0 (socket says 172.21.15.39:47710) 2024-09-13T17:07:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6effd640 1 -- 172.21.15.39:0/2136418844 learned_addr learned my addr 172.21.15.39:0/2136418844 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6f7fe640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee70151500 0x7fee701efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6e7fc640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee700c7800 0x7fee701532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6effd640 1 -- 172.21.15.39:0/2136418844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee70151500 msgr2=0x7fee701efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6effd640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee70151500 0x7fee701efb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6effd640 1 -- 172.21.15.39:0/2136418844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee700c7800 msgr2=0x7fee701532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6effd640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee700c7800 0x7fee701532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6effd640 1 -- 172.21.15.39:0/2136418844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64009420 con 0x7fee70071dd0 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee6effd640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee70071dd0 0x7fee7014e140 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fee6000c3a0 tx=0x7fee6000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee577fe640 1 -- 172.21.15.39:0/2136418844 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee600052d0 con 0x7fee70071dd0 2024-09-13T17:07:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee577fe640 1 -- 172.21.15.39:0/2136418844 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee60005450 con 0x7fee70071dd0 2024-09-13T17:07:35.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee68009b30 con 0x7fee70071dd0 2024-09-13T17:07:35.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee577fe640 1 -- 172.21.15.39:0/2136418844 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60003860 con 0x7fee70071dd0 2024-09-13T17:07:35.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee70152770 con 0x7fee70071dd0 2024-09-13T17:07:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee700c6290 con 0x7fee70071dd0 2024-09-13T17:07:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee577fe640 1 -- 172.21.15.39:0/2136418844 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee600039c0 con 0x7fee70071dd0 2024-09-13T17:07:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.009+0000 7fee577fe640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee5c066690 0x7fee5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.013+0000 7fee6e7fc640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee5c066690 0x7fee5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.013+0000 7fee577fe640 1 -- 172.21.15.39:0/2136418844 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7fee600842d0 con 0x7fee70071dd0 2024-09-13T17:07:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.013+0000 7fee6e7fc640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee5c066690 0x7fee5c068b50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fee7015e260 tx=0x7fee64009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:35.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.017+0000 7fee577fe640 1 -- 172.21.15.39:0/2136418844 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fee60050a80 con 0x7fee70071dd0 2024-09-13T17:07:35.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.221+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee700c4280 con 0x7fee70071dd0 2024-09-13T17:07:35.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.221+0000 7fee577fe640 1 -- 172.21.15.39:0/2136418844 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+5185 (secure 0 0 0) 0x7fee60017090 con 0x7fee70071dd0 2024-09-13T17:07:35.221 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:35.221 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":137,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:34.802433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:35.221 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 137 2024-09-13T17:07:35.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee5c066690 msgr2=0x7fee5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:35.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee5c066690 0x7fee5c068b50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fee7015e260 tx=0x7fee64009190 comp rx=0 tx=0).stop 2024-09-13T17:07:35.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee70071dd0 msgr2=0x7fee7014e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:35.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee70071dd0 0x7fee7014e140 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fee6000c3a0 tx=0x7fee6000c860 comp rx=0 tx=0).stop 2024-09-13T17:07:35.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 shutdown_connections 2024-09-13T17:07:35.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee5c066690 0x7fee5c068b50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee700c7800 0x7fee701532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee70151500 0x7fee701efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 --2- 172.21.15.39:0/2136418844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee70071dd0 0x7fee7014e140 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:35.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 >> 172.21.15.39:0/2136418844 conn(0x7fee700ab530 msgr2=0x7fee700c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:35.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 shutdown_connections 2024-09-13T17:07:35.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:35.225+0000 7fee755b1640 1 -- 172.21.15.39:0/2136418844 wait complete. 2024-09-13T17:07:35.240 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 137, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 137, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_7338": { "addr": "172.21.15.39:6837/2521680730", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 2521680730, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 2521680730, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7338, "incarnation": 137, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:07:34.802433+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7338 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 135, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_7230": { "addr": "172.21.15.39:6835/3941977343", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 3941977343, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 3941977343, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7230, "incarnation": 132, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 21 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:07:23.194224+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7230 } } } ], "standbys": [ { "addr": "172.21.15.84:6835/3923043380", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 3923043380, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 3923043380, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 130, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5963, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:07:37.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:37.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5bb505640 1 Processor -- start 2024-09-13T17:07:37.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5bb505640 1 -- start start 2024-09-13T17:07:37.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5bb505640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff5b40a4e70 0x7ff5b406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:37.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5bb505640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff5b406d300 0x7ff5b40c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:37.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5bb505640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b40c2780 con 0x7ff5b414e0c0 2024-09-13T17:07:37.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5bb505640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b40c2a60 con 0x7ff5b406d300 2024-09-13T17:07:37.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5bb505640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b40c2d40 con 0x7ff5b40a4e70 2024-09-13T17:07:37.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5b9aa2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff5b406d300 0x7ff5b40c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:37.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5b92a1640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff5b414e0c0 0x7ff5b40c1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56598/0 (socket says 172.21.15.39:56598) 2024-09-13T17:07:37.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5b92a1640 1 -- 172.21.15.39:0/771385919 learned_addr learned my addr 172.21.15.39:0/771385919 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:07:37.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5b8aa0640 1 --2- 172.21.15.39:0/771385919 >> v2:172.21.15.84:3300/0 conn(0x7ff5b40a4e70 0x7ff5b406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:37.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5b9aa2640 1 -- 172.21.15.39:0/771385919 >> v2:172.21.15.84:3300/0 conn(0x7ff5b40a4e70 msgr2=0x7ff5b406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:37.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5b9aa2640 1 --2- 172.21.15.39:0/771385919 >> v2:172.21.15.84:3300/0 conn(0x7ff5b40a4e70 0x7ff5b406cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5b9aa2640 1 -- 172.21.15.39:0/771385919 >> v1:172.21.15.39:6789/0 conn(0x7ff5b414e0c0 legacy=0x7ff5b40c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:37.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.373+0000 7ff5b9aa2640 1 -- 172.21.15.39:0/771385919 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b40c3020 con 0x7ff5b406d300 2024-09-13T17:07:37.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5b9aa2640 1 --2- 172.21.15.39:0/771385919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b406d300 0x7ff5b40c4740 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40c57d0 tx=0x7ff5ac01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6650ce26ee197c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:37.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5abfff640 1 -- 172.21.15.39:0/771385919 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40c5490 con 0x7ff5b406d300 2024-09-13T17:07:37.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5abfff640 1 -- 172.21.15.39:0/771385919 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac0b0600 con 0x7ff5b406d300 2024-09-13T17:07:37.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5abfff640 1 -- 172.21.15.39:0/771385919 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0b0990 con 0x7ff5b406d300 2024-09-13T17:07:37.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 -- 172.21.15.39:0/771385919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b406d300 msgr2=0x7ff5b40c4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:37.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 --2- 172.21.15.39:0/771385919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b406d300 0x7ff5b40c4740 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40c57d0 tx=0x7ff5ac01c730 comp rx=0 tx=0).stop 2024-09-13T17:07:37.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 -- 172.21.15.39:0/771385919 shutdown_connections 2024-09-13T17:07:37.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 --2- 172.21.15.39:0/771385919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b406d300 0x7ff5b40c4740 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 --2- 172.21.15.39:0/771385919 >> v2:172.21.15.84:3300/0 conn(0x7ff5b40a4e70 0x7ff5b406cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 -- 172.21.15.39:0/771385919 >> 172.21.15.39:0/771385919 conn(0x7ff5b4069e10 msgr2=0x7ff5b406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:37.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 -- 172.21.15.39:0/771385919 shutdown_connections 2024-09-13T17:07:37.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 -- 172.21.15.39:0/771385919 wait complete. 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 Processor -- start 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.377+0000 7ff5bb505640 1 -- start start 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5bb505640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5b406d300 0x7ff5b40ab300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5bb505640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b40a4e70 0x7ff5b4154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5bb505640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b414e0c0 0x7ff5b41545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5bb505640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b41449f0 con 0x7ff5b414e0c0 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5bb505640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b4152220 con 0x7ff5b40a4e70 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5bb505640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b4152870 con 0x7ff5b406d300 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b9aa2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b414e0c0 0x7ff5b41545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b9aa2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b414e0c0 0x7ff5b41545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47732/0 (socket says 172.21.15.39:47732) 2024-09-13T17:07:37.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b9aa2640 1 -- 172.21.15.39:0/754832258 learned_addr learned my addr 172.21.15.39:0/754832258 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b8aa0640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b40a4e70 0x7ff5b4154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b92a1640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5b406d300 0x7ff5b40ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b9aa2640 1 -- 172.21.15.39:0/754832258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5b406d300 msgr2=0x7ff5b40ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b9aa2640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5b406d300 0x7ff5b40ab300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b9aa2640 1 -- 172.21.15.39:0/754832258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b40a4e70 msgr2=0x7ff5b4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b9aa2640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b40a4e70 0x7ff5b4154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b9aa2640 1 -- 172.21.15.39:0/754832258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac009a70 con 0x7ff5b414e0c0 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b9aa2640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b414e0c0 0x7ff5b41545b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac01ade0 tx=0x7ff5ac0194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5a9ffb640 1 -- 172.21.15.39:0/754832258 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0b4050 con 0x7ff5b414e0c0 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5a9ffb640 1 -- 172.21.15.39:0/754832258 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac0b5040 con 0x7ff5b414e0c0 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b4151eb0 con 0x7ff5b414e0c0 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5a9ffb640 1 -- 172.21.15.39:0/754832258 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0a8800 con 0x7ff5b414e0c0 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b40ab840 con 0x7ff5b414e0c0 2024-09-13T17:07:37.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b4152390 con 0x7ff5b414e0c0 2024-09-13T17:07:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5a9ffb640 1 -- 172.21.15.39:0/754832258 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff5ac0a8960 con 0x7ff5b414e0c0 2024-09-13T17:07:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5a9ffb640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff598066690 0x7ff598068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5a9ffb640 1 -- 172.21.15.39:0/754832258 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7ff5ac120860 con 0x7ff5b414e0c0 2024-09-13T17:07:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b92a1640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff598066690 0x7ff598068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:37.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.381+0000 7ff5b92a1640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff598066690 0x7ff598068b50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0000f70 tx=0x7ff5b0007c70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:37.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.385+0000 7ff5a9ffb640 1 -- 172.21.15.39:0/754832258 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff5ac0ece90 con 0x7ff5b414e0c0 2024-09-13T17:07:37.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.581+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5b40c4740 con 0x7ff5b414e0c0 2024-09-13T17:07:37.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.581+0000 7ff5a9ffb640 1 -- 172.21.15.39:0/754832258 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+5185 (secure 0 0 0) 0x7ff5ac0f04b0 con 0x7ff5b414e0c0 2024-09-13T17:07:37.581 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:37.581 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":139,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:37.156698+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:rejoin","state_seq":5,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:37.581 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 139 2024-09-13T17:07:37.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff598066690 msgr2=0x7ff598068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:37.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff598066690 0x7ff598068b50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0000f70 tx=0x7ff5b0007c70 comp rx=0 tx=0).stop 2024-09-13T17:07:37.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b414e0c0 msgr2=0x7ff5b41545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:37.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b414e0c0 0x7ff5b41545b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac01ade0 tx=0x7ff5ac0194d0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 shutdown_connections 2024-09-13T17:07:37.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff598066690 0x7ff598068b50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5b40a4e70 0x7ff5b4154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5b414e0c0 0x7ff5b41545b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 --2- 172.21.15.39:0/754832258 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5b406d300 0x7ff5b40ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:37.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 >> 172.21.15.39:0/754832258 conn(0x7ff5b4069e10 msgr2=0x7ff5b414c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:37.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 shutdown_connections 2024-09-13T17:07:37.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:37.585+0000 7ff5bb505640 1 -- 172.21.15.39:0/754832258 wait complete. 2024-09-13T17:07:37.601 INFO:tasks.mds_thrash.fs.[a]:no change 2024-09-13T17:07:39.220 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-13T17:07:39.221+0000 7f130f341640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:07:39.601 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:39.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f258f7c3640 1 Processor -- start 2024-09-13T17:07:39.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f258f7c3640 1 -- start start 2024-09-13T17:07:39.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f258f7c3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c6060 0x7f2588150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f258f7c3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258814b000 con 0x7f258814a770 2024-09-13T17:07:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f258f7c3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25880c4760 con 0x7f258814ab50 2024-09-13T17:07:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f258f7c3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258806df80 con 0x7f25880c6060 2024-09-13T17:07:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f258d55f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f258814a770 0x7f258806ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56608/0 (socket says 172.21.15.39:56608) 2024-09-13T17:07:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f258d55f640 1 -- 172.21.15.39:0/432603955 learned_addr learned my addr 172.21.15.39:0/432603955 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:07:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938125417 0 0) 0x7f258814b000 con 0x7f258814a770 2024-09-13T17:07:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2574003530 con 0x7f258814a770 2024-09-13T17:07:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f258cd5e640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c6060 0x7f2588150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3225560067 0 0) 0x7f2574003530 con 0x7f258814a770 2024-09-13T17:07:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c6060 msgr2=0x7f2588150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c6060 0x7f2588150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 >> v1:172.21.15.84:6789/0 conn(0x7f258814ab50 legacy=0x7f25880713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588151120 con 0x7f258814a770 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f2584001a00 con 0x7f258814a770 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 >> v1:172.21.15.39:6789/0 conn(0x7f258814a770 legacy=0x7f258806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.749+0000 7f257ffff640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25740039b0 0x7f2574003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f257ffff640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f2574004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f257ffff640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c6060 0x7f257400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740054f0 con 0x7f25740047d0 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2574005660 con 0x7f25740039b0 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257400bed0 con 0x7f25880c6060 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258cd5e640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f2574004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258d55f640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25740039b0 0x7f2574003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258dd60640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c6060 0x7f257400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258cd5e640 1 -- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c6060 msgr2=0x7f257400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258cd5e640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c6060 0x7f257400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258cd5e640 1 -- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25740039b0 msgr2=0x7f2574003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:07:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258cd5e640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25740039b0 0x7f2574003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258cd5e640 1 -- 172.21.15.39:0/432603955 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574003530 con 0x7f25740047d0 2024-09-13T17:07:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258dd60640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c6060 0x7f257400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258cd5e640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f2574004c50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f257800ba40 tx=0x7f257801c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736d29c5b9f023e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780aa050 con 0x7f25740047d0 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25780ae5f0 con 0x7f25740047d0 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f257ffff640 1 -- 172.21.15.39:0/432603955 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780ae9a0 con 0x7f25740047d0 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258f7c3640 1 -- 172.21.15.39:0/432603955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 msgr2=0x7f2574004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258f7c3640 1 --2- 172.21.15.39:0/432603955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f2574004c50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f257800ba40 tx=0x7f257801c660 comp rx=0 tx=0).stop 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258dd60640 1 -- 172.21.15.39:0/432603955 reap_dead start 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258f7c3640 1 -- 172.21.15.39:0/432603955 shutdown_connections 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258f7c3640 1 -- 172.21.15.39:0/432603955 >> 172.21.15.39:0/432603955 conn(0x7f25880c32a0 msgr2=0x7f25880c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258f7c3640 1 -- 172.21.15.39:0/432603955 shutdown_connections 2024-09-13T17:07:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.753+0000 7f258f7c3640 1 -- 172.21.15.39:0/432603955 wait complete. 2024-09-13T17:07:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 Processor -- start 2024-09-13T17:07:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 -- start start 2024-09-13T17:07:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f25880bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c05b0 0x7f25880bd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25881544b0 0x7f2588154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588153980 con 0x7f25740047d0 2024-09-13T17:07:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25880bd610 con 0x7f25881544b0 2024-09-13T17:07:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2588152740 con 0x7f25880c05b0 2024-09-13T17:07:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258d55f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f25880bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258d55f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f25880bcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47752/0 (socket says 172.21.15.39:47752) 2024-09-13T17:07:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258d55f640 1 -- 172.21.15.39:0/156668132 learned_addr learned my addr 172.21.15.39:0/156668132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258cd5e640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c05b0 0x7f25880bd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258dd60640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25881544b0 0x7f2588154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258d55f640 1 -- 172.21.15.39:0/156668132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c05b0 msgr2=0x7f25880bd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258d55f640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c05b0 0x7f25880bd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258d55f640 1 -- 172.21.15.39:0/156668132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25881544b0 msgr2=0x7f2588154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258d55f640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25881544b0 0x7f2588154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258d55f640 1 -- 172.21.15.39:0/156668132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588152d00 con 0x7f25740047d0 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258d55f640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f25880bcb90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f25840079e0 tx=0x7f258400a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f257dffb640 1 -- 172.21.15.39:0/156668132 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840133f0 con 0x7f25740047d0 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f257dffb640 1 -- 172.21.15.39:0/156668132 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584013570 con 0x7f25740047d0 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25881eda70 con 0x7f25740047d0 2024-09-13T17:07:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f257dffb640 1 -- 172.21.15.39:0/156668132 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584013920 con 0x7f25740047d0 2024-09-13T17:07:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.757+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2588152990 con 0x7f25740047d0 2024-09-13T17:07:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.761+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25880619c0 con 0x7f25740047d0 2024-09-13T17:07:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.761+0000 7f257dffb640 1 -- 172.21.15.39:0/156668132 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2584013a80 con 0x7f25740047d0 2024-09-13T17:07:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.761+0000 7f257dffb640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f256c0666c0 0x7f256c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.761+0000 7f257dffb640 1 -- 172.21.15.39:0/156668132 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7f2584085ac0 con 0x7f25740047d0 2024-09-13T17:07:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.761+0000 7f258cd5e640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f256c0666c0 0x7f256c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.761+0000 7f258cd5e640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f256c0666c0 0x7f256c068b80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f25780b0000 tx=0x7f25780046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:39.765+0000 7f257dffb640 1 -- 172.21.15.39:0/156668132 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2584052210 con 0x7f25740047d0 2024-09-13T17:07:40.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.029+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25880c53e0 con 0x7f25740047d0 2024-09-13T17:07:40.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.029+0000 7f257dffb640 1 -- 172.21.15.39:0/156668132 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+5185 (secure 0 0 0) 0x7f2584085e60 con 0x7f25740047d0 2024-09-13T17:07:40.028 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:40.029 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:40.029 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 140 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.029+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f256c0666c0 msgr2=0x7f256c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.029+0000 7f258f7c3640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f256c0666c0 0x7f256c068b80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f25780b0000 tx=0x7f25780046d0 comp rx=0 tx=0).stop 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.029+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 msgr2=0x7f25880bcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.029+0000 7f258f7c3640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f25880bcb90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f25840079e0 tx=0x7f258400a6e0 comp rx=0 tx=0).stop 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.033+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 shutdown_connections 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.033+0000 7f258f7c3640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f256c0666c0 0x7f256c068b80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.033+0000 7f258f7c3640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25881544b0 0x7f2588154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.033+0000 7f258f7c3640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25880c05b0 0x7f25880bd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.033+0000 7f258f7c3640 1 --2- 172.21.15.39:0/156668132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25740047d0 0x7f25880bcb90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.033+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 >> 172.21.15.39:0/156668132 conn(0x7f25880c32a0 msgr2=0x7f2588065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.033+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 shutdown_connections 2024-09-13T17:07:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:40.033+0000 7f258f7c3640 1 -- 172.21.15.39:0/156668132 wait complete. 2024-09-13T17:07:40.050 INFO:tasks.mds_thrash.fs.[a]:mds.c has gained rank=0, replacing gid=5561 2024-09-13T17:07:40.050 INFO:tasks.mds_thrash.fs.[a]:waiting for 7 secs before reviving mds.b 2024-09-13T17:07:47.051 INFO:tasks.mds_thrash.fs.[a]:reviving mds.b 2024-09-13T17:07:47.051 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:07:47.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:07:47.055 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:07:47.055 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:07:47.140 INFO:tasks.ceph.mds.b.smithi084.stdout:starting mds.b at 2024-09-13T17:07:47.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99ec989640 1 Processor -- start 2024-09-13T17:07:47.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99ec989640 1 -- start start 2024-09-13T17:07:47.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99ec989640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80c4260 0x7f99e80c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:47.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99ec989640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e8157360 con 0x7f99e8070b40 2024-09-13T17:07:47.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99ec989640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e80c80b0 con 0x7f99e80bcfb0 2024-09-13T17:07:47.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99ec989640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e81575b0 con 0x7f99e80c4260 2024-09-13T17:07:47.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e6d9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f99e8070b40 0x7f99e815e6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34474/0 (socket says 172.21.15.39:34474) 2024-09-13T17:07:47.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e6d9d640 1 -- 172.21.15.39:0/2515574530 learned_addr learned my addr 172.21.15.39:0/2515574530 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:07:47.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e659c640 1 --2- 172.21.15.39:0/2515574530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80c4260 0x7f99e80c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:47.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e559a640 1 -- 172.21.15.39:0/2515574530 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319199941 0 0) 0x7f99e8157360 con 0x7f99e8070b40 2024-09-13T17:07:47.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e559a640 1 -- 172.21.15.39:0/2515574530 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99d0003530 con 0x7f99e8070b40 2024-09-13T17:07:47.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e559a640 1 -- 172.21.15.39:0/2515574530 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669084735 0 0) 0x7f99e80c80b0 con 0x7f99e80bcfb0 2024-09-13T17:07:47.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e559a640 1 -- 172.21.15.39:0/2515574530 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99e8157360 con 0x7f99e80bcfb0 2024-09-13T17:07:47.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e659c640 1 -- 172.21.15.39:0/2515574530 >> v1:172.21.15.84:6789/0 conn(0x7f99e80bcfb0 legacy=0x7f99e8070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:47.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e659c640 1 -- 172.21.15.39:0/2515574530 >> v1:172.21.15.39:6789/0 conn(0x7f99e8070b40 legacy=0x7f99e815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:47.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.201+0000 7f99e659c640 1 -- 172.21.15.39:0/2515574530 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8157890 con 0x7f99e80c4260 2024-09-13T17:07:47.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99e659c640 1 --2- 172.21.15.39:0/2515574530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80c4260 0x7f99e80c4640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009dc0 tx=0x7f99d801c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ab26a8c044737e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:47.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99e559a640 1 -- 172.21.15.39:0/2515574530 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80c7330 con 0x7f99e80c4260 2024-09-13T17:07:47.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99e559a640 1 -- 172.21.15.39:0/2515574530 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d80a5840 con 0x7f99e80c4260 2024-09-13T17:07:47.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99e559a640 1 -- 172.21.15.39:0/2515574530 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80a5bd0 con 0x7f99e80c4260 2024-09-13T17:07:47.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99ec989640 1 -- 172.21.15.39:0/2515574530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80c4260 msgr2=0x7f99e80c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:47.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99ec989640 1 --2- 172.21.15.39:0/2515574530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80c4260 0x7f99e80c4640 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009dc0 tx=0x7f99d801c870 comp rx=0 tx=0).stop 2024-09-13T17:07:47.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99ec989640 1 -- 172.21.15.39:0/2515574530 shutdown_connections 2024-09-13T17:07:47.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99ec989640 1 --2- 172.21.15.39:0/2515574530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80c4260 0x7f99e80c4640 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:47.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99ec989640 1 -- 172.21.15.39:0/2515574530 >> 172.21.15.39:0/2515574530 conn(0x7f99e8069e10 msgr2=0x7f99e806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:47.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99ec989640 1 -- 172.21.15.39:0/2515574530 shutdown_connections 2024-09-13T17:07:47.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.205+0000 7f99ec989640 1 -- 172.21.15.39:0/2515574530 wait complete. 2024-09-13T17:07:47.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 Processor -- start 2024-09-13T17:07:47.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 -- start start 2024-09-13T17:07:47.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80bcfb0 0x7f99e81565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:47.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e80c4260 0x7f99e8141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:47.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8141f10 0x7f99e80c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:47.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8155b00 con 0x7f99e8141f10 2024-09-13T17:07:47.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8155de0 con 0x7f99e80c4260 2024-09-13T17:07:47.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e81560c0 con 0x7f99e80bcfb0 2024-09-13T17:07:47.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8141f10 0x7f99e80c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8141f10 0x7f99e80c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45560/0 (socket says 172.21.15.39:45560) 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e6d9d640 1 -- 172.21.15.39:0/4154544919 learned_addr learned my addr 172.21.15.39:0/4154544919 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e659c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80bcfb0 0x7f99e81565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e5d9b640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e80c4260 0x7f99e8141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e6d9d640 1 -- 172.21.15.39:0/4154544919 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80bcfb0 msgr2=0x7f99e81565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e6d9d640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80bcfb0 0x7f99e81565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e6d9d640 1 -- 172.21.15.39:0/4154544919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e80c4260 msgr2=0x7f99e8141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e6d9d640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e80c4260 0x7f99e8141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e6d9d640 1 -- 172.21.15.39:0/4154544919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d8009a70 con 0x7f99e8141f10 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99e6d9d640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8141f10 0x7f99e80c9980 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f99e000c430 tx=0x7f99e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99d6ffd640 1 -- 172.21.15.39:0/4154544919 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0005cf0 con 0x7f99e8141f10 2024-09-13T17:07:47.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99d6ffd640 1 -- 172.21.15.39:0/4154544919 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0005e50 con 0x7f99e8141f10 2024-09-13T17:07:47.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e8149530 con 0x7f99e8141f10 2024-09-13T17:07:47.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99d6ffd640 1 -- 172.21.15.39:0/4154544919 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0007310 con 0x7f99e8141f10 2024-09-13T17:07:47.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.209+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e8141b60 con 0x7f99e8141f10 2024-09-13T17:07:47.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.213+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e8149750 con 0x7f99e8141f10 2024-09-13T17:07:47.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.213+0000 7f99d6ffd640 1 -- 172.21.15.39:0/4154544919 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f99e0007470 con 0x7f99e8141f10 2024-09-13T17:07:47.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.213+0000 7f99d6ffd640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99c80666c0 0x7f99c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:07:47.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.213+0000 7f99d6ffd640 1 -- 172.21.15.39:0/4154544919 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7f99e00842b0 con 0x7f99e8141f10 2024-09-13T17:07:47.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.213+0000 7f99e659c640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99c80666c0 0x7f99c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:07:47.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.213+0000 7f99e659c640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99c80666c0 0x7f99c8068b80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f99d80af000 tx=0x7f99d80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:07:47.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.217+0000 7f99d6ffd640 1 -- 172.21.15.39:0/4154544919 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f99e0050a20 con 0x7f99e8141f10 2024-09-13T17:07:47.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.457+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99e8155c40 con 0x7f99e8141f10 2024-09-13T17:07:47.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.457+0000 7f99d6ffd640 1 -- 172.21.15.39:0/4154544919 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f99e8155c40 con 0x7f99e8141f10 2024-09-13T17:07:47.456 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:07:47.456 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:07:47.456 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:07:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99c80666c0 msgr2=0x7f99c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99c80666c0 0x7f99c8068b80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f99d80af000 tx=0x7f99d80a7070 comp rx=0 tx=0).stop 2024-09-13T17:07:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8141f10 msgr2=0x7f99e80c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:07:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8141f10 0x7f99e80c9980 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f99e000c430 tx=0x7f99e000c8f0 comp rx=0 tx=0).stop 2024-09-13T17:07:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 shutdown_connections 2024-09-13T17:07:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f99c80666c0 0x7f99c8068b80 secure :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f99d80af000 tx=0x7f99d80a7070 comp rx=0 tx=0).stop 2024-09-13T17:07:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99e80c4260 0x7f99e8141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8141f10 0x7f99e80c9980 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 --2- 172.21.15.39:0/4154544919 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f99e80bcfb0 0x7f99e81565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:07:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 >> 172.21.15.39:0/4154544919 conn(0x7f99e8069e10 msgr2=0x7f99e8070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:07:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 shutdown_connections 2024-09-13T17:07:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:07:47.461+0000 7f99ec989640 1 -- 172.21.15.39:0/4154544919 wait complete. 2024-09-13T17:07:47.476 INFO:tasks.mds_thrash.fs.[a]:mds.b reported in up:standby state 2024-09-13T17:07:47.477 INFO:tasks.mds_thrash.fs.[a]:waiting for 87 secs before thrashing 2024-09-13T17:07:59.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:07:59.594 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:08:29.632 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:08:29.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:08:30.359 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:30.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.507+0000 7efefd9b9640 1 Processor -- start 2024-09-13T17:08:30.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.507+0000 7efefd9b9640 1 -- start start 2024-09-13T17:08:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.507+0000 7efefd9b9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efef814f610 0x7efef8155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.507+0000 7efefd9b9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efef806d210 0x7efef8151d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.507+0000 7efefd9b9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef80ab950 0x7efef80a8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.507+0000 7efefd9b9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef806dc30 con 0x7efef814f610 2024-09-13T17:08:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.507+0000 7efefd9b9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef814f9f0 con 0x7efef806d210 2024-09-13T17:08:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.507+0000 7efefd9b9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef81561e0 con 0x7efef80ab950 2024-09-13T17:08:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef6ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efef814f610 0x7efef8155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef6ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efef814f610 0x7efef8155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59186/0 (socket says 172.21.15.39:59186) 2024-09-13T17:08:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef6ffd640 1 -- 172.21.15.39:0/1039638182 learned_addr learned my addr 172.21.15.39:0/1039638182 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef77fe640 1 --2- 172.21.15.39:0/1039638182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef80ab950 0x7efef80a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef67fc640 1 --2- 172.21.15.39:0/1039638182 >> v2:172.21.15.84:3300/0 conn(0x7efef806d210 0x7efef8151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef6ffd640 1 -- 172.21.15.39:0/1039638182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef80ab950 msgr2=0x7efef80a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef6ffd640 1 --2- 172.21.15.39:0/1039638182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef80ab950 0x7efef80a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef6ffd640 1 -- 172.21.15.39:0/1039638182 >> v2:172.21.15.84:3300/0 conn(0x7efef806d210 msgr2=0x7efef8151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef6ffd640 1 --2- 172.21.15.39:0/1039638182 >> v2:172.21.15.84:3300/0 conn(0x7efef806d210 0x7efef8151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef6ffd640 1 -- 172.21.15.39:0/1039638182 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef81522d0 con 0x7efef814f610 2024-09-13T17:08:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef67fc640 1 --2- 172.21.15.39:0/1039638182 >> v2:172.21.15.84:3300/0 conn(0x7efef806d210 0x7efef8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:08:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef6ffd640 1 --2- 172.21.15.39:0/1039638182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef814f610 0x7efef8155ca0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efef8072670 tx=0x7efee801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe0dd5bc847798a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef5ffb640 1 -- 172.21.15.39:0/1039638182 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee80a56f0 con 0x7efef814f610 2024-09-13T17:08:30.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef5ffb640 1 -- 172.21.15.39:0/1039638182 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee80a5850 con 0x7efef814f610 2024-09-13T17:08:30.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.511+0000 7efef5ffb640 1 -- 172.21.15.39:0/1039638182 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee80a5c20 con 0x7efef814f610 2024-09-13T17:08:30.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- 172.21.15.39:0/1039638182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef814f610 msgr2=0x7efef8155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:30.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 --2- 172.21.15.39:0/1039638182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef814f610 0x7efef8155ca0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efef8072670 tx=0x7efee801c980 comp rx=0 tx=0).stop 2024-09-13T17:08:30.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- 172.21.15.39:0/1039638182 shutdown_connections 2024-09-13T17:08:30.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 --2- 172.21.15.39:0/1039638182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef80ab950 0x7efef80a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 --2- 172.21.15.39:0/1039638182 >> v2:172.21.15.84:3300/0 conn(0x7efef806d210 0x7efef8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 --2- 172.21.15.39:0/1039638182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef814f610 0x7efef8155ca0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- 172.21.15.39:0/1039638182 >> 172.21.15.39:0/1039638182 conn(0x7efef80c2a50 msgr2=0x7efef80c2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:30.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- 172.21.15.39:0/1039638182 shutdown_connections 2024-09-13T17:08:30.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- 172.21.15.39:0/1039638182 wait complete. 2024-09-13T17:08:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 Processor -- start 2024-09-13T17:08:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- start start 2024-09-13T17:08:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef806d210 0x7efef81eb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef80ab950 0x7efef8154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efef814f610 0x7efef8150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef8147bc0 con 0x7efef80ab950 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef8146700 con 0x7efef814f610 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef8146e00 con 0x7efef806d210 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef67fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef80ab950 0x7efef8154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef67fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef80ab950 0x7efef8154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59190/0 (socket says 172.21.15.39:59190) 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef67fc640 1 -- 172.21.15.39:0/4171995224 learned_addr learned my addr 172.21.15.39:0/4171995224 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef6ffd640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef806d210 0x7efef81eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef77fe640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efef814f610 0x7efef8150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef67fc640 1 -- 172.21.15.39:0/4171995224 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef806d210 msgr2=0x7efef81eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef67fc640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef806d210 0x7efef81eb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef67fc640 1 -- 172.21.15.39:0/4171995224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efef814f610 msgr2=0x7efef8150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:30.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef67fc640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efef814f610 0x7efef8150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef67fc640 1 -- 172.21.15.39:0/4171995224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee8009b30 con 0x7efef80ab950 2024-09-13T17:08:30.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef77fe640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efef814f610 0x7efef8150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:08:30.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efef67fc640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef80ab950 0x7efef8154930 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7efee000c7f0 tx=0x7efee000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efeeffff640 1 -- 172.21.15.39:0/4171995224 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0007820 con 0x7efef80ab950 2024-09-13T17:08:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efeeffff640 1 -- 172.21.15.39:0/4171995224 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee0007980 con 0x7efef80ab950 2024-09-13T17:08:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef81eaec0 con 0x7efef80ab950 2024-09-13T17:08:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efeeffff640 1 -- 172.21.15.39:0/4171995224 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0007d30 con 0x7efef80ab950 2024-09-13T17:08:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.515+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef8147330 con 0x7efef80ab950 2024-09-13T17:08:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.519+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef80a9dc0 con 0x7efef80ab950 2024-09-13T17:08:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.519+0000 7efeeffff640 1 -- 172.21.15.39:0/4171995224 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efee00113f0 con 0x7efef80ab950 2024-09-13T17:08:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.519+0000 7efeeffff640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efed8066690 0x7efed8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.519+0000 7efeeffff640 1 -- 172.21.15.39:0/4171995224 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7efee00850c0 con 0x7efef80ab950 2024-09-13T17:08:30.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.519+0000 7efef6ffd640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efed8066690 0x7efed8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.519+0000 7efef6ffd640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efed8066690 0x7efed8068b50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efee80ae000 tx=0x7efee80a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:30.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.523+0000 7efeeffff640 1 -- 172.21.15.39:0/4171995224 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efee0051830 con 0x7efef80ab950 2024-09-13T17:08:30.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.767+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efef81eb050 con 0x7efef80ab950 2024-09-13T17:08:30.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efeeffff640 1 -- 172.21.15.39:0/4171995224 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7efee001f090 con 0x7efef80ab950 2024-09-13T17:08:30.771 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:30.772 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:08:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efed8066690 msgr2=0x7efed8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efed8066690 0x7efed8068b50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efee80ae000 tx=0x7efee80a7070 comp rx=0 tx=0).stop 2024-09-13T17:08:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef80ab950 msgr2=0x7efef8154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef80ab950 0x7efef8154930 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7efee000c7f0 tx=0x7efee000ccb0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 shutdown_connections 2024-09-13T17:08:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efed8066690 0x7efed8068b50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efef814f610 0x7efef8150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef80ab950 0x7efef8154930 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 --2- 172.21.15.39:0/4171995224 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efef806d210 0x7efef81eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 >> 172.21.15.39:0/4171995224 conn(0x7efef80c2a50 msgr2=0x7efef815b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 shutdown_connections 2024-09-13T17:08:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.771+0000 7efefd9b9640 1 -- 172.21.15.39:0/4171995224 wait complete. 2024-09-13T17:08:30.792 INFO:tasks.mds_thrash.fs.[b]:kill mds.a (rank=0) 2024-09-13T17:08:30.793 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:08:30.793 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:08:30.803 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:08:30.804 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:30.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.979+0000 7f9ebf68a640 1 Processor -- start 2024-09-13T17:08:30.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.979+0000 7f9ebf68a640 1 -- start start 2024-09-13T17:08:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.979+0000 7f9ebf68a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80ab9e0 0x7f9eb80c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.979+0000 7f9ebf68a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9eb80c0ac0 0x7f9eb806d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.979+0000 7f9ebf68a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb8150e00 con 0x7f9eb8071bb0 2024-09-13T17:08:30.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.979+0000 7f9ebf68a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb80abdc0 con 0x7f9eb80c0ac0 2024-09-13T17:08:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.979+0000 7f9ebf68a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb806d830 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.979+0000 7f9ebcc25640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9eb8071bb0 0x7f9eb806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52824/0 (socket says 172.21.15.39:52824) 2024-09-13T17:08:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.979+0000 7f9ebcc25640 1 -- 172.21.15.39:0/86199340 learned_addr learned my addr 172.21.15.39:0/86199340 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:08:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9ebd426640 1 --2- 172.21.15.39:0/86199340 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80ab9e0 0x7f9eb80c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9ebdc27640 1 --2- 172.21.15.39:0/86199340 >> v2:172.21.15.84:3300/0 conn(0x7f9eb80c0ac0 0x7f9eb806d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9eaffff640 1 -- 172.21.15.39:0/86199340 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284794767 0 0) 0x7f9eb8150e00 con 0x7f9eb8071bb0 2024-09-13T17:08:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9eaffff640 1 -- 172.21.15.39:0/86199340 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ea4003530 con 0x7f9eb8071bb0 2024-09-13T17:08:30.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9ebd426640 1 -- 172.21.15.39:0/86199340 >> v2:172.21.15.84:3300/0 conn(0x7f9eb80c0ac0 msgr2=0x7f9eb806d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:30.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9ebd426640 1 --2- 172.21.15.39:0/86199340 >> v2:172.21.15.84:3300/0 conn(0x7f9eb80c0ac0 0x7f9eb806d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9ebd426640 1 -- 172.21.15.39:0/86199340 >> v1:172.21.15.39:6789/0 conn(0x7f9eb8071bb0 legacy=0x7f9eb806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:30.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9eaffff640 1 -- 172.21.15.39:0/86199340 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1297143385 0 0) 0x7f9ea4003530 con 0x7f9eb8071bb0 2024-09-13T17:08:30.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9ebd426640 1 -- 172.21.15.39:0/86199340 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb8150250 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9ebd426640 1 --2- 172.21.15.39:0/86199340 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80ab9e0 0x7f9eb80c2b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40097c0 tx=0x7f9eb401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e697d010d75e54ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:30.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9eaffff640 1 -- 172.21.15.39:0/86199340 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8001680 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.983+0000 7f9eaffff640 1 -- 172.21.15.39:0/86199340 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb40a7050 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9eaffff640 1 -- 172.21.15.39:0/86199340 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb40ab600 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 -- 172.21.15.39:0/86199340 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80ab9e0 msgr2=0x7f9eb80c2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 --2- 172.21.15.39:0/86199340 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80ab9e0 0x7f9eb80c2b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40097c0 tx=0x7f9eb401c3c0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 -- 172.21.15.39:0/86199340 shutdown_connections 2024-09-13T17:08:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 --2- 172.21.15.39:0/86199340 >> v2:172.21.15.84:3300/0 conn(0x7f9eb80c0ac0 0x7f9eb806d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 --2- 172.21.15.39:0/86199340 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80ab9e0 0x7f9eb80c2b50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 -- 172.21.15.39:0/86199340 >> 172.21.15.39:0/86199340 conn(0x7f9eb8069e10 msgr2=0x7f9eb806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 -- 172.21.15.39:0/86199340 shutdown_connections 2024-09-13T17:08:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 -- 172.21.15.39:0/86199340 wait complete. 2024-09-13T17:08:30.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 Processor -- start 2024-09-13T17:08:30.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 -- start start 2024-09-13T17:08:30.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb8071bb0 0x7f9eb814ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80ab9e0 0x7f9eb814be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80c0ac0 0x7f9eb81eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80aa5d0 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80ab1c0 con 0x7f9eb8071bb0 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebf68a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80aaf20 con 0x7f9eb80c0ac0 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.987+0000 7f9ebcc25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80ab9e0 0x7f9eb814be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebcc25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80ab9e0 0x7f9eb814be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59196/0 (socket says 172.21.15.39:59196) 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebcc25640 1 -- 172.21.15.39:0/3259119956 learned_addr learned my addr 172.21.15.39:0/3259119956 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebd426640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb8071bb0 0x7f9eb814ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebdc27640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80c0ac0 0x7f9eb81eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebcc25640 1 -- 172.21.15.39:0/3259119956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80c0ac0 msgr2=0x7f9eb81eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebcc25640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80c0ac0 0x7f9eb81eff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebcc25640 1 -- 172.21.15.39:0/3259119956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb8071bb0 msgr2=0x7f9eb814ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:30.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebcc25640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb8071bb0 0x7f9eb814ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebcc25640 1 -- 172.21.15.39:0/3259119956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4009420 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebd426640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb8071bb0 0x7f9eb814ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:08:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebcc25640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80ab9e0 0x7f9eb814be60 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800c760 tx=0x7f9ea800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9eadffb640 1 -- 172.21.15.39:0/3259119956 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8005ae0 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9eadffb640 1 -- 172.21.15.39:0/3259119956 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea8005c40 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb80aa260 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9eadffb640 1 -- 172.21.15.39:0/3259119956 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8007070 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb8148900 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb80aa780 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9eadffb640 1 -- 172.21.15.39:0/3259119956 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ea80071d0 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9eadffb640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e9c066690 0x7f9e9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9eadffb640 1 -- 172.21.15.39:0/3259119956 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7f9ea80845b0 con 0x7f9eb80ab9e0 2024-09-13T17:08:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.991+0000 7f9ebd426640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e9c066690 0x7f9e9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:30.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.995+0000 7f9ebd426640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e9c066690 0x7f9e9c068b50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40ad000 tx=0x7f9eb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:30.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:30.999+0000 7f9eadffb640 1 -- 172.21.15.39:0/3259119956 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9eb80aa780 con 0x7f9eb80ab9e0 2024-09-13T17:08:31.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.215+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eb80aa780 con 0x7f9eb80ab9e0 2024-09-13T17:08:31.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.215+0000 7f9eadffb640 1 -- 172.21.15.39:0/3259119956 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f9eb80aa780 con 0x7f9eb80ab9e0 2024-09-13T17:08:31.220 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:31.220 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:31.220 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:08:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e9c066690 msgr2=0x7f9e9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e9c066690 0x7f9e9c068b50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40ad000 tx=0x7f9eb4008040 comp rx=0 tx=0).stop 2024-09-13T17:08:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80ab9e0 msgr2=0x7f9eb814be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80ab9e0 0x7f9eb814be60 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800c760 tx=0x7f9ea800cc20 comp rx=0 tx=0).stop 2024-09-13T17:08:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 shutdown_connections 2024-09-13T17:08:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9e9c066690 0x7f9e9c068b50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9eb80c0ac0 0x7f9eb81eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80ab9e0 0x7f9eb814be60 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 --2- 172.21.15.39:0/3259119956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9eb8071bb0 0x7f9eb814ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 >> 172.21.15.39:0/3259119956 conn(0x7f9eb8069e10 msgr2=0x7f9eb806cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 shutdown_connections 2024-09-13T17:08:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:31.219+0000 7f9ebf68a640 1 -- 172.21.15.39:0/3259119956 wait complete. 2024-09-13T17:08:31.237 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:08:33.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.391+0000 7f8bc3be0640 1 Processor -- start 2024-09-13T17:08:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.391+0000 7f8bc3be0640 1 -- start start 2024-09-13T17:08:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.391+0000 7f8bc3be0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8bbc0c6060 0x7f8bbc150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.391+0000 7f8bc3be0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc0c4760 con 0x7f8bbc14a770 2024-09-13T17:08:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.391+0000 7f8bc3be0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc14afa0 con 0x7f8bbc14ab50 2024-09-13T17:08:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.391+0000 7f8bc3be0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bbc06df80 con 0x7f8bbc0c6060 2024-09-13T17:08:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.391+0000 7f8bc117b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8bbc0c6060 0x7f8bbc150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.391+0000 7f8bc117b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8bbc0c6060 0x7f8bbc150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59202/0 (socket says 172.21.15.39:59202) 2024-09-13T17:08:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.391+0000 7f8bc117b640 1 -- 172.21.15.39:0/2476475443 learned_addr learned my addr 172.21.15.39:0/2476475443 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.395+0000 7f8bc117b640 1 -- 172.21.15.39:0/2476475443 >> v1:172.21.15.84:6789/0 conn(0x7f8bbc14ab50 legacy=0x7f8bbc0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.395+0000 7f8bc117b640 1 -- 172.21.15.39:0/2476475443 >> v1:172.21.15.39:6789/0 conn(0x7f8bbc14a770 legacy=0x7f8bbc06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.395+0000 7f8bc117b640 1 -- 172.21.15.39:0/2476475443 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc151120 con 0x7f8bbc0c6060 2024-09-13T17:08:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.395+0000 7f8bc117b640 1 --2- 172.21.15.39:0/2476475443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 0x7f8bbc150ad0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009870 tx=0x7f8bac01e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddaa230425b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.395+0000 7f8bc097a640 1 -- 172.21.15.39:0/2476475443 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0ac610 con 0x7f8bbc0c6060 2024-09-13T17:08:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.395+0000 7f8bc097a640 1 -- 172.21.15.39:0/2476475443 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac0a8040 con 0x7f8bbc0c6060 2024-09-13T17:08:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.395+0000 7f8bc097a640 1 -- 172.21.15.39:0/2476475443 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0ac990 con 0x7f8bbc0c6060 2024-09-13T17:08:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.395+0000 7f8bc3be0640 1 -- 172.21.15.39:0/2476475443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 msgr2=0x7f8bbc150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.395+0000 7f8bc3be0640 1 --2- 172.21.15.39:0/2476475443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 0x7f8bbc150ad0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009870 tx=0x7f8bac01e6a0 comp rx=0 tx=0).stop 2024-09-13T17:08:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- 172.21.15.39:0/2476475443 shutdown_connections 2024-09-13T17:08:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 --2- 172.21.15.39:0/2476475443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 0x7f8bbc150ad0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- 172.21.15.39:0/2476475443 >> 172.21.15.39:0/2476475443 conn(0x7f8bbc0c32a0 msgr2=0x7f8bbc0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- 172.21.15.39:0/2476475443 shutdown_connections 2024-09-13T17:08:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- 172.21.15.39:0/2476475443 wait complete. 2024-09-13T17:08:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 Processor -- start 2024-09-13T17:08:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- start start 2024-09-13T17:08:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 0x7f8bbc0ac130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bbc14a770 0x7f8bbc0aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc14ab50 0x7f8bbc14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0abd80 con 0x7f8bbc0c6060 2024-09-13T17:08:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc14cf30 con 0x7f8bbc14ab50 2024-09-13T17:08:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc14c950 con 0x7f8bbc14a770 2024-09-13T17:08:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc197c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 0x7f8bbc0ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc197c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 0x7f8bbc0ac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59216/0 (socket says 172.21.15.39:59216) 2024-09-13T17:08:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc197c640 1 -- 172.21.15.39:0/3183581619 learned_addr learned my addr 172.21.15.39:0/3183581619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc197c640 1 -- 172.21.15.39:0/3183581619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bbc14a770 msgr2=0x7f8bbc0aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc197c640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bbc14a770 0x7f8bbc0aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc217d640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc14ab50 0x7f8bbc14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc197c640 1 -- 172.21.15.39:0/3183581619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc14ab50 msgr2=0x7f8bbc14d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc197c640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc14ab50 0x7f8bbc14d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc197c640 1 -- 172.21.15.39:0/3183581619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac0094d0 con 0x7f8bbc0c6060 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc217d640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc14ab50 0x7f8bbc14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc197c640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 0x7f8bbc0ac130 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800c7f0 tx=0x7f8bb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bb27fc640 1 -- 172.21.15.39:0/3183581619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8003b80 con 0x7f8bbc0c6060 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bb27fc640 1 -- 172.21.15.39:0/3183581619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb8003ce0 con 0x7f8bbc0c6060 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc14cbf0 con 0x7f8bbc0c6060 2024-09-13T17:08:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bb27fc640 1 -- 172.21.15.39:0/3183581619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800aa90 con 0x7f8bbc0c6060 2024-09-13T17:08:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.399+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc154cd0 con 0x7f8bbc0c6060 2024-09-13T17:08:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.403+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc0619c0 con 0x7f8bbc0c6060 2024-09-13T17:08:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.403+0000 7f8bb27fc640 1 -- 172.21.15.39:0/3183581619 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8bb800abf0 con 0x7f8bbc0c6060 2024-09-13T17:08:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.403+0000 7f8bb27fc640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba0066690 0x7f8ba0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.403+0000 7f8bb27fc640 1 -- 172.21.15.39:0/3183581619 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7f8bb8084d20 con 0x7f8bbc0c6060 2024-09-13T17:08:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.403+0000 7f8bc117b640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba0066690 0x7f8ba0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.403+0000 7f8bc117b640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba0066690 0x7f8ba0068b50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0b0000 tx=0x7f8bac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:33.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.407+0000 7f8bb27fc640 1 -- 172.21.15.39:0/3183581619 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8bb8051470 con 0x7f8bbc0c6060 2024-09-13T17:08:33.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.643+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bbc14d0a0 con 0x7f8bbc0c6060 2024-09-13T17:08:33.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.643+0000 7f8bb27fc640 1 -- 172.21.15.39:0/3183581619 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f8bbc14d0a0 con 0x7f8bbc0c6060 2024-09-13T17:08:33.647 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:33.647 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:08:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba0066690 msgr2=0x7f8ba0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba0066690 0x7f8ba0068b50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0b0000 tx=0x7f8bac008040 comp rx=0 tx=0).stop 2024-09-13T17:08:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 msgr2=0x7f8bbc0ac130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 0x7f8bbc0ac130 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800c7f0 tx=0x7f8bb800ccb0 comp rx=0 tx=0).stop 2024-09-13T17:08:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 shutdown_connections 2024-09-13T17:08:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ba0066690 0x7f8ba0068b50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bbc14ab50 0x7f8bbc14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bbc14a770 0x7f8bbc0aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:33.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 --2- 172.21.15.39:0/3183581619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bbc0c6060 0x7f8bbc0ac130 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:33.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 >> 172.21.15.39:0/3183581619 conn(0x7f8bbc0c32a0 msgr2=0x7f8bbc14e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:33.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 shutdown_connections 2024-09-13T17:08:33.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:33.647+0000 7f8bc3be0640 1 -- 172.21.15.39:0/3183581619 wait complete. 2024-09-13T17:08:33.673 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:08:35.674 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:35.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37f0b1e640 1 Processor -- start 2024-09-13T17:08:35.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37f0b1e640 1 -- start start 2024-09-13T17:08:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37f0b1e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f37ec155b60 0x7f37ec0a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37f0b1e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0a6b60 0x7f37ec0c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:35.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37f0b1e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec159070 con 0x7f37ec0c7160 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37f0b1e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec0c63d0 con 0x7f37ec155b60 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37f0b1e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec158480 con 0x7f37ec0a6b60 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37e9d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0a6b60 0x7f37ec0c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37ea59c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f37ec155b60 0x7f37ec0a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37e9d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0a6b60 0x7f37ec0c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:35558/0 (socket says 172.21.15.39:35558) 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37e9d9b640 1 -- 172.21.15.39:0/4009375326 learned_addr learned my addr 172.21.15.39:0/4009375326 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37e9d9b640 1 -- 172.21.15.39:0/4009375326 >> v2:172.21.15.84:3300/0 conn(0x7f37ec155b60 msgr2=0x7f37ec0a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37e9d9b640 1 --2- 172.21.15.39:0/4009375326 >> v2:172.21.15.84:3300/0 conn(0x7f37ec155b60 0x7f37ec0a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37e9d9b640 1 -- 172.21.15.39:0/4009375326 >> v1:172.21.15.84:6789/0 conn(0x7f37ec0c7160 legacy=0x7f37ec15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:35.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.823+0000 7f37e9d9b640 1 -- 172.21.15.39:0/4009375326 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec158740 con 0x7f37ec0a6b60 2024-09-13T17:08:35.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37e9d9b640 1 --2- 172.21.15.39:0/4009375326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0a6b60 0x7f37ec0c6c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37e00098a0 tx=0x7f37e001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f49852b0a6e473 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37e959a640 1 -- 172.21.15.39:0/4009375326 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e00ab600 con 0x7f37ec0a6b60 2024-09-13T17:08:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37e959a640 1 -- 172.21.15.39:0/4009375326 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e00a7050 con 0x7f37ec0a6b60 2024-09-13T17:08:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37e959a640 1 -- 172.21.15.39:0/4009375326 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e00ab980 con 0x7f37ec0a6b60 2024-09-13T17:08:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4009375326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0a6b60 msgr2=0x7f37ec0c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37f0b1e640 1 --2- 172.21.15.39:0/4009375326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0a6b60 0x7f37ec0c6c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37e00098a0 tx=0x7f37e001c3f0 comp rx=0 tx=0).stop 2024-09-13T17:08:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4009375326 shutdown_connections 2024-09-13T17:08:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37f0b1e640 1 --2- 172.21.15.39:0/4009375326 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0a6b60 0x7f37ec0c6c20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:35.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37f0b1e640 1 --2- 172.21.15.39:0/4009375326 >> v2:172.21.15.84:3300/0 conn(0x7f37ec155b60 0x7f37ec0a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:35.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4009375326 >> 172.21.15.39:0/4009375326 conn(0x7f37ec06c020 msgr2=0x7f37ec06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:35.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4009375326 shutdown_connections 2024-09-13T17:08:35.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.827+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4009375326 wait complete. 2024-09-13T17:08:35.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 Processor -- start 2024-09-13T17:08:35.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 -- start start 2024-09-13T17:08:35.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37ec0a6b60 0x7f37ec06ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0c7160 0x7f37ec1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37ec155b60 0x7f37ec159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1f1fa0 con 0x7f37ec155b60 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1f16d0 con 0x7f37ec0a6b60 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0bec70 con 0x7f37ec0c7160 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ead9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37ec155b60 0x7f37ec159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ead9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37ec155b60 0x7f37ec159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59228/0 (socket says 172.21.15.39:59228) 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ead9d640 1 -- 172.21.15.39:0/4157061298 learned_addr learned my addr 172.21.15.39:0/4157061298 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ea59c640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37ec0a6b60 0x7f37ec06ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37e9d9b640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0c7160 0x7f37ec1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ead9d640 1 -- 172.21.15.39:0/4157061298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0c7160 msgr2=0x7f37ec1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ead9d640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0c7160 0x7f37ec1595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ead9d640 1 -- 172.21.15.39:0/4157061298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37ec0a6b60 msgr2=0x7f37ec06ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ead9d640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37ec0a6b60 0x7f37ec06ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ead9d640 1 -- 172.21.15.39:0/4157061298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e00094d0 con 0x7f37ec155b60 2024-09-13T17:08:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37ead9d640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37ec155b60 0x7f37ec159b20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f37e400c760 tx=0x7f37e400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37daffd640 1 -- 172.21.15.39:0/4157061298 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4003b80 con 0x7f37ec155b60 2024-09-13T17:08:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec0bef50 con 0x7f37ec155b60 2024-09-13T17:08:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37daffd640 1 -- 172.21.15.39:0/4157061298 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e4003ce0 con 0x7f37ec155b60 2024-09-13T17:08:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37daffd640 1 -- 172.21.15.39:0/4157061298 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e400aa30 con 0x7f37ec155b60 2024-09-13T17:08:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec1f1c30 con 0x7f37ec155b60 2024-09-13T17:08:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.831+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec1f2110 con 0x7f37ec155b60 2024-09-13T17:08:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.835+0000 7f37daffd640 1 -- 172.21.15.39:0/4157061298 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f37e400ab90 con 0x7f37ec155b60 2024-09-13T17:08:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.835+0000 7f37daffd640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f37cc066690 0x7f37cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.835+0000 7f37daffd640 1 -- 172.21.15.39:0/4157061298 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7f37e4084ba0 con 0x7f37ec155b60 2024-09-13T17:08:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.835+0000 7f37ea59c640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f37cc066690 0x7f37cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.835+0000 7f37ea59c640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f37cc066690 0x7f37cc068b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009af0 tx=0x7f37dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:35.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:35.839+0000 7f37daffd640 1 -- 172.21.15.39:0/4157061298 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f37e4051270 con 0x7f37ec155b60 2024-09-13T17:08:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.047+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37ec0c6c20 con 0x7f37ec155b60 2024-09-13T17:08:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.047+0000 7f37daffd640 1 -- 172.21.15.39:0/4157061298 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f37e4054890 con 0x7f37ec155b60 2024-09-13T17:08:36.050 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:36.050 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:08:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f37cc066690 msgr2=0x7f37cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f37cc066690 0x7f37cc068b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009af0 tx=0x7f37dc008040 comp rx=0 tx=0).stop 2024-09-13T17:08:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37ec155b60 msgr2=0x7f37ec159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37ec155b60 0x7f37ec159b20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f37e400c760 tx=0x7f37e400cc20 comp rx=0 tx=0).stop 2024-09-13T17:08:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 shutdown_connections 2024-09-13T17:08:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f37cc066690 0x7f37cc068b50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37ec155b60 0x7f37ec159b20 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f37ec0c7160 0x7f37ec1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 --2- 172.21.15.39:0/4157061298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37ec0a6b60 0x7f37ec06ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 >> 172.21.15.39:0/4157061298 conn(0x7f37ec06c020 msgr2=0x7f37ec0a84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 shutdown_connections 2024-09-13T17:08:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:36.051+0000 7f37f0b1e640 1 -- 172.21.15.39:0/4157061298 wait complete. 2024-09-13T17:08:36.071 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:08:38.072 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:38.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c77da7640 1 Processor -- start 2024-09-13T17:08:38.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c77da7640 1 -- start start 2024-09-13T17:08:38.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c77da7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c7014aa40 0x7f8c700c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:38.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c77da7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c700c76a0 con 0x7f8c700c4710 2024-09-13T17:08:38.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c77da7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c70151570 con 0x7f8c700c1750 2024-09-13T17:08:38.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c77da7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70151b30 con 0x7f8c7014aa40 2024-09-13T17:08:38.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c76344640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8c700c4710 0x7f8c7015b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52846/0 (socket says 172.21.15.39:52846) 2024-09-13T17:08:38.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c76344640 1 -- 172.21.15.39:0/3856153710 learned_addr learned my addr 172.21.15.39:0/3856153710 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:08:38.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c75b43640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c7014aa40 0x7f8c700c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:38.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491764117 0 0) 0x7f8c700c76a0 con 0x7f8c700c4710 2024-09-13T17:08:38.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c5c003530 con 0x7f8c700c4710 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.223+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978215232 0 0) 0x7f8c70151570 con 0x7f8c700c1750 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c700c76a0 con 0x7f8c700c1750 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3494857873 0 0) 0x7f8c5c003530 con 0x7f8c700c4710 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c7014aa40 msgr2=0x7f8c700c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c7014aa40 0x7f8c700c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 >> v1:172.21.15.84:6789/0 conn(0x7f8c700c1750 legacy=0x7f8c700c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70151850 con 0x7f8c700c4710 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f8c68001a00 con 0x7f8c700c4710 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 >> v1:172.21.15.39:6789/0 conn(0x7f8c700c4710 legacy=0x7f8c7015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c5c003b40 0x7f8c5c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c7014aa40 0x7f8c5c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:38.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c5c004d10 0x7f8c5c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70151b30 con 0x7f8c5c004d10 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c00bb20 con 0x7f8c5c003b40 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c00bcb0 con 0x7f8c7014aa40 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c76344640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c5c004d10 0x7f8c5c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c75b43640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c5c003b40 0x7f8c5c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c75342640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c7014aa40 0x7f8c5c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c76344640 1 -- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c7014aa40 msgr2=0x7f8c5c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c76344640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c7014aa40 0x7f8c5c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c76344640 1 -- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c5c003b40 msgr2=0x7f8c5c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c76344640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c5c003b40 0x7f8c5c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c76344640 1 -- 172.21.15.39:0/3856153710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c003530 con 0x7f8c5c004d10 2024-09-13T17:08:38.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c76344640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c5c004d10 0x7f8c5c005190 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800a730 tx=0x7f8c6801d200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf701fac56a62eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680ad630 con 0x7f8c5c004d10 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c680a8070 con 0x7f8c5c004d10 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c74b41640 1 -- 172.21.15.39:0/3856153710 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680ad9f0 con 0x7f8c5c004d10 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c77da7640 1 -- 172.21.15.39:0/3856153710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c5c004d10 msgr2=0x7f8c5c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c77da7640 1 --2- 172.21.15.39:0/3856153710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c5c004d10 0x7f8c5c005190 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800a730 tx=0x7f8c6801d200 comp rx=0 tx=0).stop 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c76344640 1 -- 172.21.15.39:0/3856153710 reap_dead start 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c77da7640 1 -- 172.21.15.39:0/3856153710 shutdown_connections 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.227+0000 7f8c77da7640 1 -- 172.21.15.39:0/3856153710 >> 172.21.15.39:0/3856153710 conn(0x7f8c700c3310 msgr2=0x7f8c700c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 -- 172.21.15.39:0/3856153710 shutdown_connections 2024-09-13T17:08:38.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 -- 172.21.15.39:0/3856153710 wait complete. 2024-09-13T17:08:38.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 Processor -- start 2024-09-13T17:08:38.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 -- start start 2024-09-13T17:08:38.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c7014fc20 0x7f8c700a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:38.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c700a6b80 0x7f8c701ecac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:38.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c700a7000 0x7f8c701ed000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:38.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c700a76b0 con 0x7f8c700a7000 2024-09-13T17:08:38.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c700aa520 con 0x7f8c7014fc20 2024-09-13T17:08:38.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c701ea340 con 0x7f8c700a6b80 2024-09-13T17:08:38.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c76344640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c700a7000 0x7f8c701ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c76344640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c700a7000 0x7f8c701ed000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59246/0 (socket says 172.21.15.39:59246) 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c76344640 1 -- 172.21.15.39:0/3077945677 learned_addr learned my addr 172.21.15.39:0/3077945677 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c75342640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c700a6b80 0x7f8c701ecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c75b43640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c7014fc20 0x7f8c700a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c76344640 1 -- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c700a6b80 msgr2=0x7f8c701ecac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c76344640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c700a6b80 0x7f8c701ecac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c76344640 1 -- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c7014fc20 msgr2=0x7f8c700a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c76344640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c7014fc20 0x7f8c700a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c76344640 1 -- 172.21.15.39:0/3077945677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c700a7900 con 0x7f8c700a7000 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c75342640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c700a6b80 0x7f8c701ecac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c75b43640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c7014fc20 0x7f8c700a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:08:38.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c76344640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c700a7000 0x7f8c701ed000 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800a730 tx=0x7f8c5c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:38.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c667fc640 1 -- 172.21.15.39:0/3077945677 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680ba070 con 0x7f8c700a7000 2024-09-13T17:08:38.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c667fc640 1 -- 172.21.15.39:0/3077945677 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c680011d0 con 0x7f8c700a7000 2024-09-13T17:08:38.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c700a7be0 con 0x7f8c700a7000 2024-09-13T17:08:38.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c667fc640 1 -- 172.21.15.39:0/3077945677 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68001800 con 0x7f8c700a7000 2024-09-13T17:08:38.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.231+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c701447c0 con 0x7f8c700a7000 2024-09-13T17:08:38.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.235+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c700a7d70 con 0x7f8c700a7000 2024-09-13T17:08:38.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.235+0000 7f8c667fc640 1 -- 172.21.15.39:0/3077945677 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c680aa030 con 0x7f8c700a7000 2024-09-13T17:08:38.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.235+0000 7f8c667fc640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c54066690 0x7f8c54068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:38.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.235+0000 7f8c667fc640 1 -- 172.21.15.39:0/3077945677 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7f8c6811e640 con 0x7f8c700a7000 2024-09-13T17:08:38.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.235+0000 7f8c75b43640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c54066690 0x7f8c54068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:38.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.235+0000 7f8c75b43640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c54066690 0x7f8c54068b50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8c700c4eb0 tx=0x7f8c6c00b3a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.239+0000 7f8c667fc640 1 -- 172.21.15.39:0/3077945677 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8c680ead90 con 0x7f8c700a7000 2024-09-13T17:08:38.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.459+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c700c5be0 con 0x7f8c700a7000 2024-09-13T17:08:38.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c667fc640 1 -- 172.21.15.39:0/3077945677 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f8c680ee3b0 con 0x7f8c700a7000 2024-09-13T17:08:38.464 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:38.464 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:38.464 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:08:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c54066690 msgr2=0x7f8c54068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c54066690 0x7f8c54068b50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8c700c4eb0 tx=0x7f8c6c00b3a0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c700a7000 msgr2=0x7f8c701ed000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c700a7000 0x7f8c701ed000 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800a730 tx=0x7f8c5c005800 comp rx=0 tx=0).stop 2024-09-13T17:08:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 shutdown_connections 2024-09-13T17:08:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c54066690 0x7f8c54068b50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c700a7000 0x7f8c701ed000 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c700a6b80 0x7f8c701ecac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 --2- 172.21.15.39:0/3077945677 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c7014fc20 0x7f8c700a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 >> 172.21.15.39:0/3077945677 conn(0x7f8c700c3310 msgr2=0x7f8c700c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 shutdown_connections 2024-09-13T17:08:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:38.463+0000 7f8c77da7640 1 -- 172.21.15.39:0/3077945677 wait complete. 2024-09-13T17:08:38.483 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:08:40.484 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.619+0000 7fac056c1640 1 Processor -- start 2024-09-13T17:08:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.619+0000 7fac056c1640 1 -- start start 2024-09-13T17:08:40.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fac056c1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fac00065ee0 0x7fac000662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:40.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fac056c1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fac000c2ee0 0x7fac000c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:40.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fac056c1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0006ddf0 con 0x7fac000c6270 2024-09-13T17:08:40.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fac056c1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac000c4710 con 0x7fac00065ee0 2024-09-13T17:08:40.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fac056c1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac000c49d0 con 0x7fac000c2ee0 2024-09-13T17:08:40.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fabfeffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fac00065ee0 0x7fac000662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:40.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fabfeffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fac00065ee0 0x7fac000662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56230/0 (socket says 172.21.15.39:56230) 2024-09-13T17:08:40.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fabfeffd640 1 -- 172.21.15.39:0/998352284 learned_addr learned my addr 172.21.15.39:0/998352284 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:40.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fabfeffd640 1 -- 172.21.15.39:0/998352284 >> v2:172.21.15.84:3300/0 conn(0x7fac000c2ee0 msgr2=0x7fac000c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:40.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fabfeffd640 1 --2- 172.21.15.39:0/998352284 >> v2:172.21.15.84:3300/0 conn(0x7fac000c2ee0 0x7fac000c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:40.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fabfeffd640 1 -- 172.21.15.39:0/998352284 >> v1:172.21.15.84:6789/0 conn(0x7fac000c6270 legacy=0x7fac000bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:40.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.623+0000 7fabfeffd640 1 -- 172.21.15.39:0/998352284 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac00154880 con 0x7fac00065ee0 2024-09-13T17:08:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fabfeffd640 1 --2- 172.21.15.39:0/998352284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac00065ee0 0x7fac000662c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fac00151ba0 tx=0x7fabf001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d75fcc7c4b23e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fabfdffb640 1 -- 172.21.15.39:0/998352284 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf00ac410 con 0x7fac00065ee0 2024-09-13T17:08:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fabfdffb640 1 -- 172.21.15.39:0/998352284 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf00ac570 con 0x7fac00065ee0 2024-09-13T17:08:40.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fabfdffb640 1 -- 172.21.15.39:0/998352284 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf00ac900 con 0x7fac00065ee0 2024-09-13T17:08:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 -- 172.21.15.39:0/998352284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac00065ee0 msgr2=0x7fac000662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 --2- 172.21.15.39:0/998352284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac00065ee0 0x7fac000662c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fac00151ba0 tx=0x7fabf001c450 comp rx=0 tx=0).stop 2024-09-13T17:08:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 -- 172.21.15.39:0/998352284 shutdown_connections 2024-09-13T17:08:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 --2- 172.21.15.39:0/998352284 >> v2:172.21.15.84:3300/0 conn(0x7fac000c2ee0 0x7fac000c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 --2- 172.21.15.39:0/998352284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac00065ee0 0x7fac000662c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 -- 172.21.15.39:0/998352284 >> 172.21.15.39:0/998352284 conn(0x7fac000c3310 msgr2=0x7fac000c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 -- 172.21.15.39:0/998352284 shutdown_connections 2024-09-13T17:08:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 -- 172.21.15.39:0/998352284 wait complete. 2024-09-13T17:08:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 Processor -- start 2024-09-13T17:08:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 -- start start 2024-09-13T17:08:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fac00065ee0 0x7fac00143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac000c2ee0 0x7fac00144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac000c6270 0x7fac00144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00144350 con 0x7fac000c2ee0 2024-09-13T17:08:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00143db0 con 0x7fac000c6270 2024-09-13T17:08:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fac056c1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac00144070 con 0x7fac00065ee0 2024-09-13T17:08:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fabfe7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac000c2ee0 0x7fac00144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fabfe7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac000c2ee0 0x7fac00144880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56242/0 (socket says 172.21.15.39:56242) 2024-09-13T17:08:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fabfe7fc640 1 -- 172.21.15.39:0/1513453441 learned_addr learned my addr 172.21.15.39:0/1513453441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fabfeffd640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fac00065ee0 0x7fac00143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.627+0000 7fabff7fe640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac000c6270 0x7fac00144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabfe7fc640 1 -- 172.21.15.39:0/1513453441 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fac00065ee0 msgr2=0x7fac00143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabfe7fc640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fac00065ee0 0x7fac00143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabfe7fc640 1 -- 172.21.15.39:0/1513453441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac000c6270 msgr2=0x7fac00144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabfe7fc640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac000c6270 0x7fac00144dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabfe7fc640 1 -- 172.21.15.39:0/1513453441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf00094d0 con 0x7fac000c2ee0 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabfe7fc640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac000c2ee0 0x7fac00144880 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fabf400b550 tx=0x7fabf400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabe77fe640 1 -- 172.21.15.39:0/1513453441 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4010870 con 0x7fac000c2ee0 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabe77fe640 1 -- 172.21.15.39:0/1513453441 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf40109f0 con 0x7fac000c2ee0 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0014ca20 con 0x7fac000c2ee0 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabe77fe640 1 -- 172.21.15.39:0/1513453441 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4010da0 con 0x7fac000c2ee0 2024-09-13T17:08:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0014b770 con 0x7fac000c2ee0 2024-09-13T17:08:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac000662c0 con 0x7fac000c2ee0 2024-09-13T17:08:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabe77fe640 1 -- 172.21.15.39:0/1513453441 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fabf400f3c0 con 0x7fac000c2ee0 2024-09-13T17:08:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabe77fe640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabec0666c0 0x7fabec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabe77fe640 1 -- 172.21.15.39:0/1513453441 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7fabf4084d90 con 0x7fac000c2ee0 2024-09-13T17:08:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabfeffd640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabec0666c0 0x7fabec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.631+0000 7fabfeffd640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabec0666c0 0x7fabec068b80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fabf00ae000 tx=0x7fabf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.639+0000 7fabe77fe640 1 -- 172.21.15.39:0/1513453441 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fabf4051480 con 0x7fac000c2ee0 2024-09-13T17:08:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.871+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac0014ced0 con 0x7fac000c2ee0 2024-09-13T17:08:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fabe77fe640 1 -- 172.21.15.39:0/1513453441 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7fabf4014090 con 0x7fac000c2ee0 2024-09-13T17:08:40.876 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:40.876 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:40.876 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:08:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabec0666c0 msgr2=0x7fabec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabec0666c0 0x7fabec068b80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fabf00ae000 tx=0x7fabf0008040 comp rx=0 tx=0).stop 2024-09-13T17:08:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac000c2ee0 msgr2=0x7fac00144880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac000c2ee0 0x7fac00144880 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fabf400b550 tx=0x7fabf400ba10 comp rx=0 tx=0).stop 2024-09-13T17:08:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 shutdown_connections 2024-09-13T17:08:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabec0666c0 0x7fabec068b80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fac000c6270 0x7fac00144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:40.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac000c2ee0 0x7fac00144880 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 --2- 172.21.15.39:0/1513453441 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fac00065ee0 0x7fac00143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 >> 172.21.15.39:0/1513453441 conn(0x7fac000c3310 msgr2=0x7fac000bfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 shutdown_connections 2024-09-13T17:08:40.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:40.875+0000 7fac056c1640 1 -- 172.21.15.39:0/1513453441 wait complete. 2024-09-13T17:08:40.896 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:08:42.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:43.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc756f89640 1 Processor -- start 2024-09-13T17:08:43.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc756f89640 1 -- start start 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc756f89640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc75014aa40 0x7fc7500c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc756f89640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7500c1750 0x7fc7500c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc756f89640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 0x7fc7500c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc756f89640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc750151b30 con 0x7fc7500c1750 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc756f89640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7500c6510 con 0x7fc75014aa40 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc756f89640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500c76a0 con 0x7fc7500c4710 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc74ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7500c1750 0x7fc7500c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc74ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7500c1750 0x7fc7500c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56252/0 (socket says 172.21.15.39:56252) 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc74ffff640 1 -- 172.21.15.39:0/2819171592 learned_addr learned my addr 172.21.15.39:0/2819171592 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:43.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc754d25640 1 --2- 172.21.15.39:0/2819171592 >> v2:172.21.15.84:3300/0 conn(0x7fc75014aa40 0x7fc7500c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc755526640 1 --2- 172.21.15.39:0/2819171592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 0x7fc7500c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc74ffff640 1 -- 172.21.15.39:0/2819171592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 msgr2=0x7fc7500c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc74ffff640 1 --2- 172.21.15.39:0/2819171592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 0x7fc7500c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc74ffff640 1 -- 172.21.15.39:0/2819171592 >> v2:172.21.15.84:3300/0 conn(0x7fc75014aa40 msgr2=0x7fc7500c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc74ffff640 1 --2- 172.21.15.39:0/2819171592 >> v2:172.21.15.84:3300/0 conn(0x7fc75014aa40 0x7fc7500c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.039+0000 7fc74ffff640 1 -- 172.21.15.39:0/2819171592 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc750151570 con 0x7fc7500c1750 2024-09-13T17:08:43.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc74ffff640 1 --2- 172.21.15.39:0/2819171592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500c1750 0x7fc7500c5a90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc75006fde0 tx=0x7fc74401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caab930461d72d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:43.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc74f7fe640 1 -- 172.21.15.39:0/2819171592 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7440ab620 con 0x7fc7500c1750 2024-09-13T17:08:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc74f7fe640 1 -- 172.21.15.39:0/2819171592 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7440ab780 con 0x7fc7500c1750 2024-09-13T17:08:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc74f7fe640 1 -- 172.21.15.39:0/2819171592 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7440abb10 con 0x7fc7500c1750 2024-09-13T17:08:43.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc756f89640 1 -- 172.21.15.39:0/2819171592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500c1750 msgr2=0x7fc7500c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc756f89640 1 --2- 172.21.15.39:0/2819171592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500c1750 0x7fc7500c5a90 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc75006fde0 tx=0x7fc74401c460 comp rx=0 tx=0).stop 2024-09-13T17:08:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc756f89640 1 -- 172.21.15.39:0/2819171592 shutdown_connections 2024-09-13T17:08:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc756f89640 1 --2- 172.21.15.39:0/2819171592 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 0x7fc7500c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc756f89640 1 --2- 172.21.15.39:0/2819171592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500c1750 0x7fc7500c5a90 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc756f89640 1 --2- 172.21.15.39:0/2819171592 >> v2:172.21.15.84:3300/0 conn(0x7fc75014aa40 0x7fc7500c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc756f89640 1 -- 172.21.15.39:0/2819171592 >> 172.21.15.39:0/2819171592 conn(0x7fc7500c3310 msgr2=0x7fc7500c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc756f89640 1 -- 172.21.15.39:0/2819171592 shutdown_connections 2024-09-13T17:08:43.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.043+0000 7fc756f89640 1 -- 172.21.15.39:0/2819171592 wait complete. 2024-09-13T17:08:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 Processor -- start 2024-09-13T17:08:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 -- start start 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7500c1750 0x7fc7500a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 0x7fc7501ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75014aa40 0x7fc7500716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7501efe50 con 0x7fc75014aa40 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500a7f50 con 0x7fc7500c1750 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500a8230 con 0x7fc7500c4710 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc755526640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75014aa40 0x7fc7500716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc755526640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75014aa40 0x7fc7500716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56254/0 (socket says 172.21.15.39:56254) 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc755526640 1 -- 172.21.15.39:0/1003949076 learned_addr learned my addr 172.21.15.39:0/1003949076 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc74ffff640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 0x7fc7501ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:43.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc754d25640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7500c1750 0x7fc7500a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc755526640 1 -- 172.21.15.39:0/1003949076 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 msgr2=0x7fc7501ebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc755526640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 0x7fc7501ebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc755526640 1 -- 172.21.15.39:0/1003949076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7500c1750 msgr2=0x7fc7500a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc755526640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7500c1750 0x7fc7500a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc755526640 1 -- 172.21.15.39:0/1003949076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440094d0 con 0x7fc75014aa40 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc755526640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75014aa40 0x7fc7500716b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fc74800d580 tx=0x7fc74800da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc74d7fa640 1 -- 172.21.15.39:0/1003949076 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748014620 con 0x7fc75014aa40 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc74d7fa640 1 -- 172.21.15.39:0/1003949076 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748014780 con 0x7fc75014aa40 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc74d7fa640 1 -- 172.21.15.39:0/1003949076 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748014b30 con 0x7fc75014aa40 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc750145b40 con 0x7fc75014aa40 2024-09-13T17:08:43.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7501f00a0 con 0x7fc75014aa40 2024-09-13T17:08:43.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.047+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7500a6a30 con 0x7fc75014aa40 2024-09-13T17:08:43.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.051+0000 7fc74d7fa640 1 -- 172.21.15.39:0/1003949076 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc748014c90 con 0x7fc75014aa40 2024-09-13T17:08:43.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.051+0000 7fc74d7fa640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc73c066690 0x7fc73c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:43.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.051+0000 7fc74d7fa640 1 -- 172.21.15.39:0/1003949076 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7fc748085450 con 0x7fc75014aa40 2024-09-13T17:08:43.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.051+0000 7fc754d25640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc73c066690 0x7fc73c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:43.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.051+0000 7fc754d25640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc73c066690 0x7fc73c068b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc740003fd0 tx=0x7fc7400039c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.055+0000 7fc74d7fa640 1 -- 172.21.15.39:0/1003949076 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc748051b20 con 0x7fc75014aa40 2024-09-13T17:08:43.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.303+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc75014ed20 con 0x7fc75014aa40 2024-09-13T17:08:43.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.303+0000 7fc74d7fa640 1 -- 172.21.15.39:0/1003949076 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7fc74800b310 con 0x7fc75014aa40 2024-09-13T17:08:43.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:43.307 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:43.307 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:08:43.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc73c066690 msgr2=0x7fc73c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:43.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc73c066690 0x7fc73c068b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc740003fd0 tx=0x7fc7400039c0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75014aa40 msgr2=0x7fc7500716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75014aa40 0x7fc7500716b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fc74800d580 tx=0x7fc74800da40 comp rx=0 tx=0).stop 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 shutdown_connections 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc73c066690 0x7fc73c068b50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75014aa40 0x7fc7500716b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7500c4710 0x7fc7501ebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 --2- 172.21.15.39:0/1003949076 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7500c1750 0x7fc7500a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 >> 172.21.15.39:0/1003949076 conn(0x7fc7500c3310 msgr2=0x7fc7500c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 shutdown_connections 2024-09-13T17:08:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:43.307+0000 7fc756f89640 1 -- 172.21.15.39:0/1003949076 wait complete. 2024-09-13T17:08:43.328 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:08:45.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e21ad4640 1 Processor -- start 2024-09-13T17:08:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e21ad4640 1 -- start start 2024-09-13T17:08:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e21ad4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2e1c071dd0 0x7f2e1c154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e21ad4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c0c5a30 con 0x7f2e1c151500 2024-09-13T17:08:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e21ad4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c1576a0 con 0x7f2e1c0c7800 2024-09-13T17:08:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e21ad4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c157980 con 0x7f2e1c071dd0 2024-09-13T17:08:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e12ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2e1c071dd0 0x7f2e1c154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e12ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2e1c071dd0 0x7f2e1c154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56262/0 (socket says 172.21.15.39:56262) 2024-09-13T17:08:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e12ffd640 1 -- 172.21.15.39:0/3805417720 learned_addr learned my addr 172.21.15.39:0/3805417720 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e12ffd640 1 -- 172.21.15.39:0/3805417720 >> v1:172.21.15.84:6789/0 conn(0x7f2e1c0c7800 legacy=0x7f2e1c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e12ffd640 1 -- 172.21.15.39:0/3805417720 >> v1:172.21.15.39:6789/0 conn(0x7f2e1c151500 legacy=0x7f2e1c0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.471+0000 7f2e12ffd640 1 -- 172.21.15.39:0/3805417720 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c157c60 con 0x7f2e1c071dd0 2024-09-13T17:08:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e12ffd640 1 --2- 172.21.15.39:0/3805417720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c071dd0 0x7f2e1c154070 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2e040098a0 tx=0x7f2e0401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ef0267f952a6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e1affd640 1 -- 172.21.15.39:0/3805417720 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e040a6040 con 0x7f2e1c071dd0 2024-09-13T17:08:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e1affd640 1 -- 172.21.15.39:0/3805417720 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e040aa600 con 0x7f2e1c071dd0 2024-09-13T17:08:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e1affd640 1 -- 172.21.15.39:0/3805417720 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e040aa990 con 0x7f2e1c071dd0 2024-09-13T17:08:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e21ad4640 1 -- 172.21.15.39:0/3805417720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c071dd0 msgr2=0x7f2e1c154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e21ad4640 1 --2- 172.21.15.39:0/3805417720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c071dd0 0x7f2e1c154070 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2e040098a0 tx=0x7f2e0401c480 comp rx=0 tx=0).stop 2024-09-13T17:08:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e21ad4640 1 -- 172.21.15.39:0/3805417720 shutdown_connections 2024-09-13T17:08:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e21ad4640 1 --2- 172.21.15.39:0/3805417720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c071dd0 0x7f2e1c154070 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e21ad4640 1 -- 172.21.15.39:0/3805417720 >> 172.21.15.39:0/3805417720 conn(0x7f2e1c0ab530 msgr2=0x7f2e1c0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e21ad4640 1 -- 172.21.15.39:0/3805417720 shutdown_connections 2024-09-13T17:08:45.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.475+0000 7f2e21ad4640 1 -- 172.21.15.39:0/3805417720 wait complete. 2024-09-13T17:08:45.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 Processor -- start 2024-09-13T17:08:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 -- start start 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e1c071dd0 0x7f2e1c149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c0c7800 0x7f2e1c1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e1c151500 0x7f2e1c1ef460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c1533b0 con 0x7f2e1c0c7800 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c1ebe70 con 0x7f2e1c071dd0 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c147fd0 con 0x7f2e1c151500 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e12ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c0c7800 0x7f2e1c1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e12ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c0c7800 0x7f2e1c1561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56278/0 (socket says 172.21.15.39:56278) 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e12ffd640 1 -- 172.21.15.39:0/49985019 learned_addr learned my addr 172.21.15.39:0/49985019 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e1b7fe640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e1c071dd0 0x7f2e1c149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e1bfff640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e1c151500 0x7f2e1c1ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e12ffd640 1 -- 172.21.15.39:0/49985019 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e1c151500 msgr2=0x7f2e1c1ef460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e12ffd640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e1c151500 0x7f2e1c1ef460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e12ffd640 1 -- 172.21.15.39:0/49985019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e1c071dd0 msgr2=0x7f2e1c149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e12ffd640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e1c071dd0 0x7f2e1c149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e12ffd640 1 -- 172.21.15.39:0/49985019 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e040094d0 con 0x7f2e1c0c7800 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e12ffd640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c0c7800 0x7f2e1c1561a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2e04019090 tx=0x7f2e0401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e18ff9640 1 -- 172.21.15.39:0/49985019 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e040b3800 con 0x7f2e1c0c7800 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e18ff9640 1 -- 172.21.15.39:0/49985019 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e040b39a0 con 0x7f2e1c0c7800 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c152980 con 0x7f2e1c0c7800 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e18ff9640 1 -- 172.21.15.39:0/49985019 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e040b3d50 con 0x7f2e1c0c7800 2024-09-13T17:08:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c1ec350 con 0x7f2e1c0c7800 2024-09-13T17:08:45.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.479+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c1ec870 con 0x7f2e1c0c7800 2024-09-13T17:08:45.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.483+0000 7f2e18ff9640 1 -- 172.21.15.39:0/49985019 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e040c6430 con 0x7f2e1c0c7800 2024-09-13T17:08:45.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.483+0000 7f2e18ff9640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2dfc066690 0x7f2dfc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:45.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.483+0000 7f2e18ff9640 1 -- 172.21.15.39:0/49985019 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7f2e0411fbb0 con 0x7f2e1c0c7800 2024-09-13T17:08:45.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.483+0000 7f2e1b7fe640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2dfc066690 0x7f2dfc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:45.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.483+0000 7f2e1b7fe640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2dfc066690 0x7f2dfc068b50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000f70 tx=0x7f2e0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:45.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.487+0000 7f2e18ff9640 1 -- 172.21.15.39:0/49985019 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2e1c1ec870 con 0x7f2e1c0c7800 2024-09-13T17:08:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e1c1ec870 con 0x7f2e1c0c7800 2024-09-13T17:08:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e18ff9640 1 -- 172.21.15.39:0/49985019 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f2e1c1ec870 con 0x7f2e1c0c7800 2024-09-13T17:08:45.721 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:45.721 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:08:45.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2dfc066690 msgr2=0x7f2dfc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:45.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2dfc066690 0x7f2dfc068b50 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c000f70 tx=0x7f2e0c009040 comp rx=0 tx=0).stop 2024-09-13T17:08:45.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c0c7800 msgr2=0x7f2e1c1561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:45.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c0c7800 0x7f2e1c1561a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2e04019090 tx=0x7f2e0401acc0 comp rx=0 tx=0).stop 2024-09-13T17:08:45.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 shutdown_connections 2024-09-13T17:08:45.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2dfc066690 0x7f2dfc068b50 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:45.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e1c151500 0x7f2e1c1ef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:45.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1c0c7800 0x7f2e1c1561a0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:45.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 --2- 172.21.15.39:0/49985019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e1c071dd0 0x7f2e1c149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:45.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 >> 172.21.15.39:0/49985019 conn(0x7f2e1c0ab530 msgr2=0x7f2e1c15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:45.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 shutdown_connections 2024-09-13T17:08:45.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:45.719+0000 7f2e21ad4640 1 -- 172.21.15.39:0/49985019 wait complete. 2024-09-13T17:08:45.741 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:08:47.742 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:47.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.899+0000 7feb817d8640 1 Processor -- start 2024-09-13T17:08:47.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.899+0000 7feb817d8640 1 -- start start 2024-09-13T17:08:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb817d8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feb7c0c7960 0x7feb7c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb817d8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c0c5e50 con 0x7feb7c06fa50 2024-09-13T17:08:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb817d8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c06ff50 con 0x7feb7c0c6000 2024-09-13T17:08:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb817d8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7c072000 con 0x7feb7c0c7960 2024-09-13T17:08:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb7affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feb7c0c7960 0x7feb7c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb7affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feb7c0c7960 0x7feb7c155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56288/0 (socket says 172.21.15.39:56288) 2024-09-13T17:08:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb7a7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7feb7c06fa50 0x7feb7c155f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:32794/0 (socket says 172.21.15.39:32794) 2024-09-13T17:08:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb7affd640 1 -- 172.21.15.39:0/3965275302 learned_addr learned my addr 172.21.15.39:0/3965275302 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:47.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb7affd640 1 -- 172.21.15.39:0/3965275302 >> v1:172.21.15.84:6789/0 conn(0x7feb7c0c6000 legacy=0x7feb7c0c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:47.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb7affd640 1 -- 172.21.15.39:0/3965275302 >> v1:172.21.15.39:6789/0 conn(0x7feb7c06fa50 legacy=0x7feb7c155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:47.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.903+0000 7feb7affd640 1 -- 172.21.15.39:0/3965275302 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c0722e0 con 0x7feb7c0c7960 2024-09-13T17:08:47.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb7affd640 1 --2- 172.21.15.39:0/3965275302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 0x7feb7c155a20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009870 tx=0x7feb6c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f79499aa9b9f9be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:47.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb79ffb640 1 -- 172.21.15.39:0/3965275302 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0ac600 con 0x7feb7c0c7960 2024-09-13T17:08:47.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb79ffb640 1 -- 172.21.15.39:0/3965275302 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6c0a8050 con 0x7feb7c0c7960 2024-09-13T17:08:47.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb79ffb640 1 -- 172.21.15.39:0/3965275302 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0ac980 con 0x7feb7c0c7960 2024-09-13T17:08:47.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 -- 172.21.15.39:0/3965275302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 msgr2=0x7feb7c155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:47.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 --2- 172.21.15.39:0/3965275302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 0x7feb7c155a20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009870 tx=0x7feb6c01c410 comp rx=0 tx=0).stop 2024-09-13T17:08:47.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 -- 172.21.15.39:0/3965275302 shutdown_connections 2024-09-13T17:08:47.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 --2- 172.21.15.39:0/3965275302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 0x7feb7c155a20 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:47.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 -- 172.21.15.39:0/3965275302 >> 172.21.15.39:0/3965275302 conn(0x7feb7c06c020 msgr2=0x7feb7c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:47.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 -- 172.21.15.39:0/3965275302 shutdown_connections 2024-09-13T17:08:47.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 -- 172.21.15.39:0/3965275302 wait complete. 2024-09-13T17:08:47.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 Processor -- start 2024-09-13T17:08:47.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 -- start start 2024-09-13T17:08:47.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb7c06fa50 0x7feb7c0ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:47.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb7c0c6000 0x7feb7c14d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:47.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 0x7feb7c150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:47.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c14e560 con 0x7feb7c0c7960 2024-09-13T17:08:47.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c1ea4d0 con 0x7feb7c0c6000 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb817d8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c14dfa0 con 0x7feb7c06fa50 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb7b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 0x7feb7c150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb7b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 0x7feb7c150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56298/0 (socket says 172.21.15.39:56298) 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.907+0000 7feb7b7fe640 1 -- 172.21.15.39:0/2649428853 learned_addr learned my addr 172.21.15.39:0/2649428853 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7affd640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb7c06fa50 0x7feb7c0ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7a7fc640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb7c0c6000 0x7feb7c14d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7b7fe640 1 -- 172.21.15.39:0/2649428853 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb7c06fa50 msgr2=0x7feb7c0ab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7b7fe640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb7c06fa50 0x7feb7c0ab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7b7fe640 1 -- 172.21.15.39:0/2649428853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb7c0c6000 msgr2=0x7feb7c14d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7b7fe640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb7c0c6000 0x7feb7c14d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7b7fe640 1 -- 172.21.15.39:0/2649428853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6c0094d0 con 0x7feb7c0c7960 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7b7fe640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 0x7feb7c150820 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7feb7400c7a0 tx=0x7feb7400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:47.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb637fe640 1 -- 172.21.15.39:0/2649428853 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb74003ba0 con 0x7feb7c0c7960 2024-09-13T17:08:47.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb637fe640 1 -- 172.21.15.39:0/2649428853 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb74003d00 con 0x7feb7c0c7960 2024-09-13T17:08:47.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c14e280 con 0x7feb7c0c7960 2024-09-13T17:08:47.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb637fe640 1 -- 172.21.15.39:0/2649428853 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7400aa30 con 0x7feb7c0c7960 2024-09-13T17:08:47.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c1ea9c0 con 0x7feb7c0c7960 2024-09-13T17:08:47.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c0c2570 con 0x7feb7c0c7960 2024-09-13T17:08:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb637fe640 1 -- 172.21.15.39:0/2649428853 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feb7400ab90 con 0x7feb7c0c7960 2024-09-13T17:08:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb637fe640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb68066690 0x7feb68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7affd640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb68066690 0x7feb68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb637fe640 1 -- 172.21.15.39:0/2649428853 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 8873+0+0 (secure 0 0 0) 0x7feb74084c50 con 0x7feb7c0c7960 2024-09-13T17:08:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.911+0000 7feb7affd640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb68066690 0x7feb68068b50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0ae000 tx=0x7feb6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:47.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:47.915+0000 7feb637fe640 1 -- 172.21.15.39:0/2649428853 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feb7408a020 con 0x7feb7c0c7960 2024-09-13T17:08:48.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.143+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb7c14d580 con 0x7feb7c0c7960 2024-09-13T17:08:48.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.143+0000 7feb637fe640 1 -- 172.21.15.39:0/2649428853 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5943 (secure 0 0 0) 0x7feb74051310 con 0x7feb7c0c7960 2024-09-13T17:08:48.147 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:48.147 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":6071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:07:23.194224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7230},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7230":{"gid":7230,"name":"a","rank":0,"incarnation":132,"state":"up:active","state_seq":21,"addr":"172.21.15.39:6835/3941977343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3941977343},{"type":"v1","addr":"172.21.15.39:6835","nonce":3941977343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:48.147 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 141 2024-09-13T17:08:48.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb68066690 msgr2=0x7feb68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:48.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb68066690 0x7feb68068b50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0ae000 tx=0x7feb6c008040 comp rx=0 tx=0).stop 2024-09-13T17:08:48.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 msgr2=0x7feb7c150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:48.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 0x7feb7c150820 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7feb7400c7a0 tx=0x7feb7400cc60 comp rx=0 tx=0).stop 2024-09-13T17:08:48.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 shutdown_connections 2024-09-13T17:08:48.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feb68066690 0x7feb68068b50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:48.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb7c0c6000 0x7feb7c14d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:48.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7c0c7960 0x7feb7c150820 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:48.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 --2- 172.21.15.39:0/2649428853 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb7c06fa50 0x7feb7c0ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:48.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 >> 172.21.15.39:0/2649428853 conn(0x7feb7c06c020 msgr2=0x7feb7c0ac140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:48.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 shutdown_connections 2024-09-13T17:08:48.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:48.147+0000 7feb817d8640 1 -- 172.21.15.39:0/2649428853 wait complete. 2024-09-13T17:08:48.166 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:08:50.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:50.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda0d4c9640 1 Processor -- start 2024-09-13T17:08:50.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda0d4c9640 1 -- start start 2024-09-13T17:08:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda0d4c9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fda080c5070 0x7fda0806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda0d4c9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda080a5120 0x7fda080c0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda0d4c9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fda08150c20 0x7fda081510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda0d4c9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0806d7d0 con 0x7fda080c5070 2024-09-13T17:08:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda0d4c9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda080c1c60 con 0x7fda08150c20 2024-09-13T17:08:50.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda0d4c9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda080c1f40 con 0x7fda080a5120 2024-09-13T17:08:50.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda06ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fda080c5070 0x7fda0806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda06ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fda080c5070 0x7fda0806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46922/0 (socket says 172.21.15.39:46922) 2024-09-13T17:08:50.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda06ffd640 1 -- 172.21.15.39:0/3598422184 learned_addr learned my addr 172.21.15.39:0/3598422184 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda077fe640 1 --2- 172.21.15.39:0/3598422184 >> v2:172.21.15.84:3300/0 conn(0x7fda08150c20 0x7fda081510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda067fc640 1 --2- 172.21.15.39:0/3598422184 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda080a5120 0x7fda080c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda06ffd640 1 -- 172.21.15.39:0/3598422184 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda080a5120 msgr2=0x7fda080c0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda06ffd640 1 --2- 172.21.15.39:0/3598422184 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda080a5120 0x7fda080c0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda06ffd640 1 -- 172.21.15.39:0/3598422184 >> v2:172.21.15.84:3300/0 conn(0x7fda08150c20 msgr2=0x7fda081510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda06ffd640 1 --2- 172.21.15.39:0/3598422184 >> v2:172.21.15.84:3300/0 conn(0x7fda08150c20 0x7fda081510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.291+0000 7fda06ffd640 1 -- 172.21.15.39:0/3598422184 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080c2220 con 0x7fda080c5070 2024-09-13T17:08:50.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda06ffd640 1 --2- 172.21.15.39:0/3598422184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080c5070 0x7fda0806c210 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fda080c2520 tx=0x7fd9f801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca483be56f9e5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:50.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda05ffb640 1 -- 172.21.15.39:0/3598422184 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80ab600 con 0x7fda080c5070 2024-09-13T17:08:50.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda05ffb640 1 -- 172.21.15.39:0/3598422184 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f80a7050 con 0x7fda080c5070 2024-09-13T17:08:50.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda05ffb640 1 -- 172.21.15.39:0/3598422184 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80ab980 con 0x7fda080c5070 2024-09-13T17:08:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda0d4c9640 1 -- 172.21.15.39:0/3598422184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080c5070 msgr2=0x7fda0806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/3598422184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080c5070 0x7fda0806c210 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fda080c2520 tx=0x7fd9f801c490 comp rx=0 tx=0).stop 2024-09-13T17:08:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda0d4c9640 1 -- 172.21.15.39:0/3598422184 shutdown_connections 2024-09-13T17:08:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/3598422184 >> v2:172.21.15.84:3300/0 conn(0x7fda08150c20 0x7fda081510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/3598422184 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda080a5120 0x7fda080c0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/3598422184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080c5070 0x7fda0806c210 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda0d4c9640 1 -- 172.21.15.39:0/3598422184 >> 172.21.15.39:0/3598422184 conn(0x7fda080a7800 msgr2=0x7fda080a7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:50.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda0d4c9640 1 -- 172.21.15.39:0/3598422184 shutdown_connections 2024-09-13T17:08:50.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.295+0000 7fda0d4c9640 1 -- 172.21.15.39:0/3598422184 wait complete. 2024-09-13T17:08:50.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda0d4c9640 1 Processor -- start 2024-09-13T17:08:50.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda0d4c9640 1 -- start start 2024-09-13T17:08:50.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda0d4c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080a5120 0x7fda08152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda0d4c9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda080c5070 0x7fda08159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda0d4c9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda08150c20 0x7fda081572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda0d4c9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda081ef610 con 0x7fda080a5120 2024-09-13T17:08:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda06ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080a5120 0x7fda08152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda0d4c9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0815a6b0 con 0x7fda080c5070 2024-09-13T17:08:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda0d4c9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda081ef860 con 0x7fda08150c20 2024-09-13T17:08:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.299+0000 7fda06ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080a5120 0x7fda08152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46934/0 (socket says 172.21.15.39:46934) 2024-09-13T17:08:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda06ffd640 1 -- 172.21.15.39:0/384446650 learned_addr learned my addr 172.21.15.39:0/384446650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda06ffd640 1 -- 172.21.15.39:0/384446650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda08150c20 msgr2=0x7fda081572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda06ffd640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda08150c20 0x7fda081572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda06ffd640 1 -- 172.21.15.39:0/384446650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda080c5070 msgr2=0x7fda08159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda06ffd640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda080c5070 0x7fda08159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda06ffd640 1 -- 172.21.15.39:0/384446650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80094d0 con 0x7fda080a5120 2024-09-13T17:08:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda06ffd640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080a5120 0x7fda08152ae0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd9f801cd30 tx=0x7fd9f801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fd9ef7fe640 1 -- 172.21.15.39:0/384446650 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80b6420 con 0x7fda080a5120 2024-09-13T17:08:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fd9ef7fe640 1 -- 172.21.15.39:0/384446650 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f80b6580 con 0x7fda080a5120 2024-09-13T17:08:50.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda081efb40 con 0x7fda080a5120 2024-09-13T17:08:50.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fd9ef7fe640 1 -- 172.21.15.39:0/384446650 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80b6930 con 0x7fda080a5120 2024-09-13T17:08:50.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda081f0100 con 0x7fda080a5120 2024-09-13T17:08:50.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fd9ef7fe640 1 -- 172.21.15.39:0/384446650 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9f80b2040 con 0x7fda080a5120 2024-09-13T17:08:50.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda08156010 con 0x7fda080a5120 2024-09-13T17:08:50.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fd9ef7fe640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f4066690 0x7fd9f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fd9ef7fe640 1 -- 172.21.15.39:0/384446650 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fd9f811f2c0 con 0x7fda080a5120 2024-09-13T17:08:50.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.303+0000 7fda067fc640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f4066690 0x7fd9f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.307+0000 7fda067fc640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f4066690 0x7fd9f4068b50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fda081534c0 tx=0x7fd9fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:50.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.311+0000 7fd9ef7fe640 1 -- 172.21.15.39:0/384446650 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd9f80eb950 con 0x7fda080a5120 2024-09-13T17:08:50.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.591+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda0806b550 con 0x7fda080a5120 2024-09-13T17:08:50.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.591+0000 7fd9ef7fe640 1 -- 172.21.15.39:0/384446650 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) v1 ==== 95+0+5184 (secure 0 0 0) 0x7fd9f80eef70 con 0x7fda080a5120 2024-09-13T17:08:50.590 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:50.591 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":142,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:49.821137+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:50.591 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 142 2024-09-13T17:08:50.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f4066690 msgr2=0x7fd9f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f4066690 0x7fd9f4068b50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fda081534c0 tx=0x7fd9fc008040 comp rx=0 tx=0).stop 2024-09-13T17:08:50.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080a5120 msgr2=0x7fda08152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080a5120 0x7fda08152ae0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd9f801cd30 tx=0x7fd9f801acd0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 shutdown_connections 2024-09-13T17:08:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9f4066690 0x7fd9f4068b50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda080c5070 0x7fda08159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda08150c20 0x7fda081572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 --2- 172.21.15.39:0/384446650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080a5120 0x7fda08152ae0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 >> 172.21.15.39:0/384446650 conn(0x7fda080a7800 msgr2=0x7fda0806fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 shutdown_connections 2024-09-13T17:08:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.595+0000 7fda0d4c9640 1 -- 172.21.15.39:0/384446650 wait complete. 2024-09-13T17:08:50.611 INFO:tasks.mds_thrash.fs.[b]:mds.a down, removed from mdsmap 2024-09-13T17:08:50.611 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:08:50.612 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:50.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.735+0000 7f25ba390640 1 Processor -- start 2024-09-13T17:08:50.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.735+0000 7f25ba390640 1 -- start start 2024-09-13T17:08:50.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25ba390640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f25b40ca080 0x7f25b4151000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25ba390640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b4151540 0x7f25b4070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25ba390640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b40c9710 con 0x7f25b40714f0 2024-09-13T17:08:50.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25ba390640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b4157fc0 con 0x7f25b40ca080 2024-09-13T17:08:50.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25ba390640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b4071980 con 0x7f25b4151540 2024-09-13T17:08:50.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25abfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b4151540 0x7f25b4070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25b3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f25b40ca080 0x7f25b4151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25abfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b4151540 0x7f25b4070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:60496/0 (socket says 172.21.15.39:60496) 2024-09-13T17:08:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25abfff640 1 -- 172.21.15.39:0/2070287580 learned_addr learned my addr 172.21.15.39:0/2070287580 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25b892d640 1 --1- 172.21.15.39:0/2070287580 >> v1:172.21.15.84:6789/0 conn(0x7f25b40714f0 0x7f25b4153d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:43918/0 (socket says 172.21.15.39:43918) 2024-09-13T17:08:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25abfff640 1 -- 172.21.15.39:0/2070287580 >> v2:172.21.15.84:3300/0 conn(0x7f25b40ca080 msgr2=0x7f25b4151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25abfff640 1 --2- 172.21.15.39:0/2070287580 >> v2:172.21.15.84:3300/0 conn(0x7f25b40ca080 0x7f25b4151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25abfff640 1 -- 172.21.15.39:0/2070287580 >> v1:172.21.15.84:6789/0 conn(0x7f25b40714f0 legacy=0x7f25b4153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25abfff640 1 -- 172.21.15.39:0/2070287580 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4157340 con 0x7f25b4151540 2024-09-13T17:08:50.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.739+0000 7f25abfff640 1 --2- 172.21.15.39:0/2070287580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b4151540 0x7f25b4070fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f259c0098a0 tx=0x7f259c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56fd83b75fb0595f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:50.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25b37fe640 1 -- 172.21.15.39:0/2070287580 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c0ab600 con 0x7f25b4151540 2024-09-13T17:08:50.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25b37fe640 1 -- 172.21.15.39:0/2070287580 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c0a7050 con 0x7f25b4151540 2024-09-13T17:08:50.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25b37fe640 1 -- 172.21.15.39:0/2070287580 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c0ab980 con 0x7f25b4151540 2024-09-13T17:08:50.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 -- 172.21.15.39:0/2070287580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b4151540 msgr2=0x7f25b4070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 --2- 172.21.15.39:0/2070287580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b4151540 0x7f25b4070fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f259c0098a0 tx=0x7f259c01c420 comp rx=0 tx=0).stop 2024-09-13T17:08:50.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 -- 172.21.15.39:0/2070287580 shutdown_connections 2024-09-13T17:08:50.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 --2- 172.21.15.39:0/2070287580 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b4151540 0x7f25b4070fb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 --2- 172.21.15.39:0/2070287580 >> v2:172.21.15.84:3300/0 conn(0x7f25b40ca080 0x7f25b4151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 -- 172.21.15.39:0/2070287580 >> 172.21.15.39:0/2070287580 conn(0x7f25b406a210 msgr2=0x7f25b406a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:50.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 -- 172.21.15.39:0/2070287580 shutdown_connections 2024-09-13T17:08:50.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 -- 172.21.15.39:0/2070287580 wait complete. 2024-09-13T17:08:50.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 Processor -- start 2024-09-13T17:08:50.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 -- start start 2024-09-13T17:08:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25b40714f0 0x7f25b4145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b40ca080 0x7f25b4147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25b4151540 0x7f25b41565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40ad620 con 0x7f25b4151540 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40ad890 con 0x7f25b40714f0 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.743+0000 7f25ba390640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40ae8a0 con 0x7f25b40ca080 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b892d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25b4151540 0x7f25b41565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b892d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25b4151540 0x7f25b41565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46946/0 (socket says 172.21.15.39:46946) 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b892d640 1 -- 172.21.15.39:0/2505405418 learned_addr learned my addr 172.21.15.39:0/2505405418 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b3fff640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25b40714f0 0x7f25b4145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25abfff640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b40ca080 0x7f25b4147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b892d640 1 -- 172.21.15.39:0/2505405418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b40ca080 msgr2=0x7f25b4147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b892d640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b40ca080 0x7f25b4147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b892d640 1 -- 172.21.15.39:0/2505405418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25b40714f0 msgr2=0x7f25b4145db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b892d640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25b40714f0 0x7f25b4145db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b892d640 1 -- 172.21.15.39:0/2505405418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c0094d0 con 0x7f25b4151540 2024-09-13T17:08:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b892d640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25b4151540 0x7f25b41565b0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00c7a0 tx=0x7f25ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b17fa640 1 -- 172.21.15.39:0/2505405418 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac00a640 con 0x7f25b4151540 2024-09-13T17:08:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b17fa640 1 -- 172.21.15.39:0/2505405418 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac00a7a0 con 0x7f25b4151540 2024-09-13T17:08:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b414d5b0 con 0x7f25b4151540 2024-09-13T17:08:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b17fa640 1 -- 172.21.15.39:0/2505405418 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac00ab10 con 0x7f25b4151540 2024-09-13T17:08:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b414d1c0 con 0x7f25b4151540 2024-09-13T17:08:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b4151000 con 0x7f25b4151540 2024-09-13T17:08:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b17fa640 1 -- 172.21.15.39:0/2505405418 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f25ac00ac70 con 0x7f25b4151540 2024-09-13T17:08:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b17fa640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25940666c0 0x7f2594068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b17fa640 1 -- 172.21.15.39:0/2505405418 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f25ac084590 con 0x7f25b4151540 2024-09-13T17:08:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b3fff640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25940666c0 0x7f2594068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.747+0000 7f25b3fff640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25940666c0 0x7f2594068b80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f25b4153960 tx=0x7f25a4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.755+0000 7f25b17fa640 1 -- 172.21.15.39:0/2505405418 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f25ac050d40 con 0x7f25b4151540 2024-09-13T17:08:50.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25b4071180 con 0x7f25b4151540 2024-09-13T17:08:50.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25b17fa640 1 -- 172.21.15.39:0/2505405418 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+5188 (secure 0 0 0) 0x7f25ac017020 con 0x7f25b4151540 2024-09-13T17:08:50.984 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:50.984 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":143,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:50.835016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:50.985 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 143 2024-09-13T17:08:50.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25940666c0 msgr2=0x7f2594068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25940666c0 0x7f2594068b80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f25b4153960 tx=0x7f25a4009190 comp rx=0 tx=0).stop 2024-09-13T17:08:50.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25b4151540 msgr2=0x7f25b41565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25b4151540 0x7f25b41565b0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00c7a0 tx=0x7f25ac00cc60 comp rx=0 tx=0).stop 2024-09-13T17:08:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 shutdown_connections 2024-09-13T17:08:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f25940666c0 0x7f2594068b80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25b4151540 0x7f25b41565b0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25b40ca080 0x7f25b4147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 --2- 172.21.15.39:0/2505405418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25b40714f0 0x7f25b4145db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 >> 172.21.15.39:0/2505405418 conn(0x7f25b406a210 msgr2=0x7f25b406a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 shutdown_connections 2024-09-13T17:08:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:50.987+0000 7f25ba390640 1 -- 172.21.15.39:0/2505405418 wait complete. 2024-09-13T17:08:51.004 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 143, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 140, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_7338": { "addr": "172.21.15.39:6837/2521680730", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 2521680730, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 2521680730, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7338, "incarnation": 137, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:07:38.167209+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7338 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 143, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6071": { "addr": "172.21.15.84:6833/1750130579", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 1750130579, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 1750130579, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6071, "incarnation": 142, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:08:50.835016+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6071 } } } ], "standbys": [ { "addr": "172.21.15.84:6835/3923043380", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 3923043380, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 3923043380, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 130, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5963, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:08:53.005 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:53.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd376a0b640 1 Processor -- start 2024-09-13T17:08:53.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd376a0b640 1 -- start start 2024-09-13T17:08:53.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd376a0b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd37014aa40 0x7fd3700c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd376a0b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd3700c4710 0x7fd37015b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd376a0b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3700c76a0 con 0x7fd3700c1750 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd376a0b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd370151b30 con 0x7fd3700c4710 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd376a0b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd370151570 con 0x7fd37014aa40 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd3700c1750 0x7fd3700c5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57524/0 (socket says 172.21.15.39:57524) 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36f7fe640 1 -- 172.21.15.39:0/3905076052 learned_addr learned my addr 172.21.15.39:0/3905076052 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd374fa8640 1 --2- 172.21.15.39:0/3905076052 >> v2:172.21.15.84:3300/0 conn(0x7fd3700c4710 0x7fd37015b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36ffff640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd37014aa40 0x7fd3700c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641511595 0 0) 0x7fd3700c76a0 con 0x7fd3700c1750 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd358003530 con 0x7fd3700c1750 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2529146800 0 0) 0x7fd358003530 con 0x7fd3700c1750 2024-09-13T17:08:53.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd37014aa40 msgr2=0x7fd3700c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd37014aa40 0x7fd3700c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 >> v2:172.21.15.84:3300/0 conn(0x7fd3700c4710 msgr2=0x7fd37015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 --2- 172.21.15.39:0/3905076052 >> v2:172.21.15.84:3300/0 conn(0x7fd3700c4710 0x7fd37015b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370151850 con 0x7fd3700c1750 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fd364001680 con 0x7fd3700c1750 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 >> v1:172.21.15.39:6789/0 conn(0x7fd3700c1750 legacy=0x7fd3700c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd358003980 0x7fd358003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580047d0 0x7fd358004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd37014aa40 0x7fd358005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd370151b30 con 0x7fd358003980 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd370151570 con 0x7fd3580047d0 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35800b990 con 0x7fd37014aa40 2024-09-13T17:08:53.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36ffff640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd358003980 0x7fd358003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:53.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36f7fe640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580047d0 0x7fd358004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:53.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd374fa8640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd37014aa40 0x7fd358005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:53.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36ffff640 1 -- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd37014aa40 msgr2=0x7fd358005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:53.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36ffff640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd37014aa40 0x7fd358005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36ffff640 1 -- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580047d0 msgr2=0x7fd358004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:53.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36ffff640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3580047d0 0x7fd358004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.135+0000 7fd36ffff640 1 -- 172.21.15.39:0/3905076052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd358003530 con 0x7fd358003980 2024-09-13T17:08:53.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd36ffff640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd358003980 0x7fd358003da0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd360009b10 tx=0x7fd36001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e00a66a415a63c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:53.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600a5530 con 0x7fd358003980 2024-09-13T17:08:53.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3600a5690 con 0x7fd358003980 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd36effd640 1 -- 172.21.15.39:0/3905076052 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3600a5a40 con 0x7fd358003980 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd376a0b640 1 -- 172.21.15.39:0/3905076052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd358003980 msgr2=0x7fd358003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd376a0b640 1 --2- 172.21.15.39:0/3905076052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd358003980 0x7fd358003da0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd360009b10 tx=0x7fd36001ca40 comp rx=0 tx=0).stop 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd374fa8640 1 -- 172.21.15.39:0/3905076052 reap_dead start 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd376a0b640 1 -- 172.21.15.39:0/3905076052 shutdown_connections 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd376a0b640 1 -- 172.21.15.39:0/3905076052 >> 172.21.15.39:0/3905076052 conn(0x7fd3700c3310 msgr2=0x7fd3700c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd376a0b640 1 -- 172.21.15.39:0/3905076052 shutdown_connections 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd376a0b640 1 -- 172.21.15.39:0/3905076052 wait complete. 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.139+0000 7fd376a0b640 1 Processor -- start 2024-09-13T17:08:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 -- start start 2024-09-13T17:08:53.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd358003980 0x7fd370153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:53.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370065ee0 0x7fd3701f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:53.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3701f0930 0x7fd3701ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:53.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd370154f60 con 0x7fd3701f0930 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3701544c0 con 0x7fd370065ee0 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3701ec760 con 0x7fd358003980 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd374fa8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3701f0930 0x7fd3701ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd374fa8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3701f0930 0x7fd3701ecc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46954/0 (socket says 172.21.15.39:46954) 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd374fa8640 1 -- 172.21.15.39:0/811497945 learned_addr learned my addr 172.21.15.39:0/811497945 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36f7fe640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370065ee0 0x7fd3701f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36ffff640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd358003980 0x7fd370153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd374fa8640 1 -- 172.21.15.39:0/811497945 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd358003980 msgr2=0x7fd370153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd374fa8640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd358003980 0x7fd370153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd374fa8640 1 -- 172.21.15.39:0/811497945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370065ee0 msgr2=0x7fd3701f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd374fa8640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370065ee0 0x7fd3701f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd374fa8640 1 -- 172.21.15.39:0/811497945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3701546d0 con 0x7fd3701f0930 2024-09-13T17:08:53.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36ffff640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd358003980 0x7fd370153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:08:53.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd374fa8640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3701f0930 0x7fd3701ecc20 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd37006fde0 tx=0x7fd36800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:53.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36cff9640 1 -- 172.21.15.39:0/811497945 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368009b90 con 0x7fd3701f0930 2024-09-13T17:08:53.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36cff9640 1 -- 172.21.15.39:0/811497945 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368011040 con 0x7fd3701f0930 2024-09-13T17:08:53.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3701efd10 con 0x7fd3701f0930 2024-09-13T17:08:53.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36cff9640 1 -- 172.21.15.39:0/811497945 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368003be0 con 0x7fd3701f0930 2024-09-13T17:08:53.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3700ac0f0 con 0x7fd3701f0930 2024-09-13T17:08:53.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37014ae20 con 0x7fd3701f0930 2024-09-13T17:08:53.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36cff9640 1 -- 172.21.15.39:0/811497945 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd368017400 con 0x7fd3701f0930 2024-09-13T17:08:53.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36cff9640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd35c066710 0x7fd35c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:53.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36ffff640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd35c066710 0x7fd35c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:53.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.143+0000 7fd36cff9640 1 -- 172.21.15.39:0/811497945 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fd3680851e0 con 0x7fd3701f0930 2024-09-13T17:08:53.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.147+0000 7fd36ffff640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd35c066710 0x7fd35c068bd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd3600af000 tx=0x7fd3600046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:53.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.147+0000 7fd36cff9640 1 -- 172.21.15.39:0/811497945 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd368051900 con 0x7fd3701f0930 2024-09-13T17:08:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd37006ac80 con 0x7fd3701f0930 2024-09-13T17:08:53.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd36cff9640 1 -- 172.21.15.39:0/811497945 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+5185 (secure 0 0 0) 0x7fd368054f20 con 0x7fd3701f0930 2024-09-13T17:08:53.356 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:53.357 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":145,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:53.357 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 145 2024-09-13T17:08:53.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd35c066710 msgr2=0x7fd35c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:53.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd35c066710 0x7fd35c068bd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd3600af000 tx=0x7fd3600046d0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3701f0930 msgr2=0x7fd3701ecc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:53.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3701f0930 0x7fd3701ecc20 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd37006fde0 tx=0x7fd36800f4a0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 shutdown_connections 2024-09-13T17:08:53.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd35c066710 0x7fd35c068bd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd370065ee0 0x7fd3701f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3701f0930 0x7fd3701ecc20 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 --2- 172.21.15.39:0/811497945 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd358003980 0x7fd370153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:53.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 >> 172.21.15.39:0/811497945 conn(0x7fd3700c3310 msgr2=0x7fd37014f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:53.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 shutdown_connections 2024-09-13T17:08:53.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:53.359+0000 7fd376a0b640 1 -- 172.21.15.39:0/811497945 wait complete. 2024-09-13T17:08:53.380 INFO:tasks.mds_thrash.fs.[b]:mds.b has gained rank=0, replacing gid=7230 2024-09-13T17:08:53.381 INFO:tasks.mds_thrash.fs.[b]:waiting for 4 secs before reviving mds.a 2024-09-13T17:08:57.228 INFO:tasks.ceph.mds.b.smithi084.stderr:2024-09-13T17:08:57.231+0000 7fdeb4635640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:08:57.381 INFO:tasks.mds_thrash.fs.[b]:reviving mds.a 2024-09-13T17:08:57.381 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:08:57.381 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:08:57.384 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:08:57.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:57.466 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T17:08:57.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd842acd640 1 Processor -- start 2024-09-13T17:08:57.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd842acd640 1 -- start start 2024-09-13T17:08:57.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd842acd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd83c14aa40 0x7fd83c0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:57.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd842acd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd83c0c1750 0x7fd83c0c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:57.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd842acd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd83c0c4710 0x7fd83c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:57.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd842acd640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83c151b30 con 0x7fd83c0c4710 2024-09-13T17:08:57.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd842acd640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83c0c6510 con 0x7fd83c0c1750 2024-09-13T17:08:57.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd842acd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0c76a0 con 0x7fd83c14aa40 2024-09-13T17:08:57.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd84106a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd83c0c4710 0x7fd83c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:57.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd84106a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd83c0c4710 0x7fd83c0c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46970/0 (socket says 172.21.15.39:46970) 2024-09-13T17:08:57.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd84106a640 1 -- 172.21.15.39:0/3753295163 learned_addr learned my addr 172.21.15.39:0/3753295163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:57.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd840869640 1 --2- 172.21.15.39:0/3753295163 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd83c14aa40 0x7fd83c0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:57.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd83bfff640 1 --2- 172.21.15.39:0/3753295163 >> v2:172.21.15.84:3300/0 conn(0x7fd83c0c1750 0x7fd83c0c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:57.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd84106a640 1 -- 172.21.15.39:0/3753295163 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd83c14aa40 msgr2=0x7fd83c0c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:57.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd84106a640 1 --2- 172.21.15.39:0/3753295163 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd83c14aa40 0x7fd83c0c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd84106a640 1 -- 172.21.15.39:0/3753295163 >> v2:172.21.15.84:3300/0 conn(0x7fd83c0c1750 msgr2=0x7fd83c0c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:57.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd84106a640 1 --2- 172.21.15.39:0/3753295163 >> v2:172.21.15.84:3300/0 conn(0x7fd83c0c1750 0x7fd83c0c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.523+0000 7fd84106a640 1 -- 172.21.15.39:0/3753295163 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c151570 con 0x7fd83c0c4710 2024-09-13T17:08:57.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd84106a640 1 --2- 172.21.15.39:0/3753295163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c4710 0x7fd83c0c5fd0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0618a0 tx=0x7fd83401eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c03bb3a81e47364 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:57.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd83b7fe640 1 -- 172.21.15.39:0/3753295163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8340a79d0 con 0x7fd83c0c4710 2024-09-13T17:08:57.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd83b7fe640 1 -- 172.21.15.39:0/3753295163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8340a7b30 con 0x7fd83c0c4710 2024-09-13T17:08:57.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd83b7fe640 1 -- 172.21.15.39:0/3753295163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8340af3f0 con 0x7fd83c0c4710 2024-09-13T17:08:57.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 -- 172.21.15.39:0/3753295163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c4710 msgr2=0x7fd83c0c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:57.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 --2- 172.21.15.39:0/3753295163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c4710 0x7fd83c0c5fd0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0618a0 tx=0x7fd83401eb40 comp rx=0 tx=0).stop 2024-09-13T17:08:57.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 -- 172.21.15.39:0/3753295163 shutdown_connections 2024-09-13T17:08:57.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 --2- 172.21.15.39:0/3753295163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c4710 0x7fd83c0c5fd0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 --2- 172.21.15.39:0/3753295163 >> v2:172.21.15.84:3300/0 conn(0x7fd83c0c1750 0x7fd83c0c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 --2- 172.21.15.39:0/3753295163 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd83c14aa40 0x7fd83c0c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 -- 172.21.15.39:0/3753295163 >> 172.21.15.39:0/3753295163 conn(0x7fd83c0c3310 msgr2=0x7fd83c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:57.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 -- 172.21.15.39:0/3753295163 shutdown_connections 2024-09-13T17:08:57.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 -- 172.21.15.39:0/3753295163 wait complete. 2024-09-13T17:08:57.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 Processor -- start 2024-09-13T17:08:57.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.527+0000 7fd842acd640 1 -- start start 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd842acd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c1750 0x7fd83c0bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd842acd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd83c0c4710 0x7fd83c1ece70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd842acd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd83c14aa40 0x7fd83c1efc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd842acd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c1ec700 con 0x7fd83c0c1750 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd842acd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c14d7a0 con 0x7fd83c14aa40 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd842acd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0ac490 con 0x7fd83c0c4710 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd840869640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c1750 0x7fd83c0bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd840869640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c1750 0x7fd83c0bfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46982/0 (socket says 172.21.15.39:46982) 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd840869640 1 -- 172.21.15.39:0/4051941259 learned_addr learned my addr 172.21.15.39:0/4051941259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd84106a640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd83c14aa40 0x7fd83c1efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd840869640 1 -- 172.21.15.39:0/4051941259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd83c0c4710 msgr2=0x7fd83c1ece70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd840869640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd83c0c4710 0x7fd83c1ece70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd840869640 1 -- 172.21.15.39:0/4051941259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd83c14aa40 msgr2=0x7fd83c1efc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:57.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd840869640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd83c14aa40 0x7fd83c1efc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd840869640 1 -- 172.21.15.39:0/4051941259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834009b30 con 0x7fd83c0c1750 2024-09-13T17:08:57.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd840869640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c1750 0x7fd83c0bfea0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd83000e870 tx=0x7fd83000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:57.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd8397fa640 1 -- 172.21.15.39:0/4051941259 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830009510 con 0x7fd83c0c1750 2024-09-13T17:08:57.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c0ac1b0 con 0x7fd83c0c1750 2024-09-13T17:08:57.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c0ab6d0 con 0x7fd83c0c1750 2024-09-13T17:08:57.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd8397fa640 1 -- 172.21.15.39:0/4051941259 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83000eec0 con 0x7fd83c0c1750 2024-09-13T17:08:57.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd8397fa640 1 -- 172.21.15.39:0/4051941259 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830005ba0 con 0x7fd83c0c1750 2024-09-13T17:08:57.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c005260 con 0x7fd83c0c1750 2024-09-13T17:08:57.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.531+0000 7fd8397fa640 1 -- 172.21.15.39:0/4051941259 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd830005d00 con 0x7fd83c0c1750 2024-09-13T17:08:57.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.535+0000 7fd8397fa640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8200666c0 0x7fd820068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:57.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.535+0000 7fd83bfff640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8200666c0 0x7fd820068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:57.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.535+0000 7fd8397fa640 1 -- 172.21.15.39:0/4051941259 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fd830084e60 con 0x7fd83c0c1750 2024-09-13T17:08:57.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.535+0000 7fd83bfff640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8200666c0 0x7fd820068b80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd83c06b340 tx=0x7fd828008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:57.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.539+0000 7fd8397fa640 1 -- 172.21.15.39:0/4051941259 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd830051560 con 0x7fd83c0c1750 2024-09-13T17:08:57.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.759+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd81c005a90 con 0x7fd83c0c1750 2024-09-13T17:08:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.759+0000 7fd8397fa640 1 -- 172.21.15.39:0/4051941259 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+5185 (secure 0 0 0) 0x7fd830021090 con 0x7fd83c0c1750 2024-09-13T17:08:57.758 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:08:57.758 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":145,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:08:57.758 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 145 2024-09-13T17:08:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8200666c0 msgr2=0x7fd820068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8200666c0 0x7fd820068b80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd83c06b340 tx=0x7fd828008040 comp rx=0 tx=0).stop 2024-09-13T17:08:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c1750 msgr2=0x7fd83c0bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c1750 0x7fd83c0bfea0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd83000e870 tx=0x7fd83000ec70 comp rx=0 tx=0).stop 2024-09-13T17:08:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 shutdown_connections 2024-09-13T17:08:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8200666c0 0x7fd820068b80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd83c14aa40 0x7fd83c1efc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd83c0c4710 0x7fd83c1ece70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 --2- 172.21.15.39:0/4051941259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd83c0c1750 0x7fd83c0bfea0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 >> 172.21.15.39:0/4051941259 conn(0x7fd83c0c3310 msgr2=0x7fd83c0c36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 shutdown_connections 2024-09-13T17:08:57.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:57.763+0000 7fd842acd640 1 -- 172.21.15.39:0/4051941259 wait complete. 2024-09-13T17:08:57.779 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T17:08:59.780 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:08:59.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.931+0000 7f7fac4d2640 1 Processor -- start 2024-09-13T17:08:59.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.931+0000 7f7fac4d2640 1 -- start start 2024-09-13T17:08:59.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.931+0000 7f7fac4d2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7fa40c4710 0x7f7fa40c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:59.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.931+0000 7f7fac4d2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7fa40c5fd0 0x7f7fa415b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:59.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.931+0000 7f7fac4d2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40c76a0 con 0x7f7fa414aa40 2024-09-13T17:08:59.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.931+0000 7f7fac4d2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa4151b30 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.931+0000 7f7fac4d2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa4151570 con 0x7f7fa40c4710 2024-09-13T17:08:59.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.931+0000 7f7faa26e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7fa414aa40 0x7f7fa40c1210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57536/0 (socket says 172.21.15.39:57536) 2024-09-13T17:08:59.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.931+0000 7f7faa26e640 1 -- 172.21.15.39:0/1835763664 learned_addr learned my addr 172.21.15.39:0/1835763664 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:08:59.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7faaa6f640 1 --2- 172.21.15.39:0/1835763664 >> v2:172.21.15.39:3300/0 conn(0x7f7fa40c5fd0 0x7f7fa415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:59.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7fa9a6d640 1 --2- 172.21.15.39:0/1835763664 >> v2:172.21.15.84:3300/0 conn(0x7f7fa40c4710 0x7f7fa40c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:59.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7fa926c640 1 -- 172.21.15.39:0/1835763664 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718333417 0 0) 0x7f7fa40c76a0 con 0x7f7fa414aa40 2024-09-13T17:08:59.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7fa926c640 1 -- 172.21.15.39:0/1835763664 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f90003530 con 0x7f7fa414aa40 2024-09-13T17:08:59.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7faaa6f640 1 -- 172.21.15.39:0/1835763664 >> v2:172.21.15.84:3300/0 conn(0x7f7fa40c4710 msgr2=0x7f7fa40c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:08:59.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7faaa6f640 1 --2- 172.21.15.39:0/1835763664 >> v2:172.21.15.84:3300/0 conn(0x7f7fa40c4710 0x7f7fa40c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:59.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7faaa6f640 1 -- 172.21.15.39:0/1835763664 >> v1:172.21.15.39:6789/0 conn(0x7f7fa414aa40 legacy=0x7f7fa40c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:59.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7faaa6f640 1 -- 172.21.15.39:0/1835763664 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa4151850 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7faaa6f640 1 --2- 172.21.15.39:0/1835763664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 0x7f7fa415b470 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4072610 tx=0x7f7f9c01ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bf8503e86e3caa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7fa926c640 1 -- 172.21.15.39:0/1835763664 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c00b2d0 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7fa926c640 1 -- 172.21.15.39:0/1835763664 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c0a7840 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7fa926c640 1 -- 172.21.15.39:0/1835763664 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0a7bd0 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7fac4d2640 1 -- 172.21.15.39:0/1835763664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 msgr2=0x7f7fa415b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.935+0000 7f7fac4d2640 1 --2- 172.21.15.39:0/1835763664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 0x7f7fa415b470 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4072610 tx=0x7f7f9c01ea70 comp rx=0 tx=0).stop 2024-09-13T17:08:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- 172.21.15.39:0/1835763664 shutdown_connections 2024-09-13T17:08:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 --2- 172.21.15.39:0/1835763664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 0x7f7fa415b470 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 --2- 172.21.15.39:0/1835763664 >> v2:172.21.15.84:3300/0 conn(0x7f7fa40c4710 0x7f7fa40c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:59.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- 172.21.15.39:0/1835763664 >> 172.21.15.39:0/1835763664 conn(0x7f7fa40c3310 msgr2=0x7f7fa40c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:08:59.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- 172.21.15.39:0/1835763664 shutdown_connections 2024-09-13T17:08:59.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- 172.21.15.39:0/1835763664 wait complete. 2024-09-13T17:08:59.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 Processor -- start 2024-09-13T17:08:59.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- start start 2024-09-13T17:08:59.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa40c4710 0x7f7fa40bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:59.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 0x7f7fa4142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:59.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fa414aa40 0x7f7fa4142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:59.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4143400 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa41527d0 con 0x7f7fa40c4710 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4154770 con 0x7f7fa414aa40 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fa9a6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 0x7f7fa4142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fa9a6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 0x7f7fa4142980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47000/0 (socket says 172.21.15.39:47000) 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fa9a6d640 1 -- 172.21.15.39:0/70068332 learned_addr learned my addr 172.21.15.39:0/70068332 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7faaa6f640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fa414aa40 0x7f7fa4142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7faa26e640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa40c4710 0x7f7fa40bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fa9a6d640 1 -- 172.21.15.39:0/70068332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fa414aa40 msgr2=0x7f7fa4142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fa9a6d640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fa414aa40 0x7f7fa4142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fa9a6d640 1 -- 172.21.15.39:0/70068332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa40c4710 msgr2=0x7f7fa40bfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:08:59.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fa9a6d640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa40c4710 0x7f7fa40bfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:08:59.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fa9a6d640 1 -- 172.21.15.39:0/70068332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c00bc80 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7faaa6f640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fa414aa40 0x7f7fa4142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:08:59.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fa9a6d640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 0x7f7fa4142980 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400e8a0 tx=0x7f7f9400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:59.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7f9affd640 1 -- 172.21.15.39:0/70068332 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94009510 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7f9affd640 1 -- 172.21.15.39:0/70068332 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9400c940 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa414d7d0 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7f9affd640 1 -- 172.21.15.39:0/70068332 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9400ccb0 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.939+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa41ecaa0 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.943+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa414d600 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.943+0000 7f7f9affd640 1 -- 172.21.15.39:0/70068332 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7f94013470 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.943+0000 7f7f9affd640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7f88066690 0x7f7f88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:08:59.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.943+0000 7f7f9affd640 1 -- 172.21.15.39:0/70068332 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f7f94084ef0 con 0x7f7fa40c5fd0 2024-09-13T17:08:59.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.943+0000 7f7faa26e640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7f88066690 0x7f7f88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:08:59.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.943+0000 7f7faa26e640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7f88066690 0x7f7f88068b50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00011f0 tx=0x7f7fa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:08:59.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:08:59.947+0000 7f7f9affd640 1 -- 172.21.15.39:0/70068332 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7f940515f0 con 0x7f7fa40c5fd0 2024-09-13T17:09:00.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:09:00.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:09:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fa40c76a0 con 0x7f7fa40c5fd0 2024-09-13T17:09:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7f9affd640 1 -- 172.21.15.39:0/70068332 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f7f94017090 con 0x7f7fa40c5fd0 2024-09-13T17:09:00.154 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:00.154 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:00.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7f88066690 msgr2=0x7f7f88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:00.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7f88066690 0x7f7f88068b50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00011f0 tx=0x7f7fa0009040 comp rx=0 tx=0).stop 2024-09-13T17:09:00.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 msgr2=0x7f7fa4142980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:00.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 0x7f7fa4142980 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400e8a0 tx=0x7f7f9400eca0 comp rx=0 tx=0).stop 2024-09-13T17:09:00.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 shutdown_connections 2024-09-13T17:09:00.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7f88066690 0x7f7f88068b50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:00.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fa414aa40 0x7f7fa4142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:00.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fa40c5fd0 0x7f7fa4142980 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:00.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 --2- 172.21.15.39:0/70068332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fa40c4710 0x7f7fa40bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:00.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.155+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 >> 172.21.15.39:0/70068332 conn(0x7f7fa40c3310 msgr2=0x7f7fa40c0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:00.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.159+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 shutdown_connections 2024-09-13T17:09:00.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:00.159+0000 7f7fac4d2640 1 -- 172.21.15.39:0/70068332 wait complete. 2024-09-13T17:09:00.177 INFO:tasks.mds_thrash.fs.[b]:mds.a reported in up:standby state 2024-09-13T17:09:00.177 INFO:tasks.mds_thrash.fs.[b]:waiting for 12 secs before thrashing 2024-09-13T17:09:12.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:12.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.318+0000 7f3618b73640 1 Processor -- start 2024-09-13T17:09:12.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.318+0000 7f3618b73640 1 -- start start 2024-09-13T17:09:12.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.318+0000 7f3618b73640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f36141556d0 0x7f36140c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.318+0000 7f3618b73640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36140a72b0 0x7f3614071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.318+0000 7f3618b73640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361406feb0 con 0x7f36140c7610 2024-09-13T17:09:12.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f3618b73640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36140aae50 con 0x7f36141556d0 2024-09-13T17:09:12.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f3618b73640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3614155ab0 con 0x7f36140a72b0 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f361259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f36141556d0 0x7f36140c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f360bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f36140c7610 0x7f3614151b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36028/0 (socket says 172.21.15.39:36028) 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f360bfff640 1 -- 172.21.15.39:0/1668707332 learned_addr learned my addr 172.21.15.39:0/1668707332 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f3612d9d640 1 --2- 172.21.15.39:0/1668707332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36140a72b0 0x7f3614071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f361259c640 1 -- 172.21.15.39:0/1668707332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36140a72b0 msgr2=0x7f3614071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f361259c640 1 --2- 172.21.15.39:0/1668707332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36140a72b0 0x7f3614071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f361259c640 1 -- 172.21.15.39:0/1668707332 >> v1:172.21.15.39:6789/0 conn(0x7f36140c7610 legacy=0x7f3614151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f361259c640 1 -- 172.21.15.39:0/1668707332 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36140c3c20 con 0x7f36141556d0 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f3611d9b640 1 -- 172.21.15.39:0/1668707332 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591571701 0 0) 0x7f361406feb0 con 0x7f36140c7610 2024-09-13T17:09:12.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f3612d9d640 1 --2- 172.21.15.39:0/1668707332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36140a72b0 0x7f3614071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:09:12.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f361259c640 1 --2- 172.21.15.39:0/1668707332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36141556d0 0x7f36140c36e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f36040097c0 tx=0x7f360401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5465ab34ec00f97d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:12.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f3611d9b640 1 -- 172.21.15.39:0/1668707332 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36040ab600 con 0x7f36141556d0 2024-09-13T17:09:12.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f3611d9b640 1 -- 172.21.15.39:0/1668707332 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36040a7050 con 0x7f36141556d0 2024-09-13T17:09:12.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.322+0000 7f3611d9b640 1 -- 172.21.15.39:0/1668707332 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36040ab980 con 0x7f36141556d0 2024-09-13T17:09:12.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 -- 172.21.15.39:0/1668707332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36141556d0 msgr2=0x7f36140c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 --2- 172.21.15.39:0/1668707332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36141556d0 0x7f36140c36e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f36040097c0 tx=0x7f360401c380 comp rx=0 tx=0).stop 2024-09-13T17:09:12.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 -- 172.21.15.39:0/1668707332 shutdown_connections 2024-09-13T17:09:12.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 --2- 172.21.15.39:0/1668707332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36140a72b0 0x7f3614071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 --2- 172.21.15.39:0/1668707332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36141556d0 0x7f36140c36e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 -- 172.21.15.39:0/1668707332 >> 172.21.15.39:0/1668707332 conn(0x7f36140a97b0 msgr2=0x7f36140a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:12.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 -- 172.21.15.39:0/1668707332 shutdown_connections 2024-09-13T17:09:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 -- 172.21.15.39:0/1668707332 wait complete. 2024-09-13T17:09:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 Processor -- start 2024-09-13T17:09:12.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 -- start start 2024-09-13T17:09:12.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36140a72b0 0x7f3614150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36141556d0 0x7f361415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3614158f70 0x7f36141593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36141540b0 con 0x7f3614158f70 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3614151020 con 0x7f36140a72b0 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3618b73640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3614151300 con 0x7f36141556d0 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3612d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3614158f70 0x7f36141593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3612d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3614158f70 0x7f36141593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45386/0 (socket says 172.21.15.39:45386) 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3612d9d640 1 -- 172.21.15.39:0/3675294460 learned_addr learned my addr 172.21.15.39:0/3675294460 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f361259c640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36140a72b0 0x7f3614150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f360bfff640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36141556d0 0x7f361415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3612d9d640 1 -- 172.21.15.39:0/3675294460 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36141556d0 msgr2=0x7f361415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3612d9d640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36141556d0 0x7f361415ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3612d9d640 1 -- 172.21.15.39:0/3675294460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36140a72b0 msgr2=0x7f3614150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3612d9d640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36140a72b0 0x7f3614150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3612d9d640 1 -- 172.21.15.39:0/3675294460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3604009420 con 0x7f3614158f70 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f361259c640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36140a72b0 0x7f3614150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f3612d9d640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3614158f70 0x7f36141593f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f360c007e40 tx=0x7f360c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f360affd640 1 -- 172.21.15.39:0/3675294460 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c003ba0 con 0x7f3614158f70 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.326+0000 7f360affd640 1 -- 172.21.15.39:0/3675294460 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c00f040 con 0x7f3614158f70 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.330+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36140c1500 con 0x7f3614158f70 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.330+0000 7f360affd640 1 -- 172.21.15.39:0/3675294460 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c00de80 con 0x7f3614158f70 2024-09-13T17:09:12.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.330+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36140c1ac0 con 0x7f3614158f70 2024-09-13T17:09:12.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.330+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3614071530 con 0x7f3614158f70 2024-09-13T17:09:12.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.330+0000 7f360affd640 1 -- 172.21.15.39:0/3675294460 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f360c004030 con 0x7f3614158f70 2024-09-13T17:09:12.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.330+0000 7f360affd640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f35f40666c0 0x7f35f4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.330+0000 7f360affd640 1 -- 172.21.15.39:0/3675294460 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f360c084fe0 con 0x7f3614158f70 2024-09-13T17:09:12.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.330+0000 7f361259c640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f35f40666c0 0x7f35f4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.330+0000 7f361259c640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f35f40666c0 0x7f35f4068b80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f36040ad000 tx=0x7f36040a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:12.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.338+0000 7f360affd640 1 -- 172.21.15.39:0/3675294460 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f360c051790 con 0x7f3614158f70 2024-09-13T17:09:12.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.578+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36140722c0 con 0x7f3614158f70 2024-09-13T17:09:12.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.578+0000 7f360affd640 1 -- 172.21.15.39:0/3675294460 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f360c01a090 con 0x7f3614158f70 2024-09-13T17:09:12.579 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:12.579 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:12.580 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:12.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f35f40666c0 msgr2=0x7f35f4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f35f40666c0 0x7f35f4068b80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f36040ad000 tx=0x7f36040a6000 comp rx=0 tx=0).stop 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3614158f70 msgr2=0x7f36141593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3614158f70 0x7f36141593f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f360c007e40 tx=0x7f360c00d4a0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 shutdown_connections 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f35f40666c0 0x7f35f4068b80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3614158f70 0x7f36141593f0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36141556d0 0x7f361415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 --2- 172.21.15.39:0/3675294460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36140a72b0 0x7f3614150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 >> 172.21.15.39:0/3675294460 conn(0x7f36140a97b0 msgr2=0x7f36140a78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 shutdown_connections 2024-09-13T17:09:12.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.582+0000 7f3618b73640 1 -- 172.21.15.39:0/3675294460 wait complete. 2024-09-13T17:09:12.598 INFO:tasks.mds_thrash.fs.[b]:kill mds.b (rank=0) 2024-09-13T17:09:12.599 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:09:12.599 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:09:12.616 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:09:12.616 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:12.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf55175640 1 Processor -- start 2024-09-13T17:09:12.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf55175640 1 -- start start 2024-09-13T17:09:12.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf55175640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf500a4e70 0x7fbf5006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf55175640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf5006d300 0x7fbf500c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf55175640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf500c2780 con 0x7fbf5014e0c0 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf55175640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf500c2a60 con 0x7fbf5006d300 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf55175640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf500c2d40 con 0x7fbf500a4e70 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf4f59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf5006d300 0x7fbf500c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf4f59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf5006d300 0x7fbf500c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45400/0 (socket says 172.21.15.39:45400) 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf4f59e640 1 -- 172.21.15.39:0/1156252087 learned_addr learned my addr 172.21.15.39:0/1156252087 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf47fff640 1 --2- 172.21.15.39:0/1156252087 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf500a4e70 0x7fbf5006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf4f59e640 1 -- 172.21.15.39:0/1156252087 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf500a4e70 msgr2=0x7fbf5006cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf4f59e640 1 --2- 172.21.15.39:0/1156252087 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf500a4e70 0x7fbf5006cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf4f59e640 1 -- 172.21.15.39:0/1156252087 >> v1:172.21.15.84:6789/0 conn(0x7fbf5014e0c0 legacy=0x7fbf500c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.746+0000 7fbf4f59e640 1 -- 172.21.15.39:0/1156252087 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500c3020 con 0x7fbf5006d300 2024-09-13T17:09:12.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf4f59e640 1 --2- 172.21.15.39:0/1156252087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf5006d300 0x7fbf500c4740 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009e50 tx=0x7fbf4801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6acbc48124b0390 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:12.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf4e59c640 1 -- 172.21.15.39:0/1156252087 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf500c5490 con 0x7fbf5006d300 2024-09-13T17:09:12.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf4e59c640 1 -- 172.21.15.39:0/1156252087 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf480a5670 con 0x7fbf5006d300 2024-09-13T17:09:12.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf4e59c640 1 -- 172.21.15.39:0/1156252087 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480a5a00 con 0x7fbf5006d300 2024-09-13T17:09:12.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf55175640 1 -- 172.21.15.39:0/1156252087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf5006d300 msgr2=0x7fbf500c4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf55175640 1 --2- 172.21.15.39:0/1156252087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf5006d300 0x7fbf500c4740 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009e50 tx=0x7fbf4801c9a0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf55175640 1 -- 172.21.15.39:0/1156252087 shutdown_connections 2024-09-13T17:09:12.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf55175640 1 --2- 172.21.15.39:0/1156252087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf5006d300 0x7fbf500c4740 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf55175640 1 --2- 172.21.15.39:0/1156252087 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf500a4e70 0x7fbf5006cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf55175640 1 -- 172.21.15.39:0/1156252087 >> 172.21.15.39:0/1156252087 conn(0x7fbf50069e10 msgr2=0x7fbf5006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:12.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf55175640 1 -- 172.21.15.39:0/1156252087 shutdown_connections 2024-09-13T17:09:12.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.750+0000 7fbf55175640 1 -- 172.21.15.39:0/1156252087 wait complete. 2024-09-13T17:09:12.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 Processor -- start 2024-09-13T17:09:12.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 -- start start 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf5006d300 0x7fbf50143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf500a4e70 0x7fbf5014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5014e0c0 0x7fbf5014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5014b480 con 0x7fbf500a4e70 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5006fcd0 con 0x7fbf5014e0c0 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5006ffb0 con 0x7fbf5006d300 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf500a4e70 0x7fbf5014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf500a4e70 0x7fbf5014a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45414/0 (socket says 172.21.15.39:45414) 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf47fff640 1 -- 172.21.15.39:0/4109092720 learned_addr learned my addr 172.21.15.39:0/4109092720 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf4f59e640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5014e0c0 0x7fbf5014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf4ed9d640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf5006d300 0x7fbf50143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf47fff640 1 -- 172.21.15.39:0/4109092720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf5006d300 msgr2=0x7fbf50143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf47fff640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf5006d300 0x7fbf50143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf47fff640 1 -- 172.21.15.39:0/4109092720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5014e0c0 msgr2=0x7fbf5014ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf47fff640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5014e0c0 0x7fbf5014ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf47fff640 1 -- 172.21.15.39:0/4109092720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48009b30 con 0x7fbf500a4e70 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf4ed9d640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf5006d300 0x7fbf50143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf47fff640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf500a4e70 0x7fbf5014a8a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800d800 tx=0x7fbf3800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf477fe640 1 -- 172.21.15.39:0/4109092720 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800b6c0 con 0x7fbf500a4e70 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf477fe640 1 -- 172.21.15.39:0/4109092720 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3800b820 con 0x7fbf500a4e70 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf501479b0 con 0x7fbf500a4e70 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf477fe640 1 -- 172.21.15.39:0/4109092720 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800bb90 con 0x7fbf500a4e70 2024-09-13T17:09:12.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf50072150 con 0x7fbf500a4e70 2024-09-13T17:09:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf500c4740 con 0x7fbf500a4e70 2024-09-13T17:09:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.754+0000 7fbf477fe640 1 -- 172.21.15.39:0/4109092720 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf3800bcf0 con 0x7fbf500a4e70 2024-09-13T17:09:12.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.758+0000 7fbf477fe640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf30066690 0x7fbf30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.758+0000 7fbf477fe640 1 -- 172.21.15.39:0/4109092720 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fbf380853a0 con 0x7fbf500a4e70 2024-09-13T17:09:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.758+0000 7fbf4ed9d640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf30066690 0x7fbf30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:12.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.758+0000 7fbf4ed9d640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf30066690 0x7fbf30068b50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000f70 tx=0x7fbf40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:12.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:12.762+0000 7fbf477fe640 1 -- 172.21.15.39:0/4109092720 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf38051a40 con 0x7fbf500a4e70 2024-09-13T17:09:13.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.002+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf5006ccc0 con 0x7fbf500a4e70 2024-09-13T17:09:13.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.002+0000 7fbf477fe640 1 -- 172.21.15.39:0/4109092720 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7fbf38055060 con 0x7fbf500a4e70 2024-09-13T17:09:13.002 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:13.002 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:13.002 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:13.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf30066690 msgr2=0x7fbf30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:13.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf30066690 0x7fbf30068b50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000f70 tx=0x7fbf40009040 comp rx=0 tx=0).stop 2024-09-13T17:09:13.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf500a4e70 msgr2=0x7fbf5014a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:13.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf500a4e70 0x7fbf5014a8a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800d800 tx=0x7fbf3800dcc0 comp rx=0 tx=0).stop 2024-09-13T17:09:13.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 shutdown_connections 2024-09-13T17:09:13.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf30066690 0x7fbf30068b50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:13.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf5014e0c0 0x7fbf5014ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:13.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf500a4e70 0x7fbf5014a8a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:13.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 --2- 172.21.15.39:0/4109092720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf5006d300 0x7fbf50143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:13.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 >> 172.21.15.39:0/4109092720 conn(0x7fbf50069e10 msgr2=0x7fbf500aa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:13.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 shutdown_connections 2024-09-13T17:09:13.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:13.006+0000 7fbf55175640 1 -- 172.21.15.39:0/4109092720 wait complete. 2024-09-13T17:09:13.021 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:09:14.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc65b01640 1 Processor -- start 2024-09-13T17:09:14.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc65b01640 1 -- start start 2024-09-13T17:09:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc65b01640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdc60151500 0x7fdc600c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc65b01640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc60071dd0 0x7fdc60154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc65b01640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc600c5a30 con 0x7fdc600c47c0 2024-09-13T17:09:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc65b01640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc601576a0 con 0x7fdc60151500 2024-09-13T17:09:14.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc65b01640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc60157980 con 0x7fdc60071dd0 2024-09-13T17:09:14.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc5f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdc60151500 0x7fdc600c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:14.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc5f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdc60151500 0x7fdc600c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45430/0 (socket says 172.21.15.39:45430) 2024-09-13T17:09:14.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc5f7fe640 1 -- 172.21.15.39:0/1194762664 learned_addr learned my addr 172.21.15.39:0/1194762664 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:14.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc5effd640 1 --2- 172.21.15.39:0/1194762664 >> v2:172.21.15.84:3300/0 conn(0x7fdc60071dd0 0x7fdc60154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:14.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc5f7fe640 1 -- 172.21.15.39:0/1194762664 >> v2:172.21.15.84:3300/0 conn(0x7fdc60071dd0 msgr2=0x7fdc60154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:14.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc5f7fe640 1 --2- 172.21.15.39:0/1194762664 >> v2:172.21.15.84:3300/0 conn(0x7fdc60071dd0 0x7fdc60154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc5f7fe640 1 -- 172.21.15.39:0/1194762664 >> v1:172.21.15.84:6789/0 conn(0x7fdc600c47c0 legacy=0x7fdc601545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:14.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.622+0000 7fdc5f7fe640 1 -- 172.21.15.39:0/1194762664 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60157c60 con 0x7fdc60151500 2024-09-13T17:09:14.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc5f7fe640 1 --2- 172.21.15.39:0/1194762664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60151500 0x7fdc600c4280 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fdc600a9470 tx=0x7fdc5001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ae16197ba4d877 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:14.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc5e7fc640 1 -- 172.21.15.39:0/1194762664 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500a5790 con 0x7fdc60151500 2024-09-13T17:09:14.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc5e7fc640 1 -- 172.21.15.39:0/1194762664 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc500a58f0 con 0x7fdc60151500 2024-09-13T17:09:14.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc5e7fc640 1 -- 172.21.15.39:0/1194762664 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500a5c80 con 0x7fdc60151500 2024-09-13T17:09:14.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 -- 172.21.15.39:0/1194762664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60151500 msgr2=0x7fdc600c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:14.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 --2- 172.21.15.39:0/1194762664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60151500 0x7fdc600c4280 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fdc600a9470 tx=0x7fdc5001c8b0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 -- 172.21.15.39:0/1194762664 shutdown_connections 2024-09-13T17:09:14.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 --2- 172.21.15.39:0/1194762664 >> v2:172.21.15.84:3300/0 conn(0x7fdc60071dd0 0x7fdc60154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 --2- 172.21.15.39:0/1194762664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60151500 0x7fdc600c4280 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 -- 172.21.15.39:0/1194762664 >> 172.21.15.39:0/1194762664 conn(0x7fdc600ab530 msgr2=0x7fdc600ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:14.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 -- 172.21.15.39:0/1194762664 shutdown_connections 2024-09-13T17:09:14.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 -- 172.21.15.39:0/1194762664 wait complete. 2024-09-13T17:09:14.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 Processor -- start 2024-09-13T17:09:14.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 -- start start 2024-09-13T17:09:14.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60071dd0 0x7fdc600aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc600c47c0 0x7fdc60148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdc60151500 0x7fdc601485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6014e5f0 con 0x7fdc60071dd0 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6014da20 con 0x7fdc600c47c0 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.626+0000 7fdc65b01640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6014dcc0 con 0x7fdc60151500 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60071dd0 0x7fdc600aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60071dd0 0x7fdc600aa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45444/0 (socket says 172.21.15.39:45444) 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5f7fe640 1 -- 172.21.15.39:0/1429967141 learned_addr learned my addr 172.21.15.39:0/1429967141 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5effd640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc600c47c0 0x7fdc60148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5ffff640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdc60151500 0x7fdc601485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5f7fe640 1 -- 172.21.15.39:0/1429967141 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdc60151500 msgr2=0x7fdc601485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5f7fe640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdc60151500 0x7fdc601485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5f7fe640 1 -- 172.21.15.39:0/1429967141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc600c47c0 msgr2=0x7fdc60148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5f7fe640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc600c47c0 0x7fdc60148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5f7fe640 1 -- 172.21.15.39:0/1429967141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc50009b30 con 0x7fdc60071dd0 2024-09-13T17:09:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5f7fe640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60071dd0 0x7fdc600aa650 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fdc500af040 tx=0x7fdc50019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc47fff640 1 -- 172.21.15.39:0/1429967141 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500b8670 con 0x7fdc60071dd0 2024-09-13T17:09:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc47fff640 1 -- 172.21.15.39:0/1429967141 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc500b87f0 con 0x7fdc60071dd0 2024-09-13T17:09:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6014dfa0 con 0x7fdc60071dd0 2024-09-13T17:09:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc60148b00 con 0x7fdc60071dd0 2024-09-13T17:09:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc47fff640 1 -- 172.21.15.39:0/1429967141 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500b8ba0 con 0x7fdc60071dd0 2024-09-13T17:09:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc40005260 con 0x7fdc60071dd0 2024-09-13T17:09:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc47fff640 1 -- 172.21.15.39:0/1429967141 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc500b8d00 con 0x7fdc60071dd0 2024-09-13T17:09:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc47fff640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdc4c066690 0x7fdc4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc47fff640 1 -- 172.21.15.39:0/1429967141 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fdc501203b0 con 0x7fdc60071dd0 2024-09-13T17:09:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5effd640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdc4c066690 0x7fdc4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.630+0000 7fdc5effd640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdc4c066690 0x7fdc4c068b50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdc600aae70 tx=0x7fdc54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:14.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.638+0000 7fdc47fff640 1 -- 172.21.15.39:0/1429967141 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdc500ecbd0 con 0x7fdc60071dd0 2024-09-13T17:09:14.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc40005a90 con 0x7fdc60071dd0 2024-09-13T17:09:14.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc47fff640 1 -- 172.21.15.39:0/1429967141 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7fdc500f01f0 con 0x7fdc60071dd0 2024-09-13T17:09:14.909 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:14.909 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:14.909 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdc4c066690 msgr2=0x7fdc4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdc4c066690 0x7fdc4c068b50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdc600aae70 tx=0x7fdc54008040 comp rx=0 tx=0).stop 2024-09-13T17:09:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60071dd0 msgr2=0x7fdc600aa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60071dd0 0x7fdc600aa650 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fdc500af040 tx=0x7fdc50019650 comp rx=0 tx=0).stop 2024-09-13T17:09:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 shutdown_connections 2024-09-13T17:09:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdc4c066690 0x7fdc4c068b50 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc600c47c0 0x7fdc60148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdc60151500 0x7fdc601485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 --2- 172.21.15.39:0/1429967141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc60071dd0 0x7fdc600aa650 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:14.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 >> 172.21.15.39:0/1429967141 conn(0x7fdc600ab530 msgr2=0x7fdc60154410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:14.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 shutdown_connections 2024-09-13T17:09:14.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:14.910+0000 7fdc65b01640 1 -- 172.21.15.39:0/1429967141 wait complete. 2024-09-13T17:09:14.929 INFO:tasks.mds_thrash.fs.[a]:kill mds.c (rank=0) 2024-09-13T17:09:14.929 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:09:14.929 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:09:14.940 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:09:14.940 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:15.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:15.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f114e53d640 1 Processor -- start 2024-09-13T17:09:15.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f114e53d640 1 -- start start 2024-09-13T17:09:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f114e53d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f114806fba0 0x7f1148154b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f114e53d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f11480c1280 0x7f11480c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f114e53d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11480c33c0 con 0x7f11480c28d0 2024-09-13T17:09:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f114e53d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11480c1ba0 con 0x7f114806fba0 2024-09-13T17:09:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f114e53d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11480611c0 con 0x7f11480c1280 2024-09-13T17:09:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f11477fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f114806fba0 0x7f1148154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f11477fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f114806fba0 0x7f1148154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45452/0 (socket says 172.21.15.39:45452) 2024-09-13T17:09:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f11477fe640 1 -- 172.21.15.39:0/2450972645 learned_addr learned my addr 172.21.15.39:0/2450972645 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f114cada640 1 --2- 172.21.15.39:0/2450972645 >> v2:172.21.15.84:3300/0 conn(0x7f11480c1280 0x7f11480c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f11477fe640 1 -- 172.21.15.39:0/2450972645 >> v2:172.21.15.84:3300/0 conn(0x7f11480c1280 msgr2=0x7f11480c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f11477fe640 1 --2- 172.21.15.39:0/2450972645 >> v2:172.21.15.84:3300/0 conn(0x7f11480c1280 0x7f11480c1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f11477fe640 1 -- 172.21.15.39:0/2450972645 >> v1:172.21.15.84:6789/0 conn(0x7f11480c28d0 legacy=0x7f11480c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:15.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.122+0000 7f11477fe640 1 -- 172.21.15.39:0/2450972645 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11480c0ac0 con 0x7f114806fba0 2024-09-13T17:09:15.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f11477fe640 1 --2- 172.21.15.39:0/2450972645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f114806fba0 0x7f1148154b70 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f114806dd30 tx=0x7f113c01c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496cedf0f7406c94 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:15.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f1146ffd640 1 -- 172.21.15.39:0/2450972645 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c0ab600 con 0x7f114806fba0 2024-09-13T17:09:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f1146ffd640 1 -- 172.21.15.39:0/2450972645 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c0a7050 con 0x7f114806fba0 2024-09-13T17:09:15.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f1146ffd640 1 -- 172.21.15.39:0/2450972645 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c0ab980 con 0x7f114806fba0 2024-09-13T17:09:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f114e53d640 1 -- 172.21.15.39:0/2450972645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f114806fba0 msgr2=0x7f1148154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f114e53d640 1 --2- 172.21.15.39:0/2450972645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f114806fba0 0x7f1148154b70 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f114806dd30 tx=0x7f113c01c190 comp rx=0 tx=0).stop 2024-09-13T17:09:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f114e53d640 1 -- 172.21.15.39:0/2450972645 shutdown_connections 2024-09-13T17:09:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f114e53d640 1 --2- 172.21.15.39:0/2450972645 >> v2:172.21.15.84:3300/0 conn(0x7f11480c1280 0x7f11480c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f114e53d640 1 --2- 172.21.15.39:0/2450972645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f114806fba0 0x7f1148154b70 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f114e53d640 1 -- 172.21.15.39:0/2450972645 >> 172.21.15.39:0/2450972645 conn(0x7f1148069e10 msgr2=0x7f114806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f114e53d640 1 -- 172.21.15.39:0/2450972645 shutdown_connections 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.126+0000 7f114e53d640 1 -- 172.21.15.39:0/2450972645 wait complete. 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 Processor -- start 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 -- start start 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f114806fba0 0x7f1148145840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11480c1280 0x7f11481ef640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11480c28d0 0x7f11481efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11481432d0 con 0x7f11480c28d0 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1148143800 con 0x7f11480c1280 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11480c57f0 con 0x7f114806fba0 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114cada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11480c28d0 0x7f11481efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114cada640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11480c28d0 0x7f11481efb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45454/0 (socket says 172.21.15.39:45454) 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114cada640 1 -- 172.21.15.39:0/3350900368 learned_addr learned my addr 172.21.15.39:0/3350900368 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f1147fff640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f114806fba0 0x7f1148145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f11477fe640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11480c1280 0x7f11481ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114cada640 1 -- 172.21.15.39:0/3350900368 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f114806fba0 msgr2=0x7f1148145840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114cada640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f114806fba0 0x7f1148145840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114cada640 1 -- 172.21.15.39:0/3350900368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11480c1280 msgr2=0x7f11481ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114cada640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11480c1280 0x7f11481ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114cada640 1 -- 172.21.15.39:0/3350900368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c0094d0 con 0x7f11480c28d0 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114cada640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11480c28d0 0x7f11481efb80 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1140007e70 tx=0x7f114000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f1144ff9640 1 -- 172.21.15.39:0/3350900368 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140014620 con 0x7f11480c28d0 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f1144ff9640 1 -- 172.21.15.39:0/3350900368 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11400147a0 con 0x7f11480c28d0 2024-09-13T17:09:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1148143520 con 0x7f11480c28d0 2024-09-13T17:09:15.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f1144ff9640 1 -- 172.21.15.39:0/3350900368 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140014b50 con 0x7f11480c28d0 2024-09-13T17:09:15.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.130+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1148154190 con 0x7f11480c28d0 2024-09-13T17:09:15.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.134+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11480c52c0 con 0x7f11480c28d0 2024-09-13T17:09:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.134+0000 7f1144ff9640 1 -- 172.21.15.39:0/3350900368 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1140014cb0 con 0x7f11480c28d0 2024-09-13T17:09:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.134+0000 7f1144ff9640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1134066690 0x7f1134068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.134+0000 7f1144ff9640 1 -- 172.21.15.39:0/3350900368 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f1140085820 con 0x7f11480c28d0 2024-09-13T17:09:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.134+0000 7f1147fff640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1134066690 0x7f1134068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.134+0000 7f1147fff640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1134066690 0x7f1134068b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f11380098e0 tx=0x7f1138009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:15.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.138+0000 7f1144ff9640 1 -- 172.21.15.39:0/3350900368 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1140051e00 con 0x7f11480c28d0 2024-09-13T17:09:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.166+0000 7fdf13b0a640 1 Processor -- start 2024-09-13T17:09:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.166+0000 7fdf13b0a640 1 -- start start 2024-09-13T17:09:15.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.166+0000 7fdf13b0a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdf0c0c4260 0x7fdf0c0c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.166+0000 7fdf13b0a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c157360 con 0x7fdf0c0bcfb0 2024-09-13T17:09:15.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.166+0000 7fdf13b0a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0c80b0 con 0x7fdf0c070b40 2024-09-13T17:09:15.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.166+0000 7fdf13b0a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c1575b0 con 0x7fdf0c0c4260 2024-09-13T17:09:15.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.166+0000 7fdf118a6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdf0c0c4260 0x7fdf0c0c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.166+0000 7fdf110a5640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdf0c0bcfb0 0x7fdf0c070430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36042/0 (socket says 172.21.15.39:36042) 2024-09-13T17:09:15.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.166+0000 7fdf110a5640 1 -- 172.21.15.39:0/959201666 learned_addr learned my addr 172.21.15.39:0/959201666 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:15.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf118a6640 1 -- 172.21.15.39:0/959201666 >> v1:172.21.15.84:6789/0 conn(0x7fdf0c070b40 legacy=0x7fdf0c15e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:15.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf118a6640 1 -- 172.21.15.39:0/959201666 >> v1:172.21.15.39:6789/0 conn(0x7fdf0c0bcfb0 legacy=0x7fdf0c070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf108a4640 1 -- 172.21.15.39:0/959201666 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282733404 0 0) 0x7fdf0c157360 con 0x7fdf0c0bcfb0 2024-09-13T17:09:15.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf118a6640 1 -- 172.21.15.39:0/959201666 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c157890 con 0x7fdf0c0c4260 2024-09-13T17:09:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf118a6640 1 --2- 172.21.15.39:0/959201666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0c4260 0x7fdf0c0c4640 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fdf080097c0 tx=0x7fdf0801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9038b0a8b2cba9af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf108a4640 1 -- 172.21.15.39:0/959201666 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0c7330 con 0x7fdf0c0c4260 2024-09-13T17:09:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf108a4640 1 -- 172.21.15.39:0/959201666 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf080a7050 con 0x7fdf0c0c4260 2024-09-13T17:09:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf108a4640 1 -- 172.21.15.39:0/959201666 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080ab820 con 0x7fdf0c0c4260 2024-09-13T17:09:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf13b0a640 1 -- 172.21.15.39:0/959201666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0c4260 msgr2=0x7fdf0c0c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf13b0a640 1 --2- 172.21.15.39:0/959201666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0c4260 0x7fdf0c0c4640 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fdf080097c0 tx=0x7fdf0801c340 comp rx=0 tx=0).stop 2024-09-13T17:09:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf13b0a640 1 -- 172.21.15.39:0/959201666 shutdown_connections 2024-09-13T17:09:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf13b0a640 1 --2- 172.21.15.39:0/959201666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0c4260 0x7fdf0c0c4640 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf13b0a640 1 -- 172.21.15.39:0/959201666 >> 172.21.15.39:0/959201666 conn(0x7fdf0c069e10 msgr2=0x7fdf0c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf13b0a640 1 -- 172.21.15.39:0/959201666 shutdown_connections 2024-09-13T17:09:15.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.170+0000 7fdf13b0a640 1 -- 172.21.15.39:0/959201666 wait complete. 2024-09-13T17:09:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 Processor -- start 2024-09-13T17:09:15.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 -- start start 2024-09-13T17:09:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0bcfb0 0x7fdf0c1f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0be970 0x7fdf0c1edd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c1ee2b0 0x7fdf0c14a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0c3cd0 con 0x7fdf0c0be970 2024-09-13T17:09:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1f09b0 con 0x7fdf0c0bcfb0 2024-09-13T17:09:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0bde90 con 0x7fdf0c1ee2b0 2024-09-13T17:09:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf110a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0be970 0x7fdf0c1edd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf110a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0be970 0x7fdf0c1edd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45476/0 (socket says 172.21.15.39:45476) 2024-09-13T17:09:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf110a5640 1 -- 172.21.15.39:0/3336894990 learned_addr learned my addr 172.21.15.39:0/3336894990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf118a6640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0bcfb0 0x7fdf0c1f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf120a7640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c1ee2b0 0x7fdf0c14a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf110a5640 1 -- 172.21.15.39:0/3336894990 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c1ee2b0 msgr2=0x7fdf0c14a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf110a5640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c1ee2b0 0x7fdf0c14a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf110a5640 1 -- 172.21.15.39:0/3336894990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0bcfb0 msgr2=0x7fdf0c1f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf110a5640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0bcfb0 0x7fdf0c1f00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf110a5640 1 -- 172.21.15.39:0/3336894990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf08009420 con 0x7fdf0c0be970 2024-09-13T17:09:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf110a5640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0be970 0x7fdf0c1edd70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c450 tx=0x7fdefc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf027fc640 1 -- 172.21.15.39:0/3336894990 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0074e0 con 0x7fdf0c0be970 2024-09-13T17:09:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf027fc640 1 -- 172.21.15.39:0/3336894990 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc007640 con 0x7fdf0c0be970 2024-09-13T17:09:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c0be170 con 0x7fdf0c0be970 2024-09-13T17:09:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf027fc640 1 -- 172.21.15.39:0/3336894990 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0049f0 con 0x7fdf0c0be970 2024-09-13T17:09:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.174+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c1f0c90 con 0x7fdf0c0be970 2024-09-13T17:09:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.178+0000 7fdf027fc640 1 -- 172.21.15.39:0/3336894990 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdefc004b50 con 0x7fdf0c0be970 2024-09-13T17:09:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.178+0000 7fdf027fc640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef00666c0 0x7fdef0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.178+0000 7fdf027fc640 1 -- 172.21.15.39:0/3336894990 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fdefc0861e0 con 0x7fdf0c0be970 2024-09-13T17:09:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.178+0000 7fdf118a6640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef00666c0 0x7fdef0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.178+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c0c4640 con 0x7fdf0c0be970 2024-09-13T17:09:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.178+0000 7fdf118a6640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef00666c0 0x7fdef0068b80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdf080ad000 tx=0x7fdf080a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:15.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.182+0000 7fdf027fc640 1 -- 172.21.15.39:0/3336894990 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdefc018020 con 0x7fdf0c0be970 2024-09-13T17:09:15.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.354+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11480abbc0 con 0x7f11480c28d0 2024-09-13T17:09:15.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f1144ff9640 1 -- 172.21.15.39:0/3350900368 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f1140055420 con 0x7f11480c28d0 2024-09-13T17:09:15.356 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:15.356 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:15.356 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1134066690 msgr2=0x7f1134068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1134066690 0x7f1134068b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f11380098e0 tx=0x7f1138009060 comp rx=0 tx=0).stop 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11480c28d0 msgr2=0x7f11481efb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11480c28d0 0x7f11481efb80 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1140007e70 tx=0x7f114000e4b0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 shutdown_connections 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1134066690 0x7f1134068b50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f11480c1280 0x7f11481ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11480c28d0 0x7f11481efb80 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 --2- 172.21.15.39:0/3350900368 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f114806fba0 0x7f1148145840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 >> 172.21.15.39:0/3350900368 conn(0x7f1148069e10 msgr2=0x7f114806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:15.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 shutdown_connections 2024-09-13T17:09:15.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.358+0000 7f114e53d640 1 -- 172.21.15.39:0/3350900368 wait complete. 2024-09-13T17:09:15.378 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:09:15.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.430+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf0c0c6e70 con 0x7fdf0c0be970 2024-09-13T17:09:15.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.430+0000 7fdf027fc640 1 -- 172.21.15.39:0/3336894990 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7fdefc054240 con 0x7fdf0c0be970 2024-09-13T17:09:15.430 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:15.430 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:15.430 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef00666c0 msgr2=0x7fdef0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef00666c0 0x7fdef0068b80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdf080ad000 tx=0x7fdf080a6000 comp rx=0 tx=0).stop 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0be970 msgr2=0x7fdf0c1edd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0be970 0x7fdf0c1edd70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c450 tx=0x7fdefc00c910 comp rx=0 tx=0).stop 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 shutdown_connections 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdef00666c0 0x7fdef0068b80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf0c1ee2b0 0x7fdf0c14a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0be970 0x7fdf0c1edd70 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 --2- 172.21.15.39:0/3336894990 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0bcfb0 0x7fdf0c1f00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 >> 172.21.15.39:0/3336894990 conn(0x7fdf0c069e10 msgr2=0x7fdf0c0c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 shutdown_connections 2024-09-13T17:09:15.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:15.434+0000 7fdf13b0a640 1 -- 172.21.15.39:0/3336894990 wait complete. 2024-09-13T17:09:15.448 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:09:17.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:17.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:17.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84ec9ef640 1 Processor -- start 2024-09-13T17:09:17.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84ec9ef640 1 -- start start 2024-09-13T17:09:17.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84ec9ef640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f84e81556d0 0x7f84e80c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84ec9ef640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f84e80a72b0 0x7f84e8071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84ec9ef640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e806feb0 con 0x7f84e80c7610 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84ec9ef640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e80aae50 con 0x7f84e80a72b0 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84ec9ef640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e8155ab0 con 0x7f84e81556d0 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84e6d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f84e80a72b0 0x7f84e8071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84e6d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f84e80a72b0 0x7f84e8071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45490/0 (socket says 172.21.15.39:45490) 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84e6d9d640 1 -- 172.21.15.39:0/681584104 learned_addr learned my addr 172.21.15.39:0/681584104 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84e659c640 1 --2- 172.21.15.39:0/681584104 >> v2:172.21.15.84:3300/0 conn(0x7f84e81556d0 0x7f84e80c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84e6d9d640 1 -- 172.21.15.39:0/681584104 >> v2:172.21.15.84:3300/0 conn(0x7f84e81556d0 msgr2=0x7f84e80c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84e6d9d640 1 --2- 172.21.15.39:0/681584104 >> v2:172.21.15.84:3300/0 conn(0x7f84e81556d0 0x7f84e80c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84e6d9d640 1 -- 172.21.15.39:0/681584104 >> v1:172.21.15.84:6789/0 conn(0x7f84e80c7610 legacy=0x7f84e8151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.510+0000 7f84e6d9d640 1 -- 172.21.15.39:0/681584104 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e80c3c20 con 0x7f84e80a72b0 2024-09-13T17:09:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.514+0000 7f84e6d9d640 1 --2- 172.21.15.39:0/681584104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e80a72b0 0x7f84e8071530 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f84e8061cc0 tx=0x7f84e001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f18baa024a77c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e559a640 1 -- 172.21.15.39:0/681584104 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00a5640 con 0x7f84e80a72b0 2024-09-13T17:09:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e559a640 1 -- 172.21.15.39:0/681584104 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e00a57a0 con 0x7f84e80a72b0 2024-09-13T17:09:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e559a640 1 -- 172.21.15.39:0/681584104 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00a5b30 con 0x7f84e80a72b0 2024-09-13T17:09:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- 172.21.15.39:0/681584104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e80a72b0 msgr2=0x7f84e8071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 --2- 172.21.15.39:0/681584104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e80a72b0 0x7f84e8071530 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f84e8061cc0 tx=0x7f84e001c890 comp rx=0 tx=0).stop 2024-09-13T17:09:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- 172.21.15.39:0/681584104 shutdown_connections 2024-09-13T17:09:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 --2- 172.21.15.39:0/681584104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e80a72b0 0x7f84e8071530 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 --2- 172.21.15.39:0/681584104 >> v2:172.21.15.84:3300/0 conn(0x7f84e81556d0 0x7f84e80c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- 172.21.15.39:0/681584104 >> 172.21.15.39:0/681584104 conn(0x7f84e80a97b0 msgr2=0x7f84e80a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- 172.21.15.39:0/681584104 shutdown_connections 2024-09-13T17:09:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- 172.21.15.39:0/681584104 wait complete. 2024-09-13T17:09:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 Processor -- start 2024-09-13T17:09:17.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- start start 2024-09-13T17:09:17.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84e80a72b0 0x7f84e8150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e81556d0 0x7f84e81f05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84e814e250 0x7f84e81534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e806f1b0 con 0x7f84e81556d0 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e81f0b30 con 0x7f84e814e250 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8153050 con 0x7f84e80a72b0 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e81556d0 0x7f84e81f05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e81556d0 0x7f84e81f05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45506/0 (socket says 172.21.15.39:45506) 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e5d9b640 1 -- 172.21.15.39:0/4139185674 learned_addr learned my addr 172.21.15.39:0/4139185674 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e659c640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84e80a72b0 0x7f84e8150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e6d9d640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84e814e250 0x7f84e81534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e5d9b640 1 -- 172.21.15.39:0/4139185674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84e80a72b0 msgr2=0x7f84e8150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e5d9b640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84e80a72b0 0x7f84e8150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e5d9b640 1 -- 172.21.15.39:0/4139185674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84e814e250 msgr2=0x7f84e81534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e5d9b640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84e814e250 0x7f84e81534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e5d9b640 1 -- 172.21.15.39:0/4139185674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e0009b30 con 0x7f84e81556d0 2024-09-13T17:09:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e6d9d640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84e814e250 0x7f84e81534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84e5d9b640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e81556d0 0x7f84e81f05f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f84d000c450 tx=0x7f84d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84deffd640 1 -- 172.21.15.39:0/4139185674 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0012830 con 0x7f84e81556d0 2024-09-13T17:09:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84deffd640 1 -- 172.21.15.39:0/4139185674 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d00129b0 con 0x7f84e81556d0 2024-09-13T17:09:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84deffd640 1 -- 172.21.15.39:0/4139185674 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0012d60 con 0x7f84e81556d0 2024-09-13T17:09:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e8150f70 con 0x7f84e81556d0 2024-09-13T17:09:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.518+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e8151530 con 0x7f84e81556d0 2024-09-13T17:09:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.522+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e8153200 con 0x7f84e81556d0 2024-09-13T17:09:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.522+0000 7f84deffd640 1 -- 172.21.15.39:0/4139185674 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f84d001c460 con 0x7f84e81556d0 2024-09-13T17:09:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.522+0000 7f84deffd640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84c80666c0 0x7f84c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.522+0000 7f84deffd640 1 -- 172.21.15.39:0/4139185674 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f84d0085620 con 0x7f84e81556d0 2024-09-13T17:09:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.522+0000 7f84e659c640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84c80666c0 0x7f84c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.522+0000 7f84e659c640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84c80666c0 0x7f84c8068b80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f84d800b9d0 tx=0x7f84d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.530+0000 7f84deffd640 1 -- 172.21.15.39:0/4139185674 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f84d0051cb0 con 0x7f84e81556d0 2024-09-13T17:09:17.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.590+0000 7fc6f611a640 1 Processor -- start 2024-09-13T17:09:17.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.590+0000 7fc6f611a640 1 -- start start 2024-09-13T17:09:17.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.590+0000 7fc6f611a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc6f006d020 0x7fc6f00a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc6f00c4530 0x7fc6f00c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f01518f0 con 0x7fc6f0151460 2024-09-13T17:09:17.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00a8c50 con 0x7fc6f00c4530 2024-09-13T17:09:17.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f0154100 con 0x7fc6f006d020 2024-09-13T17:09:17.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6effff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc6f00c4530 0x7fc6f00c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6effff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc6f00c4530 0x7fc6f00c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45520/0 (socket says 172.21.15.39:45520) 2024-09-13T17:09:17.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6effff640 1 -- 172.21.15.39:0/120003633 learned_addr learned my addr 172.21.15.39:0/120003633 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:17.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6ef7fe640 1 --2- 172.21.15.39:0/120003633 >> v2:172.21.15.84:3300/0 conn(0x7fc6f006d020 0x7fc6f00a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6effff640 1 -- 172.21.15.39:0/120003633 >> v2:172.21.15.84:3300/0 conn(0x7fc6f006d020 msgr2=0x7fc6f00a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:17.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6effff640 1 --2- 172.21.15.39:0/120003633 >> v2:172.21.15.84:3300/0 conn(0x7fc6f006d020 0x7fc6f00a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6effff640 1 -- 172.21.15.39:0/120003633 >> v1:172.21.15.84:6789/0 conn(0x7fc6f0151460 legacy=0x7fc6f014db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:17.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6effff640 1 -- 172.21.15.39:0/120003633 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f01543e0 con 0x7fc6f00c4530 2024-09-13T17:09:17.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6effff640 1 --2- 172.21.15.39:0/120003633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f00c4530 0x7fc6f00c4940 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc6f006d820 tx=0x7fc6e801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a9a0bcbe4990ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:17.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6ee7fc640 1 -- 172.21.15.39:0/120003633 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80a56c0 con 0x7fc6f00c4530 2024-09-13T17:09:17.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6ee7fc640 1 -- 172.21.15.39:0/120003633 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e80a5820 con 0x7fc6f00c4530 2024-09-13T17:09:17.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6ee7fc640 1 -- 172.21.15.39:0/120003633 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80a5bb0 con 0x7fc6f00c4530 2024-09-13T17:09:17.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 -- 172.21.15.39:0/120003633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f00c4530 msgr2=0x7fc6f00c4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 --2- 172.21.15.39:0/120003633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f00c4530 0x7fc6f00c4940 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc6f006d820 tx=0x7fc6e801c8f0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 -- 172.21.15.39:0/120003633 shutdown_connections 2024-09-13T17:09:17.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 --2- 172.21.15.39:0/120003633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f00c4530 0x7fc6f00c4940 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 --2- 172.21.15.39:0/120003633 >> v2:172.21.15.84:3300/0 conn(0x7fc6f006d020 0x7fc6f00a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 -- 172.21.15.39:0/120003633 >> 172.21.15.39:0/120003633 conn(0x7fc6f0069e10 msgr2=0x7fc6f006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:17.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 -- 172.21.15.39:0/120003633 shutdown_connections 2024-09-13T17:09:17.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.594+0000 7fc6f611a640 1 -- 172.21.15.39:0/120003633 wait complete. 2024-09-13T17:09:17.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 Processor -- start 2024-09-13T17:09:17.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 -- start start 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6f006d020 0x7fc6f00c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6f00c4530 0x7fc6f0071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f0151460 0x7fc6f014aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f006fc70 con 0x7fc6f0151460 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f006f750 con 0x7fc6f006d020 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f006f2c0 con 0x7fc6f00c4530 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6effff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f0151460 0x7fc6f014aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6effff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f0151460 0x7fc6f014aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45522/0 (socket says 172.21.15.39:45522) 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6effff640 1 -- 172.21.15.39:0/4293921290 learned_addr learned my addr 172.21.15.39:0/4293921290 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6ef7fe640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6f006d020 0x7fc6f00c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6eeffd640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6f00c4530 0x7fc6f0071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6effff640 1 -- 172.21.15.39:0/4293921290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6f00c4530 msgr2=0x7fc6f0071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6effff640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6f00c4530 0x7fc6f0071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6effff640 1 -- 172.21.15.39:0/4293921290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6f006d020 msgr2=0x7fc6f00c09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6effff640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6f006d020 0x7fc6f00c09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6effff640 1 -- 172.21.15.39:0/4293921290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8009b30 con 0x7fc6f0151460 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6ef7fe640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6f006d020 0x7fc6f00c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6effff640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f0151460 0x7fc6f014aba0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80af040 tx=0x7fc6e8019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6d7fff640 1 -- 172.21.15.39:0/4293921290 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80b8670 con 0x7fc6f0151460 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6d7fff640 1 -- 172.21.15.39:0/4293921290 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e80b87d0 con 0x7fc6f0151460 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6d7fff640 1 -- 172.21.15.39:0/4293921290 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80b8b60 con 0x7fc6f0151460 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f0071940 con 0x7fc6f0151460 2024-09-13T17:09:17.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.598+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f00bf5a0 con 0x7fc6f0151460 2024-09-13T17:09:17.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.602+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f006f430 con 0x7fc6f0151460 2024-09-13T17:09:17.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.602+0000 7fc6d7fff640 1 -- 172.21.15.39:0/4293921290 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc6e80b8cc0 con 0x7fc6f0151460 2024-09-13T17:09:17.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.602+0000 7fc6d7fff640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6dc066690 0x7fc6dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:17.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.602+0000 7fc6d7fff640 1 -- 172.21.15.39:0/4293921290 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fc6e8120410 con 0x7fc6f0151460 2024-09-13T17:09:17.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.602+0000 7fc6ef7fe640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6dc066690 0x7fc6dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:17.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.602+0000 7fc6ef7fe640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6dc066690 0x7fc6dc068b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0007a70 tx=0x7fc6e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:17.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.606+0000 7fc6d7fff640 1 -- 172.21.15.39:0/4293921290 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc6e80ecbf0 con 0x7fc6f0151460 2024-09-13T17:09:17.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84e81531c0 con 0x7f84e81556d0 2024-09-13T17:09:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84deffd640 1 -- 172.21.15.39:0/4139185674 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f84d00552d0 con 0x7f84e81556d0 2024-09-13T17:09:17.737 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:17.737 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:17.738 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84c80666c0 msgr2=0x7f84c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84c80666c0 0x7f84c8068b80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f84d800b9d0 tx=0x7f84d8008040 comp rx=0 tx=0).stop 2024-09-13T17:09:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e81556d0 msgr2=0x7f84e81f05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e81556d0 0x7f84e81f05f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f84d000c450 tx=0x7f84d000c910 comp rx=0 tx=0).stop 2024-09-13T17:09:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 shutdown_connections 2024-09-13T17:09:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84c80666c0 0x7f84c8068b80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f84e814e250 0x7f84e81534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e81556d0 0x7f84e81f05f0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 --2- 172.21.15.39:0/4139185674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f84e80a72b0 0x7f84e8150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 >> 172.21.15.39:0/4139185674 conn(0x7f84e80a97b0 msgr2=0x7f84e80c55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 shutdown_connections 2024-09-13T17:09:17.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.738+0000 7f84ec9ef640 1 -- 172.21.15.39:0/4139185674 wait complete. 2024-09-13T17:09:17.757 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:09:17.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.810+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6f006f990 con 0x7fc6f0151460 2024-09-13T17:09:17.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.810+0000 7fc6d7fff640 1 -- 172.21.15.39:0/4293921290 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7fc6e80bc020 con 0x7fc6f0151460 2024-09-13T17:09:17.810 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:17.811 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6dc066690 msgr2=0x7fc6dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6dc066690 0x7fc6dc068b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0007a70 tx=0x7fc6e0008040 comp rx=0 tx=0).stop 2024-09-13T17:09:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f0151460 msgr2=0x7fc6f014aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f0151460 0x7fc6f014aba0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80af040 tx=0x7fc6e8019690 comp rx=0 tx=0).stop 2024-09-13T17:09:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 shutdown_connections 2024-09-13T17:09:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6dc066690 0x7fc6dc068b50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6f0151460 0x7fc6f014aba0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6f00c4530 0x7fc6f0071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 --2- 172.21.15.39:0/4293921290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6f006d020 0x7fc6f00c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:17.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 >> 172.21.15.39:0/4293921290 conn(0x7fc6f0069e10 msgr2=0x7fc6f00ab430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:17.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 shutdown_connections 2024-09-13T17:09:17.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:17.814+0000 7fc6f611a640 1 -- 172.21.15.39:0/4293921290 wait complete. 2024-09-13T17:09:17.829 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:09:19.758 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:19.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:19.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.890+0000 7f522af07640 1 Processor -- start 2024-09-13T17:09:19.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.890+0000 7f522af07640 1 -- start start 2024-09-13T17:09:19.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f522af07640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5224155910 0x7f52240c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f522af07640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522414f920 con 0x7f52240c1ab0 2024-09-13T17:09:19.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f522af07640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52240c1780 con 0x7f522414f540 2024-09-13T17:09:19.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f522af07640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52240709c0 con 0x7f5224155910 2024-09-13T17:09:19.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f5223fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f52240c1ab0 0x7f5224155200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36044/0 (socket says 172.21.15.39:36044) 2024-09-13T17:09:19.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f5223fff640 1 -- 172.21.15.39:0/2160676918 learned_addr learned my addr 172.21.15.39:0/2160676918 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:19.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52294a4640 1 --2- 172.21.15.39:0/2160676918 >> v2:172.21.15.84:3300/0 conn(0x7f5224155910 0x7f52240c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670338428 0 0) 0x7f522414f920 con 0x7f52240c1ab0 2024-09-13T17:09:19.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5214003530 con 0x7f52240c1ab0 2024-09-13T17:09:19.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1244322574 0 0) 0x7f5214003530 con 0x7f52240c1ab0 2024-09-13T17:09:19.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 >> v2:172.21.15.84:3300/0 conn(0x7f5224155910 msgr2=0x7f52240c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:19.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 --2- 172.21.15.39:0/2160676918 >> v2:172.21.15.84:3300/0 conn(0x7f5224155910 0x7f52240c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 >> v1:172.21.15.84:6789/0 conn(0x7f522414f540 legacy=0x7f522406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224071580 con 0x7f52240c1ab0 2024-09-13T17:09:19.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f5210001680 con 0x7f52240c1ab0 2024-09-13T17:09:19.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 >> v1:172.21.15.39:6789/0 conn(0x7f52240c1ab0 legacy=0x7f5224155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5214003980 0x7f5214003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52140047d0 0x7f5214004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214005190 0x7f521400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521400c000 con 0x7f5214005190 2024-09-13T17:09:19.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521400c170 con 0x7f52140047d0 2024-09-13T17:09:19.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521400c330 con 0x7f5214003980 2024-09-13T17:09:19.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52294a4640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214005190 0x7f521400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f5223fff640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52140047d0 0x7f5214004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f5228ca3640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5214003980 0x7f5214003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52294a4640 1 -- 172.21.15.39:0/2160676918 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5214003980 msgr2=0x7f5214003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:19.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52294a4640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5214003980 0x7f5214003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52294a4640 1 -- 172.21.15.39:0/2160676918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52140047d0 msgr2=0x7f5214004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:19.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52294a4640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52140047d0 0x7f5214004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52294a4640 1 -- 172.21.15.39:0/2160676918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214003530 con 0x7f5214005190 2024-09-13T17:09:19.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52294a4640 1 -- 172.21.15.39:0/2160676918 reap_dead start 2024-09-13T17:09:19.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52294a4640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214005190 0x7f521400b990 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f521c009fc0 tx=0x7f521c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc5865e24dcfb0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:19.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c0a5cd0 con 0x7f5214005190 2024-09-13T17:09:19.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c00f040 con 0x7f5214005190 2024-09-13T17:09:19.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.894+0000 7f52237fe640 1 -- 172.21.15.39:0/2160676918 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c003be0 con 0x7f5214005190 2024-09-13T17:09:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.898+0000 7f522af07640 1 -- 172.21.15.39:0/2160676918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214005190 msgr2=0x7f521400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.898+0000 7f522af07640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214005190 0x7f521400b990 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f521c009fc0 tx=0x7f521c01ccc0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.898+0000 7f522af07640 1 -- 172.21.15.39:0/2160676918 shutdown_connections 2024-09-13T17:09:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.898+0000 7f522af07640 1 --2- 172.21.15.39:0/2160676918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5214005190 0x7f521400b990 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.898+0000 7f522af07640 1 -- 172.21.15.39:0/2160676918 >> 172.21.15.39:0/2160676918 conn(0x7f52240c32a0 msgr2=0x7f52240c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.898+0000 7f522af07640 1 -- 172.21.15.39:0/2160676918 shutdown_connections 2024-09-13T17:09:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.898+0000 7f522af07640 1 -- 172.21.15.39:0/2160676918 wait complete. 2024-09-13T17:09:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.898+0000 7f522af07640 1 Processor -- start 2024-09-13T17:09:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.898+0000 7f522af07640 1 -- start start 2024-09-13T17:09:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f522af07640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5214005190 0x7f52241e8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f522af07640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52241502b0 0x7f522414bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f522af07640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522414c130 0x7f52241ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f522af07640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240a9550 con 0x7f52241502b0 2024-09-13T17:09:19.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f522af07640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522414b760 con 0x7f522414c130 2024-09-13T17:09:19.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f522af07640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52240a9a80 con 0x7f5214005190 2024-09-13T17:09:19.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5223fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52241502b0 0x7f522414bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5223fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52241502b0 0x7f522414bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45536/0 (socket says 172.21.15.39:45536) 2024-09-13T17:09:19.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5223fff640 1 -- 172.21.15.39:0/1184533787 learned_addr learned my addr 172.21.15.39:0/1184533787 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:19.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5228ca3640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5214005190 0x7f52241e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f52294a4640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522414c130 0x7f52241ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5223fff640 1 -- 172.21.15.39:0/1184533787 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5214005190 msgr2=0x7f52241e8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5223fff640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5214005190 0x7f52241e8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5223fff640 1 -- 172.21.15.39:0/1184533787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522414c130 msgr2=0x7f52241ea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5223fff640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522414c130 0x7f52241ea5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5223fff640 1 -- 172.21.15.39:0/1184533787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240a97a0 con 0x7f52241502b0 2024-09-13T17:09:19.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5223fff640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52241502b0 0x7f522414bbf0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f521000e7b0 tx=0x7f521000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:19.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f52217fa640 1 -- 172.21.15.39:0/1184533787 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521000c660 con 0x7f52241502b0 2024-09-13T17:09:19.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f52217fa640 1 -- 172.21.15.39:0/1184533787 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521000c800 con 0x7f52241502b0 2024-09-13T17:09:19.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522414b3f0 con 0x7f52241502b0 2024-09-13T17:09:19.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f52217fa640 1 -- 172.21.15.39:0/1184533787 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521000cbb0 con 0x7f52241502b0 2024-09-13T17:09:19.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522414ebe0 con 0x7f52241502b0 2024-09-13T17:09:19.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52240619c0 con 0x7f52241502b0 2024-09-13T17:09:19.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f52217fa640 1 -- 172.21.15.39:0/1184533787 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f521000cd10 con 0x7f52241502b0 2024-09-13T17:09:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f52217fa640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5208066690 0x7f5208068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f52217fa640 1 -- 172.21.15.39:0/1184533787 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f5210084ca0 con 0x7f52241502b0 2024-09-13T17:09:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5228ca3640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5208066690 0x7f5208068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.902+0000 7f5228ca3640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5208066690 0x7f5208068b50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5218001660 tx=0x7f5218009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:19.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.906+0000 7f52217fa640 1 -- 172.21.15.39:0/1184533787 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5210051320 con 0x7f52241502b0 2024-09-13T17:09:19.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff4505640 1 Processor -- start 2024-09-13T17:09:19.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff4505640 1 -- start start 2024-09-13T17:09:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff4505640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7fec06d020 0x7f7fec0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff4505640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec1518f0 con 0x7f7fec151460 2024-09-13T17:09:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff4505640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fec154100 con 0x7f7fec0c4530 2024-09-13T17:09:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff4505640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fec0a8c50 con 0x7f7fec06d020 2024-09-13T17:09:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff1aa0640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7fec151460 0x7f7fec14db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36056/0 (socket says 172.21.15.39:36056) 2024-09-13T17:09:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff1aa0640 1 -- 172.21.15.39:0/1877930128 learned_addr learned my addr 172.21.15.39:0/1877930128 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff22a1640 1 --2- 172.21.15.39:0/1877930128 >> v2:172.21.15.84:3300/0 conn(0x7f7fec06d020 0x7f7fec0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff22a1640 1 -- 172.21.15.39:0/1877930128 >> v1:172.21.15.84:6789/0 conn(0x7f7fec0c4530 legacy=0x7f7fec0c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff22a1640 1 -- 172.21.15.39:0/1877930128 >> v1:172.21.15.39:6789/0 conn(0x7f7fec151460 legacy=0x7f7fec14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.954+0000 7f7ff22a1640 1 -- 172.21.15.39:0/1877930128 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec1543e0 con 0x7f7fec06d020 2024-09-13T17:09:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff22a1640 1 --2- 172.21.15.39:0/1877930128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fec06d020 0x7f7fec0a4e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009d90 tx=0x7f7fe801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aabae397fc7f4e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff129f640 1 -- 172.21.15.39:0/1877930128 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80a5970 con 0x7f7fec06d020 2024-09-13T17:09:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff129f640 1 -- 172.21.15.39:0/1877930128 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe80a5ad0 con 0x7f7fec06d020 2024-09-13T17:09:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff129f640 1 -- 172.21.15.39:0/1877930128 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80a5e60 con 0x7f7fec06d020 2024-09-13T17:09:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff4505640 1 -- 172.21.15.39:0/1877930128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fec06d020 msgr2=0x7f7fec0a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff4505640 1 --2- 172.21.15.39:0/1877930128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fec06d020 0x7f7fec0a4e70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009d90 tx=0x7f7fe801c8b0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff4505640 1 -- 172.21.15.39:0/1877930128 shutdown_connections 2024-09-13T17:09:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff4505640 1 --2- 172.21.15.39:0/1877930128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fec06d020 0x7f7fec0a4e70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff4505640 1 -- 172.21.15.39:0/1877930128 >> 172.21.15.39:0/1877930128 conn(0x7f7fec069e10 msgr2=0x7f7fec06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:19.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff4505640 1 -- 172.21.15.39:0/1877930128 shutdown_connections 2024-09-13T17:09:19.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff4505640 1 -- 172.21.15.39:0/1877930128 wait complete. 2024-09-13T17:09:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff4505640 1 Processor -- start 2024-09-13T17:09:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.958+0000 7f7ff4505640 1 -- start start 2024-09-13T17:09:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff4505640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec06d020 0x7f7fec153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff4505640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fec0c4530 0x7f7fec146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff4505640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fec151460 0x7f7fec146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff4505640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec1f0680 con 0x7f7fec06d020 2024-09-13T17:09:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff4505640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec150dc0 con 0x7f7fec151460 2024-09-13T17:09:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff4505640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec14c5c0 con 0x7f7fec0c4530 2024-09-13T17:09:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff22a1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec06d020 0x7f7fec153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff22a1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec06d020 0x7f7fec153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45540/0 (socket says 172.21.15.39:45540) 2024-09-13T17:09:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff22a1640 1 -- 172.21.15.39:0/1870773439 learned_addr learned my addr 172.21.15.39:0/1870773439 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff1aa0640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fec0c4530 0x7f7fec146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff2aa2640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fec151460 0x7f7fec146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff22a1640 1 -- 172.21.15.39:0/1870773439 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fec0c4530 msgr2=0x7f7fec146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff22a1640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fec0c4530 0x7f7fec146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff22a1640 1 -- 172.21.15.39:0/1870773439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fec151460 msgr2=0x7f7fec146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff22a1640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fec151460 0x7f7fec146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff22a1640 1 -- 172.21.15.39:0/1870773439 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8009a70 con 0x7f7fec06d020 2024-09-13T17:09:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff22a1640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec06d020 0x7f7fec153080 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80194c0 tx=0x7f7fe8019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7fe2ffd640 1 -- 172.21.15.39:0/1870773439 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80aee10 con 0x7f7fec06d020 2024-09-13T17:09:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7fe2ffd640 1 -- 172.21.15.39:0/1870773439 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe80b7410 con 0x7f7fec06d020 2024-09-13T17:09:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fec1473d0 con 0x7f7fec06d020 2024-09-13T17:09:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7fe2ffd640 1 -- 172.21.15.39:0/1870773439 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80b77c0 con 0x7f7fec06d020 2024-09-13T17:09:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec1f03a0 con 0x7f7fec06d020 2024-09-13T17:09:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fec06d400 con 0x7f7fec06d020 2024-09-13T17:09:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7fe2ffd640 1 -- 172.21.15.39:0/1870773439 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7fe80b4050 con 0x7f7fec06d020 2024-09-13T17:09:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7fe2ffd640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7fd0066690 0x7f7fd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7fe2ffd640 1 -- 172.21.15.39:0/1870773439 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f7fe81204f0 con 0x7f7fec06d020 2024-09-13T17:09:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.962+0000 7f7ff1aa0640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7fd0066690 0x7f7fd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.966+0000 7f7ff1aa0640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7fd0066690 0x7f7fd0068b50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0bf9b0 tx=0x7f7fdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:19.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:19.974+0000 7f7fe2ffd640 1 -- 172.21.15.39:0/1870773439 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7fe80ecc90 con 0x7f7fec06d020 2024-09-13T17:09:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.106+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52240c2aa0 con 0x7f52241502b0 2024-09-13T17:09:20.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.106+0000 7f52217fa640 1 -- 172.21.15.39:0/1184533787 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f5210054940 con 0x7f52241502b0 2024-09-13T17:09:20.106 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:20.106 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:20.106 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5208066690 msgr2=0x7f5208068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5208066690 0x7f5208068b50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5218001660 tx=0x7f5218009040 comp rx=0 tx=0).stop 2024-09-13T17:09:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52241502b0 msgr2=0x7f522414bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52241502b0 0x7f522414bbf0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f521000e7b0 tx=0x7f521000ebb0 comp rx=0 tx=0).stop 2024-09-13T17:09:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 shutdown_connections 2024-09-13T17:09:20.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5208066690 0x7f5208068b50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:20.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f522414c130 0x7f52241ea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:20.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52241502b0 0x7f522414bbf0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:20.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 --2- 172.21.15.39:0/1184533787 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5214005190 0x7f52241e8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:20.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 >> 172.21.15.39:0/1184533787 conn(0x7f52240c32a0 msgr2=0x7f52240c5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:20.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 shutdown_connections 2024-09-13T17:09:20.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.110+0000 7f522af07640 1 -- 172.21.15.39:0/1184533787 wait complete. 2024-09-13T17:09:20.124 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:09:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.162+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fec0c5020 con 0x7f7fec06d020 2024-09-13T17:09:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7fe2ffd640 1 -- 172.21.15.39:0/1870773439 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f7fe80b2020 con 0x7f7fec06d020 2024-09-13T17:09:20.164 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:20.164 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:20.165 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7fd0066690 msgr2=0x7f7fd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7fd0066690 0x7f7fd0068b50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7fec0bf9b0 tx=0x7f7fdc009040 comp rx=0 tx=0).stop 2024-09-13T17:09:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec06d020 msgr2=0x7f7fec153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec06d020 0x7f7fec153080 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80194c0 tx=0x7f7fe8019650 comp rx=0 tx=0).stop 2024-09-13T17:09:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 shutdown_connections 2024-09-13T17:09:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7fd0066690 0x7f7fd0068b50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fec151460 0x7f7fec146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7fec0c4530 0x7f7fec146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 --2- 172.21.15.39:0/1870773439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fec06d020 0x7f7fec153080 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 >> 172.21.15.39:0/1870773439 conn(0x7f7fec069e10 msgr2=0x7f7fec0c4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 shutdown_connections 2024-09-13T17:09:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:20.166+0000 7f7ff4505640 1 -- 172.21.15.39:0/1870773439 wait complete. 2024-09-13T17:09:20.181 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:09:22.124 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:22.182 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:22.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.258+0000 7f478c822640 1 Processor -- start 2024-09-13T17:09:22.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.258+0000 7f478c822640 1 -- start start 2024-09-13T17:09:22.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f478814f540 0x7f47880c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f47880c1ab0 0x7f478806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478814f920 con 0x7f478806b360 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47880709c0 con 0x7f478814f540 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47880c1780 con 0x7f47880c1ab0 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478659c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f478814f540 0x7f47880c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478659c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f478814f540 0x7f47880c11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55514/0 (socket says 172.21.15.39:55514) 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478659c640 1 -- 172.21.15.39:0/1015483773 learned_addr learned my addr 172.21.15.39:0/1015483773 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478659c640 1 -- 172.21.15.39:0/1015483773 >> v2:172.21.15.84:3300/0 conn(0x7f47880c1ab0 msgr2=0x7f478806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f4785d9b640 1 --2- 172.21.15.39:0/1015483773 >> v2:172.21.15.84:3300/0 conn(0x7f47880c1ab0 0x7f478806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478659c640 1 --2- 172.21.15.39:0/1015483773 >> v2:172.21.15.84:3300/0 conn(0x7f47880c1ab0 0x7f478806ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478659c640 1 -- 172.21.15.39:0/1015483773 >> v1:172.21.15.84:6789/0 conn(0x7f478806b360 legacy=0x7f4788155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:22.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478659c640 1 -- 172.21.15.39:0/1015483773 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4788071580 con 0x7f478814f540 2024-09-13T17:09:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478659c640 1 --2- 172.21.15.39:0/1015483773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478814f540 0x7f47880c11a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f4788070380 tx=0x7f477801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3290f910cd91c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478559a640 1 -- 172.21.15.39:0/1015483773 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47780a5840 con 0x7f478814f540 2024-09-13T17:09:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478559a640 1 -- 172.21.15.39:0/1015483773 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47780a59a0 con 0x7f478814f540 2024-09-13T17:09:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478559a640 1 -- 172.21.15.39:0/1015483773 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47780a5d30 con 0x7f478814f540 2024-09-13T17:09:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 -- 172.21.15.39:0/1015483773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478814f540 msgr2=0x7f47880c11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 --2- 172.21.15.39:0/1015483773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478814f540 0x7f47880c11a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f4788070380 tx=0x7f477801c980 comp rx=0 tx=0).stop 2024-09-13T17:09:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 -- 172.21.15.39:0/1015483773 shutdown_connections 2024-09-13T17:09:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 --2- 172.21.15.39:0/1015483773 >> v2:172.21.15.84:3300/0 conn(0x7f47880c1ab0 0x7f478806ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 --2- 172.21.15.39:0/1015483773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478814f540 0x7f47880c11a0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.262+0000 7f478c822640 1 -- 172.21.15.39:0/1015483773 >> 172.21.15.39:0/1015483773 conn(0x7f47880c32a0 msgr2=0x7f47880c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 -- 172.21.15.39:0/1015483773 shutdown_connections 2024-09-13T17:09:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 -- 172.21.15.39:0/1015483773 wait complete. 2024-09-13T17:09:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 Processor -- start 2024-09-13T17:09:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 -- start start 2024-09-13T17:09:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f478806b360 0x7f47881525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47880c1ab0 0x7f47881497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f478814f540 0x7f47881ecb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478806e230 con 0x7f47880c1ab0 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788152b10 con 0x7f478814f540 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478815a700 con 0x7f478806b360 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4785d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47880c1ab0 0x7f47881497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4785d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47880c1ab0 0x7f47881497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55530/0 (socket says 172.21.15.39:55530) 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4785d9b640 1 -- 172.21.15.39:0/2035516108 learned_addr learned my addr 172.21.15.39:0/2035516108 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478659c640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f478806b360 0x7f47881525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4786d9d640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f478814f540 0x7f47881ecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4785d9b640 1 -- 172.21.15.39:0/2035516108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f478806b360 msgr2=0x7f47881525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4785d9b640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f478806b360 0x7f47881525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4785d9b640 1 -- 172.21.15.39:0/2035516108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f478814f540 msgr2=0x7f47881ecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4785d9b640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f478814f540 0x7f47881ecb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4785d9b640 1 -- 172.21.15.39:0/2035516108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778009b30 con 0x7f47880c1ab0 2024-09-13T17:09:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f4785d9b640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47880c1ab0 0x7f47881497f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f477000c7f0 tx=0x7f477000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f477effd640 1 -- 172.21.15.39:0/2035516108 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770004220 con 0x7f47880c1ab0 2024-09-13T17:09:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f477effd640 1 -- 172.21.15.39:0/2035516108 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47700043a0 con 0x7f47880c1ab0 2024-09-13T17:09:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478806e4d0 con 0x7f47880c1ab0 2024-09-13T17:09:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f477effd640 1 -- 172.21.15.39:0/2035516108 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770004510 con 0x7f47880c1ab0 2024-09-13T17:09:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478814a410 con 0x7f47880c1ab0 2024-09-13T17:09:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.266+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47880619c0 con 0x7f47880c1ab0 2024-09-13T17:09:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.270+0000 7f477effd640 1 -- 172.21.15.39:0/2035516108 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4770003a70 con 0x7f47880c1ab0 2024-09-13T17:09:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.270+0000 7f477effd640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47680666c0 0x7f4768068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.270+0000 7f477effd640 1 -- 172.21.15.39:0/2035516108 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f4770084d90 con 0x7f47880c1ab0 2024-09-13T17:09:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.270+0000 7f478659c640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47680666c0 0x7f4768068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.270+0000 7f478659c640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47680666c0 0x7f4768068b80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f47780ae000 tx=0x7f47780a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.274+0000 7f477effd640 1 -- 172.21.15.39:0/2035516108 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4770051490 con 0x7f47880c1ab0 2024-09-13T17:09:22.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa890314640 1 Processor -- start 2024-09-13T17:09:22.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa890314640 1 -- start start 2024-09-13T17:09:22.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa890314640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa888151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa890314640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8880c7470 0x7fa8880c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa890314640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88814f590 con 0x7fa88814f1b0 2024-09-13T17:09:22.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa890314640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88806f660 con 0x7fa8880c7470 2024-09-13T17:09:22.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa890314640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88806eae0 con 0x7fa88806e5e0 2024-09-13T17:09:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa88d8af640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa888151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa88d8af640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa888151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:53834/0 (socket says 172.21.15.39:53834) 2024-09-13T17:09:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa88d8af640 1 -- 172.21.15.39:0/3216418393 learned_addr learned my addr 172.21.15.39:0/3216418393 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa88e8b1640 1 --2- 172.21.15.39:0/3216418393 >> v2:172.21.15.84:3300/0 conn(0x7fa8880c7470 0x7fa8880c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa88d8af640 1 -- 172.21.15.39:0/3216418393 >> v2:172.21.15.84:3300/0 conn(0x7fa8880c7470 msgr2=0x7fa8880c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa88d8af640 1 --2- 172.21.15.39:0/3216418393 >> v2:172.21.15.84:3300/0 conn(0x7fa8880c7470 0x7fa8880c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa88d8af640 1 -- 172.21.15.39:0/3216418393 >> v1:172.21.15.84:6789/0 conn(0x7fa88814f1b0 legacy=0x7fa8880714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa88d8af640 1 -- 172.21.15.39:0/3216418393 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88806edc0 con 0x7fa88806e5e0 2024-09-13T17:09:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.306+0000 7fa88e8b1640 1 --2- 172.21.15.39:0/3216418393 >> v2:172.21.15.84:3300/0 conn(0x7fa8880c7470 0x7fa8880c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:09:22.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa88d8af640 1 --2- 172.21.15.39:0/3216418393 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa888151bb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009870 tx=0x7fa87c01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ac9af28d31a81d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:22.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa88d0ae640 1 -- 172.21.15.39:0/3216418393 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8880c69a0 con 0x7fa88806e5e0 2024-09-13T17:09:22.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa88d0ae640 1 -- 172.21.15.39:0/3216418393 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c0a7070 con 0x7fa88806e5e0 2024-09-13T17:09:22.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa88d0ae640 1 -- 172.21.15.39:0/3216418393 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0ab860 con 0x7fa88806e5e0 2024-09-13T17:09:22.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 -- 172.21.15.39:0/3216418393 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 msgr2=0x7fa888151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 --2- 172.21.15.39:0/3216418393 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa888151bb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009870 tx=0x7fa87c01c530 comp rx=0 tx=0).stop 2024-09-13T17:09:22.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 -- 172.21.15.39:0/3216418393 shutdown_connections 2024-09-13T17:09:22.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 --2- 172.21.15.39:0/3216418393 >> v2:172.21.15.84:3300/0 conn(0x7fa8880c7470 0x7fa8880c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 --2- 172.21.15.39:0/3216418393 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa888151bb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 -- 172.21.15.39:0/3216418393 >> 172.21.15.39:0/3216418393 conn(0x7fa8880a9720 msgr2=0x7fa8880a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:22.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 -- 172.21.15.39:0/3216418393 shutdown_connections 2024-09-13T17:09:22.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 -- 172.21.15.39:0/3216418393 wait complete. 2024-09-13T17:09:22.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 Processor -- start 2024-09-13T17:09:22.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.310+0000 7fa890314640 1 -- start start 2024-09-13T17:09:22.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa890314640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa88814bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa890314640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8880c7470 0x7fa8880c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa890314640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa88814f1b0 0x7fa88814a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa890314640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa888153340 con 0x7fa8880c7470 2024-09-13T17:09:22.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa890314640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88814c940 con 0x7fa88814f1b0 2024-09-13T17:09:22.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa890314640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8881efae0 con 0x7fa88806e5e0 2024-09-13T17:09:22.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88d8af640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8880c7470 0x7fa8880c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88d8af640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8880c7470 0x7fa8880c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55544/0 (socket says 172.21.15.39:55544) 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88d8af640 1 -- 172.21.15.39:0/3485718317 learned_addr learned my addr 172.21.15.39:0/3485718317 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88e0b0640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa88814bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88e8b1640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa88814f1b0 0x7fa88814a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88d8af640 1 -- 172.21.15.39:0/3485718317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 msgr2=0x7fa88814bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88d8af640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa88814bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88d8af640 1 -- 172.21.15.39:0/3485718317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa88814f1b0 msgr2=0x7fa88814a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88d8af640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa88814f1b0 0x7fa88814a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88d8af640 1 -- 172.21.15.39:0/3485718317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880009a70 con 0x7fa8880c7470 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88d8af640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8880c7470 0x7fa8880c3790 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa87c01cdd0 tx=0x7fa87c01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:22.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa87affd640 1 -- 172.21.15.39:0/3485718317 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0abea0 con 0x7fa8880c7470 2024-09-13T17:09:22.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa87affd640 1 -- 172.21.15.39:0/3485718317 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c0be3f0 con 0x7fa8880c7470 2024-09-13T17:09:22.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c0094d0 con 0x7fa8880c7470 2024-09-13T17:09:22.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa87affd640 1 -- 172.21.15.39:0/3485718317 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0be780 con 0x7fa8880c7470 2024-09-13T17:09:22.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88814c710 con 0x7fa8880c7470 2024-09-13T17:09:22.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8880c5c50 con 0x7fa8880c7470 2024-09-13T17:09:22.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa87affd640 1 -- 172.21.15.39:0/3485718317 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa87c0b43f0 con 0x7fa8880c7470 2024-09-13T17:09:22.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa87affd640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa86c066690 0x7fa86c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:22.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa87affd640 1 -- 172.21.15.39:0/3485718317 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fa87c11eb40 con 0x7fa8880c7470 2024-09-13T17:09:22.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88e0b0640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa86c066690 0x7fa86c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:22.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.314+0000 7fa88e0b0640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa86c066690 0x7fa86c068b50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa8881520d0 tx=0x7fa884009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:22.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.318+0000 7fa87affd640 1 -- 172.21.15.39:0/3485718317 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa87c0eb2f0 con 0x7fa8880c7470 2024-09-13T17:09:22.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f478814a260 con 0x7f47880c1ab0 2024-09-13T17:09:22.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f477effd640 1 -- 172.21.15.39:0/2035516108 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f477001e020 con 0x7f47880c1ab0 2024-09-13T17:09:22.473 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:22.473 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:22.473 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47680666c0 msgr2=0x7f4768068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47680666c0 0x7f4768068b80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f47780ae000 tx=0x7f47780a7070 comp rx=0 tx=0).stop 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47880c1ab0 msgr2=0x7f47881497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47880c1ab0 0x7f47881497f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f477000c7f0 tx=0x7f477000ccb0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 shutdown_connections 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47680666c0 0x7f4768068b80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f478814f540 0x7f47881ecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47880c1ab0 0x7f47881497f0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 --2- 172.21.15.39:0/2035516108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f478806b360 0x7f47881525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.474+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 >> 172.21.15.39:0/2035516108 conn(0x7f47880c32a0 msgr2=0x7f47880c0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:22.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.478+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 shutdown_connections 2024-09-13T17:09:22.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.478+0000 7f478c822640 1 -- 172.21.15.39:0/2035516108 wait complete. 2024-09-13T17:09:22.492 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:09:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.514+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa888151b70 con 0x7fa8880c7470 2024-09-13T17:09:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.514+0000 7fa87affd640 1 -- 172.21.15.39:0/3485718317 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7fa87c0ee910 con 0x7fa8880c7470 2024-09-13T17:09:22.516 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:22.516 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa86c066690 msgr2=0x7fa86c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa86c066690 0x7fa86c068b50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa8881520d0 tx=0x7fa884009040 comp rx=0 tx=0).stop 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8880c7470 msgr2=0x7fa8880c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8880c7470 0x7fa8880c3790 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa87c01cdd0 tx=0x7fa87c01ad70 comp rx=0 tx=0).stop 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 shutdown_connections 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa86c066690 0x7fa86c068b50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa88814f1b0 0x7fa88814a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8880c7470 0x7fa8880c3790 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 --2- 172.21.15.39:0/3485718317 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa88806e5e0 0x7fa88814bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 >> 172.21.15.39:0/3485718317 conn(0x7fa8880a9720 msgr2=0x7fa8881492f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 shutdown_connections 2024-09-13T17:09:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:22.518+0000 7fa890314640 1 -- 172.21.15.39:0/3485718317 wait complete. 2024-09-13T17:09:22.534 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:09:24.493 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:24.536 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.630+0000 7f95e996c640 1 Processor -- start 2024-09-13T17:09:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.630+0000 7f95e996c640 1 -- start start 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f95e40c7610 0x7f95e4151b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f95e40a72b0 0x7f95e4071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e406feb0 con 0x7f95e41556d0 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e40aae50 con 0x7f95e40a72b0 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e4155ab0 con 0x7f95e40c7610 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f95e40a72b0 0x7f95e4071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f95e40a72b0 0x7f95e4071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55560/0 (socket says 172.21.15.39:55560) 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e37fe640 1 -- 172.21.15.39:0/2688132812 learned_addr learned my addr 172.21.15.39:0/2688132812 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e27fc640 1 --2- 172.21.15.39:0/2688132812 >> v2:172.21.15.84:3300/0 conn(0x7f95e40c7610 0x7f95e4151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e37fe640 1 -- 172.21.15.39:0/2688132812 >> v2:172.21.15.84:3300/0 conn(0x7f95e40c7610 msgr2=0x7f95e4151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e37fe640 1 --2- 172.21.15.39:0/2688132812 >> v2:172.21.15.84:3300/0 conn(0x7f95e40c7610 0x7f95e4151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e37fe640 1 -- 172.21.15.39:0/2688132812 >> v1:172.21.15.84:6789/0 conn(0x7f95e41556d0 legacy=0x7f95e40c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:24.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e37fe640 1 -- 172.21.15.39:0/2688132812 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40c6c30 con 0x7f95e40a72b0 2024-09-13T17:09:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e37fe640 1 --2- 172.21.15.39:0/2688132812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 0x7f95e4071530 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f95e40ab0c0 tx=0x7f95dc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac336678944e0628 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e1ffb640 1 -- 172.21.15.39:0/2688132812 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0ad640 con 0x7f95e40a72b0 2024-09-13T17:09:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e1ffb640 1 -- 172.21.15.39:0/2688132812 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc0ad7a0 con 0x7f95e40a72b0 2024-09-13T17:09:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e1ffb640 1 -- 172.21.15.39:0/2688132812 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0adb30 con 0x7f95e40a72b0 2024-09-13T17:09:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 -- 172.21.15.39:0/2688132812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 msgr2=0x7f95e4071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 --2- 172.21.15.39:0/2688132812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 0x7f95e4071530 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f95e40ab0c0 tx=0x7f95dc01c390 comp rx=0 tx=0).stop 2024-09-13T17:09:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 -- 172.21.15.39:0/2688132812 shutdown_connections 2024-09-13T17:09:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 --2- 172.21.15.39:0/2688132812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 0x7f95e4071530 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 --2- 172.21.15.39:0/2688132812 >> v2:172.21.15.84:3300/0 conn(0x7f95e40c7610 0x7f95e4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 -- 172.21.15.39:0/2688132812 >> 172.21.15.39:0/2688132812 conn(0x7f95e40a97b0 msgr2=0x7f95e40a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.634+0000 7f95e996c640 1 -- 172.21.15.39:0/2688132812 shutdown_connections 2024-09-13T17:09:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 -- 172.21.15.39:0/2688132812 wait complete. 2024-09-13T17:09:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 Processor -- start 2024-09-13T17:09:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 -- start start 2024-09-13T17:09:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 0x7f95e40ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f95e41556d0 0x7f95e40c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e415ad70 0x7f95e40c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e41ed960 con 0x7f95e40a72b0 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40bca20 con 0x7f95e415ad70 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40be350 con 0x7f95e41556d0 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 0x7f95e40ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 0x7f95e40ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55568/0 (socket says 172.21.15.39:55568) 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e2ffd640 1 -- 172.21.15.39:0/435170682 learned_addr learned my addr 172.21.15.39:0/435170682 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e37fe640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e415ad70 0x7f95e40c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e27fc640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f95e41556d0 0x7f95e40c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e2ffd640 1 -- 172.21.15.39:0/435170682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f95e41556d0 msgr2=0x7f95e40c0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e2ffd640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f95e41556d0 0x7f95e40c0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e2ffd640 1 -- 172.21.15.39:0/435170682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e415ad70 msgr2=0x7f95e40c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e2ffd640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e415ad70 0x7f95e40c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e2ffd640 1 -- 172.21.15.39:0/435170682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0094d0 con 0x7f95e40a72b0 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e2ffd640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 0x7f95e40ac580 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f95d400b440 tx=0x7f95d400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95cbfff640 1 -- 172.21.15.39:0/435170682 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4011890 con 0x7f95e40a72b0 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95cbfff640 1 -- 172.21.15.39:0/435170682 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d40119f0 con 0x7f95e40a72b0 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e41f07e0 con 0x7f95e40a72b0 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95cbfff640 1 -- 172.21.15.39:0/435170682 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4011da0 con 0x7f95e40a72b0 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e40bcf50 con 0x7f95e40a72b0 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.638+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e40722c0 con 0x7f95e40a72b0 2024-09-13T17:09:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.642+0000 7f95cbfff640 1 -- 172.21.15.39:0/435170682 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f95d401a400 con 0x7f95e40a72b0 2024-09-13T17:09:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.642+0000 7f95cbfff640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95d0066690 0x7f95d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.642+0000 7f95e27fc640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95d0066690 0x7f95d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.642+0000 7f95cbfff640 1 -- 172.21.15.39:0/435170682 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f95d4084e00 con 0x7f95e40a72b0 2024-09-13T17:09:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.642+0000 7f95e27fc640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95d0066690 0x7f95d0068b50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f95d8003fd0 tx=0x7f95d80039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.646+0000 7f95cbfff640 1 -- 172.21.15.39:0/435170682 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f95d4051500 con 0x7f95e40a72b0 2024-09-13T17:09:24.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.662+0000 7fc79de95640 1 Processor -- start 2024-09-13T17:09:24.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.662+0000 7fc79de95640 1 -- start start 2024-09-13T17:09:24.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.662+0000 7fc79de95640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc7980c6320 0x7fc798153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.662+0000 7fc79de95640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980c6830 con 0x7fc79814a940 2024-09-13T17:09:24.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.662+0000 7fc79de95640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980c6b10 con 0x7fc798070810 2024-09-13T17:09:24.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.662+0000 7fc79de95640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980c6df0 con 0x7fc7980c6320 2024-09-13T17:09:24.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7977fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc79814a940 0x7fc7980c4290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53590/0 (socket says 172.21.15.39:53590) 2024-09-13T17:09:24.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7977fe640 1 -- 172.21.15.39:0/4292244291 learned_addr learned my addr 172.21.15.39:0/4292244291 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:24.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc796ffd640 1 --2- 172.21.15.39:0/4292244291 >> v2:172.21.15.84:3300/0 conn(0x7fc7980c6320 0x7fc798153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807129968 0 0) 0x7fc7980c6830 con 0x7fc79814a940 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc784003530 con 0x7fc79814a940 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843460559 0 0) 0x7fc784003530 con 0x7fc79814a940 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 >> v2:172.21.15.84:3300/0 conn(0x7fc7980c6320 msgr2=0x7fc798153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 --2- 172.21.15.39:0/4292244291 >> v2:172.21.15.84:3300/0 conn(0x7fc7980c6320 0x7fc798153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 >> v1:172.21.15.84:6789/0 conn(0x7fc798070810 legacy=0x7fc798070c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980c70d0 con 0x7fc79814a940 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fc788001680 con 0x7fc79814a940 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 >> v1:172.21.15.39:6789/0 conn(0x7fc79814a940 legacy=0x7fc7980c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc784003980 0x7fc784003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7840047d0 0x7fc784004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7840052b0 0x7fc78400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980c6df0 con 0x7fc784003980 2024-09-13T17:09:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78400c180 con 0x7fc7840052b0 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78400c340 con 0x7fc7840047d0 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7977fe640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc784003980 0x7fc784003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc797fff640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7840052b0 0x7fc78400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7977fe640 1 -- 172.21.15.39:0/4292244291 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7840047d0 msgr2=0x7fc784004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7977fe640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc7840047d0 0x7fc784004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7977fe640 1 -- 172.21.15.39:0/4292244291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7840052b0 msgr2=0x7fc78400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7977fe640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7840052b0 0x7fc78400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7977fe640 1 -- 172.21.15.39:0/4292244291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784003530 con 0x7fc784003980 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc797fff640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7840052b0 0x7fc78400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:24.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc797fff640 1 -- 172.21.15.39:0/4292244291 reap_dead start 2024-09-13T17:09:24.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7977fe640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc784003980 0x7fc784003da0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c210 tx=0x7fc78801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6547af3d7566d069 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:24.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788005ae0 con 0x7fc784003980 2024-09-13T17:09:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788005c60 con 0x7fc784003980 2024-09-13T17:09:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc7967fc640 1 -- 172.21.15.39:0/4292244291 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788007070 con 0x7fc784003980 2024-09-13T17:09:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc79de95640 1 -- 172.21.15.39:0/4292244291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc784003980 msgr2=0x7fc784003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.666+0000 7fc79de95640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc784003980 0x7fc784003da0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c210 tx=0x7fc78801cc40 comp rx=0 tx=0).stop 2024-09-13T17:09:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- 172.21.15.39:0/4292244291 shutdown_connections 2024-09-13T17:09:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 --2- 172.21.15.39:0/4292244291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc784003980 0x7fc784003da0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- 172.21.15.39:0/4292244291 >> 172.21.15.39:0/4292244291 conn(0x7fc7980ab460 msgr2=0x7fc7980ab870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- 172.21.15.39:0/4292244291 shutdown_connections 2024-09-13T17:09:24.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- 172.21.15.39:0/4292244291 wait complete. 2024-09-13T17:09:24.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 Processor -- start 2024-09-13T17:09:24.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- start start 2024-09-13T17:09:24.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc784003980 0x7fc7980c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7980c0b00 0x7fc798155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7980c0f60 0x7fc798156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980bde30 con 0x7fc7980c0b00 2024-09-13T17:09:24.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980c2f30 con 0x7fc7980c0f60 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980c2360 con 0x7fc784003980 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc796ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7980c0b00 0x7fc798155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc796ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7980c0b00 0x7fc798155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55576/0 (socket says 172.21.15.39:55576) 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc796ffd640 1 -- 172.21.15.39:0/850412933 learned_addr learned my addr 172.21.15.39:0/850412933 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc797fff640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7980c0f60 0x7fc798156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc796ffd640 1 -- 172.21.15.39:0/850412933 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc784003980 msgr2=0x7fc7980c34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc796ffd640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc784003980 0x7fc7980c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc796ffd640 1 -- 172.21.15.39:0/850412933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7980c0f60 msgr2=0x7fc798156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc796ffd640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7980c0f60 0x7fc798156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc796ffd640 1 -- 172.21.15.39:0/850412933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980c2600 con 0x7fc7980c0b00 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc797fff640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7980c0f60 0x7fc798156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc796ffd640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7980c0b00 0x7fc798155ec0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e840 tx=0x7fc78000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc78ffff640 1 -- 172.21.15.39:0/850412933 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780009620 con 0x7fc7980c0b00 2024-09-13T17:09:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc78ffff640 1 -- 172.21.15.39:0/850412933 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78000c8e0 con 0x7fc7980c0b00 2024-09-13T17:09:24.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7980c28e0 con 0x7fc7980c0b00 2024-09-13T17:09:24.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc78ffff640 1 -- 172.21.15.39:0/850412933 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78000cc90 con 0x7fc7980c0b00 2024-09-13T17:09:24.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7980c39f0 con 0x7fc7980c0b00 2024-09-13T17:09:24.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.670+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7980c6830 con 0x7fc7980c0b00 2024-09-13T17:09:24.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.674+0000 7fc78ffff640 1 -- 172.21.15.39:0/850412933 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc780013470 con 0x7fc7980c0b00 2024-09-13T17:09:24.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.674+0000 7fc78ffff640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7780666c0 0x7fc778068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:24.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.674+0000 7fc78ffff640 1 -- 172.21.15.39:0/850412933 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fc780085610 con 0x7fc7980c0b00 2024-09-13T17:09:24.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.674+0000 7fc7977fe640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7780666c0 0x7fc778068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:24.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.674+0000 7fc7977fe640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7780666c0 0x7fc778068b80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc7880ab000 tx=0x7fc788001a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:24.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.678+0000 7fc78ffff640 1 -- 172.21.15.39:0/850412933 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc780051d90 con 0x7fc7980c0b00 2024-09-13T17:09:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.842+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95e4071510 con 0x7f95e40a72b0 2024-09-13T17:09:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.842+0000 7f95cbfff640 1 -- 172.21.15.39:0/435170682 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f95d4015090 con 0x7f95e40a72b0 2024-09-13T17:09:24.842 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:24.842 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:24.843 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95d0066690 msgr2=0x7f95d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95d0066690 0x7f95d0068b50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f95d8003fd0 tx=0x7f95d80039a0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 msgr2=0x7f95e40ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 0x7f95e40ac580 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f95d400b440 tx=0x7f95d400b900 comp rx=0 tx=0).stop 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 shutdown_connections 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f95d0066690 0x7f95d0068b50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e415ad70 0x7f95e40c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f95e41556d0 0x7f95e40c0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 --2- 172.21.15.39:0/435170682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95e40a72b0 0x7f95e40ac580 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 >> 172.21.15.39:0/435170682 conn(0x7f95e40a97b0 msgr2=0x7f95e40a80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 shutdown_connections 2024-09-13T17:09:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.846+0000 7f95e996c640 1 -- 172.21.15.39:0/435170682 wait complete. 2024-09-13T17:09:24.860 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:09:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.870+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc798155be0 con 0x7fc7980c0b00 2024-09-13T17:09:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.870+0000 7fc78ffff640 1 -- 172.21.15.39:0/850412933 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7fc798155be0 con 0x7fc7980c0b00 2024-09-13T17:09:24.870 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:24.870 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:24.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7780666c0 msgr2=0x7fc778068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7780666c0 0x7fc778068b80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc7880ab000 tx=0x7fc788001a10 comp rx=0 tx=0).stop 2024-09-13T17:09:24.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7980c0b00 msgr2=0x7fc798155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:24.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7980c0b00 0x7fc798155ec0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fc78000e840 tx=0x7fc78000ed00 comp rx=0 tx=0).stop 2024-09-13T17:09:24.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 shutdown_connections 2024-09-13T17:09:24.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc7780666c0 0x7fc778068b80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7980c0f60 0x7fc798156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7980c0b00 0x7fc798155ec0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 --2- 172.21.15.39:0/850412933 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc784003980 0x7fc7980c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:24.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 >> 172.21.15.39:0/850412933 conn(0x7fc7980ab460 msgr2=0x7fc7981532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:24.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 shutdown_connections 2024-09-13T17:09:24.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:24.874+0000 7fc79de95640 1 -- 172.21.15.39:0/850412933 wait complete. 2024-09-13T17:09:24.887 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:09:26.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:26.888 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:26.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.994+0000 7f5f25902640 1 Processor -- start 2024-09-13T17:09:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.994+0000 7f5f25902640 1 -- start start 2024-09-13T17:09:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f25902640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5f20065ee0 0x7f5f2014f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f25902640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 0x7f5f200c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f25902640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2006ddf0 con 0x7f5f200c0a90 2024-09-13T17:09:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f25902640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f200c53d0 con 0x7f5f20065ee0 2024-09-13T17:09:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f25902640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200c5950 con 0x7f5f200c2ef0 2024-09-13T17:09:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1e7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5f200c0a90 0x7f5f200c1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53598/0 (socket says 172.21.15.39:53598) 2024-09-13T17:09:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1e7fc640 1 -- 172.21.15.39:0/3759659125 learned_addr learned my addr 172.21.15.39:0/3759659125 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1effd640 1 --2- 172.21.15.39:0/3759659125 >> v2:172.21.15.39:3300/0 conn(0x7f5f20065ee0 0x7f5f2014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1f7fe640 1 --2- 172.21.15.39:0/3759659125 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 0x7f5f200c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1dffb640 1 -- 172.21.15.39:0/3759659125 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275351145 0 0) 0x7f5f2006ddf0 con 0x7f5f200c0a90 2024-09-13T17:09:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1dffb640 1 -- 172.21.15.39:0/3759659125 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f08003530 con 0x7f5f200c0a90 2024-09-13T17:09:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1effd640 1 -- 172.21.15.39:0/3759659125 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 msgr2=0x7f5f200c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1effd640 1 --2- 172.21.15.39:0/3759659125 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 0x7f5f200c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1effd640 1 -- 172.21.15.39:0/3759659125 >> v1:172.21.15.39:6789/0 conn(0x7f5f200c0a90 legacy=0x7f5f200c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1effd640 1 -- 172.21.15.39:0/3759659125 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2006e040 con 0x7f5f20065ee0 2024-09-13T17:09:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1effd640 1 --2- 172.21.15.39:0/3759659125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 0x7f5f2014f230 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5f100097c0 tx=0x7f5f1001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac15afc7573aa3e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1dffb640 1 -- 172.21.15.39:0/3759659125 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f100ab670 con 0x7f5f20065ee0 2024-09-13T17:09:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1dffb640 1 -- 172.21.15.39:0/3759659125 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f100ab7d0 con 0x7f5f20065ee0 2024-09-13T17:09:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:26.998+0000 7f5f1dffb640 1 -- 172.21.15.39:0/3759659125 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f100abb60 con 0x7f5f20065ee0 2024-09-13T17:09:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- 172.21.15.39:0/3759659125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 msgr2=0x7f5f2014f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 --2- 172.21.15.39:0/3759659125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 0x7f5f2014f230 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5f100097c0 tx=0x7f5f1001c470 comp rx=0 tx=0).stop 2024-09-13T17:09:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- 172.21.15.39:0/3759659125 shutdown_connections 2024-09-13T17:09:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 --2- 172.21.15.39:0/3759659125 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 0x7f5f200c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 --2- 172.21.15.39:0/3759659125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 0x7f5f2014f230 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- 172.21.15.39:0/3759659125 >> 172.21.15.39:0/3759659125 conn(0x7f5f200c3310 msgr2=0x7f5f200c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- 172.21.15.39:0/3759659125 shutdown_connections 2024-09-13T17:09:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- 172.21.15.39:0/3759659125 wait complete. 2024-09-13T17:09:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 Processor -- start 2024-09-13T17:09:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- start start 2024-09-13T17:09:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 0x7f5f200abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f200c0a90 0x7f5f200bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 0x7f5f201eff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2014d7e0 con 0x7f5f20065ee0 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f20070780 con 0x7f5f200c0a90 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2014d190 con 0x7f5f200c2ef0 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 0x7f5f200abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 0x7f5f200abc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55588/0 (socket says 172.21.15.39:55588) 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1effd640 1 -- 172.21.15.39:0/84614474 learned_addr learned my addr 172.21.15.39:0/84614474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1f7fe640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 0x7f5f201eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1e7fc640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f200c0a90 0x7f5f200bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1effd640 1 -- 172.21.15.39:0/84614474 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 msgr2=0x7f5f201eff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1effd640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 0x7f5f201eff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1effd640 1 -- 172.21.15.39:0/84614474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f200c0a90 msgr2=0x7f5f200bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1effd640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f200c0a90 0x7f5f200bfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1effd640 1 -- 172.21.15.39:0/84614474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10009420 con 0x7f5f20065ee0 2024-09-13T17:09:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1e7fc640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f200c0a90 0x7f5f200bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f1effd640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 0x7f5f200abc60 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5f100ad040 tx=0x7f5f1001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f07fff640 1 -- 172.21.15.39:0/84614474 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f100b5810 con 0x7f5f20065ee0 2024-09-13T17:09:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f07fff640 1 -- 172.21.15.39:0/84614474 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f100b5990 con 0x7f5f20065ee0 2024-09-13T17:09:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2014d470 con 0x7f5f20065ee0 2024-09-13T17:09:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f07fff640 1 -- 172.21.15.39:0/84614474 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f100b5d40 con 0x7f5f20065ee0 2024-09-13T17:09:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.002+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f200c0410 con 0x7f5f20065ee0 2024-09-13T17:09:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.006+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2014f230 con 0x7f5f20065ee0 2024-09-13T17:09:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.006+0000 7f5f07fff640 1 -- 172.21.15.39:0/84614474 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f100c83f0 con 0x7f5f20065ee0 2024-09-13T17:09:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.006+0000 7f5f07fff640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f0c066690 0x7f5f0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.006+0000 7f5f1e7fc640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f0c066690 0x7f5f0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.006+0000 7f5f07fff640 1 -- 172.21.15.39:0/84614474 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f5f1011fc10 con 0x7f5f20065ee0 2024-09-13T17:09:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.006+0000 7f5f1e7fc640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f0c066690 0x7f5f0c068b50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5f2015bc50 tx=0x7f5f14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.010+0000 7f5f07fff640 1 -- 172.21.15.39:0/84614474 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5f100ec310 con 0x7f5f20065ee0 2024-09-13T17:09:27.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.030+0000 7f8baa928640 1 Processor -- start 2024-09-13T17:09:27.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.030+0000 7f8baa928640 1 -- start start 2024-09-13T17:09:27.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8ba4151500 0x7f8ba40c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8ba4071dd0 0x7f8ba4154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba40c5a30 con 0x7f8ba40c47c0 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba41576a0 con 0x7f8ba4151500 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba4157980 con 0x7f8ba4071dd0 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba3fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8ba4151500 0x7f8ba40c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba8ec5640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8ba40c47c0 0x7f8ba41545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53608/0 (socket says 172.21.15.39:53608) 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba3fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8ba4151500 0x7f8ba40c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55594/0 (socket says 172.21.15.39:55594) 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba8ec5640 1 -- 172.21.15.39:0/4039122511 learned_addr learned my addr 172.21.15.39:0/4039122511 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8b9bfff640 1 --2- 172.21.15.39:0/4039122511 >> v2:172.21.15.84:3300/0 conn(0x7f8ba4071dd0 0x7f8ba4154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba3fff640 1 -- 172.21.15.39:0/4039122511 >> v2:172.21.15.84:3300/0 conn(0x7f8ba4071dd0 msgr2=0x7f8ba4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba3fff640 1 --2- 172.21.15.39:0/4039122511 >> v2:172.21.15.84:3300/0 conn(0x7f8ba4071dd0 0x7f8ba4154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba3fff640 1 -- 172.21.15.39:0/4039122511 >> v1:172.21.15.39:6789/0 conn(0x7f8ba40c47c0 legacy=0x7f8ba41545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba3fff640 1 -- 172.21.15.39:0/4039122511 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba4157c60 con 0x7f8ba4151500 2024-09-13T17:09:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba3fff640 1 --2- 172.21.15.39:0/4039122511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4151500 0x7f8ba40c4280 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8b940097c0 tx=0x7f8b9401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec55cd8c39304aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba37fe640 1 -- 172.21.15.39:0/4039122511 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b940ac620 con 0x7f8ba4151500 2024-09-13T17:09:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba37fe640 1 -- 172.21.15.39:0/4039122511 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b940ac780 con 0x7f8ba4151500 2024-09-13T17:09:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8ba37fe640 1 -- 172.21.15.39:0/4039122511 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b940acb10 con 0x7f8ba4151500 2024-09-13T17:09:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 -- 172.21.15.39:0/4039122511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4151500 msgr2=0x7f8ba40c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 --2- 172.21.15.39:0/4039122511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4151500 0x7f8ba40c4280 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8b940097c0 tx=0x7f8b9401c3a0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 -- 172.21.15.39:0/4039122511 shutdown_connections 2024-09-13T17:09:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 --2- 172.21.15.39:0/4039122511 >> v2:172.21.15.84:3300/0 conn(0x7f8ba4071dd0 0x7f8ba4154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 --2- 172.21.15.39:0/4039122511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4151500 0x7f8ba40c4280 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.034+0000 7f8baa928640 1 -- 172.21.15.39:0/4039122511 >> 172.21.15.39:0/4039122511 conn(0x7f8ba40ab530 msgr2=0x7f8ba40ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 -- 172.21.15.39:0/4039122511 shutdown_connections 2024-09-13T17:09:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 -- 172.21.15.39:0/4039122511 wait complete. 2024-09-13T17:09:27.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 Processor -- start 2024-09-13T17:09:27.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 -- start start 2024-09-13T17:09:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4071dd0 0x7f8ba41ed130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba40c47c0 0x7f8ba40c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba4151500 0x7f8ba4153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba41efdc0 con 0x7f8ba4071dd0 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba41ed670 con 0x7f8ba4151500 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8baa928640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba4156e10 con 0x7f8ba40c47c0 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4071dd0 0x7f8ba41ed130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4071dd0 0x7f8ba41ed130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55596/0 (socket says 172.21.15.39:55596) 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba3fff640 1 -- 172.21.15.39:0/519911753 learned_addr learned my addr 172.21.15.39:0/519911753 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba8ec5640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba4151500 0x7f8ba4153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8b9bfff640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba40c47c0 0x7f8ba40c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba3fff640 1 -- 172.21.15.39:0/519911753 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba40c47c0 msgr2=0x7f8ba40c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba3fff640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba40c47c0 0x7f8ba40c6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba3fff640 1 -- 172.21.15.39:0/519911753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba4151500 msgr2=0x7f8ba4153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba3fff640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba4151500 0x7f8ba4153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba3fff640 1 -- 172.21.15.39:0/519911753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b94009420 con 0x7f8ba4071dd0 2024-09-13T17:09:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.038+0000 7f8ba3fff640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4071dd0 0x7f8ba41ed130 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8b94018fb0 tx=0x7f8b94019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8ba17fa640 1 -- 172.21.15.39:0/519911753 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b940b5410 con 0x7f8ba4071dd0 2024-09-13T17:09:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8ba17fa640 1 -- 172.21.15.39:0/519911753 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b940b5570 con 0x7f8ba4071dd0 2024-09-13T17:09:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba4156570 con 0x7f8ba4071dd0 2024-09-13T17:09:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8ba17fa640 1 -- 172.21.15.39:0/519911753 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b940b5920 con 0x7f8ba4071dd0 2024-09-13T17:09:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba4156b30 con 0x7f8ba4071dd0 2024-09-13T17:09:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba40c4280 con 0x7f8ba4071dd0 2024-09-13T17:09:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8ba17fa640 1 -- 172.21.15.39:0/519911753 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b940b3040 con 0x7f8ba4071dd0 2024-09-13T17:09:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8ba17fa640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b88066690 0x7f8b88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8ba17fa640 1 -- 172.21.15.39:0/519911753 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f8b9411f120 con 0x7f8ba4071dd0 2024-09-13T17:09:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8b9bfff640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b88066690 0x7f8b88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.042+0000 7f8b9bfff640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b88066690 0x7f8b88068b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8ba415e470 tx=0x7f8b8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:27.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.050+0000 7f8ba17fa640 1 -- 172.21.15.39:0/519911753 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8b940eb940 con 0x7f8ba4071dd0 2024-09-13T17:09:27.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.218+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f20066870 con 0x7f5f20065ee0 2024-09-13T17:09:27.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.218+0000 7f5f07fff640 1 -- 172.21.15.39:0/84614474 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f5f100ab630 con 0x7f5f20065ee0 2024-09-13T17:09:27.220 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:27.221 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:27.221 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f0c066690 msgr2=0x7f5f0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f0c066690 0x7f5f0c068b50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5f2015bc50 tx=0x7f5f14009040 comp rx=0 tx=0).stop 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 msgr2=0x7f5f200abc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 0x7f5f200abc60 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5f100ad040 tx=0x7f5f1001acb0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 shutdown_connections 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f0c066690 0x7f5f0c068b50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f200c0a90 0x7f5f200bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f200c2ef0 0x7f5f201eff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 --2- 172.21.15.39:0/84614474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f20065ee0 0x7f5f200abc60 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 >> 172.21.15.39:0/84614474 conn(0x7f5f200c3310 msgr2=0x7f5f201498d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 shutdown_connections 2024-09-13T17:09:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.222+0000 7f5f25902640 1 -- 172.21.15.39:0/84614474 wait complete. 2024-09-13T17:09:27.238 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:09:27.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ba40afb10 con 0x7f8ba4071dd0 2024-09-13T17:09:27.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8ba17fa640 1 -- 172.21.15.39:0/519911753 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f8b940eef60 con 0x7f8ba4071dd0 2024-09-13T17:09:27.249 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:27.249 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:27.249 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:27.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b88066690 msgr2=0x7f8b88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b88066690 0x7f8b88068b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8ba415e470 tx=0x7f8b8c008040 comp rx=0 tx=0).stop 2024-09-13T17:09:27.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4071dd0 msgr2=0x7f8ba41ed130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:27.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4071dd0 0x7f8ba41ed130 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8b94018fb0 tx=0x7f8b94019140 comp rx=0 tx=0).stop 2024-09-13T17:09:27.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 shutdown_connections 2024-09-13T17:09:27.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b88066690 0x7f8b88068b50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ba4151500 0x7f8ba4153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8ba40c47c0 0x7f8ba40c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 --2- 172.21.15.39:0/519911753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba4071dd0 0x7f8ba41ed130 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:27.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 >> 172.21.15.39:0/519911753 conn(0x7f8ba40ab530 msgr2=0x7f8ba4153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:27.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 shutdown_connections 2024-09-13T17:09:27.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:27.250+0000 7f8baa928640 1 -- 172.21.15.39:0/519911753 wait complete. 2024-09-13T17:09:27.267 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:09:29.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:29.269 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:29.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1291793640 1 Processor -- start 2024-09-13T17:09:29.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1291793640 1 -- start start 2024-09-13T17:09:29.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1291793640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f128c0c2ee0 0x7f128c0c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1291793640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128c06ddf0 con 0x7f128c065ee0 2024-09-13T17:09:29.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1291793640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128c0c4710 con 0x7f128c0c6270 2024-09-13T17:09:29.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1291793640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0c49d0 con 0x7f128c0c2ee0 2024-09-13T17:09:29.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128affd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f128c065ee0 0x7f128c0662c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:53612/0 (socket says 172.21.15.39:53612) 2024-09-13T17:09:29.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128affd640 1 -- 172.21.15.39:0/2376052335 learned_addr learned my addr 172.21.15.39:0/2376052335 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128a7fc640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f128c0c2ee0 0x7f128c0c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027816551 0 0) 0x7f128c06ddf0 con 0x7f128c065ee0 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1278003530 con 0x7f128c065ee0 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3134695072 0 0) 0x7f1278003530 con 0x7f128c065ee0 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f128c0c2ee0 msgr2=0x7f128c0c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f128c0c2ee0 0x7f128c0c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 >> v1:172.21.15.84:6789/0 conn(0x7f128c0c6270 legacy=0x7f128c0bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c154880 con 0x7f128c065ee0 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1284001680 con 0x7f128c065ee0 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 >> v1:172.21.15.39:6789/0 conn(0x7f128c065ee0 legacy=0x7f128c0662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12780039b0 0x7f1278003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f128c0c2ee0 0x7f12780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1278004d10 0x7f1278005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0c49d0 con 0x7f12780039b0 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127800baf0 con 0x7f1278004d10 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127800bcb0 con 0x7f128c0c2ee0 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128affd640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12780039b0 0x7f1278003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128b7fe640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1278004d10 0x7f1278005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128a7fc640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f128c0c2ee0 0x7f12780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128affd640 1 -- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f128c0c2ee0 msgr2=0x7f12780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128affd640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f128c0c2ee0 0x7f12780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128affd640 1 -- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1278004d10 msgr2=0x7f1278005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:29.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128affd640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1278004d10 0x7f1278005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128affd640 1 -- 172.21.15.39:0/2376052335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278003530 con 0x7f12780039b0 2024-09-13T17:09:29.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128b7fe640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1278004d10 0x7f1278005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:29.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.374+0000 7f128a7fc640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f128c0c2ee0 0x7f12780047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:29.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f128affd640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12780039b0 0x7f1278003dd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f128400c1c0 tx=0x7f128401cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df2bf7c2029d8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:29.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284005ae0 con 0x7f12780039b0 2024-09-13T17:09:29.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c06e490 con 0x7f12780039b0 2024-09-13T17:09:29.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1289ffb640 1 -- 172.21.15.39:0/2376052335 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12840a5880 con 0x7f12780039b0 2024-09-13T17:09:29.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1291793640 1 -- 172.21.15.39:0/2376052335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12780039b0 msgr2=0x7f1278003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1291793640 1 --2- 172.21.15.39:0/2376052335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12780039b0 0x7f1278003dd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f128400c1c0 tx=0x7f128401cb80 comp rx=0 tx=0).stop 2024-09-13T17:09:29.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f128b7fe640 1 -- 172.21.15.39:0/2376052335 reap_dead start 2024-09-13T17:09:29.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1291793640 1 -- 172.21.15.39:0/2376052335 shutdown_connections 2024-09-13T17:09:29.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1291793640 1 -- 172.21.15.39:0/2376052335 >> 172.21.15.39:0/2376052335 conn(0x7f128c0c3310 msgr2=0x7f128c0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1291793640 1 -- 172.21.15.39:0/2376052335 shutdown_connections 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1291793640 1 -- 172.21.15.39:0/2376052335 wait complete. 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1291793640 1 Processor -- start 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.378+0000 7f1291793640 1 -- start start 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f1291793640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12780039b0 0x7f128c0a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f1291793640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f128c147e90 0x7f128c1491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f1291793640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f128c0712b0 0x7f128c149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f1291793640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c148350 con 0x7f128c0712b0 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f1291793640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c148d80 con 0x7f128c147e90 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f1291793640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c1589d0 con 0x7f12780039b0 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f128c0712b0 0x7f128c149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f128c0712b0 0x7f128c149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55602/0 (socket says 172.21.15.39:55602) 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128b7fe640 1 -- 172.21.15.39:0/1350177483 learned_addr learned my addr 172.21.15.39:0/1350177483 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128a7fc640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f128c147e90 0x7f128c1491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128affd640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12780039b0 0x7f128c0a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128b7fe640 1 -- 172.21.15.39:0/1350177483 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12780039b0 msgr2=0x7f128c0a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128b7fe640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12780039b0 0x7f128c0a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128b7fe640 1 -- 172.21.15.39:0/1350177483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f128c147e90 msgr2=0x7f128c1491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128b7fe640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f128c147e90 0x7f128c1491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128b7fe640 1 -- 172.21.15.39:0/1350177483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c14a300 con 0x7f128c0712b0 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128b7fe640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f128c0712b0 0x7f128c149710 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f128c0c6650 tx=0x7f127c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f12837fe640 1 -- 172.21.15.39:0/1350177483 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c003980 con 0x7f128c0712b0 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f12837fe640 1 -- 172.21.15.39:0/1350177483 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c003b20 con 0x7f128c0712b0 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c14a5e0 con 0x7f128c0712b0 2024-09-13T17:09:29.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f12837fe640 1 -- 172.21.15.39:0/1350177483 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c005070 con 0x7f128c0712b0 2024-09-13T17:09:29.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c0a7c30 con 0x7f128c0712b0 2024-09-13T17:09:29.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c148ef0 con 0x7f128c0712b0 2024-09-13T17:09:29.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f12837fe640 1 -- 172.21.15.39:0/1350177483 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f127c0051d0 con 0x7f128c0712b0 2024-09-13T17:09:29.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f12837fe640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f126c0666c0 0x7f126c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f12837fe640 1 -- 172.21.15.39:0/1350177483 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7f127c0849f0 con 0x7f128c0712b0 2024-09-13T17:09:29.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.382+0000 7f128affd640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f126c0666c0 0x7f126c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.386+0000 7f128affd640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f126c0666c0 0x7f126c068b80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f12840ab000 tx=0x7f12840a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:29.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.390+0000 7f12837fe640 1 -- 172.21.15.39:0/1350177483 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f128c148ef0 con 0x7f128c0712b0 2024-09-13T17:09:29.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8dfc79640 1 Processor -- start 2024-09-13T17:09:29.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8dfc79640 1 -- start start 2024-09-13T17:09:29.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8dfc79640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8d80bf4f0 0x7fd8d8153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8dfc79640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d80bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8dfc79640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d80c1340 con 0x7fd8d8151860 2024-09-13T17:09:29.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8dfc79640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d81578f0 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8dfc79640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80c1590 con 0x7fd8d80611c0 2024-09-13T17:09:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8de216640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d80bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8dd214640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8d80bf4f0 0x7fd8d8153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8de216640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d80bfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:53936/0 (socket says 172.21.15.39:53936) 2024-09-13T17:09:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.402+0000 7fd8de216640 1 -- 172.21.15.39:0/3131942765 learned_addr learned my addr 172.21.15.39:0/3131942765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.406+0000 7fd8de216640 1 -- 172.21.15.39:0/3131942765 >> v2:172.21.15.84:3300/0 conn(0x7fd8d80bf4f0 msgr2=0x7fd8d8153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:29.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.406+0000 7fd8de216640 1 --2- 172.21.15.39:0/3131942765 >> v2:172.21.15.84:3300/0 conn(0x7fd8d80bf4f0 0x7fd8d8153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.406+0000 7fd8de216640 1 -- 172.21.15.39:0/3131942765 >> v1:172.21.15.84:6789/0 conn(0x7fd8d8151860 legacy=0x7fd8d8157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.406+0000 7fd8de216640 1 -- 172.21.15.39:0/3131942765 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d80c1870 con 0x7fd8d80611c0 2024-09-13T17:09:29.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.406+0000 7fd8de216640 1 --2- 172.21.15.39:0/3131942765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d80bfad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009e50 tx=0x7fd8d001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6fdb13d571e4c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dca13640 1 -- 172.21.15.39:0/3131942765 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00a5830 con 0x7fd8d80611c0 2024-09-13T17:09:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dca13640 1 -- 172.21.15.39:0/3131942765 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d00a5990 con 0x7fd8d80611c0 2024-09-13T17:09:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dca13640 1 -- 172.21.15.39:0/3131942765 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00a5d20 con 0x7fd8d80611c0 2024-09-13T17:09:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- 172.21.15.39:0/3131942765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 msgr2=0x7fd8d80bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 --2- 172.21.15.39:0/3131942765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d80bfad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009e50 tx=0x7fd8d001c970 comp rx=0 tx=0).stop 2024-09-13T17:09:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- 172.21.15.39:0/3131942765 shutdown_connections 2024-09-13T17:09:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 --2- 172.21.15.39:0/3131942765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d80bfad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 --2- 172.21.15.39:0/3131942765 >> v2:172.21.15.84:3300/0 conn(0x7fd8d80bf4f0 0x7fd8d8153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- 172.21.15.39:0/3131942765 >> 172.21.15.39:0/3131942765 conn(0x7fd8d8069e10 msgr2=0x7fd8d806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- 172.21.15.39:0/3131942765 shutdown_connections 2024-09-13T17:09:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- 172.21.15.39:0/3131942765 wait complete. 2024-09-13T17:09:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 Processor -- start 2024-09-13T17:09:29.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- start start 2024-09-13T17:09:29.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d81ed550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d80bf4f0 0x7fd8d814cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8151860 0x7fd8d81448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d8152dd0 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d81ed0c0 con 0x7fd8d8151860 2024-09-13T17:09:29.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d8145de0 con 0x7fd8d80611c0 2024-09-13T17:09:29.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dd214640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d80bf4f0 0x7fd8d814cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dd214640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d80bf4f0 0x7fd8d814cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55608/0 (socket says 172.21.15.39:55608) 2024-09-13T17:09:29.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dd214640 1 -- 172.21.15.39:0/1096497210 learned_addr learned my addr 172.21.15.39:0/1096497210 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:29.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dda15640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d81ed550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8de216640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8151860 0x7fd8d81448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dd214640 1 -- 172.21.15.39:0/1096497210 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 msgr2=0x7fd8d81ed550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dd214640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d81ed550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dd214640 1 -- 172.21.15.39:0/1096497210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8151860 msgr2=0x7fd8d81448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dd214640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8151860 0x7fd8d81448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dd214640 1 -- 172.21.15.39:0/1096497210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d0009b30 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dd214640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d80bf4f0 0x7fd8d814cec0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800e870 tx=0x7fd8c800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:29.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8ce7fc640 1 -- 172.21.15.39:0/1096497210 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8009510 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8ce7fc640 1 -- 172.21.15.39:0/1096497210 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c800eec0 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d81460c0 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.410+0000 7fd8ce7fc640 1 -- 172.21.15.39:0/1096497210 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800cae0 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.414+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d814fd00 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.414+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d80bfad0 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.414+0000 7fd8ce7fc640 1 -- 172.21.15.39:0/1096497210 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd8c800cc40 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.414+0000 7fd8ce7fc640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8bc066690 0x7fd8bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:29.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.414+0000 7fd8ce7fc640 1 -- 172.21.15.39:0/1096497210 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 8959+0+0 (secure 0 0 0) 0x7fd8c8084e50 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.414+0000 7fd8dda15640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8bc066690 0x7fd8bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:29.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.414+0000 7fd8dda15640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8bc066690 0x7fd8bc068b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4001810 tx=0x7fd8d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:29.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.418+0000 7fd8ce7fc640 1 -- 172.21.15.39:0/1096497210 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd8c8051600 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f128c148ef0 con 0x7f128c0712b0 2024-09-13T17:09:29.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f12837fe640 1 -- 172.21.15.39:0/1350177483 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7f128c148ef0 con 0x7f128c0712b0 2024-09-13T17:09:29.585 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:29.586 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:29.586 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f126c0666c0 msgr2=0x7f126c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f126c0666c0 0x7f126c068b80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f12840ab000 tx=0x7f12840a7070 comp rx=0 tx=0).stop 2024-09-13T17:09:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f128c0712b0 msgr2=0x7f128c149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f128c0712b0 0x7f128c149710 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f128c0c6650 tx=0x7f127c00d9d0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 shutdown_connections 2024-09-13T17:09:29.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f126c0666c0 0x7f126c068b80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f128c147e90 0x7f128c1491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f128c0712b0 0x7f128c149710 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 --2- 172.21.15.39:0/1350177483 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12780039b0 0x7f128c0a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 >> 172.21.15.39:0/1350177483 conn(0x7f128c0c3310 msgr2=0x7f128c0662c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.586+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 shutdown_connections 2024-09-13T17:09:29.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.590+0000 7f1291793640 1 -- 172.21.15.39:0/1350177483 wait complete. 2024-09-13T17:09:29.604 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:09:29.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.614+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8d80c0f10 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.614+0000 7fd8ce7fc640 1 -- 172.21.15.39:0/1096497210 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5943 (secure 0 0 0) 0x7fd8c8017090 con 0x7fd8d80bf4f0 2024-09-13T17:09:29.615 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:29.616 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130},{"gid":7485,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:07:38.167209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7338},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7338":{"gid":7338,"name":"c","rank":0,"incarnation":137,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6837/2521680730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2521680730},{"type":"v1","addr":"172.21.15.39:6837","nonce":2521680730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:08:52.839479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6071},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6071":{"gid":6071,"name":"b","rank":0,"incarnation":142,"state":"up:active","state_seq":19,"addr":"172.21.15.84:6833/1750130579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":1750130579},{"type":"v1","addr":"172.21.15.84:6833","nonce":1750130579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-13T17:09:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8bc066690 msgr2=0x7fd8bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8bc066690 0x7fd8bc068b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4001810 tx=0x7fd8d4009040 comp rx=0 tx=0).stop 2024-09-13T17:09:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d80bf4f0 msgr2=0x7fd8d814cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d80bf4f0 0x7fd8d814cec0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800e870 tx=0x7fd8c800ec70 comp rx=0 tx=0).stop 2024-09-13T17:09:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 shutdown_connections 2024-09-13T17:09:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8bc066690 0x7fd8bc068b50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8d8151860 0x7fd8d81448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d80bf4f0 0x7fd8d814cec0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 --2- 172.21.15.39:0/1096497210 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8d80611c0 0x7fd8d81ed550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 >> 172.21.15.39:0/1096497210 conn(0x7fd8d8069e10 msgr2=0x7fd8d80c28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 shutdown_connections 2024-09-13T17:09:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:29.618+0000 7fd8dfc79640 1 -- 172.21.15.39:0/1096497210 wait complete. 2024-09-13T17:09:29.632 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-09-13T17:09:30.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:09:30.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:09:31.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:31.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.738+0000 7fd94b5fb640 1 Processor -- start 2024-09-13T17:09:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.738+0000 7fd94b5fb640 1 -- start start 2024-09-13T17:09:31.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd94b5fb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd944151500 0x7fd9440c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd94b5fb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd944154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd94b5fb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9440c5a30 con 0x7fd9440c47c0 2024-09-13T17:09:31.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd94b5fb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9441576a0 con 0x7fd944151500 2024-09-13T17:09:31.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd94b5fb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd944157980 con 0x7fd944071dd0 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd949397640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd944151500 0x7fd9440c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd949b98640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd9440c47c0 0x7fd9441545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49368/0 (socket says 172.21.15.39:49368) 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd949b98640 1 -- 172.21.15.39:0/3196175229 learned_addr learned my addr 172.21.15.39:0/3196175229 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd948b96640 1 --2- 172.21.15.39:0/3196175229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd944154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd949397640 1 -- 172.21.15.39:0/3196175229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 msgr2=0x7fd944154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd949397640 1 --2- 172.21.15.39:0/3196175229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd944154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd949397640 1 -- 172.21.15.39:0/3196175229 >> v1:172.21.15.39:6789/0 conn(0x7fd9440c47c0 legacy=0x7fd9441545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd949397640 1 -- 172.21.15.39:0/3196175229 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944157c60 con 0x7fd944151500 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd943fff640 1 -- 172.21.15.39:0/3196175229 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456469611 0 0) 0x7fd9440c5a30 con 0x7fd9440c47c0 2024-09-13T17:09:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd948b96640 1 --2- 172.21.15.39:0/3196175229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd944154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd949397640 1 --2- 172.21.15.39:0/3196175229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 0x7fd9440c4280 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0097c0 tx=0x7fd93c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8f83b485e274cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd943fff640 1 -- 172.21.15.39:0/3196175229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0ab600 con 0x7fd944151500 2024-09-13T17:09:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd943fff640 1 -- 172.21.15.39:0/3196175229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c0ab760 con 0x7fd944151500 2024-09-13T17:09:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.742+0000 7fd943fff640 1 -- 172.21.15.39:0/3196175229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0abaf0 con 0x7fd944151500 2024-09-13T17:09:31.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3196175229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 msgr2=0x7fd9440c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 --2- 172.21.15.39:0/3196175229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 0x7fd9440c4280 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0097c0 tx=0x7fd93c01c3e0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3196175229 shutdown_connections 2024-09-13T17:09:31.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 --2- 172.21.15.39:0/3196175229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd944154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 --2- 172.21.15.39:0/3196175229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 0x7fd9440c4280 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3196175229 >> 172.21.15.39:0/3196175229 conn(0x7fd9440ab530 msgr2=0x7fd9440ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3196175229 shutdown_connections 2024-09-13T17:09:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3196175229 wait complete. 2024-09-13T17:09:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 Processor -- start 2024-09-13T17:09:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- start start 2024-09-13T17:09:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd9441f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9440c47c0 0x7fd9441f1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 0x7fd9441561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94414c960 con 0x7fd944151500 2024-09-13T17:09:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94414c150 con 0x7fd9440c47c0 2024-09-13T17:09:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441f0ae0 con 0x7fd944071dd0 2024-09-13T17:09:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949b98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 0x7fd9441561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949b98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 0x7fd9441561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59030/0 (socket says 172.21.15.39:59030) 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949b98640 1 -- 172.21.15.39:0/3030296680 learned_addr learned my addr 172.21.15.39:0/3030296680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949397640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd9441f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd948b96640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9440c47c0 0x7fd9441f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949b98640 1 -- 172.21.15.39:0/3030296680 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 msgr2=0x7fd9441f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949b98640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd9441f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949b98640 1 -- 172.21.15.39:0/3030296680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9440c47c0 msgr2=0x7fd9441f1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949b98640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9440c47c0 0x7fd9441f1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949b98640 1 -- 172.21.15.39:0/3030296680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c009420 con 0x7fd944151500 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949397640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd9441f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:31.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd949b98640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 0x7fd9441561a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c7a0 tx=0x7fd93400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd941ffb640 1 -- 172.21.15.39:0/3030296680 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93400a640 con 0x7fd944151500 2024-09-13T17:09:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd941ffb640 1 -- 172.21.15.39:0/3030296680 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93400a7a0 con 0x7fd944151500 2024-09-13T17:09:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd944156ec0 con 0x7fd944151500 2024-09-13T17:09:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd941ffb640 1 -- 172.21.15.39:0/3030296680 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93400abd0 con 0x7fd944151500 2024-09-13T17:09:31.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.746+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94414c3a0 con 0x7fd944151500 2024-09-13T17:09:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.750+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9440c4280 con 0x7fd944151500 2024-09-13T17:09:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.750+0000 7fd941ffb640 1 -- 172.21.15.39:0/3030296680 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd93400ad30 con 0x7fd944151500 2024-09-13T17:09:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.750+0000 7fd941ffb640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9300666c0 0x7fd930068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.750+0000 7fd941ffb640 1 -- 172.21.15.39:0/3030296680 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7fd934084960 con 0x7fd944151500 2024-09-13T17:09:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.750+0000 7fd949397640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9300666c0 0x7fd930068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.750+0000 7fd949397640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9300666c0 0x7fd930068b80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0af000 tx=0x7fd93c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:31.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.754+0000 7fd941ffb640 1 -- 172.21.15.39:0/3030296680 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd934051030 con 0x7fd944151500 2024-09-13T17:09:31.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.774+0000 7fc981b10640 1 Processor -- start 2024-09-13T17:09:31.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.774+0000 7fc981b10640 1 -- start start 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc981b10640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc97c0aabc0 0x7fc97c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc981b10640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c06c870 0x7fc97c0c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc981b10640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c152a80 con 0x7fc97c0c3ee0 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc981b10640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0c30f0 con 0x7fc97c0aabc0 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc981b10640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c151e00 con 0x7fc97c06c870 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc97c0aabc0 0x7fc97c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc97c0aabc0 0x7fc97c0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59032/0 (socket says 172.21.15.39:59032) 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97b7fe640 1 -- 172.21.15.39:0/4006254042 learned_addr learned my addr 172.21.15.39:0/4006254042 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97affd640 1 --2- 172.21.15.39:0/4006254042 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c06c870 0x7fc97c0c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97b7fe640 1 -- 172.21.15.39:0/4006254042 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c06c870 msgr2=0x7fc97c0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97b7fe640 1 --2- 172.21.15.39:0/4006254042 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c06c870 0x7fc97c0c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97b7fe640 1 -- 172.21.15.39:0/4006254042 >> v1:172.21.15.84:6789/0 conn(0x7fc97c0c3ee0 legacy=0x7fc97c1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:31.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97b7fe640 1 -- 172.21.15.39:0/4006254042 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c1520e0 con 0x7fc97c0aabc0 2024-09-13T17:09:31.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97b7fe640 1 --2- 172.21.15.39:0/4006254042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c0aabc0 0x7fc97c0aafa0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fc970009870 tx=0x7fc97001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7992b601378f332 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:31.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97a7fc640 1 -- 172.21.15.39:0/4006254042 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9700aa610 con 0x7fc97c0aabc0 2024-09-13T17:09:31.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97a7fc640 1 -- 172.21.15.39:0/4006254042 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9700a6040 con 0x7fc97c0aabc0 2024-09-13T17:09:31.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.778+0000 7fc97a7fc640 1 -- 172.21.15.39:0/4006254042 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9700aa990 con 0x7fc97c0aabc0 2024-09-13T17:09:31.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- 172.21.15.39:0/4006254042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c0aabc0 msgr2=0x7fc97c0aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 --2- 172.21.15.39:0/4006254042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c0aabc0 0x7fc97c0aafa0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fc970009870 tx=0x7fc97001c4f0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- 172.21.15.39:0/4006254042 shutdown_connections 2024-09-13T17:09:31.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 --2- 172.21.15.39:0/4006254042 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c06c870 0x7fc97c0c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 --2- 172.21.15.39:0/4006254042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c0aabc0 0x7fc97c0aafa0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- 172.21.15.39:0/4006254042 >> 172.21.15.39:0/4006254042 conn(0x7fc97c069e10 msgr2=0x7fc97c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:31.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- 172.21.15.39:0/4006254042 shutdown_connections 2024-09-13T17:09:31.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- 172.21.15.39:0/4006254042 wait complete. 2024-09-13T17:09:31.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 Processor -- start 2024-09-13T17:09:31.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- start start 2024-09-13T17:09:31.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c06c870 0x7fc97c1ede60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c0aabc0 0x7fc97c0a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c0c3ee0 0x7fc97c0a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0a6180 con 0x7fc97c06c870 2024-09-13T17:09:31.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0a5e30 con 0x7fc97c0c3ee0 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c072260 con 0x7fc97c0aabc0 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c06c870 0x7fc97c1ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c06c870 0x7fc97c1ede60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59046/0 (socket says 172.21.15.39:59046) 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97b7fe640 1 -- 172.21.15.39:0/1725290000 learned_addr learned my addr 172.21.15.39:0/1725290000 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97affd640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c0aabc0 0x7fc97c0a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97bfff640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c0c3ee0 0x7fc97c0a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97b7fe640 1 -- 172.21.15.39:0/1725290000 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c0aabc0 msgr2=0x7fc97c0a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97b7fe640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c0aabc0 0x7fc97c0a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97b7fe640 1 -- 172.21.15.39:0/1725290000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c0c3ee0 msgr2=0x7fc97c0a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97b7fe640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c0c3ee0 0x7fc97c0a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97b7fe640 1 -- 172.21.15.39:0/1725290000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9700094d0 con 0x7fc97c06c870 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc97b7fe640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c06c870 0x7fc97c1ede60 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc9700ac040 tx=0x7fc97001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:31.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc963fff640 1 -- 172.21.15.39:0/1725290000 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9700bd5f0 con 0x7fc97c06c870 2024-09-13T17:09:31.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc963fff640 1 -- 172.21.15.39:0/1725290000 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9700bd750 con 0x7fc97c06c870 2024-09-13T17:09:31.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc963fff640 1 -- 172.21.15.39:0/1725290000 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9700bdb00 con 0x7fc97c06c870 2024-09-13T17:09:31.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c14a270 con 0x7fc97c06c870 2024-09-13T17:09:31.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.782+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c1ee990 con 0x7fc97c06c870 2024-09-13T17:09:31.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.786+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c0a62c0 con 0x7fc97c06c870 2024-09-13T17:09:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.786+0000 7fc963fff640 1 -- 172.21.15.39:0/1725290000 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc9700b33f0 con 0x7fc97c06c870 2024-09-13T17:09:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.786+0000 7fc963fff640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc968066690 0x7fc968068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.786+0000 7fc963fff640 1 -- 172.21.15.39:0/1725290000 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7fc97011fbb0 con 0x7fc97c06c870 2024-09-13T17:09:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.786+0000 7fc97affd640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc968066690 0x7fc968068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.786+0000 7fc97affd640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc968066690 0x7fc968068b50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0c2200 tx=0x7fc96c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.790+0000 7fc963fff640 1 -- 172.21.15.39:0/1725290000 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc97c0a62c0 con 0x7fc97c06c870 2024-09-13T17:09:31.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.950+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd944060ba0 con 0x7fd944151500 2024-09-13T17:09:31.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.950+0000 7fd941ffb640 1 -- 172.21.15.39:0/3030296680 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4435 (secure 0 0 0) 0x7fd934018070 con 0x7fd944151500 2024-09-13T17:09:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:31.951 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":148,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:31.008295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:reconnect","state_seq":52,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:31.008295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:31.951 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 148 2024-09-13T17:09:31.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9300666c0 msgr2=0x7fd930068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9300666c0 0x7fd930068b80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0af000 tx=0x7fd93c0a7000 comp rx=0 tx=0).stop 2024-09-13T17:09:31.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 msgr2=0x7fd9441561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 0x7fd9441561a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c7a0 tx=0x7fd93400cc60 comp rx=0 tx=0).stop 2024-09-13T17:09:31.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 shutdown_connections 2024-09-13T17:09:31.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9300666c0 0x7fd930068b80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9440c47c0 0x7fd9441f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944151500 0x7fd9441561a0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 --2- 172.21.15.39:0/3030296680 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd944071dd0 0x7fd9441f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 >> 172.21.15.39:0/3030296680 conn(0x7fd9440ab530 msgr2=0x7fd944154280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:31.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 shutdown_connections 2024-09-13T17:09:31.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.954+0000 7fd94b5fb640 1 -- 172.21.15.39:0/3030296680 wait complete. 2024-09-13T17:09:31.970 INFO:tasks.mds_thrash.fs.[a]:mds.c down, removed from mdsmap 2024-09-13T17:09:31.971 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:09:31.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:31.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.986+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc97c152a80 con 0x7fc97c06c870 2024-09-13T17:09:31.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.986+0000 7fc963fff640 1 -- 172.21.15.39:0/1725290000 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+4435 (secure 0 0 0) 0x7fc97c152a80 con 0x7fc97c06c870 2024-09-13T17:09:31.987 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:31.987 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":148,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:31.008295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:reconnect","state_seq":52,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:31.008295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:31.987 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 148 2024-09-13T17:09:31.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc968066690 msgr2=0x7fc968068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc968066690 0x7fc968068b50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0c2200 tx=0x7fc96c008040 comp rx=0 tx=0).stop 2024-09-13T17:09:31.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c06c870 msgr2=0x7fc97c1ede60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c06c870 0x7fc97c1ede60 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc9700ac040 tx=0x7fc97001ad30 comp rx=0 tx=0).stop 2024-09-13T17:09:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 shutdown_connections 2024-09-13T17:09:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc968066690 0x7fc968068b50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c0c3ee0 0x7fc97c0a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc97c0aabc0 0x7fc97c0a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 --2- 172.21.15.39:0/1725290000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc97c06c870 0x7fc97c1ede60 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 >> 172.21.15.39:0/1725290000 conn(0x7fc97c069e10 msgr2=0x7fc97c06cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 shutdown_connections 2024-09-13T17:09:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:31.990+0000 7fc981b10640 1 -- 172.21.15.39:0/1725290000 wait complete. 2024-09-13T17:09:32.007 INFO:tasks.mds_thrash.fs.[b]:mds.b down, removed from mdsmap 2024-09-13T17:09:32.007 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:09:32.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:32.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.106+0000 7fd8cf8ce640 1 Processor -- start 2024-09-13T17:09:32.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.106+0000 7fd8cf8ce640 1 -- start start 2024-09-13T17:09:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cf8ce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd8c80c6060 0x7fd8c8150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cf8ce640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c80713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cf8ce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c80c4760 con 0x7fd8c814a770 2024-09-13T17:09:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cf8ce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c806df80 con 0x7fd8c80c6060 2024-09-13T17:09:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cf8ce640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80c1950 con 0x7fd8c814ab50 2024-09-13T17:09:32.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cce69640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd8c80c6060 0x7fd8c8150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cd66a640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd8c814a770 0x7fd8c806ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49372/0 (socket says 172.21.15.39:49372) 2024-09-13T17:09:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cd66a640 1 -- 172.21.15.39:0/575076540 learned_addr learned my addr 172.21.15.39:0/575076540 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cce69640 1 -- 172.21.15.39:0/575076540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 msgr2=0x7fd8c80713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cde6b640 1 --2- 172.21.15.39:0/575076540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c80713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8bffff640 1 -- 172.21.15.39:0/575076540 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364444169 0 0) 0x7fd8c80c4760 con 0x7fd8c814a770 2024-09-13T17:09:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cce69640 1 --2- 172.21.15.39:0/575076540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c80713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cce69640 1 -- 172.21.15.39:0/575076540 >> v1:172.21.15.39:6789/0 conn(0x7fd8c814a770 legacy=0x7fd8c806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cce69640 1 -- 172.21.15.39:0/575076540 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c814afe0 con 0x7fd8c80c6060 2024-09-13T17:09:32.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cde6b640 1 --2- 172.21.15.39:0/575076540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c80713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:32.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8cce69640 1 --2- 172.21.15.39:0/575076540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 0x7fd8c8150ad0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800ba60 tx=0x7fd8b801eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df892406cb1774ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:32.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8bffff640 1 -- 172.21.15.39:0/575076540 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80a7b80 con 0x7fd8c80c6060 2024-09-13T17:09:32.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8bffff640 1 -- 172.21.15.39:0/575076540 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b80a7ce0 con 0x7fd8c80c6060 2024-09-13T17:09:32.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.110+0000 7fd8bffff640 1 -- 172.21.15.39:0/575076540 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80af660 con 0x7fd8c80c6060 2024-09-13T17:09:32.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/575076540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 msgr2=0x7fd8c8150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 --2- 172.21.15.39:0/575076540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 0x7fd8c8150ad0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800ba60 tx=0x7fd8b801eab0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/575076540 shutdown_connections 2024-09-13T17:09:32.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 --2- 172.21.15.39:0/575076540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c80713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 --2- 172.21.15.39:0/575076540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 0x7fd8c8150ad0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/575076540 >> 172.21.15.39:0/575076540 conn(0x7fd8c80c32a0 msgr2=0x7fd8c80c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:32.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/575076540 shutdown_connections 2024-09-13T17:09:32.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/575076540 wait complete. 2024-09-13T17:09:32.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 Processor -- start 2024-09-13T17:09:32.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- start start 2024-09-13T17:09:32.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 0x7fd8c80ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c814a770 0x7fd8c8152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c81530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8154c70 con 0x7fd8c80c6060 2024-09-13T17:09:32.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c814e0c0 con 0x7fd8c814a770 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8151fe0 con 0x7fd8c814ab50 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cd66a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 0x7fd8c80ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cd66a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 0x7fd8c80ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59068/0 (socket says 172.21.15.39:59068) 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cd66a640 1 -- 172.21.15.39:0/3115600116 learned_addr learned my addr 172.21.15.39:0/3115600116 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cce69640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c814a770 0x7fd8c8152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cde6b640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c81530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cd66a640 1 -- 172.21.15.39:0/3115600116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 msgr2=0x7fd8c81530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cd66a640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c81530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cd66a640 1 -- 172.21.15.39:0/3115600116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c814a770 msgr2=0x7fd8c8152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cd66a640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c814a770 0x7fd8c8152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cd66a640 1 -- 172.21.15.39:0/3115600116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b800b6c0 con 0x7fd8c80c6060 2024-09-13T17:09:32.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cde6b640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c81530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:32.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cd66a640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 0x7fd8c80ab400 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000c480 tx=0x7fd8c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:32.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8bdffb640 1 -- 172.21.15.39:0/3115600116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00074e0 con 0x7fd8c80c6060 2024-09-13T17:09:32.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8bdffb640 1 -- 172.21.15.39:0/3115600116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0007660 con 0x7fd8c80c6060 2024-09-13T17:09:32.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c81e8f70 con 0x7fd8c80c6060 2024-09-13T17:09:32.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8bdffb640 1 -- 172.21.15.39:0/3115600116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0004a30 con 0x7fd8c80c6060 2024-09-13T17:09:32.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.114+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c8154900 con 0x7fd8c80c6060 2024-09-13T17:09:32.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.118+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c80619c0 con 0x7fd8c80c6060 2024-09-13T17:09:32.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.118+0000 7fd8bdffb640 1 -- 172.21.15.39:0/3115600116 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd8c0004b90 con 0x7fd8c80c6060 2024-09-13T17:09:32.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.118+0000 7fd8bdffb640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8ac066690 0x7fd8ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.118+0000 7fd8cce69640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8ac066690 0x7fd8ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.118+0000 7fd8bdffb640 1 -- 172.21.15.39:0/3115600116 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7fd8c00840c0 con 0x7fd8c80c6060 2024-09-13T17:09:32.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.118+0000 7fd8cce69640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8ac066690 0x7fd8ac068b50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80b1000 tx=0x7fd8b80a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:32.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.122+0000 7fd8bdffb640 1 -- 172.21.15.39:0/3115600116 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd8c00541a0 con 0x7fd8c80c6060 2024-09-13T17:09:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.162+0000 7fce7e31d640 1 Processor -- start 2024-09-13T17:09:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.162+0000 7fce7e31d640 1 -- start start 2024-09-13T17:09:32.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7e31d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c2690 0x7fce7806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7e31d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fce780c0be0 0x7fce7815e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7e31d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce78157330 con 0x7fce780c6290 2024-09-13T17:09:32.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7e31d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce780c4a20 con 0x7fce780c0be0 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7e31d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780c4740 con 0x7fce780c2690 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce777fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fce780c6290 0x7fce781577c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49376/0 (socket says 172.21.15.39:49376) 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7c8ba640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fce780c0be0 0x7fce7815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce777fe640 1 -- 172.21.15.39:0/3964486829 learned_addr learned my addr 172.21.15.39:0/3964486829 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce77fff640 1 --2- 172.21.15.39:0/3964486829 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c2690 0x7fce7806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7c8ba640 1 -- 172.21.15.39:0/3964486829 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c2690 msgr2=0x7fce7806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7c8ba640 1 --2- 172.21.15.39:0/3964486829 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c2690 0x7fce7806ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7c8ba640 1 -- 172.21.15.39:0/3964486829 >> v1:172.21.15.39:6789/0 conn(0x7fce780c6290 legacy=0x7fce781577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce76ffd640 1 -- 172.21.15.39:0/3964486829 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885314650 0 0) 0x7fce78157330 con 0x7fce780c6290 2024-09-13T17:09:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7c8ba640 1 -- 172.21.15.39:0/3964486829 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78154880 con 0x7fce780c0be0 2024-09-13T17:09:32.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce7c8ba640 1 --2- 172.21.15.39:0/3964486829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c0be0 0x7fce7815e630 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fce70009d90 tx=0x7fce7001d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442ec2f8a34ad1de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:32.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce76ffd640 1 -- 172.21.15.39:0/3964486829 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700a59b0 con 0x7fce780c0be0 2024-09-13T17:09:32.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce76ffd640 1 -- 172.21.15.39:0/3964486829 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce700a5b10 con 0x7fce780c0be0 2024-09-13T17:09:32.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.166+0000 7fce76ffd640 1 -- 172.21.15.39:0/3964486829 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700a5ea0 con 0x7fce780c0be0 2024-09-13T17:09:32.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- 172.21.15.39:0/3964486829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c0be0 msgr2=0x7fce7815e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 --2- 172.21.15.39:0/3964486829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c0be0 0x7fce7815e630 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fce70009d90 tx=0x7fce7001d330 comp rx=0 tx=0).stop 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- 172.21.15.39:0/3964486829 shutdown_connections 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 --2- 172.21.15.39:0/3964486829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c0be0 0x7fce7815e630 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 --2- 172.21.15.39:0/3964486829 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c2690 0x7fce7806ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- 172.21.15.39:0/3964486829 >> 172.21.15.39:0/3964486829 conn(0x7fce780c32a0 msgr2=0x7fce780c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- 172.21.15.39:0/3964486829 shutdown_connections 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- 172.21.15.39:0/3964486829 wait complete. 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 Processor -- start 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- start start 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c0be0 0x7fce781f0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce780c2690 0x7fce78156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c6290 0x7fce78070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce78155c50 con 0x7fce780c6290 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce781f0f10 con 0x7fce780c2690 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce78070660 con 0x7fce780c0be0 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7c8ba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c6290 0x7fce78070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7c8ba640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c6290 0x7fce78070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59084/0 (socket says 172.21.15.39:59084) 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7c8ba640 1 -- 172.21.15.39:0/1593624667 learned_addr learned my addr 172.21.15.39:0/1593624667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce777fe640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce780c2690 0x7fce78156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce77fff640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c0be0 0x7fce781f0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7c8ba640 1 -- 172.21.15.39:0/1593624667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c0be0 msgr2=0x7fce781f0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7c8ba640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c0be0 0x7fce781f0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7c8ba640 1 -- 172.21.15.39:0/1593624667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce780c2690 msgr2=0x7fce78156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7c8ba640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce780c2690 0x7fce78156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7c8ba640 1 -- 172.21.15.39:0/1593624667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce70009a70 con 0x7fce780c6290 2024-09-13T17:09:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7c8ba640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c6290 0x7fce78070c80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fce7001a500 tx=0x7fce70019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce74ff9640 1 -- 172.21.15.39:0/1593624667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700add90 con 0x7fce780c6290 2024-09-13T17:09:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce74ff9640 1 -- 172.21.15.39:0/1593624667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce70009410 con 0x7fce780c6290 2024-09-13T17:09:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce781555d0 con 0x7fce780c6290 2024-09-13T17:09:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce74ff9640 1 -- 172.21.15.39:0/1593624667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700b8660 con 0x7fce780c6290 2024-09-13T17:09:32.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.170+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce781f0c20 con 0x7fce780c6290 2024-09-13T17:09:32.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.174+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce780709a0 con 0x7fce780c6290 2024-09-13T17:09:32.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.174+0000 7fce74ff9640 1 -- 172.21.15.39:0/1593624667 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce700b4040 con 0x7fce780c6290 2024-09-13T17:09:32.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.174+0000 7fce74ff9640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce64066690 0x7fce64068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:32.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.174+0000 7fce74ff9640 1 -- 172.21.15.39:0/1593624667 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7fce7011fea0 con 0x7fce780c6290 2024-09-13T17:09:32.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.174+0000 7fce77fff640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce64066690 0x7fce64068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:32.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.174+0000 7fce77fff640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce64066690 0x7fce64068b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fce7815e650 tx=0x7fce68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:32.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.178+0000 7fce74ff9640 1 -- 172.21.15.39:0/1593624667 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce700ec590 con 0x7fce780c6290 2024-09-13T17:09:32.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.314+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8c8071d30 con 0x7fd8c80c6060 2024-09-13T17:09:32.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.314+0000 7fd8bdffb640 1 -- 172.21.15.39:0/3115600116 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) v1 ==== 95+0+4429 (secure 0 0 0) 0x7fd8c0018020 con 0x7fd8c80c6060 2024-09-13T17:09:32.316 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:32.316 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":149,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:32.019190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:rejoin","state_seq":53,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:32.019190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:rejoin","state_seq":11,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:32.316 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 149 2024-09-13T17:09:32.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8ac066690 msgr2=0x7fd8ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8ac066690 0x7fd8ac068b50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80b1000 tx=0x7fd8b80a9070 comp rx=0 tx=0).stop 2024-09-13T17:09:32.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 msgr2=0x7fd8c80ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 0x7fd8c80ab400 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000c480 tx=0x7fd8c000c940 comp rx=0 tx=0).stop 2024-09-13T17:09:32.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 shutdown_connections 2024-09-13T17:09:32.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8ac066690 0x7fd8ac068b50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c814a770 0x7fd8c8152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8c814ab50 0x7fd8c81530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 --2- 172.21.15.39:0/3115600116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8c80c6060 0x7fd8c80ab400 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 >> 172.21.15.39:0/3115600116 conn(0x7fd8c80c32a0 msgr2=0x7fd8c80a6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:32.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 shutdown_connections 2024-09-13T17:09:32.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.318+0000 7fd8cf8ce640 1 -- 172.21.15.39:0/3115600116 wait complete. 2024-09-13T17:09:32.334 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 149, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 149, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5963": { "addr": "172.21.15.84:6835/3923043380", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 3923043380, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 3923043380, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5963, "incarnation": 147, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:rejoin", "state_seq": 53 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:09:32.019190+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5963 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 149, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_7485": { "addr": "172.21.15.39:6835/1034701751", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 1034701751, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 1034701751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7485, "incarnation": 147, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:09:32.019190+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7485 } } } ], "standbys": [] } 2024-09-13T17:09:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.374+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce78155d90 con 0x7fce780c6290 2024-09-13T17:09:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.374+0000 7fce74ff9640 1 -- 172.21.15.39:0/1593624667 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) v1 ==== 95+0+4429 (secure 0 0 0) 0x7fce78155d90 con 0x7fce780c6290 2024-09-13T17:09:32.373 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:32.374 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":149,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:32.019190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:rejoin","state_seq":53,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:32.019190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:rejoin","state_seq":11,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 149 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.374+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce64066690 msgr2=0x7fce64068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.374+0000 7fce7e31d640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce64066690 0x7fce64068b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fce7815e650 tx=0x7fce68008040 comp rx=0 tx=0).stop 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.374+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c6290 msgr2=0x7fce78070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.374+0000 7fce7e31d640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c6290 0x7fce78070c80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fce7001a500 tx=0x7fce70019670 comp rx=0 tx=0).stop 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.378+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 shutdown_connections 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.378+0000 7fce7e31d640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce64066690 0x7fce64068b50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.378+0000 7fce7e31d640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce780c2690 0x7fce78156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.378+0000 7fce7e31d640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce780c6290 0x7fce78070c80 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.378+0000 7fce7e31d640 1 --2- 172.21.15.39:0/1593624667 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce780c0be0 0x7fce781f0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.378+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 >> 172.21.15.39:0/1593624667 conn(0x7fce780c32a0 msgr2=0x7fce7806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.378+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 shutdown_connections 2024-09-13T17:09:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:32.378+0000 7fce7e31d640 1 -- 172.21.15.39:0/1593624667 wait complete. 2024-09-13T17:09:32.393 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 149, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 149, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_5963": { "addr": "172.21.15.84:6835/3923043380", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 3923043380, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 3923043380, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5963, "incarnation": 147, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:rejoin", "state_seq": 53 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:09:32.019190+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5963 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 149, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_7485": { "addr": "172.21.15.39:6835/1034701751", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 1034701751, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 1034701751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7485, "incarnation": 147, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:09:32.019190+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7485 } } } ], "standbys": [] } 2024-09-13T17:09:34.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:34.393 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:34.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff808423640 1 Processor -- start 2024-09-13T17:09:34.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff808423640 1 -- start start 2024-09-13T17:09:34.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff808423640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff800071dd0 0x7ff800154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff808423640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8000c7800 0x7ff8001545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff808423640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8000c5a30 con 0x7ff800151500 2024-09-13T17:09:34.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff808423640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8001576a0 con 0x7ff800071dd0 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff808423640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff800157980 con 0x7ff8000c7800 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8061bf640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff800151500 0x7ff8000c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49386/0 (socket says 172.21.15.39:49386) 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8061bf640 1 -- 172.21.15.39:0/2326514697 learned_addr learned my addr 172.21.15.39:0/2326514697 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8059be640 1 --2- 172.21.15.39:0/2326514697 >> v2:172.21.15.84:3300/0 conn(0x7ff800071dd0 0x7ff800154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264062297 0 0) 0x7ff8000c5a30 con 0x7ff800151500 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ec003530 con 0x7ff800151500 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8069c0640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8000c7800 0x7ff8001545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2160447202 0 0) 0x7ff7ec003530 con 0x7ff800151500 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8000c7800 msgr2=0x7ff8001545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8000c7800 0x7ff8001545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 >> v2:172.21.15.84:3300/0 conn(0x7ff800071dd0 msgr2=0x7ff800154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 --2- 172.21.15.39:0/2326514697 >> v2:172.21.15.84:3300/0 conn(0x7ff800071dd0 0x7ff800154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800157c60 con 0x7ff800151500 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7ff7fc0016a0 con 0x7ff800151500 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 >> v1:172.21.15.39:6789/0 conn(0x7ff800151500 legacy=0x7ff8000c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ec003980 0x7ff7ec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff7ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8000c7800 0x7ff7ec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001576a0 con 0x7ff7ec0047d0 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff800157980 con 0x7ff7ec003980 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.470+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec00b990 con 0x7ff8000c7800 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8059be640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff7ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8061bf640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ec003980 0x7ff7ec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8069c0640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8000c7800 0x7ff7ec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8059be640 1 -- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8000c7800 msgr2=0x7ff7ec005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8059be640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8000c7800 0x7ff7ec005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8059be640 1 -- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ec003980 msgr2=0x7ff7ec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8059be640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ec003980 0x7ff7ec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8059be640 1 -- 172.21.15.39:0/2326514697 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec003530 con 0x7ff7ec0047d0 2024-09-13T17:09:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8061bf640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ec003980 0x7ff7ec003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:09:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8059be640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff7ec004c50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009ae0 tx=0x7ff7f001d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebf72e79b26bfea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800154dc0 con 0x7ff7ec0047d0 2024-09-13T17:09:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f00a5550 con 0x7ff7ec0047d0 2024-09-13T17:09:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.474+0000 7ff8051bd640 1 -- 172.21.15.39:0/2326514697 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00a58c0 con 0x7ff7ec0047d0 2024-09-13T17:09:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 -- 172.21.15.39:0/2326514697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 msgr2=0x7ff7ec004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 --2- 172.21.15.39:0/2326514697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff7ec004c50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009ae0 tx=0x7ff7f001d4a0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8069c0640 1 -- 172.21.15.39:0/2326514697 reap_dead start 2024-09-13T17:09:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 -- 172.21.15.39:0/2326514697 shutdown_connections 2024-09-13T17:09:34.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 -- 172.21.15.39:0/2326514697 >> 172.21.15.39:0/2326514697 conn(0x7ff8000ab530 msgr2=0x7ff8000ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:34.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 -- 172.21.15.39:0/2326514697 shutdown_connections 2024-09-13T17:09:34.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 -- 172.21.15.39:0/2326514697 wait complete. 2024-09-13T17:09:34.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 Processor -- start 2024-09-13T17:09:34.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 -- start start 2024-09-13T17:09:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff800156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8000bf360 0x7ff8001ef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff80014e010 0x7ff800148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000a9920 con 0x7ff7ec0047d0 2024-09-13T17:09:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000aaf60 con 0x7ff8000bf360 2024-09-13T17:09:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff808423640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80014d8a0 con 0x7ff80014e010 2024-09-13T17:09:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8061bf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff800156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8061bf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff800156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59114/0 (socket says 172.21.15.39:59114) 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8061bf640 1 -- 172.21.15.39:0/324057944 learned_addr learned my addr 172.21.15.39:0/324057944 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8069c0640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff80014e010 0x7ff800148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8059be640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8000bf360 0x7ff8001ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8061bf640 1 -- 172.21.15.39:0/324057944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff80014e010 msgr2=0x7ff800148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8061bf640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff80014e010 0x7ff800148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8061bf640 1 -- 172.21.15.39:0/324057944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8000bf360 msgr2=0x7ff8001ef600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8061bf640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8000bf360 0x7ff8001ef600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8061bf640 1 -- 172.21.15.39:0/324057944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8000ab200 con 0x7ff7ec0047d0 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8059be640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8000bf360 0x7ff8001ef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.478+0000 7ff8061bf640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff800156400 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e7f0 tx=0x7ff7fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff7f6ffd640 1 -- 172.21.15.39:0/324057944 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00c870 con 0x7ff7ec0047d0 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff7f6ffd640 1 -- 172.21.15.39:0/324057944 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc00c9d0 con 0x7ff7ec0047d0 2024-09-13T17:09:34.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff800156940 con 0x7ff7ec0047d0 2024-09-13T17:09:34.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff7f6ffd640 1 -- 172.21.15.39:0/324057944 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00cd40 con 0x7ff7ec0047d0 2024-09-13T17:09:34.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff800153100 con 0x7ff7ec0047d0 2024-09-13T17:09:34.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff800154070 con 0x7ff7ec0047d0 2024-09-13T17:09:34.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff7f6ffd640 1 -- 172.21.15.39:0/324057944 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff7fc01d460 con 0x7ff7ec0047d0 2024-09-13T17:09:34.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff7f6ffd640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7e4066690 0x7ff7e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff7f6ffd640 1 -- 172.21.15.39:0/324057944 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7ff7fc084e20 con 0x7ff7ec0047d0 2024-09-13T17:09:34.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff8059be640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7e4066690 0x7ff7e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.482+0000 7ff8059be640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7e4066690 0x7ff7e4068b50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00af000 tx=0x7ff7f00a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:34.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.486+0000 7ff7f6ffd640 1 -- 172.21.15.39:0/324057944 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff7fc051470 con 0x7ff7ec0047d0 2024-09-13T17:09:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.530+0000 7fbc5ab90640 1 Processor -- start 2024-09-13T17:09:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.530+0000 7fbc5ab90640 1 -- start start 2024-09-13T17:09:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc5ab90640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbc540bcfb0 0x7fbc540c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc5ab90640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc54157360 con 0x7fbc54070b40 2024-09-13T17:09:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc5ab90640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc540c80b0 con 0x7fbc540c4260 2024-09-13T17:09:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc5ab90640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc541575b0 con 0x7fbc540bcfb0 2024-09-13T17:09:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc5912d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbc540bcfb0 0x7fbc540c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc53fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbc54070b40 0x7fbc5415c6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49402/0 (socket says 172.21.15.39:49402) 2024-09-13T17:09:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc53fff640 1 -- 172.21.15.39:0/2327969072 learned_addr learned my addr 172.21.15.39:0/2327969072 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc5912d640 1 -- 172.21.15.39:0/2327969072 >> v1:172.21.15.84:6789/0 conn(0x7fbc540c4260 legacy=0x7fbc54070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc5912d640 1 -- 172.21.15.39:0/2327969072 >> v1:172.21.15.39:6789/0 conn(0x7fbc54070b40 legacy=0x7fbc5415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc5912d640 1 -- 172.21.15.39:0/2327969072 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc54157890 con 0x7fbc540bcfb0 2024-09-13T17:09:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc537fe640 1 -- 172.21.15.39:0/2327969072 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939825201 0 0) 0x7fbc54157360 con 0x7fbc54070b40 2024-09-13T17:09:34.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc5912d640 1 --2- 172.21.15.39:0/2327969072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc540bcfb0 0x7fbc540c4640 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009d90 tx=0x7fbc4c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67e981d087197c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:34.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc537fe640 1 -- 172.21.15.39:0/2327969072 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc540c7330 con 0x7fbc540bcfb0 2024-09-13T17:09:34.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc537fe640 1 -- 172.21.15.39:0/2327969072 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0a5820 con 0x7fbc540bcfb0 2024-09-13T17:09:34.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.534+0000 7fbc537fe640 1 -- 172.21.15.39:0/2327969072 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0a5bb0 con 0x7fbc540bcfb0 2024-09-13T17:09:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 -- 172.21.15.39:0/2327969072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc540bcfb0 msgr2=0x7fbc540c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 --2- 172.21.15.39:0/2327969072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc540bcfb0 0x7fbc540c4640 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009d90 tx=0x7fbc4c01c850 comp rx=0 tx=0).stop 2024-09-13T17:09:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 -- 172.21.15.39:0/2327969072 shutdown_connections 2024-09-13T17:09:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 --2- 172.21.15.39:0/2327969072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc540bcfb0 0x7fbc540c4640 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 -- 172.21.15.39:0/2327969072 >> 172.21.15.39:0/2327969072 conn(0x7fbc54069e10 msgr2=0x7fbc5406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 -- 172.21.15.39:0/2327969072 shutdown_connections 2024-09-13T17:09:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 -- 172.21.15.39:0/2327969072 wait complete. 2024-09-13T17:09:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 Processor -- start 2024-09-13T17:09:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 -- start start 2024-09-13T17:09:34.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc540bcfb0 0x7fbc541466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc540c4260 0x7fbc540aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc541ed910 0x7fbc540c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54146210 con 0x7fbc541ed910 2024-09-13T17:09:34.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540ac740 con 0x7fbc540c4260 2024-09-13T17:09:34.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5ab90640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54145bc0 con 0x7fbc540bcfb0 2024-09-13T17:09:34.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5912d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc541ed910 0x7fbc540c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5912d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc541ed910 0x7fbc540c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59138/0 (socket says 172.21.15.39:59138) 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5912d640 1 -- 172.21.15.39:0/3027891072 learned_addr learned my addr 172.21.15.39:0/3027891072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5892c640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc540bcfb0 0x7fbc541466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc53fff640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc540c4260 0x7fbc540aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5912d640 1 -- 172.21.15.39:0/3027891072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc540bcfb0 msgr2=0x7fbc541466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5912d640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc540bcfb0 0x7fbc541466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5912d640 1 -- 172.21.15.39:0/3027891072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc540c4260 msgr2=0x7fbc540aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5912d640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc540c4260 0x7fbc540aca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5912d640 1 -- 172.21.15.39:0/3027891072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c009a70 con 0x7fbc541ed910 2024-09-13T17:09:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.538+0000 7fbc5912d640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc541ed910 0x7fbc540c9980 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c01a480 tx=0x7fbc4c0195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc517fa640 1 -- 172.21.15.39:0/3027891072 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0b9860 con 0x7fbc541ed910 2024-09-13T17:09:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc517fa640 1 -- 172.21.15.39:0/3027891072 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0b99c0 con 0x7fbc541ed910 2024-09-13T17:09:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc54145ea0 con 0x7fbc541ed910 2024-09-13T17:09:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc517fa640 1 -- 172.21.15.39:0/3027891072 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0b9d70 con 0x7fbc541ed910 2024-09-13T17:09:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc54157260 con 0x7fbc541ed910 2024-09-13T17:09:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc54145a20 con 0x7fbc541ed910 2024-09-13T17:09:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc517fa640 1 -- 172.21.15.39:0/3027891072 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc4c0ad440 con 0x7fbc541ed910 2024-09-13T17:09:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc517fa640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc38066690 0x7fbc38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc517fa640 1 -- 172.21.15.39:0/3027891072 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7fbc4c1206a0 con 0x7fbc541ed910 2024-09-13T17:09:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc5892c640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc38066690 0x7fbc38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.542+0000 7fbc5892c640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc38066690 0x7fbc38068b50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbc48000f70 tx=0x7fbc48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:34.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.550+0000 7fbc517fa640 1 -- 172.21.15.39:0/3027891072 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbc4c0ece00 con 0x7fbc541ed910 2024-09-13T17:09:34.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.766+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff800071bb0 con 0x7ff7ec0047d0 2024-09-13T17:09:34.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.766+0000 7ff7f6ffd640 1 -- 172.21.15.39:0/324057944 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+4429 (secure 0 0 0) 0x7ff7fc054a90 con 0x7ff7ec0047d0 2024-09-13T17:09:34.767 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:34.767 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:34.767 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 150 2024-09-13T17:09:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7e4066690 msgr2=0x7ff7e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7e4066690 0x7ff7e4068b50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00af000 tx=0x7ff7f00a7040 comp rx=0 tx=0).stop 2024-09-13T17:09:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 msgr2=0x7ff800156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff800156400 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00e7f0 tx=0x7ff7fc00ecb0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 shutdown_connections 2024-09-13T17:09:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff7e4066690 0x7ff7e4068b50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8000bf360 0x7ff8001ef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff80014e010 0x7ff800148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 --2- 172.21.15.39:0/324057944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7ec0047d0 0x7ff800156400 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 >> 172.21.15.39:0/324057944 conn(0x7ff8000ab530 msgr2=0x7ff8000c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:34.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 shutdown_connections 2024-09-13T17:09:34.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.770+0000 7ff808423640 1 -- 172.21.15.39:0/324057944 wait complete. 2024-09-13T17:09:34.789 INFO:tasks.mds_thrash.fs.[a]:mds.d has gained rank=0, replacing gid=7338 2024-09-13T17:09:34.789 INFO:tasks.mds_thrash.fs.[a]:waiting for 8 secs before reviving mds.c 2024-09-13T17:09:34.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.814+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc54145a20 con 0x7fbc541ed910 2024-09-13T17:09:34.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.814+0000 7fbc517fa640 1 -- 172.21.15.39:0/3027891072 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+4429 (secure 0 0 0) 0x7fbc54145a20 con 0x7fbc541ed910 2024-09-13T17:09:34.814 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:34.814 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 150 2024-09-13T17:09:34.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc38066690 msgr2=0x7fbc38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc38066690 0x7fbc38068b50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbc48000f70 tx=0x7fbc48009040 comp rx=0 tx=0).stop 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc541ed910 msgr2=0x7fbc540c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc541ed910 0x7fbc540c9980 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c01a480 tx=0x7fbc4c0195f0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 shutdown_connections 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc38066690 0x7fbc38068b50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc540c4260 0x7fbc540aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc541ed910 0x7fbc540c9980 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 --2- 172.21.15.39:0/3027891072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc540bcfb0 0x7fbc541466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 >> 172.21.15.39:0/3027891072 conn(0x7fbc54069e10 msgr2=0x7fbc54150090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 shutdown_connections 2024-09-13T17:09:34.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:34.818+0000 7fbc5ab90640 1 -- 172.21.15.39:0/3027891072 wait complete. 2024-09-13T17:09:34.837 INFO:tasks.mds_thrash.fs.[b]:mds.a has gained rank=0, replacing gid=6071 2024-09-13T17:09:34.837 INFO:tasks.mds_thrash.fs.[b]:waiting for 2 secs before reviving mds.b 2024-09-13T17:09:36.838 INFO:tasks.mds_thrash.fs.[b]:reviving mds.b 2024-09-13T17:09:36.838 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:09:36.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:09:36.840 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:09:36.841 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:36.920 INFO:tasks.ceph.mds.b.smithi084.stdout:starting mds.b at 2024-09-13T17:09:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b1659a640 1 Processor -- start 2024-09-13T17:09:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b1659a640 1 -- start start 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b1659a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1b100abc50 0x7f1b1014f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b1659a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1b100ab340 0x7f1b1014f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b1659a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b10156270 con 0x7f1b1006cbc0 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b1659a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b10061530 con 0x7f1b100ab340 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b1659a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b101554b0 con 0x7f1b100abc50 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b14b37640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1b1006cbc0 0x7f1b1006d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49412/0 (socket says 172.21.15.39:49412) 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b0f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1b100ab340 0x7f1b1014f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b14b37640 1 -- 172.21.15.39:0/2477201807 learned_addr learned my addr 172.21.15.39:0/2477201807 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b0ffff640 1 --2- 172.21.15.39:0/2477201807 >> v2:172.21.15.84:3300/0 conn(0x7f1b100abc50 0x7f1b1014f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b0f7fe640 1 -- 172.21.15.39:0/2477201807 >> v2:172.21.15.84:3300/0 conn(0x7f1b100abc50 msgr2=0x7f1b1014f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b0f7fe640 1 --2- 172.21.15.39:0/2477201807 >> v2:172.21.15.84:3300/0 conn(0x7f1b100abc50 0x7f1b1014f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:37.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b0f7fe640 1 -- 172.21.15.39:0/2477201807 >> v1:172.21.15.39:6789/0 conn(0x7f1b1006cbc0 legacy=0x7f1b1006d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:37.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b0effd640 1 -- 172.21.15.39:0/2477201807 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153010897 0 0) 0x7f1b10156270 con 0x7f1b1006cbc0 2024-09-13T17:09:37.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.006+0000 7f1b0f7fe640 1 -- 172.21.15.39:0/2477201807 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10155790 con 0x7f1b100ab340 2024-09-13T17:09:37.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b0f7fe640 1 --2- 172.21.15.39:0/2477201807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b100ab340 0x7f1b1014f640 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1b100724d0 tx=0x7f1b0401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa20ff87b0504805 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:37.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b0effd640 1 -- 172.21.15.39:0/2477201807 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b040ad660 con 0x7f1b100ab340 2024-09-13T17:09:37.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b0effd640 1 -- 172.21.15.39:0/2477201807 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b040ad7c0 con 0x7f1b100ab340 2024-09-13T17:09:37.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b0effd640 1 -- 172.21.15.39:0/2477201807 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b040adb50 con 0x7f1b100ab340 2024-09-13T17:09:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b1659a640 1 -- 172.21.15.39:0/2477201807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b100ab340 msgr2=0x7f1b1014f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b1659a640 1 --2- 172.21.15.39:0/2477201807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b100ab340 0x7f1b1014f640 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1b100724d0 tx=0x7f1b0401c380 comp rx=0 tx=0).stop 2024-09-13T17:09:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b1659a640 1 -- 172.21.15.39:0/2477201807 shutdown_connections 2024-09-13T17:09:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b1659a640 1 --2- 172.21.15.39:0/2477201807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b100ab340 0x7f1b1014f640 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b1659a640 1 --2- 172.21.15.39:0/2477201807 >> v2:172.21.15.84:3300/0 conn(0x7f1b100abc50 0x7f1b1014f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b1659a640 1 -- 172.21.15.39:0/2477201807 >> 172.21.15.39:0/2477201807 conn(0x7f1b10069e10 msgr2=0x7f1b1006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b1659a640 1 -- 172.21.15.39:0/2477201807 shutdown_connections 2024-09-13T17:09:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.010+0000 7f1b1659a640 1 -- 172.21.15.39:0/2477201807 wait complete. 2024-09-13T17:09:37.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 Processor -- start 2024-09-13T17:09:37.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 -- start start 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b1006cbc0 0x7f1b1014d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b100ab340 0x7f1b101496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b100abc50 0x7f1b10154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b101ec120 con 0x7f1b1006cbc0 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100c2250 con 0x7f1b100ab340 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b101f03f0 con 0x7f1b100abc50 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b1006cbc0 0x7f1b1014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b1006cbc0 0x7f1b1014d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59158/0 (socket says 172.21.15.39:59158) 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0ffff640 1 -- 172.21.15.39:0/2643802042 learned_addr learned my addr 172.21.15.39:0/2643802042 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0f7fe640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b100ab340 0x7f1b101496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b14b37640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b100abc50 0x7f1b10154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:37.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0ffff640 1 -- 172.21.15.39:0/2643802042 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b100abc50 msgr2=0x7f1b10154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:37.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0ffff640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b100abc50 0x7f1b10154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:37.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0ffff640 1 -- 172.21.15.39:0/2643802042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b100ab340 msgr2=0x7f1b101496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:37.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0ffff640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b100ab340 0x7f1b101496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:37.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0ffff640 1 -- 172.21.15.39:0/2643802042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b04009420 con 0x7f1b1006cbc0 2024-09-13T17:09:37.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0ffff640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b1006cbc0 0x7f1b1014d450 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000e870 tx=0x7f1b0000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:37.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0cff9640 1 -- 172.21.15.39:0/2643802042 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00009510 con 0x7f1b1006cbc0 2024-09-13T17:09:37.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0cff9640 1 -- 172.21.15.39:0/2643802042 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0000eec0 con 0x7f1b1006cbc0 2024-09-13T17:09:37.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b100c2460 con 0x7f1b1006cbc0 2024-09-13T17:09:37.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b0cff9640 1 -- 172.21.15.39:0/2643802042 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000cb40 con 0x7f1b1006cbc0 2024-09-13T17:09:37.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b101479a0 con 0x7f1b1006cbc0 2024-09-13T17:09:37.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.014+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b100ac030 con 0x7f1b1006cbc0 2024-09-13T17:09:37.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.018+0000 7f1b0cff9640 1 -- 172.21.15.39:0/2643802042 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b0000cca0 con 0x7f1b1006cbc0 2024-09-13T17:09:37.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.018+0000 7f1b0cff9640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1afc066690 0x7f1afc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:37.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.018+0000 7f1b0cff9640 1 -- 172.21.15.39:0/2643802042 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7f1b00084f90 con 0x7f1b1006cbc0 2024-09-13T17:09:37.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.018+0000 7f1b0f7fe640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1afc066690 0x7f1afc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:37.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.018+0000 7f1b0f7fe640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1afc066690 0x7f1afc068b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1b040af000 tx=0x7f1b04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.022+0000 7f1b0cff9640 1 -- 172.21.15.39:0/2643802042 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b000b6db0 con 0x7f1b1006cbc0 2024-09-13T17:09:37.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.274+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b1006a530 con 0x7f1b1006cbc0 2024-09-13T17:09:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b0cff9640 1 -- 172.21.15.39:0/2643802042 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+5183 (secure 0 0 0) 0x7f1b00017090 con 0x7f1b1006cbc0 2024-09-13T17:09:37.276 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:37.276 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 151 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1afc066690 msgr2=0x7f1afc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1afc066690 0x7f1afc068b50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1b040af000 tx=0x7f1b04008040 comp rx=0 tx=0).stop 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b1006cbc0 msgr2=0x7f1b1014d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b1006cbc0 0x7f1b1014d450 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000e870 tx=0x7f1b0000ec70 comp rx=0 tx=0).stop 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 shutdown_connections 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1afc066690 0x7f1afc068b50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b100ab340 0x7f1b101496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1b100abc50 0x7f1b10154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 --2- 172.21.15.39:0/2643802042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b1006cbc0 0x7f1b1014d450 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 >> 172.21.15.39:0/2643802042 conn(0x7f1b10069e10 msgr2=0x7f1b1014e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 shutdown_connections 2024-09-13T17:09:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:37.278+0000 7f1b1659a640 1 -- 172.21.15.39:0/2643802042 wait complete. 2024-09-13T17:09:37.298 INFO:tasks.mds_thrash.fs.[b]:mds.b reported in up:standby state 2024-09-13T17:09:37.298 INFO:tasks.mds_thrash.fs.[b]:waiting for 48 secs before thrashing 2024-09-13T17:09:37.865 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T17:09:37.866+0000 7ff0f098f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:09:37.938 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T17:09:37.938+0000 7fc504ea6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:09:42.789 INFO:tasks.mds_thrash.fs.[a]:reviving mds.c 2024-09-13T17:09:42.790 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:09:42.790 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:09:42.793 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:09:42.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:42.866 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T17:09:42.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474ef37640 1 Processor -- start 2024-09-13T17:09:42.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474ef37640 1 -- start start 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474ef37640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748071dd0 0x7f4748154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474ef37640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f47480c7800 0x7f47481545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474ef37640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47480c5a30 con 0x7f4748151500 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474ef37640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47481576a0 con 0x7f47480c7800 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474ef37640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4748157980 con 0x7f4748071dd0 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474d4d4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f47480c7800 0x7f47481545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474d4d4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f47480c7800 0x7f47481545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34282/0 (socket says 172.21.15.39:34282) 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474d4d4640 1 -- 172.21.15.39:0/1707156594 learned_addr learned my addr 172.21.15.39:0/1707156594 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f4747fff640 1 --2- 172.21.15.39:0/1707156594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748071dd0 0x7f4748154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:42.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474d4d4640 1 -- 172.21.15.39:0/1707156594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748071dd0 msgr2=0x7f4748154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:42.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474d4d4640 1 --2- 172.21.15.39:0/1707156594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748071dd0 0x7f4748154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:42.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474d4d4640 1 -- 172.21.15.39:0/1707156594 >> v1:172.21.15.84:6789/0 conn(0x7f4748151500 legacy=0x7f47480c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:42.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474d4d4640 1 -- 172.21.15.39:0/1707156594 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4748157c60 con 0x7f47480c7800 2024-09-13T17:09:42.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.937+0000 7f474d4d4640 1 --2- 172.21.15.39:0/1707156594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47480c7800 0x7f47481545b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f4740009e50 tx=0x7f474001d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458a0a1dbd1552f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:42.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f47477fe640 1 -- 172.21.15.39:0/1707156594 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400a5850 con 0x7f47480c7800 2024-09-13T17:09:42.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f47477fe640 1 -- 172.21.15.39:0/1707156594 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47400a59b0 con 0x7f47480c7800 2024-09-13T17:09:42.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f47477fe640 1 -- 172.21.15.39:0/1707156594 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400a5d40 con 0x7f47480c7800 2024-09-13T17:09:42.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 -- 172.21.15.39:0/1707156594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47480c7800 msgr2=0x7f47481545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:42.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 --2- 172.21.15.39:0/1707156594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47480c7800 0x7f47481545b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f4740009e50 tx=0x7f474001d3d0 comp rx=0 tx=0).stop 2024-09-13T17:09:42.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 -- 172.21.15.39:0/1707156594 shutdown_connections 2024-09-13T17:09:42.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 --2- 172.21.15.39:0/1707156594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47480c7800 0x7f47481545b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:42.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 --2- 172.21.15.39:0/1707156594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748071dd0 0x7f4748154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:42.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 -- 172.21.15.39:0/1707156594 >> 172.21.15.39:0/1707156594 conn(0x7f47480ab530 msgr2=0x7f47480ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:42.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 -- 172.21.15.39:0/1707156594 shutdown_connections 2024-09-13T17:09:42.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 -- 172.21.15.39:0/1707156594 wait complete. 2024-09-13T17:09:42.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 Processor -- start 2024-09-13T17:09:42.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 -- start start 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4748071dd0 0x7f4748149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47480c7800 0x7f4748148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748151500 0x7f47481487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47481f0380 con 0x7f4748071dd0 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4748148ce0 con 0x7f47480c7800 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ef37640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47481ef700 con 0x7f4748151500 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ccd3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4748071dd0 0x7f4748149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ccd3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4748071dd0 0x7f4748149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34284/0 (socket says 172.21.15.39:34284) 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ccd3640 1 -- 172.21.15.39:0/3678883436 learned_addr learned my addr 172.21.15.39:0/3678883436 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474d4d4640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748151500 0x7f47481487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f4747fff640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47480c7800 0x7f4748148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:42.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ccd3640 1 -- 172.21.15.39:0/3678883436 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748151500 msgr2=0x7f47481487a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ccd3640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748151500 0x7f47481487a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ccd3640 1 -- 172.21.15.39:0/3678883436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47480c7800 msgr2=0x7f4748148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ccd3640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47480c7800 0x7f4748148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474ccd3640 1 -- 172.21.15.39:0/3678883436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740009b30 con 0x7f4748071dd0 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.941+0000 7f474d4d4640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748151500 0x7f47481487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f474ccd3640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4748071dd0 0x7f4748149620 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c3d0 tx=0x7f473c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f47457fa640 1 -- 172.21.15.39:0/3678883436 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0052d0 con 0x7f4748071dd0 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f47457fa640 1 -- 172.21.15.39:0/3678883436 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c005450 con 0x7f4748071dd0 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47481ef9e0 con 0x7f4748071dd0 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f47457fa640 1 -- 172.21.15.39:0/3678883436 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c003860 con 0x7f4748071dd0 2024-09-13T17:09:42.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481effa0 con 0x7f4748071dd0 2024-09-13T17:09:42.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47480c4280 con 0x7f4748071dd0 2024-09-13T17:09:42.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f47457fa640 1 -- 172.21.15.39:0/3678883436 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f473c0039c0 con 0x7f4748071dd0 2024-09-13T17:09:42.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f47457fa640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f472c0666c0 0x7f472c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:42.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f47457fa640 1 -- 172.21.15.39:0/3678883436 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7f473c084430 con 0x7f4748071dd0 2024-09-13T17:09:42.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f4747fff640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f472c0666c0 0x7f472c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:42.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.945+0000 7f4747fff640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f472c0666c0 0x7f472c068b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4748070eb0 tx=0x7f4734008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:42.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:42.949+0000 7f47457fa640 1 -- 172.21.15.39:0/3678883436 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f473c050a80 con 0x7f4748071dd0 2024-09-13T17:09:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.185+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4748071bb0 con 0x7f4748071dd0 2024-09-13T17:09:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.185+0000 7f47457fa640 1 -- 172.21.15.39:0/3678883436 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f473c017090 con 0x7f4748071dd0 2024-09-13T17:09:43.188 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:43.188 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":152}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:43.188 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-13T17:09:43.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f472c0666c0 msgr2=0x7f472c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:43.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f472c0666c0 0x7f472c068b80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4748070eb0 tx=0x7f4734008040 comp rx=0 tx=0).stop 2024-09-13T17:09:43.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4748071dd0 msgr2=0x7f4748149620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:43.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4748071dd0 0x7f4748149620 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c3d0 tx=0x7f473c00c890 comp rx=0 tx=0).stop 2024-09-13T17:09:43.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 shutdown_connections 2024-09-13T17:09:43.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f472c0666c0 0x7f472c068b80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:43.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47480c7800 0x7f4748148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:43.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4748151500 0x7f47481487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:43.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 --2- 172.21.15.39:0/3678883436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4748071dd0 0x7f4748149620 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:43.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 >> 172.21.15.39:0/3678883436 conn(0x7f47480ab530 msgr2=0x7f47480ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:43.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 shutdown_connections 2024-09-13T17:09:43.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:43.189+0000 7f474ef37640 1 -- 172.21.15.39:0/3678883436 wait complete. 2024-09-13T17:09:43.208 INFO:tasks.mds_thrash.fs.[a]:mds.c reported in up:standby state 2024-09-13T17:09:43.208 INFO:tasks.mds_thrash.fs.[a]:waiting for 1 secs before thrashing 2024-09-13T17:09:44.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:44.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a7180640 1 Processor -- start 2024-09-13T17:09:44.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a7180640 1 -- start start 2024-09-13T17:09:44.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a7180640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff8a0151500 0x7ff8a00c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a7180640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8a00c47c0 0x7ff8a0154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a7180640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a00c5a30 con 0x7ff8a0071dd0 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a7180640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a01576a0 con 0x7ff8a00c47c0 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a7180640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a0157980 con 0x7ff8a0151500 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a571d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8a00c47c0 0x7ff8a0154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a571d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8a00c47c0 0x7ff8a0154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34296/0 (socket says 172.21.15.39:34296) 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a571d640 1 -- 172.21.15.39:0/3508951309 learned_addr learned my addr 172.21.15.39:0/3508951309 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.337+0000 7ff8a4f1c640 1 --2- 172.21.15.39:0/3508951309 >> v2:172.21.15.84:3300/0 conn(0x7ff8a0151500 0x7ff8a00c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a571d640 1 -- 172.21.15.39:0/3508951309 >> v2:172.21.15.84:3300/0 conn(0x7ff8a0151500 msgr2=0x7ff8a00c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a571d640 1 --2- 172.21.15.39:0/3508951309 >> v2:172.21.15.84:3300/0 conn(0x7ff8a0151500 0x7ff8a00c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a571d640 1 -- 172.21.15.39:0/3508951309 >> v1:172.21.15.84:6789/0 conn(0x7ff8a0071dd0 legacy=0x7ff8a0154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a571d640 1 -- 172.21.15.39:0/3508951309 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a0157c60 con 0x7ff8a00c47c0 2024-09-13T17:09:44.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a571d640 1 --2- 172.21.15.39:0/3508951309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a00c47c0 0x7ff8a0154780 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff898009e50 tx=0x7ff89801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45653e6ede37a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8977fe640 1 -- 172.21.15.39:0/3508951309 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980a5810 con 0x7ff8a00c47c0 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8977fe640 1 -- 172.21.15.39:0/3508951309 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8980a5970 con 0x7ff8a00c47c0 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8977fe640 1 -- 172.21.15.39:0/3508951309 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980a5d00 con 0x7ff8a00c47c0 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a7180640 1 -- 172.21.15.39:0/3508951309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a00c47c0 msgr2=0x7ff8a0154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a7180640 1 --2- 172.21.15.39:0/3508951309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a00c47c0 0x7ff8a0154780 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff898009e50 tx=0x7ff89801c930 comp rx=0 tx=0).stop 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a7180640 1 -- 172.21.15.39:0/3508951309 shutdown_connections 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a7180640 1 --2- 172.21.15.39:0/3508951309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a00c47c0 0x7ff8a0154780 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a7180640 1 --2- 172.21.15.39:0/3508951309 >> v2:172.21.15.84:3300/0 conn(0x7ff8a0151500 0x7ff8a00c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a7180640 1 -- 172.21.15.39:0/3508951309 >> 172.21.15.39:0/3508951309 conn(0x7ff8a00ab530 msgr2=0x7ff8a00ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a7180640 1 -- 172.21.15.39:0/3508951309 shutdown_connections 2024-09-13T17:09:44.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.341+0000 7ff8a7180640 1 -- 172.21.15.39:0/3508951309 wait complete. 2024-09-13T17:09:44.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 Processor -- start 2024-09-13T17:09:44.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 -- start start 2024-09-13T17:09:44.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8a0071dd0 0x7ff8a014c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a00c47c0 0x7ff8a01469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a0151500 0x7ff8a0146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00c1830 con 0x7ff8a0151500 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01f47a0 con 0x7ff8a00c47c0 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00c3310 con 0x7ff8a0071dd0 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a571d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a0151500 0x7ff8a0146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a571d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a0151500 0x7ff8a0146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34300/0 (socket says 172.21.15.39:34300) 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a571d640 1 -- 172.21.15.39:0/3545503319 learned_addr learned my addr 172.21.15.39:0/3545503319 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff897fff640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a00c47c0 0x7ff8a01469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a4f1c640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8a0071dd0 0x7ff8a014c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a571d640 1 -- 172.21.15.39:0/3545503319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8a0071dd0 msgr2=0x7ff8a014c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a571d640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8a0071dd0 0x7ff8a014c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a571d640 1 -- 172.21.15.39:0/3545503319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a00c47c0 msgr2=0x7ff8a01469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a571d640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a00c47c0 0x7ff8a01469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a571d640 1 -- 172.21.15.39:0/3545503319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff898009b30 con 0x7ff8a0151500 2024-09-13T17:09:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a571d640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a0151500 0x7ff8a0146f30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff898019540 tx=0x7ff8980196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8957fa640 1 -- 172.21.15.39:0/3545503319 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980b8670 con 0x7ff8a0151500 2024-09-13T17:09:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8957fa640 1 -- 172.21.15.39:0/3545503319 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8980b87d0 con 0x7ff8a0151500 2024-09-13T17:09:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a00c35f0 con 0x7ff8a0151500 2024-09-13T17:09:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8957fa640 1 -- 172.21.15.39:0/3545503319 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980b8b80 con 0x7ff8a0151500 2024-09-13T17:09:44.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a014c770 con 0x7ff8a0151500 2024-09-13T17:09:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.345+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a00c4280 con 0x7ff8a0151500 2024-09-13T17:09:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.349+0000 7ff8957fa640 1 -- 172.21.15.39:0/3545503319 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8980b8ce0 con 0x7ff8a0151500 2024-09-13T17:09:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.349+0000 7ff8957fa640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff88c0666c0 0x7ff88c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.349+0000 7ff8957fa640 1 -- 172.21.15.39:0/3545503319 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7ff898120550 con 0x7ff8a0151500 2024-09-13T17:09:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.349+0000 7ff8a4f1c640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff88c0666c0 0x7ff88c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.349+0000 7ff8a4f1c640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff88c0666c0 0x7ff88c068b80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009a50 tx=0x7ff89c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.353+0000 7ff8957fa640 1 -- 172.21.15.39:0/3545503319 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff8980ecd80 con 0x7ff8a0151500 2024-09-13T17:09:44.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.569+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8a0071bb0 con 0x7ff8a0151500 2024-09-13T17:09:44.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8957fa640 1 -- 172.21.15.39:0/3545503319 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5941 (secure 0 0 0) 0x7ff8980bc090 con 0x7ff8a0151500 2024-09-13T17:09:44.572 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:44.572 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":152}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-13T17:09:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff88c0666c0 msgr2=0x7ff88c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff88c0666c0 0x7ff88c068b80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009a50 tx=0x7ff89c008040 comp rx=0 tx=0).stop 2024-09-13T17:09:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a0151500 msgr2=0x7ff8a0146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a0151500 0x7ff8a0146f30 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff898019540 tx=0x7ff8980196d0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 shutdown_connections 2024-09-13T17:09:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff88c0666c0 0x7ff88c068b80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8a00c47c0 0x7ff8a01469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a0151500 0x7ff8a0146f30 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 --2- 172.21.15.39:0/3545503319 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff8a0071dd0 0x7ff8a014c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 >> 172.21.15.39:0/3545503319 conn(0x7ff8a00ab530 msgr2=0x7ff8a014d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 shutdown_connections 2024-09-13T17:09:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.573+0000 7ff8a7180640 1 -- 172.21.15.39:0/3545503319 wait complete. 2024-09-13T17:09:44.591 INFO:tasks.mds_thrash.fs.[a]:kill mds.d (rank=0) 2024-09-13T17:09:44.591 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T17:09:44.592 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:09:44.619 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T17:09:44.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:44.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.725+0000 7fd51a675640 1 Processor -- start 2024-09-13T17:09:44.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.725+0000 7fd51a675640 1 -- start start 2024-09-13T17:09:44.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 0x7fd5140c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd5140c2ef0 0x7fd5140c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51406ddf0 con 0x7fd514065ee0 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5140c53d0 con 0x7fd5140c2ef0 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5140c5950 con 0x7fd5140c0a90 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd518c12640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd5140c2ef0 0x7fd5140c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd518c12640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd5140c2ef0 0x7fd5140c22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34314/0 (socket says 172.21.15.39:34314) 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd518c12640 1 -- 172.21.15.39:0/1183868500 learned_addr learned my addr 172.21.15.39:0/1183868500 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd50bfff640 1 --2- 172.21.15.39:0/1183868500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 0x7fd5140c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd518c12640 1 -- 172.21.15.39:0/1183868500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 msgr2=0x7fd5140c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd518c12640 1 --2- 172.21.15.39:0/1183868500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 0x7fd5140c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd518c12640 1 -- 172.21.15.39:0/1183868500 >> v1:172.21.15.84:6789/0 conn(0x7fd514065ee0 legacy=0x7fd51414f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:44.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd518c12640 1 -- 172.21.15.39:0/1183868500 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5140c2820 con 0x7fd5140c2ef0 2024-09-13T17:09:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd518c12640 1 --2- 172.21.15.39:0/1183868500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5140c2ef0 0x7fd5140c22e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009e50 tx=0x7fd50c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0291ad3f4d6634 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:44.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd5137fe640 1 -- 172.21.15.39:0/1183868500 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c0a5840 con 0x7fd5140c2ef0 2024-09-13T17:09:44.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd5137fe640 1 -- 172.21.15.39:0/1183868500 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c0a59a0 con 0x7fd5140c2ef0 2024-09-13T17:09:44.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd5137fe640 1 -- 172.21.15.39:0/1183868500 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c0a5d30 con 0x7fd5140c2ef0 2024-09-13T17:09:44.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 -- 172.21.15.39:0/1183868500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5140c2ef0 msgr2=0x7fd5140c22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 --2- 172.21.15.39:0/1183868500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5140c2ef0 0x7fd5140c22e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009e50 tx=0x7fd50c01c9c0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 -- 172.21.15.39:0/1183868500 shutdown_connections 2024-09-13T17:09:44.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 --2- 172.21.15.39:0/1183868500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5140c2ef0 0x7fd5140c22e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 --2- 172.21.15.39:0/1183868500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 0x7fd5140c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 -- 172.21.15.39:0/1183868500 >> 172.21.15.39:0/1183868500 conn(0x7fd5140c3310 msgr2=0x7fd5140c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:44.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 -- 172.21.15.39:0/1183868500 shutdown_connections 2024-09-13T17:09:44.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.729+0000 7fd51a675640 1 -- 172.21.15.39:0/1183868500 wait complete. 2024-09-13T17:09:44.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 Processor -- start 2024-09-13T17:09:44.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 -- start start 2024-09-13T17:09:44.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd514065ee0 0x7fd5141ec430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 0x7fd5140a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5140c2ef0 0x7fd514152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd514144410 con 0x7fd514065ee0 2024-09-13T17:09:44.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd514143520 con 0x7fd5140c2ef0 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd514143800 con 0x7fd5140c0a90 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd513fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd514065ee0 0x7fd5141ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd513fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd514065ee0 0x7fd5141ec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34328/0 (socket says 172.21.15.39:34328) 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd513fff640 1 -- 172.21.15.39:0/3438173048 learned_addr learned my addr 172.21.15.39:0/3438173048 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd50bfff640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 0x7fd5140a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd518c12640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5140c2ef0 0x7fd514152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd513fff640 1 -- 172.21.15.39:0/3438173048 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 msgr2=0x7fd5140a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd513fff640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 0x7fd5140a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd513fff640 1 -- 172.21.15.39:0/3438173048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5140c2ef0 msgr2=0x7fd514152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd513fff640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5140c2ef0 0x7fd514152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd513fff640 1 -- 172.21.15.39:0/3438173048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c009b30 con 0x7fd514065ee0 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd513fff640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd514065ee0 0x7fd5141ec430 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd50400b410 tx=0x7fd50400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd5117fa640 1 -- 172.21.15.39:0/3438173048 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504004020 con 0x7fd514065ee0 2024-09-13T17:09:44.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd5117fa640 1 -- 172.21.15.39:0/3438173048 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd504004180 con 0x7fd514065ee0 2024-09-13T17:09:44.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd514143ae0 con 0x7fd514065ee0 2024-09-13T17:09:44.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd5117fa640 1 -- 172.21.15.39:0/3438173048 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504011630 con 0x7fd514065ee0 2024-09-13T17:09:44.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.733+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5141440a0 con 0x7fd514065ee0 2024-09-13T17:09:44.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.737+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5140a9de0 con 0x7fd514065ee0 2024-09-13T17:09:44.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.737+0000 7fd5117fa640 1 -- 172.21.15.39:0/3438173048 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd504011790 con 0x7fd514065ee0 2024-09-13T17:09:44.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.737+0000 7fd5117fa640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4f40666c0 0x7fd4f4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:44.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.737+0000 7fd5117fa640 1 -- 172.21.15.39:0/3438173048 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7fd504085350 con 0x7fd514065ee0 2024-09-13T17:09:44.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.737+0000 7fd50bfff640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4f40666c0 0x7fd4f4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:44.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.737+0000 7fd50bfff640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4f40666c0 0x7fd4f4068b80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd514152e50 tx=0x7fd4fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:44.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.741+0000 7fd5117fa640 1 -- 172.21.15.39:0/3438173048 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd504051a20 con 0x7fd514065ee0 2024-09-13T17:09:44.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.957+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5140c5e70 con 0x7fd514065ee0 2024-09-13T17:09:44.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd5117fa640 1 -- 172.21.15.39:0/3438173048 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5941 (secure 0 0 0) 0x7fd504015020 con 0x7fd514065ee0 2024-09-13T17:09:44.960 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:44.960 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":152}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:44.961 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-13T17:09:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4f40666c0 msgr2=0x7fd4f4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4f40666c0 0x7fd4f4068b80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd514152e50 tx=0x7fd4fc008040 comp rx=0 tx=0).stop 2024-09-13T17:09:44.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd514065ee0 msgr2=0x7fd5141ec430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd514065ee0 0x7fd5141ec430 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd50400b410 tx=0x7fd50400b8d0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 shutdown_connections 2024-09-13T17:09:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4f40666c0 0x7fd4f4068b80 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5140c2ef0 0x7fd514152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5140c0a90 0x7fd5140a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 --2- 172.21.15.39:0/3438173048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd514065ee0 0x7fd5141ec430 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 >> 172.21.15.39:0/3438173048 conn(0x7fd5140c3310 msgr2=0x7fd51414fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 shutdown_connections 2024-09-13T17:09:44.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:44.961+0000 7fd51a675640 1 -- 172.21.15.39:0/3438173048 wait complete. 2024-09-13T17:09:44.978 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:09:46.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:47.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.113+0000 7f91e6035640 1 Processor -- start 2024-09-13T17:09:47.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.113+0000 7f91e6035640 1 -- start start 2024-09-13T17:09:47.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91e6035640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e0150f80 0x7f91e0153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:47.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91e6035640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f91e0153e70 0x7f91e00c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91e6035640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e01546c0 con 0x7f91e00c5330 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91e6035640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e00c19a0 con 0x7f91e0153e70 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91e6035640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00c49f0 con 0x7f91e0150f80 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91deffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f91e00c5330 0x7f91e006c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48846/0 (socket says 172.21.15.39:48846) 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91deffd640 1 -- 172.21.15.39:0/2807365106 learned_addr learned my addr 172.21.15.39:0/2807365106 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91df7fe640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e0150f80 0x7f91e0153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91dffff640 1 --2- 172.21.15.39:0/2807365106 >> v2:172.21.15.84:3300/0 conn(0x7f91e0153e70 0x7f91e00c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076789696 0 0) 0x7f91e01546c0 con 0x7f91e00c5330 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c8004640 con 0x7f91e00c5330 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3923985722 0 0) 0x7f91c8004640 con 0x7f91e00c5330 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e0150f80 msgr2=0x7f91e0153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e0150f80 0x7f91e0153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 >> v2:172.21.15.84:3300/0 conn(0x7f91e0153e70 msgr2=0x7f91e00c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:47.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 --2- 172.21.15.39:0/2807365106 >> v2:172.21.15.84:3300/0 conn(0x7f91e0153e70 0x7f91e00c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e01543e0 con 0x7f91e00c5330 2024-09-13T17:09:47.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f91d4001a00 con 0x7f91e00c5330 2024-09-13T17:09:47.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 >> v1:172.21.15.39:6789/0 conn(0x7f91e00c5330 legacy=0x7f91e006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:47.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91c8004a90 0x7f91c8004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:47.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e0150f80 0x7f91c80058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:47.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c8005e20 0x7f91c80062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:47.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00c19a0 con 0x7f91c8004a90 2024-09-13T17:09:47.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00c49f0 con 0x7f91c8005e20 2024-09-13T17:09:47.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c800caa0 con 0x7f91e0150f80 2024-09-13T17:09:47.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91df7fe640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91c8004a90 0x7f91c8004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:47.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91df7fe640 1 -- 172.21.15.39:0/2807365106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e0150f80 msgr2=0x7f91c80058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:47.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91df7fe640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e0150f80 0x7f91c80058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91df7fe640 1 -- 172.21.15.39:0/2807365106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c8005e20 msgr2=0x7f91c80062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:47.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91df7fe640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c8005e20 0x7f91c80062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.117+0000 7f91df7fe640 1 -- 172.21.15.39:0/2807365106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8004640 con 0x7f91c8004a90 2024-09-13T17:09:47.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91df7fe640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91c8004a90 0x7f91c8004eb0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009ae0 tx=0x7f91d001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eca956731246b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:47.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00a5560 con 0x7f91c8004a90 2024-09-13T17:09:47.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00a56c0 con 0x7f91c8004a90 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91de7fc640 1 -- 172.21.15.39:0/2807365106 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00a5a70 con 0x7f91c8004a90 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 -- 172.21.15.39:0/2807365106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91c8004a90 msgr2=0x7f91c8004eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 --2- 172.21.15.39:0/2807365106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91c8004a90 0x7f91c8004eb0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009ae0 tx=0x7f91d001ca80 comp rx=0 tx=0).stop 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91dffff640 1 -- 172.21.15.39:0/2807365106 reap_dead start 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 -- 172.21.15.39:0/2807365106 shutdown_connections 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 -- 172.21.15.39:0/2807365106 >> 172.21.15.39:0/2807365106 conn(0x7f91e0069e10 msgr2=0x7f91e006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 -- 172.21.15.39:0/2807365106 shutdown_connections 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 -- 172.21.15.39:0/2807365106 wait complete. 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 Processor -- start 2024-09-13T17:09:47.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 -- start start 2024-09-13T17:09:47.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c8004a90 0x7f91e01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:47.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e01efcc0 0x7f91e01ecd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:47.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e00721e0 0x7f91e01f0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:47.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00a9dc0 con 0x7f91e00721e0 2024-09-13T17:09:47.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0071ae0 con 0x7f91c8004a90 2024-09-13T17:09:47.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91e6035640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00aa590 con 0x7f91e01efcc0 2024-09-13T17:09:47.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91dffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e00721e0 0x7f91e01f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91dffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e00721e0 0x7f91e01f0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34338/0 (socket says 172.21.15.39:34338) 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91dffff640 1 -- 172.21.15.39:0/271796412 learned_addr learned my addr 172.21.15.39:0/271796412 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91df7fe640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c8004a90 0x7f91e01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91deffd640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e01efcc0 0x7f91e01ecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91dffff640 1 -- 172.21.15.39:0/271796412 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e01efcc0 msgr2=0x7f91e01ecd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91dffff640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e01efcc0 0x7f91e01ecd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91dffff640 1 -- 172.21.15.39:0/271796412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c8004a90 msgr2=0x7f91e01593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91dffff640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c8004a90 0x7f91e01593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.121+0000 7f91dffff640 1 -- 172.21.15.39:0/271796412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00a9fd0 con 0x7f91e00721e0 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91dffff640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e00721e0 0x7f91e01f0340 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f91e006ac40 tx=0x7f91d800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91c7fff640 1 -- 172.21.15.39:0/271796412 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d800c680 con 0x7f91e00721e0 2024-09-13T17:09:47.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91c7fff640 1 -- 172.21.15.39:0/271796412 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d800c800 con 0x7f91e00721e0 2024-09-13T17:09:47.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e00aa2b0 con 0x7f91e00721e0 2024-09-13T17:09:47.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91c7fff640 1 -- 172.21.15.39:0/271796412 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d800cbb0 con 0x7f91e00721e0 2024-09-13T17:09:47.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e0071d50 con 0x7f91e00721e0 2024-09-13T17:09:47.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e01ec6d0 con 0x7f91e00721e0 2024-09-13T17:09:47.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91c7fff640 1 -- 172.21.15.39:0/271796412 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f91d800cd10 con 0x7f91e00721e0 2024-09-13T17:09:47.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91c7fff640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f91cc066690 0x7f91cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:47.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91c7fff640 1 -- 172.21.15.39:0/271796412 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7f91d80847d0 con 0x7f91e00721e0 2024-09-13T17:09:47.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91df7fe640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f91cc066690 0x7f91cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:47.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.125+0000 7f91df7fe640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f91cc066690 0x7f91cc068b50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f91d00af000 tx=0x7f91d00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:47.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.129+0000 7f91c7fff640 1 -- 172.21.15.39:0/271796412 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f91e01ec6d0 con 0x7f91e00721e0 2024-09-13T17:09:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.329+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91e01ecab0 con 0x7f91e00721e0 2024-09-13T17:09:47.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91c7fff640 1 -- 172.21.15.39:0/271796412 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f91e01ecab0 con 0x7f91e00721e0 2024-09-13T17:09:47.332 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:47.332 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":152}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:47.333 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-13T17:09:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f91cc066690 msgr2=0x7f91cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f91cc066690 0x7f91cc068b50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f91d00af000 tx=0x7f91d00046d0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e00721e0 msgr2=0x7f91e01f0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e00721e0 0x7f91e01f0340 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f91e006ac40 tx=0x7f91d800ec00 comp rx=0 tx=0).stop 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 shutdown_connections 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f91cc066690 0x7f91cc068b50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e00721e0 0x7f91e01f0340 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f91e01efcc0 0x7f91e01ecd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 --2- 172.21.15.39:0/271796412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91c8004a90 0x7f91e01593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 >> 172.21.15.39:0/271796412 conn(0x7f91e0069e10 msgr2=0x7f91e006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 shutdown_connections 2024-09-13T17:09:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:47.333+0000 7f91e6035640 1 -- 172.21.15.39:0/271796412 wait complete. 2024-09-13T17:09:47.351 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:09:49.352 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.485+0000 7f5309356640 1 Processor -- start 2024-09-13T17:09:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.485+0000 7f5309356640 1 -- start start 2024-09-13T17:09:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5309356640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040c6290 0x7f53041577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5309356640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5304157330 con 0x7f53040c2690 2024-09-13T17:09:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5309356640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53040c4740 con 0x7f53040c0be0 2024-09-13T17:09:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5309356640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53040c4a20 con 0x7f53040c6290 2024-09-13T17:09:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5302ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f53040c2690 0x7f530406ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48862/0 (socket says 172.21.15.39:48862) 2024-09-13T17:09:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5302ffd640 1 -- 172.21.15.39:0/115892025 learned_addr learned my addr 172.21.15.39:0/115892025 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f53027fc640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040c6290 0x7f53041577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408011423 0 0) 0x7f5304157330 con 0x7f53040c2690 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f0003530 con 0x7f53040c2690 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2483677060 0 0) 0x7f52f0003530 con 0x7f53040c2690 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040c6290 msgr2=0x7f53041577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040c6290 0x7f53041577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 >> v1:172.21.15.84:6789/0 conn(0x7f53040c0be0 legacy=0x7f530415e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5304154880 con 0x7f53040c2690 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f52fc001680 con 0x7f53040c2690 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 >> v1:172.21.15.39:6789/0 conn(0x7f53040c2690 legacy=0x7f530406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f00039b0 0x7f52f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52f00047d0 0x7f52f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040c6290 0x7f52f00051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53040c4a20 con 0x7f52f00047d0 2024-09-13T17:09:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f000bb20 con 0x7f52f00039b0 2024-09-13T17:09:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f000bce0 con 0x7f53040c6290 2024-09-13T17:09:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f53027fc640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52f00047d0 0x7f52f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f5302ffd640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f00039b0 0x7f52f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f53027fc640 1 -- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040c6290 msgr2=0x7f52f00051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f53027fc640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040c6290 0x7f52f00051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f53027fc640 1 -- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f00039b0 msgr2=0x7f52f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f53027fc640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f00039b0 0x7f52f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.489+0000 7f53027fc640 1 -- 172.21.15.39:0/115892025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0003530 con 0x7f52f00047d0 2024-09-13T17:09:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52f00047d0 0x7f52f0004c50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f5304070340 tx=0x7f52ec01d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c188a6e2df0fb1a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0a5c00 con 0x7f52f00047d0 2024-09-13T17:09:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec00f040 con 0x7f52f00047d0 2024-09-13T17:09:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5301ffb640 1 -- 172.21.15.39:0/115892025 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0ad3f0 con 0x7f52f00047d0 2024-09-13T17:09:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- 172.21.15.39:0/115892025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52f00047d0 msgr2=0x7f52f0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 --2- 172.21.15.39:0/115892025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52f00047d0 0x7f52f0004c50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f5304070340 tx=0x7f52ec01d650 comp rx=0 tx=0).stop 2024-09-13T17:09:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53037fe640 1 -- 172.21.15.39:0/115892025 reap_dead start 2024-09-13T17:09:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- 172.21.15.39:0/115892025 shutdown_connections 2024-09-13T17:09:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- 172.21.15.39:0/115892025 >> 172.21.15.39:0/115892025 conn(0x7f53040c32a0 msgr2=0x7f53040c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- 172.21.15.39:0/115892025 shutdown_connections 2024-09-13T17:09:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- 172.21.15.39:0/115892025 wait complete. 2024-09-13T17:09:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 Processor -- start 2024-09-13T17:09:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- start start 2024-09-13T17:09:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f00047d0 0x7f530414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53040c1cb0 0x7f5304070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040a80e0 0x7f53040711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530414d2b0 con 0x7f53040c1cb0 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530414d6d0 con 0x7f52f00047d0 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53041e31f0 con 0x7f53040a80e0 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53040c1cb0 0x7f5304070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53040c1cb0 0x7f5304070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34346/0 (socket says 172.21.15.39:34346) 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 -- 172.21.15.39:0/2960841379 learned_addr learned my addr 172.21.15.39:0/2960841379 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5302ffd640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f00047d0 0x7f530414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53037fe640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040a80e0 0x7f53040711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 -- 172.21.15.39:0/2960841379 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040a80e0 msgr2=0x7f53040711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040a80e0 0x7f53040711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 -- 172.21.15.39:0/2960841379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f00047d0 msgr2=0x7f530414dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f00047d0 0x7f530414dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 -- 172.21.15.39:0/2960841379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5304066400 con 0x7f53040c1cb0 2024-09-13T17:09:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f53027fc640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53040c1cb0 0x7f5304070c80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0af040 tx=0x7f52ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f52fb7fe640 1 -- 172.21.15.39:0/2960841379 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0b6740 con 0x7f53040c1cb0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f52fb7fe640 1 -- 172.21.15.39:0/2960841379 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec0b68a0 con 0x7f53040c1cb0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53041e2f10 con 0x7f53040c1cb0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f52fb7fe640 1 -- 172.21.15.39:0/2960841379 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0b6c30 con 0x7f53040c1cb0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.493+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530414ced0 con 0x7f53040c1cb0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.497+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530414d520 con 0x7f53040c1cb0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.497+0000 7f52fb7fe640 1 -- 172.21.15.39:0/2960841379 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52ec0b6d90 con 0x7f53040c1cb0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.497+0000 7f52fb7fe640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f52e4066690 0x7f52e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.497+0000 7f5302ffd640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f52e4066690 0x7f52e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.497+0000 7f52fb7fe640 1 -- 172.21.15.39:0/2960841379 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7f52ec120600 con 0x7f53040c1cb0 2024-09-13T17:09:49.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.497+0000 7f5302ffd640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f52e4066690 0x7f52e4068b50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52fc001060 tx=0x7f52fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.501+0000 7f52fb7fe640 1 -- 172.21.15.39:0/2960841379 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f52ec0ecc50 con 0x7f53040c1cb0 2024-09-13T17:09:49.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.697+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f530406ae20 con 0x7f53040c1cb0 2024-09-13T17:09:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.697+0000 7f52fb7fe640 1 -- 172.21.15.39:0/2960841379 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f52ec0f0270 con 0x7f53040c1cb0 2024-09-13T17:09:49.698 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:49.698 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":152}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-13T17:09:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f52e4066690 msgr2=0x7f52e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f52e4066690 0x7f52e4068b50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52fc001060 tx=0x7f52fc009040 comp rx=0 tx=0).stop 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53040c1cb0 msgr2=0x7f5304070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53040c1cb0 0x7f5304070c80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f52ec0af040 tx=0x7f52ec009750 comp rx=0 tx=0).stop 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 shutdown_connections 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f52e4066690 0x7f52e4068b50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f53040a80e0 0x7f53040711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53040c1cb0 0x7f5304070c80 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 --2- 172.21.15.39:0/2960841379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52f00047d0 0x7f530414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 >> 172.21.15.39:0/2960841379 conn(0x7f53040c32a0 msgr2=0x7f5304065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 shutdown_connections 2024-09-13T17:09:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:49.701+0000 7f5309356640 1 -- 172.21.15.39:0/2960841379 wait complete. 2024-09-13T17:09:49.718 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:09:51.719 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:51.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfaac9a640 1 Processor -- start 2024-09-13T17:09:51.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfaac9a640 1 -- start start 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfaac9a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbfa414e0c0 0x7fbfa40a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfaac9a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbfa40c1f50 0x7fbfa406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfaac9a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa40c2780 con 0x7fbfa406d300 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfaac9a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa40c2a60 con 0x7fbfa40c1f50 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfaac9a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa40c2d40 con 0x7fbfa414e0c0 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa9237640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbfa406d300 0x7fbfa40c4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56132/0 (socket says 172.21.15.39:56132) 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa9237640 1 -- 172.21.15.39:0/1753784906 learned_addr learned my addr 172.21.15.39:0/1753784906 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa3fff640 1 --2- 172.21.15.39:0/1753784906 >> v2:172.21.15.39:3300/0 conn(0x7fbfa40c1f50 0x7fbfa406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa8a36640 1 --2- 172.21.15.39:0/1753784906 >> v2:172.21.15.84:3300/0 conn(0x7fbfa414e0c0 0x7fbfa40a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa37fe640 1 -- 172.21.15.39:0/1753784906 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009976561 0 0) 0x7fbfa40c2780 con 0x7fbfa406d300 2024-09-13T17:09:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa37fe640 1 -- 172.21.15.39:0/1753784906 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf8c003530 con 0x7fbfa406d300 2024-09-13T17:09:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa3fff640 1 -- 172.21.15.39:0/1753784906 >> v2:172.21.15.84:3300/0 conn(0x7fbfa414e0c0 msgr2=0x7fbfa40a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa3fff640 1 --2- 172.21.15.39:0/1753784906 >> v2:172.21.15.84:3300/0 conn(0x7fbfa414e0c0 0x7fbfa40a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa3fff640 1 -- 172.21.15.39:0/1753784906 >> v1:172.21.15.39:6789/0 conn(0x7fbfa406d300 legacy=0x7fbfa40c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.853+0000 7fbfa3fff640 1 -- 172.21.15.39:0/1753784906 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40c3020 con 0x7fbfa40c1f50 2024-09-13T17:09:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa3fff640 1 --2- 172.21.15.39:0/1753784906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40c1f50 0x7fbfa406cdc0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40c3370 tx=0x7fbf9801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da82c97b96ec2279 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa37fe640 1 -- 172.21.15.39:0/1753784906 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf980ab600 con 0x7fbfa40c1f50 2024-09-13T17:09:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa37fe640 1 -- 172.21.15.39:0/1753784906 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf980a7050 con 0x7fbfa40c1f50 2024-09-13T17:09:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa37fe640 1 -- 172.21.15.39:0/1753784906 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf980ab980 con 0x7fbfa40c1f50 2024-09-13T17:09:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 -- 172.21.15.39:0/1753784906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40c1f50 msgr2=0x7fbfa406cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 --2- 172.21.15.39:0/1753784906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40c1f50 0x7fbfa406cdc0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40c3370 tx=0x7fbf9801c3c0 comp rx=0 tx=0).stop 2024-09-13T17:09:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 -- 172.21.15.39:0/1753784906 shutdown_connections 2024-09-13T17:09:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 --2- 172.21.15.39:0/1753784906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa40c1f50 0x7fbfa406cdc0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 --2- 172.21.15.39:0/1753784906 >> v2:172.21.15.84:3300/0 conn(0x7fbfa414e0c0 0x7fbfa40a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 -- 172.21.15.39:0/1753784906 >> 172.21.15.39:0/1753784906 conn(0x7fbfa4069e10 msgr2=0x7fbfa406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 -- 172.21.15.39:0/1753784906 shutdown_connections 2024-09-13T17:09:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 -- 172.21.15.39:0/1753784906 wait complete. 2024-09-13T17:09:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 Processor -- start 2024-09-13T17:09:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 -- start start 2024-09-13T17:09:51.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa406d300 0x7fbfa41596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa40c1f50 0x7fbfa415c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfa414e0c0 0x7fbfa415ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa414d760 con 0x7fbfa406d300 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa414d480 con 0x7fbfa40c1f50 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfaac9a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa413fe30 con 0x7fbfa414e0c0 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa8a36640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa406d300 0x7fbfa41596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa8a36640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa406d300 0x7fbfa41596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35370/0 (socket says 172.21.15.39:35370) 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa8a36640 1 -- 172.21.15.39:0/3012625023 learned_addr learned my addr 172.21.15.39:0/3012625023 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa9237640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfa414e0c0 0x7fbfa415ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa3fff640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa40c1f50 0x7fbfa415c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa8a36640 1 -- 172.21.15.39:0/3012625023 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfa414e0c0 msgr2=0x7fbfa415ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa8a36640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfa414e0c0 0x7fbfa415ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa8a36640 1 -- 172.21.15.39:0/3012625023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa40c1f50 msgr2=0x7fbfa415c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:51.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa8a36640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa40c1f50 0x7fbfa415c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa8a36640 1 -- 172.21.15.39:0/3012625023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf98009420 con 0x7fbfa406d300 2024-09-13T17:09:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa3fff640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa40c1f50 0x7fbfa415c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.857+0000 7fbfa8a36640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa406d300 0x7fbfa41596f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400e870 tx=0x7fbf9400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfa17fa640 1 -- 172.21.15.39:0/3012625023 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94009510 con 0x7fbfa406d300 2024-09-13T17:09:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfa17fa640 1 -- 172.21.15.39:0/3012625023 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9400eec0 con 0x7fbfa406d300 2024-09-13T17:09:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa4140110 con 0x7fbfa406d300 2024-09-13T17:09:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfa17fa640 1 -- 172.21.15.39:0/3012625023 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400cb40 con 0x7fbfa406d300 2024-09-13T17:09:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa4159c30 con 0x7fbfa406d300 2024-09-13T17:09:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa415be00 con 0x7fbfa406d300 2024-09-13T17:09:51.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfa17fa640 1 -- 172.21.15.39:0/3012625023 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf9400cca0 con 0x7fbfa406d300 2024-09-13T17:09:51.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfa17fa640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf90066690 0x7fbf90068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:51.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfa17fa640 1 -- 172.21.15.39:0/3012625023 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7fbf94084ff0 con 0x7fbfa406d300 2024-09-13T17:09:51.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfa3fff640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf90066690 0x7fbf90068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:51.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.861+0000 7fbfa3fff640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf90066690 0x7fbf90068b50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbf980ad000 tx=0x7fbf98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:51.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:51.865+0000 7fbfa17fa640 1 -- 172.21.15.39:0/3012625023 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf940b6db0 con 0x7fbfa406d300 2024-09-13T17:09:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.065+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfa406a440 con 0x7fbfa406d300 2024-09-13T17:09:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.065+0000 7fbfa17fa640 1 -- 172.21.15.39:0/3012625023 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5941 (secure 0 0 0) 0x7fbf94021090 con 0x7fbfa406d300 2024-09-13T17:09:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:52.066 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":152}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf90066690 msgr2=0x7fbf90068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf90066690 0x7fbf90068b50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbf980ad000 tx=0x7fbf98008040 comp rx=0 tx=0).stop 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa406d300 msgr2=0x7fbfa41596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa406d300 0x7fbfa41596f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400e870 tx=0x7fbf9400ec70 comp rx=0 tx=0).stop 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 shutdown_connections 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf90066690 0x7fbf90068b50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfa40c1f50 0x7fbfa415c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbfa414e0c0 0x7fbfa415ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 --2- 172.21.15.39:0/3012625023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfa406d300 0x7fbfa41596f0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 >> 172.21.15.39:0/3012625023 conn(0x7fbfa4069e10 msgr2=0x7fbfa40a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 shutdown_connections 2024-09-13T17:09:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:52.069+0000 7fbfaac9a640 1 -- 172.21.15.39:0/3012625023 wait complete. 2024-09-13T17:09:52.086 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:09:54.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:54.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.221+0000 7fee3e55b640 1 Processor -- start 2024-09-13T17:09:54.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee3e55b640 1 -- start start 2024-09-13T17:09:54.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee3e55b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee3814e0c0 0x7fee380a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:54.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee3e55b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fee380c4740 0x7fee380c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:54.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee3e55b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee380c2780 con 0x7fee380c1f50 2024-09-13T17:09:54.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee3e55b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee380c2a60 con 0x7fee380c4740 2024-09-13T17:09:54.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee3e55b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee380c2d40 con 0x7fee3814e0c0 2024-09-13T17:09:54.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee377fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fee380c1f50 0x7fee3806cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56148/0 (socket says 172.21.15.39:56148) 2024-09-13T17:09:54.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee377fe640 1 -- 172.21.15.39:0/3989153293 learned_addr learned my addr 172.21.15.39:0/3989153293 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:54.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee37fff640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee3814e0c0 0x7fee380a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:54.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee3caf8640 1 --2- 172.21.15.39:0/3989153293 >> v2:172.21.15.84:3300/0 conn(0x7fee380c4740 0x7fee380c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:54.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794863681 0 0) 0x7fee380c2780 con 0x7fee380c1f50 2024-09-13T17:09:54.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee20003530 con 0x7fee380c1f50 2024-09-13T17:09:54.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3965024282 0 0) 0x7fee20003530 con 0x7fee380c1f50 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee3814e0c0 msgr2=0x7fee380a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee3814e0c0 0x7fee380a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 >> v2:172.21.15.84:3300/0 conn(0x7fee380c4740 msgr2=0x7fee380c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 --2- 172.21.15.39:0/3989153293 >> v2:172.21.15.84:3300/0 conn(0x7fee380c4740 0x7fee380c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380c3020 con 0x7fee380c1f50 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fee2c001680 con 0x7fee380c1f50 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee3caf8640 1 --2- 172.21.15.39:0/3989153293 >> v2:172.21.15.84:3300/0 conn(0x7fee380c4740 0x7fee380c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 >> v1:172.21.15.39:6789/0 conn(0x7fee380c1f50 legacy=0x7fee3806cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee3814e0c0 0x7fee20003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee20003f00 0x7fee200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee20004d10 0x7fee20005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380c2a60 con 0x7fee20003f00 2024-09-13T17:09:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380c2d40 con 0x7fee20004d10 2024-09-13T17:09:54.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2000b9e0 con 0x7fee3814e0c0 2024-09-13T17:09:54.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee377fe640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee20003f00 0x7fee200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:54.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee37fff640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee3814e0c0 0x7fee20003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:54.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee3caf8640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee20004d10 0x7fee20005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:54.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee377fe640 1 -- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee3814e0c0 msgr2=0x7fee20003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:54.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee377fe640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee3814e0c0 0x7fee20003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee377fe640 1 -- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee20004d10 msgr2=0x7fee20005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:54.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee377fe640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee20004d10 0x7fee20005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.225+0000 7fee377fe640 1 -- 172.21.15.39:0/3989153293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20003530 con 0x7fee20003f00 2024-09-13T17:09:54.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.229+0000 7fee377fe640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee20003f00 0x7fee200047d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c290 tx=0x7fee2c01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88845a2b30686901 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:54.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.229+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0058e0 con 0x7fee20003f00 2024-09-13T17:09:54.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.229+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c005a40 con 0x7fee20003f00 2024-09-13T17:09:54.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.229+0000 7fee36ffd640 1 -- 172.21.15.39:0/3989153293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c005df0 con 0x7fee20003f00 2024-09-13T17:09:54.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.229+0000 7fee3e55b640 1 -- 172.21.15.39:0/3989153293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee20003f00 msgr2=0x7fee200047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:54.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.229+0000 7fee3e55b640 1 --2- 172.21.15.39:0/3989153293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee20003f00 0x7fee200047d0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c290 tx=0x7fee2c01cc90 comp rx=0 tx=0).stop 2024-09-13T17:09:54.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.229+0000 7fee3caf8640 1 -- 172.21.15.39:0/3989153293 reap_dead start 2024-09-13T17:09:54.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.229+0000 7fee3e55b640 1 -- 172.21.15.39:0/3989153293 shutdown_connections 2024-09-13T17:09:54.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.229+0000 7fee3e55b640 1 -- 172.21.15.39:0/3989153293 >> 172.21.15.39:0/3989153293 conn(0x7fee38069e10 msgr2=0x7fee3806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:54.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 -- 172.21.15.39:0/3989153293 shutdown_connections 2024-09-13T17:09:54.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 -- 172.21.15.39:0/3989153293 wait complete. 2024-09-13T17:09:54.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 Processor -- start 2024-09-13T17:09:54.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 -- start start 2024-09-13T17:09:54.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee380aad60 0x7fee381ed460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:54.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee381515f0 0x7fee381ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:54.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381521b0 0x7fee38152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:54.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee381edee0 con 0x7fee380aad60 2024-09-13T17:09:54.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee381ecf60 con 0x7fee381521b0 2024-09-13T17:09:54.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee380a63b0 con 0x7fee381515f0 2024-09-13T17:09:54.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee37fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee380aad60 0x7fee381ed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee37fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee380aad60 0x7fee381ed460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35380/0 (socket says 172.21.15.39:35380) 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee37fff640 1 -- 172.21.15.39:0/2572256375 learned_addr learned my addr 172.21.15.39:0/2572256375 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3caf8640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381521b0 0x7fee38152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee377fe640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee381515f0 0x7fee381ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee37fff640 1 -- 172.21.15.39:0/2572256375 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee381515f0 msgr2=0x7fee381ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee37fff640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee381515f0 0x7fee381ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee37fff640 1 -- 172.21.15.39:0/2572256375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381521b0 msgr2=0x7fee38152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee37fff640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381521b0 0x7fee38152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee37fff640 1 -- 172.21.15.39:0/2572256375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380a6690 con 0x7fee380aad60 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3caf8640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381521b0 0x7fee38152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee37fff640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee380aad60 0x7fee381ed460 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fee280119d0 tx=0x7fee28011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:54.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee34ff9640 1 -- 172.21.15.39:0/2572256375 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2800fb50 con 0x7fee380aad60 2024-09-13T17:09:54.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee34ff9640 1 -- 172.21.15.39:0/2572256375 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2800fcb0 con 0x7fee380aad60 2024-09-13T17:09:54.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee380a6970 con 0x7fee380aad60 2024-09-13T17:09:54.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee34ff9640 1 -- 172.21.15.39:0/2572256375 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28014440 con 0x7fee380aad60 2024-09-13T17:09:54.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee380a6050 con 0x7fee380aad60 2024-09-13T17:09:54.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.233+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee380a6bb0 con 0x7fee380aad60 2024-09-13T17:09:54.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.237+0000 7fee34ff9640 1 -- 172.21.15.39:0/2572256375 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee2801d400 con 0x7fee380aad60 2024-09-13T17:09:54.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.237+0000 7fee34ff9640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee24066690 0x7fee24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:54.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.237+0000 7fee34ff9640 1 -- 172.21.15.39:0/2572256375 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7fee280857b0 con 0x7fee380aad60 2024-09-13T17:09:54.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.237+0000 7fee377fe640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee24066690 0x7fee24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:54.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.237+0000 7fee377fe640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee24066690 0x7fee24068b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0aa000 tx=0x7fee2c001a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:54.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.241+0000 7fee34ff9640 1 -- 172.21.15.39:0/2572256375 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fee28051e30 con 0x7fee380aad60 2024-09-13T17:09:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.437+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee380c4bb0 con 0x7fee380aad60 2024-09-13T17:09:54.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.437+0000 7fee34ff9640 1 -- 172.21.15.39:0/2572256375 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5941 (secure 0 0 0) 0x7fee28055450 con 0x7fee380aad60 2024-09-13T17:09:54.440 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:54.440 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":152}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:54.440 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-13T17:09:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee24066690 msgr2=0x7fee24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee24066690 0x7fee24068b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0aa000 tx=0x7fee2c001a10 comp rx=0 tx=0).stop 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee380aad60 msgr2=0x7fee381ed460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee380aad60 0x7fee381ed460 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fee280119d0 tx=0x7fee28011e90 comp rx=0 tx=0).stop 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 shutdown_connections 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fee24066690 0x7fee24068b50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee381521b0 0x7fee38152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fee381515f0 0x7fee381ed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 --2- 172.21.15.39:0/2572256375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee380aad60 0x7fee381ed460 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 >> 172.21.15.39:0/2572256375 conn(0x7fee38069e10 msgr2=0x7fee3814b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 shutdown_connections 2024-09-13T17:09:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:54.441+0000 7fee3e55b640 1 -- 172.21.15.39:0/2572256375 wait complete. 2024-09-13T17:09:54.459 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:09:56.461 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:56.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf8dcf0640 1 Processor -- start 2024-09-13T17:09:56.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf8dcf0640 1 -- start start 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf8dcf0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf8814e0c0 0x7fdf880a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf8dcf0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdf880c1f50 0x7fdf8806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf8dcf0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf880c2780 con 0x7fdf8806d300 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf8dcf0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf880c2a60 con 0x7fdf880c1f50 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf8dcf0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf880c2d40 con 0x7fdf8814e0c0 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf86ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdf880c1f50 0x7fdf8806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf87fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdf8806d300 0x7fdf880c4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56154/0 (socket says 172.21.15.39:56154) 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf87fff640 1 -- 172.21.15.39:0/2095822264 learned_addr learned my addr 172.21.15.39:0/2095822264 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf877fe640 1 --2- 172.21.15.39:0/2095822264 >> v2:172.21.15.84:3300/0 conn(0x7fdf8814e0c0 0x7fdf880a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf86ffd640 1 -- 172.21.15.39:0/2095822264 >> v2:172.21.15.84:3300/0 conn(0x7fdf8814e0c0 msgr2=0x7fdf880a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf86ffd640 1 --2- 172.21.15.39:0/2095822264 >> v2:172.21.15.84:3300/0 conn(0x7fdf8814e0c0 0x7fdf880a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf86ffd640 1 -- 172.21.15.39:0/2095822264 >> v1:172.21.15.39:6789/0 conn(0x7fdf8806d300 legacy=0x7fdf880c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.597+0000 7fdf86ffd640 1 -- 172.21.15.39:0/2095822264 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf880c3020 con 0x7fdf880c1f50 2024-09-13T17:09:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf86ffd640 1 --2- 172.21.15.39:0/2095822264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf880c1f50 0x7fdf8806cdc0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdf880c3370 tx=0x7fdf7c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac24e5f6a495975 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf867fc640 1 -- 172.21.15.39:0/2095822264 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0ab600 con 0x7fdf880c1f50 2024-09-13T17:09:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf867fc640 1 -- 172.21.15.39:0/2095822264 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c0a7050 con 0x7fdf880c1f50 2024-09-13T17:09:56.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf867fc640 1 -- 172.21.15.39:0/2095822264 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0ab980 con 0x7fdf880c1f50 2024-09-13T17:09:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/2095822264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf880c1f50 msgr2=0x7fdf8806cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 --2- 172.21.15.39:0/2095822264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf880c1f50 0x7fdf8806cdc0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdf880c3370 tx=0x7fdf7c01c3c0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/2095822264 shutdown_connections 2024-09-13T17:09:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 --2- 172.21.15.39:0/2095822264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf880c1f50 0x7fdf8806cdc0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 --2- 172.21.15.39:0/2095822264 >> v2:172.21.15.84:3300/0 conn(0x7fdf8814e0c0 0x7fdf880a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/2095822264 >> 172.21.15.39:0/2095822264 conn(0x7fdf88069e10 msgr2=0x7fdf8806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/2095822264 shutdown_connections 2024-09-13T17:09:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/2095822264 wait complete. 2024-09-13T17:09:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 Processor -- start 2024-09-13T17:09:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.601+0000 7fdf8dcf0640 1 -- start start 2024-09-13T17:09:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf8dcf0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8806d300 0x7fdf88152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf8dcf0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf880c1f50 0x7fdf88154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf8dcf0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf8814e0c0 0x7fdf881548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf8dcf0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf88154e00 con 0x7fdf8806d300 2024-09-13T17:09:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf8dcf0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf88153460 con 0x7fdf8814e0c0 2024-09-13T17:09:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf8dcf0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8806f510 con 0x7fdf880c1f50 2024-09-13T17:09:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8806d300 0x7fdf88152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8806d300 0x7fdf88152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35398/0 (socket says 172.21.15.39:35398) 2024-09-13T17:09:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf877fe640 1 -- 172.21.15.39:0/3215192692 learned_addr learned my addr 172.21.15.39:0/3215192692 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf86ffd640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf880c1f50 0x7fdf88154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf87fff640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf8814e0c0 0x7fdf881548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf877fe640 1 -- 172.21.15.39:0/3215192692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf880c1f50 msgr2=0x7fdf88154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf877fe640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf880c1f50 0x7fdf88154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf877fe640 1 -- 172.21.15.39:0/3215192692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf8814e0c0 msgr2=0x7fdf881548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf877fe640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf8814e0c0 0x7fdf881548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf877fe640 1 -- 172.21.15.39:0/3215192692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c009420 con 0x7fdf8806d300 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf87fff640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf8814e0c0 0x7fdf881548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf877fe640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8806d300 0x7fdf88152c20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800d620 tx=0x7fdf7800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf6ffff640 1 -- 172.21.15.39:0/3215192692 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780125f0 con 0x7fdf8806d300 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf6ffff640 1 -- 172.21.15.39:0/3215192692 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf78012750 con 0x7fdf8806d300 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8806fb80 con 0x7fdf8806d300 2024-09-13T17:09:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf6ffff640 1 -- 172.21.15.39:0/3215192692 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78012ae0 con 0x7fdf8806d300 2024-09-13T17:09:56.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8806f230 con 0x7fdf8806d300 2024-09-13T17:09:56.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8806cdc0 con 0x7fdf8806d300 2024-09-13T17:09:56.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf6ffff640 1 -- 172.21.15.39:0/3215192692 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf78012c40 con 0x7fdf8806d300 2024-09-13T17:09:56.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf6ffff640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf740666c0 0x7fdf74068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:56.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf6ffff640 1 -- 172.21.15.39:0/3215192692 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7fdf78085240 con 0x7fdf8806d300 2024-09-13T17:09:56.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.605+0000 7fdf86ffd640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf740666c0 0x7fdf74068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:56.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.609+0000 7fdf86ffd640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf740666c0 0x7fdf74068b80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0ad000 tx=0x7fdf7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:56.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.613+0000 7fdf6ffff640 1 -- 172.21.15.39:0/3215192692 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf78051890 con 0x7fdf8806d300 2024-09-13T17:09:56.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.813+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf8806a290 con 0x7fdf8806d300 2024-09-13T17:09:56.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.813+0000 7fdf6ffff640 1 -- 172.21.15.39:0/3215192692 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5941 (secure 0 0 0) 0x7fdf78054eb0 con 0x7fdf8806d300 2024-09-13T17:09:56.816 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:56.816 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":152}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:56.816 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-13T17:09:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf740666c0 msgr2=0x7fdf74068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf740666c0 0x7fdf74068b80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0ad000 tx=0x7fdf7c008040 comp rx=0 tx=0).stop 2024-09-13T17:09:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8806d300 msgr2=0x7fdf88152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8806d300 0x7fdf88152c20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800d620 tx=0x7fdf7800dae0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 shutdown_connections 2024-09-13T17:09:56.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf740666c0 0x7fdf74068b80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf8814e0c0 0x7fdf881548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf880c1f50 0x7fdf88154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 --2- 172.21.15.39:0/3215192692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf8806d300 0x7fdf88152c20 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 >> 172.21.15.39:0/3215192692 conn(0x7fdf88069e10 msgr2=0x7fdf8814e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 shutdown_connections 2024-09-13T17:09:56.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:56.817+0000 7fdf8dcf0640 1 -- 172.21.15.39:0/3215192692 wait complete. 2024-09-13T17:09:56.835 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:09:58.836 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:09:58.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.969+0000 7f4dafb7e640 1 Processor -- start 2024-09-13T17:09:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.969+0000 7f4dafb7e640 1 -- start start 2024-09-13T17:09:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4da806c870 0x7f4da80aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4da80c40b0 0x7f4da815c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da8152a80 con 0x7f4da80aabc0 2024-09-13T17:09:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da80c30f0 con 0x7f4da806c870 2024-09-13T17:09:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da8151e00 con 0x7f4da80c40b0 2024-09-13T17:09:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dad119640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4da806c870 0x7f4da80aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dad91a640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4da80aabc0 0x7f4da80c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56170/0 (socket says 172.21.15.39:56170) 2024-09-13T17:09:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dad91a640 1 -- 172.21.15.39:0/3670513374 learned_addr learned my addr 172.21.15.39:0/3670513374 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:09:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dae11b640 1 --2- 172.21.15.39:0/3670513374 >> v2:172.21.15.84:3300/0 conn(0x7f4da80c40b0 0x7f4da815c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dad119640 1 -- 172.21.15.39:0/3670513374 >> v2:172.21.15.84:3300/0 conn(0x7f4da80c40b0 msgr2=0x7f4da815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:09:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dad119640 1 --2- 172.21.15.39:0/3670513374 >> v2:172.21.15.84:3300/0 conn(0x7f4da80c40b0 0x7f4da815c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dad119640 1 -- 172.21.15.39:0/3670513374 >> v1:172.21.15.39:6789/0 conn(0x7f4da80aabc0 legacy=0x7f4da80c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dad119640 1 -- 172.21.15.39:0/3670513374 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da81520e0 con 0x7f4da806c870 2024-09-13T17:09:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dac918640 1 -- 172.21.15.39:0/3670513374 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167881104 0 0) 0x7f4da4001120 con 0x7f4da80aabc0 2024-09-13T17:09:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dad119640 1 --2- 172.21.15.39:0/3670513374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 0x7f4da80aafa0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4da806f220 tx=0x7f4d9c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f0382f8633408e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dac918640 1 -- 172.21.15.39:0/3670513374 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0a5930 con 0x7f4da806c870 2024-09-13T17:09:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dac918640 1 -- 172.21.15.39:0/3670513374 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c0a5a90 con 0x7f4da806c870 2024-09-13T17:09:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dac918640 1 -- 172.21.15.39:0/3670513374 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0a5e20 con 0x7f4da806c870 2024-09-13T17:09:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3670513374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 msgr2=0x7f4da80aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 --2- 172.21.15.39:0/3670513374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 0x7f4da80aafa0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4da806f220 tx=0x7f4d9c01c870 comp rx=0 tx=0).stop 2024-09-13T17:09:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3670513374 shutdown_connections 2024-09-13T17:09:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 --2- 172.21.15.39:0/3670513374 >> v2:172.21.15.84:3300/0 conn(0x7f4da80c40b0 0x7f4da815c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 --2- 172.21.15.39:0/3670513374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 0x7f4da80aafa0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.973+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3670513374 >> 172.21.15.39:0/3670513374 conn(0x7f4da8069e10 msgr2=0x7f4da806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3670513374 shutdown_connections 2024-09-13T17:09:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3670513374 wait complete. 2024-09-13T17:09:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 Processor -- start 2024-09-13T17:09:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 -- start start 2024-09-13T17:09:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 0x7f4da814b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4da80aabc0 0x7f4da814bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80c40b0 0x7f4da8150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8145b00 con 0x7f4da806c870 2024-09-13T17:09:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da814a770 con 0x7f4da80c40b0 2024-09-13T17:09:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da80a8d20 con 0x7f4da80aabc0 2024-09-13T17:09:58.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad91a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 0x7f4da814b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad91a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 0x7f4da814b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35418/0 (socket says 172.21.15.39:35418) 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad91a640 1 -- 172.21.15.39:0/3205684869 learned_addr learned my addr 172.21.15.39:0/3205684869 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad119640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4da80aabc0 0x7f4da814bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dae11b640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80c40b0 0x7f4da8150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad91a640 1 -- 172.21.15.39:0/3205684869 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4da80aabc0 msgr2=0x7f4da814bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad91a640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4da80aabc0 0x7f4da814bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad91a640 1 -- 172.21.15.39:0/3205684869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80c40b0 msgr2=0x7f4da8150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad91a640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80c40b0 0x7f4da8150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad91a640 1 -- 172.21.15.39:0/3205684869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c009a70 con 0x7f4da806c870 2024-09-13T17:09:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dad91a640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 0x7f4da814b310 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4da400c780 tx=0x7f4da400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4d9a7fc640 1 -- 172.21.15.39:0/3205684869 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da40072d0 con 0x7f4da806c870 2024-09-13T17:09:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4d9a7fc640 1 -- 172.21.15.39:0/3205684869 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da4007450 con 0x7f4da806c870 2024-09-13T17:09:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4d9a7fc640 1 -- 172.21.15.39:0/3205684869 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da40047f0 con 0x7f4da806c870 2024-09-13T17:09:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da8145820 con 0x7f4da806c870 2024-09-13T17:09:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da81535f0 con 0x7f4da806c870 2024-09-13T17:09:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.977+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da8151ad0 con 0x7f4da806c870 2024-09-13T17:09:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.981+0000 7f4d9a7fc640 1 -- 172.21.15.39:0/3205684869 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4da4004950 con 0x7f4da806c870 2024-09-13T17:09:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.981+0000 7f4d9a7fc640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d8c0666c0 0x7f4d8c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:09:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.981+0000 7f4d9a7fc640 1 -- 172.21.15.39:0/3205684869 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 9131+0+0 (secure 0 0 0) 0x7f4da4083f40 con 0x7f4da806c870 2024-09-13T17:09:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.981+0000 7f4dad119640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d8c0666c0 0x7f4d8c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:09:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.981+0000 7f4dad119640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d8c0666c0 0x7f4d8c068b80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0af000 tx=0x7f4d9c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:09:58.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:58.985+0000 7f4d9a7fc640 1 -- 172.21.15.39:0/3205684869 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4da8151ad0 con 0x7f4da806c870 2024-09-13T17:09:59.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.185+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4da8152a80 con 0x7f4da806c870 2024-09-13T17:09:59.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.185+0000 7f4d9a7fc640 1 -- 172.21.15.39:0/3205684869 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5941 (secure 0 0 0) 0x7f4da8152a80 con 0x7f4da806c870 2024-09-13T17:09:59.187 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:09:59.187 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":152}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":147,"state":"up:active","state_seq":54,"addr":"172.21.15.84:6835/3923043380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3923043380},{"type":"v1","addr":"172.21.15.84:6835","nonce":3923043380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:09:59.187 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-13T17:09:59.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d8c0666c0 msgr2=0x7f4d8c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:59.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d8c0666c0 0x7f4d8c068b80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0af000 tx=0x7f4d9c0a7070 comp rx=0 tx=0).stop 2024-09-13T17:09:59.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 msgr2=0x7f4da814b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:09:59.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 0x7f4da814b310 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4da400c780 tx=0x7f4da400cc40 comp rx=0 tx=0).stop 2024-09-13T17:09:59.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 shutdown_connections 2024-09-13T17:09:59.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4d8c0666c0 0x7f4d8c068b80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:59.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4da80c40b0 0x7f4da8150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4da80aabc0 0x7f4da814bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 --2- 172.21.15.39:0/3205684869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da806c870 0x7f4da814b310 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:09:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 >> 172.21.15.39:0/3205684869 conn(0x7f4da8069e10 msgr2=0x7f4da8152ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:09:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 shutdown_connections 2024-09-13T17:09:59.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:09:59.189+0000 7f4dafb7e640 1 -- 172.21.15.39:0/3205684869 wait complete. 2024-09-13T17:09:59.206 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:10:00.249 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:10:00.252 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:10:01.207 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ecbbb9640 1 Processor -- start 2024-09-13T17:10:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ecbbb9640 1 -- start start 2024-09-13T17:10:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ecbbb9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ecbbb9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5ec406d300 0x7f5ec40c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ecbbb9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec40c2780 con 0x7f5ec414e0c0 2024-09-13T17:10:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ecbbb9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec40c2a60 con 0x7f5ec406d300 2024-09-13T17:10:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ecbbb9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40c2d40 con 0x7f5ec40a4e70 2024-09-13T17:10:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ec9154640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5eca156640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5ec406d300 0x7f5ec40c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ec9154640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec406cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:57476/0 (socket says 172.21.15.39:57476) 2024-09-13T17:10:01.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ec9154640 1 -- 172.21.15.39:0/4234496293 learned_addr learned my addr 172.21.15.39:0/4234496293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:01.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ec9154640 1 -- 172.21.15.39:0/4234496293 >> v2:172.21.15.84:3300/0 conn(0x7f5ec406d300 msgr2=0x7f5ec40c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:01.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ec9154640 1 --2- 172.21.15.39:0/4234496293 >> v2:172.21.15.84:3300/0 conn(0x7f5ec406d300 0x7f5ec40c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ec9154640 1 -- 172.21.15.39:0/4234496293 >> v1:172.21.15.84:6789/0 conn(0x7f5ec414e0c0 legacy=0x7f5ec40c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.341+0000 7f5ec9154640 1 -- 172.21.15.39:0/4234496293 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec40c3020 con 0x7f5ec40a4e70 2024-09-13T17:10:01.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ec9154640 1 --2- 172.21.15.39:0/4234496293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec406cdc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009870 tx=0x7f5eb401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2403e5dde7c172c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:01.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ec8953640 1 -- 172.21.15.39:0/4234496293 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40c5490 con 0x7f5ec40a4e70 2024-09-13T17:10:01.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ec8953640 1 -- 172.21.15.39:0/4234496293 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb40a7050 con 0x7f5ec40a4e70 2024-09-13T17:10:01.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ec8953640 1 -- 172.21.15.39:0/4234496293 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40ab820 con 0x7f5ec40a4e70 2024-09-13T17:10:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/4234496293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 msgr2=0x7f5ec406cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 --2- 172.21.15.39:0/4234496293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec406cdc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009870 tx=0x7f5eb401c4d0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/4234496293 shutdown_connections 2024-09-13T17:10:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 --2- 172.21.15.39:0/4234496293 >> v2:172.21.15.84:3300/0 conn(0x7f5ec406d300 0x7f5ec40c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 --2- 172.21.15.39:0/4234496293 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec406cdc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/4234496293 >> 172.21.15.39:0/4234496293 conn(0x7f5ec4069e10 msgr2=0x7f5ec406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/4234496293 shutdown_connections 2024-09-13T17:10:01.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/4234496293 wait complete. 2024-09-13T17:10:01.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 Processor -- start 2024-09-13T17:10:01.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 -- start start 2024-09-13T17:10:01.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec406d300 0x7f5ec413cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec41edca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec414e0c0 0x7f5ec41ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40a53e0 con 0x7f5ec406d300 2024-09-13T17:10:01.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40ab890 con 0x7f5ec414e0c0 2024-09-13T17:10:01.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.345+0000 7f5ecbbb9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40a6770 con 0x7f5ec40a4e70 2024-09-13T17:10:01.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9955640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec406d300 0x7f5ec413cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9955640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec406d300 0x7f5ec413cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46466/0 (socket says 172.21.15.39:46466) 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9955640 1 -- 172.21.15.39:0/1851502254 learned_addr learned my addr 172.21.15.39:0/1851502254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9154640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec41edca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5eca156640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec414e0c0 0x7f5ec41ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9955640 1 -- 172.21.15.39:0/1851502254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 msgr2=0x7f5ec41edca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9955640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec41edca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9955640 1 -- 172.21.15.39:0/1851502254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec414e0c0 msgr2=0x7f5ec41ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9955640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec414e0c0 0x7f5ec41ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9955640 1 -- 172.21.15.39:0/1851502254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb40094d0 con 0x7f5ec406d300 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9955640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec406d300 0x7f5ec413cad0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c7a0 tx=0x7f5ec000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5eba7fc640 1 -- 172.21.15.39:0/1851502254 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0003b80 con 0x7f5ec406d300 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec40a6a50 con 0x7f5ec406d300 2024-09-13T17:10:01.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5eba7fc640 1 -- 172.21.15.39:0/1851502254 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec0003ce0 con 0x7f5ec406d300 2024-09-13T17:10:01.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5eba7fc640 1 -- 172.21.15.39:0/1851502254 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000aa90 con 0x7f5ec406d300 2024-09-13T17:10:01.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec41486a0 con 0x7f5ec406d300 2024-09-13T17:10:01.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec40c4740 con 0x7f5ec406d300 2024-09-13T17:10:01.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5eba7fc640 1 -- 172.21.15.39:0/1851502254 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5ec000abf0 con 0x7f5ec406d300 2024-09-13T17:10:01.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5eba7fc640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ea8066690 0x7f5ea8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5eba7fc640 1 -- 172.21.15.39:0/1851502254 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7f5ec0084c90 con 0x7f5ec406d300 2024-09-13T17:10:01.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9154640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ea8066690 0x7f5ea8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.349+0000 7f5ec9154640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ea8066690 0x7f5ea8068b50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40ad000 tx=0x7f5eb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:01.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.353+0000 7f5eba7fc640 1 -- 172.21.15.39:0/1851502254 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5ec00512b0 con 0x7f5ec406d300 2024-09-13T17:10:01.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.549+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ec40c2120 con 0x7f5ec406d300 2024-09-13T17:10:01.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.549+0000 7f5eba7fc640 1 -- 172.21.15.39:0/1851502254 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+5185 (secure 0 0 0) 0x7f5ec00548d0 con 0x7f5ec406d300 2024-09-13T17:10:01.550 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:01.551 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":154,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:01.135827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:01.551 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 154 2024-09-13T17:10:01.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ea8066690 msgr2=0x7f5ea8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ea8066690 0x7f5ea8068b50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40ad000 tx=0x7f5eb4008040 comp rx=0 tx=0).stop 2024-09-13T17:10:01.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec406d300 msgr2=0x7f5ec413cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec406d300 0x7f5ec413cad0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c7a0 tx=0x7f5ec000cc60 comp rx=0 tx=0).stop 2024-09-13T17:10:01.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 shutdown_connections 2024-09-13T17:10:01.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5ea8066690 0x7f5ea8068b50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5ec414e0c0 0x7f5ec41ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5ec40a4e70 0x7f5ec41edca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 --2- 172.21.15.39:0/1851502254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec406d300 0x7f5ec413cad0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 >> 172.21.15.39:0/1851502254 conn(0x7f5ec4069e10 msgr2=0x7f5ec415a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 shutdown_connections 2024-09-13T17:10:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.553+0000 7f5ecbbb9640 1 -- 172.21.15.39:0/1851502254 wait complete. 2024-09-13T17:10:01.570 INFO:tasks.mds_thrash.fs.[a]:mds.d down, removed from mdsmap 2024-09-13T17:10:01.570 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:10:01.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:01.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.681+0000 7f5f404c0640 1 Processor -- start 2024-09-13T17:10:01.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.681+0000 7f5f404c0640 1 -- start start 2024-09-13T17:10:01.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f404c0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5f380aafa0 0x7f5f3815d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f404c0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f38152a80 con 0x7f5f380aabc0 2024-09-13T17:10:01.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f404c0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380ab410 con 0x7f5f3806c870 2024-09-13T17:10:01.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f404c0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f380c30f0 con 0x7f5f380aafa0 2024-09-13T17:10:01.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3e25c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5f380aabc0 0x7f5f380c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33138/0 (socket says 172.21.15.39:33138) 2024-09-13T17:10:01.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3e25c640 1 -- 172.21.15.39:0/1121402888 learned_addr learned my addr 172.21.15.39:0/1121402888 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:10:01.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3ea5d640 1 --2- 172.21.15.39:0/1121402888 >> v2:172.21.15.39:3300/0 conn(0x7f5f380aafa0 0x7f5f3815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3d25a640 1 -- 172.21.15.39:0/1121402888 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452647625 0 0) 0x7f5f34001120 con 0x7f5f380aabc0 2024-09-13T17:10:01.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3d25a640 1 -- 172.21.15.39:0/1121402888 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f24003530 con 0x7f5f380aabc0 2024-09-13T17:10:01.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3ea5d640 1 -- 172.21.15.39:0/1121402888 >> v1:172.21.15.84:6789/0 conn(0x7f5f3806c870 legacy=0x7f5f380c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:01.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3ea5d640 1 -- 172.21.15.39:0/1121402888 >> v1:172.21.15.39:6789/0 conn(0x7f5f380aabc0 legacy=0x7f5f380c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3ea5d640 1 -- 172.21.15.39:0/1121402888 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f38151e00 con 0x7f5f380aafa0 2024-09-13T17:10:01.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3ea5d640 1 --2- 172.21.15.39:0/1121402888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aafa0 0x7f5f3815d480 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5f300097c0 tx=0x7f5f3001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149ac76728c7af8f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:01.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3d25a640 1 -- 172.21.15.39:0/1121402888 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300ac640 con 0x7f5f380aafa0 2024-09-13T17:10:01.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3d25a640 1 -- 172.21.15.39:0/1121402888 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f300ac7a0 con 0x7f5f380aafa0 2024-09-13T17:10:01.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f3d25a640 1 -- 172.21.15.39:0/1121402888 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300acb30 con 0x7f5f380aafa0 2024-09-13T17:10:01.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f404c0640 1 -- 172.21.15.39:0/1121402888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aafa0 msgr2=0x7f5f3815d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f404c0640 1 --2- 172.21.15.39:0/1121402888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aafa0 0x7f5f3815d480 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5f300097c0 tx=0x7f5f3001c310 comp rx=0 tx=0).stop 2024-09-13T17:10:01.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f404c0640 1 -- 172.21.15.39:0/1121402888 shutdown_connections 2024-09-13T17:10:01.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f404c0640 1 --2- 172.21.15.39:0/1121402888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aafa0 0x7f5f3815d480 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.685+0000 7f5f404c0640 1 -- 172.21.15.39:0/1121402888 >> 172.21.15.39:0/1121402888 conn(0x7f5f38069e10 msgr2=0x7f5f3806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:01.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 -- 172.21.15.39:0/1121402888 shutdown_connections 2024-09-13T17:10:01.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 -- 172.21.15.39:0/1121402888 wait complete. 2024-09-13T17:10:01.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 Processor -- start 2024-09-13T17:10:01.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 -- start start 2024-09-13T17:10:01.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f3806c870 0x7f5f381511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aabc0 0x7f5f381efc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f380aafa0 0x7f5f38141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380a9b50 con 0x7f5f380aabc0 2024-09-13T17:10:01.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f38148ba0 con 0x7f5f380aafa0 2024-09-13T17:10:01.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380a9e30 con 0x7f5f3806c870 2024-09-13T17:10:01.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3da5b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aabc0 0x7f5f381efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3da5b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aabc0 0x7f5f381efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46488/0 (socket says 172.21.15.39:46488) 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3da5b640 1 -- 172.21.15.39:0/2498813971 learned_addr learned my addr 172.21.15.39:0/2498813971 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3e25c640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f3806c870 0x7f5f381511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3ea5d640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f380aafa0 0x7f5f38141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3da5b640 1 -- 172.21.15.39:0/2498813971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f3806c870 msgr2=0x7f5f381511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3da5b640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f3806c870 0x7f5f381511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3da5b640 1 -- 172.21.15.39:0/2498813971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f380aafa0 msgr2=0x7f5f38141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3da5b640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f380aafa0 0x7f5f38141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3da5b640 1 -- 172.21.15.39:0/2498813971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30009420 con 0x7f5f380aabc0 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3ea5d640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f380aafa0 0x7f5f38141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f3da5b640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aabc0 0x7f5f381efc60 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00ba50 tx=0x7f5f2c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:01.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f2affd640 1 -- 172.21.15.39:0/2498813971 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c00cbe0 con 0x7f5f380aabc0 2024-09-13T17:10:01.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f2affd640 1 -- 172.21.15.39:0/2498813971 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2c00cd80 con 0x7f5f380aabc0 2024-09-13T17:10:01.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f38148e40 con 0x7f5f380aabc0 2024-09-13T17:10:01.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f2affd640 1 -- 172.21.15.39:0/2498813971 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c014680 con 0x7f5f380aabc0 2024-09-13T17:10:01.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f381487c0 con 0x7f5f380aabc0 2024-09-13T17:10:01.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.689+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f380c39a0 con 0x7f5f380aabc0 2024-09-13T17:10:01.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.693+0000 7f5f2affd640 1 -- 172.21.15.39:0/2498813971 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f2c0147e0 con 0x7f5f380aabc0 2024-09-13T17:10:01.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.693+0000 7f5f2affd640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f1c0666c0 0x7f5f1c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:01.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.693+0000 7f5f2affd640 1 -- 172.21.15.39:0/2498813971 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7f5f2c084d80 con 0x7f5f380aabc0 2024-09-13T17:10:01.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.693+0000 7f5f3e25c640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f1c0666c0 0x7f5f1c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:01.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.693+0000 7f5f3e25c640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f1c0666c0 0x7f5f1c068b80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5f34001280 tx=0x7f5f34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:01.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.697+0000 7f5f2affd640 1 -- 172.21.15.39:0/2498813971 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5f2c051420 con 0x7f5f380aabc0 2024-09-13T17:10:01.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.885+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f3806c650 con 0x7f5f380aabc0 2024-09-13T17:10:01.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f2affd640 1 -- 172.21.15.39:0/2498813971 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+5185 (secure 0 0 0) 0x7f5f2c019020 con 0x7f5f380aabc0 2024-09-13T17:10:01.889 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:01.889 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":154,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:01.135827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:reconnect","state_seq":6,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:01.889 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 154 2024-09-13T17:10:01.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f1c0666c0 msgr2=0x7f5f1c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f1c0666c0 0x7f5f1c068b80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5f34001280 tx=0x7f5f34009040 comp rx=0 tx=0).stop 2024-09-13T17:10:01.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aabc0 msgr2=0x7f5f381efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:01.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aabc0 0x7f5f381efc60 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00ba50 tx=0x7f5f2c00bf10 comp rx=0 tx=0).stop 2024-09-13T17:10:01.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 shutdown_connections 2024-09-13T17:10:01.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f1c0666c0 0x7f5f1c068b80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f380aafa0 0x7f5f38141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f380aabc0 0x7f5f381efc60 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 --2- 172.21.15.39:0/2498813971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f3806c870 0x7f5f381511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:01.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 >> 172.21.15.39:0/2498813971 conn(0x7f5f38069e10 msgr2=0x7f5f380c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:01.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 shutdown_connections 2024-09-13T17:10:01.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:01.889+0000 7f5f404c0640 1 -- 172.21.15.39:0/2498813971 wait complete. 2024-09-13T17:10:01.907 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 154, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 154, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_7677": { "addr": "172.21.15.39:6837/2351253009", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 2351253009, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 2351253009, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7677, "incarnation": 153, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:10:01.135827+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7677 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 150, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_7485": { "addr": "172.21.15.39:6835/1034701751", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 1034701751, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 1034701751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7485, "incarnation": 147, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 12 } }, "last_failure": 0, "last_failure_osd_epoch": 53, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:09:33.031201+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7485 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:10:03.909 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:04.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.049+0000 7f1f03e7a640 1 Processor -- start 2024-09-13T17:10:04.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.049+0000 7f1f03e7a640 1 -- start start 2024-09-13T17:10:04.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f03e7a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1efc06aac0 0x7f1efc0aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f03e7a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc0c3ae0 con 0x7f1efc0c21d0 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f03e7a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc0abb30 con 0x7f1efc14b850 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f03e7a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc0ab340 con 0x7f1efc06aac0 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f01415640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1efc06aac0 0x7f1efc0aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f01c16640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1efc0c21d0 0x7f1efc151c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33142/0 (socket says 172.21.15.39:33142) 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f01c16640 1 -- 172.21.15.39:0/2095815811 learned_addr learned my addr 172.21.15.39:0/2095815811 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f01415640 1 -- 172.21.15.39:0/2095815811 >> v1:172.21.15.84:6789/0 conn(0x7f1efc14b850 legacy=0x7f1efc15c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f01415640 1 -- 172.21.15.39:0/2095815811 >> v1:172.21.15.39:6789/0 conn(0x7f1efc0c21d0 legacy=0x7f1efc151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f01415640 1 -- 172.21.15.39:0/2095815811 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc0abe10 con 0x7f1efc06aac0 2024-09-13T17:10:04.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.053+0000 7f1f00c14640 1 -- 172.21.15.39:0/2095815811 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071430850 0 0) 0x7f1efc0c3ae0 con 0x7f1efc0c21d0 2024-09-13T17:10:04.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f01415640 1 --2- 172.21.15.39:0/2095815811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 0x7f1efc0aae00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0097c0 tx=0x7f1eec01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a431759ac93c9e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:04.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f00c14640 1 -- 172.21.15.39:0/2095815811 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0ac640 con 0x7f1efc06aac0 2024-09-13T17:10:04.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f00c14640 1 -- 172.21.15.39:0/2095815811 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec0ac7a0 con 0x7f1efc06aac0 2024-09-13T17:10:04.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f00c14640 1 -- 172.21.15.39:0/2095815811 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0acb30 con 0x7f1efc06aac0 2024-09-13T17:10:04.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f03e7a640 1 -- 172.21.15.39:0/2095815811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 msgr2=0x7f1efc0aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:04.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f03e7a640 1 --2- 172.21.15.39:0/2095815811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 0x7f1efc0aae00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0097c0 tx=0x7f1eec01c3e0 comp rx=0 tx=0).stop 2024-09-13T17:10:04.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f03e7a640 1 -- 172.21.15.39:0/2095815811 shutdown_connections 2024-09-13T17:10:04.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f03e7a640 1 --2- 172.21.15.39:0/2095815811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 0x7f1efc0aae00 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:04.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f03e7a640 1 -- 172.21.15.39:0/2095815811 >> 172.21.15.39:0/2095815811 conn(0x7f1efc069e10 msgr2=0x7f1efc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:04.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f03e7a640 1 -- 172.21.15.39:0/2095815811 shutdown_connections 2024-09-13T17:10:04.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f03e7a640 1 -- 172.21.15.39:0/2095815811 wait complete. 2024-09-13T17:10:04.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f03e7a640 1 Processor -- start 2024-09-13T17:10:04.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.057+0000 7f1f03e7a640 1 -- start start 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f03e7a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 0x7f1efc144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f03e7a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1efc0c21d0 0x7f1efc14c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f03e7a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1efc14b850 0x7f1efc14cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f03e7a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc14dfe0 con 0x7f1efc06aac0 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f03e7a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc14af80 con 0x7f1efc14b850 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f03e7a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc149c60 con 0x7f1efc0c21d0 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01c16640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 0x7f1efc144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01c16640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 0x7f1efc144490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46506/0 (socket says 172.21.15.39:46506) 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01c16640 1 -- 172.21.15.39:0/3861218620 learned_addr learned my addr 172.21.15.39:0/3861218620 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:04.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f02417640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1efc14b850 0x7f1efc14cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:04.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01415640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1efc0c21d0 0x7f1efc14c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:04.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01c16640 1 -- 172.21.15.39:0/3861218620 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1efc0c21d0 msgr2=0x7f1efc14c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:04.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01c16640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1efc0c21d0 0x7f1efc14c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:04.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01c16640 1 -- 172.21.15.39:0/3861218620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1efc14b850 msgr2=0x7f1efc14cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:04.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01c16640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1efc14b850 0x7f1efc14cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:04.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01c16640 1 -- 172.21.15.39:0/3861218620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec009420 con 0x7f1efc06aac0 2024-09-13T17:10:04.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f02417640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1efc14b850 0x7f1efc14cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:10:04.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01c16640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 0x7f1efc144490 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c4d0 tx=0x7f1ef800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:04.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1ef27fc640 1 -- 172.21.15.39:0/3861218620 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80074e0 con 0x7f1efc06aac0 2024-09-13T17:10:04.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1ef27fc640 1 -- 172.21.15.39:0/3861218620 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef8007640 con 0x7f1efc06aac0 2024-09-13T17:10:04.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc15d950 con 0x7f1efc06aac0 2024-09-13T17:10:04.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1ef27fc640 1 -- 172.21.15.39:0/3861218620 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8004a30 con 0x7f1efc06aac0 2024-09-13T17:10:04.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc1449d0 con 0x7f1efc06aac0 2024-09-13T17:10:04.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc0c3560 con 0x7f1efc06aac0 2024-09-13T17:10:04.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1ef27fc640 1 -- 172.21.15.39:0/3861218620 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ef8004b90 con 0x7f1efc06aac0 2024-09-13T17:10:04.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1ef27fc640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ee0066690 0x7f1ee0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:04.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1ef27fc640 1 -- 172.21.15.39:0/3861218620 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7f1ef8084170 con 0x7f1efc06aac0 2024-09-13T17:10:04.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.061+0000 7f1f01415640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ee0066690 0x7f1ee0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:04.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.065+0000 7f1f01415640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ee0066690 0x7f1ee0068b50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0ae000 tx=0x7f1eec0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:04.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.069+0000 7f1ef27fc640 1 -- 172.21.15.39:0/3861218620 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1ef8050760 con 0x7f1efc06aac0 2024-09-13T17:10:04.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1efc14f5c0 con 0x7f1efc06aac0 2024-09-13T17:10:04.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1ef27fc640 1 -- 172.21.15.39:0/3861218620 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+5182 (secure 0 0 0) 0x7f1ef8018070 con 0x7f1efc06aac0 2024-09-13T17:10:04.301 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:04.301 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:04.305 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 156 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ee0066690 msgr2=0x7f1ee0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ee0066690 0x7f1ee0068b50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0ae000 tx=0x7f1eec0a7000 comp rx=0 tx=0).stop 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 msgr2=0x7f1efc144490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 0x7f1efc144490 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c4d0 tx=0x7f1ef800c990 comp rx=0 tx=0).stop 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 shutdown_connections 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1ee0066690 0x7f1ee0068b50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1efc14b850 0x7f1efc14cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1efc0c21d0 0x7f1efc14c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 --2- 172.21.15.39:0/3861218620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1efc06aac0 0x7f1efc144490 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:04.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 >> 172.21.15.39:0/3861218620 conn(0x7f1efc069e10 msgr2=0x7f1efc0c60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:04.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 shutdown_connections 2024-09-13T17:10:04.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:04.301+0000 7f1f03e7a640 1 -- 172.21.15.39:0/3861218620 wait complete. 2024-09-13T17:10:04.320 INFO:tasks.mds_thrash.fs.[a]:mds.c has gained rank=0, replacing gid=5963 2024-09-13T17:10:04.320 INFO:tasks.mds_thrash.fs.[a]:waiting for 9 secs before reviving mds.d 2024-09-13T17:10:08.075 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-13T17:10:08.073+0000 7fe3e0627640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:10:13.321 INFO:tasks.mds_thrash.fs.[a]:reviving mds.d 2024-09-13T17:10:13.322 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T17:10:13.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T17:10:13.325 INFO:tasks.ceph.mds.d:Started 2024-09-13T17:10:13.325 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:13.410 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T17:10:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcbdd43640 1 Processor -- start 2024-09-13T17:10:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcbdd43640 1 -- start start 2024-09-13T17:10:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcbdd43640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbcb8065ee0 0x7fbcb814f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcbdd43640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb80c2ef0 0x7fbcb80c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcbdd43640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb806ddf0 con 0x7fbcb80c0a90 2024-09-13T17:10:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcbdd43640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb80c53d0 con 0x7fbcb8065ee0 2024-09-13T17:10:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcbdd43640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80c5950 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcb77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbcb8065ee0 0x7fbcb814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcb77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbcb8065ee0 0x7fbcb814f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53964/0 (socket says 172.21.15.39:53964) 2024-09-13T17:10:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcb77fe640 1 -- 172.21.15.39:0/521442925 learned_addr learned my addr 172.21.15.39:0/521442925 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcb7fff640 1 --2- 172.21.15.39:0/521442925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb80c2ef0 0x7fbcb80c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcb77fe640 1 -- 172.21.15.39:0/521442925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb80c2ef0 msgr2=0x7fbcb80c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcb77fe640 1 --2- 172.21.15.39:0/521442925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb80c2ef0 0x7fbcb80c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcb77fe640 1 -- 172.21.15.39:0/521442925 >> v1:172.21.15.84:6789/0 conn(0x7fbcb80c0a90 legacy=0x7fbcb80c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.477+0000 7fbcb77fe640 1 -- 172.21.15.39:0/521442925 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb806e040 con 0x7fbcb8065ee0 2024-09-13T17:10:13.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcb77fe640 1 --2- 172.21.15.39:0/521442925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb8065ee0 0x7fbcb814f230 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009870 tx=0x7fbca801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c56d97b6266d7fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:13.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcb67fc640 1 -- 172.21.15.39:0/521442925 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80a95f0 con 0x7fbcb8065ee0 2024-09-13T17:10:13.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcb67fc640 1 -- 172.21.15.39:0/521442925 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca80a9750 con 0x7fbcb8065ee0 2024-09-13T17:10:13.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcb67fc640 1 -- 172.21.15.39:0/521442925 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80a9ae0 con 0x7fbcb8065ee0 2024-09-13T17:10:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcbdd43640 1 -- 172.21.15.39:0/521442925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb8065ee0 msgr2=0x7fbcb814f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcbdd43640 1 --2- 172.21.15.39:0/521442925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb8065ee0 0x7fbcb814f230 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009870 tx=0x7fbca801c530 comp rx=0 tx=0).stop 2024-09-13T17:10:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcbdd43640 1 -- 172.21.15.39:0/521442925 shutdown_connections 2024-09-13T17:10:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcbdd43640 1 --2- 172.21.15.39:0/521442925 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb80c2ef0 0x7fbcb80c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcbdd43640 1 --2- 172.21.15.39:0/521442925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb8065ee0 0x7fbcb814f230 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcbdd43640 1 -- 172.21.15.39:0/521442925 >> 172.21.15.39:0/521442925 conn(0x7fbcb80c3310 msgr2=0x7fbcb80c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcbdd43640 1 -- 172.21.15.39:0/521442925 shutdown_connections 2024-09-13T17:10:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcbdd43640 1 -- 172.21.15.39:0/521442925 wait complete. 2024-09-13T17:10:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.481+0000 7fbcbdd43640 1 Processor -- start 2024-09-13T17:10:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 -- start start 2024-09-13T17:10:13.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb8065ee0 0x7fbcb814bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:13.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb80c0a90 0x7fbcb8148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:13.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb80c2ef0 0x7fbcb81490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:13.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80ac3d0 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80ac6b0 con 0x7fbcb80c0a90 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb814b2d0 con 0x7fbcb8065ee0 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb80c2ef0 0x7fbcb81490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb80c2ef0 0x7fbcb81490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53968/0 (socket says 172.21.15.39:53968) 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb7fff640 1 -- 172.21.15.39:0/3329668356 learned_addr learned my addr 172.21.15.39:0/3329668356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb77fe640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb8065ee0 0x7fbcb814bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb6ffd640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb80c0a90 0x7fbcb8148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb7fff640 1 -- 172.21.15.39:0/3329668356 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb8065ee0 msgr2=0x7fbcb814bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb7fff640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb8065ee0 0x7fbcb814bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb7fff640 1 -- 172.21.15.39:0/3329668356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb80c0a90 msgr2=0x7fbcb8148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb7fff640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb80c0a90 0x7fbcb8148b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb7fff640 1 -- 172.21.15.39:0/3329668356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80094d0 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcb7fff640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb80c2ef0 0x7fbcb81490c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000ef90 tx=0x7fbcb000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbc9ffff640 1 -- 172.21.15.39:0/3329668356 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0009bf0 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbc9ffff640 1 -- 172.21.15.39:0/3329668356 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb000f040 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb814b5b0 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbc9ffff640 1 -- 172.21.15.39:0/3329668356 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00146c0 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb814c2f0 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb814f230 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.485+0000 7fbc9ffff640 1 -- 172.21.15.39:0/3329668356 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbcb0014820 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.489+0000 7fbc9ffff640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbca4066690 0x7fbca4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:13.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.489+0000 7fbc9ffff640 1 -- 172.21.15.39:0/3329668356 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7fbcb0085ce0 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.489+0000 7fbcb77fe640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbca4066690 0x7fbca4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:13.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.489+0000 7fbcb77fe640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbca4066690 0x7fbca4068b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbca80ae000 tx=0x7fbca8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:13.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.493+0000 7fbc9ffff640 1 -- 172.21.15.39:0/3329668356 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbcb0052300 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcb80ac510 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbc9ffff640 1 -- 172.21.15.39:0/3329668356 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+5182 (secure 0 0 0) 0x7fbcb80ac510 con 0x7fbcb80c2ef0 2024-09-13T17:10:13.737 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:13.737 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:13.738 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 156 2024-09-13T17:10:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbca4066690 msgr2=0x7fbca4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbca4066690 0x7fbca4068b50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbca80ae000 tx=0x7fbca8009040 comp rx=0 tx=0).stop 2024-09-13T17:10:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb80c2ef0 msgr2=0x7fbcb81490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb80c2ef0 0x7fbcb81490c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000ef90 tx=0x7fbcb000c490 comp rx=0 tx=0).stop 2024-09-13T17:10:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 shutdown_connections 2024-09-13T17:10:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbca4066690 0x7fbca4068b50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbcb80c0a90 0x7fbcb8148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcb80c2ef0 0x7fbcb81490c0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 --2- 172.21.15.39:0/3329668356 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbcb8065ee0 0x7fbcb814bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 >> 172.21.15.39:0/3329668356 conn(0x7fbcb80c3310 msgr2=0x7fbcb806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 shutdown_connections 2024-09-13T17:10:13.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:13.737+0000 7fbcbdd43640 1 -- 172.21.15.39:0/3329668356 wait complete. 2024-09-13T17:10:13.757 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-09-13T17:10:15.759 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.901+0000 7f1a8afb8640 1 Processor -- start 2024-09-13T17:10:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.901+0000 7f1a8afb8640 1 -- start start 2024-09-13T17:10:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a8afb8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1a840a5260 0x7f1a84153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a8afb8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a8406dd50 0x7f1a84157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a8afb8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a841540e0 con 0x7f1a840bfeb0 2024-09-13T17:10:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a8afb8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a840c53f0 con 0x7f1a840a5260 2024-09-13T17:10:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a8afb8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8406ba10 con 0x7f1a8406dd50 2024-09-13T17:10:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a8a7b7640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1a840bfeb0 0x7f1a840bf020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:37828/0 (socket says 172.21.15.39:37828) 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a8a7b7640 1 -- 172.21.15.39:0/2151935844 learned_addr learned my addr 172.21.15.39:0/2151935844 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a897b5640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a8406dd50 0x7f1a84157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a89fb6640 1 --2- 172.21.15.39:0/2151935844 >> v2:172.21.15.84:3300/0 conn(0x7f1a840a5260 0x7f1a84153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759452826 0 0) 0x7f1a841540e0 con 0x7f1a840bfeb0 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a70003530 con 0x7f1a840bfeb0 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1026000404 0 0) 0x7f1a70003530 con 0x7f1a840bfeb0 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a8406dd50 msgr2=0x7f1a84157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a8406dd50 0x7f1a84157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 >> v2:172.21.15.84:3300/0 conn(0x7f1a840a5260 msgr2=0x7f1a84153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 --2- 172.21.15.39:0/2151935844 >> v2:172.21.15.84:3300/0 conn(0x7f1a840a5260 0x7f1a84153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a840c4e30 con 0x7f1a840bfeb0 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1a7c001a00 con 0x7f1a840bfeb0 2024-09-13T17:10:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 >> v1:172.21.15.39:6789/0 conn(0x7f1a840bfeb0 legacy=0x7f1a840bf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a70003980 0x7f1a70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a700047d0 0x7f1a70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a8406dd50 0x7f1a70005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a840c53f0 con 0x7f1a700047d0 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7000b990 con 0x7f1a70003980 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7000bb50 con 0x7f1a8406dd50 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a897b5640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a700047d0 0x7f1a70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a8a7b7640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a8406dd50 0x7f1a70005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a89fb6640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a70003980 0x7f1a70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a897b5640 1 -- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a8406dd50 msgr2=0x7f1a70005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a897b5640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a8406dd50 0x7f1a70005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:15.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a897b5640 1 -- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a70003980 msgr2=0x7f1a70003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a897b5640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a70003980 0x7f1a70003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:15.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.905+0000 7f1a897b5640 1 -- 172.21.15.39:0/2151935844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70003530 con 0x7f1a700047d0 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a897b5640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a700047d0 0x7f1a70004c50 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009ae0 tx=0x7f1a7401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e929b7cfbf2211c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740a5610 con 0x7f1a700047d0 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a740a5770 con 0x7f1a700047d0 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a88fb4640 1 -- 172.21.15.39:0/2151935844 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a740a5b20 con 0x7f1a700047d0 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2151935844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a700047d0 msgr2=0x7f1a70004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 --2- 172.21.15.39:0/2151935844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a700047d0 0x7f1a70004c50 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009ae0 tx=0x7f1a7401ca40 comp rx=0 tx=0).stop 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8a7b7640 1 -- 172.21.15.39:0/2151935844 reap_dead start 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2151935844 shutdown_connections 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2151935844 >> 172.21.15.39:0/2151935844 conn(0x7f1a840a79a0 msgr2=0x7f1a840a7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2151935844 shutdown_connections 2024-09-13T17:10:15.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2151935844 wait complete. 2024-09-13T17:10:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 Processor -- start 2024-09-13T17:10:15.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 -- start start 2024-09-13T17:10:15.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a841efb70 0x7f1a840a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:15.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a84153e80 0x7f1a841ec160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:15.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840a72d0 0x7f1a841ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:15.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841ebd10 con 0x7f1a840a72d0 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841ef300 con 0x7f1a84153e80 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8afb8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84154d30 con 0x7f1a841efb70 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.909+0000 7f1a8a7b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840a72d0 0x7f1a841ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8a7b7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840a72d0 0x7f1a841ec6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53996/0 (socket says 172.21.15.39:53996) 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8a7b7640 1 -- 172.21.15.39:0/2512955841 learned_addr learned my addr 172.21.15.39:0/2512955841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a89fb6640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a841efb70 0x7f1a840a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a897b5640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a84153e80 0x7f1a841ec160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8a7b7640 1 -- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a841efb70 msgr2=0x7f1a840a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8a7b7640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a841efb70 0x7f1a840a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8a7b7640 1 -- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a84153e80 msgr2=0x7f1a841ec160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8a7b7640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a84153e80 0x7f1a841ec160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8a7b7640 1 -- 172.21.15.39:0/2512955841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84154a50 con 0x7f1a840a72d0 2024-09-13T17:10:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a89fb6640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a841efb70 0x7f1a840a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:10:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a897b5640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a84153e80 0x7f1a841ec160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:10:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8a7b7640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840a72d0 0x7f1a841ec6a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0013e0 tx=0x7f1a7c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a7a7fc640 1 -- 172.21.15.39:0/2512955841 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c017070 con 0x7f1a840a72d0 2024-09-13T17:10:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a7a7fc640 1 -- 172.21.15.39:0/2512955841 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7c013420 con 0x7f1a840a72d0 2024-09-13T17:10:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a84151990 con 0x7f1a840a72d0 2024-09-13T17:10:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a7a7fc640 1 -- 172.21.15.39:0/2512955841 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0135b0 con 0x7f1a840a72d0 2024-09-13T17:10:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a841ecbe0 con 0x7f1a840a72d0 2024-09-13T17:10:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a841ebe80 con 0x7f1a840a72d0 2024-09-13T17:10:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a7a7fc640 1 -- 172.21.15.39:0/2512955841 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1a7c013710 con 0x7f1a840a72d0 2024-09-13T17:10:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a7a7fc640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a680666c0 0x7f1a68068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a7a7fc640 1 -- 172.21.15.39:0/2512955841 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7f1a7c084e00 con 0x7f1a840a72d0 2024-09-13T17:10:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a89fb6640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a680666c0 0x7f1a68068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.913+0000 7f1a89fb6640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a680666c0 0x7f1a68068b80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009b70 tx=0x7f1a80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:15.921+0000 7f1a7a7fc640 1 -- 172.21.15.39:0/2512955841 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1a841ebe80 con 0x7f1a840a72d0 2024-09-13T17:10:16.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.169+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a841ebe50 con 0x7f1a840a72d0 2024-09-13T17:10:16.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.169+0000 7f1a7a7fc640 1 -- 172.21.15.39:0/2512955841 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f1a7c0513f0 con 0x7f1a840a72d0 2024-09-13T17:10:16.171 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:16.171 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:16.171 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-13T17:10:16.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a680666c0 msgr2=0x7f1a68068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:16.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a680666c0 0x7f1a68068b80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009b70 tx=0x7f1a80008040 comp rx=0 tx=0).stop 2024-09-13T17:10:16.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840a72d0 msgr2=0x7f1a841ec6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:16.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840a72d0 0x7f1a841ec6a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0013e0 tx=0x7f1a7c00e530 comp rx=0 tx=0).stop 2024-09-13T17:10:16.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 shutdown_connections 2024-09-13T17:10:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1a680666c0 0x7f1a68068b80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a84153e80 0x7f1a841ec160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a840a72d0 0x7f1a841ec6a0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 --2- 172.21.15.39:0/2512955841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1a841efb70 0x7f1a840a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 >> 172.21.15.39:0/2512955841 conn(0x7f1a840a79a0 msgr2=0x7f1a8406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 shutdown_connections 2024-09-13T17:10:16.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:16.173+0000 7f1a8afb8640 1 -- 172.21.15.39:0/2512955841 wait complete. 2024-09-13T17:10:16.194 INFO:tasks.mds_thrash.fs.[a]:mds.d reported in up:standby state 2024-09-13T17:10:16.194 INFO:tasks.mds_thrash.fs.[a]:waiting for 38 secs before thrashing 2024-09-13T17:10:25.299 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:25.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.444+0000 7fe9c2311640 1 Processor -- start 2024-09-13T17:10:25.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.444+0000 7fe9c2311640 1 -- start start 2024-09-13T17:10:25.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9c2311640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9bc070fb0 0x7fe9bc071420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9c2311640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc0c9710 con 0x7fe9bc0ca080 2024-09-13T17:10:25.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9c2311640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc071960 con 0x7fe9bc153d20 2024-09-13T17:10:25.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9c2311640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc157fc0 con 0x7fe9bc070fb0 2024-09-13T17:10:25.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bbfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe9bc0ca080 0x7fe9bc151000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:47892/0 (socket says 172.21.15.39:47892) 2024-09-13T17:10:25.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bbfff640 1 -- 172.21.15.39:0/2426385451 learned_addr learned my addr 172.21.15.39:0/2426385451 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:10:25.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9b37fe640 1 --2- 172.21.15.39:0/2426385451 >> v2:172.21.15.84:3300/0 conn(0x7fe9bc070fb0 0x7fe9bc071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201215809 0 0) 0x7fe9bc0c9710 con 0x7fe9bc0ca080 2024-09-13T17:10:25.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a8003530 con 0x7fe9bc0ca080 2024-09-13T17:10:25.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 48085279 0 0) 0x7fe9a8003530 con 0x7fe9bc0ca080 2024-09-13T17:10:25.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 >> v2:172.21.15.84:3300/0 conn(0x7fe9bc070fb0 msgr2=0x7fe9bc071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:25.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 --2- 172.21.15.39:0/2426385451 >> v2:172.21.15.84:3300/0 conn(0x7fe9bc070fb0 0x7fe9bc071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 >> v1:172.21.15.84:6789/0 conn(0x7fe9bc153d20 legacy=0x7fe9bc154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc157340 con 0x7fe9bc0ca080 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fe9ac001680 con 0x7fe9bc0ca080 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 >> v1:172.21.15.39:6789/0 conn(0x7fe9bc0ca080 legacy=0x7fe9bc151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9a8003980 0x7fe9a8003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9a8005190 0x7fe9a800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc157fc0 con 0x7fe9a80047d0 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a800c0e0 con 0x7fe9a8005190 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a800c2a0 con 0x7fe9a8003980 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9b37fe640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9c08ae640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9a8005190 0x7fe9a800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9bbfff640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9a8003980 0x7fe9a8003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9b37fe640 1 -- 172.21.15.39:0/2426385451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9a8003980 msgr2=0x7fe9a8003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9b37fe640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9a8003980 0x7fe9a8003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9b37fe640 1 -- 172.21.15.39:0/2426385451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9a8005190 msgr2=0x7fe9a800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9b37fe640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9a8005190 0x7fe9a800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9b37fe640 1 -- 172.21.15.39:0/2426385451 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a8003530 con 0x7fe9a80047d0 2024-09-13T17:10:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.448+0000 7fe9c08ae640 1 -- 172.21.15.39:0/2426385451 reap_dead start 2024-09-13T17:10:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9b37fe640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9a8004c50 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009ae0 tx=0x7fe9a401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcb8af244f0a1d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40a6450 con 0x7fe9a80047d0 2024-09-13T17:10:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a40a65b0 con 0x7fe9a80047d0 2024-09-13T17:10:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bb7fe640 1 -- 172.21.15.39:0/2426385451 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40a6960 con 0x7fe9a80047d0 2024-09-13T17:10:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 -- 172.21.15.39:0/2426385451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 msgr2=0x7fe9a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9a8004c50 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009ae0 tx=0x7fe9a401ca70 comp rx=0 tx=0).stop 2024-09-13T17:10:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 -- 172.21.15.39:0/2426385451 shutdown_connections 2024-09-13T17:10:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 --2- 172.21.15.39:0/2426385451 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9a8004c50 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 -- 172.21.15.39:0/2426385451 >> 172.21.15.39:0/2426385451 conn(0x7fe9bc06a210 msgr2=0x7fe9bc06a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 -- 172.21.15.39:0/2426385451 shutdown_connections 2024-09-13T17:10:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 -- 172.21.15.39:0/2426385451 wait complete. 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 Processor -- start 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 -- start start 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9bc1eff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9bc152870 0x7fe9bc0ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9bc0ab710 0x7fe9bc145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc14b310 con 0x7fe9a80047d0 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc147440 con 0x7fe9bc0ab710 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c2311640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc147720 con 0x7fe9bc152870 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9bc1eff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9bc1eff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56058/0 (socket says 172.21.15.39:56058) 2024-09-13T17:10:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bbfff640 1 -- 172.21.15.39:0/4211184009 learned_addr learned my addr 172.21.15.39:0/4211184009 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9b37fe640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9bc152870 0x7fe9bc0ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9c08ae640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9bc0ab710 0x7fe9bc145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bbfff640 1 -- 172.21.15.39:0/4211184009 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9bc152870 msgr2=0x7fe9bc0ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bbfff640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9bc152870 0x7fe9bc0ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bbfff640 1 -- 172.21.15.39:0/4211184009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9bc0ab710 msgr2=0x7fe9bc145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bbfff640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9bc0ab710 0x7fe9bc145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.452+0000 7fe9bbfff640 1 -- 172.21.15.39:0/4211184009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc14b030 con 0x7fe9a80047d0 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9bbfff640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9bc1eff90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00e7f0 tx=0x7fe9ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9b97fa640 1 -- 172.21.15.39:0/4211184009 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac00c850 con 0x7fe9a80047d0 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9b97fa640 1 -- 172.21.15.39:0/4211184009 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac00c9b0 con 0x7fe9a80047d0 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc1f08e0 con 0x7fe9a80047d0 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9b97fa640 1 -- 172.21.15.39:0/4211184009 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac00cd60 con 0x7fe9a80047d0 2024-09-13T17:10:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc156ba0 con 0x7fe9a80047d0 2024-09-13T17:10:25.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc0718b0 con 0x7fe9a80047d0 2024-09-13T17:10:25.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9b97fa640 1 -- 172.21.15.39:0/4211184009 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe9ac01d460 con 0x7fe9a80047d0 2024-09-13T17:10:25.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9b97fa640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe99c066690 0x7fe99c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9b97fa640 1 -- 172.21.15.39:0/4211184009 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7fe9ac084ee0 con 0x7fe9a80047d0 2024-09-13T17:10:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.456+0000 7fe9b37fe640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe99c066690 0x7fe99c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.460+0000 7fe9b37fe640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe99c066690 0x7fe99c068b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40ad000 tx=0x7fe9a40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:25.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.464+0000 7fe9b97fa640 1 -- 172.21.15.39:0/4211184009 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe9ac0514d0 con 0x7fe9a80047d0 2024-09-13T17:10:25.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.680+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9bc0aaf30 con 0x7fe9a80047d0 2024-09-13T17:10:25.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.680+0000 7fe9b97fa640 1 -- 172.21.15.39:0/4211184009 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5937 (secure 0 0 0) 0x7fe9ac054af0 con 0x7fe9a80047d0 2024-09-13T17:10:25.684 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:25.684 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:25.684 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-13T17:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe99c066690 msgr2=0x7fe99c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe99c066690 0x7fe99c068b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40ad000 tx=0x7fe9a40046d0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 msgr2=0x7fe9bc1eff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9bc1eff90 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00e7f0 tx=0x7fe9ac00ecb0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 shutdown_connections 2024-09-13T17:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe99c066690 0x7fe99c068b50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9bc0ab710 0x7fe9bc145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe9bc152870 0x7fe9bc0ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 --2- 172.21.15.39:0/4211184009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9a80047d0 0x7fe9bc1eff90 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 >> 172.21.15.39:0/4211184009 conn(0x7fe9bc06a210 msgr2=0x7fe9bc156400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:25.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 shutdown_connections 2024-09-13T17:10:25.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.684+0000 7fe9c2311640 1 -- 172.21.15.39:0/4211184009 wait complete. 2024-09-13T17:10:25.704 INFO:tasks.mds_thrash.fs.[b]:kill mds.a (rank=0) 2024-09-13T17:10:25.705 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:10:25.705 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:10:25.716 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:10:25.716 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:25.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.872+0000 7f85a75d2640 1 Processor -- start 2024-09-13T17:10:25.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.872+0000 7f85a75d2640 1 -- start start 2024-09-13T17:10:25.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a75d2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f85a014a770 0x7f85a0150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a75d2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85a014ab50 0x7f85a00713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a75d2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a00c4760 con 0x7f85a00c6060 2024-09-13T17:10:25.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a75d2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a006df80 con 0x7f85a014ab50 2024-09-13T17:10:25.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a75d2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a00c1950 con 0x7f85a014a770 2024-09-13T17:10:25.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a5b6f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85a014ab50 0x7f85a00713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a5b6f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85a014ab50 0x7f85a00713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56060/0 (socket says 172.21.15.39:56060) 2024-09-13T17:10:25.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a5b6f640 1 -- 172.21.15.39:0/1278079884 learned_addr learned my addr 172.21.15.39:0/1278079884 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:25.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a536e640 1 --2- 172.21.15.39:0/1278079884 >> v2:172.21.15.84:3300/0 conn(0x7f85a014a770 0x7f85a0150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a5b6f640 1 -- 172.21.15.39:0/1278079884 >> v2:172.21.15.84:3300/0 conn(0x7f85a014a770 msgr2=0x7f85a0150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:25.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a5b6f640 1 --2- 172.21.15.39:0/1278079884 >> v2:172.21.15.84:3300/0 conn(0x7f85a014a770 0x7f85a0150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a5b6f640 1 -- 172.21.15.39:0/1278079884 >> v1:172.21.15.84:6789/0 conn(0x7f85a00c6060 legacy=0x7f85a006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:25.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a5b6f640 1 -- 172.21.15.39:0/1278079884 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a014afa0 con 0x7f85a014ab50 2024-09-13T17:10:25.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.876+0000 7f85a5b6f640 1 --2- 172.21.15.39:0/1278079884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014ab50 0x7f85a00713e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f85a00c3270 tx=0x7f859801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa79655c85fc892 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:25.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f8597fff640 1 -- 172.21.15.39:0/1278079884 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85980a57b0 con 0x7f85a014ab50 2024-09-13T17:10:25.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f8597fff640 1 -- 172.21.15.39:0/1278079884 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85980a5910 con 0x7f85a014ab50 2024-09-13T17:10:25.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f8597fff640 1 -- 172.21.15.39:0/1278079884 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85980a5ca0 con 0x7f85a014ab50 2024-09-13T17:10:25.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 -- 172.21.15.39:0/1278079884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014ab50 msgr2=0x7f85a00713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 --2- 172.21.15.39:0/1278079884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014ab50 0x7f85a00713e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f85a00c3270 tx=0x7f859801c8f0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 -- 172.21.15.39:0/1278079884 shutdown_connections 2024-09-13T17:10:25.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 --2- 172.21.15.39:0/1278079884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014ab50 0x7f85a00713e0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 --2- 172.21.15.39:0/1278079884 >> v2:172.21.15.84:3300/0 conn(0x7f85a014a770 0x7f85a0150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 -- 172.21.15.39:0/1278079884 >> 172.21.15.39:0/1278079884 conn(0x7f85a00c32a0 msgr2=0x7f85a00c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:25.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 -- 172.21.15.39:0/1278079884 shutdown_connections 2024-09-13T17:10:25.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 -- 172.21.15.39:0/1278079884 wait complete. 2024-09-13T17:10:25.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 Processor -- start 2024-09-13T17:10:25.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 -- start start 2024-09-13T17:10:25.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85a00c6060 0x7f85a00bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014a770 0x7f85a0149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85a014ab50 0x7f85a014a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00bfee0 con 0x7f85a014a770 2024-09-13T17:10:25.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a015b350 con 0x7f85a00c6060 2024-09-13T17:10:25.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a75d2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a014a5a0 con 0x7f85a014ab50 2024-09-13T17:10:25.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a4b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014a770 0x7f85a0149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a4b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014a770 0x7f85a0149b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56076/0 (socket says 172.21.15.39:56076) 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a4b6d640 1 -- 172.21.15.39:0/2066601227 learned_addr learned my addr 172.21.15.39:0/2066601227 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a536e640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85a00c6060 0x7f85a00bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a5b6f640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85a014ab50 0x7f85a014a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a4b6d640 1 -- 172.21.15.39:0/2066601227 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85a014ab50 msgr2=0x7f85a014a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a4b6d640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85a014ab50 0x7f85a014a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a4b6d640 1 -- 172.21.15.39:0/2066601227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85a00c6060 msgr2=0x7f85a00bf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a4b6d640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85a00c6060 0x7f85a00bf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.880+0000 7f85a4b6d640 1 -- 172.21.15.39:0/2066601227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8598009b30 con 0x7f85a014a770 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f85a4b6d640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014a770 0x7f85a0149b20 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f859000b410 tx=0x7f859000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f8595ffb640 1 -- 172.21.15.39:0/2066601227 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590004020 con 0x7f85a014a770 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f8595ffb640 1 -- 172.21.15.39:0/2066601227 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8590004180 con 0x7f85a014a770 2024-09-13T17:10:25.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f8595ffb640 1 -- 172.21.15.39:0/2066601227 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590011630 con 0x7f85a014a770 2024-09-13T17:10:25.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a015aa20 con 0x7f85a014a770 2024-09-13T17:10:25.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a015afe0 con 0x7f85a014a770 2024-09-13T17:10:25.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a00619c0 con 0x7f85a014a770 2024-09-13T17:10:25.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f8595ffb640 1 -- 172.21.15.39:0/2066601227 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8590011790 con 0x7f85a014a770 2024-09-13T17:10:25.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f8595ffb640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8584066690 0x7f8584068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:25.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f8595ffb640 1 -- 172.21.15.39:0/2066601227 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7f8590084ce0 con 0x7f85a014a770 2024-09-13T17:10:25.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f85a536e640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8584066690 0x7f8584068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:25.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.884+0000 7f85a536e640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8584066690 0x7f8584068b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f859c0079c0 tx=0x7f859c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:25.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:25.888+0000 7f8595ffb640 1 -- 172.21.15.39:0/2066601227 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8590051240 con 0x7f85a014a770 2024-09-13T17:10:26.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.128+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85a015b4c0 con 0x7f85a014a770 2024-09-13T17:10:26.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.128+0000 7f8595ffb640 1 -- 172.21.15.39:0/2066601227 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f8590015090 con 0x7f85a014a770 2024-09-13T17:10:26.132 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:26.132 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:26.132 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-13T17:10:26.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8584066690 msgr2=0x7f8584068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:26.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8584066690 0x7f8584068b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f859c0079c0 tx=0x7f859c008040 comp rx=0 tx=0).stop 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014a770 msgr2=0x7f85a0149b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014a770 0x7f85a0149b20 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f859000b410 tx=0x7f859000b8d0 comp rx=0 tx=0).stop 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 shutdown_connections 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8584066690 0x7f8584068b50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f85a014ab50 0x7f85a014a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85a014a770 0x7f85a0149b20 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 --2- 172.21.15.39:0/2066601227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85a00c6060 0x7f85a00bf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 >> 172.21.15.39:0/2066601227 conn(0x7f85a00c32a0 msgr2=0x7f85a00c05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 shutdown_connections 2024-09-13T17:10:26.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:26.132+0000 7f85a75d2640 1 -- 172.21.15.39:0/2066601227 wait complete. 2024-09-13T17:10:26.150 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:10:28.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f4005ec0640 1 Processor -- start 2024-09-13T17:10:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f4005ec0640 1 -- start start 2024-09-13T17:10:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f4005ec0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 0x7f400014f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f4005ec0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f40000c2ef0 0x7f40000c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f4005ec0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400006ddf0 con 0x7f40000c0a90 2024-09-13T17:10:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f4005ec0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40000c53d0 con 0x7f40000c2ef0 2024-09-13T17:10:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f4005ec0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40000c5950 con 0x7f4000065ee0 2024-09-13T17:10:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f3ffffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f40000c2ef0 0x7f40000c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f3ffffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f40000c2ef0 0x7f40000c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56082/0 (socket says 172.21.15.39:56082) 2024-09-13T17:10:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f3ffffff640 1 -- 172.21.15.39:0/692782865 learned_addr learned my addr 172.21.15.39:0/692782865 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.280+0000 7f3fff7fe640 1 --2- 172.21.15.39:0/692782865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 0x7f400014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f3ffffff640 1 -- 172.21.15.39:0/692782865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 msgr2=0x7f400014f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f3ffffff640 1 --2- 172.21.15.39:0/692782865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 0x7f400014f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f3ffffff640 1 -- 172.21.15.39:0/692782865 >> v1:172.21.15.84:6789/0 conn(0x7f40000c0a90 legacy=0x7f40000c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f3ffffff640 1 -- 172.21.15.39:0/692782865 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400006e040 con 0x7f40000c2ef0 2024-09-13T17:10:28.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f3ffffff640 1 --2- 172.21.15.39:0/692782865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c2ef0 0x7f40000c24b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009e50 tx=0x7f3ff801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a8883bc26f8124 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:28.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f3ffe7fc640 1 -- 172.21.15.39:0/692782865 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80a58d0 con 0x7f40000c2ef0 2024-09-13T17:10:28.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f3ffe7fc640 1 -- 172.21.15.39:0/692782865 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff80a5a30 con 0x7f40000c2ef0 2024-09-13T17:10:28.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f3ffe7fc640 1 -- 172.21.15.39:0/692782865 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80a5dc0 con 0x7f40000c2ef0 2024-09-13T17:10:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f4005ec0640 1 -- 172.21.15.39:0/692782865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c2ef0 msgr2=0x7f40000c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f4005ec0640 1 --2- 172.21.15.39:0/692782865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c2ef0 0x7f40000c24b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009e50 tx=0x7f3ff801ca00 comp rx=0 tx=0).stop 2024-09-13T17:10:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f4005ec0640 1 -- 172.21.15.39:0/692782865 shutdown_connections 2024-09-13T17:10:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f4005ec0640 1 --2- 172.21.15.39:0/692782865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c2ef0 0x7f40000c24b0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f4005ec0640 1 --2- 172.21.15.39:0/692782865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 0x7f400014f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f4005ec0640 1 -- 172.21.15.39:0/692782865 >> 172.21.15.39:0/692782865 conn(0x7f40000c3310 msgr2=0x7f40000c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f4005ec0640 1 -- 172.21.15.39:0/692782865 shutdown_connections 2024-09-13T17:10:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.284+0000 7f4005ec0640 1 -- 172.21.15.39:0/692782865 wait complete. 2024-09-13T17:10:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 Processor -- start 2024-09-13T17:10:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 -- start start 2024-09-13T17:10:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 0x7f40001eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:28.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c0a90 0x7f400014ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40000c2ef0 0x7f40000a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40001ea820 con 0x7f40000c0a90 2024-09-13T17:10:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40001ea210 con 0x7f40000c2ef0 2024-09-13T17:10:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4000151b40 con 0x7f4000065ee0 2024-09-13T17:10:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c0a90 0x7f400014ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c0a90 0x7f400014ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56086/0 (socket says 172.21.15.39:56086) 2024-09-13T17:10:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffeffd640 1 -- 172.21.15.39:0/780545858 learned_addr learned my addr 172.21.15.39:0/780545858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3fff7fe640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 0x7f40001eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffffff640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40000c2ef0 0x7f40000a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffeffd640 1 -- 172.21.15.39:0/780545858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 msgr2=0x7f40001eacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffeffd640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 0x7f40001eacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffeffd640 1 -- 172.21.15.39:0/780545858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40000c2ef0 msgr2=0x7f40000a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffeffd640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40000c2ef0 0x7f40000a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffeffd640 1 -- 172.21.15.39:0/780545858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff8009b30 con 0x7f40000c0a90 2024-09-13T17:10:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffffff640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40000c2ef0 0x7f40000a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:10:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3ffeffd640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c0a90 0x7f400014ccc0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c4d0 tx=0x7f3ff000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3fe7fff640 1 -- 172.21.15.39:0/780545858 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00052d0 con 0x7f40000c0a90 2024-09-13T17:10:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3fe7fff640 1 -- 172.21.15.39:0/780545858 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0005430 con 0x7f40000c0a90 2024-09-13T17:10:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f3fe7fff640 1 -- 172.21.15.39:0/780545858 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0012830 con 0x7f40000c0a90 2024-09-13T17:10:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40001ea4b0 con 0x7f40000c0a90 2024-09-13T17:10:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4000144460 con 0x7f40000c0a90 2024-09-13T17:10:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.288+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4000144680 con 0x7f40000c0a90 2024-09-13T17:10:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.292+0000 7f3fe7fff640 1 -- 172.21.15.39:0/780545858 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ff0012990 con 0x7f40000c0a90 2024-09-13T17:10:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.292+0000 7f3fe7fff640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fec066690 0x7f3fec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.292+0000 7f3fe7fff640 1 -- 172.21.15.39:0/780545858 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7f3ff0084a40 con 0x7f40000c0a90 2024-09-13T17:10:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.292+0000 7f3fff7fe640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fec066690 0x7f3fec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.292+0000 7f3fff7fe640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fec066690 0x7f3fec068b50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400b910 tx=0x7f3ff4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.296+0000 7f3fe7fff640 1 -- 172.21.15.39:0/780545858 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4000144680 con 0x7f40000c0a90 2024-09-13T17:10:28.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.520+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4000144680 con 0x7f40000c0a90 2024-09-13T17:10:28.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.520+0000 7f3fe7fff640 1 -- 172.21.15.39:0/780545858 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f4000144680 con 0x7f40000c0a90 2024-09-13T17:10:28.523 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:28.523 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:28.524 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-13T17:10:28.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fec066690 msgr2=0x7f3fec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:28.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fec066690 0x7f3fec068b50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400b910 tx=0x7f3ff4008040 comp rx=0 tx=0).stop 2024-09-13T17:10:28.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c0a90 msgr2=0x7f400014ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:28.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c0a90 0x7f400014ccc0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c4d0 tx=0x7f3ff000c990 comp rx=0 tx=0).stop 2024-09-13T17:10:28.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 shutdown_connections 2024-09-13T17:10:28.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fec066690 0x7f3fec068b50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:28.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40000c2ef0 0x7f40000a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:28.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40000c0a90 0x7f400014ccc0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:28.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 --2- 172.21.15.39:0/780545858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4000065ee0 0x7f40001eacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:28.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 >> 172.21.15.39:0/780545858 conn(0x7f40000c3310 msgr2=0x7f4000069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:28.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 shutdown_connections 2024-09-13T17:10:28.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:28.524+0000 7f4005ec0640 1 -- 172.21.15.39:0/780545858 wait complete. 2024-09-13T17:10:28.546 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:10:30.361 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:10:30.364 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:10:30.546 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:30.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44f959640 1 Processor -- start 2024-09-13T17:10:30.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44f959640 1 -- start start 2024-09-13T17:10:30.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44f959640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe448151960 0x7fe4480c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:30.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44f959640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe4480c6820 0x7fe448157f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:30.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44f959640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe448154ed0 con 0x7fe4480c7a90 2024-09-13T17:10:30.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44f959640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4480c51f0 con 0x7fe4480c6820 2024-09-13T17:10:30.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44f959640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe448158490 con 0x7fe448151960 2024-09-13T17:10:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44cef4640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe4480c7a90 0x7fe448157840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52922/0 (socket says 172.21.15.39:52922) 2024-09-13T17:10:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44cef4640 1 -- 172.21.15.39:0/2988123401 learned_addr learned my addr 172.21.15.39:0/2988123401 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:10:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44def6640 1 --2- 172.21.15.39:0/2988123401 >> v2:172.21.15.39:3300/0 conn(0x7fe4480c6820 0x7fe448157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44d6f5640 1 --2- 172.21.15.39:0/2988123401 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe448151960 0x7fe4480c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44def6640 1 -- 172.21.15.39:0/2988123401 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe448151960 msgr2=0x7fe4480c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44def6640 1 --2- 172.21.15.39:0/2988123401 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe448151960 0x7fe4480c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44def6640 1 -- 172.21.15.39:0/2988123401 >> v1:172.21.15.39:6789/0 conn(0x7fe4480c7a90 legacy=0x7fe448157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.692+0000 7fe44def6640 1 -- 172.21.15.39:0/2988123401 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4480c7330 con 0x7fe4480c6820 2024-09-13T17:10:30.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44def6640 1 --2- 172.21.15.39:0/2988123401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c6820 0x7fe448157f50 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe440009df0 tx=0x7fe44001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6e455fa363b3d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:30.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe43ffff640 1 -- 172.21.15.39:0/2988123401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400a5870 con 0x7fe4480c6820 2024-09-13T17:10:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe43ffff640 1 -- 172.21.15.39:0/2988123401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4400a59d0 con 0x7fe4480c6820 2024-09-13T17:10:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe43ffff640 1 -- 172.21.15.39:0/2988123401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400a5d60 con 0x7fe4480c6820 2024-09-13T17:10:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 -- 172.21.15.39:0/2988123401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c6820 msgr2=0x7fe448157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 --2- 172.21.15.39:0/2988123401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c6820 0x7fe448157f50 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe440009df0 tx=0x7fe44001c8c0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 -- 172.21.15.39:0/2988123401 shutdown_connections 2024-09-13T17:10:30.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 --2- 172.21.15.39:0/2988123401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c6820 0x7fe448157f50 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 --2- 172.21.15.39:0/2988123401 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe448151960 0x7fe4480c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 -- 172.21.15.39:0/2988123401 >> 172.21.15.39:0/2988123401 conn(0x7fe4480ab500 msgr2=0x7fe4480ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:30.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 -- 172.21.15.39:0/2988123401 shutdown_connections 2024-09-13T17:10:30.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 -- 172.21.15.39:0/2988123401 wait complete. 2024-09-13T17:10:30.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 Processor -- start 2024-09-13T17:10:30.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.696+0000 7fe44f959640 1 -- start start 2024-09-13T17:10:30.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44f959640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4480c6820 0x7fe4480aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:30.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44f959640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c7a90 0x7fe4480c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:30.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44f959640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe448151960 0x7fe44814cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44f959640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4480c1580 con 0x7fe4480c7a90 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44f959640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4480aa610 con 0x7fe448151960 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44f959640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4480aa840 con 0x7fe4480c6820 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44cef4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c7a90 0x7fe4480c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44cef4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c7a90 0x7fe4480c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56904/0 (socket says 172.21.15.39:56904) 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44cef4640 1 -- 172.21.15.39:0/1875074771 learned_addr learned my addr 172.21.15.39:0/1875074771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44def6640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe448151960 0x7fe44814cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44d6f5640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4480c6820 0x7fe4480aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44cef4640 1 -- 172.21.15.39:0/1875074771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4480c6820 msgr2=0x7fe4480aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44cef4640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4480c6820 0x7fe4480aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44cef4640 1 -- 172.21.15.39:0/1875074771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe448151960 msgr2=0x7fe44814cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44cef4640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe448151960 0x7fe44814cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44cef4640 1 -- 172.21.15.39:0/1875074771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe440009a70 con 0x7fe4480c7a90 2024-09-13T17:10:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44cef4640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c7a90 0x7fe4480c20e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe438001ec0 tx=0x7fe43800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:30.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe43dffb640 1 -- 172.21.15.39:0/1875074771 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438015e30 con 0x7fe4480c7a90 2024-09-13T17:10:30.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe43dffb640 1 -- 172.21.15.39:0/1875074771 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4380058c0 con 0x7fe4480c7a90 2024-09-13T17:10:30.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4480c0be0 con 0x7fe4480c7a90 2024-09-13T17:10:30.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe43dffb640 1 -- 172.21.15.39:0/1875074771 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438005c50 con 0x7fe4480c7a90 2024-09-13T17:10:30.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4480c11a0 con 0x7fe4480c7a90 2024-09-13T17:10:30.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.700+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4480c16c0 con 0x7fe4480c7a90 2024-09-13T17:10:30.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.704+0000 7fe43dffb640 1 -- 172.21.15.39:0/1875074771 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe43801b070 con 0x7fe4480c7a90 2024-09-13T17:10:30.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.704+0000 7fe43dffb640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe42c066690 0x7fe42c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:30.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.704+0000 7fe43dffb640 1 -- 172.21.15.39:0/1875074771 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7fe43808cc70 con 0x7fe4480c7a90 2024-09-13T17:10:30.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.704+0000 7fe44d6f5640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe42c066690 0x7fe42c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:30.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.704+0000 7fe44d6f5640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe42c066690 0x7fe42c068b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe44400b9d0 tx=0x7fe444008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:30.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.712+0000 7fe43dffb640 1 -- 172.21.15.39:0/1875074771 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe4480c16c0 con 0x7fe4480c7a90 2024-09-13T17:10:30.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4480c16c0 con 0x7fe4480c7a90 2024-09-13T17:10:30.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe43dffb640 1 -- 172.21.15.39:0/1875074771 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5937 (secure 0 0 0) 0x7fe4480c16c0 con 0x7fe4480c7a90 2024-09-13T17:10:30.945 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:30.946 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:30.946 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-13T17:10:30.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe42c066690 msgr2=0x7fe42c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe42c066690 0x7fe42c068b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe44400b9d0 tx=0x7fe444008040 comp rx=0 tx=0).stop 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c7a90 msgr2=0x7fe4480c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c7a90 0x7fe4480c20e0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe438001ec0 tx=0x7fe43800c960 comp rx=0 tx=0).stop 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 shutdown_connections 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe42c066690 0x7fe42c068b50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe448151960 0x7fe44814cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4480c7a90 0x7fe4480c20e0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 --2- 172.21.15.39:0/1875074771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe4480c6820 0x7fe4480aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 >> 172.21.15.39:0/1875074771 conn(0x7fe4480ab500 msgr2=0x7fe448157740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 shutdown_connections 2024-09-13T17:10:30.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:30.944+0000 7fe44f959640 1 -- 172.21.15.39:0/1875074771 wait complete. 2024-09-13T17:10:30.965 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:10:32.966 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7eda63640 1 Processor -- start 2024-09-13T17:10:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7eda63640 1 -- start start 2024-09-13T17:10:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7eda63640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 0x7fe7e8157b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7eda63640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7e806cf80 0x7fe7e8153a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7eda63640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7e8072310 0x7fe7e80c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7eda63640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e80c0480 con 0x7fe7e806cf80 2024-09-13T17:10:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7eda63640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e80c0760 con 0x7fe7e8072310 2024-09-13T17:10:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7eda63640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e8157030 con 0x7fe7e8071660 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7e7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7e806cf80 0x7fe7e8153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7e7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7e806cf80 0x7fe7e8153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56918/0 (socket says 172.21.15.39:56918) 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7e7fff640 1 -- 172.21.15.39:0/2434839118 learned_addr learned my addr 172.21.15.39:0/2434839118 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7eca61640 1 --2- 172.21.15.39:0/2434839118 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 0x7fe7e8157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7ed262640 1 --2- 172.21.15.39:0/2434839118 >> v2:172.21.15.84:3300/0 conn(0x7fe7e8072310 0x7fe7e80c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7e7fff640 1 -- 172.21.15.39:0/2434839118 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 msgr2=0x7fe7e8157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7e7fff640 1 --2- 172.21.15.39:0/2434839118 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 0x7fe7e8157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7e7fff640 1 -- 172.21.15.39:0/2434839118 >> v2:172.21.15.84:3300/0 conn(0x7fe7e8072310 msgr2=0x7fe7e80c0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7e7fff640 1 --2- 172.21.15.39:0/2434839118 >> v2:172.21.15.84:3300/0 conn(0x7fe7e8072310 0x7fe7e80c0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7e7fff640 1 -- 172.21.15.39:0/2434839118 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e8157310 con 0x7fe7e806cf80 2024-09-13T17:10:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.100+0000 7fe7ed262640 1 --2- 172.21.15.39:0/2434839118 >> v2:172.21.15.84:3300/0 conn(0x7fe7e8072310 0x7fe7e80c0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:10:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7e7fff640 1 --2- 172.21.15.39:0/2434839118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e806cf80 0x7fe7e8153a40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80c0060 tx=0x7fe7d401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263ccd272a8eeccb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7e77fe640 1 -- 172.21.15.39:0/2434839118 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40ab640 con 0x7fe7e806cf80 2024-09-13T17:10:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7e77fe640 1 -- 172.21.15.39:0/2434839118 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d40a7070 con 0x7fe7e806cf80 2024-09-13T17:10:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7e77fe640 1 -- 172.21.15.39:0/2434839118 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40ab9c0 con 0x7fe7e806cf80 2024-09-13T17:10:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 -- 172.21.15.39:0/2434839118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e806cf80 msgr2=0x7fe7e8153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 --2- 172.21.15.39:0/2434839118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e806cf80 0x7fe7e8153a40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80c0060 tx=0x7fe7d401c460 comp rx=0 tx=0).stop 2024-09-13T17:10:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 -- 172.21.15.39:0/2434839118 shutdown_connections 2024-09-13T17:10:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 --2- 172.21.15.39:0/2434839118 >> v2:172.21.15.84:3300/0 conn(0x7fe7e8072310 0x7fe7e80c0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 --2- 172.21.15.39:0/2434839118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e806cf80 0x7fe7e8153a40 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 --2- 172.21.15.39:0/2434839118 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 0x7fe7e8157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 -- 172.21.15.39:0/2434839118 >> 172.21.15.39:0/2434839118 conn(0x7fe7e80c30c0 msgr2=0x7fe7e80c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 -- 172.21.15.39:0/2434839118 shutdown_connections 2024-09-13T17:10:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 -- 172.21.15.39:0/2434839118 wait complete. 2024-09-13T17:10:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 Processor -- start 2024-09-13T17:10:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 -- start start 2024-09-13T17:10:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806cf80 0x7fe7e80abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 0x7fe7e814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e8072310 0x7fe7e813cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e81f06b0 con 0x7fe7e8072310 2024-09-13T17:10:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e81504f0 con 0x7fe7e806cf80 2024-09-13T17:10:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eda63640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e81f0ad0 con 0x7fe7e8071660 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7ed262640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e8072310 0x7fe7e813cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7ed262640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e8072310 0x7fe7e813cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56924/0 (socket says 172.21.15.39:56924) 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7ed262640 1 -- 172.21.15.39:0/1440931499 learned_addr learned my addr 172.21.15.39:0/1440931499 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7eca61640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806cf80 0x7fe7e80abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.104+0000 7fe7e7fff640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 0x7fe7e814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7ed262640 1 -- 172.21.15.39:0/1440931499 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 msgr2=0x7fe7e814cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7ed262640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 0x7fe7e814cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7ed262640 1 -- 172.21.15.39:0/1440931499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806cf80 msgr2=0x7fe7e80abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7ed262640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806cf80 0x7fe7e80abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7ed262640 1 -- 172.21.15.39:0/1440931499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d40094d0 con 0x7fe7e8072310 2024-09-13T17:10:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7ed262640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e8072310 0x7fe7e813cc40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000cdb0 tx=0x7fe7e000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:33.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7e57fa640 1 -- 172.21.15.39:0/1440931499 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e000f680 con 0x7fe7e8072310 2024-09-13T17:10:33.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7e57fa640 1 -- 172.21.15.39:0/1440931499 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e000f800 con 0x7fe7e8072310 2024-09-13T17:10:33.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e81efd10 con 0x7fe7e8072310 2024-09-13T17:10:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7e57fa640 1 -- 172.21.15.39:0/1440931499 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e000fbb0 con 0x7fe7e8072310 2024-09-13T17:10:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e81f02d0 con 0x7fe7e8072310 2024-09-13T17:10:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e81f07f0 con 0x7fe7e8072310 2024-09-13T17:10:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7e57fa640 1 -- 172.21.15.39:0/1440931499 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7e000b040 con 0x7fe7e8072310 2024-09-13T17:10:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7e57fa640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe7cc0666c0 0x7fe7cc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7e57fa640 1 -- 172.21.15.39:0/1440931499 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7fe7e00856f0 con 0x7fe7e8072310 2024-09-13T17:10:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7eca61640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe7cc0666c0 0x7fe7cc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.108+0000 7fe7eca61640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe7cc0666c0 0x7fe7cc068b80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0045f0 tx=0x7fe7dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.116+0000 7fe7e57fa640 1 -- 172.21.15.39:0/1440931499 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe7e81f07f0 con 0x7fe7e8072310 2024-09-13T17:10:33.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.328+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7e81f07f0 con 0x7fe7e8072310 2024-09-13T17:10:33.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7e57fa640 1 -- 172.21.15.39:0/1440931499 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5937 (secure 0 0 0) 0x7fe7e81f07f0 con 0x7fe7e8072310 2024-09-13T17:10:33.333 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:33.333 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-13T17:10:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe7cc0666c0 msgr2=0x7fe7cc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe7cc0666c0 0x7fe7cc068b80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0045f0 tx=0x7fe7dc008040 comp rx=0 tx=0).stop 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e8072310 msgr2=0x7fe7e813cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e8072310 0x7fe7e813cc40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000cdb0 tx=0x7fe7e000a650 comp rx=0 tx=0).stop 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 shutdown_connections 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe7cc0666c0 0x7fe7cc068b80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7e8072310 0x7fe7e813cc40 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe7e8071660 0x7fe7e814cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 --2- 172.21.15.39:0/1440931499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7e806cf80 0x7fe7e80abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 >> 172.21.15.39:0/1440931499 conn(0x7fe7e80c30c0 msgr2=0x7fe7e80c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 shutdown_connections 2024-09-13T17:10:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:33.332+0000 7fe7eda63640 1 -- 172.21.15.39:0/1440931499 wait complete. 2024-09-13T17:10:33.355 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:10:35.355 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:35.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa495775640 1 Processor -- start 2024-09-13T17:10:35.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa495775640 1 -- start start 2024-09-13T17:10:35.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa495775640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa4900c1f30 0x7fa4901523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:35.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa495775640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c2790 0x7fa4900c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa495775640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4900ac5c0 con 0x7fa4900c3c10 2024-09-13T17:10:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa495775640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4900ac840 con 0x7fa4900c1f30 2024-09-13T17:10:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa495775640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900acb20 con 0x7fa4900c2790 2024-09-13T17:10:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa48f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa4900c1f30 0x7fa4901523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa48f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa4900c1f30 0x7fa4901523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56934/0 (socket says 172.21.15.39:56934) 2024-09-13T17:10:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa48f7fe640 1 -- 172.21.15.39:0/3663055461 learned_addr learned my addr 172.21.15.39:0/3663055461 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa494f74640 1 --2- 172.21.15.39:0/3663055461 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c2790 0x7fa4900c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa48f7fe640 1 -- 172.21.15.39:0/3663055461 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c2790 msgr2=0x7fa4900c2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:35.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa48f7fe640 1 --2- 172.21.15.39:0/3663055461 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c2790 0x7fa4900c2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa48f7fe640 1 -- 172.21.15.39:0/3663055461 >> v1:172.21.15.84:6789/0 conn(0x7fa4900c3c10 legacy=0x7fa4900aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:35.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.496+0000 7fa48f7fe640 1 -- 172.21.15.39:0/3663055461 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4900ace00 con 0x7fa4900c1f30 2024-09-13T17:10:35.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa48f7fe640 1 --2- 172.21.15.39:0/3663055461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 0x7fa4901523a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa484009870 tx=0x7fa48401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22ffc4e8ffdde28 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:35.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa48effd640 1 -- 172.21.15.39:0/3663055461 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840a7070 con 0x7fa4900c1f30 2024-09-13T17:10:35.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa48effd640 1 -- 172.21.15.39:0/3663055461 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4840ab430 con 0x7fa4900c1f30 2024-09-13T17:10:35.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa48effd640 1 -- 172.21.15.39:0/3663055461 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4840ab7c0 con 0x7fa4900c1f30 2024-09-13T17:10:35.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa495775640 1 -- 172.21.15.39:0/3663055461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 msgr2=0x7fa4901523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:35.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa495775640 1 --2- 172.21.15.39:0/3663055461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 0x7fa4901523a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa484009870 tx=0x7fa48401c560 comp rx=0 tx=0).stop 2024-09-13T17:10:35.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa495775640 1 -- 172.21.15.39:0/3663055461 shutdown_connections 2024-09-13T17:10:35.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa495775640 1 --2- 172.21.15.39:0/3663055461 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c2790 0x7fa4900c2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa495775640 1 --2- 172.21.15.39:0/3663055461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 0x7fa4901523a0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa495775640 1 -- 172.21.15.39:0/3663055461 >> 172.21.15.39:0/3663055461 conn(0x7fa4900c4030 msgr2=0x7fa4900c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:35.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa495775640 1 -- 172.21.15.39:0/3663055461 shutdown_connections 2024-09-13T17:10:35.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa495775640 1 -- 172.21.15.39:0/3663055461 wait complete. 2024-09-13T17:10:35.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.500+0000 7fa495775640 1 Processor -- start 2024-09-13T17:10:35.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 -- start start 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 0x7fa4900c19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4900c2790 0x7fa4901ea810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c3c10 0x7fa4901eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901ea3f0 con 0x7fa4900c1f30 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901edd70 con 0x7fa4900c2790 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4901e9d50 con 0x7fa4900c3c10 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 0x7fa4900c19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 0x7fa4900c19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56940/0 (socket says 172.21.15.39:56940) 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48ffff640 1 -- 172.21.15.39:0/678541117 learned_addr learned my addr 172.21.15.39:0/678541117 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48f7fe640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4900c2790 0x7fa4901ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:35.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa494f74640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c3c10 0x7fa4901eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:35.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48ffff640 1 -- 172.21.15.39:0/678541117 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c3c10 msgr2=0x7fa4901eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:35.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48ffff640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c3c10 0x7fa4901eb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48ffff640 1 -- 172.21.15.39:0/678541117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4900c2790 msgr2=0x7fa4901ea810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:35.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48ffff640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4900c2790 0x7fa4901ea810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48ffff640 1 -- 172.21.15.39:0/678541117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840094d0 con 0x7fa4900c1f30 2024-09-13T17:10:35.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48f7fe640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4900c2790 0x7fa4901ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:10:35.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48ffff640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 0x7fa4900c19d0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c4e0 tx=0x7fa48000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:35.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48cff9640 1 -- 172.21.15.39:0/678541117 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4800052d0 con 0x7fa4900c1f30 2024-09-13T17:10:35.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48cff9640 1 -- 172.21.15.39:0/678541117 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa480005430 con 0x7fa4900c1f30 2024-09-13T17:10:35.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4901ee010 con 0x7fa4900c1f30 2024-09-13T17:10:35.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa48cff9640 1 -- 172.21.15.39:0/678541117 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480011a90 con 0x7fa4900c1f30 2024-09-13T17:10:35.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4901eb700 con 0x7fa4900c1f30 2024-09-13T17:10:35.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.504+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4901523a0 con 0x7fa4900c1f30 2024-09-13T17:10:35.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.508+0000 7fa48cff9640 1 -- 172.21.15.39:0/678541117 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa480011bf0 con 0x7fa4900c1f30 2024-09-13T17:10:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.508+0000 7fa48cff9640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa47c066690 0x7fa47c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.508+0000 7fa48f7fe640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa47c066690 0x7fa47c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.508+0000 7fa48cff9640 1 -- 172.21.15.39:0/678541117 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7fa480088920 con 0x7fa4900c1f30 2024-09-13T17:10:35.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.508+0000 7fa48f7fe640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa47c066690 0x7fa47c068b50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa4840af000 tx=0x7fa484008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.512+0000 7fa48cff9640 1 -- 172.21.15.39:0/678541117 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa480054f10 con 0x7fa4900c1f30 2024-09-13T17:10:35.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4900c6f70 con 0x7fa4900c1f30 2024-09-13T17:10:35.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa48cff9640 1 -- 172.21.15.39:0/678541117 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5937 (secure 0 0 0) 0x7fa480058530 con 0x7fa4900c1f30 2024-09-13T17:10:35.750 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:35.750 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:35.750 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-13T17:10:35.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa47c066690 msgr2=0x7fa47c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:35.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa47c066690 0x7fa47c068b50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa4840af000 tx=0x7fa484008040 comp rx=0 tx=0).stop 2024-09-13T17:10:35.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 msgr2=0x7fa4900c19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:35.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 0x7fa4900c19d0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fa48000c4e0 tx=0x7fa48000c9a0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 shutdown_connections 2024-09-13T17:10:35.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa47c066690 0x7fa47c068b50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4900c2790 0x7fa4901ea810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4900c3c10 0x7fa4901eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 --2- 172.21.15.39:0/678541117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4900c1f30 0x7fa4900c19d0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:35.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.748+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 >> 172.21.15.39:0/678541117 conn(0x7fa4900c4030 msgr2=0x7fa4900c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:35.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.752+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 shutdown_connections 2024-09-13T17:10:35.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:35.752+0000 7fa495775640 1 -- 172.21.15.39:0/678541117 wait complete. 2024-09-13T17:10:35.773 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:10:37.773 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:37.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f7082e7f640 1 Processor -- start 2024-09-13T17:10:37.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f7082e7f640 1 -- start start 2024-09-13T17:10:37.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f7082e7f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0c71a0 0x7f707c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:37.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f7082e7f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f707c0a71a0 0x7f707c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:37.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f7082e7f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707c072450 con 0x7f707c0c7580 2024-09-13T17:10:37.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f7082e7f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c06e6b0 con 0x7f707c0a71a0 2024-09-13T17:10:37.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f7082e7f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c154b20 con 0x7f707c0c71a0 2024-09-13T17:10:37.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f708141c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f707c0a71a0 0x7f707c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:37.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f708141c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f707c0a71a0 0x7f707c06ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56946/0 (socket says 172.21.15.39:56946) 2024-09-13T17:10:37.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f708141c640 1 -- 172.21.15.39:0/1780111957 learned_addr learned my addr 172.21.15.39:0/1780111957 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:37.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f7080c1b640 1 --2- 172.21.15.39:0/1780111957 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0c71a0 0x7f707c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:37.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f708141c640 1 -- 172.21.15.39:0/1780111957 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0c71a0 msgr2=0x7f707c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:37.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f708141c640 1 --2- 172.21.15.39:0/1780111957 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0c71a0 0x7f707c0c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:37.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f708141c640 1 -- 172.21.15.39:0/1780111957 >> v1:172.21.15.39:6789/0 conn(0x7f707c0c7580 legacy=0x7f707c071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:37.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.928+0000 7f708141c640 1 -- 172.21.15.39:0/1780111957 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c154e00 con 0x7f707c0a71a0 2024-09-13T17:10:37.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f708141c640 1 --2- 172.21.15.39:0/1780111957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0a71a0 0x7f707c06ede0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f7074009e50 tx=0x7f707401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264b1d3181a76f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:37.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f707b7fe640 1 -- 172.21.15.39:0/1780111957 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70740a5830 con 0x7f707c0a71a0 2024-09-13T17:10:37.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f707b7fe640 1 -- 172.21.15.39:0/1780111957 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70740a5990 con 0x7f707c0a71a0 2024-09-13T17:10:37.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f707b7fe640 1 -- 172.21.15.39:0/1780111957 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70740a5d20 con 0x7f707c0a71a0 2024-09-13T17:10:37.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f7082e7f640 1 -- 172.21.15.39:0/1780111957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0a71a0 msgr2=0x7f707c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:37.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f7082e7f640 1 --2- 172.21.15.39:0/1780111957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0a71a0 0x7f707c06ede0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f7074009e50 tx=0x7f707401c970 comp rx=0 tx=0).stop 2024-09-13T17:10:37.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f7082e7f640 1 -- 172.21.15.39:0/1780111957 shutdown_connections 2024-09-13T17:10:37.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f7082e7f640 1 --2- 172.21.15.39:0/1780111957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0a71a0 0x7f707c06ede0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:37.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f7082e7f640 1 --2- 172.21.15.39:0/1780111957 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0c71a0 0x7f707c0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:37.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f7082e7f640 1 -- 172.21.15.39:0/1780111957 >> 172.21.15.39:0/1780111957 conn(0x7f707c06c020 msgr2=0x7f707c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:37.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f7082e7f640 1 -- 172.21.15.39:0/1780111957 shutdown_connections 2024-09-13T17:10:37.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.932+0000 7f7082e7f640 1 -- 172.21.15.39:0/1780111957 wait complete. 2024-09-13T17:10:37.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 Processor -- start 2024-09-13T17:10:37.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 -- start start 2024-09-13T17:10:37.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0a71a0 0x7f707c0ac070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:37.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0c71a0 0x7f707c0c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:37.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f707c14b9d0 0x7f707c14be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:37.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c14c390 con 0x7f707c0c71a0 2024-09-13T17:10:37.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c0ab2b0 con 0x7f707c14b9d0 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1f07b0 con 0x7f707c0a71a0 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f707bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0c71a0 0x7f707c0c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f707bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0c71a0 0x7f707c0c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56958/0 (socket says 172.21.15.39:56958) 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f707bfff640 1 -- 172.21.15.39:0/899058515 learned_addr learned my addr 172.21.15.39:0/899058515 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7080c1b640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0a71a0 0x7f707c0ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f708141c640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f707c14b9d0 0x7f707c14be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f707bfff640 1 -- 172.21.15.39:0/899058515 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0a71a0 msgr2=0x7f707c0ac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f707bfff640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0a71a0 0x7f707c0ac070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f707bfff640 1 -- 172.21.15.39:0/899058515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f707c14b9d0 msgr2=0x7f707c14be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f707bfff640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f707c14b9d0 0x7f707c14be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:37.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f707bfff640 1 -- 172.21.15.39:0/899058515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7074009b30 con 0x7f707c0c71a0 2024-09-13T17:10:37.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f708141c640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f707c14b9d0 0x7f707c14be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:10:37.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f707bfff640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0c71a0 0x7f707c0c3920 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f706800c500 tx=0x7f706800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:37.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f70797fa640 1 -- 172.21.15.39:0/899058515 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068005220 con 0x7f707c0c71a0 2024-09-13T17:10:37.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f70797fa640 1 -- 172.21.15.39:0/899058515 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068005380 con 0x7f707c0c71a0 2024-09-13T17:10:37.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c1f0d30 con 0x7f707c0c71a0 2024-09-13T17:10:37.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f70797fa640 1 -- 172.21.15.39:0/899058515 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068007270 con 0x7f707c0c71a0 2024-09-13T17:10:37.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c0aac60 con 0x7f707c0c71a0 2024-09-13T17:10:37.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.936+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707c06ede0 con 0x7f707c0c71a0 2024-09-13T17:10:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.940+0000 7f70797fa640 1 -- 172.21.15.39:0/899058515 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f70680073d0 con 0x7f707c0c71a0 2024-09-13T17:10:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.940+0000 7f70797fa640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7060066690 0x7f7060068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.940+0000 7f70797fa640 1 -- 172.21.15.39:0/899058515 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 9217+0+0 (secure 0 0 0) 0x7f70680842b0 con 0x7f707c0c71a0 2024-09-13T17:10:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.940+0000 7f7080c1b640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7060066690 0x7f7060068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.940+0000 7f7080c1b640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7060066690 0x7f7060068b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f707c06b1f0 tx=0x7f7070008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:37.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:37.944+0000 7f70797fa640 1 -- 172.21.15.39:0/899058515 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7068050820 con 0x7f707c0c71a0 2024-09-13T17:10:38.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.200+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f707c0c0da0 con 0x7f707c0c71a0 2024-09-13T17:10:38.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f70797fa640 1 -- 172.21.15.39:0/899058515 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f7068053e40 con 0x7f707c0c71a0 2024-09-13T17:10:38.205 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:09:33.031201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":53,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7485":{"gid":7485,"name":"a","rank":0,"incarnation":147,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6835/1034701751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1034701751},{"type":"v1","addr":"172.21.15.39:6835","nonce":1034701751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:38.205 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-13T17:10:38.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7060066690 msgr2=0x7f7060068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:38.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7060066690 0x7f7060068b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f707c06b1f0 tx=0x7f7070008040 comp rx=0 tx=0).stop 2024-09-13T17:10:38.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0c71a0 msgr2=0x7f707c0c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0c71a0 0x7f707c0c3920 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f706800c500 tx=0x7f706800c9c0 comp rx=0 tx=0).stop 2024-09-13T17:10:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 shutdown_connections 2024-09-13T17:10:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7060066690 0x7f7060068b50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f707c14b9d0 0x7f707c14be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f707c0c71a0 0x7f707c0c3920 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 --2- 172.21.15.39:0/899058515 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f707c0a71a0 0x7f707c0ac070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 >> 172.21.15.39:0/899058515 conn(0x7f707c06c020 msgr2=0x7f707c15e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 shutdown_connections 2024-09-13T17:10:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:38.204+0000 7f7082e7f640 1 -- 172.21.15.39:0/899058515 wait complete. 2024-09-13T17:10:38.226 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:10:40.226 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.388+0000 7f69dd48d640 1 Processor -- start 2024-09-13T17:10:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.388+0000 7f69dd48d640 1 -- start start 2024-09-13T17:10:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.388+0000 7f69dd48d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f69d80c1730 0x7f69d80a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.388+0000 7f69dd48d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d80c1b10 0x7f69d80c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.392+0000 7f69dd48d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d80c03a0 con 0x7f69d80c4ef0 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.392+0000 7f69dd48d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d80c0d70 con 0x7f69d80c1730 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.392+0000 7f69dd48d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80c41b0 con 0x7f69d80c1b10 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.392+0000 7f69d77fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f69d80c4ef0 0x7f69d806b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:39570/0 (socket says 172.21.15.39:39570) 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.392+0000 7f69d77fe640 1 -- 172.21.15.39:0/3191662270 learned_addr learned my addr 172.21.15.39:0/3191662270 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.392+0000 7f69d6ffd640 1 --2- 172.21.15.39:0/3191662270 >> v2:172.21.15.84:3300/0 conn(0x7f69d80c1730 0x7f69d80a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.392+0000 7f69cffff640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d80c1b10 0x7f69d80c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.392+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393260625 0 0) 0x7f69d80c03a0 con 0x7f69d80c4ef0 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c4003510 con 0x7f69d80c4ef0 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3777779841 0 0) 0x7f69c4003510 con 0x7f69d80c4ef0 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d80c1b10 msgr2=0x7f69d80c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:40.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d80c1b10 0x7f69d80c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 >> v2:172.21.15.84:3300/0 conn(0x7f69d80c1730 msgr2=0x7f69d80a89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 --2- 172.21.15.39:0/3191662270 >> v2:172.21.15.84:3300/0 conn(0x7f69d80c1730 0x7f69d80a89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80c4490 con 0x7f69d80c4ef0 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f69d0001a00 con 0x7f69d80c4ef0 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 >> v1:172.21.15.39:6789/0 conn(0x7f69d80c4ef0 legacy=0x7f69d806b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d80c1b10 0x7f69c4005cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69c4006230 0x7f69c4008620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c4008c00 0x7f69c400b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80c0d70 con 0x7f69c4008c00 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80c41b0 con 0x7f69c4006230 2024-09-13T17:10:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c400b930 con 0x7f69d80c1b10 2024-09-13T17:10:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d77fe640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c4008c00 0x7f69c400b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d6ffd640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d80c1b10 0x7f69c4005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69cffff640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69c4006230 0x7f69c4008620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d77fe640 1 -- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d80c1b10 msgr2=0x7f69c4005cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d77fe640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d80c1b10 0x7f69c4005cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d77fe640 1 -- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69c4006230 msgr2=0x7f69c4008620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d77fe640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69c4006230 0x7f69c4008620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.456+0000 7f69d77fe640 1 -- 172.21.15.39:0/3191662270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c4003510 con 0x7f69c4008c00 2024-09-13T17:10:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69d77fe640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c4008c00 0x7f69c400b070 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f69d80a54f0 tx=0x7f69d001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ca494531fa4187 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d00ad630 con 0x7f69c4008c00 2024-09-13T17:10:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d00a8070 con 0x7f69c4008c00 2024-09-13T17:10:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69d67fc640 1 -- 172.21.15.39:0/3191662270 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d00ad9d0 con 0x7f69c4008c00 2024-09-13T17:10:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69dd48d640 1 -- 172.21.15.39:0/3191662270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c4008c00 msgr2=0x7f69c400b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69dd48d640 1 --2- 172.21.15.39:0/3191662270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c4008c00 0x7f69c400b070 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f69d80a54f0 tx=0x7f69d001d3c0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69d77fe640 1 -- 172.21.15.39:0/3191662270 reap_dead start 2024-09-13T17:10:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69dd48d640 1 -- 172.21.15.39:0/3191662270 shutdown_connections 2024-09-13T17:10:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69dd48d640 1 -- 172.21.15.39:0/3191662270 >> 172.21.15.39:0/3191662270 conn(0x7f69d80a75b0 msgr2=0x7f69d80a79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69dd48d640 1 -- 172.21.15.39:0/3191662270 shutdown_connections 2024-09-13T17:10:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.460+0000 7f69dd48d640 1 -- 172.21.15.39:0/3191662270 wait complete. 2024-09-13T17:10:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 Processor -- start 2024-09-13T17:10:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 -- start start 2024-09-13T17:10:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d81605a0 0x7f69d815f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69d80be8c0 0x7f69d80bed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69d815da00 0x7f69d815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80bbe40 con 0x7f69d80be8c0 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80bcf80 con 0x7f69d815da00 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80bb230 con 0x7f69d81605a0 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69d80be8c0 0x7f69d80bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69d80be8c0 0x7f69d80bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53652/0 (socket says 172.21.15.39:53652) 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cffff640 1 -- 172.21.15.39:0/1179876038 learned_addr learned my addr 172.21.15.39:0/1179876038 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69d6ffd640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d81605a0 0x7f69d815f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69d77fe640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69d815da00 0x7f69d815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cffff640 1 -- 172.21.15.39:0/1179876038 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d81605a0 msgr2=0x7f69d815f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cffff640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d81605a0 0x7f69d815f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cffff640 1 -- 172.21.15.39:0/1179876038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69d815da00 msgr2=0x7f69d815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cffff640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69d815da00 0x7f69d815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cffff640 1 -- 172.21.15.39:0/1179876038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80bb510 con 0x7f69d80be8c0 2024-09-13T17:10:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69d77fe640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69d815da00 0x7f69d815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:10:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cffff640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69d80be8c0 0x7f69d80bed20 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f69c0003980 tx=0x7f69c0003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cf7fe640 1 -- 172.21.15.39:0/1179876038 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0012040 con 0x7f69d80be8c0 2024-09-13T17:10:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cf7fe640 1 -- 172.21.15.39:0/1179876038 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c000dde0 con 0x7f69d80be8c0 2024-09-13T17:10:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d80bb7f0 con 0x7f69d80be8c0 2024-09-13T17:10:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69cf7fe640 1 -- 172.21.15.39:0/1179876038 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0004170 con 0x7f69d80be8c0 2024-09-13T17:10:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.464+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d80bf260 con 0x7f69d80be8c0 2024-09-13T17:10:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.468+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d80bbfb0 con 0x7f69d80be8c0 2024-09-13T17:10:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.468+0000 7f69cf7fe640 1 -- 172.21.15.39:0/1179876038 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69c0016630 con 0x7f69d80be8c0 2024-09-13T17:10:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.468+0000 7f69cf7fe640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69b8066690 0x7f69b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.468+0000 7f69cf7fe640 1 -- 172.21.15.39:0/1179876038 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f69c0085b60 con 0x7f69d80be8c0 2024-09-13T17:10:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.468+0000 7f69d6ffd640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69b8066690 0x7f69b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.468+0000 7f69d6ffd640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69b8066690 0x7f69b8068b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f69c800df60 tx=0x7f69c800f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.472+0000 7f69cf7fe640 1 -- 172.21.15.39:0/1179876038 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69c0052070 con 0x7f69d80be8c0 2024-09-13T17:10:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.704+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69d8066a90 con 0x7f69d80be8c0 2024-09-13T17:10:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.704+0000 7f69cf7fe640 1 -- 172.21.15.39:0/1179876038 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) v1 ==== 95+0+5178 (secure 0 0 0) 0x7f69c0055690 con 0x7f69d80be8c0 2024-09-13T17:10:40.706 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:40.706 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":158,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:39.846605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:40.707 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 158 2024-09-13T17:10:40.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69b8066690 msgr2=0x7f69b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69b8066690 0x7f69b8068b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f69c800df60 tx=0x7f69c800f040 comp rx=0 tx=0).stop 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69d80be8c0 msgr2=0x7f69d80bed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69d80be8c0 0x7f69d80bed20 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f69c0003980 tx=0x7f69c0003e40 comp rx=0 tx=0).stop 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 shutdown_connections 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69b8066690 0x7f69b8068b50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69d815da00 0x7f69d815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69d80be8c0 0x7f69d80bed20 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 --2- 172.21.15.39:0/1179876038 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69d81605a0 0x7f69d815f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 >> 172.21.15.39:0/1179876038 conn(0x7f69d80a75b0 msgr2=0x7f69d8071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 shutdown_connections 2024-09-13T17:10:40.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.708+0000 7f69dd48d640 1 -- 172.21.15.39:0/1179876038 wait complete. 2024-09-13T17:10:40.729 INFO:tasks.mds_thrash.fs.[b]:mds.a down, removed from mdsmap 2024-09-13T17:10:40.729 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:10:40.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:40.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8507991640 1 Processor -- start 2024-09-13T17:10:40.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8507991640 1 -- start start 2024-09-13T17:10:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8507991640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f850014aa40 0x7f85000c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8507991640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f85000c1750 0x7f85000c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8507991640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85000c4710 0x7f85000c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8507991640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8500151b30 con 0x7f85000c4710 2024-09-13T17:10:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8507991640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85000c6510 con 0x7f85000c1750 2024-09-13T17:10:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8507991640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85000c76a0 con 0x7f850014aa40 2024-09-13T17:10:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8505f2e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85000c4710 0x7f85000c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8505f2e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85000c4710 0x7f85000c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53664/0 (socket says 172.21.15.39:53664) 2024-09-13T17:10:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8505f2e640 1 -- 172.21.15.39:0/146412263 learned_addr learned my addr 172.21.15.39:0/146412263 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8505f2e640 1 -- 172.21.15.39:0/146412263 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f850014aa40 msgr2=0x7f85000c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8505f2e640 1 --2- 172.21.15.39:0/146412263 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f850014aa40 0x7f85000c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8505f2e640 1 -- 172.21.15.39:0/146412263 >> v2:172.21.15.84:3300/0 conn(0x7f85000c1750 msgr2=0x7f85000c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8504f2c640 1 --2- 172.21.15.39:0/146412263 >> v2:172.21.15.84:3300/0 conn(0x7f85000c1750 0x7f85000c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8505f2e640 1 --2- 172.21.15.39:0/146412263 >> v2:172.21.15.84:3300/0 conn(0x7f85000c1750 0x7f85000c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.856+0000 7f8505f2e640 1 -- 172.21.15.39:0/146412263 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500151570 con 0x7f85000c4710 2024-09-13T17:10:40.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8505f2e640 1 --2- 172.21.15.39:0/146412263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c4710 0x7f85000c5fd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f85000618a0 tx=0x7f84f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64ade77c17f22fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:40.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f84f7fff640 1 -- 172.21.15.39:0/146412263 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f80a57b0 con 0x7f85000c4710 2024-09-13T17:10:40.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f84f7fff640 1 -- 172.21.15.39:0/146412263 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f80a5910 con 0x7f85000c4710 2024-09-13T17:10:40.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f84f7fff640 1 -- 172.21.15.39:0/146412263 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f80a5ca0 con 0x7f85000c4710 2024-09-13T17:10:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 -- 172.21.15.39:0/146412263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c4710 msgr2=0x7f85000c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 --2- 172.21.15.39:0/146412263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c4710 0x7f85000c5fd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f85000618a0 tx=0x7f84f801c930 comp rx=0 tx=0).stop 2024-09-13T17:10:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 -- 172.21.15.39:0/146412263 shutdown_connections 2024-09-13T17:10:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 --2- 172.21.15.39:0/146412263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c4710 0x7f85000c5fd0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 --2- 172.21.15.39:0/146412263 >> v2:172.21.15.84:3300/0 conn(0x7f85000c1750 0x7f85000c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 --2- 172.21.15.39:0/146412263 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f850014aa40 0x7f85000c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 -- 172.21.15.39:0/146412263 >> 172.21.15.39:0/146412263 conn(0x7f85000c3310 msgr2=0x7f85000c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 -- 172.21.15.39:0/146412263 shutdown_connections 2024-09-13T17:10:40.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 -- 172.21.15.39:0/146412263 wait complete. 2024-09-13T17:10:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 Processor -- start 2024-09-13T17:10:40.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.860+0000 7f8507991640 1 -- start start 2024-09-13T17:10:40.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8507991640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c1750 0x7f85001ed890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8507991640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85000c4710 0x7f85001eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8507991640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f850014aa40 0x7f85001ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8507991640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850014bb80 con 0x7f85000c1750 2024-09-13T17:10:40.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8507991640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850014a5e0 con 0x7f85000c4710 2024-09-13T17:10:40.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8507991640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85001ee320 con 0x7f850014aa40 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f850572d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c1750 0x7f85001ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f850572d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c1750 0x7f85001ed890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53668/0 (socket says 172.21.15.39:53668) 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f850572d640 1 -- 172.21.15.39:0/1291423064 learned_addr learned my addr 172.21.15.39:0/1291423064 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8504f2c640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85000c4710 0x7f85001eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8505f2e640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f850014aa40 0x7f85001ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f850572d640 1 -- 172.21.15.39:0/1291423064 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f850014aa40 msgr2=0x7f85001ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f850572d640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f850014aa40 0x7f85001ebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f850572d640 1 -- 172.21.15.39:0/1291423064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85000c4710 msgr2=0x7f85001eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f850572d640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85000c4710 0x7f85001eb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f850572d640 1 -- 172.21.15.39:0/1291423064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f8009b30 con 0x7f85000c1750 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f850572d640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c1750 0x7f85001ed890 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00b440 tx=0x7f84fc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f84f5ffb640 1 -- 172.21.15.39:0/1291423064 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc011890 con 0x7f85000c1750 2024-09-13T17:10:40.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f84f5ffb640 1 -- 172.21.15.39:0/1291423064 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc011a10 con 0x7f85000c1750 2024-09-13T17:10:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850014b810 con 0x7f85000c1750 2024-09-13T17:10:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f84f5ffb640 1 -- 172.21.15.39:0/1291423064 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc011dc0 con 0x7f85000c1750 2024-09-13T17:10:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850014b030 con 0x7f85000c1750 2024-09-13T17:10:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850006a700 con 0x7f85000c1750 2024-09-13T17:10:40.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f84f5ffb640 1 -- 172.21.15.39:0/1291423064 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f84fc01a400 con 0x7f85000c1750 2024-09-13T17:10:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f84f5ffb640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84e4066690 0x7f84e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f84f5ffb640 1 -- 172.21.15.39:0/1291423064 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f84fc0852f0 con 0x7f85000c1750 2024-09-13T17:10:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8504f2c640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84e4066690 0x7f84e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:40.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.864+0000 7f8504f2c640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84e4066690 0x7f84e4068b50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f850014bfa0 tx=0x7f84f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:40.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:40.872+0000 7f84f5ffb640 1 -- 172.21.15.39:0/1291423064 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f84fc051890 con 0x7f85000c1750 2024-09-13T17:10:41.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.108+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f850014ae20 con 0x7f85000c1750 2024-09-13T17:10:41.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.108+0000 7f84f5ffb640 1 -- 172.21.15.39:0/1291423064 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 159 v159) v1 ==== 95+0+5181 (secure 0 0 0) 0x7f84fc015020 con 0x7f85000c1750 2024-09-13T17:10:41.111 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:41.111 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":159,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":159,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:41.002733+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:reconnect","state_seq":8,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:41.111 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 159 2024-09-13T17:10:41.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84e4066690 msgr2=0x7f84e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:41.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84e4066690 0x7f84e4068b50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f850014bfa0 tx=0x7f84f0008040 comp rx=0 tx=0).stop 2024-09-13T17:10:41.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c1750 msgr2=0x7f85001ed890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:41.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c1750 0x7f85001ed890 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00b440 tx=0x7f84fc00b900 comp rx=0 tx=0).stop 2024-09-13T17:10:41.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 shutdown_connections 2024-09-13T17:10:41.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f84e4066690 0x7f84e4068b50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:41.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85000c4710 0x7f85001eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:41.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f850014aa40 0x7f85001ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:41.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 --2- 172.21.15.39:0/1291423064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85000c1750 0x7f85001ed890 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:41.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 >> 172.21.15.39:0/1291423064 conn(0x7f85000c3310 msgr2=0x7f85000c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:41.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 shutdown_connections 2024-09-13T17:10:41.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:41.112+0000 7f8507991640 1 -- 172.21.15.39:0/1291423064 wait complete. 2024-09-13T17:10:41.131 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 159, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 156, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_7677": { "addr": "172.21.15.39:6837/2351253009", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 2351253009, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 2351253009, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7677, "incarnation": 153, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:10:03.150430+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7677 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 159, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6290": { "addr": "172.21.15.84:6835/397460031", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 397460031, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 397460031, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6290, "incarnation": 158, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:reconnect", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:10:41.002733+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6290 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:10:43.132 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:43.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.264+0000 7f2687fff640 1 Processor -- start 2024-09-13T17:10:43.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.264+0000 7f2687fff640 1 -- start start 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.264+0000 7f2687fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2688151480 0x7f26880a7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.264+0000 7f2687fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f26880a7990 0x7f26880c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.264+0000 7f2687fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 0x7f26881541d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.264+0000 7f2687fff640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26880c1790 con 0x7f2688151480 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.264+0000 7f2687fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26880aa1c0 con 0x7f26880a7990 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.264+0000 7f2687fff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26880c19a0 con 0x7f26880c1040 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2686ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2688151480 0x7f26880a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2686ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2688151480 0x7f26880a7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53680/0 (socket says 172.21.15.39:53680) 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2686ffd640 1 -- 172.21.15.39:0/2761886065 learned_addr learned my addr 172.21.15.39:0/2761886065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f267ffff640 1 --2- 172.21.15.39:0/2761886065 >> v2:172.21.15.84:3300/0 conn(0x7f26880a7990 0x7f26880c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f26877fe640 1 --2- 172.21.15.39:0/2761886065 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 0x7f26881541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:43.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2686ffd640 1 -- 172.21.15.39:0/2761886065 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 msgr2=0x7f26881541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2686ffd640 1 --2- 172.21.15.39:0/2761886065 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 0x7f26881541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2686ffd640 1 -- 172.21.15.39:0/2761886065 >> v2:172.21.15.84:3300/0 conn(0x7f26880a7990 msgr2=0x7f26880c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2686ffd640 1 --2- 172.21.15.39:0/2761886065 >> v2:172.21.15.84:3300/0 conn(0x7f26880a7990 0x7f26880c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2686ffd640 1 -- 172.21.15.39:0/2761886065 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26880c1c80 con 0x7f2688151480 2024-09-13T17:10:43.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2686ffd640 1 --2- 172.21.15.39:0/2761886065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 0x7f26880a7340 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f26880679f0 tx=0x7f267001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b372cd629e4b8b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:43.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f26867fc640 1 -- 172.21.15.39:0/2761886065 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700ab5f0 con 0x7f2688151480 2024-09-13T17:10:43.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f26867fc640 1 -- 172.21.15.39:0/2761886065 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26700ab750 con 0x7f2688151480 2024-09-13T17:10:43.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f26867fc640 1 -- 172.21.15.39:0/2761886065 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700abae0 con 0x7f2688151480 2024-09-13T17:10:43.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2687fff640 1 -- 172.21.15.39:0/2761886065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 msgr2=0x7f26880a7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:43.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2687fff640 1 --2- 172.21.15.39:0/2761886065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 0x7f26880a7340 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f26880679f0 tx=0x7f267001c520 comp rx=0 tx=0).stop 2024-09-13T17:10:43.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2687fff640 1 -- 172.21.15.39:0/2761886065 shutdown_connections 2024-09-13T17:10:43.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2687fff640 1 --2- 172.21.15.39:0/2761886065 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 0x7f26881541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2687fff640 1 --2- 172.21.15.39:0/2761886065 >> v2:172.21.15.84:3300/0 conn(0x7f26880a7990 0x7f26880c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2687fff640 1 --2- 172.21.15.39:0/2761886065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 0x7f26880a7340 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2687fff640 1 -- 172.21.15.39:0/2761886065 >> 172.21.15.39:0/2761886065 conn(0x7f268806e2b0 msgr2=0x7f268806e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:43.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2687fff640 1 -- 172.21.15.39:0/2761886065 shutdown_connections 2024-09-13T17:10:43.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.268+0000 7f2687fff640 1 -- 172.21.15.39:0/2761886065 wait complete. 2024-09-13T17:10:43.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 Processor -- start 2024-09-13T17:10:43.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 -- start start 2024-09-13T17:10:43.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26880a7990 0x7f268814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:43.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 0x7f2688156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:43.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 0x7f26880aa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:43.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881efaf0 con 0x7f2688151480 2024-09-13T17:10:43.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881efd90 con 0x7f26880a7990 2024-09-13T17:10:43.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26881f0070 con 0x7f26880c1040 2024-09-13T17:10:43.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f26877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 0x7f26880aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f26877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 0x7f26880aa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53690/0 (socket says 172.21.15.39:53690) 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f26877fe640 1 -- 172.21.15.39:0/1071780837 learned_addr learned my addr 172.21.15.39:0/1071780837 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2686ffd640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26880a7990 0x7f268814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f267ffff640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 0x7f2688156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f26877fe640 1 -- 172.21.15.39:0/1071780837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 msgr2=0x7f2688156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f26877fe640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 0x7f2688156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f26877fe640 1 -- 172.21.15.39:0/1071780837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26880a7990 msgr2=0x7f268814e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f26877fe640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26880a7990 0x7f268814e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f26877fe640 1 -- 172.21.15.39:0/1071780837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700094d0 con 0x7f2688151480 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f26877fe640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 0x7f26880aa830 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f2680007e40 tx=0x7f268000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f267f7fe640 1 -- 172.21.15.39:0/1071780837 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680014620 con 0x7f2688151480 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f267f7fe640 1 -- 172.21.15.39:0/1071780837 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268000ee90 con 0x7f2688151480 2024-09-13T17:10:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26881f0350 con 0x7f2688151480 2024-09-13T17:10:43.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f267f7fe640 1 -- 172.21.15.39:0/1071780837 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800149e0 con 0x7f2688151480 2024-09-13T17:10:43.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.272+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268814e740 con 0x7f2688151480 2024-09-13T17:10:43.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.276+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26881f0870 con 0x7f2688151480 2024-09-13T17:10:43.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.276+0000 7f267f7fe640 1 -- 172.21.15.39:0/1071780837 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2680014b40 con 0x7f2688151480 2024-09-13T17:10:43.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.276+0000 7f267f7fe640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2668066690 0x7f2668068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:43.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.276+0000 7f267f7fe640 1 -- 172.21.15.39:0/1071780837 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f2680085a80 con 0x7f2688151480 2024-09-13T17:10:43.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.276+0000 7f2686ffd640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2668066690 0x7f2668068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:43.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.276+0000 7f2686ffd640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2668066690 0x7f2668068b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f26700af000 tx=0x7f26700a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.280+0000 7f267f7fe640 1 -- 172.21.15.39:0/1071780837 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f26881f0870 con 0x7f2688151480 2024-09-13T17:10:43.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.520+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26881f0870 con 0x7f2688151480 2024-09-13T17:10:43.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f267f7fe640 1 -- 172.21.15.39:0/1071780837 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) v1 ==== 95+0+5179 (secure 0 0 0) 0x7f26881f0870 con 0x7f2688151480 2024-09-13T17:10:43.525 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:43.525 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":161,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:43.525 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 161 2024-09-13T17:10:43.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2668066690 msgr2=0x7f2668068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:43.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2668066690 0x7f2668068b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f26700af000 tx=0x7f26700a7000 comp rx=0 tx=0).stop 2024-09-13T17:10:43.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 msgr2=0x7f26880aa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:43.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 0x7f26880aa830 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f2680007e40 tx=0x7f268000e4b0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 shutdown_connections 2024-09-13T17:10:43.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2668066690 0x7f2668068b50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2688151480 0x7f26880aa830 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f26880c1040 0x7f2688156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 --2- 172.21.15.39:0/1071780837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f26880a7990 0x7f268814e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:43.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 >> 172.21.15.39:0/1071780837 conn(0x7f268806e2b0 msgr2=0x7f268806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:43.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 shutdown_connections 2024-09-13T17:10:43.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:43.524+0000 7f2687fff640 1 -- 172.21.15.39:0/1071780837 wait complete. 2024-09-13T17:10:43.544 INFO:tasks.mds_thrash.fs.[b]:mds.d has gained rank=0, replacing gid=7485 2024-09-13T17:10:43.545 INFO:tasks.mds_thrash.fs.[b]:waiting for 6 secs before reviving mds.a 2024-09-13T17:10:43.915 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T17:10:43.912+0000 7f1382c05640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:10:49.545 INFO:tasks.mds_thrash.fs.[b]:reviving mds.a 2024-09-13T17:10:49.546 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:10:49.546 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:10:49.549 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:10:49.549 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:49.637 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T17:10:49.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e753b1640 1 Processor -- start 2024-09-13T17:10:49.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e753b1640 1 -- start start 2024-09-13T17:10:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e753b1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0e70157530 0x7f0e700c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e753b1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e700c3d60 con 0x7f0e7006f840 2024-09-13T17:10:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e753b1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7006b960 con 0x7f0e70157910 2024-09-13T17:10:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e753b1640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7006be40 con 0x7f0e70157530 2024-09-13T17:10:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0e7006f840 0x7f0e700a6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:39586/0 (socket says 172.21.15.39:39586) 2024-09-13T17:10:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6f7fe640 1 -- 172.21.15.39:0/3107861259 learned_addr learned my addr 172.21.15.39:0/3107861259 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:10:49.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6effd640 1 --2- 172.21.15.39:0/3107861259 >> v2:172.21.15.84:3300/0 conn(0x7f0e70157530 0x7f0e700c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707293249 0 0) 0x7f0e700c3d60 con 0x7f0e7006f840 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e5c003530 con 0x7f0e7006f840 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1031389911 0 0) 0x7f0e5c003530 con 0x7f0e7006f840 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 >> v2:172.21.15.84:3300/0 conn(0x7f0e70157530 msgr2=0x7f0e700c2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 --2- 172.21.15.39:0/3107861259 >> v2:172.21.15.84:3300/0 conn(0x7f0e70157530 0x7f0e700c2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 >> v1:172.21.15.84:6789/0 conn(0x7f0e70157910 legacy=0x7f0e70071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e700c4000 con 0x7f0e7006f840 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f0e68001a00 con 0x7f0e7006f840 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 >> v1:172.21.15.39:6789/0 conn(0x7f0e7006f840 legacy=0x7f0e700a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:49.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c003980 0x7f0e5c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e5c0047d0 0x7f0e5c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e5c005190 0x7f0e5c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7006be40 con 0x7f0e5c003980 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c00c040 con 0x7f0e5c005190 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c00c200 con 0x7f0e5c0047d0 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6effd640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c003980 0x7f0e5c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e67fff640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e5c0047d0 0x7f0e5c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6effd640 1 -- 172.21.15.39:0/3107861259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e5c0047d0 msgr2=0x7f0e5c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6effd640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e5c0047d0 0x7f0e5c004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6effd640 1 -- 172.21.15.39:0/3107861259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e5c005190 msgr2=0x7f0e5c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6effd640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e5c005190 0x7f0e5c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6effd640 1 -- 172.21.15.39:0/3107861259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c003530 con 0x7f0e5c003980 2024-09-13T17:10:49.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.696+0000 7f0e6f7fe640 1 -- 172.21.15.39:0/3107861259 reap_dead start 2024-09-13T17:10:49.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e6effd640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c003980 0x7f0e5c003da0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009ae0 tx=0x7f0e6001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ced7b8502dc7b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600a5680 con 0x7f0e5c003980 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e600a57e0 con 0x7f0e5c003980 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e6e7fc640 1 -- 172.21.15.39:0/3107861259 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600a5b90 con 0x7f0e5c003980 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e753b1640 1 -- 172.21.15.39:0/3107861259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c003980 msgr2=0x7f0e5c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e753b1640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c003980 0x7f0e5c003da0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009ae0 tx=0x7f0e6001ca90 comp rx=0 tx=0).stop 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e753b1640 1 -- 172.21.15.39:0/3107861259 shutdown_connections 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e753b1640 1 --2- 172.21.15.39:0/3107861259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c003980 0x7f0e5c003da0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e753b1640 1 -- 172.21.15.39:0/3107861259 >> 172.21.15.39:0/3107861259 conn(0x7f0e700a97b0 msgr2=0x7f0e700a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e753b1640 1 -- 172.21.15.39:0/3107861259 shutdown_connections 2024-09-13T17:10:49.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.700+0000 7f0e753b1640 1 -- 172.21.15.39:0/3107861259 wait complete. 2024-09-13T17:10:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 Processor -- start 2024-09-13T17:10:49.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 -- start start 2024-09-13T17:10:49.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e5c003980 0x7f0e701ecab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:49.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70153470 0x7f0e701521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:49.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e701548d0 0x7f0e700a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:49.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e700ac9f0 con 0x7f0e70153470 2024-09-13T17:10:49.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70151ce0 con 0x7f0e5c003980 2024-09-13T17:10:49.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7014af10 con 0x7f0e701548d0 2024-09-13T17:10:49.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e67fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70153470 0x7f0e701521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:49.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e67fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70153470 0x7f0e701521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53736/0 (socket says 172.21.15.39:53736) 2024-09-13T17:10:49.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e67fff640 1 -- 172.21.15.39:0/1883112839 learned_addr learned my addr 172.21.15.39:0/1883112839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e6f7fe640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e701548d0 0x7f0e700a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e6effd640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e5c003980 0x7f0e701ecab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e67fff640 1 -- 172.21.15.39:0/1883112839 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e701548d0 msgr2=0x7f0e700a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e67fff640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e701548d0 0x7f0e700a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e67fff640 1 -- 172.21.15.39:0/1883112839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e5c003980 msgr2=0x7f0e701ecab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e67fff640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e5c003980 0x7f0e701ecab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e67fff640 1 -- 172.21.15.39:0/1883112839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7014b1f0 con 0x7f0e70153470 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e6f7fe640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e701548d0 0x7f0e700a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e67fff640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70153470 0x7f0e701521e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800d570 tx=0x7f0e5800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e677fe640 1 -- 172.21.15.39:0/1883112839 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58005480 con 0x7f0e70153470 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e677fe640 1 -- 172.21.15.39:0/1883112839 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e58003620 con 0x7f0e70153470 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e700acc40 con 0x7f0e70153470 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e677fe640 1 -- 172.21.15.39:0/1883112839 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58003990 con 0x7f0e70153470 2024-09-13T17:10:49.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e70152720 con 0x7f0e70153470 2024-09-13T17:10:49.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e70151e70 con 0x7f0e70153470 2024-09-13T17:10:49.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.704+0000 7f0e677fe640 1 -- 172.21.15.39:0/1883112839 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e58003af0 con 0x7f0e70153470 2024-09-13T17:10:49.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.708+0000 7f0e677fe640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e50066690 0x7f0e50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:49.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.708+0000 7f0e677fe640 1 -- 172.21.15.39:0/1883112839 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f0e58084e20 con 0x7f0e70153470 2024-09-13T17:10:49.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.708+0000 7f0e6effd640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e50066690 0x7f0e50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:49.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.708+0000 7f0e6effd640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e50066690 0x7f0e50068b50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0e600af000 tx=0x7f0e60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:49.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.712+0000 7f0e677fe640 1 -- 172.21.15.39:0/1883112839 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0e70151e70 con 0x7f0e70153470 2024-09-13T17:10:49.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.924+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e70151e20 con 0x7f0e70153470 2024-09-13T17:10:49.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.924+0000 7f0e677fe640 1 -- 172.21.15.39:0/1883112839 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) v1 ==== 95+0+5179 (secure 0 0 0) 0x7f0e580513c0 con 0x7f0e70153470 2024-09-13T17:10:49.927 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:49.927 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":161,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:49.927 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 161 2024-09-13T17:10:49.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e50066690 msgr2=0x7f0e50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e50066690 0x7f0e50068b50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0e600af000 tx=0x7f0e60008040 comp rx=0 tx=0).stop 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70153470 msgr2=0x7f0e701521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70153470 0x7f0e701521e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800d570 tx=0x7f0e5800da30 comp rx=0 tx=0).stop 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 shutdown_connections 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0e50066690 0x7f0e50068b50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0e701548d0 0x7f0e700a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70153470 0x7f0e701521e0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 --2- 172.21.15.39:0/1883112839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e5c003980 0x7f0e701ecab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 >> 172.21.15.39:0/1883112839 conn(0x7f0e700a97b0 msgr2=0x7f0e70155de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 shutdown_connections 2024-09-13T17:10:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:49.928+0000 7f0e753b1640 1 -- 172.21.15.39:0/1883112839 wait complete. 2024-09-13T17:10:49.947 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T17:10:51.948 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:52.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3957eb4640 1 Processor -- start 2024-09-13T17:10:52.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3957eb4640 1 -- start start 2024-09-13T17:10:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3957eb4640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f395006b080 0x7f395006e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3957eb4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f39500c1810 0x7f3950151390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3957eb4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39500c15d0 con 0x7f39501518d0 2024-09-13T17:10:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3957eb4640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395006d0b0 con 0x7f395006b080 2024-09-13T17:10:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3957eb4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39500c1f00 con 0x7f39500c1810 2024-09-13T17:10:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f395544f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f39500c1810 0x7f3950151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3955c50640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f395006b080 0x7f395006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3955c50640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f395006b080 0x7f395006e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:56106/0 (socket says 172.21.15.39:56106) 2024-09-13T17:10:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f395544f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f39500c1810 0x7f3950151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:38936/0 (socket says 172.21.15.39:38936) 2024-09-13T17:10:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3955c50640 1 -- 172.21.15.39:0/383354077 learned_addr learned my addr 172.21.15.39:0/383354077 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3955c50640 1 -- 172.21.15.39:0/383354077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f39500c1810 msgr2=0x7f3950151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3955c50640 1 --2- 172.21.15.39:0/383354077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f39500c1810 0x7f3950151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3955c50640 1 -- 172.21.15.39:0/383354077 >> v1:172.21.15.84:6789/0 conn(0x7f39501518d0 legacy=0x7f39501576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:52.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.088+0000 7f3955c50640 1 -- 172.21.15.39:0/383354077 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500c21a0 con 0x7f395006b080 2024-09-13T17:10:52.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3955c50640 1 --2- 172.21.15.39:0/383354077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395006b080 0x7f395006e680 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f394c009870 tx=0x7f394c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57067cc9f9bcd041 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:52.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3954c4e640 1 -- 172.21.15.39:0/383354077 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c0a6040 con 0x7f395006b080 2024-09-13T17:10:52.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3954c4e640 1 -- 172.21.15.39:0/383354077 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c0aa600 con 0x7f395006b080 2024-09-13T17:10:52.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3954c4e640 1 -- 172.21.15.39:0/383354077 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c0aa990 con 0x7f395006b080 2024-09-13T17:10:52.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 -- 172.21.15.39:0/383354077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395006b080 msgr2=0x7f395006e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:52.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 --2- 172.21.15.39:0/383354077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395006b080 0x7f395006e680 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f394c009870 tx=0x7f394c01c4a0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 -- 172.21.15.39:0/383354077 shutdown_connections 2024-09-13T17:10:52.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 --2- 172.21.15.39:0/383354077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f39500c1810 0x7f3950151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 --2- 172.21.15.39:0/383354077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f395006b080 0x7f395006e680 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 -- 172.21.15.39:0/383354077 >> 172.21.15.39:0/383354077 conn(0x7f39500c32a0 msgr2=0x7f39500c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:52.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 -- 172.21.15.39:0/383354077 shutdown_connections 2024-09-13T17:10:52.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 -- 172.21.15.39:0/383354077 wait complete. 2024-09-13T17:10:52.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 Processor -- start 2024-09-13T17:10:52.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.092+0000 7f3957eb4640 1 -- start start 2024-09-13T17:10:52.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3957eb4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f395006b080 0x7f3950071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:52.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3957eb4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39500c1810 0x7f39500a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:52.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3957eb4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39501518d0 0x7f39500a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:52.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3957eb4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500a6200 con 0x7f39501518d0 2024-09-13T17:10:52.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3957eb4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500a5f20 con 0x7f39500c1810 2024-09-13T17:10:52.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3957eb4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39500c05e0 con 0x7f395006b080 2024-09-13T17:10:52.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3956451640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39501518d0 0x7f39500a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:52.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3956451640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39501518d0 0x7f39500a7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37588/0 (socket says 172.21.15.39:37588) 2024-09-13T17:10:52.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3955c50640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f395006b080 0x7f3950071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:52.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3956451640 1 -- 172.21.15.39:0/714537914 learned_addr learned my addr 172.21.15.39:0/714537914 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:52.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f395544f640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39500c1810 0x7f39500a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:52.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3956451640 1 -- 172.21.15.39:0/714537914 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f395006b080 msgr2=0x7f3950071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:52.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3956451640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f395006b080 0x7f3950071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3956451640 1 -- 172.21.15.39:0/714537914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39500c1810 msgr2=0x7f39500a79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:52.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3956451640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39500c1810 0x7f39500a79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3956451640 1 -- 172.21.15.39:0/714537914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c0094d0 con 0x7f39501518d0 2024-09-13T17:10:52.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3956451640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39501518d0 0x7f39500a7ee0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f394800c730 tx=0x7f394800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:52.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f39467fc640 1 -- 172.21.15.39:0/714537914 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948003b80 con 0x7f39501518d0 2024-09-13T17:10:52.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f39467fc640 1 -- 172.21.15.39:0/714537914 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3948003ce0 con 0x7f39501518d0 2024-09-13T17:10:52.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f39467fc640 1 -- 172.21.15.39:0/714537914 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394800aa30 con 0x7f39501518d0 2024-09-13T17:10:52.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39501f05a0 con 0x7f39501518d0 2024-09-13T17:10:52.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3950071ba0 con 0x7f39501518d0 2024-09-13T17:10:52.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39500619c0 con 0x7f39501518d0 2024-09-13T17:10:52.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f39467fc640 1 -- 172.21.15.39:0/714537914 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f394800ab90 con 0x7f39501518d0 2024-09-13T17:10:52.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f39467fc640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3934066690 0x7f3934068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:52.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f39467fc640 1 -- 172.21.15.39:0/714537914 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f3948084be0 con 0x7f39501518d0 2024-09-13T17:10:52.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.096+0000 7f3955c50640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3934066690 0x7f3934068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:52.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.100+0000 7f3955c50640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3934066690 0x7f3934068b50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f394c0ae000 tx=0x7f394c0b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:52.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.104+0000 7f39467fc640 1 -- 172.21.15.39:0/714537914 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3948051230 con 0x7f39501518d0 2024-09-13T17:10:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3950067130 con 0x7f39501518d0 2024-09-13T17:10:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f39467fc640 1 -- 172.21.15.39:0/714537914 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f3948054850 con 0x7f39501518d0 2024-09-13T17:10:52.338 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:52.338 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:52.338 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 162 2024-09-13T17:10:52.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3934066690 msgr2=0x7f3934068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:52.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3934066690 0x7f3934068b50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f394c0ae000 tx=0x7f394c0b5040 comp rx=0 tx=0).stop 2024-09-13T17:10:52.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39501518d0 msgr2=0x7f39500a7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:52.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39501518d0 0x7f39500a7ee0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f394800c730 tx=0x7f394800cbf0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 shutdown_connections 2024-09-13T17:10:52.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3934066690 0x7f3934068b50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f39500c1810 0x7f39500a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39501518d0 0x7f39500a7ee0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 --2- 172.21.15.39:0/714537914 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f395006b080 0x7f3950071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:52.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.336+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 >> 172.21.15.39:0/714537914 conn(0x7f39500c32a0 msgr2=0x7f3950060a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:52.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.340+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 shutdown_connections 2024-09-13T17:10:52.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:52.340+0000 7f3957eb4640 1 -- 172.21.15.39:0/714537914 wait complete. 2024-09-13T17:10:52.359 INFO:tasks.mds_thrash.fs.[b]:mds.a reported in up:standby state 2024-09-13T17:10:52.359 INFO:tasks.mds_thrash.fs.[b]:waiting for 70 secs before thrashing 2024-09-13T17:10:54.195 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:54.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35e7ae640 1 Processor -- start 2024-09-13T17:10:54.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35e7ae640 1 -- start start 2024-09-13T17:10:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35e7ae640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35e7ae640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3580c7800 0x7fe3581545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35e7ae640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3580c5a30 con 0x7fe358151500 2024-09-13T17:10:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35e7ae640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3581576a0 con 0x7fe3580c7800 2024-09-13T17:10:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35e7ae640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe358157980 con 0x7fe358071dd0 2024-09-13T17:10:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe34f7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35cd4b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3580c7800 0x7fe3581545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe34f7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:38948/0 (socket says 172.21.15.39:38948) 2024-09-13T17:10:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35cd4b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3580c7800 0x7fe3581545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:56122/0 (socket says 172.21.15.39:56122) 2024-09-13T17:10:54.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe34f7fe640 1 -- 172.21.15.39:0/3300269019 learned_addr learned my addr 172.21.15.39:0/3300269019 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:54.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe357fff640 1 --1- 172.21.15.39:0/3300269019 >> v1:172.21.15.84:6789/0 conn(0x7fe358151500 0x7fe3580c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:56118/0 (socket says 172.21.15.39:56118) 2024-09-13T17:10:54.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe34f7fe640 1 -- 172.21.15.39:0/3300269019 >> v2:172.21.15.84:3300/0 conn(0x7fe3580c7800 msgr2=0x7fe3581545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:54.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe34f7fe640 1 --2- 172.21.15.39:0/3300269019 >> v2:172.21.15.84:3300/0 conn(0x7fe3580c7800 0x7fe3581545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe34f7fe640 1 -- 172.21.15.39:0/3300269019 >> v1:172.21.15.84:6789/0 conn(0x7fe358151500 legacy=0x7fe3580c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe34f7fe640 1 -- 172.21.15.39:0/3300269019 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358157c60 con 0x7fe358071dd0 2024-09-13T17:10:54.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.344+0000 7fe35cd4b640 1 --2- 172.21.15.39:0/3300269019 >> v2:172.21.15.84:3300/0 conn(0x7fe3580c7800 0x7fe3581545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:10:54.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe34f7fe640 1 --2- 172.21.15.39:0/3300269019 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358154070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe340009870 tx=0x7fe34001cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1104d21e6326ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:54.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe3577fe640 1 -- 172.21.15.39:0/3300269019 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3400aa620 con 0x7fe358071dd0 2024-09-13T17:10:54.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe3577fe640 1 -- 172.21.15.39:0/3300269019 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3400aa780 con 0x7fe358071dd0 2024-09-13T17:10:54.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe3577fe640 1 -- 172.21.15.39:0/3300269019 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3400aab10 con 0x7fe358071dd0 2024-09-13T17:10:54.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe35e7ae640 1 -- 172.21.15.39:0/3300269019 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 msgr2=0x7fe358154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe35e7ae640 1 --2- 172.21.15.39:0/3300269019 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358154070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe340009870 tx=0x7fe34001cf00 comp rx=0 tx=0).stop 2024-09-13T17:10:54.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe35e7ae640 1 -- 172.21.15.39:0/3300269019 shutdown_connections 2024-09-13T17:10:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe35e7ae640 1 --2- 172.21.15.39:0/3300269019 >> v2:172.21.15.84:3300/0 conn(0x7fe3580c7800 0x7fe3581545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe35e7ae640 1 --2- 172.21.15.39:0/3300269019 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358154070 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe35e7ae640 1 -- 172.21.15.39:0/3300269019 >> 172.21.15.39:0/3300269019 conn(0x7fe3580ab530 msgr2=0x7fe3580ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe35e7ae640 1 -- 172.21.15.39:0/3300269019 shutdown_connections 2024-09-13T17:10:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.348+0000 7fe35e7ae640 1 -- 172.21.15.39:0/3300269019 wait complete. 2024-09-13T17:10:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 Processor -- start 2024-09-13T17:10:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 -- start start 2024-09-13T17:10:54.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3580c7800 0x7fe3581f0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358151500 0x7fe3581490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe358156540 con 0x7fe358151500 2024-09-13T17:10:54.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3581f0880 con 0x7fe3580c7800 2024-09-13T17:10:54.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3581ec4d0 con 0x7fe358071dd0 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35cd4b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358151500 0x7fe3581490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35cd4b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358151500 0x7fe3581490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37594/0 (socket says 172.21.15.39:37594) 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35cd4b640 1 -- 172.21.15.39:0/209812178 learned_addr learned my addr 172.21.15.39:0/209812178 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe357fff640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe34f7fe640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3580c7800 0x7fe3581f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35cd4b640 1 -- 172.21.15.39:0/209812178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 msgr2=0x7fe358155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35cd4b640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35cd4b640 1 -- 172.21.15.39:0/209812178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3580c7800 msgr2=0x7fe3581f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35cd4b640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3580c7800 0x7fe3581f0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35cd4b640 1 -- 172.21.15.39:0/209812178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350009a70 con 0x7fe358151500 2024-09-13T17:10:54.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35cd4b640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358151500 0x7fe3581490a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe35000efc0 tx=0x7fe350009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:54.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe3557fa640 1 -- 172.21.15.39:0/209812178 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350009d70 con 0x7fe358151500 2024-09-13T17:10:54.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe3557fa640 1 -- 172.21.15.39:0/209812178 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35000f040 con 0x7fe358151500 2024-09-13T17:10:54.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3400094d0 con 0x7fe358151500 2024-09-13T17:10:54.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe3557fa640 1 -- 172.21.15.39:0/209812178 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3500039a0 con 0x7fe358151500 2024-09-13T17:10:54.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe358156a10 con 0x7fe358151500 2024-09-13T17:10:54.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.352+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3580c4280 con 0x7fe358151500 2024-09-13T17:10:54.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.356+0000 7fe3557fa640 1 -- 172.21.15.39:0/209812178 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe3500144a0 con 0x7fe358151500 2024-09-13T17:10:54.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.356+0000 7fe3557fa640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe338066690 0x7fe338068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.356+0000 7fe3557fa640 1 -- 172.21.15.39:0/209812178 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7fe350085af0 con 0x7fe358151500 2024-09-13T17:10:54.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.356+0000 7fe357fff640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe338066690 0x7fe338068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.356+0000 7fe357fff640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe338066690 0x7fe338068b50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe348000f70 tx=0x7fe348007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:54.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.360+0000 7fe3557fa640 1 -- 172.21.15.39:0/209812178 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe350052140 con 0x7fe358151500 2024-09-13T17:10:54.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.616+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe358071bb0 con 0x7fe358151500 2024-09-13T17:10:54.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.616+0000 7fe3557fa640 1 -- 172.21.15.39:0/209812178 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5937 (secure 0 0 0) 0x7fe350018020 con 0x7fe358151500 2024-09-13T17:10:54.619 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:54.619 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:54.619 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 162 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe338066690 msgr2=0x7fe338068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe338066690 0x7fe338068b50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe348000f70 tx=0x7fe348007e30 comp rx=0 tx=0).stop 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358151500 msgr2=0x7fe3581490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358151500 0x7fe3581490a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe35000efc0 tx=0x7fe350009f90 comp rx=0 tx=0).stop 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 shutdown_connections 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe338066690 0x7fe338068b50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3580c7800 0x7fe3581f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358151500 0x7fe3581490a0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 --2- 172.21.15.39:0/209812178 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe358071dd0 0x7fe358155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 >> 172.21.15.39:0/209812178 conn(0x7fe3580ab530 msgr2=0x7fe3580c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:54.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 shutdown_connections 2024-09-13T17:10:54.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.620+0000 7fe35e7ae640 1 -- 172.21.15.39:0/209812178 wait complete. 2024-09-13T17:10:54.642 INFO:tasks.mds_thrash.fs.[a]:kill mds.c (rank=0) 2024-09-13T17:10:54.642 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:10:54.642 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:10:54.652 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:10:54.653 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:54.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.828+0000 7f24ab6ee640 1 Processor -- start 2024-09-13T17:10:54.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.828+0000 7f24ab6ee640 1 -- start start 2024-09-13T17:10:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24ab6ee640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a4151500 0x7f24a40c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24ab6ee640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f24a40c47c0 0x7f24a4154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24ab6ee640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a40c5a30 con 0x7f24a4071dd0 2024-09-13T17:10:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24ab6ee640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a41576a0 con 0x7f24a40c47c0 2024-09-13T17:10:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24ab6ee640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a4157980 con 0x7f24a4151500 2024-09-13T17:10:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24a9c8b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f24a40c47c0 0x7f24a4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24a9c8b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f24a40c47c0 0x7f24a4154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37608/0 (socket says 172.21.15.39:37608) 2024-09-13T17:10:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24a9c8b640 1 -- 172.21.15.39:0/3720527374 learned_addr learned my addr 172.21.15.39:0/3720527374 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24a948a640 1 --2- 172.21.15.39:0/3720527374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a4151500 0x7f24a40c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24a9c8b640 1 -- 172.21.15.39:0/3720527374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a4151500 msgr2=0x7f24a40c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24a9c8b640 1 --2- 172.21.15.39:0/3720527374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a4151500 0x7f24a40c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24a9c8b640 1 -- 172.21.15.39:0/3720527374 >> v1:172.21.15.84:6789/0 conn(0x7f24a4071dd0 legacy=0x7f24a4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.832+0000 7f24a9c8b640 1 -- 172.21.15.39:0/3720527374 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4157c60 con 0x7f24a40c47c0 2024-09-13T17:10:54.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a9c8b640 1 --2- 172.21.15.39:0/3720527374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a40c47c0 0x7f24a4154780 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f249c009e50 tx=0x7f249c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aac56d866424118 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:54.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f249bfff640 1 -- 172.21.15.39:0/3720527374 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c0a5850 con 0x7f24a40c47c0 2024-09-13T17:10:54.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f249bfff640 1 -- 172.21.15.39:0/3720527374 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249c0a59b0 con 0x7f24a40c47c0 2024-09-13T17:10:54.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f249bfff640 1 -- 172.21.15.39:0/3720527374 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c0a5d40 con 0x7f24a40c47c0 2024-09-13T17:10:54.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3720527374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a40c47c0 msgr2=0x7f24a4154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 --2- 172.21.15.39:0/3720527374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a40c47c0 0x7f24a4154780 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f249c009e50 tx=0x7f249c01c970 comp rx=0 tx=0).stop 2024-09-13T17:10:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3720527374 shutdown_connections 2024-09-13T17:10:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 --2- 172.21.15.39:0/3720527374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a40c47c0 0x7f24a4154780 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 --2- 172.21.15.39:0/3720527374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a4151500 0x7f24a40c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3720527374 >> 172.21.15.39:0/3720527374 conn(0x7f24a40ab530 msgr2=0x7f24a40ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3720527374 shutdown_connections 2024-09-13T17:10:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3720527374 wait complete. 2024-09-13T17:10:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 Processor -- start 2024-09-13T17:10:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 -- start start 2024-09-13T17:10:54.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a4071dd0 0x7f24a41ecce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a40c47c0 0x7f24a4148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a4151500 0x7f24a414d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a4146ea0 con 0x7f24a4071dd0 2024-09-13T17:10:54.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a414d4c0 con 0x7f24a4151500 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24ab6ee640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a4148790 con 0x7f24a40c47c0 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a948a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a4071dd0 0x7f24a41ecce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a948a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a4071dd0 0x7f24a41ecce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37620/0 (socket says 172.21.15.39:37620) 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a948a640 1 -- 172.21.15.39:0/3362377606 learned_addr learned my addr 172.21.15.39:0/3362377606 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a9c8b640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a4151500 0x7f24a414d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a8c89640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a40c47c0 0x7f24a4148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a948a640 1 -- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a40c47c0 msgr2=0x7f24a4148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a948a640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a40c47c0 0x7f24a4148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a948a640 1 -- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a4151500 msgr2=0x7f24a414d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a948a640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a4151500 0x7f24a414d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:54.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a948a640 1 -- 172.21.15.39:0/3362377606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c009b30 con 0x7f24a4071dd0 2024-09-13T17:10:54.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a8c89640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a40c47c0 0x7f24a4148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:10:54.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.836+0000 7f24a9c8b640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a4151500 0x7f24a414d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:10:54.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f24a948a640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a4071dd0 0x7f24a41ecce0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f24a000d5f0 tx=0x7f24a000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:54.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f2499ffb640 1 -- 172.21.15.39:0/3362377606 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00125f0 con 0x7f24a4071dd0 2024-09-13T17:10:54.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f2499ffb640 1 -- 172.21.15.39:0/3362377606 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0012750 con 0x7f24a4071dd0 2024-09-13T17:10:54.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a4151e20 con 0x7f24a4071dd0 2024-09-13T17:10:54.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f2499ffb640 1 -- 172.21.15.39:0/3362377606 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0012ae0 con 0x7f24a4071dd0 2024-09-13T17:10:54.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a4147360 con 0x7f24a4071dd0 2024-09-13T17:10:54.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a40c4280 con 0x7f24a4071dd0 2024-09-13T17:10:54.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f2499ffb640 1 -- 172.21.15.39:0/3362377606 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f24a0012c40 con 0x7f24a4071dd0 2024-09-13T17:10:54.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f2499ffb640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2488066690 0x7f2488068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:54.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f2499ffb640 1 -- 172.21.15.39:0/3362377606 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f24a0085fe0 con 0x7f24a4071dd0 2024-09-13T17:10:54.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f24a8c89640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2488066690 0x7f2488068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:54.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.840+0000 7f24a8c89640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2488066690 0x7f2488068b50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f24a40af700 tx=0x7f2494009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:54.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:54.844+0000 7f2499ffb640 1 -- 172.21.15.39:0/3362377606 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f24a00524e0 con 0x7f24a4071dd0 2024-09-13T17:10:55.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.056+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24a4071bb0 con 0x7f24a4071dd0 2024-09-13T17:10:55.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.056+0000 7f2499ffb640 1 -- 172.21.15.39:0/3362377606 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f24a0007380 con 0x7f24a4071dd0 2024-09-13T17:10:55.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:55.059 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:55.060 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 162 2024-09-13T17:10:55.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2488066690 msgr2=0x7f2488068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:55.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2488066690 0x7f2488068b50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f24a40af700 tx=0x7f2494009040 comp rx=0 tx=0).stop 2024-09-13T17:10:55.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a4071dd0 msgr2=0x7f24a41ecce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:55.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a4071dd0 0x7f24a41ecce0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f24a000d5f0 tx=0x7f24a000dab0 comp rx=0 tx=0).stop 2024-09-13T17:10:55.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 shutdown_connections 2024-09-13T17:10:55.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2488066690 0x7f2488068b50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:55.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24a4151500 0x7f24a414d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:55.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f24a40c47c0 0x7f24a4148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:55.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 --2- 172.21.15.39:0/3362377606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24a4071dd0 0x7f24a41ecce0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:55.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 >> 172.21.15.39:0/3362377606 conn(0x7f24a40ab530 msgr2=0x7f24a40ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:55.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 shutdown_connections 2024-09-13T17:10:55.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:55.060+0000 7f24ab6ee640 1 -- 172.21.15.39:0/3362377606 wait complete. 2024-09-13T17:10:55.077 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:10:57.078 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:57.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f86435fc640 1 Processor -- start 2024-09-13T17:10:57.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f86435fc640 1 -- start start 2024-09-13T17:10:57.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f86435fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f863c0c20d0 0x7f863c0c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f86435fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f863c0c1790 0x7f863c157800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f86435fc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c06d370 0x7f863c157d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f86435fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c0c04a0 con 0x7f863c0c20d0 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f86435fc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c157370 con 0x7f863c0c1790 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f86435fc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863c158280 con 0x7f863c06d370 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8641398640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f863c0c20d0 0x7f863c0c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8641398640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f863c0c20d0 0x7f863c0c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37624/0 (socket says 172.21.15.39:37624) 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8641398640 1 -- 172.21.15.39:0/3995681364 learned_addr learned my addr 172.21.15.39:0/3995681364 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8641b99640 1 --2- 172.21.15.39:0/3995681364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c06d370 0x7f863c157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8640b97640 1 --2- 172.21.15.39:0/3995681364 >> v2:172.21.15.84:3300/0 conn(0x7f863c0c1790 0x7f863c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8641398640 1 -- 172.21.15.39:0/3995681364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c06d370 msgr2=0x7f863c157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8641398640 1 --2- 172.21.15.39:0/3995681364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c06d370 0x7f863c157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8641398640 1 -- 172.21.15.39:0/3995681364 >> v2:172.21.15.84:3300/0 conn(0x7f863c0c1790 msgr2=0x7f863c157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8641398640 1 --2- 172.21.15.39:0/3995681364 >> v2:172.21.15.84:3300/0 conn(0x7f863c0c1790 0x7f863c157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.211+0000 7f8641398640 1 -- 172.21.15.39:0/3995681364 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c153df0 con 0x7f863c0c20d0 2024-09-13T17:10:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f8641398640 1 --2- 172.21.15.39:0/3995681364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c20d0 0x7f863c0c24b0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f863c06eca0 tx=0x7f863801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81083517ff8cf0d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f8633fff640 1 -- 172.21.15.39:0/3995681364 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86380aa610 con 0x7f863c0c20d0 2024-09-13T17:10:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f8633fff640 1 -- 172.21.15.39:0/3995681364 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86380a6040 con 0x7f863c0c20d0 2024-09-13T17:10:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f8633fff640 1 -- 172.21.15.39:0/3995681364 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86380aa990 con 0x7f863c0c20d0 2024-09-13T17:10:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 -- 172.21.15.39:0/3995681364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c20d0 msgr2=0x7f863c0c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 --2- 172.21.15.39:0/3995681364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c20d0 0x7f863c0c24b0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f863c06eca0 tx=0x7f863801c4f0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 -- 172.21.15.39:0/3995681364 shutdown_connections 2024-09-13T17:10:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 --2- 172.21.15.39:0/3995681364 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c06d370 0x7f863c157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 --2- 172.21.15.39:0/3995681364 >> v2:172.21.15.84:3300/0 conn(0x7f863c0c1790 0x7f863c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 --2- 172.21.15.39:0/3995681364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c20d0 0x7f863c0c24b0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 -- 172.21.15.39:0/3995681364 >> 172.21.15.39:0/3995681364 conn(0x7f863c069e10 msgr2=0x7f863c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 -- 172.21.15.39:0/3995681364 shutdown_connections 2024-09-13T17:10:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 -- 172.21.15.39:0/3995681364 wait complete. 2024-09-13T17:10:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 Processor -- start 2024-09-13T17:10:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 -- start start 2024-09-13T17:10:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f863c06d370 0x7f863c1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c1790 0x7f863c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c0c20d0 0x7f863c158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c15a250 con 0x7f863c0c1790 2024-09-13T17:10:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1ed020 con 0x7f863c06d370 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f86435fc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1ed300 con 0x7f863c0c20d0 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f8640b97640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c1790 0x7f863c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f8640b97640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c1790 0x7f863c158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37626/0 (socket says 172.21.15.39:37626) 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f8640b97640 1 -- 172.21.15.39:0/2886700531 learned_addr learned my addr 172.21.15.39:0/2886700531 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f8641398640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f863c06d370 0x7f863c1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.215+0000 7f8641b99640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c0c20d0 0x7f863c158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8640b97640 1 -- 172.21.15.39:0/2886700531 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c0c20d0 msgr2=0x7f863c158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8640b97640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c0c20d0 0x7f863c158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8640b97640 1 -- 172.21.15.39:0/2886700531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f863c06d370 msgr2=0x7f863c1523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8640b97640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f863c06d370 0x7f863c1523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8640b97640 1 -- 172.21.15.39:0/2886700531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86380094d0 con 0x7f863c0c1790 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8640b97640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c1790 0x7f863c158830 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f862c00d5f0 tx=0x7f862c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8631ffb640 1 -- 172.21.15.39:0/2886700531 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0125f0 con 0x7f863c0c1790 2024-09-13T17:10:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8631ffb640 1 -- 172.21.15.39:0/2886700531 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c012750 con 0x7f863c0c1790 2024-09-13T17:10:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c15a4a0 con 0x7f863c0c1790 2024-09-13T17:10:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8631ffb640 1 -- 172.21.15.39:0/2886700531 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c012ae0 con 0x7f863c0c1790 2024-09-13T17:10:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c1592b0 con 0x7f863c0c1790 2024-09-13T17:10:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c0c24b0 con 0x7f863c0c1790 2024-09-13T17:10:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8631ffb640 1 -- 172.21.15.39:0/2886700531 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f862c012c40 con 0x7f863c0c1790 2024-09-13T17:10:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8631ffb640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8620066690 0x7f8620068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8641398640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8620066690 0x7f8620068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8631ffb640 1 -- 172.21.15.39:0/2886700531 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f862c085460 con 0x7f863c0c1790 2024-09-13T17:10:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.219+0000 7f8641398640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8620066690 0x7f8620068b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f86380ae000 tx=0x7f86380a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:57.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.223+0000 7f8631ffb640 1 -- 172.21.15.39:0/2886700531 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f862c051a00 con 0x7f863c0c1790 2024-09-13T17:10:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.423+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f863c1ed4c0 con 0x7f863c0c1790 2024-09-13T17:10:57.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.423+0000 7f8631ffb640 1 -- 172.21.15.39:0/2886700531 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f863c1ed4c0 con 0x7f863c0c1790 2024-09-13T17:10:57.427 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:57.427 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:57.427 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 162 2024-09-13T17:10:57.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8620066690 msgr2=0x7f8620068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:57.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8620066690 0x7f8620068b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f86380ae000 tx=0x7f86380a7000 comp rx=0 tx=0).stop 2024-09-13T17:10:57.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c1790 msgr2=0x7f863c158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:57.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c1790 0x7f863c158830 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f862c00d5f0 tx=0x7f862c00dab0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 shutdown_connections 2024-09-13T17:10:57.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8620066690 0x7f8620068b50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f863c0c20d0 0x7f863c158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f863c0c1790 0x7f863c158830 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 --2- 172.21.15.39:0/2886700531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f863c06d370 0x7f863c1523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:57.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 >> 172.21.15.39:0/2886700531 conn(0x7f863c069e10 msgr2=0x7f863c06f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:57.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 shutdown_connections 2024-09-13T17:10:57.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:57.427+0000 7f86435fc640 1 -- 172.21.15.39:0/2886700531 wait complete. 2024-09-13T17:10:57.446 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:10:59.447 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:10:59.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc9e01640 1 Processor -- start 2024-09-13T17:10:59.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc9e01640 1 -- start start 2024-09-13T17:10:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc9e01640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7bc40c5330 0x7f7bc40c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc9e01640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7bc406c090 0x7f7bc406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc9e01640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc41546c0 con 0x7f7bc4150f80 2024-09-13T17:10:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc9e01640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc40c19a0 con 0x7f7bc406c090 2024-09-13T17:10:59.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc9e01640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc40c49f0 con 0x7f7bc40c5330 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc37fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7bc4150f80 0x7f7bc4153930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57348/0 (socket says 172.21.15.39:57348) 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc37fe640 1 -- 172.21.15.39:0/2955183447 learned_addr learned my addr 172.21.15.39:0/2955183447 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc3fff640 1 --2- 172.21.15.39:0/2955183447 >> v2:172.21.15.39:3300/0 conn(0x7f7bc406c090 0x7f7bc406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc2ffd640 1 --2- 172.21.15.39:0/2955183447 >> v2:172.21.15.84:3300/0 conn(0x7f7bc40c5330 0x7f7bc40c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc27fc640 1 -- 172.21.15.39:0/2955183447 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092081715 0 0) 0x7f7bc41546c0 con 0x7f7bc4150f80 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc27fc640 1 -- 172.21.15.39:0/2955183447 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb0003530 con 0x7f7bc4150f80 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc3fff640 1 -- 172.21.15.39:0/2955183447 >> v2:172.21.15.84:3300/0 conn(0x7f7bc40c5330 msgr2=0x7f7bc40c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc3fff640 1 --2- 172.21.15.39:0/2955183447 >> v2:172.21.15.84:3300/0 conn(0x7f7bc40c5330 0x7f7bc40c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc3fff640 1 -- 172.21.15.39:0/2955183447 >> v1:172.21.15.39:6789/0 conn(0x7f7bc4150f80 legacy=0x7f7bc4153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:59.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.575+0000 7f7bc3fff640 1 -- 172.21.15.39:0/2955183447 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc41543e0 con 0x7f7bc406c090 2024-09-13T17:10:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc3fff640 1 --2- 172.21.15.39:0/2955183447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc406c090 0x7f7bc406c540 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40097c0 tx=0x7f7bb401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b357ad0cb4e871e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc27fc640 1 -- 172.21.15.39:0/2955183447 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb40ab600 con 0x7f7bc406c090 2024-09-13T17:10:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc27fc640 1 -- 172.21.15.39:0/2955183447 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb40a7050 con 0x7f7bc406c090 2024-09-13T17:10:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc27fc640 1 -- 172.21.15.39:0/2955183447 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb40ab980 con 0x7f7bc406c090 2024-09-13T17:10:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 -- 172.21.15.39:0/2955183447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc406c090 msgr2=0x7f7bc406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 --2- 172.21.15.39:0/2955183447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc406c090 0x7f7bc406c540 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40097c0 tx=0x7f7bb401c3a0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 -- 172.21.15.39:0/2955183447 shutdown_connections 2024-09-13T17:10:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 --2- 172.21.15.39:0/2955183447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc406c090 0x7f7bc406c540 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 --2- 172.21.15.39:0/2955183447 >> v2:172.21.15.84:3300/0 conn(0x7f7bc40c5330 0x7f7bc40c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 -- 172.21.15.39:0/2955183447 >> 172.21.15.39:0/2955183447 conn(0x7f7bc4069e10 msgr2=0x7f7bc406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 -- 172.21.15.39:0/2955183447 shutdown_connections 2024-09-13T17:10:59.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 -- 172.21.15.39:0/2955183447 wait complete. 2024-09-13T17:10:59.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 Processor -- start 2024-09-13T17:10:59.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 -- start start 2024-09-13T17:10:59.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7bc406c090 0x7f7bc406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:59.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc40c5330 0x7f7bc4159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:59.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bc4150f80 0x7f7bc41ec410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc4071a40 con 0x7f7bc40c5330 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40aae60 con 0x7f7bc4150f80 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc9e01640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40ab140 con 0x7f7bc406c090 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc40c5330 0x7f7bc4159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc40c5330 0x7f7bc4159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37640/0 (socket says 172.21.15.39:37640) 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.579+0000 7f7bc2ffd640 1 -- 172.21.15.39:0/176591807 learned_addr learned my addr 172.21.15.39:0/176591807 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc3fff640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bc4150f80 0x7f7bc41ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc37fe640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7bc406c090 0x7f7bc406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc2ffd640 1 -- 172.21.15.39:0/176591807 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7bc406c090 msgr2=0x7f7bc406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc2ffd640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7bc406c090 0x7f7bc406fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc2ffd640 1 -- 172.21.15.39:0/176591807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bc4150f80 msgr2=0x7f7bc41ec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:59.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc2ffd640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bc4150f80 0x7f7bc41ec410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc2ffd640 1 -- 172.21.15.39:0/176591807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4009420 con 0x7f7bc40c5330 2024-09-13T17:10:59.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc3fff640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bc4150f80 0x7f7bc41ec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:10:59.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc2ffd640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc40c5330 0x7f7bc4159090 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00afd0 tx=0x7f7bac00eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:59.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bbbfff640 1 -- 172.21.15.39:0/176591807 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00ca30 con 0x7f7bc40c5330 2024-09-13T17:10:59.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bbbfff640 1 -- 172.21.15.39:0/176591807 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00cb90 con 0x7f7bc40c5330 2024-09-13T17:10:59.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc40716d0 con 0x7f7bc40c5330 2024-09-13T17:10:59.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bbbfff640 1 -- 172.21.15.39:0/176591807 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0113f0 con 0x7f7bc40c5330 2024-09-13T17:10:59.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc4071380 con 0x7f7bc40c5330 2024-09-13T17:10:59.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc4071bf0 con 0x7f7bc40c5330 2024-09-13T17:10:59.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bbbfff640 1 -- 172.21.15.39:0/176591807 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7bac011550 con 0x7f7bc40c5330 2024-09-13T17:10:59.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bbbfff640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ba4066690 0x7f7ba4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:10:59.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bbbfff640 1 -- 172.21.15.39:0/176591807 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f7bac084c20 con 0x7f7bc40c5330 2024-09-13T17:10:59.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc37fe640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ba4066690 0x7f7ba4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:10:59.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.583+0000 7f7bc37fe640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ba4066690 0x7f7ba4068b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc001570 tx=0x7f7bbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:10:59.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.587+0000 7f7bbbfff640 1 -- 172.21.15.39:0/176591807 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7bc4071bf0 con 0x7f7bc40c5330 2024-09-13T17:10:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.783+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bc4071bf0 con 0x7f7bc40c5330 2024-09-13T17:10:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bbbfff640 1 -- 172.21.15.39:0/176591807 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f7bc4071bf0 con 0x7f7bc40c5330 2024-09-13T17:10:59.795 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:10:59.795 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:10:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 162 2024-09-13T17:10:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ba4066690 msgr2=0x7f7ba4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ba4066690 0x7f7ba4068b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc001570 tx=0x7f7bbc009040 comp rx=0 tx=0).stop 2024-09-13T17:10:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc40c5330 msgr2=0x7f7bc4159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:10:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc40c5330 0x7f7bc4159090 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00afd0 tx=0x7f7bac00eb40 comp rx=0 tx=0).stop 2024-09-13T17:10:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 shutdown_connections 2024-09-13T17:10:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ba4066690 0x7f7ba4068b50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7bc4150f80 0x7f7bc41ec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bc40c5330 0x7f7bc4159090 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 --2- 172.21.15.39:0/176591807 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7bc406c090 0x7f7bc406fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:10:59.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 >> 172.21.15.39:0/176591807 conn(0x7f7bc4069e10 msgr2=0x7f7bc4152340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:10:59.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 shutdown_connections 2024-09-13T17:10:59.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:10:59.787+0000 7f7bc9e01640 1 -- 172.21.15.39:0/176591807 wait complete. 2024-09-13T17:10:59.809 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:11:00.422 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:11:00.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:11:01.809 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:01.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.939+0000 7f5f8b197640 1 Processor -- start 2024-09-13T17:11:01.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.939+0000 7f5f8b197640 1 -- start start 2024-09-13T17:11:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f8b197640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5f840c1ab0 0x7f5f840c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f8b197640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8414f920 con 0x7f5f84155200 2024-09-13T17:11:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f8b197640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f840c1780 con 0x7f5f8414f540 2024-09-13T17:11:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f8b197640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f840709c0 con 0x7f5f840c1ab0 2024-09-13T17:11:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f89734640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5f84155200 0x7f5f84155670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34082/0 (socket says 172.21.15.39:34082) 2024-09-13T17:11:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f89734640 1 -- 172.21.15.39:0/890645801 learned_addr learned my addr 172.21.15.39:0/890645801 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f83fff640 1 --2- 172.21.15.39:0/890645801 >> v2:172.21.15.84:3300/0 conn(0x7f5f840c1ab0 0x7f5f840c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407117882 0 0) 0x7f5f8414f920 con 0x7f5f84155200 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f6c003530 con 0x7f5f84155200 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2200730673 0 0) 0x7f5f6c003530 con 0x7f5f84155200 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 >> v2:172.21.15.84:3300/0 conn(0x7f5f840c1ab0 msgr2=0x7f5f840c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 --2- 172.21.15.39:0/890645801 >> v2:172.21.15.84:3300/0 conn(0x7f5f840c1ab0 0x7f5f840c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 >> v1:172.21.15.84:6789/0 conn(0x7f5f8414f540 legacy=0x7f5f8406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f84071580 con 0x7f5f84155200 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f5f7c001680 con 0x7f5f84155200 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 >> v1:172.21.15.39:6789/0 conn(0x7f5f84155200 legacy=0x7f5f84155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f6c003980 0x7f5f6c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6c0047d0 0x7f5f6c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f6c005190 0x7f5f6c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c00c000 con 0x7f5f6c0047d0 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c00c170 con 0x7f5f6c003980 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c00c330 con 0x7f5f6c005190 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f83fff640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6c0047d0 0x7f5f6c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f88f33640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f6c003980 0x7f5f6c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f89734640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f6c005190 0x7f5f6c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f83fff640 1 -- 172.21.15.39:0/890645801 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f6c005190 msgr2=0x7f5f6c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f83fff640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f6c005190 0x7f5f6c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f83fff640 1 -- 172.21.15.39:0/890645801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f6c003980 msgr2=0x7f5f6c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f83fff640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f6c003980 0x7f5f6c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f89734640 1 -- 172.21.15.39:0/890645801 reap_dead start 2024-09-13T17:11:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.943+0000 7f5f83fff640 1 -- 172.21.15.39:0/890645801 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c003530 con 0x7f5f6c0047d0 2024-09-13T17:11:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f83fff640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6c0047d0 0x7f5f6c004c50 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009ae0 tx=0x7f5f7801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab3d6903ec85450 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780a5550 con 0x7f5f6c0047d0 2024-09-13T17:11:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f780a56b0 con 0x7f5f6c0047d0 2024-09-13T17:11:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f837fe640 1 -- 172.21.15.39:0/890645801 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780a5a60 con 0x7f5f6c0047d0 2024-09-13T17:11:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 -- 172.21.15.39:0/890645801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6c0047d0 msgr2=0x7f5f6c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6c0047d0 0x7f5f6c004c50 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009ae0 tx=0x7f5f7801ca70 comp rx=0 tx=0).stop 2024-09-13T17:11:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 -- 172.21.15.39:0/890645801 shutdown_connections 2024-09-13T17:11:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 --2- 172.21.15.39:0/890645801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6c0047d0 0x7f5f6c004c50 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 -- 172.21.15.39:0/890645801 >> 172.21.15.39:0/890645801 conn(0x7f5f840c32a0 msgr2=0x7f5f840c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 -- 172.21.15.39:0/890645801 shutdown_connections 2024-09-13T17:11:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 -- 172.21.15.39:0/890645801 wait complete. 2024-09-13T17:11:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 Processor -- start 2024-09-13T17:11:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 -- start start 2024-09-13T17:11:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f6c0047d0 0x7f5f84146250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f84145d90 0x7f5f8414acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f840bc740 0x7f5f840bcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f840c08a0 con 0x7f5f84145d90 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f840c0af0 con 0x7f5f840bc740 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f8b197640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f841eb380 con 0x7f5f6c0047d0 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f83fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f84145d90 0x7f5f8414acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f83fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f84145d90 0x7f5f8414acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38434/0 (socket says 172.21.15.39:38434) 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.947+0000 7f5f83fff640 1 -- 172.21.15.39:0/3539550922 learned_addr learned my addr 172.21.15.39:0/3539550922 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f88f33640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f6c0047d0 0x7f5f84146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f89734640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f840bc740 0x7f5f840bcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f83fff640 1 -- 172.21.15.39:0/3539550922 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f6c0047d0 msgr2=0x7f5f84146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f83fff640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f6c0047d0 0x7f5f84146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f83fff640 1 -- 172.21.15.39:0/3539550922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f840bc740 msgr2=0x7f5f840bcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f83fff640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f840bc740 0x7f5f840bcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f83fff640 1 -- 172.21.15.39:0/3539550922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f841eb660 con 0x7f5f84145d90 2024-09-13T17:11:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f83fff640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f84145d90 0x7f5f8414acd0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5f7801c8a0 tx=0x7f5f78019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f817fa640 1 -- 172.21.15.39:0/3539550922 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78004020 con 0x7f5f84145d90 2024-09-13T17:11:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f817fa640 1 -- 172.21.15.39:0/3539550922 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f780041c0 con 0x7f5f84145d90 2024-09-13T17:11:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f817fa640 1 -- 172.21.15.39:0/3539550922 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f780b8660 con 0x7f5f84145d90 2024-09-13T17:11:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8414b210 con 0x7f5f84145d90 2024-09-13T17:11:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f840bba70 con 0x7f5f84145d90 2024-09-13T17:11:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f840619c0 con 0x7f5f84145d90 2024-09-13T17:11:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f817fa640 1 -- 172.21.15.39:0/3539550922 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f780b4070 con 0x7f5f84145d90 2024-09-13T17:11:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f817fa640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f70066690 0x7f5f70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f817fa640 1 -- 172.21.15.39:0/3539550922 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f5f7811f440 con 0x7f5f84145d90 2024-09-13T17:11:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f88f33640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f70066690 0x7f5f70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.951+0000 7f5f88f33640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f70066690 0x7f5f70068b50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5f74000f90 tx=0x7f5f74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:01.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:01.955+0000 7f5f817fa640 1 -- 172.21.15.39:0/3539550922 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5f780eba90 con 0x7f5f84145d90 2024-09-13T17:11:02.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.155+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f841eb7f0 con 0x7f5f84145d90 2024-09-13T17:11:02.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f817fa640 1 -- 172.21.15.39:0/3539550922 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f5f780bc090 con 0x7f5f84145d90 2024-09-13T17:11:02.161 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:02.161 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:02.162 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 162 2024-09-13T17:11:02.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f70066690 msgr2=0x7f5f70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f70066690 0x7f5f70068b50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5f74000f90 tx=0x7f5f74009040 comp rx=0 tx=0).stop 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f84145d90 msgr2=0x7f5f8414acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f84145d90 0x7f5f8414acd0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5f7801c8a0 tx=0x7f5f78019810 comp rx=0 tx=0).stop 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 shutdown_connections 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5f70066690 0x7f5f70068b50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f840bc740 0x7f5f840bcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f84145d90 0x7f5f8414acd0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 --2- 172.21.15.39:0/3539550922 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5f6c0047d0 0x7f5f84146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 >> 172.21.15.39:0/3539550922 conn(0x7f5f840c32a0 msgr2=0x7f5f8406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 shutdown_connections 2024-09-13T17:11:02.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:02.159+0000 7f5f8b197640 1 -- 172.21.15.39:0/3539550922 wait complete. 2024-09-13T17:11:02.181 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:11:04.181 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:04.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7efff1b0c640 1 Processor -- start 2024-09-13T17:11:04.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7efff1b0c640 1 -- start start 2024-09-13T17:11:04.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7efff1b0c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c2690 0x7effec06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:04.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7efff1b0c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7effec0c6290 0x7effec1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:04.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7efff1b0c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7effec0c0be0 0x7effec157d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:04.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7efff1b0c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effec0c4a20 con 0x7effec0c0be0 2024-09-13T17:11:04.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7efff1b0c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effec157330 con 0x7effec0c6290 2024-09-13T17:11:04.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7efff1b0c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec158240 con 0x7effec0c2690 2024-09-13T17:11:04.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effebfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7effec0c0be0 0x7effec157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:04.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effebfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7effec0c0be0 0x7effec157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38450/0 (socket says 172.21.15.39:38450) 2024-09-13T17:11:04.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effebfff640 1 -- 172.21.15.39:0/627071086 learned_addr learned my addr 172.21.15.39:0/627071086 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:04.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effeb7fe640 1 --2- 172.21.15.39:0/627071086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c2690 0x7effec06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:04.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effeaffd640 1 --2- 172.21.15.39:0/627071086 >> v2:172.21.15.84:3300/0 conn(0x7effec0c6290 0x7effec1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:04.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effebfff640 1 -- 172.21.15.39:0/627071086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c2690 msgr2=0x7effec06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:04.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effebfff640 1 --2- 172.21.15.39:0/627071086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c2690 0x7effec06ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effebfff640 1 -- 172.21.15.39:0/627071086 >> v2:172.21.15.84:3300/0 conn(0x7effec0c6290 msgr2=0x7effec1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:04.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effebfff640 1 --2- 172.21.15.39:0/627071086 >> v2:172.21.15.84:3300/0 conn(0x7effec0c6290 0x7effec1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.315+0000 7effebfff640 1 -- 172.21.15.39:0/627071086 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec0c4740 con 0x7effec0c0be0 2024-09-13T17:11:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7effebfff640 1 --2- 172.21.15.39:0/627071086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c0be0 0x7effec157d00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7effec0c77e0 tx=0x7effe401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c100c82930aa2d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7effea7fc640 1 -- 172.21.15.39:0/627071086 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe40a57b0 con 0x7effec0c0be0 2024-09-13T17:11:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7effea7fc640 1 -- 172.21.15.39:0/627071086 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe40a5910 con 0x7effec0c0be0 2024-09-13T17:11:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7effea7fc640 1 -- 172.21.15.39:0/627071086 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe40a5ca0 con 0x7effec0c0be0 2024-09-13T17:11:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 -- 172.21.15.39:0/627071086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c0be0 msgr2=0x7effec157d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 --2- 172.21.15.39:0/627071086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c0be0 0x7effec157d00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7effec0c77e0 tx=0x7effe401c930 comp rx=0 tx=0).stop 2024-09-13T17:11:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 -- 172.21.15.39:0/627071086 shutdown_connections 2024-09-13T17:11:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 --2- 172.21.15.39:0/627071086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c0be0 0x7effec157d00 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 --2- 172.21.15.39:0/627071086 >> v2:172.21.15.84:3300/0 conn(0x7effec0c6290 0x7effec1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 --2- 172.21.15.39:0/627071086 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c2690 0x7effec06ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 -- 172.21.15.39:0/627071086 >> 172.21.15.39:0/627071086 conn(0x7effec0c32a0 msgr2=0x7effec0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 -- 172.21.15.39:0/627071086 shutdown_connections 2024-09-13T17:11:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 -- 172.21.15.39:0/627071086 wait complete. 2024-09-13T17:11:04.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 Processor -- start 2024-09-13T17:11:04.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.319+0000 7efff1b0c640 1 -- start start 2024-09-13T17:11:04.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7efff1b0c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c0be0 0x7effec14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:04.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7efff1b0c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c2690 0x7effec14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:04.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7efff1b0c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec0c6290 0x7effec156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:04.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7efff1b0c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec153790 con 0x7effec0c2690 2024-09-13T17:11:04.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7efff1b0c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec14e8e0 con 0x7effec0c6290 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7efff1b0c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec14d950 con 0x7effec0c0be0 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c2690 0x7effec14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c2690 0x7effec14c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38464/0 (socket says 172.21.15.39:38464) 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeaffd640 1 -- 172.21.15.39:0/3097932982 learned_addr learned my addr 172.21.15.39:0/3097932982 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeb7fe640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c0be0 0x7effec14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effebfff640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec0c6290 0x7effec156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeaffd640 1 -- 172.21.15.39:0/3097932982 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c0be0 msgr2=0x7effec14e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeaffd640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c0be0 0x7effec14e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeaffd640 1 -- 172.21.15.39:0/3097932982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec0c6290 msgr2=0x7effec156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeaffd640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec0c6290 0x7effec156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeaffd640 1 -- 172.21.15.39:0/3097932982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe4009b30 con 0x7effec0c2690 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effeaffd640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c2690 0x7effec14c790 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7effe000d830 tx=0x7effe000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effd3fff640 1 -- 172.21.15.39:0/3097932982 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0004030 con 0x7effec0c2690 2024-09-13T17:11:04.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effd3fff640 1 -- 172.21.15.39:0/3097932982 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe0004190 con 0x7effec0c2690 2024-09-13T17:11:04.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec1ed3b0 con 0x7effec0c2690 2024-09-13T17:11:04.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effd3fff640 1 -- 172.21.15.39:0/3097932982 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0004520 con 0x7effec0c2690 2024-09-13T17:11:04.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec14dba0 con 0x7effec0c2690 2024-09-13T17:11:04.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec066a20 con 0x7effec0c2690 2024-09-13T17:11:04.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effd3fff640 1 -- 172.21.15.39:0/3097932982 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7effe000b8d0 con 0x7effec0c2690 2024-09-13T17:11:04.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.323+0000 7effd3fff640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd80666c0 0x7effd8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:04.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.327+0000 7effd3fff640 1 -- 172.21.15.39:0/3097932982 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7effe0085290 con 0x7effec0c2690 2024-09-13T17:11:04.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.327+0000 7effeb7fe640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd80666c0 0x7effd8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:04.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.327+0000 7effeb7fe640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd80666c0 0x7effd8068b80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7effec14c4f0 tx=0x7effdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:04.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.331+0000 7effd3fff640 1 -- 172.21.15.39:0/3097932982 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7effe0051830 con 0x7effec0c2690 2024-09-13T17:11:04.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.527+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effec0c0470 con 0x7effec0c2690 2024-09-13T17:11:04.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.527+0000 7effd3fff640 1 -- 172.21.15.39:0/3097932982 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5937 (secure 0 0 0) 0x7effe0054e50 con 0x7effec0c2690 2024-09-13T17:11:04.530 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:04.531 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:04.531 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 162 2024-09-13T17:11:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd80666c0 msgr2=0x7effd8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd80666c0 0x7effd8068b80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7effec14c4f0 tx=0x7effdc008040 comp rx=0 tx=0).stop 2024-09-13T17:11:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c2690 msgr2=0x7effec14c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c2690 0x7effec14c790 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7effe000d830 tx=0x7effe000dcf0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 shutdown_connections 2024-09-13T17:11:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7effd80666c0 0x7effd8068b80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7effec0c6290 0x7effec156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effec0c2690 0x7effec14c790 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 --2- 172.21.15.39:0/3097932982 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7effec0c0be0 0x7effec14e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:04.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 >> 172.21.15.39:0/3097932982 conn(0x7effec0c32a0 msgr2=0x7effec0c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:04.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 shutdown_connections 2024-09-13T17:11:04.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:04.531+0000 7efff1b0c640 1 -- 172.21.15.39:0/3097932982 wait complete. 2024-09-13T17:11:04.550 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:11:06.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:06.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47f517640 1 Processor -- start 2024-09-13T17:11:06.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47f517640 1 -- start start 2024-09-13T17:11:06.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47f517640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb47806c870 0x7fb4780aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:06.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47f517640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb478152a80 con 0x7fb4780aabc0 2024-09-13T17:11:06.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47f517640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4780c30f0 con 0x7fb4780c40b0 2024-09-13T17:11:06.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47f517640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb478151e00 con 0x7fb47806c870 2024-09-13T17:11:06.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47d2b3640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb4780aabc0 0x7fb4780c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34086/0 (socket says 172.21.15.39:34086) 2024-09-13T17:11:06.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47d2b3640 1 -- 172.21.15.39:0/519004041 learned_addr learned my addr 172.21.15.39:0/519004041 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:06.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47cab2640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb47806c870 0x7fb4780aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:06.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833033849 0 0) 0x7fb474001120 con 0x7fb4780aabc0 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb464003530 con 0x7fb4780aabc0 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3117100267 0 0) 0x7fb464003530 con 0x7fb4780aabc0 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb47806c870 msgr2=0x7fb4780aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb47806c870 0x7fb4780aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 >> v1:172.21.15.84:6789/0 conn(0x7fb4780c40b0 legacy=0x7fb47815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4781520e0 con 0x7fb4780aabc0 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fb474001810 con 0x7fb4780aabc0 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 >> v1:172.21.15.39:6789/0 conn(0x7fb4780aabc0 legacy=0x7fb4780c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4640039b0 0x7fb464003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640047d0 0x7fb464004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb47806c870 0x7fb464005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478151e00 con 0x7fb4640039b0 2024-09-13T17:11:06.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46400ba70 con 0x7fb4640047d0 2024-09-13T17:11:06.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46400bc30 con 0x7fb47806c870 2024-09-13T17:11:06.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47d2b3640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4640039b0 0x7fb464003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:06.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47dab4640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb47806c870 0x7fb464005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:06.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47cab2640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640047d0 0x7fb464004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:06.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47d2b3640 1 -- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb47806c870 msgr2=0x7fb464005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:06.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47d2b3640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb47806c870 0x7fb464005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47d2b3640 1 -- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640047d0 msgr2=0x7fb464004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:06.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47d2b3640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4640047d0 0x7fb464004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.683+0000 7fb47d2b3640 1 -- 172.21.15.39:0/519004041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb464003530 con 0x7fb4640039b0 2024-09-13T17:11:06.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47d2b3640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4640039b0 0x7fb464003dd0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c160 tx=0x7fb47401cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7012fd4e1d55003 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474005ae0 con 0x7fb4640039b0 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb474005c60 con 0x7fb4640039b0 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb46ffff640 1 -- 172.21.15.39:0/519004041 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740a58b0 con 0x7fb4640039b0 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47f517640 1 -- 172.21.15.39:0/519004041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4640039b0 msgr2=0x7fb464003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47f517640 1 --2- 172.21.15.39:0/519004041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4640039b0 0x7fb464003dd0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c160 tx=0x7fb47401cbb0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47dab4640 1 -- 172.21.15.39:0/519004041 reap_dead start 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47f517640 1 -- 172.21.15.39:0/519004041 shutdown_connections 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47f517640 1 -- 172.21.15.39:0/519004041 >> 172.21.15.39:0/519004041 conn(0x7fb478069e10 msgr2=0x7fb47806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47f517640 1 -- 172.21.15.39:0/519004041 shutdown_connections 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47f517640 1 -- 172.21.15.39:0/519004041 wait complete. 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47f517640 1 Processor -- start 2024-09-13T17:11:06.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.687+0000 7fb47f517640 1 -- start start 2024-09-13T17:11:06.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47f517640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4640039b0 0x7fb4780a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47f517640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4780aa4e0 0x7fb4781f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47f517640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4781ecc50 0x7fb4781ed340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47f517640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780a8200 con 0x7fb4780aa4e0 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47f517640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780725b0 con 0x7fb4781ecc50 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47f517640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47814e820 con 0x7fb4640039b0 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47cab2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4780aa4e0 0x7fb4781f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47cab2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4780aa4e0 0x7fb4781f0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38474/0 (socket says 172.21.15.39:38474) 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47cab2640 1 -- 172.21.15.39:0/3587545136 learned_addr learned my addr 172.21.15.39:0/3587545136 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47d2b3640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4640039b0 0x7fb4780a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47dab4640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4781ecc50 0x7fb4781ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47cab2640 1 -- 172.21.15.39:0/3587545136 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4640039b0 msgr2=0x7fb4780a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:06.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47cab2640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4640039b0 0x7fb4780a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47cab2640 1 -- 172.21.15.39:0/3587545136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4781ecc50 msgr2=0x7fb4781ed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47cab2640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4781ecc50 0x7fb4781ed340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47cab2640 1 -- 172.21.15.39:0/3587545136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4781f0360 con 0x7fb4780aa4e0 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47cab2640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4780aa4e0 0x7fb4781f0840 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb468010870 tx=0x7fb468010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb46dffb640 1 -- 172.21.15.39:0/3587545136 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4680095f0 con 0x7fb4780aa4e0 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb46dffb640 1 -- 172.21.15.39:0/3587545136 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46800eb00 con 0x7fb4780aa4e0 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47814df80 con 0x7fb4780aa4e0 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb46dffb640 1 -- 172.21.15.39:0/3587545136 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46800ee90 con 0x7fb4780aa4e0 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47814e540 con 0x7fb4780aa4e0 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4781f01c0 con 0x7fb4780aa4e0 2024-09-13T17:11:06.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb46dffb640 1 -- 172.21.15.39:0/3587545136 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb46801c400 con 0x7fb4780aa4e0 2024-09-13T17:11:06.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb46dffb640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb45c0666c0 0x7fb45c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:06.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb46dffb640 1 -- 172.21.15.39:0/3587545136 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7fb4680852c0 con 0x7fb4780aa4e0 2024-09-13T17:11:06.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47d2b3640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb45c0666c0 0x7fb45c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:06.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.691+0000 7fb47d2b3640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb45c0666c0 0x7fb45c068b80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb4740ab000 tx=0x7fb4740a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:06.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.695+0000 7fb46dffb640 1 -- 172.21.15.39:0/3587545136 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb4781f01c0 con 0x7fb4780aa4e0 2024-09-13T17:11:06.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.887+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4781f01c0 con 0x7fb4780aa4e0 2024-09-13T17:11:06.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.887+0000 7fb46dffb640 1 -- 172.21.15.39:0/3587545136 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5937 (secure 0 0 0) 0x7fb4781f01c0 con 0x7fb4780aa4e0 2024-09-13T17:11:06.891 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:06.891 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:06.892 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 162 2024-09-13T17:11:06.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb45c0666c0 msgr2=0x7fb45c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb45c0666c0 0x7fb45c068b80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb4740ab000 tx=0x7fb4740a7070 comp rx=0 tx=0).stop 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4780aa4e0 msgr2=0x7fb4781f0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4780aa4e0 0x7fb4781f0840 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb468010870 tx=0x7fb468010d30 comp rx=0 tx=0).stop 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 shutdown_connections 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb45c0666c0 0x7fb45c068b80 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4781ecc50 0x7fb4781ed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4780aa4e0 0x7fb4781f0840 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 --2- 172.21.15.39:0/3587545136 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb4640039b0 0x7fb4780a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 >> 172.21.15.39:0/3587545136 conn(0x7fb478069e10 msgr2=0x7fb478151320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 shutdown_connections 2024-09-13T17:11:06.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:06.891+0000 7fb47f517640 1 -- 172.21.15.39:0/3587545136 wait complete. 2024-09-13T17:11:06.910 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:11:08.911 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:09.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.039+0000 7f20616f3640 1 Processor -- start 2024-09-13T17:11:09.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.039+0000 7f20616f3640 1 -- start start 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f20616f3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f205c0aabc0 0x7f205c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f20616f3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c0c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f20616f3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f205c0c3ee0 0x7f205c1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f20616f3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c152a80 con 0x7f205c0c3ee0 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f20616f3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c0c30f0 con 0x7f205c0aabc0 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f20616f3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c151e00 con 0x7f205c06c870 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f205c0c3ee0 0x7f205c1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f205c0c3ee0 0x7f205c1550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38490/0 (socket says 172.21.15.39:38490) 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205b7fe640 1 -- 172.21.15.39:0/2388470559 learned_addr learned my addr 172.21.15.39:0/2388470559 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205a7fc640 1 --2- 172.21.15.39:0/2388470559 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c0c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:09.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205affd640 1 --2- 172.21.15.39:0/2388470559 >> v2:172.21.15.84:3300/0 conn(0x7f205c0aabc0 0x7f205c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:09.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205b7fe640 1 -- 172.21.15.39:0/2388470559 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 msgr2=0x7f205c0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:09.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205b7fe640 1 --2- 172.21.15.39:0/2388470559 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c0c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205b7fe640 1 -- 172.21.15.39:0/2388470559 >> v2:172.21.15.84:3300/0 conn(0x7f205c0aabc0 msgr2=0x7f205c0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:09.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205b7fe640 1 --2- 172.21.15.39:0/2388470559 >> v2:172.21.15.84:3300/0 conn(0x7f205c0aabc0 0x7f205c0aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205b7fe640 1 -- 172.21.15.39:0/2388470559 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c1520e0 con 0x7f205c0c3ee0 2024-09-13T17:11:09.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205affd640 1 --2- 172.21.15.39:0/2388470559 >> v2:172.21.15.84:3300/0 conn(0x7f205c0aabc0 0x7f205c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:11:09.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205a7fc640 1 --2- 172.21.15.39:0/2388470559 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c0c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:11:09.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f205b7fe640 1 --2- 172.21.15.39:0/2388470559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 0x7f205c1550e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f205c0c33a0 tx=0x7f204c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c04ab923d6a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:09.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f2059ffb640 1 -- 172.21.15.39:0/2388470559 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0a5840 con 0x7f205c0c3ee0 2024-09-13T17:11:09.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f2059ffb640 1 -- 172.21.15.39:0/2388470559 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c0a59a0 con 0x7f205c0c3ee0 2024-09-13T17:11:09.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.043+0000 7f2059ffb640 1 -- 172.21.15.39:0/2388470559 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0a5d30 con 0x7f205c0c3ee0 2024-09-13T17:11:09.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- 172.21.15.39:0/2388470559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 msgr2=0x7f205c1550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:09.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 --2- 172.21.15.39:0/2388470559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 0x7f205c1550e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f205c0c33a0 tx=0x7f204c01c9c0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- 172.21.15.39:0/2388470559 shutdown_connections 2024-09-13T17:11:09.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 --2- 172.21.15.39:0/2388470559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 0x7f205c1550e0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 --2- 172.21.15.39:0/2388470559 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c0c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 --2- 172.21.15.39:0/2388470559 >> v2:172.21.15.84:3300/0 conn(0x7f205c0aabc0 0x7f205c0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- 172.21.15.39:0/2388470559 >> 172.21.15.39:0/2388470559 conn(0x7f205c069e10 msgr2=0x7f205c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:09.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- 172.21.15.39:0/2388470559 shutdown_connections 2024-09-13T17:11:09.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- 172.21.15.39:0/2388470559 wait complete. 2024-09-13T17:11:09.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 Processor -- start 2024-09-13T17:11:09.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- start start 2024-09-13T17:11:09.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c14ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:09.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c0aabc0 0x7f205c1eba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:09.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 0x7f205c1ebf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:09.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c1ea590 con 0x7f205c0c3ee0 2024-09-13T17:11:09.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c1eae10 con 0x7f205c0aabc0 2024-09-13T17:11:09.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c1ee510 con 0x7f205c06c870 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 0x7f205c1ebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 0x7f205c1ebf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38498/0 (socket says 172.21.15.39:38498) 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205b7fe640 1 -- 172.21.15.39:0/1351537887 learned_addr learned my addr 172.21.15.39:0/1351537887 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205a7fc640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c0aabc0 0x7f205c1eba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205affd640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c14ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205b7fe640 1 -- 172.21.15.39:0/1351537887 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 msgr2=0x7f205c14ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205b7fe640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c14ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205b7fe640 1 -- 172.21.15.39:0/1351537887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c0aabc0 msgr2=0x7f205c1eba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205b7fe640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c0aabc0 0x7f205c1eba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205b7fe640 1 -- 172.21.15.39:0/1351537887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c009b30 con 0x7f205c0c3ee0 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205affd640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c14ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f205b7fe640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 0x7f205c1ebf40 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f204c01b070 tx=0x7f204c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20537fe640 1 -- 172.21.15.39:0/1351537887 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0b8670 con 0x7f205c0c3ee0 2024-09-13T17:11:09.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20537fe640 1 -- 172.21.15.39:0/1351537887 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c0b87d0 con 0x7f205c0c3ee0 2024-09-13T17:11:09.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20537fe640 1 -- 172.21.15.39:0/1351537887 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0b8b60 con 0x7f205c0c3ee0 2024-09-13T17:11:09.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c1ee7f0 con 0x7f205c0c3ee0 2024-09-13T17:11:09.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.047+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c1eaaa0 con 0x7f205c0c3ee0 2024-09-13T17:11:09.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.051+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c1eafc0 con 0x7f205c0c3ee0 2024-09-13T17:11:09.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.051+0000 7f20537fe640 1 -- 172.21.15.39:0/1351537887 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f204c0b8cc0 con 0x7f205c0c3ee0 2024-09-13T17:11:09.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.051+0000 7f20537fe640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f203c066690 0x7f203c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:09.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.051+0000 7f20537fe640 1 -- 172.21.15.39:0/1351537887 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 9303+0+0 (secure 0 0 0) 0x7f204c120570 con 0x7f205c0c3ee0 2024-09-13T17:11:09.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.051+0000 7f205affd640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f203c066690 0x7f203c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:09.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.051+0000 7f205affd640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f203c066690 0x7f203c068b50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2054009970 tx=0x7f2054008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:09.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.055+0000 7f20537fe640 1 -- 172.21.15.39:0/1351537887 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f205c1eafc0 con 0x7f205c0c3ee0 2024-09-13T17:11:09.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.251+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f205c1eaf80 con 0x7f205c0c3ee0 2024-09-13T17:11:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.251+0000 7f20537fe640 1 -- 172.21.15.39:0/1351537887 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5937 (secure 0 0 0) 0x7f204c0bc020 con 0x7f205c0c3ee0 2024-09-13T17:11:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:09.256 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:10:03.150430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7677},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7677":{"gid":7677,"name":"c","rank":0,"incarnation":153,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6837/2351253009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":2351253009},{"type":"v1","addr":"172.21.15.39:6837","nonce":2351253009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:09.256 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 162 2024-09-13T17:11:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f203c066690 msgr2=0x7f203c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f203c066690 0x7f203c068b50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2054009970 tx=0x7f2054008040 comp rx=0 tx=0).stop 2024-09-13T17:11:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 msgr2=0x7f205c1ebf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 0x7f205c1ebf40 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f204c01b070 tx=0x7f204c019760 comp rx=0 tx=0).stop 2024-09-13T17:11:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 shutdown_connections 2024-09-13T17:11:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f203c066690 0x7f203c068b50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c0aabc0 0x7f205c1eba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f205c0c3ee0 0x7f205c1ebf40 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 --2- 172.21.15.39:0/1351537887 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f205c06c870 0x7f205c14ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 >> 172.21.15.39:0/1351537887 conn(0x7f205c069e10 msgr2=0x7f205c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 shutdown_connections 2024-09-13T17:11:09.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:09.255+0000 7f20616f3640 1 -- 172.21.15.39:0/1351537887 wait complete. 2024-09-13T17:11:09.276 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:11:11.278 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:11.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.415+0000 7f59079b2640 1 Processor -- start 2024-09-13T17:11:11.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.415+0000 7f59079b2640 1 -- start start 2024-09-13T17:11:11.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f59079b2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000c20d0 0x7f59000c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f59079b2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59000c1790 0x7f5900157800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f59079b2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f590006d370 0x7f5900157d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f59079b2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59000c04a0 con 0x7f59000c1790 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f59079b2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5900157370 con 0x7f590006d370 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f59079b2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5900158280 con 0x7f59000c20d0 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5904f4d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59000c1790 0x7f5900157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5904f4d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59000c1790 0x7f5900157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43988/0 (socket says 172.21.15.39:43988) 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5904f4d640 1 -- 172.21.15.39:0/61951674 learned_addr learned my addr 172.21.15.39:0/61951674 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f590574e640 1 --2- 172.21.15.39:0/61951674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000c20d0 0x7f59000c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5905f4f640 1 --2- 172.21.15.39:0/61951674 >> v2:172.21.15.84:3300/0 conn(0x7f590006d370 0x7f5900157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5904f4d640 1 -- 172.21.15.39:0/61951674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000c20d0 msgr2=0x7f59000c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5904f4d640 1 --2- 172.21.15.39:0/61951674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000c20d0 0x7f59000c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5904f4d640 1 -- 172.21.15.39:0/61951674 >> v2:172.21.15.84:3300/0 conn(0x7f590006d370 msgr2=0x7f5900157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5904f4d640 1 --2- 172.21.15.39:0/61951674 >> v2:172.21.15.84:3300/0 conn(0x7f590006d370 0x7f5900157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5904f4d640 1 -- 172.21.15.39:0/61951674 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900153df0 con 0x7f59000c1790 2024-09-13T17:11:11.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f5904f4d640 1 --2- 172.21.15.39:0/61951674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c1790 0x7f5900157800 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f59000abb70 tx=0x7f58f001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95850631caa1c55e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:11.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.419+0000 7f58f7fff640 1 -- 172.21.15.39:0/61951674 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f00aa610 con 0x7f59000c1790 2024-09-13T17:11:11.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f58f7fff640 1 -- 172.21.15.39:0/61951674 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f00a6040 con 0x7f59000c1790 2024-09-13T17:11:11.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f58f7fff640 1 -- 172.21.15.39:0/61951674 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f00aa990 con 0x7f59000c1790 2024-09-13T17:11:11.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- 172.21.15.39:0/61951674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c1790 msgr2=0x7f5900157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 --2- 172.21.15.39:0/61951674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c1790 0x7f5900157800 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f59000abb70 tx=0x7f58f001c4f0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- 172.21.15.39:0/61951674 shutdown_connections 2024-09-13T17:11:11.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 --2- 172.21.15.39:0/61951674 >> v2:172.21.15.84:3300/0 conn(0x7f590006d370 0x7f5900157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 --2- 172.21.15.39:0/61951674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c1790 0x7f5900157800 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 --2- 172.21.15.39:0/61951674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000c20d0 0x7f59000c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- 172.21.15.39:0/61951674 >> 172.21.15.39:0/61951674 conn(0x7f5900069e10 msgr2=0x7f590006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:11.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- 172.21.15.39:0/61951674 shutdown_connections 2024-09-13T17:11:11.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- 172.21.15.39:0/61951674 wait complete. 2024-09-13T17:11:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 Processor -- start 2024-09-13T17:11:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- start start 2024-09-13T17:11:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f590006d370 0x7f590014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000c1790 0x7f5900147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c20d0 0x7f5900147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590006f640 con 0x7f59000c20d0 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900070130 con 0x7f59000c1790 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590015c940 con 0x7f590006d370 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5905f4f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c20d0 0x7f5900147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5905f4f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c20d0 0x7f5900147ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43996/0 (socket says 172.21.15.39:43996) 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5905f4f640 1 -- 172.21.15.39:0/2179303254 learned_addr learned my addr 172.21.15.39:0/2179303254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f590574e640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f590006d370 0x7f590014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5904f4d640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000c1790 0x7f5900147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5905f4f640 1 -- 172.21.15.39:0/2179303254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f590006d370 msgr2=0x7f590014af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5905f4f640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f590006d370 0x7f590014af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5905f4f640 1 -- 172.21.15.39:0/2179303254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000c1790 msgr2=0x7f5900147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5905f4f640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000c1790 0x7f5900147580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5905f4f640 1 -- 172.21.15.39:0/2179303254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f00094d0 con 0x7f59000c20d0 2024-09-13T17:11:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5904f4d640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000c1790 0x7f5900147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f5905f4f640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c20d0 0x7f5900147ac0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f58f8007d10 tx=0x7f58f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f58f5ffb640 1 -- 172.21.15.39:0/2179303254 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8014420 con 0x7f59000c20d0 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f58f5ffb640 1 -- 172.21.15.39:0/2179303254 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8014580 con 0x7f59000c20d0 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f58f5ffb640 1 -- 172.21.15.39:0/2179303254 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8014930 con 0x7f59000c20d0 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.423+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590006f890 con 0x7f59000c20d0 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.427+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590006fe50 con 0x7f59000c20d0 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.427+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590006a760 con 0x7f59000c20d0 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.427+0000 7f58f5ffb640 1 -- 172.21.15.39:0/2179303254 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58f8014a90 con 0x7f59000c20d0 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.427+0000 7f58f5ffb640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58e40666c0 0x7f58e4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.427+0000 7f58f5ffb640 1 -- 172.21.15.39:0/2179303254 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7f58f8085820 con 0x7f59000c20d0 2024-09-13T17:11:11.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.427+0000 7f590574e640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58e40666c0 0x7f58e4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.427+0000 7f590574e640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58e40666c0 0x7f58e4068b80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f58fc007a60 tx=0x7f58fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:11.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.431+0000 7f58f5ffb640 1 -- 172.21.15.39:0/2179303254 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f58f8051ce0 con 0x7f59000c20d0 2024-09-13T17:11:11.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59000c24b0 con 0x7f59000c20d0 2024-09-13T17:11:11.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f58f5ffb640 1 -- 172.21.15.39:0/2179303254 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+5182 (secure 0 0 0) 0x7f58f80098c0 con 0x7f59000c20d0 2024-09-13T17:11:11.630 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:11.631 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:11.216387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:11.631 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-13T17:11:11.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58e40666c0 msgr2=0x7f58e4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58e40666c0 0x7f58e4068b80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f58fc007a60 tx=0x7f58fc008040 comp rx=0 tx=0).stop 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c20d0 msgr2=0x7f5900147ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c20d0 0x7f5900147ac0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f58f8007d10 tx=0x7f58f800e4b0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 shutdown_connections 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58e40666c0 0x7f58e4068b80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000c1790 0x7f5900147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59000c20d0 0x7f5900147ac0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 --2- 172.21.15.39:0/2179303254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f590006d370 0x7f590014af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 >> 172.21.15.39:0/2179303254 conn(0x7f5900069e10 msgr2=0x7f5900151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.627+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 shutdown_connections 2024-09-13T17:11:11.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.631+0000 7f59079b2640 1 -- 172.21.15.39:0/2179303254 wait complete. 2024-09-13T17:11:11.650 INFO:tasks.mds_thrash.fs.[a]:mds.c down, removed from mdsmap 2024-09-13T17:11:11.650 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:11:11.650 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:11.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.759+0000 7fd4f8344640 1 Processor -- start 2024-09-13T17:11:11.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.759+0000 7fd4f8344640 1 -- start start 2024-09-13T17:11:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f8344640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f00c5330 0x7fd4f00c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f8344640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f01546c0 con 0x7fd4f006c090 2024-09-13T17:11:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f8344640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f00c49f0 con 0x7fd4f0150f80 2024-09-13T17:11:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f8344640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00c19a0 con 0x7fd4f00c5330 2024-09-13T17:11:11.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f68e1640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd4f006c090 0x7fd4f006c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52568/0 (socket says 172.21.15.39:52568) 2024-09-13T17:11:11.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f68e1640 1 -- 172.21.15.39:0/1471052294 learned_addr learned my addr 172.21.15.39:0/1471052294 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:11.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f58df640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f00c5330 0x7fd4f00c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444960880 0 0) 0x7fd4f01546c0 con 0x7fd4f006c090 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4dc003530 con 0x7fd4f006c090 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3260714895 0 0) 0x7fd4dc003530 con 0x7fd4f006c090 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f00c5330 msgr2=0x7fd4f00c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f00c5330 0x7fd4f00c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 >> v1:172.21.15.84:6789/0 conn(0x7fd4f0150f80 legacy=0x7fd4f0153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f01543e0 con 0x7fd4f006c090 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fd4e8001a00 con 0x7fd4f006c090 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 >> v1:172.21.15.39:6789/0 conn(0x7fd4f006c090 legacy=0x7fd4f006c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f00c5330 0x7fd4dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4dc003f30 0x7fd4dc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4dc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00c19a0 con 0x7fd4dc004d10 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc00bb20 con 0x7fd4dc003f30 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc00bce0 con 0x7fd4f00c5330 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f68e1640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4dc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f58df640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4dc003f30 0x7fd4dc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f68e1640 1 -- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f00c5330 msgr2=0x7fd4dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f68e1640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f00c5330 0x7fd4dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f68e1640 1 -- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4dc003f30 msgr2=0x7fd4dc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f68e1640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4dc003f30 0x7fd4dc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f68e1640 1 -- 172.21.15.39:0/1471052294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc003530 con 0x7fd4dc004d10 2024-09-13T17:11:11.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.763+0000 7fd4f58df640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4dc003f30 0x7fd4dc0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f68e1640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4dc005190 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800a410 tx=0x7fd4e8021cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56456e73f798a89f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80ab8f0 con 0x7fd4dc004d10 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e80aba70 con 0x7fd4dc004d10 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f50de640 1 -- 172.21.15.39:0/1471052294 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80abe20 con 0x7fd4dc004d10 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- 172.21.15.39:0/1471052294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 msgr2=0x7fd4dc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 --2- 172.21.15.39:0/1471052294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4dc005190 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800a410 tx=0x7fd4e8021cd0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f68e1640 1 -- 172.21.15.39:0/1471052294 reap_dead start 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- 172.21.15.39:0/1471052294 shutdown_connections 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- 172.21.15.39:0/1471052294 >> 172.21.15.39:0/1471052294 conn(0x7fd4f0069e10 msgr2=0x7fd4f006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- 172.21.15.39:0/1471052294 shutdown_connections 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- 172.21.15.39:0/1471052294 wait complete. 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 Processor -- start 2024-09-13T17:11:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- start start 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4f0146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f014c740 0x7fd4f014cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4f01e51d0 0x7fd4f01e5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f006fac0 con 0x7fd4dc004d10 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00aaa80 con 0x7fd4f01e51d0 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00717d0 con 0x7fd4f014c740 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f60e0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4f0146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f60e0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4f0146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44018/0 (socket says 172.21.15.39:44018) 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f60e0640 1 -- 172.21.15.39:0/134007491 learned_addr learned my addr 172.21.15.39:0/134007491 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f58df640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f014c740 0x7fd4f014cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f68e1640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4f01e51d0 0x7fd4f01e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f60e0640 1 -- 172.21.15.39:0/134007491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f014c740 msgr2=0x7fd4f014cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f60e0640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f014c740 0x7fd4f014cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f60e0640 1 -- 172.21.15.39:0/134007491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4f01e51d0 msgr2=0x7fd4f01e5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f60e0640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4f01e51d0 0x7fd4f01e5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f60e0640 1 -- 172.21.15.39:0/134007491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f006f570 con 0x7fd4dc004d10 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f58df640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f014c740 0x7fd4f014cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f60e0640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4f0146f70 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c500 tx=0x7fd4ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4e6ffd640 1 -- 172.21.15.39:0/134007491 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0052d0 con 0x7fd4dc004d10 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4e6ffd640 1 -- 172.21.15.39:0/134007491 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec005430 con 0x7fd4dc004d10 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f00aaf10 con 0x7fd4dc004d10 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4e6ffd640 1 -- 172.21.15.39:0/134007491 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec012870 con 0x7fd4dc004d10 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.767+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f006f830 con 0x7fd4dc004d10 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.771+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f006d540 con 0x7fd4dc004d10 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.771+0000 7fd4e6ffd640 1 -- 172.21.15.39:0/134007491 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4ec0129d0 con 0x7fd4dc004d10 2024-09-13T17:11:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.771+0000 7fd4e6ffd640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4d4066690 0x7fd4d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.771+0000 7fd4e6ffd640 1 -- 172.21.15.39:0/134007491 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7fd4ec084b30 con 0x7fd4dc004d10 2024-09-13T17:11:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.771+0000 7fd4f58df640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4d4066690 0x7fd4d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.771+0000 7fd4f58df640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4d4066690 0x7fd4d4068b50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0148b70 tx=0x7fd4e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:11.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.775+0000 7fd4e6ffd640 1 -- 172.21.15.39:0/134007491 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd4ec051080 con 0x7fd4dc004d10 2024-09-13T17:11:11.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.963+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4f0153930 con 0x7fd4dc004d10 2024-09-13T17:11:11.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4e6ffd640 1 -- 172.21.15.39:0/134007491 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+5182 (secure 0 0 0) 0x7fd4ec0546a0 con 0x7fd4dc004d10 2024-09-13T17:11:11.969 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:11.969 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:11.216387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:11.970 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-13T17:11:11.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4d4066690 msgr2=0x7fd4d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4d4066690 0x7fd4d4068b50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0148b70 tx=0x7fd4e0008040 comp rx=0 tx=0).stop 2024-09-13T17:11:11.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 msgr2=0x7fd4f0146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:11.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4f0146f70 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c500 tx=0x7fd4ec00c9c0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 shutdown_connections 2024-09-13T17:11:11.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd4d4066690 0x7fd4d4068b50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4f01e51d0 0x7fd4f01e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd4f014c740 0x7fd4f014cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 --2- 172.21.15.39:0/134007491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4dc004d10 0x7fd4f0146f70 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:11.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 >> 172.21.15.39:0/134007491 conn(0x7fd4f0069e10 msgr2=0x7fd4f0151620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:11.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 shutdown_connections 2024-09-13T17:11:11.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:11.967+0000 7fd4f8344640 1 -- 172.21.15.39:0/134007491 wait complete. 2024-09-13T17:11:11.989 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 164, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 164, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_7851": { "addr": "172.21.15.39:6835/3334565883", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 3334565883, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 3334565883, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7851, "incarnation": 163, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 56, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:11:11.216387+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7851 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 161, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6290": { "addr": "172.21.15.84:6835/397460031", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 397460031, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 397460031, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6290, "incarnation": 158, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:10:43.027964+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6290 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:11:13.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:14.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.139+0000 7f51b0e55640 1 Processor -- start 2024-09-13T17:11:14.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.139+0000 7f51b0e55640 1 -- start start 2024-09-13T17:11:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51b0e55640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51ac154b70 0x7f51ac14df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51b0e55640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac0c33c0 con 0x7f51ac06fba0 2024-09-13T17:11:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51b0e55640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac0c0ac0 con 0x7f51ac0c28d0 2024-09-13T17:11:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51b0e55640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac0611c0 con 0x7f51ac154b70 2024-09-13T17:11:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a9d9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f51ac06fba0 0x7f51ac0c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52578/0 (socket says 172.21.15.39:52578) 2024-09-13T17:11:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a9d9b640 1 -- 172.21.15.39:0/3658938313 learned_addr learned my addr 172.21.15.39:0/3658938313 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aad9d640 1 --2- 172.21.15.39:0/3658938313 >> v2:172.21.15.84:3300/0 conn(0x7f51ac154b70 0x7f51ac14df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902495747 0 0) 0x7f51a0001120 con 0x7f51ac06fba0 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5194003530 con 0x7f51ac06fba0 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1515528785 0 0) 0x7f5194003530 con 0x7f51ac06fba0 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 >> v2:172.21.15.84:3300/0 conn(0x7f51ac154b70 msgr2=0x7f51ac14df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 --2- 172.21.15.39:0/3658938313 >> v2:172.21.15.84:3300/0 conn(0x7f51ac154b70 0x7f51ac14df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 >> v1:172.21.15.84:6789/0 conn(0x7f51ac0c28d0 legacy=0x7f51ac0c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac154f50 con 0x7f51ac06fba0 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f51a0001810 con 0x7f51ac06fba0 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 >> v1:172.21.15.39:6789/0 conn(0x7f51ac06fba0 legacy=0x7f51ac0c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5194003980 0x7f5194003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51940047d0 0x7f5194004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194005190 0x7f519400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0611c0 con 0x7f5194003980 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519400c130 con 0x7f51940047d0 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519400c2f0 con 0x7f5194005190 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aa59c640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5194003980 0x7f5194003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a9d9b640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51940047d0 0x7f5194004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aad9d640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194005190 0x7f519400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aa59c640 1 -- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194005190 msgr2=0x7f519400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aa59c640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194005190 0x7f519400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aa59c640 1 -- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51940047d0 msgr2=0x7f5194004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aa59c640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51940047d0 0x7f5194004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aa59c640 1 -- 172.21.15.39:0/3658938313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194003530 con 0x7f5194003980 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aad9d640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194005190 0x7f519400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51a9d9b640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51940047d0 0x7f5194004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:11:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.143+0000 7f51aad9d640 1 -- 172.21.15.39:0/3658938313 reap_dead start 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51aa59c640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5194003980 0x7f5194003da0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f519c001780 tx=0x7f519c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bf198c6b52679 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c0a4ea0 con 0x7f5194003980 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c0a7050 con 0x7f5194003980 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51a959a640 1 -- 172.21.15.39:0/3658938313 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c003600 con 0x7f5194003980 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51b0e55640 1 -- 172.21.15.39:0/3658938313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5194003980 msgr2=0x7f5194003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51b0e55640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5194003980 0x7f5194003da0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f519c001780 tx=0x7f519c01c400 comp rx=0 tx=0).stop 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51b0e55640 1 -- 172.21.15.39:0/3658938313 shutdown_connections 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51b0e55640 1 --2- 172.21.15.39:0/3658938313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5194003980 0x7f5194003da0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51b0e55640 1 -- 172.21.15.39:0/3658938313 >> 172.21.15.39:0/3658938313 conn(0x7f51ac069e10 msgr2=0x7f51ac06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51b0e55640 1 -- 172.21.15.39:0/3658938313 shutdown_connections 2024-09-13T17:11:14.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51b0e55640 1 -- 172.21.15.39:0/3658938313 wait complete. 2024-09-13T17:11:14.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51b0e55640 1 Processor -- start 2024-09-13T17:11:14.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.147+0000 7f51b0e55640 1 -- start start 2024-09-13T17:11:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51b0e55640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194003980 0x7f51ac0c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51b0e55640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0c58c0 0x7f51ac070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51b0e55640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac1eda90 0x7f51ac1edf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51b0e55640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0c4de0 con 0x7f51ac0c58c0 2024-09-13T17:11:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51b0e55640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac142fe0 con 0x7f51ac1eda90 2024-09-13T17:11:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51b0e55640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac13c990 con 0x7f5194003980 2024-09-13T17:11:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0c58c0 0x7f51ac070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51a9d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0c58c0 0x7f51ac070210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44032/0 (socket says 172.21.15.39:44032) 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51a9d9b640 1 -- 172.21.15.39:0/1806621644 learned_addr learned my addr 172.21.15.39:0/1806621644 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51aa59c640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194003980 0x7f51ac0c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51aad9d640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac1eda90 0x7f51ac1edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51a9d9b640 1 -- 172.21.15.39:0/1806621644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194003980 msgr2=0x7f51ac0c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51a9d9b640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194003980 0x7f51ac0c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51a9d9b640 1 -- 172.21.15.39:0/1806621644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac1eda90 msgr2=0x7f51ac1edf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51a9d9b640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac1eda90 0x7f51ac1edf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51a9d9b640 1 -- 172.21.15.39:0/1806621644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac070750 con 0x7f51ac0c58c0 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51a9d9b640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0c58c0 0x7f51ac070210 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f51a0005e20 tx=0x7f51a0007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f519affd640 1 -- 172.21.15.39:0/1806621644 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0007760 con 0x7f51ac0c58c0 2024-09-13T17:11:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f519affd640 1 -- 172.21.15.39:0/1806621644 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a000e6e0 con 0x7f51ac0c58c0 2024-09-13T17:11:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac070e80 con 0x7f51ac0c58c0 2024-09-13T17:11:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f519affd640 1 -- 172.21.15.39:0/1806621644 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a000eab0 con 0x7f51ac0c58c0 2024-09-13T17:11:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac070b30 con 0x7f51ac0c58c0 2024-09-13T17:11:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac0c55e0 con 0x7f51ac0c58c0 2024-09-13T17:11:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f519affd640 1 -- 172.21.15.39:0/1806621644 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f51a000ec10 con 0x7f51ac0c58c0 2024-09-13T17:11:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f519affd640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f518c066690 0x7f518c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f51aa59c640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f518c066690 0x7f518c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.151+0000 7f519affd640 1 -- 172.21.15.39:0/1806621644 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7f51a0084c60 con 0x7f51ac0c58c0 2024-09-13T17:11:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.155+0000 7f51aa59c640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f518c066690 0x7f518c068b50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f519c001750 tx=0x7f519c0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.159+0000 7f519affd640 1 -- 172.21.15.39:0/1806621644 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f51ac0c55e0 con 0x7f51ac0c58c0 2024-09-13T17:11:14.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.383+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51ac0c55e0 con 0x7f51ac0c58c0 2024-09-13T17:11:14.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f519affd640 1 -- 172.21.15.39:0/1806621644 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+5179 (secure 0 0 0) 0x7f51ac0c55e0 con 0x7f51ac0c58c0 2024-09-13T17:11:14.389 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:14.389 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":166,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:14.389 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 166 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f518c066690 msgr2=0x7f518c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f518c066690 0x7f518c068b50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f519c001750 tx=0x7f519c0011a0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0c58c0 msgr2=0x7f51ac070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0c58c0 0x7f51ac070210 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f51a0005e20 tx=0x7f51a0007420 comp rx=0 tx=0).stop 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 shutdown_connections 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f518c066690 0x7f518c068b50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51ac1eda90 0x7f51ac1edf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0c58c0 0x7f51ac070210 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 --2- 172.21.15.39:0/1806621644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5194003980 0x7f51ac0c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 >> 172.21.15.39:0/1806621644 conn(0x7f51ac069e10 msgr2=0x7f51ac146d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:14.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 shutdown_connections 2024-09-13T17:11:14.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:14.387+0000 7f51b0e55640 1 -- 172.21.15.39:0/1806621644 wait complete. 2024-09-13T17:11:14.411 INFO:tasks.mds_thrash.fs.[a]:mds.a has gained rank=0, replacing gid=7677 2024-09-13T17:11:14.411 INFO:tasks.mds_thrash.fs.[a]:waiting for 4 secs before reviving mds.c 2024-09-13T17:11:15.059 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T17:11:15.059+0000 7f80d21ca640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:11:18.412 INFO:tasks.mds_thrash.fs.[a]:reviving mds.c 2024-09-13T17:11:18.412 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:11:18.412 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:11:18.419 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:11:18.420 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:18.501 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T17:11:18.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62fabb640 1 Processor -- start 2024-09-13T17:11:18.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62fabb640 1 -- start start 2024-09-13T17:11:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62fabb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6280611c0 0x7ff6280bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62fabb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6280c1340 con 0x7ff628151860 2024-09-13T17:11:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62fabb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6281578f0 con 0x7ff6280bf4f0 2024-09-13T17:11:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62fabb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6280c1590 con 0x7ff6280611c0 2024-09-13T17:11:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62d857640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff628151860 0x7ff628157d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52592/0 (socket says 172.21.15.39:52592) 2024-09-13T17:11:18.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62d857640 1 -- 172.21.15.39:0/1247411048 learned_addr learned my addr 172.21.15.39:0/1247411048 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:18.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62e058640 1 --2- 172.21.15.39:0/1247411048 >> v2:172.21.15.84:3300/0 conn(0x7ff6280611c0 0x7ff6280bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:18.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62c855640 1 -- 172.21.15.39:0/1247411048 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030579575 0 0) 0x7ff6280c1340 con 0x7ff628151860 2024-09-13T17:11:18.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62c855640 1 -- 172.21.15.39:0/1247411048 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff614003530 con 0x7ff628151860 2024-09-13T17:11:18.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62e058640 1 -- 172.21.15.39:0/1247411048 >> v1:172.21.15.84:6789/0 conn(0x7ff6280bf4f0 legacy=0x7ff628153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:18.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62e058640 1 -- 172.21.15.39:0/1247411048 >> v1:172.21.15.39:6789/0 conn(0x7ff628151860 legacy=0x7ff628157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:18.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62c855640 1 -- 172.21.15.39:0/1247411048 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1818374696 0 0) 0x7ff614003530 con 0x7ff628151860 2024-09-13T17:11:18.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.571+0000 7ff62e058640 1 -- 172.21.15.39:0/1247411048 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280c1870 con 0x7ff6280611c0 2024-09-13T17:11:18.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62e058640 1 --2- 172.21.15.39:0/1247411048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6280611c0 0x7ff6280bfad0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff620009d90 tx=0x7ff62001c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=467b1d00df6ef289 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:18.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62c855640 1 -- 172.21.15.39:0/1247411048 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624001680 con 0x7ff6280611c0 2024-09-13T17:11:18.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62c855640 1 -- 172.21.15.39:0/1247411048 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6200a55f0 con 0x7ff6280611c0 2024-09-13T17:11:18.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62c855640 1 -- 172.21.15.39:0/1247411048 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200a5980 con 0x7ff6280611c0 2024-09-13T17:11:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62fabb640 1 -- 172.21.15.39:0/1247411048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6280611c0 msgr2=0x7ff6280bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62fabb640 1 --2- 172.21.15.39:0/1247411048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6280611c0 0x7ff6280bfad0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff620009d90 tx=0x7ff62001c7d0 comp rx=0 tx=0).stop 2024-09-13T17:11:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62fabb640 1 -- 172.21.15.39:0/1247411048 shutdown_connections 2024-09-13T17:11:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62fabb640 1 --2- 172.21.15.39:0/1247411048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6280611c0 0x7ff6280bfad0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62fabb640 1 -- 172.21.15.39:0/1247411048 >> 172.21.15.39:0/1247411048 conn(0x7ff628069e10 msgr2=0x7ff62806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62fabb640 1 -- 172.21.15.39:0/1247411048 shutdown_connections 2024-09-13T17:11:18.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.575+0000 7ff62fabb640 1 -- 172.21.15.39:0/1247411048 wait complete. 2024-09-13T17:11:18.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 Processor -- start 2024-09-13T17:11:18.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 -- start start 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6280611c0 0x7ff62814bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6280bf4f0 0x7ff62814c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff628151860 0x7ff6280aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280a92a0 con 0x7ff6280bf4f0 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62814c590 con 0x7ff6280611c0 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6281ed5d0 con 0x7ff628151860 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d056640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6280bf4f0 0x7ff62814c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d056640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6280bf4f0 0x7ff62814c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44052/0 (socket says 172.21.15.39:44052) 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d056640 1 -- 172.21.15.39:0/1573251328 learned_addr learned my addr 172.21.15.39:0/1573251328 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:18.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d857640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6280611c0 0x7ff62814bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:18.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d056640 1 -- 172.21.15.39:0/1573251328 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff628151860 msgr2=0x7ff6280aafc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:11:18.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d056640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff628151860 0x7ff6280aafc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:18.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d056640 1 -- 172.21.15.39:0/1573251328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6280611c0 msgr2=0x7ff62814bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:18.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d056640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6280611c0 0x7ff62814bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:18.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d056640 1 -- 172.21.15.39:0/1573251328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff620009a70 con 0x7ff6280bf4f0 2024-09-13T17:11:18.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62d056640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6280bf4f0 0x7ff62814c050 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff61800c3c0 tx=0x7ff61800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:18.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff61e7fc640 1 -- 172.21.15.39:0/1573251328 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180052d0 con 0x7ff6280bf4f0 2024-09-13T17:11:18.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff61e7fc640 1 -- 172.21.15.39:0/1573251328 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff618005430 con 0x7ff6280bf4f0 2024-09-13T17:11:18.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6281550c0 con 0x7ff6280bf4f0 2024-09-13T17:11:18.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff61e7fc640 1 -- 172.21.15.39:0/1573251328 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618003840 con 0x7ff6280bf4f0 2024-09-13T17:11:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.579+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6281ed890 con 0x7ff6280bf4f0 2024-09-13T17:11:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.583+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6280a9410 con 0x7ff6280bf4f0 2024-09-13T17:11:18.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.583+0000 7ff61e7fc640 1 -- 172.21.15.39:0/1573251328 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff6180039a0 con 0x7ff6280bf4f0 2024-09-13T17:11:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.583+0000 7ff61e7fc640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff60c066690 0x7ff60c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.583+0000 7ff61e7fc640 1 -- 172.21.15.39:0/1573251328 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7ff618084450 con 0x7ff6280bf4f0 2024-09-13T17:11:18.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.583+0000 7ff62d857640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff60c066690 0x7ff60c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:18.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.583+0000 7ff62d857640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff60c066690 0x7ff60c068b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff624000f70 tx=0x7ff624001d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:18.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.591+0000 7ff61e7fc640 1 -- 172.21.15.39:0/1573251328 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff618050a50 con 0x7ff6280bf4f0 2024-09-13T17:11:18.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.811+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6280bfad0 con 0x7ff6280bf4f0 2024-09-13T17:11:18.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff61e7fc640 1 -- 172.21.15.39:0/1573251328 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+5179 (secure 0 0 0) 0x7ff618017090 con 0x7ff6280bf4f0 2024-09-13T17:11:18.813 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:18.813 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":166,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:18.813 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 166 2024-09-13T17:11:18.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff60c066690 msgr2=0x7ff60c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:18.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff60c066690 0x7ff60c068b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff624000f70 tx=0x7ff624001d70 comp rx=0 tx=0).stop 2024-09-13T17:11:18.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6280bf4f0 msgr2=0x7ff62814c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:18.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6280bf4f0 0x7ff62814c050 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff61800c3c0 tx=0x7ff61800c880 comp rx=0 tx=0).stop 2024-09-13T17:11:18.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 shutdown_connections 2024-09-13T17:11:18.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff60c066690 0x7ff60c068b50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:18.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff628151860 0x7ff6280aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:18.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6280bf4f0 0x7ff62814c050 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:18.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 --2- 172.21.15.39:0/1573251328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6280611c0 0x7ff62814bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:18.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 >> 172.21.15.39:0/1573251328 conn(0x7ff628069e10 msgr2=0x7ff6280c2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:18.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 shutdown_connections 2024-09-13T17:11:18.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:18.815+0000 7ff62fabb640 1 -- 172.21.15.39:0/1573251328 wait complete. 2024-09-13T17:11:18.832 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-09-13T17:11:20.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:20.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc75822640 1 Processor -- start 2024-09-13T17:11:20.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc75822640 1 -- start start 2024-09-13T17:11:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc75822640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc700c2690 0x7efc7006ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc75822640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c6290 0x7efc701577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc75822640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc70157330 con 0x7efc700c0be0 2024-09-13T17:11:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc75822640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc700c4a20 con 0x7efc700c2690 2024-09-13T17:11:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc75822640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc700c4740 con 0x7efc700c6290 2024-09-13T17:11:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc6effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc700c2690 0x7efc7006ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:20.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc6effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efc700c2690 0x7efc7006ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34612/0 (socket says 172.21.15.39:34612) 2024-09-13T17:11:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc6effd640 1 -- 172.21.15.39:0/1778803937 learned_addr learned my addr 172.21.15.39:0/1778803937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc6e7fc640 1 --2- 172.21.15.39:0/1778803937 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c6290 0x7efc701577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc6effd640 1 -- 172.21.15.39:0/1778803937 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c6290 msgr2=0x7efc701577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc6effd640 1 --2- 172.21.15.39:0/1778803937 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c6290 0x7efc701577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc6effd640 1 -- 172.21.15.39:0/1778803937 >> v1:172.21.15.84:6789/0 conn(0x7efc700c0be0 legacy=0x7efc70065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:20.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.979+0000 7efc6effd640 1 -- 172.21.15.39:0/1778803937 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc70154880 con 0x7efc700c2690 2024-09-13T17:11:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc6effd640 1 --2- 172.21.15.39:0/1778803937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c2690 0x7efc7006ae20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7efc58009870 tx=0x7efc5801cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c447b9997416f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc6dffb640 1 -- 172.21.15.39:0/1778803937 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc580aa620 con 0x7efc700c2690 2024-09-13T17:11:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc6dffb640 1 -- 172.21.15.39:0/1778803937 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc580aa780 con 0x7efc700c2690 2024-09-13T17:11:20.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc6dffb640 1 -- 172.21.15.39:0/1778803937 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc580aab10 con 0x7efc700c2690 2024-09-13T17:11:20.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 -- 172.21.15.39:0/1778803937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c2690 msgr2=0x7efc7006ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:20.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 --2- 172.21.15.39:0/1778803937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c2690 0x7efc7006ae20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7efc58009870 tx=0x7efc5801cf00 comp rx=0 tx=0).stop 2024-09-13T17:11:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 -- 172.21.15.39:0/1778803937 shutdown_connections 2024-09-13T17:11:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 --2- 172.21.15.39:0/1778803937 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c6290 0x7efc701577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 --2- 172.21.15.39:0/1778803937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c2690 0x7efc7006ae20 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 -- 172.21.15.39:0/1778803937 >> 172.21.15.39:0/1778803937 conn(0x7efc700c32a0 msgr2=0x7efc700c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 -- 172.21.15.39:0/1778803937 shutdown_connections 2024-09-13T17:11:20.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 -- 172.21.15.39:0/1778803937 wait complete. 2024-09-13T17:11:20.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 Processor -- start 2024-09-13T17:11:20.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.983+0000 7efc75822640 1 -- start start 2024-09-13T17:11:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc75822640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc700c0be0 0x7efc7015a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc75822640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c2690 0x7efc701e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc75822640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c6290 0x7efc70150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc75822640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc700ac300 con 0x7efc700c6290 2024-09-13T17:11:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc75822640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc701e71f0 con 0x7efc700c0be0 2024-09-13T17:11:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc75822640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7015a8a0 con 0x7efc700c2690 2024-09-13T17:11:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c6290 0x7efc70150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c6290 0x7efc70150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34618/0 (socket says 172.21.15.39:34618) 2024-09-13T17:11:20.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6f7fe640 1 -- 172.21.15.39:0/1468899522 learned_addr learned my addr 172.21.15.39:0/1468899522 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6effd640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc700c0be0 0x7efc7015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6e7fc640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c2690 0x7efc701e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6f7fe640 1 -- 172.21.15.39:0/1468899522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c2690 msgr2=0x7efc701e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6f7fe640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c2690 0x7efc701e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6f7fe640 1 -- 172.21.15.39:0/1468899522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc700c0be0 msgr2=0x7efc7015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6f7fe640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc700c0be0 0x7efc7015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6f7fe640 1 -- 172.21.15.39:0/1468899522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc580094d0 con 0x7efc700c6290 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6e7fc640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c2690 0x7efc701e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6f7fe640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c6290 0x7efc70150a80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c790 tx=0x7efc6800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc67fff640 1 -- 172.21.15.39:0/1468899522 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68006d60 con 0x7efc700c6290 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc67fff640 1 -- 172.21.15.39:0/1468899522 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc68005020 con 0x7efc700c6290 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc67fff640 1 -- 172.21.15.39:0/1468899522 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc680053d0 con 0x7efc700c6290 2024-09-13T17:11:20.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc701ed620 con 0x7efc700c6290 2024-09-13T17:11:20.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc701e6f10 con 0x7efc700c6290 2024-09-13T17:11:20.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7006ae20 con 0x7efc700c6290 2024-09-13T17:11:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc67fff640 1 -- 172.21.15.39:0/1468899522 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efc68005530 con 0x7efc700c6290 2024-09-13T17:11:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc67fff640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc50066690 0x7efc50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc67fff640 1 -- 172.21.15.39:0/1468899522 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7efc680848f0 con 0x7efc700c6290 2024-09-13T17:11:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.987+0000 7efc6effd640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc50066690 0x7efc50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:20.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.991+0000 7efc6effd640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc50066690 0x7efc50068b50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efc580ae000 tx=0x7efc580b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:20.995+0000 7efc67fff640 1 -- 172.21.15.39:0/1468899522 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efc68050e40 con 0x7efc700c6290 2024-09-13T17:11:21.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.243+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc700c2d20 con 0x7efc700c6290 2024-09-13T17:11:21.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc67fff640 1 -- 172.21.15.39:0/1468899522 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) v1 ==== 95+0+5934 (secure 0 0 0) 0x7efc68054460 con 0x7efc700c6290 2024-09-13T17:11:21.246 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:21.246 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":167,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 167 2024-09-13T17:11:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc50066690 msgr2=0x7efc50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc50066690 0x7efc50068b50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efc580ae000 tx=0x7efc580b5040 comp rx=0 tx=0).stop 2024-09-13T17:11:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c6290 msgr2=0x7efc70150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c6290 0x7efc70150a80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7efc6800c790 tx=0x7efc6800cc50 comp rx=0 tx=0).stop 2024-09-13T17:11:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 shutdown_connections 2024-09-13T17:11:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efc50066690 0x7efc50068b50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc700c6290 0x7efc70150a80 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efc700c2690 0x7efc701e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 --2- 172.21.15.39:0/1468899522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efc700c0be0 0x7efc7015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 >> 172.21.15.39:0/1468899522 conn(0x7efc700c32a0 msgr2=0x7efc700a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 shutdown_connections 2024-09-13T17:11:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:21.247+0000 7efc75822640 1 -- 172.21.15.39:0/1468899522 wait complete. 2024-09-13T17:11:21.267 INFO:tasks.mds_thrash.fs.[a]:mds.c reported in up:standby state 2024-09-13T17:11:21.267 INFO:tasks.mds_thrash.fs.[a]:waiting for 24 secs before thrashing 2024-09-13T17:11:30.492 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:11:30.495 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:11:45.268 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7aa6595640 1 Processor -- start 2024-09-13T17:11:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7aa6595640 1 -- start start 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7aa6595640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01556d0 0x7f7aa00c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7aa6595640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7aa00c7610 0x7f7aa0151b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7aa6595640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa006feb0 con 0x7f7aa00a72b0 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7aa6595640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa00aae50 con 0x7f7aa00c7610 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7aa6595640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa0155ab0 con 0x7f7aa01556d0 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7aa4b32640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7aa00a72b0 0x7f7aa0071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41586/0 (socket says 172.21.15.39:41586) 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7a9f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7aa00c7610 0x7f7aa0151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7aa4b32640 1 -- 172.21.15.39:0/1116535674 learned_addr learned my addr 172.21.15.39:0/1116535674 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7a9ffff640 1 --2- 172.21.15.39:0/1116535674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01556d0 0x7f7aa00c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7a9effd640 1 -- 172.21.15.39:0/1116535674 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851441049 0 0) 0x7f7aa006feb0 con 0x7f7aa00a72b0 2024-09-13T17:11:45.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7a9effd640 1 -- 172.21.15.39:0/1116535674 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a88003530 con 0x7f7aa00a72b0 2024-09-13T17:11:45.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7a9f7fe640 1 -- 172.21.15.39:0/1116535674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01556d0 msgr2=0x7f7aa00c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:45.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7a9f7fe640 1 --2- 172.21.15.39:0/1116535674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01556d0 0x7f7aa00c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7a9f7fe640 1 -- 172.21.15.39:0/1116535674 >> v1:172.21.15.39:6789/0 conn(0x7f7aa00a72b0 legacy=0x7f7aa0071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.430+0000 7f7a9f7fe640 1 -- 172.21.15.39:0/1116535674 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa00c3c20 con 0x7f7aa00c7610 2024-09-13T17:11:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7a9f7fe640 1 --2- 172.21.15.39:0/1116535674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa00c7610 0x7f7aa0151b80 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7a940097c0 tx=0x7f7a9401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110570ccb44b93b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7a9effd640 1 -- 172.21.15.39:0/1116535674 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940ab600 con 0x7f7aa00c7610 2024-09-13T17:11:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7a9effd640 1 -- 172.21.15.39:0/1116535674 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a940a7050 con 0x7f7aa00c7610 2024-09-13T17:11:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7a9effd640 1 -- 172.21.15.39:0/1116535674 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940ab980 con 0x7f7aa00c7610 2024-09-13T17:11:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 -- 172.21.15.39:0/1116535674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa00c7610 msgr2=0x7f7aa0151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 --2- 172.21.15.39:0/1116535674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa00c7610 0x7f7aa0151b80 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7a940097c0 tx=0x7f7a9401c380 comp rx=0 tx=0).stop 2024-09-13T17:11:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 -- 172.21.15.39:0/1116535674 shutdown_connections 2024-09-13T17:11:45.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 --2- 172.21.15.39:0/1116535674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa00c7610 0x7f7aa0151b80 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 --2- 172.21.15.39:0/1116535674 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01556d0 0x7f7aa00c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 -- 172.21.15.39:0/1116535674 >> 172.21.15.39:0/1116535674 conn(0x7f7aa00a97b0 msgr2=0x7f7aa00a9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:45.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 -- 172.21.15.39:0/1116535674 shutdown_connections 2024-09-13T17:11:45.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 -- 172.21.15.39:0/1116535674 wait complete. 2024-09-13T17:11:45.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 Processor -- start 2024-09-13T17:11:45.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 -- start start 2024-09-13T17:11:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00a72b0 0x7f7aa0149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa01556d0 0x7f7aa014bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01438d0 0x7f7aa014c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa0155310 con 0x7f7aa01556d0 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01550c0 con 0x7f7aa00a72b0 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa6595640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01483b0 con 0x7f7aa01438d0 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7a9f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa01556d0 0x7f7aa014bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7a9f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa01556d0 0x7f7aa014bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49144/0 (socket says 172.21.15.39:49144) 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7a9f7fe640 1 -- 172.21.15.39:0/250276492 learned_addr learned my addr 172.21.15.39:0/250276492 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7a9ffff640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00a72b0 0x7f7aa0149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.434+0000 7f7aa4b32640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01438d0 0x7f7aa014c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9f7fe640 1 -- 172.21.15.39:0/250276492 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01438d0 msgr2=0x7f7aa014c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9f7fe640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01438d0 0x7f7aa014c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9f7fe640 1 -- 172.21.15.39:0/250276492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00a72b0 msgr2=0x7f7aa0149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9f7fe640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00a72b0 0x7f7aa0149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9f7fe640 1 -- 172.21.15.39:0/250276492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94009420 con 0x7f7aa01556d0 2024-09-13T17:11:45.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9f7fe640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa01556d0 0x7f7aa014bf60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7a940ae040 tx=0x7f7a94019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:45.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9cff9640 1 -- 172.21.15.39:0/250276492 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940b4430 con 0x7f7aa01556d0 2024-09-13T17:11:45.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9cff9640 1 -- 172.21.15.39:0/250276492 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a940b45b0 con 0x7f7aa01556d0 2024-09-13T17:11:45.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa0148690 con 0x7f7aa01556d0 2024-09-13T17:11:45.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9cff9640 1 -- 172.21.15.39:0/250276492 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a940b4960 con 0x7f7aa01556d0 2024-09-13T17:11:45.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa006d8f0 con 0x7f7aa01556d0 2024-09-13T17:11:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa014bc80 con 0x7f7aa01556d0 2024-09-13T17:11:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9cff9640 1 -- 172.21.15.39:0/250276492 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a940b2040 con 0x7f7aa01556d0 2024-09-13T17:11:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9cff9640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a8c066690 0x7f7a8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9cff9640 1 -- 172.21.15.39:0/250276492 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7f7a9411ede0 con 0x7f7aa01556d0 2024-09-13T17:11:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9ffff640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a8c066690 0x7f7a8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.438+0000 7f7a9ffff640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a8c066690 0x7f7a8c068b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7aa014a760 tx=0x7f7a90009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.442+0000 7f7a9cff9640 1 -- 172.21.15.39:0/250276492 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7aa014bc80 con 0x7f7aa01556d0 2024-09-13T17:11:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.678+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7aa014bc80 con 0x7f7aa01556d0 2024-09-13T17:11:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7a9cff9640 1 -- 172.21.15.39:0/250276492 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f7aa014bc80 con 0x7f7aa01556d0 2024-09-13T17:11:45.681 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:45.681 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":167,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 167 2024-09-13T17:11:45.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a8c066690 msgr2=0x7f7a8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a8c066690 0x7f7a8c068b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7aa014a760 tx=0x7f7a90009190 comp rx=0 tx=0).stop 2024-09-13T17:11:45.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa01556d0 msgr2=0x7f7aa014bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa01556d0 0x7f7aa014bf60 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7a940ae040 tx=0x7f7a94019120 comp rx=0 tx=0).stop 2024-09-13T17:11:45.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 shutdown_connections 2024-09-13T17:11:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a8c066690 0x7f7a8c068b50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7aa01438d0 0x7f7aa014c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7aa01556d0 0x7f7aa014bf60 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 --2- 172.21.15.39:0/250276492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00a72b0 0x7f7aa0149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 >> 172.21.15.39:0/250276492 conn(0x7f7aa00a97b0 msgr2=0x7f7aa00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 shutdown_connections 2024-09-13T17:11:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.682+0000 7f7aa6595640 1 -- 172.21.15.39:0/250276492 wait complete. 2024-09-13T17:11:45.704 INFO:tasks.mds_thrash.fs.[a]:kill mds.a (rank=0) 2024-09-13T17:11:45.704 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:11:45.704 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:11:45.717 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:11:45.717 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:45.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.878+0000 7f4c91474640 1 Processor -- start 2024-09-13T17:11:45.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.878+0000 7f4c91474640 1 -- start start 2024-09-13T17:11:45.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.878+0000 7f4c91474640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c070b40 0x7f4c8c0c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.878+0000 7f4c91474640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c8c157360 con 0x7f4c8c0c4260 2024-09-13T17:11:45.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.878+0000 7f4c91474640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c8c0c80b0 con 0x7f4c8c0bcfb0 2024-09-13T17:11:45.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.878+0000 7f4c91474640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c1575b0 con 0x7f4c8c070b40 2024-09-13T17:11:45.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.878+0000 7f4c8affd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4c8c0c4260 0x7f4c8c070430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41590/0 (socket says 172.21.15.39:41590) 2024-09-13T17:11:45.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.878+0000 7f4c8affd640 1 -- 172.21.15.39:0/2728269720 learned_addr learned my addr 172.21.15.39:0/2728269720 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:45.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8a7fc640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c070b40 0x7f4c8c0c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249062144 0 0) 0x7f4c8c157360 con 0x7f4c8c0c4260 2024-09-13T17:11:45.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c74003530 con 0x7f4c8c0c4260 2024-09-13T17:11:45.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2473550322 0 0) 0x7f4c74003530 con 0x7f4c8c0c4260 2024-09-13T17:11:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c070b40 msgr2=0x7f4c8c0c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c070b40 0x7f4c8c0c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 >> v1:172.21.15.84:6789/0 conn(0x7f4c8c0bcfb0 legacy=0x7f4c8c15e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c157890 con 0x7f4c8c0c4260 2024-09-13T17:11:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4c7c001a00 con 0x7f4c8c0c4260 2024-09-13T17:11:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 >> v1:172.21.15.39:6789/0 conn(0x7f4c8c0c4260 legacy=0x7f4c8c070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c740039b0 0x7f4c74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c74004310 0x7f4c740047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c74005090 0x7f4c7400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c1575b0 con 0x7f4c74004310 2024-09-13T17:11:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7400bfb0 con 0x7f4c74005090 2024-09-13T17:11:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7400c170 con 0x7f4c740039b0 2024-09-13T17:11:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8a7fc640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c74004310 0x7f4c740047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8b7fe640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c74005090 0x7f4c7400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8affd640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c740039b0 0x7f4c74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8a7fc640 1 -- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c740039b0 msgr2=0x7f4c74003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:45.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8a7fc640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c740039b0 0x7f4c74003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8a7fc640 1 -- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c74005090 msgr2=0x7f4c7400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:45.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8a7fc640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c74005090 0x7f4c7400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8a7fc640 1 -- 172.21.15.39:0/2728269720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c74003530 con 0x7f4c74004310 2024-09-13T17:11:45.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c8a7fc640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c74004310 0x7f4c740047f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009ae0 tx=0x7f4c8001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c031980bef272b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:45.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.882+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0c7330 con 0x7f4c74004310 2024-09-13T17:11:45.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c800a5550 con 0x7f4c74004310 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c89ffb640 1 -- 172.21.15.39:0/2728269720 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c800a5900 con 0x7f4c74004310 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 -- 172.21.15.39:0/2728269720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c74004310 msgr2=0x7f4c740047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 --2- 172.21.15.39:0/2728269720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c74004310 0x7f4c740047f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009ae0 tx=0x7f4c8001ca70 comp rx=0 tx=0).stop 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8b7fe640 1 -- 172.21.15.39:0/2728269720 reap_dead start 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 -- 172.21.15.39:0/2728269720 shutdown_connections 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 -- 172.21.15.39:0/2728269720 >> 172.21.15.39:0/2728269720 conn(0x7f4c8c069e10 msgr2=0x7f4c8c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 -- 172.21.15.39:0/2728269720 shutdown_connections 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 -- 172.21.15.39:0/2728269720 wait complete. 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 Processor -- start 2024-09-13T17:11:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 -- start start 2024-09-13T17:11:45.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c74004310 0x7f4c8c154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c156f00 0x7f4c8c143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c15f920 0x7f4c8c1ec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c153ad0 con 0x7f4c8c156f00 2024-09-13T17:11:45.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c1f0a60 con 0x7f4c74004310 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c91474640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c1efe50 con 0x7f4c8c15f920 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c156f00 0x7f4c8c143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c156f00 0x7f4c8c143870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49168/0 (socket says 172.21.15.39:49168) 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8a7fc640 1 -- 172.21.15.39:0/2395296059 learned_addr learned my addr 172.21.15.39:0/2395296059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8affd640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c74004310 0x7f4c8c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8b7fe640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c15f920 0x7f4c8c1ec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8a7fc640 1 -- 172.21.15.39:0/2395296059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c15f920 msgr2=0x7f4c8c1ec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8a7fc640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c15f920 0x7f4c8c1ec8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8a7fc640 1 -- 172.21.15.39:0/2395296059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c74004310 msgr2=0x7f4c8c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8a7fc640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c74004310 0x7f4c8c154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:45.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8a7fc640 1 -- 172.21.15.39:0/2395296059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c1f0130 con 0x7f4c8c156f00 2024-09-13T17:11:45.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8b7fe640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c15f920 0x7f4c8c1ec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:11:45.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.886+0000 7f4c8a7fc640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c156f00 0x7f4c8c143870 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f4c800b0040 tx=0x7f4c80019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:45.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c737fe640 1 -- 172.21.15.39:0/2395296059 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000f000 con 0x7f4c8c156f00 2024-09-13T17:11:45.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c737fe640 1 -- 172.21.15.39:0/2395296059 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80004020 con 0x7f4c8c156f00 2024-09-13T17:11:45.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c737fe640 1 -- 172.21.15.39:0/2395296059 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c800043d0 con 0x7f4c8c156f00 2024-09-13T17:11:45.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c1f0410 con 0x7f4c8c156f00 2024-09-13T17:11:45.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c154550 con 0x7f4c8c156f00 2024-09-13T17:11:45.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c067a00 con 0x7f4c8c156f00 2024-09-13T17:11:45.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c737fe640 1 -- 172.21.15.39:0/2395296059 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c800b4070 con 0x7f4c8c156f00 2024-09-13T17:11:45.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c737fe640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c780666c0 0x7f4c78068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:45.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c737fe640 1 -- 172.21.15.39:0/2395296059 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7f4c8011eff0 con 0x7f4c8c156f00 2024-09-13T17:11:45.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c8affd640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c780666c0 0x7f4c78068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:45.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.890+0000 7f4c8affd640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c780666c0 0x7f4c78068b80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c001470 tx=0x7f4c7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:45.898+0000 7f4c737fe640 1 -- 172.21.15.39:0/2395296059 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c800eb540 con 0x7f4c8c156f00 2024-09-13T17:11:46.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.138+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c8c070430 con 0x7f4c8c156f00 2024-09-13T17:11:46.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.138+0000 7f4c737fe640 1 -- 172.21.15.39:0/2395296059 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f4c800b2090 con 0x7f4c8c156f00 2024-09-13T17:11:46.139 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:46.139 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":167,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:46.140 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 167 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c780666c0 msgr2=0x7f4c78068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c780666c0 0x7f4c78068b80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c001470 tx=0x7f4c7c009040 comp rx=0 tx=0).stop 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c156f00 msgr2=0x7f4c8c143870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c156f00 0x7f4c8c143870 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f4c800b0040 tx=0x7f4c80019810 comp rx=0 tx=0).stop 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 shutdown_connections 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c780666c0 0x7f4c78068b80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4c8c15f920 0x7f4c8c1ec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c156f00 0x7f4c8c143870 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 --2- 172.21.15.39:0/2395296059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c74004310 0x7f4c8c154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 >> 172.21.15.39:0/2395296059 conn(0x7f4c8c069e10 msgr2=0x7f4c8c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 shutdown_connections 2024-09-13T17:11:46.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:46.142+0000 7f4c91474640 1 -- 172.21.15.39:0/2395296059 wait complete. 2024-09-13T17:11:46.160 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:11:48.160 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:48.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc702604640 1 Processor -- start 2024-09-13T17:11:48.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc702604640 1 -- start start 2024-09-13T17:11:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc702604640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc6fc0c5330 0x7fc6fc0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc702604640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc0c2300 0x7fc6fc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc702604640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6fc1546c0 con 0x7fc6fc150f80 2024-09-13T17:11:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc702604640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6fc0c19a0 con 0x7fc6fc0c5330 2024-09-13T17:11:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc702604640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0c49f0 con 0x7fc6fc0c2300 2024-09-13T17:11:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fbfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc6fc150f80 0x7fc6fc153930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41598/0 (socket says 172.21.15.39:41598) 2024-09-13T17:11:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fbfff640 1 -- 172.21.15.39:0/883200014 learned_addr learned my addr 172.21.15.39:0/883200014 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fb7fe640 1 --2- 172.21.15.39:0/883200014 >> v2:172.21.15.84:3300/0 conn(0x7fc6fc0c5330 0x7fc6fc0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc700ba1640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc0c2300 0x7fc6fc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658999592 0 0) 0x7fc6fc1546c0 con 0x7fc6fc150f80 2024-09-13T17:11:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6e4003530 con 0x7fc6fc150f80 2024-09-13T17:11:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 13306316 0 0) 0x7fc6e4003530 con 0x7fc6fc150f80 2024-09-13T17:11:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc0c2300 msgr2=0x7fc6fc06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc0c2300 0x7fc6fc06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 >> v2:172.21.15.84:3300/0 conn(0x7fc6fc0c5330 msgr2=0x7fc6fc0c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 --2- 172.21.15.39:0/883200014 >> v2:172.21.15.84:3300/0 conn(0x7fc6fc0c5330 0x7fc6fc0c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc1543e0 con 0x7fc6fc150f80 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fc6ec001680 con 0x7fc6fc150f80 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 >> v1:172.21.15.39:6789/0 conn(0x7fc6fc150f80 legacy=0x7fc6fc153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6e4003980 0x7fc6e4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6e40047d0 0x7fc6e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc0c2300 0x7fc6e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0c19a0 con 0x7fc6e4003980 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0c49f0 con 0x7fc6e40047d0 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e400b9e0 con 0x7fc6fc0c2300 2024-09-13T17:11:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fbfff640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6e4003980 0x7fc6e4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc700ba1640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc0c2300 0x7fc6e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fbfff640 1 -- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc0c2300 msgr2=0x7fc6e4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fbfff640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc0c2300 0x7fc6e4005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fbfff640 1 -- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6e40047d0 msgr2=0x7fc6e4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fb7fe640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6e40047d0 0x7fc6e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fbfff640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6e40047d0 0x7fc6e4004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.294+0000 7fc6fbfff640 1 -- 172.21.15.39:0/883200014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4003530 con 0x7fc6e4003980 2024-09-13T17:11:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc6fbfff640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6e4003980 0x7fc6e4003da0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c240 tx=0x7fc6ec01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1627336a76cf465 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec005ae0 con 0x7fc6e4003980 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec005c40 con 0x7fc6e4003980 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc6faffd640 1 -- 172.21.15.39:0/883200014 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0a5830 con 0x7fc6e4003980 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc702604640 1 -- 172.21.15.39:0/883200014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6e4003980 msgr2=0x7fc6e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc702604640 1 --2- 172.21.15.39:0/883200014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6e4003980 0x7fc6e4003da0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c240 tx=0x7fc6ec01cc30 comp rx=0 tx=0).stop 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc700ba1640 1 -- 172.21.15.39:0/883200014 reap_dead start 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc702604640 1 -- 172.21.15.39:0/883200014 shutdown_connections 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc702604640 1 -- 172.21.15.39:0/883200014 >> 172.21.15.39:0/883200014 conn(0x7fc6fc069e10 msgr2=0x7fc6fc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc702604640 1 -- 172.21.15.39:0/883200014 shutdown_connections 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc702604640 1 -- 172.21.15.39:0/883200014 wait complete. 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc702604640 1 Processor -- start 2024-09-13T17:11:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.298+0000 7fc702604640 1 -- start start 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc702604640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6e4003980 0x7fc6fc1eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc702604640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc1eded0 0x7fc6fc1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc702604640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6fc156af0 0x7fc6fc146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc702604640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc1ea5b0 con 0x7fc6fc156af0 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc702604640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc156190 con 0x7fc6e4003980 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc702604640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc154bc0 con 0x7fc6fc1eded0 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc700ba1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6fc156af0 0x7fc6fc146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc700ba1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6fc156af0 0x7fc6fc146570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49180/0 (socket says 172.21.15.39:49180) 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc700ba1640 1 -- 172.21.15.39:0/2975386993 learned_addr learned my addr 172.21.15.39:0/2975386993 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc6fbfff640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6e4003980 0x7fc6fc1eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc6fb7fe640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc1eded0 0x7fc6fc1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc700ba1640 1 -- 172.21.15.39:0/2975386993 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc1eded0 msgr2=0x7fc6fc1565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc700ba1640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc1eded0 0x7fc6fc1565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc700ba1640 1 -- 172.21.15.39:0/2975386993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6e4003980 msgr2=0x7fc6fc1eaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc700ba1640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6e4003980 0x7fc6fc1eaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc700ba1640 1 -- 172.21.15.39:0/2975386993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc154ea0 con 0x7fc6fc156af0 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc700ba1640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6fc156af0 0x7fc6fc146570 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0c49c0 tx=0x7fc6f400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc6f8ff9640 1 -- 172.21.15.39:0/2975386993 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4017400 con 0x7fc6fc156af0 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc6f8ff9640 1 -- 172.21.15.39:0/2975386993 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f4011040 con 0x7fc6fc156af0 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc6f8ff9640 1 -- 172.21.15.39:0/2975386993 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40177c0 con 0x7fc6fc156af0 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc1ea240 con 0x7fc6fc156af0 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc1e9f30 con 0x7fc6fc156af0 2024-09-13T17:11:48.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6fc1ea760 con 0x7fc6fc156af0 2024-09-13T17:11:48.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc6f8ff9640 1 -- 172.21.15.39:0/2975386993 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc6f4021440 con 0x7fc6fc156af0 2024-09-13T17:11:48.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.302+0000 7fc6f8ff9640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6e80666c0 0x7fc6e8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:48.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.306+0000 7fc6fbfff640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6e80666c0 0x7fc6e8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:48.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.306+0000 7fc6f8ff9640 1 -- 172.21.15.39:0/2975386993 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7fc6f40852c0 con 0x7fc6fc156af0 2024-09-13T17:11:48.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.306+0000 7fc6fbfff640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6e80666c0 0x7fc6e8068b80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0ab000 tx=0x7fc6ec0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:48.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.310+0000 7fc6f8ff9640 1 -- 172.21.15.39:0/2975386993 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc6fc1ea760 con 0x7fc6fc156af0 2024-09-13T17:11:48.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6fc1ea760 con 0x7fc6fc156af0 2024-09-13T17:11:48.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc6f8ff9640 1 -- 172.21.15.39:0/2975386993 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) v1 ==== 95+0+5934 (secure 0 0 0) 0x7fc6fc1ea760 con 0x7fc6fc156af0 2024-09-13T17:11:48.510 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:48.511 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":167,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:48.511 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 167 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6e80666c0 msgr2=0x7fc6e8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6e80666c0 0x7fc6e8068b80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0ab000 tx=0x7fc6ec0a7070 comp rx=0 tx=0).stop 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6fc156af0 msgr2=0x7fc6fc146570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6fc156af0 0x7fc6fc146570 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0c49c0 tx=0x7fc6f400f4a0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 shutdown_connections 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc6e80666c0 0x7fc6e8068b80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6fc156af0 0x7fc6fc146570 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc6fc1eded0 0x7fc6fc1565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 --2- 172.21.15.39:0/2975386993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6e4003980 0x7fc6fc1eaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 >> 172.21.15.39:0/2975386993 conn(0x7fc6fc069e10 msgr2=0x7fc6fc0aab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 shutdown_connections 2024-09-13T17:11:48.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:48.510+0000 7fc702604640 1 -- 172.21.15.39:0/2975386993 wait complete. 2024-09-13T17:11:48.529 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:11:50.529 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef16541640 1 Processor -- start 2024-09-13T17:11:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef16541640 1 -- start start 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef16541640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef10070fb0 0x7fef100713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef16541640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef100c9710 con 0x7fef100ca080 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef16541640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef10157340 con 0x7fef10153d20 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef16541640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef10157fc0 con 0x7fef10070fb0 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fef100ca080 0x7fef10151000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44758/0 (socket says 172.21.15.39:44758) 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0ffff640 1 -- 172.21.15.39:0/2319914762 learned_addr learned my addr 172.21.15.39:0/2319914762 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef10070fb0 0x7fef100713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422749375 0 0) 0x7fef100c9710 con 0x7fef100ca080 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feefc003530 con 0x7fef100ca080 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1300784125 0 0) 0x7feefc003530 con 0x7fef100ca080 2024-09-13T17:11:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef10070fb0 msgr2=0x7fef100713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef10070fb0 0x7fef100713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 >> v1:172.21.15.84:6789/0 conn(0x7fef10153d20 legacy=0x7fef101541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10157620 con 0x7fef100ca080 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fef08001680 con 0x7fef100ca080 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 >> v1:172.21.15.39:6789/0 conn(0x7fef100ca080 legacy=0x7fef10151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc0039b0 0x7feefc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7feefc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef10070fb0 0x7feefc005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef10157fc0 con 0x7feefc0047d0 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc00bbc0 con 0x7feefc0039b0 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc00bd80 con 0x7fef10070fb0 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7feefc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef14ade640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef10070fb0 0x7feefc005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0f7fe640 1 -- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef10070fb0 msgr2=0x7feefc005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef10070fb0 0x7feefc005290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0ffff640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc0039b0 0x7feefc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0f7fe640 1 -- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc0039b0 msgr2=0x7feefc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feefc0039b0 0x7feefc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.666+0000 7fef0f7fe640 1 -- 172.21.15.39:0/2319914762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc003530 con 0x7feefc0047d0 2024-09-13T17:11:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7feefc004c50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7feef8009ae0 tx=0x7feef801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25b7c9e9f3a8994 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef80a5530 con 0x7feefc0047d0 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef80a5690 con 0x7feefc0047d0 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef0effd640 1 -- 172.21.15.39:0/2319914762 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef80a5a40 con 0x7feefc0047d0 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef16541640 1 -- 172.21.15.39:0/2319914762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 msgr2=0x7feefc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef16541640 1 --2- 172.21.15.39:0/2319914762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7feefc004c50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7feef8009ae0 tx=0x7feef801ca70 comp rx=0 tx=0).stop 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef14ade640 1 -- 172.21.15.39:0/2319914762 reap_dead start 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef16541640 1 -- 172.21.15.39:0/2319914762 shutdown_connections 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef16541640 1 -- 172.21.15.39:0/2319914762 >> 172.21.15.39:0/2319914762 conn(0x7fef1006a210 msgr2=0x7fef1006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef16541640 1 -- 172.21.15.39:0/2319914762 shutdown_connections 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef16541640 1 -- 172.21.15.39:0/2319914762 wait complete. 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef16541640 1 Processor -- start 2024-09-13T17:11:50.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.670+0000 7fef16541640 1 -- start start 2024-09-13T17:11:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef16541640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7fef101efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef16541640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef101f0110 0x7fef10148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef16541640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef10150b40 0x7fef10149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef16541640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1015d520 con 0x7feefc0047d0 2024-09-13T17:11:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef16541640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1014a330 con 0x7fef10150b40 2024-09-13T17:11:50.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef16541640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1014a610 con 0x7fef101f0110 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7fef101efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7fef101efbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40088/0 (socket says 172.21.15.39:40088) 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0ffff640 1 -- 172.21.15.39:0/1920634821 learned_addr learned my addr 172.21.15.39:0/1920634821 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef101f0110 0x7fef10148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef14ade640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef10150b40 0x7fef10149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0ffff640 1 -- 172.21.15.39:0/1920634821 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef101f0110 msgr2=0x7fef10148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef101f0110 0x7fef10148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0ffff640 1 -- 172.21.15.39:0/1920634821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef10150b40 msgr2=0x7fef10149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef10150b40 0x7fef10149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0ffff640 1 -- 172.21.15.39:0/1920634821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1014a8f0 con 0x7feefc0047d0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef14ade640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef10150b40 0x7fef10149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0ffff640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7fef101efbd0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fef0800d810 tx=0x7fef0800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0cff9640 1 -- 172.21.15.39:0/1920634821 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080074e0 con 0x7feefc0047d0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0cff9640 1 -- 172.21.15.39:0/1920634821 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef08007640 con 0x7feefc0047d0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef10151df0 con 0x7feefc0047d0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0cff9640 1 -- 172.21.15.39:0/1920634821 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08004a10 con 0x7feefc0047d0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef10153860 con 0x7feefc0047d0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1014aa80 con 0x7feefc0047d0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0cff9640 1 -- 172.21.15.39:0/1920634821 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef08004b70 con 0x7feefc0047d0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0cff9640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feef00666c0 0x7feef0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0cff9640 1 -- 172.21.15.39:0/1920634821 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7fef08084d50 con 0x7feefc0047d0 2024-09-13T17:11:50.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.674+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feef00666c0 0x7feef0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:50.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.678+0000 7fef0f7fe640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feef00666c0 0x7feef0068b80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7feef80af000 tx=0x7feef80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:50.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.682+0000 7fef0cff9640 1 -- 172.21.15.39:0/1920634821 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fef08088030 con 0x7feefc0047d0 2024-09-13T17:11:50.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.874+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef100717a0 con 0x7feefc0047d0 2024-09-13T17:11:50.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef0cff9640 1 -- 172.21.15.39:0/1920634821 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) v1 ==== 95+0+5934 (secure 0 0 0) 0x7fef08019020 con 0x7feefc0047d0 2024-09-13T17:11:50.878 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:50.878 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":167,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:50.878 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 167 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feef00666c0 msgr2=0x7feef0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feef00666c0 0x7feef0068b80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7feef80af000 tx=0x7feef80046d0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 msgr2=0x7fef101efbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7fef101efbd0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fef0800d810 tx=0x7fef0800dcd0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 shutdown_connections 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feef00666c0 0x7feef0068b80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef10150b40 0x7fef10149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fef101f0110 0x7fef10148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 --2- 172.21.15.39:0/1920634821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0047d0 0x7fef101efbd0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 >> 172.21.15.39:0/1920634821 conn(0x7fef1006a210 msgr2=0x7fef100c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 shutdown_connections 2024-09-13T17:11:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:50.878+0000 7fef16541640 1 -- 172.21.15.39:0/1920634821 wait complete. 2024-09-13T17:11:50.897 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:11:52.898 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:53.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f0138e03640 1 Processor -- start 2024-09-13T17:11:53.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f0138e03640 1 -- start start 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f0138e03640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f013414aa40 0x7f01340c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f0138e03640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01340c76a0 con 0x7f01340c4710 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f0138e03640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0134151570 con 0x7f01340c1750 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f0138e03640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0134151b30 con 0x7f013414aa40 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f013259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f013414aa40 0x7f01340c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f013259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f013414aa40 0x7f01340c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40104/0 (socket says 172.21.15.39:40104) 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f013259c640 1 -- 172.21.15.39:0/3245454849 learned_addr learned my addr 172.21.15.39:0/3245454849 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f013259c640 1 -- 172.21.15.39:0/3245454849 >> v1:172.21.15.84:6789/0 conn(0x7f01340c1750 legacy=0x7f01340c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f013259c640 1 -- 172.21.15.39:0/3245454849 >> v1:172.21.15.39:6789/0 conn(0x7f01340c4710 legacy=0x7f013415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:53.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.042+0000 7f013259c640 1 -- 172.21.15.39:0/3245454849 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0134151850 con 0x7f013414aa40 2024-09-13T17:11:53.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f013259c640 1 --2- 172.21.15.39:0/3245454849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f013414aa40 0x7f01340c1210 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f012c009e50 tx=0x7f012c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978ed9c668bdede2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:53.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f013159a640 1 -- 172.21.15.39:0/3245454849 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0a57f0 con 0x7f013414aa40 2024-09-13T17:11:53.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f013159a640 1 -- 172.21.15.39:0/3245454849 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c0a5950 con 0x7f013414aa40 2024-09-13T17:11:53.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f013159a640 1 -- 172.21.15.39:0/3245454849 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0a5ce0 con 0x7f013414aa40 2024-09-13T17:11:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 -- 172.21.15.39:0/3245454849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f013414aa40 msgr2=0x7f01340c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 --2- 172.21.15.39:0/3245454849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f013414aa40 0x7f01340c1210 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f012c009e50 tx=0x7f012c01c930 comp rx=0 tx=0).stop 2024-09-13T17:11:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 -- 172.21.15.39:0/3245454849 shutdown_connections 2024-09-13T17:11:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 --2- 172.21.15.39:0/3245454849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f013414aa40 0x7f01340c1210 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 -- 172.21.15.39:0/3245454849 >> 172.21.15.39:0/3245454849 conn(0x7f01340c3310 msgr2=0x7f01340c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 -- 172.21.15.39:0/3245454849 shutdown_connections 2024-09-13T17:11:53.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 -- 172.21.15.39:0/3245454849 wait complete. 2024-09-13T17:11:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 Processor -- start 2024-09-13T17:11:53.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 -- start start 2024-09-13T17:11:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01340c1750 0x7f0134147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c4710 0x7f0134065ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f013414aa40 0x7f0134143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134144780 con 0x7f01340c4710 2024-09-13T17:11:53.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134147640 con 0x7f01340c1750 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0138e03640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0134144990 con 0x7f013414aa40 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0131d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c4710 0x7f0134065ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0131d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c4710 0x7f0134065ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40110/0 (socket says 172.21.15.39:40110) 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0131d9b640 1 -- 172.21.15.39:0/1103802482 learned_addr learned my addr 172.21.15.39:0/1103802482 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f013259c640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01340c1750 0x7f0134147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0132d9d640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f013414aa40 0x7f0134143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0131d9b640 1 -- 172.21.15.39:0/1103802482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f013414aa40 msgr2=0x7f0134143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0131d9b640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f013414aa40 0x7f0134143990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0131d9b640 1 -- 172.21.15.39:0/1103802482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01340c1750 msgr2=0x7f0134147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0131d9b640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01340c1750 0x7f0134147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:53.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0131d9b640 1 -- 172.21.15.39:0/1103802482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c009b30 con 0x7f01340c4710 2024-09-13T17:11:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.046+0000 7f0131d9b640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c4710 0x7f0134065ee0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f012800b410 tx=0x7f012800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f0122ffd640 1 -- 172.21.15.39:0/1103802482 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128004020 con 0x7f01340c4710 2024-09-13T17:11:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f0122ffd640 1 -- 172.21.15.39:0/1103802482 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0128004180 con 0x7f01340c4710 2024-09-13T17:11:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0134144c70 con 0x7f01340c4710 2024-09-13T17:11:53.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f0122ffd640 1 -- 172.21.15.39:0/1103802482 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128011630 con 0x7f01340c4710 2024-09-13T17:11:53.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0134145230 con 0x7f01340c4710 2024-09-13T17:11:53.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013414ae20 con 0x7f01340c4710 2024-09-13T17:11:53.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f0122ffd640 1 -- 172.21.15.39:0/1103802482 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0128011790 con 0x7f01340c4710 2024-09-13T17:11:53.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f0122ffd640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0114066690 0x7f0114068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f0122ffd640 1 -- 172.21.15.39:0/1103802482 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7f0128084e10 con 0x7f01340c4710 2024-09-13T17:11:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f013259c640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0114066690 0x7f0114068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.050+0000 7f013259c640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0114066690 0x7f0114068b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f012c0ae000 tx=0x7f012c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:53.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.054+0000 7f0122ffd640 1 -- 172.21.15.39:0/1103802482 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f01280512d0 con 0x7f01340c4710 2024-09-13T17:11:53.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f013406aa40 con 0x7f01340c4710 2024-09-13T17:11:53.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0122ffd640 1 -- 172.21.15.39:0/1103802482 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f0128015090 con 0x7f01340c4710 2024-09-13T17:11:53.259 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:53.259 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":167,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:53.259 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 167 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0114066690 msgr2=0x7f0114068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0114066690 0x7f0114068b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f012c0ae000 tx=0x7f012c0a7070 comp rx=0 tx=0).stop 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c4710 msgr2=0x7f0134065ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c4710 0x7f0134065ee0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f012800b410 tx=0x7f012800b8d0 comp rx=0 tx=0).stop 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 shutdown_connections 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0114066690 0x7f0114068b50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f013414aa40 0x7f0134143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01340c4710 0x7f0134065ee0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 --2- 172.21.15.39:0/1103802482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01340c1750 0x7f0134147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:53.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 >> 172.21.15.39:0/1103802482 conn(0x7f01340c3310 msgr2=0x7f013406a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:53.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.258+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 shutdown_connections 2024-09-13T17:11:53.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:53.262+0000 7f0138e03640 1 -- 172.21.15.39:0/1103802482 wait complete. 2024-09-13T17:11:53.278 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:11:55.278 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:55.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.414+0000 7fbc95328640 1 Processor -- start 2024-09-13T17:11:55.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.414+0000 7fbc95328640 1 -- start start 2024-09-13T17:11:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc95328640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc900c3e20 0x7fbc9006c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc95328640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc900c0d60 con 0x7fbc9006d3e0 2024-09-13T17:11:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc95328640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc900c42b0 con 0x7fbc901558a0 2024-09-13T17:11:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc95328640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc90158ea0 con 0x7fbc900c3e20 2024-09-13T17:11:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8e7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbc9006d3e0 0x7fbc9006bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44764/0 (socket says 172.21.15.39:44764) 2024-09-13T17:11:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8e7fc640 1 -- 172.21.15.39:0/1949597738 learned_addr learned my addr 172.21.15.39:0/1949597738 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8f7fe640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc900c3e20 0x7fbc9006c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078149465 0 0) 0x7fbc900c0d60 con 0x7fbc9006d3e0 2024-09-13T17:11:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc78003530 con 0x7fbc9006d3e0 2024-09-13T17:11:55.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4262306276 0 0) 0x7fbc78003530 con 0x7fbc9006d3e0 2024-09-13T17:11:55.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc900c3e20 msgr2=0x7fbc9006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:55.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc900c3e20 0x7fbc9006c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:55.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 >> v1:172.21.15.84:6789/0 conn(0x7fbc901558a0 legacy=0x7fbc900c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:55.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90158220 con 0x7fbc9006d3e0 2024-09-13T17:11:55.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fbc84001680 con 0x7fbc9006d3e0 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 >> v1:172.21.15.39:6789/0 conn(0x7fbc9006d3e0 legacy=0x7fbc9006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780039b0 0x7fbc78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc900c3e20 0x7fbc780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc78004d10 0x7fbc78005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc90158ea0 con 0x7fbc78004d10 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7800b9c0 con 0x7fbc780039b0 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7800bb80 con 0x7fbc900c3e20 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8f7fe640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc78004d10 0x7fbc78005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8effd640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780039b0 0x7fbc78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8e7fc640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc900c3e20 0x7fbc780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:55.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8f7fe640 1 -- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc900c3e20 msgr2=0x7fbc780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:55.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8f7fe640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc900c3e20 0x7fbc780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:55.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8f7fe640 1 -- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780039b0 msgr2=0x7fbc78003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:55.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8f7fe640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780039b0 0x7fbc78003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:55.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8f7fe640 1 -- 172.21.15.39:0/1949597738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78003530 con 0x7fbc78004d10 2024-09-13T17:11:55.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.418+0000 7fbc8effd640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc780039b0 0x7fbc78003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:11:55.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc78004d10 0x7fbc78005190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009fc0 tx=0x7fbc8801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126082adb2c4e327 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:55.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc880a5b80 con 0x7fbc78004d10 2024-09-13T17:11:55.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8800f040 con 0x7fbc78004d10 2024-09-13T17:11:55.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8dffb640 1 -- 172.21.15.39:0/1949597738 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc880ad440 con 0x7fbc78004d10 2024-09-13T17:11:55.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 -- 172.21.15.39:0/1949597738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc78004d10 msgr2=0x7fbc78005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:55.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 --2- 172.21.15.39:0/1949597738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc78004d10 0x7fbc78005190 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009fc0 tx=0x7fbc8801cc80 comp rx=0 tx=0).stop 2024-09-13T17:11:55.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 -- 172.21.15.39:0/1949597738 reap_dead start 2024-09-13T17:11:55.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 -- 172.21.15.39:0/1949597738 shutdown_connections 2024-09-13T17:11:55.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 -- 172.21.15.39:0/1949597738 >> 172.21.15.39:0/1949597738 conn(0x7fbc90069e10 msgr2=0x7fbc9006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:55.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 -- 172.21.15.39:0/1949597738 shutdown_connections 2024-09-13T17:11:55.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 -- 172.21.15.39:0/1949597738 wait complete. 2024-09-13T17:11:55.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 Processor -- start 2024-09-13T17:11:55.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 -- start start 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc90157470 0x7fbc900aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc90159590 0x7fbc9015b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc9015a260 0x7fbc901f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015c570 con 0x7fbc9015a260 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015c020 con 0x7fbc90157470 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc95328640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015c290 con 0x7fbc90159590 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc9015a260 0x7fbc901f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc9015a260 0x7fbc901f0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40124/0 (socket says 172.21.15.39:40124) 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 -- 172.21.15.39:0/989962759 learned_addr learned my addr 172.21.15.39:0/989962759 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8e7fc640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc90159590 0x7fbc9015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8effd640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc90157470 0x7fbc900aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 -- 172.21.15.39:0/989962759 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc90159590 msgr2=0x7fbc9015b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:55.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc90159590 0x7fbc9015b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 -- 172.21.15.39:0/989962759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc90157470 msgr2=0x7fbc900aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc90157470 0x7fbc900aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 -- 172.21.15.39:0/989962759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900a91b0 con 0x7fbc9015a260 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc8f7fe640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc9015a260 0x7fbc901f0a40 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbc880b0040 tx=0x7fbc88009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc777fe640 1 -- 172.21.15.39:0/989962759 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc880039a0 con 0x7fbc9015a260 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc777fe640 1 -- 172.21.15.39:0/989962759 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc88003b00 con 0x7fbc9015a260 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.422+0000 7fbc777fe640 1 -- 172.21.15.39:0/989962759 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88003e70 con 0x7fbc9015a260 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.426+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc90159290 con 0x7fbc9015a260 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.426+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc90157850 con 0x7fbc9015a260 2024-09-13T17:11:55.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.426+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc900a96d0 con 0x7fbc9015a260 2024-09-13T17:11:55.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.426+0000 7fbc777fe640 1 -- 172.21.15.39:0/989962759 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc880b4040 con 0x7fbc9015a260 2024-09-13T17:11:55.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.426+0000 7fbc777fe640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc7c066710 0x7fbc7c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:55.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.426+0000 7fbc777fe640 1 -- 172.21.15.39:0/989962759 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7fbc8811ecf0 con 0x7fbc9015a260 2024-09-13T17:11:55.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.426+0000 7fbc8effd640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc7c066710 0x7fbc7c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:55.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.426+0000 7fbc8effd640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc7c066710 0x7fbc7c068bd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbc800054c0 tx=0x7fbc80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:55.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.430+0000 7fbc777fe640 1 -- 172.21.15.39:0/989962759 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbc900a96d0 con 0x7fbc9015a260 2024-09-13T17:11:55.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.626+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc900a96d0 con 0x7fbc9015a260 2024-09-13T17:11:55.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.626+0000 7fbc777fe640 1 -- 172.21.15.39:0/989962759 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) v1 ==== 95+0+5934 (secure 0 0 0) 0x7fbc900a96d0 con 0x7fbc9015a260 2024-09-13T17:11:55.629 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:55.629 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":167,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:55.629 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 167 2024-09-13T17:11:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc7c066710 msgr2=0x7fbc7c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc7c066710 0x7fbc7c068bd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbc800054c0 tx=0x7fbc80009040 comp rx=0 tx=0).stop 2024-09-13T17:11:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc9015a260 msgr2=0x7fbc901f0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc9015a260 0x7fbc901f0a40 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbc880b0040 tx=0x7fbc88009750 comp rx=0 tx=0).stop 2024-09-13T17:11:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 shutdown_connections 2024-09-13T17:11:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbc7c066710 0x7fbc7c068bd0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc9015a260 0x7fbc901f0a40 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbc90159590 0x7fbc9015b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 --2- 172.21.15.39:0/989962759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc90157470 0x7fbc900aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:55.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 >> 172.21.15.39:0/989962759 conn(0x7fbc90069e10 msgr2=0x7fbc9006c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:55.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 shutdown_connections 2024-09-13T17:11:55.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:55.630+0000 7fbc95328640 1 -- 172.21.15.39:0/989962759 wait complete. 2024-09-13T17:11:55.648 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:11:57.650 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:11:57.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.786+0000 7fd7610cc640 1 Processor -- start 2024-09-13T17:11:57.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.786+0000 7fd7610cc640 1 -- start start 2024-09-13T17:11:57.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd7610cc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd75c0a4e70 0x7fd75c151460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:57.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd7610cc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75c154100 con 0x7fd75c0c4530 2024-09-13T17:11:57.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd7610cc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75c1543e0 con 0x7fd75c06d020 2024-09-13T17:11:57.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd7610cc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75c0a8c50 con 0x7fd75c0a4e70 2024-09-13T17:11:57.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd75a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd75c0a4e70 0x7fd75c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:57.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd75b59e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd75c0c4530 0x7fd75c0c4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44772/0 (socket says 172.21.15.39:44772) 2024-09-13T17:11:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd75b59e640 1 -- 172.21.15.39:0/1161094400 learned_addr learned my addr 172.21.15.39:0/1161094400 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:11:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd75a59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd75c0a4e70 0x7fd75c151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40130/0 (socket says 172.21.15.39:40130) 2024-09-13T17:11:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd75a59c640 1 -- 172.21.15.39:0/1161094400 >> v1:172.21.15.84:6789/0 conn(0x7fd75c06d020 legacy=0x7fd75c14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:11:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd75a59c640 1 -- 172.21.15.39:0/1161094400 >> v1:172.21.15.39:6789/0 conn(0x7fd75c0c4530 legacy=0x7fd75c0c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd75a59c640 1 -- 172.21.15.39:0/1161094400 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c1546c0 con 0x7fd75c0a4e70 2024-09-13T17:11:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd75a59c640 1 --2- 172.21.15.39:0/1161094400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 0x7fd75c151460 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fd7500097c0 tx=0x7fd75001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6756458584851b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:57.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd759d9b640 1 -- 172.21.15.39:0/1161094400 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500ab600 con 0x7fd75c0a4e70 2024-09-13T17:11:57.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd759d9b640 1 -- 172.21.15.39:0/1161094400 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7500a7050 con 0x7fd75c0a4e70 2024-09-13T17:11:57.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.790+0000 7fd759d9b640 1 -- 172.21.15.39:0/1161094400 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500ab980 con 0x7fd75c0a4e70 2024-09-13T17:11:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- 172.21.15.39:0/1161094400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 msgr2=0x7fd75c151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 --2- 172.21.15.39:0/1161094400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 0x7fd75c151460 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fd7500097c0 tx=0x7fd75001c3d0 comp rx=0 tx=0).stop 2024-09-13T17:11:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- 172.21.15.39:0/1161094400 shutdown_connections 2024-09-13T17:11:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 --2- 172.21.15.39:0/1161094400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 0x7fd75c151460 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- 172.21.15.39:0/1161094400 >> 172.21.15.39:0/1161094400 conn(0x7fd75c069e10 msgr2=0x7fd75c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- 172.21.15.39:0/1161094400 shutdown_connections 2024-09-13T17:11:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- 172.21.15.39:0/1161094400 wait complete. 2024-09-13T17:11:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 Processor -- start 2024-09-13T17:11:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- start start 2024-09-13T17:11:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd75c06d020 0x7fd75c146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 0x7fd75c147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd75c0c4530 0x7fd75c147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c153120 con 0x7fd75c0a4e70 2024-09-13T17:11:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c152080 con 0x7fd75c06d020 2024-09-13T17:11:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c152360 con 0x7fd75c0c4530 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 0x7fd75c147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 0x7fd75c147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40138/0 (socket says 172.21.15.39:40138) 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75a59c640 1 -- 172.21.15.39:0/567877281 learned_addr learned my addr 172.21.15.39:0/567877281 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd75c06d020 0x7fd75c146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75b59e640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd75c0c4530 0x7fd75c147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75a59c640 1 -- 172.21.15.39:0/567877281 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd75c0c4530 msgr2=0x7fd75c147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75a59c640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd75c0c4530 0x7fd75c147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75a59c640 1 -- 172.21.15.39:0/567877281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd75c06d020 msgr2=0x7fd75c146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75a59c640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd75c06d020 0x7fd75c146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75a59c640 1 -- 172.21.15.39:0/567877281 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd750009420 con 0x7fd75c0a4e70 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd75a59c640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 0x7fd75c147660 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd750018fe0 tx=0x7fd750019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7437fe640 1 -- 172.21.15.39:0/567877281 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500b4410 con 0x7fd75c0a4e70 2024-09-13T17:11:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7437fe640 1 -- 172.21.15.39:0/567877281 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7500b4570 con 0x7fd75c0a4e70 2024-09-13T17:11:57.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7437fe640 1 -- 172.21.15.39:0/567877281 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500b4920 con 0x7fd75c0a4e70 2024-09-13T17:11:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75c152640 con 0x7fd75c0a4e70 2024-09-13T17:11:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.794+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75c152c00 con 0x7fd75c0a4e70 2024-09-13T17:11:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.798+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd75c072600 con 0x7fd75c0a4e70 2024-09-13T17:11:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.798+0000 7fd7437fe640 1 -- 172.21.15.39:0/567877281 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd7500b2040 con 0x7fd75c0a4e70 2024-09-13T17:11:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.798+0000 7fd7437fe640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd748066690 0x7fd748068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:11:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.798+0000 7fd7437fe640 1 -- 172.21.15.39:0/567877281 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 9389+0+0 (secure 0 0 0) 0x7fd75011f1d0 con 0x7fd75c0a4e70 2024-09-13T17:11:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.798+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd748066690 0x7fd748068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:11:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.798+0000 7fd75ad9d640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd748066690 0x7fd748068b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd74c004e30 tx=0x7fd74c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:11:57.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.802+0000 7fd7437fe640 1 -- 172.21.15.39:0/567877281 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd7500eb690 con 0x7fd75c0a4e70 2024-09-13T17:11:57.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd75c151460 con 0x7fd75c0a4e70 2024-09-13T17:11:57.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7437fe640 1 -- 172.21.15.39:0/567877281 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) v1 ==== 95+0+5934 (secure 0 0 0) 0x7fd7500b0090 con 0x7fd75c0a4e70 2024-09-13T17:11:57.998 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:11:57.998 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":167,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":167}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:13.221168+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7851},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7851":{"gid":7851,"name":"a","rank":0,"incarnation":163,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/3334565883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3334565883},{"type":"v1","addr":"172.21.15.39:6835","nonce":3334565883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:11:57.999 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 167 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd748066690 msgr2=0x7fd748068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd748066690 0x7fd748068b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd74c004e30 tx=0x7fd74c009040 comp rx=0 tx=0).stop 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 msgr2=0x7fd75c147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 0x7fd75c147660 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fd750018fe0 tx=0x7fd750019170 comp rx=0 tx=0).stop 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 shutdown_connections 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd748066690 0x7fd748068b50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd75c0c4530 0x7fd75c147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75c0a4e70 0x7fd75c147660 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 --2- 172.21.15.39:0/567877281 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd75c06d020 0x7fd75c146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:11:58.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 >> 172.21.15.39:0/567877281 conn(0x7fd75c069e10 msgr2=0x7fd75c14f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:11:58.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 shutdown_connections 2024-09-13T17:11:58.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:11:57.998+0000 7fd7610cc640 1 -- 172.21.15.39:0/567877281 wait complete. 2024-09-13T17:11:58.017 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:11:58.163 INFO:tasks.workunit.client.1.smithi084.stdout:fs setup took 1294 secs 2024-09-13T17:11:59.510 INFO:tasks.workunit.client.1.smithi084.stdout:Syncing()...1 sec 2024-09-13T17:11:59.511 INFO:tasks.workunit.client.1.smithi084.stdout:Starting Actual Benchmark At: Fri Sep 13 17:11:59 2024 2024-09-13T17:11:59.511 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:12:00.019 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.182+0000 7f3f4f4da640 1 Processor -- start 2024-09-13T17:12:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.182+0000 7f3f4f4da640 1 -- start start 2024-09-13T17:12:00.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.182+0000 7f3f4f4da640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f480c6c20 0x7f3f480c7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.182+0000 7f3f4f4da640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f48159070 con 0x7f3f4815b9f0 2024-09-13T17:12:00.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.182+0000 7f3f4f4da640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f48158480 con 0x7f3f48155b60 2024-09-13T17:12:00.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.182+0000 7f3f4f4da640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f480c63d0 con 0x7f3f480c6c20 2024-09-13T17:12:00.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f4ca75640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f480c6c20 0x7f3f480c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f4d276640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3f48155b60 0x7f3f480a6620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:38320/0 (socket says 172.21.15.39:38320) 2024-09-13T17:12:00.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f4d276640 1 -- 172.21.15.39:0/1102662376 learned_addr learned my addr 172.21.15.39:0/1102662376 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:12:00.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f4ca75640 1 -- 172.21.15.39:0/1102662376 >> v1:172.21.15.84:6789/0 conn(0x7f3f48155b60 legacy=0x7f3f480a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f4ca75640 1 -- 172.21.15.39:0/1102662376 >> v1:172.21.15.39:6789/0 conn(0x7f3f4815b9f0 legacy=0x7f3f4815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:00.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f4ca75640 1 -- 172.21.15.39:0/1102662376 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f48158740 con 0x7f3f480c6c20 2024-09-13T17:12:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f4ca75640 1 --2- 172.21.15.39:0/1102662376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f480c6c20 0x7f3f480c7060 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3f440097c0 tx=0x7f3f4401c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=200d1ebbfa8578df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f3ffff640 1 -- 172.21.15.39:0/1102662376 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440ab600 con 0x7f3f480c6c20 2024-09-13T17:12:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f3ffff640 1 -- 172.21.15.39:0/1102662376 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f440a7050 con 0x7f3f480c6c20 2024-09-13T17:12:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.186+0000 7f3f3ffff640 1 -- 172.21.15.39:0/1102662376 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440ab980 con 0x7f3f480c6c20 2024-09-13T17:12:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 -- 172.21.15.39:0/1102662376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f480c6c20 msgr2=0x7f3f480c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 --2- 172.21.15.39:0/1102662376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f480c6c20 0x7f3f480c7060 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3f440097c0 tx=0x7f3f4401c340 comp rx=0 tx=0).stop 2024-09-13T17:12:00.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 -- 172.21.15.39:0/1102662376 shutdown_connections 2024-09-13T17:12:00.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 --2- 172.21.15.39:0/1102662376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f480c6c20 0x7f3f480c7060 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 -- 172.21.15.39:0/1102662376 >> 172.21.15.39:0/1102662376 conn(0x7f3f4806c020 msgr2=0x7f3f4806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:00.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 -- 172.21.15.39:0/1102662376 shutdown_connections 2024-09-13T17:12:00.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 -- 172.21.15.39:0/1102662376 wait complete. 2024-09-13T17:12:00.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 Processor -- start 2024-09-13T17:12:00.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 -- start start 2024-09-13T17:12:00.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f480c6c20 0x7f3f4806ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f48155b60 0x7f3f480bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3f4815b9f0 0x7f3f4814fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4806b570 con 0x7f3f480c6c20 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f480c3c90 con 0x7f3f48155b60 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4f4da640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4806af20 con 0x7f3f4815b9f0 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4d276640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f480c6c20 0x7f3f4806ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4ca75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f48155b60 0x7f3f480bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4d276640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f480c6c20 0x7f3f4806ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35328/0 (socket says 172.21.15.39:35328) 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4ca75640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f48155b60 0x7f3f480bd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:47252/0 (socket says 172.21.15.39:47252) 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4d276640 1 -- 172.21.15.39:0/813059286 learned_addr learned my addr 172.21.15.39:0/813059286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:00.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.190+0000 7f3f4da77640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3f4815b9f0 0x7f3f4814fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f4da77640 1 -- 172.21.15.39:0/813059286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f48155b60 msgr2=0x7f3f480bd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f4da77640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f48155b60 0x7f3f480bd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f4da77640 1 -- 172.21.15.39:0/813059286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f480c6c20 msgr2=0x7f3f4806ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f4da77640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f480c6c20 0x7f3f4806ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f4da77640 1 -- 172.21.15.39:0/813059286 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f44009420 con 0x7f3f4815b9f0 2024-09-13T17:12:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f4ca75640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f48155b60 0x7f3f480bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:12:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f4da77640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3f4815b9f0 0x7f3f4814fd80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800b3e0 tx=0x7f3f3800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f3dffb640 1 -- 172.21.15.39:0/813059286 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38011830 con 0x7f3f4815b9f0 2024-09-13T17:12:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f3dffb640 1 -- 172.21.15.39:0/813059286 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f38011990 con 0x7f3f4815b9f0 2024-09-13T17:12:00.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4806b200 con 0x7f3f4815b9f0 2024-09-13T17:12:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f3dffb640 1 -- 172.21.15.39:0/813059286 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f38011d40 con 0x7f3f4815b9f0 2024-09-13T17:12:00.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.194+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f48151a50 con 0x7f3f4815b9f0 2024-09-13T17:12:00.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.198+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4806b720 con 0x7f3f4815b9f0 2024-09-13T17:12:00.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.198+0000 7f3f3dffb640 1 -- 172.21.15.39:0/813059286 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f3801a400 con 0x7f3f4815b9f0 2024-09-13T17:12:00.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.198+0000 7f3f3dffb640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3f2c0666c0 0x7f3f2c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.198+0000 7f3f3dffb640 1 -- 172.21.15.39:0/813059286 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f3f380850e0 con 0x7f3f4815b9f0 2024-09-13T17:12:00.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.198+0000 7f3f4d276640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3f2c0666c0 0x7f3f2c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.198+0000 7f3f4d276640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3f2c0666c0 0x7f3f2c068b80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3f40005c10 tx=0x7f3f40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:00.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.210+0000 7f3f3dffb640 1 -- 172.21.15.39:0/813059286 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3f4806b720 con 0x7f3f4815b9f0 2024-09-13T17:12:00.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.530+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f4806b720 con 0x7f3f4815b9f0 2024-09-13T17:12:00.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f3dffb640 1 -- 172.21.15.39:0/813059286 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+5176 (secure 0 0 0) 0x7f3f4806b720 con 0x7f3f4815b9f0 2024-09-13T17:12:00.534 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:00.534 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:59.866587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:00.534 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 168 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3f2c0666c0 msgr2=0x7f3f2c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3f2c0666c0 0x7f3f2c068b80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3f40005c10 tx=0x7f3f40009040 comp rx=0 tx=0).stop 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3f4815b9f0 msgr2=0x7f3f4814fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3f4815b9f0 0x7f3f4814fd80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800b3e0 tx=0x7f3f3800b8a0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 shutdown_connections 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3f2c0666c0 0x7f3f2c068b80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f48155b60 0x7f3f480bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3f4815b9f0 0x7f3f4814fd80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 --2- 172.21.15.39:0/813059286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f480c6c20 0x7f3f4806ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.534+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 >> 172.21.15.39:0/813059286 conn(0x7f3f4806c020 msgr2=0x7f3f480a6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:00.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.538+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 shutdown_connections 2024-09-13T17:12:00.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.538+0000 7f3f4f4da640 1 -- 172.21.15.39:0/813059286 wait complete. 2024-09-13T17:12:00.558 INFO:tasks.mds_thrash.fs.[a]:mds.a down, removed from mdsmap 2024-09-13T17:12:00.559 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:12:00.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:00.560 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:12:00.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:12:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe580ec3640 1 Processor -- start 2024-09-13T17:12:00.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe580ec3640 1 -- start start 2024-09-13T17:12:00.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe580ec3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe57c0c6290 0x7fe57c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe580ec3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c0be0 0x7fe57c157d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe580ec3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c157330 con 0x7fe57c0c2690 2024-09-13T17:12:00.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe580ec3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c0c4a20 con 0x7fe57c0c6290 2024-09-13T17:12:00.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe580ec3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c158240 con 0x7fe57c0c0be0 2024-09-13T17:12:00.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe579d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe57c0c6290 0x7fe57c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe579d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe57c0c6290 0x7fe57c1577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35330/0 (socket says 172.21.15.39:35330) 2024-09-13T17:12:00.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe579d9b640 1 -- 172.21.15.39:0/2849907408 learned_addr learned my addr 172.21.15.39:0/2849907408 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:00.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe57ad9d640 1 --2- 172.21.15.39:0/2849907408 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c0be0 0x7fe57c157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe579d9b640 1 -- 172.21.15.39:0/2849907408 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c0be0 msgr2=0x7fe57c157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:00.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe579d9b640 1 --2- 172.21.15.39:0/2849907408 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c0be0 0x7fe57c157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe579d9b640 1 -- 172.21.15.39:0/2849907408 >> v1:172.21.15.84:6789/0 conn(0x7fe57c0c2690 legacy=0x7fe57c06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.698+0000 7fe579d9b640 1 -- 172.21.15.39:0/2849907408 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0c4740 con 0x7fe57c0c6290 2024-09-13T17:12:00.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe579d9b640 1 --2- 172.21.15.39:0/2849907408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c6290 0x7fe57c1577c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fe570009870 tx=0x7fe57001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb923cf00d90dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:00.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe57959a640 1 -- 172.21.15.39:0/2849907408 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700a6040 con 0x7fe57c0c6290 2024-09-13T17:12:00.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe57959a640 1 -- 172.21.15.39:0/2849907408 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5700aa600 con 0x7fe57c0c6290 2024-09-13T17:12:00.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe57959a640 1 -- 172.21.15.39:0/2849907408 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700aa990 con 0x7fe57c0c6290 2024-09-13T17:12:00.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe580ec3640 1 -- 172.21.15.39:0/2849907408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c6290 msgr2=0x7fe57c1577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe580ec3640 1 --2- 172.21.15.39:0/2849907408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c6290 0x7fe57c1577c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fe570009870 tx=0x7fe57001c4a0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe580ec3640 1 -- 172.21.15.39:0/2849907408 shutdown_connections 2024-09-13T17:12:00.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe580ec3640 1 --2- 172.21.15.39:0/2849907408 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c0be0 0x7fe57c157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe580ec3640 1 --2- 172.21.15.39:0/2849907408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c6290 0x7fe57c1577c0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe580ec3640 1 -- 172.21.15.39:0/2849907408 >> 172.21.15.39:0/2849907408 conn(0x7fe57c0c32a0 msgr2=0x7fe57c0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:00.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe580ec3640 1 -- 172.21.15.39:0/2849907408 shutdown_connections 2024-09-13T17:12:00.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe580ec3640 1 -- 172.21.15.39:0/2849907408 wait complete. 2024-09-13T17:12:00.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.702+0000 7fe580ec3640 1 Processor -- start 2024-09-13T17:12:00.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 -- start start 2024-09-13T17:12:00.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0c0be0 0x7fe57c15c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c2690 0x7fe57c0aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c6290 0x7fe57c152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0aca10 con 0x7fe57c0c2690 2024-09-13T17:12:00.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c15cac0 con 0x7fe57c0c0be0 2024-09-13T17:12:00.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c15d020 con 0x7fe57c0c6290 2024-09-13T17:12:00.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe579d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c2690 0x7fe57c0aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe579d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c2690 0x7fe57c0aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35344/0 (socket says 172.21.15.39:35344) 2024-09-13T17:12:00.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe579d9b640 1 -- 172.21.15.39:0/2010460184 learned_addr learned my addr 172.21.15.39:0/2010460184 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:00.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe57a59c640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0c0be0 0x7fe57c15c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe57ad9d640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c6290 0x7fe57c152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe579d9b640 1 -- 172.21.15.39:0/2010460184 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c6290 msgr2=0x7fe57c152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe579d9b640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c6290 0x7fe57c152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe579d9b640 1 -- 172.21.15.39:0/2010460184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0c0be0 msgr2=0x7fe57c15c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe579d9b640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0c0be0 0x7fe57c15c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe579d9b640 1 -- 172.21.15.39:0/2010460184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5700094d0 con 0x7fe57c0c2690 2024-09-13T17:12:00.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe57a59c640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0c0be0 0x7fe57c15c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:12:00.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe579d9b640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c2690 0x7fe57c0aa4c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe57001ab50 tx=0x7fe570019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:00.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe56affd640 1 -- 172.21.15.39:0/2010460184 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700b3600 con 0x7fe57c0c2690 2024-09-13T17:12:00.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe56affd640 1 -- 172.21.15.39:0/2010460184 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5700b3780 con 0x7fe57c0c2690 2024-09-13T17:12:00.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c15d300 con 0x7fe57c0c2690 2024-09-13T17:12:00.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe56affd640 1 -- 172.21.15.39:0/2010460184 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700b3b30 con 0x7fe57c0c2690 2024-09-13T17:12:00.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c1543d0 con 0x7fe57c0c2690 2024-09-13T17:12:00.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c06ae20 con 0x7fe57c0c2690 2024-09-13T17:12:00.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe56affd640 1 -- 172.21.15.39:0/2010460184 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe5700b3c90 con 0x7fe57c0c2690 2024-09-13T17:12:00.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe56affd640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe55c066690 0x7fe55c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:00.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.706+0000 7fe56affd640 1 -- 172.21.15.39:0/2010460184 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fe57011f920 con 0x7fe57c0c2690 2024-09-13T17:12:00.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.710+0000 7fe57a59c640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe55c066690 0x7fe55c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:00.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.710+0000 7fe57a59c640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe55c066690 0x7fe55c068b50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000f70 tx=0x7fe56c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:00.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.714+0000 7fe56affd640 1 -- 172.21.15.39:0/2010460184 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe5700ebec0 con 0x7fe57c0c2690 2024-09-13T17:12:00.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.986+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe57c0c0470 con 0x7fe57c0c2690 2024-09-13T17:12:00.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.986+0000 7fe56affd640 1 -- 172.21.15.39:0/2010460184 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+5176 (secure 0 0 0) 0x7fe5700ef4e0 con 0x7fe57c0c2690 2024-09-13T17:12:00.989 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:00.989 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:11:59.866587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:00.989 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 168 2024-09-13T17:12:00.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe55c066690 msgr2=0x7fe55c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe55c066690 0x7fe55c068b50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000f70 tx=0x7fe56c009040 comp rx=0 tx=0).stop 2024-09-13T17:12:00.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c2690 msgr2=0x7fe57c0aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:00.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c2690 0x7fe57c0aa4c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe57001ab50 tx=0x7fe570019240 comp rx=0 tx=0).stop 2024-09-13T17:12:00.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 shutdown_connections 2024-09-13T17:12:00.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe55c066690 0x7fe55c068b50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe57c0c6290 0x7fe57c152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe57c0c2690 0x7fe57c0aa4c0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 --2- 172.21.15.39:0/2010460184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe57c0c0be0 0x7fe57c15c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:00.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 >> 172.21.15.39:0/2010460184 conn(0x7fe57c0c32a0 msgr2=0x7fe57c0a7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:00.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 shutdown_connections 2024-09-13T17:12:00.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:00.990+0000 7fe580ec3640 1 -- 172.21.15.39:0/2010460184 wait complete. 2024-09-13T17:12:01.010 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 168, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 168, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_7932": { "addr": "172.21.15.39:6837/282472862", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 282472862, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 282472862, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7932, "incarnation": 168, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:11:59.866587+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7932 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 161, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6290": { "addr": "172.21.15.84:6835/397460031", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 397460031, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 397460031, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6290, "incarnation": 158, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:10:43.027964+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6290 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:12:02.360 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:02.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.510+0000 7f2598ac9640 1 Processor -- start 2024-09-13T17:12:02.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.510+0000 7f2598ac9640 1 -- start start 2024-09-13T17:12:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.510+0000 7f2598ac9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f259414aa40 0x7f25940c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.510+0000 7f2598ac9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f25940c1750 0x7f25940c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.510+0000 7f2598ac9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f25940c4710 0x7f25940c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.510+0000 7f2598ac9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2594151b30 con 0x7f25940c1750 2024-09-13T17:12:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.510+0000 7f2598ac9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25940c6510 con 0x7f25940c4710 2024-09-13T17:12:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.510+0000 7f2598ac9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25940c76a0 con 0x7f259414aa40 2024-09-13T17:12:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f258bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f25940c1750 0x7f25940c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2592d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f25940c4710 0x7f25940c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f258bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f25940c1750 0x7f25940c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35346/0 (socket says 172.21.15.39:35346) 2024-09-13T17:12:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f259259c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f259414aa40 0x7f25940c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2592d9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f25940c4710 0x7f25940c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:47282/0 (socket says 172.21.15.39:47282) 2024-09-13T17:12:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f258bfff640 1 -- 172.21.15.39:0/2355976719 learned_addr learned my addr 172.21.15.39:0/2355976719 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2592d9d640 1 -- 172.21.15.39:0/2355976719 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f259414aa40 msgr2=0x7f25940c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2592d9d640 1 --2- 172.21.15.39:0/2355976719 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f259414aa40 0x7f25940c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2592d9d640 1 -- 172.21.15.39:0/2355976719 >> v2:172.21.15.39:3300/0 conn(0x7f25940c1750 msgr2=0x7f25940c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2592d9d640 1 --2- 172.21.15.39:0/2355976719 >> v2:172.21.15.39:3300/0 conn(0x7f25940c1750 0x7f25940c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2592d9d640 1 -- 172.21.15.39:0/2355976719 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2594151570 con 0x7f25940c4710 2024-09-13T17:12:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f259259c640 1 --2- 172.21.15.39:0/2355976719 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f259414aa40 0x7f25940c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:12:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f258bfff640 1 --2- 172.21.15.39:0/2355976719 >> v2:172.21.15.39:3300/0 conn(0x7f25940c1750 0x7f25940c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:12:02.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2592d9d640 1 --2- 172.21.15.39:0/2355976719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25940c4710 0x7f25940c5fd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2594072610 tx=0x7f258c01c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd8a5b9416df4ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:02.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2591d9b640 1 -- 172.21.15.39:0/2355976719 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0a58f0 con 0x7f25940c4710 2024-09-13T17:12:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2591d9b640 1 -- 172.21.15.39:0/2355976719 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c0a5a50 con 0x7f25940c4710 2024-09-13T17:12:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.514+0000 7f2591d9b640 1 -- 172.21.15.39:0/2355976719 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0a5de0 con 0x7f25940c4710 2024-09-13T17:12:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 -- 172.21.15.39:0/2355976719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25940c4710 msgr2=0x7f25940c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 --2- 172.21.15.39:0/2355976719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25940c4710 0x7f25940c5fd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2594072610 tx=0x7f258c01c870 comp rx=0 tx=0).stop 2024-09-13T17:12:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 -- 172.21.15.39:0/2355976719 shutdown_connections 2024-09-13T17:12:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 --2- 172.21.15.39:0/2355976719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f25940c4710 0x7f25940c5fd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 --2- 172.21.15.39:0/2355976719 >> v2:172.21.15.39:3300/0 conn(0x7f25940c1750 0x7f25940c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 --2- 172.21.15.39:0/2355976719 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f259414aa40 0x7f25940c1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 -- 172.21.15.39:0/2355976719 >> 172.21.15.39:0/2355976719 conn(0x7f25940c3310 msgr2=0x7f25940c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 -- 172.21.15.39:0/2355976719 shutdown_connections 2024-09-13T17:12:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 -- 172.21.15.39:0/2355976719 wait complete. 2024-09-13T17:12:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 Processor -- start 2024-09-13T17:12:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 -- start start 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25940c1750 0x7f25940702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25940c4710 0x7f259414b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f259414aa40 0x7f25941f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940ab7e0 con 0x7f25940c4710 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940ac350 con 0x7f259414aa40 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f2598ac9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25940ac630 con 0x7f25940c1750 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f258bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25940c4710 0x7f259414b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f258bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25940c4710 0x7f259414b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35360/0 (socket says 172.21.15.39:35360) 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.518+0000 7f258bfff640 1 -- 172.21.15.39:0/714643080 learned_addr learned my addr 172.21.15.39:0/714643080 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f259259c640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25940c1750 0x7f25940702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f2592d9d640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f259414aa40 0x7f25941f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258bfff640 1 -- 172.21.15.39:0/714643080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25940c1750 msgr2=0x7f25940702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258bfff640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25940c1750 0x7f25940702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258bfff640 1 -- 172.21.15.39:0/714643080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f259414aa40 msgr2=0x7f25941f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258bfff640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f259414aa40 0x7f25941f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258bfff640 1 -- 172.21.15.39:0/714643080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0094d0 con 0x7f25940c4710 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f2592d9d640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f259414aa40 0x7f25941f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258bfff640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25940c4710 0x7f259414b470 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f257c00e890 tx=0x7f257c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258affd640 1 -- 172.21.15.39:0/714643080 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c004240 con 0x7f25940c4710 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258affd640 1 -- 172.21.15.39:0/714643080 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c0043a0 con 0x7f25940c4710 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258affd640 1 -- 172.21.15.39:0/714643080 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00c970 con 0x7f25940c4710 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2584009420 con 0x7f25940c4710 2024-09-13T17:12:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c009a70 con 0x7f25940c4710 2024-09-13T17:12:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259414ae20 con 0x7f25940c4710 2024-09-13T17:12:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258affd640 1 -- 172.21.15.39:0/714643080 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f257c00cad0 con 0x7f25940c4710 2024-09-13T17:12:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258affd640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2574066690 0x7f2574068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f258affd640 1 -- 172.21.15.39:0/714643080 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f257c085420 con 0x7f25940c4710 2024-09-13T17:12:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f259259c640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2574066690 0x7f2574068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.522+0000 7f259259c640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2574066690 0x7f2574068b50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2584009c50 tx=0x7f2584009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.530+0000 7f258affd640 1 -- 172.21.15.39:0/714643080 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f257c051890 con 0x7f25940c4710 2024-09-13T17:12:02.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25940ac7a0 con 0x7f25940c4710 2024-09-13T17:12:02.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f258affd640 1 -- 172.21.15.39:0/714643080 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+5177 (secure 0 0 0) 0x7f257c054eb0 con 0x7f25940c4710 2024-09-13T17:12:02.819 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:02.819 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:02.034189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:02.819 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 170 2024-09-13T17:12:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2574066690 msgr2=0x7f2574068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2574066690 0x7f2574068b50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2584009c50 tx=0x7f2584009190 comp rx=0 tx=0).stop 2024-09-13T17:12:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25940c4710 msgr2=0x7f259414b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25940c4710 0x7f259414b470 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f257c00e890 tx=0x7f257c00ed50 comp rx=0 tx=0).stop 2024-09-13T17:12:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 shutdown_connections 2024-09-13T17:12:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2574066690 0x7f2574068b50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f259414aa40 0x7f25941f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25940c4710 0x7f259414b470 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 --2- 172.21.15.39:0/714643080 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f25940c1750 0x7f25940702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:02.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 >> 172.21.15.39:0/714643080 conn(0x7f25940c3310 msgr2=0x7f259414e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:02.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 shutdown_connections 2024-09-13T17:12:02.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:02.818+0000 7f2598ac9640 1 -- 172.21.15.39:0/714643080 wait complete. 2024-09-13T17:12:02.844 INFO:tasks.mds_thrash.fs.[b]:kill mds.d (rank=0) 2024-09-13T17:12:02.844 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T17:12:02.844 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:12:02.857 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T17:12:02.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:03.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58aadd4640 1 Processor -- start 2024-09-13T17:12:03.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58aadd4640 1 -- start start 2024-09-13T17:12:03.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58aadd4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a414f540 0x7f58a40c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58aadd4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a414f920 con 0x7f58a4155200 2024-09-13T17:12:03.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58aadd4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a40c1780 con 0x7f58a40c1ab0 2024-09-13T17:12:03.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58aadd4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40709c0 con 0x7f58a414f540 2024-09-13T17:12:03.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58a9371640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f58a4155200 0x7f58a415c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:45854/0 (socket says 172.21.15.39:45854) 2024-09-13T17:12:03.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58a9371640 1 -- 172.21.15.39:0/765780454 learned_addr learned my addr 172.21.15.39:0/765780454 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:12:03.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58a8b70640 1 --2- 172.21.15.39:0/765780454 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a414f540 0x7f58a40c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58a8b70640 1 -- 172.21.15.39:0/765780454 >> v1:172.21.15.84:6789/0 conn(0x7f58a40c1ab0 legacy=0x7f58a406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58a8b70640 1 -- 172.21.15.39:0/765780454 >> v1:172.21.15.39:6789/0 conn(0x7f58a4155200 legacy=0x7f58a415c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.006+0000 7f58a8b70640 1 -- 172.21.15.39:0/765780454 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a4071580 con 0x7f58a414f540 2024-09-13T17:12:03.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58a8b70640 1 --2- 172.21.15.39:0/765780454 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a414f540 0x7f58a40c11a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58980097f0 tx=0x7f589801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c7b8cc074be969b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:03.010 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:03.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58a37fe640 1 -- 172.21.15.39:0/765780454 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58980aa610 con 0x7f58a414f540 2024-09-13T17:12:03.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58a37fe640 1 -- 172.21.15.39:0/765780454 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58980a6040 con 0x7f58a414f540 2024-09-13T17:12:03.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58a37fe640 1 -- 172.21.15.39:0/765780454 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58980aa990 con 0x7f58a414f540 2024-09-13T17:12:03.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58aadd4640 1 -- 172.21.15.39:0/765780454 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a414f540 msgr2=0x7f58a40c11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58aadd4640 1 --2- 172.21.15.39:0/765780454 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a414f540 0x7f58a40c11a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58980097f0 tx=0x7f589801c4b0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58aadd4640 1 -- 172.21.15.39:0/765780454 shutdown_connections 2024-09-13T17:12:03.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58aadd4640 1 --2- 172.21.15.39:0/765780454 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a414f540 0x7f58a40c11a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58aadd4640 1 -- 172.21.15.39:0/765780454 >> 172.21.15.39:0/765780454 conn(0x7f58a40c32a0 msgr2=0x7f58a40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:03.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58aadd4640 1 -- 172.21.15.39:0/765780454 shutdown_connections 2024-09-13T17:12:03.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.010+0000 7f58aadd4640 1 -- 172.21.15.39:0/765780454 wait complete. 2024-09-13T17:12:03.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 Processor -- start 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 -- start start 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a40c1ab0 0x7f58a414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58a414f540 0x7f58a4154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a4155200 0x7f58a4148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a414d410 con 0x7f58a414f540 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a414c060 con 0x7f58a40c1ab0 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a41f42e0 con 0x7f58a4155200 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58a414f540 0x7f58a4154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58a414f540 0x7f58a4154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35364/0 (socket says 172.21.15.39:35364) 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a3fff640 1 -- 172.21.15.39:0/4122574833 learned_addr learned my addr 172.21.15.39:0/4122574833 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:03.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a9371640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a4155200 0x7f58a4148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a8b70640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a40c1ab0 0x7f58a414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a3fff640 1 -- 172.21.15.39:0/4122574833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a4155200 msgr2=0x7f58a4148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a3fff640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a4155200 0x7f58a4148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a3fff640 1 -- 172.21.15.39:0/4122574833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a40c1ab0 msgr2=0x7f58a414bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a3fff640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a40c1ab0 0x7f58a414bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a3fff640 1 -- 172.21.15.39:0/4122574833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898009420 con 0x7f58a414f540 2024-09-13T17:12:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a3fff640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58a414f540 0x7f58a4154520 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f589000c4d0 tx=0x7f589000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a17fa640 1 -- 172.21.15.39:0/4122574833 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58900052d0 con 0x7f58a414f540 2024-09-13T17:12:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a17fa640 1 -- 172.21.15.39:0/4122574833 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890005430 con 0x7f58a414f540 2024-09-13T17:12:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a41f45c0 con 0x7f58a414f540 2024-09-13T17:12:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58a17fa640 1 -- 172.21.15.39:0/4122574833 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890012830 con 0x7f58a414f540 2024-09-13T17:12:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.014+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a41f4b80 con 0x7f58a414f540 2024-09-13T17:12:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.018+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a40619c0 con 0x7f58a414f540 2024-09-13T17:12:03.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.018+0000 7f58a17fa640 1 -- 172.21.15.39:0/4122574833 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5890012990 con 0x7f58a414f540 2024-09-13T17:12:03.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.018+0000 7f58a17fa640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5888066690 0x7f5888068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.018+0000 7f58a17fa640 1 -- 172.21.15.39:0/4122574833 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f58900849e0 con 0x7f58a414f540 2024-09-13T17:12:03.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.018+0000 7f58a8b70640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5888066690 0x7f5888068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.018+0000 7f58a8b70640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5888066690 0x7f5888068b50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f58980ae000 tx=0x7f58980a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:03.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.026+0000 7f58a17fa640 1 -- 172.21.15.39:0/4122574833 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5890050f20 con 0x7f58a414f540 2024-09-13T17:12:03.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.178+0000 7f78d0ab2640 1 Processor -- start 2024-09-13T17:12:03.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.178+0000 7f78d0ab2640 1 -- start start 2024-09-13T17:12:03.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.178+0000 7f78d0ab2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78c4049a00 0x7f78c40d61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78cb7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f78c40d37d0 0x7f78c40076a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:45864/0 (socket says 172.21.15.39:45864) 2024-09-13T17:12:03.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78cb7fe640 1 -- 172.21.15.39:0/714785774 learned_addr learned my addr 172.21.15.39:0/714785774 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:12:03.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78caffd640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78c4049a00 0x7f78c40d61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.178+0000 7f78d0ab2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c40da240 con 0x7f78c40d37d0 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78d0ab2640 1 -- 172.21.15.39:0/714785774 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c4046520 con 0x7f78c403f190 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78d0ab2640 1 -- 172.21.15.39:0/714785774 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c4046ec0 con 0x7f78c4049a00 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653147260 0 0) 0x7f78c40da240 con 0x7f78c40d37d0 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78bc003530 con 0x7f78c40d37d0 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 184840076 0 0) 0x7f78bc003530 con 0x7f78c40d37d0 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78c4049a00 msgr2=0x7f78c40d61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78c4049a00 0x7f78c40d61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 >> v1:172.21.15.84:6789/0 conn(0x7f78c403f190 legacy=0x7f78c40d66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c4046800 con 0x7f78c40d37d0 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f78c0001730 con 0x7f78c40d37d0 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 >> v1:172.21.15.39:6789/0 conn(0x7f78c40d37d0 legacy=0x7f78c40076a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78bc003a00 0x7f78bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc0047d0 0x7f78bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78bc0052b0 0x7f78bc005960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c4046ec0 con 0x7f78bc003a00 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc006150 con 0x7f78bc0047d0 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc006310 con 0x7f78bc0052b0 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78cbfff640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78bc0052b0 0x7f78bc005960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78cb7fe640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78bc003a00 0x7f78bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78caffd640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc0047d0 0x7f78bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78cb7fe640 1 -- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78bc0052b0 msgr2=0x7f78bc005960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78cb7fe640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78bc0052b0 0x7f78bc005960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78cb7fe640 1 -- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc0047d0 msgr2=0x7f78bc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:03.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78cb7fe640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc0047d0 0x7f78bc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.182+0000 7f78cb7fe640 1 -- 172.21.15.39:0/714785774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc003530 con 0x7f78bc003a00 2024-09-13T17:12:03.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78cb7fe640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78bc003a00 0x7f78bc003e20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f78c0007f10 tx=0x7f78c001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6cf7b47a4363ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:03.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0007270 con 0x7f78bc003a00 2024-09-13T17:12:03.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c00073d0 con 0x7f78bc003a00 2024-09-13T17:12:03.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78ca7fc640 1 -- 172.21.15.39:0/714785774 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c00057b0 con 0x7f78bc003a00 2024-09-13T17:12:03.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78d0ab2640 1 -- 172.21.15.39:0/714785774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78bc003a00 msgr2=0x7f78bc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78d0ab2640 1 --2- 172.21.15.39:0/714785774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78bc003a00 0x7f78bc003e20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f78c0007f10 tx=0x7f78c001ca50 comp rx=0 tx=0).stop 2024-09-13T17:12:03.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78cbfff640 1 -- 172.21.15.39:0/714785774 reap_dead start 2024-09-13T17:12:03.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78d0ab2640 1 -- 172.21.15.39:0/714785774 shutdown_connections 2024-09-13T17:12:03.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78d0ab2640 1 -- 172.21.15.39:0/714785774 >> 172.21.15.39:0/714785774 conn(0x7f78c4048580 msgr2=0x7f78c4048990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:03.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78d0ab2640 1 -- 172.21.15.39:0/714785774 shutdown_connections 2024-09-13T17:12:03.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78d0ab2640 1 -- 172.21.15.39:0/714785774 wait complete. 2024-09-13T17:12:03.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.186+0000 7f78d0ab2640 1 Processor -- start 2024-09-13T17:12:03.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78d0ab2640 1 -- start start 2024-09-13T17:12:03.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78d0ab2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc003a00 0x7f78c416e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78d0ab2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78c40c91c0 0x7f78c40d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78d0ab2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78c40c8890 0x7f78c40d8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78d0ab2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c4046be0 con 0x7f78c40c8890 2024-09-13T17:12:03.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc003a00 0x7f78c416e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc003a00 0x7f78c416e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:47312/0 (socket says 172.21.15.39:47312) 2024-09-13T17:12:03.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cb7fe640 1 -- 172.21.15.39:0/1614988587 learned_addr learned my addr 172.21.15.39:0/1614988587 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:03.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78d0ab2640 1 -- 172.21.15.39:0/1614988587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c416e4c0 con 0x7f78bc003a00 2024-09-13T17:12:03.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cbfff640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78c40c8890 0x7f78c40d8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78d0ab2640 1 -- 172.21.15.39:0/1614988587 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40d2790 con 0x7f78c40c91c0 2024-09-13T17:12:03.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cbfff640 1 -- 172.21.15.39:0/1614988587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78c40c91c0 msgr2=0x7f78c40d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cbfff640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78c40c91c0 0x7f78c40d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cbfff640 1 -- 172.21.15.39:0/1614988587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc003a00 msgr2=0x7f78c416e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cbfff640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc003a00 0x7f78c416e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cbfff640 1 -- 172.21.15.39:0/1614988587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40d2a70 con 0x7f78c40c8890 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cb7fe640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc003a00 0x7f78c416e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78cbfff640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78c40c8890 0x7f78c40d8db0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f78cc068f10 tx=0x7f78cc068670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78b7fff640 1 -- 172.21.15.39:0/1614988587 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc08f6a0 con 0x7f78c40c8890 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78b7fff640 1 -- 172.21.15.39:0/1614988587 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc08f800 con 0x7f78c40c8890 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78b7fff640 1 -- 172.21.15.39:0/1614988587 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc08fb70 con 0x7f78c40c8890 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78d0ab2640 1 -- 172.21.15.39:0/1614988587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c416e150 con 0x7f78c40c8890 2024-09-13T17:12:03.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.190+0000 7f78d0ab2640 1 -- 172.21.15.39:0/1614988587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c40d2fb0 con 0x7f78c40c8890 2024-09-13T17:12:03.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.194+0000 7f78b5ffb640 1 -- 172.21.15.39:0/1614988587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a4005260 con 0x7f78c40c8890 2024-09-13T17:12:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.194+0000 7f78b7fff640 1 -- 172.21.15.39:0/1614988587 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f78cc08fcd0 con 0x7f78c40c8890 2024-09-13T17:12:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.194+0000 7f78b7fff640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78b006aaa0 0x7f78b006cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.194+0000 7f78b7fff640 1 -- 172.21.15.39:0/1614988587 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f78cc0fe9e0 con 0x7f78c40c8890 2024-09-13T17:12:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.194+0000 7f78cb7fe640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78b006aaa0 0x7f78b006cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.194+0000 7f78cb7fe640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78b006aaa0 0x7f78b006cf60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f78c0000c10 tx=0x7f78c00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:03.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.198+0000 7f78b7fff640 1 -- 172.21.15.39:0/1614988587 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f78cc0cae70 con 0x7f78c40c8890 2024-09-13T17:12:03.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.386+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58a40c2aa0 con 0x7f58a414f540 2024-09-13T17:12:03.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.390+0000 7f58a17fa640 1 -- 172.21.15.39:0/4122574833 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5177 (secure 0 0 0) 0x7f5890054540 con 0x7f58a414f540 2024-09-13T17:12:03.390 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:03.390 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:03.390 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 171 2024-09-13T17:12:03.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.390+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5888066690 msgr2=0x7f5888068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.390+0000 7f58aadd4640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5888066690 0x7f5888068b50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f58980ae000 tx=0x7f58980a7000 comp rx=0 tx=0).stop 2024-09-13T17:12:03.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.390+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58a414f540 msgr2=0x7f58a4154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.390+0000 7f58aadd4640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58a414f540 0x7f58a4154520 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f589000c4d0 tx=0x7f589000c990 comp rx=0 tx=0).stop 2024-09-13T17:12:03.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.394+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 shutdown_connections 2024-09-13T17:12:03.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.394+0000 7f58aadd4640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5888066690 0x7f5888068b50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.394+0000 7f58aadd4640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58a4155200 0x7f58a4148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.394+0000 7f58aadd4640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58a414f540 0x7f58a4154520 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.394+0000 7f58aadd4640 1 --2- 172.21.15.39:0/4122574833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58a40c1ab0 0x7f58a414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.394+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 >> 172.21.15.39:0/4122574833 conn(0x7f58a40c32a0 msgr2=0x7f58a4069f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:03.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.394+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 shutdown_connections 2024-09-13T17:12:03.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.394+0000 7f58aadd4640 1 -- 172.21.15.39:0/4122574833 wait complete. 2024-09-13T17:12:03.424 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:12:03.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.538+0000 7f78b5ffb640 1 -- 172.21.15.39:0/1614988587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78a4005a90 con 0x7f78c40c8890 2024-09-13T17:12:03.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.538+0000 7f78b7fff640 1 -- 172.21.15.39:0/1614988587 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5177 (secure 0 0 0) 0x7f78cc097070 con 0x7f78c40c8890 2024-09-13T17:12:03.540 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:03.540 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:03.541 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 171 2024-09-13T17:12:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 -- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78b006aaa0 msgr2=0x7f78b006cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78b006aaa0 0x7f78b006cf60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f78c0000c10 tx=0x7f78c00a7070 comp rx=0 tx=0).stop 2024-09-13T17:12:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 -- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78c40c8890 msgr2=0x7f78c40d8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78c40c8890 0x7f78c40d8db0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f78cc068f10 tx=0x7f78cc068670 comp rx=0 tx=0).stop 2024-09-13T17:12:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 -- 172.21.15.39:0/1614988587 shutdown_connections 2024-09-13T17:12:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f78b006aaa0 0x7f78b006cf60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78c40c8890 0x7f78c40d8db0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f78c40c91c0 0x7f78c40d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 --2- 172.21.15.39:0/1614988587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78bc003a00 0x7f78c416e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 -- 172.21.15.39:0/1614988587 >> 172.21.15.39:0/1614988587 conn(0x7f78c4048580 msgr2=0x7f78c40da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 -- 172.21.15.39:0/1614988587 shutdown_connections 2024-09-13T17:12:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:03.542+0000 7f78b5ffb640 1 -- 172.21.15.39:0/1614988587 wait complete. 2024-09-13T17:12:03.567 INFO:tasks.mds_thrash.fs.[a]:mds.c has gained rank=0, replacing gid=7851 2024-09-13T17:12:03.567 INFO:tasks.mds_thrash.fs.[a]:waiting for 7 secs before reviving mds.a 2024-09-13T17:12:03.913 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-13T17:12:03.914+0000 7fe2dc474640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:12:04.712 INFO:tasks.workunit.client.0.smithi039.stdout:fs setup took 1320 secs 2024-09-13T17:12:05.425 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.590+0000 7fd90409c640 1 Processor -- start 2024-09-13T17:12:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd90409c640 1 -- start start 2024-09-13T17:12:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd90409c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8fc0c1730 0x7fd8fc0a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd90409c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc0c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd90409c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0c03a0 con 0x7fd8fc0c4ef0 2024-09-13T17:12:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd90409c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8fc0c0d70 con 0x7fd8fc0c1730 2024-09-13T17:12:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd90409c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0c41b0 con 0x7fd8fc0c1b10 2024-09-13T17:12:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd901637640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc0c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd901637640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc0c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:40762/0 (socket says 172.21.15.39:40762) 2024-09-13T17:12:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd901637640 1 -- 172.21.15.39:0/2459616083 learned_addr learned my addr 172.21.15.39:0/2459616083 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd900e36640 1 -- 172.21.15.39:0/2459616083 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769228965 0 0) 0x7fd8fc0c03a0 con 0x7fd8fc0c4ef0 2024-09-13T17:12:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.646+0000 7fd900e36640 1 -- 172.21.15.39:0/2459616083 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e8003510 con 0x7fd8fc0c4ef0 2024-09-13T17:12:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.594+0000 7fd901e38640 1 --2- 172.21.15.39:0/2459616083 >> v2:172.21.15.84:3300/0 conn(0x7fd8fc0c1730 0x7fd8fc0a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.646+0000 7fd901e38640 1 -- 172.21.15.39:0/2459616083 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 msgr2=0x7fd8fc0c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.646+0000 7fd901e38640 1 --2- 172.21.15.39:0/2459616083 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc0c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.646+0000 7fd901e38640 1 -- 172.21.15.39:0/2459616083 >> v1:172.21.15.84:6789/0 conn(0x7fd8fc0c4ef0 legacy=0x7fd8fc06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.646+0000 7fd901e38640 1 -- 172.21.15.39:0/2459616083 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0c4490 con 0x7fd8fc0c1730 2024-09-13T17:12:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.646+0000 7fd901637640 1 --2- 172.21.15.39:0/2459616083 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:12:05.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.650+0000 7fd901e38640 1 --2- 172.21.15.39:0/2459616083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc0c1730 0x7fd8fc0a89b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80097c0 tx=0x7fd8f801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3e82e6e30e371b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:05.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.650+0000 7fd900e36640 1 -- 172.21.15.39:0/2459616083 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80ab600 con 0x7fd8fc0c1730 2024-09-13T17:12:05.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.650+0000 7fd900e36640 1 -- 172.21.15.39:0/2459616083 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f80a7050 con 0x7fd8fc0c1730 2024-09-13T17:12:05.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.650+0000 7fd900e36640 1 -- 172.21.15.39:0/2459616083 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80ab980 con 0x7fd8fc0c1730 2024-09-13T17:12:05.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.654+0000 7fd90409c640 1 -- 172.21.15.39:0/2459616083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc0c1730 msgr2=0x7fd8fc0a89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:05.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.654+0000 7fd90409c640 1 --2- 172.21.15.39:0/2459616083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc0c1730 0x7fd8fc0a89b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80097c0 tx=0x7fd8f801c460 comp rx=0 tx=0).stop 2024-09-13T17:12:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.654+0000 7fd90409c640 1 -- 172.21.15.39:0/2459616083 shutdown_connections 2024-09-13T17:12:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.654+0000 7fd90409c640 1 --2- 172.21.15.39:0/2459616083 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.654+0000 7fd90409c640 1 --2- 172.21.15.39:0/2459616083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc0c1730 0x7fd8fc0a89b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.654+0000 7fd90409c640 1 -- 172.21.15.39:0/2459616083 >> 172.21.15.39:0/2459616083 conn(0x7fd8fc0a75b0 msgr2=0x7fd8fc0a79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.654+0000 7fd90409c640 1 -- 172.21.15.39:0/2459616083 shutdown_connections 2024-09-13T17:12:05.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.658+0000 7fd90409c640 1 -- 172.21.15.39:0/2459616083 wait complete. 2024-09-13T17:12:05.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.658+0000 7fd90409c640 1 Processor -- start 2024-09-13T17:12:05.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.658+0000 7fd90409c640 1 -- start start 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd90409c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8fc0c1730 0x7fd8fc071040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd90409c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc1630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd90409c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc0c4ef0 0x7fd8fc15f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd90409c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc071580 con 0x7fd8fc0c1730 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd90409c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc16a6c0 con 0x7fd8fc0c4ef0 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd90409c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc169d90 con 0x7fd8fc0c1b10 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901e38640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8fc0c1730 0x7fd8fc071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901e38640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8fc0c1730 0x7fd8fc071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35378/0 (socket says 172.21.15.39:35378) 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901e38640 1 -- 172.21.15.39:0/3397455144 learned_addr learned my addr 172.21.15.39:0/3397455144 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901637640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc1630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd902639640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc0c4ef0 0x7fd8fc15f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901e38640 1 -- 172.21.15.39:0/3397455144 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 msgr2=0x7fd8fc1630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901e38640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc1630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901e38640 1 -- 172.21.15.39:0/3397455144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc0c4ef0 msgr2=0x7fd8fc15f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901e38640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc0c4ef0 0x7fd8fc15f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901e38640 1 -- 172.21.15.39:0/3397455144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec008050 con 0x7fd8fc0c1730 2024-09-13T17:12:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd901e38640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8fc0c1730 0x7fd8fc071040 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80097c0 tx=0x7fd8f8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd8f27fc640 1 -- 172.21.15.39:0/3397455144 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80b4400 con 0x7fd8fc0c1730 2024-09-13T17:12:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd8f27fc640 1 -- 172.21.15.39:0/3397455144 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f80b4560 con 0x7fd8fc0c1730 2024-09-13T17:12:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f8009420 con 0x7fd8fc0c1730 2024-09-13T17:12:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc16a350 con 0x7fd8fc0c1730 2024-09-13T17:12:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.662+0000 7fd8f27fc640 1 -- 172.21.15.39:0/3397455144 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80b48d0 con 0x7fd8fc0c1730 2024-09-13T17:12:05.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.666+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e4005260 con 0x7fd8fc0c1730 2024-09-13T17:12:05.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.666+0000 7fd8f27fc640 1 -- 172.21.15.39:0/3397455144 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd8f80b2040 con 0x7fd8fc0c1730 2024-09-13T17:12:05.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.666+0000 7fd8f27fc640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8e0066690 0x7fd8e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:05.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.666+0000 7fd8f27fc640 1 -- 172.21.15.39:0/3397455144 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fd8f811ebf0 con 0x7fd8fc0c1730 2024-09-13T17:12:05.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.666+0000 7fd901637640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8e0066690 0x7fd8e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.666+0000 7fd901637640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8e0066690 0x7fd8e0068b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0c4150 tx=0x7fd8ec00c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:05.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.670+0000 7fd8f27fc640 1 -- 172.21.15.39:0/3397455144 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd8f80eb020 con 0x7fd8fc0c1730 2024-09-13T17:12:05.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.965+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8e4005a90 con 0x7fd8fc0c1730 2024-09-13T17:12:05.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.965+0000 7fd8f27fc640 1 -- 172.21.15.39:0/3397455144 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5177 (secure 0 0 0) 0x7fd8f80b0070 con 0x7fd8fc0c1730 2024-09-13T17:12:05.969 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:05.969 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:05.969 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 171 2024-09-13T17:12:05.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8e0066690 msgr2=0x7fd8e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:05.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8e0066690 0x7fd8e0068b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0c4150 tx=0x7fd8ec00c040 comp rx=0 tx=0).stop 2024-09-13T17:12:05.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8fc0c1730 msgr2=0x7fd8fc071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:05.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8fc0c1730 0x7fd8fc071040 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80097c0 tx=0x7fd8f8019200 comp rx=0 tx=0).stop 2024-09-13T17:12:05.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 shutdown_connections 2024-09-13T17:12:05.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd8e0066690 0x7fd8e0068b50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:05.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc0c4ef0 0x7fd8fc15f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:05.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8fc0c1b10 0x7fd8fc1630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:05.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 --2- 172.21.15.39:0/3397455144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8fc0c1730 0x7fd8fc071040 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:05.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 >> 172.21.15.39:0/3397455144 conn(0x7fd8fc0a75b0 msgr2=0x7fd8fc0a6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:05.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 shutdown_connections 2024-09-13T17:12:05.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:05.969+0000 7fd90409c640 1 -- 172.21.15.39:0/3397455144 wait complete. 2024-09-13T17:12:05.987 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:12:06.360 INFO:tasks.workunit.client.0.smithi039.stdout:Syncing()...1 sec 2024-09-13T17:12:06.361 INFO:tasks.workunit.client.0.smithi039.stdout:Starting Actual Benchmark At: Fri Sep 13 17:12:06 2024 2024-09-13T17:12:06.361 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:12:07.987 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:08.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff117667640 1 Processor -- start 2024-09-13T17:12:08.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff117667640 1 -- start start 2024-09-13T17:12:08.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff117667640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff11006c1a0 0x7ff11006d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:08.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff117667640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff1100c4060 0x7ff1101578d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:08.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff117667640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11006ce60 con 0x7ff11006c580 2024-09-13T17:12:08.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff117667640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1100c1c60 con 0x7ff1100c4060 2024-09-13T17:12:08.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff117667640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1100c0210 con 0x7ff11006c1a0 2024-09-13T17:12:08.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff115403640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff11006c1a0 0x7ff11006d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:08.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff115403640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff11006c1a0 0x7ff11006d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:47338/0 (socket says 172.21.15.39:47338) 2024-09-13T17:12:08.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff115403640 1 -- 172.21.15.39:0/2022067805 learned_addr learned my addr 172.21.15.39:0/2022067805 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:08.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff115c04640 1 --2- 172.21.15.39:0/2022067805 >> v2:172.21.15.39:3300/0 conn(0x7ff1100c4060 0x7ff1101578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:08.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff115403640 1 -- 172.21.15.39:0/2022067805 >> v2:172.21.15.39:3300/0 conn(0x7ff1100c4060 msgr2=0x7ff1101578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:08.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff115403640 1 --2- 172.21.15.39:0/2022067805 >> v2:172.21.15.39:3300/0 conn(0x7ff1100c4060 0x7ff1101578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:08.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff115403640 1 -- 172.21.15.39:0/2022067805 >> v1:172.21.15.84:6789/0 conn(0x7ff11006c580 legacy=0x7ff1100c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:08.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff115403640 1 -- 172.21.15.39:0/2022067805 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11006d830 con 0x7ff11006c1a0 2024-09-13T17:12:08.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.169+0000 7ff115c04640 1 --2- 172.21.15.39:0/2022067805 >> v2:172.21.15.39:3300/0 conn(0x7ff1100c4060 0x7ff1101578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:12:08.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff115403640 1 --2- 172.21.15.39:0/2022067805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 0x7ff11006d2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff110072680 tx=0x7ff10c01c160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47dd707816913b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:08.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff107fff640 1 -- 172.21.15.39:0/2022067805 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0ab600 con 0x7ff11006c1a0 2024-09-13T17:12:08.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff107fff640 1 -- 172.21.15.39:0/2022067805 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c0a7050 con 0x7ff11006c1a0 2024-09-13T17:12:08.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff107fff640 1 -- 172.21.15.39:0/2022067805 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0ab980 con 0x7ff11006c1a0 2024-09-13T17:12:08.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff117667640 1 -- 172.21.15.39:0/2022067805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 msgr2=0x7ff11006d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:08.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff117667640 1 --2- 172.21.15.39:0/2022067805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 0x7ff11006d2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff110072680 tx=0x7ff10c01c160 comp rx=0 tx=0).stop 2024-09-13T17:12:08.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff117667640 1 -- 172.21.15.39:0/2022067805 shutdown_connections 2024-09-13T17:12:08.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff117667640 1 --2- 172.21.15.39:0/2022067805 >> v2:172.21.15.39:3300/0 conn(0x7ff1100c4060 0x7ff1101578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:08.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff117667640 1 --2- 172.21.15.39:0/2022067805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 0x7ff11006d2f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:08.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.173+0000 7ff117667640 1 -- 172.21.15.39:0/2022067805 >> 172.21.15.39:0/2022067805 conn(0x7ff1100a7800 msgr2=0x7ff1100a7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:08.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 -- 172.21.15.39:0/2022067805 shutdown_connections 2024-09-13T17:12:08.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 -- 172.21.15.39:0/2022067805 wait complete. 2024-09-13T17:12:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 Processor -- start 2024-09-13T17:12:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 -- start start 2024-09-13T17:12:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 0x7ff110154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff11006c580 0x7ff1101524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1100c4060 0x7ff1101ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110155a30 con 0x7ff1100c4060 2024-09-13T17:12:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11014db60 con 0x7ff11006c1a0 2024-09-13T17:12:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff117667640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff110153950 con 0x7ff11006c580 2024-09-13T17:12:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff115c04640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1100c4060 0x7ff1101ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff115c04640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1100c4060 0x7ff1101ef9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35410/0 (socket says 172.21.15.39:35410) 2024-09-13T17:12:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff115c04640 1 -- 172.21.15.39:0/2953750396 learned_addr learned my addr 172.21.15.39:0/2953750396 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff115403640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 0x7ff110154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff114c02640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff11006c580 0x7ff1101524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff115403640 1 -- 172.21.15.39:0/2953750396 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff11006c580 msgr2=0x7ff1101524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff115403640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff11006c580 0x7ff1101524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff115403640 1 -- 172.21.15.39:0/2953750396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1100c4060 msgr2=0x7ff1101ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff115403640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1100c4060 0x7ff1101ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.177+0000 7ff115403640 1 -- 172.21.15.39:0/2953750396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff108009a70 con 0x7ff11006c1a0 2024-09-13T17:12:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.181+0000 7ff115403640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 0x7ff110154a50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0ab000 tx=0x7ff10c018f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.181+0000 7ff105ffb640 1 -- 172.21.15.39:0/2953750396 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0abd70 con 0x7ff11006c1a0 2024-09-13T17:12:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.181+0000 7ff105ffb640 1 -- 172.21.15.39:0/2953750396 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c0b4460 con 0x7ff11006c1a0 2024-09-13T17:12:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.181+0000 7ff105ffb640 1 -- 172.21.15.39:0/2953750396 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0b47d0 con 0x7ff11006c1a0 2024-09-13T17:12:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.185+0000 7ff117667640 1 -- 172.21.15.39:0/2953750396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10c0094d0 con 0x7ff11006c1a0 2024-09-13T17:12:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.185+0000 7ff117667640 1 -- 172.21.15.39:0/2953750396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff110154f90 con 0x7ff11006c1a0 2024-09-13T17:12:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.185+0000 7ff117667640 1 -- 172.21.15.39:0/2953750396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1100c5820 con 0x7ff11006c1a0 2024-09-13T17:12:08.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.185+0000 7ff105ffb640 1 -- 172.21.15.39:0/2953750396 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff10c0b2070 con 0x7ff11006c1a0 2024-09-13T17:12:08.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.189+0000 7ff105ffb640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0f4066690 0x7ff0f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.189+0000 7ff105ffb640 1 -- 172.21.15.39:0/2953750396 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7ff10c11e2d0 con 0x7ff11006c1a0 2024-09-13T17:12:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.189+0000 7ff114c02640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0f4066690 0x7ff0f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.189+0000 7ff114c02640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0f4066690 0x7ff0f4068b50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff11015db50 tx=0x7ff100007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.193+0000 7ff105ffb640 1 -- 172.21.15.39:0/2953750396 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff10c0ea890 con 0x7ff11006c1a0 2024-09-13T17:12:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.597+0000 7ff117667640 1 -- 172.21.15.39:0/2953750396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1100a8640 con 0x7ff11006c1a0 2024-09-13T17:12:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.597+0000 7ff105ffb640 1 -- 172.21.15.39:0/2953750396 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5177 (secure 0 0 0) 0x7ff10c0c2090 con 0x7ff11006c1a0 2024-09-13T17:12:08.601 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:08.601 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 171 2024-09-13T17:12:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 -- 172.21.15.39:0/2953750396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0f4066690 msgr2=0x7ff0f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0f4066690 0x7ff0f4068b50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff11015db50 tx=0x7ff100007e40 comp rx=0 tx=0).stop 2024-09-13T17:12:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 -- 172.21.15.39:0/2953750396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 msgr2=0x7ff110154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 0x7ff110154a50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0ab000 tx=0x7ff10c018f00 comp rx=0 tx=0).stop 2024-09-13T17:12:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 -- 172.21.15.39:0/2953750396 shutdown_connections 2024-09-13T17:12:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff0f4066690 0x7ff0f4068b50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1100c4060 0x7ff1101ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff11006c580 0x7ff1101524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 --2- 172.21.15.39:0/2953750396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff11006c1a0 0x7ff110154a50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 -- 172.21.15.39:0/2953750396 >> 172.21.15.39:0/2953750396 conn(0x7ff1100a7800 msgr2=0x7ff1100ab790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 -- 172.21.15.39:0/2953750396 shutdown_connections 2024-09-13T17:12:08.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:08.601+0000 7ff0f37fe640 1 -- 172.21.15.39:0/2953750396 wait complete. 2024-09-13T17:12:08.640 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:12:10.568 INFO:tasks.mds_thrash.fs.[a]:reviving mds.a 2024-09-13T17:12:10.568 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:12:10.569 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:12:10.571 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:12:10.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:10.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:10.726 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T17:12:10.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa7844ea640 1 Processor -- start 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa7844ea640 1 -- start start 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa7844ea640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa77c0c1730 0x7fa77c0a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa7844ea640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa77c0c1b10 0x7fa77c0c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa7844ea640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c06b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa7844ea640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0c0d70 con 0x7fa77c0c1730 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa7844ea640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0c03a0 con 0x7fa77c0c1b10 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa7844ea640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0c41b0 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa782286640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa77c0c1730 0x7fa77c0a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa782286640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa77c0c1730 0x7fa77c0a89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49780/0 (socket says 172.21.15.39:49780) 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa781a85640 1 --2- 172.21.15.39:0/566967372 >> v2:172.21.15.84:3300/0 conn(0x7fa77c0c1b10 0x7fa77c0c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa782a87640 1 --2- 172.21.15.39:0/566967372 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c06b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.781+0000 7fa782286640 1 -- 172.21.15.39:0/566967372 learned_addr learned my addr 172.21.15.39:0/566967372 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:10.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.789+0000 7fa782286640 1 -- 172.21.15.39:0/566967372 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 msgr2=0x7fa77c06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:10.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.789+0000 7fa782286640 1 --2- 172.21.15.39:0/566967372 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c06b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.789+0000 7fa782286640 1 -- 172.21.15.39:0/566967372 >> v2:172.21.15.84:3300/0 conn(0x7fa77c0c1b10 msgr2=0x7fa77c0c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:10.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.789+0000 7fa782286640 1 --2- 172.21.15.39:0/566967372 >> v2:172.21.15.84:3300/0 conn(0x7fa77c0c1b10 0x7fa77c0c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.789+0000 7fa782286640 1 -- 172.21.15.39:0/566967372 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0c4490 con 0x7fa77c0c1730 2024-09-13T17:12:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa782286640 1 --2- 172.21.15.39:0/566967372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1730 0x7fa77c0a89b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0a54f0 tx=0x7fa774024980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4daed606ec14607 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:10.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa781284640 1 -- 172.21.15.39:0/566967372 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77400d2d0 con 0x7fa77c0c1730 2024-09-13T17:12:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa781284640 1 -- 172.21.15.39:0/566967372 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7740ad660 con 0x7fa77c0c1730 2024-09-13T17:12:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa7844ea640 1 -- 172.21.15.39:0/566967372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1730 msgr2=0x7fa77c0a89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:10.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa7844ea640 1 --2- 172.21.15.39:0/566967372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1730 0x7fa77c0a89b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0a54f0 tx=0x7fa774024980 comp rx=0 tx=0).stop 2024-09-13T17:12:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa7844ea640 1 -- 172.21.15.39:0/566967372 shutdown_connections 2024-09-13T17:12:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa7844ea640 1 --2- 172.21.15.39:0/566967372 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c06b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa7844ea640 1 --2- 172.21.15.39:0/566967372 >> v2:172.21.15.84:3300/0 conn(0x7fa77c0c1b10 0x7fa77c0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa7844ea640 1 --2- 172.21.15.39:0/566967372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1730 0x7fa77c0a89b0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.793+0000 7fa7844ea640 1 -- 172.21.15.39:0/566967372 >> 172.21.15.39:0/566967372 conn(0x7fa77c0a75b0 msgr2=0x7fa77c0a79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:10.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 -- 172.21.15.39:0/566967372 shutdown_connections 2024-09-13T17:12:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 -- 172.21.15.39:0/566967372 wait complete. 2024-09-13T17:12:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 Processor -- start 2024-09-13T17:12:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 -- start start 2024-09-13T17:12:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1b10 0x7fa77c0bec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c0bf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa77c169140 0x7fa77c1695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c1fc1f0 con 0x7fa77c0c1b10 2024-09-13T17:12:10.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c168cb0 con 0x7fa77c169140 2024-09-13T17:12:10.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.797+0000 7fa7844ea640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0bf6a0 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa781a85640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c0bf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa782a87640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa77c169140 0x7fa77c1695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa781a85640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c0bf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:36112/0 (socket says 172.21.15.39:36112) 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa781a85640 1 -- 172.21.15.39:0/2706901695 learned_addr learned my addr 172.21.15.39:0/2706901695 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa781a85640 1 -- 172.21.15.39:0/2706901695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa77c169140 msgr2=0x7fa77c1695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa781a85640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa77c169140 0x7fa77c1695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa781a85640 1 -- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1b10 msgr2=0x7fa77c0bec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa782286640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1b10 0x7fa77c0bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa781a85640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1b10 0x7fa77c0bec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa781a85640 1 -- 172.21.15.39:0/2706901695 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77400d970 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa781a85640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c0bf160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa77000fa50 tx=0x7fa77000ff10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa782286640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1b10 0x7fa77c0bec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa76effd640 1 -- 172.21.15.39:0/2706901695 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77000de50 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa76effd640 1 -- 172.21.15.39:0/2706901695 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa770018410 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c0aaf40 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa76effd640 1 -- 172.21.15.39:0/2706901695 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa770018780 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c168940 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.801+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c168e40 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.805+0000 7fa76effd640 1 -- 172.21.15.39:0/2706901695 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa77000c050 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.805+0000 7fa76effd640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7600666e0 0x7fa760068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.805+0000 7fa76effd640 1 -- 172.21.15.39:0/2706901695 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fa770085f40 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.805+0000 7fa782286640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7600666e0 0x7fa760068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.805+0000 7fa782286640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7600666e0 0x7fa760068ba0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b6000 tx=0x7fa7740af070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:10.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.809+0000 7fa76effd640 1 -- 172.21.15.39:0/2706901695 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa7700524b0 con 0x7fa77c0c4ef0 2024-09-13T17:12:10.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb8b03640 1 Processor -- start 2024-09-13T17:12:10.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb8b03640 1 -- start start 2024-09-13T17:12:10.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb8b03640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3cb4154b70 0x7f3cb414df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb8b03640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb40c33c0 con 0x7f3cb40c28d0 2024-09-13T17:12:10.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb8b03640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb40c0ac0 con 0x7f3cb406fba0 2024-09-13T17:12:10.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb8b03640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb40611c0 con 0x7f3cb4154b70 2024-09-13T17:12:10.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb1d9b640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3cb406fba0 0x7f3cb40c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:48152/0 (socket says 172.21.15.39:48152) 2024-09-13T17:12:10.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb1d9b640 1 -- 172.21.15.39:0/3988218164 learned_addr learned my addr 172.21.15.39:0/3988218164 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:12:10.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb2d9d640 1 --2- 172.21.15.39:0/3988218164 >> v2:172.21.15.84:3300/0 conn(0x7f3cb4154b70 0x7f3cb414df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb159a640 1 -- 172.21.15.39:0/3988218164 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140751524 0 0) 0x7f3cb40c0ac0 con 0x7f3cb406fba0 2024-09-13T17:12:10.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb159a640 1 -- 172.21.15.39:0/3988218164 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c9c003530 con 0x7f3cb406fba0 2024-09-13T17:12:10.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb2d9d640 1 -- 172.21.15.39:0/3988218164 >> v1:172.21.15.84:6789/0 conn(0x7f3cb406fba0 legacy=0x7f3cb40c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:10.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb2d9d640 1 -- 172.21.15.39:0/3988218164 >> v1:172.21.15.39:6789/0 conn(0x7f3cb40c28d0 legacy=0x7f3cb40c2cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:12:10.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.861+0000 7f3cb2d9d640 1 -- 172.21.15.39:0/3988218164 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4154f50 con 0x7f3cb4154b70 2024-09-13T17:12:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb2d9d640 1 --2- 172.21.15.39:0/3988218164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cb4154b70 0x7f3cb414df30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009d90 tx=0x7f3cac01c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1037b7d8332efeb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb159a640 1 -- 172.21.15.39:0/3988218164 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac0a5760 con 0x7f3cb4154b70 2024-09-13T17:12:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb159a640 1 -- 172.21.15.39:0/3988218164 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac0a58c0 con 0x7f3cb4154b70 2024-09-13T17:12:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb159a640 1 -- 172.21.15.39:0/3988218164 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac0a5c50 con 0x7f3cb4154b70 2024-09-13T17:12:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3988218164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cb4154b70 msgr2=0x7f3cb414df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb8b03640 1 --2- 172.21.15.39:0/3988218164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cb4154b70 0x7f3cb414df30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009d90 tx=0x7f3cac01c8a0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3988218164 shutdown_connections 2024-09-13T17:12:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb8b03640 1 --2- 172.21.15.39:0/3988218164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cb4154b70 0x7f3cb414df30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3988218164 >> 172.21.15.39:0/3988218164 conn(0x7f3cb4069e10 msgr2=0x7f3cb406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3988218164 shutdown_connections 2024-09-13T17:12:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.865+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3988218164 wait complete. 2024-09-13T17:12:10.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.869+0000 7f3cb8b03640 1 Processor -- start 2024-09-13T17:12:10.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.869+0000 7f3cb8b03640 1 -- start start 2024-09-13T17:12:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.869+0000 7f3cb8b03640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cb406fba0 0x7f3cb4144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.869+0000 7f3cb8b03640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3cb40c28d0 0x7f3cb40bf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.869+0000 7f3cb8b03640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cb4154b70 0x7f3cb4152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.869+0000 7f3cb8b03640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4152650 con 0x7f3cb406fba0 2024-09-13T17:12:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.869+0000 7f3cb8b03640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb41e9d10 con 0x7f3cb4154b70 2024-09-13T17:12:10.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.869+0000 7f3cb8b03640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb41e9ff0 con 0x7f3cb40c28d0 2024-09-13T17:12:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb1d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3cb40c28d0 0x7f3cb40bf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb1d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3cb40c28d0 0x7f3cb40bf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:36118/0 (socket says 172.21.15.39:36118) 2024-09-13T17:12:10.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb1d9b640 1 -- 172.21.15.39:0/3373426290 learned_addr learned my addr 172.21.15.39:0/3373426290 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:10.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb259c640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cb406fba0 0x7f3cb4144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb1d9b640 1 -- 172.21.15.39:0/3373426290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cb4154b70 msgr2=0x7f3cb4152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:10.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb1d9b640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cb4154b70 0x7f3cb4152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb1d9b640 1 -- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cb406fba0 msgr2=0x7f3cb4144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:10.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb1d9b640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cb406fba0 0x7f3cb4144490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:10.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb1d9b640 1 -- 172.21.15.39:0/3373426290 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac009a70 con 0x7f3cb40c28d0 2024-09-13T17:12:10.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb259c640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cb406fba0 0x7f3cb4144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:12:10.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb1d9b640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3cb40c28d0 0x7f3cb40bf790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8001ec0 tx=0x7f3ca800c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:10.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3ca2ffd640 1 -- 172.21.15.39:0/3373426290 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8005cf0 con 0x7f3cb40c28d0 2024-09-13T17:12:10.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3ca2ffd640 1 -- 172.21.15.39:0/3373426290 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca8005e50 con 0x7f3cb40c28d0 2024-09-13T17:12:10.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb4152000 con 0x7f3cb40c28d0 2024-09-13T17:12:10.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb4148320 con 0x7f3cb40c28d0 2024-09-13T17:12:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb406aaf0 con 0x7f3cb40c28d0 2024-09-13T17:12:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.873+0000 7f3ca2ffd640 1 -- 172.21.15.39:0/3373426290 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8007270 con 0x7f3cb40c28d0 2024-09-13T17:12:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.877+0000 7f3ca2ffd640 1 -- 172.21.15.39:0/3373426290 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ca80073d0 con 0x7f3cb40c28d0 2024-09-13T17:12:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.877+0000 7f3ca2ffd640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c9406aba0 0x7f3c9406d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.877+0000 7f3ca2ffd640 1 -- 172.21.15.39:0/3373426290 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f3ca80849e0 con 0x7f3cb40c28d0 2024-09-13T17:12:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.877+0000 7f3cb259c640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c9406aba0 0x7f3c9406d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.877+0000 7f3cb259c640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c9406aba0 0x7f3c9406d060 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4006fd0 tx=0x7f3ca4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:10.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:10.881+0000 7f3ca2ffd640 1 -- 172.21.15.39:0/3373426290 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3ca8054a30 con 0x7f3cb40c28d0 2024-09-13T17:12:11.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.213+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa77c066a90 con 0x7fa77c0c4ef0 2024-09-13T17:12:11.214 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:11.214 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6503,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:11.214 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-13T17:12:11.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.213+0000 7fa76effd640 1 -- 172.21.15.39:0/2706901695 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5932 (secure 0 0 0) 0x7fa770055ad0 con 0x7fa77c0c4ef0 2024-09-13T17:12:11.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7600666e0 msgr2=0x7fa760068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:11.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7600666e0 0x7fa760068ba0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b6000 tx=0x7fa7740af070 comp rx=0 tx=0).stop 2024-09-13T17:12:11.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 msgr2=0x7fa77c0bf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:11.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c0bf160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa77000fa50 tx=0x7fa77000ff10 comp rx=0 tx=0).stop 2024-09-13T17:12:11.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 shutdown_connections 2024-09-13T17:12:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa7600666e0 0x7fa760068ba0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa77c169140 0x7fa77c1695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa77c0c4ef0 0x7fa77c0bf160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 --2- 172.21.15.39:0/2706901695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77c0c1b10 0x7fa77c0bec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 >> 172.21.15.39:0/2706901695 conn(0x7fa77c0a75b0 msgr2=0x7fa77c0a7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 shutdown_connections 2024-09-13T17:12:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.221+0000 7fa7844ea640 1 -- 172.21.15.39:0/2706901695 wait complete. 2024-09-13T17:12:11.246 INFO:tasks.mds_thrash.fs.[a]:mds.a reported in up:standby state 2024-09-13T17:12:11.246 INFO:tasks.mds_thrash.fs.[a]:waiting for 94 secs before thrashing 2024-09-13T17:12:11.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.285+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cb41527c0 con 0x7f3cb40c28d0 2024-09-13T17:12:11.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.285+0000 7f3ca2ffd640 1 -- 172.21.15.39:0/3373426290 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f3ca8021090 con 0x7f3cb40c28d0 2024-09-13T17:12:11.287 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:11.288 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6503,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:11.288 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-13T17:12:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c9406aba0 msgr2=0x7f3c9406d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c9406aba0 0x7f3c9406d060 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4006fd0 tx=0x7f3ca4008040 comp rx=0 tx=0).stop 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3cb40c28d0 msgr2=0x7f3cb40bf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3cb40c28d0 0x7f3cb40bf790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8001ec0 tx=0x7f3ca800c960 comp rx=0 tx=0).stop 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 shutdown_connections 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3c9406aba0 0x7f3c9406d060 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cb4154b70 0x7f3cb4152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3cb40c28d0 0x7f3cb40bf790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 --2- 172.21.15.39:0/3373426290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cb406fba0 0x7f3cb4144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 >> 172.21.15.39:0/3373426290 conn(0x7f3cb4069e10 msgr2=0x7f3cb406ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 shutdown_connections 2024-09-13T17:12:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:11.289+0000 7f3cb8b03640 1 -- 172.21.15.39:0/3373426290 wait complete. 2024-09-13T17:12:11.319 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:12:13.321 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:13.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.497+0000 7f857b3ad640 1 Processor -- start 2024-09-13T17:12:13.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.497+0000 7f857b3ad640 1 -- start start 2024-09-13T17:12:13.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f857b3ad640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85740c4710 0x7f85740c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:13.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f857b3ad640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f85740c5fd0 0x7f857415b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:13.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f857b3ad640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85740c76a0 con 0x7f857414aa40 2024-09-13T17:12:13.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f857b3ad640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8574151b30 con 0x7f85740c4710 2024-09-13T17:12:13.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f857b3ad640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8574151570 con 0x7f85740c5fd0 2024-09-13T17:12:13.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f8578948640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85740c4710 0x7f85740c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:13.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f857994a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f85740c5fd0 0x7f857415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:13.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f8578948640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85740c4710 0x7f85740c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49804/0 (socket says 172.21.15.39:49804) 2024-09-13T17:12:13.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f8578948640 1 -- 172.21.15.39:0/1796411702 learned_addr learned my addr 172.21.15.39:0/1796411702 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:13.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f8578948640 1 -- 172.21.15.39:0/1796411702 >> v2:172.21.15.84:3300/0 conn(0x7f85740c5fd0 msgr2=0x7f857415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:13.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f8578948640 1 --2- 172.21.15.39:0/1796411702 >> v2:172.21.15.84:3300/0 conn(0x7f85740c5fd0 0x7f857415b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:13.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f8578948640 1 -- 172.21.15.39:0/1796411702 >> v1:172.21.15.84:6789/0 conn(0x7f857414aa40 legacy=0x7f85740c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:13.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.501+0000 7f8578948640 1 -- 172.21.15.39:0/1796411702 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574151850 con 0x7f85740c4710 2024-09-13T17:12:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.505+0000 7f8578948640 1 --2- 172.21.15.39:0/1796411702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85740c4710 0x7f85740c5a90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f85740c24a0 tx=0x7f856401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a1521eaba8fb79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:13.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.505+0000 7f8573fff640 1 -- 172.21.15.39:0/1796411702 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85640a56f0 con 0x7f85740c4710 2024-09-13T17:12:13.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.505+0000 7f8573fff640 1 -- 172.21.15.39:0/1796411702 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85640a5850 con 0x7f85740c4710 2024-09-13T17:12:13.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.505+0000 7f857b3ad640 1 -- 172.21.15.39:0/1796411702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85740c4710 msgr2=0x7f85740c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:13.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.505+0000 7f857b3ad640 1 --2- 172.21.15.39:0/1796411702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85740c4710 0x7f85740c5a90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f85740c24a0 tx=0x7f856401c930 comp rx=0 tx=0).stop 2024-09-13T17:12:13.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.505+0000 7f857b3ad640 1 -- 172.21.15.39:0/1796411702 shutdown_connections 2024-09-13T17:12:13.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.505+0000 7f857b3ad640 1 --2- 172.21.15.39:0/1796411702 >> v2:172.21.15.84:3300/0 conn(0x7f85740c5fd0 0x7f857415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:13.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.505+0000 7f857b3ad640 1 --2- 172.21.15.39:0/1796411702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85740c4710 0x7f85740c5a90 secure :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f85740c24a0 tx=0x7f856401c930 comp rx=0 tx=0).stop 2024-09-13T17:12:13.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.505+0000 7f857b3ad640 1 -- 172.21.15.39:0/1796411702 >> 172.21.15.39:0/1796411702 conn(0x7f85740c3310 msgr2=0x7f85740c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:13.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.509+0000 7f857b3ad640 1 -- 172.21.15.39:0/1796411702 shutdown_connections 2024-09-13T17:12:13.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.509+0000 7f857b3ad640 1 -- 172.21.15.39:0/1796411702 wait complete. 2024-09-13T17:12:13.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.509+0000 7f857b3ad640 1 Processor -- start 2024-09-13T17:12:13.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.509+0000 7f857b3ad640 1 -- start start 2024-09-13T17:12:13.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857b3ad640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85740c5fd0 0x7f8574153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857b3ad640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f857414aa40 0x7f8574143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857b3ad640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8574149220 0x7f85741455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857b3ad640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85741469a0 con 0x7f857414aa40 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857b3ad640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8574153800 con 0x7f85740c5fd0 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857b3ad640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8574154160 con 0x7f8574149220 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8578948640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f857414aa40 0x7f8574143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8579149640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85740c5fd0 0x7f8574153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8578948640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f857414aa40 0x7f8574143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49806/0 (socket says 172.21.15.39:49806) 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8578948640 1 -- 172.21.15.39:0/228946254 learned_addr learned my addr 172.21.15.39:0/228946254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857994a640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8574149220 0x7f85741455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8578948640 1 -- 172.21.15.39:0/228946254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8574149220 msgr2=0x7f85741455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8578948640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8574149220 0x7f85741455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:13.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8578948640 1 -- 172.21.15.39:0/228946254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85740c5fd0 msgr2=0x7f8574153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:13.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8578948640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85740c5fd0 0x7f8574153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:13.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8578948640 1 -- 172.21.15.39:0/228946254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8564009b30 con 0x7f857414aa40 2024-09-13T17:12:13.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857994a640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8574149220 0x7f85741455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:12:13.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8578948640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f857414aa40 0x7f8574143890 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f856401ce10 tx=0x7f85640196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:13.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8571ffb640 1 -- 172.21.15.39:0/228946254 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85640090d0 con 0x7f857414aa40 2024-09-13T17:12:13.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8571ffb640 1 -- 172.21.15.39:0/228946254 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8564009230 con 0x7f857414aa40 2024-09-13T17:12:13.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f8571ffb640 1 -- 172.21.15.39:0/228946254 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564007d50 con 0x7f857414aa40 2024-09-13T17:12:13.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8574146400 con 0x7f857414aa40 2024-09-13T17:12:13.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.513+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8574153420 con 0x7f857414aa40 2024-09-13T17:12:13.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.517+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85740c1910 con 0x7f857414aa40 2024-09-13T17:12:13.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.517+0000 7f8571ffb640 1 -- 172.21.15.39:0/228946254 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85640a5900 con 0x7f857414aa40 2024-09-13T17:12:13.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.517+0000 7f8571ffb640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85600666e0 0x7f8560068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:13.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.517+0000 7f8571ffb640 1 -- 172.21.15.39:0/228946254 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7f856411fd20 con 0x7f857414aa40 2024-09-13T17:12:13.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.517+0000 7f8579149640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85600666e0 0x7f8560068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:13.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.521+0000 7f8579149640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85600666e0 0x7f8560068ba0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8568007fd0 tx=0x7f8568009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:13.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.529+0000 7f8571ffb640 1 -- 172.21.15.39:0/228946254 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f85640ec230 con 0x7f857414aa40 2024-09-13T17:12:13.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.933+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8574153280 con 0x7f857414aa40 2024-09-13T17:12:13.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.933+0000 7f8571ffb640 1 -- 172.21.15.39:0/228946254 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5932 (secure 0 0 0) 0x7f8574153280 con 0x7f857414aa40 2024-09-13T17:12:13.937 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:13.937 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6503,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:13.937 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-13T17:12:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85600666e0 msgr2=0x7f8560068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85600666e0 0x7f8560068ba0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8568007fd0 tx=0x7f8568009040 comp rx=0 tx=0).stop 2024-09-13T17:12:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f857414aa40 msgr2=0x7f8574143890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f857414aa40 0x7f8574143890 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f856401ce10 tx=0x7f85640196d0 comp rx=0 tx=0).stop 2024-09-13T17:12:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 shutdown_connections 2024-09-13T17:12:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f85600666e0 0x7f8560068ba0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8574149220 0x7f85741455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f857414aa40 0x7f8574143890 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 --2- 172.21.15.39:0/228946254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85740c5fd0 0x7f8574153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 >> 172.21.15.39:0/228946254 conn(0x7f85740c3310 msgr2=0x7f85740c0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 shutdown_connections 2024-09-13T17:12:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:13.937+0000 7f857b3ad640 1 -- 172.21.15.39:0/228946254 wait complete. 2024-09-13T17:12:13.962 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:12:15.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:16.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.125+0000 7fed30ce9640 1 Processor -- start 2024-09-13T17:12:16.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.125+0000 7fed30ce9640 1 -- start start 2024-09-13T17:12:16.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.125+0000 7fed30ce9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fed2c157b20 0x7fed2c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:16.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.125+0000 7fed30ce9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2c06b280 con 0x7fed2c06ae20 2024-09-13T17:12:16.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.125+0000 7fed30ce9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2c0c23a0 con 0x7fed2c071790 2024-09-13T17:12:16.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.125+0000 7fed30ce9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2c0c2a20 con 0x7fed2c157b20 2024-09-13T17:12:16.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.129+0000 7fed2ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fed2c157b20 0x7fed2c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:16.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.129+0000 7fed2ad9d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fed2c157b20 0x7fed2c157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:56458/0 (socket says 172.21.15.39:56458) 2024-09-13T17:12:16.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.129+0000 7fed2ad9d640 1 -- 172.21.15.39:0/3916026508 learned_addr learned my addr 172.21.15.39:0/3916026508 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:16.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.129+0000 7fed2ad9d640 1 -- 172.21.15.39:0/3916026508 >> v1:172.21.15.84:6789/0 conn(0x7fed2c071790 legacy=0x7fed2c0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:16.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.129+0000 7fed2ad9d640 1 -- 172.21.15.39:0/3916026508 >> v1:172.21.15.39:6789/0 conn(0x7fed2c06ae20 legacy=0x7fed2c0c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:16.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.129+0000 7fed2ad9d640 1 -- 172.21.15.39:0/3916026508 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c0c72a0 con 0x7fed2c157b20 2024-09-13T17:12:16.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.129+0000 7fed2ad9d640 1 --2- 172.21.15.39:0/3916026508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed2c157b20 0x7fed2c157fb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fed24009e50 tx=0x7fed2401c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=108c5db71c880e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:16.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.129+0000 7fed2959a640 1 -- 172.21.15.39:0/3916026508 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240a5770 con 0x7fed2c157b20 2024-09-13T17:12:16.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed2959a640 1 -- 172.21.15.39:0/3916026508 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed240a58d0 con 0x7fed2c157b20 2024-09-13T17:12:16.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed2959a640 1 -- 172.21.15.39:0/3916026508 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240a5c60 con 0x7fed2c157b20 2024-09-13T17:12:16.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed30ce9640 1 -- 172.21.15.39:0/3916026508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed2c157b20 msgr2=0x7fed2c157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:16.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed30ce9640 1 --2- 172.21.15.39:0/3916026508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed2c157b20 0x7fed2c157fb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fed24009e50 tx=0x7fed2401c9a0 comp rx=0 tx=0).stop 2024-09-13T17:12:16.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed30ce9640 1 -- 172.21.15.39:0/3916026508 shutdown_connections 2024-09-13T17:12:16.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed30ce9640 1 --2- 172.21.15.39:0/3916026508 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed2c157b20 0x7fed2c157fb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:16.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed30ce9640 1 -- 172.21.15.39:0/3916026508 >> 172.21.15.39:0/3916026508 conn(0x7fed2c0a96b0 msgr2=0x7fed2c0a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:16.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed30ce9640 1 -- 172.21.15.39:0/3916026508 shutdown_connections 2024-09-13T17:12:16.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed30ce9640 1 -- 172.21.15.39:0/3916026508 wait complete. 2024-09-13T17:12:16.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed30ce9640 1 Processor -- start 2024-09-13T17:12:16.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.133+0000 7fed30ce9640 1 -- start start 2024-09-13T17:12:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed30ce9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c06ae20 0x7fed2c0abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed30ce9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed2c071790 0x7fed2c0ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed30ce9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fed2c157b20 0x7fed2c1f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed30ce9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1f4bc0 con 0x7fed2c06ae20 2024-09-13T17:12:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed30ce9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1f48e0 con 0x7fed2c071790 2024-09-13T17:12:16.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed30ce9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c156150 con 0x7fed2c157b20 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2ad9d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fed2c157b20 0x7fed2c1f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c06ae20 0x7fed2c0abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed29d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed2c071790 0x7fed2c0ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2ad9d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fed2c157b20 0x7fed2c1f3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:36146/0 (socket says 172.21.15.39:36146) 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2ad9d640 1 -- 172.21.15.39:0/2369491726 learned_addr learned my addr 172.21.15.39:0/2369491726 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2ad9d640 1 -- 172.21.15.39:0/2369491726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed2c071790 msgr2=0x7fed2c0ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2ad9d640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed2c071790 0x7fed2c0ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2ad9d640 1 -- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c06ae20 msgr2=0x7fed2c0abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2ad9d640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c06ae20 0x7fed2c0abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2ad9d640 1 -- 172.21.15.39:0/2369491726 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed24009b30 con 0x7fed2c157b20 2024-09-13T17:12:16.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2a59c640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c06ae20 0x7fed2c0abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:12:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed2ad9d640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fed2c157b20 0x7fed2c1f3cf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed240af040 tx=0x7fed24019740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed1affd640 1 -- 172.21.15.39:0/2369491726 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240b8880 con 0x7fed2c157b20 2024-09-13T17:12:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed1affd640 1 -- 172.21.15.39:0/2369491726 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed240b89e0 con 0x7fed2c157b20 2024-09-13T17:12:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed30ce9640 1 -- 172.21.15.39:0/2369491726 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c156430 con 0x7fed2c157b20 2024-09-13T17:12:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed1affd640 1 -- 172.21.15.39:0/2369491726 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240b8dd0 con 0x7fed2c157b20 2024-09-13T17:12:16.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.137+0000 7fed30ce9640 1 -- 172.21.15.39:0/2369491726 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c14fcc0 con 0x7fed2c157b20 2024-09-13T17:12:16.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.141+0000 7fed18ff9640 1 -- 172.21.15.39:0/2369491726 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed04005260 con 0x7fed2c157b20 2024-09-13T17:12:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.141+0000 7fed1affd640 1 -- 172.21.15.39:0/2369491726 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fed240073f0 con 0x7fed2c157b20 2024-09-13T17:12:16.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.141+0000 7fed1affd640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fed0c066690 0x7fed0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:16.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.141+0000 7fed2a59c640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fed0c066690 0x7fed0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:16.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.141+0000 7fed1affd640 1 -- 172.21.15.39:0/2369491726 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fed24120640 con 0x7fed2c157b20 2024-09-13T17:12:16.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.141+0000 7fed2a59c640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fed0c066690 0x7fed0c068b50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fed1c007fd0 tx=0x7fed1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:16.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.145+0000 7fed1affd640 1 -- 172.21.15.39:0/2369491726 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fed240ecd80 con 0x7fed2c157b20 2024-09-13T17:12:16.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.505+0000 7fed18ff9640 1 -- 172.21.15.39:0/2369491726 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed04005a90 con 0x7fed2c157b20 2024-09-13T17:12:16.510 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:16.510 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6503,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:16.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.509+0000 7fed1affd640 1 -- 172.21.15.39:0/2369491726 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5932 (secure 0 0 0) 0x7fed240f03a0 con 0x7fed2c157b20 2024-09-13T17:12:16.511 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-13T17:12:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.509+0000 7fed18ff9640 1 -- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fed0c066690 msgr2=0x7fed0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.509+0000 7fed18ff9640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fed0c066690 0x7fed0c068b50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fed1c007fd0 tx=0x7fed1c009040 comp rx=0 tx=0).stop 2024-09-13T17:12:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.509+0000 7fed18ff9640 1 -- 172.21.15.39:0/2369491726 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fed2c157b20 msgr2=0x7fed2c1f3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.509+0000 7fed18ff9640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fed2c157b20 0x7fed2c1f3cf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed240af040 tx=0x7fed24019740 comp rx=0 tx=0).stop 2024-09-13T17:12:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.513+0000 7fed18ff9640 1 -- 172.21.15.39:0/2369491726 shutdown_connections 2024-09-13T17:12:16.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.513+0000 7fed18ff9640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fed0c066690 0x7fed0c068b50 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.513+0000 7fed18ff9640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed2c071790 0x7fed2c0ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.513+0000 7fed18ff9640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fed2c157b20 0x7fed2c1f3cf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.513+0000 7fed18ff9640 1 --2- 172.21.15.39:0/2369491726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c06ae20 0x7fed2c0abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.513+0000 7fed18ff9640 1 -- 172.21.15.39:0/2369491726 >> 172.21.15.39:0/2369491726 conn(0x7fed2c0a96b0 msgr2=0x7fed2c0c6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.513+0000 7fed18ff9640 1 -- 172.21.15.39:0/2369491726 shutdown_connections 2024-09-13T17:12:16.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:16.513+0000 7fed18ff9640 1 -- 172.21.15.39:0/2369491726 wait complete. 2024-09-13T17:12:16.543 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:12:18.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:18.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe13a44f640 1 Processor -- start 2024-09-13T17:12:18.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe13a44f640 1 -- start start 2024-09-13T17:12:18.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe13a44f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe13406cb10 0x7fe134153e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:18.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe13a44f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe1340a80d0 0x7fe1340a8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:18.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe13a44f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1340c4960 con 0x7fe1340c4110 2024-09-13T17:12:18.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe13a44f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1340c4c40 con 0x7fe1340a80d0 2024-09-13T17:12:18.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe13a44f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1340c4f20 con 0x7fe13406cb10 2024-09-13T17:12:18.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe1337fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe1340a80d0 0x7fe1340a8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:18.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe1337fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe1340a80d0 0x7fe1340a8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49824/0 (socket says 172.21.15.39:49824) 2024-09-13T17:12:18.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe1337fe640 1 -- 172.21.15.39:0/2099384816 learned_addr learned my addr 172.21.15.39:0/2099384816 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:18.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe133fff640 1 --2- 172.21.15.39:0/2099384816 >> v2:172.21.15.84:3300/0 conn(0x7fe13406cb10 0x7fe134153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:18.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe133fff640 1 -- 172.21.15.39:0/2099384816 >> v2:172.21.15.39:3300/0 conn(0x7fe1340a80d0 msgr2=0x7fe1340a8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe133fff640 1 --2- 172.21.15.39:0/2099384816 >> v2:172.21.15.39:3300/0 conn(0x7fe1340a80d0 0x7fe1340a8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe133fff640 1 -- 172.21.15.39:0/2099384816 >> v1:172.21.15.39:6789/0 conn(0x7fe1340c4110 legacy=0x7fe1340a9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe133fff640 1 -- 172.21.15.39:0/2099384816 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1340a9c40 con 0x7fe13406cb10 2024-09-13T17:12:18.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.705+0000 7fe132ffd640 1 -- 172.21.15.39:0/2099384816 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584729328 0 0) 0x7fe1340c4960 con 0x7fe1340c4110 2024-09-13T17:12:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.709+0000 7fe133fff640 1 --2- 172.21.15.39:0/2099384816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe13406cb10 0x7fe134153e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe134066d00 tx=0x7fe12801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3ab602441bf8f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.713+0000 7fe132ffd640 1 -- 172.21.15.39:0/2099384816 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1280a6040 con 0x7fe13406cb10 2024-09-13T17:12:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.713+0000 7fe132ffd640 1 -- 172.21.15.39:0/2099384816 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1280aa600 con 0x7fe13406cb10 2024-09-13T17:12:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.713+0000 7fe132ffd640 1 -- 172.21.15.39:0/2099384816 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1280aa990 con 0x7fe13406cb10 2024-09-13T17:12:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.713+0000 7fe13a44f640 1 -- 172.21.15.39:0/2099384816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe13406cb10 msgr2=0x7fe134153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:18.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.713+0000 7fe13a44f640 1 --2- 172.21.15.39:0/2099384816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe13406cb10 0x7fe134153e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe134066d00 tx=0x7fe12801c480 comp rx=0 tx=0).stop 2024-09-13T17:12:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.713+0000 7fe13a44f640 1 -- 172.21.15.39:0/2099384816 shutdown_connections 2024-09-13T17:12:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.713+0000 7fe13a44f640 1 --2- 172.21.15.39:0/2099384816 >> v2:172.21.15.39:3300/0 conn(0x7fe1340a80d0 0x7fe1340a8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.713+0000 7fe13a44f640 1 --2- 172.21.15.39:0/2099384816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe13406cb10 0x7fe134153e50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:18.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.713+0000 7fe13a44f640 1 -- 172.21.15.39:0/2099384816 >> 172.21.15.39:0/2099384816 conn(0x7fe1340a75b0 msgr2=0x7fe1340a79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 -- 172.21.15.39:0/2099384816 shutdown_connections 2024-09-13T17:12:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 -- 172.21.15.39:0/2099384816 wait complete. 2024-09-13T17:12:18.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 Processor -- start 2024-09-13T17:12:18.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 -- start start 2024-09-13T17:12:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13406cb10 0x7fe13414d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1340a80d0 0x7fe1340babe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:18.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340c4110 0x7fe134071870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13414d7d0 con 0x7fe13406cb10 2024-09-13T17:12:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13414ea00 con 0x7fe1340c4110 2024-09-13T17:12:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe13a44f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13414e440 con 0x7fe1340a80d0 2024-09-13T17:12:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1389ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340c4110 0x7fe134071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1340a80d0 0x7fe1340babe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1340a80d0 0x7fe1340babe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:36156/0 (socket says 172.21.15.39:36156) 2024-09-13T17:12:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1389ec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340c4110 0x7fe134071870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:56474/0 (socket says 172.21.15.39:56474) 2024-09-13T17:12:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1337fe640 1 -- 172.21.15.39:0/1704736750 learned_addr learned my addr 172.21.15.39:0/1704736750 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:18.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe133fff640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13406cb10 0x7fe13414d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1389ec640 1 -- 172.21.15.39:0/1704736750 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1340a80d0 msgr2=0x7fe1340babe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1389ec640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1340a80d0 0x7fe1340babe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1389ec640 1 -- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13406cb10 msgr2=0x7fe13414d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1389ec640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13406cb10 0x7fe13414d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1389ec640 1 -- 172.21.15.39:0/1704736750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128009420 con 0x7fe1340c4110 2024-09-13T17:12:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe1337fe640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1340a80d0 0x7fe1340babe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:12:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.717+0000 7fe133fff640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13406cb10 0x7fe13414d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:12:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.721+0000 7fe1389ec640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340c4110 0x7fe134071870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c410 tx=0x7fe12c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.721+0000 7fe130ff9640 1 -- 172.21.15.39:0/1704736750 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0054e0 con 0x7fe1340c4110 2024-09-13T17:12:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.721+0000 7fe130ff9640 1 -- 172.21.15.39:0/1704736750 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c003600 con 0x7fe1340c4110 2024-09-13T17:12:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.721+0000 7fe130ff9640 1 -- 172.21.15.39:0/1704736750 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0039f0 con 0x7fe1340c4110 2024-09-13T17:12:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.721+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13414e720 con 0x7fe1340c4110 2024-09-13T17:12:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.721+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1340a4e70 con 0x7fe1340c4110 2024-09-13T17:12:18.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.725+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe134153e50 con 0x7fe1340c4110 2024-09-13T17:12:18.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.725+0000 7fe130ff9640 1 -- 172.21.15.39:0/1704736750 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe12c003b50 con 0x7fe1340c4110 2024-09-13T17:12:18.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.725+0000 7fe130ff9640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 0x7fe120068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:18.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.725+0000 7fe130ff9640 1 -- 172.21.15.39:0/1704736750 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 9475+0+0 (secure 0 0 0) 0x7fe12c084930 con 0x7fe1340c4110 2024-09-13T17:12:18.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.725+0000 7fe133fff640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 0x7fe120068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:18.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.729+0000 7fe133fff640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 0x7fe120068b50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe1280ae000 tx=0x7fe1280a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:18.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:18.729+0000 7fe130ff9640 1 -- 172.21.15.39:0/1704736750 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe12c050ef0 con 0x7fe1340c4110 2024-09-13T17:12:19.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.081+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1340c16c0 con 0x7fe1340c4110 2024-09-13T17:12:19.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.081+0000 7fe130ff9640 1 -- 172.21.15.39:0/1704736750 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5932 (secure 0 0 0) 0x7fe12c017070 con 0x7fe1340c4110 2024-09-13T17:12:19.085 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:19.085 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6503,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":172}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:10:43.027964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6290},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6290":{"gid":6290,"name":"d","rank":0,"incarnation":158,"state":"up:active","state_seq":10,"addr":"172.21.15.84:6835/397460031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":397460031},{"type":"v1","addr":"172.21.15.84:6835","nonce":397460031}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:19.085 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-13T17:12:19.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 msgr2=0x7fe120068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:19.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 0x7fe120068b50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe1280ae000 tx=0x7fe1280a7000 comp rx=0 tx=0).stop 2024-09-13T17:12:19.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340c4110 msgr2=0x7fe134071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:19.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340c4110 0x7fe134071870 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c410 tx=0x7fe12c00c810 comp rx=0 tx=0).stop 2024-09-13T17:12:19.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 shutdown_connections 2024-09-13T17:12:19.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe120066690 0x7fe120068b50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:19.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1340c4110 0x7fe134071870 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:19.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1340a80d0 0x7fe1340babe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:19.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 --2- 172.21.15.39:0/1704736750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13406cb10 0x7fe13414d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:19.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 >> 172.21.15.39:0/1704736750 conn(0x7fe1340a75b0 msgr2=0x7fe1340a8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:19.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 shutdown_connections 2024-09-13T17:12:19.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:19.085+0000 7fe13a44f640 1 -- 172.21.15.39:0/1704736750 wait complete. 2024-09-13T17:12:19.114 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:12:21.115 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.289+0000 7f8f1176b640 1 Processor -- start 2024-09-13T17:12:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.289+0000 7f8f1176b640 1 -- start start 2024-09-13T17:12:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f1176b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c14e0c0 0x7f8f0c0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f1176b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8f0c0c1f50 0x7f8f0c06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f1176b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c0c2780 con 0x7f8f0c06d300 2024-09-13T17:12:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f1176b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c0c2a60 con 0x7f8f0c0c1f50 2024-09-13T17:12:21.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f1176b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0c2d40 con 0x7f8f0c14e0c0 2024-09-13T17:12:21.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f0a7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8f0c0c1f50 0x7f8f0c06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f0a7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8f0c0c1f50 0x7f8f0c06cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46292/0 (socket says 172.21.15.39:46292) 2024-09-13T17:12:21.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f0a7fc640 1 -- 172.21.15.39:0/2518851500 learned_addr learned my addr 172.21.15.39:0/2518851500 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:21.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f0a7fc640 1 -- 172.21.15.39:0/2518851500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c14e0c0 msgr2=0x7f8f0c0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:21.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f0a7fc640 1 --2- 172.21.15.39:0/2518851500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c14e0c0 0x7f8f0c0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f0a7fc640 1 -- 172.21.15.39:0/2518851500 >> v1:172.21.15.84:6789/0 conn(0x7f8f0c06d300 legacy=0x7f8f0c0c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:21.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.293+0000 7f8f0a7fc640 1 -- 172.21.15.39:0/2518851500 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0c3020 con 0x7f8f0c0c1f50 2024-09-13T17:12:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f0a7fc640 1 --2- 172.21.15.39:0/2518851500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c0c1f50 0x7f8f0c06cdc0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000bd70 tx=0x7f8f0001ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89227a25487a95e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f09ffb640 1 -- 172.21.15.39:0/2518851500 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000a7c40 con 0x7f8f0c0c1f50 2024-09-13T17:12:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f09ffb640 1 -- 172.21.15.39:0/2518851500 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f000a7da0 con 0x7f8f0c0c1f50 2024-09-13T17:12:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f09ffb640 1 -- 172.21.15.39:0/2518851500 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000af660 con 0x7f8f0c0c1f50 2024-09-13T17:12:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f1176b640 1 -- 172.21.15.39:0/2518851500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c0c1f50 msgr2=0x7f8f0c06cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f1176b640 1 --2- 172.21.15.39:0/2518851500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c0c1f50 0x7f8f0c06cdc0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000bd70 tx=0x7f8f0001ed90 comp rx=0 tx=0).stop 2024-09-13T17:12:21.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f1176b640 1 -- 172.21.15.39:0/2518851500 shutdown_connections 2024-09-13T17:12:21.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f1176b640 1 --2- 172.21.15.39:0/2518851500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c0c1f50 0x7f8f0c06cdc0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f1176b640 1 --2- 172.21.15.39:0/2518851500 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c14e0c0 0x7f8f0c0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.297+0000 7f8f1176b640 1 -- 172.21.15.39:0/2518851500 >> 172.21.15.39:0/2518851500 conn(0x7f8f0c069e10 msgr2=0x7f8f0c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:21.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 -- 172.21.15.39:0/2518851500 shutdown_connections 2024-09-13T17:12:21.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 -- 172.21.15.39:0/2518851500 wait complete. 2024-09-13T17:12:21.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 Processor -- start 2024-09-13T17:12:21.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 -- start start 2024-09-13T17:12:21.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c06d300 0x7f8f0c145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c0c1f50 0x7f8f0c070070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0c14e0c0 0x7f8f0c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c146220 con 0x7f8f0c06d300 2024-09-13T17:12:21.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c1f5080 con 0x7f8f0c14e0c0 2024-09-13T17:12:21.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c1f4ae0 con 0x7f8f0c0c1f50 2024-09-13T17:12:21.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c06d300 0x7f8f0c145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c06d300 0x7f8f0c145ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46298/0 (socket says 172.21.15.39:46298) 2024-09-13T17:12:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0affd640 1 -- 172.21.15.39:0/1948377978 learned_addr learned my addr 172.21.15.39:0/1948377978 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0b7fe640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0c14e0c0 0x7f8f0c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0affd640 1 -- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c0c1f50 msgr2=0x7f8f0c070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0a7fc640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c0c1f50 0x7f8f0c070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0affd640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c0c1f50 0x7f8f0c070070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0affd640 1 -- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0c14e0c0 msgr2=0x7f8f0c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0affd640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0c14e0c0 0x7f8f0c154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0affd640 1 -- 172.21.15.39:0/1948377978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0000b770 con 0x7f8f0c06d300 2024-09-13T17:12:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0b7fe640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0c14e0c0 0x7f8f0c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:12:21.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0affd640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c06d300 0x7f8f0c145ce0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00b440 tx=0x7f8efc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:21.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8ef37fe640 1 -- 172.21.15.39:0/1948377978 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc011890 con 0x7f8f0c06d300 2024-09-13T17:12:21.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f0a7fc640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c0c1f50 0x7f8f0c070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:12:21.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8ef37fe640 1 -- 172.21.15.39:0/1948377978 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc0119f0 con 0x7f8f0c06d300 2024-09-13T17:12:21.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.301+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c1f4da0 con 0x7f8f0c06d300 2024-09-13T17:12:21.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.305+0000 7f8ef37fe640 1 -- 172.21.15.39:0/1948377978 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc011da0 con 0x7f8f0c06d300 2024-09-13T17:12:21.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.305+0000 7f8ef37fe640 1 -- 172.21.15.39:0/1948377978 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8efc01a400 con 0x7f8f0c06d300 2024-09-13T17:12:21.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.305+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c144fd0 con 0x7f8f0c06d300 2024-09-13T17:12:21.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.305+0000 7f8ef37fe640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ef8066690 0x7f8ef8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.305+0000 7f8ef37fe640 1 -- 172.21.15.39:0/1948377978 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f8efc086200 con 0x7f8f0c06d300 2024-09-13T17:12:21.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.305+0000 7f8f0a7fc640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ef8066690 0x7f8ef8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.305+0000 7f8f0a7fc640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ef8066690 0x7f8ef8068b50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8f000b1000 tx=0x7f8f000a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:21.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.305+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c06c8c0 con 0x7f8f0c06d300 2024-09-13T17:12:21.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.313+0000 7f8ef37fe640 1 -- 172.21.15.39:0/1948377978 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8efc015020 con 0x7f8f0c06d300 2024-09-13T17:12:21.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.693+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f0c070e10 con 0x7f8f0c06d300 2024-09-13T17:12:21.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.693+0000 7f8ef37fe640 1 -- 172.21.15.39:0/1948377978 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+5179 (secure 0 0 0) 0x7f8efc050a90 con 0x7f8f0c06d300 2024-09-13T17:12:21.697 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:21.697 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":174,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:21.109267+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:21.697 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 174 2024-09-13T17:12:21.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ef8066690 msgr2=0x7f8ef8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:21.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ef8066690 0x7f8ef8068b50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8f000b1000 tx=0x7f8f000a9070 comp rx=0 tx=0).stop 2024-09-13T17:12:21.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c06d300 msgr2=0x7f8f0c145ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:21.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c06d300 0x7f8f0c145ce0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00b440 tx=0x7f8efc00b900 comp rx=0 tx=0).stop 2024-09-13T17:12:21.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 shutdown_connections 2024-09-13T17:12:21.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8ef8066690 0x7f8ef8068b50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f0c14e0c0 0x7f8f0c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8f0c0c1f50 0x7f8f0c070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 --2- 172.21.15.39:0/1948377978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f0c06d300 0x7f8f0c145ce0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 >> 172.21.15.39:0/1948377978 conn(0x7f8f0c069e10 msgr2=0x7f8f0c0c4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 shutdown_connections 2024-09-13T17:12:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.697+0000 7f8f1176b640 1 -- 172.21.15.39:0/1948377978 wait complete. 2024-09-13T17:12:21.725 INFO:tasks.mds_thrash.fs.[b]:mds.d down, removed from mdsmap 2024-09-13T17:12:21.725 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:12:21.725 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:21.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690759e640 1 Processor -- start 2024-09-13T17:12:21.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690759e640 1 -- start start 2024-09-13T17:12:21.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690759e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f690814de90 0x7f6908154760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690759e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f690806e920 0x7f690806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690759e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6908070a30 con 0x7f690806b360 2024-09-13T17:12:21.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690759e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69080715a0 con 0x7f690814de90 2024-09-13T17:12:21.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690759e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69080c2830 con 0x7f690806e920 2024-09-13T17:12:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690659c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f690814de90 0x7f6908154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690659c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f690814de90 0x7f6908154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46314/0 (socket says 172.21.15.39:46314) 2024-09-13T17:12:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690659c640 1 -- 172.21.15.39:0/4068773665 learned_addr learned my addr 172.21.15.39:0/4068773665 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f6905d9b640 1 --2- 172.21.15.39:0/4068773665 >> v2:172.21.15.84:3300/0 conn(0x7f690806e920 0x7f690806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690659c640 1 -- 172.21.15.39:0/4068773665 >> v2:172.21.15.84:3300/0 conn(0x7f690806e920 msgr2=0x7f690806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690659c640 1 --2- 172.21.15.39:0/4068773665 >> v2:172.21.15.84:3300/0 conn(0x7f690806e920 0x7f690806ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690659c640 1 -- 172.21.15.39:0/4068773665 >> v1:172.21.15.84:6789/0 conn(0x7f690806b360 legacy=0x7f6908150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:21.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.889+0000 7f690659c640 1 -- 172.21.15.39:0/4068773665 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908154ca0 con 0x7f690814de90 2024-09-13T17:12:21.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690659c640 1 --2- 172.21.15.39:0/4068773665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690814de90 0x7f6908154760 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6908071dd0 tx=0x7f68fc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7032830e49063b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:21.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690559a640 1 -- 172.21.15.39:0/4068773665 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0ab600 con 0x7f690814de90 2024-09-13T17:12:21.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690559a640 1 -- 172.21.15.39:0/4068773665 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc0a7050 con 0x7f690814de90 2024-09-13T17:12:21.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690559a640 1 -- 172.21.15.39:0/4068773665 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0ab980 con 0x7f690814de90 2024-09-13T17:12:21.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690759e640 1 -- 172.21.15.39:0/4068773665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690814de90 msgr2=0x7f6908154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:21.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690759e640 1 --2- 172.21.15.39:0/4068773665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690814de90 0x7f6908154760 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6908071dd0 tx=0x7f68fc01c400 comp rx=0 tx=0).stop 2024-09-13T17:12:21.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690759e640 1 -- 172.21.15.39:0/4068773665 shutdown_connections 2024-09-13T17:12:21.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690759e640 1 --2- 172.21.15.39:0/4068773665 >> v2:172.21.15.84:3300/0 conn(0x7f690806e920 0x7f690806ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690759e640 1 --2- 172.21.15.39:0/4068773665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690814de90 0x7f6908154760 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690759e640 1 -- 172.21.15.39:0/4068773665 >> 172.21.15.39:0/4068773665 conn(0x7f69080c30c0 msgr2=0x7f69080c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:21.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690759e640 1 -- 172.21.15.39:0/4068773665 shutdown_connections 2024-09-13T17:12:21.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.893+0000 7f690759e640 1 -- 172.21.15.39:0/4068773665 wait complete. 2024-09-13T17:12:21.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690759e640 1 Processor -- start 2024-09-13T17:12:21.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690759e640 1 -- start start 2024-09-13T17:12:21.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690759e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f690806b360 0x7f6908152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690806e920 0x7f690814cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690759e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690814de90 0x7f690814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690759e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690814e5d0 con 0x7f690806e920 2024-09-13T17:12:21.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690759e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690814e820 con 0x7f690814de90 2024-09-13T17:12:21.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690759e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69080ac1d0 con 0x7f690806b360 2024-09-13T17:12:21.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f6906d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690814de90 0x7f690814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f6906d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690814de90 0x7f690814f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:41048/0 (socket says 172.21.15.39:41048) 2024-09-13T17:12:21.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f6906d9d640 1 -- 172.21.15.39:0/2225112200 learned_addr learned my addr 172.21.15.39:0/2225112200 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:21.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f6905d9b640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690806e920 0x7f690814cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690659c640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f690806b360 0x7f6908152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f6906d9d640 1 -- 172.21.15.39:0/2225112200 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f690806b360 msgr2=0x7f6908152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:21.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f6906d9d640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f690806b360 0x7f6908152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f6906d9d640 1 -- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690806e920 msgr2=0x7f690814cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:21.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f6906d9d640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690806e920 0x7f690814cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:21.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f6906d9d640 1 -- 172.21.15.39:0/2225112200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0094d0 con 0x7f690814de90 2024-09-13T17:12:21.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.897+0000 7f690659c640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f690806b360 0x7f6908152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:12:21.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f6906d9d640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690814de90 0x7f690814f140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f690000c730 tx=0x7f690000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:21.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f68f6ffd640 1 -- 172.21.15.39:0/2225112200 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900003b80 con 0x7f690814de90 2024-09-13T17:12:21.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f68f6ffd640 1 -- 172.21.15.39:0/2225112200 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900003ce0 con 0x7f690814de90 2024-09-13T17:12:21.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6908152700 con 0x7f690814de90 2024-09-13T17:12:21.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f68f6ffd640 1 -- 172.21.15.39:0/2225112200 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690000aa30 con 0x7f690814de90 2024-09-13T17:12:21.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690814d1e0 con 0x7f690814de90 2024-09-13T17:12:21.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6908152560 con 0x7f690814de90 2024-09-13T17:12:21.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f68f6ffd640 1 -- 172.21.15.39:0/2225112200 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f690000ab90 con 0x7f690814de90 2024-09-13T17:12:21.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f68f6ffd640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68e8066690 0x7f68e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:21.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f68f6ffd640 1 -- 172.21.15.39:0/2225112200 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f69000852f0 con 0x7f690814de90 2024-09-13T17:12:21.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f690659c640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68e8066690 0x7f68e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:21.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.901+0000 7f690659c640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68e8066690 0x7f68e8068b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0ad000 tx=0x7f68fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:21.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:21.913+0000 7f68f6ffd640 1 -- 172.21.15.39:0/2225112200 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6900051810 con 0x7f690814de90 2024-09-13T17:12:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.273+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6908152560 con 0x7f690814de90 2024-09-13T17:12:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.273+0000 7f68f6ffd640 1 -- 172.21.15.39:0/2225112200 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+5176 (secure 0 0 0) 0x7f6908152560 con 0x7f690814de90 2024-09-13T17:12:22.277 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:22.277 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":175,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:22.117622+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:rejoin","state_seq":5,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 175 2024-09-13T17:12:22.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68e8066690 msgr2=0x7f68e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:22.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68e8066690 0x7f68e8068b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0ad000 tx=0x7f68fc008040 comp rx=0 tx=0).stop 2024-09-13T17:12:22.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690814de90 msgr2=0x7f690814f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:22.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690814de90 0x7f690814f140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f690000c730 tx=0x7f690000cbf0 comp rx=0 tx=0).stop 2024-09-13T17:12:22.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 shutdown_connections 2024-09-13T17:12:22.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f68e8066690 0x7f68e8068b50 secure :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0ad000 tx=0x7f68fc008040 comp rx=0 tx=0).stop 2024-09-13T17:12:22.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f690814de90 0x7f690814f140 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:22.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690806e920 0x7f690814cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:22.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 --2- 172.21.15.39:0/2225112200 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f690806b360 0x7f6908152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:22.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 >> 172.21.15.39:0/2225112200 conn(0x7f69080c30c0 msgr2=0x7f690814b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:22.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 shutdown_connections 2024-09-13T17:12:22.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:22.277+0000 7f690759e640 1 -- 172.21.15.39:0/2225112200 wait complete. 2024-09-13T17:12:22.305 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 175, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 171, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_7932": { "addr": "172.21.15.39:6837/282472862", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 282472862, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 282472862, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7932, "incarnation": 168, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:12:03.044383+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7932 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 175, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6503": { "addr": "172.21.15.39:6835/305129652", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 305129652, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 305129652, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6503, "incarnation": 173, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 58, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:12:22.117622+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6503 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:12:24.306 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.629+0000 7f7aa7fbe640 1 Processor -- start 2024-09-13T17:12:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa7fbe640 1 -- start start 2024-09-13T17:12:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa7fbe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7a980db810 0x7f7a9800aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa7fbe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7a980dec50 0x7f7a980e8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa7fbe640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9803e270 con 0x7f7a9800b1e0 2024-09-13T17:12:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa7fbe640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9803e530 con 0x7f7a980dec50 2024-09-13T17:12:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa7fbe640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a98030780 con 0x7f7a980db810 2024-09-13T17:12:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa655b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7a980dec50 0x7f7a980e8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa655b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7a980dec50 0x7f7a980e8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46338/0 (socket says 172.21.15.39:46338) 2024-09-13T17:12:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa655b640 1 -- 172.21.15.39:0/3142518257 learned_addr learned my addr 172.21.15.39:0/3142518257 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa5d5a640 1 --2- 172.21.15.39:0/3142518257 >> v2:172.21.15.84:3300/0 conn(0x7f7a980db810 0x7f7a9800aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa655b640 1 -- 172.21.15.39:0/3142518257 >> v2:172.21.15.84:3300/0 conn(0x7f7a980db810 msgr2=0x7f7a9800aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:12:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa655b640 1 --2- 172.21.15.39:0/3142518257 >> v2:172.21.15.84:3300/0 conn(0x7f7a980db810 0x7f7a9800aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa655b640 1 -- 172.21.15.39:0/3142518257 >> v1:172.21.15.84:6789/0 conn(0x7f7a9800b1e0 legacy=0x7f7a980de540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.633+0000 7f7aa655b640 1 -- 172.21.15.39:0/3142518257 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98030a60 con 0x7f7a980dec50 2024-09-13T17:12:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.637+0000 7f7aa655b640 1 --2- 172.21.15.39:0/3142518257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a980dec50 0x7f7a980e8970 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0068a40 tx=0x7f7aa0096950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386b49153569dde3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.645+0000 7f7aa4d58640 1 -- 172.21.15.39:0/3142518257 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0123710 con 0x7f7a980dec50 2024-09-13T17:12:24.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.645+0000 7f7aa4d58640 1 -- 172.21.15.39:0/3142518257 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa0123870 con 0x7f7a980dec50 2024-09-13T17:12:24.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.645+0000 7f7aa4d58640 1 -- 172.21.15.39:0/3142518257 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0123c60 con 0x7f7a980dec50 2024-09-13T17:12:24.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.645+0000 7f7aa7fbe640 1 -- 172.21.15.39:0/3142518257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a980dec50 msgr2=0x7f7a980e8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:24.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.645+0000 7f7aa7fbe640 1 --2- 172.21.15.39:0/3142518257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a980dec50 0x7f7a980e8970 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0068a40 tx=0x7f7aa0096950 comp rx=0 tx=0).stop 2024-09-13T17:12:24.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.645+0000 7f7aa7fbe640 1 -- 172.21.15.39:0/3142518257 shutdown_connections 2024-09-13T17:12:24.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.645+0000 7f7aa7fbe640 1 --2- 172.21.15.39:0/3142518257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a980dec50 0x7f7a980e8970 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:24.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.645+0000 7f7aa7fbe640 1 --2- 172.21.15.39:0/3142518257 >> v2:172.21.15.84:3300/0 conn(0x7f7a980db810 0x7f7a9800aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:24.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.645+0000 7f7aa7fbe640 1 -- 172.21.15.39:0/3142518257 >> 172.21.15.39:0/3142518257 conn(0x7f7a98047d20 msgr2=0x7f7a98048130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:24.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.649+0000 7f7aa7fbe640 1 -- 172.21.15.39:0/3142518257 shutdown_connections 2024-09-13T17:12:24.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.649+0000 7f7aa7fbe640 1 -- 172.21.15.39:0/3142518257 wait complete. 2024-09-13T17:12:24.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.649+0000 7f7aa7fbe640 1 Processor -- start 2024-09-13T17:12:24.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa7fbe640 1 -- start start 2024-09-13T17:12:24.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa7fbe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9800b1e0 0x7f7a9802f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa7fbe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a980db810 0x7f7a98195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa7fbe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a980dec50 0x7f7a98196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa7fbe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980d6ff0 con 0x7f7a980db810 2024-09-13T17:12:24.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa7fbe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980443a0 con 0x7f7a980dec50 2024-09-13T17:12:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa7fbe640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a98044960 con 0x7f7a9800b1e0 2024-09-13T17:12:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa5d5a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9800b1e0 0x7f7a9802f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa5d5a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9800b1e0 0x7f7a9802f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:59968/0 (socket says 172.21.15.39:59968) 2024-09-13T17:12:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa5d5a640 1 -- 172.21.15.39:0/1378494743 learned_addr learned my addr 172.21.15.39:0/1378494743 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa655b640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a980dec50 0x7f7a98196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa5d5a640 1 -- 172.21.15.39:0/1378494743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a980dec50 msgr2=0x7f7a98196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa5d5a640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a980dec50 0x7f7a98196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa5d5a640 1 -- 172.21.15.39:0/1378494743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a980db810 msgr2=0x7f7a98195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa5d5a640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a980db810 0x7f7a98195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa5d5a640 1 -- 172.21.15.39:0/1378494743 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98196650 con 0x7f7a9800b1e0 2024-09-13T17:12:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa5d5a640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9800b1e0 0x7f7a9802f620 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e9b0 tx=0x7f7a9c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7a967fc640 1 -- 172.21.15.39:0/1378494743 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00c9a0 con 0x7f7a9800b1e0 2024-09-13T17:12:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7a967fc640 1 -- 172.21.15.39:0/1378494743 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c00cb00 con 0x7f7a9800b1e0 2024-09-13T17:12:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa7fbe640 1 -- 172.21.15.39:0/1378494743 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a98199530 con 0x7f7a9800b1e0 2024-09-13T17:12:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7a967fc640 1 -- 172.21.15.39:0/1378494743 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00ce90 con 0x7f7a9800b1e0 2024-09-13T17:12:24.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.653+0000 7f7aa7fbe640 1 -- 172.21.15.39:0/1378494743 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a980445f0 con 0x7f7a9800b1e0 2024-09-13T17:12:24.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.657+0000 7f7a967fc640 1 -- 172.21.15.39:0/1378494743 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a9c013470 con 0x7f7a9800b1e0 2024-09-13T17:12:24.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.657+0000 7f7a967fc640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a840666e0 0x7f7a84068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:24.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.657+0000 7f7aa5559640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a840666e0 0x7f7a84068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:24.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.661+0000 7f7aa7fbe640 1 -- 172.21.15.39:0/1378494743 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9803e990 con 0x7f7a9800b1e0 2024-09-13T17:12:24.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.661+0000 7f7aa5559640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a840666e0 0x7f7a84068ba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7a90000f70 tx=0x7f7a900079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:24.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.661+0000 7f7a967fc640 1 -- 172.21.15.39:0/1378494743 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f7a9c085620 con 0x7f7a9800b1e0 2024-09-13T17:12:24.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:24.669+0000 7f7a967fc640 1 -- 172.21.15.39:0/1378494743 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7a9c051ab0 con 0x7f7a9800b1e0 2024-09-13T17:12:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.517+0000 7f7aa7fbe640 1 -- 172.21.15.39:0/1378494743 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a98007050 con 0x7f7a9800b1e0 2024-09-13T17:12:25.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.521+0000 7f7a967fc640 1 -- 172.21.15.39:0/1378494743 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+5176 (secure 0 0 0) 0x7f7a9c021090 con 0x7f7a9800b1e0 2024-09-13T17:12:25.525 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:25.525 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":176,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:25.525 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 176 2024-09-13T17:12:25.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.525+0000 7f7a83fff640 1 -- 172.21.15.39:0/1378494743 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a840666e0 msgr2=0x7f7a84068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:25.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.525+0000 7f7a83fff640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a840666e0 0x7f7a84068ba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7a90000f70 tx=0x7f7a900079c0 comp rx=0 tx=0).stop 2024-09-13T17:12:25.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.525+0000 7f7a83fff640 1 -- 172.21.15.39:0/1378494743 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9800b1e0 msgr2=0x7f7a9802f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:25.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.525+0000 7f7a83fff640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9800b1e0 0x7f7a9802f620 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e9b0 tx=0x7f7a9c00ee70 comp rx=0 tx=0).stop 2024-09-13T17:12:25.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.525+0000 7f7a83fff640 1 -- 172.21.15.39:0/1378494743 shutdown_connections 2024-09-13T17:12:25.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.525+0000 7f7a83fff640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a840666e0 0x7f7a84068ba0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:25.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.525+0000 7f7a83fff640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a980dec50 0x7f7a98196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:25.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.525+0000 7f7a83fff640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a980db810 0x7f7a98195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:25.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.525+0000 7f7a83fff640 1 --2- 172.21.15.39:0/1378494743 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a9800b1e0 0x7f7a9802f620 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:25.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.529+0000 7f7a83fff640 1 -- 172.21.15.39:0/1378494743 >> 172.21.15.39:0/1378494743 conn(0x7f7a98047d20 msgr2=0x7f7a9804a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.529+0000 7f7a83fff640 1 -- 172.21.15.39:0/1378494743 shutdown_connections 2024-09-13T17:12:25.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:25.529+0000 7f7a83fff640 1 -- 172.21.15.39:0/1378494743 wait complete. 2024-09-13T17:12:25.588 INFO:tasks.mds_thrash.fs.[b]:mds.a has gained rank=0, replacing gid=6290 2024-09-13T17:12:25.589 INFO:tasks.mds_thrash.fs.[b]:waiting for 7 secs before reviving mds.d 2024-09-13T17:12:26.084 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T17:12:26.081+0000 7f0dccbfd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:12:30.627 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:12:30.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:12:32.590 INFO:tasks.mds_thrash.fs.[b]:reviving mds.d 2024-09-13T17:12:32.590 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T17:12:32.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T17:12:32.592 INFO:tasks.ceph.mds.d:Started 2024-09-13T17:12:32.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:12:32.747 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T17:12:32.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.825+0000 7f4a23245640 1 Processor -- start 2024-09-13T17:12:32.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.825+0000 7f4a23245640 1 -- start start 2024-09-13T17:12:32.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a23245640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4a1c155af0 0x7f4a1c0c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:32.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a23245640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c0c0a40 0x7f4a1c0c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:32.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a23245640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c06c1b0 con 0x7f4a1c0c1920 2024-09-13T17:12:32.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a23245640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c06c500 con 0x7f4a1c155af0 2024-09-13T17:12:32.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a23245640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0c4350 con 0x7f4a1c0c0a40 2024-09-13T17:12:32.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a1bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c0c0a40 0x7f4a1c0c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:32.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a217e2640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4a1c0c1920 0x7f4a1c0c3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56610/0 (socket says 172.21.15.39:56610) 2024-09-13T17:12:32.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a217e2640 1 -- 172.21.15.39:0/2787226277 learned_addr learned my addr 172.21.15.39:0/2787226277 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:12:32.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a1bfff640 1 -- 172.21.15.39:0/2787226277 >> v2:172.21.15.39:3300/0 conn(0x7f4a1c155af0 msgr2=0x7f4a1c0c13e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:12:32.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a1bfff640 1 --2- 172.21.15.39:0/2787226277 >> v2:172.21.15.39:3300/0 conn(0x7f4a1c155af0 0x7f4a1c0c13e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:32.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a1bfff640 1 -- 172.21.15.39:0/2787226277 >> v1:172.21.15.39:6789/0 conn(0x7f4a1c0c1920 legacy=0x7f4a1c0c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:32.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.829+0000 7f4a1bfff640 1 -- 172.21.15.39:0/2787226277 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c159220 con 0x7f4a1c0c0a40 2024-09-13T17:12:32.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.833+0000 7f4a1bfff640 1 --2- 172.21.15.39:0/2787226277 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c0c0a40 0x7f4a1c0c3700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a080097c0 tx=0x7f4a0801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd5520b18a59bb54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:32.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.833+0000 7f4a1b7fe640 1 -- 172.21.15.39:0/2787226277 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a080ab600 con 0x7f4a1c0c0a40 2024-09-13T17:12:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.833+0000 7f4a1b7fe640 1 -- 172.21.15.39:0/2787226277 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a080a7050 con 0x7f4a1c0c0a40 2024-09-13T17:12:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.833+0000 7f4a23245640 1 -- 172.21.15.39:0/2787226277 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c0c0a40 msgr2=0x7f4a1c0c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.833+0000 7f4a23245640 1 --2- 172.21.15.39:0/2787226277 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c0c0a40 0x7f4a1c0c3700 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a080097c0 tx=0x7f4a0801c450 comp rx=0 tx=0).stop 2024-09-13T17:12:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.833+0000 7f4a23245640 1 -- 172.21.15.39:0/2787226277 shutdown_connections 2024-09-13T17:12:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.833+0000 7f4a23245640 1 --2- 172.21.15.39:0/2787226277 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c0c0a40 0x7f4a1c0c3700 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.833+0000 7f4a23245640 1 --2- 172.21.15.39:0/2787226277 >> v2:172.21.15.39:3300/0 conn(0x7f4a1c155af0 0x7f4a1c0c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.833+0000 7f4a23245640 1 -- 172.21.15.39:0/2787226277 >> 172.21.15.39:0/2787226277 conn(0x7f4a1c0a75b0 msgr2=0x7f4a1c0a79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.837+0000 7f4a23245640 1 -- 172.21.15.39:0/2787226277 shutdown_connections 2024-09-13T17:12:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.837+0000 7f4a23245640 1 -- 172.21.15.39:0/2787226277 wait complete. 2024-09-13T17:12:32.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.841+0000 7f4a23245640 1 Processor -- start 2024-09-13T17:12:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.841+0000 7f4a23245640 1 -- start start 2024-09-13T17:12:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.841+0000 7f4a23245640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a1c0c0a40 0x7f4a1c1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.841+0000 7f4a23245640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1c0c1920 0x7f4a1c0715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.841+0000 7f4a23245640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c1f9a90 0x7f4a1c071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.841+0000 7f4a23245640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0ab7e0 con 0x7f4a1c0c0a40 2024-09-13T17:12:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.841+0000 7f4a23245640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0701f0 con 0x7f4a1c0c1920 2024-09-13T17:12:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.841+0000 7f4a23245640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0a4990 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a217e2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c1f9a90 0x7f4a1c071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:32.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a217e2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c1f9a90 0x7f4a1c071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:58660/0 (socket says 172.21.15.39:58660) 2024-09-13T17:12:32.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a217e2640 1 -- 172.21.15.39:0/3188097154 learned_addr learned my addr 172.21.15.39:0/3188097154 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:12:32.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a20fe1640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a1c0c0a40 0x7f4a1c1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:32.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a217e2640 1 -- 172.21.15.39:0/3188097154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1c0c1920 msgr2=0x7f4a1c0715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:32.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a217e2640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1c0c1920 0x7f4a1c0715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:32.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a217e2640 1 -- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a1c0c0a40 msgr2=0x7f4a1c1549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:32.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a217e2640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a1c0c0a40 0x7f4a1c1549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:32.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a217e2640 1 -- 172.21.15.39:0/3188097154 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a08009420 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a20fe1640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a1c0c0a40 0x7f4a1c1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:12:32.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a217e2640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c1f9a90 0x7f4a1c071ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400ca70 tx=0x7f4a1400cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a197fa640 1 -- 172.21.15.39:0/3188097154 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400abd0 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.845+0000 7f4a197fa640 1 -- 172.21.15.39:0/3188097154 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1400ad30 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.849+0000 7f4a197fa640 1 -- 172.21.15.39:0/3188097154 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400aea0 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.849+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c1f6ce0 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.849+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c0aaf40 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.849+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c0c13e0 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.849+0000 7f4a197fa640 1 -- 172.21.15.39:0/3188097154 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4a14007070 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.849+0000 7f4a197fa640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a000666c0 0x7f4a00068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:12:32.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.849+0000 7f4a197fa640 1 -- 172.21.15.39:0/3188097154 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f4a14084e50 con 0x7f4a1c1f9a90 2024-09-13T17:12:32.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.849+0000 7f4a20fe1640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a000666c0 0x7f4a00068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:12:32.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.853+0000 7f4a20fe1640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a000666c0 0x7f4a00068b80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c06ae10 tx=0x7f4a10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:12:32.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:32.861+0000 7f4a197fa640 1 -- 172.21.15.39:0/3188097154 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4a14051220 con 0x7f4a1c1f9a90 2024-09-13T17:12:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.449+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a1c1f6cc0 con 0x7f4a1c1f9a90 2024-09-13T17:12:33.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.453+0000 7f4a197fa640 1 -- 172.21.15.39:0/3188097154 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f4a14017030 con 0x7f4a1c1f9a90 2024-09-13T17:12:33.475 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:12:33.475 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:12:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 177 2024-09-13T17:12:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a000666c0 msgr2=0x7f4a00068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a000666c0 0x7f4a00068b80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c06ae10 tx=0x7f4a10008040 comp rx=0 tx=0).stop 2024-09-13T17:12:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c1f9a90 msgr2=0x7f4a1c071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:12:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c1f9a90 0x7f4a1c071ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400ca70 tx=0x7f4a1400cf30 comp rx=0 tx=0).stop 2024-09-13T17:12:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 shutdown_connections 2024-09-13T17:12:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a000666c0 0x7f4a00068b80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a1c0c1920 0x7f4a1c0715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:33.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a1c1f9a90 0x7f4a1c071ae0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:33.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 --2- 172.21.15.39:0/3188097154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a1c0c0a40 0x7f4a1c1549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:12:33.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.457+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 >> 172.21.15.39:0/3188097154 conn(0x7f4a1c0a75b0 msgr2=0x7f4a1c162410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:12:33.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.461+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 shutdown_connections 2024-09-13T17:12:33.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:12:33.461+0000 7f4a23245640 1 -- 172.21.15.39:0/3188097154 wait complete. 2024-09-13T17:12:33.526 INFO:tasks.mds_thrash.fs.[b]:mds.d reported in up:standby state 2024-09-13T17:12:33.526 INFO:tasks.mds_thrash.fs.[b]:waiting for 118 secs before thrashing 2024-09-13T17:13:00.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:13:00.922 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:13:31.233 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:13:31.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:13:45.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:13:45.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabd0c4a640 1 Processor -- start 2024-09-13T17:13:45.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabd0c4a640 1 -- start start 2024-09-13T17:13:45.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabd0c4a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fabcc155af0 0x7fabcc0c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabd0c4a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabcc06c1b0 con 0x7fabcc0c0a40 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabd0c4a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabcc159220 con 0x7fabcc0c1920 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabd0c4a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabcc06c500 con 0x7fabcc155af0 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabc9d9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fabcc0c0a40 0x7fabcc0c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:58478/0 (socket says 172.21.15.39:58478) 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabc9d9b640 1 -- 172.21.15.39:0/1545705916 learned_addr learned my addr 172.21.15.39:0/1545705916 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabca59c640 1 --2- 172.21.15.39:0/1545705916 >> v2:172.21.15.84:3300/0 conn(0x7fabcc155af0 0x7fabcc0c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabca59c640 1 -- 172.21.15.39:0/1545705916 >> v1:172.21.15.84:6789/0 conn(0x7fabcc0c1920 legacy=0x7fabcc162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabca59c640 1 -- 172.21.15.39:0/1545705916 >> v1:172.21.15.39:6789/0 conn(0x7fabcc0c0a40 legacy=0x7fabcc0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.883+0000 7fabca59c640 1 -- 172.21.15.39:0/1545705916 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc0c02a0 con 0x7fabcc155af0 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.887+0000 7fabca59c640 1 --2- 172.21.15.39:0/1545705916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabcc155af0 0x7fabcc0c13e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fabc00097c0 tx=0x7fabc001c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1189f70eab4eacb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.887+0000 7fabc959a640 1 -- 172.21.15.39:0/1545705916 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00ab600 con 0x7fabcc155af0 2024-09-13T17:13:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.887+0000 7fabc959a640 1 -- 172.21.15.39:0/1545705916 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc00a7050 con 0x7fabcc155af0 2024-09-13T17:13:45.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.887+0000 7fabc959a640 1 -- 172.21.15.39:0/1545705916 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00ab980 con 0x7fabcc155af0 2024-09-13T17:13:45.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.895+0000 7fabd0c4a640 1 -- 172.21.15.39:0/1545705916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabcc155af0 msgr2=0x7fabcc0c13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:45.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.895+0000 7fabd0c4a640 1 --2- 172.21.15.39:0/1545705916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabcc155af0 0x7fabcc0c13e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fabc00097c0 tx=0x7fabc001c430 comp rx=0 tx=0).stop 2024-09-13T17:13:45.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.895+0000 7fabd0c4a640 1 -- 172.21.15.39:0/1545705916 shutdown_connections 2024-09-13T17:13:45.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.895+0000 7fabd0c4a640 1 --2- 172.21.15.39:0/1545705916 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabcc155af0 0x7fabcc0c13e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:45.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.895+0000 7fabd0c4a640 1 -- 172.21.15.39:0/1545705916 >> 172.21.15.39:0/1545705916 conn(0x7fabcc0a75b0 msgr2=0x7fabcc0a79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:45.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.899+0000 7fabd0c4a640 1 -- 172.21.15.39:0/1545705916 shutdown_connections 2024-09-13T17:13:45.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.903+0000 7fabd0c4a640 1 -- 172.21.15.39:0/1545705916 wait complete. 2024-09-13T17:13:45.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.903+0000 7fabd0c4a640 1 Processor -- start 2024-09-13T17:13:45.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.903+0000 7fabd0c4a640 1 -- start start 2024-09-13T17:13:45.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.903+0000 7fabd0c4a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabcc0c1920 0x7fabcc15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:45.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.903+0000 7fabd0c4a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabcc155af0 0x7fabcc06fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:45.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.903+0000 7fabd0c4a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fabcc15a680 0x7fabcc0719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:45.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabcad9d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fabcc15a680 0x7fabcc0719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:45.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabcad9d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fabcc15a680 0x7fabcc0719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:56464/0 (socket says 172.21.15.39:56464) 2024-09-13T17:13:45.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabd0c4a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc159490 con 0x7fabcc0c1920 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabd0c4a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc159770 con 0x7fabcc155af0 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabd0c4a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc0a5200 con 0x7fabcc15a680 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabcad9d640 1 -- 172.21.15.39:0/358067831 learned_addr learned my addr 172.21.15.39:0/358067831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabcad9d640 1 -- 172.21.15.39:0/358067831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabcc155af0 msgr2=0x7fabcc06fdf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabcad9d640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabcc155af0 0x7fabcc06fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabcad9d640 1 -- 172.21.15.39:0/358067831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabcc0c1920 msgr2=0x7fabcc15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabcad9d640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabcc0c1920 0x7fabcc15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabcad9d640 1 -- 172.21.15.39:0/358067831 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc0009420 con 0x7fabcc15a680 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabcad9d640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fabcc15a680 0x7fabcc0719b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c370 tx=0x7fabbc00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabbaffd640 1 -- 172.21.15.39:0/358067831 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0052d0 con 0x7fabcc15a680 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabcc0706a0 con 0x7fabcc15a680 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabcc155580 con 0x7fabcc15a680 2024-09-13T17:13:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabbaffd640 1 -- 172.21.15.39:0/358067831 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc005430 con 0x7fabcc15a680 2024-09-13T17:13:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.907+0000 7fabbaffd640 1 -- 172.21.15.39:0/358067831 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc003820 con 0x7fabcc15a680 2024-09-13T17:13:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.911+0000 7fabbaffd640 1 -- 172.21.15.39:0/358067831 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fabbc003980 con 0x7fabcc15a680 2024-09-13T17:13:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.911+0000 7fabbaffd640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabac0666c0 0x7fabac068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.911+0000 7fabbaffd640 1 -- 172.21.15.39:0/358067831 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7fabbc084430 con 0x7fabcc15a680 2024-09-13T17:13:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.911+0000 7fabca59c640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabac0666c0 0x7fabac068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.911+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb0005260 con 0x7fabcc15a680 2024-09-13T17:13:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.911+0000 7fabca59c640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabac0666c0 0x7fabac068b80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0c0030 tx=0x7fabc00ad040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:45.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:45.935+0000 7fabbaffd640 1 -- 172.21.15.39:0/358067831 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fabbc017080 con 0x7fabcc15a680 2024-09-13T17:13:46.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.647+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabb0005a90 con 0x7fabcc15a680 2024-09-13T17:13:46.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.647+0000 7fabbaffd640 1 -- 172.21.15.39:0/358067831 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5931 (secure 0 0 0) 0x7fabbc081e60 con 0x7fabcc15a680 2024-09-13T17:13:46.648 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:13:46.649 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:13:46.649 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 177 2024-09-13T17:13:46.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabac0666c0 msgr2=0x7fabac068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:46.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabac0666c0 0x7fabac068b80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0c0030 tx=0x7fabc00ad040 comp rx=0 tx=0).stop 2024-09-13T17:13:46.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fabcc15a680 msgr2=0x7fabcc0719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:46.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fabcc15a680 0x7fabcc0719b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00c370 tx=0x7fabbc00c830 comp rx=0 tx=0).stop 2024-09-13T17:13:46.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 shutdown_connections 2024-09-13T17:13:46.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fabac0666c0 0x7fabac068b80 secure :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0c0030 tx=0x7fabc00ad040 comp rx=0 tx=0).stop 2024-09-13T17:13:46.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabcc155af0 0x7fabcc06fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:46.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fabcc15a680 0x7fabcc0719b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:46.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 --2- 172.21.15.39:0/358067831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabcc0c1920 0x7fabcc15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:46.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.659+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 >> 172.21.15.39:0/358067831 conn(0x7fabcc0a75b0 msgr2=0x7fabcc146ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:46.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.663+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 shutdown_connections 2024-09-13T17:13:46.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:46.663+0000 7fabd0c4a640 1 -- 172.21.15.39:0/358067831 wait complete. 2024-09-13T17:13:46.843 INFO:tasks.mds_thrash.fs.[a]:kill mds.c (rank=0) 2024-09-13T17:13:46.843 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:13:46.843 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:13:46.862 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:13:46.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:13:47.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.355+0000 7f40265f3640 1 Processor -- start 2024-09-13T17:13:47.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.355+0000 7f40265f3640 1 -- start start 2024-09-13T17:13:47.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.355+0000 7f40265f3640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f40180d19f0 0x7f4018047e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:47.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.355+0000 7f40265f3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4018048390 0x7f401802f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:47.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.355+0000 7f40265f3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401802fc00 con 0x7f4018040b60 2024-09-13T17:13:47.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.355+0000 7f40265f3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4018049190 con 0x7f4018048390 2024-09-13T17:13:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.355+0000 7f40265f3640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40180d1dd0 con 0x7f40180d19f0 2024-09-13T17:13:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.355+0000 7f401f7fe640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4018040b60 0x7f401802efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:50940/0 (socket says 172.21.15.39:50940) 2024-09-13T17:13:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.355+0000 7f401f7fe640 1 -- 172.21.15.39:0/3490603364 learned_addr learned my addr 172.21.15.39:0/3490603364 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:13:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.359+0000 7f401ffff640 1 --2- 172.21.15.39:0/3490603364 >> v2:172.21.15.84:3300/0 conn(0x7f40180d19f0 0x7f4018047e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.359+0000 7f401ffff640 1 -- 172.21.15.39:0/3490603364 >> v2:172.21.15.39:3300/0 conn(0x7f4018048390 msgr2=0x7f401802f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:13:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.359+0000 7f401ffff640 1 --2- 172.21.15.39:0/3490603364 >> v2:172.21.15.39:3300/0 conn(0x7f4018048390 0x7f401802f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.359+0000 7f401ffff640 1 -- 172.21.15.39:0/3490603364 >> v1:172.21.15.84:6789/0 conn(0x7f4018040b60 legacy=0x7f401802efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.359+0000 7f401ffff640 1 -- 172.21.15.39:0/3490603364 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40180daf00 con 0x7f40180d19f0 2024-09-13T17:13:47.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f401ffff640 1 --2- 172.21.15.39:0/3490603364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40180d19f0 0x7f4018047e50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f40180d5750 tx=0x7f400c024950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cfdbff91a984524 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:47.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f401effd640 1 -- 172.21.15.39:0/3490603364 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c0ad7a0 con 0x7f40180d19f0 2024-09-13T17:13:47.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f401effd640 1 -- 172.21.15.39:0/3490603364 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c0ad900 con 0x7f40180d19f0 2024-09-13T17:13:47.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f40265f3640 1 -- 172.21.15.39:0/3490603364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40180d19f0 msgr2=0x7f4018047e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:47.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f40265f3640 1 --2- 172.21.15.39:0/3490603364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40180d19f0 0x7f4018047e50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f40180d5750 tx=0x7f400c024950 comp rx=0 tx=0).stop 2024-09-13T17:13:47.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f40265f3640 1 -- 172.21.15.39:0/3490603364 shutdown_connections 2024-09-13T17:13:47.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f40265f3640 1 --2- 172.21.15.39:0/3490603364 >> v2:172.21.15.39:3300/0 conn(0x7f4018048390 0x7f401802f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:47.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f40265f3640 1 --2- 172.21.15.39:0/3490603364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40180d19f0 0x7f4018047e50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:47.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f40265f3640 1 -- 172.21.15.39:0/3490603364 >> 172.21.15.39:0/3490603364 conn(0x7f401804d4d0 msgr2=0x7f401804d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f40265f3640 1 -- 172.21.15.39:0/3490603364 shutdown_connections 2024-09-13T17:13:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.363+0000 7f40265f3640 1 -- 172.21.15.39:0/3490603364 wait complete. 2024-09-13T17:13:47.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f40265f3640 1 Processor -- start 2024-09-13T17:13:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f40265f3640 1 -- start start 2024-09-13T17:13:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f40265f3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4018040b60 0x7f40180d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f40265f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4018048390 0x7f40180da3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f40265f3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f40180d19f0 0x7f40180cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f40265f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401816e1d0 con 0x7f4018048390 2024-09-13T17:13:47.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f40265f3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40180d3250 con 0x7f4018040b60 2024-09-13T17:13:47.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f40265f3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40181717c0 con 0x7f40180d19f0 2024-09-13T17:13:47.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f401ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4018040b60 0x7f40180d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:47.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f401ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4018040b60 0x7f40180d2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:60888/0 (socket says 172.21.15.39:60888) 2024-09-13T17:13:47.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f401ffff640 1 -- 172.21.15.39:0/3147017611 learned_addr learned my addr 172.21.15.39:0/3147017611 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:13:47.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f401f7fe640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4018048390 0x7f40180da3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:47.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f401ffff640 1 -- 172.21.15.39:0/3147017611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f40180d19f0 msgr2=0x7f40180cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:47.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.367+0000 7f401ffff640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f40180d19f0 0x7f40180cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:47.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.371+0000 7f401ffff640 1 -- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4018048390 msgr2=0x7f40180da3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:47.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.371+0000 7f401ffff640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4018048390 0x7f40180da3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:47.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.371+0000 7f401ffff640 1 -- 172.21.15.39:0/3147017611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c00f610 con 0x7f4018040b60 2024-09-13T17:13:47.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.371+0000 7f401f7fe640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4018048390 0x7f40180da3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:13:47.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.375+0000 7f401ffff640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4018040b60 0x7f40180d2d10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f400c021560 tx=0x7f400c0216f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:47.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.375+0000 7f401cff9640 1 -- 172.21.15.39:0/3147017611 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c0b6e10 con 0x7f4018040b60 2024-09-13T17:13:47.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.375+0000 7f401cff9640 1 -- 172.21.15.39:0/3147017611 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c0be410 con 0x7f4018040b60 2024-09-13T17:13:47.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.375+0000 7f401cff9640 1 -- 172.21.15.39:0/3147017611 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c0be7c0 con 0x7f4018040b60 2024-09-13T17:13:47.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.375+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4018171aa0 con 0x7f4018040b60 2024-09-13T17:13:47.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.375+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40180da8f0 con 0x7f4018040b60 2024-09-13T17:13:47.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.375+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4018171620 con 0x7f4018040b60 2024-09-13T17:13:47.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.379+0000 7f401cff9640 1 -- 172.21.15.39:0/3147017611 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f400c0bc040 con 0x7f4018040b60 2024-09-13T17:13:47.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.379+0000 7f401cff9640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4004066690 0x7f4004068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:47.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.379+0000 7f401cff9640 1 -- 172.21.15.39:0/3147017611 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f400c127c90 con 0x7f4018040b60 2024-09-13T17:13:47.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.379+0000 7f401f7fe640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4004066690 0x7f4004068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:47.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.383+0000 7f401f7fe640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4004066690 0x7f4004068b50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4018033440 tx=0x7f4014009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:47.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:47.391+0000 7f401cff9640 1 -- 172.21.15.39:0/3147017611 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4018171620 con 0x7f4018040b60 2024-09-13T17:13:48.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.211+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4018171fc0 con 0x7f4018040b60 2024-09-13T17:13:48.214 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:13:48.214 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:13:48.215 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 177 2024-09-13T17:13:48.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.211+0000 7f401cff9640 1 -- 172.21.15.39:0/3147017611 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f4018171fc0 con 0x7f4018040b60 2024-09-13T17:13:48.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4004066690 msgr2=0x7f4004068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:48.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4004066690 0x7f4004068b50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4018033440 tx=0x7f4014009040 comp rx=0 tx=0).stop 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4018040b60 msgr2=0x7f40180d2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4018040b60 0x7f40180d2d10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f400c021560 tx=0x7f400c0216f0 comp rx=0 tx=0).stop 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 shutdown_connections 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4004066690 0x7f4004068b50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f40180d19f0 0x7f40180cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4018048390 0x7f40180da3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 --2- 172.21.15.39:0/3147017611 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4018040b60 0x7f40180d2d10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 >> 172.21.15.39:0/3147017611 conn(0x7f401804d4d0 msgr2=0x7f40180d5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 shutdown_connections 2024-09-13T17:13:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:48.215+0000 7f40265f3640 1 -- 172.21.15.39:0/3147017611 wait complete. 2024-09-13T17:13:48.264 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:13:50.266 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:13:50.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.591+0000 7f09acd59640 1 Processor -- start 2024-09-13T17:13:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.591+0000 7f09acd59640 1 -- start start 2024-09-13T17:13:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09acd59640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a806e4b0 0x7f09a8158e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09acd59640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f09a80c53d0 0x7f09a815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09acd59640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09a815be20 0x7f09a806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09acd59640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a81589a0 con 0x7f09a80c53d0 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09acd59640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a8158370 con 0x7f09a815be20 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09acd59640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a815c330 con 0x7f09a806e4b0 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a5d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f09a80c53d0 0x7f09a815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a659c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a806e4b0 0x7f09a8158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a659c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a806e4b0 0x7f09a8158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:55166/0 (socket says 172.21.15.39:55166) 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a5d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f09a80c53d0 0x7f09a815b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54448/0 (socket says 172.21.15.39:54448) 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a659c640 1 -- 172.21.15.39:0/3782477867 learned_addr learned my addr 172.21.15.39:0/3782477867 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:13:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a6d9d640 1 --2- 172.21.15.39:0/3782477867 >> v2:172.21.15.84:3300/0 conn(0x7f09a815be20 0x7f09a806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:50.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a659c640 1 -- 172.21.15.39:0/3782477867 >> v2:172.21.15.84:3300/0 conn(0x7f09a815be20 msgr2=0x7f09a806ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:13:50.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a659c640 1 --2- 172.21.15.39:0/3782477867 >> v2:172.21.15.84:3300/0 conn(0x7f09a815be20 0x7f09a806ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:50.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a659c640 1 -- 172.21.15.39:0/3782477867 >> v2:172.21.15.39:3300/0 conn(0x7f09a80c53d0 msgr2=0x7f09a815b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:13:50.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a659c640 1 --2- 172.21.15.39:0/3782477867 >> v2:172.21.15.39:3300/0 conn(0x7f09a80c53d0 0x7f09a815b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:50.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.595+0000 7f09a659c640 1 -- 172.21.15.39:0/3782477867 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a815c610 con 0x7f09a806e4b0 2024-09-13T17:13:50.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.599+0000 7f09a659c640 1 --2- 172.21.15.39:0/3782477867 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a806e4b0 0x7f09a8158e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a80a9470 tx=0x7f099801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b84b85cdeb74cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:50.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.599+0000 7f09a559a640 1 -- 172.21.15.39:0/3782477867 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980ab600 con 0x7f09a806e4b0 2024-09-13T17:13:50.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.599+0000 7f09a559a640 1 -- 172.21.15.39:0/3782477867 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09980a7050 con 0x7f09a806e4b0 2024-09-13T17:13:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.599+0000 7f09a559a640 1 -- 172.21.15.39:0/3782477867 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980ab9c0 con 0x7f09a806e4b0 2024-09-13T17:13:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.599+0000 7f09acd59640 1 -- 172.21.15.39:0/3782477867 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a806e4b0 msgr2=0x7f09a8158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.599+0000 7f09acd59640 1 --2- 172.21.15.39:0/3782477867 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a806e4b0 0x7f09a8158e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a80a9470 tx=0x7f099801c4a0 comp rx=0 tx=0).stop 2024-09-13T17:13:50.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.603+0000 7f09acd59640 1 -- 172.21.15.39:0/3782477867 shutdown_connections 2024-09-13T17:13:50.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.603+0000 7f09acd59640 1 --2- 172.21.15.39:0/3782477867 >> v2:172.21.15.84:3300/0 conn(0x7f09a815be20 0x7f09a806ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:50.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.603+0000 7f09acd59640 1 --2- 172.21.15.39:0/3782477867 >> v2:172.21.15.39:3300/0 conn(0x7f09a80c53d0 0x7f09a815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:50.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.603+0000 7f09acd59640 1 --2- 172.21.15.39:0/3782477867 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a806e4b0 0x7f09a8158e30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:50.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.603+0000 7f09acd59640 1 -- 172.21.15.39:0/3782477867 >> 172.21.15.39:0/3782477867 conn(0x7f09a80c32a0 msgr2=0x7f09a80c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:50.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.607+0000 7f09acd59640 1 -- 172.21.15.39:0/3782477867 shutdown_connections 2024-09-13T17:13:50.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.607+0000 7f09acd59640 1 -- 172.21.15.39:0/3782477867 wait complete. 2024-09-13T17:13:50.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.607+0000 7f09acd59640 1 Processor -- start 2024-09-13T17:13:50.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.607+0000 7f09acd59640 1 -- start start 2024-09-13T17:13:50.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09acd59640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a806e4b0 0x7f09a8156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:50.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09acd59640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a80c53d0 0x7f09a81f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:50.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09acd59640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a815be20 0x7f09a81f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:50.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09acd59640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a814aff0 con 0x7f09a80c53d0 2024-09-13T17:13:50.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09acd59640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80a7e30 con 0x7f09a806e4b0 2024-09-13T17:13:50.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09acd59640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80a8110 con 0x7f09a815be20 2024-09-13T17:13:50.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a80c53d0 0x7f09a81f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:50.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a80c53d0 0x7f09a81f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54450/0 (socket says 172.21.15.39:54450) 2024-09-13T17:13:50.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a5d9b640 1 -- 172.21.15.39:0/2792656215 learned_addr learned my addr 172.21.15.39:0/2792656215 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:13:50.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a659c640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a806e4b0 0x7f09a8156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:50.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a659c640 1 -- 172.21.15.39:0/2792656215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a815be20 msgr2=0x7f09a81f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:50.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a659c640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a815be20 0x7f09a81f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:50.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a659c640 1 -- 172.21.15.39:0/2792656215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a80c53d0 msgr2=0x7f09a81f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:50.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a659c640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a80c53d0 0x7f09a81f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:50.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a659c640 1 -- 172.21.15.39:0/2792656215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09980094d0 con 0x7f09a806e4b0 2024-09-13T17:13:50.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f09a659c640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a806e4b0 0x7f09a8156b00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f09a815c470 tx=0x7f0998019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:50.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f0996ffd640 1 -- 172.21.15.39:0/2792656215 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980b5400 con 0x7f09a806e4b0 2024-09-13T17:13:50.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.611+0000 7f0996ffd640 1 -- 172.21.15.39:0/2792656215 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09980b5580 con 0x7f09a806e4b0 2024-09-13T17:13:50.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.615+0000 7f0996ffd640 1 -- 172.21.15.39:0/2792656215 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980b5930 con 0x7f09a806e4b0 2024-09-13T17:13:50.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.615+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a81502a0 con 0x7f09a806e4b0 2024-09-13T17:13:50.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.615+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a8157040 con 0x7f09a806e4b0 2024-09-13T17:13:50.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.615+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a8150100 con 0x7f09a806e4b0 2024-09-13T17:13:50.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.615+0000 7f0996ffd640 1 -- 172.21.15.39:0/2792656215 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09980b5a90 con 0x7f09a806e4b0 2024-09-13T17:13:50.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.615+0000 7f0996ffd640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:50.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.619+0000 7f09a5d9b640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:50.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.619+0000 7f09a5d9b640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f09a80c1780 tx=0x7f099c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:50.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.619+0000 7f0996ffd640 1 -- 172.21.15.39:0/2792656215 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f09980f0640 con 0x7f09a806e4b0 2024-09-13T17:13:50.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:50.623+0000 7f0996ffd640 1 -- 172.21.15.39:0/2792656215 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09980ec280 con 0x7f09a806e4b0 2024-09-13T17:13:51.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.175+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09a81507c0 con 0x7f09a806e4b0 2024-09-13T17:13:51.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.179+0000 7f0996ffd640 1 -- 172.21.15.39:0/2792656215 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f09a81507c0 con 0x7f09a806e4b0 2024-09-13T17:13:51.179 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:13:51.179 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:13:51.179 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 177 2024-09-13T17:13:51.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.183+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 msgr2=0x7f0988068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:51.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.183+0000 7f09acd59640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f09a80c1780 tx=0x7f099c008040 comp rx=0 tx=0).stop 2024-09-13T17:13:51.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.183+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a806e4b0 msgr2=0x7f09a8156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:51.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.183+0000 7f09acd59640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a806e4b0 0x7f09a8156b00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f09a815c470 tx=0x7f0998019240 comp rx=0 tx=0).stop 2024-09-13T17:13:51.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.183+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 shutdown_connections 2024-09-13T17:13:51.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.183+0000 7f09acd59640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:51.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.187+0000 7f09acd59640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a815be20 0x7f09a81f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:51.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.187+0000 7f09acd59640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a80c53d0 0x7f09a81f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:51.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.187+0000 7f09acd59640 1 --2- 172.21.15.39:0/2792656215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a806e4b0 0x7f09a8156b00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:51.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.187+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 >> 172.21.15.39:0/2792656215 conn(0x7f09a80c32a0 msgr2=0x7f09a80c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:51.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.187+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 shutdown_connections 2024-09-13T17:13:51.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:51.187+0000 7f09acd59640 1 -- 172.21.15.39:0/2792656215 wait complete. 2024-09-13T17:13:51.293 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:13:53.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:13:53.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f46414b0640 1 Processor -- start 2024-09-13T17:13:53.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f46414b0640 1 -- start start 2024-09-13T17:13:53.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f46414b0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f463c0c7470 0x7f463c151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:53.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f46414b0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c14f590 con 0x7f463c06e5e0 2024-09-13T17:13:53.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f46414b0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463c06eae0 con 0x7f463c14f1b0 2024-09-13T17:13:53.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f46414b0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463c06f660 con 0x7f463c0c7470 2024-09-13T17:13:53.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f463b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f463c0c7470 0x7f463c151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:53.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f4633fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f463c06e5e0 0x7f463c0c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54340/0 (socket says 172.21.15.39:54340) 2024-09-13T17:13:53.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f4633fff640 1 -- 172.21.15.39:0/3065720113 learned_addr learned my addr 172.21.15.39:0/3065720113 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:13:53.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f463a7fc640 1 -- 172.21.15.39:0/3065720113 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689154629 0 0) 0x7f463c14f590 con 0x7f463c06e5e0 2024-09-13T17:13:53.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f463a7fc640 1 -- 172.21.15.39:0/3065720113 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4628003530 con 0x7f463c06e5e0 2024-09-13T17:13:53.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f463b7fe640 1 -- 172.21.15.39:0/3065720113 >> v1:172.21.15.84:6789/0 conn(0x7f463c14f1b0 legacy=0x7f463c0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:53.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f463b7fe640 1 -- 172.21.15.39:0/3065720113 >> v1:172.21.15.39:6789/0 conn(0x7f463c06e5e0 legacy=0x7f463c0c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:53.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f463a7fc640 1 -- 172.21.15.39:0/3065720113 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413920546 0 0) 0x7f463c06eae0 con 0x7f463c14f1b0 2024-09-13T17:13:53.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.815+0000 7f463b7fe640 1 -- 172.21.15.39:0/3065720113 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c06edc0 con 0x7f463c0c7470 2024-09-13T17:13:53.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.819+0000 7f463b7fe640 1 --2- 172.21.15.39:0/3065720113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c0c7470 0x7f463c151bb0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f4634009d90 tx=0x7f463401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8387d17686290b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:53.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.819+0000 7f463a7fc640 1 -- 172.21.15.39:0/3065720113 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46340a5a00 con 0x7f463c0c7470 2024-09-13T17:13:53.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.819+0000 7f463a7fc640 1 -- 172.21.15.39:0/3065720113 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46340a5b60 con 0x7f463c0c7470 2024-09-13T17:13:53.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.819+0000 7f463a7fc640 1 -- 172.21.15.39:0/3065720113 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46340ad440 con 0x7f463c0c7470 2024-09-13T17:13:53.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.823+0000 7f46414b0640 1 -- 172.21.15.39:0/3065720113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c0c7470 msgr2=0x7f463c151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:53.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.823+0000 7f46414b0640 1 --2- 172.21.15.39:0/3065720113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c0c7470 0x7f463c151bb0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f4634009d90 tx=0x7f463401c940 comp rx=0 tx=0).stop 2024-09-13T17:13:53.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.823+0000 7f46414b0640 1 -- 172.21.15.39:0/3065720113 shutdown_connections 2024-09-13T17:13:53.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.823+0000 7f46414b0640 1 --2- 172.21.15.39:0/3065720113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c0c7470 0x7f463c151bb0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:53.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.823+0000 7f46414b0640 1 -- 172.21.15.39:0/3065720113 >> 172.21.15.39:0/3065720113 conn(0x7f463c0a9720 msgr2=0x7f463c0a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:53.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.823+0000 7f46414b0640 1 -- 172.21.15.39:0/3065720113 shutdown_connections 2024-09-13T17:13:53.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.823+0000 7f46414b0640 1 -- 172.21.15.39:0/3065720113 wait complete. 2024-09-13T17:13:53.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.823+0000 7f46414b0640 1 Processor -- start 2024-09-13T17:13:53.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f46414b0640 1 -- start start 2024-09-13T17:13:53.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f46414b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463c06e5e0 0x7f463c141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:53.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f46414b0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f463c0c7470 0x7f463c14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f46414b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c14f1b0 0x7f463c1eb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f46414b0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c141bb0 con 0x7f463c14f1b0 2024-09-13T17:13:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f46414b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c1ea9e0 con 0x7f463c06e5e0 2024-09-13T17:13:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f46414b0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463c1ea110 con 0x7f463c0c7470 2024-09-13T17:13:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f463affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463c06e5e0 0x7f463c141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f463affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463c06e5e0 0x7f463c141670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:41336/0 (socket says 172.21.15.39:41336) 2024-09-13T17:13:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.827+0000 7f463affd640 1 -- 172.21.15.39:0/623682285 learned_addr learned my addr 172.21.15.39:0/623682285 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:13:53.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f463affd640 1 -- 172.21.15.39:0/623682285 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f463c0c7470 msgr2=0x7f463c14db60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:13:53.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f463affd640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f463c0c7470 0x7f463c14db60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:53.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f463affd640 1 -- 172.21.15.39:0/623682285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c14f1b0 msgr2=0x7f463c1eb580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:13:53.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f463affd640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c14f1b0 0x7f463c1eb580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:53.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f463affd640 1 -- 172.21.15.39:0/623682285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634009a70 con 0x7f463c06e5e0 2024-09-13T17:13:53.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f463affd640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463c06e5e0 0x7f463c141670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c470 tx=0x7f462c00c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f46337fe640 1 -- 172.21.15.39:0/623682285 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c005480 con 0x7f463c06e5e0 2024-09-13T17:13:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f46337fe640 1 -- 172.21.15.39:0/623682285 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c003600 con 0x7f463c06e5e0 2024-09-13T17:13:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f46337fe640 1 -- 172.21.15.39:0/623682285 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c0039b0 con 0x7f463c06e5e0 2024-09-13T17:13:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c1ee3e0 con 0x7f463c06e5e0 2024-09-13T17:13:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c1ea670 con 0x7f463c06e5e0 2024-09-13T17:13:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.831+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4620005260 con 0x7f463c06e5e0 2024-09-13T17:13:53.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.835+0000 7f46337fe640 1 -- 172.21.15.39:0/623682285 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f462c003b10 con 0x7f463c06e5e0 2024-09-13T17:13:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.835+0000 7f46337fe640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f461c0666e0 0x7f461c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.835+0000 7f46337fe640 1 -- 172.21.15.39:0/623682285 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f462c0849d0 con 0x7f463c06e5e0 2024-09-13T17:13:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.835+0000 7f4633fff640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f461c0666e0 0x7f461c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:53.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.835+0000 7f4633fff640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f461c0666e0 0x7f461c068ba0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4624007fd0 tx=0x7f4624007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:53.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:53.843+0000 7f46337fe640 1 -- 172.21.15.39:0/623682285 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f462c050eb0 con 0x7f463c06e5e0 2024-09-13T17:13:54.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.139+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4620005a90 con 0x7f463c06e5e0 2024-09-13T17:13:54.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.139+0000 7f46337fe640 1 -- 172.21.15.39:0/623682285 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f462c0544d0 con 0x7f463c06e5e0 2024-09-13T17:13:54.140 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:13:54.140 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:13:54.140 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 177 2024-09-13T17:13:54.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f461c0666e0 msgr2=0x7f461c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:54.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f461c0666e0 0x7f461c068ba0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4624007fd0 tx=0x7f4624007900 comp rx=0 tx=0).stop 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463c06e5e0 msgr2=0x7f463c141670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463c06e5e0 0x7f463c141670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c470 tx=0x7f462c00c930 comp rx=0 tx=0).stop 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 shutdown_connections 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f461c0666e0 0x7f461c068ba0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f463c14f1b0 0x7f463c1eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f463c0c7470 0x7f463c14db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 --2- 172.21.15.39:0/623682285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f463c06e5e0 0x7f463c141670 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 >> 172.21.15.39:0/623682285 conn(0x7f463c0a9720 msgr2=0x7f463c14bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 shutdown_connections 2024-09-13T17:13:54.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:54.143+0000 7f46414b0640 1 -- 172.21.15.39:0/623682285 wait complete. 2024-09-13T17:13:54.164 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:13:56.165 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:13:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84f691640 1 Processor -- start 2024-09-13T17:13:56.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84f691640 1 -- start start 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84f691640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd84806e5e0 0x7fd848151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84f691640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84814f590 con 0x7fd84814f1b0 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84f691640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84806eae0 con 0x7fd8480c7470 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84f691640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd84806f660 con 0x7fd84806e5e0 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84d42d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd84814f1b0 0x7fd8480714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54350/0 (socket says 172.21.15.39:54350) 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84d42d640 1 -- 172.21.15.39:0/1745317717 learned_addr learned my addr 172.21.15.39:0/1745317717 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84cc2c640 1 --2- 172.21.15.39:0/1745317717 >> v2:172.21.15.84:3300/0 conn(0x7fd84806e5e0 0x7fd848151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102266450 0 0) 0x7fd84814f590 con 0x7fd84814f1b0 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd834003530 con 0x7fd84814f1b0 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823619026 0 0) 0x7fd84806eae0 con 0x7fd8480c7470 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd84814f590 con 0x7fd8480c7470 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1102383636 0 0) 0x7fd834003530 con 0x7fd84814f1b0 2024-09-13T17:13:56.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 >> v2:172.21.15.84:3300/0 conn(0x7fd84806e5e0 msgr2=0x7fd848151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 --2- 172.21.15.39:0/1745317717 >> v2:172.21.15.84:3300/0 conn(0x7fd84806e5e0 0x7fd848151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 >> v1:172.21.15.84:6789/0 conn(0x7fd8480c7470 legacy=0x7fd8480c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84806edc0 con 0x7fd84814f1b0 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fd844001680 con 0x7fd84814f1b0 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 >> v1:172.21.15.39:6789/0 conn(0x7fd84814f1b0 legacy=0x7fd8480714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd834003b10 0x7fd834003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8340047d0 0x7fd834004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340052b0 0x7fd83400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84806f660 con 0x7fd8340047d0 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83400c1b0 con 0x7fd834003b10 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83400c370 con 0x7fd8340052b0 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84cc2c640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8340047d0 0x7fd834004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84d42d640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd834003b10 0x7fd834003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84dc2e640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340052b0 0x7fd83400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84cc2c640 1 -- 172.21.15.39:0/1745317717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340052b0 msgr2=0x7fd83400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:13:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84cc2c640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340052b0 0x7fd83400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84cc2c640 1 -- 172.21.15.39:0/1745317717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd834003b10 msgr2=0x7fd834003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:13:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84cc2c640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd834003b10 0x7fd834003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84cc2c640 1 -- 172.21.15.39:0/1745317717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834003530 con 0x7fd8340047d0 2024-09-13T17:13:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84dc2e640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340052b0 0x7fd83400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:13:56.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.594+0000 7fd84dc2e640 1 -- 172.21.15.39:0/1745317717 reap_dead start 2024-09-13T17:13:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84cc2c640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8340047d0 0x7fd834004c50 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fd838006fd0 tx=0x7fd83801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714781ebd6290ea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8380ab660 con 0x7fd8340047d0 2024-09-13T17:13:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8380ab7c0 con 0x7fd8340047d0 2024-09-13T17:13:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd83ffff640 1 -- 172.21.15.39:0/1745317717 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8380abb70 con 0x7fd8340047d0 2024-09-13T17:13:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84f691640 1 -- 172.21.15.39:0/1745317717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8340047d0 msgr2=0x7fd834004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84f691640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8340047d0 0x7fd834004c50 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fd838006fd0 tx=0x7fd83801c510 comp rx=0 tx=0).stop 2024-09-13T17:13:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84f691640 1 -- 172.21.15.39:0/1745317717 shutdown_connections 2024-09-13T17:13:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84f691640 1 --2- 172.21.15.39:0/1745317717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8340047d0 0x7fd834004c50 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84f691640 1 -- 172.21.15.39:0/1745317717 >> 172.21.15.39:0/1745317717 conn(0x7fd8480a9720 msgr2=0x7fd8480a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84f691640 1 -- 172.21.15.39:0/1745317717 shutdown_connections 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84f691640 1 -- 172.21.15.39:0/1745317717 wait complete. 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84f691640 1 Processor -- start 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.598+0000 7fd84f691640 1 -- start start 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84f691640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340047d0 0x7fd84814aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84f691640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd84814c9e0 0x7fd84814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84f691640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd848149f00 0x7fd8481439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84f691640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8481487f0 con 0x7fd848149f00 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84f691640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84814af40 con 0x7fd84814c9e0 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84f691640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84814e200 con 0x7fd8340047d0 2024-09-13T17:13:56.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84dc2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd848149f00 0x7fd8481439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84d42d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340047d0 0x7fd84814aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84dc2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd848149f00 0x7fd8481439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54486/0 (socket says 172.21.15.39:54486) 2024-09-13T17:13:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84dc2e640 1 -- 172.21.15.39:0/1282579387 learned_addr learned my addr 172.21.15.39:0/1282579387 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84dc2e640 1 -- 172.21.15.39:0/1282579387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340047d0 msgr2=0x7fd84814aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84dc2e640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340047d0 0x7fd84814aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84dc2e640 1 -- 172.21.15.39:0/1282579387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd84814c9e0 msgr2=0x7fd84814b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84dc2e640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd84814c9e0 0x7fd84814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84dc2e640 1 -- 172.21.15.39:0/1282579387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8481e5ae0 con 0x7fd848149f00 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84dc2e640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd848149f00 0x7fd8481439c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd8480c71a0 tx=0x7fd84000f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd83dffb640 1 -- 172.21.15.39:0/1282579387 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000fd40 con 0x7fd848149f00 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84814e4a0 con 0x7fd848149f00 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd848148430 con 0x7fd848149f00 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd83dffb640 1 -- 172.21.15.39:0/1282579387 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84000fea0 con 0x7fd848149f00 2024-09-13T17:13:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd83dffb640 1 -- 172.21.15.39:0/1282579387 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840015650 con 0x7fd848149f00 2024-09-13T17:13:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.602+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd828005260 con 0x7fd848149f00 2024-09-13T17:13:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.606+0000 7fd83dffb640 1 -- 172.21.15.39:0/1282579387 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd8400157b0 con 0x7fd848149f00 2024-09-13T17:13:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.606+0000 7fd83dffb640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd82c066690 0x7fd82c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.606+0000 7fd83dffb640 1 -- 172.21.15.39:0/1282579387 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7fd840088f90 con 0x7fd848149f00 2024-09-13T17:13:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.610+0000 7fd84d42d640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd82c066690 0x7fd82c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.610+0000 7fd83dffb640 1 -- 172.21.15.39:0/1282579387 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd8400553c0 con 0x7fd848149f00 2024-09-13T17:13:56.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.610+0000 7fd84d42d640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd82c066690 0x7fd82c068b50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd844001ec0 tx=0x7fd844001e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.890+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8280057e0 con 0x7fd848149f00 2024-09-13T17:13:56.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.890+0000 7fd83dffb640 1 -- 172.21.15.39:0/1282579387 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5931 (secure 0 0 0) 0x7fd8400589e0 con 0x7fd848149f00 2024-09-13T17:13:56.893 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:13:56.893 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:13:56.893 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 177 2024-09-13T17:13:56.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd82c066690 msgr2=0x7fd82c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:56.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd82c066690 0x7fd82c068b50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd844001ec0 tx=0x7fd844001e30 comp rx=0 tx=0).stop 2024-09-13T17:13:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd848149f00 msgr2=0x7fd8481439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd848149f00 0x7fd8481439c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd8480c71a0 tx=0x7fd84000f540 comp rx=0 tx=0).stop 2024-09-13T17:13:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 shutdown_connections 2024-09-13T17:13:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd82c066690 0x7fd82c068b50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd84814c9e0 0x7fd84814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd848149f00 0x7fd8481439c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 --2- 172.21.15.39:0/1282579387 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd8340047d0 0x7fd84814aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:56.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 >> 172.21.15.39:0/1282579387 conn(0x7fd8480a9720 msgr2=0x7fd848149540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:56.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 shutdown_connections 2024-09-13T17:13:56.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:56.894+0000 7fd84f691640 1 -- 172.21.15.39:0/1282579387 wait complete. 2024-09-13T17:13:56.918 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:13:58.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:13:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.094+0000 7f4bc4ecf640 1 Processor -- start 2024-09-13T17:13:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.094+0000 7f4bc4ecf640 1 -- start start 2024-09-13T17:13:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bc4ecf640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4bc014acf0 0x7f4bc0151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bc4ecf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4bc00c4ba0 0x7f4bc0153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bc4ecf640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc0154ab0 con 0x7f4bc00c4710 2024-09-13T17:13:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bc4ecf640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc0154d90 con 0x7f4bc00c4ba0 2024-09-13T17:13:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bc4ecf640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc0150d50 con 0x7f4bc014acf0 2024-09-13T17:13:59.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bbed9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4bc00c4ba0 0x7f4bc0153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bbed9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4bc00c4ba0 0x7f4bc0153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54496/0 (socket says 172.21.15.39:54496) 2024-09-13T17:13:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bbed9d640 1 -- 172.21.15.39:0/2620331156 learned_addr learned my addr 172.21.15.39:0/2620331156 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:13:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bbd59a640 1 -- 172.21.15.39:0/2620331156 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755266079 0 0) 0x7f4bc0154ab0 con 0x7f4bc00c4710 2024-09-13T17:13:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bbed9d640 1 -- 172.21.15.39:0/2620331156 >> v2:172.21.15.84:3300/0 conn(0x7f4bc014acf0 msgr2=0x7f4bc0151550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:13:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bbed9d640 1 --2- 172.21.15.39:0/2620331156 >> v2:172.21.15.84:3300/0 conn(0x7f4bc014acf0 0x7f4bc0151550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bbed9d640 1 -- 172.21.15.39:0/2620331156 >> v1:172.21.15.84:6789/0 conn(0x7f4bc00c4710 legacy=0x7f4bc006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.098+0000 7f4bbed9d640 1 -- 172.21.15.39:0/2620331156 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc0151030 con 0x7f4bc00c4ba0 2024-09-13T17:13:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bbed9d640 1 --2- 172.21.15.39:0/2620331156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4ba0 0x7f4bc0153ff0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800d150 tx=0x7f4bb801eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e40880f2468665 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bbd59a640 1 -- 172.21.15.39:0/2620331156 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800b2d0 con 0x7f4bc00c4ba0 2024-09-13T17:13:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bbd59a640 1 -- 172.21.15.39:0/2620331156 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb800b430 con 0x7f4bc00c4ba0 2024-09-13T17:13:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bbd59a640 1 -- 172.21.15.39:0/2620331156 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80a7b30 con 0x7f4bc00c4ba0 2024-09-13T17:13:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2620331156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4ba0 msgr2=0x7f4bc0153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bc4ecf640 1 --2- 172.21.15.39:0/2620331156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4ba0 0x7f4bc0153ff0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800d150 tx=0x7f4bb801eb20 comp rx=0 tx=0).stop 2024-09-13T17:13:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2620331156 shutdown_connections 2024-09-13T17:13:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bc4ecf640 1 --2- 172.21.15.39:0/2620331156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4ba0 0x7f4bc0153ff0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bc4ecf640 1 --2- 172.21.15.39:0/2620331156 >> v2:172.21.15.84:3300/0 conn(0x7f4bc014acf0 0x7f4bc0151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2620331156 >> 172.21.15.39:0/2620331156 conn(0x7f4bc00c32a0 msgr2=0x7f4bc00c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:59.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2620331156 shutdown_connections 2024-09-13T17:13:59.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.102+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2620331156 wait complete. 2024-09-13T17:13:59.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 Processor -- start 2024-09-13T17:13:59.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 -- start start 2024-09-13T17:13:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4710 0x7f4bc01e9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4bc00c4ba0 0x7f4bc0070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc014acf0 0x7f4bc0070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc01538b0 con 0x7f4bc00c4710 2024-09-13T17:13:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0153c60 con 0x7f4bc014acf0 2024-09-13T17:13:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc0153f40 con 0x7f4bc00c4ba0 2024-09-13T17:13:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbe59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4710 0x7f4bc01e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbe59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4710 0x7f4bc01e9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54506/0 (socket says 172.21.15.39:54506) 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbe59c640 1 -- 172.21.15.39:0/2882398287 learned_addr learned my addr 172.21.15.39:0/2882398287 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbed9d640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc014acf0 0x7f4bc0070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbdd9b640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4bc00c4ba0 0x7f4bc0070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbe59c640 1 -- 172.21.15.39:0/2882398287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4bc00c4ba0 msgr2=0x7f4bc0070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbe59c640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4bc00c4ba0 0x7f4bc0070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbe59c640 1 -- 172.21.15.39:0/2882398287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc014acf0 msgr2=0x7f4bc0070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbe59c640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc014acf0 0x7f4bc0070fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbe59c640 1 -- 172.21.15.39:0/2882398287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb800bd40 con 0x7f4bc00c4710 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbdd9b640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4bc00c4ba0 0x7f4bc0070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:13:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bbe59c640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4710 0x7f4bc01e9330 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400b3e0 tx=0x7f4bb400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4baeffd640 1 -- 172.21.15.39:0/2882398287 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4011830 con 0x7f4bc00c4710 2024-09-13T17:13:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4baeffd640 1 -- 172.21.15.39:0/2882398287 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb40119b0 con 0x7f4bc00c4710 2024-09-13T17:13:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4baeffd640 1 -- 172.21.15.39:0/2882398287 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4011d60 con 0x7f4bc00c4710 2024-09-13T17:13:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc0154220 con 0x7f4bc00c4710 2024-09-13T17:13:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.106+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc01e8e10 con 0x7f4bc00c4710 2024-09-13T17:13:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.110+0000 7f4baeffd640 1 -- 172.21.15.39:0/2882398287 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4bb401a400 con 0x7f4bc00c4710 2024-09-13T17:13:59.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.110+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc0154ab0 con 0x7f4bc00c4710 2024-09-13T17:13:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.110+0000 7f4baeffd640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ba00666c0 0x7f4ba0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:13:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.110+0000 7f4baeffd640 1 -- 172.21.15.39:0/2882398287 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f4bb40850a0 con 0x7f4bc00c4710 2024-09-13T17:13:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.110+0000 7f4bbdd9b640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ba00666c0 0x7f4ba0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:13:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.114+0000 7f4bbdd9b640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ba00666c0 0x7f4ba0068b80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0007fd0 tx=0x7f4bb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:13:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.118+0000 7f4baeffd640 1 -- 172.21.15.39:0/2882398287 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4bb4051540 con 0x7f4bc00c4710 2024-09-13T17:13:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.398+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bc00a5ac0 con 0x7f4bc00c4710 2024-09-13T17:13:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.398+0000 7f4baeffd640 1 -- 172.21.15.39:0/2882398287 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f4bb4015090 con 0x7f4bc00c4710 2024-09-13T17:13:59.399 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:13:59.399 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:13:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 177 2024-09-13T17:13:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ba00666c0 msgr2=0x7f4ba0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ba00666c0 0x7f4ba0068b80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0007fd0 tx=0x7f4bb0009040 comp rx=0 tx=0).stop 2024-09-13T17:13:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4710 msgr2=0x7f4bc01e9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:13:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4710 0x7f4bc01e9330 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400b3e0 tx=0x7f4bb400b8a0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 shutdown_connections 2024-09-13T17:13:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4ba00666c0 0x7f4ba0068b80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc014acf0 0x7f4bc0070fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4bc00c4ba0 0x7f4bc0070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 --2- 172.21.15.39:0/2882398287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bc00c4710 0x7f4bc01e9330 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:13:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 >> 172.21.15.39:0/2882398287 conn(0x7f4bc00c32a0 msgr2=0x7f4bc0069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:13:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 shutdown_connections 2024-09-13T17:13:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:13:59.402+0000 7f4bc4ecf640 1 -- 172.21.15.39:0/2882398287 wait complete. 2024-09-13T17:13:59.429 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:14:01.303 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:14:01.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:14:01.431 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:01.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1f5b6640 1 Processor -- start 2024-09-13T17:14:01.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1f5b6640 1 -- start start 2024-09-13T17:14:01.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1f5b6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4d1814a770 0x7f4d18150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1f5b6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d180c6060 0x7f4d1814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1f5b6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d180c4760 con 0x7f4d1806ae20 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1f5b6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1806df80 con 0x7f4d1814a770 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1f5b6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180c1950 con 0x7f4d180c6060 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1d352640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4d1814a770 0x7f4d18150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1d352640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4d1814a770 0x7f4d18150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46932/0 (socket says 172.21.15.39:46932) 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1d352640 1 -- 172.21.15.39:0/799341945 learned_addr learned my addr 172.21.15.39:0/799341945 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.590+0000 7f4d1cb51640 1 --2- 172.21.15.39:0/799341945 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d180c6060 0x7f4d1814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d1d352640 1 -- 172.21.15.39:0/799341945 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d180c6060 msgr2=0x7f4d1814ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d1d352640 1 --2- 172.21.15.39:0/799341945 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d180c6060 0x7f4d1814ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d1d352640 1 -- 172.21.15.39:0/799341945 >> v1:172.21.15.84:6789/0 conn(0x7f4d1806ae20 legacy=0x7f4d180713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:01.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d1d352640 1 -- 172.21.15.39:0/799341945 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d18151120 con 0x7f4d1814a770 2024-09-13T17:14:01.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d1d352640 1 --2- 172.21.15.39:0/799341945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d1814a770 0x7f4d18150ad0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d1401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9a491c58046f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:01.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d0ffff640 1 -- 172.21.15.39:0/799341945 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d140ac410 con 0x7f4d1814a770 2024-09-13T17:14:01.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d0ffff640 1 -- 172.21.15.39:0/799341945 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d140ac570 con 0x7f4d1814a770 2024-09-13T17:14:01.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d0ffff640 1 -- 172.21.15.39:0/799341945 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d140ac900 con 0x7f4d1814a770 2024-09-13T17:14:01.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/799341945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d1814a770 msgr2=0x7f4d18150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:01.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.594+0000 7f4d1f5b6640 1 --2- 172.21.15.39:0/799341945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d1814a770 0x7f4d18150ad0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d1401c4a0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.598+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/799341945 shutdown_connections 2024-09-13T17:14:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.598+0000 7f4d1f5b6640 1 --2- 172.21.15.39:0/799341945 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d180c6060 0x7f4d1814ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.598+0000 7f4d1f5b6640 1 --2- 172.21.15.39:0/799341945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d1814a770 0x7f4d18150ad0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.598+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/799341945 >> 172.21.15.39:0/799341945 conn(0x7f4d180c32a0 msgr2=0x7f4d180c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.598+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/799341945 shutdown_connections 2024-09-13T17:14:01.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.598+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/799341945 wait complete. 2024-09-13T17:14:01.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 Processor -- start 2024-09-13T17:14:01.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 -- start start 2024-09-13T17:14:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d1806ae20 0x7f4d180aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d180c6060 0x7f4d18147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1814a770 0x7f4d18147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d181eacf0 con 0x7f4d180c6060 2024-09-13T17:14:01.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180a9900 con 0x7f4d1814a770 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d18153ed0 con 0x7f4d1806ae20 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1cb51640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d180c6060 0x7f4d18147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1cb51640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d180c6060 0x7f4d18147510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46944/0 (socket says 172.21.15.39:46944) 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1cb51640 1 -- 172.21.15.39:0/252765094 learned_addr learned my addr 172.21.15.39:0/252765094 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1db53640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1814a770 0x7f4d18147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1d352640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d1806ae20 0x7f4d180aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1cb51640 1 -- 172.21.15.39:0/252765094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d1806ae20 msgr2=0x7f4d180aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1cb51640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d1806ae20 0x7f4d180aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1cb51640 1 -- 172.21.15.39:0/252765094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1814a770 msgr2=0x7f4d18147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1cb51640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1814a770 0x7f4d18147a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1cb51640 1 -- 172.21.15.39:0/252765094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d140094d0 con 0x7f4d180c6060 2024-09-13T17:14:01.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1cb51640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d180c6060 0x7f4d18147510 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800e900 tx=0x7f4d0800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:01.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d0dffb640 1 -- 172.21.15.39:0/252765094 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08009690 con 0x7f4d180c6060 2024-09-13T17:14:01.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d0dffb640 1 -- 172.21.15.39:0/252765094 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0800ca00 con 0x7f4d180c6060 2024-09-13T17:14:01.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d0dffb640 1 -- 172.21.15.39:0/252765094 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800cd90 con 0x7f4d180c6060 2024-09-13T17:14:01.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d18154190 con 0x7f4d180c6060 2024-09-13T17:14:01.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.602+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d18154470 con 0x7f4d180c6060 2024-09-13T17:14:01.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.606+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d180619c0 con 0x7f4d180c6060 2024-09-13T17:14:01.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.606+0000 7f4d0dffb640 1 -- 172.21.15.39:0/252765094 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4d08013470 con 0x7f4d180c6060 2024-09-13T17:14:01.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.606+0000 7f4d0dffb640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cfc066690 0x7f4cfc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:01.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.606+0000 7f4d0dffb640 1 -- 172.21.15.39:0/252765094 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f4d080851e0 con 0x7f4d180c6060 2024-09-13T17:14:01.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.606+0000 7f4d1d352640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cfc066690 0x7f4cfc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:01.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.606+0000 7f4d1d352640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cfc066690 0x7f4cfc068b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4d140ae000 tx=0x7f4d14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:01.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.614+0000 7f4d0dffb640 1 -- 172.21.15.39:0/252765094 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4d08051730 con 0x7f4d180c6060 2024-09-13T17:14:01.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.898+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d1814e040 con 0x7f4d180c6060 2024-09-13T17:14:01.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.898+0000 7f4d0dffb640 1 -- 172.21.15.39:0/252765094 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f4d08017090 con 0x7f4d180c6060 2024-09-13T17:14:01.901 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:01.901 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:01.902 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 177 2024-09-13T17:14:01.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cfc066690 msgr2=0x7f4cfc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:01.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cfc066690 0x7f4cfc068b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4d140ae000 tx=0x7f4d14008040 comp rx=0 tx=0).stop 2024-09-13T17:14:01.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d180c6060 msgr2=0x7f4d18147510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:01.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d180c6060 0x7f4d18147510 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800e900 tx=0x7f4d0800edc0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 shutdown_connections 2024-09-13T17:14:01.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cfc066690 0x7f4cfc068b50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1814a770 0x7f4d18147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d180c6060 0x7f4d18147510 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 --2- 172.21.15.39:0/252765094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4d1806ae20 0x7f4d180aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:01.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 >> 172.21.15.39:0/252765094 conn(0x7f4d180c32a0 msgr2=0x7f4d180c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:01.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 shutdown_connections 2024-09-13T17:14:01.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:01.902+0000 7f4d1f5b6640 1 -- 172.21.15.39:0/252765094 wait complete. 2024-09-13T17:14:01.936 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:14:03.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:04.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.082+0000 7f694014d640 1 Processor -- start 2024-09-13T17:14:04.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f694014d640 1 -- start start 2024-09-13T17:14:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f694014d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938150f80 0x7f6938153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f694014d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f69380c5330 0x7f693806c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693e6ea640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6938153e70 0x7f693806c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:43984/0 (socket says 172.21.15.39:43984) 2024-09-13T17:14:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693e6ea640 1 -- 172.21.15.39:0/3449775593 learned_addr learned my addr 172.21.15.39:0/3449775593 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:14:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693d6e8640 1 --2- 172.21.15.39:0/3449775593 >> v2:172.21.15.39:3300/0 conn(0x7f69380c5330 0x7f693806c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f694014d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69381546c0 con 0x7f6938153e70 2024-09-13T17:14:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f694014d640 1 -- 172.21.15.39:0/3449775593 --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69380c19a0 con 0x7f69380c5330 2024-09-13T17:14:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f694014d640 1 -- 172.21.15.39:0/3449775593 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69380c49f0 con 0x7f6938150f80 2024-09-13T17:14:04.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693dee9640 1 --2- 172.21.15.39:0/3449775593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938150f80 0x7f6938153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:04.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693cee7640 1 -- 172.21.15.39:0/3449775593 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261495572 0 0) 0x7f69381546c0 con 0x7f6938153e70 2024-09-13T17:14:04.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693cee7640 1 -- 172.21.15.39:0/3449775593 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6924003450 con 0x7f6938153e70 2024-09-13T17:14:04.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693d6e8640 1 -- 172.21.15.39:0/3449775593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938150f80 msgr2=0x7f6938153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:04.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693d6e8640 1 --2- 172.21.15.39:0/3449775593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938150f80 0x7f6938153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693d6e8640 1 -- 172.21.15.39:0/3449775593 >> v1:172.21.15.84:6789/0 conn(0x7f6938153e70 legacy=0x7f693806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:04.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693d6e8640 1 -- 172.21.15.39:0/3449775593 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69381543e0 con 0x7f69380c5330 2024-09-13T17:14:04.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.086+0000 7f693dee9640 1 --2- 172.21.15.39:0/3449775593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938150f80 0x7f6938153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:14:04.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f693d6e8640 1 --2- 172.21.15.39:0/3449775593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69380c5330 0x7f693806c090 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f6930009d90 tx=0x7f693001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5c340e754ad87c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:04.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f693cee7640 1 -- 172.21.15.39:0/3449775593 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69300a5800 con 0x7f69380c5330 2024-09-13T17:14:04.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f693cee7640 1 -- 172.21.15.39:0/3449775593 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69300a5960 con 0x7f69380c5330 2024-09-13T17:14:04.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f693cee7640 1 -- 172.21.15.39:0/3449775593 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69300a5cf0 con 0x7f69380c5330 2024-09-13T17:14:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 -- 172.21.15.39:0/3449775593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69380c5330 msgr2=0x7f693806c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 --2- 172.21.15.39:0/3449775593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69380c5330 0x7f693806c090 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f6930009d90 tx=0x7f693001c8b0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 -- 172.21.15.39:0/3449775593 shutdown_connections 2024-09-13T17:14:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 --2- 172.21.15.39:0/3449775593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69380c5330 0x7f693806c090 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 --2- 172.21.15.39:0/3449775593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938150f80 0x7f6938153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 -- 172.21.15.39:0/3449775593 >> 172.21.15.39:0/3449775593 conn(0x7f6938069e10 msgr2=0x7f693806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 -- 172.21.15.39:0/3449775593 shutdown_connections 2024-09-13T17:14:04.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 -- 172.21.15.39:0/3449775593 wait complete. 2024-09-13T17:14:04.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 Processor -- start 2024-09-13T17:14:04.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.090+0000 7f694014d640 1 -- start start 2024-09-13T17:14:04.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f694014d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69380c5330 0x7f69380a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:04.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f694014d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6938150f80 0x7f6938148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:04.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f694014d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938153e70 0x7f69381efc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:04.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f694014d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69381ed3d0 con 0x7f6938150f80 2024-09-13T17:14:04.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f694014d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69381ec750 con 0x7f69380c5330 2024-09-13T17:14:04.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f694014d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69381eca30 con 0x7f6938153e70 2024-09-13T17:14:04.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693d6e8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6938150f80 0x7f6938148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:04.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693d6e8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6938150f80 0x7f6938148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46972/0 (socket says 172.21.15.39:46972) 2024-09-13T17:14:04.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693d6e8640 1 -- 172.21.15.39:0/1918290440 learned_addr learned my addr 172.21.15.39:0/1918290440 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:04.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693e6ea640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938153e70 0x7f69381efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:04.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693dee9640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69380c5330 0x7f69380a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:04.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693e6ea640 1 -- 172.21.15.39:0/1918290440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69380c5330 msgr2=0x7f69380a7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:04.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693e6ea640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69380c5330 0x7f69380a7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693e6ea640 1 -- 172.21.15.39:0/1918290440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6938150f80 msgr2=0x7f6938148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:04.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693e6ea640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6938150f80 0x7f6938148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693e6ea640 1 -- 172.21.15.39:0/1918290440 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6930009a70 con 0x7f6938153e70 2024-09-13T17:14:04.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f693e6ea640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938153e70 0x7f69381efc50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f692800c3f0 tx=0x7f692800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f692e7fc640 1 -- 172.21.15.39:0/1918290440 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928005cf0 con 0x7f6938153e70 2024-09-13T17:14:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f692e7fc640 1 -- 172.21.15.39:0/1918290440 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6928005e50 con 0x7f6938153e70 2024-09-13T17:14:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69381ecd10 con 0x7f6938153e70 2024-09-13T17:14:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f692e7fc640 1 -- 172.21.15.39:0/1918290440 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280072d0 con 0x7f6938153e70 2024-09-13T17:14:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69380a83d0 con 0x7f6938153e70 2024-09-13T17:14:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f692e7fc640 1 -- 172.21.15.39:0/1918290440 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6928007430 con 0x7f6938153e70 2024-09-13T17:14:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69381ef970 con 0x7f6938153e70 2024-09-13T17:14:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.094+0000 7f692e7fc640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f691c066690 0x7f691c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.098+0000 7f692e7fc640 1 -- 172.21.15.39:0/1918290440 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 9561+0+0 (secure 0 0 0) 0x7f6928084490 con 0x7f6938153e70 2024-09-13T17:14:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.098+0000 7f693dee9640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f691c066690 0x7f691c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:04.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.098+0000 7f693dee9640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f691c066690 0x7f691c068b50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6934006e90 tx=0x7f6934008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:04.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.102+0000 7f692e7fc640 1 -- 172.21.15.39:0/1918290440 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69381ef970 con 0x7f6938153e70 2024-09-13T17:14:04.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.394+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69381ef970 con 0x7f6938153e70 2024-09-13T17:14:04.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.394+0000 7f692e7fc640 1 -- 172.21.15.39:0/1918290440 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5931 (secure 0 0 0) 0x7f69381ef970 con 0x7f6938153e70 2024-09-13T17:14:04.398 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:04.398 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:12:03.044383+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7932":{"gid":7932,"name":"c","rank":0,"incarnation":168,"state":"up:active","state_seq":14,"addr":"172.21.15.39:6837/282472862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":282472862},{"type":"v1","addr":"172.21.15.39:6837","nonce":282472862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:04.398 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 177 2024-09-13T17:14:04.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f691c066690 msgr2=0x7f691c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:04.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f691c066690 0x7f691c068b50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6934006e90 tx=0x7f6934008040 comp rx=0 tx=0).stop 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938153e70 msgr2=0x7f69381efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938153e70 0x7f69381efc50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f692800c3f0 tx=0x7f692800c8b0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 shutdown_connections 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f691c066690 0x7f691c068b50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6938153e70 0x7f69381efc50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6938150f80 0x7f6938148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 --2- 172.21.15.39:0/1918290440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69380c5330 0x7f69380a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 >> 172.21.15.39:0/1918290440 conn(0x7f6938069e10 msgr2=0x7f693806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 shutdown_connections 2024-09-13T17:14:04.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:04.398+0000 7f694014d640 1 -- 172.21.15.39:0/1918290440 wait complete. 2024-09-13T17:14:04.434 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:14:06.434 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:06.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4c719640 1 Processor -- start 2024-09-13T17:14:06.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4c719640 1 -- start start 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4c719640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fda440c5870 0x7fda440c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4c719640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda44153e30 con 0x7fda44154bb0 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4c719640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440a7000 con 0x7fda440c2370 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4c719640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda440720f0 con 0x7fda440c5870 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4acb6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fda440c5870 0x7fda440c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda49cb4640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fda44154bb0 0x7fda4415c7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:50116/0 (socket says 172.21.15.39:50116) 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda49cb4640 1 -- 172.21.15.39:0/4110678140 learned_addr learned my addr 172.21.15.39:0/4110678140 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4acb6640 1 -- 172.21.15.39:0/4110678140 >> v1:172.21.15.84:6789/0 conn(0x7fda440c2370 legacy=0x7fda440c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4acb6640 1 -- 172.21.15.39:0/4110678140 >> v1:172.21.15.39:6789/0 conn(0x7fda44154bb0 legacy=0x7fda4415c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda4acb6640 1 -- 172.21.15.39:0/4110678140 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda441540d0 con 0x7fda440c5870 2024-09-13T17:14:06.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.582+0000 7fda494b3640 1 -- 172.21.15.39:0/4110678140 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155275275 0 0) 0x7fda44153e30 con 0x7fda44154bb0 2024-09-13T17:14:06.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.586+0000 7fda4acb6640 1 --2- 172.21.15.39:0/4110678140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c5870 0x7fda440c2750 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009d90 tx=0x7fda3c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ab047328b7e01b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:06.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.586+0000 7fda494b3640 1 -- 172.21.15.39:0/4110678140 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0a57a0 con 0x7fda440c5870 2024-09-13T17:14:06.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.586+0000 7fda494b3640 1 -- 172.21.15.39:0/4110678140 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c0a5900 con 0x7fda440c5870 2024-09-13T17:14:06.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.586+0000 7fda494b3640 1 -- 172.21.15.39:0/4110678140 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0a5c90 con 0x7fda440c5870 2024-09-13T17:14:06.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.586+0000 7fda4c719640 1 -- 172.21.15.39:0/4110678140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c5870 msgr2=0x7fda440c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:06.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.586+0000 7fda4c719640 1 --2- 172.21.15.39:0/4110678140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c5870 0x7fda440c2750 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009d90 tx=0x7fda3c01c8e0 comp rx=0 tx=0).stop 2024-09-13T17:14:06.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.586+0000 7fda4c719640 1 -- 172.21.15.39:0/4110678140 shutdown_connections 2024-09-13T17:14:06.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.586+0000 7fda4c719640 1 --2- 172.21.15.39:0/4110678140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c5870 0x7fda440c2750 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:06.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.586+0000 7fda4c719640 1 -- 172.21.15.39:0/4110678140 >> 172.21.15.39:0/4110678140 conn(0x7fda440a9720 msgr2=0x7fda440a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:06.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 -- 172.21.15.39:0/4110678140 shutdown_connections 2024-09-13T17:14:06.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 -- 172.21.15.39:0/4110678140 wait complete. 2024-09-13T17:14:06.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 Processor -- start 2024-09-13T17:14:06.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 -- start start 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c2370 0x7fda441effa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda440c5870 0x7fda441f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda44154bb0 0x7fda4406e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4414b5e0 con 0x7fda440c2370 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4415fa90 con 0x7fda440c5870 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4c719640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4406dc90 con 0x7fda44154bb0 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4a4b5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c2370 0x7fda441effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4a4b5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c2370 0x7fda441effa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46992/0 (socket says 172.21.15.39:46992) 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4a4b5640 1 -- 172.21.15.39:0/2417084446 learned_addr learned my addr 172.21.15.39:0/2417084446 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda4acb6640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda44154bb0 0x7fda4406e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:06.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.590+0000 7fda49cb4640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda440c5870 0x7fda441f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:06.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda49cb4640 1 -- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda44154bb0 msgr2=0x7fda4406e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda49cb4640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda44154bb0 0x7fda4406e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda49cb4640 1 -- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c2370 msgr2=0x7fda441effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda49cb4640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c2370 0x7fda441effa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda49cb4640 1 -- 172.21.15.39:0/2417084446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c009a70 con 0x7fda440c5870 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda4acb6640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda44154bb0 0x7fda4406e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda4a4b5640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c2370 0x7fda441effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda49cb4640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda440c5870 0x7fda441f04e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fda44154fc0 tx=0x7fda3400c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda3affd640 1 -- 172.21.15.39:0/2417084446 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda340074e0 con 0x7fda440c5870 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda3affd640 1 -- 172.21.15.39:0/2417084446 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda34007660 con 0x7fda440c5870 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4414d680 con 0x7fda440c5870 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda3affd640 1 -- 172.21.15.39:0/2417084446 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34004a30 con 0x7fda440c5870 2024-09-13T17:14:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda441f0b30 con 0x7fda440c5870 2024-09-13T17:14:06.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c005260 con 0x7fda440c5870 2024-09-13T17:14:06.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda3affd640 1 -- 172.21.15.39:0/2417084446 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda34004c50 con 0x7fda440c5870 2024-09-13T17:14:06.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda3affd640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fda280666e0 0x7fda28068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:06.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda4a4b5640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fda280666e0 0x7fda28068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:06.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.594+0000 7fda3affd640 1 -- 172.21.15.39:0/2417084446 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7fda34055580 con 0x7fda440c5870 2024-09-13T17:14:06.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.598+0000 7fda4a4b5640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fda280666e0 0x7fda28068ba0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fda40001ca0 tx=0x7fda40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:06.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.606+0000 7fda3affd640 1 -- 172.21.15.39:0/2417084446 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fda340518c0 con 0x7fda440c5870 2024-09-13T17:14:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.874+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda2c0057e0 con 0x7fda440c5870 2024-09-13T17:14:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.874+0000 7fda3affd640 1 -- 172.21.15.39:0/2417084446 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5179 (secure 0 0 0) 0x7fda34054ee0 con 0x7fda440c5870 2024-09-13T17:14:06.876 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:06.877 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:06.332363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:06.877 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 179 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fda280666e0 msgr2=0x7fda28068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fda280666e0 0x7fda28068ba0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fda40001ca0 tx=0x7fda40009040 comp rx=0 tx=0).stop 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda440c5870 msgr2=0x7fda441f04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda440c5870 0x7fda441f04e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fda44154fc0 tx=0x7fda3400c870 comp rx=0 tx=0).stop 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 shutdown_connections 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fda280666e0 0x7fda28068ba0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda440c5870 0x7fda441f04e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fda44154bb0 0x7fda4406e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 --2- 172.21.15.39:0/2417084446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda440c2370 0x7fda441effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 >> 172.21.15.39:0/2417084446 conn(0x7fda440a9720 msgr2=0x7fda44149940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 shutdown_connections 2024-09-13T17:14:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:06.878+0000 7fda4c719640 1 -- 172.21.15.39:0/2417084446 wait complete. 2024-09-13T17:14:06.901 INFO:tasks.mds_thrash.fs.[a]:mds.c down, removed from mdsmap 2024-09-13T17:14:06.901 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:14:06.901 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c45a9640 1 Processor -- start 2024-09-13T17:14:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c45a9640 1 -- start start 2024-09-13T17:14:07.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c45a9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4bc06b080 0x7fa4bc06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:07.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c45a9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4bc1576e0 0x7fa4bc157b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c45a9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4bc0c15d0 con 0x7fa4bc0c1810 2024-09-13T17:14:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c45a9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4bc06d0b0 con 0x7fa4bc06b080 2024-09-13T17:14:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c45a9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc0c1f00 con 0x7fa4bc1576e0 2024-09-13T17:14:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c1b44640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa4bc0c1810 0x7fa4bc151390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:44006/0 (socket says 172.21.15.39:44006) 2024-09-13T17:14:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c2345640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4bc06b080 0x7fa4bc06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.050+0000 7fa4c1b44640 1 -- 172.21.15.39:0/26409770 learned_addr learned my addr 172.21.15.39:0/26409770 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:14:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c2345640 1 -- 172.21.15.39:0/26409770 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4bc1576e0 msgr2=0x7fa4bc157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c2345640 1 --2- 172.21.15.39:0/26409770 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4bc1576e0 0x7fa4bc157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c2345640 1 -- 172.21.15.39:0/26409770 >> v1:172.21.15.84:6789/0 conn(0x7fa4bc0c1810 legacy=0x7fa4bc151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:07.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c2345640 1 -- 172.21.15.39:0/26409770 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc0c21a0 con 0x7fa4bc06b080 2024-09-13T17:14:07.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c2345640 1 --2- 172.21.15.39:0/26409770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4bc06b080 0x7fa4bc06e680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80097c0 tx=0x7fa4b801c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=750a501a1bfa35d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:07.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c1343640 1 -- 172.21.15.39:0/26409770 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b80ad680 con 0x7fa4bc06b080 2024-09-13T17:14:07.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c1343640 1 -- 172.21.15.39:0/26409770 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b80ad7e0 con 0x7fa4bc06b080 2024-09-13T17:14:07.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c1343640 1 -- 172.21.15.39:0/26409770 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b80adb70 con 0x7fa4bc06b080 2024-09-13T17:14:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c45a9640 1 -- 172.21.15.39:0/26409770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4bc06b080 msgr2=0x7fa4bc06e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c45a9640 1 --2- 172.21.15.39:0/26409770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4bc06b080 0x7fa4bc06e680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80097c0 tx=0x7fa4b801c490 comp rx=0 tx=0).stop 2024-09-13T17:14:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c45a9640 1 -- 172.21.15.39:0/26409770 shutdown_connections 2024-09-13T17:14:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c45a9640 1 --2- 172.21.15.39:0/26409770 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4bc1576e0 0x7fa4bc157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c45a9640 1 --2- 172.21.15.39:0/26409770 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4bc06b080 0x7fa4bc06e680 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.054+0000 7fa4c45a9640 1 -- 172.21.15.39:0/26409770 >> 172.21.15.39:0/26409770 conn(0x7fa4bc0c32a0 msgr2=0x7fa4bc0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 -- 172.21.15.39:0/26409770 shutdown_connections 2024-09-13T17:14:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 -- 172.21.15.39:0/26409770 wait complete. 2024-09-13T17:14:07.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 Processor -- start 2024-09-13T17:14:07.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 -- start start 2024-09-13T17:14:07.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4bc06b080 0x7fa4bc1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:07.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4bc0c1810 0x7fa4bc14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:07.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4bc1576e0 0x7fa4bc14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:07.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc153f30 con 0x7fa4bc0c1810 2024-09-13T17:14:07.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc0bfef0 con 0x7fa4bc1576e0 2024-09-13T17:14:07.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc0bd480 con 0x7fa4bc06b080 2024-09-13T17:14:07.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c1b44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4bc0c1810 0x7fa4bc14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:07.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c1b44640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4bc0c1810 0x7fa4bc14cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47004/0 (socket says 172.21.15.39:47004) 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c1b44640 1 -- 172.21.15.39:0/507482149 learned_addr learned my addr 172.21.15.39:0/507482149 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c2345640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4bc06b080 0x7fa4bc1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c2b46640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4bc1576e0 0x7fa4bc14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c1b44640 1 -- 172.21.15.39:0/507482149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4bc06b080 msgr2=0x7fa4bc1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c1b44640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4bc06b080 0x7fa4bc1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c1b44640 1 -- 172.21.15.39:0/507482149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4bc1576e0 msgr2=0x7fa4bc14d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c1b44640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4bc1576e0 0x7fa4bc14d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c1b44640 1 -- 172.21.15.39:0/507482149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b8009420 con 0x7fa4bc0c1810 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c1b44640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4bc0c1810 0x7fa4bc14cc50 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00c4d0 tx=0x7fa4ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4b2ffd640 1 -- 172.21.15.39:0/507482149 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac005cf0 con 0x7fa4bc0c1810 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4b2ffd640 1 -- 172.21.15.39:0/507482149 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ac005e50 con 0x7fa4bc0c1810 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4b2ffd640 1 -- 172.21.15.39:0/507482149 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0072b0 con 0x7fa4bc0c1810 2024-09-13T17:14:07.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4bc1522b0 con 0x7fa4bc0c1810 2024-09-13T17:14:07.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.058+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4bc152870 con 0x7fa4bc0c1810 2024-09-13T17:14:07.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.062+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4bc152d90 con 0x7fa4bc0c1810 2024-09-13T17:14:07.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.062+0000 7fa4b2ffd640 1 -- 172.21.15.39:0/507482149 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa4ac007410 con 0x7fa4bc0c1810 2024-09-13T17:14:07.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.062+0000 7fa4b2ffd640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4a0066690 0x7fa4a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:07.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.062+0000 7fa4b2ffd640 1 -- 172.21.15.39:0/507482149 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7fa4ac084530 con 0x7fa4bc0c1810 2024-09-13T17:14:07.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.062+0000 7fa4c2345640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4a0066690 0x7fa4a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:07.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.062+0000 7fa4c2345640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4a0066690 0x7fa4a0068b50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80af000 tx=0x7fa4b80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:07.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.066+0000 7fa4b2ffd640 1 -- 172.21.15.39:0/507482149 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa4ac050a30 con 0x7fa4bc0c1810 2024-09-13T17:14:07.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.374+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4bc06e680 con 0x7fa4bc0c1810 2024-09-13T17:14:07.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4b2ffd640 1 -- 172.21.15.39:0/507482149 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5179 (secure 0 0 0) 0x7fa4ac017020 con 0x7fa4bc0c1810 2024-09-13T17:14:07.378 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:07.379 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:06.332363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:07.379 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 179 2024-09-13T17:14:07.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4a0066690 msgr2=0x7fa4a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:07.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4a0066690 0x7fa4a0068b50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80af000 tx=0x7fa4b80a7000 comp rx=0 tx=0).stop 2024-09-13T17:14:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4bc0c1810 msgr2=0x7fa4bc14cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4bc0c1810 0x7fa4bc14cc50 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00c4d0 tx=0x7fa4ac00c990 comp rx=0 tx=0).stop 2024-09-13T17:14:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 shutdown_connections 2024-09-13T17:14:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa4a0066690 0x7fa4a0068b50 secure :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80af000 tx=0x7fa4b80a7000 comp rx=0 tx=0).stop 2024-09-13T17:14:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4bc1576e0 0x7fa4bc14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4bc0c1810 0x7fa4bc14cc50 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 --2- 172.21.15.39:0/507482149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa4bc06b080 0x7fa4bc1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:07.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.378+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 >> 172.21.15.39:0/507482149 conn(0x7fa4bc0c32a0 msgr2=0x7fa4bc071f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:07.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.382+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 shutdown_connections 2024-09-13T17:14:07.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:07.382+0000 7fa4c45a9640 1 -- 172.21.15.39:0/507482149 wait complete. 2024-09-13T17:14:07.409 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 179, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 179, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6530": { "addr": "172.21.15.84:6835/380008541", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 380008541, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 380008541, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6530, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:reconnect", "state_seq": 25 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:14:06.332363+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6530 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 176, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6503": { "addr": "172.21.15.39:6835/305129652", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 305129652, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 305129652, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6503, "incarnation": 173, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 58, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:12:23.135929+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6503 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:14:09.410 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:09.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.830+0000 7f18bfe30640 1 Processor -- start 2024-09-13T17:14:09.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.830+0000 7f18bfe30640 1 -- start start 2024-09-13T17:14:09.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.830+0000 7f18bfe30640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18b0007300 0x7f18b004a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:09.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.830+0000 7f18bfe30640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18b00ddf90 0x7f18b00de3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:09.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.830+0000 7f18bfe30640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b0047120 con 0x7f18b00d8070 2024-09-13T17:14:09.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.830+0000 7f18bfe30640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b002b070 con 0x7f18b0007300 2024-09-13T17:14:09.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.830+0000 7f18bfe30640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b00d8450 con 0x7f18b00ddf90 2024-09-13T17:14:09.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.834+0000 7f18be3cd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18b00ddf90 0x7f18b00de3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:09.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.834+0000 7f18bd3cb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18b0007300 0x7f18b004a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:09.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.834+0000 7f18bd3cb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18b0007300 0x7f18b004a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47010/0 (socket says 172.21.15.39:47010) 2024-09-13T17:14:09.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.834+0000 7f18bd3cb640 1 -- 172.21.15.39:0/3496438135 learned_addr learned my addr 172.21.15.39:0/3496438135 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:09.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.838+0000 7f18be3cd640 1 -- 172.21.15.39:0/3496438135 >> v2:172.21.15.39:3300/0 conn(0x7f18b0007300 msgr2=0x7f18b004a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:09.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.838+0000 7f18be3cd640 1 --2- 172.21.15.39:0/3496438135 >> v2:172.21.15.39:3300/0 conn(0x7f18b0007300 0x7f18b004a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:09.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.838+0000 7f18be3cd640 1 -- 172.21.15.39:0/3496438135 >> v1:172.21.15.84:6789/0 conn(0x7f18b00d8070 legacy=0x7f18b000b0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:14:09.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.842+0000 7f18be3cd640 1 -- 172.21.15.39:0/3496438135 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b004ad00 con 0x7f18b00ddf90 2024-09-13T17:14:09.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.842+0000 7f18bd3cb640 1 --2- 172.21.15.39:0/3496438135 >> v2:172.21.15.39:3300/0 conn(0x7f18b0007300 0x7f18b004a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:14:09.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.850+0000 7f18be3cd640 1 --2- 172.21.15.39:0/3496438135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00ddf90 0x7f18b00de3a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f18b0007790 tx=0x7f18b401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b98aa1e7931230d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:09.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.850+0000 7f18bcbca640 1 -- 172.21.15.39:0/3496438135 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b40ab610 con 0x7f18b00ddf90 2024-09-13T17:14:09.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.850+0000 7f18bcbca640 1 -- 172.21.15.39:0/3496438135 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b40ab770 con 0x7f18b00ddf90 2024-09-13T17:14:09.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.850+0000 7f18bfe30640 1 -- 172.21.15.39:0/3496438135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00ddf90 msgr2=0x7f18b00de3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:09.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.850+0000 7f18bfe30640 1 --2- 172.21.15.39:0/3496438135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00ddf90 0x7f18b00de3a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f18b0007790 tx=0x7f18b401c410 comp rx=0 tx=0).stop 2024-09-13T17:14:09.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.850+0000 7f18bfe30640 1 -- 172.21.15.39:0/3496438135 shutdown_connections 2024-09-13T17:14:09.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.850+0000 7f18bfe30640 1 --2- 172.21.15.39:0/3496438135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00ddf90 0x7f18b00de3a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:09.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.850+0000 7f18bfe30640 1 --2- 172.21.15.39:0/3496438135 >> v2:172.21.15.39:3300/0 conn(0x7f18b0007300 0x7f18b004a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:09.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.850+0000 7f18bfe30640 1 -- 172.21.15.39:0/3496438135 >> 172.21.15.39:0/3496438135 conn(0x7f18b0048170 msgr2=0x7f18b0048580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:09.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.862+0000 7f18bfe30640 1 -- 172.21.15.39:0/3496438135 shutdown_connections 2024-09-13T17:14:09.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.862+0000 7f18bfe30640 1 -- 172.21.15.39:0/3496438135 wait complete. 2024-09-13T17:14:09.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.862+0000 7f18bfe30640 1 Processor -- start 2024-09-13T17:14:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.870+0000 7f18bfe30640 1 -- start start 2024-09-13T17:14:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.870+0000 7f18bfe30640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b0007300 0x7f18b0005610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.870+0000 7f18bfe30640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00d8070 0x7f18b00da310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.870+0000 7f18bfe30640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18b00d8650 0x7f18b00d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.870+0000 7f18bfe30640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00db330 con 0x7f18b0007300 2024-09-13T17:14:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.870+0000 7f18bfe30640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00410e0 con 0x7f18b00d8070 2024-09-13T17:14:09.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.870+0000 7f18bfe30640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00413c0 con 0x7f18b00d8650 2024-09-13T17:14:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bdbcc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b0007300 0x7f18b0005610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bdbcc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b0007300 0x7f18b0005610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47022/0 (socket says 172.21.15.39:47022) 2024-09-13T17:14:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bd3cb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00d8070 0x7f18b00da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bd3cb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00d8070 0x7f18b00da310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:60862/0 (socket says 172.21.15.39:60862) 2024-09-13T17:14:09.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bd3cb640 1 -- 172.21.15.39:0/2619967717 learned_addr learned my addr 172.21.15.39:0/2619967717 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bdbcc640 1 -- 172.21.15.39:0/2619967717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18b00d8650 msgr2=0x7f18b00d8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bdbcc640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18b00d8650 0x7f18b00d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bdbcc640 1 -- 172.21.15.39:0/2619967717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00d8070 msgr2=0x7f18b00da310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:09.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bdbcc640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00d8070 0x7f18b00da310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:09.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.874+0000 7f18bdbcc640 1 -- 172.21.15.39:0/2619967717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80094d0 con 0x7f18b0007300 2024-09-13T17:14:09.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.878+0000 7f18bdbcc640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b0007300 0x7f18b0005610 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f18b8063c00 tx=0x7f18b8067a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:09.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.878+0000 7f18ae7fc640 1 -- 172.21.15.39:0/2619967717 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8072460 con 0x7f18b0007300 2024-09-13T17:14:09.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.878+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b4009420 con 0x7f18b0007300 2024-09-13T17:14:09.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.878+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b0005b50 con 0x7f18b0007300 2024-09-13T17:14:09.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.882+0000 7f18ae7fc640 1 -- 172.21.15.39:0/2619967717 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b80725e0 con 0x7f18b0007300 2024-09-13T17:14:09.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.882+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b00db470 con 0x7f18b0007300 2024-09-13T17:14:09.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.882+0000 7f18ae7fc640 1 -- 172.21.15.39:0/2619967717 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b806f8d0 con 0x7f18b0007300 2024-09-13T17:14:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.882+0000 7f18ae7fc640 1 -- 172.21.15.39:0/2619967717 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18b806fab0 con 0x7f18b0007300 2024-09-13T17:14:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.882+0000 7f18ae7fc640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f189c0666e0 0x7f189c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:09.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.882+0000 7f18bd3cb640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f189c0666e0 0x7f189c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:09.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.882+0000 7f18bd3cb640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f189c0666e0 0x7f189c068ba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f18b00d4cd0 tx=0x7f18a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:09.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.890+0000 7f18ae7fc640 1 -- 172.21.15.39:0/2619967717 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7f18b80cf6e0 con 0x7f18b0007300 2024-09-13T17:14:09.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:09.910+0000 7f18ae7fc640 1 -- 172.21.15.39:0/2619967717 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f18b00db470 con 0x7f18b0007300 2024-09-13T17:14:10.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.718+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18b00db470 con 0x7f18b0007300 2024-09-13T17:14:10.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.722+0000 7f18ae7fc640 1 -- 172.21.15.39:0/2619967717 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+5176 (secure 0 0 0) 0x7f18b00db470 con 0x7f18b0007300 2024-09-13T17:14:10.723 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:10.723 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:10.723 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 181 2024-09-13T17:14:10.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f189c0666e0 msgr2=0x7f189c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:10.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f189c0666e0 0x7f189c068ba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f18b00d4cd0 tx=0x7f18a8008040 comp rx=0 tx=0).stop 2024-09-13T17:14:10.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b0007300 msgr2=0x7f18b0005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:10.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b0007300 0x7f18b0005610 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f18b8063c00 tx=0x7f18b8067a80 comp rx=0 tx=0).stop 2024-09-13T17:14:10.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 shutdown_connections 2024-09-13T17:14:10.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f189c0666e0 0x7f189c068ba0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:10.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b00d8070 0x7f18b00da310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:10.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18b00d8650 0x7f18b00d8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:10.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 --2- 172.21.15.39:0/2619967717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b0007300 0x7f18b0005610 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:10.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 >> 172.21.15.39:0/2619967717 conn(0x7f18b0048170 msgr2=0x7f18b00d6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:10.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 shutdown_connections 2024-09-13T17:14:10.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:10.726+0000 7f18bfe30640 1 -- 172.21.15.39:0/2619967717 wait complete. 2024-09-13T17:14:10.787 INFO:tasks.mds_thrash.fs.[a]:mds.d has gained rank=0, replacing gid=7932 2024-09-13T17:14:10.787 INFO:tasks.mds_thrash.fs.[a]:waiting for 6 secs before reviving mds.c 2024-09-13T17:14:13.493 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T17:14:13.304+0000 7f15e4e5c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:14:16.787 INFO:tasks.mds_thrash.fs.[a]:reviving mds.c 2024-09-13T17:14:16.788 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:14:16.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:14:16.790 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:14:16.790 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:17.079 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T17:14:17.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f674138c640 1 Processor -- start 2024-09-13T17:14:17.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f674138c640 1 -- start start 2024-09-13T17:14:17.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f674138c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f673c0c0c10 0x7f673c06f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:17.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f674138c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f673c06dc30 0x7f673c06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:17.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f674138c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673c06d7a0 con 0x7f673c0c3af0 2024-09-13T17:14:17.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f674138c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c0c5530 con 0x7f673c06dc30 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f674138c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c06ce90 con 0x7f673c0c0c10 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f673b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f673c06dc30 0x7f673c06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f673bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f673c0c0c10 0x7f673c06f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f673b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f673c06dc30 0x7f673c06fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50700/0 (socket says 172.21.15.39:50700) 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f673b7fe640 1 -- 172.21.15.39:0/406289007 learned_addr learned my addr 172.21.15.39:0/406289007 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f673bfff640 1 -- 172.21.15.39:0/406289007 >> v2:172.21.15.39:3300/0 conn(0x7f673c06dc30 msgr2=0x7f673c06fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f673bfff640 1 --2- 172.21.15.39:0/406289007 >> v2:172.21.15.39:3300/0 conn(0x7f673c06dc30 0x7f673c06fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f673bfff640 1 -- 172.21.15.39:0/406289007 >> v1:172.21.15.84:6789/0 conn(0x7f673c0c3af0 legacy=0x7f673c0c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:17.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.206+0000 7f673bfff640 1 -- 172.21.15.39:0/406289007 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c0c5740 con 0x7f673c0c0c10 2024-09-13T17:14:17.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.210+0000 7f673bfff640 1 --2- 172.21.15.39:0/406289007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c0c10 0x7f673c06f9a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f673c0672c0 tx=0x7f672c01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f6dfedbbf375605 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:17.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.210+0000 7f673affd640 1 -- 172.21.15.39:0/406289007 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c0ab600 con 0x7f673c0c0c10 2024-09-13T17:14:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.210+0000 7f673affd640 1 -- 172.21.15.39:0/406289007 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c0a7050 con 0x7f673c0c0c10 2024-09-13T17:14:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.210+0000 7f673affd640 1 -- 172.21.15.39:0/406289007 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c0ab980 con 0x7f673c0c0c10 2024-09-13T17:14:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.214+0000 7f674138c640 1 -- 172.21.15.39:0/406289007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c0c10 msgr2=0x7f673c06f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:17.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.214+0000 7f674138c640 1 --2- 172.21.15.39:0/406289007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c0c10 0x7f673c06f9a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f673c0672c0 tx=0x7f672c01c440 comp rx=0 tx=0).stop 2024-09-13T17:14:17.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.218+0000 7f674138c640 1 -- 172.21.15.39:0/406289007 shutdown_connections 2024-09-13T17:14:17.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.218+0000 7f674138c640 1 --2- 172.21.15.39:0/406289007 >> v2:172.21.15.39:3300/0 conn(0x7f673c06dc30 0x7f673c06fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:17.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.218+0000 7f674138c640 1 --2- 172.21.15.39:0/406289007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c0c10 0x7f673c06f9a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:17.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.222+0000 7f674138c640 1 -- 172.21.15.39:0/406289007 >> 172.21.15.39:0/406289007 conn(0x7f673c0a79a0 msgr2=0x7f673c0a7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:17.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.246+0000 7f674138c640 1 -- 172.21.15.39:0/406289007 shutdown_connections 2024-09-13T17:14:17.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.246+0000 7f674138c640 1 -- 172.21.15.39:0/406289007 wait complete. 2024-09-13T17:14:17.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.250+0000 7f674138c640 1 Processor -- start 2024-09-13T17:14:17.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.250+0000 7f674138c640 1 -- start start 2024-09-13T17:14:17.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.250+0000 7f674138c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f673c06dc30 0x7f673c1f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:17.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f674138c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f673c0c0c10 0x7f673c155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f674138c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c3af0 0x7f673c0a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f673b7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f673c0c0c10 0x7f673c155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f674138c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c158240 con 0x7f673c06dc30 2024-09-13T17:14:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f6740b8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c3af0 0x7f673c0a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f6740b8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c3af0 0x7f673c0a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:56396/0 (socket says 172.21.15.39:56396) 2024-09-13T17:14:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f6740b8b640 1 -- 172.21.15.39:0/4157600303 learned_addr learned my addr 172.21.15.39:0/4157600303 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f674138c640 1 -- 172.21.15.39:0/4157600303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c157a70 con 0x7f673c0c3af0 2024-09-13T17:14:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f674138c640 1 -- 172.21.15.39:0/4157600303 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c1f04e0 con 0x7f673c0c0c10 2024-09-13T17:14:17.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f673bfff640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f673c06dc30 0x7f673c1f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:17.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f673bfff640 1 -- 172.21.15.39:0/4157600303 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f673c0c0c10 msgr2=0x7f673c155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:17.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f673bfff640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f673c0c0c10 0x7f673c155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:17.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f673bfff640 1 -- 172.21.15.39:0/4157600303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c3af0 msgr2=0x7f673c0a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:17.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f673bfff640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c3af0 0x7f673c0a6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:17.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f673bfff640 1 -- 172.21.15.39:0/4157600303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c009420 con 0x7f673c06dc30 2024-09-13T17:14:17.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f673bfff640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f673c06dc30 0x7f673c1f0ef0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f672c0ab000 tx=0x7f672c0191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:17.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f6738ff9640 1 -- 172.21.15.39:0/4157600303 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c0b4400 con 0x7f673c06dc30 2024-09-13T17:14:17.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f674138c640 1 -- 172.21.15.39:0/4157600303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c1f1430 con 0x7f673c06dc30 2024-09-13T17:14:17.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.254+0000 7f674138c640 1 -- 172.21.15.39:0/4157600303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c157c80 con 0x7f673c06dc30 2024-09-13T17:14:17.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.258+0000 7f6738ff9640 1 -- 172.21.15.39:0/4157600303 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c0b4560 con 0x7f673c06dc30 2024-09-13T17:14:17.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.258+0000 7f6738ff9640 1 -- 172.21.15.39:0/4157600303 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c0b4910 con 0x7f673c06dc30 2024-09-13T17:14:17.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.262+0000 7f674138c640 1 -- 172.21.15.39:0/4157600303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c005260 con 0x7f673c06dc30 2024-09-13T17:14:17.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.262+0000 7f6738ff9640 1 -- 172.21.15.39:0/4157600303 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f672c0b2040 con 0x7f673c06dc30 2024-09-13T17:14:17.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.262+0000 7f6738ff9640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6728066690 0x7f6728068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:17.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.262+0000 7f6738ff9640 1 -- 172.21.15.39:0/4157600303 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7f672c11ec80 con 0x7f673c06dc30 2024-09-13T17:14:17.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.270+0000 7f673b7fe640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6728066690 0x7f6728068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:17.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.270+0000 7f673b7fe640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6728066690 0x7f6728068b50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f673000d630 tx=0x7f6730008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:17.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:17.282+0000 7f6738ff9640 1 -- 172.21.15.39:0/4157600303 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f672c125080 con 0x7f673c06dc30 2024-09-13T17:14:18.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.058+0000 7f674138c640 1 -- 172.21.15.39:0/4157600303 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f671c005a90 con 0x7f673c06dc30 2024-09-13T17:14:18.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.058+0000 7f6738ff9640 1 -- 172.21.15.39:0/4157600303 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f672c125080 con 0x7f673c06dc30 2024-09-13T17:14:18.065 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:18.065 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 182 2024-09-13T17:14:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.062+0000 7f67227fc640 1 -- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6728066690 msgr2=0x7f6728068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.062+0000 7f67227fc640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6728066690 0x7f6728068b50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f673000d630 tx=0x7f6730008040 comp rx=0 tx=0).stop 2024-09-13T17:14:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.062+0000 7f67227fc640 1 -- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f673c06dc30 msgr2=0x7f673c1f0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.062+0000 7f67227fc640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f673c06dc30 0x7f673c1f0ef0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f672c0ab000 tx=0x7f672c0191e0 comp rx=0 tx=0).stop 2024-09-13T17:14:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.062+0000 7f67227fc640 1 -- 172.21.15.39:0/4157600303 shutdown_connections 2024-09-13T17:14:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.062+0000 7f67227fc640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6728066690 0x7f6728068b50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.066+0000 7f67227fc640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f673c0c3af0 0x7f673c0a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.066+0000 7f67227fc640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f673c0c0c10 0x7f673c155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.066+0000 7f67227fc640 1 --2- 172.21.15.39:0/4157600303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f673c06dc30 0x7f673c1f0ef0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.066+0000 7f67227fc640 1 -- 172.21.15.39:0/4157600303 >> 172.21.15.39:0/4157600303 conn(0x7f673c0a79a0 msgr2=0x7f673c0c3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:18.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.070+0000 7f67227fc640 1 -- 172.21.15.39:0/4157600303 shutdown_connections 2024-09-13T17:14:18.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:18.070+0000 7f67227fc640 1 -- 172.21.15.39:0/4157600303 wait complete. 2024-09-13T17:14:18.148 INFO:tasks.mds_thrash.fs.[a]:mds.c reported in up:standby state 2024-09-13T17:14:18.148 INFO:tasks.mds_thrash.fs.[a]:waiting for 119 secs before thrashing 2024-09-13T17:14:31.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:31.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.986+0000 7f3592f5b640 1 Processor -- start 2024-09-13T17:14:31.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.986+0000 7f3592f5b640 1 -- start start 2024-09-13T17:14:31.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.994+0000 7f3591f59640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f358c0aebf0 0x7f358c157a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41912/0 (socket says 172.21.15.39:41912) 2024-09-13T17:14:31.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3592f5b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f358c0bcdb0 0x7f358c153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:31.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3592f5b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f358c070de0 0x7f358c0c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:31.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3592f5b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c158150 con 0x7f358c0aebf0 2024-09-13T17:14:31.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3592f5b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c0c5350 con 0x7f358c0bcdb0 2024-09-13T17:14:31.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3592f5b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1575b0 con 0x7f358c070de0 2024-09-13T17:14:31.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3591f59640 1 -- 172.21.15.39:0/2866439720 learned_addr learned my addr 172.21.15.39:0/2866439720 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:14:31.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648894025 0 0) 0x7f357c001120 con 0x7f358c0aebf0 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3578003530 con 0x7f358c0aebf0 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1844172116 0 0) 0x7f3578003530 con 0x7f358c0aebf0 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f358c070de0 msgr2=0x7f358c0c40a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 --2- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f358c070de0 0x7f358c0c40a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 >> v2:172.21.15.39:3300/0 conn(0x7f358c0bcdb0 msgr2=0x7f358c153bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 --2- 172.21.15.39:0/2866439720 >> v2:172.21.15.39:3300/0 conn(0x7f358c0bcdb0 0x7f358c153bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0c7990 con 0x7f358c0aebf0 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f357c0017f0 con 0x7f358c0aebf0 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 >> v1:172.21.15.39:6789/0 conn(0x7f358c0aebf0 legacy=0x7f358c157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 --2- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35780039e0 0x7f3578003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:32.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 --2- 172.21.15.39:0/2866439720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35780047d0 0x7f3578004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:32.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 --2- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3578005390 0x7f357800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:32.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1575b0 con 0x7f35780047d0 2024-09-13T17:14:32.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357800c260 con 0x7f35780039e0 2024-09-13T17:14:32.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357800c420 con 0x7f3578005390 2024-09-13T17:14:32.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f359275a640 1 --2- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3578005390 0x7f357800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:32.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f359275a640 1 -- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35780039e0 msgr2=0x7f3578003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:32.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f359275a640 1 --2- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35780039e0 0x7f3578003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f359275a640 1 -- 172.21.15.39:0/2866439720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35780047d0 msgr2=0x7f3578004c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:14:32.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f359275a640 1 --2- 172.21.15.39:0/2866439720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35780047d0 0x7f3578004c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:31.998+0000 7f359275a640 1 -- 172.21.15.39:0/2866439720 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0c7990 con 0x7f3578005390 2024-09-13T17:14:32.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.002+0000 7f359275a640 1 --2- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3578005390 0x7f357800b9d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f358c071290 tx=0x7f358401cdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=678fca5c84200a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:32.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.002+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840a5be0 con 0x7f3578005390 2024-09-13T17:14:32.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.002+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840a5d40 con 0x7f3578005390 2024-09-13T17:14:32.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.002+0000 7f3590f57640 1 -- 172.21.15.39:0/2866439720 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840ad680 con 0x7f3578005390 2024-09-13T17:14:32.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.014+0000 7f3592f5b640 1 -- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3578005390 msgr2=0x7f357800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:32.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.014+0000 7f3592f5b640 1 --2- 172.21.15.39:0/2866439720 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3578005390 0x7f357800b9d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f358c071290 tx=0x7f358401cdf0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.014+0000 7f359275a640 1 -- 172.21.15.39:0/2866439720 reap_dead start 2024-09-13T17:14:32.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.014+0000 7f3592f5b640 1 -- 172.21.15.39:0/2866439720 shutdown_connections 2024-09-13T17:14:32.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.014+0000 7f3592f5b640 1 -- 172.21.15.39:0/2866439720 >> 172.21.15.39:0/2866439720 conn(0x7f358c069fe0 msgr2=0x7f358c06a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:32.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.014+0000 7f3592f5b640 1 -- 172.21.15.39:0/2866439720 shutdown_connections 2024-09-13T17:14:32.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.014+0000 7f3592f5b640 1 -- 172.21.15.39:0/2866439720 wait complete. 2024-09-13T17:14:32.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.018+0000 7f3592f5b640 1 Processor -- start 2024-09-13T17:14:32.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.018+0000 7f3592f5b640 1 -- start start 2024-09-13T17:14:32.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f3592f5b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f358c1484d0 0x7f358c1ec750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f3592f5b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f358c1488b0 0x7f358c152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f3592f5b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f358c152fb0 0x7f358c14dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f3592f5b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c15a070 con 0x7f358c1484d0 2024-09-13T17:14:32.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f3592f5b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c150b00 con 0x7f358c152fb0 2024-09-13T17:14:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f3592f5b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1ecc90 con 0x7f358c1488b0 2024-09-13T17:14:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f359275a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f358c152fb0 0x7f358c14dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f359275a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f358c152fb0 0x7f358c14dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:37350/0 (socket says 172.21.15.39:37350) 2024-09-13T17:14:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f359275a640 1 -- 172.21.15.39:0/2396542285 learned_addr learned my addr 172.21.15.39:0/2396542285 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:32.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f359275a640 1 -- 172.21.15.39:0/2396542285 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f358c1488b0 msgr2=0x7f358c152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f359275a640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f358c1488b0 0x7f358c152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f359275a640 1 -- 172.21.15.39:0/2396542285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f358c1484d0 msgr2=0x7f358c1ec750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f359275a640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f358c1484d0 0x7f358c1ec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f359275a640 1 -- 172.21.15.39:0/2396542285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c150090 con 0x7f358c152fb0 2024-09-13T17:14:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f359275a640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f358c152fb0 0x7f358c14dc50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3584098900 tx=0x7f3578005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f35827fc640 1 -- 172.21.15.39:0/2396542285 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840a5be0 con 0x7f358c152fb0 2024-09-13T17:14:32.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c150820 con 0x7f358c152fb0 2024-09-13T17:14:32.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c14fd20 con 0x7f358c152fb0 2024-09-13T17:14:32.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f35827fc640 1 -- 172.21.15.39:0/2396542285 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840adba0 con 0x7f358c152fb0 2024-09-13T17:14:32.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.022+0000 7f35827fc640 1 -- 172.21.15.39:0/2396542285 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840b7410 con 0x7f358c152fb0 2024-09-13T17:14:32.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.026+0000 7f35827fc640 1 -- 172.21.15.39:0/2396542285 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f35840b4050 con 0x7f358c152fb0 2024-09-13T17:14:32.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.026+0000 7f35827fc640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3570066790 0x7f3570068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:32.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.026+0000 7f3591f59640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3570066790 0x7f3570068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:32.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.030+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3574005260 con 0x7f358c152fb0 2024-09-13T17:14:32.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.030+0000 7f3591f59640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3570066790 0x7f3570068c50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f357c007b70 tx=0x7f357c001ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:32.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.030+0000 7f35827fc640 1 -- 172.21.15.39:0/2396542285 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7f358411f300 con 0x7f358c152fb0 2024-09-13T17:14:32.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.038+0000 7f35827fc640 1 -- 172.21.15.39:0/2396542285 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f35840eb700 con 0x7f358c152fb0 2024-09-13T17:14:32.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:14:32.156 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:14:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.726+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3574005a90 con 0x7f358c152fb0 2024-09-13T17:14:32.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.726+0000 7f35827fc640 1 -- 172.21.15.39:0/2396542285 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f35840b2090 con 0x7f358c152fb0 2024-09-13T17:14:32.730 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:32.730 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:32.730 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 182 2024-09-13T17:14:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.730+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3570066790 msgr2=0x7f3570068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.730+0000 7f3592f5b640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3570066790 0x7f3570068c50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f357c007b70 tx=0x7f357c001ba0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.730+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f358c152fb0 msgr2=0x7f358c14dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.730+0000 7f3592f5b640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f358c152fb0 0x7f358c14dc50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3584098900 tx=0x7f3578005800 comp rx=0 tx=0).stop 2024-09-13T17:14:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.730+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 shutdown_connections 2024-09-13T17:14:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.730+0000 7f3592f5b640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3570066790 0x7f3570068c50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.730+0000 7f3592f5b640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f358c152fb0 0x7f358c14dc50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.730+0000 7f3592f5b640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f358c1488b0 0x7f358c152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.734+0000 7f3592f5b640 1 --2- 172.21.15.39:0/2396542285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f358c1484d0 0x7f358c1ec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.734+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 >> 172.21.15.39:0/2396542285 conn(0x7f358c069fe0 msgr2=0x7f358c14cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.734+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 shutdown_connections 2024-09-13T17:14:32.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:32.734+0000 7f3592f5b640 1 -- 172.21.15.39:0/2396542285 wait complete. 2024-09-13T17:14:32.796 INFO:tasks.mds_thrash.fs.[b]:kill mds.a (rank=0) 2024-09-13T17:14:32.796 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:14:32.796 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:14:32.834 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:14:32.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.398+0000 7f2e67fff640 1 Processor -- start 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.398+0000 7f2e67fff640 1 -- start start 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e600d5210 0x7f2e60049e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e67fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2e600d8630 0x7f2e6000c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e67fff640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e600d81a0 con 0x7f2e60047da0 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e67fff640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e60049270 con 0x7f2e600d8630 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e67fff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6002ce00 con 0x7f2e600d5210 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e677fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2e60047da0 0x7f2e600db0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41916/0 (socket says 172.21.15.39:41916) 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e677fe640 1 -- 172.21.15.39:0/2129144837 learned_addr learned my addr 172.21.15.39:0/2129144837 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e667fc640 1 --2- 172.21.15.39:0/2129144837 >> v2:172.21.15.39:3300/0 conn(0x7f2e600d8630 0x7f2e6000c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e66ffd640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e600d5210 0x7f2e60049e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:33.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737106955 0 0) 0x7f2e600d81a0 con 0x7f2e60047da0 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e54003530 con 0x7f2e60047da0 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 48793679 0 0) 0x7f2e54003530 con 0x7f2e60047da0 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e600d5210 msgr2=0x7f2e60049e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e600d5210 0x7f2e60049e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 >> v2:172.21.15.39:3300/0 conn(0x7f2e600d8630 msgr2=0x7f2e6000c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 --2- 172.21.15.39:0/2129144837 >> v2:172.21.15.39:3300/0 conn(0x7f2e600d8630 0x7f2e6000c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e600d8a50 con 0x7f2e60047da0 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f2e6804e740 con 0x7f2e60047da0 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 >> v1:172.21.15.39:6789/0 conn(0x7f2e60047da0 legacy=0x7f2e600db0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:33.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e540039b0 0x7f2e54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e54004350 0x7f2e54004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e54004e50 0x7f2e5402d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e60049270 con 0x7f2e54004e50 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6002ce00 con 0x7f2e540039b0 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5402dee0 con 0x7f2e54004350 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e677fe640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e54004e50 0x7f2e5402d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e677fe640 1 -- 172.21.15.39:0/2129144837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e54004350 msgr2=0x7f2e54004830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e677fe640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e54004350 0x7f2e54004830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e677fe640 1 -- 172.21.15.39:0/2129144837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e540039b0 msgr2=0x7f2e54003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e677fe640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e540039b0 0x7f2e54003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.402+0000 7f2e677fe640 1 -- 172.21.15.39:0/2129144837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e54003530 con 0x7f2e54004e50 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.410+0000 7f2e677fe640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e54004e50 0x7f2e5402d9a0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2e6804f070 tx=0x7f2e6808e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a856790ee918cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.410+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e681186c0 con 0x7f2e54004e50 2024-09-13T17:14:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.410+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e68118860 con 0x7f2e54004e50 2024-09-13T17:14:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.410+0000 7f2e65ffb640 1 -- 172.21.15.39:0/2129144837 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68118c90 con 0x7f2e54004e50 2024-09-13T17:14:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.410+0000 7f2e67fff640 1 -- 172.21.15.39:0/2129144837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e54004e50 msgr2=0x7f2e5402d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.410+0000 7f2e67fff640 1 --2- 172.21.15.39:0/2129144837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e54004e50 0x7f2e5402d9a0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2e6804f070 tx=0x7f2e6808e8f0 comp rx=0 tx=0).stop 2024-09-13T17:14:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.410+0000 7f2e677fe640 1 -- 172.21.15.39:0/2129144837 reap_dead start 2024-09-13T17:14:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 -- 172.21.15.39:0/2129144837 shutdown_connections 2024-09-13T17:14:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 -- 172.21.15.39:0/2129144837 >> 172.21.15.39:0/2129144837 conn(0x7f2e6002f010 msgr2=0x7f2e6002f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:33.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 -- 172.21.15.39:0/2129144837 shutdown_connections 2024-09-13T17:14:33.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 -- 172.21.15.39:0/2129144837 wait complete. 2024-09-13T17:14:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 Processor -- start 2024-09-13T17:14:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 -- start start 2024-09-13T17:14:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e54004e50 0x7f2e6016c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e600db660 0x7f2e600da780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e600dacc0 0x7f2e600cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e600d96e0 con 0x7f2e600dacc0 2024-09-13T17:14:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e600d99c0 con 0x7f2e600db660 2024-09-13T17:14:33.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.418+0000 7f2e67fff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e600d9ca0 con 0x7f2e54004e50 2024-09-13T17:14:33.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e667fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e600db660 0x7f2e600da780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:33.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e667fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e600db660 0x7f2e600da780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:37356/0 (socket says 172.21.15.39:37356) 2024-09-13T17:14:33.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e667fc640 1 -- 172.21.15.39:0/3829499993 learned_addr learned my addr 172.21.15.39:0/3829499993 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:33.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e677fe640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e600dacc0 0x7f2e600cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:33.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e677fe640 1 -- 172.21.15.39:0/3829499993 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e54004e50 msgr2=0x7f2e6016c050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:14:33.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e677fe640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e54004e50 0x7f2e6016c050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:33.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e677fe640 1 -- 172.21.15.39:0/3829499993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e600db660 msgr2=0x7f2e600da780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:33.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e677fe640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e600db660 0x7f2e600da780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:33.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e677fe640 1 -- 172.21.15.39:0/3829499993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e600d9f80 con 0x7f2e600dacc0 2024-09-13T17:14:33.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e677fe640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e600dacc0 0x7f2e600cdd50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2e6804e960 tx=0x7f2e68065a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:33.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e5ffff640 1 -- 172.21.15.39:0/3829499993 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68086550 con 0x7f2e600dacc0 2024-09-13T17:14:33.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e5ffff640 1 -- 172.21.15.39:0/3829499993 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e680866b0 con 0x7f2e600dacc0 2024-09-13T17:14:33.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e67fff640 1 -- 172.21.15.39:0/3829499993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e600da260 con 0x7f2e600dacc0 2024-09-13T17:14:33.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e5ffff640 1 -- 172.21.15.39:0/3829499993 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68086a20 con 0x7f2e600dacc0 2024-09-13T17:14:33.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e67fff640 1 -- 172.21.15.39:0/3829499993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e600d9300 con 0x7f2e600dacc0 2024-09-13T17:14:33.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.422+0000 7f2e67fff640 1 -- 172.21.15.39:0/3829499993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e600d7a60 con 0x7f2e600dacc0 2024-09-13T17:14:33.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.430+0000 7f2e5ffff640 1 -- 172.21.15.39:0/3829499993 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e6807f610 con 0x7f2e600dacc0 2024-09-13T17:14:33.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.430+0000 7f2e5ffff640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2e48066690 0x7f2e48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:33.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.430+0000 7f2e5ffff640 1 -- 172.21.15.39:0/3829499993 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7f2e68188370 con 0x7f2e600dacc0 2024-09-13T17:14:33.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.430+0000 7f2e5ffff640 1 -- 172.21.15.39:0/3829499993 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2e681546d0 con 0x7f2e600dacc0 2024-09-13T17:14:33.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.438+0000 7f2e66ffd640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2e48066690 0x7f2e48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:33.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:33.450+0000 7f2e66ffd640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2e48066690 0x7f2e48068b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000f1a0 tx=0x7f2e50006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:34.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.141+0000 7f2e67fff640 1 -- 172.21.15.39:0/3829499993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e6002fb30 con 0x7f2e600dacc0 2024-09-13T17:14:34.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.145+0000 7f2e5ffff640 1 -- 172.21.15.39:0/3829499993 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f2e68125070 con 0x7f2e600dacc0 2024-09-13T17:14:34.150 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:34.150 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:34.150 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 182 2024-09-13T17:14:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 -- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2e48066690 msgr2=0x7f2e48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2e48066690 0x7f2e48068b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000f1a0 tx=0x7f2e50006f90 comp rx=0 tx=0).stop 2024-09-13T17:14:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 -- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e600dacc0 msgr2=0x7f2e600cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e600dacc0 0x7f2e600cdd50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2e6804e960 tx=0x7f2e68065a90 comp rx=0 tx=0).stop 2024-09-13T17:14:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 -- 172.21.15.39:0/3829499993 shutdown_connections 2024-09-13T17:14:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2e48066690 0x7f2e48068b50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e600db660 0x7f2e600da780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e600dacc0 0x7f2e600cdd50 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 --2- 172.21.15.39:0/3829499993 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2e54004e50 0x7f2e6016c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 -- 172.21.15.39:0/3829499993 >> 172.21.15.39:0/3829499993 conn(0x7f2e6002f010 msgr2=0x7f2e6002d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 -- 172.21.15.39:0/3829499993 shutdown_connections 2024-09-13T17:14:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:34.153+0000 7f2e5dffb640 1 -- 172.21.15.39:0/3829499993 wait complete. 2024-09-13T17:14:34.203 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:14:36.204 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:36.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.581+0000 7f10fcd9e640 1 Processor -- start 2024-09-13T17:14:36.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.581+0000 7f10fcd9e640 1 -- start start 2024-09-13T17:14:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10fcd9e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f10f00d1a20 0x7f10f0047e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10fcd9e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f10f0040b90 0x7f10f002efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:36.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10fcd9e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00483c0 0x7f10f002f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10fcd9e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00491c0 con 0x7f10f00d1a20 2024-09-13T17:14:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10fcd9e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00d1e00 con 0x7f10f0040b90 2024-09-13T17:14:36.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10fcd9e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00daf30 con 0x7f10f00483c0 2024-09-13T17:14:36.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f6d9d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00483c0 0x7f10f002f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:36.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f6d9d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00483c0 0x7f10f002f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:35838/0 (socket says 172.21.15.39:35838) 2024-09-13T17:14:36.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f6d9d640 1 -- 172.21.15.39:0/2980176456 learned_addr learned my addr 172.21.15.39:0/2980176456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:36.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f5d9b640 1 --2- 172.21.15.39:0/2980176456 >> v2:172.21.15.84:3300/0 conn(0x7f10f0040b90 0x7f10f002efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:36.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f659c640 1 --2- 172.21.15.39:0/2980176456 >> v2:172.21.15.39:3300/0 conn(0x7f10f00d1a20 0x7f10f0047e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:36.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f6d9d640 1 -- 172.21.15.39:0/2980176456 >> v2:172.21.15.84:3300/0 conn(0x7f10f0040b90 msgr2=0x7f10f002efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:36.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f6d9d640 1 --2- 172.21.15.39:0/2980176456 >> v2:172.21.15.84:3300/0 conn(0x7f10f0040b90 0x7f10f002efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:36.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f6d9d640 1 -- 172.21.15.39:0/2980176456 >> v2:172.21.15.39:3300/0 conn(0x7f10f00d1a20 msgr2=0x7f10f0047e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:36.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f6d9d640 1 --2- 172.21.15.39:0/2980176456 >> v2:172.21.15.39:3300/0 conn(0x7f10f00d1a20 0x7f10f0047e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:36.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f6d9d640 1 -- 172.21.15.39:0/2980176456 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00db210 con 0x7f10f00483c0 2024-09-13T17:14:36.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.597+0000 7f10f659c640 1 --2- 172.21.15.39:0/2980176456 >> v2:172.21.15.39:3300/0 conn(0x7f10f00d1a20 0x7f10f0047e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:14:36.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.601+0000 7f10f6d9d640 1 --2- 172.21.15.39:0/2980176456 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00483c0 0x7f10f002f520 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10f8068a40 tx=0x7f10f8096910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64564de7aad1aa95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:36.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.613+0000 7f10f559a640 1 -- 172.21.15.39:0/2980176456 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f81207c0 con 0x7f10f00483c0 2024-09-13T17:14:36.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.613+0000 7f10f559a640 1 -- 172.21.15.39:0/2980176456 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f8120920 con 0x7f10f00483c0 2024-09-13T17:14:36.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.613+0000 7f10f559a640 1 -- 172.21.15.39:0/2980176456 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8120d10 con 0x7f10f00483c0 2024-09-13T17:14:36.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.613+0000 7f10fcd9e640 1 -- 172.21.15.39:0/2980176456 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00483c0 msgr2=0x7f10f002f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:36.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.613+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/2980176456 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00483c0 0x7f10f002f520 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10f8068a40 tx=0x7f10f8096910 comp rx=0 tx=0).stop 2024-09-13T17:14:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.621+0000 7f10fcd9e640 1 -- 172.21.15.39:0/2980176456 shutdown_connections 2024-09-13T17:14:36.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.621+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/2980176456 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00483c0 0x7f10f002f520 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:36.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.621+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/2980176456 >> v2:172.21.15.84:3300/0 conn(0x7f10f0040b90 0x7f10f002efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:36.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.621+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/2980176456 >> v2:172.21.15.39:3300/0 conn(0x7f10f00d1a20 0x7f10f0047e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:36.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.621+0000 7f10fcd9e640 1 -- 172.21.15.39:0/2980176456 >> 172.21.15.39:0/2980176456 conn(0x7f10f004d500 msgr2=0x7f10f004d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:36.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.625+0000 7f10fcd9e640 1 -- 172.21.15.39:0/2980176456 shutdown_connections 2024-09-13T17:14:36.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.625+0000 7f10fcd9e640 1 -- 172.21.15.39:0/2980176456 wait complete. 2024-09-13T17:14:36.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.625+0000 7f10fcd9e640 1 Processor -- start 2024-09-13T17:14:36.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.625+0000 7f10fcd9e640 1 -- start start 2024-09-13T17:14:36.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10fcd9e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f0040b90 0x7f10f0042c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:36.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10fcd9e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f00483c0 0x7f10f0043660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:36.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10fcd9e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00d1a20 0x7f10f0041ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:36.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10fcd9e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0040f70 con 0x7f10f00483c0 2024-09-13T17:14:36.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10f5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f00483c0 0x7f10f0043660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:36.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10fcd9e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00445d0 con 0x7f10f0040b90 2024-09-13T17:14:36.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10fcd9e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00ce930 con 0x7f10f00d1a20 2024-09-13T17:14:36.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10f5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f00483c0 0x7f10f0043660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45852/0 (socket says 172.21.15.39:45852) 2024-09-13T17:14:36.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10f5d9b640 1 -- 172.21.15.39:0/1932418395 learned_addr learned my addr 172.21.15.39:0/1932418395 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:36.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10f5d9b640 1 -- 172.21.15.39:0/1932418395 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00d1a20 msgr2=0x7f10f0041ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:14:36.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10f5d9b640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00d1a20 0x7f10f0041ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:36.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10f5d9b640 1 -- 172.21.15.39:0/1932418395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f0040b90 msgr2=0x7f10f0042c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:36.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10f5d9b640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f0040b90 0x7f10f0042c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:36.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.629+0000 7f10f5d9b640 1 -- 172.21.15.39:0/1932418395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8009420 con 0x7f10f00483c0 2024-09-13T17:14:36.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.633+0000 7f10f5d9b640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f00483c0 0x7f10f0043660 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00eb10 tx=0x7f10ec00c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:36.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.633+0000 7f10e6ffd640 1 -- 172.21.15.39:0/1932418395 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00c950 con 0x7f10f00483c0 2024-09-13T17:14:36.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.633+0000 7f10e6ffd640 1 -- 172.21.15.39:0/1932418395 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec00cab0 con 0x7f10f00483c0 2024-09-13T17:14:36.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.633+0000 7f10e6ffd640 1 -- 172.21.15.39:0/1932418395 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00ce20 con 0x7f10f00483c0 2024-09-13T17:14:36.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.637+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f00411c0 con 0x7f10f00483c0 2024-09-13T17:14:36.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.637+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f0041780 con 0x7f10f00483c0 2024-09-13T17:14:36.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.637+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f002f290 con 0x7f10f00483c0 2024-09-13T17:14:36.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.649+0000 7f10e6ffd640 1 -- 172.21.15.39:0/1932418395 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f10ec013470 con 0x7f10f00483c0 2024-09-13T17:14:36.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.649+0000 7f10e6ffd640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10d80666c0 0x7f10d8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:36.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.649+0000 7f10e6ffd640 1 -- 172.21.15.39:0/1932418395 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7f10ec085aa0 con 0x7f10f00483c0 2024-09-13T17:14:36.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.649+0000 7f10f659c640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10d80666c0 0x7f10d8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:36.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.649+0000 7f10f659c640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10d80666c0 0x7f10d8068b80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f10f00d1740 tx=0x7f10e8009610 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:36.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:36.657+0000 7f10e6ffd640 1 -- 172.21.15.39:0/1932418395 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f10ec051f10 con 0x7f10f00483c0 2024-09-13T17:14:37.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.253+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10f0041970 con 0x7f10f00483c0 2024-09-13T17:14:37.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.253+0000 7f10e6ffd640 1 -- 172.21.15.39:0/1932418395 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f10ec021020 con 0x7f10f00483c0 2024-09-13T17:14:37.259 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:37.259 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:37.259 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 182 2024-09-13T17:14:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10d80666c0 msgr2=0x7f10d8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10d80666c0 0x7f10d8068b80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f10f00d1740 tx=0x7f10e8009610 comp rx=0 tx=0).stop 2024-09-13T17:14:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f00483c0 msgr2=0x7f10f0043660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f00483c0 0x7f10f0043660 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00eb10 tx=0x7f10ec00c150 comp rx=0 tx=0).stop 2024-09-13T17:14:37.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 shutdown_connections 2024-09-13T17:14:37.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f10d80666c0 0x7f10d8068b80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:37.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f10f00d1a20 0x7f10f0041ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:37.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10f00483c0 0x7f10f0043660 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:37.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 --2- 172.21.15.39:0/1932418395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10f0040b90 0x7f10f0042c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 >> 172.21.15.39:0/1932418395 conn(0x7f10f004d500 msgr2=0x7f10f004d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 shutdown_connections 2024-09-13T17:14:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:37.261+0000 7f10fcd9e640 1 -- 172.21.15.39:0/1932418395 wait complete. 2024-09-13T17:14:37.335 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:14:39.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a385a640 1 Processor -- start 2024-09-13T17:14:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a385a640 1 -- start start 2024-09-13T17:14:39.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a385a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc3940d0980 0x7fc394046d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a1df7640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc3940484c0 0x7fc394008710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41926/0 (socket says 172.21.15.39:41926) 2024-09-13T17:14:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a1df7640 1 -- 172.21.15.39:0/3903091781 learned_addr learned my addr 172.21.15.39:0/3903091781 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:14:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a385a640 1 -- 172.21.15.39:0/3903091781 --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3940d6c70 con 0x7fc3940484c0 2024-09-13T17:14:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a385a640 1 -- 172.21.15.39:0/3903091781 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc39403fc00 con 0x7fc39404a2d0 2024-09-13T17:14:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a385a640 1 -- 172.21.15.39:0/3903091781 --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc394007800 con 0x7fc3940d0980 2024-09-13T17:14:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc393fff640 1 -- 172.21.15.39:0/3903091781 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702863987 0 0) 0x7fc3940d6c70 con 0x7fc3940484c0 2024-09-13T17:14:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc393fff640 1 -- 172.21.15.39:0/3903091781 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc388003530 con 0x7fc3940484c0 2024-09-13T17:14:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a15f6640 1 --2- 172.21.15.39:0/3903091781 >> v2:172.21.15.84:3300/0 conn(0x7fc3940d0980 0x7fc394046d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:39.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a15f6640 1 -- 172.21.15.39:0/3903091781 >> v1:172.21.15.84:6789/0 conn(0x7fc39404a2d0 legacy=0x7fc394008000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:39.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a15f6640 1 -- 172.21.15.39:0/3903091781 >> v1:172.21.15.39:6789/0 conn(0x7fc3940484c0 legacy=0x7fc394008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:39.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc3a15f6640 1 -- 172.21.15.39:0/3903091781 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394007ae0 con 0x7fc3940d0980 2024-09-13T17:14:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.593+0000 7fc393fff640 1 -- 172.21.15.39:0/3903091781 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 659937150 0 0) 0x7fc388003530 con 0x7fc3940484c0 2024-09-13T17:14:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.601+0000 7fc3a15f6640 1 --2- 172.21.15.39:0/3903091781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3940d0980 0x7fc394046d50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0097c0 tx=0x7fc38c01bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8f14b068f2761fc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:39.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.601+0000 7fc393fff640 1 -- 172.21.15.39:0/3903091781 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c068160 con 0x7fc3940d0980 2024-09-13T17:14:39.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.601+0000 7fc393fff640 1 -- 172.21.15.39:0/3903091781 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c0a9470 con 0x7fc3940d0980 2024-09-13T17:14:39.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.601+0000 7fc393fff640 1 -- 172.21.15.39:0/3903091781 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0a9800 con 0x7fc3940d0980 2024-09-13T17:14:39.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.605+0000 7fc3a385a640 1 -- 172.21.15.39:0/3903091781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3940d0980 msgr2=0x7fc394046d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:39.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.605+0000 7fc3a385a640 1 --2- 172.21.15.39:0/3903091781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3940d0980 0x7fc394046d50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0097c0 tx=0x7fc38c01bf80 comp rx=0 tx=0).stop 2024-09-13T17:14:39.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.605+0000 7fc3a385a640 1 -- 172.21.15.39:0/3903091781 shutdown_connections 2024-09-13T17:14:39.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.605+0000 7fc3a385a640 1 --2- 172.21.15.39:0/3903091781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc3940d0980 0x7fc394046d50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:39.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.605+0000 7fc3a385a640 1 -- 172.21.15.39:0/3903091781 >> 172.21.15.39:0/3903091781 conn(0x7fc394048e50 msgr2=0x7fc394049260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:39.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.613+0000 7fc3a385a640 1 -- 172.21.15.39:0/3903091781 shutdown_connections 2024-09-13T17:14:39.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.613+0000 7fc3a385a640 1 -- 172.21.15.39:0/3903091781 wait complete. 2024-09-13T17:14:39.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.613+0000 7fc3a385a640 1 Processor -- start 2024-09-13T17:14:39.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.613+0000 7fc3a385a640 1 -- start start 2024-09-13T17:14:39.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.613+0000 7fc3a385a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3940484c0 0x7fc39402fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:39.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.613+0000 7fc3a385a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc39404a2d0 0x7fc394042330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:39.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.613+0000 7fc3a385a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3940d0980 0x7fc3940d1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.613+0000 7fc3a385a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940d4360 con 0x7fc3940484c0 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a385a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39402f8a0 con 0x7fc39404a2d0 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a385a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940311e0 con 0x7fc3940d0980 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a1df7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3940d0980 0x7fc3940d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a1df7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3940d0980 0x7fc3940d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:35860/0 (socket says 172.21.15.39:35860) 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a1df7640 1 -- 172.21.15.39:0/966483217 learned_addr learned my addr 172.21.15.39:0/966483217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a15f6640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3940484c0 0x7fc39402fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a1df7640 1 -- 172.21.15.39:0/966483217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc39404a2d0 msgr2=0x7fc394042330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a1df7640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc39404a2d0 0x7fc394042330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a1df7640 1 -- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3940484c0 msgr2=0x7fc39402fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a1df7640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3940484c0 0x7fc39402fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a1df7640 1 -- 172.21.15.39:0/966483217 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c009420 con 0x7fc3940d0980 2024-09-13T17:14:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a15f6640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3940484c0 0x7fc39402fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:14:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.617+0000 7fc3a1df7640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3940d0980 0x7fc3940d1000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc39c02c980 tx=0x7fc39c067a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.621+0000 7fc391ffb640 1 -- 172.21.15.39:0/966483217 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c06f630 con 0x7fc3940d0980 2024-09-13T17:14:39.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.621+0000 7fc3a385a640 1 -- 172.21.15.39:0/966483217 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3940d3da0 con 0x7fc3940d0980 2024-09-13T17:14:39.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.621+0000 7fc3a385a640 1 -- 172.21.15.39:0/966483217 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39402f530 con 0x7fc3940d0980 2024-09-13T17:14:39.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.621+0000 7fc391ffb640 1 -- 172.21.15.39:0/966483217 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c06f790 con 0x7fc3940d0980 2024-09-13T17:14:39.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.621+0000 7fc391ffb640 1 -- 172.21.15.39:0/966483217 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c06fb80 con 0x7fc3940d0980 2024-09-13T17:14:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.625+0000 7fc3a385a640 1 -- 172.21.15.39:0/966483217 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c005260 con 0x7fc3940d0980 2024-09-13T17:14:39.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.625+0000 7fc391ffb640 1 -- 172.21.15.39:0/966483217 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc39c06fce0 con 0x7fc3940d0980 2024-09-13T17:14:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.625+0000 7fc391ffb640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc388066a10 0x7fc388068ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.625+0000 7fc391ffb640 1 -- 172.21.15.39:0/966483217 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7fc39c0ff120 con 0x7fc3940d0980 2024-09-13T17:14:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.625+0000 7fc3a15f6640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc388066a10 0x7fc388068ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.629+0000 7fc3a15f6640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc388066a10 0x7fc388068ed0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0ad000 tx=0x7fc38c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:39.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:39.633+0000 7fc391ffb640 1 -- 172.21.15.39:0/966483217 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc39c0cb590 con 0x7fc3940d0980 2024-09-13T17:14:40.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.453+0000 7fc3a385a640 1 -- 172.21.15.39:0/966483217 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc37c005a90 con 0x7fc3940d0980 2024-09-13T17:14:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.453+0000 7fc391ffb640 1 -- 172.21.15.39:0/966483217 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5934 (secure 0 0 0) 0x7fc39c08f090 con 0x7fc3940d0980 2024-09-13T17:14:40.462 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:40.462 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 182 2024-09-13T17:14:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.461+0000 7fc3877fe640 1 -- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc388066a10 msgr2=0x7fc388068ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.461+0000 7fc3877fe640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc388066a10 0x7fc388068ed0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0ad000 tx=0x7fc38c0a6000 comp rx=0 tx=0).stop 2024-09-13T17:14:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.461+0000 7fc3877fe640 1 -- 172.21.15.39:0/966483217 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3940d0980 msgr2=0x7fc3940d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.461+0000 7fc3877fe640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3940d0980 0x7fc3940d1000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc39c02c980 tx=0x7fc39c067a80 comp rx=0 tx=0).stop 2024-09-13T17:14:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.461+0000 7fc3877fe640 1 -- 172.21.15.39:0/966483217 shutdown_connections 2024-09-13T17:14:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.461+0000 7fc3877fe640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc388066a10 0x7fc388068ed0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.461+0000 7fc3877fe640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc39404a2d0 0x7fc394042330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:40.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.461+0000 7fc3877fe640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3940d0980 0x7fc3940d1000 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.469+0000 7fc3877fe640 1 --2- 172.21.15.39:0/966483217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3940484c0 0x7fc39402fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.469+0000 7fc3877fe640 1 -- 172.21.15.39:0/966483217 >> 172.21.15.39:0/966483217 conn(0x7fc394048e50 msgr2=0x7fc39404a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.469+0000 7fc3877fe640 1 -- 172.21.15.39:0/966483217 shutdown_connections 2024-09-13T17:14:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:40.469+0000 7fc3877fe640 1 -- 172.21.15.39:0/966483217 wait complete. 2024-09-13T17:14:40.536 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:14:42.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:42.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fda795640 1 Processor -- start 2024-09-13T17:14:42.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fda795640 1 -- start start 2024-09-13T17:14:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fda795640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc046600 0x7f5fcc0d9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fda795640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fcc031230 con 0x7f5fcc005a30 2024-09-13T17:14:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fda795640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fcc0da160 con 0x7f5fcc0d3540 2024-09-13T17:14:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fda795640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc04b120 con 0x7f5fcc046600 2024-09-13T17:14:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fd9f94640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5fcc005a30 0x7f5fcc005ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56920/0 (socket says 172.21.15.39:56920) 2024-09-13T17:14:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fd9f94640 1 -- 172.21.15.39:0/3265612588 learned_addr learned my addr 172.21.15.39:0/3265612588 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:14:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fd8f92640 1 --2- 172.21.15.39:0/3265612588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc046600 0x7f5fcc0d9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fc7fff640 1 -- 172.21.15.39:0/3265612588 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153467086 0 0) 0x7f5fd4068160 con 0x7f5fcc005a30 2024-09-13T17:14:42.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fc7fff640 1 -- 172.21.15.39:0/3265612588 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc0003530 con 0x7f5fcc005a30 2024-09-13T17:14:42.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fc7fff640 1 -- 172.21.15.39:0/3265612588 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261892500 0 0) 0x7f5fcc0da160 con 0x7f5fcc0d3540 2024-09-13T17:14:42.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fc7fff640 1 -- 172.21.15.39:0/3265612588 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd4068160 con 0x7f5fcc0d3540 2024-09-13T17:14:42.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fd8f92640 1 -- 172.21.15.39:0/3265612588 >> v1:172.21.15.84:6789/0 conn(0x7f5fcc0d3540 legacy=0x7f5fcc00ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:42.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fd8f92640 1 -- 172.21.15.39:0/3265612588 >> v1:172.21.15.39:6789/0 conn(0x7f5fcc005a30 legacy=0x7f5fcc005ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:42.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.729+0000 7f5fd8f92640 1 -- 172.21.15.39:0/3265612588 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc0065b0 con 0x7f5fcc046600 2024-09-13T17:14:42.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.733+0000 7f5fd8f92640 1 --2- 172.21.15.39:0/3265612588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc046600 0x7f5fcc0d9c20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0da2f0 tx=0x7f5fd001e8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a108c0c00740273 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:42.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.733+0000 7f5fc7fff640 1 -- 172.21.15.39:0/3265612588 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00a7980 con 0x7f5fcc046600 2024-09-13T17:14:42.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.733+0000 7f5fc7fff640 1 -- 172.21.15.39:0/3265612588 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd00ab070 con 0x7f5fcc046600 2024-09-13T17:14:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.733+0000 7f5fc7fff640 1 -- 172.21.15.39:0/3265612588 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00a7d00 con 0x7f5fcc046600 2024-09-13T17:14:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.733+0000 7f5fda795640 1 -- 172.21.15.39:0/3265612588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc046600 msgr2=0x7f5fcc0d9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:42.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.733+0000 7f5fda795640 1 --2- 172.21.15.39:0/3265612588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc046600 0x7f5fcc0d9c20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0da2f0 tx=0x7f5fd001e8b0 comp rx=0 tx=0).stop 2024-09-13T17:14:42.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.737+0000 7f5fda795640 1 -- 172.21.15.39:0/3265612588 shutdown_connections 2024-09-13T17:14:42.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.737+0000 7f5fda795640 1 --2- 172.21.15.39:0/3265612588 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc046600 0x7f5fcc0d9c20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:42.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.737+0000 7f5fda795640 1 -- 172.21.15.39:0/3265612588 >> 172.21.15.39:0/3265612588 conn(0x7f5fcc047cf0 msgr2=0x7f5fcc048100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:42.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.737+0000 7f5fda795640 1 -- 172.21.15.39:0/3265612588 shutdown_connections 2024-09-13T17:14:42.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fda795640 1 -- 172.21.15.39:0/3265612588 wait complete. 2024-09-13T17:14:42.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fda795640 1 Processor -- start 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fda795640 1 -- start start 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fda795640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc005a30 0x7f5fcc044c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fda795640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fcc046600 0x7f5fcc042340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fda795640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc0d3540 0x7f5fcc0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fda795640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc03fe80 con 0x7f5fcc0d3540 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fda795640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc041940 con 0x7f5fcc046600 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fda795640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc041360 con 0x7f5fcc005a30 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fd9793640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc005a30 0x7f5fcc044c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fd9793640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc005a30 0x7f5fcc044c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:59098/0 (socket says 172.21.15.39:59098) 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fd9793640 1 -- 172.21.15.39:0/3500851784 learned_addr learned my addr 172.21.15.39:0/3500851784 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fd9793640 1 -- 172.21.15.39:0/3500851784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fcc046600 msgr2=0x7f5fcc042340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:14:42.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fd9793640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fcc046600 0x7f5fcc042340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fd9793640 1 -- 172.21.15.39:0/3500851784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc0d3540 msgr2=0x7f5fcc0cd680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fd9793640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc0d3540 0x7f5fcc0cd680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.741+0000 7f5fd9793640 1 -- 172.21.15.39:0/3500851784 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd000b6b0 con 0x7f5fcc005a30 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.745+0000 7f5fd9793640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc005a30 0x7f5fcc044c10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c470 tx=0x7f5fc800c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.745+0000 7f5fc5ffb640 1 -- 172.21.15.39:0/3500851784 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc80052d0 con 0x7f5fcc005a30 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.745+0000 7f5fc5ffb640 1 -- 172.21.15.39:0/3500851784 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc8005430 con 0x7f5fcc005a30 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.745+0000 7f5fc5ffb640 1 -- 172.21.15.39:0/3500851784 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8003840 con 0x7f5fcc005a30 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.745+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc03f8c0 con 0x7f5fcc005a30 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.745+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc0415d0 con 0x7f5fcc005a30 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.745+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fcc0d9c20 con 0x7f5fcc005a30 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.749+0000 7f5fc5ffb640 1 -- 172.21.15.39:0/3500851784 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5fc80039a0 con 0x7f5fcc005a30 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.749+0000 7f5fc5ffb640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fb8066710 0x7f5fb8068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:42.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.749+0000 7f5fc5ffb640 1 -- 172.21.15.39:0/3500851784 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7f5fc8084670 con 0x7f5fcc005a30 2024-09-13T17:14:42.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.749+0000 7f5fd8f92640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fb8066710 0x7f5fb8068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:42.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.749+0000 7f5fd8f92640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fb8066710 0x7f5fb8068bd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00b2000 tx=0x7f5fd00a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:42.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:42.757+0000 7f5fc5ffb640 1 -- 172.21.15.39:0/3500851784 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5fc8017080 con 0x7f5fcc005a30 2024-09-13T17:14:43.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.173+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fcc005f40 con 0x7f5fcc005a30 2024-09-13T17:14:43.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fc5ffb640 1 -- 172.21.15.39:0/3500851784 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f5fc8050a60 con 0x7f5fcc005a30 2024-09-13T17:14:43.179 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:43.179 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 182 2024-09-13T17:14:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fb8066710 msgr2=0x7f5fb8068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fb8066710 0x7f5fb8068bd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00b2000 tx=0x7f5fd00a9070 comp rx=0 tx=0).stop 2024-09-13T17:14:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc005a30 msgr2=0x7f5fcc044c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc005a30 0x7f5fcc044c10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c470 tx=0x7f5fc800c930 comp rx=0 tx=0).stop 2024-09-13T17:14:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 shutdown_connections 2024-09-13T17:14:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5fb8066710 0x7f5fb8068bd0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fcc046600 0x7f5fcc042340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc0d3540 0x7f5fcc0cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:43.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 --2- 172.21.15.39:0/3500851784 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5fcc005a30 0x7f5fcc044c10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:43.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 >> 172.21.15.39:0/3500851784 conn(0x7f5fcc047cf0 msgr2=0x7f5fcc02c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:43.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.177+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 shutdown_connections 2024-09-13T17:14:43.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:43.181+0000 7f5fda795640 1 -- 172.21.15.39:0/3500851784 wait complete. 2024-09-13T17:14:43.227 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:14:45.228 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfcd941640 1 Processor -- start 2024-09-13T17:14:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfcd941640 1 -- start start 2024-09-13T17:14:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfcd941640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc8155aa0 0x7fcfc80c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfcd941640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcfc80c78c0 0x7fcfc815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfcd941640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc81592d0 con 0x7fcfc80ca130 2024-09-13T17:14:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfcd941640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc80bd3b0 con 0x7fcfc80c78c0 2024-09-13T17:14:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfcd941640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc815c8a0 con 0x7fcfc8155aa0 2024-09-13T17:14:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfbffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcfc80c78c0 0x7fcfc815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfc77fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcfc80ca130 0x7fcfc815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56928/0 (socket says 172.21.15.39:56928) 2024-09-13T17:14:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfc77fe640 1 -- 172.21.15.39:0/357225004 learned_addr learned my addr 172.21.15.39:0/357225004 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:14:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfc6ffd640 1 --2- 172.21.15.39:0/357225004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc8155aa0 0x7fcfc80c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:45.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfc67fc640 1 -- 172.21.15.39:0/357225004 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847176488 0 0) 0x7fcfc81592d0 con 0x7fcfc80ca130 2024-09-13T17:14:45.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfc67fc640 1 -- 172.21.15.39:0/357225004 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb4003530 con 0x7fcfc80ca130 2024-09-13T17:14:45.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfc6ffd640 1 -- 172.21.15.39:0/357225004 >> v2:172.21.15.39:3300/0 conn(0x7fcfc80c78c0 msgr2=0x7fcfc815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:45.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfc6ffd640 1 --2- 172.21.15.39:0/357225004 >> v2:172.21.15.39:3300/0 conn(0x7fcfc80c78c0 0x7fcfc815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfc6ffd640 1 -- 172.21.15.39:0/357225004 >> v1:172.21.15.39:6789/0 conn(0x7fcfc80ca130 legacy=0x7fcfc815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:45.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.421+0000 7fcfc6ffd640 1 -- 172.21.15.39:0/357225004 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8158750 con 0x7fcfc8155aa0 2024-09-13T17:14:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.429+0000 7fcfc6ffd640 1 --2- 172.21.15.39:0/357225004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc8155aa0 0x7fcfc80c9bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00097c0 tx=0x7fcfb001c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dea7e36a45f013d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.429+0000 7fcfc67fc640 1 -- 172.21.15.39:0/357225004 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0004230 con 0x7fcfc8155aa0 2024-09-13T17:14:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.429+0000 7fcfc67fc640 1 -- 172.21.15.39:0/357225004 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb00a7050 con 0x7fcfc8155aa0 2024-09-13T17:14:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.429+0000 7fcfc67fc640 1 -- 172.21.15.39:0/357225004 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0003620 con 0x7fcfc8155aa0 2024-09-13T17:14:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.429+0000 7fcfcd941640 1 -- 172.21.15.39:0/357225004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc8155aa0 msgr2=0x7fcfc80c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.429+0000 7fcfcd941640 1 --2- 172.21.15.39:0/357225004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc8155aa0 0x7fcfc80c9bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00097c0 tx=0x7fcfb001c350 comp rx=0 tx=0).stop 2024-09-13T17:14:45.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.433+0000 7fcfcd941640 1 -- 172.21.15.39:0/357225004 shutdown_connections 2024-09-13T17:14:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.433+0000 7fcfcd941640 1 --2- 172.21.15.39:0/357225004 >> v2:172.21.15.39:3300/0 conn(0x7fcfc80c78c0 0x7fcfc815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.433+0000 7fcfcd941640 1 --2- 172.21.15.39:0/357225004 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc8155aa0 0x7fcfc80c9bf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.433+0000 7fcfcd941640 1 -- 172.21.15.39:0/357225004 >> 172.21.15.39:0/357225004 conn(0x7fcfc8069fe0 msgr2=0x7fcfc806a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.433+0000 7fcfcd941640 1 -- 172.21.15.39:0/357225004 shutdown_connections 2024-09-13T17:14:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.433+0000 7fcfcd941640 1 -- 172.21.15.39:0/357225004 wait complete. 2024-09-13T17:14:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.437+0000 7fcfcd941640 1 Processor -- start 2024-09-13T17:14:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.437+0000 7fcfcd941640 1 -- start start 2024-09-13T17:14:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.437+0000 7fcfcd941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc80c78c0 0x7fcfc80c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.437+0000 7fcfcd941640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc80ca130 0x7fcfc8153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:45.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.437+0000 7fcfc6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc80c78c0 0x7fcfc80c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:45.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.437+0000 7fcfc6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc80c78c0 0x7fcfc80c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:38064/0 (socket says 172.21.15.39:38064) 2024-09-13T17:14:45.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.437+0000 7fcfcd941640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfc8155aa0 0x7fcfc81541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:45.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.437+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc80ca130 0x7fcfc8153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:45.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.437+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc80ca130 0x7fcfc8153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:59122/0 (socket says 172.21.15.39:59122) 2024-09-13T17:14:45.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc6ffd640 1 -- 172.21.15.39:0/1428221639 learned_addr learned my addr 172.21.15.39:0/1428221639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:45.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc8153840 con 0x7fcfc8155aa0 2024-09-13T17:14:45.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc81534d0 con 0x7fcfc80c78c0 2024-09-13T17:14:45.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc80af690 con 0x7fcfc80ca130 2024-09-13T17:14:45.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc77fe640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfc8155aa0 0x7fcfc81541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc77fe640 1 -- 172.21.15.39:0/1428221639 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc80ca130 msgr2=0x7fcfc8153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc77fe640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc80ca130 0x7fcfc8153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc77fe640 1 -- 172.21.15.39:0/1428221639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc80c78c0 msgr2=0x7fcfc80c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc77fe640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc80c78c0 0x7fcfc80c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc77fe640 1 -- 172.21.15.39:0/1428221639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb0009420 con 0x7fcfc8155aa0 2024-09-13T17:14:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc77fe640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfc8155aa0 0x7fcfc81541a0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000cad0 tx=0x7fcfc000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc4ff9640 1 -- 172.21.15.39:0/1428221639 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000ab70 con 0x7fcfc8155aa0 2024-09-13T17:14:45.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc80af3b0 con 0x7fcfc8155aa0 2024-09-13T17:14:45.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc4ff9640 1 -- 172.21.15.39:0/1428221639 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc000acd0 con 0x7fcfc8155aa0 2024-09-13T17:14:45.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc8156600 con 0x7fcfc8155aa0 2024-09-13T17:14:45.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.441+0000 7fcfc4ff9640 1 -- 172.21.15.39:0/1428221639 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0012620 con 0x7fcfc8155aa0 2024-09-13T17:14:45.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.445+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc80674c0 con 0x7fcfc8155aa0 2024-09-13T17:14:45.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.457+0000 7fcfc4ff9640 1 -- 172.21.15.39:0/1428221639 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcfc0007070 con 0x7fcfc8155aa0 2024-09-13T17:14:45.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.461+0000 7fcfc4ff9640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfa8066690 0x7fcfa8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.461+0000 7fcfc6ffd640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfa8066690 0x7fcfa8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.461+0000 7fcfc4ff9640 1 -- 172.21.15.39:0/1428221639 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7fcfc0085310 con 0x7fcfc8155aa0 2024-09-13T17:14:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.461+0000 7fcfc6ffd640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfa8066690 0x7fcfa8068b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80c3fc0 tx=0x7fcfb00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.461+0000 7fcfc4ff9640 1 -- 172.21.15.39:0/1428221639 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcfc0085790 con 0x7fcfc8155aa0 2024-09-13T17:14:45.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.865+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfc8153980 con 0x7fcfc8155aa0 2024-09-13T17:14:45.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.869+0000 7fcfc4ff9640 1 -- 172.21.15.39:0/1428221639 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5934 (secure 0 0 0) 0x7fcfc8153980 con 0x7fcfc8155aa0 2024-09-13T17:14:45.871 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:45.871 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:45.872 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 182 2024-09-13T17:14:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfa8066690 msgr2=0x7fcfa8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfa8066690 0x7fcfa8068b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80c3fc0 tx=0x7fcfb00a6000 comp rx=0 tx=0).stop 2024-09-13T17:14:45.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfc8155aa0 msgr2=0x7fcfc81541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfc8155aa0 0x7fcfc81541a0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000cad0 tx=0x7fcfc000ced0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 shutdown_connections 2024-09-13T17:14:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfa8066690 0x7fcfa8068b50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfc8155aa0 0x7fcfc81541a0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfc80ca130 0x7fcfc8153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 --2- 172.21.15.39:0/1428221639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfc80c78c0 0x7fcfc80c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 >> 172.21.15.39:0/1428221639 conn(0x7fcfc8069fe0 msgr2=0x7fcfc80c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 shutdown_connections 2024-09-13T17:14:45.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:45.873+0000 7fcfcd941640 1 -- 172.21.15.39:0/1428221639 wait complete. 2024-09-13T17:14:45.904 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:14:47.905 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:48.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d5e59e640 1 Processor -- start 2024-09-13T17:14:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d5e59e640 1 -- start start 2024-09-13T17:14:48.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d5e59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d580c2db0 0x7f0d580bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:48.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d5e59e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d580c00b0 con 0x7f0d580611c0 2024-09-13T17:14:48.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d5e59e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d58153e70 con 0x7f0d5806cbc0 2024-09-13T17:14:48.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d5e59e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d580c0300 con 0x7f0d580c2db0 2024-09-13T17:14:48.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d57fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d580c2db0 0x7f0d580bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:48.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d57fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d580c2db0 0x7f0d580bf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36494/0 (socket says 172.21.15.39:36494) 2024-09-13T17:14:48.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d57fff640 1 -- 172.21.15.39:0/2361806327 learned_addr learned my addr 172.21.15.39:0/2361806327 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:48.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d57fff640 1 -- 172.21.15.39:0/2361806327 >> v1:172.21.15.84:6789/0 conn(0x7f0d5806cbc0 legacy=0x7f0d581512b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:14:48.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d57fff640 1 -- 172.21.15.39:0/2361806327 >> v1:172.21.15.39:6789/0 conn(0x7f0d580611c0 legacy=0x7f0d581545e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:14:48.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.089+0000 7f0d57fff640 1 -- 172.21.15.39:0/2361806327 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d580c05e0 con 0x7f0d580c2db0 2024-09-13T17:14:48.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.101+0000 7f0d57fff640 1 --2- 172.21.15.39:0/2361806327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580c2db0 0x7f0d580bf4f0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009ad0 tx=0x7f0d4801c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7297a1373755be55 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:48.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.101+0000 7f0d577fe640 1 -- 172.21.15.39:0/2361806327 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d480aa650 con 0x7f0d580c2db0 2024-09-13T17:14:48.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.101+0000 7f0d577fe640 1 -- 172.21.15.39:0/2361806327 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d480aa7b0 con 0x7f0d580c2db0 2024-09-13T17:14:48.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.101+0000 7f0d577fe640 1 -- 172.21.15.39:0/2361806327 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d480aab40 con 0x7f0d580c2db0 2024-09-13T17:14:48.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.105+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2361806327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580c2db0 msgr2=0x7f0d580bf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:48.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.105+0000 7f0d5e59e640 1 --2- 172.21.15.39:0/2361806327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580c2db0 0x7f0d580bf4f0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009ad0 tx=0x7f0d4801c620 comp rx=0 tx=0).stop 2024-09-13T17:14:48.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.105+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2361806327 shutdown_connections 2024-09-13T17:14:48.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.105+0000 7f0d5e59e640 1 --2- 172.21.15.39:0/2361806327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580c2db0 0x7f0d580bf4f0 secure :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009ad0 tx=0x7f0d4801c620 comp rx=0 tx=0).stop 2024-09-13T17:14:48.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.105+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2361806327 >> 172.21.15.39:0/2361806327 conn(0x7f0d58069e10 msgr2=0x7f0d5806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:48.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.109+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2361806327 shutdown_connections 2024-09-13T17:14:48.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.109+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2361806327 wait complete. 2024-09-13T17:14:48.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5e59e640 1 Processor -- start 2024-09-13T17:14:48.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5e59e640 1 -- start start 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5e59e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d580611c0 0x7f0d58157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5e59e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d5806cbc0 0x7f0d58155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5e59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580a9380 0x7f0d58157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d4ffff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d5806cbc0 0x7f0d58155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5cb3b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580a9380 0x7f0d58157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d4ffff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d5806cbc0 0x7f0d58155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:59130/0 (socket says 172.21.15.39:59130) 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d4ffff640 1 -- 172.21.15.39:0/2451290469 learned_addr learned my addr 172.21.15.39:0/2451290469 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d580611c0 0x7f0d58157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58157910 con 0x7f0d580a9380 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d580aa0b0 con 0x7f0d580611c0 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d581557a0 con 0x7f0d5806cbc0 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d4ffff640 1 -- 172.21.15.39:0/2451290469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d580611c0 msgr2=0x7f0d58157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:48.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d4ffff640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d580611c0 0x7f0d58157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:48.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d4ffff640 1 -- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580a9380 msgr2=0x7f0d58157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:48.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d4ffff640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580a9380 0x7f0d58157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:48.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d4ffff640 1 -- 172.21.15.39:0/2451290469 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d480094d0 con 0x7f0d5806cbc0 2024-09-13T17:14:48.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d5cb3b640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580a9380 0x7f0d58157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:14:48.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.113+0000 7f0d4ffff640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d5806cbc0 0x7f0d58155260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b4d0 tx=0x7f0d4000b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:48.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.117+0000 7f0d557fa640 1 -- 172.21.15.39:0/2451290469 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d40010870 con 0x7f0d5806cbc0 2024-09-13T17:14:48.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.117+0000 7f0d557fa640 1 -- 172.21.15.39:0/2451290469 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d400109d0 con 0x7f0d5806cbc0 2024-09-13T17:14:48.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.117+0000 7f0d557fa640 1 -- 172.21.15.39:0/2451290469 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d40010d80 con 0x7f0d5806cbc0 2024-09-13T17:14:48.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.117+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d580a9810 con 0x7f0d5806cbc0 2024-09-13T17:14:48.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.117+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d580a9dd0 con 0x7f0d5806cbc0 2024-09-13T17:14:48.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.117+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d58157660 con 0x7f0d5806cbc0 2024-09-13T17:14:48.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.121+0000 7f0d557fa640 1 -- 172.21.15.39:0/2451290469 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d40019450 con 0x7f0d5806cbc0 2024-09-13T17:14:48.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.121+0000 7f0d557fa640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d38066760 0x7f0d38068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:48.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.121+0000 7f0d57fff640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d38066760 0x7f0d38068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:48.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.121+0000 7f0d57fff640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d38066760 0x7f0d38068c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ae000 tx=0x7f0d480b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:48.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.121+0000 7f0d557fa640 1 -- 172.21.15.39:0/2451290469 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 9647+0+0 (secure 0 0 0) 0x7f0d40085270 con 0x7f0d5806cbc0 2024-09-13T17:14:48.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.125+0000 7f0d557fa640 1 -- 172.21.15.39:0/2451290469 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d40051700 con 0x7f0d5806cbc0 2024-09-13T17:14:48.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.561+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d580721f0 con 0x7f0d5806cbc0 2024-09-13T17:14:48.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.561+0000 7f0d557fa640 1 -- 172.21.15.39:0/2451290469 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f0d4001e090 con 0x7f0d5806cbc0 2024-09-13T17:14:48.568 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:48.568 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:12:23.135929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6503":{"gid":6503,"name":"a","rank":0,"incarnation":173,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6835/305129652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":305129652},{"type":"v1","addr":"172.21.15.39:6835","nonce":305129652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 182 2024-09-13T17:14:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d38066760 msgr2=0x7f0d38068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d38066760 0x7f0d38068c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ae000 tx=0x7f0d480b5040 comp rx=0 tx=0).stop 2024-09-13T17:14:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d5806cbc0 msgr2=0x7f0d58155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d5806cbc0 0x7f0d58155260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b4d0 tx=0x7f0d4000b990 comp rx=0 tx=0).stop 2024-09-13T17:14:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 shutdown_connections 2024-09-13T17:14:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0d38066760 0x7f0d38068c20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d580a9380 0x7f0d58157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0d5806cbc0 0x7f0d58155260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 --2- 172.21.15.39:0/2451290469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d580611c0 0x7f0d58157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 >> 172.21.15.39:0/2451290469 conn(0x7f0d58069e10 msgr2=0x7f0d58070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 shutdown_connections 2024-09-13T17:14:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:48.569+0000 7f0d5e59e640 1 -- 172.21.15.39:0/2451290469 wait complete. 2024-09-13T17:14:48.612 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:14:50.613 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3edbbc640 1 Processor -- start 2024-09-13T17:14:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3edbbc640 1 -- start start 2024-09-13T17:14:50.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3edbbc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 0x7ff3e80662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3edbbc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3e80c6270 0x7ff3e80bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3edbbc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e806ddf0 con 0x7ff3e80c2ee0 2024-09-13T17:14:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3edbbc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e80c4710 con 0x7ff3e80c6270 2024-09-13T17:14:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3edbbc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80c49d0 con 0x7ff3e8065ee0 2024-09-13T17:14:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3e7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3e80c6270 0x7ff3e80bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3e7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3e80c6270 0x7ff3e80bfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:33124/0 (socket says 172.21.15.39:33124) 2024-09-13T17:14:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3e7fff640 1 -- 172.21.15.39:0/3530438901 learned_addr learned my addr 172.21.15.39:0/3530438901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.973+0000 7ff3e6ffd640 1 --1- 172.21.15.39:0/3530438901 >> v1:172.21.15.84:6789/0 conn(0x7ff3e80c2ee0 0x7ff3e80c2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:45044/0 (socket says 172.21.15.39:45044) 2024-09-13T17:14:50.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.977+0000 7ff3e7fff640 1 -- 172.21.15.39:0/3530438901 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 msgr2=0x7ff3e80662c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:14:50.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.977+0000 7ff3e7fff640 1 --2- 172.21.15.39:0/3530438901 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 0x7ff3e80662c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:50.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.977+0000 7ff3e7fff640 1 -- 172.21.15.39:0/3530438901 >> v1:172.21.15.84:6789/0 conn(0x7ff3e80c2ee0 legacy=0x7ff3e80c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:50.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.977+0000 7ff3e7fff640 1 -- 172.21.15.39:0/3530438901 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e8154880 con 0x7ff3e80c6270 2024-09-13T17:14:50.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.977+0000 7ff3e7fff640 1 --2- 172.21.15.39:0/3530438901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80c6270 0x7ff3e80bfc40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009e50 tx=0x7ff3e001c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0f8f4430cf8adae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:50.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.977+0000 7ff3e67fc640 1 -- 172.21.15.39:0/3530438901 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00a5560 con 0x7ff3e80c6270 2024-09-13T17:14:50.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.977+0000 7ff3e67fc640 1 -- 172.21.15.39:0/3530438901 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e00a56c0 con 0x7ff3e80c6270 2024-09-13T17:14:50.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.977+0000 7ff3e67fc640 1 -- 172.21.15.39:0/3530438901 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00a5a50 con 0x7ff3e80c6270 2024-09-13T17:14:50.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.981+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3530438901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80c6270 msgr2=0x7ff3e80bfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:50.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.981+0000 7ff3edbbc640 1 --2- 172.21.15.39:0/3530438901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80c6270 0x7ff3e80bfc40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009e50 tx=0x7ff3e001c850 comp rx=0 tx=0).stop 2024-09-13T17:14:50.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.981+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3530438901 shutdown_connections 2024-09-13T17:14:50.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.981+0000 7ff3edbbc640 1 --2- 172.21.15.39:0/3530438901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80c6270 0x7ff3e80bfc40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:50.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.981+0000 7ff3edbbc640 1 --2- 172.21.15.39:0/3530438901 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 0x7ff3e80662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:50.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.981+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3530438901 >> 172.21.15.39:0/3530438901 conn(0x7ff3e80c3310 msgr2=0x7ff3e80c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:50.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.981+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3530438901 shutdown_connections 2024-09-13T17:14:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.981+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3530438901 wait complete. 2024-09-13T17:14:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3edbbc640 1 Processor -- start 2024-09-13T17:14:50.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3edbbc640 1 -- start start 2024-09-13T17:14:50.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3edbbc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 0x7ff3e8150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:50.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3edbbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e80c2ee0 0x7ff3e81f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:50.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3edbbc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80c6270 0x7ff3e81f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:50.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3edbbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80a9e40 con 0x7ff3e80c2ee0 2024-09-13T17:14:50.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3edbbc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e81f73e0 con 0x7ff3e80c6270 2024-09-13T17:14:50.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3edbbc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80c24f0 con 0x7ff3e8065ee0 2024-09-13T17:14:50.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 0x7ff3e8150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:50.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e77fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 0x7ff3e8150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:49216/0 (socket says 172.21.15.39:49216) 2024-09-13T17:14:50.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e77fe640 1 -- 172.21.15.39:0/3750262540 learned_addr learned my addr 172.21.15.39:0/3750262540 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:50.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e7fff640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80c6270 0x7ff3e81f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e77fe640 1 -- 172.21.15.39:0/3750262540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80c6270 msgr2=0x7ff3e81f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e77fe640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80c6270 0x7ff3e81f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e77fe640 1 -- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e80c2ee0 msgr2=0x7ff3e81f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e6ffd640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e80c2ee0 0x7ff3e81f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e77fe640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e80c2ee0 0x7ff3e81f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.985+0000 7ff3e77fe640 1 -- 172.21.15.39:0/3750262540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e0009b30 con 0x7ff3e8065ee0 2024-09-13T17:14:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3e77fe640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 0x7ff3e8150c20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800b4a0 tx=0x7ff3d800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3e6ffd640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e80c2ee0 0x7ff3e81f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:14:50.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3cffff640 1 -- 172.21.15.39:0/3750262540 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8010610 con 0x7ff3e8065ee0 2024-09-13T17:14:50.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3cffff640 1 -- 172.21.15.39:0/3750262540 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d8010790 con 0x7ff3e8065ee0 2024-09-13T17:14:50.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3cffff640 1 -- 172.21.15.39:0/3750262540 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8010b20 con 0x7ff3e8065ee0 2024-09-13T17:14:50.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e8156730 con 0x7ff3e8065ee0 2024-09-13T17:14:50.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e81fd630 con 0x7ff3e8065ee0 2024-09-13T17:14:50.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3cffff640 1 -- 172.21.15.39:0/3750262540 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3d8010c80 con 0x7ff3e8065ee0 2024-09-13T17:14:50.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3cffff640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3d4066710 0x7ff3d4068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:50.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3cffff640 1 -- 172.21.15.39:0/3750262540 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7ff3d8084bb0 con 0x7ff3e8065ee0 2024-09-13T17:14:50.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.989+0000 7ff3e6ffd640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3d4066710 0x7ff3d4068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:50.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:50.993+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e80662c0 con 0x7ff3e8065ee0 2024-09-13T17:14:51.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.001+0000 7ff3cffff640 1 -- 172.21.15.39:0/3750262540 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff3d801e080 con 0x7ff3e8065ee0 2024-09-13T17:14:51.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.005+0000 7ff3e6ffd640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3d4066710 0x7ff3d4068bd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80c2850 tx=0x7ff3dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:51.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.557+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3e81f96b0 con 0x7ff3e8065ee0 2024-09-13T17:14:51.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.561+0000 7ff3cffff640 1 -- 172.21.15.39:0/3750262540 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) v1 ==== 95+0+5179 (secure 0 0 0) 0x7ff3d8014020 con 0x7ff3e8065ee0 2024-09-13T17:14:51.563 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:51.563 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":183,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:49.920790+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:51.563 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 183 2024-09-13T17:14:51.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.561+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3d4066710 msgr2=0x7ff3d4068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:51.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.561+0000 7ff3edbbc640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3d4066710 0x7ff3d4068bd0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80c2850 tx=0x7ff3dc009040 comp rx=0 tx=0).stop 2024-09-13T17:14:51.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.561+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 msgr2=0x7ff3e8150c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:51.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.561+0000 7ff3edbbc640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 0x7ff3e8150c20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800b4a0 tx=0x7ff3d800b960 comp rx=0 tx=0).stop 2024-09-13T17:14:51.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.565+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 shutdown_connections 2024-09-13T17:14:51.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.565+0000 7ff3edbbc640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff3d4066710 0x7ff3d4068bd0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:51.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.565+0000 7ff3edbbc640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3e80c6270 0x7ff3e81f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:51.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.565+0000 7ff3edbbc640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e80c2ee0 0x7ff3e81f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:51.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.565+0000 7ff3edbbc640 1 --2- 172.21.15.39:0/3750262540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3e8065ee0 0x7ff3e8150c20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:51.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.565+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 >> 172.21.15.39:0/3750262540 conn(0x7ff3e80c3310 msgr2=0x7ff3e8069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:51.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.569+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 shutdown_connections 2024-09-13T17:14:51.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.569+0000 7ff3edbbc640 1 -- 172.21.15.39:0/3750262540 wait complete. 2024-09-13T17:14:51.620 INFO:tasks.mds_thrash.fs.[b]:mds.a down, removed from mdsmap 2024-09-13T17:14:51.621 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:14:51.621 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:51.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5aef51640 1 Processor -- start 2024-09-13T17:14:51.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5aef51640 1 -- start start 2024-09-13T17:14:51.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5aef51640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff5a80ab9e0 0x7ff5a80c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:51.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5aef51640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a8071bb0 0x7ff5a806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:51.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5aef51640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a8150e00 con 0x7ff5a80c0ac0 2024-09-13T17:14:51.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5aef51640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80abdc0 con 0x7ff5a80ab9e0 2024-09-13T17:14:51.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5aef51640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a806d830 con 0x7ff5a8071bb0 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5acced640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff5a80ab9e0 0x7ff5a80c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5acced640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff5a80ab9e0 0x7ff5a80c2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34064/0 (socket says 172.21.15.39:34064) 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5acced640 1 -- 172.21.15.39:0/3149156190 learned_addr learned my addr 172.21.15.39:0/3149156190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5a77fe640 1 -- 172.21.15.39:0/3149156190 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506890205 0 0) 0x7ff5a8150e00 con 0x7ff5a80c0ac0 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5a77fe640 1 -- 172.21.15.39:0/3149156190 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff598003530 con 0x7ff5a80c0ac0 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5a7fff640 1 --2- 172.21.15.39:0/3149156190 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a8071bb0 0x7ff5a806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5acced640 1 -- 172.21.15.39:0/3149156190 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a8071bb0 msgr2=0x7ff5a806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5acced640 1 --2- 172.21.15.39:0/3149156190 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a8071bb0 0x7ff5a806cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5acced640 1 -- 172.21.15.39:0/3149156190 >> v1:172.21.15.39:6789/0 conn(0x7ff5a80c0ac0 legacy=0x7ff5a806d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:51.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.821+0000 7ff5acced640 1 -- 172.21.15.39:0/3149156190 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a8150250 con 0x7ff5a80ab9e0 2024-09-13T17:14:51.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5acced640 1 --2- 172.21.15.39:0/3149156190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a80ab9e0 0x7ff5a80c2b50 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009870 tx=0x7ff59c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c7fa7494fb6097 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:51.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5a77fe640 1 -- 172.21.15.39:0/3149156190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0ad660 con 0x7ff5a80ab9e0 2024-09-13T17:14:51.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5a77fe640 1 -- 172.21.15.39:0/3149156190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c0ad7c0 con 0x7ff5a80ab9e0 2024-09-13T17:14:51.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5a77fe640 1 -- 172.21.15.39:0/3149156190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0adb50 con 0x7ff5a80ab9e0 2024-09-13T17:14:51.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5aef51640 1 -- 172.21.15.39:0/3149156190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a80ab9e0 msgr2=0x7ff5a80c2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:51.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5aef51640 1 --2- 172.21.15.39:0/3149156190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a80ab9e0 0x7ff5a80c2b50 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009870 tx=0x7ff59c01c560 comp rx=0 tx=0).stop 2024-09-13T17:14:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5aef51640 1 -- 172.21.15.39:0/3149156190 shutdown_connections 2024-09-13T17:14:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5aef51640 1 --2- 172.21.15.39:0/3149156190 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a8071bb0 0x7ff5a806cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5aef51640 1 --2- 172.21.15.39:0/3149156190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a80ab9e0 0x7ff5a80c2b50 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:51.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5aef51640 1 -- 172.21.15.39:0/3149156190 >> 172.21.15.39:0/3149156190 conn(0x7ff5a8069e10 msgr2=0x7ff5a806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:51.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5aef51640 1 -- 172.21.15.39:0/3149156190 shutdown_connections 2024-09-13T17:14:51.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.825+0000 7ff5aef51640 1 -- 172.21.15.39:0/3149156190 wait complete. 2024-09-13T17:14:51.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5aef51640 1 Processor -- start 2024-09-13T17:14:51.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5aef51640 1 -- start start 2024-09-13T17:14:51.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5aef51640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a8071bb0 0x7ff5a81457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:51.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5acced640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a8071bb0 0x7ff5a81457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:51.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5acced640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a8071bb0 0x7ff5a81457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34078/0 (socket says 172.21.15.39:34078) 2024-09-13T17:14:51.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5acced640 1 -- 172.21.15.39:0/163686865 learned_addr learned my addr 172.21.15.39:0/163686865 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:51.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5aef51640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5a80ab9e0 0x7ff5a814cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:51.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5aef51640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a80c0ac0 0x7ff5a814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:51.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5aef51640 1 -- 172.21.15.39:0/163686865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a814d190 con 0x7ff5a8071bb0 2024-09-13T17:14:51.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5aef51640 1 -- 172.21.15.39:0/163686865 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a81462b0 con 0x7ff5a80ab9e0 2024-09-13T17:14:51.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5aef51640 1 -- 172.21.15.39:0/163686865 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8145320 con 0x7ff5a80c0ac0 2024-09-13T17:14:51.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5ad4ee640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a80c0ac0 0x7ff5a814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5acced640 1 -- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a80c0ac0 msgr2=0x7ff5a814b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5acced640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a80c0ac0 0x7ff5a814b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5acced640 1 -- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5a80ab9e0 msgr2=0x7ff5a814cc50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5acced640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5a80ab9e0 0x7ff5a814cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.833+0000 7ff5acced640 1 -- 172.21.15.39:0/163686865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c0094d0 con 0x7ff5a8071bb0 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff5ad4ee640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a80c0ac0 0x7ff5a814b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff5acced640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a8071bb0 0x7ff5a81457b0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff59c019190 tx=0x7ff59c019320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff5a57fa640 1 -- 172.21.15.39:0/163686865 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0b7640 con 0x7ff5a8071bb0 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff5a57fa640 1 -- 172.21.15.39:0/163686865 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c0b77e0 con 0x7ff5a8071bb0 2024-09-13T17:14:51.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff5a57fa640 1 -- 172.21.15.39:0/163686865 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0b7b90 con 0x7ff5a8071bb0 2024-09-13T17:14:51.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff5aef51640 1 -- 172.21.15.39:0/163686865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a8145ff0 con 0x7ff5a8071bb0 2024-09-13T17:14:51.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff5aef51640 1 -- 172.21.15.39:0/163686865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a814d710 con 0x7ff5a8071bb0 2024-09-13T17:14:51.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff592ffd640 1 -- 172.21.15.39:0/163686865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff580005260 con 0x7ff5a8071bb0 2024-09-13T17:14:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff5a57fa640 1 -- 172.21.15.39:0/163686865 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff59c0b7cf0 con 0x7ff5a8071bb0 2024-09-13T17:14:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.837+0000 7ff5a57fa640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff58c066690 0x7ff58c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.841+0000 7ff5a57fa640 1 -- 172.21.15.39:0/163686865 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7ff59c11fa60 con 0x7ff5a8071bb0 2024-09-13T17:14:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.841+0000 7ff5a7fff640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff58c066690 0x7ff58c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.841+0000 7ff5a7fff640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff58c066690 0x7ff58c068b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff5a814fc90 tx=0x7ff594008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:51.853+0000 7ff5a57fa640 1 -- 172.21.15.39:0/163686865 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff59c0ebe50 con 0x7ff5a8071bb0 2024-09-13T17:14:52.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.337+0000 7ff592ffd640 1 -- 172.21.15.39:0/163686865 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff580005a90 con 0x7ff5a8071bb0 2024-09-13T17:14:52.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.337+0000 7ff5a57fa640 1 -- 172.21.15.39:0/163686865 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+5183 (secure 0 0 0) 0x7ff59c0b2090 con 0x7ff5a8071bb0 2024-09-13T17:14:52.341 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:52.341 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":184,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:51.932338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:52.341 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 184 2024-09-13T17:14:52.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 -- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff58c066690 msgr2=0x7ff58c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:52.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff58c066690 0x7ff58c068b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff5a814fc90 tx=0x7ff594008040 comp rx=0 tx=0).stop 2024-09-13T17:14:52.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 -- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a8071bb0 msgr2=0x7ff5a81457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:52.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a8071bb0 0x7ff5a81457b0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7ff59c019190 tx=0x7ff59c019320 comp rx=0 tx=0).stop 2024-09-13T17:14:52.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 -- 172.21.15.39:0/163686865 shutdown_connections 2024-09-13T17:14:52.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff58c066690 0x7ff58c068b50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:52.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5a80ab9e0 0x7ff5a814cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:52.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff5a80c0ac0 0x7ff5a814b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:52.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 --2- 172.21.15.39:0/163686865 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5a8071bb0 0x7ff5a81457b0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:52.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 -- 172.21.15.39:0/163686865 >> 172.21.15.39:0/163686865 conn(0x7ff5a8069e10 msgr2=0x7ff5a814ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:52.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 -- 172.21.15.39:0/163686865 shutdown_connections 2024-09-13T17:14:52.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:52.341+0000 7ff592ffd640 1 -- 172.21.15.39:0/163686865 wait complete. 2024-09-13T17:14:52.386 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 184, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 181, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6530": { "addr": "172.21.15.84:6835/380008541", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 380008541, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 380008541, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6530, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 27 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:14:08.379706+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6530 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 184, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_7375": { "addr": "172.21.15.39:6837/1355102409", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 1355102409, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 1355102409, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7375, "incarnation": 183, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 60, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:14:51.932338+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7375 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:14:54.386 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:14:54.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.625+0000 7f7a7955c640 1 Processor -- start 2024-09-13T17:14:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.625+0000 7f7a7955c640 1 -- start start 2024-09-13T17:14:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.629+0000 7f7a7955c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a74153cc0 0x7f7a740a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.629+0000 7f7a7955c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7a740a9130 0x7f7a740c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.629+0000 7f7a7955c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a740c5340 con 0x7f7a7406cc80 2024-09-13T17:14:54.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.629+0000 7f7a7955c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a740c1da0 con 0x7f7a740a9130 2024-09-13T17:14:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.629+0000 7f7a7955c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a740c5060 con 0x7f7a74153cc0 2024-09-13T17:14:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.629+0000 7f7a6bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7a740a9130 0x7f7a740c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:54.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.629+0000 7f7a6bfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7a740a9130 0x7f7a740c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:33146/0 (socket says 172.21.15.39:33146) 2024-09-13T17:14:54.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.629+0000 7f7a6bfff640 1 -- 172.21.15.39:0/2609332971 learned_addr learned my addr 172.21.15.39:0/2609332971 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:54.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.633+0000 7f7a6bfff640 1 -- 172.21.15.39:0/2609332971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a74153cc0 msgr2=0x7f7a740a8bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:14:54.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.633+0000 7f7a6bfff640 1 --2- 172.21.15.39:0/2609332971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a74153cc0 0x7f7a740a8bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:54.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.633+0000 7f7a6bfff640 1 -- 172.21.15.39:0/2609332971 >> v1:172.21.15.84:6789/0 conn(0x7f7a7406cc80 legacy=0x7f7a740c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:54.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.633+0000 7f7a6bfff640 1 -- 172.21.15.39:0/2609332971 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7406c1a0 con 0x7f7a740a9130 2024-09-13T17:14:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.641+0000 7f7a6bfff640 1 --2- 172.21.15.39:0/2609332971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a740a9130 0x7f7a740c3a20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0091e0 tx=0x7f7a5c01d780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a82250d7a94c5e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a727fc640 1 -- 172.21.15.39:0/2609332971 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0a6830 con 0x7f7a740a9130 2024-09-13T17:14:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a727fc640 1 -- 172.21.15.39:0/2609332971 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c0a6990 con 0x7f7a740a9130 2024-09-13T17:14:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a7955c640 1 -- 172.21.15.39:0/2609332971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a740a9130 msgr2=0x7f7a740c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a7955c640 1 --2- 172.21.15.39:0/2609332971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a740a9130 0x7f7a740c3a20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0091e0 tx=0x7f7a5c01d780 comp rx=0 tx=0).stop 2024-09-13T17:14:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a7955c640 1 -- 172.21.15.39:0/2609332971 shutdown_connections 2024-09-13T17:14:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a7955c640 1 --2- 172.21.15.39:0/2609332971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a740a9130 0x7f7a740c3a20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a7955c640 1 --2- 172.21.15.39:0/2609332971 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a74153cc0 0x7f7a740a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a7955c640 1 -- 172.21.15.39:0/2609332971 >> 172.21.15.39:0/2609332971 conn(0x7f7a740a72c0 msgr2=0x7f7a740a76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a7955c640 1 -- 172.21.15.39:0/2609332971 shutdown_connections 2024-09-13T17:14:54.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.645+0000 7f7a7955c640 1 -- 172.21.15.39:0/2609332971 wait complete. 2024-09-13T17:14:54.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.649+0000 7f7a7955c640 1 Processor -- start 2024-09-13T17:14:54.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a7955c640 1 -- start start 2024-09-13T17:14:54.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a7955c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a7406cc80 0x7f7a7420d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:54.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a7955c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a740a9130 0x7f7a7420d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:54.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a7955c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a74153cc0 0x7f7a74150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a7955c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a74156ef0 con 0x7f7a740a9130 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a7955c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a740bdd30 con 0x7f7a74153cc0 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a7955c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7420ddd0 con 0x7f7a7406cc80 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a7406cc80 0x7f7a7420d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a74153cc0 0x7f7a74150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a72ffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a7406cc80 0x7f7a7420d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:49242/0 (socket says 172.21.15.39:49242) 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.653+0000 7f7a72ffd640 1 -- 172.21.15.39:0/3691067424 learned_addr learned my addr 172.21.15.39:0/3691067424 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a72ffd640 1 -- 172.21.15.39:0/3691067424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a74153cc0 msgr2=0x7f7a74150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a72ffd640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a74153cc0 0x7f7a74150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a72ffd640 1 -- 172.21.15.39:0/3691067424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a740a9130 msgr2=0x7f7a7420d7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:14:54.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a72ffd640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a740a9130 0x7f7a7420d7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:54.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a72ffd640 1 -- 172.21.15.39:0/3691067424 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c009d60 con 0x7f7a7406cc80 2024-09-13T17:14:54.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a737fe640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a74153cc0 0x7f7a74150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:14:54.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a72ffd640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a7406cc80 0x7f7a7420d290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400b4a0 tx=0x7f7a6400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:54.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a6b7fe640 1 -- 172.21.15.39:0/3691067424 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64010610 con 0x7f7a7406cc80 2024-09-13T17:14:54.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7420e0b0 con 0x7f7a7406cc80 2024-09-13T17:14:54.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a740bd9c0 con 0x7f7a7406cc80 2024-09-13T17:14:54.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a6b7fe640 1 -- 172.21.15.39:0/3691067424 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a64010770 con 0x7f7a7406cc80 2024-09-13T17:14:54.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a6b7fe640 1 -- 172.21.15.39:0/3691067424 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64010b60 con 0x7f7a7406cc80 2024-09-13T17:14:54.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a740c1520 con 0x7f7a7406cc80 2024-09-13T17:14:54.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.657+0000 7f7a6b7fe640 1 -- 172.21.15.39:0/3691067424 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a64010cc0 con 0x7f7a7406cc80 2024-09-13T17:14:54.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.661+0000 7f7a6b7fe640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a540666e0 0x7f7a54068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:14:54.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.661+0000 7f7a6bfff640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a540666e0 0x7f7a54068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:14:54.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.661+0000 7f7a6bfff640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a540666e0 0x7f7a54068ba0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0b0000 tx=0x7f7a5c0a8070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:14:54.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.661+0000 7f7a6b7fe640 1 -- 172.21.15.39:0/3691067424 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7f7a64084f70 con 0x7f7a7406cc80 2024-09-13T17:14:54.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:54.705+0000 7f7a6b7fe640 1 -- 172.21.15.39:0/3691067424 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7a64051430 con 0x7f7a7406cc80 2024-09-13T17:14:55.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.645+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a740bfa40 con 0x7f7a7406cc80 2024-09-13T17:14:55.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.645+0000 7f7a6b7fe640 1 -- 172.21.15.39:0/3691067424 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 186 v186) v1 ==== 95+0+5180 (secure 0 0 0) 0x7f7a6401e090 con 0x7f7a7406cc80 2024-09-13T17:14:55.649 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:14:55.650 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":186,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:53.952217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:14:55.650 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 186 2024-09-13T17:14:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.657+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a540666e0 msgr2=0x7f7a54068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.657+0000 7f7a7955c640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a540666e0 0x7f7a54068ba0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0b0000 tx=0x7f7a5c0a8070 comp rx=0 tx=0).stop 2024-09-13T17:14:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.657+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a7406cc80 msgr2=0x7f7a7420d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:14:55.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.661+0000 7f7a7955c640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a7406cc80 0x7f7a7420d290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400b4a0 tx=0x7f7a6400b960 comp rx=0 tx=0).stop 2024-09-13T17:14:55.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.661+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 shutdown_connections 2024-09-13T17:14:55.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.661+0000 7f7a7955c640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a540666e0 0x7f7a54068ba0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:55.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.661+0000 7f7a7955c640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a74153cc0 0x7f7a74150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:55.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.661+0000 7f7a7955c640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a740a9130 0x7f7a7420d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:55.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.661+0000 7f7a7955c640 1 --2- 172.21.15.39:0/3691067424 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a7406cc80 0x7f7a7420d290 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:14:55.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.661+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 >> 172.21.15.39:0/3691067424 conn(0x7f7a740a72c0 msgr2=0x7f7a740be810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:14:55.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.661+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 shutdown_connections 2024-09-13T17:14:55.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:14:55.661+0000 7f7a7955c640 1 -- 172.21.15.39:0/3691067424 wait complete. 2024-09-13T17:14:55.732 INFO:tasks.mds_thrash.fs.[b]:mds.c has gained rank=0, replacing gid=6503 2024-09-13T17:14:55.732 INFO:tasks.mds_thrash.fs.[b]:waiting for 4 secs before reviving mds.a 2024-09-13T17:14:57.461 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-13T17:14:57.457+0000 7fa2c82c2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:14:59.733 INFO:tasks.mds_thrash.fs.[b]:reviving mds.a 2024-09-13T17:14:59.734 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:14:59.734 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:14:59.739 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:14:59.739 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:00.021 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T17:15:00.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9fa4fb640 1 Processor -- start 2024-09-13T17:15:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9fa4fb640 1 -- start start 2024-09-13T17:15:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9fa4fb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd9ec0d3540 0x7fd9ec0d9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9fa4fb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec046600 0x7fd9ec00ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9fa4fb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec031230 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9fa4fb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec04b120 con 0x7fd9ec0d3540 2024-09-13T17:15:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9fa4fb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0da160 con 0x7fd9ec046600 2024-09-13T17:15:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9f8cf8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec046600 0x7fd9ec00ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9f9cfa640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd9ec00b1b0 0x7fd9ec005a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48204/0 (socket says 172.21.15.39:48204) 2024-09-13T17:15:00.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9f9cfa640 1 -- 172.21.15.39:0/434369048 learned_addr learned my addr 172.21.15.39:0/434369048 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:15:00.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9f8cf8640 1 -- 172.21.15.39:0/434369048 >> v2:172.21.15.84:3300/0 conn(0x7fd9ec0d3540 msgr2=0x7fd9ec0d9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:00.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9f94f9640 1 --2- 172.21.15.39:0/434369048 >> v2:172.21.15.84:3300/0 conn(0x7fd9ec0d3540 0x7fd9ec0d9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:00.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9f8cf8640 1 --2- 172.21.15.39:0/434369048 >> v2:172.21.15.84:3300/0 conn(0x7fd9ec0d3540 0x7fd9ec0d9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:00.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9f8cf8640 1 -- 172.21.15.39:0/434369048 >> v1:172.21.15.39:6789/0 conn(0x7fd9ec00b1b0 legacy=0x7fd9ec005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:00.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9ebfff640 1 -- 172.21.15.39:0/434369048 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247646539 0 0) 0x7fd9f4068160 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9f8cf8640 1 -- 172.21.15.39:0/434369048 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec0d3920 con 0x7fd9ec046600 2024-09-13T17:15:00.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.113+0000 7fd9f94f9640 1 --2- 172.21.15.39:0/434369048 >> v2:172.21.15.84:3300/0 conn(0x7fd9ec0d3540 0x7fd9ec0d9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:00.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.121+0000 7fd9f8cf8640 1 --2- 172.21.15.39:0/434369048 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec046600 0x7fd9ec00ac70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40097c0 tx=0x7fd9e401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67224504593b763 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.141+0000 7fd9ebfff640 1 -- 172.21.15.39:0/434369048 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e40ac640 con 0x7fd9ec046600 2024-09-13T17:15:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.141+0000 7fd9ebfff640 1 -- 172.21.15.39:0/434369048 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e40ac7a0 con 0x7fd9ec046600 2024-09-13T17:15:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.141+0000 7fd9ebfff640 1 -- 172.21.15.39:0/434369048 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e40acb30 con 0x7fd9ec046600 2024-09-13T17:15:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.141+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/434369048 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec046600 msgr2=0x7fd9ec00ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.141+0000 7fd9fa4fb640 1 --2- 172.21.15.39:0/434369048 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec046600 0x7fd9ec00ac70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40097c0 tx=0x7fd9e401c3e0 comp rx=0 tx=0).stop 2024-09-13T17:15:00.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.145+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/434369048 shutdown_connections 2024-09-13T17:15:00.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.145+0000 7fd9fa4fb640 1 --2- 172.21.15.39:0/434369048 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec046600 0x7fd9ec00ac70 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40097c0 tx=0x7fd9e401c3e0 comp rx=0 tx=0).stop 2024-09-13T17:15:00.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.145+0000 7fd9fa4fb640 1 --2- 172.21.15.39:0/434369048 >> v2:172.21.15.84:3300/0 conn(0x7fd9ec0d3540 0x7fd9ec0d9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:00.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.145+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/434369048 >> 172.21.15.39:0/434369048 conn(0x7fd9ec047cf0 msgr2=0x7fd9ec048100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.149+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/434369048 shutdown_connections 2024-09-13T17:15:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.149+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/434369048 wait complete. 2024-09-13T17:15:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.153+0000 7fd9fa4fb640 1 Processor -- start 2024-09-13T17:15:00.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.153+0000 7fd9fa4fb640 1 -- start start 2024-09-13T17:15:00.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.153+0000 7fd9fa4fb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ec00b1b0 0x7fd9ec03f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:00.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.153+0000 7fd9fa4fb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec0d3540 0x7fd9ec17f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:00.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.153+0000 7fd9fa4fb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec0480d0 0x7fd9ec17fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:00.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.153+0000 7fd9fa4fb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec041d70 con 0x7fd9ec0480d0 2024-09-13T17:15:00.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.153+0000 7fd9fa4fb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0d2a60 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.153+0000 7fd9fa4fb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0d9140 con 0x7fd9ec0d3540 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f9cfa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec0480d0 0x7fd9ec17fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f94f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ec00b1b0 0x7fd9ec03f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f9cfa640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec0480d0 0x7fd9ec17fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34080/0 (socket says 172.21.15.39:34080) 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f9cfa640 1 -- 172.21.15.39:0/80084908 learned_addr learned my addr 172.21.15.39:0/80084908 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f94f9640 1 -- 172.21.15.39:0/80084908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec0d3540 msgr2=0x7fd9ec17f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f94f9640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec0d3540 0x7fd9ec17f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f94f9640 1 -- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec0480d0 msgr2=0x7fd9ec17fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f94f9640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec0480d0 0x7fd9ec17fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f94f9640 1 -- 172.21.15.39:0/80084908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e4009420 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f9cfa640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec0480d0 0x7fd9ec17fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:15:00.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.157+0000 7fd9f94f9640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ec00b1b0 0x7fd9ec03f410 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000c7c0 tx=0x7fd9f000cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:00.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.161+0000 7fd9e9ffb640 1 -- 172.21.15.39:0/80084908 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0007820 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.161+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec0d2f20 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.161+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec0d8af0 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.161+0000 7fd9e9ffb640 1 -- 172.21.15.39:0/80084908 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f0007980 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.161+0000 7fd9e9ffb640 1 -- 172.21.15.39:0/80084908 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0007d10 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.165+0000 7fd9e9ffb640 1 -- 172.21.15.39:0/80084908 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9f00113f0 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.169+0000 7fd9e9ffb640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9e0066850 0x7fd9e0068d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:00.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.169+0000 7fd9e9ffb640 1 -- 172.21.15.39:0/80084908 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7fd9f0085050 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.169+0000 7fd9f8cf8640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9e0066850 0x7fd9e0068d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:00.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.185+0000 7fd9f8cf8640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9e0066850 0x7fd9e0068d10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40ae000 tx=0x7fd9e4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:00.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.209+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d4005260 con 0x7fd9ec00b1b0 2024-09-13T17:15:00.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:00.217+0000 7fd9e9ffb640 1 -- 172.21.15.39:0/80084908 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd9f0055060 con 0x7fd9ec00b1b0 2024-09-13T17:15:01.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.273+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9d4005a90 con 0x7fd9ec00b1b0 2024-09-13T17:15:01.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.277+0000 7fd9e9ffb640 1 -- 172.21.15.39:0/80084908 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5938 (secure 0 0 0) 0x7fd9f001f030 con 0x7fd9ec00b1b0 2024-09-13T17:15:01.282 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:01.282 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":187}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:53.952217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:01.282 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 187 2024-09-13T17:15:01.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.281+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9e0066850 msgr2=0x7fd9e0068d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:01.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.281+0000 7fd9fa4fb640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9e0066850 0x7fd9e0068d10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40ae000 tx=0x7fd9e4004740 comp rx=0 tx=0).stop 2024-09-13T17:15:01.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.281+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ec00b1b0 msgr2=0x7fd9ec03f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:01.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.281+0000 7fd9fa4fb640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ec00b1b0 0x7fd9ec03f410 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000c7c0 tx=0x7fd9f000cc80 comp rx=0 tx=0).stop 2024-09-13T17:15:01.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.285+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 shutdown_connections 2024-09-13T17:15:01.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.285+0000 7fd9fa4fb640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd9e0066850 0x7fd9e0068d10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:01.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.285+0000 7fd9fa4fb640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec0480d0 0x7fd9ec17fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:01.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.285+0000 7fd9fa4fb640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd9ec0d3540 0x7fd9ec17f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:01.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.285+0000 7fd9fa4fb640 1 --2- 172.21.15.39:0/80084908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ec00b1b0 0x7fd9ec03f410 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:01.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.285+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 >> 172.21.15.39:0/80084908 conn(0x7fd9ec047cf0 msgr2=0x7fd9ec009c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:01.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.285+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 shutdown_connections 2024-09-13T17:15:01.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:01.285+0000 7fd9fa4fb640 1 -- 172.21.15.39:0/80084908 wait complete. 2024-09-13T17:15:01.354 INFO:tasks.mds_thrash.fs.[b]:mds.a reported in up:standby state 2024-09-13T17:15:01.355 INFO:tasks.mds_thrash.fs.[b]:waiting for 13 secs before thrashing 2024-09-13T17:15:05.096 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:15:05.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:15:14.355 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:14.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.941+0000 7fc5bbfff640 1 Processor -- start 2024-09-13T17:15:14.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.941+0000 7fc5bbfff640 1 -- start start 2024-09-13T17:15:14.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.945+0000 7fc5bbfff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc5bc0aa210 0x7fc5bc0aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:14.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.945+0000 7fc5bbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc5bc14ff40 0x7fc5bc1503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:14.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.945+0000 7fc5bbfff640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc0c0240 con 0x7fc5bc0c2940 2024-09-13T17:15:14.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.945+0000 7fc5bbfff640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc06da70 con 0x7fc5bc14ff40 2024-09-13T17:15:14.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.945+0000 7fc5bbfff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc06d4f0 con 0x7fc5bc0aa210 2024-09-13T17:15:14.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.945+0000 7fc5baffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc5bc0c2940 0x7fc5bc06b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42414/0 (socket says 172.21.15.39:42414) 2024-09-13T17:15:14.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.945+0000 7fc5baffd640 1 -- 172.21.15.39:0/4171510123 learned_addr learned my addr 172.21.15.39:0/4171510123 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:15:14.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.945+0000 7fc5bb7fe640 1 --2- 172.21.15.39:0/4171510123 >> v2:172.21.15.39:3300/0 conn(0x7fc5bc14ff40 0x7fc5bc1503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:14.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.949+0000 7fc5ba7fc640 1 --2- 172.21.15.39:0/4171510123 >> v2:172.21.15.84:3300/0 conn(0x7fc5bc0aa210 0x7fc5bc0aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:14.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.949+0000 7fc5bb7fe640 1 -- 172.21.15.39:0/4171510123 >> v2:172.21.15.84:3300/0 conn(0x7fc5bc0aa210 msgr2=0x7fc5bc0aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:14.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.949+0000 7fc5bb7fe640 1 --2- 172.21.15.39:0/4171510123 >> v2:172.21.15.84:3300/0 conn(0x7fc5bc0aa210 0x7fc5bc0aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:14.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.949+0000 7fc5bb7fe640 1 -- 172.21.15.39:0/4171510123 >> v1:172.21.15.39:6789/0 conn(0x7fc5bc0c2940 legacy=0x7fc5bc06b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:14.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.957+0000 7fc5bb7fe640 1 -- 172.21.15.39:0/4171510123 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0c5300 con 0x7fc5bc14ff40 2024-09-13T17:15:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.965+0000 7fc5bb7fe640 1 --2- 172.21.15.39:0/4171510123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 0x7fc5bc1503f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc14d9a0 tx=0x7fc5b001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fc78d674d17048 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.965+0000 7fc5b9ffb640 1 -- 172.21.15.39:0/4171510123 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0004230 con 0x7fc5bc14ff40 2024-09-13T17:15:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.965+0000 7fc5b9ffb640 1 -- 172.21.15.39:0/4171510123 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b00a7050 con 0x7fc5bc14ff40 2024-09-13T17:15:14.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.965+0000 7fc5bbfff640 1 -- 172.21.15.39:0/4171510123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 msgr2=0x7fc5bc1503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.969+0000 7fc5b9ffb640 1 -- 172.21.15.39:0/4171510123 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0004230 con 0x7fc5bc14ff40 2024-09-13T17:15:14.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.969+0000 7fc5bbfff640 1 --2- 172.21.15.39:0/4171510123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 0x7fc5bc1503f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc14d9a0 tx=0x7fc5b001c4e0 comp rx=0 tx=0).stop 2024-09-13T17:15:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.969+0000 7fc5bbfff640 1 -- 172.21.15.39:0/4171510123 shutdown_connections 2024-09-13T17:15:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.969+0000 7fc5bbfff640 1 --2- 172.21.15.39:0/4171510123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 0x7fc5bc1503f0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.969+0000 7fc5bbfff640 1 --2- 172.21.15.39:0/4171510123 >> v2:172.21.15.84:3300/0 conn(0x7fc5bc0aa210 0x7fc5bc0aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:14.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.969+0000 7fc5bbfff640 1 -- 172.21.15.39:0/4171510123 >> 172.21.15.39:0/4171510123 conn(0x7fc5bc0c5510 msgr2=0x7fc5bc0c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.973+0000 7fc5bbfff640 1 -- 172.21.15.39:0/4171510123 shutdown_connections 2024-09-13T17:15:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.973+0000 7fc5bbfff640 1 -- 172.21.15.39:0/4171510123 wait complete. 2024-09-13T17:15:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.973+0000 7fc5bbfff640 1 Processor -- start 2024-09-13T17:15:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5bbfff640 1 -- start start 2024-09-13T17:15:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5bbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5bc0aa210 0x7fc5bc1f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5bbfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5bc0c2940 0x7fc5bc06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5bbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 0x7fc5bc0a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5bbfff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0a30e0 con 0x7fc5bc14ff40 2024-09-13T17:15:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5bbfff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0a2460 con 0x7fc5bc0aa210 2024-09-13T17:15:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5bbfff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0a2740 con 0x7fc5bc0c2940 2024-09-13T17:15:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5bc0aa210 0x7fc5bc1f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:14.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5bc0aa210 0x7fc5bc1f04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:50842/0 (socket says 172.21.15.39:50842) 2024-09-13T17:15:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5baffd640 1 -- 172.21.15.39:0/992124185 learned_addr learned my addr 172.21.15.39:0/992124185 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5ba7fc640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5bc0c2940 0x7fc5bc06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5baffd640 1 -- 172.21.15.39:0/992124185 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5bc0c2940 msgr2=0x7fc5bc06f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5baffd640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5bc0c2940 0x7fc5bc06f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5baffd640 1 -- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 msgr2=0x7fc5bc0a4460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:15:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5bb7fe640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 0x7fc5bc0a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5baffd640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 0x7fc5bc0a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5baffd640 1 -- 172.21.15.39:0/992124185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b0009420 con 0x7fc5bc0aa210 2024-09-13T17:15:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5ba7fc640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5bc0c2940 0x7fc5bc06f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.977+0000 7fc5baffd640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5bc0aa210 0x7fc5bc1f04e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400caf0 tx=0x7fc5b40021a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.981+0000 7fc5bb7fe640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 0x7fc5bc0a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.989+0000 7fc5a3fff640 1 -- 172.21.15.39:0/992124185 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400a7c0 con 0x7fc5bc0aa210 2024-09-13T17:15:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.989+0000 7fc5a3fff640 1 -- 172.21.15.39:0/992124185 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b400a920 con 0x7fc5bc0aa210 2024-09-13T17:15:14.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.989+0000 7fc5a3fff640 1 -- 172.21.15.39:0/992124185 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400acd0 con 0x7fc5bc0aa210 2024-09-13T17:15:14.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.989+0000 7fc5bbfff640 1 -- 172.21.15.39:0/992124185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc0a2a20 con 0x7fc5bc0aa210 2024-09-13T17:15:14.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.989+0000 7fc5bbfff640 1 -- 172.21.15.39:0/992124185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc0a49a0 con 0x7fc5bc0aa210 2024-09-13T17:15:15.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.997+0000 7fc5a3fff640 1 -- 172.21.15.39:0/992124185 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc5b40070d0 con 0x7fc5bc0aa210 2024-09-13T17:15:15.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.997+0000 7fc5a3fff640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5a8066790 0x7fc5a8068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:15.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:14.997+0000 7fc5a3fff640 1 -- 172.21.15.39:0/992124185 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7fc5b4084cc0 con 0x7fc5bc0aa210 2024-09-13T17:15:15.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.001+0000 7fc5bbfff640 1 -- 172.21.15.39:0/992124185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59c005260 con 0x7fc5bc0aa210 2024-09-13T17:15:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.013+0000 7fc5ba7fc640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5a8066790 0x7fc5a8068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:15.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.017+0000 7fc5ba7fc640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5a8066790 0x7fc5a8068c50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc151040 tx=0x7fc5ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:15.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.021+0000 7fc5a3fff640 1 -- 172.21.15.39:0/992124185 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc5b4018070 con 0x7fc5bc0aa210 2024-09-13T17:15:15.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.844+0000 7fc5bbfff640 1 -- 172.21.15.39:0/992124185 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc59c005a90 con 0x7fc5bc0aa210 2024-09-13T17:15:15.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.844+0000 7fc5a3fff640 1 -- 172.21.15.39:0/992124185 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5938 (secure 0 0 0) 0x7fc5b4054b60 con 0x7fc5bc0aa210 2024-09-13T17:15:15.853 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:15.853 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":187}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:53.952217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:15.853 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 187 2024-09-13T17:15:15.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 -- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5a8066790 msgr2=0x7fc5a8068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5a8066790 0x7fc5a8068c50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc151040 tx=0x7fc5ac008040 comp rx=0 tx=0).stop 2024-09-13T17:15:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 -- 172.21.15.39:0/992124185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5bc0aa210 msgr2=0x7fc5bc1f04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5bc0aa210 0x7fc5bc1f04e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400caf0 tx=0x7fc5b40021a0 comp rx=0 tx=0).stop 2024-09-13T17:15:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 -- 172.21.15.39:0/992124185 shutdown_connections 2024-09-13T17:15:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5a8066790 0x7fc5a8068c50 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5bc14ff40 0x7fc5bc0a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5bc0c2940 0x7fc5bc06f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 --2- 172.21.15.39:0/992124185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5bc0aa210 0x7fc5bc1f04e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:15.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.860+0000 7fc5a1ffb640 1 -- 172.21.15.39:0/992124185 >> 172.21.15.39:0/992124185 conn(0x7fc5bc0c5510 msgr2=0x7fc5bc0a8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:15.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.868+0000 7fc5a1ffb640 1 -- 172.21.15.39:0/992124185 shutdown_connections 2024-09-13T17:15:15.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:15.868+0000 7fc5a1ffb640 1 -- 172.21.15.39:0/992124185 wait complete. 2024-09-13T17:15:16.052 INFO:tasks.mds_thrash.fs.[b]:kill mds.c (rank=0) 2024-09-13T17:15:16.052 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:15:16.052 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:15:16.083 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:15:16.083 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:16.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2880cca640 1 Processor -- start 2024-09-13T17:15:16.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2880cca640 1 -- start start 2024-09-13T17:15:16.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2880cca640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c153cc0 0x7f287c0a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:16.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2880cca640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f287c0a9130 0x7f287c0c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2880cca640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f287c06cc80 0x7f287c0c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2880cca640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c0c1da0 con 0x7f287c06cc80 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2880cca640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c0c5340 con 0x7f287c0a9130 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2880cca640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0c5060 con 0x7f287c153cc0 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2879d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f287c0a9130 0x7f287c0c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2879d9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f287c0a9130 0x7f287c0c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:50850/0 (socket says 172.21.15.39:50850) 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.772+0000 7f2879d9b640 1 -- 172.21.15.39:0/1128048458 learned_addr learned my addr 172.21.15.39:0/1128048458 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.776+0000 7f2879d9b640 1 -- 172.21.15.39:0/1128048458 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c153cc0 msgr2=0x7f287c0a8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.776+0000 7f2879d9b640 1 --2- 172.21.15.39:0/1128048458 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c153cc0 0x7f287c0a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.776+0000 7f2879d9b640 1 -- 172.21.15.39:0/1128048458 >> v2:172.21.15.39:3300/0 conn(0x7f287c06cc80 msgr2=0x7f287c0c3f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.776+0000 7f2879d9b640 1 --2- 172.21.15.39:0/1128048458 >> v2:172.21.15.39:3300/0 conn(0x7f287c06cc80 0x7f287c0c3f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.776+0000 7f2879d9b640 1 -- 172.21.15.39:0/1128048458 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c06c1a0 con 0x7f287c0a9130 2024-09-13T17:15:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.784+0000 7f2879d9b640 1 --2- 172.21.15.39:0/1128048458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f287c0a9130 0x7f287c0c3a20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f287c0c64f0 tx=0x7f286c020a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=766160460c6b67e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.788+0000 7f287959a640 1 -- 172.21.15.39:0/1128048458 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c0a9af0 con 0x7f287c0a9130 2024-09-13T17:15:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.788+0000 7f287959a640 1 -- 172.21.15.39:0/1128048458 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c0a9c50 con 0x7f287c0a9130 2024-09-13T17:15:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.788+0000 7f2880cca640 1 -- 172.21.15.39:0/1128048458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f287c0a9130 msgr2=0x7f287c0c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.788+0000 7f2880cca640 1 --2- 172.21.15.39:0/1128048458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f287c0a9130 0x7f287c0c3a20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f287c0c64f0 tx=0x7f286c020a60 comp rx=0 tx=0).stop 2024-09-13T17:15:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 -- 172.21.15.39:0/1128048458 shutdown_connections 2024-09-13T17:15:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 --2- 172.21.15.39:0/1128048458 >> v2:172.21.15.39:3300/0 conn(0x7f287c06cc80 0x7f287c0c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 --2- 172.21.15.39:0/1128048458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f287c0a9130 0x7f287c0c3a20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 --2- 172.21.15.39:0/1128048458 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c153cc0 0x7f287c0a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 -- 172.21.15.39:0/1128048458 >> 172.21.15.39:0/1128048458 conn(0x7f287c0a72c0 msgr2=0x7f287c0a76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 -- 172.21.15.39:0/1128048458 shutdown_connections 2024-09-13T17:15:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 -- 172.21.15.39:0/1128048458 wait complete. 2024-09-13T17:15:16.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 Processor -- start 2024-09-13T17:15:16.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 -- start start 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f287c06cc80 0x7f287c0a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c0a9130 0x7f287c06fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f287c153cc0 0x7f287c1f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1f1940 con 0x7f287c06cc80 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1f1e30 con 0x7f287c153cc0 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2880cca640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0a2340 con 0x7f287c0a9130 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2879d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c0a9130 0x7f287c06fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2879d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c0a9130 0x7f287c06fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:60106/0 (socket says 172.21.15.39:60106) 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2879d9b640 1 -- 172.21.15.39:0/2726151366 learned_addr learned my addr 172.21.15.39:0/2726151366 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:16.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f287a59c640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f287c06cc80 0x7f287c0a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:16.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2879d9b640 1 -- 172.21.15.39:0/2726151366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f287c153cc0 msgr2=0x7f287c1f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:16.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2879d9b640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f287c153cc0 0x7f287c1f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:16.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2879d9b640 1 -- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f287c06cc80 msgr2=0x7f287c0a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:16.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2879d9b640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f287c06cc80 0x7f287c0a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:16.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.792+0000 7f2879d9b640 1 -- 172.21.15.39:0/2726151366 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c009d60 con 0x7f287c0a9130 2024-09-13T17:15:16.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.796+0000 7f2879d9b640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c0a9130 0x7f287c06fd40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f286c021300 tx=0x7f287c0c0f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:16.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.796+0000 7f286affd640 1 -- 172.21.15.39:0/2726151366 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c013420 con 0x7f287c0a9130 2024-09-13T17:15:16.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.796+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c150750 con 0x7f287c0a9130 2024-09-13T17:15:16.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.796+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c0a2820 con 0x7f287c0a9130 2024-09-13T17:15:16.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.796+0000 7f286affd640 1 -- 172.21.15.39:0/2726151366 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c013580 con 0x7f287c0a9130 2024-09-13T17:15:16.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.796+0000 7f286affd640 1 -- 172.21.15.39:0/2726151366 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c00f6b0 con 0x7f287c0a9130 2024-09-13T17:15:16.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.800+0000 7f286affd640 1 -- 172.21.15.39:0/2726151366 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f286c00f810 con 0x7f287c0a9130 2024-09-13T17:15:16.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.800+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c0a8bf0 con 0x7f287c0a9130 2024-09-13T17:15:16.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.800+0000 7f286affd640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f285c066760 0x7f285c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:16.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.800+0000 7f286affd640 1 -- 172.21.15.39:0/2726151366 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7f286c11eed0 con 0x7f287c0a9130 2024-09-13T17:15:16.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.800+0000 7f287a59c640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f285c066760 0x7f285c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:16.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.800+0000 7f287a59c640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f285c066760 0x7f285c068c20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f287400a400 tx=0x7f287400d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:16.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:16.832+0000 7f286affd640 1 -- 172.21.15.39:0/2726151366 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f286c0eb390 con 0x7f287c0a9130 2024-09-13T17:15:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.060+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f287c0c14a0 con 0x7f287c0a9130 2024-09-13T17:15:18.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.076+0000 7f286affd640 1 -- 172.21.15.39:0/2726151366 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5938 (secure 0 0 0) 0x7f286c0b6090 con 0x7f287c0a9130 2024-09-13T17:15:18.081 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:18.081 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":187}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:53.952217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:18.081 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 187 2024-09-13T17:15:18.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.080+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f285c066760 msgr2=0x7f285c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:18.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.080+0000 7f2880cca640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f285c066760 0x7f285c068c20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f287400a400 tx=0x7f287400d040 comp rx=0 tx=0).stop 2024-09-13T17:15:18.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.080+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c0a9130 msgr2=0x7f287c06fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:18.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.080+0000 7f2880cca640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c0a9130 0x7f287c06fd40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f286c021300 tx=0x7f287c0c0f20 comp rx=0 tx=0).stop 2024-09-13T17:15:18.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.080+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 shutdown_connections 2024-09-13T17:15:18.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.084+0000 7f2880cca640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f285c066760 0x7f285c068c20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:18.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.084+0000 7f2880cca640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f287c153cc0 0x7f287c1f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:18.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.084+0000 7f2880cca640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f287c0a9130 0x7f287c06fd40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:18.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.084+0000 7f2880cca640 1 --2- 172.21.15.39:0/2726151366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f287c06cc80 0x7f287c0a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:18.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.084+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 >> 172.21.15.39:0/2726151366 conn(0x7f287c0a72c0 msgr2=0x7f287c0c3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:18.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.084+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 shutdown_connections 2024-09-13T17:15:18.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:18.084+0000 7f2880cca640 1 -- 172.21.15.39:0/2726151366 wait complete. 2024-09-13T17:15:18.164 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:15:20.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.472+0000 7fcf17fff640 1 Processor -- start 2024-09-13T17:15:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.472+0000 7fcf17fff640 1 -- start start 2024-09-13T17:15:20.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.472+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18150650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:20.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.472+0000 7fcf17fff640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf180c0240 con 0x7fcf180aa210 2024-09-13T17:15:20.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.472+0000 7fcf17fff640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1806da70 con 0x7fcf180c2940 2024-09-13T17:15:20.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.472+0000 7fcf17fff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1806d4f0 con 0x7fcf180aa760 2024-09-13T17:15:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.476+0000 7fcf167fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcf180aa210 0x7fcf1814ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48380/0 (socket says 172.21.15.39:48380) 2024-09-13T17:15:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.476+0000 7fcf167fc640 1 -- 172.21.15.39:0/2041378488 learned_addr learned my addr 172.21.15.39:0/2041378488 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:15:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.476+0000 7fcf177fe640 1 --2- 172.21.15.39:0/2041378488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:20.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.480+0000 7fcf15ffb640 1 -- 172.21.15.39:0/2041378488 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283139839 0 0) 0x7fcf1806da70 con 0x7fcf180c2940 2024-09-13T17:15:20.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.480+0000 7fcf15ffb640 1 -- 172.21.15.39:0/2041378488 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf00003450 con 0x7fcf180c2940 2024-09-13T17:15:20.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.480+0000 7fcf15ffb640 1 -- 172.21.15.39:0/2041378488 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153698130 0 0) 0x7fcf180c0240 con 0x7fcf180aa210 2024-09-13T17:15:20.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.480+0000 7fcf15ffb640 1 -- 172.21.15.39:0/2041378488 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf1806da70 con 0x7fcf180aa210 2024-09-13T17:15:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.480+0000 7fcf177fe640 1 -- 172.21.15.39:0/2041378488 >> v1:172.21.15.84:6789/0 conn(0x7fcf180c2940 legacy=0x7fcf1806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.480+0000 7fcf177fe640 1 -- 172.21.15.39:0/2041378488 >> v1:172.21.15.39:6789/0 conn(0x7fcf180aa210 legacy=0x7fcf1814ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.480+0000 7fcf177fe640 1 -- 172.21.15.39:0/2041378488 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf180c5300 con 0x7fcf180aa760 2024-09-13T17:15:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.484+0000 7fcf177fe640 1 --2- 172.21.15.39:0/2041378488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18150650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009dc0 tx=0x7fcf1001c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970543c7d9375644 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.484+0000 7fcf15ffb640 1 -- 172.21.15.39:0/2041378488 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf100a58d0 con 0x7fcf180aa760 2024-09-13T17:15:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.484+0000 7fcf15ffb640 1 -- 172.21.15.39:0/2041378488 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf100a5a30 con 0x7fcf180aa760 2024-09-13T17:15:20.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.488+0000 7fcf17fff640 1 -- 172.21.15.39:0/2041378488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 msgr2=0x7fcf18150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:20.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.488+0000 7fcf17fff640 1 --2- 172.21.15.39:0/2041378488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18150650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009dc0 tx=0x7fcf1001c900 comp rx=0 tx=0).stop 2024-09-13T17:15:20.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.488+0000 7fcf17fff640 1 -- 172.21.15.39:0/2041378488 shutdown_connections 2024-09-13T17:15:20.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.488+0000 7fcf17fff640 1 --2- 172.21.15.39:0/2041378488 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18150650 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:20.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.488+0000 7fcf17fff640 1 -- 172.21.15.39:0/2041378488 >> 172.21.15.39:0/2041378488 conn(0x7fcf180c5510 msgr2=0x7fcf180c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:20.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.488+0000 7fcf17fff640 1 -- 172.21.15.39:0/2041378488 shutdown_connections 2024-09-13T17:15:20.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.488+0000 7fcf17fff640 1 -- 172.21.15.39:0/2041378488 wait complete. 2024-09-13T17:15:20.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.488+0000 7fcf17fff640 1 Processor -- start 2024-09-13T17:15:20.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.488+0000 7fcf17fff640 1 -- start start 2024-09-13T17:15:20.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf180aa210 0x7fcf181eb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:20.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:20.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf180c2940 0x7fcf181496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:20.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf17fff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18072120 con 0x7fcf180aa210 2024-09-13T17:15:20.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf17fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181ebe50 con 0x7fcf180c2940 2024-09-13T17:15:20.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf17fff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18072540 con 0x7fcf180aa760 2024-09-13T17:15:20.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf167fc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:20.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf167fc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18149190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:55040/0 (socket says 172.21.15.39:55040) 2024-09-13T17:15:20.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf167fc640 1 -- 172.21.15.39:0/1794418509 learned_addr learned my addr 172.21.15.39:0/1794418509 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:20.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf16ffd640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf180aa210 0x7fcf181eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:20.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf16ffd640 1 -- 172.21.15.39:0/1794418509 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 msgr2=0x7fcf18149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:20.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf16ffd640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:20.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf16ffd640 1 -- 172.21.15.39:0/1794418509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf180c2940 msgr2=0x7fcf181496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:20.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf16ffd640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf180c2940 0x7fcf181496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:20.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf16ffd640 1 -- 172.21.15.39:0/1794418509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10009a70 con 0x7fcf180aa210 2024-09-13T17:15:20.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.492+0000 7fcf16ffd640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf180aa210 0x7fcf181eb170 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c390 tx=0x7fcf0c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:20.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.496+0000 7fceff7fe640 1 -- 172.21.15.39:0/1794418509 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0052d0 con 0x7fcf180aa210 2024-09-13T17:15:20.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.496+0000 7fceff7fe640 1 -- 172.21.15.39:0/1794418509 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c005450 con 0x7fcf180aa210 2024-09-13T17:15:20.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.496+0000 7fceff7fe640 1 -- 172.21.15.39:0/1794418509 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c003840 con 0x7fcf180aa210 2024-09-13T17:15:20.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.500+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf18153a10 con 0x7fcf180aa210 2024-09-13T17:15:20.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.500+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1814ccd0 con 0x7fcf180aa210 2024-09-13T17:15:20.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.504+0000 7fceff7fe640 1 -- 172.21.15.39:0/1794418509 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf0c0039a0 con 0x7fcf180aa210 2024-09-13T17:15:20.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.504+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef8005260 con 0x7fcf180aa210 2024-09-13T17:15:20.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.504+0000 7fceff7fe640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf04066760 0x7fcf04068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:20.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.504+0000 7fceff7fe640 1 -- 172.21.15.39:0/1794418509 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7fcf0c0845a0 con 0x7fcf180aa210 2024-09-13T17:15:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.512+0000 7fcf167fc640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf04066760 0x7fcf04068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:20.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.512+0000 7fcf167fc640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf04066760 0x7fcf04068c20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcf08001230 tx=0x7fcf080011c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:20.520+0000 7fceff7fe640 1 -- 172.21.15.39:0/1794418509 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf0c050a60 con 0x7fcf180aa210 2024-09-13T17:15:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.288+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcef8005a90 con 0x7fcf180aa210 2024-09-13T17:15:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.288+0000 7fceff7fe640 1 -- 172.21.15.39:0/1794418509 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5938 (secure 0 0 0) 0x7fcf0c054080 con 0x7fcf180aa210 2024-09-13T17:15:21.294 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:21.294 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":187}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:53.952217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 187 2024-09-13T17:15:21.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf04066760 msgr2=0x7fcf04068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:21.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf04066760 0x7fcf04068c20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcf08001230 tx=0x7fcf080011c0 comp rx=0 tx=0).stop 2024-09-13T17:15:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf180aa210 msgr2=0x7fcf181eb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf180aa210 0x7fcf181eb170 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00c390 tx=0x7fcf0c00c850 comp rx=0 tx=0).stop 2024-09-13T17:15:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 shutdown_connections 2024-09-13T17:15:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcf04066760 0x7fcf04068c20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcf180c2940 0x7fcf181496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcf180aa760 0x7fcf18149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 --2- 172.21.15.39:0/1794418509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf180aa210 0x7fcf181eb170 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 >> 172.21.15.39:0/1794418509 conn(0x7fcf180c5510 msgr2=0x7fcf180a7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 shutdown_connections 2024-09-13T17:15:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:21.292+0000 7fcf17fff640 1 -- 172.21.15.39:0/1794418509 wait complete. 2024-09-13T17:15:21.333 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:15:23.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:23.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8ee25640 1 Processor -- start 2024-09-13T17:15:23.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8ee25640 1 -- start start 2024-09-13T17:15:23.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8ee25640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d8814a770 0x7f1d88150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:23.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8ee25640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d880c6060 0x7f1d8814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8ee25640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1d8806ae20 0x7f1d8806b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8ee25640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8806df80 con 0x7f1d8806ae20 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8cbc1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d8814a770 0x7f1d88150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8cbc1640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d8814a770 0x7f1d88150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:52856/0 (socket says 172.21.15.39:52856) 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8cbc1640 1 -- 172.21.15.39:0/4246923956 learned_addr learned my addr 172.21.15.39:0/4246923956 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d87fff640 1 --2- 172.21.15.39:0/4246923956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d880c6060 0x7f1d8814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8ee25640 1 -- 172.21.15.39:0/4246923956 --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d880c1950 con 0x7f1d8814a770 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8d3c2640 1 --2- 172.21.15.39:0/4246923956 >> v2:172.21.15.39:3300/0 conn(0x7f1d8806ae20 0x7f1d8806b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8ee25640 1 -- 172.21.15.39:0/4246923956 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880c4760 con 0x7f1d880c6060 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8d3c2640 1 -- 172.21.15.39:0/4246923956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d880c6060 msgr2=0x7f1d8814ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8d3c2640 1 --2- 172.21.15.39:0/4246923956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d880c6060 0x7f1d8814ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8d3c2640 1 -- 172.21.15.39:0/4246923956 >> v2:172.21.15.84:3300/0 conn(0x7f1d8814a770 msgr2=0x7f1d88150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8d3c2640 1 --2- 172.21.15.39:0/4246923956 >> v2:172.21.15.84:3300/0 conn(0x7f1d8814a770 0x7f1d88150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d8d3c2640 1 -- 172.21.15.39:0/4246923956 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d88151120 con 0x7f1d8806ae20 2024-09-13T17:15:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.692+0000 7f1d87fff640 1 --2- 172.21.15.39:0/4246923956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d880c6060 0x7f1d8814ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:15:23.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8d3c2640 1 --2- 172.21.15.39:0/4246923956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 0x7f1d8806b2d0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1d880c3270 tx=0x7f1d8001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f810aef43e66bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:23.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d877fe640 1 -- 172.21.15.39:0/4246923956 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800a58f0 con 0x7f1d8806ae20 2024-09-13T17:15:23.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d877fe640 1 -- 172.21.15.39:0/4246923956 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d800a5a50 con 0x7f1d8806ae20 2024-09-13T17:15:23.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d877fe640 1 -- 172.21.15.39:0/4246923956 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800a5de0 con 0x7f1d8806ae20 2024-09-13T17:15:23.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8ee25640 1 -- 172.21.15.39:0/4246923956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 msgr2=0x7f1d8806b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:23.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/4246923956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 0x7f1d8806b2d0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1d880c3270 tx=0x7f1d8001c870 comp rx=0 tx=0).stop 2024-09-13T17:15:23.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8ee25640 1 -- 172.21.15.39:0/4246923956 shutdown_connections 2024-09-13T17:15:23.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/4246923956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 0x7f1d8806b2d0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:23.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/4246923956 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d880c6060 0x7f1d8814ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:23.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/4246923956 >> v2:172.21.15.84:3300/0 conn(0x7f1d8814a770 0x7f1d88150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:23.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8ee25640 1 -- 172.21.15.39:0/4246923956 >> 172.21.15.39:0/4246923956 conn(0x7f1d880c32a0 msgr2=0x7f1d880c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:23.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8ee25640 1 -- 172.21.15.39:0/4246923956 shutdown_connections 2024-09-13T17:15:23.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.700+0000 7f1d8ee25640 1 -- 172.21.15.39:0/4246923956 wait complete. 2024-09-13T17:15:23.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8ee25640 1 Processor -- start 2024-09-13T17:15:23.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8ee25640 1 -- start start 2024-09-13T17:15:23.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8ee25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 0x7f1d8814b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:23.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8ee25640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d880c6060 0x7f1d880abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:23.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8cbc1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 0x7f1d8814b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:23.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8cbc1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 0x7f1d8814b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49096/0 (socket says 172.21.15.39:49096) 2024-09-13T17:15:23.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8cbc1640 1 -- 172.21.15.39:0/591724108 learned_addr learned my addr 172.21.15.39:0/591724108 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:23.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d8814a770 0x7f1d880ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:23.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8814e0e0 con 0x7f1d8806ae20 2024-09-13T17:15:23.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880a9ba0 con 0x7f1d880c6060 2024-09-13T17:15:23.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880ac790 con 0x7f1d8814a770 2024-09-13T17:15:23.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.704+0000 7f1d8d3c2640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d8814a770 0x7f1d880ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:23.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d8cbc1640 1 -- 172.21.15.39:0/591724108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d8814a770 msgr2=0x7f1d880ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:23.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d8cbc1640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d8814a770 0x7f1d880ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:23.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d8cbc1640 1 -- 172.21.15.39:0/591724108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d880c6060 msgr2=0x7f1d880abd10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:15:23.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d8cbc1640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d880c6060 0x7f1d880abd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:23.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d8cbc1640 1 -- 172.21.15.39:0/591724108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d74009420 con 0x7f1d8806ae20 2024-09-13T17:15:23.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d8cbc1640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 0x7f1d8814b550 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00e890 tx=0x7f1d7c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:23.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d857fa640 1 -- 172.21.15.39:0/591724108 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c009620 con 0x7f1d8806ae20 2024-09-13T17:15:23.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d857fa640 1 -- 172.21.15.39:0/591724108 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c00c940 con 0x7f1d8806ae20 2024-09-13T17:15:23.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d857fa640 1 -- 172.21.15.39:0/591724108 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00ccb0 con 0x7f1d8806ae20 2024-09-13T17:15:23.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d80009a70 con 0x7f1d8806ae20 2024-09-13T17:15:23.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d857fa640 1 -- 172.21.15.39:0/591724108 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d7c013470 con 0x7f1d8806ae20 2024-09-13T17:15:23.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d857fa640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d6c066710 0x7f1d6c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:23.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d87fff640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d6c066710 0x7f1d6c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:23.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d881e90a0 con 0x7f1d8806ae20 2024-09-13T17:15:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.708+0000 7f1d857fa640 1 -- 172.21.15.39:0/591724108 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7f1d7c054b00 con 0x7f1d8806ae20 2024-09-13T17:15:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.712+0000 7f1d87fff640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d6c066710 0x7f1d6c068bd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1d88071100 tx=0x7f1d74009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.712+0000 7f1d72ffd640 1 -- 172.21.15.39:0/591724108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d60005260 con 0x7f1d8806ae20 2024-09-13T17:15:23.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:23.728+0000 7f1d857fa640 1 -- 172.21.15.39:0/591724108 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1d7c017020 con 0x7f1d8806ae20 2024-09-13T17:15:24.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.248+0000 7f1d72ffd640 1 -- 172.21.15.39:0/591724108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d60005510 con 0x7f1d8806ae20 2024-09-13T17:15:24.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.248+0000 7f1d857fa640 1 -- 172.21.15.39:0/591724108 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5938 (secure 0 0 0) 0x7f1d7c054640 con 0x7f1d8806ae20 2024-09-13T17:15:24.257 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:24.257 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":187}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:53.952217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:24.257 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 187 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d6c066710 msgr2=0x7f1d6c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d6c066710 0x7f1d6c068bd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1d88071100 tx=0x7f1d74009190 comp rx=0 tx=0).stop 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 msgr2=0x7f1d8814b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 0x7f1d8814b550 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00e890 tx=0x7f1d7c00ed50 comp rx=0 tx=0).stop 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 shutdown_connections 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d6c066710 0x7f1d6c068bd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d880c6060 0x7f1d880abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d8814a770 0x7f1d880ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 --2- 172.21.15.39:0/591724108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d8806ae20 0x7f1d8814b550 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 >> 172.21.15.39:0/591724108 conn(0x7f1d880c32a0 msgr2=0x7f1d880a7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 shutdown_connections 2024-09-13T17:15:24.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:24.260+0000 7f1d8ee25640 1 -- 172.21.15.39:0/591724108 wait complete. 2024-09-13T17:15:24.330 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:15:26.331 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:26.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.592+0000 7f26241cb640 1 Processor -- start 2024-09-13T17:15:26.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.600+0000 7f26241cb640 1 -- start start 2024-09-13T17:15:26.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.600+0000 7f26241cb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f261c071900 0x7f261c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:26.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.600+0000 7f26241cb640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f261c0611c0 0x7f261c155530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:26.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.600+0000 7f26241cb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c15c380 con 0x7f261c155a70 2024-09-13T17:15:26.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.600+0000 7f26241cb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261c0701b0 con 0x7f261c071900 2024-09-13T17:15:26.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.600+0000 7f26241cb640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261c15b840 con 0x7f261c0611c0 2024-09-13T17:15:26.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.600+0000 7f2622768640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f261c155a70 0x7f261c06cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:59080/0 (socket says 172.21.15.39:59080) 2024-09-13T17:15:26.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.600+0000 7f2622768640 1 -- 172.21.15.39:0/2706844130 learned_addr learned my addr 172.21.15.39:0/2706844130 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:15:26.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.604+0000 7f2621766640 1 --2- 172.21.15.39:0/2706844130 >> v2:172.21.15.84:3300/0 conn(0x7f261c0611c0 0x7f261c155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:26.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.604+0000 7f2620f65640 1 -- 172.21.15.39:0/2706844130 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709971073 0 0) 0x7f261c15c380 con 0x7f261c155a70 2024-09-13T17:15:26.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.604+0000 7f2620f65640 1 -- 172.21.15.39:0/2706844130 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2604003530 con 0x7f261c155a70 2024-09-13T17:15:26.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.604+0000 7f2621766640 1 -- 172.21.15.39:0/2706844130 >> v2:172.21.15.39:3300/0 conn(0x7f261c071900 msgr2=0x7f261c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:26.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.604+0000 7f2620f65640 1 -- 172.21.15.39:0/2706844130 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3013597545 0 0) 0x7f2604003530 con 0x7f261c155a70 2024-09-13T17:15:26.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.604+0000 7f2621f67640 1 --2- 172.21.15.39:0/2706844130 >> v2:172.21.15.39:3300/0 conn(0x7f261c071900 0x7f261c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:26.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.604+0000 7f2621766640 1 --2- 172.21.15.39:0/2706844130 >> v2:172.21.15.39:3300/0 conn(0x7f261c071900 0x7f261c06fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:26.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.604+0000 7f2621766640 1 -- 172.21.15.39:0/2706844130 >> v1:172.21.15.84:6789/0 conn(0x7f261c155a70 legacy=0x7f261c06cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:26.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.604+0000 7f2621766640 1 -- 172.21.15.39:0/2706844130 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c155f00 con 0x7f261c0611c0 2024-09-13T17:15:26.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f2621f67640 1 --2- 172.21.15.39:0/2706844130 >> v2:172.21.15.39:3300/0 conn(0x7f261c071900 0x7f261c06fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:26.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f2621766640 1 --2- 172.21.15.39:0/2706844130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c0611c0 0x7f261c155530 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f261c06b890 tx=0x7f260c01c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4ce73e368fdb0ea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:26.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f2620f65640 1 -- 172.21.15.39:0/2706844130 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614001680 con 0x7f261c0611c0 2024-09-13T17:15:26.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f2620f65640 1 -- 172.21.15.39:0/2706844130 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c0b0400 con 0x7f261c0611c0 2024-09-13T17:15:26.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 -- 172.21.15.39:0/2706844130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c0611c0 msgr2=0x7f261c155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:26.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 --2- 172.21.15.39:0/2706844130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c0611c0 0x7f261c155530 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f261c06b890 tx=0x7f260c01c750 comp rx=0 tx=0).stop 2024-09-13T17:15:26.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 -- 172.21.15.39:0/2706844130 shutdown_connections 2024-09-13T17:15:26.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 --2- 172.21.15.39:0/2706844130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c0611c0 0x7f261c155530 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:26.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 --2- 172.21.15.39:0/2706844130 >> v2:172.21.15.39:3300/0 conn(0x7f261c071900 0x7f261c06fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:26.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 -- 172.21.15.39:0/2706844130 >> 172.21.15.39:0/2706844130 conn(0x7f261c069e10 msgr2=0x7f261c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:26.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 -- 172.21.15.39:0/2706844130 shutdown_connections 2024-09-13T17:15:26.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 -- 172.21.15.39:0/2706844130 wait complete. 2024-09-13T17:15:26.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 Processor -- start 2024-09-13T17:15:26.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.608+0000 7f26241cb640 1 -- start start 2024-09-13T17:15:26.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f26241cb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f261c071900 0x7f261c161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:26.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f26241cb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f261c155a70 0x7f261c157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:26.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f26241cb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c15fa10 0x7f261c15fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:26.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f26241cb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c14cea0 con 0x7f261c155a70 2024-09-13T17:15:26.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f26241cb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c15dc70 con 0x7f261c15fa10 2024-09-13T17:15:26.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f26241cb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c162560 con 0x7f261c071900 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2622768640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c15fa10 0x7f261c15fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2621f67640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f261c071900 0x7f261c161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2622768640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c15fa10 0x7f261c15fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:52880/0 (socket says 172.21.15.39:52880) 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2622768640 1 -- 172.21.15.39:0/3155026631 learned_addr learned my addr 172.21.15.39:0/3155026631 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2621766640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f261c155a70 0x7f261c157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2622768640 1 -- 172.21.15.39:0/3155026631 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f261c071900 msgr2=0x7f261c161960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2622768640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f261c071900 0x7f261c161960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2622768640 1 -- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f261c155a70 msgr2=0x7f261c157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2622768640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f261c155a70 0x7f261c157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:26.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2622768640 1 -- 172.21.15.39:0/3155026631 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c009a70 con 0x7f261c15fa10 2024-09-13T17:15:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2621f67640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f261c071900 0x7f261c161960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:15:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f2622768640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c15fa10 0x7f261c15fec0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f261400c750 tx=0x7f261400cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:26.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f261a7fc640 1 -- 172.21.15.39:0/3155026631 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614005ae0 con 0x7f261c15fa10 2024-09-13T17:15:26.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f261a7fc640 1 -- 172.21.15.39:0/3155026631 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614005c40 con 0x7f261c15fa10 2024-09-13T17:15:26.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f261a7fc640 1 -- 172.21.15.39:0/3155026631 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614007070 con 0x7f261c15fa10 2024-09-13T17:15:26.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c162840 con 0x7f261c15fa10 2024-09-13T17:15:26.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.612+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261c15e160 con 0x7f261c15fa10 2024-09-13T17:15:26.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.616+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261c15e350 con 0x7f261c15fa10 2024-09-13T17:15:26.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.628+0000 7f261a7fc640 1 -- 172.21.15.39:0/3155026631 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f26140071d0 con 0x7f261c15fa10 2024-09-13T17:15:26.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.628+0000 7f261a7fc640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2608066710 0x7f2608068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:26.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.628+0000 7f261a7fc640 1 -- 172.21.15.39:0/3155026631 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7f2614085820 con 0x7f261c15fa10 2024-09-13T17:15:26.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.628+0000 7f261a7fc640 1 -- 172.21.15.39:0/3155026631 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2614055220 con 0x7f261c15fa10 2024-09-13T17:15:26.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.628+0000 7f2621f67640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2608066710 0x7f2608068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:26.648+0000 7f2621f67640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2608066710 0x7f2608068bd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f261c1588e0 tx=0x7f2610008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:27.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.072+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f261c1629d0 con 0x7f261c15fa10 2024-09-13T17:15:27.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.072+0000 7f261a7fc640 1 -- 172.21.15.39:0/3155026631 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5938 (secure 0 0 0) 0x7f2614055400 con 0x7f261c15fa10 2024-09-13T17:15:27.078 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:27.078 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":187}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:53.952217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:27.079 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 187 2024-09-13T17:15:27.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2608066710 msgr2=0x7f2608068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:27.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2608066710 0x7f2608068bd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f261c1588e0 tx=0x7f2610008040 comp rx=0 tx=0).stop 2024-09-13T17:15:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c15fa10 msgr2=0x7f261c15fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c15fa10 0x7f261c15fec0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f261400c750 tx=0x7f261400cc10 comp rx=0 tx=0).stop 2024-09-13T17:15:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 shutdown_connections 2024-09-13T17:15:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2608066710 0x7f2608068bd0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f261c15fa10 0x7f261c15fec0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:27.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f261c155a70 0x7f261c157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 --2- 172.21.15.39:0/3155026631 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f261c071900 0x7f261c161960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 >> 172.21.15.39:0/3155026631 conn(0x7f261c069e10 msgr2=0x7f261c15ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:27.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 shutdown_connections 2024-09-13T17:15:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:27.088+0000 7f26241cb640 1 -- 172.21.15.39:0/3155026631 wait complete. 2024-09-13T17:15:27.130 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:15:29.131 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29adb2e640 1 Processor -- start 2024-09-13T17:15:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29adb2e640 1 -- start start 2024-09-13T17:15:29.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29adb2e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29a80c7800 0x7f29a8154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:29.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29adb2e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a80c5a30 con 0x7f29a8071dd0 2024-09-13T17:15:29.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29adb2e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a81576a0 con 0x7f29a8151500 2024-09-13T17:15:29.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29adb2e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a8157980 con 0x7f29a80c7800 2024-09-13T17:15:29.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29a7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29a80c7800 0x7f29a8154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:29.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f299ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f29a8071dd0 0x7f29a8154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48382/0 (socket says 172.21.15.39:48382) 2024-09-13T17:15:29.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f299ffff640 1 -- 172.21.15.39:0/2283859532 learned_addr learned my addr 172.21.15.39:0/2283859532 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:15:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29a6ffd640 1 -- 172.21.15.39:0/2283859532 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285153586 0 0) 0x7f29a81576a0 con 0x7f29a8151500 2024-09-13T17:15:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29a6ffd640 1 -- 172.21.15.39:0/2283859532 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2994003530 con 0x7f29a8151500 2024-09-13T17:15:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29a7fff640 1 -- 172.21.15.39:0/2283859532 >> v1:172.21.15.84:6789/0 conn(0x7f29a8151500 legacy=0x7f29a80c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29a7fff640 1 -- 172.21.15.39:0/2283859532 >> v1:172.21.15.39:6789/0 conn(0x7f29a8071dd0 legacy=0x7f29a8154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:29.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.328+0000 7f29a7fff640 1 -- 172.21.15.39:0/2283859532 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a8157c60 con 0x7f29a80c7800 2024-09-13T17:15:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29a7fff640 1 --2- 172.21.15.39:0/2283859532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a80c7800 0x7f29a8154780 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009dc0 tx=0x7f29a001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e4625e345bd462 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29a6ffd640 1 -- 172.21.15.39:0/2283859532 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a00a5970 con 0x7f29a80c7800 2024-09-13T17:15:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29a6ffd640 1 -- 172.21.15.39:0/2283859532 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a00a5ad0 con 0x7f29a80c7800 2024-09-13T17:15:29.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29a6ffd640 1 -- 172.21.15.39:0/2283859532 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a00a5e60 con 0x7f29a80c7800 2024-09-13T17:15:29.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29adb2e640 1 -- 172.21.15.39:0/2283859532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a80c7800 msgr2=0x7f29a8154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29adb2e640 1 --2- 172.21.15.39:0/2283859532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a80c7800 0x7f29a8154780 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009dc0 tx=0x7f29a001c890 comp rx=0 tx=0).stop 2024-09-13T17:15:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29adb2e640 1 -- 172.21.15.39:0/2283859532 shutdown_connections 2024-09-13T17:15:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29adb2e640 1 --2- 172.21.15.39:0/2283859532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a80c7800 0x7f29a8154780 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29adb2e640 1 -- 172.21.15.39:0/2283859532 >> 172.21.15.39:0/2283859532 conn(0x7f29a80ab530 msgr2=0x7f29a80ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29adb2e640 1 -- 172.21.15.39:0/2283859532 shutdown_connections 2024-09-13T17:15:29.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.332+0000 7f29adb2e640 1 -- 172.21.15.39:0/2283859532 wait complete. 2024-09-13T17:15:29.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29adb2e640 1 Processor -- start 2024-09-13T17:15:29.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29adb2e640 1 -- start start 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29adb2e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a8071dd0 0x7f29a814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29adb2e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29a80c7800 0x7f29a80aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a8071dd0 0x7f29a814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29adb2e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a8151500 0x7f29a8147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a8071dd0 0x7f29a814d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49134/0 (socket says 172.21.15.39:49134) 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a814ac60 con 0x7f29a8071dd0 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a77fe640 1 -- 172.21.15.39:0/4276554485 learned_addr learned my addr 172.21.15.39:0/4276554485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a81ecea0 con 0x7f29a8151500 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a81490a0 con 0x7f29a80c7800 2024-09-13T17:15:29.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f299ffff640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29a80c7800 0x7f29a80aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a7fff640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a8151500 0x7f29a8147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a77fe640 1 -- 172.21.15.39:0/4276554485 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29a80c7800 msgr2=0x7f29a80aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a77fe640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29a80c7800 0x7f29a80aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a77fe640 1 -- 172.21.15.39:0/4276554485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a8151500 msgr2=0x7f29a8147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a77fe640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a8151500 0x7f29a8147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a77fe640 1 -- 172.21.15.39:0/4276554485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009a70 con 0x7f29a8071dd0 2024-09-13T17:15:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f299ffff640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29a80c7800 0x7f29a80aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:15:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.336+0000 7f29a77fe640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a8071dd0 0x7f29a814d2c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f299800c410 tx=0x7f299800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:29.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f29a4ff9640 1 -- 172.21.15.39:0/4276554485 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980052d0 con 0x7f29a8071dd0 2024-09-13T17:15:29.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f29a4ff9640 1 -- 172.21.15.39:0/4276554485 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2998005450 con 0x7f29a8071dd0 2024-09-13T17:15:29.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f29a4ff9640 1 -- 172.21.15.39:0/4276554485 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998003820 con 0x7f29a8071dd0 2024-09-13T17:15:29.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a8149380 con 0x7f29a8071dd0 2024-09-13T17:15:29.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a81ecb90 con 0x7f29a8071dd0 2024-09-13T17:15:29.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f29a4ff9640 1 -- 172.21.15.39:0/4276554485 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2998003980 con 0x7f29a8071dd0 2024-09-13T17:15:29.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a8148ef0 con 0x7f29a8071dd0 2024-09-13T17:15:29.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f29a4ff9640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29880666e0 0x7f2988068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:29.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f29a4ff9640 1 -- 172.21.15.39:0/4276554485 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 9733+0+0 (secure 0 0 0) 0x7f29980845e0 con 0x7f29a8071dd0 2024-09-13T17:15:29.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.340+0000 7f299ffff640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29880666e0 0x7f2988068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:29.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.344+0000 7f299ffff640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29880666e0 0x7f2988068ba0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f29a81e9f80 tx=0x7f2990009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:29.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.360+0000 7f29a4ff9640 1 -- 172.21.15.39:0/4276554485 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2998050c00 con 0x7f29a8071dd0 2024-09-13T17:15:29.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.792+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29a8157320 con 0x7f29a8071dd0 2024-09-13T17:15:29.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.796+0000 7f29a4ff9640 1 -- 172.21.15.39:0/4276554485 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5938 (secure 0 0 0) 0x7f2998017020 con 0x7f29a8071dd0 2024-09-13T17:15:29.802 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:29.802 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":187}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:14:53.952217+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7375":{"gid":7375,"name":"c","rank":0,"incarnation":183,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6837/1355102409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1355102409},{"type":"v1","addr":"172.21.15.39:6837","nonce":1355102409}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 187 2024-09-13T17:15:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.800+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29880666e0 msgr2=0x7f2988068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.800+0000 7f29adb2e640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29880666e0 0x7f2988068ba0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f29a81e9f80 tx=0x7f2990009040 comp rx=0 tx=0).stop 2024-09-13T17:15:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.804+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a8071dd0 msgr2=0x7f29a814d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:29.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.804+0000 7f29adb2e640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a8071dd0 0x7f29a814d2c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f299800c410 tx=0x7f299800c8d0 comp rx=0 tx=0).stop 2024-09-13T17:15:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.808+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 shutdown_connections 2024-09-13T17:15:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.808+0000 7f29adb2e640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29880666e0 0x7f2988068ba0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.808+0000 7f29adb2e640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29a8151500 0x7f29a8147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.808+0000 7f29adb2e640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29a80c7800 0x7f29a80aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.808+0000 7f29adb2e640 1 --2- 172.21.15.39:0/4276554485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29a8071dd0 0x7f29a814d2c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.808+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 >> 172.21.15.39:0/4276554485 conn(0x7f29a80ab530 msgr2=0x7f29a80ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.808+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 shutdown_connections 2024-09-13T17:15:29.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:29.808+0000 7f29adb2e640 1 -- 172.21.15.39:0/4276554485 wait complete. 2024-09-13T17:15:29.859 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-09-13T17:15:31.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:32.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.092+0000 7f820a947640 1 Processor -- start 2024-09-13T17:15:32.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.092+0000 7f820a947640 1 -- start start 2024-09-13T17:15:32.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f820a947640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f82040c1480 0x7f820414d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:32.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f820a947640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82040c0270 0x7f8204150110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:32.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f820a947640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820406d1a0 con 0x7f820406b6c0 2024-09-13T17:15:32.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f820a947640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8204151020 con 0x7f82040c1480 2024-09-13T17:15:32.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f820a947640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820414dd60 con 0x7f82040c0270 2024-09-13T17:15:32.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f8209144640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82040c0270 0x7f8204150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:32.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f820a146640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f820406b6c0 0x7f8204153680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34156/0 (socket says 172.21.15.39:34156) 2024-09-13T17:15:32.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f820a146640 1 -- 172.21.15.39:0/4192824427 learned_addr learned my addr 172.21.15.39:0/4192824427 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:15:32.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f8209945640 1 --2- 172.21.15.39:0/4192824427 >> v2:172.21.15.39:3300/0 conn(0x7f82040c1480 0x7f820414d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:32.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f8209144640 1 -- 172.21.15.39:0/4192824427 >> v2:172.21.15.39:3300/0 conn(0x7f82040c1480 msgr2=0x7f820414d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:32.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f8209144640 1 --2- 172.21.15.39:0/4192824427 >> v2:172.21.15.39:3300/0 conn(0x7f82040c1480 0x7f820414d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f8209144640 1 -- 172.21.15.39:0/4192824427 >> v1:172.21.15.39:6789/0 conn(0x7f820406b6c0 legacy=0x7f8204153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:32.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f8209144640 1 -- 172.21.15.39:0/4192824427 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040c31f0 con 0x7f82040c0270 2024-09-13T17:15:32.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.096+0000 7f8208943640 1 -- 172.21.15.39:0/4192824427 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608232470 0 0) 0x7f820406d1a0 con 0x7f820406b6c0 2024-09-13T17:15:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.100+0000 7f8209144640 1 --2- 172.21.15.39:0/4192824427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c0270 0x7f8204150110 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f82040a4cf0 tx=0x7f820001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eba7002e937a3b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.100+0000 7f8208943640 1 -- 172.21.15.39:0/4192824427 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000ab600 con 0x7f82040c0270 2024-09-13T17:15:32.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.100+0000 7f8208943640 1 -- 172.21.15.39:0/4192824427 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82000a7050 con 0x7f82040c0270 2024-09-13T17:15:32.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.100+0000 7f8208943640 1 -- 172.21.15.39:0/4192824427 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000ab980 con 0x7f82040c0270 2024-09-13T17:15:32.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.100+0000 7f820a947640 1 -- 172.21.15.39:0/4192824427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c0270 msgr2=0x7f8204150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:32.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.100+0000 7f820a947640 1 --2- 172.21.15.39:0/4192824427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c0270 0x7f8204150110 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f82040a4cf0 tx=0x7f820001c330 comp rx=0 tx=0).stop 2024-09-13T17:15:32.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.108+0000 7f820a947640 1 -- 172.21.15.39:0/4192824427 shutdown_connections 2024-09-13T17:15:32.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.108+0000 7f820a947640 1 --2- 172.21.15.39:0/4192824427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c0270 0x7f8204150110 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.108+0000 7f820a947640 1 --2- 172.21.15.39:0/4192824427 >> v2:172.21.15.39:3300/0 conn(0x7f82040c1480 0x7f820414d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.108+0000 7f820a947640 1 -- 172.21.15.39:0/4192824427 >> 172.21.15.39:0/4192824427 conn(0x7f82040c5870 msgr2=0x7f82040c5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:32.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 -- 172.21.15.39:0/4192824427 shutdown_connections 2024-09-13T17:15:32.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 -- 172.21.15.39:0/4192824427 wait complete. 2024-09-13T17:15:32.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 Processor -- start 2024-09-13T17:15:32.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 -- start start 2024-09-13T17:15:32.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820406b6c0 0x7f82041f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:32.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f82040c0270 0x7f820413fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:32.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c1480 0x7f820413ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:32.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8204072510 con 0x7f820406b6c0 2024-09-13T17:15:32.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82041f1a10 con 0x7f82040c1480 2024-09-13T17:15:32.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a947640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82041f46c0 con 0x7f82040c0270 2024-09-13T17:15:32.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a146640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c1480 0x7f820413ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:32.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a146640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c1480 0x7f820413ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55034/0 (socket says 172.21.15.39:55034) 2024-09-13T17:15:32.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.116+0000 7f820a146640 1 -- 172.21.15.39:0/3873469103 learned_addr learned my addr 172.21.15.39:0/3873469103 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f8209945640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820406b6c0 0x7f82041f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f820a146640 1 -- 172.21.15.39:0/3873469103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f82040c0270 msgr2=0x7f820413fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f820a146640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f82040c0270 0x7f820413fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f820a146640 1 -- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820406b6c0 msgr2=0x7f82041f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f820a146640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820406b6c0 0x7f82041f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f820a146640 1 -- 172.21.15.39:0/3873469103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8200009420 con 0x7f82040c1480 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f8209945640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820406b6c0 0x7f82041f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f820a146640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c1480 0x7f820413ff70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00c780 tx=0x7f81fc00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f81fa7fc640 1 -- 172.21.15.39:0/3873469103 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00ce90 con 0x7f82040c1480 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f81fa7fc640 1 -- 172.21.15.39:0/3873469103 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc00a640 con 0x7f82040c1480 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82041f1c60 con 0x7f82040c1480 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f81fa7fc640 1 -- 172.21.15.39:0/3873469103 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00a9d0 con 0x7f82040c1480 2024-09-13T17:15:32.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82040721a0 con 0x7f82040c1480 2024-09-13T17:15:32.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f81fa7fc640 1 -- 172.21.15.39:0/3873469103 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f81fc00ab30 con 0x7f82040c1480 2024-09-13T17:15:32.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f81fa7fc640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81e80666c0 0x7f81e8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:32.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f81fa7fc640 1 -- 172.21.15.39:0/3873469103 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7f81fc084900 con 0x7f82040c1480 2024-09-13T17:15:32.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.120+0000 7f8209945640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81e80666c0 0x7f81e8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:32.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.124+0000 7f8209945640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81e80666c0 0x7f81e8068b80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009ae0 tx=0x7f81f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:32.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.124+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8204150110 con 0x7f82040c1480 2024-09-13T17:15:32.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.136+0000 7f81fa7fc640 1 -- 172.21.15.39:0/3873469103 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f81fc018020 con 0x7f82040c1480 2024-09-13T17:15:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.712+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82041f2110 con 0x7f82040c1480 2024-09-13T17:15:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.716+0000 7f81fa7fc640 1 -- 172.21.15.39:0/3873469103 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+5182 (secure 0 0 0) 0x7f81fc050ca0 con 0x7f82040c1480 2024-09-13T17:15:32.719 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:32.719 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":189,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:32.142963+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:reconnect","state_seq":9,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:32.719 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 189 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.716+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81e80666c0 msgr2=0x7f81e8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.716+0000 7f820a947640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81e80666c0 0x7f81e8068b80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009ae0 tx=0x7f81f4008040 comp rx=0 tx=0).stop 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c1480 msgr2=0x7f820413ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c1480 0x7f820413ff70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00c780 tx=0x7f81fc00cc40 comp rx=0 tx=0).stop 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 shutdown_connections 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f81e80666c0 0x7f81e8068b80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82040c1480 0x7f820413ff70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f82040c0270 0x7f820413fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 --2- 172.21.15.39:0/3873469103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820406b6c0 0x7f82041f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 >> 172.21.15.39:0/3873469103 conn(0x7f82040c5870 msgr2=0x7f820415e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:32.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 shutdown_connections 2024-09-13T17:15:32.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.720+0000 7f820a947640 1 -- 172.21.15.39:0/3873469103 wait complete. 2024-09-13T17:15:32.781 INFO:tasks.mds_thrash.fs.[b]:mds.c down, removed from mdsmap 2024-09-13T17:15:32.781 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:15:32.781 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:32.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410fafe640 1 Processor -- start 2024-09-13T17:15:32.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410fafe640 1 -- start start 2024-09-13T17:15:32.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410fafe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f41080c20d0 0x7f41080c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:32.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410fafe640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4108157370 con 0x7f410806d370 2024-09-13T17:15:32.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410fafe640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4108153df0 con 0x7f41080c1790 2024-09-13T17:15:32.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410fafe640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080c04a0 con 0x7f41080c20d0 2024-09-13T17:15:32.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410d89a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f41080c20d0 0x7f41080c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:32.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410d89a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f41080c20d0 0x7f41080c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55038/0 (socket says 172.21.15.39:55038) 2024-09-13T17:15:32.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410d89a640 1 -- 172.21.15.39:0/2076756417 learned_addr learned my addr 172.21.15.39:0/2076756417 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:32.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410d89a640 1 -- 172.21.15.39:0/2076756417 >> v1:172.21.15.84:6789/0 conn(0x7f41080c1790 legacy=0x7f4108157800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:15:32.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410d89a640 1 -- 172.21.15.39:0/2076756417 >> v1:172.21.15.39:6789/0 conn(0x7f410806d370 legacy=0x7f410815e700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:15:32.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.984+0000 7f410d89a640 1 -- 172.21.15.39:0/2076756417 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410806c700 con 0x7f41080c20d0 2024-09-13T17:15:32.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.992+0000 7f410d89a640 1 --2- 172.21.15.39:0/2076756417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c20d0 0x7f41080c24b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4104009870 tx=0x7f410401c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f48f96a32b9ac11c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:32.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.992+0000 7f410c898640 1 -- 172.21.15.39:0/2076756417 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040ac620 con 0x7f41080c20d0 2024-09-13T17:15:32.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.992+0000 7f410c898640 1 -- 172.21.15.39:0/2076756417 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41040ac780 con 0x7f41080c20d0 2024-09-13T17:15:32.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.992+0000 7f410c898640 1 -- 172.21.15.39:0/2076756417 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040acb10 con 0x7f41080c20d0 2024-09-13T17:15:32.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 -- 172.21.15.39:0/2076756417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c20d0 msgr2=0x7f41080c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:32.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 --2- 172.21.15.39:0/2076756417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c20d0 0x7f41080c24b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4104009870 tx=0x7f410401c4b0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 -- 172.21.15.39:0/2076756417 shutdown_connections 2024-09-13T17:15:32.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 --2- 172.21.15.39:0/2076756417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c20d0 0x7f41080c24b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:32.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 -- 172.21.15.39:0/2076756417 >> 172.21.15.39:0/2076756417 conn(0x7f4108069e10 msgr2=0x7f410806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:32.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 -- 172.21.15.39:0/2076756417 shutdown_connections 2024-09-13T17:15:32.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 -- 172.21.15.39:0/2076756417 wait complete. 2024-09-13T17:15:33.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 Processor -- start 2024-09-13T17:15:33.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 -- start start 2024-09-13T17:15:33.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f410806d370 0x7f410814a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:33.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c1790 0x7f41080a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:33.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f41081512b0 0x7f4108147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:33.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410806f5a0 con 0x7f410806d370 2024-09-13T17:15:33.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410806f7f0 con 0x7f41080c1790 2024-09-13T17:15:33.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410fafe640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410806fad0 con 0x7f41081512b0 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410d099640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c1790 0x7f41080a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410e09b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f41081512b0 0x7f4108147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410d099640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c1790 0x7f41080a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:55050/0 (socket says 172.21.15.39:55050) 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410d099640 1 -- 172.21.15.39:0/2692532079 learned_addr learned my addr 172.21.15.39:0/2692532079 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410d099640 1 -- 172.21.15.39:0/2692532079 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f41081512b0 msgr2=0x7f4108147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410d099640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f41081512b0 0x7f4108147580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410d099640 1 -- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f410806d370 msgr2=0x7f410814a660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410d89a640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f410806d370 0x7f410814a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410d099640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f410806d370 0x7f410814a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:32.996+0000 7f410d099640 1 -- 172.21.15.39:0/2692532079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41040094d0 con 0x7f41080c1790 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f410d89a640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f410806d370 0x7f410814a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:33.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f410d099640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c1790 0x7f41080a6770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f40f800b4a0 tx=0x7f40f800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:33.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f40fe7fc640 1 -- 172.21.15.39:0/2692532079 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8010610 con 0x7f41080c1790 2024-09-13T17:15:33.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f40fe7fc640 1 -- 172.21.15.39:0/2692532079 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f8010770 con 0x7f41080c1790 2024-09-13T17:15:33.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f40fe7fc640 1 -- 172.21.15.39:0/2692532079 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8010b60 con 0x7f41080c1790 2024-09-13T17:15:33.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410806fdb0 con 0x7f41080c1790 2024-09-13T17:15:33.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41080a6cb0 con 0x7f41080c1790 2024-09-13T17:15:33.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f40fe7fc640 1 -- 172.21.15.39:0/2692532079 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f40f8010cc0 con 0x7f41080c1790 2024-09-13T17:15:33.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f40fe7fc640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f40ec0666e0 0x7f40ec068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:33.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f40fe7fc640 1 -- 172.21.15.39:0/2692532079 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7f40f8084d70 con 0x7f41080c1790 2024-09-13T17:15:33.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.000+0000 7f410d89a640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f40ec0666e0 0x7f40ec068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:33.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.004+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410806f930 con 0x7f41080c1790 2024-09-13T17:15:33.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.004+0000 7f410d89a640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f40ec0666e0 0x7f40ec068ba0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f41040ae000 tx=0x7f4104008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:33.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.012+0000 7f40fe7fc640 1 -- 172.21.15.39:0/2692532079 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f410806f930 con 0x7f41080c1790 2024-09-13T17:15:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.548+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41080702d0 con 0x7f41080c1790 2024-09-13T17:15:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.552+0000 7f40fe7fc640 1 -- 172.21.15.39:0/2692532079 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) v1 ==== 95+0+5180 (secure 0 0 0) 0x7f41080702d0 con 0x7f41080c1790 2024-09-13T17:15:33.556 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:33.556 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":190,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:33.152024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 190 2024-09-13T17:15:33.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f40ec0666e0 msgr2=0x7f40ec068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:33.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f40ec0666e0 0x7f40ec068ba0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f41040ae000 tx=0x7f4104008040 comp rx=0 tx=0).stop 2024-09-13T17:15:33.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c1790 msgr2=0x7f41080a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:33.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c1790 0x7f41080a6770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f40f800b4a0 tx=0x7f40f800b960 comp rx=0 tx=0).stop 2024-09-13T17:15:33.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 shutdown_connections 2024-09-13T17:15:33.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f40ec0666e0 0x7f40ec068ba0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:33.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41080c1790 0x7f41080a6770 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:33.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f41081512b0 0x7f4108147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:33.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 --2- 172.21.15.39:0/2692532079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f410806d370 0x7f410814a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:33.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.556+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 >> 172.21.15.39:0/2692532079 conn(0x7f4108069e10 msgr2=0x7f4108157800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:33.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.560+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 shutdown_connections 2024-09-13T17:15:33.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:33.560+0000 7f410fafe640 1 -- 172.21.15.39:0/2692532079 wait complete. 2024-09-13T17:15:33.591 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 190, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 181, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6530": { "addr": "172.21.15.84:6835/380008541", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 380008541, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 380008541, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6530, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 27 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:14:08.379706+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6530 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 190, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_8220": { "addr": "172.21.15.39:6835/3721635324", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 3721635324, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 3721635324, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8220, "incarnation": 188, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 61, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:15:33.152024+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8220 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:15:35.359 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:15:35.361 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:15:35.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:36.004 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T17:15:36.000+0000 7f4256448640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:15:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.128+0000 7f4a4d865640 1 Processor -- start 2024-09-13T17:15:36.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.128+0000 7f4a4d865640 1 -- start start 2024-09-13T17:15:36.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.132+0000 7f4a4d865640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4a480c4520 0x7f4a480a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:36.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.132+0000 7f4a4d865640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480c38a0 con 0x7f4a48153a30 2024-09-13T17:15:36.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.132+0000 7f4a4d865640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480c02a0 con 0x7f4a480c4c10 2024-09-13T17:15:36.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.132+0000 7f4a4d865640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a480c4390 con 0x7f4a480c4520 2024-09-13T17:15:36.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.140+0000 7f4a477fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4a48153a30 0x7f4a48153e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34162/0 (socket says 172.21.15.39:34162) 2024-09-13T17:15:36.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.140+0000 7f4a477fe640 1 -- 172.21.15.39:0/2449577168 learned_addr learned my addr 172.21.15.39:0/2449577168 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:15:36.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068121989 0 0) 0x7f4a480c38a0 con 0x7f4a48153a30 2024-09-13T17:15:36.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a30003530 con 0x7f4a48153a30 2024-09-13T17:15:36.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 214054669 0 0) 0x7f4a480c02a0 con 0x7f4a480c4c10 2024-09-13T17:15:36.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a480c38a0 con 0x7f4a480c4c10 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 71394103 0 0) 0x7f4a30003530 con 0x7f4a48153a30 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 >> v2:172.21.15.39:3300/0 conn(0x7f4a480c4520 msgr2=0x7f4a480a8db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 --2- 172.21.15.39:0/2449577168 >> v2:172.21.15.39:3300/0 conn(0x7f4a480c4520 0x7f4a480a8db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 >> v1:172.21.15.84:6789/0 conn(0x7f4a480c4c10 legacy=0x7f4a480c0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480c0580 con 0x7f4a48153a30 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4a3c001600 con 0x7f4a48153a30 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 >> v1:172.21.15.39:6789/0 conn(0x7f4a48153a30 legacy=0x7f4a48153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 --2- 172.21.15.39:0/2449577168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a30003770 0x7f4a30003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 --2- 172.21.15.39:0/2449577168 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a30004150 0x7f4a3000b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 --2- 172.21.15.39:0/2449577168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3000b6f0 0x7f4a3000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480c4390 con 0x7f4a3000b6f0 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000e470 con 0x7f4a30003770 2024-09-13T17:15:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000e630 con 0x7f4a30004150 2024-09-13T17:15:36.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.144+0000 7f4a477fe640 1 --2- 172.21.15.39:0/2449577168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3000b6f0 0x7f4a3000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.148+0000 7f4a477fe640 1 -- 172.21.15.39:0/2449577168 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a30004150 msgr2=0x7f4a3000b020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:15:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.148+0000 7f4a477fe640 1 --2- 172.21.15.39:0/2449577168 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a30004150 0x7f4a3000b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.148+0000 7f4a477fe640 1 -- 172.21.15.39:0/2449577168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a30003770 msgr2=0x7f4a30003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.148+0000 7f4a477fe640 1 --2- 172.21.15.39:0/2449577168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a30003770 0x7f4a30003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.148+0000 7f4a477fe640 1 -- 172.21.15.39:0/2449577168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30003530 con 0x7f4a3000b6f0 2024-09-13T17:15:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.148+0000 7f4a477fe640 1 -- 172.21.15.39:0/2449577168 reap_dead start 2024-09-13T17:15:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.160+0000 7f4a477fe640 1 --2- 172.21.15.39:0/2449577168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3000b6f0 0x7f4a3000db90 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c007f30 tx=0x7f4a3c01cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb36af8919c1e1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c005ae0 con 0x7f4a3000b6f0 2024-09-13T17:15:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c005c60 con 0x7f4a3000b6f0 2024-09-13T17:15:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a45ffb640 1 -- 172.21.15.39:0/2449577168 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0a57e0 con 0x7f4a3000b6f0 2024-09-13T17:15:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a4d865640 1 -- 172.21.15.39:0/2449577168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3000b6f0 msgr2=0x7f4a3000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a4d865640 1 --2- 172.21.15.39:0/2449577168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3000b6f0 0x7f4a3000db90 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c007f30 tx=0x7f4a3c01cae0 comp rx=0 tx=0).stop 2024-09-13T17:15:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a4d865640 1 -- 172.21.15.39:0/2449577168 shutdown_connections 2024-09-13T17:15:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a4d865640 1 --2- 172.21.15.39:0/2449577168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a3000b6f0 0x7f4a3000db90 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a4d865640 1 -- 172.21.15.39:0/2449577168 >> 172.21.15.39:0/2449577168 conn(0x7f4a48069e10 msgr2=0x7f4a4806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a4d865640 1 -- 172.21.15.39:0/2449577168 shutdown_connections 2024-09-13T17:15:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.164+0000 7f4a4d865640 1 -- 172.21.15.39:0/2449577168 wait complete. 2024-09-13T17:15:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.168+0000 7f4a4d865640 1 Processor -- start 2024-09-13T17:15:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.168+0000 7f4a4d865640 1 -- start start 2024-09-13T17:15:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.168+0000 7f4a4d865640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a3000b6f0 0x7f4a480a8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.168+0000 7f4a4d865640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a480a44f0 0x7f4a480a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.168+0000 7f4a4d865640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a480a73d0 0x7f4a4814fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.168+0000 7f4a4d865640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480a6470 con 0x7f4a480a44f0 2024-09-13T17:15:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.168+0000 7f4a4d865640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480702c0 con 0x7f4a480a73d0 2024-09-13T17:15:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.168+0000 7f4a4d865640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48070b60 con 0x7f4a3000b6f0 2024-09-13T17:15:36.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.172+0000 7f4a46ffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a3000b6f0 0x7f4a480a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:36.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.172+0000 7f4a46ffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a3000b6f0 0x7f4a480a8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:58280/0 (socket says 172.21.15.39:58280) 2024-09-13T17:15:36.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.172+0000 7f4a46ffd640 1 -- 172.21.15.39:0/1577552452 learned_addr learned my addr 172.21.15.39:0/1577552452 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:36.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.172+0000 7f4a46ffd640 1 -- 172.21.15.39:0/1577552452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a480a73d0 msgr2=0x7f4a4814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:36.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.172+0000 7f4a46ffd640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a480a73d0 0x7f4a4814fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:36.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.172+0000 7f4a46ffd640 1 -- 172.21.15.39:0/1577552452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a480a44f0 msgr2=0x7f4a480a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:36.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.172+0000 7f4a46ffd640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a480a44f0 0x7f4a480a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:36.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.172+0000 7f4a46ffd640 1 -- 172.21.15.39:0/1577552452 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480a6660 con 0x7f4a3000b6f0 2024-09-13T17:15:36.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.172+0000 7f4a46ffd640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a3000b6f0 0x7f4a480a8050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000c090 tx=0x7f4a4000c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:36.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.180+0000 7f4a2ffff640 1 -- 172.21.15.39:0/1577552452 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40003b80 con 0x7f4a3000b6f0 2024-09-13T17:15:36.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.192+0000 7f4a2ffff640 1 -- 172.21.15.39:0/1577552452 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40003ce0 con 0x7f4a3000b6f0 2024-09-13T17:15:36.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.192+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a48070510 con 0x7f4a3000b6f0 2024-09-13T17:15:36.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.192+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a480a8590 con 0x7f4a3000b6f0 2024-09-13T17:15:36.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.192+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a480c3ff0 con 0x7f4a3000b6f0 2024-09-13T17:15:36.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.192+0000 7f4a2ffff640 1 -- 172.21.15.39:0/1577552452 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40005270 con 0x7f4a3000b6f0 2024-09-13T17:15:36.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.204+0000 7f4a2ffff640 1 -- 172.21.15.39:0/1577552452 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4a400053d0 con 0x7f4a3000b6f0 2024-09-13T17:15:36.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.204+0000 7f4a2ffff640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a34066760 0x7f4a34068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:36.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.204+0000 7f4a2ffff640 1 -- 172.21.15.39:0/1577552452 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7f4a400879b0 con 0x7f4a3000b6f0 2024-09-13T17:15:36.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.204+0000 7f4a467fc640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a34066760 0x7f4a34068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:36.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.204+0000 7f4a2ffff640 1 -- 172.21.15.39:0/1577552452 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4a400b2e50 con 0x7f4a3000b6f0 2024-09-13T17:15:36.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:36.208+0000 7f4a467fc640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a34066760 0x7f4a34068c20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4a481eaee0 tx=0x7f4a38006be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:37.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.404+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a48061470 con 0x7f4a3000b6f0 2024-09-13T17:15:37.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.408+0000 7f4a2ffff640 1 -- 172.21.15.39:0/1577552452 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) v1 ==== 95+0+5180 (secure 0 0 0) 0x7f4a40023090 con 0x7f4a3000b6f0 2024-09-13T17:15:37.424 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:37.424 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":191,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:34.158161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:active","state_seq":11,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:37.425 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 191 2024-09-13T17:15:37.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a34066760 msgr2=0x7f4a34068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:37.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a34066760 0x7f4a34068c20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4a481eaee0 tx=0x7f4a38006be0 comp rx=0 tx=0).stop 2024-09-13T17:15:37.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a3000b6f0 msgr2=0x7f4a480a8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:37.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a3000b6f0 0x7f4a480a8050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000c090 tx=0x7f4a4000c550 comp rx=0 tx=0).stop 2024-09-13T17:15:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 shutdown_connections 2024-09-13T17:15:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4a34066760 0x7f4a34068c20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4a480a73d0 0x7f4a4814fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a480a44f0 0x7f4a480a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 --2- 172.21.15.39:0/1577552452 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4a3000b6f0 0x7f4a480a8050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 >> 172.21.15.39:0/1577552452 conn(0x7f4a48069e10 msgr2=0x7f4a480c2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:37.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 shutdown_connections 2024-09-13T17:15:37.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:37.464+0000 7f4a4d865640 1 -- 172.21.15.39:0/1577552452 wait complete. 2024-09-13T17:15:37.588 INFO:tasks.mds_thrash.fs.[b]:mds.a has gained rank=0, replacing gid=7375 2024-09-13T17:15:37.588 INFO:tasks.mds_thrash.fs.[b]:waiting for 5 secs before reviving mds.c 2024-09-13T17:15:42.589 INFO:tasks.mds_thrash.fs.[b]:reviving mds.c 2024-09-13T17:15:42.589 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:15:42.589 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:15:42.592 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:15:42.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:42.899 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T17:15:43.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.048+0000 7f29d69dc640 1 Processor -- start 2024-09-13T17:15:43.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.048+0000 7f29d69dc640 1 -- start start 2024-09-13T17:15:43.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.048+0000 7f29d69dc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d015e270 0x7f29d00ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:43.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.048+0000 7f29d69dc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f29d0164ad0 0x7f29d016b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:43.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.048+0000 7f29d69dc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d00c8f10 con 0x7f29d00c9330 2024-09-13T17:15:43.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.048+0000 7f29d69dc640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d0163f70 con 0x7f29d0164ad0 2024-09-13T17:15:43.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.048+0000 7f29d69dc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01614b0 con 0x7f29d015e270 2024-09-13T17:15:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.052+0000 7f29d59da640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d015e270 0x7f29d00ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.052+0000 7f29d59da640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d015e270 0x7f29d00ab610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:40622/0 (socket says 172.21.15.39:40622) 2024-09-13T17:15:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.052+0000 7f29d59da640 1 -- 172.21.15.39:0/2833417301 learned_addr learned my addr 172.21.15.39:0/2833417301 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.052+0000 7f29d59da640 1 -- 172.21.15.39:0/2833417301 >> v2:172.21.15.84:3300/0 conn(0x7f29d0164ad0 msgr2=0x7f29d016b1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:15:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.052+0000 7f29d59da640 1 --2- 172.21.15.39:0/2833417301 >> v2:172.21.15.84:3300/0 conn(0x7f29d0164ad0 0x7f29d016b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.052+0000 7f29d59da640 1 -- 172.21.15.39:0/2833417301 >> v1:172.21.15.84:6789/0 conn(0x7f29d00c9330 legacy=0x7f29d01643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.052+0000 7f29d59da640 1 -- 172.21.15.39:0/2833417301 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d0161790 con 0x7f29d015e270 2024-09-13T17:15:43.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.052+0000 7f29d59da640 1 --2- 172.21.15.39:0/2833417301 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d015e270 0x7f29d00ab610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009870 tx=0x7f29cc01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b569ea0ba3645119 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:43.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.056+0000 7f29d49d8640 1 -- 172.21.15.39:0/2833417301 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0a6040 con 0x7f29d015e270 2024-09-13T17:15:43.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.056+0000 7f29d49d8640 1 -- 172.21.15.39:0/2833417301 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc0aa600 con 0x7f29d015e270 2024-09-13T17:15:43.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.056+0000 7f29d49d8640 1 -- 172.21.15.39:0/2833417301 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0aa9d0 con 0x7f29d015e270 2024-09-13T17:15:43.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.068+0000 7f29d69dc640 1 -- 172.21.15.39:0/2833417301 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d015e270 msgr2=0x7f29d00ab610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:43.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.068+0000 7f29d69dc640 1 --2- 172.21.15.39:0/2833417301 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d015e270 0x7f29d00ab610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009870 tx=0x7f29cc01c480 comp rx=0 tx=0).stop 2024-09-13T17:15:43.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.072+0000 7f29d69dc640 1 -- 172.21.15.39:0/2833417301 shutdown_connections 2024-09-13T17:15:43.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.072+0000 7f29d69dc640 1 --2- 172.21.15.39:0/2833417301 >> v2:172.21.15.84:3300/0 conn(0x7f29d0164ad0 0x7f29d016b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:43.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.072+0000 7f29d69dc640 1 --2- 172.21.15.39:0/2833417301 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d015e270 0x7f29d00ab610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:43.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.072+0000 7f29d69dc640 1 -- 172.21.15.39:0/2833417301 >> 172.21.15.39:0/2833417301 conn(0x7f29d00c9b10 msgr2=0x7f29d00c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:43.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.072+0000 7f29d69dc640 1 -- 172.21.15.39:0/2833417301 shutdown_connections 2024-09-13T17:15:43.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d69dc640 1 -- 172.21.15.39:0/2833417301 wait complete. 2024-09-13T17:15:43.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d69dc640 1 Processor -- start 2024-09-13T17:15:43.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d69dc640 1 -- start start 2024-09-13T17:15:43.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d69dc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d00c9330 0x7f29d015d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:43.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d69dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d015e270 0x7f29d015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:43.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d59da640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d00c9330 0x7f29d015d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:43.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d59da640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d00c9330 0x7f29d015d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41282/0 (socket says 172.21.15.39:41282) 2024-09-13T17:15:43.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d59da640 1 -- 172.21.15.39:0/2365355557 learned_addr learned my addr 172.21.15.39:0/2365355557 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:43.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d69dc640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d0164ad0 0x7f29d00bd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:43.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d69dc640 1 -- 172.21.15.39:0/2365355557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d015dc10 con 0x7f29d00c9330 2024-09-13T17:15:43.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d69dc640 1 -- 172.21.15.39:0/2365355557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01fa500 con 0x7f29d015e270 2024-09-13T17:15:43.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.076+0000 7f29d69dc640 1 -- 172.21.15.39:0/2365355557 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01faa30 con 0x7f29d0164ad0 2024-09-13T17:15:43.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d51d9640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d015e270 0x7f29d015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:43.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d59da640 1 -- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d0164ad0 msgr2=0x7f29d00bd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:43.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d61db640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d0164ad0 0x7f29d00bd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:43.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d59da640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d0164ad0 0x7f29d00bd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:43.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d59da640 1 -- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d015e270 msgr2=0x7f29d015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d59da640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d015e270 0x7f29d015d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d59da640 1 -- 172.21.15.39:0/2365355557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc0094d0 con 0x7f29d00c9330 2024-09-13T17:15:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d61db640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d0164ad0 0x7f29d00bd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d59da640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d00c9330 0x7f29d015d190 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f29cc01ab30 tx=0x7f29cc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29c67fc640 1 -- 172.21.15.39:0/2365355557 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0b3600 con 0x7f29d00c9330 2024-09-13T17:15:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d69dc640 1 -- 172.21.15.39:0/2365355557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d00c00e0 con 0x7f29d00c9330 2024-09-13T17:15:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d69dc640 1 -- 172.21.15.39:0/2365355557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d00bfa90 con 0x7f29d00c9330 2024-09-13T17:15:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29c67fc640 1 -- 172.21.15.39:0/2365355557 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc0b3760 con 0x7f29d00c9330 2024-09-13T17:15:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29c67fc640 1 -- 172.21.15.39:0/2365355557 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0b3ad0 con 0x7f29d00c9330 2024-09-13T17:15:43.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.080+0000 7f29d51d9640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d015e270 0x7f29d015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:15:43.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.088+0000 7f29c67fc640 1 -- 172.21.15.39:0/2365355557 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29cc0b3c30 con 0x7f29d00c9330 2024-09-13T17:15:43.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.096+0000 7f29c67fc640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:43.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.096+0000 7f29c67fc640 1 -- 172.21.15.39:0/2365355557 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7f29cc11fbc0 con 0x7f29d00c9330 2024-09-13T17:15:43.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.100+0000 7f29d51d9640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:43.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.100+0000 7f29d51d9640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f29d00d2d30 tx=0x7f29c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:43.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.116+0000 7f29d69dc640 1 -- 172.21.15.39:0/2365355557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b8005260 con 0x7f29d00c9330 2024-09-13T17:15:43.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:43.152+0000 7f29c67fc640 1 -- 172.21.15.39:0/2365355557 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f29cc0ebfb0 con 0x7f29d00c9330 2024-09-13T17:15:44.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.120+0000 7f29d69dc640 1 -- 172.21.15.39:0/2365355557 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29b8005a90 con 0x7f29d00c9330 2024-09-13T17:15:44.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.120+0000 7f29c67fc640 1 -- 172.21.15.39:0/2365355557 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+5938 (secure 0 0 0) 0x7f29cc0ef5d0 con 0x7f29d00c9330 2024-09-13T17:15:44.126 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:44.126 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6638,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:34.158161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:active","state_seq":11,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:44.126 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 192 2024-09-13T17:15:44.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 -- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 msgr2=0x7f29b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:44.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f29d00d2d30 tx=0x7f29c0009040 comp rx=0 tx=0).stop 2024-09-13T17:15:44.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 -- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d00c9330 msgr2=0x7f29d015d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:44.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d00c9330 0x7f29d015d190 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f29cc01ab30 tx=0x7f29cc01acc0 comp rx=0 tx=0).stop 2024-09-13T17:15:44.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 -- 172.21.15.39:0/2365355557 shutdown_connections 2024-09-13T17:15:44.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f29b4066690 0x7f29b4068b50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:44.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29d015e270 0x7f29d015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:44.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f29d0164ad0 0x7f29d00bd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:44.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 --2- 172.21.15.39:0/2365355557 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d00c9330 0x7f29d015d190 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:44.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 -- 172.21.15.39:0/2365355557 >> 172.21.15.39:0/2365355557 conn(0x7f29d00c9b10 msgr2=0x7f29d00af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:44.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 -- 172.21.15.39:0/2365355557 shutdown_connections 2024-09-13T17:15:44.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:44.128+0000 7f29b3fff640 1 -- 172.21.15.39:0/2365355557 wait complete. 2024-09-13T17:15:44.178 INFO:tasks.mds_thrash.fs.[b]:mds.c reported in up:standby state 2024-09-13T17:15:44.178 INFO:tasks.mds_thrash.fs.[b]:waiting for 5 secs before thrashing 2024-09-13T17:15:49.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb3fff640 1 Processor -- start 2024-09-13T17:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb3fff640 1 -- start start 2024-09-13T17:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb3fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbbb406d320 0x7fbbb406b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb3fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbbb40c4810 0x7fbbb408a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb3fff640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb40c4d40 con 0x7fbbb408aa10 2024-09-13T17:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb3fff640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb40c07b0 con 0x7fbbb406d320 2024-09-13T17:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb3fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb40c29b0 con 0x7fbbb40c4810 2024-09-13T17:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb27fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbbb40c4810 0x7fbbb408a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb27fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbbb40c4810 0x7fbbb408a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:34048/0 (socket says 172.21.15.39:34048) 2024-09-13T17:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.452+0000 7fbbb27fc640 1 -- 172.21.15.39:0/1869758424 learned_addr learned my addr 172.21.15.39:0/1869758424 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.456+0000 7fbbb27fc640 1 -- 172.21.15.39:0/1869758424 >> v2:172.21.15.39:3300/0 conn(0x7fbbb406d320 msgr2=0x7fbbb406b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.456+0000 7fbbb27fc640 1 --2- 172.21.15.39:0/1869758424 >> v2:172.21.15.39:3300/0 conn(0x7fbbb406d320 0x7fbbb406b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.456+0000 7fbbb27fc640 1 -- 172.21.15.39:0/1869758424 >> v1:172.21.15.39:6789/0 conn(0x7fbbb408aa10 legacy=0x7fbbb40c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.456+0000 7fbbb27fc640 1 -- 172.21.15.39:0/1869758424 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb4161650 con 0x7fbbb40c4810 2024-09-13T17:15:49.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.472+0000 7fbbb27fc640 1 --2- 172.21.15.39:0/1869758424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb40c4810 0x7fbbb408a4d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4061a30 tx=0x7fbbac0212e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46175c0836da00ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:49.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb1ffb640 1 -- 172.21.15.39:0/1869758424 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0a9e50 con 0x7fbbb40c4810 2024-09-13T17:15:49.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb1ffb640 1 -- 172.21.15.39:0/1869758424 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac0b1440 con 0x7fbbb40c4810 2024-09-13T17:15:49.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb1ffb640 1 -- 172.21.15.39:0/1869758424 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0b17b0 con 0x7fbbb40c4810 2024-09-13T17:15:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb3fff640 1 -- 172.21.15.39:0/1869758424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb40c4810 msgr2=0x7fbbb408a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb3fff640 1 --2- 172.21.15.39:0/1869758424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb40c4810 0x7fbbb408a4d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4061a30 tx=0x7fbbac0212e0 comp rx=0 tx=0).stop 2024-09-13T17:15:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb3fff640 1 -- 172.21.15.39:0/1869758424 shutdown_connections 2024-09-13T17:15:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb3fff640 1 --2- 172.21.15.39:0/1869758424 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb40c4810 0x7fbbb408a4d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb3fff640 1 --2- 172.21.15.39:0/1869758424 >> v2:172.21.15.39:3300/0 conn(0x7fbbb406d320 0x7fbbb406b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb3fff640 1 -- 172.21.15.39:0/1869758424 >> 172.21.15.39:0/1869758424 conn(0x7fbbb40c5510 msgr2=0x7fbbb40c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb3fff640 1 -- 172.21.15.39:0/1869758424 shutdown_connections 2024-09-13T17:15:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.476+0000 7fbbb3fff640 1 -- 172.21.15.39:0/1869758424 wait complete. 2024-09-13T17:15:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.480+0000 7fbbb3fff640 1 Processor -- start 2024-09-13T17:15:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.480+0000 7fbbb3fff640 1 -- start start 2024-09-13T17:15:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb406d320 0x7fbbb4203ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb408aa10 0x7fbbb4204020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb3fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbbb40c4810 0x7fbbb41fc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb3fff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41634f0 con 0x7fbbb406d320 2024-09-13T17:15:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb3fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41649f0 con 0x7fbbb408aa10 2024-09-13T17:15:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb3fff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb40ba6f0 con 0x7fbbb40c4810 2024-09-13T17:15:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb37fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbbb40c4810 0x7fbbb41fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb37fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbbb40c4810 0x7fbbb41fc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:40628/0 (socket says 172.21.15.39:40628) 2024-09-13T17:15:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb37fe640 1 -- 172.21.15.39:0/2557873794 learned_addr learned my addr 172.21.15.39:0/2557873794 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb27fc640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb408aa10 0x7fbbb4204020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb27fc640 1 -- 172.21.15.39:0/2557873794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbbb40c4810 msgr2=0x7fbbb41fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb27fc640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbbb40c4810 0x7fbbb41fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb27fc640 1 -- 172.21.15.39:0/2557873794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb406d320 msgr2=0x7fbbb4203ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:15:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb27fc640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb406d320 0x7fbbb4203ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb27fc640 1 -- 172.21.15.39:0/2557873794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac008b20 con 0x7fbbb408aa10 2024-09-13T17:15:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbbb27fc640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb408aa10 0x7fbbb4204020 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0b0000 tx=0x7fbbac01e080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbb9b7fe640 1 -- 172.21.15.39:0/2557873794 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0aaea0 con 0x7fbbb408aa10 2024-09-13T17:15:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.484+0000 7fbb9b7fe640 1 -- 172.21.15.39:0/2557873794 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac013560 con 0x7fbbb408aa10 2024-09-13T17:15:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.488+0000 7fbb9b7fe640 1 -- 172.21.15.39:0/2557873794 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac004280 con 0x7fbbb408aa10 2024-09-13T17:15:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.488+0000 7fbbb3fff640 1 -- 172.21.15.39:0/2557873794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb40bc990 con 0x7fbbb408aa10 2024-09-13T17:15:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.488+0000 7fbbb3fff640 1 -- 172.21.15.39:0/2557873794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb4164430 con 0x7fbbb408aa10 2024-09-13T17:15:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.488+0000 7fbb997fa640 1 -- 172.21.15.39:0/2557873794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c005260 con 0x7fbbb408aa10 2024-09-13T17:15:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.496+0000 7fbb9b7fe640 1 -- 172.21.15.39:0/2557873794 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbbac0043e0 con 0x7fbbb408aa10 2024-09-13T17:15:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.496+0000 7fbb9b7fe640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbba0066710 0x7fbba0068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.496+0000 7fbb9b7fe640 1 -- 172.21.15.39:0/2557873794 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7fbbac11fc30 con 0x7fbbb408aa10 2024-09-13T17:15:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.500+0000 7fbbb2ffd640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbba0066710 0x7fbba0068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:49.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.500+0000 7fbbb2ffd640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbba0066710 0x7fbba0068bd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009700 tx=0x7fbba4006c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:49.508+0000 7fbb9b7fe640 1 -- 172.21.15.39:0/2557873794 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbbac0ec110 con 0x7fbbb408aa10 2024-09-13T17:15:50.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.496+0000 7fbb997fa640 1 -- 172.21.15.39:0/2557873794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb8c0057e0 con 0x7fbbb408aa10 2024-09-13T17:15:50.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.500+0000 7fbb9b7fe640 1 -- 172.21.15.39:0/2557873794 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+5938 (secure 0 0 0) 0x7fbbac0b6070 con 0x7fbbb408aa10 2024-09-13T17:15:50.506 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:50.506 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6638,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:34.158161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:active","state_seq":11,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:50.506 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 192 2024-09-13T17:15:50.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.508+0000 7fbb997fa640 1 -- 172.21.15.39:0/2557873794 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbba0066710 msgr2=0x7fbba0068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.508+0000 7fbb997fa640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbba0066710 0x7fbba0068bd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009700 tx=0x7fbba4006c20 comp rx=0 tx=0).stop 2024-09-13T17:15:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.508+0000 7fbb997fa640 1 -- 172.21.15.39:0/2557873794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb408aa10 msgr2=0x7fbbb4204020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.508+0000 7fbb997fa640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb408aa10 0x7fbbb4204020 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0b0000 tx=0x7fbbac01e080 comp rx=0 tx=0).stop 2024-09-13T17:15:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.512+0000 7fbb997fa640 1 -- 172.21.15.39:0/2557873794 shutdown_connections 2024-09-13T17:15:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.512+0000 7fbb997fa640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbba0066710 0x7fbba0068bd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.512+0000 7fbb997fa640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbbb408aa10 0x7fbbb4204020 secure :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0b0000 tx=0x7fbbac01e080 comp rx=0 tx=0).stop 2024-09-13T17:15:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.512+0000 7fbb997fa640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbbb40c4810 0x7fbbb41fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.512+0000 7fbb997fa640 1 --2- 172.21.15.39:0/2557873794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb406d320 0x7fbbb4203ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.512+0000 7fbb997fa640 1 -- 172.21.15.39:0/2557873794 >> 172.21.15.39:0/2557873794 conn(0x7fbbb40c5510 msgr2=0x7fbbb415f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:50.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.520+0000 7fbb997fa640 1 -- 172.21.15.39:0/2557873794 shutdown_connections 2024-09-13T17:15:50.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:50.532+0000 7fbb997fa640 1 -- 172.21.15.39:0/2557873794 wait complete. 2024-09-13T17:15:50.609 INFO:tasks.mds_thrash.fs.[b]:kill mds.a (rank=0) 2024-09-13T17:15:50.609 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:15:50.609 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:15:50.626 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:15:50.626 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:51.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.048+0000 7f9e09d9c640 1 Processor -- start 2024-09-13T17:15:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.048+0000 7f9e09d9c640 1 -- start start 2024-09-13T17:15:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.052+0000 7f9e09d9c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9dfc046600 0x7f9dfc0d9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.052+0000 7f9e09d9c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc031230 con 0x7f9dfc0d3540 2024-09-13T17:15:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.052+0000 7f9e09d9c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc0da160 con 0x7f9dfc005a30 2024-09-13T17:15:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.052+0000 7f9e09d9c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dfc04b120 con 0x7f9dfc046600 2024-09-13T17:15:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.052+0000 7f9e08d9a640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9dfc0d3540 0x7f9dfc00ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42064/0 (socket says 172.21.15.39:42064) 2024-09-13T17:15:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.052+0000 7f9e08d9a640 1 -- 172.21.15.39:0/619189382 learned_addr learned my addr 172.21.15.39:0/619189382 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:15:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844303051 0 0) 0x7f9df0001120 con 0x7f9dfc0d3540 2024-09-13T17:15:51.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df4003530 con 0x7f9dfc0d3540 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2151854158 0 0) 0x7f9df4003530 con 0x7f9dfc0d3540 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 >> v2:172.21.15.84:3300/0 conn(0x7f9dfc046600 msgr2=0x7f9dfc0d9c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 --2- 172.21.15.39:0/619189382 >> v2:172.21.15.84:3300/0 conn(0x7f9dfc046600 0x7f9dfc0d9c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 >> v1:172.21.15.84:6789/0 conn(0x7f9dfc005a30 legacy=0x7f9dfc005e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc006570 con 0x7f9dfc0d3540 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f9df0001810 con 0x7f9dfc0d3540 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 >> v1:172.21.15.39:6789/0 conn(0x7f9dfc0d3540 legacy=0x7f9dfc00ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df4003a50 0x7f9df4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9df40047d0 0x7f9df4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df40053e0 0x7f9df400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:51.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df400c2c0 con 0x7f9df40053e0 2024-09-13T17:15:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df400c430 con 0x7f9df4003a50 2024-09-13T17:15:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df400c5f0 con 0x7f9df40047d0 2024-09-13T17:15:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e08d9a640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df4003a50 0x7f9df4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e03fff640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9df40047d0 0x7f9df4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e08d9a640 1 -- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9df40047d0 msgr2=0x7f9df4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e08d9a640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9df40047d0 0x7f9df4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e08d9a640 1 -- 172.21.15.39:0/619189382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df40053e0 msgr2=0x7f9df400ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:15:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e08d9a640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df40053e0 0x7f9df400ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:51.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.056+0000 7f9e08d9a640 1 -- 172.21.15.39:0/619189382 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4003530 con 0x7f9df4003a50 2024-09-13T17:15:51.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.064+0000 7f9e08d9a640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df4003a50 0x7f9df4003e30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9df0008e60 tx=0x7f9df001f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4f9b69f58aecd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:51.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.064+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0005ba0 con 0x7f9df4003a50 2024-09-13T17:15:51.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.064+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0005d00 con 0x7f9df4003a50 2024-09-13T17:15:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.064+0000 7f9e037fe640 1 -- 172.21.15.39:0/619189382 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df0008350 con 0x7f9df4003a50 2024-09-13T17:15:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.064+0000 7f9e0959b640 1 -- 172.21.15.39:0/619189382 reap_dead start 2024-09-13T17:15:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.072+0000 7f9e09d9c640 1 -- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df4003a50 msgr2=0x7f9df4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:51.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.072+0000 7f9e09d9c640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df4003a50 0x7f9df4003e30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9df0008e60 tx=0x7f9df001f8d0 comp rx=0 tx=0).stop 2024-09-13T17:15:51.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.076+0000 7f9e09d9c640 1 -- 172.21.15.39:0/619189382 shutdown_connections 2024-09-13T17:15:51.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.076+0000 7f9e09d9c640 1 --2- 172.21.15.39:0/619189382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9df4003a50 0x7f9df4003e30 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9df0008e60 tx=0x7f9df001f8d0 comp rx=0 tx=0).stop 2024-09-13T17:15:51.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.076+0000 7f9e09d9c640 1 -- 172.21.15.39:0/619189382 >> 172.21.15.39:0/619189382 conn(0x7f9dfc047cf0 msgr2=0x7f9dfc048100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:51.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.096+0000 7f9e09d9c640 1 -- 172.21.15.39:0/619189382 shutdown_connections 2024-09-13T17:15:51.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.096+0000 7f9e09d9c640 1 -- 172.21.15.39:0/619189382 wait complete. 2024-09-13T17:15:51.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.096+0000 7f9e09d9c640 1 Processor -- start 2024-09-13T17:15:51.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.096+0000 7f9e09d9c640 1 -- start start 2024-09-13T17:15:51.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e09d9c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dfc02fd30 0x7f9dfc0415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e08d9a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dfc02fd30 0x7f9dfc0415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e08d9a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dfc02fd30 0x7f9dfc0415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44872/0 (socket says 172.21.15.39:44872) 2024-09-13T17:15:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e09d9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9dfc0dc270 0x7f9dfc170830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e08d9a640 1 -- 172.21.15.39:0/4192363909 learned_addr learned my addr 172.21.15.39:0/4192363909 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e09d9c640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dfc170d70 0x7f9dfc0db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc173a70 con 0x7f9dfc02fd30 2024-09-13T17:15:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc041b30 con 0x7f9dfc170d70 2024-09-13T17:15:51.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc173790 con 0x7f9dfc0dc270 2024-09-13T17:15:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e03fff640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9dfc0dc270 0x7f9dfc170830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e08d9a640 1 -- 172.21.15.39:0/4192363909 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9dfc0dc270 msgr2=0x7f9dfc170830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e08d9a640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9dfc0dc270 0x7f9dfc170830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e08d9a640 1 -- 172.21.15.39:0/4192363909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dfc170d70 msgr2=0x7f9dfc0db1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:15:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e08d9a640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dfc170d70 0x7f9dfc0db1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:51.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e08d9a640 1 -- 172.21.15.39:0/4192363909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0da700 con 0x7f9dfc02fd30 2024-09-13T17:15:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e08d9a640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dfc02fd30 0x7f9dfc0415f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9df0020170 tx=0x7f9df4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e017fa640 1 -- 172.21.15.39:0/4192363909 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00bd070 con 0x7f9dfc02fd30 2024-09-13T17:15:51.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.100+0000 7f9e017fa640 1 -- 172.21.15.39:0/4192363909 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0001a30 con 0x7f9dfc02fd30 2024-09-13T17:15:51.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.104+0000 7f9e017fa640 1 -- 172.21.15.39:0/4192363909 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00047f0 con 0x7f9dfc02fd30 2024-09-13T17:15:51.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.108+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc0da9e0 con 0x7f9dfc02fd30 2024-09-13T17:15:51.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.112+0000 7f9e017fa640 1 -- 172.21.15.39:0/4192363909 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9df00059a0 con 0x7f9dfc02fd30 2024-09-13T17:15:51.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.112+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc02c940 con 0x7f9dfc02fd30 2024-09-13T17:15:51.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.112+0000 7f9e017fa640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9de80666c0 0x7f9de8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:51.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.112+0000 7f9e017fa640 1 -- 172.21.15.39:0/4192363909 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7f9df0125170 con 0x7f9dfc02fd30 2024-09-13T17:15:51.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.112+0000 7f9e03fff640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9de80666c0 0x7f9de8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:51.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.116+0000 7f9e03fff640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9de80666c0 0x7f9de8068b80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9e04063da0 tx=0x7f9e0408c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:51.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.116+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc0da560 con 0x7f9dfc02fd30 2024-09-13T17:15:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:51.132+0000 7f9e017fa640 1 -- 172.21.15.39:0/4192363909 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9dfc0da560 con 0x7f9dfc02fd30 2024-09-13T17:15:52.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.284+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dfc031230 con 0x7f9dfc02fd30 2024-09-13T17:15:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.288+0000 7f9e017fa640 1 -- 172.21.15.39:0/4192363909 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+5938 (secure 0 0 0) 0x7f9dfc031230 con 0x7f9dfc02fd30 2024-09-13T17:15:52.298 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:52.299 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6638,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:34.158161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:active","state_seq":11,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:52.299 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 192 2024-09-13T17:15:52.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9de80666c0 msgr2=0x7f9de8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:52.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9de80666c0 0x7f9de8068b80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9e04063da0 tx=0x7f9e0408c040 comp rx=0 tx=0).stop 2024-09-13T17:15:52.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dfc02fd30 msgr2=0x7f9dfc0415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:52.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dfc02fd30 0x7f9dfc0415f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f9df0020170 tx=0x7f9df4005800 comp rx=0 tx=0).stop 2024-09-13T17:15:52.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 shutdown_connections 2024-09-13T17:15:52.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9de80666c0 0x7f9de8068b80 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:52.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dfc170d70 0x7f9dfc0db1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:52.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9dfc0dc270 0x7f9dfc170830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:52.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 --2- 172.21.15.39:0/4192363909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9dfc02fd30 0x7f9dfc0415f0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:52.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 >> 172.21.15.39:0/4192363909 conn(0x7f9dfc047cf0 msgr2=0x7f9dfc0d1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:52.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 shutdown_connections 2024-09-13T17:15:52.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:52.304+0000 7f9e09d9c640 1 -- 172.21.15.39:0/4192363909 wait complete. 2024-09-13T17:15:52.342 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:15:54.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:54.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624ecfd640 1 Processor -- start 2024-09-13T17:15:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624ecfd640 1 -- start start 2024-09-13T17:15:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624ecfd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62480a5f60 0x7f624814df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624ecfd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f624814e450 0x7f624806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624ecfd640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62480c5810 con 0x7f624806ddc0 2024-09-13T17:15:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624ecfd640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62480c5a80 con 0x7f624814e450 2024-09-13T17:15:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624ecfd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6248069eb0 con 0x7f62480a5f60 2024-09-13T17:15:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624d29a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f624814e450 0x7f624806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624d29a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f624814e450 0x7f624806a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44878/0 (socket says 172.21.15.39:44878) 2024-09-13T17:15:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f6247fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62480a5f60 0x7f624814df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624d29a640 1 -- 172.21.15.39:0/3864187794 learned_addr learned my addr 172.21.15.39:0/3864187794 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624d29a640 1 -- 172.21.15.39:0/3864187794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62480a5f60 msgr2=0x7f624814df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624d29a640 1 --2- 172.21.15.39:0/3864187794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62480a5f60 0x7f624814df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.736+0000 7f624d29a640 1 -- 172.21.15.39:0/3864187794 >> v1:172.21.15.39:6789/0 conn(0x7f624806ddc0 legacy=0x7f6248154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:54.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.740+0000 7f624d29a640 1 -- 172.21.15.39:0/3864187794 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624806a190 con 0x7f624814e450 2024-09-13T17:15:54.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.740+0000 7f6247fff640 1 --2- 172.21.15.39:0/3864187794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62480a5f60 0x7f624814df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:54.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.748+0000 7f624d29a640 1 --2- 172.21.15.39:0/3864187794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 0x7f624806a990 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6230009870 tx=0x7f623001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30eca5413d2e143 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:54.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.748+0000 7f62477fe640 1 -- 172.21.15.39:0/3864187794 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62300ac640 con 0x7f624814e450 2024-09-13T17:15:54.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.748+0000 7f62477fe640 1 -- 172.21.15.39:0/3864187794 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62300ac7a0 con 0x7f624814e450 2024-09-13T17:15:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.752+0000 7f624ecfd640 1 -- 172.21.15.39:0/3864187794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 msgr2=0x7f624806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:54.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.752+0000 7f624ecfd640 1 --2- 172.21.15.39:0/3864187794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 0x7f624806a990 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6230009870 tx=0x7f623001c4d0 comp rx=0 tx=0).stop 2024-09-13T17:15:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.752+0000 7f624ecfd640 1 -- 172.21.15.39:0/3864187794 shutdown_connections 2024-09-13T17:15:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.752+0000 7f624ecfd640 1 --2- 172.21.15.39:0/3864187794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 0x7f624806a990 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:54.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.752+0000 7f624ecfd640 1 --2- 172.21.15.39:0/3864187794 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f62480a5f60 0x7f624814df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:54.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.752+0000 7f624ecfd640 1 -- 172.21.15.39:0/3864187794 >> 172.21.15.39:0/3864187794 conn(0x7f62480a86b0 msgr2=0x7f62480a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:54.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.760+0000 7f624ecfd640 1 -- 172.21.15.39:0/3864187794 shutdown_connections 2024-09-13T17:15:54.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.760+0000 7f624ecfd640 1 -- 172.21.15.39:0/3864187794 wait complete. 2024-09-13T17:15:54.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f624ecfd640 1 Processor -- start 2024-09-13T17:15:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f624ecfd640 1 -- start start 2024-09-13T17:15:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f624ecfd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f624806ddc0 0x7f62480c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f624ecfd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62480a5f60 0x7f62481f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f624ecfd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 0x7f62481f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f624ecfd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480c10e0 con 0x7f624814e450 2024-09-13T17:15:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f624ecfd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480c1330 con 0x7f62480a5f60 2024-09-13T17:15:54.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f624ecfd640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62480c1610 con 0x7f624806ddc0 2024-09-13T17:15:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f6247fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62480a5f60 0x7f62481f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f6247fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62480a5f60 0x7f62481f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:41888/0 (socket says 172.21.15.39:41888) 2024-09-13T17:15:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f6247fff640 1 -- 172.21.15.39:0/3952805985 learned_addr learned my addr 172.21.15.39:0/3952805985 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f6247fff640 1 -- 172.21.15.39:0/3952805985 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f624806ddc0 msgr2=0x7f62480c2660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:15:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f6247fff640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f624806ddc0 0x7f62480c2660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f6247fff640 1 -- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 msgr2=0x7f62481f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f624d29a640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 0x7f62481f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:54.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f6247fff640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 0x7f62481f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:54.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.764+0000 7f6247fff640 1 -- 172.21.15.39:0/3952805985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300094d0 con 0x7f62480a5f60 2024-09-13T17:15:54.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.768+0000 7f624d29a640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 0x7f62481f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:54.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.768+0000 7f6247fff640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62480a5f60 0x7f62481f63f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f623800c7c0 tx=0x7f623800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:54.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.768+0000 7f62457fa640 1 -- 172.21.15.39:0/3952805985 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238007820 con 0x7f62480a5f60 2024-09-13T17:15:54.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.768+0000 7f62457fa640 1 -- 172.21.15.39:0/3952805985 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6238007980 con 0x7f62480a5f60 2024-09-13T17:15:54.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.768+0000 7f62457fa640 1 -- 172.21.15.39:0/3952805985 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238007cf0 con 0x7f62480a5f60 2024-09-13T17:15:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.776+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62480c18f0 con 0x7f62480a5f60 2024-09-13T17:15:54.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.776+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62480c2ba0 con 0x7f62480a5f60 2024-09-13T17:15:54.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.784+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62480c1e10 con 0x7f62480a5f60 2024-09-13T17:15:54.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.788+0000 7f62457fa640 1 -- 172.21.15.39:0/3952805985 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f623801a450 con 0x7f62480a5f60 2024-09-13T17:15:54.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.792+0000 7f62457fa640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f622c066690 0x7f622c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:54.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.792+0000 7f62457fa640 1 -- 172.21.15.39:0/3952805985 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7f62380852b0 con 0x7f62480a5f60 2024-09-13T17:15:54.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.808+0000 7f624ca99640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f622c066690 0x7f622c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:54.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.812+0000 7f624ca99640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f622c066690 0x7f622c068b50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6240007fd0 tx=0x7f6240007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:54.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:54.816+0000 7f62457fa640 1 -- 172.21.15.39:0/3952805985 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f62380515e0 con 0x7f62480a5f60 2024-09-13T17:15:55.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.464+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62480c1e10 con 0x7f62480a5f60 2024-09-13T17:15:55.472 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:55.473 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6638,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:34.158161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:active","state_seq":11,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:55.473 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 192 2024-09-13T17:15:55.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.468+0000 7f62457fa640 1 -- 172.21.15.39:0/3952805985 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+5938 (secure 0 0 0) 0x7f62480c1e10 con 0x7f62480a5f60 2024-09-13T17:15:55.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f622c066690 msgr2=0x7f622c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:55.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f622c066690 0x7f622c068b50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6240007fd0 tx=0x7f6240007d40 comp rx=0 tx=0).stop 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62480a5f60 msgr2=0x7f62481f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62480a5f60 0x7f62481f63f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f623800c7c0 tx=0x7f623800cc80 comp rx=0 tx=0).stop 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 shutdown_connections 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f622c066690 0x7f622c068b50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62480a5f60 0x7f62481f63f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f624814e450 0x7f62481f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 --2- 172.21.15.39:0/3952805985 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f624806ddc0 0x7f62480c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 >> 172.21.15.39:0/3952805985 conn(0x7f62480a86b0 msgr2=0x7f6248065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 shutdown_connections 2024-09-13T17:15:55.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:55.472+0000 7f624ecfd640 1 -- 172.21.15.39:0/3952805985 wait complete. 2024-09-13T17:15:55.573 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:15:57.574 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:15:57.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.780+0000 7fa079270640 1 Processor -- start 2024-09-13T17:15:57.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa079270640 1 -- start start 2024-09-13T17:15:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa079270640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa0740bf4f0 0x7fa074153df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa079270640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0740611c0 0x7fa0740bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa0727fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa0740bf4f0 0x7fa074153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa0727fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa0740bf4f0 0x7fa074153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44900/0 (socket says 172.21.15.39:44900) 2024-09-13T17:15:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa072ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa074151860 0x7fa074157d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:42084/0 (socket says 172.21.15.39:42084) 2024-09-13T17:15:57.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa079270640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0740c1340 con 0x7fa074151860 2024-09-13T17:15:57.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa079270640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0741578f0 con 0x7fa0740bf4f0 2024-09-13T17:15:57.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa079270640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740c1590 con 0x7fa0740611c0 2024-09-13T17:15:57.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa0727fc640 1 -- 172.21.15.39:0/3675263571 learned_addr learned my addr 172.21.15.39:0/3675263571 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:57.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa0737fe640 1 --2- 172.21.15.39:0/3675263571 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0740611c0 0x7fa0740bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa0727fc640 1 -- 172.21.15.39:0/3675263571 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0740611c0 msgr2=0x7fa0740bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:15:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa0727fc640 1 --2- 172.21.15.39:0/3675263571 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0740611c0 0x7fa0740bfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa0727fc640 1 -- 172.21.15.39:0/3675263571 >> v1:172.21.15.39:6789/0 conn(0x7fa074151860 legacy=0x7fa074157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.784+0000 7fa0727fc640 1 -- 172.21.15.39:0/3675263571 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0740c1870 con 0x7fa0740bf4f0 2024-09-13T17:15:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.788+0000 7fa0727fc640 1 --2- 172.21.15.39:0/3675263571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740bf4f0 0x7fa074153df0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa068009870 tx=0x7fa06801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bf7914b250cd08 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.788+0000 7fa071ffb640 1 -- 172.21.15.39:0/3675263571 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0680a6040 con 0x7fa0740bf4f0 2024-09-13T17:15:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.788+0000 7fa071ffb640 1 -- 172.21.15.39:0/3675263571 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0680aa600 con 0x7fa0740bf4f0 2024-09-13T17:15:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.788+0000 7fa079270640 1 -- 172.21.15.39:0/3675263571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740bf4f0 msgr2=0x7fa074153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.788+0000 7fa071ffb640 1 -- 172.21.15.39:0/3675263571 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0680a6040 con 0x7fa0740bf4f0 2024-09-13T17:15:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.788+0000 7fa079270640 1 --2- 172.21.15.39:0/3675263571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740bf4f0 0x7fa074153df0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa068009870 tx=0x7fa06801c4a0 comp rx=0 tx=0).stop 2024-09-13T17:15:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 -- 172.21.15.39:0/3675263571 shutdown_connections 2024-09-13T17:15:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 --2- 172.21.15.39:0/3675263571 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa0740611c0 0x7fa0740bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 --2- 172.21.15.39:0/3675263571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740bf4f0 0x7fa074153df0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 -- 172.21.15.39:0/3675263571 >> 172.21.15.39:0/3675263571 conn(0x7fa074069e10 msgr2=0x7fa07406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 -- 172.21.15.39:0/3675263571 shutdown_connections 2024-09-13T17:15:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 -- 172.21.15.39:0/3675263571 wait complete. 2024-09-13T17:15:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 Processor -- start 2024-09-13T17:15:57.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 -- start start 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740611c0 0x7fa074153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0740bf4f0 0x7fa0741444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa074151860 0x7fa074141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa074154010 con 0x7fa0740611c0 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa074151fd0 con 0x7fa0740bf4f0 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa079270640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0741539c0 con 0x7fa074151860 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa074151860 0x7fa074141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa0737fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa074151860 0x7fa074141c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:40860/0 (socket says 172.21.15.39:40860) 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.792+0000 7fa0737fe640 1 -- 172.21.15.39:0/615577112 learned_addr learned my addr 172.21.15.39:0/615577112 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa0737fe640 1 -- 172.21.15.39:0/615577112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0740bf4f0 msgr2=0x7fa0741444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:57.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa0737fe640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0740bf4f0 0x7fa0741444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa0737fe640 1 -- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740611c0 msgr2=0x7fa074153050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:15:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa072ffd640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740611c0 0x7fa074153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa0737fe640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740611c0 0x7fa074153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa0737fe640 1 -- 172.21.15.39:0/615577112 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0680094d0 con 0x7fa074151860 2024-09-13T17:15:57.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa072ffd640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740611c0 0x7fa074153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:15:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa0737fe640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa074151860 0x7fa074141c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa06c007e60 tx=0x7fa06c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa05b7fe640 1 -- 172.21.15.39:0/615577112 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c014420 con 0x7fa074151860 2024-09-13T17:15:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa074153ca0 con 0x7fa074151860 2024-09-13T17:15:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa074151c60 con 0x7fa074151860 2024-09-13T17:15:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa05b7fe640 1 -- 172.21.15.39:0/615577112 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c0145a0 con 0x7fa074151860 2024-09-13T17:15:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa05b7fe640 1 -- 172.21.15.39:0/615577112 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c014950 con 0x7fa074151860 2024-09-13T17:15:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa054005260 con 0x7fa074151860 2024-09-13T17:15:57.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa05b7fe640 1 -- 172.21.15.39:0/615577112 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa06c014ab0 con 0x7fa074151860 2024-09-13T17:15:57.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa05b7fe640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa060066690 0x7fa060068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:15:57.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.796+0000 7fa05b7fe640 1 -- 172.21.15.39:0/615577112 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7fa06c0865f0 con 0x7fa074151860 2024-09-13T17:15:57.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.800+0000 7fa072ffd640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa060066690 0x7fa060068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:15:57.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.800+0000 7fa072ffd640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa060066690 0x7fa060068b50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa064007fd0 tx=0x7fa064009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:15:57.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:57.804+0000 7fa05b7fe640 1 -- 172.21.15.39:0/615577112 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa06c0529b0 con 0x7fa074151860 2024-09-13T17:15:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.352+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0540057e0 con 0x7fa074151860 2024-09-13T17:15:58.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.360+0000 7fa05b7fe640 1 -- 172.21.15.39:0/615577112 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+5938 (secure 0 0 0) 0x7fa06c055fd0 con 0x7fa074151860 2024-09-13T17:15:58.365 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:15:58.365 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6638,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:34.158161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:active","state_seq":11,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:15:58.366 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 192 2024-09-13T17:15:58.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.372+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa060066690 msgr2=0x7fa060068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:58.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.372+0000 7fa079270640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa060066690 0x7fa060068b50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa064007fd0 tx=0x7fa064009040 comp rx=0 tx=0).stop 2024-09-13T17:15:58.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.372+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa074151860 msgr2=0x7fa074141c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:15:58.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.372+0000 7fa079270640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa074151860 0x7fa074141c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa06c007e60 tx=0x7fa06c00e4b0 comp rx=0 tx=0).stop 2024-09-13T17:15:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.376+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 shutdown_connections 2024-09-13T17:15:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.376+0000 7fa079270640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa060066690 0x7fa060068b50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.376+0000 7fa079270640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0740bf4f0 0x7fa0741444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.376+0000 7fa079270640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa074151860 0x7fa074141c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.376+0000 7fa079270640 1 --2- 172.21.15.39:0/615577112 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0740611c0 0x7fa074153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:15:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.376+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 >> 172.21.15.39:0/615577112 conn(0x7fa074069e10 msgr2=0x7fa07414e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:15:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.376+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 shutdown_connections 2024-09-13T17:15:58.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:15:58.376+0000 7fa079270640 1 -- 172.21.15.39:0/615577112 wait complete. 2024-09-13T17:15:58.416 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:16:00.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:00.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f2775794640 1 Processor -- start 2024-09-13T17:16:00.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f2775794640 1 -- start start 2024-09-13T17:16:00.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f2775794640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2770155a40 0x7f27700c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:00.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f2775794640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f277015b810 0x7f27700c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:00.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f2775794640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27700adb00 0x7f27700c6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:00.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f2775794640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700c6110 con 0x7f2770155a40 2024-09-13T17:16:00.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f2775794640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700c4b30 con 0x7f277015b810 2024-09-13T17:16:00.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f2775794640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27700c4e10 con 0x7f27700adb00 2024-09-13T17:16:00.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f276f7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27700adb00 0x7f27700c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:00.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f276f7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27700adb00 0x7f27700c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:44634/0 (socket says 172.21.15.39:44634) 2024-09-13T17:16:00.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f276f7fe640 1 -- 172.21.15.39:0/1665192059 learned_addr learned my addr 172.21.15.39:0/1665192059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:00.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f276f7fe640 1 -- 172.21.15.39:0/1665192059 >> v2:172.21.15.84:3300/0 conn(0x7f277015b810 msgr2=0x7f27700c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:00.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f276f7fe640 1 --2- 172.21.15.39:0/1665192059 >> v2:172.21.15.84:3300/0 conn(0x7f277015b810 0x7f27700c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:00.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f276f7fe640 1 -- 172.21.15.39:0/1665192059 >> v2:172.21.15.39:3300/0 conn(0x7f2770155a40 msgr2=0x7f27700c5bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:00.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f276f7fe640 1 --2- 172.21.15.39:0/1665192059 >> v2:172.21.15.39:3300/0 conn(0x7f2770155a40 0x7f27700c5bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.639+0000 7f276f7fe640 1 -- 172.21.15.39:0/1665192059 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700c50f0 con 0x7f27700adb00 2024-09-13T17:16:00.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.651+0000 7f276f7fe640 1 --2- 172.21.15.39:0/1665192059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27700adb00 0x7f27700c6370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27700ad880 tx=0x7f276001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f611ab562bcc990c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:00.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.651+0000 7f276dffb640 1 -- 172.21.15.39:0/1665192059 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600ab5f0 con 0x7f27700adb00 2024-09-13T17:16:00.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.651+0000 7f276dffb640 1 -- 172.21.15.39:0/1665192059 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27600ab750 con 0x7f27700adb00 2024-09-13T17:16:00.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.651+0000 7f276dffb640 1 -- 172.21.15.39:0/1665192059 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600abae0 con 0x7f27700adb00 2024-09-13T17:16:00.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.651+0000 7f2775794640 1 -- 172.21.15.39:0/1665192059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27700adb00 msgr2=0x7f27700c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:00.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.651+0000 7f2775794640 1 --2- 172.21.15.39:0/1665192059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27700adb00 0x7f27700c6370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27700ad880 tx=0x7f276001c540 comp rx=0 tx=0).stop 2024-09-13T17:16:00.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 -- 172.21.15.39:0/1665192059 shutdown_connections 2024-09-13T17:16:00.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 --2- 172.21.15.39:0/1665192059 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f27700adb00 0x7f27700c6370 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:00.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 --2- 172.21.15.39:0/1665192059 >> v2:172.21.15.84:3300/0 conn(0x7f277015b810 0x7f27700c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:00.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 --2- 172.21.15.39:0/1665192059 >> v2:172.21.15.39:3300/0 conn(0x7f2770155a40 0x7f27700c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:00.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 -- 172.21.15.39:0/1665192059 >> 172.21.15.39:0/1665192059 conn(0x7f2770069e10 msgr2=0x7f277006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:00.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 -- 172.21.15.39:0/1665192059 shutdown_connections 2024-09-13T17:16:00.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 -- 172.21.15.39:0/1665192059 wait complete. 2024-09-13T17:16:00.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 Processor -- start 2024-09-13T17:16:00.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 -- start start 2024-09-13T17:16:00.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27700adb00 0x7f27700c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:00.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2770155a40 0x7f27700c1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:00.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f277015b810 0x7f27701f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:00.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2770158a40 con 0x7f27700adb00 2024-09-13T17:16:00.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27701f5a70 con 0x7f277015b810 2024-09-13T17:16:00.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f2775794640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277015af20 con 0x7f2770155a40 2024-09-13T17:16:00.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.655+0000 7f276f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f277015b810 0x7f27701f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:00.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f277015b810 0x7f27701f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:49004/0 (socket says 172.21.15.39:49004) 2024-09-13T17:16:00.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276f7fe640 1 -- 172.21.15.39:0/3605262701 learned_addr learned my addr 172.21.15.39:0/3605262701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:00.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276f7fe640 1 -- 172.21.15.39:0/3605262701 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2770155a40 msgr2=0x7f27700c1440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:16:00.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276f7fe640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2770155a40 0x7f27700c1440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:00.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276f7fe640 1 -- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27700adb00 msgr2=0x7f27700c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:00.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276effd640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27700adb00 0x7f27700c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:00.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276f7fe640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27700adb00 0x7f27700c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:00.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276f7fe640 1 -- 172.21.15.39:0/3605262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600094d0 con 0x7f277015b810 2024-09-13T17:16:00.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276effd640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27700adb00 0x7f27700c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:16:00.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.659+0000 7f276f7fe640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f277015b810 0x7f27701f4db0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f27600097c0 tx=0x7f276001ad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.663+0000 7f27577fe640 1 -- 172.21.15.39:0/3605262701 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600b4600 con 0x7f277015b810 2024-09-13T17:16:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.663+0000 7f27577fe640 1 -- 172.21.15.39:0/3605262701 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27600b4760 con 0x7f277015b810 2024-09-13T17:16:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.663+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277015ac40 con 0x7f277015b810 2024-09-13T17:16:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.663+0000 7f27577fe640 1 -- 172.21.15.39:0/3605262701 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27600b4af0 con 0x7f277015b810 2024-09-13T17:16:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.663+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27701f5700 con 0x7f277015b810 2024-09-13T17:16:00.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.663+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27700c5bd0 con 0x7f277015b810 2024-09-13T17:16:00.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.663+0000 7f27577fe640 1 -- 172.21.15.39:0/3605262701 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27600b4c50 con 0x7f277015b810 2024-09-13T17:16:00.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.663+0000 7f27577fe640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f275c066710 0x7f275c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:00.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.663+0000 7f27577fe640 1 -- 172.21.15.39:0/3605262701 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7f27600a8050 con 0x7f277015b810 2024-09-13T17:16:00.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.667+0000 7f276effd640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f275c066710 0x7f275c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:00.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.667+0000 7f276effd640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f275c066710 0x7f275c068bd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f277006a520 tx=0x7f27680073c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:00.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:00.679+0000 7f27577fe640 1 -- 172.21.15.39:0/3605262701 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27600ec4e0 con 0x7f277015b810 2024-09-13T17:16:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.163+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27700c3340 con 0x7f277015b810 2024-09-13T17:16:01.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.163+0000 7f27577fe640 1 -- 172.21.15.39:0/3605262701 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+5938 (secure 0 0 0) 0x7f27600efb00 con 0x7f277015b810 2024-09-13T17:16:01.170 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:01.170 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6638,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:34.158161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:active","state_seq":11,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:01.170 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 192 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f275c066710 msgr2=0x7f275c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f275c066710 0x7f275c068bd0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f277006a520 tx=0x7f27680073c0 comp rx=0 tx=0).stop 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f277015b810 msgr2=0x7f27701f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f277015b810 0x7f27701f4db0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f27600097c0 tx=0x7f276001ad80 comp rx=0 tx=0).stop 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 shutdown_connections 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f275c066710 0x7f275c068bd0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f277015b810 0x7f27701f4db0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2770155a40 0x7f27700c1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 --2- 172.21.15.39:0/3605262701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27700adb00 0x7f27700c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 >> 172.21.15.39:0/3605262701 conn(0x7f2770069e10 msgr2=0x7f27700aa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:01.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 shutdown_connections 2024-09-13T17:16:01.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:01.167+0000 7f2775794640 1 -- 172.21.15.39:0/3605262701 wait complete. 2024-09-13T17:16:01.201 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:16:03.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:03.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.399+0000 7f590db28640 1 Processor -- start 2024-09-13T17:16:03.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.399+0000 7f590db28640 1 -- start start 2024-09-13T17:16:03.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f590db28640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f59000d0980 0x7f5900046d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:03.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f5907fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f590004a2d0 0x7f5900008000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34058/0 (socket says 172.21.15.39:34058) 2024-09-13T17:16:03.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f5907fff640 1 -- 172.21.15.39:0/2987380494 learned_addr learned my addr 172.21.15.39:0/2987380494 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:16:03.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f590cb26640 1 --2- 172.21.15.39:0/2987380494 >> v2:172.21.15.84:3300/0 conn(0x7f59000d0980 0x7f5900046d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:03.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f590db28640 1 -- 172.21.15.39:0/2987380494 --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000d6c70 con 0x7f590004a2d0 2024-09-13T17:16:03.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f590db28640 1 -- 172.21.15.39:0/2987380494 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590003fc00 con 0x7f59000484c0 2024-09-13T17:16:03.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f590db28640 1 -- 172.21.15.39:0/2987380494 --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5900007800 con 0x7f59000d0980 2024-09-13T17:16:03.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f59077fe640 1 -- 172.21.15.39:0/2987380494 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780098560 0 0) 0x7f590003fc00 con 0x7f59000484c0 2024-09-13T17:16:03.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f59077fe640 1 -- 172.21.15.39:0/2987380494 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58f8003530 con 0x7f59000484c0 2024-09-13T17:16:03.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f59077fe640 1 -- 172.21.15.39:0/2987380494 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598349681 0 0) 0x7f59000d6c70 con 0x7f590004a2d0 2024-09-13T17:16:03.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f59077fe640 1 -- 172.21.15.39:0/2987380494 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f590003fc00 con 0x7f590004a2d0 2024-09-13T17:16:03.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f590cb26640 1 -- 172.21.15.39:0/2987380494 >> v1:172.21.15.84:6789/0 conn(0x7f59000484c0 legacy=0x7f5900008710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:03.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f590cb26640 1 -- 172.21.15.39:0/2987380494 >> v1:172.21.15.39:6789/0 conn(0x7f590004a2d0 legacy=0x7f5900008000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:03.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.403+0000 7f590cb26640 1 -- 172.21.15.39:0/2987380494 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900007ae0 con 0x7f59000d0980 2024-09-13T17:16:03.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.407+0000 7f590cb26640 1 --2- 172.21.15.39:0/2987380494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 0x7f5900046d50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0097c0 tx=0x7f58fc01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701c4979e1c86682 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.411+0000 7f59077fe640 1 -- 172.21.15.39:0/2987380494 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0ab600 con 0x7f59000d0980 2024-09-13T17:16:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.411+0000 7f59077fe640 1 -- 172.21.15.39:0/2987380494 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc0a7050 con 0x7f59000d0980 2024-09-13T17:16:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.411+0000 7f59077fe640 1 -- 172.21.15.39:0/2987380494 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0ab980 con 0x7f59000d0980 2024-09-13T17:16:03.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.411+0000 7f590db28640 1 -- 172.21.15.39:0/2987380494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 msgr2=0x7f5900046d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:03.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.411+0000 7f590db28640 1 --2- 172.21.15.39:0/2987380494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 0x7f5900046d50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0097c0 tx=0x7f58fc01c380 comp rx=0 tx=0).stop 2024-09-13T17:16:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.411+0000 7f590db28640 1 -- 172.21.15.39:0/2987380494 shutdown_connections 2024-09-13T17:16:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.411+0000 7f590db28640 1 --2- 172.21.15.39:0/2987380494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 0x7f5900046d50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.411+0000 7f590db28640 1 -- 172.21.15.39:0/2987380494 >> 172.21.15.39:0/2987380494 conn(0x7f5900048e50 msgr2=0x7f5900049260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:03.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.415+0000 7f590db28640 1 -- 172.21.15.39:0/2987380494 shutdown_connections 2024-09-13T17:16:03.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.415+0000 7f590db28640 1 -- 172.21.15.39:0/2987380494 wait complete. 2024-09-13T17:16:03.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.415+0000 7f590db28640 1 Processor -- start 2024-09-13T17:16:03.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.415+0000 7f590db28640 1 -- start start 2024-09-13T17:16:03.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590db28640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000484c0 0x7f59000d1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:03.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590db28640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f590004a2d0 0x7f5900171810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:03.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590cb26640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000484c0 0x7f59000d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:03.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590cb26640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000484c0 0x7f59000d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:44654/0 (socket says 172.21.15.39:44654) 2024-09-13T17:16:03.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590cb26640 1 -- 172.21.15.39:0/1923872213 learned_addr learned my addr 172.21.15.39:0/1923872213 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:03.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590db28640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 0x7f5900041080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:03.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f5907fff640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f590004a2d0 0x7f5900171810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:03.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900040910 con 0x7f590004a2d0 2024-09-13T17:16:03.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900040bf0 con 0x7f59000d0980 2024-09-13T17:16:03.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590d327640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 0x7f5900041080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:03.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.419+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59000d3940 con 0x7f59000484c0 2024-09-13T17:16:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f590cb26640 1 -- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 msgr2=0x7f5900041080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f590cb26640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 0x7f5900041080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f590cb26640 1 -- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f590004a2d0 msgr2=0x7f5900171810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f590cb26640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f590004a2d0 0x7f5900171810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f590cb26640 1 -- 172.21.15.39:0/1923872213 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc009420 con 0x7f59000484c0 2024-09-13T17:16:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f590d327640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 0x7f5900041080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:16:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f5907fff640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f590004a2d0 0x7f5900171810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:16:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f590cb26640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000484c0 0x7f59000d1000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58fc019fb0 tx=0x7f58fc019120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f59057fa640 1 -- 172.21.15.39:0/1923872213 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0abea0 con 0x7f59000484c0 2024-09-13T17:16:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f59057fa640 1 -- 172.21.15.39:0/1923872213 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc0b4620 con 0x7f59000484c0 2024-09-13T17:16:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f59057fa640 1 -- 172.21.15.39:0/1923872213 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0b49d0 con 0x7f59000484c0 2024-09-13T17:16:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59000ce3e0 con 0x7f59000484c0 2024-09-13T17:16:03.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.423+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5900172060 con 0x7f59000484c0 2024-09-13T17:16:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.427+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5900005f40 con 0x7f59000484c0 2024-09-13T17:16:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.427+0000 7f59057fa640 1 -- 172.21.15.39:0/1923872213 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58fc0b2040 con 0x7f59000484c0 2024-09-13T17:16:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.427+0000 7f59057fa640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58ec0666e0 0x7f58ec068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:03.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.427+0000 7f59057fa640 1 -- 172.21.15.39:0/1923872213 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 9819+0+0 (secure 0 0 0) 0x7f58fc11f260 con 0x7f59000484c0 2024-09-13T17:16:03.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.427+0000 7f5907fff640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58ec0666e0 0x7f58ec068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:03.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.431+0000 7f5907fff640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58ec0666e0 0x7f58ec068ba0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f59000ce9f0 tx=0x7f58f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:03.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.435+0000 7f59057fa640 1 -- 172.21.15.39:0/1923872213 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f58fc0eb5a0 con 0x7f59000484c0 2024-09-13T17:16:03.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.895+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f590004ad90 con 0x7f59000484c0 2024-09-13T17:16:03.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.899+0000 7f59057fa640 1 -- 172.21.15.39:0/1923872213 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+5938 (secure 0 0 0) 0x7f58fc0b9090 con 0x7f59000484c0 2024-09-13T17:16:03.903 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:03.903 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6638,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":192}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:15:34.158161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8220},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8220":{"gid":8220,"name":"a","rank":0,"incarnation":188,"state":"up:active","state_seq":11,"addr":"172.21.15.39:6835/3721635324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3721635324},{"type":"v1","addr":"172.21.15.39:6835","nonce":3721635324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:03.903 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 192 2024-09-13T17:16:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58ec0666e0 msgr2=0x7f58ec068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58ec0666e0 0x7f58ec068ba0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f59000ce9f0 tx=0x7f58f4009040 comp rx=0 tx=0).stop 2024-09-13T17:16:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000484c0 msgr2=0x7f59000d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000484c0 0x7f59000d1000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58fc019fb0 tx=0x7f58fc019120 comp rx=0 tx=0).stop 2024-09-13T17:16:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 shutdown_connections 2024-09-13T17:16:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f58ec0666e0 0x7f58ec068ba0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59000d0980 0x7f5900041080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f590004a2d0 0x7f5900171810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 --2- 172.21.15.39:0/1923872213 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f59000484c0 0x7f59000d1000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 >> 172.21.15.39:0/1923872213 conn(0x7f5900048e50 msgr2=0x7f5900004e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 shutdown_connections 2024-09-13T17:16:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:03.907+0000 7f590db28640 1 -- 172.21.15.39:0/1923872213 wait complete. 2024-09-13T17:16:03.969 INFO:tasks.mds_thrash.fs.[b]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:16:05.475 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:16:05.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:16:05.970 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd8f5f640 1 Processor -- start 2024-09-13T17:16:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd8f5f640 1 -- start start 2024-09-13T17:16:06.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd8f5f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcdcc04a2d0 0x7fcdcc0d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:06.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd8f5f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcdcc0484c0 0x7fcdcc046d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:06.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd8f5f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdcc0dd2a0 con 0x7fcdcc047290 2024-09-13T17:16:06.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd8f5f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdcc0d0ec0 con 0x7fcdcc04a2d0 2024-09-13T17:16:06.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd8f5f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdcc0d6c70 con 0x7fcdcc0484c0 2024-09-13T17:16:06.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd3fff640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcdcc047290 0x7fcdcc008000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:39922/0 (socket says 172.21.15.39:39922) 2024-09-13T17:16:06.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd3fff640 1 -- 172.21.15.39:0/2643700538 learned_addr learned my addr 172.21.15.39:0/2643700538 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:16:06.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449189491 0 0) 0x7fcdcc0dd2a0 con 0x7fcdcc047290 2024-09-13T17:16:06.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc4003530 con 0x7fcdcc047290 2024-09-13T17:16:06.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 147764839 0 0) 0x7fcdc4003530 con 0x7fcdcc047290 2024-09-13T17:16:06.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 >> v2:172.21.15.84:3300/0 conn(0x7fcdcc0484c0 msgr2=0x7fcdcc046d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:06.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd27fc640 1 --2- 172.21.15.39:0/2643700538 >> v2:172.21.15.84:3300/0 conn(0x7fcdcc0484c0 0x7fcdcc046d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 >> v2:172.21.15.39:3300/0 conn(0x7fcdcc04a2d0 msgr2=0x7fcdcc0d0980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:06.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd27fc640 1 --2- 172.21.15.39:0/2643700538 >> v2:172.21.15.39:3300/0 conn(0x7fcdcc04a2d0 0x7fcdcc0d0980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.259+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc03fc00 con 0x7fcdcc047290 2024-09-13T17:16:06.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fcdd404e740 con 0x7fcdcc047290 2024-09-13T17:16:06.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 >> v1:172.21.15.84:6789/0 conn(0x7fcdcc047290 legacy=0x7fcdcc008000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:06.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd27fc640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcdc4003980 0x7fcdc4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:06.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd27fc640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdc40047d0 0x7fcdc4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:06.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd27fc640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdc401c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:06.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0d0ec0 con 0x7fcdc40047d0 2024-09-13T17:16:06.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0d6c70 con 0x7fcdc4005190 2024-09-13T17:16:06.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc401d120 con 0x7fcdc4003980 2024-09-13T17:16:06.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd3fff640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdc401c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:06.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd2ffd640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdc40047d0 0x7fcdc4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:06.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd3fff640 1 -- 172.21.15.39:0/2643700538 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcdc4003980 msgr2=0x7fcdc4003d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:06.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd3fff640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcdc4003980 0x7fcdc4003d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd3fff640 1 -- 172.21.15.39:0/2643700538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdc40047d0 msgr2=0x7fcdc4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:06.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd3fff640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdc40047d0 0x7fcdc4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd3fff640 1 -- 172.21.15.39:0/2643700538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc4003530 con 0x7fcdc4005190 2024-09-13T17:16:06.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.263+0000 7fcdd3fff640 1 -- 172.21.15.39:0/2643700538 reap_dead start 2024-09-13T17:16:06.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.267+0000 7fcdd2ffd640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdc40047d0 0x7fcdc4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:16:06.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.267+0000 7fcdd3fff640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdc401c9a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcdd404f070 tx=0x7fcdd40977b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d773f8c59fd66714 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:06.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.267+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4127600 con 0x7fcdc4005190 2024-09-13T17:16:06.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.267+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd41277a0 con 0x7fcdc4005190 2024-09-13T17:16:06.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.267+0000 7fcdd27fc640 1 -- 172.21.15.39:0/2643700538 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4127b50 con 0x7fcdc4005190 2024-09-13T17:16:06.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.267+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/2643700538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 msgr2=0x7fcdc401c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:06.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.267+0000 7fcdd8f5f640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdc401c9a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcdd404f070 tx=0x7fcdd40977b0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.283+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/2643700538 shutdown_connections 2024-09-13T17:16:06.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.283+0000 7fcdd8f5f640 1 --2- 172.21.15.39:0/2643700538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdc401c9a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.283+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/2643700538 >> 172.21.15.39:0/2643700538 conn(0x7fcdcc048e50 msgr2=0x7fcdcc049260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:06.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.283+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/2643700538 shutdown_connections 2024-09-13T17:16:06.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.283+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/2643700538 wait complete. 2024-09-13T17:16:06.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd8f5f640 1 Processor -- start 2024-09-13T17:16:06.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd8f5f640 1 -- start start 2024-09-13T17:16:06.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd8f5f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdcc02d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:06.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd8f5f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcdcc0cd060 0x7fcdcc0c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:06.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd8f5f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdcc0cd4a0 0x7fcdcc171350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:06.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd8f5f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0e0fd0 con 0x7fcdcc0cd4a0 2024-09-13T17:16:06.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd8f5f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc02d000 con 0x7fcdc4005190 2024-09-13T17:16:06.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd8f5f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdcc0404a0 con 0x7fcdcc0cd060 2024-09-13T17:16:06.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdcc02d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:06.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdcc02d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:49026/0 (socket says 172.21.15.39:49026) 2024-09-13T17:16:06.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd37fe640 1 -- 172.21.15.39:0/577760617 learned_addr learned my addr 172.21.15.39:0/577760617 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:06.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd3fff640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdcc0cd4a0 0x7fcdcc171350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:06.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd2ffd640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcdcc0cd060 0x7fcdcc0c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:06.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd2ffd640 1 -- 172.21.15.39:0/577760617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 msgr2=0x7fcdcc02d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:06.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd2ffd640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdcc02d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd2ffd640 1 -- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdcc0cd4a0 msgr2=0x7fcdcc171350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:06.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd2ffd640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdcc0cd4a0 0x7fcdcc171350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd2ffd640 1 -- 172.21.15.39:0/577760617 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc0e11e0 con 0x7fcdcc0cd060 2024-09-13T17:16:06.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd37fe640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdcc02d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:16:06.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.287+0000 7fcdd3fff640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdcc0cd4a0 0x7fcdcc171350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:16:06.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.291+0000 7fcdd2ffd640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcdcc0cd060 0x7fcdcc0c55e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0042d0 tx=0x7fcdc000c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:06.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.291+0000 7fcdbffff640 1 -- 172.21.15.39:0/577760617 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0011800 con 0x7fcdcc0cd060 2024-09-13T17:16:06.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.291+0000 7fcdbffff640 1 -- 172.21.15.39:0/577760617 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc0011960 con 0x7fcdcc0cd060 2024-09-13T17:16:06.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.291+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdcc02dab0 con 0x7fcdcc0cd060 2024-09-13T17:16:06.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.291+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdcc0c7230 con 0x7fcdcc0cd060 2024-09-13T17:16:06.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.291+0000 7fcdbffff640 1 -- 172.21.15.39:0/577760617 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0011d10 con 0x7fcdcc0cd060 2024-09-13T17:16:06.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.291+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb0005260 con 0x7fcdcc0cd060 2024-09-13T17:16:06.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.295+0000 7fcdbffff640 1 -- 172.21.15.39:0/577760617 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcdc001a450 con 0x7fcdcc0cd060 2024-09-13T17:16:06.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.295+0000 7fcdbffff640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcdb8066690 0x7fcdb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:06.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.295+0000 7fcdbffff640 1 -- 172.21.15.39:0/577760617 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7fcdc00850e0 con 0x7fcdcc0cd060 2024-09-13T17:16:06.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.295+0000 7fcdd37fe640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcdb8066690 0x7fcdb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:06.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.295+0000 7fcdd37fe640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcdb8066690 0x7fcdb8068b50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009da0 tx=0x7fcdc8009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:06.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.299+0000 7fcdbffff640 1 -- 172.21.15.39:0/577760617 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcdc00514f0 con 0x7fcdcc0cd060 2024-09-13T17:16:06.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.951+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdb00057e0 con 0x7fcdcc0cd060 2024-09-13T17:16:06.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.955+0000 7fcdbffff640 1 -- 172.21.15.39:0/577760617 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+5182 (secure 0 0 0) 0x7fcdc001f090 con 0x7fcdcc0cd060 2024-09-13T17:16:06.960 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:06.960 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":194,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":194,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:06.949945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:06.961 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 194 2024-09-13T17:16:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcdb8066690 msgr2=0x7fcdb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcdb8066690 0x7fcdb8068b50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009da0 tx=0x7fcdc8009110 comp rx=0 tx=0).stop 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcdcc0cd060 msgr2=0x7fcdcc0c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcdcc0cd060 0x7fcdcc0c55e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0042d0 tx=0x7fcdc000c7d0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 shutdown_connections 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcdb8066690 0x7fcdb8068b50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdcc0cd4a0 0x7fcdcc171350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcdcc0cd060 0x7fcdcc0c55e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 --2- 172.21.15.39:0/577760617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdc4005190 0x7fcdcc02d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 >> 172.21.15.39:0/577760617 conn(0x7fcdcc048e50 msgr2=0x7fcdcc006e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 shutdown_connections 2024-09-13T17:16:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:06.959+0000 7fcdd8f5f640 1 -- 172.21.15.39:0/577760617 wait complete. 2024-09-13T17:16:07.017 INFO:tasks.mds_thrash.fs.[b]:mds.a down, removed from mdsmap 2024-09-13T17:16:07.018 INFO:tasks.mds_thrash.fs.[b]:waiting for mds cluster to stabilize... 2024-09-13T17:16:07.018 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:07.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f4778b70640 1 Processor -- start 2024-09-13T17:16:07.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f4778b70640 1 -- start start 2024-09-13T17:16:07.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f4778b70640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c042cc0 0x7f476c006d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:07.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f4778b70640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c0da470 con 0x7f476c0d7420 2024-09-13T17:16:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f4778b70640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c00f390 con 0x7f476c007290 2024-09-13T17:16:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f4778b70640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c00f950 con 0x7f476c042cc0 2024-09-13T17:16:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f47737fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f476c0d7420 0x7f476c00dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:34060/0 (socket says 172.21.15.39:34060) 2024-09-13T17:16:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f47737fe640 1 -- 172.21.15.39:0/2620839443 learned_addr learned my addr 172.21.15.39:0/2620839443 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:16:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362791675 0 0) 0x7f476c00f390 con 0x7f476c007290 2024-09-13T17:16:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.263+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4764003530 con 0x7f476c007290 2024-09-13T17:16:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f4772ffd640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c042cc0 0x7f476c006d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:07.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 <== mon.1 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2118355790 0 0) 0x7f4764003530 con 0x7f476c007290 2024-09-13T17:16:07.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c042cc0 msgr2=0x7f476c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:07.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c042cc0 0x7f476c006d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:07.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 >> v1:172.21.15.39:6789/0 conn(0x7f476c0d7420 legacy=0x7f476c00dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:07.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c00f670 con 0x7f476c007290 2024-09-13T17:16:07.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 <== mon.1 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4768001680 con 0x7f476c007290 2024-09-13T17:16:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 >> v1:172.21.15.84:6789/0 conn(0x7f476c007290 legacy=0x7f476c0e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4764003980 0x7f4764003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47640047d0 0x7f4764004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c042cc0 0x7f476400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c00f950 con 0x7f4764003980 2024-09-13T17:16:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4764005550 con 0x7f47640047d0 2024-09-13T17:16:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476400bed0 con 0x7f476c042cc0 2024-09-13T17:16:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f4772ffd640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47640047d0 0x7f4764004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f4772ffd640 1 -- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c042cc0 msgr2=0x7f476400b990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:07.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f4772ffd640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c042cc0 0x7f476400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:07.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f4772ffd640 1 -- 172.21.15.39:0/2620839443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4764003980 msgr2=0x7f4764003da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:07.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47737fe640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4764003980 0x7f4764003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:07.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f4772ffd640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4764003980 0x7f4764003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:07.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f4772ffd640 1 -- 172.21.15.39:0/2620839443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c00f670 con 0x7f47640047d0 2024-09-13T17:16:07.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.267+0000 7f47737fe640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4764003980 0x7f4764003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:16:07.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f4772ffd640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47640047d0 0x7f4764004c50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f476000ba30 tx=0x7f476001c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f876f20cb0426fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:07.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600ae650 con 0x7f47640047d0 2024-09-13T17:16:07.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47600ae7b0 con 0x7f47640047d0 2024-09-13T17:16:07.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f47727fc640 1 -- 172.21.15.39:0/2620839443 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600aeb60 con 0x7f47640047d0 2024-09-13T17:16:07.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f4778b70640 1 -- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47640047d0 msgr2=0x7f4764004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:07.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f4778b70640 1 --2- 172.21.15.39:0/2620839443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47640047d0 0x7f4764004c50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f476000ba30 tx=0x7f476001c660 comp rx=0 tx=0).stop 2024-09-13T17:16:07.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f4773fff640 1 -- 172.21.15.39:0/2620839443 reap_dead start 2024-09-13T17:16:07.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f4778b70640 1 -- 172.21.15.39:0/2620839443 shutdown_connections 2024-09-13T17:16:07.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f4778b70640 1 -- 172.21.15.39:0/2620839443 >> 172.21.15.39:0/2620839443 conn(0x7f476c04c0b0 msgr2=0x7f476c04c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:07.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f4778b70640 1 -- 172.21.15.39:0/2620839443 shutdown_connections 2024-09-13T17:16:07.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f4778b70640 1 -- 172.21.15.39:0/2620839443 wait complete. 2024-09-13T17:16:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.271+0000 7f4778b70640 1 Processor -- start 2024-09-13T17:16:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4778b70640 1 -- start start 2024-09-13T17:16:07.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4778b70640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c0d8e90 0x7f476c0d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:07.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4778b70640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f476c0cacb0 0x7f476c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:07.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4778b70640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f476c0cb670 0x7f476c0c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:07.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4778b70640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0d3360 con 0x7f476c0cb670 2024-09-13T17:16:07.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4778b70640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0d7b20 con 0x7f476c0cacb0 2024-09-13T17:16:07.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4778b70640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0d7e00 con 0x7f476c0d8e90 2024-09-13T17:16:07.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4772ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f476c0cacb0 0x7f476c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:07.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4772ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f476c0cacb0 0x7f476c0cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:49044/0 (socket says 172.21.15.39:49044) 2024-09-13T17:16:07.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4772ffd640 1 -- 172.21.15.39:0/3039519554 learned_addr learned my addr 172.21.15.39:0/3039519554 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4773fff640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f476c0cb670 0x7f476c0c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f47737fe640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c0d8e90 0x7f476c0d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4772ffd640 1 -- 172.21.15.39:0/3039519554 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c0d8e90 msgr2=0x7f476c0d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4772ffd640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c0d8e90 0x7f476c0d2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.275+0000 7f4772ffd640 1 -- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f476c0cb670 msgr2=0x7f476c0c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f4772ffd640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f476c0cb670 0x7f476c0c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f4772ffd640 1 -- 172.21.15.39:0/3039519554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0d80e0 con 0x7f476c0cacb0 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f47737fe640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c0d8e90 0x7f476c0d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f4773fff640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f476c0cb670 0x7f476c0c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f4772ffd640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f476c0cacb0 0x7f476c0cb130 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f476000ba30 tx=0x7f4764005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f475ffff640 1 -- 172.21.15.39:0/3039519554 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760004020 con 0x7f476c0cacb0 2024-09-13T17:16:07.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f475ffff640 1 -- 172.21.15.39:0/3039519554 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760004180 con 0x7f476c0cacb0 2024-09-13T17:16:07.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f475ffff640 1 -- 172.21.15.39:0/3039519554 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600044f0 con 0x7f476c0cacb0 2024-09-13T17:16:07.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f4778b70640 1 -- 172.21.15.39:0/3039519554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c0d3570 con 0x7f476c0cacb0 2024-09-13T17:16:07.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f4778b70640 1 -- 172.21.15.39:0/3039519554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c0d53a0 con 0x7f476c0cacb0 2024-09-13T17:16:07.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.279+0000 7f4778b70640 1 -- 172.21.15.39:0/3039519554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c00dd00 con 0x7f476c0cacb0 2024-09-13T17:16:07.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.283+0000 7f475ffff640 1 -- 172.21.15.39:0/3039519554 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47600b5040 con 0x7f476c0cacb0 2024-09-13T17:16:07.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.283+0000 7f475ffff640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4758066790 0x7f4758068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:07.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.283+0000 7f475ffff640 1 -- 172.21.15.39:0/3039519554 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7f47600aa050 con 0x7f476c0cacb0 2024-09-13T17:16:07.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.283+0000 7f47737fe640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4758066790 0x7f4758068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:07.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.283+0000 7f47737fe640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4758066790 0x7f4758068c50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f477404fbc0 tx=0x7f477406c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:07.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.287+0000 7f475ffff640 1 -- 172.21.15.39:0/3039519554 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f47600eb730 con 0x7f476c0cacb0 2024-09-13T17:16:07.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.867+0000 7f4778b70640 1 -- 172.21.15.39:0/3039519554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f476c0d7c60 con 0x7f476c0cacb0 2024-09-13T17:16:07.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475ffff640 1 -- 172.21.15.39:0/3039519554 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+5182 (secure 0 0 0) 0x7f476c0d7c60 con 0x7f476c0cacb0 2024-09-13T17:16:07.877 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:07.877 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":194,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":194,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:06.949945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 194 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 -- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4758066790 msgr2=0x7f4758068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4758066790 0x7f4758068c50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f477404fbc0 tx=0x7f477406c040 comp rx=0 tx=0).stop 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 -- 172.21.15.39:0/3039519554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f476c0cacb0 msgr2=0x7f476c0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f476c0cacb0 0x7f476c0cb130 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f476000ba30 tx=0x7f4764005800 comp rx=0 tx=0).stop 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 -- 172.21.15.39:0/3039519554 shutdown_connections 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4758066790 0x7f4758068c50 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f476c0cacb0 0x7f476c0cb130 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f476c0cb670 0x7f476c0c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 --2- 172.21.15.39:0/3039519554 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f476c0d8e90 0x7f476c0d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 -- 172.21.15.39:0/3039519554 >> 172.21.15.39:0/3039519554 conn(0x7f476c04c0b0 msgr2=0x7f476c005180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 -- 172.21.15.39:0/3039519554 shutdown_connections 2024-09-13T17:16:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:07.871+0000 7f475dffb640 1 -- 172.21.15.39:0/3039519554 wait complete. 2024-09-13T17:16:07.917 INFO:tasks.mds_thrash.fs.[b]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 194, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 181, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_6530": { "addr": "172.21.15.84:6835/380008541", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 380008541, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 380008541, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6530, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 27 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:14:08.379706+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6530 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 194, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6638": { "addr": "172.21.15.39:6837/3255731814", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 3255731814, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 3255731814, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6638, "incarnation": 193, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:reconnect", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 62, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:16:06.949945+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6638 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:16:09.918 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:10.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.399+0000 7ff6a8dc8640 1 Processor -- start 2024-09-13T17:16:10.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a8dc8640 1 -- start start 2024-09-13T17:16:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a8dc8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff69c0d7420 0x7ff69c00dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a8dc8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff69c042cc0 0x7ff69c006d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a8dc8640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff69c00e240 0x7ff69c0dd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a8dc8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69c00f950 con 0x7ff69c0d7420 2024-09-13T17:16:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a8dc8640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69c0d9c10 con 0x7ff69c00e240 2024-09-13T17:16:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a8dc8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0da470 con 0x7ff69c042cc0 2024-09-13T17:16:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff69c0d7420 0x7ff69c00dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff69c0d7420 0x7ff69c00dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33832/0 (socket says 172.21.15.39:33832) 2024-09-13T17:16:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.407+0000 7ff6a37fe640 1 -- 172.21.15.39:0/798756182 learned_addr learned my addr 172.21.15.39:0/798756182 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.411+0000 7ff6a37fe640 1 -- 172.21.15.39:0/798756182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff69c042cc0 msgr2=0x7ff69c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.411+0000 7ff6a37fe640 1 --2- 172.21.15.39:0/798756182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff69c042cc0 0x7ff69c006d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.411+0000 7ff6a37fe640 1 -- 172.21.15.39:0/798756182 >> v2:172.21.15.84:3300/0 conn(0x7ff69c00e240 msgr2=0x7ff69c0dd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.411+0000 7ff6a37fe640 1 --2- 172.21.15.39:0/798756182 >> v2:172.21.15.84:3300/0 conn(0x7ff69c00e240 0x7ff69c0dd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.411+0000 7ff6a37fe640 1 -- 172.21.15.39:0/798756182 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c00f390 con 0x7ff69c0d7420 2024-09-13T17:16:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.411+0000 7ff6a37fe640 1 --2- 172.21.15.39:0/798756182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c0d7420 0x7ff69c00dd00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0de2a0 tx=0x7ff69001c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bd1cd50c6d0840 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.415+0000 7ff6a27fc640 1 -- 172.21.15.39:0/798756182 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900a9680 con 0x7ff69c0d7420 2024-09-13T17:16:10.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.415+0000 7ff6a27fc640 1 -- 172.21.15.39:0/798756182 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6900a97e0 con 0x7ff69c0d7420 2024-09-13T17:16:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.415+0000 7ff6a27fc640 1 -- 172.21.15.39:0/798756182 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900a9b70 con 0x7ff69c0d7420 2024-09-13T17:16:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.419+0000 7ff6a8dc8640 1 -- 172.21.15.39:0/798756182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c0d7420 msgr2=0x7ff69c00dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:10.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.419+0000 7ff6a8dc8640 1 --2- 172.21.15.39:0/798756182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c0d7420 0x7ff69c00dd00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0de2a0 tx=0x7ff69001c750 comp rx=0 tx=0).stop 2024-09-13T17:16:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.423+0000 7ff6a8dc8640 1 -- 172.21.15.39:0/798756182 shutdown_connections 2024-09-13T17:16:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.423+0000 7ff6a8dc8640 1 --2- 172.21.15.39:0/798756182 >> v2:172.21.15.84:3300/0 conn(0x7ff69c00e240 0x7ff69c0dd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.423+0000 7ff6a8dc8640 1 --2- 172.21.15.39:0/798756182 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff69c042cc0 0x7ff69c006d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.423+0000 7ff6a8dc8640 1 --2- 172.21.15.39:0/798756182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c0d7420 0x7ff69c00dd00 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.423+0000 7ff6a8dc8640 1 -- 172.21.15.39:0/798756182 >> 172.21.15.39:0/798756182 conn(0x7ff69c04c0b0 msgr2=0x7ff69c04c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.427+0000 7ff6a8dc8640 1 -- 172.21.15.39:0/798756182 shutdown_connections 2024-09-13T17:16:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.427+0000 7ff6a8dc8640 1 -- 172.21.15.39:0/798756182 wait complete. 2024-09-13T17:16:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a8dc8640 1 Processor -- start 2024-09-13T17:16:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a8dc8640 1 -- start start 2024-09-13T17:16:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a8dc8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff69c00e240 0x7ff69c043230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a8dc8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c042cc0 0x7ff69c0d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a8dc8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff69c0d7420 0x7ff69c048f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a8dc8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0d07c0 con 0x7ff69c042cc0 2024-09-13T17:16:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a8dc8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c00dbd0 con 0x7ff69c00e240 2024-09-13T17:16:10.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a8dc8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c045a40 con 0x7ff69c0d7420 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c042cc0 0x7ff69c0d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c042cc0 0x7ff69c0d2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33846/0 (socket says 172.21.15.39:33846) 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a2ffd640 1 -- 172.21.15.39:0/1598101407 learned_addr learned my addr 172.21.15.39:0/1598101407 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a3fff640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff69c0d7420 0x7ff69c048f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a2ffd640 1 -- 172.21.15.39:0/1598101407 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff69c0d7420 msgr2=0x7ff69c048f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a2ffd640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff69c0d7420 0x7ff69c048f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a2ffd640 1 -- 172.21.15.39:0/1598101407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff69c00e240 msgr2=0x7ff69c043230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a2ffd640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff69c00e240 0x7ff69c043230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a2ffd640 1 -- 172.21.15.39:0/1598101407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900094d0 con 0x7ff69c042cc0 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff6a2ffd640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c042cc0 0x7ff69c0d2bb0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff69800b550 tx=0x7ff69800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff68ffff640 1 -- 172.21.15.39:0/1598101407 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698010870 con 0x7ff69c042cc0 2024-09-13T17:16:10.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.431+0000 7ff68ffff640 1 -- 172.21.15.39:0/1598101407 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6980109f0 con 0x7ff69c042cc0 2024-09-13T17:16:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.435+0000 7ff68ffff640 1 -- 172.21.15.39:0/1598101407 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698010d80 con 0x7ff69c042cc0 2024-09-13T17:16:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.435+0000 7ff6a8dc8640 1 -- 172.21.15.39:0/1598101407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c045d20 con 0x7ff69c042cc0 2024-09-13T17:16:10.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.435+0000 7ff6a8dc8640 1 -- 172.21.15.39:0/1598101407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c0d30f0 con 0x7ff69c042cc0 2024-09-13T17:16:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.435+0000 7ff68dffb640 1 -- 172.21.15.39:0/1598101407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c005260 con 0x7ff69c042cc0 2024-09-13T17:16:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.435+0000 7ff68ffff640 1 -- 172.21.15.39:0/1598101407 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff69800f3c0 con 0x7ff69c042cc0 2024-09-13T17:16:10.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.439+0000 7ff68ffff640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff688066690 0x7ff688068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:10.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.439+0000 7ff68ffff640 1 -- 172.21.15.39:0/1598101407 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7ff698085220 con 0x7ff69c042cc0 2024-09-13T17:16:10.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.443+0000 7ff6a37fe640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff688066690 0x7ff688068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:10.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.447+0000 7ff6a37fe640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff688066690 0x7ff688068b50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff6900ae000 tx=0x7ff6900a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:10.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:10.455+0000 7ff68ffff640 1 -- 172.21.15.39:0/1598101407 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff698051470 con 0x7ff69c042cc0 2024-09-13T17:16:11.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.471+0000 7ff68dffb640 1 -- 172.21.15.39:0/1598101407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff67c005a90 con 0x7ff69c042cc0 2024-09-13T17:16:11.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.471+0000 7ff68ffff640 1 -- 172.21.15.39:0/1598101407 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) v1 ==== 95+0+5179 (secure 0 0 0) 0x7ff698054a90 con 0x7ff69c042cc0 2024-09-13T17:16:11.476 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:11.476 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":196,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:11.476 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 196 2024-09-13T17:16:11.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.475+0000 7ff68dffb640 1 -- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff688066690 msgr2=0x7ff688068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:11.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.475+0000 7ff68dffb640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff688066690 0x7ff688068b50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff6900ae000 tx=0x7ff6900a7000 comp rx=0 tx=0).stop 2024-09-13T17:16:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.475+0000 7ff68dffb640 1 -- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c042cc0 msgr2=0x7ff69c0d2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.475+0000 7ff68dffb640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c042cc0 0x7ff69c0d2bb0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff69800b550 tx=0x7ff69800ba10 comp rx=0 tx=0).stop 2024-09-13T17:16:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.475+0000 7ff68dffb640 1 -- 172.21.15.39:0/1598101407 shutdown_connections 2024-09-13T17:16:11.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.475+0000 7ff68dffb640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff688066690 0x7ff688068b50 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:11.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.479+0000 7ff68dffb640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff69c0d7420 0x7ff69c048f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:11.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.479+0000 7ff68dffb640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff69c042cc0 0x7ff69c0d2bb0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:11.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.479+0000 7ff68dffb640 1 --2- 172.21.15.39:0/1598101407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff69c00e240 0x7ff69c043230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:11.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.479+0000 7ff68dffb640 1 -- 172.21.15.39:0/1598101407 >> 172.21.15.39:0/1598101407 conn(0x7ff69c04c0b0 msgr2=0x7ff69c0e3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:11.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.483+0000 7ff68dffb640 1 -- 172.21.15.39:0/1598101407 shutdown_connections 2024-09-13T17:16:11.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:11.483+0000 7ff68dffb640 1 -- 172.21.15.39:0/1598101407 wait complete. 2024-09-13T17:16:11.595 INFO:tasks.mds_thrash.fs.[b]:mds.c has gained rank=0, replacing gid=8220 2024-09-13T17:16:11.595 INFO:tasks.mds_thrash.fs.[b]:waiting for 2 secs before reviving mds.a 2024-09-13T17:16:13.234 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-13T17:16:13.227+0000 7f918cfd0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:16:13.596 INFO:tasks.mds_thrash.fs.[b]:reviving mds.a 2024-09-13T17:16:13.597 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:16:13.597 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:16:13.600 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:16:13.600 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:13.749 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T17:16:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.035+0000 7f2dbd423640 1 Processor -- start 2024-09-13T17:16:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.035+0000 7f2dbd423640 1 -- start start 2024-09-13T17:16:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.039+0000 7f2dbd423640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2db81540b0 0x7f2db80a8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.039+0000 7f2dbd423640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 0x7f2db80c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:14.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.039+0000 7f2dbd423640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db8154490 con 0x7f2db806cb10 2024-09-13T17:16:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.039+0000 7f2dbd423640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db80c4680 con 0x7f2db81540b0 2024-09-13T17:16:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.039+0000 7f2dbd423640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db80c4960 con 0x7f2db80c4110 2024-09-13T17:16:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.039+0000 7f2db67fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2db806cb10 0x7f2db80a9530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33978/0 (socket says 172.21.15.39:33978) 2024-09-13T17:16:14.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.039+0000 7f2db67fc640 1 -- 172.21.15.39:0/1371035332 learned_addr learned my addr 172.21.15.39:0/1371035332 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:16:14.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.043+0000 7f2db77fe640 1 --2- 172.21.15.39:0/1371035332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 0x7f2db80c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:14.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.043+0000 7f2db77fe640 1 -- 172.21.15.39:0/1371035332 >> v2:172.21.15.39:3300/0 conn(0x7f2db81540b0 msgr2=0x7f2db80a8ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:14.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.059+0000 7f2db5ffb640 1 -- 172.21.15.39:0/1371035332 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596967913 0 0) 0x7f2db8154490 con 0x7f2db806cb10 2024-09-13T17:16:14.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.095+0000 7f2db6ffd640 1 --2- 172.21.15.39:0/1371035332 >> v2:172.21.15.39:3300/0 conn(0x7f2db81540b0 0x7f2db80a8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:14.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.095+0000 7f2db77fe640 1 --2- 172.21.15.39:0/1371035332 >> v2:172.21.15.39:3300/0 conn(0x7f2db81540b0 0x7f2db80a8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:14.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.095+0000 7f2db77fe640 1 -- 172.21.15.39:0/1371035332 >> v1:172.21.15.39:6789/0 conn(0x7f2db806cb10 legacy=0x7f2db80a9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:14.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.095+0000 7f2db77fe640 1 -- 172.21.15.39:0/1371035332 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db80c4c40 con 0x7f2db80c4110 2024-09-13T17:16:14.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.099+0000 7f2db77fe640 1 --2- 172.21.15.39:0/1371035332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 0x7f2db80c0480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2da80097c0 tx=0x7f2da801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e1e1ecede7997da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:14.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.099+0000 7f2db5ffb640 1 -- 172.21.15.39:0/1371035332 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80ab670 con 0x7f2db80c4110 2024-09-13T17:16:14.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.103+0000 7f2db5ffb640 1 -- 172.21.15.39:0/1371035332 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da80ab7d0 con 0x7f2db80c4110 2024-09-13T17:16:14.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.103+0000 7f2dbd423640 1 -- 172.21.15.39:0/1371035332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 msgr2=0x7f2db80c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:14.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.103+0000 7f2dbd423640 1 --2- 172.21.15.39:0/1371035332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 0x7f2db80c0480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2da80097c0 tx=0x7f2da801c450 comp rx=0 tx=0).stop 2024-09-13T17:16:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.127+0000 7f2dbd423640 1 -- 172.21.15.39:0/1371035332 shutdown_connections 2024-09-13T17:16:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.127+0000 7f2dbd423640 1 --2- 172.21.15.39:0/1371035332 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 0x7f2db80c0480 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.127+0000 7f2dbd423640 1 --2- 172.21.15.39:0/1371035332 >> v2:172.21.15.39:3300/0 conn(0x7f2db81540b0 0x7f2db80a8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.127+0000 7f2dbd423640 1 -- 172.21.15.39:0/1371035332 >> 172.21.15.39:0/1371035332 conn(0x7f2db80a75b0 msgr2=0x7f2db80a79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.151+0000 7f2dbd423640 1 -- 172.21.15.39:0/1371035332 shutdown_connections 2024-09-13T17:16:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.151+0000 7f2dbd423640 1 -- 172.21.15.39:0/1371035332 wait complete. 2024-09-13T17:16:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.155+0000 7f2dbd423640 1 Processor -- start 2024-09-13T17:16:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.155+0000 7f2dbd423640 1 -- start start 2024-09-13T17:16:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.155+0000 7f2dbd423640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db806cb10 0x7f2db80ba820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.155+0000 7f2dbd423640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 0x7f2db80a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.155+0000 7f2dbd423640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2db81f9590 0x7f2db806f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.155+0000 7f2dbd423640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db81507c0 con 0x7f2db806cb10 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.155+0000 7f2dbd423640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db81fd510 con 0x7f2db81f9590 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.155+0000 7f2dbd423640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db8150960 con 0x7f2db80c4110 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2db6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db806cb10 0x7f2db80ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2db6ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db806cb10 0x7f2db80ba820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33880/0 (socket says 172.21.15.39:33880) 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2db6ffd640 1 -- 172.21.15.39:0/950378593 learned_addr learned my addr 172.21.15.39:0/950378593 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2db6ffd640 1 -- 172.21.15.39:0/950378593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 msgr2=0x7f2db80a66d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2db6ffd640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 0x7f2db80a66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2db6ffd640 1 -- 172.21.15.39:0/950378593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2db81f9590 msgr2=0x7f2db806f270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2db6ffd640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2db81f9590 0x7f2db806f270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2db6ffd640 1 -- 172.21.15.39:0/950378593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da8009420 con 0x7f2db806cb10 2024-09-13T17:16:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2db6ffd640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db806cb10 0x7f2db80ba820 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f2db0007f70 tx=0x7f2db000a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2d9f7fe640 1 -- 172.21.15.39:0/950378593 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0003ba0 con 0x7f2db806cb10 2024-09-13T17:16:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db81fd9a0 con 0x7f2db806cb10 2024-09-13T17:16:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db8150bb0 con 0x7f2db806cb10 2024-09-13T17:16:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2d9f7fe640 1 -- 172.21.15.39:0/950378593 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db0003d00 con 0x7f2db806cb10 2024-09-13T17:16:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.159+0000 7f2d9f7fe640 1 -- 172.21.15.39:0/950378593 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0004260 con 0x7f2db806cb10 2024-09-13T17:16:14.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.163+0000 7f2d9f7fe640 1 -- 172.21.15.39:0/950378593 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2db00043c0 con 0x7f2db806cb10 2024-09-13T17:16:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.163+0000 7f2d9f7fe640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2da4066690 0x7f2da4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.167+0000 7f2d9f7fe640 1 -- 172.21.15.39:0/950378593 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7f2db0012070 con 0x7f2db806cb10 2024-09-13T17:16:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.167+0000 7f2db67fc640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2da4066690 0x7f2da4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:14.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.167+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d98005260 con 0x7f2db806cb10 2024-09-13T17:16:14.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.167+0000 7f2db67fc640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2da4066690 0x7f2da4068b50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2dac000c10 tx=0x7f2dac0078a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:14.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:14.179+0000 7f2d9f7fe640 1 -- 172.21.15.39:0/950378593 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2db0051e00 con 0x7f2db806cb10 2024-09-13T17:16:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.319+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d98005a90 con 0x7f2db806cb10 2024-09-13T17:16:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.323+0000 7f2d9f7fe640 1 -- 172.21.15.39:0/950378593 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f2db0055420 con 0x7f2db806cb10 2024-09-13T17:16:15.335 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:15.336 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:15.336 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 197 2024-09-13T17:16:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2da4066690 msgr2=0x7f2da4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:15.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2da4066690 0x7f2da4068b50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2dac000c10 tx=0x7f2dac0078a0 comp rx=0 tx=0).stop 2024-09-13T17:16:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db806cb10 msgr2=0x7f2db80ba820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db806cb10 0x7f2db80ba820 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f2db0007f70 tx=0x7f2db000a720 comp rx=0 tx=0).stop 2024-09-13T17:16:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 shutdown_connections 2024-09-13T17:16:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2da4066690 0x7f2da4068b50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:15.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2db81f9590 0x7f2db806f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:15.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2db80c4110 0x7f2db80a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:15.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 --2- 172.21.15.39:0/950378593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2db806cb10 0x7f2db80ba820 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:15.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 >> 172.21.15.39:0/950378593 conn(0x7f2db80a75b0 msgr2=0x7f2db80c2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:15.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 shutdown_connections 2024-09-13T17:16:15.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:15.335+0000 7f2dbd423640 1 -- 172.21.15.39:0/950378593 wait complete. 2024-09-13T17:16:15.386 INFO:tasks.mds_thrash.fs.[b]:mds.a reported in up:standby state 2024-09-13T17:16:15.387 INFO:tasks.mds_thrash.fs.[b]:waiting for 109 secs before thrashing 2024-09-13T17:16:17.150 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:17.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d98b5c640 1 Processor -- start 2024-09-13T17:16:17.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d98b5c640 1 -- start start 2024-09-13T17:16:17.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d98b5c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d94158c10 0x7f1d940bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:17.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d98b5c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1d940c4a20 0x7f1d940c0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:17.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d98b5c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d9406bc40 0x7f1d9406c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:17.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d98b5c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940a8220 con 0x7f1d940c4a20 2024-09-13T17:16:17.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d98b5c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9406b7b0 con 0x7f1d9406bc40 2024-09-13T17:16:17.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d98b5c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940c3fc0 con 0x7f1d94158c10 2024-09-13T17:16:17.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d91d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1d940c4a20 0x7f1d940c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:17.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d91d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1d940c4a20 0x7f1d940c0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33886/0 (socket says 172.21.15.39:33886) 2024-09-13T17:16:17.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d91d9b640 1 -- 172.21.15.39:0/149541735 learned_addr learned my addr 172.21.15.39:0/149541735 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:17.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d92d9d640 1 --2- 172.21.15.39:0/149541735 >> v2:172.21.15.84:3300/0 conn(0x7f1d9406bc40 0x7f1d9406c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:17.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d91d9b640 1 -- 172.21.15.39:0/149541735 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d94158c10 msgr2=0x7f1d940bff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:17.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d91d9b640 1 --2- 172.21.15.39:0/149541735 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d94158c10 0x7f1d940bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:17.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d91d9b640 1 -- 172.21.15.39:0/149541735 >> v2:172.21.15.84:3300/0 conn(0x7f1d9406bc40 msgr2=0x7f1d9406c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:17.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d91d9b640 1 --2- 172.21.15.39:0/149541735 >> v2:172.21.15.84:3300/0 conn(0x7f1d9406bc40 0x7f1d9406c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:17.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.611+0000 7f1d91d9b640 1 -- 172.21.15.39:0/149541735 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940c42a0 con 0x7f1d940c4a20 2024-09-13T17:16:17.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.615+0000 7f1d91d9b640 1 --2- 172.21.15.39:0/149541735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d940c4a20 0x7f1d940c0450 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f1d9406d300 tx=0x7f1d8c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dda623d61728db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:17.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.615+0000 7f1d9159a640 1 -- 172.21.15.39:0/149541735 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c0a5550 con 0x7f1d940c4a20 2024-09-13T17:16:17.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.615+0000 7f1d9159a640 1 -- 172.21.15.39:0/149541735 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c0a56b0 con 0x7f1d940c4a20 2024-09-13T17:16:17.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.619+0000 7f1d98b5c640 1 -- 172.21.15.39:0/149541735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d940c4a20 msgr2=0x7f1d940c0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:17.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.619+0000 7f1d98b5c640 1 --2- 172.21.15.39:0/149541735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d940c4a20 0x7f1d940c0450 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f1d9406d300 tx=0x7f1d8c01c8d0 comp rx=0 tx=0).stop 2024-09-13T17:16:17.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.619+0000 7f1d98b5c640 1 -- 172.21.15.39:0/149541735 shutdown_connections 2024-09-13T17:16:17.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.619+0000 7f1d98b5c640 1 --2- 172.21.15.39:0/149541735 >> v2:172.21.15.84:3300/0 conn(0x7f1d9406bc40 0x7f1d9406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:17.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.619+0000 7f1d98b5c640 1 --2- 172.21.15.39:0/149541735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d940c4a20 0x7f1d940c0450 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:17.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.619+0000 7f1d98b5c640 1 --2- 172.21.15.39:0/149541735 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d94158c10 0x7f1d940bff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:17.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.619+0000 7f1d98b5c640 1 -- 172.21.15.39:0/149541735 >> 172.21.15.39:0/149541735 conn(0x7f1d940a72c0 msgr2=0x7f1d940a76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:17.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.619+0000 7f1d98b5c640 1 -- 172.21.15.39:0/149541735 shutdown_connections 2024-09-13T17:16:17.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.623+0000 7f1d98b5c640 1 -- 172.21.15.39:0/149541735 wait complete. 2024-09-13T17:16:17.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d98b5c640 1 Processor -- start 2024-09-13T17:16:17.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d98b5c640 1 -- start start 2024-09-13T17:16:17.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d98b5c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d9406bc40 0x7f1d941612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:17.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d98b5c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d940c4a20 0x7f1d940aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:17.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d98b5c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d94158c10 0x7f1d941604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:17.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d98b5c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d942026b0 con 0x7f1d94158c10 2024-09-13T17:16:17.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d98b5c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94202b40 con 0x7f1d940c4a20 2024-09-13T17:16:17.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d98b5c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d94161830 con 0x7f1d9406bc40 2024-09-13T17:16:17.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d92d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d94158c10 0x7f1d941604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:17.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d92d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d94158c10 0x7f1d941604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33888/0 (socket says 172.21.15.39:33888) 2024-09-13T17:16:17.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d92d9d640 1 -- 172.21.15.39:0/3223384687 learned_addr learned my addr 172.21.15.39:0/3223384687 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:17.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d92d9d640 1 -- 172.21.15.39:0/3223384687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d9406bc40 msgr2=0x7f1d941612f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:16:17.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d92d9d640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d9406bc40 0x7f1d941612f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:17.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d92d9d640 1 -- 172.21.15.39:0/3223384687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d940c4a20 msgr2=0x7f1d940aa1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:16:17.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d92d9d640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d940c4a20 0x7f1d940aa1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:17.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d92d9d640 1 -- 172.21.15.39:0/3223384687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c009b30 con 0x7f1d94158c10 2024-09-13T17:16:17.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.627+0000 7f1d92d9d640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d94158c10 0x7f1d941604f0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400e7f0 tx=0x7f1d8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:17.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.631+0000 7f1d82ffd640 1 -- 172.21.15.39:0/3223384687 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400c730 con 0x7f1d94158c10 2024-09-13T17:16:17.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.631+0000 7f1d82ffd640 1 -- 172.21.15.39:0/3223384687 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8400c890 con 0x7f1d94158c10 2024-09-13T17:16:17.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.631+0000 7f1d82ffd640 1 -- 172.21.15.39:0/3223384687 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400cc20 con 0x7f1d94158c10 2024-09-13T17:16:17.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.639+0000 7f1d98b5c640 1 -- 172.21.15.39:0/3223384687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9408aa20 con 0x7f1d94158c10 2024-09-13T17:16:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.639+0000 7f1d98b5c640 1 -- 172.21.15.39:0/3223384687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9415c7e0 con 0x7f1d94158c10 2024-09-13T17:16:17.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.639+0000 7f1d98b5c640 1 -- 172.21.15.39:0/3223384687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9406c050 con 0x7f1d94158c10 2024-09-13T17:16:17.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.643+0000 7f1d82ffd640 1 -- 172.21.15.39:0/3223384687 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d8400cd80 con 0x7f1d94158c10 2024-09-13T17:16:17.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.647+0000 7f1d82ffd640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d74066760 0x7f1d74068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:17.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.647+0000 7f1d82ffd640 1 -- 172.21.15.39:0/3223384687 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7f1d840851b0 con 0x7f1d94158c10 2024-09-13T17:16:17.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.651+0000 7f1d9259c640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d74066760 0x7f1d74068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:17.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.651+0000 7f1d9259c640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d74066760 0x7f1d74068c20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009670 tx=0x7f1d88006b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:17.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:17.675+0000 7f1d82ffd640 1 -- 172.21.15.39:0/3223384687 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1d84051510 con 0x7f1d94158c10 2024-09-13T17:16:18.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.851+0000 7f1d98b5c640 1 -- 172.21.15.39:0/3223384687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d940a4320 con 0x7f1d94158c10 2024-09-13T17:16:18.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.851+0000 7f1d82ffd640 1 -- 172.21.15.39:0/3223384687 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f1d84054b30 con 0x7f1d94158c10 2024-09-13T17:16:18.859 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:18.859 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:18.859 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 197 2024-09-13T17:16:18.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 -- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d74066760 msgr2=0x7f1d74068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:18.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d74066760 0x7f1d74068c20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009670 tx=0x7f1d88006b90 comp rx=0 tx=0).stop 2024-09-13T17:16:18.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 -- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d94158c10 msgr2=0x7f1d941604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:18.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d94158c10 0x7f1d941604f0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400e7f0 tx=0x7f1d8400ecb0 comp rx=0 tx=0).stop 2024-09-13T17:16:18.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 -- 172.21.15.39:0/3223384687 shutdown_connections 2024-09-13T17:16:18.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1d74066760 0x7f1d74068c20 secure :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009670 tx=0x7f1d88006b90 comp rx=0 tx=0).stop 2024-09-13T17:16:18.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d940c4a20 0x7f1d940aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:18.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d94158c10 0x7f1d941604f0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:18.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 --2- 172.21.15.39:0/3223384687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1d9406bc40 0x7f1d941612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:18.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.863+0000 7f1d80ff9640 1 -- 172.21.15.39:0/3223384687 >> 172.21.15.39:0/3223384687 conn(0x7f1d940a72c0 msgr2=0x7f1d940bfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:18.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.879+0000 7f1d80ff9640 1 -- 172.21.15.39:0/3223384687 shutdown_connections 2024-09-13T17:16:18.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:18.883+0000 7f1d80ff9640 1 -- 172.21.15.39:0/3223384687 wait complete. 2024-09-13T17:16:19.019 INFO:tasks.mds_thrash.fs.[a]:kill mds.d (rank=0) 2024-09-13T17:16:19.019 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T17:16:19.020 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:16:19.036 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T17:16:19.036 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.799+0000 7f03a1021640 1 Processor -- start 2024-09-13T17:16:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.799+0000 7f03a1021640 1 -- start start 2024-09-13T17:16:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.799+0000 7f03a1021640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 0x7f039c162b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.799+0000 7f03a1021640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c158e40 con 0x7f039c06caa0 2024-09-13T17:16:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.799+0000 7f03a1021640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c0a5080 con 0x7f039c0bf790 2024-09-13T17:16:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.799+0000 7f03a1021640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c158230 con 0x7f039c0a4bd0 2024-09-13T17:16:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.803+0000 7f039bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f039c06caa0 0x7f039c0c3510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:33990/0 (socket says 172.21.15.39:33990) 2024-09-13T17:16:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.803+0000 7f039bfff640 1 -- 172.21.15.39:0/1457189085 learned_addr learned my addr 172.21.15.39:0/1457189085 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:16:19.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.803+0000 7f03a0820640 1 --2- 172.21.15.39:0/1457189085 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 0x7f039c162b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:19.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.803+0000 7f03a0820640 1 -- 172.21.15.39:0/1457189085 >> v1:172.21.15.84:6789/0 conn(0x7f039c0bf790 legacy=0x7f039c0c3c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:19.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.803+0000 7f03a0820640 1 -- 172.21.15.39:0/1457189085 >> v1:172.21.15.39:6789/0 conn(0x7f039c06caa0 legacy=0x7f039c0c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:19.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.803+0000 7f03a0820640 1 -- 172.21.15.39:0/1457189085 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c158510 con 0x7f039c0a4bd0 2024-09-13T17:16:19.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.811+0000 7f03a0820640 1 --2- 172.21.15.39:0/1457189085 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 0x7f039c162b60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f038c0097f0 tx=0x7f038c01c2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3046555ff2058291 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:19.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.811+0000 7f039affd640 1 -- 172.21.15.39:0/1457189085 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0ab600 con 0x7f039c0a4bd0 2024-09-13T17:16:19.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.811+0000 7f039affd640 1 -- 172.21.15.39:0/1457189085 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c0a7050 con 0x7f039c0a4bd0 2024-09-13T17:16:19.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.811+0000 7f03a1021640 1 -- 172.21.15.39:0/1457189085 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 msgr2=0x7f039c162b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:19.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.811+0000 7f03a1021640 1 --2- 172.21.15.39:0/1457189085 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 0x7f039c162b60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f038c0097f0 tx=0x7f038c01c2a0 comp rx=0 tx=0).stop 2024-09-13T17:16:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.815+0000 7f03a1021640 1 -- 172.21.15.39:0/1457189085 shutdown_connections 2024-09-13T17:16:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.815+0000 7f03a1021640 1 --2- 172.21.15.39:0/1457189085 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 0x7f039c162b60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.815+0000 7f03a1021640 1 -- 172.21.15.39:0/1457189085 >> 172.21.15.39:0/1457189085 conn(0x7f039c0c5510 msgr2=0x7f039c0c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.815+0000 7f03a1021640 1 -- 172.21.15.39:0/1457189085 shutdown_connections 2024-09-13T17:16:19.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.815+0000 7f03a1021640 1 -- 172.21.15.39:0/1457189085 wait complete. 2024-09-13T17:16:19.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.819+0000 7f03a1021640 1 Processor -- start 2024-09-13T17:16:19.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a1021640 1 -- start start 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a1021640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039c06caa0 0x7f039c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a1021640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 0x7f039c15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a1021640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f039c0bf790 0x7f039c0a9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a1021640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c141c20 con 0x7f039c06caa0 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a1021640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c141f00 con 0x7f039c0bf790 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a1021640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c140da0 con 0x7f039c0a4bd0 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a0820640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f039c0bf790 0x7f039c0a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a0820640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f039c0bf790 0x7f039c0a9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:48752/0 (socket says 172.21.15.39:48752) 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a0820640 1 -- 172.21.15.39:0/3728956821 learned_addr learned my addr 172.21.15.39:0/3728956821 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a0820640 1 -- 172.21.15.39:0/3728956821 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 msgr2=0x7f039c15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a0820640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 0x7f039c15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:19.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a0820640 1 -- 172.21.15.39:0/3728956821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039c06caa0 msgr2=0x7f039c157b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:16:19.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a0820640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039c06caa0 0x7f039c157b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:19.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a0820640 1 -- 172.21.15.39:0/3728956821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c009420 con 0x7f039c0bf790 2024-09-13T17:16:19.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.823+0000 7f03a0820640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f039c0bf790 0x7f039c0a9120 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f038c0097f0 tx=0x7f038c019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:19.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.827+0000 7f0398ff9640 1 -- 172.21.15.39:0/3728956821 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0aa970 con 0x7f039c0bf790 2024-09-13T17:16:19.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.827+0000 7f0398ff9640 1 -- 172.21.15.39:0/3728956821 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c0aaaf0 con 0x7f039c0bf790 2024-09-13T17:16:19.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.827+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c141080 con 0x7f039c0bf790 2024-09-13T17:16:19.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.827+0000 7f0398ff9640 1 -- 172.21.15.39:0/3728956821 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0aaea0 con 0x7f039c0bf790 2024-09-13T17:16:19.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.827+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c0aa0c0 con 0x7f039c0bf790 2024-09-13T17:16:19.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.827+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c0c3510 con 0x7f039c0bf790 2024-09-13T17:16:19.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.835+0000 7f0398ff9640 1 -- 172.21.15.39:0/3728956821 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f038c0b2040 con 0x7f039c0bf790 2024-09-13T17:16:19.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.835+0000 7f0398ff9640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0388066690 0x7f0388068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:19.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.835+0000 7f0398ff9640 1 -- 172.21.15.39:0/3728956821 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7f038c11fd60 con 0x7f039c0bf790 2024-09-13T17:16:19.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.835+0000 7f039bfff640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0388066690 0x7f0388068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:19.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.835+0000 7f0398ff9640 1 -- 172.21.15.39:0/3728956821 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f038c0ec030 con 0x7f039c0bf790 2024-09-13T17:16:19.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:19.839+0000 7f039bfff640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0388066690 0x7f0388068b50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0394001e30 tx=0x7f0394007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:20.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.575+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f039c142040 con 0x7f039c0bf790 2024-09-13T17:16:20.579 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:20.579 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:20.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.575+0000 7f0398ff9640 1 -- 172.21.15.39:0/3728956821 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f039c142040 con 0x7f039c0bf790 2024-09-13T17:16:20.580 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 197 2024-09-13T17:16:20.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0388066690 msgr2=0x7f0388068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:20.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0388066690 0x7f0388068b50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0394001e30 tx=0x7f0394007d30 comp rx=0 tx=0).stop 2024-09-13T17:16:20.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f039c0bf790 msgr2=0x7f039c0a9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:20.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f039c0bf790 0x7f039c0a9120 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f038c0097f0 tx=0x7f038c019040 comp rx=0 tx=0).stop 2024-09-13T17:16:20.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 shutdown_connections 2024-09-13T17:16:20.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0388066690 0x7f0388068b50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:20.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f039c0bf790 0x7f039c0a9120 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:20.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f039c0a4bd0 0x7f039c15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:20.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 --2- 172.21.15.39:0/3728956821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039c06caa0 0x7f039c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:20.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.579+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 >> 172.21.15.39:0/3728956821 conn(0x7f039c0c5510 msgr2=0x7f039c0a5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:20.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.583+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 shutdown_connections 2024-09-13T17:16:20.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:20.583+0000 7f03a1021640 1 -- 172.21.15.39:0/3728956821 wait complete. 2024-09-13T17:16:20.737 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:16:22.738 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:23.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.251+0000 7f36f7fff640 1 Processor -- start 2024-09-13T17:16:23.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.251+0000 7f36f7fff640 1 -- start start 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.251+0000 7f36f7fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f36f80aa210 0x7f36f80aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.251+0000 7f36f7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f36f814ff40 0x7f36f81503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.251+0000 7f36f7fff640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f80c0240 con 0x7f36f80c2940 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.251+0000 7f36f7fff640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f806da70 con 0x7f36f814ff40 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.251+0000 7f36f7fff640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f806d4f0 con 0x7f36f80aa210 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.255+0000 7f36f77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f36f814ff40 0x7f36f81503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.255+0000 7f36f77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f36f814ff40 0x7f36f81503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54416/0 (socket says 172.21.15.39:54416) 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.255+0000 7f36f6ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f36f80c2940 0x7f36f806b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54692/0 (socket says 172.21.15.39:54692) 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.255+0000 7f36f77fe640 1 -- 172.21.15.39:0/4101242245 learned_addr learned my addr 172.21.15.39:0/4101242245 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.255+0000 7f36f77fe640 1 -- 172.21.15.39:0/4101242245 >> v2:172.21.15.84:3300/0 conn(0x7f36f80aa210 msgr2=0x7f36f80aa760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.255+0000 7f36f77fe640 1 --2- 172.21.15.39:0/4101242245 >> v2:172.21.15.84:3300/0 conn(0x7f36f80aa210 0x7f36f80aa760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.255+0000 7f36f77fe640 1 -- 172.21.15.39:0/4101242245 >> v1:172.21.15.39:6789/0 conn(0x7f36f80c2940 legacy=0x7f36f806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.255+0000 7f36f77fe640 1 -- 172.21.15.39:0/4101242245 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f80c5300 con 0x7f36f814ff40 2024-09-13T17:16:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.259+0000 7f36f77fe640 1 --2- 172.21.15.39:0/4101242245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 0x7f36f81503f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f36f814d9a0 tx=0x7f36e0020790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4753025d9835d749 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.259+0000 7f36f5ffb640 1 -- 172.21.15.39:0/4101242245 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0007ad0 con 0x7f36f814ff40 2024-09-13T17:16:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.259+0000 7f36f5ffb640 1 -- 172.21.15.39:0/4101242245 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0007c30 con 0x7f36f814ff40 2024-09-13T17:16:23.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.259+0000 7f36f5ffb640 1 -- 172.21.15.39:0/4101242245 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e00090d0 con 0x7f36f814ff40 2024-09-13T17:16:23.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.259+0000 7f36f7fff640 1 -- 172.21.15.39:0/4101242245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 msgr2=0x7f36f81503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:23.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.259+0000 7f36f7fff640 1 --2- 172.21.15.39:0/4101242245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 0x7f36f81503f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f36f814d9a0 tx=0x7f36e0020790 comp rx=0 tx=0).stop 2024-09-13T17:16:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.267+0000 7f36f7fff640 1 -- 172.21.15.39:0/4101242245 shutdown_connections 2024-09-13T17:16:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.267+0000 7f36f7fff640 1 --2- 172.21.15.39:0/4101242245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 0x7f36f81503f0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.267+0000 7f36f7fff640 1 --2- 172.21.15.39:0/4101242245 >> v2:172.21.15.84:3300/0 conn(0x7f36f80aa210 0x7f36f80aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:23.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.267+0000 7f36f7fff640 1 -- 172.21.15.39:0/4101242245 >> 172.21.15.39:0/4101242245 conn(0x7f36f80c5510 msgr2=0x7f36f80c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:23.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.279+0000 7f36f7fff640 1 -- 172.21.15.39:0/4101242245 shutdown_connections 2024-09-13T17:16:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.283+0000 7f36f7fff640 1 -- 172.21.15.39:0/4101242245 wait complete. 2024-09-13T17:16:23.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.283+0000 7f36f7fff640 1 Processor -- start 2024-09-13T17:16:23.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.283+0000 7f36f7fff640 1 -- start start 2024-09-13T17:16:23.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.283+0000 7f36f7fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36f80aa210 0x7f36f80bcb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:23.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36f80c2940 0x7f36f80bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:23.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 0x7f36f80bc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:23.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f7fff640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f80be0f0 con 0x7f36f814ff40 2024-09-13T17:16:23.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f7fff640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f81fcfb0 con 0x7f36f80c2940 2024-09-13T17:16:23.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f7fff640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f81fd4e0 con 0x7f36f80aa210 2024-09-13T17:16:23.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 0x7f36f80bc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:23.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 0x7f36f80bc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54428/0 (socket says 172.21.15.39:54428) 2024-09-13T17:16:23.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f77fe640 1 -- 172.21.15.39:0/1286399764 learned_addr learned my addr 172.21.15.39:0/1286399764 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:23.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f77fe640 1 -- 172.21.15.39:0/1286399764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36f80aa210 msgr2=0x7f36f80bcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:23.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f77fe640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36f80aa210 0x7f36f80bcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:23.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f77fe640 1 -- 172.21.15.39:0/1286399764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36f80c2940 msgr2=0x7f36f80bbcb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:16:23.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f77fe640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36f80c2940 0x7f36f80bbcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:23.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f77fe640 1 -- 172.21.15.39:0/1286399764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e000d040 con 0x7f36f814ff40 2024-09-13T17:16:23.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.287+0000 7f36f77fe640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 0x7f36f80bc1f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f36e001e3c0 tx=0x7f36e001d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.295+0000 7f36ef7fe640 1 -- 172.21.15.39:0/1286399764 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0013360 con 0x7f36f814ff40 2024-09-13T17:16:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.295+0000 7f36ef7fe640 1 -- 172.21.15.39:0/1286399764 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e00134c0 con 0x7f36f814ff40 2024-09-13T17:16:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.295+0000 7f36ef7fe640 1 -- 172.21.15.39:0/1286399764 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e00a9750 con 0x7f36f814ff40 2024-09-13T17:16:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.295+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f81fd200 con 0x7f36f814ff40 2024-09-13T17:16:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.295+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f80bb4b0 con 0x7f36f814ff40 2024-09-13T17:16:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.299+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f80aa760 con 0x7f36f814ff40 2024-09-13T17:16:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.299+0000 7f36ef7fe640 1 -- 172.21.15.39:0/1286399764 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f36e00a98b0 con 0x7f36f814ff40 2024-09-13T17:16:23.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.303+0000 7f36ef7fe640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f36e40668f0 0x7f36e4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:23.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.303+0000 7f36f6ffd640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f36e40668f0 0x7f36e4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.303+0000 7f36f6ffd640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f36e40668f0 0x7f36e4068db0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f36f0004ed0 tx=0x7f36f00079f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.303+0000 7f36ef7fe640 1 -- 172.21.15.39:0/1286399764 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7f36e011f5e0 con 0x7f36f814ff40 2024-09-13T17:16:23.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:23.311+0000 7f36ef7fe640 1 -- 172.21.15.39:0/1286399764 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f36e00eb940 con 0x7f36f814ff40 2024-09-13T17:16:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.135+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36f8150aa0 con 0x7f36f814ff40 2024-09-13T17:16:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.135+0000 7f36ef7fe640 1 -- 172.21.15.39:0/1286399764 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f36e00eef60 con 0x7f36f814ff40 2024-09-13T17:16:24.149 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:24.149 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:24.150 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 197 2024-09-13T17:16:24.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.147+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f36e40668f0 msgr2=0x7f36e4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:24.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.147+0000 7f36f7fff640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f36e40668f0 0x7f36e4068db0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f36f0004ed0 tx=0x7f36f00079f0 comp rx=0 tx=0).stop 2024-09-13T17:16:24.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.147+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 msgr2=0x7f36f80bc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:24.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.147+0000 7f36f7fff640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 0x7f36f80bc1f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f36e001e3c0 tx=0x7f36e001d530 comp rx=0 tx=0).stop 2024-09-13T17:16:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.147+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 shutdown_connections 2024-09-13T17:16:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.147+0000 7f36f7fff640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f36e40668f0 0x7f36e4068db0 secure :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f36f0004ed0 tx=0x7f36f00079f0 comp rx=0 tx=0).stop 2024-09-13T17:16:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.147+0000 7f36f7fff640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36f80c2940 0x7f36f80bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:24.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.147+0000 7f36f7fff640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36f814ff40 0x7f36f80bc1f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:24.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.147+0000 7f36f7fff640 1 --2- 172.21.15.39:0/1286399764 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f36f80aa210 0x7f36f80bcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:24.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.155+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 >> 172.21.15.39:0/1286399764 conn(0x7f36f80c5510 msgr2=0x7f36f8150350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:24.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.155+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 shutdown_connections 2024-09-13T17:16:24.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:24.155+0000 7f36f7fff640 1 -- 172.21.15.39:0/1286399764 wait complete. 2024-09-13T17:16:24.225 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:16:26.226 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:26.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f680d1640 1 Processor -- start 2024-09-13T17:16:26.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f680d1640 1 -- start start 2024-09-13T17:16:26.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f680d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1f60151500 0x7f1f600c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:26.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f680d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f600c5a30 con 0x7f1f600c47c0 2024-09-13T17:16:26.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f680d1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f601576a0 con 0x7f1f60071dd0 2024-09-13T17:16:26.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f6666e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f1f600c47c0 0x7f1f60154780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54700/0 (socket says 172.21.15.39:54700) 2024-09-13T17:16:26.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f6666e640 1 -- 172.21.15.39:0/3042133461 learned_addr learned my addr 172.21.15.39:0/3042133461 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:16:26.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f680d1640 1 -- 172.21.15.39:0/3042133461 --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f60157980 con 0x7f1f60151500 2024-09-13T17:16:26.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f65e6d640 1 --2- 172.21.15.39:0/3042133461 >> v2:172.21.15.39:3300/0 conn(0x7f1f60151500 0x7f1f600c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:26.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.355+0000 7f1f64e6b640 1 -- 172.21.15.39:0/3042133461 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736579550 0 0) 0x7f1f601576a0 con 0x7f1f60071dd0 2024-09-13T17:16:26.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.359+0000 7f1f64e6b640 1 -- 172.21.15.39:0/3042133461 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f4c003530 con 0x7f1f60071dd0 2024-09-13T17:16:26.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.359+0000 7f1f64e6b640 1 -- 172.21.15.39:0/3042133461 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077654127 0 0) 0x7f1f600c5a30 con 0x7f1f600c47c0 2024-09-13T17:16:26.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.359+0000 7f1f64e6b640 1 -- 172.21.15.39:0/3042133461 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f601576a0 con 0x7f1f600c47c0 2024-09-13T17:16:26.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.359+0000 7f1f65e6d640 1 -- 172.21.15.39:0/3042133461 >> v1:172.21.15.84:6789/0 conn(0x7f1f60071dd0 legacy=0x7f1f60154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:26.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.359+0000 7f1f65e6d640 1 -- 172.21.15.39:0/3042133461 >> v1:172.21.15.39:6789/0 conn(0x7f1f600c47c0 legacy=0x7f1f60154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:26.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.359+0000 7f1f65e6d640 1 -- 172.21.15.39:0/3042133461 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60157c60 con 0x7f1f60151500 2024-09-13T17:16:26.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f65e6d640 1 --2- 172.21.15.39:0/3042133461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 0x7f1f600c4280 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097f0 tx=0x7f1f5c01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812aa3a5a8134e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:26.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f64e6b640 1 -- 172.21.15.39:0/3042133461 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0ab600 con 0x7f1f60151500 2024-09-13T17:16:26.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f64e6b640 1 -- 172.21.15.39:0/3042133461 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c0ab760 con 0x7f1f60151500 2024-09-13T17:16:26.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f64e6b640 1 -- 172.21.15.39:0/3042133461 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0abaf0 con 0x7f1f60151500 2024-09-13T17:16:26.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 -- 172.21.15.39:0/3042133461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 msgr2=0x7f1f600c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:26.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 --2- 172.21.15.39:0/3042133461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 0x7f1f600c4280 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0097f0 tx=0x7f1f5c01ce40 comp rx=0 tx=0).stop 2024-09-13T17:16:26.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 -- 172.21.15.39:0/3042133461 shutdown_connections 2024-09-13T17:16:26.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 --2- 172.21.15.39:0/3042133461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 0x7f1f600c4280 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:26.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 -- 172.21.15.39:0/3042133461 >> 172.21.15.39:0/3042133461 conn(0x7f1f600ab530 msgr2=0x7f1f600ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:26.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 -- 172.21.15.39:0/3042133461 shutdown_connections 2024-09-13T17:16:26.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 -- 172.21.15.39:0/3042133461 wait complete. 2024-09-13T17:16:26.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 Processor -- start 2024-09-13T17:16:26.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 -- start start 2024-09-13T17:16:26.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f60071dd0 0x7f1f60150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:26.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f600c47c0 0x7f1f60156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:26.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 0x7f1f600a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:26.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600aaf40 con 0x7f1f60151500 2024-09-13T17:16:26.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f601554d0 con 0x7f1f600c47c0 2024-09-13T17:16:26.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f680d1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f601551f0 con 0x7f1f60071dd0 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f6666e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 0x7f1f600a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f6666e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 0x7f1f600a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54452/0 (socket says 172.21.15.39:54452) 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.363+0000 7f1f6666e640 1 -- 172.21.15.39:0/1486875992 learned_addr learned my addr 172.21.15.39:0/1486875992 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f6566c640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f600c47c0 0x7f1f60156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f65e6d640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f60071dd0 0x7f1f60150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f6666e640 1 -- 172.21.15.39:0/1486875992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f60071dd0 msgr2=0x7f1f60150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f6666e640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f60071dd0 0x7f1f60150270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f6666e640 1 -- 172.21.15.39:0/1486875992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f600c47c0 msgr2=0x7f1f60156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f6666e640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f600c47c0 0x7f1f60156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f6666e640 1 -- 172.21.15.39:0/1486875992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c009420 con 0x7f1f60151500 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f6566c640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f600c47c0 0x7f1f60156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:16:26.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f6666e640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 0x7f1f600a93a0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800cbc0 tx=0x7f1f5800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:26.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f567fc640 1 -- 172.21.15.39:0/1486875992 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800ad40 con 0x7f1f60151500 2024-09-13T17:16:26.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f567fc640 1 -- 172.21.15.39:0/1486875992 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f58012400 con 0x7f1f60151500 2024-09-13T17:16:26.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f60153810 con 0x7f1f60151500 2024-09-13T17:16:26.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f567fc640 1 -- 172.21.15.39:0/1486875992 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800aea0 con 0x7f1f60151500 2024-09-13T17:16:26.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f601507b0 con 0x7f1f60151500 2024-09-13T17:16:26.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f601f0a30 con 0x7f1f60151500 2024-09-13T17:16:26.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f567fc640 1 -- 172.21.15.39:0/1486875992 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f58007070 con 0x7f1f60151500 2024-09-13T17:16:26.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f567fc640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1f440666c0 0x7f1f44068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:26.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f567fc640 1 -- 172.21.15.39:0/1486875992 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7f1f58085410 con 0x7f1f60151500 2024-09-13T17:16:26.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f65e6d640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1f440666c0 0x7f1f44068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:26.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.367+0000 7f1f65e6d640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1f440666c0 0x7f1f44068b80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0af000 tx=0x7f1f5c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:26.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.375+0000 7f1f567fc640 1 -- 172.21.15.39:0/1486875992 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f58051790 con 0x7f1f60151500 2024-09-13T17:16:26.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.667+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f601f0a30 con 0x7f1f60151500 2024-09-13T17:16:26.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.667+0000 7f1f567fc640 1 -- 172.21.15.39:0/1486875992 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f1f601f0a30 con 0x7f1f60151500 2024-09-13T17:16:26.670 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:26.670 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:26.670 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 197 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1f440666c0 msgr2=0x7f1f44068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1f440666c0 0x7f1f44068b80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0af000 tx=0x7f1f5c004740 comp rx=0 tx=0).stop 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 msgr2=0x7f1f600a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 0x7f1f600a93a0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800cbc0 tx=0x7f1f5800a430 comp rx=0 tx=0).stop 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 shutdown_connections 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1f440666c0 0x7f1f44068b80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1f600c47c0 0x7f1f60156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f60151500 0x7f1f600a93a0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 --2- 172.21.15.39:0/1486875992 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1f60071dd0 0x7f1f60150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 >> 172.21.15.39:0/1486875992 conn(0x7f1f600ab530 msgr2=0x7f1f60154070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:26.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 shutdown_connections 2024-09-13T17:16:26.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:26.671+0000 7f1f680d1640 1 -- 172.21.15.39:0/1486875992 wait complete. 2024-09-13T17:16:26.691 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:16:28.692 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.847+0000 7efd02820640 1 Processor -- start 2024-09-13T17:16:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.847+0000 7efd02820640 1 -- start start 2024-09-13T17:16:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efd02820640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efcfc065ee0 0x7efcfc0662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efd02820640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efcfc0c2ee0 0x7efcfc0c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efd02820640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc06ddf0 con 0x7efcfc0c6270 2024-09-13T17:16:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efd02820640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc0c4710 con 0x7efcfc065ee0 2024-09-13T17:16:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efd02820640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc0c49d0 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efd00dbd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7efcfc0c6270 0x7efcfc0bfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:54716/0 (socket says 172.21.15.39:54716) 2024-09-13T17:16:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efd00dbd640 1 -- 172.21.15.39:0/3975306436 learned_addr learned my addr 172.21.15.39:0/3975306436 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:16:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efcfb7fe640 1 --2- 172.21.15.39:0/3975306436 >> v2:172.21.15.84:3300/0 conn(0x7efcfc0c2ee0 0x7efcfc0c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efcfaffd640 1 -- 172.21.15.39:0/3975306436 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246039011 0 0) 0x7efcfc06ddf0 con 0x7efcfc0c6270 2024-09-13T17:16:28.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efcfaffd640 1 -- 172.21.15.39:0/3975306436 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce8003530 con 0x7efcfc0c6270 2024-09-13T17:16:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efcfb7fe640 1 -- 172.21.15.39:0/3975306436 >> v2:172.21.15.39:3300/0 conn(0x7efcfc065ee0 msgr2=0x7efcfc0662c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efcfb7fe640 1 --2- 172.21.15.39:0/3975306436 >> v2:172.21.15.39:3300/0 conn(0x7efcfc065ee0 0x7efcfc0662c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efcfb7fe640 1 -- 172.21.15.39:0/3975306436 >> v1:172.21.15.39:6789/0 conn(0x7efcfc0c6270 legacy=0x7efcfc0bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:28.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.851+0000 7efcfaffd640 1 -- 172.21.15.39:0/3975306436 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1540347028 0 0) 0x7efce8003530 con 0x7efcfc0c6270 2024-09-13T17:16:28.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.855+0000 7efcfb7fe640 1 -- 172.21.15.39:0/3975306436 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc154880 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.855+0000 7efcfb7fe640 1 --2- 172.21.15.39:0/3975306436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcfc0c2ee0 0x7efcfc0c2220 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efcfc151ba0 tx=0x7efce401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd7c0c1b404d28b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:28.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.855+0000 7efcfaffd640 1 -- 172.21.15.39:0/3975306436 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce40a4ea0 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.855+0000 7efcfaffd640 1 -- 172.21.15.39:0/3975306436 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce40a7050 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.855+0000 7efcfaffd640 1 -- 172.21.15.39:0/3975306436 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec001680 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 -- 172.21.15.39:0/3975306436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcfc0c2ee0 msgr2=0x7efcfc0c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:28.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 --2- 172.21.15.39:0/3975306436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcfc0c2ee0 0x7efcfc0c2220 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efcfc151ba0 tx=0x7efce401c330 comp rx=0 tx=0).stop 2024-09-13T17:16:28.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 -- 172.21.15.39:0/3975306436 shutdown_connections 2024-09-13T17:16:28.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 --2- 172.21.15.39:0/3975306436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcfc0c2ee0 0x7efcfc0c2220 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:28.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 --2- 172.21.15.39:0/3975306436 >> v2:172.21.15.39:3300/0 conn(0x7efcfc065ee0 0x7efcfc0662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:28.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 -- 172.21.15.39:0/3975306436 >> 172.21.15.39:0/3975306436 conn(0x7efcfc0c3310 msgr2=0x7efcfc0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:28.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 -- 172.21.15.39:0/3975306436 shutdown_connections 2024-09-13T17:16:28.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 -- 172.21.15.39:0/3975306436 wait complete. 2024-09-13T17:16:28.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 Processor -- start 2024-09-13T17:16:28.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 -- start start 2024-09-13T17:16:28.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efcfc065ee0 0x7efcfc14cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:28.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcfc0c2ee0 0x7efcfc155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:28.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcfc0c6270 0x7efcfc156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:28.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc158190 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc155b50 con 0x7efcfc0c6270 2024-09-13T17:16:28.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd02820640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0aa680 con 0x7efcfc065ee0 2024-09-13T17:16:28.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcfc0c2ee0 0x7efcfc155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:28.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcfc0c2ee0 0x7efcfc155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54456/0 (socket says 172.21.15.39:54456) 2024-09-13T17:16:28.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efcfb7fe640 1 -- 172.21.15.39:0/2953537231 learned_addr learned my addr 172.21.15.39:0/2953537231 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:28.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efcfbfff640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efcfc065ee0 0x7efcfc14cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:28.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.859+0000 7efd00dbd640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcfc0c6270 0x7efcfc156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:28.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efcfb7fe640 1 -- 172.21.15.39:0/2953537231 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efcfc065ee0 msgr2=0x7efcfc14cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:28.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efcfb7fe640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efcfc065ee0 0x7efcfc14cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:28.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efcfb7fe640 1 -- 172.21.15.39:0/2953537231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcfc0c6270 msgr2=0x7efcfc156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:28.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efcfb7fe640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcfc0c6270 0x7efcfc156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:28.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efcfb7fe640 1 -- 172.21.15.39:0/2953537231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce4009420 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efcfb7fe640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcfc0c2ee0 0x7efcfc155610 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7efce40ae040 tx=0x7efce401ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:28.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efcf8ff9640 1 -- 172.21.15.39:0/2953537231 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4006e20 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efcf8ff9640 1 -- 172.21.15.39:0/2953537231 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce40b7450 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efcf8ff9640 1 -- 172.21.15.39:0/2953537231 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce40b7820 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc1f0590 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.863+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc157e20 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.867+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc0662c0 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.867+0000 7efcf8ff9640 1 -- 172.21.15.39:0/2953537231 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efce40b7980 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.867+0000 7efcf8ff9640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efcdc0666e0 0x7efcdc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.867+0000 7efcf8ff9640 1 -- 172.21.15.39:0/2953537231 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7efce411fa60 con 0x7efcfc0c2ee0 2024-09-13T17:16:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.867+0000 7efcfbfff640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efcdc0666e0 0x7efcdc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.871+0000 7efcfbfff640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efcdc0666e0 0x7efcdc068ba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efcfc1ed610 tx=0x7efcf400d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:28.875+0000 7efcf8ff9640 1 -- 172.21.15.39:0/2953537231 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efce40ebed0 con 0x7efcfc0c2ee0 2024-09-13T17:16:29.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.167+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcfc15f4d0 con 0x7efcfc0c2ee0 2024-09-13T17:16:29.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.167+0000 7efcf8ff9640 1 -- 172.21.15.39:0/2953537231 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5934 (secure 0 0 0) 0x7efce40b0090 con 0x7efcfc0c2ee0 2024-09-13T17:16:29.170 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:29.170 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:29.171 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 197 2024-09-13T17:16:29.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efcdc0666e0 msgr2=0x7efcdc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:29.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efcdc0666e0 0x7efcdc068ba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efcfc1ed610 tx=0x7efcf400d040 comp rx=0 tx=0).stop 2024-09-13T17:16:29.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcfc0c2ee0 msgr2=0x7efcfc155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:29.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcfc0c2ee0 0x7efcfc155610 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7efce40ae040 tx=0x7efce401ab70 comp rx=0 tx=0).stop 2024-09-13T17:16:29.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 shutdown_connections 2024-09-13T17:16:29.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efcdc0666e0 0x7efcdc068ba0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:29.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efcfc0c6270 0x7efcfc156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:29.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcfc0c2ee0 0x7efcfc155610 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:29.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 --2- 172.21.15.39:0/2953537231 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efcfc065ee0 0x7efcfc14cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:29.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 >> 172.21.15.39:0/2953537231 conn(0x7efcfc0c3310 msgr2=0x7efcfc069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:29.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 shutdown_connections 2024-09-13T17:16:29.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:29.171+0000 7efd02820640 1 -- 172.21.15.39:0/2953537231 wait complete. 2024-09-13T17:16:29.200 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:16:31.201 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efee1b74640 1 Processor -- start 2024-09-13T17:16:31.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efee1b74640 1 -- start start 2024-09-13T17:16:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efee1b74640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efedc0c0a90 0x7efedc0c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efee1b74640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efedc0c2ef0 0x7efedc0c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efee1b74640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efedc06ddf0 con 0x7efedc065ee0 2024-09-13T17:16:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efedbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efedc0c2ef0 0x7efedc0c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:31.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efedbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efedc0c2ef0 0x7efedc0c22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50410/0 (socket says 172.21.15.39:50410) 2024-09-13T17:16:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efedbfff640 1 -- 172.21.15.39:0/3185113596 learned_addr learned my addr 172.21.15.39:0/3185113596 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efee1b74640 1 -- 172.21.15.39:0/3185113596 --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0c53d0 con 0x7efedc0c2ef0 2024-09-13T17:16:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.355+0000 7efee1b74640 1 -- 172.21.15.39:0/3185113596 --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0c5950 con 0x7efedc0c0a90 2024-09-13T17:16:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.359+0000 7efedbfff640 1 -- 172.21.15.39:0/3185113596 >> v2:172.21.15.84:3300/0 conn(0x7efedc0c0a90 msgr2=0x7efedc0c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.359+0000 7efedbfff640 1 --2- 172.21.15.39:0/3185113596 >> v2:172.21.15.84:3300/0 conn(0x7efedc0c0a90 0x7efedc0c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.359+0000 7efedbfff640 1 -- 172.21.15.39:0/3185113596 >> v1:172.21.15.84:6789/0 conn(0x7efedc065ee0 legacy=0x7efedc14f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:31.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.359+0000 7efedbfff640 1 -- 172.21.15.39:0/3185113596 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc0c2820 con 0x7efedc0c2ef0 2024-09-13T17:16:31.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.359+0000 7efedbfff640 1 --2- 172.21.15.39:0/3185113596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c2ef0 0x7efedc0c22e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7efedc156430 tx=0x7efed401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec5fd7ef5f71b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:31.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.359+0000 7efedaffd640 1 -- 172.21.15.39:0/3185113596 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40a5840 con 0x7efedc0c2ef0 2024-09-13T17:16:31.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.359+0000 7efedaffd640 1 -- 172.21.15.39:0/3185113596 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed40a59a0 con 0x7efedc0c2ef0 2024-09-13T17:16:31.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.359+0000 7efedaffd640 1 -- 172.21.15.39:0/3185113596 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40a5d30 con 0x7efedc0c2ef0 2024-09-13T17:16:31.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 -- 172.21.15.39:0/3185113596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c2ef0 msgr2=0x7efedc0c22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:31.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 --2- 172.21.15.39:0/3185113596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c2ef0 0x7efedc0c22e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7efedc156430 tx=0x7efed401c980 comp rx=0 tx=0).stop 2024-09-13T17:16:31.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 -- 172.21.15.39:0/3185113596 shutdown_connections 2024-09-13T17:16:31.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 --2- 172.21.15.39:0/3185113596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c2ef0 0x7efedc0c22e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 --2- 172.21.15.39:0/3185113596 >> v2:172.21.15.84:3300/0 conn(0x7efedc0c0a90 0x7efedc0c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 -- 172.21.15.39:0/3185113596 >> 172.21.15.39:0/3185113596 conn(0x7efedc0c3310 msgr2=0x7efedc0c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:31.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 -- 172.21.15.39:0/3185113596 shutdown_connections 2024-09-13T17:16:31.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 -- 172.21.15.39:0/3185113596 wait complete. 2024-09-13T17:16:31.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 Processor -- start 2024-09-13T17:16:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.363+0000 7efee1b74640 1 -- start start 2024-09-13T17:16:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efee1b74640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc065ee0 0x7efedc1ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efee1b74640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c0a90 0x7efedc0c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efee1b74640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efedc0c2ef0 0x7efedc0ab540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efee1b74640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc1ef970 con 0x7efedc0c0a90 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efee1b74640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc1f07d0 con 0x7efedc065ee0 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efee1b74640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc1efbc0 con 0x7efedc0c2ef0 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c0a90 0x7efedc0c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c0a90 0x7efedc0c1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50424/0 (socket says 172.21.15.39:50424) 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed3fff640 1 -- 172.21.15.39:0/1335095634 learned_addr learned my addr 172.21.15.39:0/1335095634 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efedbfff640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efedc0c2ef0 0x7efedc0ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed3fff640 1 -- 172.21.15.39:0/1335095634 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efedc0c2ef0 msgr2=0x7efedc0ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed3fff640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efedc0c2ef0 0x7efedc0ab540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed3fff640 1 -- 172.21.15.39:0/1335095634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc065ee0 msgr2=0x7efedc1ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed3fff640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc065ee0 0x7efedc1ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed3fff640 1 -- 172.21.15.39:0/1335095634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4009b30 con 0x7efedc0c0a90 2024-09-13T17:16:31.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed3fff640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c0a90 0x7efedc0c1f50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7efec400b580 tx=0x7efec400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:31.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed8ff9640 1 -- 172.21.15.39:0/1335095634 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4010870 con 0x7efedc0c0a90 2024-09-13T17:16:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed8ff9640 1 -- 172.21.15.39:0/1335095634 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec40109f0 con 0x7efedc0c0a90 2024-09-13T17:16:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc1efea0 con 0x7efedc0c0a90 2024-09-13T17:16:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc1f0460 con 0x7efedc0c0a90 2024-09-13T17:16:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed8ff9640 1 -- 172.21.15.39:0/1335095634 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4010da0 con 0x7efedc0c0a90 2024-09-13T17:16:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc0c5de0 con 0x7efedc0c0a90 2024-09-13T17:16:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed8ff9640 1 -- 172.21.15.39:0/1335095634 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efec400f3c0 con 0x7efedc0c0a90 2024-09-13T17:16:31.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed8ff9640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efebc0666e0 0x7efebc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:31.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efed8ff9640 1 -- 172.21.15.39:0/1335095634 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7efec40850f0 con 0x7efedc0c0a90 2024-09-13T17:16:31.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.367+0000 7efedb7fe640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efebc0666e0 0x7efebc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:31.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.371+0000 7efedb7fe640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efebc0666e0 0x7efebc068ba0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7efecc007fd0 tx=0x7efecc0079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:31.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.379+0000 7efed8ff9640 1 -- 172.21.15.39:0/1335095634 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efec4054f70 con 0x7efedc0c0a90 2024-09-13T17:16:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.687+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efedc0c48c0 con 0x7efedc0c0a90 2024-09-13T17:16:31.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.687+0000 7efed8ff9640 1 -- 172.21.15.39:0/1335095634 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5934 (secure 0 0 0) 0x7efec4054b00 con 0x7efedc0c0a90 2024-09-13T17:16:31.689 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:31.690 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:31.690 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 197 2024-09-13T17:16:31.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efebc0666e0 msgr2=0x7efebc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efebc0666e0 0x7efebc068ba0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7efecc007fd0 tx=0x7efecc0079a0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c0a90 msgr2=0x7efedc0c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c0a90 0x7efedc0c1f50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7efec400b580 tx=0x7efec400ba40 comp rx=0 tx=0).stop 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 shutdown_connections 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7efebc0666e0 0x7efebc068ba0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7efedc0c2ef0 0x7efedc0ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efedc0c0a90 0x7efedc0c1f50 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 --2- 172.21.15.39:0/1335095634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efedc065ee0 0x7efedc1ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 >> 172.21.15.39:0/1335095634 conn(0x7efedc0c3310 msgr2=0x7efedc14fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 shutdown_connections 2024-09-13T17:16:31.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:31.691+0000 7efee1b74640 1 -- 172.21.15.39:0/1335095634 wait complete. 2024-09-13T17:16:31.717 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:16:33.718 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12b18ab640 1 Processor -- start 2024-09-13T17:16:33.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12b18ab640 1 -- start start 2024-09-13T17:16:33.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12b18ab640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f12ac14f540 0x7f12ac0c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:33.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12aa7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f12ac0c1ab0 0x7f12ac06ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:36026/0 (socket says 172.21.15.39:36026) 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12aa7fc640 1 -- 172.21.15.39:0/3006340670 learned_addr learned my addr 172.21.15.39:0/3006340670 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12b18ab640 1 -- 172.21.15.39:0/3006340670 --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac14f920 con 0x7f12ac0c1ab0 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12b18ab640 1 -- 172.21.15.39:0/3006340670 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac0c1780 con 0x7f12ac155200 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12b18ab640 1 -- 172.21.15.39:0/3006340670 --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ac0709c0 con 0x7f12ac14f540 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12aaffd640 1 --2- 172.21.15.39:0/3006340670 >> v2:172.21.15.39:3300/0 conn(0x7f12ac14f540 0x7f12ac0c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255658718 0 0) 0x7f12ac14f920 con 0x7f12ac0c1ab0 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1294003530 con 0x7f12ac0c1ab0 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3179089542 0 0) 0x7f1294003530 con 0x7f12ac0c1ab0 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 >> v2:172.21.15.39:3300/0 conn(0x7f12ac14f540 msgr2=0x7f12ac0c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 --2- 172.21.15.39:0/3006340670 >> v2:172.21.15.39:3300/0 conn(0x7f12ac14f540 0x7f12ac0c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:33.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 >> v1:172.21.15.84:6789/0 conn(0x7f12ac155200 legacy=0x7f12ac155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac071580 con 0x7f12ac0c1ab0 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f129c001680 con 0x7f12ac0c1ab0 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 >> v1:172.21.15.39:6789/0 conn(0x7f12ac0c1ab0 legacy=0x7f12ac06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294003980 0x7f1294003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12940047d0 0x7f1294004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1294005190 0x7f129400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129400bfe0 con 0x7f1294003980 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129400c150 con 0x7f1294005190 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129400c310 con 0x7f12940047d0 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12aa7fc640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12940047d0 0x7f1294004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:33.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.863+0000 7f12aaffd640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294003980 0x7f1294003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:33.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.867+0000 7f12aa7fc640 1 -- 172.21.15.39:0/3006340670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1294005190 msgr2=0x7f129400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:33.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.867+0000 7f12aa7fc640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1294005190 0x7f129400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:33.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.867+0000 7f12aa7fc640 1 -- 172.21.15.39:0/3006340670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294003980 msgr2=0x7f1294003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:33.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.867+0000 7f12aa7fc640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294003980 0x7f1294003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:33.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.867+0000 7f12aa7fc640 1 -- 172.21.15.39:0/3006340670 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294003530 con 0x7f12940047d0 2024-09-13T17:16:33.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.867+0000 7f12ab7fe640 1 -- 172.21.15.39:0/3006340670 reap_dead start 2024-09-13T17:16:33.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.867+0000 7f12aaffd640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1294003980 0x7f1294003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:16:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.867+0000 7f12aa7fc640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12940047d0 0x7f1294004c50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f129c007fa0 tx=0x7f129c021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e56ccee90005f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.871+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0a9e30 con 0x7f12940047d0 2024-09-13T17:16:33.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.871+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c0058e0 con 0x7f12940047d0 2024-09-13T17:16:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.871+0000 7f12a9ffb640 1 -- 172.21.15.39:0/3006340670 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c005d10 con 0x7f12940047d0 2024-09-13T17:16:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.871+0000 7f12b18ab640 1 -- 172.21.15.39:0/3006340670 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12940047d0 msgr2=0x7f1294004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:33.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.871+0000 7f12b18ab640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12940047d0 0x7f1294004c50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f129c007fa0 tx=0x7f129c021040 comp rx=0 tx=0).stop 2024-09-13T17:16:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.875+0000 7f12b18ab640 1 -- 172.21.15.39:0/3006340670 shutdown_connections 2024-09-13T17:16:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.875+0000 7f12b18ab640 1 --2- 172.21.15.39:0/3006340670 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12940047d0 0x7f1294004c50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:33.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.875+0000 7f12b18ab640 1 -- 172.21.15.39:0/3006340670 >> 172.21.15.39:0/3006340670 conn(0x7f12ac0c32a0 msgr2=0x7f12ac0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:33.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.875+0000 7f12b18ab640 1 -- 172.21.15.39:0/3006340670 shutdown_connections 2024-09-13T17:16:33.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.875+0000 7f12b18ab640 1 -- 172.21.15.39:0/3006340670 wait complete. 2024-09-13T17:16:33.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 Processor -- start 2024-09-13T17:16:33.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 -- start start 2024-09-13T17:16:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12940047d0 0x7f12ac0a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12ac1534d0 0x7f12ac153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12ac0a77c0 0x7f12ac154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac0bdba0 con 0x7f12ac0a77c0 2024-09-13T17:16:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac153b30 con 0x7f12940047d0 2024-09-13T17:16:33.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac0bddf0 con 0x7f12ac1534d0 2024-09-13T17:16:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12ab7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12ac0a77c0 0x7f12ac154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12ab7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12ac0a77c0 0x7f12ac154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50438/0 (socket says 172.21.15.39:50438) 2024-09-13T17:16:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12ab7fe640 1 -- 172.21.15.39:0/1039900324 learned_addr learned my addr 172.21.15.39:0/1039900324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12aaffd640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12940047d0 0x7f12ac0a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12aa7fc640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12ac1534d0 0x7f12ac153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12ab7fe640 1 -- 172.21.15.39:0/1039900324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12ac1534d0 msgr2=0x7f12ac153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12ab7fe640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12ac1534d0 0x7f12ac153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12ab7fe640 1 -- 172.21.15.39:0/1039900324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12940047d0 msgr2=0x7f12ac0a7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12ab7fe640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12940047d0 0x7f12ac0a7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12ab7fe640 1 -- 172.21.15.39:0/1039900324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac1522c0 con 0x7f12ac0a77c0 2024-09-13T17:16:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12ab7fe640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12ac0a77c0 0x7f12ac154500 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f1294004750 tx=0x7f12a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f1293fff640 1 -- 172.21.15.39:0/1039900324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4007170 con 0x7f12ac0a77c0 2024-09-13T17:16:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f1293fff640 1 -- 172.21.15.39:0/1039900324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a40072d0 con 0x7f12ac0a77c0 2024-09-13T17:16:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac1525a0 con 0x7f12ac0a77c0 2024-09-13T17:16:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f1293fff640 1 -- 172.21.15.39:0/1039900324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4007680 con 0x7f12ac0a77c0 2024-09-13T17:16:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.879+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac1ebeb0 con 0x7f12ac0a77c0 2024-09-13T17:16:33.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.883+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ac0619c0 con 0x7f12ac0a77c0 2024-09-13T17:16:33.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.883+0000 7f1293fff640 1 -- 172.21.15.39:0/1039900324 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f12a40077e0 con 0x7f12ac0a77c0 2024-09-13T17:16:33.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.883+0000 7f1293fff640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1298066690 0x7f1298068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:33.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.883+0000 7f1293fff640 1 -- 172.21.15.39:0/1039900324 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 9905+0+0 (secure 0 0 0) 0x7f12a4084d70 con 0x7f12ac0a77c0 2024-09-13T17:16:33.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.883+0000 7f12aaffd640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1298066690 0x7f1298068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:33.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.883+0000 7f12aaffd640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1298066690 0x7f1298068b50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0c0db0 tx=0x7f12a0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:33.887+0000 7f1293fff640 1 -- 172.21.15.39:0/1039900324 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f12a4051110 con 0x7f12ac0a77c0 2024-09-13T17:16:34.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.191+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12ac067130 con 0x7f12ac0a77c0 2024-09-13T17:16:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.191+0000 7f1293fff640 1 -- 172.21.15.39:0/1039900324 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5934 (secure 0 0 0) 0x7f12a4054730 con 0x7f12ac0a77c0 2024-09-13T17:16:34.194 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:34.194 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:14:08.379706+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6530":{"gid":6530,"name":"d","rank":0,"incarnation":178,"state":"up:active","state_seq":27,"addr":"172.21.15.84:6835/380008541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":380008541},{"type":"v1","addr":"172.21.15.84:6835","nonce":380008541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:34.195 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 197 2024-09-13T17:16:34.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1298066690 msgr2=0x7f1298068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:34.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1298066690 0x7f1298068b50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0c0db0 tx=0x7f12a0009190 comp rx=0 tx=0).stop 2024-09-13T17:16:34.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12ac0a77c0 msgr2=0x7f12ac154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:34.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12ac0a77c0 0x7f12ac154500 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f1294004750 tx=0x7f12a400c8c0 comp rx=0 tx=0).stop 2024-09-13T17:16:34.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 shutdown_connections 2024-09-13T17:16:34.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1298066690 0x7f1298068b50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:34.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12ac0a77c0 0x7f12ac154500 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:34.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f12ac1534d0 0x7f12ac153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:34.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 --2- 172.21.15.39:0/1039900324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f12940047d0 0x7f12ac0a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:34.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 >> 172.21.15.39:0/1039900324 conn(0x7f12ac0c32a0 msgr2=0x7f12ac069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:34.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 shutdown_connections 2024-09-13T17:16:34.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:34.195+0000 7f12b18ab640 1 -- 172.21.15.39:0/1039900324 wait complete. 2024-09-13T17:16:34.221 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-09-13T17:16:35.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:16:35.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:16:36.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:36.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe947fda640 1 Processor -- start 2024-09-13T17:16:36.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe947fda640 1 -- start start 2024-09-13T17:16:36.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe947fda640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 0x7fe940151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe947fda640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe94006e5e0 0x7fe9400714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe947fda640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9400c7470 0x7fe9400c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe947fda640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94006f660 con 0x7fe9400c7470 2024-09-13T17:16:36.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe947fda640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94014f590 con 0x7fe94006e5e0 2024-09-13T17:16:36.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe947fda640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94006eae0 con 0x7fe94014f1b0 2024-09-13T17:16:36.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945575640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe94006e5e0 0x7fe9400714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945d76640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 0x7fe940151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945575640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe94006e5e0 0x7fe9400714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:52224/0 (socket says 172.21.15.39:52224) 2024-09-13T17:16:36.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945575640 1 -- 172.21.15.39:0/4114434227 learned_addr learned my addr 172.21.15.39:0/4114434227 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:36.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945575640 1 -- 172.21.15.39:0/4114434227 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 msgr2=0x7fe940151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:36.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945575640 1 --2- 172.21.15.39:0/4114434227 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 0x7fe940151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945575640 1 -- 172.21.15.39:0/4114434227 >> v2:172.21.15.39:3300/0 conn(0x7fe9400c7470 msgr2=0x7fe9400c5c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:36.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945575640 1 --2- 172.21.15.39:0/4114434227 >> v2:172.21.15.39:3300/0 conn(0x7fe9400c7470 0x7fe9400c5c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945575640 1 -- 172.21.15.39:0/4114434227 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94006edc0 con 0x7fe94006e5e0 2024-09-13T17:16:36.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.355+0000 7fe945d76640 1 --2- 172.21.15.39:0/4114434227 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 0x7fe940151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:16:36.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.359+0000 7fe945575640 1 --2- 172.21.15.39:0/4114434227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe94006e5e0 0x7fe9400714e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe94006d0e0 tx=0x7fe93001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f558f81f72b3a568 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:36.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.359+0000 7fe944d74640 1 -- 172.21.15.39:0/4114434227 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9400c69a0 con 0x7fe94006e5e0 2024-09-13T17:16:36.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.359+0000 7fe944d74640 1 -- 172.21.15.39:0/4114434227 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9300aa430 con 0x7fe94006e5e0 2024-09-13T17:16:36.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.359+0000 7fe944d74640 1 -- 172.21.15.39:0/4114434227 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9300aa7c0 con 0x7fe94006e5e0 2024-09-13T17:16:36.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.359+0000 7fe947fda640 1 -- 172.21.15.39:0/4114434227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe94006e5e0 msgr2=0x7fe9400714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:36.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.359+0000 7fe947fda640 1 --2- 172.21.15.39:0/4114434227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe94006e5e0 0x7fe9400714e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe94006d0e0 tx=0x7fe93001c510 comp rx=0 tx=0).stop 2024-09-13T17:16:36.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 -- 172.21.15.39:0/4114434227 shutdown_connections 2024-09-13T17:16:36.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 --2- 172.21.15.39:0/4114434227 >> v2:172.21.15.39:3300/0 conn(0x7fe9400c7470 0x7fe9400c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 --2- 172.21.15.39:0/4114434227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe94006e5e0 0x7fe9400714e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 --2- 172.21.15.39:0/4114434227 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 0x7fe940151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 -- 172.21.15.39:0/4114434227 >> 172.21.15.39:0/4114434227 conn(0x7fe9400a9720 msgr2=0x7fe9400a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:36.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 -- 172.21.15.39:0/4114434227 shutdown_connections 2024-09-13T17:16:36.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 -- 172.21.15.39:0/4114434227 wait complete. 2024-09-13T17:16:36.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 Processor -- start 2024-09-13T17:16:36.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 -- start start 2024-09-13T17:16:36.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe94006e5e0 0x7fe9400bfd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9400c7470 0x7fe94014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 0x7fe94014a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400ad6f0 con 0x7fe9400c7470 2024-09-13T17:16:36.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400c02a0 con 0x7fe94006e5e0 2024-09-13T17:16:36.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe947fda640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400ad940 con 0x7fe94014f1b0 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe945575640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9400c7470 0x7fe94014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe945575640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9400c7470 0x7fe94014de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50442/0 (socket says 172.21.15.39:50442) 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe945575640 1 -- 172.21.15.39:0/1546588073 learned_addr learned my addr 172.21.15.39:0/1546588073 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe945d76640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe94006e5e0 0x7fe9400bfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.363+0000 7fe946577640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 0x7fe94014a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe945575640 1 -- 172.21.15.39:0/1546588073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 msgr2=0x7fe94014a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe945575640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 0x7fe94014a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe945575640 1 -- 172.21.15.39:0/1546588073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe94006e5e0 msgr2=0x7fe9400bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe945575640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe94006e5e0 0x7fe9400bfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe945575640 1 -- 172.21.15.39:0/1546588073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c009420 con 0x7fe9400c7470 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe945575640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9400c7470 0x7fe94014de30 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe9300ac040 tx=0x7fe93001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:36.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe9367fc640 1 -- 172.21.15.39:0/1546588073 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9300b3600 con 0x7fe9400c7470 2024-09-13T17:16:36.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe9367fc640 1 -- 172.21.15.39:0/1546588073 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9300b3760 con 0x7fe9400c7470 2024-09-13T17:16:36.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9300094d0 con 0x7fe9400c7470 2024-09-13T17:16:36.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe9367fc640 1 -- 172.21.15.39:0/1546588073 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9300b3af0 con 0x7fe9400c7470 2024-09-13T17:16:36.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9400c0f10 con 0x7fe9400c7470 2024-09-13T17:16:36.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9400a9490 con 0x7fe9400c7470 2024-09-13T17:16:36.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe9367fc640 1 -- 172.21.15.39:0/1546588073 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe9300b3c50 con 0x7fe9400c7470 2024-09-13T17:16:36.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe9367fc640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe924066690 0x7fe924068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe9367fc640 1 -- 172.21.15.39:0/1546588073 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fe93011fc40 con 0x7fe9400c7470 2024-09-13T17:16:36.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe945d76640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe924066690 0x7fe924068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.367+0000 7fe945d76640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe924066690 0x7fe924068b50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009c50 tx=0x7fe93c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:36.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.375+0000 7fe9367fc640 1 -- 172.21.15.39:0/1546588073 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe9300ec100 con 0x7fe9400c7470 2024-09-13T17:16:36.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.659+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9400a9490 con 0x7fe9400c7470 2024-09-13T17:16:36.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.659+0000 7fe9367fc640 1 -- 172.21.15.39:0/1546588073 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 199 v199) v1 ==== 95+0+5181 (secure 0 0 0) 0x7fe9300ef720 con 0x7fe9400c7470 2024-09-13T17:16:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:36.662 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":199,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":199,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:36.164111+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:36.662 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 199 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe924066690 msgr2=0x7fe924068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe924066690 0x7fe924068b50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009c50 tx=0x7fe93c009190 comp rx=0 tx=0).stop 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9400c7470 msgr2=0x7fe94014de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9400c7470 0x7fe94014de30 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe9300ac040 tx=0x7fe93001ad50 comp rx=0 tx=0).stop 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 shutdown_connections 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe924066690 0x7fe924068b50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe94014f1b0 0x7fe94014a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9400c7470 0x7fe94014de30 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 --2- 172.21.15.39:0/1546588073 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe94006e5e0 0x7fe9400bfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 >> 172.21.15.39:0/1546588073 conn(0x7fe9400a9720 msgr2=0x7fe9400a9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 shutdown_connections 2024-09-13T17:16:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.663+0000 7fe947fda640 1 -- 172.21.15.39:0/1546588073 wait complete. 2024-09-13T17:16:36.684 INFO:tasks.mds_thrash.fs.[a]:mds.d down, removed from mdsmap 2024-09-13T17:16:36.684 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:16:36.685 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:36.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.819+0000 7f450951e640 1 Processor -- start 2024-09-13T17:16:36.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.819+0000 7f450951e640 1 -- start start 2024-09-13T17:16:36.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.823+0000 7f450951e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f45040c1810 0x7f450406e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.823+0000 7f450951e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45041576e0 0x7f4504157b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.823+0000 7f450951e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45040c15d0 con 0x7f450406b080 2024-09-13T17:16:36.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.823+0000 7f450951e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450406d0b0 con 0x7f45040c1810 2024-09-13T17:16:36.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.823+0000 7f450951e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040c1f00 con 0x7f45041576e0 2024-09-13T17:16:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.823+0000 7f45027fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f45040c1810 0x7f450406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.823+0000 7f45027fc640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f45040c1810 0x7f450406e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:52238/0 (socket says 172.21.15.39:52238) 2024-09-13T17:16:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.823+0000 7f45027fc640 1 -- 172.21.15.39:0/3224557417 learned_addr learned my addr 172.21.15.39:0/3224557417 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.827+0000 7f45037fe640 1 --2- 172.21.15.39:0/3224557417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45041576e0 0x7f4504157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.827+0000 7f45037fe640 1 -- 172.21.15.39:0/3224557417 >> v2:172.21.15.84:3300/0 conn(0x7f45040c1810 msgr2=0x7f450406e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.827+0000 7f45037fe640 1 --2- 172.21.15.39:0/3224557417 >> v2:172.21.15.84:3300/0 conn(0x7f45040c1810 0x7f450406e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.827+0000 7f45037fe640 1 -- 172.21.15.39:0/3224557417 >> v1:172.21.15.84:6789/0 conn(0x7f450406b080 legacy=0x7f4504151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.827+0000 7f45037fe640 1 -- 172.21.15.39:0/3224557417 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040c21a0 con 0x7f45041576e0 2024-09-13T17:16:36.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.831+0000 7f45037fe640 1 --2- 172.21.15.39:0/3224557417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45041576e0 0x7f4504157b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f450406be40 tx=0x7f44fc01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70244ee2fedeb1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:36.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f4501ffb640 1 -- 172.21.15.39:0/3224557417 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0a5870 con 0x7f45041576e0 2024-09-13T17:16:36.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f4501ffb640 1 -- 172.21.15.39:0/3224557417 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc0a59d0 con 0x7f45041576e0 2024-09-13T17:16:36.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f4501ffb640 1 -- 172.21.15.39:0/3224557417 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0a5da0 con 0x7f45041576e0 2024-09-13T17:16:36.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 -- 172.21.15.39:0/3224557417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45041576e0 msgr2=0x7f4504157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:36.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 --2- 172.21.15.39:0/3224557417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45041576e0 0x7f4504157b30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f450406be40 tx=0x7f44fc01c8e0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 -- 172.21.15.39:0/3224557417 shutdown_connections 2024-09-13T17:16:36.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 --2- 172.21.15.39:0/3224557417 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45041576e0 0x7f4504157b30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 --2- 172.21.15.39:0/3224557417 >> v2:172.21.15.84:3300/0 conn(0x7f45040c1810 0x7f450406e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 -- 172.21.15.39:0/3224557417 >> 172.21.15.39:0/3224557417 conn(0x7f45040c32a0 msgr2=0x7f45040c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:36.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 -- 172.21.15.39:0/3224557417 shutdown_connections 2024-09-13T17:16:36.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 -- 172.21.15.39:0/3224557417 wait complete. 2024-09-13T17:16:36.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 Processor -- start 2024-09-13T17:16:36.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.835+0000 7f450951e640 1 -- start start 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f450951e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f450406b080 0x7f45040bdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f450951e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45040c1810 0x7f4504065ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f450951e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45041576e0 0x7f450414c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f450951e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040bfe20 con 0x7f45041576e0 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f450951e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450414bd00 con 0x7f450406b080 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f450951e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450414e950 con 0x7f45040c1810 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f45027fc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45040c1810 0x7f4504065ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f45037fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45041576e0 0x7f450414c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f4502ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f450406b080 0x7f45040bdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f4502ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f450406b080 0x7f45040bdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:52250/0 (socket says 172.21.15.39:52250) 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f45037fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45041576e0 0x7f450414c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50448/0 (socket says 172.21.15.39:50448) 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f4502ffd640 1 -- 172.21.15.39:0/2117951089 learned_addr learned my addr 172.21.15.39:0/2117951089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f45037fe640 1 -- 172.21.15.39:0/2117951089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45040c1810 msgr2=0x7f4504065ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f45037fe640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45040c1810 0x7f4504065ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f45037fe640 1 -- 172.21.15.39:0/2117951089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f450406b080 msgr2=0x7f45040bdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:36.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f45037fe640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f450406b080 0x7f45040bdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f45037fe640 1 -- 172.21.15.39:0/2117951089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc009a70 con 0x7f45041576e0 2024-09-13T17:16:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f4502ffd640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f450406b080 0x7f45040bdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:16:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f45037fe640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45041576e0 0x7f450414c190 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0ac000 tx=0x7f44fc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f44eb7fe640 1 -- 172.21.15.39:0/2117951089 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0ad850 con 0x7f45041576e0 2024-09-13T17:16:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f44eb7fe640 1 -- 172.21.15.39:0/2117951089 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc0ad9b0 con 0x7f45041576e0 2024-09-13T17:16:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450414b9d0 con 0x7f45041576e0 2024-09-13T17:16:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f44eb7fe640 1 -- 172.21.15.39:0/2117951089 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0adda0 con 0x7f45041576e0 2024-09-13T17:16:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.839+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450414da40 con 0x7f45041576e0 2024-09-13T17:16:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.843+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45040619c0 con 0x7f45041576e0 2024-09-13T17:16:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.843+0000 7f44eb7fe640 1 -- 172.21.15.39:0/2117951089 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44fc0b4050 con 0x7f45041576e0 2024-09-13T17:16:36.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.843+0000 7f44eb7fe640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44f0066690 0x7f44f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:36.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.843+0000 7f44eb7fe640 1 -- 172.21.15.39:0/2117951089 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f44fc1263b0 con 0x7f45041576e0 2024-09-13T17:16:36.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.843+0000 7f4502ffd640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44f0066690 0x7f44f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:36.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.843+0000 7f4502ffd640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44f0066690 0x7f44f0068b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009da0 tx=0x7f44f8009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:36.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:36.855+0000 7f44eb7fe640 1 -- 172.21.15.39:0/2117951089 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f44fc0eb7d0 con 0x7f45041576e0 2024-09-13T17:16:37.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.131+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f450406e680 con 0x7f45041576e0 2024-09-13T17:16:37.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.131+0000 7f44eb7fe640 1 -- 172.21.15.39:0/2117951089 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 199 v199) v1 ==== 95+0+5181 (secure 0 0 0) 0x7f44fc0b2070 con 0x7f45041576e0 2024-09-13T17:16:37.132 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:37.132 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":199,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":199,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:36.164111+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:reconnect","state_seq":7,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:37.133 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 199 2024-09-13T17:16:37.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.131+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44f0066690 msgr2=0x7f44f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:37.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.131+0000 7f450951e640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44f0066690 0x7f44f0068b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009da0 tx=0x7f44f8009110 comp rx=0 tx=0).stop 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.131+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45041576e0 msgr2=0x7f450414c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.131+0000 7f450951e640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45041576e0 0x7f450414c190 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0ac000 tx=0x7f44fc019680 comp rx=0 tx=0).stop 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.135+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 shutdown_connections 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.135+0000 7f450951e640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f44f0066690 0x7f44f0068b50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.135+0000 7f450951e640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45041576e0 0x7f450414c190 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.135+0000 7f450951e640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f45040c1810 0x7f4504065ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.135+0000 7f450951e640 1 --2- 172.21.15.39:0/2117951089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f450406b080 0x7f45040bdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.135+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 >> 172.21.15.39:0/2117951089 conn(0x7f45040c32a0 msgr2=0x7f45040c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.135+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 shutdown_connections 2024-09-13T17:16:37.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:37.135+0000 7f450951e640 1 -- 172.21.15.39:0/2117951089 wait complete. 2024-09-13T17:16:37.157 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 199, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 199, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_8319": { "addr": "172.21.15.39:6835/980326904", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6834", "nonce": 980326904, "type": "v2" }, { "addr": "172.21.15.39:6835", "nonce": 980326904, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8319, "incarnation": 198, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 63, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:16:36.164111+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8319 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 196, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6638": { "addr": "172.21.15.39:6837/3255731814", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 3255731814, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 3255731814, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6638, "incarnation": 193, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 62, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:16:08.963924+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6638 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:16:39.025 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-13T17:16:39.023+0000 7fc6f2553640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:16:39.159 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00c20d640 1 Processor -- start 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00c20d640 1 -- start start 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00c20d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbffc0db7e0 0x7fbffc00ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00c20d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc0e8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00c20d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc03e240 con 0x7fbffc00b1b0 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00c20d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc03e500 con 0x7fbffc0db7e0 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00c20d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc030750 con 0x7fbffc0dec20 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00a7aa640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc0e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00a7aa640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc0e8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:48838/0 (socket says 172.21.15.39:48838) 2024-09-13T17:16:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.795+0000 7fc00a7aa640 1 -- 172.21.15.39:0/1944565302 learned_addr learned my addr 172.21.15.39:0/1944565302 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.803+0000 7fc00a7aa640 1 -- 172.21.15.39:0/1944565302 >> v2:172.21.15.39:3300/0 conn(0x7fbffc0db7e0 msgr2=0x7fbffc00ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.803+0000 7fc00a7aa640 1 --2- 172.21.15.39:0/1944565302 >> v2:172.21.15.39:3300/0 conn(0x7fbffc0db7e0 0x7fbffc00ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.803+0000 7fc00a7aa640 1 -- 172.21.15.39:0/1944565302 >> v1:172.21.15.84:6789/0 conn(0x7fbffc00b1b0 legacy=0x7fbffc0de510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:16:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.811+0000 7fc00a7aa640 1 -- 172.21.15.39:0/1944565302 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc030a30 con 0x7fbffc0dec20 2024-09-13T17:16:39.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.815+0000 7fc00a7aa640 1 --2- 172.21.15.39:0/1944565302 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc0e8940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc000009870 tx=0x7fc00001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13b71c51666c39ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:39.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.819+0000 7fc008fa7640 1 -- 172.21.15.39:0/1944565302 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000ab600 con 0x7fbffc0dec20 2024-09-13T17:16:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.819+0000 7fc008fa7640 1 -- 172.21.15.39:0/1944565302 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0000a7050 con 0x7fbffc0dec20 2024-09-13T17:16:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.819+0000 7fc00c20d640 1 -- 172.21.15.39:0/1944565302 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 msgr2=0x7fbffc0e8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.819+0000 7fc00c20d640 1 --2- 172.21.15.39:0/1944565302 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc0e8940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc000009870 tx=0x7fc00001c440 comp rx=0 tx=0).stop 2024-09-13T17:16:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.823+0000 7fc00c20d640 1 -- 172.21.15.39:0/1944565302 shutdown_connections 2024-09-13T17:16:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.823+0000 7fc00c20d640 1 --2- 172.21.15.39:0/1944565302 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc0e8940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.823+0000 7fc00c20d640 1 --2- 172.21.15.39:0/1944565302 >> v2:172.21.15.39:3300/0 conn(0x7fbffc0db7e0 0x7fbffc00ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.823+0000 7fc00c20d640 1 -- 172.21.15.39:0/1944565302 >> 172.21.15.39:0/1944565302 conn(0x7fbffc047cf0 msgr2=0x7fbffc048100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.823+0000 7fc00c20d640 1 -- 172.21.15.39:0/1944565302 shutdown_connections 2024-09-13T17:16:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.827+0000 7fc00c20d640 1 -- 172.21.15.39:0/1944565302 wait complete. 2024-09-13T17:16:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.827+0000 7fc00c20d640 1 Processor -- start 2024-09-13T17:16:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.827+0000 7fc00c20d640 1 -- start start 2024-09-13T17:16:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00c20d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc00b1b0 0x7fbffc0407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00c20d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbffc0db7e0 0x7fbffc040cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00c20d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc042340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00c20d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc044a80 con 0x7fbffc00b1b0 2024-09-13T17:16:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00c20d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0d0fb0 con 0x7fbffc0db7e0 2024-09-13T17:16:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc009fa9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc00b1b0 0x7fbffc0407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc009fa9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc00b1b0 0x7fbffc0407b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50454/0 (socket says 172.21.15.39:50454) 2024-09-13T17:16:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc009fa9640 1 -- 172.21.15.39:0/553524295 learned_addr learned my addr 172.21.15.39:0/553524295 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00c20d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc044cd0 con 0x7fbffc0dec20 2024-09-13T17:16:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00a7aa640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc042340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00a7aa640 1 -- 172.21.15.39:0/553524295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbffc0db7e0 msgr2=0x7fbffc040cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00a7aa640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbffc0db7e0 0x7fbffc040cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00a7aa640 1 -- 172.21.15.39:0/553524295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc00b1b0 msgr2=0x7fbffc0407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00a7aa640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc00b1b0 0x7fbffc0407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00a7aa640 1 -- 172.21.15.39:0/553524295 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000094d0 con 0x7fbffc0dec20 2024-09-13T17:16:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fc00a7aa640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc042340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc0000ab000 tx=0x7fc0000191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fbffa7fc640 1 -- 172.21.15.39:0/553524295 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000b58e0 con 0x7fbffc0dec20 2024-09-13T17:16:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fbffa7fc640 1 -- 172.21.15.39:0/553524295 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0000b5a80 con 0x7fbffc0dec20 2024-09-13T17:16:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.831+0000 7fbffa7fc640 1 -- 172.21.15.39:0/553524295 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000b5e30 con 0x7fbffc0dec20 2024-09-13T17:16:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.835+0000 7fc00c20d640 1 -- 172.21.15.39:0/553524295 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc0d0cd0 con 0x7fbffc0dec20 2024-09-13T17:16:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.835+0000 7fc00c20d640 1 -- 172.21.15.39:0/553524295 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc040290 con 0x7fbffc0dec20 2024-09-13T17:16:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.835+0000 7fbfe7fff640 1 -- 172.21.15.39:0/553524295 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc005d60 con 0x7fbffc0dec20 2024-09-13T17:16:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.839+0000 7fbffa7fc640 1 -- 172.21.15.39:0/553524295 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc0000b2050 con 0x7fbffc0dec20 2024-09-13T17:16:39.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.839+0000 7fbffa7fc640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbfe80666e0 0x7fbfe8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:39.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.839+0000 7fbffa7fc640 1 -- 172.21.15.39:0/553524295 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fc00011f570 con 0x7fbffc0dec20 2024-09-13T17:16:39.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.843+0000 7fc009fa9640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbfe80666e0 0x7fbfe8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:39.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.843+0000 7fc009fa9640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbfe80666e0 0x7fbfe8068ba0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009a50 tx=0x7fbff4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:39.855+0000 7fbffa7fc640 1 -- 172.21.15.39:0/553524295 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc0000eb880 con 0x7fbffc0dec20 2024-09-13T17:16:40.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.967+0000 7fbfe7fff640 1 -- 172.21.15.39:0/553524295 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbffc040480 con 0x7fbffc0dec20 2024-09-13T17:16:40.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.967+0000 7fbffa7fc640 1 -- 172.21.15.39:0/553524295 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 201 v201) v1 ==== 95+0+5178 (secure 0 0 0) 0x7fc0000eeea0 con 0x7fbffc0dec20 2024-09-13T17:16:40.971 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:40.971 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":201,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:40.971 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 201 2024-09-13T17:16:40.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 -- 172.21.15.39:0/553524295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbfe80666e0 msgr2=0x7fbfe8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:40.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbfe80666e0 0x7fbfe8068ba0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009a50 tx=0x7fbff4009210 comp rx=0 tx=0).stop 2024-09-13T17:16:40.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 -- 172.21.15.39:0/553524295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 msgr2=0x7fbffc042340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:40.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc042340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc0000ab000 tx=0x7fc0000191e0 comp rx=0 tx=0).stop 2024-09-13T17:16:40.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 -- 172.21.15.39:0/553524295 shutdown_connections 2024-09-13T17:16:40.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbfe80666e0 0x7fbfe8068ba0 secure :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009a50 tx=0x7fbff4009210 comp rx=0 tx=0).stop 2024-09-13T17:16:40.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbffc0db7e0 0x7fbffc040cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:40.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbffc0dec20 0x7fbffc042340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:40.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 --2- 172.21.15.39:0/553524295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbffc00b1b0 0x7fbffc0407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:40.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.979+0000 7fc00c20d640 1 -- 172.21.15.39:0/553524295 >> 172.21.15.39:0/553524295 conn(0x7fbffc047cf0 msgr2=0x7fbffc0dd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.991+0000 7fc00c20d640 1 -- 172.21.15.39:0/553524295 shutdown_connections 2024-09-13T17:16:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:40.991+0000 7fc00c20d640 1 -- 172.21.15.39:0/553524295 wait complete. 2024-09-13T17:16:41.103 INFO:tasks.mds_thrash.fs.[a]:mds.a has gained rank=0, replacing gid=6530 2024-09-13T17:16:41.103 INFO:tasks.mds_thrash.fs.[a]:waiting for 4 secs before reviving mds.d 2024-09-13T17:16:45.104 INFO:tasks.mds_thrash.fs.[a]:reviving mds.d 2024-09-13T17:16:45.104 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T17:16:45.105 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T17:16:45.106 INFO:tasks.ceph.mds.d:Started 2024-09-13T17:16:45.107 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:16:45.557 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T17:16:46.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.131+0000 7f598fab7640 1 Processor -- start 2024-09-13T17:16:46.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598fab7640 1 -- start start 2024-09-13T17:16:46.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598fab7640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5988153cc0 0x7f59880a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:46.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598fab7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59880a9130 0x7f59880c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:46.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598fab7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59880c5340 con 0x7f598806cc80 2024-09-13T17:16:46.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598fab7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59880c1da0 con 0x7f59880a9130 2024-09-13T17:16:46.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598fab7640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59880c5060 con 0x7f5988153cc0 2024-09-13T17:16:46.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598d853640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5988153cc0 0x7f59880a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598d853640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5988153cc0 0x7f59880a8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:47076/0 (socket says 172.21.15.39:47076) 2024-09-13T17:16:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598d853640 1 -- 172.21.15.39:0/1856838025 learned_addr learned my addr 172.21.15.39:0/1856838025 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598d853640 1 -- 172.21.15.39:0/1856838025 >> v2:172.21.15.39:3300/0 conn(0x7f59880a9130 msgr2=0x7f59880c3a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598d853640 1 --2- 172.21.15.39:0/1856838025 >> v2:172.21.15.39:3300/0 conn(0x7f59880a9130 0x7f59880c3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:46.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598d853640 1 -- 172.21.15.39:0/1856838025 >> v1:172.21.15.39:6789/0 conn(0x7f598806cc80 legacy=0x7f59880c3f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:16:46.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.135+0000 7f598d853640 1 -- 172.21.15.39:0/1856838025 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598806c1a0 con 0x7f5988153cc0 2024-09-13T17:16:46.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.139+0000 7f598d853640 1 --2- 172.21.15.39:0/1856838025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5988153cc0 0x7f59880a8bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5988074800 tx=0x7f598001ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a5dd76e4464e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:46.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.175+0000 7f598c851640 1 -- 172.21.15.39:0/1856838025 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800a7da0 con 0x7f5988153cc0 2024-09-13T17:16:46.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.175+0000 7f598c851640 1 -- 172.21.15.39:0/1856838025 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59800ab070 con 0x7f5988153cc0 2024-09-13T17:16:46.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.179+0000 7f598fab7640 1 -- 172.21.15.39:0/1856838025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5988153cc0 msgr2=0x7f59880a8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:46.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.179+0000 7f598fab7640 1 --2- 172.21.15.39:0/1856838025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5988153cc0 0x7f59880a8bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5988074800 tx=0x7f598001ecb0 comp rx=0 tx=0).stop 2024-09-13T17:16:46.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.179+0000 7f598fab7640 1 -- 172.21.15.39:0/1856838025 shutdown_connections 2024-09-13T17:16:46.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.179+0000 7f598fab7640 1 --2- 172.21.15.39:0/1856838025 >> v2:172.21.15.39:3300/0 conn(0x7f59880a9130 0x7f59880c3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:46.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.179+0000 7f598fab7640 1 --2- 172.21.15.39:0/1856838025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5988153cc0 0x7f59880a8bf0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.179+0000 7f598fab7640 1 -- 172.21.15.39:0/1856838025 >> 172.21.15.39:0/1856838025 conn(0x7f59880a72c0 msgr2=0x7f59880a76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 -- 172.21.15.39:0/1856838025 shutdown_connections 2024-09-13T17:16:46.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 -- 172.21.15.39:0/1856838025 wait complete. 2024-09-13T17:16:46.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 Processor -- start 2024-09-13T17:16:46.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 -- start start 2024-09-13T17:16:46.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f598806cc80 0x7f598814dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:46.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880a9130 0x7f598814eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:46.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f598814f430 0x7f59881f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:46.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59881f1420 con 0x7f598814f430 2024-09-13T17:16:46.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988145ca0 con 0x7f59880a9130 2024-09-13T17:16:46.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598fab7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59881f0e80 con 0x7f598806cc80 2024-09-13T17:16:46.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598d853640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f598806cc80 0x7f598814dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:46.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598d853640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f598806cc80 0x7f598814dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:35588/0 (socket says 172.21.15.39:35588) 2024-09-13T17:16:46.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.183+0000 7f598d853640 1 -- 172.21.15.39:0/2516309122 learned_addr learned my addr 172.21.15.39:0/2516309122 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:16:46.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.187+0000 7f598d052640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880a9130 0x7f598814eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:46.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.187+0000 7f598d853640 1 -- 172.21.15.39:0/2516309122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880a9130 msgr2=0x7f598814eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:46.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.187+0000 7f598d853640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880a9130 0x7f598814eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:46.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.187+0000 7f598d853640 1 -- 172.21.15.39:0/2516309122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f598814f430 msgr2=0x7f59881f3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:16:46.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.187+0000 7f598d853640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f598814f430 0x7f59881f3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:46.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.187+0000 7f598d853640 1 -- 172.21.15.39:0/2516309122 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598000d050 con 0x7f598806cc80 2024-09-13T17:16:46.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.187+0000 7f598d853640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f598806cc80 0x7f598814dbe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f59800b3040 tx=0x7f598001ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:46.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.207+0000 7f597e7fc640 1 -- 172.21.15.39:0/2516309122 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980007e30 con 0x7f598806cc80 2024-09-13T17:16:46.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.207+0000 7f597e7fc640 1 -- 172.21.15.39:0/2516309122 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59800ab050 con 0x7f598806cc80 2024-09-13T17:16:46.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.207+0000 7f597e7fc640 1 -- 172.21.15.39:0/2516309122 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59800b1c50 con 0x7f598806cc80 2024-09-13T17:16:46.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.207+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5988145eb0 con 0x7f598806cc80 2024-09-13T17:16:46.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.207+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598814e9d0 con 0x7f598806cc80 2024-09-13T17:16:46.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.211+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59880c1520 con 0x7f598806cc80 2024-09-13T17:16:46.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.215+0000 7f597e7fc640 1 -- 172.21.15.39:0/2516309122 <== mon.2 v2:172.21.15.84:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5980008040 con 0x7f598806cc80 2024-09-13T17:16:46.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.215+0000 7f597e7fc640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f596c0666e0 0x7f596c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:16:46.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.215+0000 7f597e7fc640 1 -- 172.21.15.39:0/2516309122 <== mon.2 v2:172.21.15.84:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f598011fe50 con 0x7f598806cc80 2024-09-13T17:16:46.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.215+0000 7f598d052640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f596c0666e0 0x7f596c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:16:46.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.215+0000 7f598d052640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f596c0666e0 0x7f596c068ba0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5988150720 tx=0x7f5984009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:16:46.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:46.255+0000 7f597e7fc640 1 -- 172.21.15.39:0/2516309122 <== mon.2 v2:172.21.15.84:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59800ec180 con 0x7f598806cc80 2024-09-13T17:16:47.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.882+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f598814e830 con 0x7f598806cc80 2024-09-13T17:16:47.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.882+0000 7f597e7fc640 1 -- 172.21.15.39:0/2516309122 <== mon.2 v2:172.21.15.84:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f598814e830 con 0x7f598806cc80 2024-09-13T17:16:47.884 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:16:47.884 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:16:47.885 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-13T17:16:47.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.918+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f596c0666e0 msgr2=0x7f596c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:47.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.918+0000 7f598fab7640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f596c0666e0 0x7f596c068ba0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5988150720 tx=0x7f5984009060 comp rx=0 tx=0).stop 2024-09-13T17:16:47.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.918+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f598806cc80 msgr2=0x7f598814dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:16:47.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.918+0000 7f598fab7640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f598806cc80 0x7f598814dbe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f59800b3040 tx=0x7f598001ba50 comp rx=0 tx=0).stop 2024-09-13T17:16:47.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.954+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 shutdown_connections 2024-09-13T17:16:47.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.954+0000 7f598fab7640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f596c0666e0 0x7f596c068ba0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:47.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.954+0000 7f598fab7640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f59880a9130 0x7f598814eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:47.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.954+0000 7f598fab7640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f598814f430 0x7f59881f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:47.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.954+0000 7f598fab7640 1 --2- 172.21.15.39:0/2516309122 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f598806cc80 0x7f598814dbe0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:16:47.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.954+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 >> 172.21.15.39:0/2516309122 conn(0x7f59880a72c0 msgr2=0x7f5988151360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:16:47.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.954+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 shutdown_connections 2024-09-13T17:16:47.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:16:47.954+0000 7f598fab7640 1 -- 172.21.15.39:0/2516309122 wait complete. 2024-09-13T17:16:48.136 INFO:tasks.mds_thrash.fs.[a]:mds.d reported in up:standby state 2024-09-13T17:16:48.136 INFO:tasks.mds_thrash.fs.[a]:waiting for 40 secs before thrashing 2024-09-13T17:17:06.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:17:06.132 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:17:06.924 INFO:tasks.workunit.client.1.smithi084.stdout:Syncing()...2 sec 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout:FFSB benchmark finished at: Fri Sep 13 17:17:06 2024 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout:Results: 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout:Benchmark took 307.41 sec 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout:Total Results 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout:=============== 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout: ======= ============ ========= ======= =========== ========== 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout: write : 1338880 4355.30 100.000% 100.000% 17MB/sec 2024-09-13T17:17:06.925 INFO:tasks.workunit.client.1.smithi084.stdout:- 2024-09-13T17:17:06.926 INFO:tasks.workunit.client.1.smithi084.stdout:4355.30 Transactions per Second 2024-09-13T17:17:06.926 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.926 INFO:tasks.workunit.client.1.smithi084.stdout:Throughput Results 2024-09-13T17:17:06.926 INFO:tasks.workunit.client.1.smithi084.stdout:=================== 2024-09-13T17:17:06.926 INFO:tasks.workunit.client.1.smithi084.stdout:Write Throughput: 17MB/sec 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout:System Call Latency statistics in millisecs 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout:===== 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout: Min Avg Max Total Calls 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout: ======== ======== ======== ============ 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout:[ open] 0.432000 2032.420822 21179.187500 1046 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-13T17:17:06.930 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[5] 100.000000 - 200.000000 : 3 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[6] 50.000000 - 100.000000 : 44 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[7] 20.000000 - 50.000000 : 342 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[8] 10.000000 - 20.000000 : 284 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[9] 5.000000 - 10.000000 : 125 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[10] 2.000000 - 5.000000 : 66 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[11] 1.000000 - 2.000000 : 39 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[12] 0.500000 - 1.000000 : 13 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[13] 0.200000 - 0.500000 : 2 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T17:17:06.931 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout:[ write] 0.086000 5.650996 258.040985 1338880 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[4] 200.000000 - 500.000000 : 32 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[5] 100.000000 - 200.000000 : 439 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[6] 50.000000 - 100.000000 : 9316 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[7] 20.000000 - 50.000000 : 77207 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[8] 10.000000 - 20.000000 : 141873 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[9] 5.000000 - 10.000000 : 211761 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[10] 2.000000 - 5.000000 : 245035 2024-09-13T17:17:06.932 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[11] 1.000000 - 2.000000 : 104935 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[12] 0.500000 - 1.000000 : 72626 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[13] 0.200000 - 0.500000 : 257256 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[14] 0.100000 - 0.200000 : 212058 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[15] 0.050000 - 0.100000 : 6342 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout:[ lseek] 0.000000 0.001027 4.412000 1338880 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T17:17:06.933 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[10] 2.000000 - 5.000000 : 6 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[11] 1.000000 - 2.000000 : 7 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[12] 0.500000 - 1.000000 : 8 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[13] 0.200000 - 0.500000 : 37 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[14] 0.100000 - 0.200000 : 49 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[15] 0.050000 - 0.100000 : 43 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[16] 0.020000 - 0.050000 : 313 2024-09-13T17:17:06.934 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[17] 0.010000 - 0.020000 : 826 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[18] 0.000000 - 0.010000 : 1337591 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout:[ close] 0.019000 2.345762 68.063004 1046 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[6] 50.000000 - 100.000000 : 1 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[7] 20.000000 - 50.000000 : 27 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[8] 10.000000 - 20.000000 : 36 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[9] 5.000000 - 10.000000 : 66 2024-09-13T17:17:06.935 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[10] 2.000000 - 5.000000 : 98 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[11] 1.000000 - 2.000000 : 79 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[12] 0.500000 - 1.000000 : 63 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[13] 0.200000 - 0.500000 : 73 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[14] 0.100000 - 0.200000 : 172 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[15] 0.050000 - 0.100000 : 100 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[16] 0.020000 - 0.050000 : 329 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[17] 0.010000 - 0.020000 : 2 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout: 2024-09-13T17:17:06.936 INFO:tasks.workunit.client.1.smithi084.stdout:0.3% User Time 2024-09-13T17:17:06.937 INFO:tasks.workunit.client.1.smithi084.stdout:54.2% System Time 2024-09-13T17:17:06.937 INFO:tasks.workunit.client.1.smithi084.stdout:54.5% CPU Utilization 2024-09-13T17:17:11.696 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-13T17:17:11.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-09-13T17:17:11.744 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.1... 2024-09-13T17:17:11.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-09-13T17:17:12.206 DEBUG:teuthology.parallel:result is None 2024-09-13T17:17:15.231 INFO:tasks.workunit.client.0.smithi039.stdout:Syncing()...5 sec 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout:FFSB benchmark finished at: Fri Sep 13 17:17:15 2024 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout:Results: 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout:Benchmark took 308.87 sec 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout:Total Results 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout:=============== 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout: ======= ============ ========= ======= =========== ========== 2024-09-13T17:17:15.232 INFO:tasks.workunit.client.0.smithi039.stdout: write : 1488640 4819.64 100.000% 100.000% 18.8MB/sec 2024-09-13T17:17:15.233 INFO:tasks.workunit.client.0.smithi039.stdout:- 2024-09-13T17:17:15.233 INFO:tasks.workunit.client.0.smithi039.stdout:4819.64 Transactions per Second 2024-09-13T17:17:15.233 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.233 INFO:tasks.workunit.client.0.smithi039.stdout:Throughput Results 2024-09-13T17:17:15.233 INFO:tasks.workunit.client.0.smithi039.stdout:=================== 2024-09-13T17:17:15.233 INFO:tasks.workunit.client.0.smithi039.stdout:Write Throughput: 18.8MB/sec 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout:System Call Latency statistics in millisecs 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout:===== 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout: Min Avg Max Total Calls 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout: ======== ======== ======== ============ 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout:[ open] 0.489000 906.318433 21498.064453 1163 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T17:17:15.240 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[5] 100.000000 - 200.000000 : 21 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[6] 50.000000 - 100.000000 : 42 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[7] 20.000000 - 50.000000 : 344 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[8] 10.000000 - 20.000000 : 422 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[9] 5.000000 - 10.000000 : 155 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[10] 2.000000 - 5.000000 : 81 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[11] 1.000000 - 2.000000 : 29 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[12] 0.500000 - 1.000000 : 6 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[13] 0.200000 - 0.500000 : 1 2024-09-13T17:17:15.241 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout:[ write] 0.088000 5.779916 19176.740234 1488640 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[4] 200.000000 - 500.000000 : 1 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[5] 100.000000 - 200.000000 : 306 2024-09-13T17:17:15.242 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[6] 50.000000 - 100.000000 : 9298 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[7] 20.000000 - 50.000000 : 82883 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[8] 10.000000 - 20.000000 : 156515 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[9] 5.000000 - 10.000000 : 261048 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[10] 2.000000 - 5.000000 : 331765 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[11] 1.000000 - 2.000000 : 138263 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[12] 0.500000 - 1.000000 : 69532 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[13] 0.200000 - 0.500000 : 243810 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[14] 0.100000 - 0.200000 : 194462 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[15] 0.050000 - 0.100000 : 755 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout:[ lseek] 0.000000 0.001047 7.427000 1488640 2024-09-13T17:17:15.243 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[9] 5.000000 - 10.000000 : 2 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[10] 2.000000 - 5.000000 : 5 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[11] 1.000000 - 2.000000 : 4 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[12] 0.500000 - 1.000000 : 10 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[13] 0.200000 - 0.500000 : 14 2024-09-13T17:17:15.244 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[14] 0.100000 - 0.200000 : 37 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[15] 0.050000 - 0.100000 : 39 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[16] 0.020000 - 0.050000 : 350 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[17] 0.010000 - 0.020000 : 832 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[18] 0.000000 - 0.010000 : 1487347 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout:[ close] 0.020000 2.674328 64.897003 1163 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[6] 50.000000 - 100.000000 : 2 2024-09-13T17:17:15.245 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[7] 20.000000 - 50.000000 : 36 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[8] 10.000000 - 20.000000 : 49 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[9] 5.000000 - 10.000000 : 78 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[10] 2.000000 - 5.000000 : 145 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[11] 1.000000 - 2.000000 : 116 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[12] 0.500000 - 1.000000 : 66 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[13] 0.200000 - 0.500000 : 76 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[14] 0.100000 - 0.200000 : 146 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[15] 0.050000 - 0.100000 : 136 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[16] 0.020000 - 0.050000 : 313 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.246 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.247 INFO:tasks.workunit.client.0.smithi039.stdout: 2024-09-13T17:17:15.247 INFO:tasks.workunit.client.0.smithi039.stdout:0.4% User Time 2024-09-13T17:17:15.247 INFO:tasks.workunit.client.0.smithi039.stdout:59.8% System Time 2024-09-13T17:17:15.247 INFO:tasks.workunit.client.0.smithi039.stdout:60.2% CPU Utilization 2024-09-13T17:17:19.205 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-13T17:17:19.206 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-13T17:17:19.262 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-13T17:17:19.262 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-13T17:17:19.751 DEBUG:teuthology.parallel:result is None 2024-09-13T17:17:19.752 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T17:17:19.780 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T17:17:19.781 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-09-13T17:17:19.805 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-09-13T17:17:19.805 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2024-09-13T17:17:19.814 INFO:tasks.mds_thrash:joining mds_thrasher 2024-09-13T17:17:19.814 INFO:tasks.mds_thrash.fs.[b]:stat['max_mds'] = 0 2024-09-13T17:17:19.814 INFO:tasks.mds_thrash.fs.[b]:stat['deactivate'] = 0 2024-09-13T17:17:19.814 INFO:tasks.mds_thrash.fs.[b]:stat['kill'] = 21 2024-09-13T17:17:19.815 INFO:tasks.mds_thrash:done joining 2024-09-13T17:17:19.815 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T17:17:19.824 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T17:17:19.825 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T17:17:19.825 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:17:19.836 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T17:17:19.836 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:17:19.889 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:17:20.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53eb04640 1 Processor -- start 2024-09-13T17:17:20.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53eb04640 1 -- start start 2024-09-13T17:17:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53eb04640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380c4260 0x7fb5380c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53eb04640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5380bcfb0 0x7fb538070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53eb04640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb538157360 con 0x7fb5380c77e0 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53eb04640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5380c80b0 con 0x7fb5380bcfb0 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53eb04640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5381575b0 con 0x7fb5380c4260 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53c8a0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380c4260 0x7fb5380c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb537fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5380bcfb0 0x7fb538070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53c8a0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380c4260 0x7fb5380c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:34862/0 (socket says 172.21.15.39:34862) 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53c8a0640 1 -- 172.21.15.39:0/1542210362 learned_addr learned my addr 172.21.15.39:0/1542210362 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53c8a0640 1 -- 172.21.15.39:0/1542210362 >> v2:172.21.15.84:3300/0 conn(0x7fb5380bcfb0 msgr2=0x7fb538070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53c8a0640 1 --2- 172.21.15.39:0/1542210362 >> v2:172.21.15.84:3300/0 conn(0x7fb5380bcfb0 0x7fb538070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53c8a0640 1 -- 172.21.15.39:0/1542210362 >> v1:172.21.15.84:6789/0 conn(0x7fb5380c77e0 legacy=0x7fb538153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53c8a0640 1 -- 172.21.15.39:0/1542210362 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb538157890 con 0x7fb5380c4260 2024-09-13T17:17:20.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb537fff640 1 --2- 172.21.15.39:0/1542210362 >> v2:172.21.15.84:3300/0 conn(0x7fb5380bcfb0 0x7fb538070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:17:20.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.102+0000 7fb53c8a0640 1 --2- 172.21.15.39:0/1542210362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380c4260 0x7fb5380c4640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb528009e50 tx=0x7fb52801c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a50935596b11cec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:20.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb5377fe640 1 -- 172.21.15.39:0/1542210362 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538154d10 con 0x7fb5380c4260 2024-09-13T17:17:20.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb5377fe640 1 -- 172.21.15.39:0/1542210362 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5280a55d0 con 0x7fb5380c4260 2024-09-13T17:17:20.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb5377fe640 1 -- 172.21.15.39:0/1542210362 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5280a5960 con 0x7fb5380c4260 2024-09-13T17:17:20.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- 172.21.15.39:0/1542210362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380c4260 msgr2=0x7fb5380c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 --2- 172.21.15.39:0/1542210362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380c4260 0x7fb5380c4640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb528009e50 tx=0x7fb52801c910 comp rx=0 tx=0).stop 2024-09-13T17:17:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- 172.21.15.39:0/1542210362 shutdown_connections 2024-09-13T17:17:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 --2- 172.21.15.39:0/1542210362 >> v2:172.21.15.84:3300/0 conn(0x7fb5380bcfb0 0x7fb538070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 --2- 172.21.15.39:0/1542210362 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380c4260 0x7fb5380c4640 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- 172.21.15.39:0/1542210362 >> 172.21.15.39:0/1542210362 conn(0x7fb538069e10 msgr2=0x7fb53806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- 172.21.15.39:0/1542210362 shutdown_connections 2024-09-13T17:17:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- 172.21.15.39:0/1542210362 wait complete. 2024-09-13T17:17:20.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 Processor -- start 2024-09-13T17:17:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- start start 2024-09-13T17:17:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380bcfb0 0x7fb5381f0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5380c4260 0x7fb5380c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5380c77e0 0x7fb5380c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5380c1a90 con 0x7fb5380c4260 2024-09-13T17:17:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5381f0e70 con 0x7fb5380c77e0 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5381f0b90 con 0x7fb5380bcfb0 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb537fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5380c4260 0x7fb5380c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53c8a0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380bcfb0 0x7fb5381f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53d0a1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5380c77e0 0x7fb5380c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb537fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5380c4260 0x7fb5380c1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42396/0 (socket says 172.21.15.39:42396) 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb537fff640 1 -- 172.21.15.39:0/2014118254 learned_addr learned my addr 172.21.15.39:0/2014118254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb537fff640 1 -- 172.21.15.39:0/2014118254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380bcfb0 msgr2=0x7fb5381f0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb537fff640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380bcfb0 0x7fb5381f0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb537fff640 1 -- 172.21.15.39:0/2014118254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5380c77e0 msgr2=0x7fb5380c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb537fff640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5380c77e0 0x7fb5380c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb537fff640 1 -- 172.21.15.39:0/2014118254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520009420 con 0x7fb5380c4260 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb537fff640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5380c4260 0x7fb5380c1e40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e960 tx=0x7fb52000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb5357fa640 1 -- 172.21.15.39:0/2014118254 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000ca00 con 0x7fb5380c4260 2024-09-13T17:17:20.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb5357fa640 1 -- 172.21.15.39:0/2014118254 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52000cb60 con 0x7fb5380c4260 2024-09-13T17:17:20.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb528009b30 con 0x7fb5380c4260 2024-09-13T17:17:20.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb5357fa640 1 -- 172.21.15.39:0/2014118254 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520013470 con 0x7fb5380c4260 2024-09-13T17:17:20.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.106+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5380bf200 con 0x7fb5380c4260 2024-09-13T17:17:20.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.110+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb538070430 con 0x7fb5380c4260 2024-09-13T17:17:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.110+0000 7fb5357fa640 1 -- 172.21.15.39:0/2014118254 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5200135d0 con 0x7fb5380c4260 2024-09-13T17:17:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.110+0000 7fb5357fa640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb518066690 0x7fb518068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.110+0000 7fb5357fa640 1 -- 172.21.15.39:0/2014118254 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fb520085470 con 0x7fb5380c4260 2024-09-13T17:17:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.110+0000 7fb53c8a0640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb518066690 0x7fb518068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.110+0000 7fb53c8a0640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb518066690 0x7fb518068b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb5280ae000 tx=0x7fb5280a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:20.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.118+0000 7fb5357fa640 1 -- 172.21.15.39:0/2014118254 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb520051810 con 0x7fb5380c4260 2024-09-13T17:17:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.470+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5380c6e70 con 0x7fb5380c4260 2024-09-13T17:17:20.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.470+0000 7fb5357fa640 1 -- 172.21.15.39:0/2014118254 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+25425 (secure 0 0 0) 0x7fb520017090 con 0x7fb5380c4260 2024-09-13T17:17:20.475 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:20.476 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":63,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T17:16:34.950618+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6835/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6834/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.39:6834/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6837/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6835/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.84:6834/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.39:6836/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.39:6835/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.84:6835/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.39:6835/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.84:6834/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.39:6834/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.39:6834/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.39:6836/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.84:6832/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.84:6835/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.84:6832/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6836/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.84:6834/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.39:6835/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.84:6832/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6834/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6835/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.84:6834/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6835/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.39:6835/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6836/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6833/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.39:6835/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.39:6837/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6834/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6835/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6834/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.39:6835/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1750130579":"2024-09-14T17:09:29.828673+0000","172.21.15.39:6834/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.84:6832/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.39:6834/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.39:6837/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6836/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6835/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6834/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.84:6835/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.84:6833/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.84:6833/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6835/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.39:6834/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.84:6832/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6833/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6834/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6835/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6836/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6834/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6833/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6835/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6834/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.39:6834/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6835/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6836/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.39:6835/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.84:6833/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6836/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6837/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.84:6834/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.84:6834/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.84:6835/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.39:6837/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.39:6837/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6835/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.84:6832/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6835/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/1750130579":"2024-09-14T17:09:29.828673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:17:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb518066690 msgr2=0x7fb518068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb518066690 0x7fb518068b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb5280ae000 tx=0x7fb5280a7070 comp rx=0 tx=0).stop 2024-09-13T17:17:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5380c4260 msgr2=0x7fb5380c1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5380c4260 0x7fb5380c1e40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e960 tx=0x7fb52000ee20 comp rx=0 tx=0).stop 2024-09-13T17:17:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 shutdown_connections 2024-09-13T17:17:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb518066690 0x7fb518068b50 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5380c77e0 0x7fb5380c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5380c4260 0x7fb5380c1e40 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 --2- 172.21.15.39:0/2014118254 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5380bcfb0 0x7fb5381f0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 >> 172.21.15.39:0/2014118254 conn(0x7fb538069e10 msgr2=0x7fb5381500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 shutdown_connections 2024-09-13T17:17:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.478+0000 7fb53eb04640 1 -- 172.21.15.39:0/2014118254 wait complete. 2024-09-13T17:17:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:17:20.513 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:17:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffe5de640 1 Processor -- start 2024-09-13T17:17:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffe5de640 1 -- start start 2024-09-13T17:17:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffe5de640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2ff80c42b0 0x7f2ff80c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffe5de640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2ff80c7110 0x7f2ff806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffe5de640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff8155d80 con 0x7f2ff80c6590 2024-09-13T17:17:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffe5de640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff80c2290 con 0x7f2ff80c7110 2024-09-13T17:17:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffe5de640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80c6430 con 0x7f2ff80c42b0 2024-09-13T17:17:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffcb7b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2ff80c7110 0x7f2ff806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffcb7b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2ff80c7110 0x7f2ff806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54228/0 (socket says 172.21.15.39:54228) 2024-09-13T17:17:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ff7fff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2ff80c42b0 0x7f2ff80c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffcb7b640 1 -- 172.21.15.39:0/712067379 learned_addr learned my addr 172.21.15.39:0/712067379 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffcb7b640 1 -- 172.21.15.39:0/712067379 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2ff80c42b0 msgr2=0x7f2ff80c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffcb7b640 1 --2- 172.21.15.39:0/712067379 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2ff80c42b0 0x7f2ff80c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffcb7b640 1 -- 172.21.15.39:0/712067379 >> v1:172.21.15.84:6789/0 conn(0x7f2ff80c6590 legacy=0x7f2ff80c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.706+0000 7f2ffcb7b640 1 -- 172.21.15.39:0/712067379 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80c2ac0 con 0x7f2ff80c7110 2024-09-13T17:17:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffcb7b640 1 --2- 172.21.15.39:0/712067379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ff80c7110 0x7f2ff806e720 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009e50 tx=0x7f2ff001c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3143239d98d9bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:20.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ff77fe640 1 -- 172.21.15.39:0/712067379 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00a56e0 con 0x7f2ff80c7110 2024-09-13T17:17:20.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ff77fe640 1 -- 172.21.15.39:0/712067379 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff00a5840 con 0x7f2ff80c7110 2024-09-13T17:17:20.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ff77fe640 1 -- 172.21.15.39:0/712067379 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00a5bd0 con 0x7f2ff80c7110 2024-09-13T17:17:20.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 -- 172.21.15.39:0/712067379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ff80c7110 msgr2=0x7f2ff806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 --2- 172.21.15.39:0/712067379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ff80c7110 0x7f2ff806e720 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009e50 tx=0x7f2ff001c9a0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 -- 172.21.15.39:0/712067379 shutdown_connections 2024-09-13T17:17:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 --2- 172.21.15.39:0/712067379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ff80c7110 0x7f2ff806e720 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 --2- 172.21.15.39:0/712067379 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2ff80c42b0 0x7f2ff80c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 -- 172.21.15.39:0/712067379 >> 172.21.15.39:0/712067379 conn(0x7f2ff80c7820 msgr2=0x7f2ff80c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 -- 172.21.15.39:0/712067379 shutdown_connections 2024-09-13T17:17:20.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 -- 172.21.15.39:0/712067379 wait complete. 2024-09-13T17:17:20.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 Processor -- start 2024-09-13T17:17:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 -- start start 2024-09-13T17:17:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ff80c42b0 0x7f2ff8152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff80c6590 0x7f2ff8155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2ff80c7110 0x7f2ff814ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80a94b0 con 0x7f2ff80c6590 2024-09-13T17:17:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff81516e0 con 0x7f2ff80c42b0 2024-09-13T17:17:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.710+0000 7f2ffe5de640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff81f1120 con 0x7f2ff80c7110 2024-09-13T17:17:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.714+0000 7f2feffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff80c6590 0x7f2ff8155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.714+0000 7f2feffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff80c6590 0x7f2ff8155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42406/0 (socket says 172.21.15.39:42406) 2024-09-13T17:17:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.714+0000 7f2feffff640 1 -- 172.21.15.39:0/1622194825 learned_addr learned my addr 172.21.15.39:0/1622194825 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.714+0000 7f2feffff640 1 -- 172.21.15.39:0/1622194825 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2ff80c7110 msgr2=0x7f2ff814ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.714+0000 7f2feffff640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2ff80c7110 0x7f2ff814ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.714+0000 7f2feffff640 1 -- 172.21.15.39:0/1622194825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ff80c42b0 msgr2=0x7f2ff8152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.714+0000 7f2feffff640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ff80c42b0 0x7f2ff8152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.714+0000 7f2feffff640 1 -- 172.21.15.39:0/1622194825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0009b30 con 0x7f2ff80c6590 2024-09-13T17:17:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.714+0000 7f2feffff640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff80c6590 0x7f2ff8155270 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c3c0 tx=0x7f2fe000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.718+0000 7f2ff57fa640 1 -- 172.21.15.39:0/1622194825 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00052d0 con 0x7f2ff80c6590 2024-09-13T17:17:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.718+0000 7f2ffe5de640 1 -- 172.21.15.39:0/1622194825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff814bf90 con 0x7f2ff80c6590 2024-09-13T17:17:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.718+0000 7f2ffe5de640 1 -- 172.21.15.39:0/1622194825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff8154820 con 0x7f2ff80c6590 2024-09-13T17:17:20.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.718+0000 7f2ff57fa640 1 -- 172.21.15.39:0/1622194825 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe0005430 con 0x7f2ff80c6590 2024-09-13T17:17:20.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.718+0000 7f2ff57fa640 1 -- 172.21.15.39:0/1622194825 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0003840 con 0x7f2ff80c6590 2024-09-13T17:17:20.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.722+0000 7f2ff57fa640 1 -- 172.21.15.39:0/1622194825 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2fe0003a60 con 0x7f2ff80c6590 2024-09-13T17:17:20.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.722+0000 7f2ff57fa640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fd80666e0 0x7f2fd8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:20.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.722+0000 7f2ff57fa640 1 -- 172.21.15.39:0/1622194825 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f2fe0084690 con 0x7f2ff80c6590 2024-09-13T17:17:20.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.722+0000 7f2ff7fff640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fd80666e0 0x7f2fd8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:20.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.722+0000 7f2ffe5de640 1 -- 172.21.15.39:0/1622194825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc005260 con 0x7f2ff80c6590 2024-09-13T17:17:20.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.722+0000 7f2ff7fff640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fd80666e0 0x7f2fd8068ba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8006aa0 tx=0x7f2fe8006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:20.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:20.738+0000 7f2ff57fa640 1 -- 172.21.15.39:0/1622194825 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2fe0050a30 con 0x7f2ff80c6590 2024-09-13T17:17:21.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.106+0000 7f2ffe5de640 1 -- 172.21.15.39:0/1622194825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2fdc005a90 con 0x7f2ff80c6590 2024-09-13T17:17:21.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.110+0000 7f2ff57fa640 1 -- 172.21.15.39:0/1622194825 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2fe0017020 con 0x7f2ff80c6590 2024-09-13T17:17:21.112 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T17:17:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 -- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fd80666e0 msgr2=0x7f2fd8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fd80666e0 0x7f2fd8068ba0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8006aa0 tx=0x7f2fe8006a30 comp rx=0 tx=0).stop 2024-09-13T17:17:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 -- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff80c6590 msgr2=0x7f2ff8155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff80c6590 0x7f2ff8155270 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c3c0 tx=0x7f2fe000c880 comp rx=0 tx=0).stop 2024-09-13T17:17:21.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 -- 172.21.15.39:0/1622194825 shutdown_connections 2024-09-13T17:17:21.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2fd80666e0 0x7f2fd8068ba0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f2ff80c7110 0x7f2ff814ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff80c6590 0x7f2ff8155270 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 --2- 172.21.15.39:0/1622194825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ff80c42b0 0x7f2ff8152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 -- 172.21.15.39:0/1622194825 >> 172.21.15.39:0/1622194825 conn(0x7f2ff80c7820 msgr2=0x7f2ff806e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:21.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 -- 172.21.15.39:0/1622194825 shutdown_connections 2024-09-13T17:17:21.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.114+0000 7f2fee7fc640 1 -- 172.21.15.39:0/1622194825 wait complete. 2024-09-13T17:17:21.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T17:17:21.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.326+0000 7fb1dbcd2640 1 Processor -- start 2024-09-13T17:17:21.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.326+0000 7fb1dbcd2640 1 -- start start 2024-09-13T17:17:21.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1dbcd2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb1d40bf5e0 0x7fb1d4154a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1dbcd2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1d40bff20 0x7fb1d40c0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1dbcd2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1d406ca90 0x7fb1d4151260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1dbcd2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d406c600 con 0x7fb1d406ca90 2024-09-13T17:17:21.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1dbcd2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d406bcd0 con 0x7fb1d40bf5e0 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1dbcd2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d406bfb0 con 0x7fb1d40bff20 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1da26f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1d406ca90 0x7fb1d4151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1da26f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1d406ca90 0x7fb1d4151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42414/0 (socket says 172.21.15.39:42414) 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1da26f640 1 -- 172.21.15.39:0/2860283126 learned_addr learned my addr 172.21.15.39:0/2860283126 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1d926d640 1 --2- 172.21.15.39:0/2860283126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1d40bff20 0x7fb1d40c0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1d9a6e640 1 --2- 172.21.15.39:0/2860283126 >> v2:172.21.15.84:3300/0 conn(0x7fb1d40bf5e0 0x7fb1d4154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1da26f640 1 -- 172.21.15.39:0/2860283126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1d40bff20 msgr2=0x7fb1d40c0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1da26f640 1 --2- 172.21.15.39:0/2860283126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1d40bff20 0x7fb1d40c0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1da26f640 1 -- 172.21.15.39:0/2860283126 >> v2:172.21.15.84:3300/0 conn(0x7fb1d40bf5e0 msgr2=0x7fb1d4154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1da26f640 1 --2- 172.21.15.39:0/2860283126 >> v2:172.21.15.84:3300/0 conn(0x7fb1d40bf5e0 0x7fb1d4154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1da26f640 1 -- 172.21.15.39:0/2860283126 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d406c290 con 0x7fb1d406ca90 2024-09-13T17:17:21.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1d9a6e640 1 --2- 172.21.15.39:0/2860283126 >> v2:172.21.15.84:3300/0 conn(0x7fb1d40bf5e0 0x7fb1d4154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:21.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1da26f640 1 --2- 172.21.15.39:0/2860283126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d406ca90 0x7fb1d4151260 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb1d406f2d0 tx=0x7fb1cc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c09f89075b6e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:21.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1d8a6c640 1 -- 172.21.15.39:0/2860283126 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0a56f0 con 0x7fb1d406ca90 2024-09-13T17:17:21.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1d8a6c640 1 -- 172.21.15.39:0/2860283126 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc0a5850 con 0x7fb1d406ca90 2024-09-13T17:17:21.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.330+0000 7fb1d8a6c640 1 -- 172.21.15.39:0/2860283126 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0a5be0 con 0x7fb1d406ca90 2024-09-13T17:17:21.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.338+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/2860283126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d406ca90 msgr2=0x7fb1d4151260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.338+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/2860283126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d406ca90 0x7fb1d4151260 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb1d406f2d0 tx=0x7fb1cc01c960 comp rx=0 tx=0).stop 2024-09-13T17:17:21.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.338+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/2860283126 shutdown_connections 2024-09-13T17:17:21.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.338+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/2860283126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d406ca90 0x7fb1d4151260 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.338+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/2860283126 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1d40bff20 0x7fb1d40c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.338+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/2860283126 >> v2:172.21.15.84:3300/0 conn(0x7fb1d40bf5e0 0x7fb1d4154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.338+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/2860283126 >> 172.21.15.39:0/2860283126 conn(0x7fb1d4069e10 msgr2=0x7fb1d406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:21.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.338+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/2860283126 shutdown_connections 2024-09-13T17:17:21.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.338+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/2860283126 wait complete. 2024-09-13T17:17:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1dbcd2640 1 Processor -- start 2024-09-13T17:17:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1dbcd2640 1 -- start start 2024-09-13T17:17:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1dbcd2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d406ca90 0x7fb1d40c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1dbcd2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1d40bf5e0 0x7fb1d41569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1dbcd2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d40bff20 0x7fb1d41533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1dbcd2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d41440e0 con 0x7fb1d406ca90 2024-09-13T17:17:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1dbcd2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d41f0740 con 0x7fb1d40bff20 2024-09-13T17:17:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1dbcd2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d4155450 con 0x7fb1d40bf5e0 2024-09-13T17:17:21.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1da26f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d40bff20 0x7fb1d41533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1da26f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d40bff20 0x7fb1d41533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54248/0 (socket says 172.21.15.39:54248) 2024-09-13T17:17:21.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1da26f640 1 -- 172.21.15.39:0/884274308 learned_addr learned my addr 172.21.15.39:0/884274308 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:21.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1da26f640 1 -- 172.21.15.39:0/884274308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1d40bf5e0 msgr2=0x7fb1d41569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1da26f640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1d40bf5e0 0x7fb1d41569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1da26f640 1 -- 172.21.15.39:0/884274308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d406ca90 msgr2=0x7fb1d40c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1da26f640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d406ca90 0x7fb1d40c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1da26f640 1 -- 172.21.15.39:0/884274308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc009b30 con 0x7fb1d40bff20 2024-09-13T17:17:21.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1da26f640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d40bff20 0x7fb1d41533a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc01d200 tx=0x7fb1cc019700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:21.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1ca7fc640 1 -- 172.21.15.39:0/884274308 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0b8670 con 0x7fb1d40bff20 2024-09-13T17:17:21.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1ca7fc640 1 -- 172.21.15.39:0/884274308 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc0b87d0 con 0x7fb1d40bff20 2024-09-13T17:17:21.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.342+0000 7fb1ca7fc640 1 -- 172.21.15.39:0/884274308 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0b8b40 con 0x7fb1d40bff20 2024-09-13T17:17:21.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.346+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d406d110 con 0x7fb1d40bff20 2024-09-13T17:17:21.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.346+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d4155a10 con 0x7fb1d40bff20 2024-09-13T17:17:21.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.346+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d41f0910 con 0x7fb1d40bff20 2024-09-13T17:17:21.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.346+0000 7fb1ca7fc640 1 -- 172.21.15.39:0/884274308 <== mon.1 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb1cc0b8ca0 con 0x7fb1d40bff20 2024-09-13T17:17:21.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.346+0000 7fb1ca7fc640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1b8066690 0x7fb1b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.346+0000 7fb1ca7fc640 1 -- 172.21.15.39:0/884274308 <== mon.1 v2:172.21.15.84:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fb1cc1206b0 con 0x7fb1d40bff20 2024-09-13T17:17:21.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.350+0000 7fb1d9a6e640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1b8066690 0x7fb1b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.350+0000 7fb1d9a6e640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1b8066690 0x7fb1b8068b50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000a8d0 tx=0x7fb1d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:21.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.362+0000 7fb1ca7fc640 1 -- 172.21.15.39:0/884274308 <== mon.1 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb1d41f0910 con 0x7fb1d40bff20 2024-09-13T17:17:21.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.702+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1d41f0910 con 0x7fb1d40bff20 2024-09-13T17:17:21.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.702+0000 7fb1ca7fc640 1 -- 172.21.15.39:0/884274308 <== mon.1 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v63) v1 ==== 99+0+11 (secure 0 0 0) 0x7fb1d41f0910 con 0x7fb1d40bff20 2024-09-13T17:17:21.706 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T17:17:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.706+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1b8066690 msgr2=0x7fb1b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.706+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1b8066690 0x7fb1b8068b50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000a8d0 tx=0x7fb1d0008040 comp rx=0 tx=0).stop 2024-09-13T17:17:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.706+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d40bff20 msgr2=0x7fb1d41533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.706+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d40bff20 0x7fb1d41533a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc01d200 tx=0x7fb1cc019700 comp rx=0 tx=0).stop 2024-09-13T17:17:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.706+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 shutdown_connections 2024-09-13T17:17:21.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.710+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb1b8066690 0x7fb1b8068b50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.710+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1d40bff20 0x7fb1d41533a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.710+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb1d40bf5e0 0x7fb1d41569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.710+0000 7fb1dbcd2640 1 --2- 172.21.15.39:0/884274308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d406ca90 0x7fb1d40c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.710+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 >> 172.21.15.39:0/884274308 conn(0x7fb1d4069e10 msgr2=0x7fb1d4070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:21.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.710+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 shutdown_connections 2024-09-13T17:17:21.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.710+0000 7fb1dbcd2640 1 -- 172.21.15.39:0/884274308 wait complete. 2024-09-13T17:17:21.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T17:17:21.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.910+0000 7f6c5f25e640 1 Processor -- start 2024-09-13T17:17:21.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.910+0000 7f6c5f25e640 1 -- start start 2024-09-13T17:17:21.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.910+0000 7f6c5f25e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c581510e0 0x7f6c5814ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.910+0000 7f6c5f25e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5806cc10 con 0x7f6c580c4950 2024-09-13T17:17:21.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.910+0000 7f6c5f25e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5806ce80 con 0x7f6c5806b180 2024-09-13T17:17:21.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.910+0000 7f6c5f25e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c580c5390 con 0x7f6c581510e0 2024-09-13T17:17:21.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.914+0000 7f6c5ea5d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c581510e0 0x7f6c5814ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.914+0000 7f6c5ea5d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6c581510e0 0x7f6c5814ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54250/0 (socket says 172.21.15.39:54250) 2024-09-13T17:17:21.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.914+0000 7f6c5ea5d640 1 -- 172.21.15.39:0/3348714930 learned_addr learned my addr 172.21.15.39:0/3348714930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:21.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.914+0000 7f6c5ea5d640 1 -- 172.21.15.39:0/3348714930 >> v1:172.21.15.84:6789/0 conn(0x7f6c5806b180 legacy=0x7f6c58150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.914+0000 7f6c5ea5d640 1 -- 172.21.15.39:0/3348714930 >> v1:172.21.15.39:6789/0 conn(0x7f6c580c4950 legacy=0x7f6c58153bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:17:21.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.914+0000 7f6c5ea5d640 1 -- 172.21.15.39:0/3348714930 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5806d160 con 0x7f6c581510e0 2024-09-13T17:17:21.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.922+0000 7f6c5ea5d640 1 --2- 172.21.15.39:0/3348714930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 0x7f6c5814ce20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800a910 tx=0x7f6c48020ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a9668f94767c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:21.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.926+0000 7f6c5d25a640 1 -- 172.21.15.39:0/3348714930 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480a9b60 con 0x7f6c581510e0 2024-09-13T17:17:21.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.926+0000 7f6c5d25a640 1 -- 172.21.15.39:0/3348714930 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c480ad070 con 0x7f6c581510e0 2024-09-13T17:17:21.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.926+0000 7f6c5d25a640 1 -- 172.21.15.39:0/3348714930 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480b3400 con 0x7f6c581510e0 2024-09-13T17:17:21.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.926+0000 7f6c5f25e640 1 -- 172.21.15.39:0/3348714930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 msgr2=0x7f6c5814ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.926+0000 7f6c5f25e640 1 --2- 172.21.15.39:0/3348714930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 0x7f6c5814ce20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800a910 tx=0x7f6c48020ba0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.930+0000 7f6c5f25e640 1 -- 172.21.15.39:0/3348714930 shutdown_connections 2024-09-13T17:17:21.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.930+0000 7f6c5f25e640 1 --2- 172.21.15.39:0/3348714930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 0x7f6c5814ce20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.930+0000 7f6c5f25e640 1 -- 172.21.15.39:0/3348714930 >> 172.21.15.39:0/3348714930 conn(0x7f6c580c5510 msgr2=0x7f6c580c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:21.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.930+0000 7f6c5f25e640 1 -- 172.21.15.39:0/3348714930 shutdown_connections 2024-09-13T17:17:21.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5f25e640 1 -- 172.21.15.39:0/3348714930 wait complete. 2024-09-13T17:17:21.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5f25e640 1 Processor -- start 2024-09-13T17:17:21.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5f25e640 1 -- start start 2024-09-13T17:17:21.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5f25e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c5806b180 0x7f6c580baef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5f25e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c580c4950 0x7f6c580bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5f25e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 0x7f6c580bd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5f25e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580701e0 con 0x7f6c5806b180 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5f25e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580beb80 con 0x7f6c581510e0 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5f25e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580bbc10 con 0x7f6c580c4950 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5e25c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c5806b180 0x7f6c580baef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5e25c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c5806b180 0x7f6c580baef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42440/0 (socket says 172.21.15.39:42440) 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5e25c640 1 -- 172.21.15.39:0/93401858 learned_addr learned my addr 172.21.15.39:0/93401858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5da5b640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c580c4950 0x7f6c580bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.934+0000 7f6c5ea5d640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 0x7f6c580bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5e25c640 1 -- 172.21.15.39:0/93401858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c580c4950 msgr2=0x7f6c580bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5e25c640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c580c4950 0x7f6c580bd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5e25c640 1 -- 172.21.15.39:0/93401858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 msgr2=0x7f6c580bd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5e25c640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 0x7f6c580bd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5e25c640 1 -- 172.21.15.39:0/93401858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4800a570 con 0x7f6c5806b180 2024-09-13T17:17:21.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5ea5d640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 0x7f6c580bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:21.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5e25c640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c5806b180 0x7f6c580baef0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000ba50 tx=0x7f6c5000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:21.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c4effd640 1 -- 172.21.15.39:0/93401858 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000cbe0 con 0x7f6c5806b180 2024-09-13T17:17:21.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c4effd640 1 -- 172.21.15.39:0/93401858 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5000cd80 con 0x7f6c5806b180 2024-09-13T17:17:21.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c580bb930 con 0x7f6c5806b180 2024-09-13T17:17:21.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c4effd640 1 -- 172.21.15.39:0/93401858 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50014680 con 0x7f6c5806b180 2024-09-13T17:17:21.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c581f1c80 con 0x7f6c5806b180 2024-09-13T17:17:21.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.938+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c58070350 con 0x7f6c5806b180 2024-09-13T17:17:21.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.950+0000 7f6c4effd640 1 -- 172.21.15.39:0/93401858 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c500147e0 con 0x7f6c5806b180 2024-09-13T17:17:21.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.950+0000 7f6c4effd640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c3c066690 0x7f6c3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:21.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.950+0000 7f6c4effd640 1 -- 172.21.15.39:0/93401858 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f6c50084ba0 con 0x7f6c5806b180 2024-09-13T17:17:21.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.950+0000 7f6c5da5b640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c3c066690 0x7f6c3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:21.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.950+0000 7f6c5da5b640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c3c066690 0x7f6c3c068b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6c5814ca10 tx=0x7f6c54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:21.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:21.954+0000 7f6c4effd640 1 -- 172.21.15.39:0/93401858 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6c500514d0 con 0x7f6c5806b180 2024-09-13T17:17:22.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.206+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f6c5814ce20 con 0x7f6c5806b180 2024-09-13T17:17:22.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.206+0000 7f6c4effd640 1 -- 172.21.15.39:0/93401858 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v63) v1 ==== 95+0+11 (secure 0 0 0) 0x7f6c500190a0 con 0x7f6c5806b180 2024-09-13T17:17:22.210 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T17:17:22.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c3c066690 msgr2=0x7f6c3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c3c066690 0x7f6c3c068b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6c5814ca10 tx=0x7f6c54009040 comp rx=0 tx=0).stop 2024-09-13T17:17:22.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c5806b180 msgr2=0x7f6c580baef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c5806b180 0x7f6c580baef0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000ba50 tx=0x7f6c5000bf10 comp rx=0 tx=0).stop 2024-09-13T17:17:22.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 shutdown_connections 2024-09-13T17:17:22.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6c3c066690 0x7f6c3c068b50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6c581510e0 0x7f6c580bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6c580c4950 0x7f6c580bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 --2- 172.21.15.39:0/93401858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c5806b180 0x7f6c580baef0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 >> 172.21.15.39:0/93401858 conn(0x7f6c580c5510 msgr2=0x7f6c5814af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:22.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 shutdown_connections 2024-09-13T17:17:22.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.210+0000 7f6c5f25e640 1 -- 172.21.15.39:0/93401858 wait complete. 2024-09-13T17:17:22.228 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T17:17:22.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.346+0000 7fdcc96c7640 1 Processor -- start 2024-09-13T17:17:22.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.346+0000 7fdcc96c7640 1 -- start start 2024-09-13T17:17:22.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc96c7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc4072120 0x7fdcc41545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc96c7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc406ec80 con 0x7fdcc406ddd0 2024-09-13T17:17:22.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc96c7640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc40c0e60 con 0x7fdcc41516d0 2024-09-13T17:17:22.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc96c7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4071d80 con 0x7fdcc4072120 2024-09-13T17:17:22.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcbbfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdcc406ddd0 0x7fdcc4153e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41484/0 (socket says 172.21.15.39:41484) 2024-09-13T17:17:22.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcbbfff640 1 -- 172.21.15.39:0/4148178587 learned_addr learned my addr 172.21.15.39:0/4148178587 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:22.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc37fe640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc4072120 0x7fdcc41545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134647519 0 0) 0x7fdcc406ec80 con 0x7fdcc406ddd0 2024-09-13T17:17:22.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb0003530 con 0x7fdcc406ddd0 2024-09-13T17:17:22.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2530523310 0 0) 0x7fdcb0003530 con 0x7fdcc406ddd0 2024-09-13T17:17:22.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc4072120 msgr2=0x7fdcc41545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:22.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc4072120 0x7fdcc41545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 >> v1:172.21.15.84:6789/0 conn(0x7fdcc41516d0 legacy=0x7fdcc40c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40c1100 con 0x7fdcc406ddd0 2024-09-13T17:17:22.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fdcac001680 con 0x7fdcc406ddd0 2024-09-13T17:17:22.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 >> v1:172.21.15.39:6789/0 conn(0x7fdcc406ddd0 legacy=0x7fdcc4153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb00047d0 0x7fdcb0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc4072120 0x7fdcb00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb000ba90 con 0x7fdcb00039b0 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb000bc00 con 0x7fdcb00047d0 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb000bdc0 con 0x7fdcc4072120 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc2ffd640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcbbfff640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb00047d0 0x7fdcb0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc37fe640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc4072120 0x7fdcb00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc2ffd640 1 -- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc4072120 msgr2=0x7fdcb00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc2ffd640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc4072120 0x7fdcb00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc2ffd640 1 -- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb00047d0 msgr2=0x7fdcb0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:22.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc2ffd640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcb00047d0 0x7fdcb0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc2ffd640 1 -- 172.21.15.39:0/4148178587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb0003530 con 0x7fdcb00039b0 2024-09-13T17:17:22.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.350+0000 7fdcc37fe640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc4072120 0x7fdcb00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:22.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcb0003dd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4001780 tx=0x7fdcb401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e29a18de5e6f06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:22.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40052d0 con 0x7fdcb00039b0 2024-09-13T17:17:22.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb40a7050 con 0x7fdcb00039b0 2024-09-13T17:17:22.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc27fc640 1 -- 172.21.15.39:0/4148178587 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4003600 con 0x7fdcb00039b0 2024-09-13T17:17:22.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 -- 172.21.15.39:0/4148178587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 msgr2=0x7fdcb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 --2- 172.21.15.39:0/4148178587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcb0003dd0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4001780 tx=0x7fdcb401c420 comp rx=0 tx=0).stop 2024-09-13T17:17:22.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc37fe640 1 -- 172.21.15.39:0/4148178587 reap_dead start 2024-09-13T17:17:22.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 -- 172.21.15.39:0/4148178587 shutdown_connections 2024-09-13T17:17:22.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 -- 172.21.15.39:0/4148178587 >> 172.21.15.39:0/4148178587 conn(0x7fdcc406c020 msgr2=0x7fdcc406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:22.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 -- 172.21.15.39:0/4148178587 shutdown_connections 2024-09-13T17:17:22.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 -- 172.21.15.39:0/4148178587 wait complete. 2024-09-13T17:17:22.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 Processor -- start 2024-09-13T17:17:22.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 -- start start 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcc41ed010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc41584f0 0x7fdcc414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc4145e90 0x7fdcc4146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc414a200 con 0x7fdcb00039b0 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40c49a0 con 0x7fdcc4145e90 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc96c7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40c4c80 con 0x7fdcc41584f0 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcc41ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcc41ed010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42456/0 (socket says 172.21.15.39:42456) 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 -- 172.21.15.39:0/3358728545 learned_addr learned my addr 172.21.15.39:0/3358728545 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcbbfff640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc41584f0 0x7fdcc414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 -- 172.21.15.39:0/3358728545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc41584f0 msgr2=0x7fdcc414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc41584f0 0x7fdcc414f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 -- 172.21.15.39:0/3358728545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc4145e90 msgr2=0x7fdcc4146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc4145e90 0x7fdcc4146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 -- 172.21.15.39:0/3358728545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc414f6a0 con 0x7fdcb00039b0 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.354+0000 7fdcc2ffd640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcc41ed010 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4005040 tx=0x7fdcb0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcbb7fe640 1 -- 172.21.15.39:0/3358728545 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4007040 con 0x7fdcb00039b0 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcbb7fe640 1 -- 172.21.15.39:0/3358728545 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb40071a0 con 0x7fdcb00039b0 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc414a200 con 0x7fdcb00039b0 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcbb7fe640 1 -- 172.21.15.39:0/3358728545 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4007590 con 0x7fdcb00039b0 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc4071b80 con 0x7fdcb00039b0 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc41549b0 con 0x7fdcb00039b0 2024-09-13T17:17:22.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcbb7fe640 1 -- 172.21.15.39:0/3358728545 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdcb40067e0 con 0x7fdcb00039b0 2024-09-13T17:17:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcbb7fe640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdca4066690 0x7fdca4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcbbfff640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdca4066690 0x7fdca4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcbb7fe640 1 -- 172.21.15.39:0/3358728545 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fdcb411f5d0 con 0x7fdcb00039b0 2024-09-13T17:17:22.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.358+0000 7fdcbbfff640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdca4066690 0x7fdca4068b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdcc41f0bc0 tx=0x7fdcac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:22.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.362+0000 7fdcbb7fe640 1 -- 172.21.15.39:0/3358728545 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdcb40eb9e0 con 0x7fdcb00039b0 2024-09-13T17:17:22.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.514+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcc40a71c0 con 0x7fdcb00039b0 2024-09-13T17:17:22.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.514+0000 7fdcbb7fe640 1 -- 172.21.15.39:0/3358728545 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v63) v1 ==== 99+0+11 (secure 0 0 0) 0x7fdcb40ef000 con 0x7fdcb00039b0 2024-09-13T17:17:22.518 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T17:17:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdca4066690 msgr2=0x7fdca4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdca4066690 0x7fdca4068b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdcc41f0bc0 tx=0x7fdcac009040 comp rx=0 tx=0).stop 2024-09-13T17:17:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 msgr2=0x7fdcc41ed010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcc41ed010 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4005040 tx=0x7fdcb0005800 comp rx=0 tx=0).stop 2024-09-13T17:17:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 shutdown_connections 2024-09-13T17:17:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdca4066690 0x7fdca4068b50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdcc4145e90 0x7fdcc4146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdcc41584f0 0x7fdcc414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 --2- 172.21.15.39:0/3358728545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb00039b0 0x7fdcc41ed010 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 >> 172.21.15.39:0/3358728545 conn(0x7fdcc406c020 msgr2=0x7fdcc414d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:22.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 shutdown_connections 2024-09-13T17:17:22.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.518+0000 7fdcc96c7640 1 -- 172.21.15.39:0/3358728545 wait complete. 2024-09-13T17:17:22.536 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T17:17:22.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.658+0000 7fc229016640 1 Processor -- start 2024-09-13T17:17:22.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.658+0000 7fc229016640 1 -- start start 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc229016640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc2240aa2d0 0x7fc2240aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc229016640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc224151cf0 0x7fc224152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc229016640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc224155d20 con 0x7fc2240a4d60 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc229016640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc224155b30 con 0x7fc2240aa2d0 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc229016640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2240c4750 con 0x7fc224151cf0 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc22259c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc2240a4d60 0x7fc22414ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41492/0 (socket says 172.21.15.39:41492) 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc22259c640 1 -- 172.21.15.39:0/973286883 learned_addr learned my addr 172.21.15.39:0/973286883 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc222d9d640 1 --2- 172.21.15.39:0/973286883 >> v2:172.21.15.39:3300/0 conn(0x7fc2240aa2d0 0x7fc2240aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc22359e640 1 --2- 172.21.15.39:0/973286883 >> v2:172.21.15.84:3300/0 conn(0x7fc224151cf0 0x7fc224152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc221d9b640 1 -- 172.21.15.39:0/973286883 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693088679 0 0) 0x7fc224155d20 con 0x7fc2240a4d60 2024-09-13T17:17:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc221d9b640 1 -- 172.21.15.39:0/973286883 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc20c003530 con 0x7fc2240a4d60 2024-09-13T17:17:22.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc222d9d640 1 -- 172.21.15.39:0/973286883 >> v2:172.21.15.84:3300/0 conn(0x7fc224151cf0 msgr2=0x7fc224152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:22.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc222d9d640 1 --2- 172.21.15.39:0/973286883 >> v2:172.21.15.84:3300/0 conn(0x7fc224151cf0 0x7fc224152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc222d9d640 1 -- 172.21.15.39:0/973286883 >> v1:172.21.15.39:6789/0 conn(0x7fc2240a4d60 legacy=0x7fc22414ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc222d9d640 1 -- 172.21.15.39:0/973286883 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc224155fc0 con 0x7fc2240aa2d0 2024-09-13T17:17:22.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc22359e640 1 --2- 172.21.15.39:0/973286883 >> v2:172.21.15.84:3300/0 conn(0x7fc224151cf0 0x7fc224152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:22.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc222d9d640 1 --2- 172.21.15.39:0/973286883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 0x7fc2240aa6b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc2240c6640 tx=0x7fc21401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683302beb927f9b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:22.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc221d9b640 1 -- 172.21.15.39:0/973286883 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140ab600 con 0x7fc2240aa2d0 2024-09-13T17:17:22.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc221d9b640 1 -- 172.21.15.39:0/973286883 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2140a7050 con 0x7fc2240aa2d0 2024-09-13T17:17:22.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc221d9b640 1 -- 172.21.15.39:0/973286883 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140ab960 con 0x7fc2240aa2d0 2024-09-13T17:17:22.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc229016640 1 -- 172.21.15.39:0/973286883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 msgr2=0x7fc2240aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc229016640 1 --2- 172.21.15.39:0/973286883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 0x7fc2240aa6b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc2240c6640 tx=0x7fc21401c3c0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.662+0000 7fc229016640 1 -- 172.21.15.39:0/973286883 shutdown_connections 2024-09-13T17:17:22.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 --2- 172.21.15.39:0/973286883 >> v2:172.21.15.84:3300/0 conn(0x7fc224151cf0 0x7fc224152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 --2- 172.21.15.39:0/973286883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 0x7fc2240aa6b0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- 172.21.15.39:0/973286883 >> 172.21.15.39:0/973286883 conn(0x7fc2240c5790 msgr2=0x7fc2240c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:22.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- 172.21.15.39:0/973286883 shutdown_connections 2024-09-13T17:17:22.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- 172.21.15.39:0/973286883 wait complete. 2024-09-13T17:17:22.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 Processor -- start 2024-09-13T17:17:22.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- start start 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2240a4d60 0x7fc224142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 0x7fc22414c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc224151cf0 0x7fc2240709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc224070230 con 0x7fc2240aa2d0 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240718e0 con 0x7fc224151cf0 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc224071bc0 con 0x7fc2240a4d60 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc22259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 0x7fc22414c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc22259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 0x7fc22414c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42478/0 (socket says 172.21.15.39:42478) 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc22259c640 1 -- 172.21.15.39:0/609418659 learned_addr learned my addr 172.21.15.39:0/609418659 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc22259c640 1 -- 172.21.15.39:0/609418659 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2240a4d60 msgr2=0x7fc224142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc22259c640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2240a4d60 0x7fc224142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc22259c640 1 -- 172.21.15.39:0/609418659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc224151cf0 msgr2=0x7fc2240709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc22259c640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc224151cf0 0x7fc2240709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc22259c640 1 -- 172.21.15.39:0/609418659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214009420 con 0x7fc2240aa2d0 2024-09-13T17:17:22.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc22259c640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 0x7fc22414c9c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c450 tx=0x7fc21800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:22.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc20b7fe640 1 -- 172.21.15.39:0/609418659 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218005cf0 con 0x7fc2240aa2d0 2024-09-13T17:17:22.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc20b7fe640 1 -- 172.21.15.39:0/609418659 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc218005e50 con 0x7fc2240aa2d0 2024-09-13T17:17:22.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc224071ea0 con 0x7fc2240aa2d0 2024-09-13T17:17:22.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc20b7fe640 1 -- 172.21.15.39:0/609418659 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218007290 con 0x7fc2240aa2d0 2024-09-13T17:17:22.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc224142c60 con 0x7fc2240aa2d0 2024-09-13T17:17:22.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc224071630 con 0x7fc2240aa2d0 2024-09-13T17:17:22.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.666+0000 7fc20b7fe640 1 -- 172.21.15.39:0/609418659 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2180073f0 con 0x7fc2240aa2d0 2024-09-13T17:17:22.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.670+0000 7fc20b7fe640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2100666c0 0x7fc210068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.670+0000 7fc20b7fe640 1 -- 172.21.15.39:0/609418659 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fc2180847c0 con 0x7fc2240aa2d0 2024-09-13T17:17:22.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.670+0000 7fc222d9d640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2100666c0 0x7fc210068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.670+0000 7fc222d9d640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2100666c0 0x7fc210068b80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc2140ad000 tx=0x7fc214004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:22.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.674+0000 7fc20b7fe640 1 -- 172.21.15.39:0/609418659 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc218050ab0 con 0x7fc2240aa2d0 2024-09-13T17:17:22.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.826+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fc2240aa6b0 con 0x7fc2240aa2d0 2024-09-13T17:17:22.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.826+0000 7fc20b7fe640 1 -- 172.21.15.39:0/609418659 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v63) v1 ==== 95+0+11 (secure 0 0 0) 0x7fc218017070 con 0x7fc2240aa2d0 2024-09-13T17:17:22.830 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2100666c0 msgr2=0x7fc210068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2100666c0 0x7fc210068b80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc2140ad000 tx=0x7fc214004740 comp rx=0 tx=0).stop 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 msgr2=0x7fc22414c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 0x7fc22414c9c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc21800c450 tx=0x7fc21800c910 comp rx=0 tx=0).stop 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 shutdown_connections 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2100666c0 0x7fc210068b80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc224151cf0 0x7fc2240709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2240aa2d0 0x7fc22414c9c0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 --2- 172.21.15.39:0/609418659 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2240a4d60 0x7fc224142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 >> 172.21.15.39:0/609418659 conn(0x7fc2240c5790 msgr2=0x7fc2241513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 shutdown_connections 2024-09-13T17:17:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.830+0000 7fc229016640 1 -- 172.21.15.39:0/609418659 wait complete. 2024-09-13T17:17:22.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:17:22.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b4f248640 1 Processor -- start 2024-09-13T17:17:22.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b4f248640 1 -- start start 2024-09-13T17:17:22.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b4f248640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0b48151010 0x7f0b480c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b4f248640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b480c08f0 0x7f0b480c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b4f248640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b480c20f0 con 0x7f0b480c2da0 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b4f248640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4806c9e0 con 0x7f0b48151010 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b4f248640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480c18e0 con 0x7f0b480c08f0 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b47fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0b480c2da0 0x7f0b480c40a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41506/0 (socket says 172.21.15.39:41506) 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b47fff640 1 -- 172.21.15.39:0/944410938 learned_addr learned my addr 172.21.15.39:0/944410938 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b4d7e5640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b480c08f0 0x7f0b480c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477591169 0 0) 0x7f0b34001120 con 0x7f0b480c2da0 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b4cfe4640 1 --2- 172.21.15.39:0/944410938 >> v2:172.21.15.84:3300/0 conn(0x7f0b48151010 0x7f0b480c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.970+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b38004640 con 0x7f0b480c2da0 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1125636758 0 0) 0x7f0b38004640 con 0x7f0b480c2da0 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b480c08f0 msgr2=0x7f0b480c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:22.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b480c08f0 0x7f0b480c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 >> v2:172.21.15.84:3300/0 conn(0x7f0b48151010 msgr2=0x7f0b480c03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 --2- 172.21.15.39:0/944410938 >> v2:172.21.15.84:3300/0 conn(0x7f0b48151010 0x7f0b480c03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4806b640 con 0x7f0b480c2da0 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f0b34001810 con 0x7f0b480c2da0 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 >> v1:172.21.15.39:6789/0 conn(0x7f0b480c2da0 legacy=0x7f0b480c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38004a90 0x7f0b38004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b380058e0 0x7f0b38005d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b480c08f0 0x7f0b380062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480c18e0 con 0x7f0b38004a90 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b380034e0 con 0x7f0b380058e0 2024-09-13T17:17:22.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b380036a0 con 0x7f0b480c08f0 2024-09-13T17:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4cfe4640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38004a90 0x7f0b38004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b47fff640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b380058e0 0x7f0b38005d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4d7e5640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b480c08f0 0x7f0b380062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4cfe4640 1 -- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b480c08f0 msgr2=0x7f0b380062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4cfe4640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b480c08f0 0x7f0b380062a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4cfe4640 1 -- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b380058e0 msgr2=0x7f0b38005d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4cfe4640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b380058e0 0x7f0b38005d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4cfe4640 1 -- 172.21.15.39:0/944410938 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b38004640 con 0x7f0b38004a90 2024-09-13T17:17:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b47fff640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b380058e0 0x7f0b38005d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:22.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4cfe4640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38004a90 0x7f0b38004eb0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00bfc0 tx=0x7f0b3c01ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabb42ff2d6d9d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:22.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0a7ac0 con 0x7f0b38004a90 2024-09-13T17:17:22.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c011040 con 0x7f0b38004a90 2024-09-13T17:17:22.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b477fe640 1 -- 172.21.15.39:0/944410938 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0a7e60 con 0x7f0b38004a90 2024-09-13T17:17:22.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4f248640 1 -- 172.21.15.39:0/944410938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38004a90 msgr2=0x7f0b38004eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4f248640 1 --2- 172.21.15.39:0/944410938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b38004a90 0x7f0b38004eb0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00bfc0 tx=0x7f0b3c01ecc0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4d7e5640 1 -- 172.21.15.39:0/944410938 reap_dead start 2024-09-13T17:17:22.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4f248640 1 -- 172.21.15.39:0/944410938 shutdown_connections 2024-09-13T17:17:22.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4f248640 1 -- 172.21.15.39:0/944410938 >> 172.21.15.39:0/944410938 conn(0x7f0b480c5790 msgr2=0x7f0b480c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:22.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4f248640 1 -- 172.21.15.39:0/944410938 shutdown_connections 2024-09-13T17:17:22.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.974+0000 7f0b4f248640 1 -- 172.21.15.39:0/944410938 wait complete. 2024-09-13T17:17:22.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 Processor -- start 2024-09-13T17:17:22.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 -- start start 2024-09-13T17:17:22.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b38004a90 0x7f0b480bee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b480bf360 0x7f0b4814d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b4814de70 0x7f0b480a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481577c0 con 0x7f0b480bf360 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480aa8a0 con 0x7f0b38004a90 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b481edaa0 con 0x7f0b4814de70 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b480bf360 0x7f0b4814d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b47fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b480bf360 0x7f0b4814d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42490/0 (socket says 172.21.15.39:42490) 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b47fff640 1 -- 172.21.15.39:0/2725445292 learned_addr learned my addr 172.21.15.39:0/2725445292 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4cfe4640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b38004a90 0x7f0b480bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4d7e5640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b4814de70 0x7f0b480a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b47fff640 1 -- 172.21.15.39:0/2725445292 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b4814de70 msgr2=0x7f0b480a8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b47fff640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b4814de70 0x7f0b480a8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b47fff640 1 -- 172.21.15.39:0/2725445292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b38004a90 msgr2=0x7f0b480bee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b47fff640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b38004a90 0x7f0b480bee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b47fff640 1 -- 172.21.15.39:0/2725445292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b480a99a0 con 0x7f0b480bf360 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b47fff640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b480bf360 0x7f0b4814d930 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400d760 tx=0x7f0b3400dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b457fa640 1 -- 172.21.15.39:0/2725445292 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340074e0 con 0x7f0b480bf360 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b457fa640 1 -- 172.21.15.39:0/2725445292 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b34007640 con 0x7f0b480bf360 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b480a9c80 con 0x7f0b480bf360 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b457fa640 1 -- 172.21.15.39:0/2725445292 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34004a30 con 0x7f0b480bf360 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b48157a30 con 0x7f0b480bf360 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b480674c0 con 0x7f0b480bf360 2024-09-13T17:17:22.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b457fa640 1 -- 172.21.15.39:0/2725445292 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b34004b90 con 0x7f0b480bf360 2024-09-13T17:17:22.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.978+0000 7f0b457fa640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0b2c066690 0x7f0b2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:22.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.982+0000 7f0b4cfe4640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0b2c066690 0x7f0b2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:22.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.982+0000 7f0b457fa640 1 -- 172.21.15.39:0/2725445292 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f0b34084c30 con 0x7f0b480bf360 2024-09-13T17:17:22.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.982+0000 7f0b4cfe4640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0b2c066690 0x7f0b2c068b50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0b1000 tx=0x7f0b3c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:22.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:22.982+0000 7f0b457fa640 1 -- 172.21.15.39:0/2725445292 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0b34050fd0 con 0x7f0b480bf360 2024-09-13T17:17:23.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.146+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0b480c20b0 con 0x7f0b480bf360 2024-09-13T17:17:23.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b457fa640 1 -- 172.21.15.39:0/2725445292 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 78 entries v63) v1 ==== 81+0+4738 (secure 0 0 0) 0x7f0b34019070 con 0x7f0b480bf360 2024-09-13T17:17:23.151 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/380008541 2024-09-14T17:16:34.950555+0000 2024-09-13T17:17:23.151 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3721635324 2024-09-14T17:16:04.944309+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1355102409 2024-09-14T17:15:29.935985+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/305129652 2024-09-14T17:14:49.920017+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/282472862 2024-09-14T17:14:04.899010+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/397460031 2024-09-14T17:12:19.869778+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/397460031 2024-09-14T17:12:19.869778+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/2351253009 2024-09-14T17:11:09.852639+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3334565883 2024-09-14T17:11:59.865711+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3923043380 2024-09-14T17:09:59.836463+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1334276441 2024-09-14T16:59:04.677958+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/1326467723 2024-09-14T17:00:02.030979+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1334276441 2024-09-14T16:59:04.677958+0000 2024-09-13T17:17:23.152 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4200926406 2024-09-14T17:05:59.776545+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1392182763 2024-09-14T17:07:19.797311+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/3122796453 2024-09-14T16:59:54.690333+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/1326467723 2024-09-14T17:00:02.030979+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2740003107 2024-09-14T16:57:14.654506+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1355102409 2024-09-14T17:15:29.935985+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/1969975539 2024-09-14T16:52:59.589819+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3721635324 2024-09-14T17:16:04.944309+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/2351253009 2024-09-14T17:11:09.852639+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2144264453 2024-09-14T16:52:24.581694+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4094657757 2024-09-14T16:56:04.630964+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3600322972 2024-09-14T16:55:14.618334+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/380008541 2024-09-14T17:16:34.950555+0000 2024-09-13T17:17:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1572373831 2024-09-14T16:51:14.567493+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4246622083 2024-09-14T17:01:24.709561+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/282472862 2024-09-14T17:14:04.899010+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1355376480 2024-09-14T16:54:09.605050+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3334565883 2024-09-14T17:11:59.865711+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3941977343 2024-09-14T17:08:49.820117+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/2521680730 2024-09-14T17:09:29.828780+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/661906662 2024-09-14T16:57:59.663485+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4094657757 2024-09-14T16:56:04.630964+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3540656099 2024-09-14T16:50:14.555020+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2065196144 2024-09-14T16:53:14.593565+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1750130579 2024-09-14T17:09:29.828673+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3163580917 2024-09-14T16:49:44.548920+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1355376480 2024-09-14T16:54:09.605050+0000 2024-09-13T17:17:23.154 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2065196144 2024-09-14T16:53:14.593565+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1392588044 2024-09-14T16:50:49.562750+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1034701751 2024-09-14T17:10:39.845194+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/3122796453 2024-09-14T16:59:54.690333+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/3105501456 2024-09-14T17:03:49.742996+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/2521680730 2024-09-14T17:09:29.828780+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/661906662 2024-09-14T16:57:59.663485+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1572373831 2024-09-14T16:51:14.567493+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/1969975539 2024-09-14T16:52:59.589819+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1392588044 2024-09-14T16:50:49.562750+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2144264453 2024-09-14T16:52:24.581694+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3163580917 2024-09-14T16:49:44.548920+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3941977343 2024-09-14T17:08:49.820117+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2638959497 2024-09-14T17:01:24.709473+0000 2024-09-13T17:17:23.155 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2740003107 2024-09-14T16:57:14.654506+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4246622083 2024-09-14T17:01:24.709561+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/305129652 2024-09-14T17:14:49.920017+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/835951775 2024-09-14T17:02:59.730250+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/828722052 2024-09-14T17:03:19.734465+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2638959497 2024-09-14T17:01:24.709473+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/828722052 2024-09-14T17:03:19.734465+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3600322972 2024-09-14T16:55:14.618334+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/611669592 2024-09-14T17:04:24.751785+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/611669592 2024-09-14T17:04:24.751785+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1590039204 2024-09-14T17:05:09.765210+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4200926406 2024-09-14T17:05:59.776545+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/440028459 2024-09-14T17:07:34.801438+0000 2024-09-13T17:17:23.156 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/3105501456 2024-09-14T17:03:49.742996+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1590039204 2024-09-14T17:05:09.765210+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/263561599 2024-09-14T17:05:54.774622+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3923043380 2024-09-14T17:09:59.836463+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/263561599 2024-09-14T17:05:54.774622+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1392182763 2024-09-14T17:07:19.797311+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/835951775 2024-09-14T17:02:59.730250+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3540656099 2024-09-14T16:50:14.555020+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/440028459 2024-09-14T17:07:34.801438+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1034701751 2024-09-14T17:10:39.845194+0000 2024-09-13T17:17:23.157 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1750130579 2024-09-14T17:09:29.828673+0000 2024-09-13T17:17:23.158 INFO:teuthology.orchestra.run.smithi039.stderr:listed 78 entries 2024-09-13T17:17:23.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0b2c066690 msgr2=0x7f0b2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0b2c066690 0x7f0b2c068b50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0b1000 tx=0x7f0b3c007040 comp rx=0 tx=0).stop 2024-09-13T17:17:23.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b480bf360 msgr2=0x7f0b4814d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b480bf360 0x7f0b4814d930 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400d760 tx=0x7f0b3400dc20 comp rx=0 tx=0).stop 2024-09-13T17:17:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 shutdown_connections 2024-09-13T17:17:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0b2c066690 0x7f0b2c068b50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0b4814de70 0x7f0b480a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b480bf360 0x7f0b4814d930 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 --2- 172.21.15.39:0/2725445292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0b38004a90 0x7f0b480bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 >> 172.21.15.39:0/2725445292 conn(0x7f0b480c5790 msgr2=0x7f0b48153fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 shutdown_connections 2024-09-13T17:17:23.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.150+0000 7f0b4f248640 1 -- 172.21.15.39:0/2725445292 wait complete. 2024-09-13T17:17:23.169 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-13T17:17:23.169 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:17:23.175 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:17:23.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.326+0000 7fc3092d3640 1 Processor -- start 2024-09-13T17:17:23.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.326+0000 7fc3092d3640 1 -- start start 2024-09-13T17:17:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc3040a4e70 0x7fc30406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc30406d300 0x7fc3040c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3040c2780 con 0x7fc30414e0c0 2024-09-13T17:17:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3040c2a60 con 0x7fc3040a4e70 2024-09-13T17:17:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040c2d40 con 0x7fc30406d300 2024-09-13T17:17:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3027fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc3040a4e70 0x7fc30406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc302ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc30414e0c0 0x7fc3040c1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41514/0 (socket says 172.21.15.39:41514) 2024-09-13T17:17:23.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc302ffd640 1 -- 172.21.15.39:0/3344790741 learned_addr learned my addr 172.21.15.39:0/3344790741 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:23.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3027fc640 1 -- 172.21.15.39:0/3344790741 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc30406d300 msgr2=0x7fc3040c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:23.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3037fe640 1 --2- 172.21.15.39:0/3344790741 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc30406d300 0x7fc3040c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3027fc640 1 --2- 172.21.15.39:0/3344790741 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc30406d300 0x7fc3040c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3027fc640 1 -- 172.21.15.39:0/3344790741 >> v1:172.21.15.39:6789/0 conn(0x7fc30414e0c0 legacy=0x7fc3040c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc301ffb640 1 -- 172.21.15.39:0/3344790741 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235575079 0 0) 0x7fc3040c2780 con 0x7fc30414e0c0 2024-09-13T17:17:23.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3027fc640 1 -- 172.21.15.39:0/3344790741 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3040c3020 con 0x7fc3040a4e70 2024-09-13T17:17:23.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3027fc640 1 --2- 172.21.15.39:0/3344790741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3040a4e70 0x7fc30406cdc0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80097c0 tx=0x7fc2f801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9114dfe69e9e0c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc301ffb640 1 -- 172.21.15.39:0/3344790741 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3040c5490 con 0x7fc3040a4e70 2024-09-13T17:17:23.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc301ffb640 1 -- 172.21.15.39:0/3344790741 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f80a7050 con 0x7fc3040a4e70 2024-09-13T17:17:23.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc301ffb640 1 -- 172.21.15.39:0/3344790741 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80ab820 con 0x7fc3040a4e70 2024-09-13T17:17:23.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 -- 172.21.15.39:0/3344790741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3040a4e70 msgr2=0x7fc30406cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 --2- 172.21.15.39:0/3344790741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3040a4e70 0x7fc30406cdc0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80097c0 tx=0x7fc2f801c410 comp rx=0 tx=0).stop 2024-09-13T17:17:23.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 -- 172.21.15.39:0/3344790741 shutdown_connections 2024-09-13T17:17:23.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 --2- 172.21.15.39:0/3344790741 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc30406d300 0x7fc3040c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 --2- 172.21.15.39:0/3344790741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3040a4e70 0x7fc30406cdc0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 -- 172.21.15.39:0/3344790741 >> 172.21.15.39:0/3344790741 conn(0x7fc304069e10 msgr2=0x7fc30406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:23.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 -- 172.21.15.39:0/3344790741 shutdown_connections 2024-09-13T17:17:23.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.330+0000 7fc3092d3640 1 -- 172.21.15.39:0/3344790741 wait complete. 2024-09-13T17:17:23.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 Processor -- start 2024-09-13T17:17:23.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 -- start start 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc30406d300 0x7fc30414ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3040a4e70 0x7fc3040bec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc30414e0c0 0x7fc304140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040bf560 con 0x7fc30406d300 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041ef8b0 con 0x7fc30414e0c0 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041efb90 con 0x7fc3040a4e70 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc30406d300 0x7fc30414ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc302ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc30406d300 0x7fc30414ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42504/0 (socket says 172.21.15.39:42504) 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc302ffd640 1 -- 172.21.15.39:0/3185633858 learned_addr learned my addr 172.21.15.39:0/3185633858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:23.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3027fc640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3040a4e70 0x7fc3040bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc302ffd640 1 -- 172.21.15.39:0/3185633858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3040a4e70 msgr2=0x7fc3040bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc302ffd640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3040a4e70 0x7fc3040bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc302ffd640 1 -- 172.21.15.39:0/3185633858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc30414e0c0 msgr2=0x7fc304140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3037fe640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc30414e0c0 0x7fc304140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc302ffd640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc30414e0c0 0x7fc304140b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc302ffd640 1 -- 172.21.15.39:0/3185633858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f8009420 con 0x7fc30406d300 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3027fc640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3040a4e70 0x7fc3040bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc302ffd640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc30406d300 0x7fc30414ef40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400c480 tx=0x7fc2f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc2eb7fe640 1 -- 172.21.15.39:0/3185633858 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40074e0 con 0x7fc30406d300 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc2eb7fe640 1 -- 172.21.15.39:0/3185633858 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f4007660 con 0x7fc30406d300 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3041efe70 con 0x7fc30406d300 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc2eb7fe640 1 -- 172.21.15.39:0/3185633858 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4004a30 con 0x7fc30406d300 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304151600 con 0x7fc30406d300 2024-09-13T17:17:23.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3041f00b0 con 0x7fc30406d300 2024-09-13T17:17:23.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.334+0000 7fc2eb7fe640 1 -- 172.21.15.39:0/3185633858 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2f4004b90 con 0x7fc30406d300 2024-09-13T17:17:23.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.338+0000 7fc2eb7fe640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2f00666c0 0x7fc2f0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.338+0000 7fc2eb7fe640 1 -- 172.21.15.39:0/3185633858 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fc2f4084390 con 0x7fc30406d300 2024-09-13T17:17:23.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.338+0000 7fc3027fc640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2f00666c0 0x7fc2f0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.338+0000 7fc3027fc640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2f00666c0 0x7fc2f0068b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80ad000 tx=0x7fc2f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.342+0000 7fc2eb7fe640 1 -- 172.21.15.39:0/3185633858 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc2f4054110 con 0x7fc30406d300 2024-09-13T17:17:23.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.494+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3041ef6e0 con 0x7fc30406d300 2024-09-13T17:17:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc2eb7fe640 1 -- 172.21.15.39:0/3185633858 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+25425 (secure 0 0 0) 0x7fc2f4018020 con 0x7fc30406d300 2024-09-13T17:17:23.500 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:23.500 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":63,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T17:16:34.950618+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6835/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6834/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.39:6834/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6837/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6835/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.84:6834/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.39:6836/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.39:6835/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.84:6835/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.39:6835/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.84:6834/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.39:6834/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.39:6834/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.39:6836/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.84:6832/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.84:6835/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.84:6832/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6836/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.84:6834/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.39:6835/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.84:6832/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6834/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6835/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.84:6834/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6835/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.39:6835/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6836/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6833/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.39:6835/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.39:6837/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6834/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6835/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6834/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.39:6835/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1750130579":"2024-09-14T17:09:29.828673+0000","172.21.15.39:6834/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.84:6832/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.39:6834/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.39:6837/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6836/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6835/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6834/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.84:6835/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.84:6833/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.84:6833/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6835/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.39:6834/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.84:6832/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6833/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6834/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6835/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6836/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6834/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6833/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6835/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6834/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.39:6834/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6835/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6836/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.39:6835/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.84:6833/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6836/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6837/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.84:6834/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.84:6834/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.84:6835/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.39:6837/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.39:6837/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6835/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.84:6832/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6835/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/1750130579":"2024-09-14T17:09:29.828673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2f00666c0 msgr2=0x7fc2f0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2f00666c0 0x7fc2f0068b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80ad000 tx=0x7fc2f8008040 comp rx=0 tx=0).stop 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc30406d300 msgr2=0x7fc30414ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc30406d300 0x7fc30414ef40 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400c480 tx=0x7fc2f400c940 comp rx=0 tx=0).stop 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 shutdown_connections 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc2f00666c0 0x7fc2f0068b80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc30414e0c0 0x7fc304140b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc3040a4e70 0x7fc3040bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 --2- 172.21.15.39:0/3185633858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc30406d300 0x7fc30414ef40 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 >> 172.21.15.39:0/3185633858 conn(0x7fc304069e10 msgr2=0x7fc30406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 shutdown_connections 2024-09-13T17:17:23.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.498+0000 7fc3092d3640 1 -- 172.21.15.39:0/3185633858 wait complete. 2024-09-13T17:17:23.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:17:23.518 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:17:23.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.642+0000 7f4777ea0640 1 Processor -- start 2024-09-13T17:17:23.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.642+0000 7f4777ea0640 1 -- start start 2024-09-13T17:17:23.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4777ea0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f47700aa2d0 0x7f47700aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4777ea0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f47700a4d60 0x7f477014ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4777ea0640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4770155d20 con 0x7f477014f4b0 2024-09-13T17:17:23.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4777ea0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4770155b30 con 0x7f47700aa2d0 2024-09-13T17:17:23.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4777ea0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47700c4750 con 0x7f47700a4d60 2024-09-13T17:17:23.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4775c3c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f47700aa2d0 0x7f47700aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f477543b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f47700a4d60 0x7f477014ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4775c3c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f47700aa2d0 0x7f47700aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54344/0 (socket says 172.21.15.39:54344) 2024-09-13T17:17:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4775c3c640 1 -- 172.21.15.39:0/2028034545 learned_addr learned my addr 172.21.15.39:0/2028034545 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f477543b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f47700a4d60 0x7f477014ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:35010/0 (socket says 172.21.15.39:35010) 2024-09-13T17:17:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4775c3c640 1 -- 172.21.15.39:0/2028034545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f47700a4d60 msgr2=0x7f477014ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4775c3c640 1 --2- 172.21.15.39:0/2028034545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f47700a4d60 0x7f477014ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4775c3c640 1 -- 172.21.15.39:0/2028034545 >> v1:172.21.15.84:6789/0 conn(0x7f477014f4b0 legacy=0x7f4770151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f4775c3c640 1 -- 172.21.15.39:0/2028034545 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4770155fc0 con 0x7f47700aa2d0 2024-09-13T17:17:23.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.646+0000 7f477543b640 1 --2- 172.21.15.39:0/2028034545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f47700a4d60 0x7f477014ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:17:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4775c3c640 1 --2- 172.21.15.39:0/2028034545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47700aa2d0 0x7f47700aa6b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f476401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2776bf25ac241d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4774c3a640 1 -- 172.21.15.39:0/2028034545 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640ac410 con 0x7f47700aa2d0 2024-09-13T17:17:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4774c3a640 1 -- 172.21.15.39:0/2028034545 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47640ac570 con 0x7f47700aa2d0 2024-09-13T17:17:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4774c3a640 1 -- 172.21.15.39:0/2028034545 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640ac900 con 0x7f47700aa2d0 2024-09-13T17:17:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 -- 172.21.15.39:0/2028034545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47700aa2d0 msgr2=0x7f47700aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 --2- 172.21.15.39:0/2028034545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47700aa2d0 0x7f47700aa6b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f476401c4a0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 -- 172.21.15.39:0/2028034545 shutdown_connections 2024-09-13T17:17:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 --2- 172.21.15.39:0/2028034545 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f47700a4d60 0x7f477014ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 --2- 172.21.15.39:0/2028034545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47700aa2d0 0x7f47700aa6b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 -- 172.21.15.39:0/2028034545 >> 172.21.15.39:0/2028034545 conn(0x7f47700c5790 msgr2=0x7f47700c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 -- 172.21.15.39:0/2028034545 shutdown_connections 2024-09-13T17:17:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 -- 172.21.15.39:0/2028034545 wait complete. 2024-09-13T17:17:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 Processor -- start 2024-09-13T17:17:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 -- start start 2024-09-13T17:17:23.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47700a4d60 0x7f4770071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700aa2d0 0x7f47701ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f477014f4b0 0x7f47701eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4770149040 con 0x7f47700aa2d0 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4770148d60 con 0x7f47700a4d60 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f4777ea0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4770146760 con 0x7f477014f4b0 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f477543b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700aa2d0 0x7f47701ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f477543b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700aa2d0 0x7f47701ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42518/0 (socket says 172.21.15.39:42518) 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.650+0000 7f477543b640 1 -- 172.21.15.39:0/994935157 learned_addr learned my addr 172.21.15.39:0/994935157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f4775c3c640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47700a4d60 0x7f4770071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f477643d640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f477014f4b0 0x7f47701eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f477543b640 1 -- 172.21.15.39:0/994935157 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f477014f4b0 msgr2=0x7f47701eaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f477543b640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f477014f4b0 0x7f47701eaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f477543b640 1 -- 172.21.15.39:0/994935157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47700a4d60 msgr2=0x7f4770071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f477543b640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47700a4d60 0x7f4770071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f477543b640 1 -- 172.21.15.39:0/994935157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c009420 con 0x7f47700aa2d0 2024-09-13T17:17:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f477543b640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700aa2d0 0x7f47701ea9a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e980 tx=0x7f476c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f47627fc640 1 -- 172.21.15.39:0/994935157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00ca00 con 0x7f47700aa2d0 2024-09-13T17:17:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f47627fc640 1 -- 172.21.15.39:0/994935157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c00cb60 con 0x7f47700aa2d0 2024-09-13T17:17:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f47627fc640 1 -- 172.21.15.39:0/994935157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c013470 con 0x7f47700aa2d0 2024-09-13T17:17:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47640094d0 con 0x7f47700aa2d0 2024-09-13T17:17:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4770072030 con 0x7f47700aa2d0 2024-09-13T17:17:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47701edb60 con 0x7f47700aa2d0 2024-09-13T17:17:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f47627fc640 1 -- 172.21.15.39:0/994935157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f476c0135d0 con 0x7f47700aa2d0 2024-09-13T17:17:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f47627fc640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4754066690 0x7f4754068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f47627fc640 1 -- 172.21.15.39:0/994935157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f476c085b00 con 0x7f47700aa2d0 2024-09-13T17:17:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f4775c3c640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4754066690 0x7f4754068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.654+0000 7f4775c3c640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4754066690 0x7f4754068b50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f47640ae000 tx=0x7f4764009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.658+0000 7f47627fc640 1 -- 172.21.15.39:0/994935157 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f476c051f20 con 0x7f47700aa2d0 2024-09-13T17:17:23.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.826+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f477014ef70 con 0x7f47700aa2d0 2024-09-13T17:17:23.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.826+0000 7f47627fc640 1 -- 172.21.15.39:0/994935157 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f476c021020 con 0x7f47700aa2d0 2024-09-13T17:17:23.830 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4754066690 msgr2=0x7f4754068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4754066690 0x7f4754068b50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f47640ae000 tx=0x7f4764009040 comp rx=0 tx=0).stop 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700aa2d0 msgr2=0x7f47701ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700aa2d0 0x7f47701ea9a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e980 tx=0x7f476c00ee40 comp rx=0 tx=0).stop 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 shutdown_connections 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4754066690 0x7f4754068b50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f477014f4b0 0x7f47701eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700aa2d0 0x7f47701ea9a0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 --2- 172.21.15.39:0/994935157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47700a4d60 0x7f4770071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 >> 172.21.15.39:0/994935157 conn(0x7f47700c5790 msgr2=0x7f47700c3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 shutdown_connections 2024-09-13T17:17:23.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.830+0000 7f4777ea0640 1 -- 172.21.15.39:0/994935157 wait complete. 2024-09-13T17:17:23.850 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T17:17:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4fffe0f640 1 Processor -- start 2024-09-13T17:17:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4fffe0f640 1 -- start start 2024-09-13T17:17:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4fffe0f640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4ff8157370 0x7f4ff8154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4fffe0f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff8155270 con 0x7f4ff81514d0 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4fffe0f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff8153d80 con 0x7f4ff80bcf90 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4fffe0f640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff8157870 con 0x7f4ff8157370 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4ffdbab640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4ff81514d0 0x7f4ff80c41f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41530/0 (socket says 172.21.15.39:41530) 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4ffdbab640 1 -- 172.21.15.39:0/1999188561 learned_addr learned my addr 172.21.15.39:0/1999188561 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4ffe3ac640 1 --2- 172.21.15.39:0/1999188561 >> v2:172.21.15.84:3300/0 conn(0x7f4ff8157370 0x7f4ff8154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388197883 0 0) 0x7f4ff8155270 con 0x7f4ff81514d0 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.982+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe4003530 con 0x7f4ff81514d0 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4246002953 0 0) 0x7f4fe4003530 con 0x7f4ff81514d0 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 >> v2:172.21.15.84:3300/0 conn(0x7f4ff8157370 msgr2=0x7f4ff8154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 --2- 172.21.15.39:0/1999188561 >> v2:172.21.15.84:3300/0 conn(0x7f4ff8157370 0x7f4ff8154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 >> v1:172.21.15.84:6789/0 conn(0x7f4ff80bcf90 legacy=0x7f4ff8154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff8157b50 con 0x7f4ff81514d0 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4ff4001680 con 0x7f4ff81514d0 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 >> v1:172.21.15.39:6789/0 conn(0x7f4ff81514d0 legacy=0x7f4ff80c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fe4003980 0x7f4fe4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4fe40047d0 0x7f4fe4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4005190 0x7f4fe400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8157870 con 0x7f4fe4003980 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe400c130 con 0x7f4fe4005190 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe400c2f0 con 0x7f4fe40047d0 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffdbab640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fe4003980 0x7f4fe4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffe3ac640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4005190 0x7f4fe400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffd3aa640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4fe40047d0 0x7f4fe4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffdbab640 1 -- 172.21.15.39:0/1999188561 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4fe40047d0 msgr2=0x7f4fe4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffdbab640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4fe40047d0 0x7f4fe4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffdbab640 1 -- 172.21.15.39:0/1999188561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4005190 msgr2=0x7f4fe400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffdbab640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4005190 0x7f4fe400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffdbab640 1 -- 172.21.15.39:0/1999188561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4003530 con 0x7f4fe4003980 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffe3ac640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe4005190 0x7f4fe400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffe3ac640 1 -- 172.21.15.39:0/1999188561 reap_dead start 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffdbab640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fe4003980 0x7f4fe4003da0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c240 tx=0x7f4ff401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fbd83830753743 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40058e0 con 0x7f4fe4003980 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4005a60 con 0x7f4fe4003980 2024-09-13T17:17:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4ffcba9640 1 -- 172.21.15.39:0/1999188561 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4005e10 con 0x7f4fe4003980 2024-09-13T17:17:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1999188561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fe4003980 msgr2=0x7f4fe4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4fffe0f640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fe4003980 0x7f4fe4003da0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400c240 tx=0x7f4ff401cc20 comp rx=0 tx=0).stop 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1999188561 shutdown_connections 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4fffe0f640 1 --2- 172.21.15.39:0/1999188561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4fe4003980 0x7f4fe4003da0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1999188561 >> 172.21.15.39:0/1999188561 conn(0x7f4ff806a210 msgr2=0x7f4ff806a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1999188561 shutdown_connections 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.986+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1999188561 wait complete. 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 Processor -- start 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 -- start start 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4fe4003980 0x7f4ff8149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ff81efc40 0x7f4ff80c0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff80c0e80 0x7f4ff8147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8148090 con 0x7f4ff80c0e80 2024-09-13T17:17:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80c9320 con 0x7f4ff81efc40 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff80c3820 con 0x7f4fe4003980 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffe3ac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff80c0e80 0x7f4ff8147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffe3ac640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff80c0e80 0x7f4ff8147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42536/0 (socket says 172.21.15.39:42536) 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffe3ac640 1 -- 172.21.15.39:0/1065611190 learned_addr learned my addr 172.21.15.39:0/1065611190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffdbab640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4fe4003980 0x7f4ff8149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffd3aa640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ff81efc40 0x7f4ff80c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffe3ac640 1 -- 172.21.15.39:0/1065611190 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4fe4003980 msgr2=0x7f4ff8149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffe3ac640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4fe4003980 0x7f4ff8149b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffe3ac640 1 -- 172.21.15.39:0/1065611190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ff81efc40 msgr2=0x7f4ff80c0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffe3ac640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ff81efc40 0x7f4ff80c0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffe3ac640 1 -- 172.21.15.39:0/1065611190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff8156280 con 0x7f4ff80c0e80 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffd3aa640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ff81efc40 0x7f4ff80c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4ffe3ac640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff80c0e80 0x7f4ff8147030 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4ff81572d0 tx=0x7f4ff00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fee7fc640 1 -- 172.21.15.39:0/1065611190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000e070 con 0x7f4ff80c0e80 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fee7fc640 1 -- 172.21.15.39:0/1065611190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff0007e40 con 0x7f4ff80c0e80 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff80c3540 con 0x7f4ff80c0e80 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fee7fc640 1 -- 172.21.15.39:0/1065611190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0011db0 con 0x7f4ff80c0e80 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff81524c0 con 0x7f4ff80c0e80 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.990+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff81548d0 con 0x7f4ff80c0e80 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.994+0000 7f4fee7fc640 1 -- 172.21.15.39:0/1065611190 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ff00039a0 con 0x7f4ff80c0e80 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.994+0000 7f4fee7fc640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4fdc0666c0 0x7f4fdc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.994+0000 7f4fee7fc640 1 -- 172.21.15.39:0/1065611190 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f4ff00858e0 con 0x7f4ff80c0e80 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.994+0000 7f4ffdbab640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4fdc0666c0 0x7f4fdc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:23.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.994+0000 7f4ffdbab640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4fdc0666c0 0x7f4fdc068b80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40ab000 tx=0x7f4ff40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:23.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:23.998+0000 7f4fee7fc640 1 -- 172.21.15.39:0/1065611190 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4ff0051bd0 con 0x7f4ff80c0e80 2024-09-13T17:17:24.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.158+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ff8070690 con 0x7f4ff80c0e80 2024-09-13T17:17:24.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.158+0000 7f4fee7fc640 1 -- 172.21.15.39:0/1065611190 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v63) v1 ==== 99+0+11 (secure 0 0 0) 0x7f4ff0014350 con 0x7f4ff80c0e80 2024-09-13T17:17:24.161 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T17:17:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.158+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4fdc0666c0 msgr2=0x7f4fdc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.158+0000 7f4fffe0f640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4fdc0666c0 0x7f4fdc068b80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40ab000 tx=0x7f4ff40a7070 comp rx=0 tx=0).stop 2024-09-13T17:17:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.158+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff80c0e80 msgr2=0x7f4ff8147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.158+0000 7f4fffe0f640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff80c0e80 0x7f4ff8147030 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4ff81572d0 tx=0x7f4ff00115a0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.162+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 shutdown_connections 2024-09-13T17:17:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.162+0000 7f4fffe0f640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4fdc0666c0 0x7f4fdc068b80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.162+0000 7f4fffe0f640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ff81efc40 0x7f4ff80c0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.162+0000 7f4fffe0f640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff80c0e80 0x7f4ff8147030 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.162+0000 7f4fffe0f640 1 --2- 172.21.15.39:0/1065611190 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4fe4003980 0x7f4ff8149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.162+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 >> 172.21.15.39:0/1065611190 conn(0x7f4ff806a210 msgr2=0x7f4ff814d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.162+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 shutdown_connections 2024-09-13T17:17:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.162+0000 7f4fffe0f640 1 -- 172.21.15.39:0/1065611190 wait complete. 2024-09-13T17:17:24.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T17:17:24.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2f2d0640 1 Processor -- start 2024-09-13T17:17:24.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2f2d0640 1 -- start start 2024-09-13T17:17:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2f2d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b281514d0 0x7f8b280c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2f2d0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b280bcf90 0x7f8b28154240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2f2d0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b28155270 con 0x7f8b28157370 2024-09-13T17:17:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2f2d0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b28153d80 con 0x7f8b280bcf90 2024-09-13T17:17:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2f2d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28157870 con 0x7f8b281514d0 2024-09-13T17:17:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2d86d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8b28157370 0x7f8b28154780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41544/0 (socket says 172.21.15.39:41544) 2024-09-13T17:17:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2d86d640 1 -- 172.21.15.39:0/1846614161 learned_addr learned my addr 172.21.15.39:0/1846614161 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2c86b640 1 --2- 172.21.15.39:0/1846614161 >> v2:172.21.15.84:3300/0 conn(0x7f8b280bcf90 0x7f8b28154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2d06c640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b281514d0 0x7f8b280c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960809360 0 0) 0x7f8b28155270 con 0x7f8b28157370 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b14003530 con 0x7f8b28157370 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2877669434 0 0) 0x7f8b14003530 con 0x7f8b28157370 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b281514d0 msgr2=0x7f8b280c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b281514d0 0x7f8b280c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 >> v2:172.21.15.84:3300/0 conn(0x7f8b280bcf90 msgr2=0x7f8b28154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 --2- 172.21.15.39:0/1846614161 >> v2:172.21.15.84:3300/0 conn(0x7f8b280bcf90 0x7f8b28154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28157b50 con 0x7f8b28157370 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f8b20001a00 con 0x7f8b28157370 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 >> v1:172.21.15.39:6789/0 conn(0x7f8b28157370 legacy=0x7f8b28154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b281514d0 0x7f8b14003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b14004d10 0x7f8b14005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28153d80 con 0x7f8b14003f00 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28157870 con 0x7f8b14004d10 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1400ba90 con 0x7f8b281514d0 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2c86b640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2d06c640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b281514d0 0x7f8b14003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2d86d640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b14004d10 0x7f8b14005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2c86b640 1 -- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b281514d0 msgr2=0x7f8b14003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2c86b640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b281514d0 0x7f8b14003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2c86b640 1 -- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b14004d10 msgr2=0x7f8b14005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2c86b640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b14004d10 0x7f8b14005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2c86b640 1 -- 172.21.15.39:0/1846614161 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b14003530 con 0x7f8b14003f00 2024-09-13T17:17:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.306+0000 7f8b2d06c640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b281514d0 0x7f8b14003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2c86b640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b140047d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8b18009ae0 tx=0x7f8b1801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924988c0ea3eac63 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180a5530 con 0x7f8b14003f00 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b180a5690 con 0x7f8b14003f00 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b1ffff640 1 -- 172.21.15.39:0/1846614161 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180a5840 con 0x7f8b14003f00 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1846614161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 msgr2=0x7f8b140047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2f2d0640 1 --2- 172.21.15.39:0/1846614161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b140047d0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f8b18009ae0 tx=0x7f8b1801ca70 comp rx=0 tx=0).stop 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2d86d640 1 -- 172.21.15.39:0/1846614161 reap_dead start 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1846614161 shutdown_connections 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1846614161 >> 172.21.15.39:0/1846614161 conn(0x7f8b2806a210 msgr2=0x7f8b2806a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1846614161 shutdown_connections 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1846614161 wait complete. 2024-09-13T17:17:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2f2d0640 1 Processor -- start 2024-09-13T17:17:24.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.310+0000 7f8b2f2d0640 1 -- start start 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2f2d0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b28148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2f2d0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b280c9020 0x7f8b28150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2f2d0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28146c10 0x7f8b281470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2f2d0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281ed0a0 con 0x7f8b14003f00 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2f2d0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281ec2e0 con 0x7f8b28146c10 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2f2d0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281ec5c0 con 0x7f8b280c9020 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d06c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b28148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d06c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b28148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42562/0 (socket says 172.21.15.39:42562) 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d06c640 1 -- 172.21.15.39:0/1628299695 learned_addr learned my addr 172.21.15.39:0/1628299695 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2c86b640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b280c9020 0x7f8b28150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d86d640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28146c10 0x7f8b281470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d06c640 1 -- 172.21.15.39:0/1628299695 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b280c9020 msgr2=0x7f8b28150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d06c640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b280c9020 0x7f8b28150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d06c640 1 -- 172.21.15.39:0/1628299695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28146c10 msgr2=0x7f8b281470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d06c640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28146c10 0x7f8b281470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d06c640 1 -- 172.21.15.39:0/1628299695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b281ec8a0 con 0x7f8b14003f00 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2d06c640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b28148d20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400f510 tx=0x7f8b2400f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b1dffb640 1 -- 172.21.15.39:0/1628299695 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24014870 con 0x7f8b14003f00 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b1dffb640 1 -- 172.21.15.39:0/1628299695 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b240149d0 con 0x7f8b14003f00 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b281ecb80 con 0x7f8b14003f00 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b1dffb640 1 -- 172.21.15.39:0/1628299695 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24014d60 con 0x7f8b14003f00 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b28149260 con 0x7f8b14003f00 2024-09-13T17:17:24.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b280af880 con 0x7f8b14003f00 2024-09-13T17:17:24.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b1dffb640 1 -- 172.21.15.39:0/1628299695 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b24015410 con 0x7f8b14003f00 2024-09-13T17:17:24.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b1dffb640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b0c0666c0 0x7f8b0c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2c86b640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b0c0666c0 0x7f8b0c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b1dffb640 1 -- 172.21.15.39:0/1628299695 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f8b24085ad0 con 0x7f8b14003f00 2024-09-13T17:17:24.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.314+0000 7f8b2c86b640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b0c0666c0 0x7f8b0c068b80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8b180af000 tx=0x7f8b180046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.318+0000 7f8b1dffb640 1 -- 172.21.15.39:0/1628299695 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8b24051dc0 con 0x7f8b14003f00 2024-09-13T17:17:24.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.482+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f8b28154240 con 0x7f8b14003f00 2024-09-13T17:17:24.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.482+0000 7f8b1dffb640 1 -- 172.21.15.39:0/1628299695 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v63) v1 ==== 95+0+11 (secure 0 0 0) 0x7f8b240553e0 con 0x7f8b14003f00 2024-09-13T17:17:24.484 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T17:17:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b0c0666c0 msgr2=0x7f8b0c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b0c0666c0 0x7f8b0c068b80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8b180af000 tx=0x7f8b180046d0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 msgr2=0x7f8b28148d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b28148d20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400f510 tx=0x7f8b2400f9d0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 shutdown_connections 2024-09-13T17:17:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8b0c0666c0 0x7f8b0c068b80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b28146c10 0x7f8b281470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8b280c9020 0x7f8b28150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 --2- 172.21.15.39:0/1628299695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b14003f00 0x7f8b28148d20 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 >> 172.21.15.39:0/1628299695 conn(0x7f8b2806a210 msgr2=0x7f8b281534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:24.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 shutdown_connections 2024-09-13T17:17:24.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.486+0000 7f8b2f2d0640 1 -- 172.21.15.39:0/1628299695 wait complete. 2024-09-13T17:17:24.504 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T17:17:24.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184bd74640 1 Processor -- start 2024-09-13T17:17:24.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184bd74640 1 -- start start 2024-09-13T17:17:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184bd74640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1844070630 0x7f184415d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184bd74640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1844151bd0 con 0x7f18440aa630 2024-09-13T17:17:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184bd74640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440a8680 con 0x7f184414f580 2024-09-13T17:17:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184bd74640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1844061910 con 0x7f1844070630 2024-09-13T17:17:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184930f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f18440aa630 0x7f1844152320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41548/0 (socket says 172.21.15.39:41548) 2024-09-13T17:17:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184930f640 1 -- 172.21.15.39:0/796718159 learned_addr learned my addr 172.21.15.39:0/796718159 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184a311640 1 --2- 172.21.15.39:0/796718159 >> v2:172.21.15.84:3300/0 conn(0x7f1844070630 0x7f184415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548672754 0 0) 0x7f1844151bd0 con 0x7f18440aa630 2024-09-13T17:17:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1830003530 con 0x7f18440aa630 2024-09-13T17:17:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2813883301 0 0) 0x7f1830003530 con 0x7f18440aa630 2024-09-13T17:17:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 >> v2:172.21.15.84:3300/0 conn(0x7f1844070630 msgr2=0x7f184415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 --2- 172.21.15.39:0/796718159 >> v2:172.21.15.84:3300/0 conn(0x7f1844070630 0x7f184415d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 >> v1:172.21.15.84:6789/0 conn(0x7f184414f580 legacy=0x7f18440aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1844061630 con 0x7f18440aa630 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f1840001b80 con 0x7f18440aa630 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 >> v1:172.21.15.39:6789/0 conn(0x7f18440aa630 legacy=0x7f1844152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1830003980 0x7f1830003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f1830004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1830005190 0x7f183000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1844061910 con 0x7f18300047d0 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183000c130 con 0x7f1830003980 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183000c2f0 con 0x7f1830005190 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184930f640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f1830004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f1849b10640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1830003980 0x7f1830003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184a311640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1830005190 0x7f183000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184930f640 1 -- 172.21.15.39:0/796718159 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1830005190 msgr2=0x7f183000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184930f640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f1830005190 0x7f183000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184930f640 1 -- 172.21.15.39:0/796718159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1830003980 msgr2=0x7f1830003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184930f640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1830003980 0x7f1830003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184930f640 1 -- 172.21.15.39:0/796718159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830003530 con 0x7f18300047d0 2024-09-13T17:17:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.630+0000 7f184a311640 1 -- 172.21.15.39:0/796718159 reap_dead start 2024-09-13T17:17:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f184930f640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f1830004c50 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f184000c4f0 tx=0x7f184001cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b5bfc1d3ab20f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840005b30 con 0x7f18300047d0 2024-09-13T17:17:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1840005c90 con 0x7f18300047d0 2024-09-13T17:17:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f1848b0e640 1 -- 172.21.15.39:0/796718159 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18400a5d10 con 0x7f18300047d0 2024-09-13T17:17:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f184bd74640 1 -- 172.21.15.39:0/796718159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 msgr2=0x7f1830004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f184bd74640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f1830004c50 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f184000c4f0 tx=0x7f184001cf20 comp rx=0 tx=0).stop 2024-09-13T17:17:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f184bd74640 1 -- 172.21.15.39:0/796718159 shutdown_connections 2024-09-13T17:17:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f184bd74640 1 --2- 172.21.15.39:0/796718159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f1830004c50 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f184bd74640 1 -- 172.21.15.39:0/796718159 >> 172.21.15.39:0/796718159 conn(0x7f1844069e10 msgr2=0x7f184406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f184bd74640 1 -- 172.21.15.39:0/796718159 shutdown_connections 2024-09-13T17:17:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.634+0000 7f184bd74640 1 -- 172.21.15.39:0/796718159 wait complete. 2024-09-13T17:17:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 Processor -- start 2024-09-13T17:17:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 -- start start 2024-09-13T17:17:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f18441ec590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18440c55d0 0x7f18441e18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18440c4550 0x7f18441f0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18441efe70 con 0x7f18300047d0 2024-09-13T17:17:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184406ce40 con 0x7f18440c55d0 2024-09-13T17:17:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184406d120 con 0x7f18440c4550 2024-09-13T17:17:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f1849b10640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f18441ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f1849b10640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f18441ec590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42582/0 (socket says 172.21.15.39:42582) 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f1849b10640 1 -- 172.21.15.39:0/1398772501 learned_addr learned my addr 172.21.15.39:0/1398772501 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184a311640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18440c4550 0x7f18441f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184930f640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18440c55d0 0x7f18441e18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f1849b10640 1 -- 172.21.15.39:0/1398772501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18440c4550 msgr2=0x7f18441f0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f1849b10640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18440c4550 0x7f18441f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f1849b10640 1 -- 172.21.15.39:0/1398772501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18440c55d0 msgr2=0x7f18441e18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f1849b10640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18440c55d0 0x7f18441e18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f1849b10640 1 -- 172.21.15.39:0/1398772501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184406d400 con 0x7f18300047d0 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f1849b10640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f18441ec590 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f183800e4b0 tx=0x7f183800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f18367fc640 1 -- 172.21.15.39:0/1398772501 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18380052e0 con 0x7f18300047d0 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f18367fc640 1 -- 172.21.15.39:0/1398772501 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1838005440 con 0x7f18300047d0 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f18367fc640 1 -- 172.21.15.39:0/1398772501 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838003840 con 0x7f18300047d0 2024-09-13T17:17:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184406d6e0 con 0x7f18300047d0 2024-09-13T17:17:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18441e1e00 con 0x7f18300047d0 2024-09-13T17:17:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1844152320 con 0x7f18300047d0 2024-09-13T17:17:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f18367fc640 1 -- 172.21.15.39:0/1398772501 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18380039a0 con 0x7f18300047d0 2024-09-13T17:17:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f18367fc640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1828066690 0x7f1828068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f18367fc640 1 -- 172.21.15.39:0/1398772501 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f1838084fa0 con 0x7f18300047d0 2024-09-13T17:17:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.638+0000 7f184930f640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1828066690 0x7f1828068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.642+0000 7f184930f640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1828066690 0x7f1828068b50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f18400aa000 tx=0x7f18400b0040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.642+0000 7f18367fc640 1 -- 172.21.15.39:0/1398772501 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f18380b6db0 con 0x7f18300047d0 2024-09-13T17:17:24.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1844061370 con 0x7f18300047d0 2024-09-13T17:17:24.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f18367fc640 1 -- 172.21.15.39:0/1398772501 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v63) v1 ==== 99+0+11 (secure 0 0 0) 0x7f1838051290 con 0x7f18300047d0 2024-09-13T17:17:24.799 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T17:17:24.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1828066690 msgr2=0x7f1828068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1828066690 0x7f1828068b50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f18400aa000 tx=0x7f18400b0040 comp rx=0 tx=0).stop 2024-09-13T17:17:24.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 msgr2=0x7f18441ec590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f18441ec590 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f183800e4b0 tx=0x7f183800e970 comp rx=0 tx=0).stop 2024-09-13T17:17:24.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 shutdown_connections 2024-09-13T17:17:24.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f1828066690 0x7f1828068b50 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18440c55d0 0x7f18441e18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18440c4550 0x7f18441f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 --2- 172.21.15.39:0/1398772501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18300047d0 0x7f18441ec590 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 >> 172.21.15.39:0/1398772501 conn(0x7f1844069e10 msgr2=0x7f184414b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:24.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 shutdown_connections 2024-09-13T17:17:24.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.798+0000 7f184bd74640 1 -- 172.21.15.39:0/1398772501 wait complete. 2024-09-13T17:17:24.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T17:17:24.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf44141640 1 Processor -- start 2024-09-13T17:17:24.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf44141640 1 -- start start 2024-09-13T17:17:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf44141640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf3c0a4d60 0x7fdf3c14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf44141640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c155d20 con 0x7fdf3c14f4b0 2024-09-13T17:17:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf44141640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c0c4750 con 0x7fdf3c0aa2d0 2024-09-13T17:17:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf44141640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c155b30 con 0x7fdf3c0a4d60 2024-09-13T17:17:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf426de640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdf3c14f4b0 0x7fdf3c151cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41556/0 (socket says 172.21.15.39:41556) 2024-09-13T17:17:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf426de640 1 -- 172.21.15.39:0/3428119597 learned_addr learned my addr 172.21.15.39:0/3428119597 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf416dc640 1 --2- 172.21.15.39:0/3428119597 >> v2:172.21.15.84:3300/0 conn(0x7fdf3c0a4d60 0x7fdf3c14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558334663 0 0) 0x7fdf3c155d20 con 0x7fdf3c14f4b0 2024-09-13T17:17:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf24003530 con 0x7fdf3c14f4b0 2024-09-13T17:17:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2651195933 0 0) 0x7fdf24003530 con 0x7fdf3c14f4b0 2024-09-13T17:17:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 >> v2:172.21.15.84:3300/0 conn(0x7fdf3c0a4d60 msgr2=0x7fdf3c14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 --2- 172.21.15.39:0/3428119597 >> v2:172.21.15.84:3300/0 conn(0x7fdf3c0a4d60 0x7fdf3c14ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 >> v1:172.21.15.84:6789/0 conn(0x7fdf3c0aa2d0 legacy=0x7fdf3c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c155fc0 con 0x7fdf3c14f4b0 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fdf2c001680 con 0x7fdf3c14f4b0 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 >> v1:172.21.15.39:6789/0 conn(0x7fdf3c14f4b0 legacy=0x7fdf3c151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf24003980 0x7fdf24003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf24004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24005190 0x7fdf2400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2400c100 con 0x7fdf240047d0 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2400c270 con 0x7fdf24003980 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2400c430 con 0x7fdf24005190 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf416dc640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf24004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf41edd640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf24003980 0x7fdf24003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf426de640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24005190 0x7fdf2400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf416dc640 1 -- 172.21.15.39:0/3428119597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24005190 msgr2=0x7fdf2400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf416dc640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24005190 0x7fdf2400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf416dc640 1 -- 172.21.15.39:0/3428119597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf24003980 msgr2=0x7fdf24003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf416dc640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf24003980 0x7fdf24003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf416dc640 1 -- 172.21.15.39:0/3428119597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf24003530 con 0x7fdf240047d0 2024-09-13T17:17:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf426de640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf24005190 0x7fdf2400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.942+0000 7fdf426de640 1 -- 172.21.15.39:0/3428119597 reap_dead start 2024-09-13T17:17:24.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf416dc640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf24004c50 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009ae0 tx=0x7fdf3001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6bb8218c5dc28e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf300a6450 con 0x7fdf240047d0 2024-09-13T17:17:24.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf300a65b0 con 0x7fdf240047d0 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf40edb640 1 -- 172.21.15.39:0/3428119597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf300a6960 con 0x7fdf240047d0 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf44141640 1 -- 172.21.15.39:0/3428119597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 msgr2=0x7fdf24004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf44141640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf24004c50 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009ae0 tx=0x7fdf3001ca70 comp rx=0 tx=0).stop 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf44141640 1 -- 172.21.15.39:0/3428119597 shutdown_connections 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf44141640 1 --2- 172.21.15.39:0/3428119597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf24004c50 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf44141640 1 -- 172.21.15.39:0/3428119597 >> 172.21.15.39:0/3428119597 conn(0x7fdf3c0c5790 msgr2=0x7fdf3c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf44141640 1 -- 172.21.15.39:0/3428119597 shutdown_connections 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf44141640 1 -- 172.21.15.39:0/3428119597 wait complete. 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.946+0000 7fdf44141640 1 Processor -- start 2024-09-13T17:17:24.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 -- start start 2024-09-13T17:17:24.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf3c0a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf3c0c0aa0 0x7fdf3c0a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3c141800 0x7fdf3c141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0c10e0 con 0x7fdf240047d0 2024-09-13T17:17:24.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0a7e70 con 0x7fdf3c0c0aa0 2024-09-13T17:17:24.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0a6070 con 0x7fdf3c141800 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf41edd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf3c0a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf41edd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf3c0a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42598/0 (socket says 172.21.15.39:42598) 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf41edd640 1 -- 172.21.15.39:0/649901396 learned_addr learned my addr 172.21.15.39:0/649901396 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf416dc640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf3c0c0aa0 0x7fdf3c0a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf426de640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3c141800 0x7fdf3c141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf41edd640 1 -- 172.21.15.39:0/649901396 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3c141800 msgr2=0x7fdf3c141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf41edd640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3c141800 0x7fdf3c141c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf41edd640 1 -- 172.21.15.39:0/649901396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf3c0c0aa0 msgr2=0x7fdf3c0a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf41edd640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf3c0c0aa0 0x7fdf3c0a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf41edd640 1 -- 172.21.15.39:0/649901396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0a5d90 con 0x7fdf240047d0 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf41edd640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf3c0a66b0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400e900 tx=0x7fdf3400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf3a7fc640 1 -- 172.21.15.39:0/649901396 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34003b90 con 0x7fdf240047d0 2024-09-13T17:17:24.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf3a7fc640 1 -- 172.21.15.39:0/649901396 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf34003cf0 con 0x7fdf240047d0 2024-09-13T17:17:24.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c0a7510 con 0x7fdf240047d0 2024-09-13T17:17:24.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf3a7fc640 1 -- 172.21.15.39:0/649901396 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34005310 con 0x7fdf240047d0 2024-09-13T17:17:24.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c0a7b00 con 0x7fdf240047d0 2024-09-13T17:17:24.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c152830 con 0x7fdf240047d0 2024-09-13T17:17:24.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf3a7fc640 1 -- 172.21.15.39:0/649901396 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf34005470 con 0x7fdf240047d0 2024-09-13T17:17:24.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf3a7fc640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf28066690 0x7fdf28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:24.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf3a7fc640 1 -- 172.21.15.39:0/649901396 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fdf34085370 con 0x7fdf240047d0 2024-09-13T17:17:24.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.950+0000 7fdf416dc640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf28066690 0x7fdf28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:24.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.954+0000 7fdf416dc640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf28066690 0x7fdf28068b50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdf300ad000 tx=0x7fdf300046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:24.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:24.954+0000 7fdf3a7fc640 1 -- 172.21.15.39:0/649901396 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf34051660 con 0x7fdf240047d0 2024-09-13T17:17:25.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fdf3c14e8b0 con 0x7fdf240047d0 2024-09-13T17:17:25.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf3a7fc640 1 -- 172.21.15.39:0/649901396 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v63) v1 ==== 95+0+11 (secure 0 0 0) 0x7fdf34003e60 con 0x7fdf240047d0 2024-09-13T17:17:25.119 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T17:17:25.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf28066690 msgr2=0x7fdf28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:25.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf28066690 0x7fdf28068b50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdf300ad000 tx=0x7fdf300046d0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 msgr2=0x7fdf3c0a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf3c0a66b0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400e900 tx=0x7fdf3400edc0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 shutdown_connections 2024-09-13T17:17:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdf28066690 0x7fdf28068b50 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf3c0c0aa0 0x7fdf3c0a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdf3c141800 0x7fdf3c141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 --2- 172.21.15.39:0/649901396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf240047d0 0x7fdf3c0a66b0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 >> 172.21.15.39:0/649901396 conn(0x7fdf3c0c5790 msgr2=0x7fdf3c0c3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 shutdown_connections 2024-09-13T17:17:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.118+0000 7fdf44141640 1 -- 172.21.15.39:0/649901396 wait complete. 2024-09-13T17:17:25.140 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:17:25.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd36df9c640 1 Processor -- start 2024-09-13T17:17:25.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd36df9c640 1 -- start start 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd36df9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368071dd0 0x7fd368154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd36df9c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd3680c7800 0x7fd3681545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd36df9c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3680c5a30 con 0x7fd368151500 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd36df9c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3681576a0 con 0x7fd3680c7800 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd36df9c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd368157980 con 0x7fd368071dd0 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd366ffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368071dd0 0x7fd368154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd367fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd3680c7800 0x7fd3681545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd367fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd3680c7800 0x7fd3681545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:54470/0 (socket says 172.21.15.39:54470) 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd367fff640 1 -- 172.21.15.39:0/1276470371 learned_addr learned my addr 172.21.15.39:0/1276470371 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd366ffd640 1 -- 172.21.15.39:0/1276470371 >> v2:172.21.15.84:3300/0 conn(0x7fd3680c7800 msgr2=0x7fd3681545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd366ffd640 1 --2- 172.21.15.39:0/1276470371 >> v2:172.21.15.84:3300/0 conn(0x7fd3680c7800 0x7fd3681545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd366ffd640 1 -- 172.21.15.39:0/1276470371 >> v1:172.21.15.84:6789/0 conn(0x7fd368151500 legacy=0x7fd3680c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:25.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.258+0000 7fd366ffd640 1 -- 172.21.15.39:0/1276470371 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd368157c60 con 0x7fd368071dd0 2024-09-13T17:17:25.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd366ffd640 1 --2- 172.21.15.39:0/1276470371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368071dd0 0x7fd368154070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097c0 tx=0x7fd35c01ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddb4ec54ab3f437a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:25.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd3667fc640 1 -- 172.21.15.39:0/1276470371 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0ab600 con 0x7fd368071dd0 2024-09-13T17:17:25.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd3667fc640 1 -- 172.21.15.39:0/1276470371 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c0ab760 con 0x7fd368071dd0 2024-09-13T17:17:25.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd3667fc640 1 -- 172.21.15.39:0/1276470371 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0abaf0 con 0x7fd368071dd0 2024-09-13T17:17:25.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 -- 172.21.15.39:0/1276470371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368071dd0 msgr2=0x7fd368154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:25.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 --2- 172.21.15.39:0/1276470371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368071dd0 0x7fd368154070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0097c0 tx=0x7fd35c01ce40 comp rx=0 tx=0).stop 2024-09-13T17:17:25.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 -- 172.21.15.39:0/1276470371 shutdown_connections 2024-09-13T17:17:25.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 --2- 172.21.15.39:0/1276470371 >> v2:172.21.15.84:3300/0 conn(0x7fd3680c7800 0x7fd3681545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 --2- 172.21.15.39:0/1276470371 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368071dd0 0x7fd368154070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 -- 172.21.15.39:0/1276470371 >> 172.21.15.39:0/1276470371 conn(0x7fd3680ab530 msgr2=0x7fd3680ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:25.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 -- 172.21.15.39:0/1276470371 shutdown_connections 2024-09-13T17:17:25.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 -- 172.21.15.39:0/1276470371 wait complete. 2024-09-13T17:17:25.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 Processor -- start 2024-09-13T17:17:25.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 -- start start 2024-09-13T17:17:25.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd368071dd0 0x7fd3681495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:25.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3680c7800 0x7fd3681efef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:25.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368151500 0x7fd3681f0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:25.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36814bcc0 con 0x7fd3680c7800 2024-09-13T17:17:25.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3681529a0 con 0x7fd368071dd0 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd36df9c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3681ec5a0 con 0x7fd368151500 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd366ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3680c7800 0x7fd3681efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd366ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3680c7800 0x7fd3681efef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42602/0 (socket says 172.21.15.39:42602) 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd366ffd640 1 -- 172.21.15.39:0/946305758 learned_addr learned my addr 172.21.15.39:0/946305758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd367fff640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368151500 0x7fd3681f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.262+0000 7fd3677fe640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd368071dd0 0x7fd3681495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd366ffd640 1 -- 172.21.15.39:0/946305758 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368151500 msgr2=0x7fd3681f0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd366ffd640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368151500 0x7fd3681f0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd366ffd640 1 -- 172.21.15.39:0/946305758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd368071dd0 msgr2=0x7fd3681495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd366ffd640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd368071dd0 0x7fd3681495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd366ffd640 1 -- 172.21.15.39:0/946305758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c009420 con 0x7fd3680c7800 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd366ffd640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3680c7800 0x7fd3681efef0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd35c01aa90 tx=0x7fd35c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd34ffff640 1 -- 172.21.15.39:0/946305758 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0bf610 con 0x7fd3680c7800 2024-09-13T17:17:25.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd34ffff640 1 -- 172.21.15.39:0/946305758 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c0bf770 con 0x7fd3680c7800 2024-09-13T17:17:25.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd368152bd0 con 0x7fd3680c7800 2024-09-13T17:17:25.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd34ffff640 1 -- 172.21.15.39:0/946305758 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0bfb20 con 0x7fd3680c7800 2024-09-13T17:17:25.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd368153190 con 0x7fd3680c7800 2024-09-13T17:17:25.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3680c4280 con 0x7fd3680c7800 2024-09-13T17:17:25.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd34ffff640 1 -- 172.21.15.39:0/946305758 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd35c0b5400 con 0x7fd3680c7800 2024-09-13T17:17:25.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd34ffff640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd354066690 0x7fd354068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:25.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd3677fe640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd354066690 0x7fd354068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:25.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd34ffff640 1 -- 172.21.15.39:0/946305758 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fd35c11fbd0 con 0x7fd3680c7800 2024-09-13T17:17:25.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.266+0000 7fd3677fe640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd354066690 0x7fd354068b50 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd3580012f0 tx=0x7fd358009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:25.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.270+0000 7fd34ffff640 1 -- 172.21.15.39:0/946305758 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd35c0ebe30 con 0x7fd3680c7800 2024-09-13T17:17:25.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.442+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd368071bb0 con 0x7fd3680c7800 2024-09-13T17:17:25.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.442+0000 7fd34ffff640 1 -- 172.21.15.39:0/946305758 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 78 entries v63) v1 ==== 81+0+4738 (secure 0 0 0) 0x7fd35c0ef450 con 0x7fd3680c7800 2024-09-13T17:17:25.446 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/380008541 2024-09-14T17:16:34.950555+0000 2024-09-13T17:17:25.446 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3721635324 2024-09-14T17:16:04.944309+0000 2024-09-13T17:17:25.446 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1355102409 2024-09-14T17:15:29.935985+0000 2024-09-13T17:17:25.446 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/305129652 2024-09-14T17:14:49.920017+0000 2024-09-13T17:17:25.446 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/282472862 2024-09-14T17:14:04.899010+0000 2024-09-13T17:17:25.446 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/397460031 2024-09-14T17:12:19.869778+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/397460031 2024-09-14T17:12:19.869778+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/2351253009 2024-09-14T17:11:09.852639+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3334565883 2024-09-14T17:11:59.865711+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3923043380 2024-09-14T17:09:59.836463+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1334276441 2024-09-14T16:59:04.677958+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/1326467723 2024-09-14T17:00:02.030979+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1334276441 2024-09-14T16:59:04.677958+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4200926406 2024-09-14T17:05:59.776545+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1392182763 2024-09-14T17:07:19.797311+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/3122796453 2024-09-14T16:59:54.690333+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/1326467723 2024-09-14T17:00:02.030979+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2740003107 2024-09-14T16:57:14.654506+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1355102409 2024-09-14T17:15:29.935985+0000 2024-09-13T17:17:25.447 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/1969975539 2024-09-14T16:52:59.589819+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3721635324 2024-09-14T17:16:04.944309+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/2351253009 2024-09-14T17:11:09.852639+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2144264453 2024-09-14T16:52:24.581694+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4094657757 2024-09-14T16:56:04.630964+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3600322972 2024-09-14T16:55:14.618334+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/380008541 2024-09-14T17:16:34.950555+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1572373831 2024-09-14T16:51:14.567493+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4246622083 2024-09-14T17:01:24.709561+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/282472862 2024-09-14T17:14:04.899010+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1355376480 2024-09-14T16:54:09.605050+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3334565883 2024-09-14T17:11:59.865711+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3941977343 2024-09-14T17:08:49.820117+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/2521680730 2024-09-14T17:09:29.828780+0000 2024-09-13T17:17:25.448 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/661906662 2024-09-14T16:57:59.663485+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4094657757 2024-09-14T16:56:04.630964+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3540656099 2024-09-14T16:50:14.555020+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2065196144 2024-09-14T16:53:14.593565+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1750130579 2024-09-14T17:09:29.828673+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3163580917 2024-09-14T16:49:44.548920+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1355376480 2024-09-14T16:54:09.605050+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2065196144 2024-09-14T16:53:14.593565+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1392588044 2024-09-14T16:50:49.562750+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1034701751 2024-09-14T17:10:39.845194+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/3122796453 2024-09-14T16:59:54.690333+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/3105501456 2024-09-14T17:03:49.742996+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/2521680730 2024-09-14T17:09:29.828780+0000 2024-09-13T17:17:25.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/661906662 2024-09-14T16:57:59.663485+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1572373831 2024-09-14T16:51:14.567493+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/1969975539 2024-09-14T16:52:59.589819+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1392588044 2024-09-14T16:50:49.562750+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2144264453 2024-09-14T16:52:24.581694+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3163580917 2024-09-14T16:49:44.548920+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3941977343 2024-09-14T17:08:49.820117+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2638959497 2024-09-14T17:01:24.709473+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2740003107 2024-09-14T16:57:14.654506+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4246622083 2024-09-14T17:01:24.709561+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/305129652 2024-09-14T17:14:49.920017+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/835951775 2024-09-14T17:02:59.730250+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/828722052 2024-09-14T17:03:19.734465+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2638959497 2024-09-14T17:01:24.709473+0000 2024-09-13T17:17:25.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/828722052 2024-09-14T17:03:19.734465+0000 2024-09-13T17:17:25.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3600322972 2024-09-14T16:55:14.618334+0000 2024-09-13T17:17:25.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/611669592 2024-09-14T17:04:24.751785+0000 2024-09-13T17:17:25.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/611669592 2024-09-14T17:04:24.751785+0000 2024-09-13T17:17:25.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1590039204 2024-09-14T17:05:09.765210+0000 2024-09-13T17:17:25.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4200926406 2024-09-14T17:05:59.776545+0000 2024-09-13T17:17:25.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/440028459 2024-09-14T17:07:34.801438+0000 2024-09-13T17:17:25.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/3105501456 2024-09-14T17:03:49.742996+0000 2024-09-13T17:17:25.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1590039204 2024-09-14T17:05:09.765210+0000 2024-09-13T17:17:25.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/263561599 2024-09-14T17:05:54.774622+0000 2024-09-13T17:17:25.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3923043380 2024-09-14T17:09:59.836463+0000 2024-09-13T17:17:25.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/263561599 2024-09-14T17:05:54.774622+0000 2024-09-13T17:17:25.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1392182763 2024-09-14T17:07:19.797311+0000 2024-09-13T17:17:25.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/835951775 2024-09-14T17:02:59.730250+0000 2024-09-13T17:17:25.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3540656099 2024-09-14T16:50:14.555020+0000 2024-09-13T17:17:25.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/440028459 2024-09-14T17:07:34.801438+0000 2024-09-13T17:17:25.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1034701751 2024-09-14T17:10:39.845194+0000 2024-09-13T17:17:25.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1750130579 2024-09-14T17:09:29.828673+0000 2024-09-13T17:17:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:listed 78 entries 2024-09-13T17:17:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd354066690 msgr2=0x7fd354068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd354066690 0x7fd354068b50 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd3580012f0 tx=0x7fd358009040 comp rx=0 tx=0).stop 2024-09-13T17:17:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3680c7800 msgr2=0x7fd3681efef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3680c7800 0x7fd3681efef0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd35c01aa90 tx=0x7fd35c01ac20 comp rx=0 tx=0).stop 2024-09-13T17:17:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 shutdown_connections 2024-09-13T17:17:25.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd354066690 0x7fd354068b50 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd368151500 0x7fd3681f0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3680c7800 0x7fd3681efef0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 --2- 172.21.15.39:0/946305758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd368071dd0 0x7fd3681495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 >> 172.21.15.39:0/946305758 conn(0x7fd3680ab530 msgr2=0x7fd3680ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 shutdown_connections 2024-09-13T17:17:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:25.446+0000 7fd36df9c640 1 -- 172.21.15.39:0/946305758 wait complete. 2024-09-13T17:17:25.464 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi039.front.sepia.ceph.com... 2024-09-13T17:17:25.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:25.465 DEBUG:teuthology.orchestra.run.smithi039:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:17:25.492 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:17:25.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi039.stderr:ceph-fuse[36878]: fuse finished with error 0 and tester_r 0 2024-09-13T17:17:28.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:28.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff28277d640 1 Processor -- start 2024-09-13T17:17:28.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff28277d640 1 -- start start 2024-09-13T17:17:28.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff28277d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c0a4e70 0x7ff27c151460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff28277d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff27c0c4530 0x7ff27c0c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff28277d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27c154100 con 0x7ff27c06d020 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff28277d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c0a8c50 con 0x7ff27c0c4530 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff28277d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1519a0 con 0x7ff27c0a4e70 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff280d1a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff27c0c4530 0x7ff27c0c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff27bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff27c06d020 0x7ff27c14db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41560/0 (socket says 172.21.15.39:41560) 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff27bfff640 1 -- 172.21.15.39:0/2571780116 learned_addr learned my addr 172.21.15.39:0/2571780116 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff27b7fe640 1 --2- 172.21.15.39:0/2571780116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c0a4e70 0x7ff27c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff280d1a640 1 -- 172.21.15.39:0/2571780116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c0a4e70 msgr2=0x7ff27c151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff280d1a640 1 --2- 172.21.15.39:0/2571780116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c0a4e70 0x7ff27c151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff280d1a640 1 -- 172.21.15.39:0/2571780116 >> v1:172.21.15.39:6789/0 conn(0x7ff27c06d020 legacy=0x7ff27c14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff280d1a640 1 -- 172.21.15.39:0/2571780116 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c1543e0 con 0x7ff27c0c4530 2024-09-13T17:17:28.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.270+0000 7ff27b7fe640 1 --2- 172.21.15.39:0/2571780116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c0a4e70 0x7ff27c151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:28.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff280d1a640 1 --2- 172.21.15.39:0/2571780116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0c4530 0x7ff27c0c49a0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7ff2740097c0 tx=0x7ff27401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75b976c3ef9c8ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:28.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27affd640 1 -- 172.21.15.39:0/2571780116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740ab600 con 0x7ff27c0c4530 2024-09-13T17:17:28.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27affd640 1 -- 172.21.15.39:0/2571780116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2740a7050 con 0x7ff27c0c4530 2024-09-13T17:17:28.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27affd640 1 -- 172.21.15.39:0/2571780116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740ab980 con 0x7ff27c0c4530 2024-09-13T17:17:28.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 -- 172.21.15.39:0/2571780116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0c4530 msgr2=0x7ff27c0c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 --2- 172.21.15.39:0/2571780116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0c4530 0x7ff27c0c49a0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7ff2740097c0 tx=0x7ff27401c410 comp rx=0 tx=0).stop 2024-09-13T17:17:28.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 -- 172.21.15.39:0/2571780116 shutdown_connections 2024-09-13T17:17:28.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 --2- 172.21.15.39:0/2571780116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0c4530 0x7ff27c0c49a0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 --2- 172.21.15.39:0/2571780116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c0a4e70 0x7ff27c151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 -- 172.21.15.39:0/2571780116 >> 172.21.15.39:0/2571780116 conn(0x7ff27c069e10 msgr2=0x7ff27c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:28.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 -- 172.21.15.39:0/2571780116 shutdown_connections 2024-09-13T17:17:28.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 -- 172.21.15.39:0/2571780116 wait complete. 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 Processor -- start 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 -- start start 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c06d020 0x7ff27c14ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0a4e70 0x7ff27c146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff27c0c4530 0x7ff27c147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c1ec690 con 0x7ff27c0a4e70 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c15c370 con 0x7ff27c0c4530 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff28277d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c15c650 con 0x7ff27c06d020 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0a4e70 0x7ff27c146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0a4e70 0x7ff27c146ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42612/0 (socket says 172.21.15.39:42612) 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27b7fe640 1 -- 172.21.15.39:0/1259467851 learned_addr learned my addr 172.21.15.39:0/1259467851 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27bfff640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c06d020 0x7ff27c14ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff280d1a640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff27c0c4530 0x7ff27c147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27b7fe640 1 -- 172.21.15.39:0/1259467851 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c06d020 msgr2=0x7ff27c14ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27b7fe640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c06d020 0x7ff27c14ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27b7fe640 1 -- 172.21.15.39:0/1259467851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff27c0c4530 msgr2=0x7ff27c147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27b7fe640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff27c0c4530 0x7ff27c147000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.274+0000 7ff27b7fe640 1 -- 172.21.15.39:0/1259467851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274009420 con 0x7ff27c0a4e70 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff27b7fe640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0a4e70 0x7ff27c146ac0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff27000ce00 tx=0x7ff27000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff278ff9640 1 -- 172.21.15.39:0/1259467851 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000f680 con 0x7ff27c0a4e70 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff278ff9640 1 -- 172.21.15.39:0/1259467851 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27000f7e0 con 0x7ff27c0a4e70 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c0bf820 con 0x7ff27c0a4e70 2024-09-13T17:17:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff278ff9640 1 -- 172.21.15.39:0/1259467851 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000fbb0 con 0x7ff27c0a4e70 2024-09-13T17:17:28.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c1ec8a0 con 0x7ff27c0a4e70 2024-09-13T17:17:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27c151460 con 0x7ff27c0a4e70 2024-09-13T17:17:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff278ff9640 1 -- 172.21.15.39:0/1259467851 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff27000b040 con 0x7ff27c0a4e70 2024-09-13T17:17:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff278ff9640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff2680666e0 0x7ff268068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff278ff9640 1 -- 172.21.15.39:0/1259467851 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7ff270085bc0 con 0x7ff27c0a4e70 2024-09-13T17:17:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff27bfff640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff2680666e0 0x7ff268068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.278+0000 7ff27bfff640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff2680666e0 0x7ff268068ba0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff26c005890 tx=0x7ff26c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.282+0000 7ff278ff9640 1 -- 172.21.15.39:0/1259467851 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff270051fe0 con 0x7ff27c0a4e70 2024-09-13T17:17:28.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.482+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff27c0c14d0 con 0x7ff27c0a4e70 2024-09-13T17:17:28.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.482+0000 7ff278ff9640 1 -- 172.21.15.39:0/1259467851 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5936 (secure 0 0 0) 0x7ff270055600 con 0x7ff27c0a4e70 2024-09-13T17:17:28.484 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:28.484 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:28.485 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-13T17:17:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff2680666e0 msgr2=0x7ff268068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff2680666e0 0x7ff268068ba0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff26c005890 tx=0x7ff26c009040 comp rx=0 tx=0).stop 2024-09-13T17:17:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0a4e70 msgr2=0x7ff27c146ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0a4e70 0x7ff27c146ac0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff27000ce00 tx=0x7ff27000a650 comp rx=0 tx=0).stop 2024-09-13T17:17:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 shutdown_connections 2024-09-13T17:17:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff2680666e0 0x7ff268068ba0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff27c0c4530 0x7ff27c147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff27c0a4e70 0x7ff27c146ac0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 --2- 172.21.15.39:0/1259467851 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff27c06d020 0x7ff27c14ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 >> 172.21.15.39:0/1259467851 conn(0x7ff27c069e10 msgr2=0x7ff27c14caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 shutdown_connections 2024-09-13T17:17:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.486+0000 7ff28277d640 1 -- 172.21.15.39:0/1259467851 wait complete. 2024-09-13T17:17:28.505 INFO:tasks.mds_thrash.fs.[a]:kill mds.a (rank=0) 2024-09-13T17:17:28.505 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:17:28.505 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:17:28.516 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:17:28.516 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae7eed640 1 Processor -- start 2024-09-13T17:17:28.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae7eed640 1 -- start start 2024-09-13T17:17:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae7eed640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcae01526b0 0x7fcae00c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae7eed640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae014f790 con 0x7fcae006c940 2024-09-13T17:17:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae7eed640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae0152a90 con 0x7fcae01555f0 2024-09-13T17:17:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae7eed640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae006ce40 con 0x7fcae01526b0 2024-09-13T17:17:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae5488640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcae006c940 0x7fcae00aabc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:41576/0 (socket says 172.21.15.39:41576) 2024-09-13T17:17:28.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae5488640 1 -- 172.21.15.39:0/3005123425 learned_addr learned my addr 172.21.15.39:0/3005123425 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:28.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae5c89640 1 --2- 172.21.15.39:0/3005123425 >> v2:172.21.15.39:3300/0 conn(0x7fcae01526b0 0x7fcae00c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae4c87640 1 -- 172.21.15.39:0/3005123425 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186874769 0 0) 0x7fcae014f790 con 0x7fcae006c940 2024-09-13T17:17:28.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae4c87640 1 -- 172.21.15.39:0/3005123425 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcacc003530 con 0x7fcae006c940 2024-09-13T17:17:28.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae5c89640 1 -- 172.21.15.39:0/3005123425 >> v1:172.21.15.84:6789/0 conn(0x7fcae01555f0 legacy=0x7fcae0155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae5c89640 1 -- 172.21.15.39:0/3005123425 >> v1:172.21.15.39:6789/0 conn(0x7fcae006c940 legacy=0x7fcae00aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.686+0000 7fcae5c89640 1 -- 172.21.15.39:0/3005123425 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae006d120 con 0x7fcae01526b0 2024-09-13T17:17:28.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae5c89640 1 --2- 172.21.15.39:0/3005123425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 0x7fcae00c12b0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0097c0 tx=0x7fcadc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a9e580518fe341 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:28.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae4c87640 1 -- 172.21.15.39:0/3005123425 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0a8050 con 0x7fcae01526b0 2024-09-13T17:17:28.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae4c87640 1 -- 172.21.15.39:0/3005123425 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc0ac5f0 con 0x7fcae01526b0 2024-09-13T17:17:28.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae4c87640 1 -- 172.21.15.39:0/3005123425 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc0ac980 con 0x7fcae01526b0 2024-09-13T17:17:28.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae7eed640 1 -- 172.21.15.39:0/3005123425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 msgr2=0x7fcae00c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae7eed640 1 --2- 172.21.15.39:0/3005123425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 0x7fcae00c12b0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0097c0 tx=0x7fcadc01c4a0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae7eed640 1 -- 172.21.15.39:0/3005123425 shutdown_connections 2024-09-13T17:17:28.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae7eed640 1 --2- 172.21.15.39:0/3005123425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 0x7fcae00c12b0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae7eed640 1 -- 172.21.15.39:0/3005123425 >> 172.21.15.39:0/3005123425 conn(0x7fcae0069e10 msgr2=0x7fcae006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae7eed640 1 -- 172.21.15.39:0/3005123425 shutdown_connections 2024-09-13T17:17:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae7eed640 1 -- 172.21.15.39:0/3005123425 wait complete. 2024-09-13T17:17:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae7eed640 1 Processor -- start 2024-09-13T17:17:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.690+0000 7fcae7eed640 1 -- start start 2024-09-13T17:17:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae7eed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcae006c940 0x7fcae0147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae7eed640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 0x7fcae01ece00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae7eed640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcae01555f0 0x7fcae00bec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae7eed640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae014f000 con 0x7fcae01526b0 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae7eed640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00a4f00 con 0x7fcae006c940 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae7eed640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01ec420 con 0x7fcae01555f0 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5488640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 0x7fcae01ece00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5488640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 0x7fcae01ece00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42634/0 (socket says 172.21.15.39:42634) 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5488640 1 -- 172.21.15.39:0/1053287995 learned_addr learned my addr 172.21.15.39:0/1053287995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae648a640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcae01555f0 0x7fcae00bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5c89640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcae006c940 0x7fcae0147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5488640 1 -- 172.21.15.39:0/1053287995 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcae01555f0 msgr2=0x7fcae00bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5488640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcae01555f0 0x7fcae00bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5488640 1 -- 172.21.15.39:0/1053287995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcae006c940 msgr2=0x7fcae0147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5488640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcae006c940 0x7fcae0147f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5488640 1 -- 172.21.15.39:0/1053287995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc009420 con 0x7fcae01526b0 2024-09-13T17:17:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae648a640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcae01555f0 0x7fcae00bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5488640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 0x7fcae01ece00 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c490 tx=0x7fcad000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcad67fc640 1 -- 172.21.15.39:0/1053287995 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0005cf0 con 0x7fcae01526b0 2024-09-13T17:17:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcad67fc640 1 -- 172.21.15.39:0/1053287995 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad0005e50 con 0x7fcae01526b0 2024-09-13T17:17:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae0072350 con 0x7fcae01526b0 2024-09-13T17:17:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcad67fc640 1 -- 172.21.15.39:0/1053287995 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0007270 con 0x7fcae01526b0 2024-09-13T17:17:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae014e760 con 0x7fcae01526b0 2024-09-13T17:17:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae00a5160 con 0x7fcae01526b0 2024-09-13T17:17:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcad67fc640 1 -- 172.21.15.39:0/1053287995 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcad00073d0 con 0x7fcae01526b0 2024-09-13T17:17:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcad67fc640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcac4066690 0x7fcac4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcae5c89640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcac4066690 0x7fcac4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.694+0000 7fcad67fc640 1 -- 172.21.15.39:0/1053287995 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fcad00846b0 con 0x7fcae01526b0 2024-09-13T17:17:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.698+0000 7fcae5c89640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcac4066690 0x7fcac4068b50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0ae000 tx=0x7fcadc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:28.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.698+0000 7fcad67fc640 1 -- 172.21.15.39:0/1053287995 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcad00509a0 con 0x7fcae01526b0 2024-09-13T17:17:28.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.894+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcae006c670 con 0x7fcae01526b0 2024-09-13T17:17:28.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.894+0000 7fcad67fc640 1 -- 172.21.15.39:0/1053287995 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5936 (secure 0 0 0) 0x7fcad0017090 con 0x7fcae01526b0 2024-09-13T17:17:28.898 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:28.898 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:28.898 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-13T17:17:28.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcac4066690 msgr2=0x7fcac4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcac4066690 0x7fcac4068b50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0ae000 tx=0x7fcadc008040 comp rx=0 tx=0).stop 2024-09-13T17:17:28.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 msgr2=0x7fcae01ece00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:28.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 0x7fcae01ece00 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c490 tx=0x7fcad000c950 comp rx=0 tx=0).stop 2024-09-13T17:17:28.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 shutdown_connections 2024-09-13T17:17:28.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcac4066690 0x7fcac4068b50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcae01555f0 0x7fcae00bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcae01526b0 0x7fcae01ece00 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 --2- 172.21.15.39:0/1053287995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcae006c940 0x7fcae0147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:28.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 >> 172.21.15.39:0/1053287995 conn(0x7fcae0069e10 msgr2=0x7fcae00c1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:28.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 shutdown_connections 2024-09-13T17:17:28.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:28.898+0000 7fcae7eed640 1 -- 172.21.15.39:0/1053287995 wait complete. 2024-09-13T17:17:28.917 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:17:30.917 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:31.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89d5c39640 1 Processor -- start 2024-09-13T17:17:31.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89d5c39640 1 -- start start 2024-09-13T17:17:31.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89d5c39640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f89d00c20d0 0x7f89d00c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:31.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89d5c39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f89d00c1790 0x7f89d0157800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89d5c39640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d0157370 con 0x7f89d006d370 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89d5c39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d00c04a0 con 0x7f89d00c1790 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89d5c39640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d0153df0 con 0x7f89d00c20d0 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89ceffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f89d00c1790 0x7f89d0157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89ceffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f89d00c1790 0x7f89d0157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54700/0 (socket says 172.21.15.39:54700) 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89ceffd640 1 -- 172.21.15.39:0/2062893056 learned_addr learned my addr 172.21.15.39:0/2062893056 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89cf7fe640 1 --2- 172.21.15.39:0/2062893056 >> v2:172.21.15.84:3300/0 conn(0x7f89d00c20d0 0x7f89d00c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89ceffd640 1 -- 172.21.15.39:0/2062893056 >> v2:172.21.15.84:3300/0 conn(0x7f89d00c20d0 msgr2=0x7f89d00c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89ceffd640 1 --2- 172.21.15.39:0/2062893056 >> v2:172.21.15.84:3300/0 conn(0x7f89d00c20d0 0x7f89d00c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89ceffd640 1 -- 172.21.15.39:0/2062893056 >> v1:172.21.15.84:6789/0 conn(0x7f89d006d370 legacy=0x7f89d0157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:31.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.050+0000 7f89ceffd640 1 -- 172.21.15.39:0/2062893056 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d006c700 con 0x7f89d00c1790 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89ceffd640 1 --2- 172.21.15.39:0/2062893056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 0x7f89d0157800 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f89d00c2f50 tx=0x7f89b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7410331256d2fa25 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89ce7fc640 1 -- 172.21.15.39:0/2062893056 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b80aa650 con 0x7f89d00c1790 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89ce7fc640 1 -- 172.21.15.39:0/2062893056 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b80aa7b0 con 0x7f89d00c1790 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89ce7fc640 1 -- 172.21.15.39:0/2062893056 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b80aab40 con 0x7f89d00c1790 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89d5c39640 1 -- 172.21.15.39:0/2062893056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 msgr2=0x7f89d0157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89d5c39640 1 --2- 172.21.15.39:0/2062893056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 0x7f89d0157800 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f89d00c2f50 tx=0x7f89b801c4f0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89d5c39640 1 -- 172.21.15.39:0/2062893056 shutdown_connections 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89d5c39640 1 --2- 172.21.15.39:0/2062893056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 0x7f89d0157800 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89d5c39640 1 --2- 172.21.15.39:0/2062893056 >> v2:172.21.15.84:3300/0 conn(0x7f89d00c20d0 0x7f89d00c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89d5c39640 1 -- 172.21.15.39:0/2062893056 >> 172.21.15.39:0/2062893056 conn(0x7f89d0069e10 msgr2=0x7f89d006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89d5c39640 1 -- 172.21.15.39:0/2062893056 shutdown_connections 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.054+0000 7f89d5c39640 1 -- 172.21.15.39:0/2062893056 wait complete. 2024-09-13T17:17:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 Processor -- start 2024-09-13T17:17:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 -- start start 2024-09-13T17:17:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89d006d370 0x7f89d0152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 0x7f89d014a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f89d00c20d0 0x7f89d014aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01ecbe0 con 0x7f89d00c1790 2024-09-13T17:17:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01ed110 con 0x7f89d006d370 2024-09-13T17:17:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d01ece30 con 0x7f89d00c20d0 2024-09-13T17:17:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89ceffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 0x7f89d014a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89ceffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 0x7f89d014a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54706/0 (socket says 172.21.15.39:54706) 2024-09-13T17:17:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89ceffd640 1 -- 172.21.15.39:0/1978086234 learned_addr learned my addr 172.21.15.39:0/1978086234 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89cf7fe640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89d006d370 0x7f89d0152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89cffff640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f89d00c20d0 0x7f89d014aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89ceffd640 1 -- 172.21.15.39:0/1978086234 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f89d00c20d0 msgr2=0x7f89d014aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89ceffd640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f89d00c20d0 0x7f89d014aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89ceffd640 1 -- 172.21.15.39:0/1978086234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89d006d370 msgr2=0x7f89d0152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89ceffd640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89d006d370 0x7f89d0152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89ceffd640 1 -- 172.21.15.39:0/1978086234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b80094d0 con 0x7f89d00c1790 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89ceffd640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 0x7f89d014a4f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f89b80ac040 tx=0x7f89b801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89c7fff640 1 -- 172.21.15.39:0/1978086234 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b80b3800 con 0x7f89d00c1790 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89c7fff640 1 -- 172.21.15.39:0/1978086234 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b80b3960 con 0x7f89d00c1790 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d00a81f0 con 0x7f89d00c1790 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89c7fff640 1 -- 172.21.15.39:0/1978086234 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b80b3cf0 con 0x7f89d00c1790 2024-09-13T17:17:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d014af70 con 0x7f89d00c1790 2024-09-13T17:17:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d00c24b0 con 0x7f89d00c1790 2024-09-13T17:17:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89c7fff640 1 -- 172.21.15.39:0/1978086234 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f89b80c6440 con 0x7f89d00c1790 2024-09-13T17:17:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89c7fff640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f89b0066690 0x7f89b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89c7fff640 1 -- 172.21.15.39:0/1978086234 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f89b811ffe0 con 0x7f89d00c1790 2024-09-13T17:17:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.058+0000 7f89cf7fe640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f89b0066690 0x7f89b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.062+0000 7f89cf7fe640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f89b0066690 0x7f89b0068b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f89c800b9d0 tx=0x7f89c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:31.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.066+0000 7f89c7fff640 1 -- 172.21.15.39:0/1978086234 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f89b80ec2d0 con 0x7f89d00c1790 2024-09-13T17:17:31.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.254+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89d014d690 con 0x7f89d00c1790 2024-09-13T17:17:31.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89c7fff640 1 -- 172.21.15.39:0/1978086234 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f89b80ef8f0 con 0x7f89d00c1790 2024-09-13T17:17:31.262 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:31.262 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:31.263 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-13T17:17:31.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f89b0066690 msgr2=0x7f89b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:31.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f89b0066690 0x7f89b0068b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f89c800b9d0 tx=0x7f89c8008040 comp rx=0 tx=0).stop 2024-09-13T17:17:31.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 msgr2=0x7f89d014a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:31.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 0x7f89d014a4f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f89b80ac040 tx=0x7f89b801ad30 comp rx=0 tx=0).stop 2024-09-13T17:17:31.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 shutdown_connections 2024-09-13T17:17:31.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f89b0066690 0x7f89b0068b50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f89d00c20d0 0x7f89d014aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d00c1790 0x7f89d014a4f0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 --2- 172.21.15.39:0/1978086234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89d006d370 0x7f89d0152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:31.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 >> 172.21.15.39:0/1978086234 conn(0x7f89d0069e10 msgr2=0x7f89d00c3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:31.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 shutdown_connections 2024-09-13T17:17:31.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:31.258+0000 7f89d5c39640 1 -- 172.21.15.39:0/1978086234 wait complete. 2024-09-13T17:17:31.277 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:17:31.594 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T17:17:31.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:31.594 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:17:31.602 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T17:17:31.602 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:17:31.868 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T17:17:31.868 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:17:31.918 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:17:32.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc2a5f5f640 1 Processor -- start 2024-09-13T17:17:32.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc2a5f5f640 1 -- start start 2024-09-13T17:17:32.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc2a5f5f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc2a01526b0 0x7fc2a00c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc2a5f5f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a006c940 0x7fc2a00aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc2a5f5f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a014f790 con 0x7fc2a00ab100 2024-09-13T17:17:32.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc2a5f5f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a0152a90 con 0x7fc2a01526b0 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc2a5f5f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a006ce40 con 0x7fc2a006c940 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc2a01526b0 0x7fc2a00c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc2a00ab100 0x7fc2a01555f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51780/0 (socket says 172.21.15.39:51780) 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc2a01526b0 0x7fc2a00c12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54722/0 (socket says 172.21.15.39:54722) 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29ffff640 1 -- 172.21.15.39:0/1058610840 learned_addr learned my addr 172.21.15.39:0/1058610840 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29effd640 1 --2- 172.21.15.39:0/1058610840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a006c940 0x7fc2a00aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29f7fe640 1 -- 172.21.15.39:0/1058610840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a006c940 msgr2=0x7fc2a00aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29f7fe640 1 --2- 172.21.15.39:0/1058610840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a006c940 0x7fc2a00aabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29f7fe640 1 -- 172.21.15.39:0/1058610840 >> v1:172.21.15.39:6789/0 conn(0x7fc2a00ab100 legacy=0x7fc2a01555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29f7fe640 1 -- 172.21.15.39:0/1058610840 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a006d120 con 0x7fc2a01526b0 2024-09-13T17:17:32.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.050+0000 7fc29e7fc640 1 -- 172.21.15.39:0/1058610840 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078350880 0 0) 0x7fc2a014f790 con 0x7fc2a00ab100 2024-09-13T17:17:32.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc29f7fe640 1 --2- 172.21.15.39:0/1058610840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a01526b0 0x7fc2a00c12b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc290009d90 tx=0x7fc29001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fd60d6edcebba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:32.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc29e7fc640 1 -- 172.21.15.39:0/1058610840 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2900a5830 con 0x7fc2a01526b0 2024-09-13T17:17:32.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc29e7fc640 1 -- 172.21.15.39:0/1058610840 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2900a5990 con 0x7fc2a01526b0 2024-09-13T17:17:32.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc29e7fc640 1 -- 172.21.15.39:0/1058610840 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2900a5d20 con 0x7fc2a01526b0 2024-09-13T17:17:32.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/1058610840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a01526b0 msgr2=0x7fc2a00c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 --2- 172.21.15.39:0/1058610840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a01526b0 0x7fc2a00c12b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc290009d90 tx=0x7fc29001c970 comp rx=0 tx=0).stop 2024-09-13T17:17:32.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/1058610840 shutdown_connections 2024-09-13T17:17:32.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 --2- 172.21.15.39:0/1058610840 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a006c940 0x7fc2a00aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 --2- 172.21.15.39:0/1058610840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a01526b0 0x7fc2a00c12b0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/1058610840 >> 172.21.15.39:0/1058610840 conn(0x7fc2a0069e10 msgr2=0x7fc2a006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:32.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/1058610840 shutdown_connections 2024-09-13T17:17:32.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/1058610840 wait complete. 2024-09-13T17:17:32.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 Processor -- start 2024-09-13T17:17:32.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 -- start start 2024-09-13T17:17:32.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a006c940 0x7fc2a00bee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2a00ab100 0x7fc2a01efd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a01526b0 0x7fc2a01f02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00aaa80 con 0x7fc2a006c940 2024-09-13T17:17:32.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a014be80 con 0x7fc2a00ab100 2024-09-13T17:17:32.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc2a5f5f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a014c0b0 con 0x7fc2a01526b0 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc29f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a006c940 0x7fc2a00bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc29f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a006c940 0x7fc2a00bee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54728/0 (socket says 172.21.15.39:54728) 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.054+0000 7fc29f7fe640 1 -- 172.21.15.39:0/123565574 learned_addr learned my addr 172.21.15.39:0/123565574 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29ffff640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a01526b0 0x7fc2a01f02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29effd640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2a00ab100 0x7fc2a01efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29f7fe640 1 -- 172.21.15.39:0/123565574 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a01526b0 msgr2=0x7fc2a01f02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29f7fe640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a01526b0 0x7fc2a01f02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29f7fe640 1 -- 172.21.15.39:0/123565574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2a00ab100 msgr2=0x7fc2a01efd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29f7fe640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2a00ab100 0x7fc2a01efd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29f7fe640 1 -- 172.21.15.39:0/123565574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290009a70 con 0x7fc2a006c940 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29f7fe640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a006c940 0x7fc2a00bee90 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc2900b0040 tx=0x7fc290019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc287fff640 1 -- 172.21.15.39:0/123565574 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2900ad850 con 0x7fc2a006c940 2024-09-13T17:17:32.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc287fff640 1 -- 172.21.15.39:0/123565574 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2900ad9b0 con 0x7fc2a006c940 2024-09-13T17:17:32.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc287fff640 1 -- 172.21.15.39:0/123565574 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2900add20 con 0x7fc2a006c940 2024-09-13T17:17:32.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a014c390 con 0x7fc2a006c940 2024-09-13T17:17:32.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a00bf3d0 con 0x7fc2a006c940 2024-09-13T17:17:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a01efbc0 con 0x7fc2a006c940 2024-09-13T17:17:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc287fff640 1 -- 172.21.15.39:0/123565574 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2900b4050 con 0x7fc2a006c940 2024-09-13T17:17:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc287fff640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc28c0666c0 0x7fc28c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc287fff640 1 -- 172.21.15.39:0/123565574 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fc29011fd10 con 0x7fc2a006c940 2024-09-13T17:17:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29effd640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc28c0666c0 0x7fc28c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.058+0000 7fc29effd640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc28c0666c0 0x7fc28c068b80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc2a00c5210 tx=0x7fc294008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:32.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.062+0000 7fc287fff640 1 -- 172.21.15.39:0/123565574 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc2900ec000 con 0x7fc2a006c940 2024-09-13T17:17:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.218+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2a00aabc0 con 0x7fc2a006c940 2024-09-13T17:17:32.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc287fff640 1 -- 172.21.15.39:0/123565574 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+25425 (secure 0 0 0) 0x7fc2900b2020 con 0x7fc2a006c940 2024-09-13T17:17:32.224 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:32.224 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":63,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T17:16:34.950618+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6835/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6834/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.39:6834/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6837/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6835/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.84:6834/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.39:6836/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.39:6835/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.84:6835/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.39:6835/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.84:6834/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.39:6834/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.39:6834/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.39:6836/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.84:6832/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.84:6835/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.84:6832/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6836/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.84:6834/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.39:6835/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.84:6832/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6834/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6835/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.84:6834/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6835/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.39:6835/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6836/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6833/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.39:6835/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.39:6837/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6834/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6835/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6834/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.39:6835/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1750130579":"2024-09-14T17:09:29.828673+0000","172.21.15.39:6834/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.84:6832/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.39:6834/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.39:6837/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6836/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6835/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6834/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.84:6835/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.84:6833/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.84:6833/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6835/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.39:6834/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.84:6832/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6833/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6834/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6835/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6836/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6834/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6833/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6835/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6834/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.39:6834/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6835/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6836/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.39:6835/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.84:6833/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6836/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6837/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.84:6834/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.84:6834/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.84:6835/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.39:6837/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.39:6837/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6835/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.84:6832/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6835/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/1750130579":"2024-09-14T17:09:29.828673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc28c0666c0 msgr2=0x7fc28c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc28c0666c0 0x7fc28c068b80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc2a00c5210 tx=0x7fc294008040 comp rx=0 tx=0).stop 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a006c940 msgr2=0x7fc2a00bee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a006c940 0x7fc2a00bee90 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc2900b0040 tx=0x7fc290019710 comp rx=0 tx=0).stop 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 shutdown_connections 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc28c0666c0 0x7fc28c068b80 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc2a00ab100 0x7fc2a01efd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc2a01526b0 0x7fc2a01f02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 --2- 172.21.15.39:0/123565574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2a006c940 0x7fc2a00bee90 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 >> 172.21.15.39:0/123565574 conn(0x7fc2a0069e10 msgr2=0x7fc2a00c2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:32.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 shutdown_connections 2024-09-13T17:17:32.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.222+0000 7fc2a5f5f640 1 -- 172.21.15.39:0/123565574 wait complete. 2024-09-13T17:17:32.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:17:32.243 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:17:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e933d640 1 Processor -- start 2024-09-13T17:17:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e933d640 1 -- start start 2024-09-13T17:17:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e933d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e41540e0 0x7f18e40c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e933d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18e40c4ce0 0x7f18e4151070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e933d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e40c1d70 con 0x7f18e414e530 2024-09-13T17:17:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e933d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e4150b70 con 0x7f18e40c4ce0 2024-09-13T17:17:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e933d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4154570 con 0x7f18e41540e0 2024-09-13T17:17:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18e40c4ce0 0x7f18e4151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18e40c4ce0 0x7f18e4151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54740/0 (socket says 172.21.15.39:54740) 2024-09-13T17:17:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e37fe640 1 -- 172.21.15.39:0/3797495654 learned_addr learned my addr 172.21.15.39:0/3797495654 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e27fc640 1 --2- 172.21.15.39:0/3797495654 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e41540e0 0x7f18e40c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e37fe640 1 -- 172.21.15.39:0/3797495654 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e41540e0 msgr2=0x7f18e40c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e37fe640 1 --2- 172.21.15.39:0/3797495654 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e41540e0 0x7f18e40c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e37fe640 1 -- 172.21.15.39:0/3797495654 >> v1:172.21.15.84:6789/0 conn(0x7f18e414e530 legacy=0x7f18e40c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.366+0000 7f18e37fe640 1 -- 172.21.15.39:0/3797495654 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e4154850 con 0x7f18e40c4ce0 2024-09-13T17:17:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e37fe640 1 --2- 172.21.15.39:0/3797495654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 0x7f18e4151070 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009e50 tx=0x7f18dc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc2372ef06258ff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e1ffb640 1 -- 172.21.15.39:0/3797495654 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0a5730 con 0x7f18e40c4ce0 2024-09-13T17:17:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e1ffb640 1 -- 172.21.15.39:0/3797495654 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc0a5890 con 0x7f18e40c4ce0 2024-09-13T17:17:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e1ffb640 1 -- 172.21.15.39:0/3797495654 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0a5c20 con 0x7f18e40c4ce0 2024-09-13T17:17:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 -- 172.21.15.39:0/3797495654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 msgr2=0x7f18e4151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 --2- 172.21.15.39:0/3797495654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 0x7f18e4151070 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009e50 tx=0x7f18dc01c970 comp rx=0 tx=0).stop 2024-09-13T17:17:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 -- 172.21.15.39:0/3797495654 shutdown_connections 2024-09-13T17:17:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 --2- 172.21.15.39:0/3797495654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 0x7f18e4151070 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 --2- 172.21.15.39:0/3797495654 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e41540e0 0x7f18e40c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 -- 172.21.15.39:0/3797495654 >> 172.21.15.39:0/3797495654 conn(0x7f18e40c5790 msgr2=0x7f18e40c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 -- 172.21.15.39:0/3797495654 shutdown_connections 2024-09-13T17:17:32.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 -- 172.21.15.39:0/3797495654 wait complete. 2024-09-13T17:17:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 Processor -- start 2024-09-13T17:17:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 -- start start 2024-09-13T17:17:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 0x7f18e414ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e414e530 0x7f18e414d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e4153c00 0x7f18e414d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e414a8c0 con 0x7f18e40c4ce0 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e41413c0 con 0x7f18e414e530 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e933d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e414dd50 con 0x7f18e4153c00 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.370+0000 7f18e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 0x7f18e414ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 0x7f18e414ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54754/0 (socket says 172.21.15.39:54754) 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e2ffd640 1 -- 172.21.15.39:0/4001298651 learned_addr learned my addr 172.21.15.39:0/4001298651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e37fe640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e4153c00 0x7f18e414d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e27fc640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e414e530 0x7f18e414d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e2ffd640 1 -- 172.21.15.39:0/4001298651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e4153c00 msgr2=0x7f18e414d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e2ffd640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e4153c00 0x7f18e414d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e2ffd640 1 -- 172.21.15.39:0/4001298651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e414e530 msgr2=0x7f18e414d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e2ffd640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e414e530 0x7f18e414d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e2ffd640 1 -- 172.21.15.39:0/4001298651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009b30 con 0x7f18e40c4ce0 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e37fe640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e4153c00 0x7f18e414d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e2ffd640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 0x7f18e414ec40 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c500 tx=0x7f18d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18cb7fe640 1 -- 172.21.15.39:0/4001298651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4005220 con 0x7f18e40c4ce0 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18cb7fe640 1 -- 172.21.15.39:0/4001298651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4005380 con 0x7f18e40c4ce0 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18cb7fe640 1 -- 172.21.15.39:0/4001298651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4007270 con 0x7f18e40c4ce0 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e4151b40 con 0x7f18e40c4ce0 2024-09-13T17:17:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e4140a20 con 0x7f18e40c4ce0 2024-09-13T17:17:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e40c42e0 con 0x7f18e40c4ce0 2024-09-13T17:17:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18cb7fe640 1 -- 172.21.15.39:0/4001298651 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18d40073d0 con 0x7f18e40c4ce0 2024-09-13T17:17:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18cb7fe640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f18d0066690 0x7f18d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e27fc640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f18d0066690 0x7f18d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18cb7fe640 1 -- 172.21.15.39:0/4001298651 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f18d4084860 con 0x7f18e40c4ce0 2024-09-13T17:17:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.374+0000 7f18e27fc640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f18d0066690 0x7f18d0068b50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f18e40c6210 tx=0x7f18d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:32.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.378+0000 7f18cb7fe640 1 -- 172.21.15.39:0/4001298651 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f18d4050ad0 con 0x7f18e40c4ce0 2024-09-13T17:17:32.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.538+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18e40bfb30 con 0x7f18e40c4ce0 2024-09-13T17:17:32.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18cb7fe640 1 -- 172.21.15.39:0/4001298651 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18d40540f0 con 0x7f18e40c4ce0 2024-09-13T17:17:32.543 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T17:17:32.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f18d0066690 msgr2=0x7f18d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f18d0066690 0x7f18d0068b50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f18e40c6210 tx=0x7f18d8008040 comp rx=0 tx=0).stop 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 msgr2=0x7f18e414ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 0x7f18e414ec40 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c500 tx=0x7f18d400c9c0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 shutdown_connections 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f18d0066690 0x7f18d0068b50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18e414e530 0x7f18e414d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f18e4153c00 0x7f18e414d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 --2- 172.21.15.39:0/4001298651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e40c4ce0 0x7f18e414ec40 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 >> 172.21.15.39:0/4001298651 conn(0x7f18e40c5790 msgr2=0x7f18e40a9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 shutdown_connections 2024-09-13T17:17:32.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.542+0000 7f18e933d640 1 -- 172.21.15.39:0/4001298651 wait complete. 2024-09-13T17:17:32.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T17:17:32.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf876b2640 1 Processor -- start 2024-09-13T17:17:32.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf876b2640 1 -- start start 2024-09-13T17:17:32.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf876b2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbf801514d0 0x7fbf800c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf876b2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf80155270 con 0x7fbf800bcf90 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf876b2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf80153d80 con 0x7fbf80157370 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf876b2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf80157870 con 0x7fbf801514d0 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf84c4d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbf800bcf90 0x7fbf80154240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51796/0 (socket says 172.21.15.39:51796) 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf84c4d640 1 -- 172.21.15.39:0/3882764920 learned_addr learned my addr 172.21.15.39:0/3882764920 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf8544e640 1 --2- 172.21.15.39:0/3882764920 >> v2:172.21.15.84:3300/0 conn(0x7fbf801514d0 0x7fbf800c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031927017 0 0) 0x7fbf80155270 con 0x7fbf800bcf90 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf6c003530 con 0x7fbf800bcf90 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1330189151 0 0) 0x7fbf6c003530 con 0x7fbf800bcf90 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 >> v2:172.21.15.84:3300/0 conn(0x7fbf801514d0 msgr2=0x7fbf800c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 --2- 172.21.15.39:0/3882764920 >> v2:172.21.15.84:3300/0 conn(0x7fbf801514d0 0x7fbf800c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 >> v1:172.21.15.84:6789/0 conn(0x7fbf80157370 legacy=0x7fbf8015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf80157b50 con 0x7fbf800bcf90 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fbf70001680 con 0x7fbf800bcf90 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 >> v1:172.21.15.39:6789/0 conn(0x7fbf800bcf90 legacy=0x7fbf80154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf6c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf6c0047d0 0x7fbf6c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf6c005190 0x7fbf6c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80157870 con 0x7fbf6c003980 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c00c130 con 0x7fbf6c005190 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c00c2f0 con 0x7fbf6c0047d0 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf8544e640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf6c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf8544e640 1 -- 172.21.15.39:0/3882764920 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf6c0047d0 msgr2=0x7fbf6c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf8544e640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf6c0047d0 0x7fbf6c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf8544e640 1 -- 172.21.15.39:0/3882764920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf6c005190 msgr2=0x7fbf6c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:32.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf85c4f640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf6c005190 0x7fbf6c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf8544e640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf6c005190 0x7fbf6c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf8544e640 1 -- 172.21.15.39:0/3882764920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c003530 con 0x7fbf6c003980 2024-09-13T17:17:32.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf85c4f640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf6c005190 0x7fbf6c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:32.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.694+0000 7fbf85c4f640 1 -- 172.21.15.39:0/3882764920 reap_dead start 2024-09-13T17:17:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf8544e640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf6c003da0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009ae0 tx=0x7fbf7c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461d3ef23bab5678 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0a5530 con 0x7fbf6c003980 2024-09-13T17:17:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c0a5690 con 0x7fbf6c003980 2024-09-13T17:17:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf77fff640 1 -- 172.21.15.39:0/3882764920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0a5840 con 0x7fbf6c003980 2024-09-13T17:17:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 -- 172.21.15.39:0/3882764920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 msgr2=0x7fbf6c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf6c003da0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009ae0 tx=0x7fbf7c01ca70 comp rx=0 tx=0).stop 2024-09-13T17:17:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 -- 172.21.15.39:0/3882764920 shutdown_connections 2024-09-13T17:17:32.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 --2- 172.21.15.39:0/3882764920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf6c003da0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 -- 172.21.15.39:0/3882764920 >> 172.21.15.39:0/3882764920 conn(0x7fbf8006a210 msgr2=0x7fbf8006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 -- 172.21.15.39:0/3882764920 shutdown_connections 2024-09-13T17:17:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 -- 172.21.15.39:0/3882764920 wait complete. 2024-09-13T17:17:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 Processor -- start 2024-09-13T17:17:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 -- start start 2024-09-13T17:17:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf8014c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf800c85e0 0x7fbf800bd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf800c80c0 0x7fbf80146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80156850 con 0x7fbf6c003980 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf800bf5a0 con 0x7fbf800c80c0 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.698+0000 7fbf876b2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf800c8dc0 con 0x7fbf800c85e0 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf8544e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf8014c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf8544e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf8014c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54774/0 (socket says 172.21.15.39:54774) 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf8544e640 1 -- 172.21.15.39:0/2255056657 learned_addr learned my addr 172.21.15.39:0/2255056657 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf85c4f640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf800c80c0 0x7fbf80146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf84c4d640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf800c85e0 0x7fbf800bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf8544e640 1 -- 172.21.15.39:0/2255056657 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf800c85e0 msgr2=0x7fbf800bd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf8544e640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf800c85e0 0x7fbf800bd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf8544e640 1 -- 172.21.15.39:0/2255056657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf800c80c0 msgr2=0x7fbf80146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf8544e640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf800c80c0 0x7fbf80146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf8544e640 1 -- 172.21.15.39:0/2255056657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf800c8ae0 con 0x7fbf6c003980 2024-09-13T17:17:32.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf84c4d640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf800c85e0 0x7fbf800bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:32.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf8544e640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf8014c4c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c01c8a0 tx=0x7fbf7c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:32.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf75ffb640 1 -- 172.21.15.39:0/2255056657 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0a5530 con 0x7fbf6c003980 2024-09-13T17:17:32.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf75ffb640 1 -- 172.21.15.39:0/2255056657 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c00f000 con 0x7fbf6c003980 2024-09-13T17:17:32.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf800c75e0 con 0x7fbf6c003980 2024-09-13T17:17:32.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf75ffb640 1 -- 172.21.15.39:0/2255056657 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c004260 con 0x7fbf6c003980 2024-09-13T17:17:32.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf800c7ba0 con 0x7fbf6c003980 2024-09-13T17:17:32.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf800c7440 con 0x7fbf6c003980 2024-09-13T17:17:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf75ffb640 1 -- 172.21.15.39:0/2255056657 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf7c0b4070 con 0x7fbf6c003980 2024-09-13T17:17:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf75ffb640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf640666c0 0x7fbf64068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf75ffb640 1 -- 172.21.15.39:0/2255056657 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fbf7c11f710 con 0x7fbf6c003980 2024-09-13T17:17:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf84c4d640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf640666c0 0x7fbf64068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:32.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.702+0000 7fbf84c4d640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf640666c0 0x7fbf64068b80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbf800c5a30 tx=0x7fbf70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:32.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.706+0000 7fbf75ffb640 1 -- 172.21.15.39:0/2255056657 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf800c7440 con 0x7fbf6c003980 2024-09-13T17:17:32.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf800c7440 con 0x7fbf6c003980 2024-09-13T17:17:32.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf75ffb640 1 -- 172.21.15.39:0/2255056657 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v63) v1 ==== 99+0+11 (secure 0 0 0) 0x7fbf800c7440 con 0x7fbf6c003980 2024-09-13T17:17:32.868 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T17:17:32.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf640666c0 msgr2=0x7fbf64068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf640666c0 0x7fbf64068b80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbf800c5a30 tx=0x7fbf70009040 comp rx=0 tx=0).stop 2024-09-13T17:17:32.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 msgr2=0x7fbf8014c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:32.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf8014c4c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c01c8a0 tx=0x7fbf7c019810 comp rx=0 tx=0).stop 2024-09-13T17:17:32.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 shutdown_connections 2024-09-13T17:17:32.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbf640666c0 0x7fbf64068b80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf800c80c0 0x7fbf80146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbf800c85e0 0x7fbf800bd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 --2- 172.21.15.39:0/2255056657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf6c003980 0x7fbf8014c4c0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:32.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 >> 172.21.15.39:0/2255056657 conn(0x7fbf8006a210 msgr2=0x7fbf800c6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:32.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 shutdown_connections 2024-09-13T17:17:32.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:32.866+0000 7fbf876b2640 1 -- 172.21.15.39:0/2255056657 wait complete. 2024-09-13T17:17:32.887 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T17:17:33.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.002+0000 7f3473de4640 1 Processor -- start 2024-09-13T17:17:33.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.002+0000 7f3473de4640 1 -- start start 2024-09-13T17:17:33.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3473de4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f346c151cf0 0x7f346c152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3473de4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346c155d20 con 0x7f346c0aa2d0 2024-09-13T17:17:33.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3473de4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346c0c4750 con 0x7f346c0a4d60 2024-09-13T17:17:33.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3473de4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346c155b30 con 0x7f346c151cf0 2024-09-13T17:17:33.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3472381640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f346c151cf0 0x7f346c152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3471b80640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f346c0aa2d0 0x7f346c0aa6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51804/0 (socket says 172.21.15.39:51804) 2024-09-13T17:17:33.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3471b80640 1 -- 172.21.15.39:0/1410667186 learned_addr learned my addr 172.21.15.39:0/1410667186 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:33.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3472381640 1 -- 172.21.15.39:0/1410667186 >> v1:172.21.15.84:6789/0 conn(0x7f346c0a4d60 legacy=0x7f346c14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:33.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3472381640 1 -- 172.21.15.39:0/1410667186 >> v1:172.21.15.39:6789/0 conn(0x7f346c0aa2d0 legacy=0x7f346c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3472381640 1 -- 172.21.15.39:0/1410667186 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c155fc0 con 0x7f346c151cf0 2024-09-13T17:17:33.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3470b7e640 1 -- 172.21.15.39:0/1410667186 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768406176 0 0) 0x7f346c155d20 con 0x7f346c0aa2d0 2024-09-13T17:17:33.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.006+0000 7f3472381640 1 --2- 172.21.15.39:0/1410667186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 0x7f346c152140 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f34640097c0 tx=0x7f346401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb44634e20c27a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3470b7e640 1 -- 172.21.15.39:0/1410667186 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640ab600 con 0x7f346c151cf0 2024-09-13T17:17:33.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3470b7e640 1 -- 172.21.15.39:0/1410667186 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34640a7050 con 0x7f346c151cf0 2024-09-13T17:17:33.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3470b7e640 1 -- 172.21.15.39:0/1410667186 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640ab980 con 0x7f346c151cf0 2024-09-13T17:17:33.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- 172.21.15.39:0/1410667186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 msgr2=0x7f346c152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 --2- 172.21.15.39:0/1410667186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 0x7f346c152140 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f34640097c0 tx=0x7f346401c3d0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- 172.21.15.39:0/1410667186 shutdown_connections 2024-09-13T17:17:33.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 --2- 172.21.15.39:0/1410667186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 0x7f346c152140 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- 172.21.15.39:0/1410667186 >> 172.21.15.39:0/1410667186 conn(0x7f346c0c5790 msgr2=0x7f346c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- 172.21.15.39:0/1410667186 shutdown_connections 2024-09-13T17:17:33.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- 172.21.15.39:0/1410667186 wait complete. 2024-09-13T17:17:33.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 Processor -- start 2024-09-13T17:17:33.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- start start 2024-09-13T17:17:33.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f346c0a4d60 0x7f346c0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f346c0aa2d0 0x7f346c0c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 0x7f346c1f0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c0c28e0 con 0x7f346c151cf0 2024-09-13T17:17:33.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c0c2bc0 con 0x7f346c0a4d60 2024-09-13T17:17:33.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c1efb00 con 0x7f346c0aa2d0 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3472381640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 0x7f346c1f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3472381640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 0x7f346c1f0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54788/0 (socket says 172.21.15.39:54788) 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3472381640 1 -- 172.21.15.39:0/1786460603 learned_addr learned my addr 172.21.15.39:0/1786460603 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3471b80640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f346c0a4d60 0x7f346c0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f347137f640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f346c0aa2d0 0x7f346c0c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3472381640 1 -- 172.21.15.39:0/1786460603 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f346c0aa2d0 msgr2=0x7f346c0c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3472381640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f346c0aa2d0 0x7f346c0c10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3472381640 1 -- 172.21.15.39:0/1786460603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f346c0a4d60 msgr2=0x7f346c0c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3472381640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f346c0a4d60 0x7f346c0c21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3472381640 1 -- 172.21.15.39:0/1786460603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3464009420 con 0x7f346c151cf0 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3472381640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 0x7f346c1f0300 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f34640ae040 tx=0x7f346401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f34627fc640 1 -- 172.21.15.39:0/1786460603 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640b7650 con 0x7f346c151cf0 2024-09-13T17:17:33.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f34627fc640 1 -- 172.21.15.39:0/1786460603 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34640b77b0 con 0x7f346c151cf0 2024-09-13T17:17:33.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f34627fc640 1 -- 172.21.15.39:0/1786460603 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640b7b40 con 0x7f346c151cf0 2024-09-13T17:17:33.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c1efde0 con 0x7f346c151cf0 2024-09-13T17:17:33.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.010+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c06d190 con 0x7f346c151cf0 2024-09-13T17:17:33.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.014+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346c0724e0 con 0x7f346c151cf0 2024-09-13T17:17:33.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.014+0000 7f34627fc640 1 -- 172.21.15.39:0/1786460603 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f34640b7ca0 con 0x7f346c151cf0 2024-09-13T17:17:33.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.014+0000 7f34627fc640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3450066690 0x7f3450068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.014+0000 7f34627fc640 1 -- 172.21.15.39:0/1786460603 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f346411fb50 con 0x7f346c151cf0 2024-09-13T17:17:33.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.014+0000 7f3471b80640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3450066690 0x7f3450068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.014+0000 7f3471b80640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3450066690 0x7f3450068b50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f34680012f0 tx=0x7f3468009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.018+0000 7f34627fc640 1 -- 172.21.15.39:0/1786460603 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f34640ebf60 con 0x7f346c151cf0 2024-09-13T17:17:33.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.178+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7f346c1effd0 con 0x7f346c151cf0 2024-09-13T17:17:33.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.178+0000 7f34627fc640 1 -- 172.21.15.39:0/1786460603 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v63) v1 ==== 95+0+11 (secure 0 0 0) 0x7f34640b0020 con 0x7f346c151cf0 2024-09-13T17:17:33.181 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T17:17:33.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3450066690 msgr2=0x7f3450068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3450066690 0x7f3450068b50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f34680012f0 tx=0x7f3468009040 comp rx=0 tx=0).stop 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 msgr2=0x7f346c1f0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 0x7f346c1f0300 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f34640ae040 tx=0x7f346401ac10 comp rx=0 tx=0).stop 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 shutdown_connections 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3450066690 0x7f3450068b50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f346c151cf0 0x7f346c1f0300 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f346c0aa2d0 0x7f346c0c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 --2- 172.21.15.39:0/1786460603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f346c0a4d60 0x7f346c0c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 >> 172.21.15.39:0/1786460603 conn(0x7f346c0c5790 msgr2=0x7f346c0c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 shutdown_connections 2024-09-13T17:17:33.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.182+0000 7f3473de4640 1 -- 172.21.15.39:0/1786460603 wait complete. 2024-09-13T17:17:33.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T17:17:33.278 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:33.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.322+0000 7f8c36187640 1 Processor -- start 2024-09-13T17:17:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.322+0000 7f8c36187640 1 -- start start 2024-09-13T17:17:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.322+0000 7f8c36187640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8c3014acf0 0x7f8c30151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4ba0 0x7f8c30153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c30154ab0 con 0x7f8c300c4710 2024-09-13T17:17:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c30154d90 con 0x7f8c3014acf0 2024-09-13T17:17:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30150d50 con 0x7f8c300c4ba0 2024-09-13T17:17:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8c3014acf0 0x7f8c30151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8c3014acf0 0x7f8c30151550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54798/0 (socket says 172.21.15.39:54798) 2024-09-13T17:17:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2f7fe640 1 -- 172.21.15.39:0/2588540 learned_addr learned my addr 172.21.15.39:0/2588540 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2ffff640 1 --2- 172.21.15.39:0/2588540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4ba0 0x7f8c30153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2f7fe640 1 -- 172.21.15.39:0/2588540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4ba0 msgr2=0x7f8c30153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2f7fe640 1 --2- 172.21.15.39:0/2588540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4ba0 0x7f8c30153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2f7fe640 1 -- 172.21.15.39:0/2588540 >> v1:172.21.15.84:6789/0 conn(0x7f8c300c4710 legacy=0x7f8c3006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2f7fe640 1 -- 172.21.15.39:0/2588540 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30151030 con 0x7f8c3014acf0 2024-09-13T17:17:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2f7fe640 1 --2- 172.21.15.39:0/2588540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 0x7f8c30151550 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8c200098a0 tx=0x7f8c2001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf6432cf99149e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2e7fc640 1 -- 172.21.15.39:0/2588540 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c200ab5f0 con 0x7f8c3014acf0 2024-09-13T17:17:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2e7fc640 1 -- 172.21.15.39:0/2588540 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c200ab750 con 0x7f8c3014acf0 2024-09-13T17:17:33.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c2e7fc640 1 -- 172.21.15.39:0/2588540 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c200abae0 con 0x7f8c3014acf0 2024-09-13T17:17:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 -- 172.21.15.39:0/2588540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 msgr2=0x7f8c30151550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 --2- 172.21.15.39:0/2588540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 0x7f8c30151550 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8c200098a0 tx=0x7f8c2001c410 comp rx=0 tx=0).stop 2024-09-13T17:17:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 -- 172.21.15.39:0/2588540 shutdown_connections 2024-09-13T17:17:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 --2- 172.21.15.39:0/2588540 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4ba0 0x7f8c30153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 --2- 172.21.15.39:0/2588540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 0x7f8c30151550 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 -- 172.21.15.39:0/2588540 >> 172.21.15.39:0/2588540 conn(0x7f8c300c32a0 msgr2=0x7f8c300c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 -- 172.21.15.39:0/2588540 shutdown_connections 2024-09-13T17:17:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.326+0000 7f8c36187640 1 -- 172.21.15.39:0/2588540 wait complete. 2024-09-13T17:17:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 Processor -- start 2024-09-13T17:17:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 -- start start 2024-09-13T17:17:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4710 0x7f8c3014f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c300c4ba0 0x7f8c30148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 0x7f8c30148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301ec7b0 con 0x7f8c3014acf0 2024-09-13T17:17:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301ec9c0 con 0x7f8c300c4ba0 2024-09-13T17:17:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301ecca0 con 0x7f8c300c4710 2024-09-13T17:17:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 0x7f8c30148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 0x7f8c30148e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54804/0 (socket says 172.21.15.39:54804) 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2ffff640 1 -- 172.21.15.39:0/1814858639 learned_addr learned my addr 172.21.15.39:0/1814858639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2f7fe640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4710 0x7f8c3014f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2effd640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c300c4ba0 0x7f8c30148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2ffff640 1 -- 172.21.15.39:0/1814858639 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4710 msgr2=0x7f8c3014f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2ffff640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4710 0x7f8c3014f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2ffff640 1 -- 172.21.15.39:0/1814858639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c300c4ba0 msgr2=0x7f8c30148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2ffff640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c300c4ba0 0x7f8c30148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2ffff640 1 -- 172.21.15.39:0/1814858639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200094d0 con 0x7f8c3014acf0 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c2ffff640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 0x7f8c30148e90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f8c28007d80 tx=0x7f8c2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c17fff640 1 -- 172.21.15.39:0/1814858639 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28014420 con 0x7f8c3014acf0 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c17fff640 1 -- 172.21.15.39:0/1814858639 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c28014580 con 0x7f8c3014acf0 2024-09-13T17:17:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c301ecf80 con 0x7f8c3014acf0 2024-09-13T17:17:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c17fff640 1 -- 172.21.15.39:0/1814858639 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28014930 con 0x7f8c3014acf0 2024-09-13T17:17:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c301ed540 con 0x7f8c3014acf0 2024-09-13T17:17:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.330+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c30153ff0 con 0x7f8c3014acf0 2024-09-13T17:17:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.334+0000 7f8c17fff640 1 -- 172.21.15.39:0/1814858639 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c28014a90 con 0x7f8c3014acf0 2024-09-13T17:17:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.334+0000 7f8c17fff640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c1c066690 0x7f8c1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.334+0000 7f8c17fff640 1 -- 172.21.15.39:0/1814858639 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f8c28085c20 con 0x7f8c3014acf0 2024-09-13T17:17:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.334+0000 7f8c2f7fe640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c1c066690 0x7f8c1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.334+0000 7f8c2f7fe640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c1c066690 0x7f8c1c068b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8c200af000 tx=0x7f8c200a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.338+0000 7f8c17fff640 1 -- 172.21.15.39:0/1814858639 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8c28051e30 con 0x7f8c3014acf0 2024-09-13T17:17:33.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.406+0000 7f97fa02c640 1 Processor -- start 2024-09-13T17:17:33.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.406+0000 7f97fa02c640 1 -- start start 2024-09-13T17:17:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f97f40c6130 0x7f97f40c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f97f40c0240 0x7f97f4157d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f40c6510 con 0x7f97f4153de0 2024-09-13T17:17:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f40c1c80 con 0x7f97f40c6130 2024-09-13T17:17:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f4158290 con 0x7f97f40c0240 2024-09-13T17:17:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f97f40c6130 0x7f97f40c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f97f40c6130 0x7f97f40c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54816/0 (socket says 172.21.15.39:54816) 2024-09-13T17:17:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f37fe640 1 -- 172.21.15.39:0/1086877920 learned_addr learned my addr 172.21.15.39:0/1086877920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f2ffd640 1 --2- 172.21.15.39:0/1086877920 >> v2:172.21.15.84:3300/0 conn(0x7f97f40c0240 0x7f97f4157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f37fe640 1 -- 172.21.15.39:0/1086877920 >> v2:172.21.15.84:3300/0 conn(0x7f97f40c0240 msgr2=0x7f97f4157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f37fe640 1 --2- 172.21.15.39:0/1086877920 >> v2:172.21.15.84:3300/0 conn(0x7f97f40c0240 0x7f97f4157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f37fe640 1 -- 172.21.15.39:0/1086877920 >> v1:172.21.15.84:6789/0 conn(0x7f97f4153de0 legacy=0x7f97f406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f37fe640 1 -- 172.21.15.39:0/1086877920 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f40c2bc0 con 0x7f97f40c6130 2024-09-13T17:17:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f2ffd640 1 --2- 172.21.15.39:0/1086877920 >> v2:172.21.15.84:3300/0 conn(0x7f97f40c0240 0x7f97f4157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f37fe640 1 --2- 172.21.15.39:0/1086877920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f40c6130 0x7f97f40c24c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f97f40abb70 tx=0x7f97ec01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32816b52d37e3de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f27fc640 1 -- 172.21.15.39:0/1086877920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0a5890 con 0x7f97f40c6130 2024-09-13T17:17:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f27fc640 1 -- 172.21.15.39:0/1086877920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec0a59f0 con 0x7f97f40c6130 2024-09-13T17:17:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97f27fc640 1 -- 172.21.15.39:0/1086877920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0a5d80 con 0x7f97f40c6130 2024-09-13T17:17:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 -- 172.21.15.39:0/1086877920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f40c6130 msgr2=0x7f97f40c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 --2- 172.21.15.39:0/1086877920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f40c6130 0x7f97f40c24c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f97f40abb70 tx=0x7f97ec01c9b0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 -- 172.21.15.39:0/1086877920 shutdown_connections 2024-09-13T17:17:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 --2- 172.21.15.39:0/1086877920 >> v2:172.21.15.84:3300/0 conn(0x7f97f40c0240 0x7f97f4157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 --2- 172.21.15.39:0/1086877920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f40c6130 0x7f97f40c24c0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 -- 172.21.15.39:0/1086877920 >> 172.21.15.39:0/1086877920 conn(0x7f97f40c32a0 msgr2=0x7f97f40c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 -- 172.21.15.39:0/1086877920 shutdown_connections 2024-09-13T17:17:33.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.410+0000 7f97fa02c640 1 -- 172.21.15.39:0/1086877920 wait complete. 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 Processor -- start 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 -- start start 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97f40c0240 0x7f97f4150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97f40c6130 0x7f97f4159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f4153de0 0x7f97f41ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f414e940 con 0x7f97f4153de0 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f406d080 con 0x7f97f40c6130 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f41f0020 con 0x7f97f40c0240 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f4153de0 0x7f97f41ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f4153de0 0x7f97f41ef7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54824/0 (socket says 172.21.15.39:54824) 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f3fff640 1 -- 172.21.15.39:0/2946807841 learned_addr learned my addr 172.21.15.39:0/2946807841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f37fe640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97f40c0240 0x7f97f4150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f2ffd640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97f40c6130 0x7f97f4159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f3fff640 1 -- 172.21.15.39:0/2946807841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97f40c0240 msgr2=0x7f97f4150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f3fff640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97f40c0240 0x7f97f4150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f3fff640 1 -- 172.21.15.39:0/2946807841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97f40c6130 msgr2=0x7f97f4159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f3fff640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97f40c6130 0x7f97f4159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f3fff640 1 -- 172.21.15.39:0/2946807841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec009b30 con 0x7f97f4153de0 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f2ffd640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97f40c6130 0x7f97f4159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97f3fff640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f4153de0 0x7f97f41ef7d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f97e400b410 tx=0x7f97e400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97ebfff640 1 -- 172.21.15.39:0/2946807841 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4011830 con 0x7f97f4153de0 2024-09-13T17:17:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97ebfff640 1 -- 172.21.15.39:0/2946807841 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e40119d0 con 0x7f97f4153de0 2024-09-13T17:17:33.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f41f02c0 con 0x7f97f4153de0 2024-09-13T17:17:33.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97ebfff640 1 -- 172.21.15.39:0/2946807841 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4011d80 con 0x7f97f4153de0 2024-09-13T17:17:33.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f406d5e0 con 0x7f97f4153de0 2024-09-13T17:17:33.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f406d800 con 0x7f97f4153de0 2024-09-13T17:17:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.414+0000 7f97ebfff640 1 -- 172.21.15.39:0/2946807841 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f97e4023430 con 0x7f97f4153de0 2024-09-13T17:17:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.418+0000 7f97ebfff640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f97d4066690 0x7f97d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.418+0000 7f97ebfff640 1 -- 172.21.15.39:0/2946807841 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f97e4085340 con 0x7f97f4153de0 2024-09-13T17:17:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.418+0000 7f97f37fe640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f97d4066690 0x7f97d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.418+0000 7f97f37fe640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f97d4066690 0x7f97d4068b50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0ae000 tx=0x7f97ec0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.418+0000 7f97ebfff640 1 -- 172.21.15.39:0/2946807841 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f97e40516b0 con 0x7f97f4153de0 2024-09-13T17:17:33.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.502+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c300a5ac0 con 0x7f8c3014acf0 2024-09-13T17:17:33.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.502+0000 7f8c17fff640 1 -- 172.21.15.39:0/1814858639 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v63) v1 ==== 99+0+11 (secure 0 0 0) 0x7f8c28055450 con 0x7f8c3014acf0 2024-09-13T17:17:33.503 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.502+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c1c066690 msgr2=0x7f8c1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.502+0000 7f8c36187640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c1c066690 0x7f8c1c068b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8c200af000 tx=0x7f8c200a7000 comp rx=0 tx=0).stop 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.502+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 msgr2=0x7f8c30148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.502+0000 7f8c36187640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 0x7f8c30148e90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f8c28007d80 tx=0x7f8c2800e4b0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.506+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 shutdown_connections 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.506+0000 7f8c36187640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8c1c066690 0x7f8c1c068b50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.506+0000 7f8c36187640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c300c4ba0 0x7f8c30148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.506+0000 7f8c36187640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3014acf0 0x7f8c30148e90 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.506+0000 7f8c36187640 1 --2- 172.21.15.39:0/1814858639 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8c300c4710 0x7f8c3014f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.506+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 >> 172.21.15.39:0/1814858639 conn(0x7f8c300c32a0 msgr2=0x7f8c300bfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.506+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 shutdown_connections 2024-09-13T17:17:33.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.506+0000 7f8c36187640 1 -- 172.21.15.39:0/1814858639 wait complete. 2024-09-13T17:17:33.528 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T17:17:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.622+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97f4154270 con 0x7f97f4153de0 2024-09-13T17:17:33.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.622+0000 7f97ebfff640 1 -- 172.21.15.39:0/2946807841 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f97e4054cd0 con 0x7f97f4153de0 2024-09-13T17:17:33.626 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:33.626 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:33.627 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-13T17:17:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f97d4066690 msgr2=0x7f97d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f97d4066690 0x7f97d4068b50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0ae000 tx=0x7f97ec0a7070 comp rx=0 tx=0).stop 2024-09-13T17:17:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f4153de0 msgr2=0x7f97f41ef7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f4153de0 0x7f97f41ef7d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f97e400b410 tx=0x7f97e400b8d0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 shutdown_connections 2024-09-13T17:17:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f97d4066690 0x7f97d4068b50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97f40c6130 0x7f97f4159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97f4153de0 0x7f97f41ef7d0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 --2- 172.21.15.39:0/2946807841 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f97f40c0240 0x7f97f4150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 >> 172.21.15.39:0/2946807841 conn(0x7f97f40c32a0 msgr2=0x7f97f415de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 shutdown_connections 2024-09-13T17:17:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.626+0000 7f97fa02c640 1 -- 172.21.15.39:0/2946807841 wait complete. 2024-09-13T17:17:33.644 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:17:33.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12cd05640 1 Processor -- start 2024-09-13T17:17:33.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12cd05640 1 -- start start 2024-09-13T17:17:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12cd05640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1280bcf90 0x7fc128154240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12cd05640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc128155270 con 0x7fc128157370 2024-09-13T17:17:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12cd05640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc128153d80 con 0x7fc1281514d0 2024-09-13T17:17:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12cd05640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc128157870 con 0x7fc1280bcf90 2024-09-13T17:17:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc126d9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc128157370 0x7fc128154780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51812/0 (socket says 172.21.15.39:51812) 2024-09-13T17:17:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc126d9d640 1 -- 172.21.15.39:0/3347797615 learned_addr learned my addr 172.21.15.39:0/3347797615 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc125d9b640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1280bcf90 0x7fc128154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828311448 0 0) 0x7fc128155270 con 0x7fc128157370 2024-09-13T17:17:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc110003530 con 0x7fc128157370 2024-09-13T17:17:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3269686668 0 0) 0x7fc110003530 con 0x7fc128157370 2024-09-13T17:17:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1280bcf90 msgr2=0x7fc128154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1280bcf90 0x7fc128154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 >> v1:172.21.15.84:6789/0 conn(0x7fc1281514d0 legacy=0x7fc1280c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc128157b50 con 0x7fc128157370 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fc120001a00 con 0x7fc128157370 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 >> v1:172.21.15.39:6789/0 conn(0x7fc128157370 legacy=0x7fc128154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1100039b0 0x7fc110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1100047d0 0x7fc110004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1280bcf90 0x7fc1100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc128157870 con 0x7fc1100039b0 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11000baf0 con 0x7fc1100047d0 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11000bcb0 con 0x7fc1280bcf90 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12659c640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1100039b0 0x7fc110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc126d9d640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1280bcf90 0x7fc1100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc125d9b640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1100047d0 0x7fc110004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12659c640 1 -- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1280bcf90 msgr2=0x7fc1100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12659c640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1280bcf90 0x7fc1100051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12659c640 1 -- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1100047d0 msgr2=0x7fc110004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12659c640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1100047d0 0x7fc110004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc12659c640 1 -- 172.21.15.39:0/3347797615 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110003530 con 0x7fc1100039b0 2024-09-13T17:17:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.650+0000 7fc125d9b640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1100047d0 0x7fc110004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12659c640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1100039b0 0x7fc110003dd0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fc118001780 tx=0x7fc11801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cb49aafb772f22 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180052d0 con 0x7fc1100039b0 2024-09-13T17:17:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1180a7050 con 0x7fc1100039b0 2024-09-13T17:17:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12559a640 1 -- 172.21.15.39:0/3347797615 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118003600 con 0x7fc1100039b0 2024-09-13T17:17:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 -- 172.21.15.39:0/3347797615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1100039b0 msgr2=0x7fc110003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 --2- 172.21.15.39:0/3347797615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1100039b0 0x7fc110003dd0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fc118001780 tx=0x7fc11801c420 comp rx=0 tx=0).stop 2024-09-13T17:17:33.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 -- 172.21.15.39:0/3347797615 reap_dead start 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 -- 172.21.15.39:0/3347797615 shutdown_connections 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 -- 172.21.15.39:0/3347797615 >> 172.21.15.39:0/3347797615 conn(0x7fc12806a210 msgr2=0x7fc12806a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 -- 172.21.15.39:0/3347797615 shutdown_connections 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 -- 172.21.15.39:0/3347797615 wait complete. 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 Processor -- start 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 -- start start 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1100039b0 0x7fc12815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1280bd370 0x7fc1281ee150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1281ee690 0x7fc1280c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1281f0f00 con 0x7fc1281ee690 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1281f0b90 con 0x7fc1280bd370 2024-09-13T17:17:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12cd05640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280c92e0 con 0x7fc1100039b0 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1281ee690 0x7fc1280c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1281ee690 0x7fc1280c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54842/0 (socket says 172.21.15.39:54842) 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 -- 172.21.15.39:0/3635828149 learned_addr learned my addr 172.21.15.39:0/3635828149 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc125d9b640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1280bd370 0x7fc1281ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc12659c640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1100039b0 0x7fc12815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 -- 172.21.15.39:0/3635828149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1100039b0 msgr2=0x7fc12815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1100039b0 0x7fc12815eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 -- 172.21.15.39:0/3635828149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1280bd370 msgr2=0x7fc1281ee150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1280bd370 0x7fc1281ee150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 -- 172.21.15.39:0/3635828149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12814c960 con 0x7fc1281ee690 2024-09-13T17:17:33.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.654+0000 7fc126d9d640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1281ee690 0x7fc1280c7420 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc110004750 tx=0x7fc12000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc116ffd640 1 -- 172.21.15.39:0/3635828149 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120004d70 con 0x7fc1281ee690 2024-09-13T17:17:33.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc116ffd640 1 -- 172.21.15.39:0/3635828149 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12000bb80 con 0x7fc1281ee690 2024-09-13T17:17:33.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12815f0c0 con 0x7fc1281ee690 2024-09-13T17:17:33.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc116ffd640 1 -- 172.21.15.39:0/3635828149 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120007110 con 0x7fc1281ee690 2024-09-13T17:17:33.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12814cc70 con 0x7fc1281ee690 2024-09-13T17:17:33.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12815a510 con 0x7fc1281ee690 2024-09-13T17:17:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc116ffd640 1 -- 172.21.15.39:0/3635828149 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc120007270 con 0x7fc1281ee690 2024-09-13T17:17:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc116ffd640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc108066690 0x7fc108068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc12659c640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc108066690 0x7fc108068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc116ffd640 1 -- 172.21.15.39:0/3635828149 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fc1200850d0 con 0x7fc1281ee690 2024-09-13T17:17:33.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.658+0000 7fc12659c640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc108066690 0x7fc108068b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc118001750 tx=0x7fc1180011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:33.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.662+0000 7fc116ffd640 1 -- 172.21.15.39:0/3635828149 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc1200513c0 con 0x7fc1281ee690 2024-09-13T17:17:33.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.822+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7fc1281f1040 con 0x7fc1281ee690 2024-09-13T17:17:33.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.822+0000 7fc116ffd640 1 -- 172.21.15.39:0/3635828149 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v63) v1 ==== 95+0+11 (secure 0 0 0) 0x7fc1200549e0 con 0x7fc1281ee690 2024-09-13T17:17:33.825 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T17:17:33.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc108066690 msgr2=0x7fc108068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc108066690 0x7fc108068b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc118001750 tx=0x7fc1180011a0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1281ee690 msgr2=0x7fc1280c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:33.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1281ee690 0x7fc1280c7420 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc110004750 tx=0x7fc12000b4a0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 shutdown_connections 2024-09-13T17:17:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc108066690 0x7fc108068b50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1280bd370 0x7fc1281ee150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1281ee690 0x7fc1280c7420 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 --2- 172.21.15.39:0/3635828149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc1100039b0 0x7fc12815eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 >> 172.21.15.39:0/3635828149 conn(0x7fc12806a210 msgr2=0x7fc128156730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 shutdown_connections 2024-09-13T17:17:33.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:33.826+0000 7fc12cd05640 1 -- 172.21.15.39:0/3635828149 wait complete. 2024-09-13T17:17:33.844 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:17:34.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69f52d5640 1 Processor -- start 2024-09-13T17:17:34.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69f52d5640 1 -- start start 2024-09-13T17:17:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69f52d5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f69f00aaaf0 0x7f69f0152780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69f52d5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f0153110 con 0x7f69f00aa2d0 2024-09-13T17:17:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69f52d5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f00c47f0 con 0x7f69f0155c20 2024-09-13T17:17:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69f52d5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f0151c20 con 0x7f69f00aaaf0 2024-09-13T17:17:34.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69ef7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f69f00aaaf0 0x7f69f0152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69ef7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f69f00aaaf0 0x7f69f0152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54852/0 (socket says 172.21.15.39:54852) 2024-09-13T17:17:34.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69ef7fe640 1 -- 172.21.15.39:0/3693539778 learned_addr learned my addr 172.21.15.39:0/3693539778 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:34.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69ef7fe640 1 -- 172.21.15.39:0/3693539778 >> v1:172.21.15.84:6789/0 conn(0x7f69f0155c20 legacy=0x7f69f0152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:34.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69ef7fe640 1 -- 172.21.15.39:0/3693539778 >> v1:172.21.15.39:6789/0 conn(0x7f69f00aa2d0 legacy=0x7f69f0154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.006+0000 7f69ef7fe640 1 -- 172.21.15.39:0/3693539778 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f00c4a60 con 0x7f69f00aaaf0 2024-09-13T17:17:34.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69ef7fe640 1 --2- 172.21.15.39:0/3693539778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 0x7f69f0152780 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009870 tx=0x7f69e001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e7c8e9bed2944b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69edffb640 1 -- 172.21.15.39:0/3693539778 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00ab680 con 0x7f69f00aaaf0 2024-09-13T17:17:34.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69edffb640 1 -- 172.21.15.39:0/3693539778 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e00ab7e0 con 0x7f69f00aaaf0 2024-09-13T17:17:34.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69edffb640 1 -- 172.21.15.39:0/3693539778 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00abb70 con 0x7f69f00aaaf0 2024-09-13T17:17:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69f52d5640 1 -- 172.21.15.39:0/3693539778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 msgr2=0x7f69f0152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69f52d5640 1 --2- 172.21.15.39:0/3693539778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 0x7f69f0152780 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009870 tx=0x7f69e001c460 comp rx=0 tx=0).stop 2024-09-13T17:17:34.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69f52d5640 1 -- 172.21.15.39:0/3693539778 shutdown_connections 2024-09-13T17:17:34.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69f52d5640 1 --2- 172.21.15.39:0/3693539778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 0x7f69f0152780 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69f52d5640 1 -- 172.21.15.39:0/3693539778 >> 172.21.15.39:0/3693539778 conn(0x7f69f00c5790 msgr2=0x7f69f00c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:34.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69f52d5640 1 -- 172.21.15.39:0/3693539778 shutdown_connections 2024-09-13T17:17:34.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69f52d5640 1 -- 172.21.15.39:0/3693539778 wait complete. 2024-09-13T17:17:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69f52d5640 1 Processor -- start 2024-09-13T17:17:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.010+0000 7f69f52d5640 1 -- start start 2024-09-13T17:17:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69f52d5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69f00aa2d0 0x7f69f006b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69f52d5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 0x7f69f01434c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69f52d5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f00a7fd0 0x7f69f014aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69f52d5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00a7650 con 0x7f69f00aaaf0 2024-09-13T17:17:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69f52d5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f006d2c0 con 0x7f69f00a7fd0 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69f52d5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f0144000 con 0x7f69f00aa2d0 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ee7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 0x7f69f01434c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ee7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 0x7f69f01434c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54868/0 (socket says 172.21.15.39:54868) 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ee7fc640 1 -- 172.21.15.39:0/4093482530 learned_addr learned my addr 172.21.15.39:0/4093482530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ef7fe640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f00a7fd0 0x7f69f014aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69eeffd640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69f00aa2d0 0x7f69f006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ee7fc640 1 -- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69f00aa2d0 msgr2=0x7f69f006b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ee7fc640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69f00aa2d0 0x7f69f006b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ee7fc640 1 -- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f00a7fd0 msgr2=0x7f69f014aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ee7fc640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f00a7fd0 0x7f69f014aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ee7fc640 1 -- 172.21.15.39:0/4093482530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e00094d0 con 0x7f69f00aaaf0 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69eeffd640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69f00aa2d0 0x7f69f006b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ef7fe640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f00a7fd0 0x7f69f014aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69ee7fc640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 0x7f69f01434c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f69e400b410 tx=0x7f69e400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69d77fe640 1 -- 172.21.15.39:0/4093482530 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4004020 con 0x7f69f00aaaf0 2024-09-13T17:17:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69d77fe640 1 -- 172.21.15.39:0/4093482530 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e4004180 con 0x7f69f00aaaf0 2024-09-13T17:17:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f00a7400 con 0x7f69f00aaaf0 2024-09-13T17:17:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69d77fe640 1 -- 172.21.15.39:0/4093482530 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4011630 con 0x7f69f00aaaf0 2024-09-13T17:17:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f014a510 con 0x7f69f00aaaf0 2024-09-13T17:17:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f00674c0 con 0x7f69f00aaaf0 2024-09-13T17:17:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69d77fe640 1 -- 172.21.15.39:0/4093482530 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69e4011790 con 0x7f69f00aaaf0 2024-09-13T17:17:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69d77fe640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69dc066690 0x7f69dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69d77fe640 1 -- 172.21.15.39:0/4093482530 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f69e4084fa0 con 0x7f69f00aaaf0 2024-09-13T17:17:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69eeffd640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69dc066690 0x7f69dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.014+0000 7f69eeffd640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69dc066690 0x7f69dc068b50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f69e8007fd0 tx=0x7f69e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.018+0000 7f69d77fe640 1 -- 172.21.15.39:0/4093482530 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69e40b6db0 con 0x7f69f00aaaf0 2024-09-13T17:17:34.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.182+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f69f0141740 con 0x7f69f00aaaf0 2024-09-13T17:17:34.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.182+0000 7f69d77fe640 1 -- 172.21.15.39:0/4093482530 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 78 entries v63) v1 ==== 81+0+4738 (secure 0 0 0) 0x7f69e4051290 con 0x7f69f00aaaf0 2024-09-13T17:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/380008541 2024-09-14T17:16:34.950555+0000 2024-09-13T17:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3721635324 2024-09-14T17:16:04.944309+0000 2024-09-13T17:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1355102409 2024-09-14T17:15:29.935985+0000 2024-09-13T17:17:34.186 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/305129652 2024-09-14T17:14:49.920017+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/282472862 2024-09-14T17:14:04.899010+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/397460031 2024-09-14T17:12:19.869778+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/397460031 2024-09-14T17:12:19.869778+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/2351253009 2024-09-14T17:11:09.852639+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3334565883 2024-09-14T17:11:59.865711+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3923043380 2024-09-14T17:09:59.836463+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1334276441 2024-09-14T16:59:04.677958+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/1326467723 2024-09-14T17:00:02.030979+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1334276441 2024-09-14T16:59:04.677958+0000 2024-09-13T17:17:34.187 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4200926406 2024-09-14T17:05:59.776545+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1392182763 2024-09-14T17:07:19.797311+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/3122796453 2024-09-14T16:59:54.690333+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/1326467723 2024-09-14T17:00:02.030979+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2740003107 2024-09-14T16:57:14.654506+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1355102409 2024-09-14T17:15:29.935985+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/1969975539 2024-09-14T16:52:59.589819+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3721635324 2024-09-14T17:16:04.944309+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/2351253009 2024-09-14T17:11:09.852639+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2144264453 2024-09-14T16:52:24.581694+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4094657757 2024-09-14T16:56:04.630964+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3600322972 2024-09-14T16:55:14.618334+0000 2024-09-13T17:17:34.188 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/380008541 2024-09-14T17:16:34.950555+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1572373831 2024-09-14T16:51:14.567493+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4246622083 2024-09-14T17:01:24.709561+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/282472862 2024-09-14T17:14:04.899010+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1355376480 2024-09-14T16:54:09.605050+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3334565883 2024-09-14T17:11:59.865711+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3941977343 2024-09-14T17:08:49.820117+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/2521680730 2024-09-14T17:09:29.828780+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/661906662 2024-09-14T16:57:59.663485+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4094657757 2024-09-14T16:56:04.630964+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3540656099 2024-09-14T16:50:14.555020+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2065196144 2024-09-14T16:53:14.593565+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1750130579 2024-09-14T17:09:29.828673+0000 2024-09-13T17:17:34.189 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3163580917 2024-09-14T16:49:44.548920+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1355376480 2024-09-14T16:54:09.605050+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2065196144 2024-09-14T16:53:14.593565+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1392588044 2024-09-14T16:50:49.562750+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1034701751 2024-09-14T17:10:39.845194+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/3122796453 2024-09-14T16:59:54.690333+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/3105501456 2024-09-14T17:03:49.742996+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/2521680730 2024-09-14T17:09:29.828780+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/661906662 2024-09-14T16:57:59.663485+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1572373831 2024-09-14T16:51:14.567493+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/1969975539 2024-09-14T16:52:59.589819+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1392588044 2024-09-14T16:50:49.562750+0000 2024-09-13T17:17:34.190 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2144264453 2024-09-14T16:52:24.581694+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3163580917 2024-09-14T16:49:44.548920+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3941977343 2024-09-14T17:08:49.820117+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2638959497 2024-09-14T17:01:24.709473+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2740003107 2024-09-14T16:57:14.654506+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4246622083 2024-09-14T17:01:24.709561+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/305129652 2024-09-14T17:14:49.920017+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/835951775 2024-09-14T17:02:59.730250+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/828722052 2024-09-14T17:03:19.734465+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2638959497 2024-09-14T17:01:24.709473+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/828722052 2024-09-14T17:03:19.734465+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3600322972 2024-09-14T16:55:14.618334+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/611669592 2024-09-14T17:04:24.751785+0000 2024-09-13T17:17:34.191 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/611669592 2024-09-14T17:04:24.751785+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1590039204 2024-09-14T17:05:09.765210+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4200926406 2024-09-14T17:05:59.776545+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/440028459 2024-09-14T17:07:34.801438+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/3105501456 2024-09-14T17:03:49.742996+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1590039204 2024-09-14T17:05:09.765210+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/263561599 2024-09-14T17:05:54.774622+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3923043380 2024-09-14T17:09:59.836463+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/263561599 2024-09-14T17:05:54.774622+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1392182763 2024-09-14T17:07:19.797311+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/835951775 2024-09-14T17:02:59.730250+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3540656099 2024-09-14T16:50:14.555020+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/440028459 2024-09-14T17:07:34.801438+0000 2024-09-13T17:17:34.192 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1034701751 2024-09-14T17:10:39.845194+0000 2024-09-13T17:17:34.193 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1750130579 2024-09-14T17:09:29.828673+0000 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:listed 78 entries 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69dc066690 msgr2=0x7f69dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69dc066690 0x7f69dc068b50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f69e8007fd0 tx=0x7f69e8009040 comp rx=0 tx=0).stop 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 msgr2=0x7f69f01434c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 0x7f69f01434c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f69e400b410 tx=0x7f69e400b8d0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 shutdown_connections 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f69dc066690 0x7f69dc068b50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69f00a7fd0 0x7f69f014aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69f00aaaf0 0x7f69f01434c0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 --2- 172.21.15.39:0/4093482530 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f69f00aa2d0 0x7f69f006b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 >> 172.21.15.39:0/4093482530 conn(0x7f69f00c5790 msgr2=0x7f69f00c3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:34.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 shutdown_connections 2024-09-13T17:17:34.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.186+0000 7f69f52d5640 1 -- 172.21.15.39:0/4093482530 wait complete. 2024-09-13T17:17:34.204 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-13T17:17:34.205 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:17:34.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:17:34.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.330+0000 7f09ac24f640 1 Processor -- start 2024-09-13T17:17:34.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09ac24f640 1 -- start start 2024-09-13T17:17:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09ac24f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a414e0c0 0x7f09a40a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09ac24f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f09a40c4740 0x7f09a40c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09ac24f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40c2780 con 0x7f09a40c1f50 2024-09-13T17:17:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09ac24f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40c2a60 con 0x7f09a40c4740 2024-09-13T17:17:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09ac24f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40c2d40 con 0x7f09a414e0c0 2024-09-13T17:17:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09aa7ec640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f09a40c4740 0x7f09a40c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09aa7ec640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f09a40c4740 0x7f09a40c4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54874/0 (socket says 172.21.15.39:54874) 2024-09-13T17:17:34.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09aa7ec640 1 -- 172.21.15.39:0/790475990 learned_addr learned my addr 172.21.15.39:0/790475990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:34.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09a9feb640 1 --2- 172.21.15.39:0/790475990 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a414e0c0 0x7f09a40a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09aa7ec640 1 -- 172.21.15.39:0/790475990 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a414e0c0 msgr2=0x7f09a40a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:34.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09aa7ec640 1 --2- 172.21.15.39:0/790475990 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a414e0c0 0x7f09a40a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09aa7ec640 1 -- 172.21.15.39:0/790475990 >> v1:172.21.15.84:6789/0 conn(0x7f09a40c1f50 legacy=0x7f09a406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:34.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09aa7ec640 1 -- 172.21.15.39:0/790475990 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40c3020 con 0x7f09a40c4740 2024-09-13T17:17:34.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09aa7ec640 1 --2- 172.21.15.39:0/790475990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a40c4740 0x7f09a40c4bf0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f099c009e50 tx=0x7f099c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5596ec3afac2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09a8fe9640 1 -- 172.21.15.39:0/790475990 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0a56e0 con 0x7f09a40c4740 2024-09-13T17:17:34.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09a8fe9640 1 -- 172.21.15.39:0/790475990 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c0a5840 con 0x7f09a40c4740 2024-09-13T17:17:34.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.334+0000 7f09a8fe9640 1 -- 172.21.15.39:0/790475990 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0a5bd0 con 0x7f09a40c4740 2024-09-13T17:17:34.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- 172.21.15.39:0/790475990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a40c4740 msgr2=0x7f09a40c4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 --2- 172.21.15.39:0/790475990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a40c4740 0x7f09a40c4bf0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f099c009e50 tx=0x7f099c01c930 comp rx=0 tx=0).stop 2024-09-13T17:17:34.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- 172.21.15.39:0/790475990 shutdown_connections 2024-09-13T17:17:34.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 --2- 172.21.15.39:0/790475990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a40c4740 0x7f09a40c4bf0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 --2- 172.21.15.39:0/790475990 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a414e0c0 0x7f09a40a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- 172.21.15.39:0/790475990 >> 172.21.15.39:0/790475990 conn(0x7f09a4069e10 msgr2=0x7f09a406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:34.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- 172.21.15.39:0/790475990 shutdown_connections 2024-09-13T17:17:34.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- 172.21.15.39:0/790475990 wait complete. 2024-09-13T17:17:34.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 Processor -- start 2024-09-13T17:17:34.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- start start 2024-09-13T17:17:34.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a40c1f50 0x7f09a4150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a40c4740 0x7f09a4140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a414e0c0 0x7f09a4141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4070200 con 0x7f09a414e0c0 2024-09-13T17:17:34.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4071c50 con 0x7f09a40c4740 2024-09-13T17:17:34.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a406fc40 con 0x7f09a40c1f50 2024-09-13T17:17:34.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09aa7ec640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a414e0c0 0x7f09a4141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09aa7ec640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a414e0c0 0x7f09a4141040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54888/0 (socket says 172.21.15.39:54888) 2024-09-13T17:17:34.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09aa7ec640 1 -- 172.21.15.39:0/3340055710 learned_addr learned my addr 172.21.15.39:0/3340055710 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:34.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09a9feb640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a40c1f50 0x7f09a4150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09a97ea640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a40c4740 0x7f09a4140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09aa7ec640 1 -- 172.21.15.39:0/3340055710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a40c1f50 msgr2=0x7f09a4150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09aa7ec640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a40c1f50 0x7f09a4150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09aa7ec640 1 -- 172.21.15.39:0/3340055710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a40c4740 msgr2=0x7f09a4140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09aa7ec640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a40c4740 0x7f09a4140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09aa7ec640 1 -- 172.21.15.39:0/3340055710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c009b30 con 0x7f09a414e0c0 2024-09-13T17:17:34.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09aa7ec640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a414e0c0 0x7f09a4141040 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f099c01afe0 tx=0x7f099c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09967fc640 1 -- 172.21.15.39:0/3340055710 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0b8670 con 0x7f09a414e0c0 2024-09-13T17:17:34.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09967fc640 1 -- 172.21.15.39:0/3340055710 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c0b87f0 con 0x7f09a414e0c0 2024-09-13T17:17:34.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a406ff20 con 0x7f09a414e0c0 2024-09-13T17:17:34.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09967fc640 1 -- 172.21.15.39:0/3340055710 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0b8ba0 con 0x7f09a414e0c0 2024-09-13T17:17:34.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.338+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a40718e0 con 0x7f09a414e0c0 2024-09-13T17:17:34.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.342+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a40c4bf0 con 0x7f09a414e0c0 2024-09-13T17:17:34.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.342+0000 7f09967fc640 1 -- 172.21.15.39:0/3340055710 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f099c0b8d00 con 0x7f09a414e0c0 2024-09-13T17:17:34.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.342+0000 7f09967fc640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.342+0000 7f09967fc640 1 -- 172.21.15.39:0/3340055710 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f099c120a70 con 0x7f09a414e0c0 2024-09-13T17:17:34.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.342+0000 7f09a9feb640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.342+0000 7f09a9feb640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f09980096d0 tx=0x7f0998008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.346+0000 7f09967fc640 1 -- 172.21.15.39:0/3340055710 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f099c0eceb0 con 0x7f09a414e0c0 2024-09-13T17:17:34.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.510+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09a4150a50 con 0x7f09a414e0c0 2024-09-13T17:17:34.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.510+0000 7f09967fc640 1 -- 172.21.15.39:0/3340055710 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+25425 (secure 0 0 0) 0x7f099c0bc070 con 0x7f09a414e0c0 2024-09-13T17:17:34.514 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:34.515 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":63,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T17:16:34.950618+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6835/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6834/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.39:6834/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6837/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6835/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.84:6834/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.39:6836/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.39:6835/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.84:6835/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.39:6835/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.84:6834/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.39:6834/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.39:6834/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.39:6836/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.84:6832/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.84:6835/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.84:6832/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6836/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.84:6834/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.39:6835/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.84:6832/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6834/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6835/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.84:6834/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6835/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.39:6835/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6836/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6833/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.39:6835/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.39:6837/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6834/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6835/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6834/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.39:6835/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1750130579":"2024-09-14T17:09:29.828673+0000","172.21.15.39:6834/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.84:6832/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.39:6834/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.39:6837/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6836/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6835/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6834/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.84:6835/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.84:6833/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.84:6833/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6835/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.39:6834/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.84:6832/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6833/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6834/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6835/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6836/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6834/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6833/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6835/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6834/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.39:6834/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6835/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6836/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.39:6835/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.84:6833/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6836/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6837/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.84:6834/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.84:6834/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.84:6835/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.39:6837/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.39:6837/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6835/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.84:6832/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6835/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/1750130579":"2024-09-14T17:09:29.828673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:17:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 msgr2=0x7f0988068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f09980096d0 tx=0x7f0998008040 comp rx=0 tx=0).stop 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a414e0c0 msgr2=0x7f09a4141040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a414e0c0 0x7f09a4141040 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f099c01afe0 tx=0x7f099c0196d0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 shutdown_connections 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0988066690 0x7f0988068b50 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09a40c4740 0x7f09a4140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09a414e0c0 0x7f09a4141040 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 --2- 172.21.15.39:0/3340055710 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09a40c1f50 0x7f09a4150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 >> 172.21.15.39:0/3340055710 conn(0x7f09a4069e10 msgr2=0x7f09a414ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 shutdown_connections 2024-09-13T17:17:34.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.514+0000 7f09ac24f640 1 -- 172.21.15.39:0/3340055710 wait complete. 2024-09-13T17:17:34.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T16:47:47.637217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'a_metadata', 'create_time': '2024-09-13T16:47:57.579177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'a'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'a_data', 'create_time': '2024-09-13T16:47:58.000542+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'a'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'b_metadata', 'create_time': '2024-09-13T16:48:06.063601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'b'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'b_data', 'create_time': '2024-09-13T16:48:07.109684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'b'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:17:34.535 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:17:34.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb12fc5640 1 Processor -- start 2024-09-13T17:17:34.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb12fc5640 1 -- start start 2024-09-13T17:17:34.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb12fc5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c0aa630 0x7feb0c152320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb12fc5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb0c070630 0x7feb0c15c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb12fc5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c151bd0 con 0x7feb0c14f580 2024-09-13T17:17:34.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb12fc5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c061910 con 0x7feb0c070630 2024-09-13T17:17:34.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb12fc5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0a8aa0 con 0x7feb0c0aa630 2024-09-13T17:17:34.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb10d61640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7feb0c14f580 0x7feb0c0aace0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51832/0 (socket says 172.21.15.39:51832) 2024-09-13T17:17:34.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb10d61640 1 -- 172.21.15.39:0/2812373595 learned_addr learned my addr 172.21.15.39:0/2812373595 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0bfff640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c0aa630 0x7feb0c152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675716326 0 0) 0x7feb0c151bd0 con 0x7feb0c14f580 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feafc003530 con 0x7feb0c14f580 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb11562640 1 --2- 172.21.15.39:0/2812373595 >> v2:172.21.15.84:3300/0 conn(0x7feb0c070630 0x7feb0c15c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 330504561 0 0) 0x7feafc003530 con 0x7feb0c14f580 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c0aa630 msgr2=0x7feb0c152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c0aa630 0x7feb0c152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 >> v2:172.21.15.84:3300/0 conn(0x7feb0c070630 msgr2=0x7feb0c15c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 --2- 172.21.15.39:0/2812373595 >> v2:172.21.15.84:3300/0 conn(0x7feb0c070630 0x7feb0c15c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0a8680 con 0x7feb0c14f580 2024-09-13T17:17:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7feb00001680 con 0x7feb0c14f580 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 >> v1:172.21.15.39:6789/0 conn(0x7feb0c14f580 legacy=0x7feb0c0aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feafc003980 0x7feafc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feafc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c0aa630 0x7feafc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c061910 con 0x7feafc0047d0 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc00b990 con 0x7feafc003980 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc00bb00 con 0x7feb0c0aa630 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0bfff640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feafc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb10d61640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feafc003980 0x7feafc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb11562640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c0aa630 0x7feafc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0bfff640 1 -- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c0aa630 msgr2=0x7feafc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0bfff640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c0aa630 0x7feafc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0bfff640 1 -- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feafc003980 msgr2=0x7feafc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0bfff640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feafc003980 0x7feafc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.662+0000 7feb0bfff640 1 -- 172.21.15.39:0/2812373595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc003530 con 0x7feafc0047d0 2024-09-13T17:17:34.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb0bfff640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feafc004c50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009ac0 tx=0x7feaf801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0250b017f5a06de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80059a0 con 0x7feafc0047d0 2024-09-13T17:17:34.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf8005b00 con 0x7feafc0047d0 2024-09-13T17:17:34.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb0b7fe640 1 -- 172.21.15.39:0/2812373595 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf8006030 con 0x7feafc0047d0 2024-09-13T17:17:34.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 -- 172.21.15.39:0/2812373595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 msgr2=0x7feafc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 --2- 172.21.15.39:0/2812373595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feafc004c50 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009ac0 tx=0x7feaf801c9c0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb11562640 1 -- 172.21.15.39:0/2812373595 reap_dead start 2024-09-13T17:17:34.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 -- 172.21.15.39:0/2812373595 shutdown_connections 2024-09-13T17:17:34.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 -- 172.21.15.39:0/2812373595 >> 172.21.15.39:0/2812373595 conn(0x7feb0c069e10 msgr2=0x7feb0c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:34.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 -- 172.21.15.39:0/2812373595 shutdown_connections 2024-09-13T17:17:34.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 -- 172.21.15.39:0/2812373595 wait complete. 2024-09-13T17:17:34.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 Processor -- start 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 -- start start 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feb0c14a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c14aca0 0x7feb0c157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0c157880 0x7feb0c0bf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c14a2f0 con 0x7feafc0047d0 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c06bd80 con 0x7feb0c157880 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb12fc5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c149830 con 0x7feb0c14aca0 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb10d61640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feb0c14a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb10d61640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feb0c14a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54908/0 (socket says 172.21.15.39:54908) 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.666+0000 7feb10d61640 1 -- 172.21.15.39:0/1653358433 learned_addr learned my addr 172.21.15.39:0/1653358433 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb0bfff640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c14aca0 0x7feb0c157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb11562640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0c157880 0x7feb0c0bf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb10d61640 1 -- 172.21.15.39:0/1653358433 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c14aca0 msgr2=0x7feb0c157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb10d61640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c14aca0 0x7feb0c157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb10d61640 1 -- 172.21.15.39:0/1653358433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0c157880 msgr2=0x7feb0c0bf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb10d61640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0c157880 0x7feb0c0bf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb10d61640 1 -- 172.21.15.39:0/1653358433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0ab570 con 0x7feafc0047d0 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb10d61640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feb0c14a760 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7feb0000e760 tx=0x7feb0000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb097fa640 1 -- 172.21.15.39:0/1653358433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0000ee70 con 0x7feafc0047d0 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb097fa640 1 -- 172.21.15.39:0/1653358433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0000c8d0 con 0x7feafc0047d0 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c153280 con 0x7feafc0047d0 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb097fa640 1 -- 172.21.15.39:0/1653358433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0000cc80 con 0x7feafc0047d0 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c14b6e0 con 0x7feafc0047d0 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c0c0480 con 0x7feafc0047d0 2024-09-13T17:17:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb097fa640 1 -- 172.21.15.39:0/1653358433 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feb0001d460 con 0x7feafc0047d0 2024-09-13T17:17:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb097fa640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaf00666c0 0x7feaf0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb0bfff640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaf00666c0 0x7feaf0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb097fa640 1 -- 172.21.15.39:0/1653358433 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7feb000851a0 con 0x7feafc0047d0 2024-09-13T17:17:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.670+0000 7feb0bfff640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaf00666c0 0x7feaf0068b80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7feaf80ae000 tx=0x7feaf80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.674+0000 7feb097fa640 1 -- 172.21.15.39:0/1653358433 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feb00051490 con 0x7feafc0047d0 2024-09-13T17:17:34.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.838+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb0c061370 con 0x7feafc0047d0 2024-09-13T17:17:34.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb097fa640 1 -- 172.21.15.39:0/1653358433 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb00054ab0 con 0x7feafc0047d0 2024-09-13T17:17:34.843 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-13T17:17:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaf00666c0 msgr2=0x7feaf0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaf00666c0 0x7feaf0068b80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7feaf80ae000 tx=0x7feaf80046d0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 msgr2=0x7feb0c14a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feb0c14a760 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7feb0000e760 tx=0x7feb0000ec20 comp rx=0 tx=0).stop 2024-09-13T17:17:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 shutdown_connections 2024-09-13T17:17:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7feaf00666c0 0x7feaf0068b80 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0c157880 0x7feb0c0bf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7feb0c14aca0 0x7feb0c157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 --2- 172.21.15.39:0/1653358433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feafc0047d0 0x7feb0c14a760 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 >> 172.21.15.39:0/1653358433 conn(0x7feb0c069e10 msgr2=0x7feb0c0c29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 shutdown_connections 2024-09-13T17:17:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.842+0000 7feb12fc5640 1 -- 172.21.15.39:0/1653358433 wait complete. 2024-09-13T17:17:34.863 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_metadata pg_num 2024-09-13T17:17:34.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.982+0000 7f5d511f1640 1 Processor -- start 2024-09-13T17:17:34.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.982+0000 7f5d511f1640 1 -- start start 2024-09-13T17:17:34.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d511f1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d4c0a4d60 0x7f5d4c14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d511f1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c14f4b0 0x7f5d4c151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d511f1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4c155d20 con 0x7f5d4c0aa2d0 2024-09-13T17:17:34.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d511f1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c155b30 con 0x7f5d4c0a4d60 2024-09-13T17:17:34.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d511f1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0aadc0 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d43fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d4c0a4d60 0x7f5d4c14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d4ad9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5d4c0aa2d0 0x7f5d4c0aa6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51848/0 (socket says 172.21.15.39:51848) 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d4ad9d640 1 -- 172.21.15.39:0/2185029137 learned_addr learned my addr 172.21.15.39:0/2185029137 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d4b59e640 1 --2- 172.21.15.39:0/2185029137 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c14f4b0 0x7f5d4c151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d4a59c640 1 -- 172.21.15.39:0/2185029137 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747984662 0 0) 0x7f5d4c155d20 con 0x7f5d4c0aa2d0 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d4a59c640 1 -- 172.21.15.39:0/2185029137 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d38003530 con 0x7f5d4c0aa2d0 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d43fff640 1 -- 172.21.15.39:0/2185029137 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c14f4b0 msgr2=0x7f5d4c151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d43fff640 1 --2- 172.21.15.39:0/2185029137 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c14f4b0 0x7f5d4c151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d43fff640 1 -- 172.21.15.39:0/2185029137 >> v1:172.21.15.39:6789/0 conn(0x7f5d4c0aa2d0 legacy=0x7f5d4c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d43fff640 1 -- 172.21.15.39:0/2185029137 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0c4750 con 0x7f5d4c0a4d60 2024-09-13T17:17:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d43fff640 1 --2- 172.21.15.39:0/2185029137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c0a4d60 0x7f5d4c14ef70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5d340097f0 tx=0x7f5d3401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ab9adfffdbba94 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d4a59c640 1 -- 172.21.15.39:0/2185029137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c152a40 con 0x7f5d4c0a4d60 2024-09-13T17:17:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d4a59c640 1 -- 172.21.15.39:0/2185029137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d340a7050 con 0x7f5d4c0a4d60 2024-09-13T17:17:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.986+0000 7f5d4a59c640 1 -- 172.21.15.39:0/2185029137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d340ab800 con 0x7f5d4c0a4d60 2024-09-13T17:17:34.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- 172.21.15.39:0/2185029137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c0a4d60 msgr2=0x7f5d4c14ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 --2- 172.21.15.39:0/2185029137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c0a4d60 0x7f5d4c14ef70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5d340097f0 tx=0x7f5d3401c3e0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- 172.21.15.39:0/2185029137 shutdown_connections 2024-09-13T17:17:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 --2- 172.21.15.39:0/2185029137 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c14f4b0 0x7f5d4c151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 --2- 172.21.15.39:0/2185029137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c0a4d60 0x7f5d4c14ef70 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- 172.21.15.39:0/2185029137 >> 172.21.15.39:0/2185029137 conn(0x7f5d4c0c5790 msgr2=0x7f5d4c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- 172.21.15.39:0/2185029137 shutdown_connections 2024-09-13T17:17:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- 172.21.15.39:0/2185029137 wait complete. 2024-09-13T17:17:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 Processor -- start 2024-09-13T17:17:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- start start 2024-09-13T17:17:34.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d4c0a4d60 0x7f5d4c0c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c0aa2d0 0x7f5d4c0bee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c14f4b0 0x7f5d4c06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0712e0 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c06f320 con 0x7f5d4c0a4d60 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0717c0 con 0x7f5d4c0aa2d0 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c14f4b0 0x7f5d4c06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c14f4b0 0x7f5d4c06f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54930/0 (socket says 172.21.15.39:54930) 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4b59e640 1 -- 172.21.15.39:0/1576984358 learned_addr learned my addr 172.21.15.39:0/1576984358 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d43fff640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c0aa2d0 0x7f5d4c0bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4ad9d640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d4c0a4d60 0x7f5d4c0c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4b59e640 1 -- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c0aa2d0 msgr2=0x7f5d4c0bee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4b59e640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c0aa2d0 0x7f5d4c0bee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4b59e640 1 -- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d4c0a4d60 msgr2=0x7f5d4c0c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:34.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4b59e640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d4c0a4d60 0x7f5d4c0c2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4b59e640 1 -- 172.21.15.39:0/1576984358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d34009420 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d43fff640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c0aa2d0 0x7f5d4c0bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4ad9d640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d4c0a4d60 0x7f5d4c0c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d4b59e640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c14f4b0 0x7f5d4c06f7b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f5d44007e60 tx=0x7f5d4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d437fe640 1 -- 172.21.15.39:0/1576984358 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44014420 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d437fe640 1 -- 172.21.15.39:0/1576984358 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d440145a0 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c071aa0 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d437fe640 1 -- 172.21.15.39:0/1576984358 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44014950 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.990+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c072060 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.994+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c06f4b0 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.994+0000 7f5d437fe640 1 -- 172.21.15.39:0/1576984358 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d44014ab0 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.994+0000 7f5d437fe640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d2c066690 0x7f5d2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.994+0000 7f5d437fe640 1 -- 172.21.15.39:0/1576984358 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f5d44085bd0 con 0x7f5d4c14f4b0 2024-09-13T17:17:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.994+0000 7f5d4ad9d640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d2c066690 0x7f5d2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.994+0000 7f5d4ad9d640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d2c066690 0x7f5d2c068b50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005890 tx=0x7f5d3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:34.998+0000 7f5d437fe640 1 -- 172.21.15.39:0/1576984358 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d44051e60 con 0x7f5d4c14f4b0 2024-09-13T17:17:35.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.162+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d4c14ef70 con 0x7f5d4c14f4b0 2024-09-13T17:17:35.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.162+0000 7f5d437fe640 1 -- 172.21.15.39:0/1576984358 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_metadata", "var": "pg_num"}]=0 v63) v1 ==== 99+0+11 (secure 0 0 0) 0x7f5d44055480 con 0x7f5d4c14f4b0 2024-09-13T17:17:35.164 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T17:17:35.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.162+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d2c066690 msgr2=0x7f5d2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.162+0000 7f5d511f1640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d2c066690 0x7f5d2c068b50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005890 tx=0x7f5d3c009040 comp rx=0 tx=0).stop 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.162+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c14f4b0 msgr2=0x7f5d4c06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.162+0000 7f5d511f1640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c14f4b0 0x7f5d4c06f7b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f5d44007e60 tx=0x7f5d4400e4b0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.166+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 shutdown_connections 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.166+0000 7f5d511f1640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f5d2c066690 0x7f5d2c068b50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.166+0000 7f5d511f1640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d4c14f4b0 0x7f5d4c06f7b0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.166+0000 7f5d511f1640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f5d4c0aa2d0 0x7f5d4c0bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.166+0000 7f5d511f1640 1 --2- 172.21.15.39:0/1576984358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d4c0a4d60 0x7f5d4c0c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.166+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 >> 172.21.15.39:0/1576984358 conn(0x7f5d4c0c5790 msgr2=0x7f5d4c14c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.166+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 shutdown_connections 2024-09-13T17:17:35.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.166+0000 7f5d511f1640 1 -- 172.21.15.39:0/1576984358 wait complete. 2024-09-13T17:17:35.184 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get a_data pg_num 2024-09-13T17:17:35.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecc112d640 1 Processor -- start 2024-09-13T17:17:35.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecc112d640 1 -- start start 2024-09-13T17:17:35.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecc112d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc14acf0 0x7fecbc151550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecc112d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fecbc0c4710 0x7fecbc06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecc112d640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc154ab0 con 0x7fecbc0c4b70 2024-09-13T17:17:35.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecc112d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc154d90 con 0x7fecbc0c4710 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecc112d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc150d50 con 0x7fecbc14acf0 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecba59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fecbc0c4710 0x7fecbc06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecba59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fecbc0c4710 0x7fecbc06ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54938/0 (socket says 172.21.15.39:54938) 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecba59c640 1 -- 172.21.15.39:0/2879035866 learned_addr learned my addr 172.21.15.39:0/2879035866 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecbad9d640 1 --2- 172.21.15.39:0/2879035866 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc14acf0 0x7fecbc151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecba59c640 1 -- 172.21.15.39:0/2879035866 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc14acf0 msgr2=0x7fecbc151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecba59c640 1 --2- 172.21.15.39:0/2879035866 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc14acf0 0x7fecbc151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecba59c640 1 -- 172.21.15.39:0/2879035866 >> v1:172.21.15.84:6789/0 conn(0x7fecbc0c4b70 legacy=0x7fecbc153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecba59c640 1 -- 172.21.15.39:0/2879035866 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc151030 con 0x7fecbc0c4710 2024-09-13T17:17:35.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.306+0000 7fecbad9d640 1 --2- 172.21.15.39:0/2879035866 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc14acf0 0x7fecbc151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:35.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecba59c640 1 --2- 172.21.15.39:0/2879035866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4710 0x7fecbc06ae20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7feca400bb40 tx=0x7feca401eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5454c69342203e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecb9d9b640 1 -- 172.21.15.39:0/2879035866 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca40a7a90 con 0x7fecbc0c4710 2024-09-13T17:17:35.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecb9d9b640 1 -- 172.21.15.39:0/2879035866 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca40a7bf0 con 0x7fecbc0c4710 2024-09-13T17:17:35.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecb9d9b640 1 -- 172.21.15.39:0/2879035866 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca40af440 con 0x7fecbc0c4710 2024-09-13T17:17:35.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecc112d640 1 -- 172.21.15.39:0/2879035866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4710 msgr2=0x7fecbc06ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecc112d640 1 --2- 172.21.15.39:0/2879035866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4710 0x7fecbc06ae20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7feca400bb40 tx=0x7feca401eae0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecc112d640 1 -- 172.21.15.39:0/2879035866 shutdown_connections 2024-09-13T17:17:35.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecc112d640 1 --2- 172.21.15.39:0/2879035866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4710 0x7fecbc06ae20 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecc112d640 1 --2- 172.21.15.39:0/2879035866 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc14acf0 0x7fecbc151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecc112d640 1 -- 172.21.15.39:0/2879035866 >> 172.21.15.39:0/2879035866 conn(0x7fecbc0c32a0 msgr2=0x7fecbc0c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecc112d640 1 -- 172.21.15.39:0/2879035866 shutdown_connections 2024-09-13T17:17:35.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.310+0000 7fecc112d640 1 -- 172.21.15.39:0/2879035866 wait complete. 2024-09-13T17:17:35.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 Processor -- start 2024-09-13T17:17:35.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 -- start start 2024-09-13T17:17:35.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc0c4710 0x7fecbc0c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4b70 0x7fecbc071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecbc14acf0 0x7fecbc1443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc154140 con 0x7fecbc0c4b70 2024-09-13T17:17:35.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc070cb0 con 0x7fecbc14acf0 2024-09-13T17:17:35.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc071f40 con 0x7fecbc0c4710 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecba59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4b70 0x7fecbc071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecba59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4b70 0x7fecbc071310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54946/0 (socket says 172.21.15.39:54946) 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecba59c640 1 -- 172.21.15.39:0/3603566602 learned_addr learned my addr 172.21.15.39:0/3603566602 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecbb59e640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecbc14acf0 0x7fecbc1443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecba59c640 1 -- 172.21.15.39:0/3603566602 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc0c4710 msgr2=0x7fecbc0c0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecbad9d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc0c4710 0x7fecbc0c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecba59c640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc0c4710 0x7fecbc0c0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecba59c640 1 -- 172.21.15.39:0/3603566602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecbc14acf0 msgr2=0x7fecbc1443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecba59c640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecbc14acf0 0x7fecbc1443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecba59c640 1 -- 172.21.15.39:0/3603566602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca400b770 con 0x7fecbc0c4b70 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecbad9d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc0c4710 0x7fecbc0c0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:35.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecba59c640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4b70 0x7fecbc071310 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7feca400b2c0 tx=0x7feca401b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecb37fe640 1 -- 172.21.15.39:0/3603566602 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca40a7a90 con 0x7fecbc0c4b70 2024-09-13T17:17:35.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecb37fe640 1 -- 172.21.15.39:0/3603566602 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca40ba420 con 0x7fecbc0c4b70 2024-09-13T17:17:35.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc154880 con 0x7fecbc0c4b70 2024-09-13T17:17:35.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecb37fe640 1 -- 172.21.15.39:0/3603566602 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca40ba7f0 con 0x7fecbc0c4b70 2024-09-13T17:17:35.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc153280 con 0x7fecbc0c4b70 2024-09-13T17:17:35.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc070a80 con 0x7fecbc0c4b70 2024-09-13T17:17:35.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecb37fe640 1 -- 172.21.15.39:0/3603566602 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feca40b6040 con 0x7fecbc0c4b70 2024-09-13T17:17:35.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecb37fe640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec9c0666c0 0x7fec9c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecb37fe640 1 -- 172.21.15.39:0/3603566602 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7feca4122940 con 0x7fecbc0c4b70 2024-09-13T17:17:35.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.314+0000 7fecbad9d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec9c0666c0 0x7fec9c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.318+0000 7fecbad9d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec9c0666c0 0x7fec9c068b80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fecac008890 tx=0x7fecac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.318+0000 7fecb37fe640 1 -- 172.21.15.39:0/3603566602 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fecbc070a80 con 0x7fecbc0c4b70 2024-09-13T17:17:35.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.482+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"} v 0) v1 -- 0x7fecbc070e20 con 0x7fecbc0c4b70 2024-09-13T17:17:35.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.482+0000 7fecb37fe640 1 -- 172.21.15.39:0/3603566602 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "a_data", "var": "pg_num"}]=0 v63) v1 ==== 95+0+11 (secure 0 0 0) 0x7fecbc070e20 con 0x7fecbc0c4b70 2024-09-13T17:17:35.485 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T17:17:35.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec9c0666c0 msgr2=0x7fec9c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec9c0666c0 0x7fec9c068b80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fecac008890 tx=0x7fecac008040 comp rx=0 tx=0).stop 2024-09-13T17:17:35.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4b70 msgr2=0x7fecbc071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4b70 0x7fecbc071310 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7feca400b2c0 tx=0x7feca401b8e0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 shutdown_connections 2024-09-13T17:17:35.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fec9c0666c0 0x7fec9c068b80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fecbc14acf0 0x7fecbc1443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecbc0c4b70 0x7fecbc071310 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 --2- 172.21.15.39:0/3603566602 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fecbc0c4710 0x7fecbc0c0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 >> 172.21.15.39:0/3603566602 conn(0x7fecbc0c32a0 msgr2=0x7fecbc1491a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 shutdown_connections 2024-09-13T17:17:35.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.486+0000 7fecc112d640 1 -- 172.21.15.39:0/3603566602 wait complete. 2024-09-13T17:17:35.504 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_metadata pg_num 2024-09-13T17:17:35.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.630+0000 7f2278de8640 1 Processor -- start 2024-09-13T17:17:35.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.630+0000 7f2278de8640 1 -- start start 2024-09-13T17:17:35.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f2278de8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740a4d60 0x7f227414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f2278de8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2274155d20 con 0x7f22740aa2d0 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f2278de8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22740c4750 con 0x7f227414f4b0 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f2278de8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2274155b30 con 0x7f22740a4d60 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227259c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f22740aa2d0 0x7f22740aa6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51860/0 (socket says 172.21.15.39:51860) 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227259c640 1 -- 172.21.15.39:0/1651156979 learned_addr learned my addr 172.21.15.39:0/1651156979 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f2271d9b640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740a4d60 0x7f227414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503499930 0 0) 0x7f2274155d20 con 0x7f22740aa2d0 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f225c003530 con 0x7f22740aa2d0 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4211833019 0 0) 0x7f225c003530 con 0x7f22740aa2d0 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740a4d60 msgr2=0x7f227414ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740a4d60 0x7f227414ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 >> v1:172.21.15.84:6789/0 conn(0x7f227414f4b0 legacy=0x7f2274151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274155fc0 con 0x7f22740aa2d0 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f2264001680 con 0x7f22740aa2d0 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 >> v1:172.21.15.39:6789/0 conn(0x7f22740aa2d0 legacy=0x7f22740aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f225c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740a4d60 0x7f225c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f225c004d10 0x7f225c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c00ba90 con 0x7f225c0039b0 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c00bc00 con 0x7f225c004d10 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c00bdc0 con 0x7f22740a4d60 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227259c640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f225c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f2271d9b640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740a4d60 0x7f225c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f2272d9d640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f225c004d10 0x7f225c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227259c640 1 -- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740a4d60 msgr2=0x7f225c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227259c640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740a4d60 0x7f225c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227259c640 1 -- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f225c004d10 msgr2=0x7f225c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227259c640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f225c004d10 0x7f225c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.634+0000 7f227259c640 1 -- 172.21.15.39:0/1651156979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c003530 con 0x7f225c0039b0 2024-09-13T17:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227259c640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f225c003dd0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f226400c1c0 tx=0x7f226401cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395f53d35d7433c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264005ae0 con 0x7f225c0039b0 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2264005c40 con 0x7f225c0039b0 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227159a640 1 -- 172.21.15.39:0/1651156979 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264007070 con 0x7f225c0039b0 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 -- 172.21.15.39:0/1651156979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 msgr2=0x7f225c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 --2- 172.21.15.39:0/1651156979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f225c003dd0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f226400c1c0 tx=0x7f226401cb80 comp rx=0 tx=0).stop 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2272d9d640 1 -- 172.21.15.39:0/1651156979 reap_dead start 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 -- 172.21.15.39:0/1651156979 shutdown_connections 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 -- 172.21.15.39:0/1651156979 >> 172.21.15.39:0/1651156979 conn(0x7f22740c5790 msgr2=0x7f22740c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 -- 172.21.15.39:0/1651156979 shutdown_connections 2024-09-13T17:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 -- 172.21.15.39:0/1651156979 wait complete. 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 Processor -- start 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 -- start start 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f2274154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22740c2b80 0x7f22741ea650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740c0700 0x7f22741eab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22740c04f0 con 0x7f225c0039b0 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2274154d80 con 0x7f22740c2b80 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2278de8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22741550b0 con 0x7f22740c0700 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f2274154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f2274154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54974/0 (socket says 172.21.15.39:54974) 2024-09-13T17:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227259c640 1 -- 172.21.15.39:0/3437395618 learned_addr learned my addr 172.21.15.39:0/3437395618 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2272d9d640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740c0700 0x7f22741eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2271d9b640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22740c2b80 0x7f22741ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227259c640 1 -- 172.21.15.39:0/3437395618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740c0700 msgr2=0x7f22741eab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227259c640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740c0700 0x7f22741eab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227259c640 1 -- 172.21.15.39:0/3437395618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22740c2b80 msgr2=0x7f22741ea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227259c640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22740c2b80 0x7f22741ea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f227259c640 1 -- 172.21.15.39:0/3437395618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22740c0210 con 0x7f225c0039b0 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.638+0000 7f2271d9b640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22740c2b80 0x7f22741ea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f227259c640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f2274154220 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f226401c9b0 tx=0x7f225c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2262ffd640 1 -- 172.21.15.39:0/3437395618 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22640b4070 con 0x7f225c0039b0 2024-09-13T17:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2262ffd640 1 -- 172.21.15.39:0/3437395618 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2264001440 con 0x7f225c0039b0 2024-09-13T17:17:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2262ffd640 1 -- 172.21.15.39:0/3437395618 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22640070a0 con 0x7f225c0039b0 2024-09-13T17:17:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2274154760 con 0x7f225c0039b0 2024-09-13T17:17:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22741eb0d0 con 0x7f225c0039b0 2024-09-13T17:17:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22740aa6b0 con 0x7f225c0039b0 2024-09-13T17:17:35.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2262ffd640 1 -- 172.21.15.39:0/3437395618 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2264007200 con 0x7f225c0039b0 2024-09-13T17:17:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2262ffd640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22540666c0 0x7f2254068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2262ffd640 1 -- 172.21.15.39:0/3437395618 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f226411ecd0 con 0x7f225c0039b0 2024-09-13T17:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2271d9b640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22540666c0 0x7f2254068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.642+0000 7f2271d9b640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22540666c0 0x7f2254068b80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2274151e60 tx=0x7f2268008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.646+0000 7f2262ffd640 1 -- 172.21.15.39:0/3437395618 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f22640eea50 con 0x7f225c0039b0 2024-09-13T17:17:35.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.758+0000 7ff34c700640 1 Processor -- start 2024-09-13T17:17:35.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.758+0000 7ff34c700640 1 -- start start 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34c700640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff34414aa40 0x7ff3440c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34c700640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3440c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34c700640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff3440c4710 0x7ff3440c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34c700640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff344151b30 con 0x7ff34414aa40 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34c700640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3440c6510 con 0x7ff3440c4710 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34c700640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3440c76a0 con 0x7ff3440c1750 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34a49c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff34414aa40 0x7ff3440c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34a49c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff34414aa40 0x7ff3440c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54984/0 (socket says 172.21.15.39:54984) 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34a49c640 1 -- 172.21.15.39:0/1941780495 learned_addr learned my addr 172.21.15.39:0/1941780495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff349c9b640 1 --2- 172.21.15.39:0/1941780495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3440c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34ac9d640 1 --2- 172.21.15.39:0/1941780495 >> v2:172.21.15.84:3300/0 conn(0x7ff3440c4710 0x7ff3440c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34a49c640 1 -- 172.21.15.39:0/1941780495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 msgr2=0x7ff3440c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:35.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34a49c640 1 --2- 172.21.15.39:0/1941780495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3440c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34a49c640 1 -- 172.21.15.39:0/1941780495 >> v2:172.21.15.84:3300/0 conn(0x7ff3440c4710 msgr2=0x7ff3440c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:35.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34a49c640 1 --2- 172.21.15.39:0/1941780495 >> v2:172.21.15.84:3300/0 conn(0x7ff3440c4710 0x7ff3440c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34a49c640 1 -- 172.21.15.39:0/1941780495 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff344151570 con 0x7ff34414aa40 2024-09-13T17:17:35.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff349c9b640 1 --2- 172.21.15.39:0/1941780495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3440c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:35.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34a49c640 1 --2- 172.21.15.39:0/1941780495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 0x7ff3440c1210 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ff344072610 tx=0x7ff34001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f8949bcfea4266 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34949a640 1 -- 172.21.15.39:0/1941780495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3400ab620 con 0x7ff34414aa40 2024-09-13T17:17:35.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34949a640 1 -- 172.21.15.39:0/1941780495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3400ab780 con 0x7ff34414aa40 2024-09-13T17:17:35.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34949a640 1 -- 172.21.15.39:0/1941780495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3400abb10 con 0x7ff34414aa40 2024-09-13T17:17:35.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34c700640 1 -- 172.21.15.39:0/1941780495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 msgr2=0x7ff3440c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.762+0000 7ff34c700640 1 --2- 172.21.15.39:0/1941780495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 0x7ff3440c1210 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ff344072610 tx=0x7ff34001c460 comp rx=0 tx=0).stop 2024-09-13T17:17:35.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- 172.21.15.39:0/1941780495 shutdown_connections 2024-09-13T17:17:35.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 --2- 172.21.15.39:0/1941780495 >> v2:172.21.15.84:3300/0 conn(0x7ff3440c4710 0x7ff3440c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 --2- 172.21.15.39:0/1941780495 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3440c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 --2- 172.21.15.39:0/1941780495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 0x7ff3440c1210 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- 172.21.15.39:0/1941780495 >> 172.21.15.39:0/1941780495 conn(0x7ff3440c3310 msgr2=0x7ff3440c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- 172.21.15.39:0/1941780495 shutdown_connections 2024-09-13T17:17:35.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- 172.21.15.39:0/1941780495 wait complete. 2024-09-13T17:17:35.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 Processor -- start 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- start start 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3441ec880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3440c4710 0x7ff3441495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 0x7ff344149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3441ea520 con 0x7ff34414aa40 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3441ea800 con 0x7ff3440c4710 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34414e150 con 0x7ff3440c1750 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34ac9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 0x7ff344149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34ac9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 0x7ff344149ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54996/0 (socket says 172.21.15.39:54996) 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34ac9d640 1 -- 172.21.15.39:0/97271273 learned_addr learned my addr 172.21.15.39:0/97271273 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff349c9b640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3440c4710 0x7ff3441495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34a49c640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3441ec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34ac9d640 1 -- 172.21.15.39:0/97271273 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 msgr2=0x7ff3441ec880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34ac9d640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3441ec880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34ac9d640 1 -- 172.21.15.39:0/97271273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3440c4710 msgr2=0x7ff3441495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34ac9d640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3440c4710 0x7ff3441495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34ac9d640 1 -- 172.21.15.39:0/97271273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400094d0 con 0x7ff34414aa40 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34a49c640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3441ec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34ac9d640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 0x7ff344149ae0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00cd90 tx=0x7ff33c00e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff33affd640 1 -- 172.21.15.39:0/97271273 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c019070 con 0x7ff34414aa40 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff33affd640 1 -- 172.21.15.39:0/97271273 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c00ed40 con 0x7ff34414aa40 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3440ac0f0 con 0x7ff34414aa40 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff33affd640 1 -- 172.21.15.39:0/97271273 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c011070 con 0x7ff34414aa40 2024-09-13T17:17:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3440abe10 con 0x7ff34414aa40 2024-09-13T17:17:35.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.766+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3441ea970 con 0x7ff34414aa40 2024-09-13T17:17:35.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.770+0000 7ff33affd640 1 -- 172.21.15.39:0/97271273 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff33c015430 con 0x7ff34414aa40 2024-09-13T17:17:35.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.770+0000 7ff33affd640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff328066690 0x7ff328068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.770+0000 7ff34a49c640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff328066690 0x7ff328068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.770+0000 7ff33affd640 1 -- 172.21.15.39:0/97271273 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7ff33c087150 con 0x7ff34414aa40 2024-09-13T17:17:35.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.770+0000 7ff34a49c640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff328066690 0x7ff328068b50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff3400af000 tx=0x7ff340008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.774+0000 7ff33affd640 1 -- 172.21.15.39:0/97271273 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff33c053500 con 0x7ff34414aa40 2024-09-13T17:17:35.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.802+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22740614e0 con 0x7f225c0039b0 2024-09-13T17:17:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.802+0000 7f2262ffd640 1 -- 172.21.15.39:0/3437395618 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_metadata", "var": "pg_num"}]=0 v63) v1 ==== 99+0+11 (secure 0 0 0) 0x7f22640ee5e0 con 0x7f225c0039b0 2024-09-13T17:17:35.805 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-13T17:17:35.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22540666c0 msgr2=0x7f2254068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22540666c0 0x7f2254068b80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2274151e60 tx=0x7f2268008040 comp rx=0 tx=0).stop 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 msgr2=0x7f2274154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f2274154220 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f226401c9b0 tx=0x7f225c005800 comp rx=0 tx=0).stop 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 shutdown_connections 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f22540666c0 0x7f2254068b80 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22740c2b80 0x7f22741ea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f22740c0700 0x7f22741eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 --2- 172.21.15.39:0/3437395618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0039b0 0x7f2274154220 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 >> 172.21.15.39:0/3437395618 conn(0x7f22740c5790 msgr2=0x7f227414f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 shutdown_connections 2024-09-13T17:17:35.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.806+0000 7f2278de8640 1 -- 172.21.15.39:0/3437395618 wait complete. 2024-09-13T17:17:35.823 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get b_data pg_num 2024-09-13T17:17:35.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63e017b640 1 Processor -- start 2024-09-13T17:17:35.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63e017b640 1 -- start start 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63e017b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63d806d7f0 0x7f63d814f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63e017b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63d814f660 0x7f63d8151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63e017b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d80c0a50 con 0x7f63d80c6200 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63e017b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d8061410 con 0x7f63d806d7f0 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63e017b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d80616b0 con 0x7f63d814f660 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63dd716640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63d806d7f0 0x7f63d814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63dd716640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63d806d7f0 0x7f63d814f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55002/0 (socket says 172.21.15.39:55002) 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63dd716640 1 -- 172.21.15.39:0/3854595667 learned_addr learned my addr 172.21.15.39:0/3854595667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63de718640 1 --2- 172.21.15.39:0/3854595667 >> v2:172.21.15.84:3300/0 conn(0x7f63d814f660 0x7f63d8151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63dd716640 1 -- 172.21.15.39:0/3854595667 >> v2:172.21.15.84:3300/0 conn(0x7f63d814f660 msgr2=0x7f63d8151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:35.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63dd716640 1 --2- 172.21.15.39:0/3854595667 >> v2:172.21.15.84:3300/0 conn(0x7f63d814f660 0x7f63d8151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63dd716640 1 -- 172.21.15.39:0/3854595667 >> v1:172.21.15.84:6789/0 conn(0x7f63d80c6200 legacy=0x7f63d806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:35.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.934+0000 7f63dd716640 1 -- 172.21.15.39:0/3854595667 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80c0ca0 con 0x7f63d806d7f0 2024-09-13T17:17:35.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 --2- 172.21.15.39:0/3854595667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d806d7f0 0x7f63d814f120 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f63d80c70f0 tx=0x7f63c801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31964e33bf77ed32 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dcf15640 1 -- 172.21.15.39:0/3854595667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80ab600 con 0x7f63d806d7f0 2024-09-13T17:17:35.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dcf15640 1 -- 172.21.15.39:0/3854595667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c80a7050 con 0x7f63d806d7f0 2024-09-13T17:17:35.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dcf15640 1 -- 172.21.15.39:0/3854595667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80ab980 con 0x7f63d806d7f0 2024-09-13T17:17:35.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 -- 172.21.15.39:0/3854595667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d806d7f0 msgr2=0x7f63d814f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 --2- 172.21.15.39:0/3854595667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d806d7f0 0x7f63d814f120 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f63d80c70f0 tx=0x7f63c801c390 comp rx=0 tx=0).stop 2024-09-13T17:17:35.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 -- 172.21.15.39:0/3854595667 shutdown_connections 2024-09-13T17:17:35.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 --2- 172.21.15.39:0/3854595667 >> v2:172.21.15.84:3300/0 conn(0x7f63d814f660 0x7f63d8151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 --2- 172.21.15.39:0/3854595667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d806d7f0 0x7f63d814f120 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 -- 172.21.15.39:0/3854595667 >> 172.21.15.39:0/3854595667 conn(0x7f63d80a8860 msgr2=0x7f63d806bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 -- 172.21.15.39:0/3854595667 shutdown_connections 2024-09-13T17:17:35.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 -- 172.21.15.39:0/3854595667 wait complete. 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 Processor -- start 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 -- start start 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63d806d7f0 0x7f63d80c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d80c6200 0x7f63d8143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63d814f660 0x7f63d8143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8150c70 con 0x7f63d80c6200 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d814fe90 con 0x7f63d806d7f0 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63e017b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d806ad30 con 0x7f63d814f660 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d80c6200 0x7f63d8143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d80c6200 0x7f63d8143940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55008/0 (socket says 172.21.15.39:55008) 2024-09-13T17:17:35.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 -- 172.21.15.39:0/1874093354 learned_addr learned my addr 172.21.15.39:0/1874093354 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63ddf17640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63d806d7f0 0x7f63d80c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63de718640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63d814f660 0x7f63d8143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 -- 172.21.15.39:0/1874093354 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63d814f660 msgr2=0x7f63d8143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63d814f660 0x7f63d8143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 -- 172.21.15.39:0/1874093354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63d806d7f0 msgr2=0x7f63d80c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63d806d7f0 0x7f63d80c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 -- 172.21.15.39:0/1874093354 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c80094d0 con 0x7f63d80c6200 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.938+0000 7f63dd716640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d80c6200 0x7f63d8143940 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f63c80ae040 tx=0x7f63c8019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63ce7fc640 1 -- 172.21.15.39:0/1874093354 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80b4440 con 0x7f63d80c6200 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63ce7fc640 1 -- 172.21.15.39:0/1874093354 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c80b45a0 con 0x7f63d80c6200 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d8151b80 con 0x7f63d80c6200 2024-09-13T17:17:35.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63ce7fc640 1 -- 172.21.15.39:0/1874093354 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80b4930 con 0x7f63d80c6200 2024-09-13T17:17:35.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d814fb40 con 0x7f63d80c6200 2024-09-13T17:17:35.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d814f120 con 0x7f63d80c6200 2024-09-13T17:17:35.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63ce7fc640 1 -- 172.21.15.39:0/1874093354 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63c80b2070 con 0x7f63d80c6200 2024-09-13T17:17:35.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63ce7fc640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63bc066690 0x7f63bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:35.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63ce7fc640 1 -- 172.21.15.39:0/1874093354 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f63c811f1f0 con 0x7f63d80c6200 2024-09-13T17:17:35.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63ddf17640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63bc066690 0x7f63bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:35.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.942+0000 7f63ddf17640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63bc066690 0x7f63bc068b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f63d4003d60 tx=0x7f63d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:35.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.950+0000 7f63ce7fc640 1 -- 172.21.15.39:0/1874093354 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f63c80eb450 con 0x7f63d80c6200 2024-09-13T17:17:35.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff34414ae20 con 0x7ff34414aa40 2024-09-13T17:17:35.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff33affd640 1 -- 172.21.15.39:0/97271273 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5936 (secure 0 0 0) 0x7ff33c056b20 con 0x7ff34414aa40 2024-09-13T17:17:35.975 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:35.975 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:35.976 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-13T17:17:35.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff328066690 msgr2=0x7ff328068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff328066690 0x7ff328068b50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff3400af000 tx=0x7ff340008040 comp rx=0 tx=0).stop 2024-09-13T17:17:35.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 msgr2=0x7ff344149ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:35.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 0x7ff344149ae0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00cd90 tx=0x7ff33c00e420 comp rx=0 tx=0).stop 2024-09-13T17:17:35.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 shutdown_connections 2024-09-13T17:17:35.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7ff328066690 0x7ff328068b50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3440c4710 0x7ff3441495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34414aa40 0x7ff344149ae0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 --2- 172.21.15.39:0/97271273 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7ff3440c1750 0x7ff3441ec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:35.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 >> 172.21.15.39:0/97271273 conn(0x7ff3440c3310 msgr2=0x7ff3440c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:35.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 shutdown_connections 2024-09-13T17:17:35.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:35.974+0000 7ff34c700640 1 -- 172.21.15.39:0/97271273 wait complete. 2024-09-13T17:17:35.994 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:17:36.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.110+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"} v 0) v1 -- 0x7f63d814bc60 con 0x7f63d80c6200 2024-09-13T17:17:36.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63ce7fc640 1 -- 172.21.15.39:0/1874093354 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "b_data", "var": "pg_num"}]=0 v63) v1 ==== 95+0+11 (secure 0 0 0) 0x7f63c80f3e60 con 0x7f63d80c6200 2024-09-13T17:17:36.115 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-13T17:17:36.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63bc066690 msgr2=0x7f63bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:36.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63bc066690 0x7f63bc068b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f63d4003d60 tx=0x7f63d4009040 comp rx=0 tx=0).stop 2024-09-13T17:17:36.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d80c6200 msgr2=0x7f63d8143940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:36.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d80c6200 0x7f63d8143940 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f63c80ae040 tx=0x7f63c8019130 comp rx=0 tx=0).stop 2024-09-13T17:17:36.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 shutdown_connections 2024-09-13T17:17:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f63bc066690 0x7f63bc068b50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f63d814f660 0x7f63d8143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63d80c6200 0x7f63d8143940 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 --2- 172.21.15.39:0/1874093354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63d806d7f0 0x7f63d80c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 >> 172.21.15.39:0/1874093354 conn(0x7f63d80a8860 msgr2=0x7f63d814acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 shutdown_connections 2024-09-13T17:17:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.114+0000 7f63e017b640 1 -- 172.21.15.39:0/1874093354 wait complete. 2024-09-13T17:17:36.136 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:17:36.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac955b640 1 Processor -- start 2024-09-13T17:17:36.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac955b640 1 -- start start 2024-09-13T17:17:36.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac955b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0ac414ad30 0x7f0ac4154160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:36.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac955b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac4150c80 con 0x7f0ac40c1c30 2024-09-13T17:17:36.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac955b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac40ac140 con 0x7f0ac406fb60 2024-09-13T17:17:36.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac955b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac406b8d0 con 0x7f0ac414ad30 2024-09-13T17:17:36.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac2ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0ac40c1c30 0x7f0ac4151440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51874/0 (socket says 172.21.15.39:51874) 2024-09-13T17:17:36.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac2ffd640 1 -- 172.21.15.39:0/1055814974 learned_addr learned my addr 172.21.15.39:0/1055814974 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac37fe640 1 --2- 172.21.15.39:0/1055814974 >> v2:172.21.15.39:3300/0 conn(0x7f0ac414ad30 0x7f0ac4154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac1ffb640 1 -- 172.21.15.39:0/1055814974 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482275482 0 0) 0x7f0ac4150c80 con 0x7f0ac40c1c30 2024-09-13T17:17:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac1ffb640 1 -- 172.21.15.39:0/1055814974 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa4003530 con 0x7f0ac40c1c30 2024-09-13T17:17:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac37fe640 1 -- 172.21.15.39:0/1055814974 >> v1:172.21.15.84:6789/0 conn(0x7f0ac406fb60 legacy=0x7f0ac414a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac37fe640 1 -- 172.21.15.39:0/1055814974 >> v1:172.21.15.39:6789/0 conn(0x7f0ac40c1c30 legacy=0x7f0ac4151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac1ffb640 1 -- 172.21.15.39:0/1055814974 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565308273 0 0) 0x7f0ac40ac140 con 0x7f0ac406fb60 2024-09-13T17:17:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.258+0000 7f0ac37fe640 1 -- 172.21.15.39:0/1055814974 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac406bbb0 con 0x7f0ac414ad30 2024-09-13T17:17:36.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac37fe640 1 --2- 172.21.15.39:0/1055814974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac414ad30 0x7f0ac4154160 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40097f0 tx=0x7f0ab401c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233d943d32a8b814 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:36.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac1ffb640 1 -- 172.21.15.39:0/1055814974 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac001680 con 0x7f0ac414ad30 2024-09-13T17:17:36.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac1ffb640 1 -- 172.21.15.39:0/1055814974 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab40a7050 con 0x7f0ac414ad30 2024-09-13T17:17:36.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac1ffb640 1 -- 172.21.15.39:0/1055814974 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40ab600 con 0x7f0ac414ad30 2024-09-13T17:17:36.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac955b640 1 -- 172.21.15.39:0/1055814974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac414ad30 msgr2=0x7f0ac4154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:36.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac955b640 1 --2- 172.21.15.39:0/1055814974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac414ad30 0x7f0ac4154160 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40097f0 tx=0x7f0ab401c2f0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac955b640 1 -- 172.21.15.39:0/1055814974 shutdown_connections 2024-09-13T17:17:36.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac955b640 1 --2- 172.21.15.39:0/1055814974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac414ad30 0x7f0ac4154160 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac955b640 1 -- 172.21.15.39:0/1055814974 >> 172.21.15.39:0/1055814974 conn(0x7f0ac40c2ac0 msgr2=0x7f0ac40c2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:36.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac955b640 1 -- 172.21.15.39:0/1055814974 shutdown_connections 2024-09-13T17:17:36.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.262+0000 7f0ac955b640 1 -- 172.21.15.39:0/1055814974 wait complete. 2024-09-13T17:17:36.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 Processor -- start 2024-09-13T17:17:36.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 -- start start 2024-09-13T17:17:36.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac406fb60 0x7f0ac4144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac40c1c30 0x7f0ac4149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0ac414ad30 0x7f0ac4151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4144d30 con 0x7f0ac40c1c30 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41ea820 con 0x7f0ac406fb60 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41eab00 con 0x7f0ac414ad30 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac40c1c30 0x7f0ac4149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac27fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac40c1c30 0x7f0ac4149950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55022/0 (socket says 172.21.15.39:55022) 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac27fc640 1 -- 172.21.15.39:0/683701630 learned_addr learned my addr 172.21.15.39:0/683701630 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac2ffd640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac406fb60 0x7f0ac4144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac37fe640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0ac414ad30 0x7f0ac4151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac27fc640 1 -- 172.21.15.39:0/683701630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0ac414ad30 msgr2=0x7f0ac4151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:36.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac27fc640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0ac414ad30 0x7f0ac4151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac27fc640 1 -- 172.21.15.39:0/683701630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac406fb60 msgr2=0x7f0ac4144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac27fc640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac406fb60 0x7f0ac4144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac27fc640 1 -- 172.21.15.39:0/683701630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4009420 con 0x7f0ac40c1c30 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac37fe640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0ac414ad30 0x7f0ac4151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac27fc640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac40c1c30 0x7f0ac4149950 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c7a0 tx=0x7f0abc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0abb7fe640 1 -- 172.21.15.39:0/683701630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc003db0 con 0x7f0ac40c1c30 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0abb7fe640 1 -- 172.21.15.39:0/683701630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc005070 con 0x7f0ac40c1c30 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0abb7fe640 1 -- 172.21.15.39:0/683701630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc005420 con 0x7f0ac40c1c30 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac40c55a0 con 0x7f0ac40c1c30 2024-09-13T17:17:36.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac41448d0 con 0x7f0ac40c1c30 2024-09-13T17:17:36.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac41ea680 con 0x7f0ac40c1c30 2024-09-13T17:17:36.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0abb7fe640 1 -- 172.21.15.39:0/683701630 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0abc005580 con 0x7f0ac40c1c30 2024-09-13T17:17:36.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0abb7fe640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0aa8066690 0x7f0aa8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:36.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0abb7fe640 1 -- 172.21.15.39:0/683701630 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f0abc084970 con 0x7f0ac40c1c30 2024-09-13T17:17:36.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.266+0000 7f0ac2ffd640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0aa8066690 0x7f0aa8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:36.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.270+0000 7f0ac2ffd640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0aa8066690 0x7f0aa8068b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0016b0 tx=0x7f0aac001a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:36.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.274+0000 7f0abb7fe640 1 -- 172.21.15.39:0/683701630 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0ac41ea680 con 0x7f0ac40c1c30 2024-09-13T17:17:36.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.446+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0ac41ea680 con 0x7f0ac40c1c30 2024-09-13T17:17:36.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.446+0000 7f0abb7fe640 1 -- 172.21.15.39:0/683701630 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 78 entries v63) v1 ==== 81+0+4738 (secure 0 0 0) 0x7f0ac41ea680 con 0x7f0ac40c1c30 2024-09-13T17:17:36.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/380008541 2024-09-14T17:16:34.950555+0000 2024-09-13T17:17:36.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3721635324 2024-09-14T17:16:04.944309+0000 2024-09-13T17:17:36.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1355102409 2024-09-14T17:15:29.935985+0000 2024-09-13T17:17:36.450 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/305129652 2024-09-14T17:14:49.920017+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/282472862 2024-09-14T17:14:04.899010+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/397460031 2024-09-14T17:12:19.869778+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/397460031 2024-09-14T17:12:19.869778+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/2351253009 2024-09-14T17:11:09.852639+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3334565883 2024-09-14T17:11:59.865711+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3923043380 2024-09-14T17:09:59.836463+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1334276441 2024-09-14T16:59:04.677958+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/1326467723 2024-09-14T17:00:02.030979+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1334276441 2024-09-14T16:59:04.677958+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4200926406 2024-09-14T17:05:59.776545+0000 2024-09-13T17:17:36.451 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1392182763 2024-09-14T17:07:19.797311+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/3122796453 2024-09-14T16:59:54.690333+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/1326467723 2024-09-14T17:00:02.030979+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2740003107 2024-09-14T16:57:14.654506+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1355102409 2024-09-14T17:15:29.935985+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/1969975539 2024-09-14T16:52:59.589819+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3721635324 2024-09-14T17:16:04.944309+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/2351253009 2024-09-14T17:11:09.852639+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2144264453 2024-09-14T16:52:24.581694+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4094657757 2024-09-14T16:56:04.630964+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3600322972 2024-09-14T16:55:14.618334+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/380008541 2024-09-14T17:16:34.950555+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1572373831 2024-09-14T16:51:14.567493+0000 2024-09-13T17:17:36.452 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4246622083 2024-09-14T17:01:24.709561+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/282472862 2024-09-14T17:14:04.899010+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1355376480 2024-09-14T16:54:09.605050+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3334565883 2024-09-14T17:11:59.865711+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3941977343 2024-09-14T17:08:49.820117+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/2521680730 2024-09-14T17:09:29.828780+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/661906662 2024-09-14T16:57:59.663485+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4094657757 2024-09-14T16:56:04.630964+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3540656099 2024-09-14T16:50:14.555020+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2065196144 2024-09-14T16:53:14.593565+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1750130579 2024-09-14T17:09:29.828673+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3163580917 2024-09-14T16:49:44.548920+0000 2024-09-13T17:17:36.453 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1355376480 2024-09-14T16:54:09.605050+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2065196144 2024-09-14T16:53:14.593565+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/1392588044 2024-09-14T16:50:49.562750+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1034701751 2024-09-14T17:10:39.845194+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/3122796453 2024-09-14T16:59:54.690333+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/3105501456 2024-09-14T17:03:49.742996+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/2521680730 2024-09-14T17:09:29.828780+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/661906662 2024-09-14T16:57:59.663485+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1572373831 2024-09-14T16:51:14.567493+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/1969975539 2024-09-14T16:52:59.589819+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1392588044 2024-09-14T16:50:49.562750+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2144264453 2024-09-14T16:52:24.581694+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3163580917 2024-09-14T16:49:44.548920+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3941977343 2024-09-14T17:08:49.820117+0000 2024-09-13T17:17:36.454 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/2638959497 2024-09-14T17:01:24.709473+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2740003107 2024-09-14T16:57:14.654506+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4246622083 2024-09-14T17:01:24.709561+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/305129652 2024-09-14T17:14:49.920017+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/835951775 2024-09-14T17:02:59.730250+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/828722052 2024-09-14T17:03:19.734465+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/2638959497 2024-09-14T17:01:24.709473+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/828722052 2024-09-14T17:03:19.734465+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3600322972 2024-09-14T16:55:14.618334+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/611669592 2024-09-14T17:04:24.751785+0000 2024-09-13T17:17:36.455 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/611669592 2024-09-14T17:04:24.751785+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/1590039204 2024-09-14T17:05:09.765210+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/4200926406 2024-09-14T17:05:59.776545+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/440028459 2024-09-14T17:07:34.801438+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6836/3105501456 2024-09-14T17:03:49.742996+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1590039204 2024-09-14T17:05:09.765210+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/263561599 2024-09-14T17:05:54.774622+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6834/3923043380 2024-09-14T17:09:59.836463+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/263561599 2024-09-14T17:05:54.774622+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/1392182763 2024-09-14T17:07:19.797311+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6837/835951775 2024-09-14T17:02:59.730250+0000 2024-09-13T17:17:36.456 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6835/3540656099 2024-09-14T16:50:14.555020+0000 2024-09-13T17:17:36.457 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6832/440028459 2024-09-14T17:07:34.801438+0000 2024-09-13T17:17:36.457 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1034701751 2024-09-14T17:10:39.845194+0000 2024-09-13T17:17:36.457 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.84:6833/1750130579 2024-09-14T17:09:29.828673+0000 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:listed 78 entries 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0aa8066690 msgr2=0x7f0aa8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0aa8066690 0x7f0aa8068b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0016b0 tx=0x7f0aac001a20 comp rx=0 tx=0).stop 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac40c1c30 msgr2=0x7f0ac4149950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac40c1c30 0x7f0ac4149950 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00c7a0 tx=0x7f0abc00cc60 comp rx=0 tx=0).stop 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 shutdown_connections 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0aa8066690 0x7f0aa8068b50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0ac414ad30 0x7f0ac4151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac40c1c30 0x7f0ac4149950 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 --2- 172.21.15.39:0/683701630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ac406fb60 0x7f0ac4144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 >> 172.21.15.39:0/683701630 conn(0x7f0ac40c2ac0 msgr2=0x7f0ac4071650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:36.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 shutdown_connections 2024-09-13T17:17:36.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:36.450+0000 7f0ac955b640 1 -- 172.21.15.39:0/683701630 wait complete. 2024-09-13T17:17:36.469 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2024-09-13T17:17:36.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:36.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T17:17:36.556 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:17:36.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi084.stderr:ceph-fuse[34829]: fuse finished with error 0 and tester_r 0 2024-09-13T17:17:36.736 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:17:36.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:17:37.995 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.125+0000 7fc9d1bcc640 1 Processor -- start 2024-09-13T17:17:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.125+0000 7fc9d1bcc640 1 -- start start 2024-09-13T17:17:38.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9d1bcc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc1526b0 0x7fc9cc0c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9d1bcc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc9cc06c940 0x7fc9cc0aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9d1bcc640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc14f790 con 0x7fc9cc0ab100 2024-09-13T17:17:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9d1bcc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc152a90 con 0x7fc9cc06c940 2024-09-13T17:17:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9d1bcc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc06ce40 con 0x7fc9cc1526b0 2024-09-13T17:17:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9caffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc9cc06c940 0x7fc9cc0aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9caffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc9cc06c940 0x7fc9cc0aabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55032/0 (socket says 172.21.15.39:55032) 2024-09-13T17:17:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9cb7fe640 1 --2- 172.21.15.39:0/259602923 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc1526b0 0x7fc9cc0c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9caffd640 1 -- 172.21.15.39:0/259602923 learned_addr learned my addr 172.21.15.39:0/259602923 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:38.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9caffd640 1 -- 172.21.15.39:0/259602923 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc1526b0 msgr2=0x7fc9cc0c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:38.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9caffd640 1 --2- 172.21.15.39:0/259602923 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc1526b0 0x7fc9cc0c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9caffd640 1 -- 172.21.15.39:0/259602923 >> v1:172.21.15.84:6789/0 conn(0x7fc9cc0ab100 legacy=0x7fc9cc1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:38.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9caffd640 1 -- 172.21.15.39:0/259602923 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc06d120 con 0x7fc9cc06c940 2024-09-13T17:17:38.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9caffd640 1 --2- 172.21.15.39:0/259602923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc06c940 0x7fc9cc0aabc0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009870 tx=0x7fc9c001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ffe52f1840ac6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:38.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9ca7fc640 1 -- 172.21.15.39:0/259602923 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00ad660 con 0x7fc9cc06c940 2024-09-13T17:17:38.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9ca7fc640 1 -- 172.21.15.39:0/259602923 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c00ad7c0 con 0x7fc9cc06c940 2024-09-13T17:17:38.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.129+0000 7fc9ca7fc640 1 -- 172.21.15.39:0/259602923 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00adb50 con 0x7fc9cc06c940 2024-09-13T17:17:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/259602923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc06c940 msgr2=0x7fc9cc0aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 --2- 172.21.15.39:0/259602923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc06c940 0x7fc9cc0aabc0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009870 tx=0x7fc9c001c560 comp rx=0 tx=0).stop 2024-09-13T17:17:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/259602923 shutdown_connections 2024-09-13T17:17:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 --2- 172.21.15.39:0/259602923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc06c940 0x7fc9cc0aabc0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 --2- 172.21.15.39:0/259602923 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc1526b0 0x7fc9cc0c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/259602923 >> 172.21.15.39:0/259602923 conn(0x7fc9cc069e10 msgr2=0x7fc9cc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/259602923 shutdown_connections 2024-09-13T17:17:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/259602923 wait complete. 2024-09-13T17:17:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 Processor -- start 2024-09-13T17:17:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- start start 2024-09-13T17:17:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc06c940 0x7fc9cc14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc0ab100 0x7fc9cc150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9cc1526b0 0x7fc9cc150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0a7d40 con 0x7fc9cc0ab100 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0a7fb0 con 0x7fc9cc1526b0 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0a8440 con 0x7fc9cc06c940 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9caffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc0ab100 0x7fc9cc150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9caffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc0ab100 0x7fc9cc150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55040/0 (socket says 172.21.15.39:55040) 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9caffd640 1 -- 172.21.15.39:0/1642775091 learned_addr learned my addr 172.21.15.39:0/1642775091 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9cb7fe640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc06c940 0x7fc9cc14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9cbfff640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9cc1526b0 0x7fc9cc150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9caffd640 1 -- 172.21.15.39:0/1642775091 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc06c940 msgr2=0x7fc9cc14ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9caffd640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc06c940 0x7fc9cc14ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9caffd640 1 -- 172.21.15.39:0/1642775091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9cc1526b0 msgr2=0x7fc9cc150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9caffd640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9cc1526b0 0x7fc9cc150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9caffd640 1 -- 172.21.15.39:0/1642775091 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c00094d0 con 0x7fc9cc0ab100 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9cbfff640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9cc1526b0 0x7fc9cc150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9caffd640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc0ab100 0x7fc9cc150150 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00b0040 tx=0x7fc9c0019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9b3fff640 1 -- 172.21.15.39:0/1642775091 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00b6660 con 0x7fc9cc0ab100 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9b3fff640 1 -- 172.21.15.39:0/1642775091 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c00b67e0 con 0x7fc9cc0ab100 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc1447e0 con 0x7fc9cc0ab100 2024-09-13T17:17:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9b3fff640 1 -- 172.21.15.39:0/1642775091 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00b6b90 con 0x7fc9cc0ab100 2024-09-13T17:17:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.133+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc1ee9a0 con 0x7fc9cc0ab100 2024-09-13T17:17:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.137+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc0a8120 con 0x7fc9cc0ab100 2024-09-13T17:17:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.137+0000 7fc9b3fff640 1 -- 172.21.15.39:0/1642775091 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc9c00b4070 con 0x7fc9cc0ab100 2024-09-13T17:17:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.137+0000 7fc9b3fff640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9b8066690 0x7fc9b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.137+0000 7fc9b3fff640 1 -- 172.21.15.39:0/1642775091 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fc9c011f420 con 0x7fc9cc0ab100 2024-09-13T17:17:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.137+0000 7fc9cb7fe640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9b8066690 0x7fc9b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.137+0000 7fc9cb7fe640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9b8066690 0x7fc9b8068b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc150e50 tx=0x7fc9bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:38.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.141+0000 7fc9b3fff640 1 -- 172.21.15.39:0/1642775091 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc9c00eb710 con 0x7fc9cc0ab100 2024-09-13T17:17:38.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.333+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9cc0aabc0 con 0x7fc9cc0ab100 2024-09-13T17:17:38.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9b3fff640 1 -- 172.21.15.39:0/1642775091 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5936 (secure 0 0 0) 0x7fc9c00eed30 con 0x7fc9cc0ab100 2024-09-13T17:17:38.339 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:38.339 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:38.339 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-13T17:17:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9b8066690 msgr2=0x7fc9b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9b8066690 0x7fc9b8068b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc150e50 tx=0x7fc9bc008040 comp rx=0 tx=0).stop 2024-09-13T17:17:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc0ab100 msgr2=0x7fc9cc150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc0ab100 0x7fc9cc150150 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00b0040 tx=0x7fc9c0019300 comp rx=0 tx=0).stop 2024-09-13T17:17:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 shutdown_connections 2024-09-13T17:17:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc9b8066690 0x7fc9b8068b50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9cc1526b0 0x7fc9cc150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9cc0ab100 0x7fc9cc150150 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 --2- 172.21.15.39:0/1642775091 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc9cc06c940 0x7fc9cc14ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:38.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 >> 172.21.15.39:0/1642775091 conn(0x7fc9cc069e10 msgr2=0x7fc9cc148a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:38.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 shutdown_connections 2024-09-13T17:17:38.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:38.337+0000 7fc9d1bcc640 1 -- 172.21.15.39:0/1642775091 wait complete. 2024-09-13T17:17:38.357 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:17:40.358 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.485+0000 7f7d0b4c6640 1 Processor -- start 2024-09-13T17:17:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.485+0000 7f7d0b4c6640 1 -- start start 2024-09-13T17:17:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.485+0000 7f7d0b4c6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d040c5330 0x7f7d040c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.485+0000 7f7d0b4c6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d041546c0 con 0x7f7d04150f80 2024-09-13T17:17:40.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.485+0000 7f7d0b4c6640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d040c49f0 con 0x7f7d0406c090 2024-09-13T17:17:40.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.485+0000 7f7d0b4c6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040c19a0 con 0x7f7d040c5330 2024-09-13T17:17:40.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.485+0000 7f7d09262640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7d04150f80 0x7f7d04153930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40164/0 (socket says 172.21.15.39:40164) 2024-09-13T17:17:40.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.485+0000 7f7d09262640 1 -- 172.21.15.39:0/2585505859 learned_addr learned my addr 172.21.15.39:0/2585505859 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:40.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d08a61640 1 --2- 172.21.15.39:0/2585505859 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d040c5330 0x7f7d040c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:40.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7cfbfff640 1 -- 172.21.15.39:0/2585505859 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589090736 0 0) 0x7f7d041546c0 con 0x7f7d04150f80 2024-09-13T17:17:40.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7cfbfff640 1 -- 172.21.15.39:0/2585505859 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf0004640 con 0x7f7d04150f80 2024-09-13T17:17:40.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7cfbfff640 1 -- 172.21.15.39:0/2585505859 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070989468 0 0) 0x7f7d040c49f0 con 0x7f7d0406c090 2024-09-13T17:17:40.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7cfbfff640 1 -- 172.21.15.39:0/2585505859 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d041546c0 con 0x7f7d0406c090 2024-09-13T17:17:40.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7cfbfff640 1 -- 172.21.15.39:0/2585505859 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3195525721 0 0) 0x7f7cf0004640 con 0x7f7d04150f80 2024-09-13T17:17:40.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d08a61640 1 -- 172.21.15.39:0/2585505859 >> v1:172.21.15.84:6789/0 conn(0x7f7d0406c090 legacy=0x7f7d0406c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:40.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d08a61640 1 -- 172.21.15.39:0/2585505859 >> v1:172.21.15.39:6789/0 conn(0x7f7d04150f80 legacy=0x7f7d04153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:40.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d08a61640 1 -- 172.21.15.39:0/2585505859 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d041543e0 con 0x7f7d040c5330 2024-09-13T17:17:40.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d08a61640 1 --2- 172.21.15.39:0/2585505859 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d040c5330 0x7f7d040c1210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7d000097f0 tx=0x7f7d0001c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f22b48fa283bd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7cfbfff640 1 -- 172.21.15.39:0/2585505859 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4001680 con 0x7f7d040c5330 2024-09-13T17:17:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7cfbfff640 1 -- 172.21.15.39:0/2585505859 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d000a7050 con 0x7f7d040c5330 2024-09-13T17:17:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7cfbfff640 1 -- 172.21.15.39:0/2585505859 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000ab600 con 0x7f7d040c5330 2024-09-13T17:17:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2585505859 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d040c5330 msgr2=0x7f7d040c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d0b4c6640 1 --2- 172.21.15.39:0/2585505859 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d040c5330 0x7f7d040c1210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7d000097f0 tx=0x7f7d0001c340 comp rx=0 tx=0).stop 2024-09-13T17:17:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2585505859 shutdown_connections 2024-09-13T17:17:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d0b4c6640 1 --2- 172.21.15.39:0/2585505859 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d040c5330 0x7f7d040c1210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.489+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2585505859 >> 172.21.15.39:0/2585505859 conn(0x7f7d04069e10 msgr2=0x7f7d0406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2585505859 shutdown_connections 2024-09-13T17:17:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2585505859 wait complete. 2024-09-13T17:17:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 Processor -- start 2024-09-13T17:17:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 -- start start 2024-09-13T17:17:40.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d0406c090 0x7f7d0414e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d040c5330 0x7f7d041ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d04150f80 0x7f7d041edef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0406f970 con 0x7f7d040c5330 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0406f3f0 con 0x7f7d04150f80 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041ee430 con 0x7f7d0406c090 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d08a61640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d040c5330 0x7f7d041ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d08a61640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d040c5330 0x7f7d041ed9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40546/0 (socket says 172.21.15.39:40546) 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d08a61640 1 -- 172.21.15.39:0/2872482496 learned_addr learned my addr 172.21.15.39:0/2872482496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d09262640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d0406c090 0x7f7d0414e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d09a63640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d04150f80 0x7f7d041edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d08a61640 1 -- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d0406c090 msgr2=0x7f7d0414e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d08a61640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d0406c090 0x7f7d0414e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d08a61640 1 -- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d04150f80 msgr2=0x7f7d041edef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d08a61640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d04150f80 0x7f7d041edef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d08a61640 1 -- 172.21.15.39:0/2872482496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00009420 con 0x7f7d040c5330 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d09a63640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d04150f80 0x7f7d041edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d09262640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d0406c090 0x7f7d0414e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d08a61640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d040c5330 0x7f7d041ed9b0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7d000097f0 tx=0x7f7d000190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7cf9ffb640 1 -- 172.21.15.39:0/2872482496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00006e20 con 0x7f7d040c5330 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7cf9ffb640 1 -- 172.21.15.39:0/2872482496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d000b4400 con 0x7f7d040c5330 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0406f690 con 0x7f7d040c5330 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7cf9ffb640 1 -- 172.21.15.39:0/2872482496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000b4770 con 0x7f7d040c5330 2024-09-13T17:17:40.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d04071d90 con 0x7f7d040c5330 2024-09-13T17:17:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.493+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d040718e0 con 0x7f7d040c5330 2024-09-13T17:17:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.497+0000 7f7cf9ffb640 1 -- 172.21.15.39:0/2872482496 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d000b2040 con 0x7f7d040c5330 2024-09-13T17:17:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.497+0000 7f7cf9ffb640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ce8066690 0x7f7ce8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.497+0000 7f7cf9ffb640 1 -- 172.21.15.39:0/2872482496 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f7d0011f590 con 0x7f7d040c5330 2024-09-13T17:17:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.497+0000 7f7d09262640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ce8066690 0x7f7ce8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.497+0000 7f7d09262640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ce8066690 0x7f7ce8068b50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4000fe0 tx=0x7f7cf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:40.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.501+0000 7f7cf9ffb640 1 -- 172.21.15.39:0/2872482496 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7d000eb800 con 0x7f7d040c5330 2024-09-13T17:17:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.697+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d04071f80 con 0x7f7d040c5330 2024-09-13T17:17:40.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7cf9ffb640 1 -- 172.21.15.39:0/2872482496 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f7d000eee20 con 0x7f7d040c5330 2024-09-13T17:17:40.703 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:40.703 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:40.703 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-13T17:17:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ce8066690 msgr2=0x7f7ce8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ce8066690 0x7f7ce8068b50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4000fe0 tx=0x7f7cf4009040 comp rx=0 tx=0).stop 2024-09-13T17:17:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d040c5330 msgr2=0x7f7d041ed9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d040c5330 0x7f7d041ed9b0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7d000097f0 tx=0x7f7d000190e0 comp rx=0 tx=0).stop 2024-09-13T17:17:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 shutdown_connections 2024-09-13T17:17:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7ce8066690 0x7f7ce8068b50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d04150f80 0x7f7d041edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d040c5330 0x7f7d041ed9b0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 --2- 172.21.15.39:0/2872482496 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d0406c090 0x7f7d0414e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 >> 172.21.15.39:0/2872482496 conn(0x7f7d04069e10 msgr2=0x7f7d0406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 shutdown_connections 2024-09-13T17:17:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:40.701+0000 7f7d0b4c6640 1 -- 172.21.15.39:0/2872482496 wait complete. 2024-09-13T17:17:40.723 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:17:42.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T17:17:42.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:42.658 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:17:42.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:42.666 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-13T17:17:42.724 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:42.726 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:17:42.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:42.726 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:17:42.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:42.781 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-09-13T17:17:42.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f6584f00640 1 Processor -- start 2024-09-13T17:17:42.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f6584f00640 1 -- start start 2024-09-13T17:17:42.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f6584f00640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f65800c21d0 0x7f65800aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:42.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f6584f00640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658006aac0 0x7f6580151c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:42.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f6584f00640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65800c3ae0 con 0x7f658014b850 2024-09-13T17:17:42.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f6584f00640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65800ab340 con 0x7f65800c21d0 2024-09-13T17:17:42.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f6584f00640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65800abb30 con 0x7f658006aac0 2024-09-13T17:17:42.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f657e59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f65800c21d0 0x7f65800aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:42.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f657ed9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f658014b850 0x7f6580155230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40168/0 (socket says 172.21.15.39:40168) 2024-09-13T17:17:42.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f657ed9d640 1 -- 172.21.15.39:0/2835365287 learned_addr learned my addr 172.21.15.39:0/2835365287 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:42.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f6577fff640 1 --2- 172.21.15.39:0/2835365287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658006aac0 0x7f6580151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:42.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f657e59c640 1 -- 172.21.15.39:0/2835365287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658006aac0 msgr2=0x7f6580151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:42.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f657e59c640 1 --2- 172.21.15.39:0/2835365287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658006aac0 0x7f6580151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:42.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f657e59c640 1 -- 172.21.15.39:0/2835365287 >> v1:172.21.15.39:6789/0 conn(0x7f658014b850 legacy=0x7f6580155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:42.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f657e59c640 1 -- 172.21.15.39:0/2835365287 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65800abe10 con 0x7f65800c21d0 2024-09-13T17:17:42.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f657dd9b640 1 -- 172.21.15.39:0/2835365287 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230941894 0 0) 0x7f65800c3ae0 con 0x7f658014b850 2024-09-13T17:17:42.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f6577fff640 1 --2- 172.21.15.39:0/2835365287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658006aac0 0x7f6580151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:42.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.897+0000 7f657e59c640 1 --2- 172.21.15.39:0/2835365287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65800c21d0 0x7f65800aae00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f65700097c0 tx=0x7f657001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e0ee0c02d640b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657dd9b640 1 -- 172.21.15.39:0/2835365287 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700ac640 con 0x7f65800c21d0 2024-09-13T17:17:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657dd9b640 1 -- 172.21.15.39:0/2835365287 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65700ac7a0 con 0x7f65800c21d0 2024-09-13T17:17:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657dd9b640 1 -- 172.21.15.39:0/2835365287 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700acb30 con 0x7f65800c21d0 2024-09-13T17:17:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- 172.21.15.39:0/2835365287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65800c21d0 msgr2=0x7f65800aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:42.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 --2- 172.21.15.39:0/2835365287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65800c21d0 0x7f65800aae00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f65700097c0 tx=0x7f657001c3e0 comp rx=0 tx=0).stop 2024-09-13T17:17:42.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- 172.21.15.39:0/2835365287 shutdown_connections 2024-09-13T17:17:42.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 --2- 172.21.15.39:0/2835365287 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658006aac0 0x7f6580151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:42.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 --2- 172.21.15.39:0/2835365287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65800c21d0 0x7f65800aae00 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:42.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- 172.21.15.39:0/2835365287 >> 172.21.15.39:0/2835365287 conn(0x7f6580069e10 msgr2=0x7f658006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:42.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- 172.21.15.39:0/2835365287 shutdown_connections 2024-09-13T17:17:42.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- 172.21.15.39:0/2835365287 wait complete. 2024-09-13T17:17:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 Processor -- start 2024-09-13T17:17:42.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- start start 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f658006aac0 0x7f6580070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65800c21d0 0x7f6580154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658014b850 0x7f658015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658014c890 con 0x7f658006aac0 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658014c2d0 con 0x7f65800c21d0 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658014c5b0 con 0x7f658014b850 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f658006aac0 0x7f6580070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f658006aac0 0x7f6580070820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40556/0 (socket says 172.21.15.39:40556) 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657e59c640 1 -- 172.21.15.39:0/385269908 learned_addr learned my addr 172.21.15.39:0/385269908 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6577fff640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65800c21d0 0x7f6580154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657ed9d640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658014b850 0x7f658015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:42.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657e59c640 1 -- 172.21.15.39:0/385269908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658014b850 msgr2=0x7f658015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657e59c640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658014b850 0x7f658015b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657e59c640 1 -- 172.21.15.39:0/385269908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65800c21d0 msgr2=0x7f6580154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657e59c640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65800c21d0 0x7f6580154880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657e59c640 1 -- 172.21.15.39:0/385269908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570009420 con 0x7f658006aac0 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f657e59c640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f658006aac0 0x7f6580070820 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f65700af040 tx=0x7f657001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6576ffd640 1 -- 172.21.15.39:0/385269908 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700b8860 con 0x7f658006aac0 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6576ffd640 1 -- 172.21.15.39:0/385269908 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65700b89c0 con 0x7f658006aac0 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658015a930 con 0x7f658006aac0 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6576ffd640 1 -- 172.21.15.39:0/385269908 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65700b8d70 con 0x7f658006aac0 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.901+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6580154dc0 con 0x7f658006aac0 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.905+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658015a790 con 0x7f658006aac0 2024-09-13T17:17:42.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.905+0000 7f6576ffd640 1 -- 172.21.15.39:0/385269908 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65700b6400 con 0x7f658006aac0 2024-09-13T17:17:42.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.905+0000 7f6576ffd640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65600666c0 0x7f6560068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:42.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.905+0000 7f6576ffd640 1 -- 172.21.15.39:0/385269908 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f657011fff0 con 0x7f658006aac0 2024-09-13T17:17:42.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.905+0000 7f6577fff640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65600666c0 0x7f6560068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:42.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.905+0000 7f6577fff640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65600666c0 0x7f6560068b80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6568006fd0 tx=0x7f6568008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:42.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:42.909+0000 7f6576ffd640 1 -- 172.21.15.39:0/385269908 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f658015a790 con 0x7f658006aac0 2024-09-13T17:17:42.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:42.992 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-09-13T17:17:43.041 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-13T17:17:43.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:43.042 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:17:43.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:17:43.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2024-09-13T17:17:43.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.105+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f658015a790 con 0x7f658006aac0 2024-09-13T17:17:43.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.105+0000 7f6576ffd640 1 -- 172.21.15.39:0/385269908 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5936 (secure 0 0 0) 0x7f658015a790 con 0x7f658006aac0 2024-09-13T17:17:43.108 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:43.109 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":7555,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":202}],"filesystems":[{"mdsmap":{"epoch":201,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:16:38.184766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"a","rank":0,"incarnation":198,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6835/980326904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":980326904},{"type":"v1","addr":"172.21.15.39:6835","nonce":980326904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-13T17:17:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65600666c0 msgr2=0x7f6560068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65600666c0 0x7f6560068b80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6568006fd0 tx=0x7f6568008040 comp rx=0 tx=0).stop 2024-09-13T17:17:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f658006aac0 msgr2=0x7f6580070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f658006aac0 0x7f6580070820 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f65700af040 tx=0x7f657001ac20 comp rx=0 tx=0).stop 2024-09-13T17:17:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 shutdown_connections 2024-09-13T17:17:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f65600666c0 0x7f6560068b80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65800c21d0 0x7f6580154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f658014b850 0x7f658015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 --2- 172.21.15.39:0/385269908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f658006aac0 0x7f6580070820 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 >> 172.21.15.39:0/385269908 conn(0x7f6580069e10 msgr2=0x7f65800c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:43.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 shutdown_connections 2024-09-13T17:17:43.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.109+0000 7f6584f00640 1 -- 172.21.15.39:0/385269908 wait complete. 2024-09-13T17:17:43.128 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-09-13T17:17:43.262 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T17:17:43.275 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T17:17:43.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:17:43.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.401+0000 7f054c919640 1 Processor -- start 2024-09-13T17:17:43.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054c919640 1 -- start start 2024-09-13T17:17:43.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054c919640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f054806c870 0x7f05480aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054c919640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0548152a80 con 0x7f05480aabc0 2024-09-13T17:17:43.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054c919640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05480c30f0 con 0x7f05480c40b0 2024-09-13T17:17:43.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054c919640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0548151e00 con 0x7f054806c870 2024-09-13T17:17:43.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054659c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f05480aabc0 0x7f05480c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40184/0 (socket says 172.21.15.39:40184) 2024-09-13T17:17:43.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054659c640 1 -- 172.21.15.39:0/1526729644 learned_addr learned my addr 172.21.15.39:0/1526729644 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0545d9b640 1 --2- 172.21.15.39:0/1526729644 >> v2:172.21.15.84:3300/0 conn(0x7f054806c870 0x7f05480aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869568070 0 0) 0x7f0538001120 con 0x7f05480aabc0 2024-09-13T17:17:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0530003530 con 0x7f05480aabc0 2024-09-13T17:17:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3284005406 0 0) 0x7f0530003530 con 0x7f05480aabc0 2024-09-13T17:17:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 >> v2:172.21.15.84:3300/0 conn(0x7f054806c870 msgr2=0x7f05480aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 --2- 172.21.15.39:0/1526729644 >> v2:172.21.15.84:3300/0 conn(0x7f054806c870 0x7f05480aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 >> v1:172.21.15.84:6789/0 conn(0x7f05480c40b0 legacy=0x7f054815c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05481520e0 con 0x7f05480aabc0 2024-09-13T17:17:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f0538001810 con 0x7f05480aabc0 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 >> v1:172.21.15.39:6789/0 conn(0x7f05480aabc0 legacy=0x7f05480c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0530003980 0x7f0530003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f0530004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05300052b0 0x7f053000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548151e00 con 0x7f05300047d0 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053000c180 con 0x7f0530003980 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053000c340 con 0x7f05300052b0 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0545d9b640 1 --2- 172.21.15.39:0/1526729644 >> v2:172.21.15.84:3300/0 conn(0x7f054806c870 0x7f05480aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0545d9b640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f0530004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f054659c640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0530003980 0x7f0530003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0546d9d640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05300052b0 0x7f053000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0545d9b640 1 -- 172.21.15.39:0/1526729644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05300052b0 msgr2=0x7f053000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0545d9b640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05300052b0 0x7f053000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0545d9b640 1 -- 172.21.15.39:0/1526729644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0530003980 msgr2=0x7f0530003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:43.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0545d9b640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0530003980 0x7f0530003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0545d9b640 1 -- 172.21.15.39:0/1526729644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530003530 con 0x7f05300047d0 2024-09-13T17:17:43.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.405+0000 7f0546d9d640 1 -- 172.21.15.39:0/1526729644 reap_dead start 2024-09-13T17:17:43.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f0545d9b640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f0530004c50 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f053c009880 tx=0x7f053c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d545e397e9adfd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:43.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c004220 con 0x7f05300047d0 2024-09-13T17:17:43.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c004380 con 0x7f05300047d0 2024-09-13T17:17:43.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054559a640 1 -- 172.21.15.39:0/1526729644 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c003840 con 0x7f05300047d0 2024-09-13T17:17:43.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 -- 172.21.15.39:0/1526729644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 msgr2=0x7f0530004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f0530004c50 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f053c009880 tx=0x7f053c01c520 comp rx=0 tx=0).stop 2024-09-13T17:17:43.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 -- 172.21.15.39:0/1526729644 shutdown_connections 2024-09-13T17:17:43.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 --2- 172.21.15.39:0/1526729644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f0530004c50 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 -- 172.21.15.39:0/1526729644 >> 172.21.15.39:0/1526729644 conn(0x7f0548069e10 msgr2=0x7f054806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:43.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 -- 172.21.15.39:0/1526729644 shutdown_connections 2024-09-13T17:17:43.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 -- 172.21.15.39:0/1526729644 wait complete. 2024-09-13T17:17:43.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 Processor -- start 2024-09-13T17:17:43.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 -- start start 2024-09-13T17:17:43.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f054806cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05480bee10 0x7f05481ea980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05480bf290 0x7f05481eaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05481ee890 con 0x7f05300047d0 2024-09-13T17:17:43.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480718a0 con 0x7f05480bf290 2024-09-13T17:17:43.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054c919640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05481eb400 con 0x7f05480bee10 2024-09-13T17:17:43.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f054806cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f054806cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40572/0 (socket says 172.21.15.39:40572) 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054659c640 1 -- 172.21.15.39:0/2106367650 learned_addr learned my addr 172.21.15.39:0/2106367650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f0545d9b640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05480bee10 0x7f05481ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f0546d9d640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05480bf290 0x7f05481eaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054659c640 1 -- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05480bee10 msgr2=0x7f05481ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054659c640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05480bee10 0x7f05481ea980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054659c640 1 -- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05480bf290 msgr2=0x7f05481eaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054659c640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05480bf290 0x7f05481eaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054659c640 1 -- 172.21.15.39:0/2106367650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480bf7a0 con 0x7f05300047d0 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f0546d9d640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05480bf290 0x7f05481eaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:43.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f0545d9b640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05480bee10 0x7f05481ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:43.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.409+0000 7f054659c640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f054806cfb0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0538001060 tx=0x7f053800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:43.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f0536ffd640 1 -- 172.21.15.39:0/2106367650 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053800c870 con 0x7f05300047d0 2024-09-13T17:17:43.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f0536ffd640 1 -- 172.21.15.39:0/2106367650 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053800c9f0 con 0x7f05300047d0 2024-09-13T17:17:43.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05481edef0 con 0x7f05300047d0 2024-09-13T17:17:43.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f0536ffd640 1 -- 172.21.15.39:0/2106367650 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053800cda0 con 0x7f05300047d0 2024-09-13T17:17:43.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05481ee4b0 con 0x7f05300047d0 2024-09-13T17:17:43.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05481ee9d0 con 0x7f05300047d0 2024-09-13T17:17:43.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f0536ffd640 1 -- 172.21.15.39:0/2106367650 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0538013420 con 0x7f05300047d0 2024-09-13T17:17:43.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f0536ffd640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05280666c0 0x7f0528068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f0536ffd640 1 -- 172.21.15.39:0/2106367650 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f0538084c40 con 0x7f05300047d0 2024-09-13T17:17:43.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f0545d9b640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05280666c0 0x7f0528068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.413+0000 7f0545d9b640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05280666c0 0x7f0528068b80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f053c0ad000 tx=0x7f053c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:43.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.417+0000 7f0536ffd640 1 -- 172.21.15.39:0/2106367650 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f05481ee9d0 con 0x7f05300047d0 2024-09-13T17:17:43.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.577+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05480c5560 con 0x7f05280666c0 2024-09-13T17:17:43.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.617+0000 7f0536ffd640 1 -- 172.21.15.39:0/2106367650 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528671 (secure 0 0 0) 0x7f053c0b7040 con 0x7f05280666c0 2024-09-13T17:17:43.624 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:43.628 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T17:17:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05280666c0 msgr2=0x7f0528068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05280666c0 0x7f0528068b80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f053c0ad000 tx=0x7f053c0046d0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 msgr2=0x7f054806cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f054806cfb0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0538001060 tx=0x7f053800ed50 comp rx=0 tx=0).stop 2024-09-13T17:17:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 shutdown_connections 2024-09-13T17:17:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f05280666c0 0x7f0528068b80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05480bf290 0x7f05481eaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f05480bee10 0x7f05481ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 --2- 172.21.15.39:0/2106367650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05300047d0 0x7f054806cfb0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 >> 172.21.15.39:0/2106367650 conn(0x7f0548069e10 msgr2=0x7f0548154410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 shutdown_connections 2024-09-13T17:17:43.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.621+0000 7f054c919640 1 -- 172.21.15.39:0/2106367650 wait complete. 2024-09-13T17:17:43.644 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":953,"stamp":"2024-09-13T17:17:42.345888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39268818,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":3083,"num_read_kb":397491,"num_write":2818764,"num_write_kb":37531879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522488,"ondisk_log_size":522488,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6804280,"kb_used_data":83192,"kb_used_omap":12,"kb_used_meta":6721011,"kb_avail":748170440,"statfs":{"total":773094113280,"available":766126530560,"internally_reserved":0,"allocated":85188608,"data_stored":81998836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":6882315835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":429,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004893"},"pg_stats":[{"pgid":"5.18","version":"63'4385","reported_seq":4447,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058432+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T17:17:14.058432+0000","last_peered":"2024-09-13T17:17:14.058432+0000","last_clean":"2024-09-13T17:17:14.058432+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T17:17:14.058432+0000","last_undegraded":"2024-09-13T17:17:14.058432+0000","last_fullsized":"2024-09-13T17:17:14.058432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":23692,"num_write_kb":324330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"63'916","reported_seq":1025,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096259+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T17:17:22.096259+0000","last_peered":"2024-09-13T17:17:22.096259+0000","last_clean":"2024-09-13T17:17:22.096259+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T17:17:22.096259+0000","last_undegraded":"2024-09-13T17:17:22.096259+0000","last_fullsized":"2024-09-13T17:17:22.096259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":6289,"num_write":894,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'3737","reported_seq":3804,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.100845+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T17:17:22.100845+0000","last_peered":"2024-09-13T17:17:22.100845+0000","last_clean":"2024-09-13T17:17:22.100845+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T17:17:22.100845+0000","last_undegraded":"2024-09-13T17:17:22.100845+0000","last_fullsized":"2024-09-13T17:17:22.100845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3737,"log_dups_size":0,"ondisk_log_size":3737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20454,"num_write_kb":270405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.021643+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T17:17:14.021643+0000","last_peered":"2024-09-13T17:17:14.021643+0000","last_clean":"2024-09-13T17:17:14.021643+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T17:17:14.021643+0000","last_undegraded":"2024-09-13T17:17:14.021643+0000","last_fullsized":"2024-09-13T17:17:14.021643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"63'4331","reported_seq":4394,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.043747+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T17:17:14.043747+0000","last_peered":"2024-09-13T17:17:14.043747+0000","last_clean":"2024-09-13T17:17:14.043747+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T17:17:14.043747+0000","last_undegraded":"2024-09-13T17:17:14.043747+0000","last_fullsized":"2024-09-13T17:17:14.043747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":22875,"num_write_kb":325320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"63'23","reported_seq":91,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096300+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T17:17:22.096300+0000","last_peered":"2024-09-13T17:17:22.096300+0000","last_clean":"2024-09-13T17:17:22.096300+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T17:17:22.096300+0000","last_undegraded":"2024-09-13T17:17:22.096300+0000","last_fullsized":"2024-09-13T17:17:22.096300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"63'5559","reported_seq":5626,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.132606+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T17:17:22.132606+0000","last_peered":"2024-09-13T17:17:22.132606+0000","last_clean":"2024-09-13T17:17:22.132606+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T17:17:22.132606+0000","last_undegraded":"2024-09-13T17:17:22.132606+0000","last_fullsized":"2024-09-13T17:17:22.132606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5559,"log_dups_size":0,"ondisk_log_size":5559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30370,"num_write_kb":379865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"63'2","reported_seq":64,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:09.573144+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T17:17:09.573144+0000","last_peered":"2024-09-13T17:17:09.573144+0000","last_clean":"2024-09-13T17:17:09.573144+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T17:17:09.573144+0000","last_undegraded":"2024-09-13T17:17:09.573144+0000","last_fullsized":"2024-09-13T17:17:09.573144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"63'3581","reported_seq":3641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.986895+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T17:17:13.986895+0000","last_peered":"2024-09-13T17:17:13.986895+0000","last_clean":"2024-09-13T17:17:13.986895+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T17:17:13.986895+0000","last_undegraded":"2024-09-13T17:17:13.986895+0000","last_fullsized":"2024-09-13T17:17:13.986895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":268344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"63'40","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.099149+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T17:17:22.099149+0000","last_peered":"2024-09-13T17:17:22.099149+0000","last_clean":"2024-09-13T17:17:22.099149+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T17:17:22.099149+0000","last_undegraded":"2024-09-13T17:17:22.099149+0000","last_fullsized":"2024-09-13T17:17:22.099149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'3493","reported_seq":3559,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.063521+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T17:17:22.063521+0000","last_peered":"2024-09-13T17:17:22.063521+0000","last_clean":"2024-09-13T17:17:22.063521+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T17:17:22.063521+0000","last_undegraded":"2024-09-13T17:17:22.063521+0000","last_fullsized":"2024-09-13T17:17:22.063521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18921,"num_write_kb":256078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'726","reported_seq":812,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.996155+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T17:17:13.996155+0000","last_peered":"2024-09-13T17:17:13.996155+0000","last_clean":"2024-09-13T17:17:13.996155+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T17:17:13.996155+0000","last_undegraded":"2024-09-13T17:17:13.996155+0000","last_fullsized":"2024-09-13T17:17:13.996155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":75389,"num_write":714,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"63'3135","reported_seq":3195,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.003272+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T17:17:14.003272+0000","last_peered":"2024-09-13T17:17:14.003272+0000","last_clean":"2024-09-13T17:17:14.003272+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T17:17:14.003272+0000","last_undegraded":"2024-09-13T17:17:14.003272+0000","last_fullsized":"2024-09-13T17:17:14.003272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3135,"log_dups_size":0,"ondisk_log_size":3135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":232447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"63'5","reported_seq":106,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:16:38.283375+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T17:16:38.283375+0000","last_peered":"2024-09-13T17:16:38.283375+0000","last_clean":"2024-09-13T17:16:38.283375+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T17:16:38.283375+0000","last_undegraded":"2024-09-13T17:16:38.283375+0000","last_fullsized":"2024-09-13T17:16:38.283375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"63'5195","reported_seq":5259,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095099+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T17:17:22.095099+0000","last_peered":"2024-09-13T17:17:22.095099+0000","last_clean":"2024-09-13T17:17:22.095099+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T17:17:22.095099+0000","last_undegraded":"2024-09-13T17:17:22.095099+0000","last_fullsized":"2024-09-13T17:17:22.095099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29369,"num_write_kb":359470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"63'210","reported_seq":271,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:27.922744+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T17:17:27.922744+0000","last_peered":"2024-09-13T17:17:27.922744+0000","last_clean":"2024-09-13T17:17:27.922744+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T17:17:27.922744+0000","last_undegraded":"2024-09-13T17:17:27.922744+0000","last_fullsized":"2024-09-13T17:17:27.922744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153565,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"63'3570","reported_seq":3634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.050021+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T17:17:14.050021+0000","last_peered":"2024-09-13T17:17:14.050021+0000","last_clean":"2024-09-13T17:17:14.050021+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T17:17:14.050021+0000","last_undegraded":"2024-09-13T17:17:14.050021+0000","last_fullsized":"2024-09-13T17:17:14.050021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":19362,"num_write_kb":282506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"63'4274","reported_seq":4340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096597+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T17:17:22.096597+0000","last_peered":"2024-09-13T17:17:22.096597+0000","last_clean":"2024-09-13T17:17:22.096597+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T17:17:22.096597+0000","last_undegraded":"2024-09-13T17:17:22.096597+0000","last_fullsized":"2024-09-13T17:17:22.096597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23751,"num_write_kb":312338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"63'1418","reported_seq":1528,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.108614+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T17:17:22.108614+0000","last_peered":"2024-09-13T17:17:22.108614+0000","last_clean":"2024-09-13T17:17:22.108614+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T17:17:22.108614+0000","last_undegraded":"2024-09-13T17:17:22.108614+0000","last_fullsized":"2024-09-13T17:17:22.108614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1418,"log_dups_size":0,"ondisk_log_size":1418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":38147,"num_write":1396,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"63'6","reported_seq":90,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.030550+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T17:17:14.030550+0000","last_peered":"2024-09-13T17:17:14.030550+0000","last_clean":"2024-09-13T17:17:14.030550+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T17:17:14.030550+0000","last_undegraded":"2024-09-13T17:17:14.030550+0000","last_fullsized":"2024-09-13T17:17:14.030550+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"63'4212","reported_seq":4277,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.051215+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T17:17:14.051215+0000","last_peered":"2024-09-13T17:17:14.051215+0000","last_clean":"2024-09-13T17:17:14.051215+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T17:17:14.051215+0000","last_undegraded":"2024-09-13T17:17:14.051215+0000","last_fullsized":"2024-09-13T17:17:14.051215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":17,"num_write":23343,"num_write_kb":314959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"63'3296","reported_seq":3362,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.125915+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T17:17:22.125915+0000","last_peered":"2024-09-13T17:17:22.125915+0000","last_clean":"2024-09-13T17:17:22.125915+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T17:17:22.125915+0000","last_undegraded":"2024-09-13T17:17:22.125915+0000","last_fullsized":"2024-09-13T17:17:22.125915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3296,"log_dups_size":0,"ondisk_log_size":3296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17758,"num_write_kb":230990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'3","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.059668+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T17:17:22.059668+0000","last_peered":"2024-09-13T17:17:22.059668+0000","last_clean":"2024-09-13T17:17:22.059668+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T17:17:22.059668+0000","last_undegraded":"2024-09-13T17:17:22.059668+0000","last_fullsized":"2024-09-13T17:17:22.059668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'5","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.027550+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T17:17:14.027550+0000","last_peered":"2024-09-13T17:17:14.027550+0000","last_clean":"2024-09-13T17:17:14.027550+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T17:17:14.027550+0000","last_undegraded":"2024-09-13T17:17:14.027550+0000","last_fullsized":"2024-09-13T17:17:14.027550+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"63'3371","reported_seq":3434,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022230+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T17:17:14.022230+0000","last_peered":"2024-09-13T17:17:14.022230+0000","last_clean":"2024-09-13T17:17:14.022230+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T17:17:14.022230+0000","last_undegraded":"2024-09-13T17:17:14.022230+0000","last_fullsized":"2024-09-13T17:17:14.022230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3371,"log_dups_size":0,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18156,"num_write_kb":249012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"63'3752","reported_seq":3818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.984671+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T17:17:21.984671+0000","last_peered":"2024-09-13T17:17:21.984671+0000","last_clean":"2024-09-13T17:17:21.984671+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T17:17:21.984671+0000","last_undegraded":"2024-09-13T17:17:21.984671+0000","last_fullsized":"2024-09-13T17:17:21.984671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21127,"num_write_kb":268998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"63'2","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.056813+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T17:17:22.056813+0000","last_peered":"2024-09-13T17:17:22.056813+0000","last_clean":"2024-09-13T17:17:22.056813+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T17:17:22.056813+0000","last_undegraded":"2024-09-13T17:17:22.056813+0000","last_fullsized":"2024-09-13T17:17:22.056813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"63'4","reported_seq":88,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.029328+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T17:17:14.029328+0000","last_peered":"2024-09-13T17:17:14.029328+0000","last_clean":"2024-09-13T17:17:14.029328+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T17:17:14.029328+0000","last_undegraded":"2024-09-13T17:17:14.029328+0000","last_fullsized":"2024-09-13T17:17:14.029328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"63'4499","reported_seq":4565,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.023958+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T17:17:14.023958+0000","last_peered":"2024-09-13T17:17:14.023958+0000","last_clean":"2024-09-13T17:17:14.023958+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T17:17:14.023958+0000","last_undegraded":"2024-09-13T17:17:14.023958+0000","last_fullsized":"2024-09-13T17:17:14.023958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":24341,"num_write_kb":343418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"63'4043","reported_seq":4109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.908339+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T17:17:21.908339+0000","last_peered":"2024-09-13T17:17:21.908339+0000","last_clean":"2024-09-13T17:17:21.908339+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T17:17:21.908339+0000","last_undegraded":"2024-09-13T17:17:21.908339+0000","last_fullsized":"2024-09-13T17:17:21.908339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22005,"num_write_kb":288910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"63'5","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.010552+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T17:17:22.010552+0000","last_peered":"2024-09-13T17:17:22.010552+0000","last_clean":"2024-09-13T17:17:22.010552+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T17:17:22.010552+0000","last_undegraded":"2024-09-13T17:17:22.010552+0000","last_fullsized":"2024-09-13T17:17:22.010552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"63'7","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024599+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T17:17:14.024599+0000","last_peered":"2024-09-13T17:17:14.024599+0000","last_clean":"2024-09-13T17:17:14.024599+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T17:17:14.024599+0000","last_undegraded":"2024-09-13T17:17:14.024599+0000","last_fullsized":"2024-09-13T17:17:14.024599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"63'3470","reported_seq":3533,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.032577+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T17:17:14.032577+0000","last_peered":"2024-09-13T17:17:14.032577+0000","last_clean":"2024-09-13T17:17:14.032577+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T17:17:14.032577+0000","last_undegraded":"2024-09-13T17:17:14.032577+0000","last_fullsized":"2024-09-13T17:17:14.032577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19441,"num_write_kb":258288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'4844","reported_seq":4911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.060853+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T17:17:22.060853+0000","last_peered":"2024-09-13T17:17:22.060853+0000","last_clean":"2024-09-13T17:17:22.060853+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T17:17:22.060853+0000","last_undegraded":"2024-09-13T17:17:22.060853+0000","last_fullsized":"2024-09-13T17:17:22.060853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":26635,"num_write_kb":336207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"63'9","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.115612+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T17:17:22.115612+0000","last_peered":"2024-09-13T17:17:22.115612+0000","last_clean":"2024-09-13T17:17:22.115612+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T17:17:22.115612+0000","last_undegraded":"2024-09-13T17:17:22.115612+0000","last_fullsized":"2024-09-13T17:17:22.115612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"63'48","reported_seq":175,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:24.143768+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T17:17:24.143768+0000","last_peered":"2024-09-13T17:17:24.143768+0000","last_clean":"2024-09-13T17:17:24.143768+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T17:17:24.143768+0000","last_undegraded":"2024-09-13T17:17:24.143768+0000","last_fullsized":"2024-09-13T17:17:24.143768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92726,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":118,"num_write":43,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"63'3635","reported_seq":3699,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.001656+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T17:17:14.001656+0000","last_peered":"2024-09-13T17:17:14.001656+0000","last_clean":"2024-09-13T17:17:14.001656+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T17:17:14.001656+0000","last_undegraded":"2024-09-13T17:17:14.001656+0000","last_fullsized":"2024-09-13T17:17:14.001656+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19787,"num_write_kb":259620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'3639","reported_seq":3705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.121330+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T17:17:22.121330+0000","last_peered":"2024-09-13T17:17:22.121330+0000","last_clean":"2024-09-13T17:17:22.121330+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T17:17:22.121330+0000","last_undegraded":"2024-09-13T17:17:22.121330+0000","last_fullsized":"2024-09-13T17:17:22.121330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3639,"log_dups_size":0,"ondisk_log_size":3639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20066,"num_write_kb":240267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"63'265","reported_seq":350,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.104288+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T17:17:22.104288+0000","last_peered":"2024-09-13T17:17:22.104288+0000","last_clean":"2024-09-13T17:17:22.104288+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T17:17:22.104288+0000","last_undegraded":"2024-09-13T17:17:22.104288+0000","last_fullsized":"2024-09-13T17:17:22.104288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":46,"num_write":634,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"63'11","reported_seq":75,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.026249+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T17:17:14.026249+0000","last_peered":"2024-09-13T17:17:14.026249+0000","last_clean":"2024-09-13T17:17:14.026249+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T17:17:14.026249+0000","last_undegraded":"2024-09-13T17:17:14.026249+0000","last_fullsized":"2024-09-13T17:17:14.026249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"63'3190","reported_seq":3253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.040510+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T17:17:14.040510+0000","last_peered":"2024-09-13T17:17:14.040510+0000","last_clean":"2024-09-13T17:17:14.040510+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T17:17:14.040510+0000","last_undegraded":"2024-09-13T17:17:14.040510+0000","last_fullsized":"2024-09-13T17:17:14.040510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3190,"log_dups_size":0,"ondisk_log_size":3190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":254866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"63'4701","reported_seq":4768,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.071944+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T17:17:22.071944+0000","last_peered":"2024-09-13T17:17:22.071944+0000","last_clean":"2024-09-13T17:17:22.071944+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T17:17:22.071944+0000","last_undegraded":"2024-09-13T17:17:22.071944+0000","last_fullsized":"2024-09-13T17:17:22.071944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26171,"num_write_kb":338588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'2","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:19.802813+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T17:17:19.802813+0000","last_peered":"2024-09-13T17:17:19.802813+0000","last_clean":"2024-09-13T17:17:19.802813+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T17:17:19.802813+0000","last_undegraded":"2024-09-13T17:17:19.802813+0000","last_fullsized":"2024-09-13T17:17:19.802813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"63'3","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.988743+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T17:17:13.988743+0000","last_peered":"2024-09-13T17:17:13.988743+0000","last_clean":"2024-09-13T17:17:13.988743+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T17:17:13.988743+0000","last_undegraded":"2024-09-13T17:17:13.988743+0000","last_fullsized":"2024-09-13T17:17:13.988743+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"63'3859","reported_seq":3922,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.945566+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T17:17:13.945566+0000","last_peered":"2024-09-13T17:17:13.945566+0000","last_clean":"2024-09-13T17:17:13.945566+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T17:17:13.945566+0000","last_undegraded":"2024-09-13T17:17:13.945566+0000","last_fullsized":"2024-09-13T17:17:13.945566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21286,"num_write_kb":290222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"63'3876","reported_seq":3943,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.089484+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T17:17:22.089484+0000","last_peered":"2024-09-13T17:17:22.089484+0000","last_clean":"2024-09-13T17:17:22.089484+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T17:17:22.089484+0000","last_undegraded":"2024-09-13T17:17:22.089484+0000","last_fullsized":"2024-09-13T17:17:22.089484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21201,"num_write_kb":273354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"63'2","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.094910+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T17:17:22.094910+0000","last_peered":"2024-09-13T17:17:22.094910+0000","last_clean":"2024-09-13T17:17:22.094910+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T17:17:22.094910+0000","last_undegraded":"2024-09-13T17:17:22.094910+0000","last_fullsized":"2024-09-13T17:17:22.094910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'5","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.944152+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T17:17:12.944152+0000","last_peered":"2024-09-13T17:17:12.944152+0000","last_clean":"2024-09-13T17:17:12.944152+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T17:17:12.944152+0000","last_undegraded":"2024-09-13T17:17:12.944152+0000","last_fullsized":"2024-09-13T17:17:12.944152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"63'3422","reported_seq":3486,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058428+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T17:17:14.058428+0000","last_peered":"2024-09-13T17:17:14.058428+0000","last_clean":"2024-09-13T17:17:14.058428+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T17:17:14.058428+0000","last_undegraded":"2024-09-13T17:17:14.058428+0000","last_fullsized":"2024-09-13T17:17:14.058428+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":7,"num_write":19042,"num_write_kb":260722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"63'4802","reported_seq":4869,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.992037+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T17:17:21.992037+0000","last_peered":"2024-09-13T17:17:21.992037+0000","last_clean":"2024-09-13T17:17:21.992037+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T17:17:21.992037+0000","last_undegraded":"2024-09-13T17:17:21.992037+0000","last_fullsized":"2024-09-13T17:17:21.992037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26419,"num_write_kb":335470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"63'4","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.258399+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T17:17:14.258399+0000","last_peered":"2024-09-13T17:17:14.258399+0000","last_clean":"2024-09-13T17:17:14.258399+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T17:17:14.258399+0000","last_undegraded":"2024-09-13T17:17:14.258399+0000","last_fullsized":"2024-09-13T17:17:14.258399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"63'2","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.026552+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T17:17:14.026552+0000","last_peered":"2024-09-13T17:17:14.026552+0000","last_clean":"2024-09-13T17:17:14.026552+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T17:17:14.026552+0000","last_undegraded":"2024-09-13T17:17:14.026552+0000","last_fullsized":"2024-09-13T17:17:14.026552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"63'4215","reported_seq":4278,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.984815+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T17:17:13.984815+0000","last_peered":"2024-09-13T17:17:13.984815+0000","last_clean":"2024-09-13T17:17:13.984815+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T17:17:13.984815+0000","last_undegraded":"2024-09-13T17:17:13.984815+0000","last_fullsized":"2024-09-13T17:17:13.984815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":8,"num_write":22461,"num_write_kb":319331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'3790","reported_seq":3857,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107689+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T17:17:22.107689+0000","last_peered":"2024-09-13T17:17:22.107689+0000","last_clean":"2024-09-13T17:17:22.107689+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T17:17:22.107689+0000","last_undegraded":"2024-09-13T17:17:22.107689+0000","last_fullsized":"2024-09-13T17:17:22.107689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21077,"num_write_kb":260732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"63'40","reported_seq":162,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.062885+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T17:17:22.062885+0000","last_peered":"2024-09-13T17:17:22.062885+0000","last_clean":"2024-09-13T17:17:22.062885+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T17:17:22.062885+0000","last_undegraded":"2024-09-13T17:17:22.062885+0000","last_fullsized":"2024-09-13T17:17:22.062885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84545,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":54,"num_read_kb":105,"num_write":36,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"63'9","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.040704+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T17:17:14.040704+0000","last_peered":"2024-09-13T17:17:14.040704+0000","last_clean":"2024-09-13T17:17:14.040704+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T17:17:14.040704+0000","last_undegraded":"2024-09-13T17:17:14.040704+0000","last_fullsized":"2024-09-13T17:17:14.040704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"63'3128","reported_seq":3190,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.042016+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T17:17:14.042016+0000","last_peered":"2024-09-13T17:17:14.042016+0000","last_clean":"2024-09-13T17:17:14.042016+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T17:17:14.042016+0000","last_undegraded":"2024-09-13T17:17:14.042016+0000","last_fullsized":"2024-09-13T17:17:14.042016+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3128,"log_dups_size":0,"ondisk_log_size":3128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17776,"num_write_kb":243283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"63'3657","reported_seq":3724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.977555+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T17:17:21.977555+0000","last_peered":"2024-09-13T17:17:21.977555+0000","last_clean":"2024-09-13T17:17:21.977555+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T17:17:21.977555+0000","last_undegraded":"2024-09-13T17:17:21.977555+0000","last_fullsized":"2024-09-13T17:17:21.977555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19816,"num_write_kb":255512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'10","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.101605+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T17:17:22.101605+0000","last_peered":"2024-09-13T17:17:22.101605+0000","last_clean":"2024-09-13T17:17:22.101605+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T17:17:22.101605+0000","last_undegraded":"2024-09-13T17:17:22.101605+0000","last_fullsized":"2024-09-13T17:17:22.101605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'251","reported_seq":335,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.918664+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T17:17:22.918664+0000","last_peered":"2024-09-13T17:17:22.918664+0000","last_clean":"2024-09-13T17:17:22.918664+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T17:17:22.918664+0000","last_undegraded":"2024-09-13T17:17:22.918664+0000","last_fullsized":"2024-09-13T17:17:22.918664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":57,"num_write":597,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"63'4141","reported_seq":4205,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022107+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T17:17:14.022107+0000","last_peered":"2024-09-13T17:17:14.022107+0000","last_clean":"2024-09-13T17:17:14.022107+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T17:17:14.022107+0000","last_undegraded":"2024-09-13T17:17:14.022107+0000","last_fullsized":"2024-09-13T17:17:14.022107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22849,"num_write_kb":317008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'4103","reported_seq":4178,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.906868+0000","last_change":"2024-09-13T16:48:41.389774+0000","last_active":"2024-09-13T17:17:21.906868+0000","last_peered":"2024-09-13T17:17:21.906868+0000","last_clean":"2024-09-13T17:17:21.906868+0000","last_became_active":"2024-09-13T16:48:41.389507+0000","last_became_peered":"2024-09-13T16:48:41.389507+0000","last_unstale":"2024-09-13T17:17:21.906868+0000","last_undegraded":"2024-09-13T17:17:21.906868+0000","last_fullsized":"2024-09-13T17:17:21.906868+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:07:23.114010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22342,"num_write_kb":278194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"63'3","reported_seq":71,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058621+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T17:17:22.058621+0000","last_peered":"2024-09-13T17:17:22.058621+0000","last_clean":"2024-09-13T17:17:22.058621+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T17:17:22.058621+0000","last_undegraded":"2024-09-13T17:17:22.058621+0000","last_fullsized":"2024-09-13T17:17:22.058621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T17:16:04.972860+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T17:16:04.972860+0000","last_peered":"2024-09-13T17:16:04.972860+0000","last_clean":"2024-09-13T17:16:04.972860+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T17:16:04.972860+0000","last_undegraded":"2024-09-13T17:16:04.972860+0000","last_fullsized":"2024-09-13T17:16:04.972860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"63'2","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.946275+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T17:17:12.946275+0000","last_peered":"2024-09-13T17:17:12.946275+0000","last_clean":"2024-09-13T17:17:12.946275+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T17:17:12.946275+0000","last_undegraded":"2024-09-13T17:17:12.946275+0000","last_fullsized":"2024-09-13T17:17:12.946275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"63'2904","reported_seq":2967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058460+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T17:17:14.058460+0000","last_peered":"2024-09-13T17:17:14.058460+0000","last_clean":"2024-09-13T17:17:14.058460+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T17:17:14.058460+0000","last_undegraded":"2024-09-13T17:17:14.058460+0000","last_fullsized":"2024-09-13T17:17:14.058460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":2,"num_write":15607,"num_write_kb":214254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'4284","reported_seq":4350,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117903+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T17:17:22.117903+0000","last_peered":"2024-09-13T17:17:22.117903+0000","last_clean":"2024-09-13T17:17:22.117903+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T17:17:22.117903+0000","last_undegraded":"2024-09-13T17:17:22.117903+0000","last_fullsized":"2024-09-13T17:17:22.117903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23392,"num_write_kb":310963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"63'5","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096955+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T17:17:22.096955+0000","last_peered":"2024-09-13T17:17:22.096955+0000","last_clean":"2024-09-13T17:17:22.096955+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T17:17:22.096955+0000","last_undegraded":"2024-09-13T17:17:22.096955+0000","last_fullsized":"2024-09-13T17:17:22.096955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"63'3","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.991304+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T17:17:13.991304+0000","last_peered":"2024-09-13T17:17:13.991304+0000","last_clean":"2024-09-13T17:17:13.991304+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T17:17:13.991304+0000","last_undegraded":"2024-09-13T17:17:13.991304+0000","last_fullsized":"2024-09-13T17:17:13.991304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"63'4138","reported_seq":4200,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.019219+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T17:17:14.019219+0000","last_peered":"2024-09-13T17:17:14.019219+0000","last_clean":"2024-09-13T17:17:14.019219+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T17:17:14.019219+0000","last_undegraded":"2024-09-13T17:17:14.019219+0000","last_fullsized":"2024-09-13T17:17:14.019219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":309086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"63'4584","reported_seq":4643,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.128999+0000","last_change":"2024-09-13T16:48:40.767044+0000","last_active":"2024-09-13T17:17:22.128999+0000","last_peered":"2024-09-13T17:17:22.128999+0000","last_clean":"2024-09-13T17:17:22.128999+0000","last_became_active":"2024-09-13T16:48:40.765861+0000","last_became_peered":"2024-09-13T16:48:40.765861+0000","last_unstale":"2024-09-13T17:17:22.128999+0000","last_undegraded":"2024-09-13T17:17:22.128999+0000","last_fullsized":"2024-09-13T17:17:22.128999+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:39.408967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24690,"num_write_kb":320186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"63'6","reported_seq":91,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.999257+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T17:17:21.999257+0000","last_peered":"2024-09-13T17:17:21.999257+0000","last_clean":"2024-09-13T17:17:21.999257+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T17:17:21.999257+0000","last_undegraded":"2024-09-13T17:17:21.999257+0000","last_fullsized":"2024-09-13T17:17:21.999257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'1425","reported_seq":1536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.059648+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T17:17:14.059648+0000","last_peered":"2024-09-13T17:17:14.059648+0000","last_clean":"2024-09-13T17:17:14.059648+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T17:17:14.059648+0000","last_undegraded":"2024-09-13T17:17:14.059648+0000","last_fullsized":"2024-09-13T17:17:14.059648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":38459,"num_write":1400,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"63'3637","reported_seq":3700,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.967851+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T17:17:13.967851+0000","last_peered":"2024-09-13T17:17:13.967851+0000","last_clean":"2024-09-13T17:17:13.967851+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T17:17:13.967851+0000","last_undegraded":"2024-09-13T17:17:13.967851+0000","last_fullsized":"2024-09-13T17:17:13.967851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19635,"num_write_kb":259018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'5067","reported_seq":5126,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.041483+0000","last_change":"2024-09-13T16:48:40.831954+0000","last_active":"2024-09-13T17:17:22.041483+0000","last_peered":"2024-09-13T17:17:22.041483+0000","last_clean":"2024-09-13T17:17:22.041483+0000","last_became_active":"2024-09-13T16:48:40.831696+0000","last_became_peered":"2024-09-13T16:48:40.831696+0000","last_unstale":"2024-09-13T17:17:22.041483+0000","last_undegraded":"2024-09-13T17:17:22.041483+0000","last_fullsized":"2024-09-13T17:17:22.041483+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:53.452702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27028,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"63'7","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.064989+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T17:17:22.064989+0000","last_peered":"2024-09-13T17:17:22.064989+0000","last_clean":"2024-09-13T17:17:22.064989+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T17:17:22.064989+0000","last_undegraded":"2024-09-13T17:17:22.064989+0000","last_fullsized":"2024-09-13T17:17:22.064989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"63'5","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.951351+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T17:17:12.951351+0000","last_peered":"2024-09-13T17:17:12.951351+0000","last_clean":"2024-09-13T17:17:12.951351+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T17:17:12.951351+0000","last_undegraded":"2024-09-13T17:17:12.951351+0000","last_fullsized":"2024-09-13T17:17:12.951351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"63'2915","reported_seq":2977,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.931098+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T17:17:13.931098+0000","last_peered":"2024-09-13T17:17:13.931098+0000","last_clean":"2024-09-13T17:17:13.931098+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T17:17:13.931098+0000","last_undegraded":"2024-09-13T17:17:13.931098+0000","last_fullsized":"2024-09-13T17:17:13.931098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2915,"log_dups_size":0,"ondisk_log_size":2915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":234494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"63'3697","reported_seq":3763,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.091815+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T17:17:22.091815+0000","last_peered":"2024-09-13T17:17:22.091815+0000","last_clean":"2024-09-13T17:17:22.091815+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T17:17:22.091815+0000","last_undegraded":"2024-09-13T17:17:22.091815+0000","last_fullsized":"2024-09-13T17:17:22.091815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20206,"num_write_kb":253140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"63'4","reported_seq":89,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107701+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T17:17:22.107701+0000","last_peered":"2024-09-13T17:17:22.107701+0000","last_clean":"2024-09-13T17:17:22.107701+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T17:17:22.107701+0000","last_undegraded":"2024-09-13T17:17:22.107701+0000","last_fullsized":"2024-09-13T17:17:22.107701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"63'2","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.991145+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T17:17:13.991145+0000","last_peered":"2024-09-13T17:17:13.991145+0000","last_clean":"2024-09-13T17:17:13.991145+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T17:17:13.991145+0000","last_undegraded":"2024-09-13T17:17:13.991145+0000","last_fullsized":"2024-09-13T17:17:13.991145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"63'3522","reported_seq":3583,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.999295+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T17:17:13.999295+0000","last_peered":"2024-09-13T17:17:13.999295+0000","last_clean":"2024-09-13T17:17:13.999295+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T17:17:13.999295+0000","last_undegraded":"2024-09-13T17:17:13.999295+0000","last_fullsized":"2024-09-13T17:17:13.999295+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18828,"num_write_kb":272277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"63'3874","reported_seq":3941,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.067632+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T17:17:22.067632+0000","last_peered":"2024-09-13T17:17:22.067632+0000","last_clean":"2024-09-13T17:17:22.067632+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T17:17:22.067632+0000","last_undegraded":"2024-09-13T17:17:22.067632+0000","last_fullsized":"2024-09-13T17:17:22.067632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21413,"num_write_kb":278431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"63'14","reported_seq":170,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095666+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T17:17:22.095666+0000","last_peered":"2024-09-13T17:17:22.095666+0000","last_clean":"2024-09-13T17:17:22.095666+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T17:17:22.095666+0000","last_undegraded":"2024-09-13T17:17:22.095666+0000","last_fullsized":"2024-09-13T17:17:22.095666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":270,"num_read_kb":270,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036103+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T17:17:14.036103+0000","last_peered":"2024-09-13T17:17:14.036103+0000","last_clean":"2024-09-13T17:17:14.036103+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T17:17:14.036103+0000","last_undegraded":"2024-09-13T17:17:14.036103+0000","last_fullsized":"2024-09-13T17:17:14.036103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"63'3622","reported_seq":3686,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058427+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T17:17:14.058427+0000","last_peered":"2024-09-13T17:17:14.058427+0000","last_clean":"2024-09-13T17:17:14.058427+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T17:17:14.058427+0000","last_undegraded":"2024-09-13T17:17:14.058427+0000","last_fullsized":"2024-09-13T17:17:14.058427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":19996,"num_write_kb":293269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"63'3240","reported_seq":3305,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.013971+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T17:17:22.013971+0000","last_peered":"2024-09-13T17:17:22.013971+0000","last_clean":"2024-09-13T17:17:22.013971+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T17:17:22.013971+0000","last_undegraded":"2024-09-13T17:17:22.013971+0000","last_fullsized":"2024-09-13T17:17:22.013971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3240,"log_dups_size":0,"ondisk_log_size":3240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":237245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"63'5","reported_seq":71,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.098797+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T17:17:22.098797+0000","last_peered":"2024-09-13T17:17:22.098797+0000","last_clean":"2024-09-13T17:17:22.098797+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T17:17:22.098797+0000","last_undegraded":"2024-09-13T17:17:22.098797+0000","last_fullsized":"2024-09-13T17:17:22.098797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'5","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.992012+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T17:17:13.992012+0000","last_peered":"2024-09-13T17:17:13.992012+0000","last_clean":"2024-09-13T17:17:13.992012+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T17:17:13.992012+0000","last_undegraded":"2024-09-13T17:17:13.992012+0000","last_fullsized":"2024-09-13T17:17:13.992012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"63'3376","reported_seq":3437,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.779058+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T17:17:13.779058+0000","last_peered":"2024-09-13T17:17:13.779058+0000","last_clean":"2024-09-13T17:17:13.779058+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T17:17:13.779058+0000","last_undegraded":"2024-09-13T17:17:13.779058+0000","last_fullsized":"2024-09-13T17:17:13.779058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18879,"num_write_kb":251906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"63'4848","reported_seq":4913,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.099538+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T17:17:22.099538+0000","last_peered":"2024-09-13T17:17:22.099538+0000","last_clean":"2024-09-13T17:17:22.099538+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T17:17:22.099538+0000","last_undegraded":"2024-09-13T17:17:22.099538+0000","last_fullsized":"2024-09-13T17:17:22.099538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26963,"num_write_kb":321221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"63'3","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117357+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T17:17:22.117357+0000","last_peered":"2024-09-13T17:17:22.117357+0000","last_clean":"2024-09-13T17:17:22.117357+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T17:17:22.117357+0000","last_undegraded":"2024-09-13T17:17:22.117357+0000","last_fullsized":"2024-09-13T17:17:22.117357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"63'389","reported_seq":558,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:09.910889+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T17:17:09.910889+0000","last_peered":"2024-09-13T17:17:09.910889+0000","last_clean":"2024-09-13T17:17:09.910889+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T17:17:09.910889+0000","last_undegraded":"2024-09-13T17:17:09.910889+0000","last_fullsized":"2024-09-13T17:17:09.910889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":191,"num_read_kb":82094,"num_write":385,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'4856","reported_seq":4919,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.025593+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T17:17:14.025593+0000","last_peered":"2024-09-13T17:17:14.025593+0000","last_clean":"2024-09-13T17:17:14.025593+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T17:17:14.025593+0000","last_undegraded":"2024-09-13T17:17:14.025593+0000","last_fullsized":"2024-09-13T17:17:14.025593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":26453,"num_write_kb":368210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'4063","reported_seq":4128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.081187+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T17:17:22.081187+0000","last_peered":"2024-09-13T17:17:22.081187+0000","last_clean":"2024-09-13T17:17:22.081187+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T17:17:22.081187+0000","last_undegraded":"2024-09-13T17:17:22.081187+0000","last_fullsized":"2024-09-13T17:17:22.081187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4063,"log_dups_size":0,"ondisk_log_size":4063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21999,"num_write_kb":293018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"63'8","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.105763+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T17:17:22.105763+0000","last_peered":"2024-09-13T17:17:22.105763+0000","last_clean":"2024-09-13T17:17:22.105763+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T17:17:22.105763+0000","last_undegraded":"2024-09-13T17:17:22.105763+0000","last_fullsized":"2024-09-13T17:17:22.105763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":144,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'30","reported_seq":155,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.143808+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T17:17:14.143808+0000","last_peered":"2024-09-13T17:17:14.143808+0000","last_clean":"2024-09-13T17:17:14.143808+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T17:17:14.143808+0000","last_undegraded":"2024-09-13T17:17:14.143808+0000","last_fullsized":"2024-09-13T17:17:14.143808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":31,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"63'4239","reported_seq":4300,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.052220+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T17:17:14.052220+0000","last_peered":"2024-09-13T17:17:14.052220+0000","last_clean":"2024-09-13T17:17:14.052220+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T17:17:14.052220+0000","last_undegraded":"2024-09-13T17:17:14.052220+0000","last_fullsized":"2024-09-13T17:17:14.052220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23316,"num_write_kb":318961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'3718","reported_seq":3783,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.786167+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T17:17:21.786167+0000","last_peered":"2024-09-13T17:17:21.786167+0000","last_clean":"2024-09-13T17:17:21.786167+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T17:17:21.786167+0000","last_undegraded":"2024-09-13T17:17:21.786167+0000","last_fullsized":"2024-09-13T17:17:21.786167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3718,"log_dups_size":0,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20956,"num_write_kb":260218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"63'390","reported_seq":546,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:17.006685+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T17:17:17.006685+0000","last_peered":"2024-09-13T17:17:17.006685+0000","last_clean":"2024-09-13T17:17:17.006685+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T17:17:17.006685+0000","last_undegraded":"2024-09-13T17:17:17.006685+0000","last_fullsized":"2024-09-13T17:17:17.006685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":73872,"num_write":388,"num_write_kb":4310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.039214+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T17:17:14.039214+0000","last_peered":"2024-09-13T17:17:14.039214+0000","last_clean":"2024-09-13T17:17:14.039214+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T17:17:14.039214+0000","last_undegraded":"2024-09-13T17:17:14.039214+0000","last_fullsized":"2024-09-13T17:17:14.039214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"63'3522","reported_seq":3584,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.982810+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T17:17:13.982810+0000","last_peered":"2024-09-13T17:17:13.982810+0000","last_clean":"2024-09-13T17:17:13.982810+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T17:17:13.982810+0000","last_undegraded":"2024-09-13T17:17:13.982810+0000","last_fullsized":"2024-09-13T17:17:13.982810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":4,"num_write":19373,"num_write_kb":266222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"63'5138","reported_seq":5203,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095588+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T17:17:22.095588+0000","last_peered":"2024-09-13T17:17:22.095588+0000","last_clean":"2024-09-13T17:17:22.095588+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T17:17:22.095588+0000","last_undegraded":"2024-09-13T17:17:22.095588+0000","last_fullsized":"2024-09-13T17:17:22.095588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27863,"num_write_kb":353454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"63'23","reported_seq":125,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.131774+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T17:17:22.131774+0000","last_peered":"2024-09-13T17:17:22.131774+0000","last_clean":"2024-09-13T17:17:22.131774+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T17:17:22.131774+0000","last_undegraded":"2024-09-13T17:17:22.131774+0000","last_fullsized":"2024-09-13T17:17:22.131774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"63'7","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031857+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T17:17:14.031857+0000","last_peered":"2024-09-13T17:17:14.031857+0000","last_clean":"2024-09-13T17:17:14.031857+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T17:17:14.031857+0000","last_undegraded":"2024-09-13T17:17:14.031857+0000","last_fullsized":"2024-09-13T17:17:14.031857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":168,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"63'3602","reported_seq":3666,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.014764+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T17:17:14.014764+0000","last_peered":"2024-09-13T17:17:14.014764+0000","last_clean":"2024-09-13T17:17:14.014764+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T17:17:14.014764+0000","last_undegraded":"2024-09-13T17:17:14.014764+0000","last_fullsized":"2024-09-13T17:17:14.014764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":20290,"num_write_kb":278255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'3743","reported_seq":3808,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084809+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T17:17:22.084809+0000","last_peered":"2024-09-13T17:17:22.084809+0000","last_clean":"2024-09-13T17:17:22.084809+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T17:17:22.084809+0000","last_undegraded":"2024-09-13T17:17:22.084809+0000","last_fullsized":"2024-09-13T17:17:22.084809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20404,"num_write_kb":278673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"63'3","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.103865+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T17:17:22.103865+0000","last_peered":"2024-09-13T17:17:22.103865+0000","last_clean":"2024-09-13T17:17:22.103865+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T17:17:22.103865+0000","last_undegraded":"2024-09-13T17:17:22.103865+0000","last_fullsized":"2024-09-13T17:17:22.103865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"63'15","reported_seq":185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024454+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T17:17:14.024454+0000","last_peered":"2024-09-13T17:17:14.024454+0000","last_clean":"2024-09-13T17:17:14.024454+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T17:17:14.024454+0000","last_undegraded":"2024-09-13T17:17:14.024454+0000","last_fullsized":"2024-09-13T17:17:14.024454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":324,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"63'3189","reported_seq":3250,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.950668+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T17:17:13.950668+0000","last_peered":"2024-09-13T17:17:13.950668+0000","last_clean":"2024-09-13T17:17:13.950668+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T17:17:13.950668+0000","last_undegraded":"2024-09-13T17:17:13.950668+0000","last_fullsized":"2024-09-13T17:17:13.950668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17342,"num_write_kb":249797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"63'4223","reported_seq":4288,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117238+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T17:17:22.117238+0000","last_peered":"2024-09-13T17:17:22.117238+0000","last_clean":"2024-09-13T17:17:22.117238+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T17:17:22.117238+0000","last_undegraded":"2024-09-13T17:17:22.117238+0000","last_fullsized":"2024-09-13T17:17:22.117238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4223,"log_dups_size":0,"ondisk_log_size":4223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":295025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"63'4","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.059446+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T17:17:22.059446+0000","last_peered":"2024-09-13T17:17:22.059446+0000","last_clean":"2024-09-13T17:17:22.059446+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T17:17:22.059446+0000","last_undegraded":"2024-09-13T17:17:22.059446+0000","last_fullsized":"2024-09-13T17:17:22.059446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"63'5","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.021576+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T17:17:14.021576+0000","last_peered":"2024-09-13T17:17:14.021576+0000","last_clean":"2024-09-13T17:17:14.021576+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T17:17:14.021576+0000","last_undegraded":"2024-09-13T17:17:14.021576+0000","last_fullsized":"2024-09-13T17:17:14.021576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'5357","reported_seq":5423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058245+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T17:17:14.058245+0000","last_peered":"2024-09-13T17:17:14.058245+0000","last_clean":"2024-09-13T17:17:14.058245+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T17:17:14.058245+0000","last_undegraded":"2024-09-13T17:17:14.058245+0000","last_fullsized":"2024-09-13T17:17:14.058245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":39,"num_write":29637,"num_write_kb":393569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"63'26","reported_seq":90,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.989101+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T17:17:13.989101+0000","last_peered":"2024-09-13T17:17:13.989101+0000","last_clean":"2024-09-13T17:17:13.989101+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T17:17:13.989101+0000","last_undegraded":"2024-09-13T17:17:13.989101+0000","last_fullsized":"2024-09-13T17:17:13.989101+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"63'5122","reported_seq":5187,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.106963+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T17:17:22.106963+0000","last_peered":"2024-09-13T17:17:22.106963+0000","last_clean":"2024-09-13T17:17:22.106963+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T17:17:22.106963+0000","last_undegraded":"2024-09-13T17:17:22.106963+0000","last_fullsized":"2024-09-13T17:17:22.106963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":348704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"63'2","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:04.209346+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T17:17:04.209346+0000","last_peered":"2024-09-13T17:17:04.209346+0000","last_clean":"2024-09-13T17:17:04.209346+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T17:17:04.209346+0000","last_undegraded":"2024-09-13T17:17:04.209346+0000","last_fullsized":"2024-09-13T17:17:04.209346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"63'3554","reported_seq":3617,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036436+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T17:17:14.036436+0000","last_peered":"2024-09-13T17:17:14.036436+0000","last_clean":"2024-09-13T17:17:14.036436+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T17:17:14.036436+0000","last_undegraded":"2024-09-13T17:17:14.036436+0000","last_fullsized":"2024-09-13T17:17:14.036436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":268389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"63'1091","reported_seq":1205,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.012135+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T17:17:14.012135+0000","last_peered":"2024-09-13T17:17:14.012135+0000","last_clean":"2024-09-13T17:17:14.012135+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T17:17:14.012135+0000","last_undegraded":"2024-09-13T17:17:14.012135+0000","last_fullsized":"2024-09-13T17:17:14.012135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":9436,"num_write":1065,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"63'4714","reported_seq":4779,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.121909+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T17:17:22.121909+0000","last_peered":"2024-09-13T17:17:22.121909+0000","last_clean":"2024-09-13T17:17:22.121909+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T17:17:22.121909+0000","last_undegraded":"2024-09-13T17:17:22.121909+0000","last_fullsized":"2024-09-13T17:17:22.121909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25459,"num_write_kb":343662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"63'3","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.098646+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T17:17:22.098646+0000","last_peered":"2024-09-13T17:17:22.098646+0000","last_clean":"2024-09-13T17:17:22.098646+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T17:17:22.098646+0000","last_undegraded":"2024-09-13T17:17:22.098646+0000","last_fullsized":"2024-09-13T17:17:22.098646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"63'4328","reported_seq":4388,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.013223+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T17:17:14.013223+0000","last_peered":"2024-09-13T17:17:14.013223+0000","last_clean":"2024-09-13T17:17:14.013223+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T17:17:14.013223+0000","last_undegraded":"2024-09-13T17:17:14.013223+0000","last_fullsized":"2024-09-13T17:17:14.013223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24456,"num_write_kb":315374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"63'396","reported_seq":463,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:23.808734+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T17:17:23.808734+0000","last_peered":"2024-09-13T17:17:23.808734+0000","last_clean":"2024-09-13T17:17:23.808734+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T17:17:23.808734+0000","last_undegraded":"2024-09-13T17:17:23.808734+0000","last_fullsized":"2024-09-13T17:17:23.808734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2790242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":465,"num_write":372,"num_write_kb":2945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"63'4155","reported_seq":4219,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.033063+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T17:17:22.033063+0000","last_peered":"2024-09-13T17:17:22.033063+0000","last_clean":"2024-09-13T17:17:22.033063+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T17:17:22.033063+0000","last_undegraded":"2024-09-13T17:17:22.033063+0000","last_fullsized":"2024-09-13T17:17:22.033063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22247,"num_write_kb":285891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"63'5","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:16:34.958339+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T17:16:34.958339+0000","last_peered":"2024-09-13T17:16:34.958339+0000","last_clean":"2024-09-13T17:16:34.958339+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T17:16:34.958339+0000","last_undegraded":"2024-09-13T17:16:34.958339+0000","last_fullsized":"2024-09-13T17:16:34.958339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"63'3426","reported_seq":3489,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.986253+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T17:17:13.986253+0000","last_peered":"2024-09-13T17:17:13.986253+0000","last_clean":"2024-09-13T17:17:13.986253+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T17:17:13.986253+0000","last_undegraded":"2024-09-13T17:17:13.986253+0000","last_fullsized":"2024-09-13T17:17:13.986253+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3426,"log_dups_size":0,"ondisk_log_size":3426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":18510,"num_write_kb":250514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'712","reported_seq":799,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058400+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T17:17:22.058400+0000","last_peered":"2024-09-13T17:17:22.058400+0000","last_clean":"2024-09-13T17:17:22.058400+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T17:17:22.058400+0000","last_undegraded":"2024-09-13T17:17:22.058400+0000","last_fullsized":"2024-09-13T17:17:22.058400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":70578,"num_write":702,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"63'4316","reported_seq":4381,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.050313+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T17:17:22.050313+0000","last_peered":"2024-09-13T17:17:22.050313+0000","last_clean":"2024-09-13T17:17:22.050313+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T17:17:22.050313+0000","last_undegraded":"2024-09-13T17:17:22.050313+0000","last_fullsized":"2024-09-13T17:17:22.050313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22350,"num_write_kb":290228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'44","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031452+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T17:17:14.031452+0000","last_peered":"2024-09-13T17:17:14.031452+0000","last_clean":"2024-09-13T17:17:14.031452+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T17:17:14.031452+0000","last_undegraded":"2024-09-13T17:17:14.031452+0000","last_fullsized":"2024-09-13T17:17:14.031452+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"63'3129","reported_seq":3191,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031759+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T17:17:14.031759+0000","last_peered":"2024-09-13T17:17:14.031759+0000","last_clean":"2024-09-13T17:17:14.031759+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T17:17:14.031759+0000","last_undegraded":"2024-09-13T17:17:14.031759+0000","last_fullsized":"2024-09-13T17:17:14.031759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17279,"num_write_kb":257601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"63'4785","reported_seq":4852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.086557+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T17:17:22.086557+0000","last_peered":"2024-09-13T17:17:22.086557+0000","last_clean":"2024-09-13T17:17:22.086557+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T17:17:22.086557+0000","last_undegraded":"2024-09-13T17:17:22.086557+0000","last_fullsized":"2024-09-13T17:17:22.086557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26302,"num_write_kb":310249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"63'4231","reported_seq":4294,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.009044+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T17:17:14.009044+0000","last_peered":"2024-09-13T17:17:14.009044+0000","last_clean":"2024-09-13T17:17:14.009044+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T17:17:14.009044+0000","last_undegraded":"2024-09-13T17:17:14.009044+0000","last_fullsized":"2024-09-13T17:17:14.009044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22604,"num_write_kb":324065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"63'3874","reported_seq":3942,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.078497+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T17:17:22.078497+0000","last_peered":"2024-09-13T17:17:22.078497+0000","last_clean":"2024-09-13T17:17:22.078497+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T17:17:22.078497+0000","last_undegraded":"2024-09-13T17:17:22.078497+0000","last_fullsized":"2024-09-13T17:17:22.078497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20920,"num_write_kb":276387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"63'3865","reported_seq":3928,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.006270+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T17:17:14.006270+0000","last_peered":"2024-09-13T17:17:14.006270+0000","last_clean":"2024-09-13T17:17:14.006270+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T17:17:14.006270+0000","last_undegraded":"2024-09-13T17:17:14.006270+0000","last_fullsized":"2024-09-13T17:17:14.006270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21432,"num_write_kb":286796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"63'4552","reported_seq":4621,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.103214+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T17:17:22.103214+0000","last_peered":"2024-09-13T17:17:22.103214+0000","last_clean":"2024-09-13T17:17:22.103214+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T17:17:22.103214+0000","last_undegraded":"2024-09-13T17:17:22.103214+0000","last_fullsized":"2024-09-13T17:17:22.103214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":24031,"num_write_kb":313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"63'4071","reported_seq":4134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.013587+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T17:17:14.013587+0000","last_peered":"2024-09-13T17:17:14.013587+0000","last_clean":"2024-09-13T17:17:14.013587+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T17:17:14.013587+0000","last_undegraded":"2024-09-13T17:17:14.013587+0000","last_fullsized":"2024-09-13T17:17:14.013587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":22061,"num_write_kb":309650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"63'3506","reported_seq":3574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.087622+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T17:17:22.087622+0000","last_peered":"2024-09-13T17:17:22.087622+0000","last_clean":"2024-09-13T17:17:22.087622+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T17:17:22.087622+0000","last_undegraded":"2024-09-13T17:17:22.087622+0000","last_fullsized":"2024-09-13T17:17:22.087622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3506,"log_dups_size":0,"ondisk_log_size":3506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19493,"num_write_kb":254380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"63'4523","reported_seq":4586,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.039988+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T17:17:14.039988+0000","last_peered":"2024-09-13T17:17:14.039988+0000","last_clean":"2024-09-13T17:17:14.039988+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T17:17:14.039988+0000","last_undegraded":"2024-09-13T17:17:14.039988+0000","last_fullsized":"2024-09-13T17:17:14.039988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4523,"log_dups_size":0,"ondisk_log_size":4523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24221,"num_write_kb":338849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"63'4422","reported_seq":4489,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.072016+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T17:17:22.072016+0000","last_peered":"2024-09-13T17:17:22.072016+0000","last_clean":"2024-09-13T17:17:22.072016+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T17:17:22.072016+0000","last_undegraded":"2024-09-13T17:17:22.072016+0000","last_fullsized":"2024-09-13T17:17:22.072016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":24862,"num_write_kb":304752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"63'3771","reported_seq":3835,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022654+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T17:17:14.022654+0000","last_peered":"2024-09-13T17:17:14.022654+0000","last_clean":"2024-09-13T17:17:14.022654+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T17:17:14.022654+0000","last_undegraded":"2024-09-13T17:17:14.022654+0000","last_fullsized":"2024-09-13T17:17:14.022654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21135,"num_write_kb":285540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"63'4461","reported_seq":4527,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095053+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T17:17:22.095053+0000","last_peered":"2024-09-13T17:17:22.095053+0000","last_clean":"2024-09-13T17:17:22.095053+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T17:17:22.095053+0000","last_undegraded":"2024-09-13T17:17:22.095053+0000","last_fullsized":"2024-09-13T17:17:22.095053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25388,"num_write_kb":319094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"63'4193","reported_seq":4256,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.014710+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T17:17:14.014710+0000","last_peered":"2024-09-13T17:17:14.014710+0000","last_clean":"2024-09-13T17:17:14.014710+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T17:17:14.014710+0000","last_undegraded":"2024-09-13T17:17:14.014710+0000","last_fullsized":"2024-09-13T17:17:14.014710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4193,"log_dups_size":0,"ondisk_log_size":4193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23066,"num_write_kb":313713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"63'3978","reported_seq":4044,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.100942+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T17:17:22.100942+0000","last_peered":"2024-09-13T17:17:22.100942+0000","last_clean":"2024-09-13T17:17:22.100942+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T17:17:22.100942+0000","last_undegraded":"2024-09-13T17:17:22.100942+0000","last_fullsized":"2024-09-13T17:17:22.100942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22157,"num_write_kb":281285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"63'4050","reported_seq":4115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.048739+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T17:17:14.048739+0000","last_peered":"2024-09-13T17:17:14.048739+0000","last_clean":"2024-09-13T17:17:14.048739+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T17:17:14.048739+0000","last_undegraded":"2024-09-13T17:17:14.048739+0000","last_fullsized":"2024-09-13T17:17:14.048739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4050,"log_dups_size":0,"ondisk_log_size":4050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21731,"num_write_kb":291923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"63'5211","reported_seq":5272,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.046515+0000","last_change":"2024-09-13T16:48:40.831835+0000","last_active":"2024-09-13T17:17:22.046515+0000","last_peered":"2024-09-13T17:17:22.046515+0000","last_clean":"2024-09-13T17:17:22.046515+0000","last_became_active":"2024-09-13T16:48:40.831649+0000","last_became_peered":"2024-09-13T16:48:40.831649+0000","last_unstale":"2024-09-13T17:17:22.046515+0000","last_undegraded":"2024-09-13T17:17:22.046515+0000","last_fullsized":"2024-09-13T17:17:22.046515+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:54.247988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":28065,"num_write_kb":366501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"63'3806","reported_seq":3870,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024614+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T17:17:14.024614+0000","last_peered":"2024-09-13T17:17:14.024614+0000","last_clean":"2024-09-13T17:17:14.024614+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T17:17:14.024614+0000","last_undegraded":"2024-09-13T17:17:14.024614+0000","last_fullsized":"2024-09-13T17:17:14.024614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4,"num_write":20286,"num_write_kb":286237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"63'3659","reported_seq":3726,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126330+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T17:17:22.126330+0000","last_peered":"2024-09-13T17:17:22.126330+0000","last_clean":"2024-09-13T17:17:22.126330+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T17:17:22.126330+0000","last_undegraded":"2024-09-13T17:17:22.126330+0000","last_fullsized":"2024-09-13T17:17:22.126330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19936,"num_write_kb":264332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"63'5003","reported_seq":5065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.034159+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T17:17:14.034159+0000","last_peered":"2024-09-13T17:17:14.034159+0000","last_clean":"2024-09-13T17:17:14.034159+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T17:17:14.034159+0000","last_undegraded":"2024-09-13T17:17:14.034159+0000","last_fullsized":"2024-09-13T17:17:14.034159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27137,"num_write_kb":387189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"63'4068","reported_seq":4134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.106118+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T17:17:22.106118+0000","last_peered":"2024-09-13T17:17:22.106118+0000","last_clean":"2024-09-13T17:17:22.106118+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T17:17:22.106118+0000","last_undegraded":"2024-09-13T17:17:22.106118+0000","last_fullsized":"2024-09-13T17:17:22.106118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22149,"num_write_kb":293545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"63'3283","reported_seq":3345,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.057875+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T17:17:14.057875+0000","last_peered":"2024-09-13T17:17:14.057875+0000","last_clean":"2024-09-13T17:17:14.057875+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T17:17:14.057875+0000","last_undegraded":"2024-09-13T17:17:14.057875+0000","last_fullsized":"2024-09-13T17:17:14.057875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":248866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"63'4697","reported_seq":4763,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.910034+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T17:17:21.910034+0000","last_peered":"2024-09-13T17:17:21.910034+0000","last_clean":"2024-09-13T17:17:21.910034+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T17:17:21.910034+0000","last_undegraded":"2024-09-13T17:17:21.910034+0000","last_fullsized":"2024-09-13T17:17:21.910034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25800,"num_write_kb":320606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"63'3820","reported_seq":3882,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.997809+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T17:17:13.997809+0000","last_peered":"2024-09-13T17:17:13.997809+0000","last_clean":"2024-09-13T17:17:13.997809+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T17:17:13.997809+0000","last_undegraded":"2024-09-13T17:17:13.997809+0000","last_fullsized":"2024-09-13T17:17:13.997809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3820,"log_dups_size":0,"ondisk_log_size":3820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20813,"num_write_kb":300503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"63'4719","reported_seq":4787,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117962+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T17:17:22.117962+0000","last_peered":"2024-09-13T17:17:22.117962+0000","last_clean":"2024-09-13T17:17:22.117962+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T17:17:22.117962+0000","last_undegraded":"2024-09-13T17:17:22.117962+0000","last_fullsized":"2024-09-13T17:17:22.117962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":25263,"num_write_kb":330973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"63'3722","reported_seq":3784,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.001243+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T17:17:14.001243+0000","last_peered":"2024-09-13T17:17:14.001243+0000","last_clean":"2024-09-13T17:17:14.001243+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T17:17:14.001243+0000","last_undegraded":"2024-09-13T17:17:14.001243+0000","last_fullsized":"2024-09-13T17:17:14.001243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20022,"num_write_kb":289122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"63'3813","reported_seq":3879,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.064518+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T17:17:22.064518+0000","last_peered":"2024-09-13T17:17:22.064518+0000","last_clean":"2024-09-13T17:17:22.064518+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T17:17:22.064518+0000","last_undegraded":"2024-09-13T17:17:22.064518+0000","last_fullsized":"2024-09-13T17:17:22.064518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21202,"num_write_kb":265282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"63'4479","reported_seq":4545,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.060514+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T17:17:14.060514+0000","last_peered":"2024-09-13T17:17:14.060514+0000","last_clean":"2024-09-13T17:17:14.060514+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T17:17:14.060514+0000","last_undegraded":"2024-09-13T17:17:14.060514+0000","last_fullsized":"2024-09-13T17:17:14.060514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":24556,"num_write_kb":340437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"63'4549","reported_seq":4615,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058661+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T17:17:22.058661+0000","last_peered":"2024-09-13T17:17:22.058661+0000","last_clean":"2024-09-13T17:17:22.058661+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T17:17:22.058661+0000","last_undegraded":"2024-09-13T17:17:22.058661+0000","last_fullsized":"2024-09-13T17:17:22.058661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24549,"num_write_kb":319591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"63'3479","reported_seq":3542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.019701+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T17:17:14.019701+0000","last_peered":"2024-09-13T17:17:14.019701+0000","last_clean":"2024-09-13T17:17:14.019701+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T17:17:14.019701+0000","last_undegraded":"2024-09-13T17:17:14.019701+0000","last_fullsized":"2024-09-13T17:17:14.019701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3479,"log_dups_size":0,"ondisk_log_size":3479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18966,"num_write_kb":268432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"63'3573","reported_seq":3639,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084131+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T17:17:22.084131+0000","last_peered":"2024-09-13T17:17:22.084131+0000","last_clean":"2024-09-13T17:17:22.084131+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T17:17:22.084131+0000","last_undegraded":"2024-09-13T17:17:22.084131+0000","last_fullsized":"2024-09-13T17:17:22.084131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19348,"num_write_kb":270113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"63'4058","reported_seq":4120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036359+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T17:17:14.036359+0000","last_peered":"2024-09-13T17:17:14.036359+0000","last_clean":"2024-09-13T17:17:14.036359+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T17:17:14.036359+0000","last_undegraded":"2024-09-13T17:17:14.036359+0000","last_fullsized":"2024-09-13T17:17:14.036359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22331,"num_write_kb":302585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"63'5839","reported_seq":5904,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117408+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T17:17:22.117408+0000","last_peered":"2024-09-13T17:17:22.117408+0000","last_clean":"2024-09-13T17:17:22.117408+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T17:17:22.117408+0000","last_undegraded":"2024-09-13T17:17:22.117408+0000","last_fullsized":"2024-09-13T17:17:22.117408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31756,"num_write_kb":397521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"63'3835","reported_seq":3897,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.035326+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T17:17:14.035326+0000","last_peered":"2024-09-13T17:17:14.035326+0000","last_clean":"2024-09-13T17:17:14.035326+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T17:17:14.035326+0000","last_undegraded":"2024-09-13T17:17:14.035326+0000","last_fullsized":"2024-09-13T17:17:14.035326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":21185,"num_write_kb":306173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"63'4134","reported_seq":4199,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.079721+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T17:17:22.079721+0000","last_peered":"2024-09-13T17:17:22.079721+0000","last_clean":"2024-09-13T17:17:22.079721+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T17:17:22.079721+0000","last_undegraded":"2024-09-13T17:17:22.079721+0000","last_fullsized":"2024-09-13T17:17:22.079721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22268,"num_write_kb":281801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"63'5073","reported_seq":5136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058407+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T17:17:14.058407+0000","last_peered":"2024-09-13T17:17:14.058407+0000","last_clean":"2024-09-13T17:17:14.058407+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T17:17:14.058407+0000","last_undegraded":"2024-09-13T17:17:14.058407+0000","last_fullsized":"2024-09-13T17:17:14.058407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":28693,"num_write_kb":393724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"63'3822","reported_seq":3887,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107201+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T17:17:22.107201+0000","last_peered":"2024-09-13T17:17:22.107201+0000","last_clean":"2024-09-13T17:17:22.107201+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T17:17:22.107201+0000","last_undegraded":"2024-09-13T17:17:22.107201+0000","last_fullsized":"2024-09-13T17:17:22.107201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20823,"num_write_kb":275830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"63'2778","reported_seq":2842,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.048760+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T17:17:14.048760+0000","last_peered":"2024-09-13T17:17:14.048760+0000","last_clean":"2024-09-13T17:17:14.048760+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T17:17:14.048760+0000","last_undegraded":"2024-09-13T17:17:14.048760+0000","last_fullsized":"2024-09-13T17:17:14.048760+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":34,"num_write":15212,"num_write_kb":208573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"63'3975","reported_seq":4040,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.131512+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T17:17:22.131512+0000","last_peered":"2024-09-13T17:17:22.131512+0000","last_clean":"2024-09-13T17:17:22.131512+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T17:17:22.131512+0000","last_undegraded":"2024-09-13T17:17:22.131512+0000","last_fullsized":"2024-09-13T17:17:22.131512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3975,"log_dups_size":0,"ondisk_log_size":3975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21351,"num_write_kb":278200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"63'3844","reported_seq":3907,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.966555+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T17:17:13.966555+0000","last_peered":"2024-09-13T17:17:13.966555+0000","last_clean":"2024-09-13T17:17:13.966555+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T17:17:13.966555+0000","last_undegraded":"2024-09-13T17:17:13.966555+0000","last_fullsized":"2024-09-13T17:17:13.966555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21085,"num_write_kb":289508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"63'3324","reported_seq":3389,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.087163+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T17:17:22.087163+0000","last_peered":"2024-09-13T17:17:22.087163+0000","last_clean":"2024-09-13T17:17:22.087163+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T17:17:22.087163+0000","last_undegraded":"2024-09-13T17:17:22.087163+0000","last_fullsized":"2024-09-13T17:17:22.087163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3324,"log_dups_size":0,"ondisk_log_size":3324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18810,"num_write_kb":235312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"63'3743","reported_seq":3805,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.055690+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T17:17:14.055690+0000","last_peered":"2024-09-13T17:17:14.055690+0000","last_clean":"2024-09-13T17:17:14.055690+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T17:17:14.055690+0000","last_undegraded":"2024-09-13T17:17:14.055690+0000","last_fullsized":"2024-09-13T17:17:14.055690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20790,"num_write_kb":280180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"63'3173","reported_seq":3238,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.083735+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T17:17:22.083735+0000","last_peered":"2024-09-13T17:17:22.083735+0000","last_clean":"2024-09-13T17:17:22.083735+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T17:17:22.083735+0000","last_undegraded":"2024-09-13T17:17:22.083735+0000","last_fullsized":"2024-09-13T17:17:22.083735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17044,"num_write_kb":219853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"63'3460","reported_seq":3522,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.008254+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T17:17:14.008254+0000","last_peered":"2024-09-13T17:17:14.008254+0000","last_clean":"2024-09-13T17:17:14.008254+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T17:17:14.008254+0000","last_undegraded":"2024-09-13T17:17:14.008254+0000","last_fullsized":"2024-09-13T17:17:14.008254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":19788,"num_write_kb":263976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"63'3747","reported_seq":3812,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.008211+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T17:17:22.008211+0000","last_peered":"2024-09-13T17:17:22.008211+0000","last_clean":"2024-09-13T17:17:22.008211+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T17:17:22.008211+0000","last_undegraded":"2024-09-13T17:17:22.008211+0000","last_fullsized":"2024-09-13T17:17:22.008211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20231,"num_write_kb":269532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"63'3422","reported_seq":3483,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.003363+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T17:17:14.003363+0000","last_peered":"2024-09-13T17:17:14.003363+0000","last_clean":"2024-09-13T17:17:14.003363+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T17:17:14.003363+0000","last_undegraded":"2024-09-13T17:17:14.003363+0000","last_fullsized":"2024-09-13T17:17:14.003363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18722,"num_write_kb":267617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"63'4803","reported_seq":4868,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084790+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T17:17:22.084790+0000","last_peered":"2024-09-13T17:17:22.084790+0000","last_clean":"2024-09-13T17:17:22.084790+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T17:17:22.084790+0000","last_undegraded":"2024-09-13T17:17:22.084790+0000","last_fullsized":"2024-09-13T17:17:22.084790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26967,"num_write_kb":333429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"63'3840","reported_seq":3901,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.021162+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T17:17:14.021162+0000","last_peered":"2024-09-13T17:17:14.021162+0000","last_clean":"2024-09-13T17:17:14.021162+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T17:17:14.021162+0000","last_undegraded":"2024-09-13T17:17:14.021162+0000","last_fullsized":"2024-09-13T17:17:14.021162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21113,"num_write_kb":293770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"63'5433","reported_seq":5500,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.040459+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T17:17:22.040459+0000","last_peered":"2024-09-13T17:17:22.040459+0000","last_clean":"2024-09-13T17:17:22.040459+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T17:17:22.040459+0000","last_undegraded":"2024-09-13T17:17:22.040459+0000","last_fullsized":"2024-09-13T17:17:22.040459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30664,"num_write_kb":389336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"63'4741","reported_seq":4806,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.053779+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T17:17:14.053779+0000","last_peered":"2024-09-13T17:17:14.053779+0000","last_clean":"2024-09-13T17:17:14.053779+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T17:17:14.053779+0000","last_undegraded":"2024-09-13T17:17:14.053779+0000","last_fullsized":"2024-09-13T17:17:14.053779+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":15,"num_write":25588,"num_write_kb":340391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"63'4281","reported_seq":4345,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.981625+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T17:17:21.981625+0000","last_peered":"2024-09-13T17:17:21.981625+0000","last_clean":"2024-09-13T17:17:21.981625+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T17:17:21.981625+0000","last_undegraded":"2024-09-13T17:17:21.981625+0000","last_fullsized":"2024-09-13T17:17:21.981625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23373,"num_write_kb":302635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"63'3575","reported_seq":3641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.054561+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T17:17:14.054561+0000","last_peered":"2024-09-13T17:17:14.054561+0000","last_clean":"2024-09-13T17:17:14.054561+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T17:17:14.054561+0000","last_undegraded":"2024-09-13T17:17:14.054561+0000","last_fullsized":"2024-09-13T17:17:14.054561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":7,"num_write":20117,"num_write_kb":289677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"63'4404","reported_seq":4468,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.129331+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T17:17:22.129331+0000","last_peered":"2024-09-13T17:17:22.129331+0000","last_clean":"2024-09-13T17:17:22.129331+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T17:17:22.129331+0000","last_undegraded":"2024-09-13T17:17:22.129331+0000","last_fullsized":"2024-09-13T17:17:22.129331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24308,"num_write_kb":306423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"63'3318","reported_seq":3382,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.949311+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T17:17:13.949311+0000","last_peered":"2024-09-13T17:17:13.949311+0000","last_clean":"2024-09-13T17:17:13.949311+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T17:17:13.949311+0000","last_undegraded":"2024-09-13T17:17:13.949311+0000","last_fullsized":"2024-09-13T17:17:13.949311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3318,"log_dups_size":0,"ondisk_log_size":3318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18295,"num_write_kb":253537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"63'4013","reported_seq":4077,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.861950+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T17:17:21.861950+0000","last_peered":"2024-09-13T17:17:21.861950+0000","last_clean":"2024-09-13T17:17:21.861950+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T17:17:21.861950+0000","last_undegraded":"2024-09-13T17:17:21.861950+0000","last_fullsized":"2024-09-13T17:17:21.861950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21393,"num_write_kb":262042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"63'3658","reported_seq":3722,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.048939+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T17:17:14.048939+0000","last_peered":"2024-09-13T17:17:14.048939+0000","last_clean":"2024-09-13T17:17:14.048939+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T17:17:14.048939+0000","last_undegraded":"2024-09-13T17:17:14.048939+0000","last_fullsized":"2024-09-13T17:17:14.048939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19946,"num_write_kb":268446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"63'3910","reported_seq":3974,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.108541+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T17:17:22.108541+0000","last_peered":"2024-09-13T17:17:22.108541+0000","last_clean":"2024-09-13T17:17:22.108541+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T17:17:22.108541+0000","last_undegraded":"2024-09-13T17:17:22.108541+0000","last_fullsized":"2024-09-13T17:17:22.108541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3910,"log_dups_size":0,"ondisk_log_size":3910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":278651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"63'3074","reported_seq":3134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.837771+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T17:17:13.837771+0000","last_peered":"2024-09-13T17:17:13.837771+0000","last_clean":"2024-09-13T17:17:13.837771+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T17:17:13.837771+0000","last_undegraded":"2024-09-13T17:17:13.837771+0000","last_fullsized":"2024-09-13T17:17:13.837771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":246226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"63'3515","reported_seq":3583,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.005699+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T17:17:22.005699+0000","last_peered":"2024-09-13T17:17:22.005699+0000","last_clean":"2024-09-13T17:17:22.005699+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T17:17:22.005699+0000","last_undegraded":"2024-09-13T17:17:22.005699+0000","last_fullsized":"2024-09-13T17:17:22.005699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19512,"num_write_kb":258445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"63'3439","reported_seq":3500,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.038223+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T17:17:14.038223+0000","last_peered":"2024-09-13T17:17:14.038223+0000","last_clean":"2024-09-13T17:17:14.038223+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T17:17:14.038223+0000","last_undegraded":"2024-09-13T17:17:14.038223+0000","last_fullsized":"2024-09-13T17:17:14.038223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":17681,"num_write_kb":263603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"63'3892","reported_seq":3960,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.067970+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T17:17:22.067970+0000","last_peered":"2024-09-13T17:17:22.067970+0000","last_clean":"2024-09-13T17:17:22.067970+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T17:17:22.067970+0000","last_undegraded":"2024-09-13T17:17:22.067970+0000","last_fullsized":"2024-09-13T17:17:22.067970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21304,"num_write_kb":286190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"63'3622","reported_seq":3682,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.941100+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T17:17:13.941100+0000","last_peered":"2024-09-13T17:17:13.941100+0000","last_clean":"2024-09-13T17:17:13.941100+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T17:17:13.941100+0000","last_undegraded":"2024-09-13T17:17:13.941100+0000","last_fullsized":"2024-09-13T17:17:13.941100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19729,"num_write_kb":275691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"63'5036","reported_seq":5103,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.072290+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T17:17:22.072290+0000","last_peered":"2024-09-13T17:17:22.072290+0000","last_clean":"2024-09-13T17:17:22.072290+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T17:17:22.072290+0000","last_undegraded":"2024-09-13T17:17:22.072290+0000","last_fullsized":"2024-09-13T17:17:22.072290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27469,"num_write_kb":368239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"63'3689","reported_seq":3749,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.059558+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T17:17:14.059558+0000","last_peered":"2024-09-13T17:17:14.059558+0000","last_clean":"2024-09-13T17:17:14.059558+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T17:17:14.059558+0000","last_undegraded":"2024-09-13T17:17:14.059558+0000","last_fullsized":"2024-09-13T17:17:14.059558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3689,"log_dups_size":0,"ondisk_log_size":3689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20818,"num_write_kb":284187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"63'4047","reported_seq":4115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126604+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T17:17:22.126604+0000","last_peered":"2024-09-13T17:17:22.126604+0000","last_clean":"2024-09-13T17:17:22.126604+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T17:17:22.126604+0000","last_undegraded":"2024-09-13T17:17:22.126604+0000","last_fullsized":"2024-09-13T17:17:22.126604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22743,"num_write_kb":291925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":197,"num_write":1329762,"num_write_kb":18444058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242925,"ondisk_log_size":242925,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19024977,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1583,"num_read_kb":206738,"num_write":4472,"num_write_kb":20527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38109184,"data_stored":38049954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4352,"ondisk_log_size":4352,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":1479870,"num_write_kb":19043750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270762,"ondisk_log_size":270762,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":19653473,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1296,"num_read_kb":190456,"num_write":4501,"num_write_kb":21128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39370752,"data_stored":39306946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4332,"ondisk_log_size":4332,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":942768,"kb_used_data":13672,"kb_used_omap":1,"kb_used_meta":929086,"kb_avail":93429072,"statfs":{"total":96636764160,"available":95671369728,"internally_reserved":0,"allocated":14000128,"data_stored":13606602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":951384523},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":1.9890000000000001,"15min":1.212},"min":{"1min":0.24299999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":8.0960000000000001,"5min":16.571999999999999,"15min":16.571999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":2.3239999999999998,"5min":2.1040000000000001,"15min":1.3080000000000001},"min":{"1min":0.26600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":9.7799999999999994,"5min":16.523,"15min":16.523},"last":0.95399999999999996}]},{"osd":1,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":5.859,"5min":2.754,"15min":1.486},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":59.911999999999999,"5min":59.911999999999999,"15min":59.911999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":5.8680000000000003,"5min":2.8519999999999999,"15min":1.5660000000000001},"min":{"1min":0.28699999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":59.895000000000003,"5min":59.895000000000003,"15min":59.895000000000003},"last":1.1080000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.113,"5min":1.6659999999999999,"15min":1.1040000000000001},"min":{"1min":0.28999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.6869999999999994,"5min":16.023,"15min":16.023},"last":0.90000000000000002},{"interface":"front","average":{"1min":2.173,"5min":1.659,"15min":1.1559999999999999},"min":{"1min":0.28299999999999997,"5min":0.249,"15min":0.249},"max":{"1min":9.5619999999999994,"5min":10.654999999999999,"15min":10.654999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.2200000000000002,"5min":1.944,"15min":1.2210000000000001},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":9.7370000000000001,"5min":23.225000000000001,"15min":23.225000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":2.2959999999999998,"5min":1.944,"15min":1.2789999999999999},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.891,"5min":12.045999999999999,"15min":12.045999999999999},"last":1.0169999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":2.0659999999999998,"15min":1.274},"min":{"1min":0.25900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":14.209,"5min":18.004999999999999,"15min":18.004999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":3.9510000000000001,"5min":2.339,"15min":1.391},"min":{"1min":0.28299999999999997,"5min":0.245,"15min":0.245},"max":{"1min":22.315999999999999,"5min":22.315999999999999,"15min":22.315999999999999},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":4.9809999999999999,"5min":2.2959999999999998,"15min":1.419},"min":{"1min":0.32200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":37.302999999999997,"5min":37.302999999999997,"15min":37.302999999999997},"last":0.86599999999999999},{"interface":"front","average":{"1min":2.6560000000000001,"5min":1.8100000000000001,"15min":1.27},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":16.867999999999999,"5min":16.867999999999999,"15min":16.867999999999999},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":1.788,"15min":1.292},"min":{"1min":0.38100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":9.1660000000000004,"5min":17.968,"15min":17.968},"last":0.81200000000000006},{"interface":"front","average":{"1min":3.29,"5min":1.9910000000000001,"15min":1.3500000000000001},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.025,"5min":19.244,"15min":19.244},"last":0.99399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575208,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673180,"kb_used_data":11732,"kb_used_omap":1,"kb_used_meta":661438,"kb_avail":93698660,"statfs":{"total":96636764160,"available":95947427840,"internally_reserved":0,"allocated":12013568,"data_stored":11612788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":677312968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.97999999999999998,"15min":0.86399999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":2.3839999999999999,"5min":6.335,"15min":6.335},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.878,"5min":1.111,"15min":0.89300000000000002},"min":{"1min":0.24199999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.5299999999999998,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2130000000000001,"15min":0.95899999999999996},"min":{"1min":0.28899999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":6.3230000000000004,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":0.82599999999999996},{"interface":"front","average":{"1min":2.0379999999999998,"5min":1.5169999999999999,"15min":1.034},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":9.3629999999999995,"5min":10.111000000000001,"15min":10.111000000000001},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.98199999999999998,"15min":0.90700000000000003},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":6.2160000000000002,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.1850000000000001,"15min":1.006},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.1130000000000004,"5min":6.6079999999999997,"15min":6.6079999999999997},"last":1.1419999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5800000000000001,"15min":1.1220000000000001},"min":{"1min":0.308,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":15.068,"5min":15.247999999999999,"15min":15.247999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.782,"5min":1.5249999999999999,"15min":1.127},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":15.179,"5min":15.179,"15min":15.179},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":1.222,"15min":1.042},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4790000000000001,"5min":15.326000000000001,"15min":15.326000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.454,"15min":1.1259999999999999},"min":{"1min":0.24399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":11.48,"5min":15.026,"15min":15.026},"last":1.175}]},{"osd":5,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.5089999999999999,"15min":1.2},"min":{"1min":0.38200000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":9.6080000000000005,"5min":11.566000000000001,"15min":11.566000000000001},"last":1.256},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.4139999999999999,"15min":1.1639999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":10.223000000000001,"5min":10.223000000000001,"15min":10.223000000000001},"last":1.4159999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.583,"15min":1.2250000000000001},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":9.8960000000000008,"5min":13.112,"15min":13.112},"last":1.3149999999999999},{"interface":"front","average":{"1min":2.0470000000000002,"5min":1.7470000000000001,"15min":1.28},"min":{"1min":0.33200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":14.4,"5min":14.4,"15min":14.4},"last":1.4490000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575209,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":957772,"kb_used_data":4996,"kb_used_omap":1,"kb_used_meta":952766,"kb_avail":93414068,"statfs":{"total":96636764160,"available":95656005632,"internally_reserved":0,"allocated":5115904,"data_stored":4720442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":975632838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":3.762,"5min":1.978,"15min":1.1919999999999999},"min":{"1min":0.246,"5min":0.215,"15min":0.215},"max":{"1min":27.952999999999999,"5min":27.952999999999999,"15min":27.952999999999999},"last":1.042},{"interface":"front","average":{"1min":2.5059999999999998,"5min":1.629,"15min":1.121},"min":{"1min":0.28100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":23.619,"5min":23.619,"15min":23.619},"last":1.0980000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.8210000000000002,"5min":1.8109999999999999,"15min":1.1379999999999999},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":13.515000000000001,"5min":13.515000000000001,"15min":13.515000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.621,"5min":1.9299999999999999,"15min":1.1839999999999999},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":6.9290000000000003,"5min":35.186999999999998,"15min":35.186999999999998},"last":0.84099999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":2.306,"15min":1.3420000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.609,"5min":51.18,"15min":51.18},"last":0.873},{"interface":"front","average":{"1min":1.732,"5min":1.929,"15min":1.2949999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.8659999999999997,"5min":48.756,"15min":48.756},"last":0.99199999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.7480000000000002,"5min":2.0219999999999998,"15min":1.2789999999999999},"min":{"1min":0.39200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":15.946999999999999,"5min":19.800000000000001,"15min":19.800000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.7649999999999999,"15min":1.256},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":9.1020000000000003,"5min":9.4789999999999992,"15min":9.4789999999999992},"last":0.80000000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":4.2270000000000003,"5min":2.4260000000000002,"15min":1.379},"min":{"1min":0.40699999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":20.713999999999999,"5min":22.361999999999998,"15min":22.361999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":4.1710000000000003,"5min":2.9940000000000002,"15min":1.641},"min":{"1min":0.35299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":28.062999999999999,"5min":31.843,"15min":31.843},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.427,"15min":1.131},"min":{"1min":0.375,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.3300000000000001,"5min":8.1910000000000007,"15min":8.1910000000000007},"last":1.1220000000000001},{"interface":"front","average":{"1min":2.052,"5min":1.7649999999999999,"15min":1.26},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.1059999999999999,"5min":13.031000000000001,"15min":13.031000000000001},"last":0.92700000000000005}]},{"osd":7,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":2.1890000000000001,"15min":1.4139999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":11.239000000000001,"5min":35.124000000000002,"15min":35.124000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":2.0550000000000002,"5min":2.5089999999999999,"15min":1.546},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.99,"5min":32.328000000000003,"15min":32.328000000000003},"last":0.89300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575209,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1011316,"kb_used_data":13932,"kb_used_omap":1,"kb_used_meta":997374,"kb_avail":93360524,"statfs":{"total":96636764160,"available":95601176576,"internally_reserved":0,"allocated":14266368,"data_stored":13858021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1021311430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.4910000000000001,"15min":1.0369999999999999},"min":{"1min":0.25,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":13.18,"5min":13.18,"15min":13.18},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.829,"5min":1.575,"15min":1.141},"min":{"1min":0.25700000000000001,"5min":0.251,"15min":0.251},"max":{"1min":13.163,"5min":13.163,"15min":13.163},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":3.1299999999999999,"5min":1.73,"15min":1.1060000000000001},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":27.242000000000001,"5min":27.242000000000001,"15min":27.242000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":2.9119999999999999,"5min":1.978,"15min":1.232},"min":{"1min":0.34399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":26.545999999999999,"5min":26.545999999999999,"15min":26.545999999999999},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.577,"15min":1.1060000000000001},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":7.2619999999999996,"5min":11.803000000000001,"15min":11.803000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.573,"15min":1.131},"min":{"1min":0.35299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3319999999999999,"5min":8.6709999999999994,"15min":8.6709999999999994},"last":0.83699999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":2.1899999999999999,"15min":1.343},"min":{"1min":0.32000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":10.583,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":2.875,"5min":1.9770000000000001,"15min":1.298},"min":{"1min":0.39300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":20.468,"5min":20.468,"15min":20.468},"last":0.94599999999999995}]},{"osd":5,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.4079999999999999,"15min":1.087},"min":{"1min":0.314,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.109,"5min":13.146000000000001,"15min":13.146000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.823,"5min":1.8080000000000001,"15min":1.264},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":14.202,"5min":15.044,"15min":15.044},"last":0.99199999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4359999999999999,"15min":1.1279999999999999},"min":{"1min":0.35299999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":7.2030000000000003,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":1.232},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":1.234},"min":{"1min":0.38400000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.7320000000000002,"5min":7.5860000000000003,"15min":7.5860000000000003},"last":1.069}]},{"osd":7,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.8169999999999999,"15min":1.345},"min":{"1min":0.33500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":6.3449999999999998,"5min":24.518999999999998,"15min":24.518999999999998},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.595,"5min":1.7170000000000001,"15min":1.306},"min":{"1min":0.372,"5min":0.312,"15min":0.312},"max":{"1min":9.0519999999999996,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":1.139}]}]},{"osd":3,"up_from":13,"seq":55834575209,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":958468,"kb_used_data":19388,"kb_used_omap":1,"kb_used_meta":939070,"kb_avail":93413372,"statfs":{"total":96636764160,"available":95655292928,"internally_reserved":0,"allocated":19853312,"data_stored":19449342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":961608136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":2.702,"5min":1.4350000000000001,"15min":0.91700000000000004},"min":{"1min":0.19800000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":15.791,"5min":15.791,"15min":15.791},"last":0.36499999999999999},{"interface":"front","average":{"1min":4.133,"5min":1.508,"15min":0.96899999999999997},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":20.699999999999999,"5min":20.699999999999999,"15min":20.699999999999999},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.8999999999999999,"5min":1.391,"15min":0.91100000000000003},"min":{"1min":0.28499999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":10.933,"5min":16.100000000000001,"15min":16.100000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.377,"15min":0.98299999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":8.2919999999999998,"5min":10.417,"15min":10.417},"last":0.432}]},{"osd":2,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.196,"15min":0.93000000000000005},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":5.1219999999999999,"5min":8.5950000000000006,"15min":8.5950000000000006},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2410000000000001,"15min":0.94599999999999995},"min":{"1min":0.24099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":10.737,"5min":15.433999999999999,"15min":15.433999999999999},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":2.2000000000000002,"5min":1.627,"15min":1.1799999999999999},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":10.835000000000001,"5min":10.835000000000001,"15min":10.835000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.9379999999999999,"5min":1.401,"15min":1.1240000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":7.0439999999999996,"5min":9.2759999999999998,"15min":9.2759999999999998},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":2.6880000000000002,"5min":1.841,"15min":1.3},"min":{"1min":0.42199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":31.187999999999999,"5min":31.187999999999999,"15min":31.187999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":2.508,"5min":1.802,"15min":1.29},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":17.466000000000001,"5min":17.466000000000001,"15min":17.466000000000001},"last":0.69999999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.764,"5min":1.768,"15min":1.286},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":10.785,"5min":12.567,"15min":12.567},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.998,"5min":2.1499999999999999,"15min":1.464},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.872,"5min":15.882,"15min":15.882},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.6579999999999999,"15min":1.2929999999999999},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":9.2720000000000002,"5min":11.94,"15min":11.94},"last":0.625},{"interface":"front","average":{"1min":3.3420000000000001,"5min":2.1520000000000001,"15min":1.4339999999999999},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":25.652999999999999,"5min":25.652999999999999,"15min":25.652999999999999},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575209,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662268,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":661374,"kb_avail":93709572,"statfs":{"total":96636764160,"available":95958601728,"internally_reserved":0,"allocated":905216,"data_stored":517265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":677247430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.89500000000000002,"15min":0.70699999999999996},"min":{"1min":0.189,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":3.3319999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.4830000000000001,"15min":0.96999999999999997},"min":{"1min":0.218,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":9.2729999999999997,"5min":9.2729999999999997,"15min":9.2729999999999997},"last":0.85399999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.3069999999999999,"5min":1.766,"15min":1.0680000000000001},"min":{"1min":0.29499999999999998,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":11.319000000000001,"5min":33.579999999999998,"15min":33.579999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":2.4359999999999999,"5min":1.7509999999999999,"15min":1.089},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":14.624000000000001,"5min":23.965,"15min":23.965},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.7949999999999999,"5min":2.1779999999999999,"15min":1.266},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.245},"max":{"1min":11.837999999999999,"5min":22.760000000000002,"15min":22.760000000000002},"last":0.998},{"interface":"front","average":{"1min":2.5449999999999999,"5min":1.6499999999999999,"15min":1.093},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":10.372,"5min":10.372,"15min":10.372},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.1520000000000001,"5min":2.012,"15min":1.292},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":10.324999999999999,"5min":20.673999999999999,"15min":20.673999999999999},"last":1.024},{"interface":"front","average":{"1min":2.0289999999999999,"5min":1.7669999999999999,"15min":1.24},"min":{"1min":0.42199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":6.4059999999999997,"5min":16.5,"15min":16.5},"last":0.97199999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":4.601,"5min":2.5329999999999999,"15min":1.5149999999999999},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":21.722000000000001,"5min":21.722000000000001,"15min":21.722000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":3.2240000000000002,"5min":2.238,"15min":1.415},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":19.242999999999999,"5min":29.620999999999999,"15min":29.620999999999999},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.6739999999999999,"15min":1.254},"min":{"1min":0.44,"5min":0.38,"15min":0.38},"max":{"1min":5.3239999999999998,"5min":21.882999999999999,"15min":21.882999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":2.2669999999999999,"5min":2.1070000000000002,"15min":1.429},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.29,"5min":25.274999999999999,"15min":25.274999999999999},"last":0.94899999999999995}]},{"osd":7,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":5.9109999999999996,"5min":3.2639999999999998,"15min":1.825},"min":{"1min":0.45700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":28.094999999999999,"5min":43.649000000000001,"15min":43.649000000000001},"last":1.147},{"interface":"front","average":{"1min":6.6379999999999999,"5min":3.1709999999999998,"15min":1.8460000000000001},"min":{"1min":0.44600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":27.417999999999999,"5min":27.417999999999999,"15min":27.417999999999999},"last":1.1140000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575209,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798968,"kb_used_data":9008,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572872,"statfs":{"total":96636764160,"available":95818620928,"internally_reserved":0,"allocated":9224192,"data_stored":8822112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":808909254},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.8919999999999999,"5min":1.8089999999999999,"15min":1.04},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":9.3499999999999996,"5min":15.81,"15min":15.81},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.7050000000000001,"15min":1.056},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.8360000000000003,"5min":15.823,"15min":15.823},"last":0.627}]},{"osd":2,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":1.4139999999999999,"15min":0.91500000000000004},"min":{"1min":0.23699999999999999,"5min":0.193,"15min":0.193},"max":{"1min":9.4420000000000002,"5min":11.672000000000001,"15min":11.672000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.1919999999999999,"15min":0.90900000000000003},"min":{"1min":0.26000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.3440000000000003,"5min":11.722,"15min":11.722},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3719999999999999,"5min":1.901,"15min":1.171},"min":{"1min":0.26300000000000001,"5min":0.22,"15min":0.22},"max":{"1min":10.151999999999999,"5min":16.507999999999999,"15min":16.507999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.4790000000000001,"15min":1.0029999999999999},"min":{"1min":0.25600000000000001,"5min":0.218,"15min":0.218},"max":{"1min":5.024,"5min":12.047000000000001,"15min":12.047000000000001},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.6160000000000001,"5min":2.1040000000000001,"15min":1.2649999999999999},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":10.176,"5min":22.234000000000002,"15min":22.234000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":2.4510000000000001,"5min":2.2080000000000002,"15min":1.381},"min":{"1min":0.377,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":10.77,"5min":20.448,"15min":20.448},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.649,"15min":1.1910000000000001},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.7839999999999998,"5min":15.788,"15min":15.788},"last":0.58999999999999997},{"interface":"front","average":{"1min":2.105,"5min":1.8180000000000001,"15min":1.256},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":9.3000000000000007,"5min":11.731999999999999,"15min":11.731999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.4119999999999999,"15min":1.141},"min":{"1min":0.42499999999999999,"5min":0.255,"15min":0.255},"max":{"1min":9.1760000000000002,"5min":12.019,"15min":12.019},"last":0.72199999999999998},{"interface":"front","average":{"1min":2.1760000000000002,"5min":1.768,"15min":1.258},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":9.3339999999999996,"5min":11.628,"15min":11.628},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3809999999999998,"5min":1.9279999999999999,"15min":1.371},"min":{"1min":0.435,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":8.9689999999999994,"5min":21.161000000000001,"15min":21.161000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":4.2720000000000002,"5min":2.504,"15min":1.552},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":29.323,"5min":29.323,"15min":29.323},"last":0.78900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575209,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799540,"kb_used_data":9580,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572300,"statfs":{"total":96636764160,"available":95818035200,"internally_reserved":0,"allocated":9809920,"data_stored":9412264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":808909256},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":1.474,"15min":0.92700000000000005},"min":{"1min":0.23300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":2.5870000000000002,"5min":59.031999999999996,"15min":59.031999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":2.0249999999999999,"5min":2.222,"15min":1.196},"min":{"1min":0.20200000000000001,"5min":0.188,"15min":0.188},"max":{"1min":9.7110000000000003,"5min":57.186999999999998,"15min":57.186999999999998},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":0.93799999999999994,"15min":0.81200000000000006},"min":{"1min":0.28000000000000003,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":4.0919999999999996,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":2.0960000000000001,"5min":1.1899999999999999,"15min":0.84699999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":12.052,"5min":12.052,"15min":12.052},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.901,"5min":2.5739999999999998,"15min":1.353},"min":{"1min":0.26100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.6769999999999996,"5min":65.215000000000003,"15min":65.215000000000003},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.03,"5min":1.679,"15min":1.133},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.319,"5min":62.722999999999999,"15min":62.722999999999999},"last":0.84599999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":3.23,"5min":1.857,"15min":1.202},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":14.694000000000001,"5min":14.694000000000001,"15min":14.694000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.994,"5min":1.3360000000000001,"15min":1.095},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":9.3499999999999996,"5min":10.036,"15min":10.036},"last":0.92200000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":2.8300000000000001,"5min":1.748,"15min":1.1919999999999999},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":21.940999999999999,"5min":21.940999999999999,"15min":21.940999999999999},"last":1.038},{"interface":"front","average":{"1min":2.1440000000000001,"5min":1.962,"15min":1.3029999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.0620000000000003,"5min":9.3420000000000005,"15min":9.3420000000000005},"last":0.89900000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.7410000000000001,"5min":1.242,"15min":1.119},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":9.3840000000000003,"5min":9.3840000000000003,"15min":9.3840000000000003},"last":1.0229999999999999},{"interface":"front","average":{"1min":2.012,"5min":1.458,"15min":1.161},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.7060000000000004,"5min":8.157,"15min":8.157},"last":1.3560000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6140000000000001,"15min":1.2589999999999999},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":5.3209999999999997,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":1.2749999999999999},{"interface":"front","average":{"1min":2.601,"5min":2.677,"15min":1.5920000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40300000000000002},"max":{"1min":8.7210000000000001,"5min":32.069000000000003,"15min":32.069000000000003},"last":1.1910000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":84545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8482816,"data_stored":8473825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7000064,"data_stored":6984672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6995968,"data_stored":6985218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10542867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4296704,"data_stored":4287702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:17:43.653 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:17:43.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.809+0000 7f3192af1640 1 Processor -- start 2024-09-13T17:17:43.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.809+0000 7f3192af1640 1 -- start start 2024-09-13T17:17:43.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f3192af1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f318c06d370 0x7f318c157f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f3192af1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318c157370 con 0x7f318c0c1790 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f3192af1640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318c153df0 con 0x7f318c0c20d0 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f3192af1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318c0c04a0 con 0x7f318c06d370 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f318bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f318c0c1790 0x7f318c15c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40188/0 (socket says 172.21.15.39:40188) 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f319108e640 1 --2- 172.21.15.39:0/951795440 >> v2:172.21.15.39:3300/0 conn(0x7f318c06d370 0x7f318c157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f318bfff640 1 -- 172.21.15.39:0/951795440 learned_addr learned my addr 172.21.15.39:0/951795440 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f318b7fe640 1 -- 172.21.15.39:0/951795440 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974754296 0 0) 0x7f318c157370 con 0x7f318c0c1790 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f318b7fe640 1 -- 172.21.15.39:0/951795440 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3178003530 con 0x7f318c0c1790 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f319108e640 1 -- 172.21.15.39:0/951795440 >> v1:172.21.15.84:6789/0 conn(0x7f318c0c20d0 legacy=0x7f318c157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f319108e640 1 -- 172.21.15.39:0/951795440 >> v1:172.21.15.39:6789/0 conn(0x7f318c0c1790 legacy=0x7f318c15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f319108e640 1 -- 172.21.15.39:0/951795440 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c06c700 con 0x7f318c06d370 2024-09-13T17:17:43.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f319108e640 1 --2- 172.21.15.39:0/951795440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c06d370 0x7f318c157f10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f3184009d90 tx=0x7f318401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a7ec95f89bd363 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:43.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f318b7fe640 1 -- 172.21.15.39:0/951795440 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31840a5a20 con 0x7f318c06d370 2024-09-13T17:17:43.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f318b7fe640 1 -- 172.21.15.39:0/951795440 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31840a5b80 con 0x7f318c06d370 2024-09-13T17:17:43.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.813+0000 7f318b7fe640 1 -- 172.21.15.39:0/951795440 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31840ad440 con 0x7f318c06d370 2024-09-13T17:17:43.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- 172.21.15.39:0/951795440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c06d370 msgr2=0x7f318c157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 --2- 172.21.15.39:0/951795440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c06d370 0x7f318c157f10 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f3184009d90 tx=0x7f318401c940 comp rx=0 tx=0).stop 2024-09-13T17:17:43.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- 172.21.15.39:0/951795440 shutdown_connections 2024-09-13T17:17:43.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 --2- 172.21.15.39:0/951795440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c06d370 0x7f318c157f10 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- 172.21.15.39:0/951795440 >> 172.21.15.39:0/951795440 conn(0x7f318c069e10 msgr2=0x7f318c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:43.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- 172.21.15.39:0/951795440 shutdown_connections 2024-09-13T17:17:43.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- 172.21.15.39:0/951795440 wait complete. 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 Processor -- start 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- start start 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f318c06d370 0x7f318c144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c0c1790 0x7f318c145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f318c0c20d0 0x7f318c14b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c0abc00 con 0x7f318c0c1790 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c151010 con 0x7f318c0c20d0 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1512f0 con 0x7f318c06d370 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f318bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c0c1790 0x7f318c145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f318bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c0c1790 0x7f318c145cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40580/0 (socket says 172.21.15.39:40580) 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f318bfff640 1 -- 172.21.15.39:0/1766579516 learned_addr learned my addr 172.21.15.39:0/1766579516 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f319108e640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f318c0c20d0 0x7f318c14b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f319088d640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f318c06d370 0x7f318c144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f318bfff640 1 -- 172.21.15.39:0/1766579516 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f318c06d370 msgr2=0x7f318c144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f318bfff640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f318c06d370 0x7f318c144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f318bfff640 1 -- 172.21.15.39:0/1766579516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f318c0c20d0 msgr2=0x7f318c14b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f318bfff640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f318c0c20d0 0x7f318c14b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f318bfff640 1 -- 172.21.15.39:0/1766579516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184009a70 con 0x7f318c0c1790 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f319088d640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f318c06d370 0x7f318c144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f318bfff640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c0c1790 0x7f318c145cc0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f317400c420 tx=0x7f317400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f31897fa640 1 -- 172.21.15.39:0/1766579516 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3174005cf0 con 0x7f318c0c1790 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f31897fa640 1 -- 172.21.15.39:0/1766579516 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3174005e50 con 0x7f318c0c1790 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c145210 con 0x7f318c0c1790 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f31897fa640 1 -- 172.21.15.39:0/1766579516 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31740072d0 con 0x7f318c0c1790 2024-09-13T17:17:43.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c14a420 con 0x7f318c0c1790 2024-09-13T17:17:43.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.817+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c157800 con 0x7f318c0c1790 2024-09-13T17:17:43.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.821+0000 7f31897fa640 1 -- 172.21.15.39:0/1766579516 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3174007430 con 0x7f318c0c1790 2024-09-13T17:17:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.821+0000 7f31897fa640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f316c066690 0x7f316c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.821+0000 7f31897fa640 1 -- 172.21.15.39:0/1766579516 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f3174084680 con 0x7f318c0c1790 2024-09-13T17:17:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.821+0000 7f319088d640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f316c066690 0x7f316c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.821+0000 7f319088d640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f316c066690 0x7f316c068b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f318c153d60 tx=0x7f317c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:43.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.825+0000 7f31897fa640 1 -- 172.21.15.39:0/1766579516 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3174050ac0 con 0x7f318c0c1790 2024-09-13T17:17:43.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.985+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f318c144af0 con 0x7f316c066690 2024-09-13T17:17:44.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:43.997+0000 7f31897fa640 1 -- 172.21.15.39:0/1766579516 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528671 (secure 0 0 0) 0x7f318c144af0 con 0x7f316c066690 2024-09-13T17:17:44.003 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T17:17:44.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f316c066690 msgr2=0x7f316c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f316c066690 0x7f316c068b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f318c153d60 tx=0x7f317c009040 comp rx=0 tx=0).stop 2024-09-13T17:17:44.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c0c1790 msgr2=0x7f318c145cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c0c1790 0x7f318c145cc0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f317400c420 tx=0x7f317400c8e0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 shutdown_connections 2024-09-13T17:17:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f316c066690 0x7f316c068b50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f318c0c20d0 0x7f318c14b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f318c0c1790 0x7f318c145cc0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 --2- 172.21.15.39:0/1766579516 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f318c06d370 0x7f318c144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 >> 172.21.15.39:0/1766579516 conn(0x7f318c069e10 msgr2=0x7f318c155590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 shutdown_connections 2024-09-13T17:17:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.005+0000 7f3192af1640 1 -- 172.21.15.39:0/1766579516 wait complete. 2024-09-13T17:17:44.025 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":953,"stamp":"2024-09-13T17:17:42.345888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39268818,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":3083,"num_read_kb":397491,"num_write":2818764,"num_write_kb":37531879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522488,"ondisk_log_size":522488,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6804280,"kb_used_data":83192,"kb_used_omap":12,"kb_used_meta":6721011,"kb_avail":748170440,"statfs":{"total":773094113280,"available":766126530560,"internally_reserved":0,"allocated":85188608,"data_stored":81998836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":6882315835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":429,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004893"},"pg_stats":[{"pgid":"5.18","version":"63'4385","reported_seq":4447,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058432+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T17:17:14.058432+0000","last_peered":"2024-09-13T17:17:14.058432+0000","last_clean":"2024-09-13T17:17:14.058432+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T17:17:14.058432+0000","last_undegraded":"2024-09-13T17:17:14.058432+0000","last_fullsized":"2024-09-13T17:17:14.058432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":23692,"num_write_kb":324330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"63'916","reported_seq":1025,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096259+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T17:17:22.096259+0000","last_peered":"2024-09-13T17:17:22.096259+0000","last_clean":"2024-09-13T17:17:22.096259+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T17:17:22.096259+0000","last_undegraded":"2024-09-13T17:17:22.096259+0000","last_fullsized":"2024-09-13T17:17:22.096259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":6289,"num_write":894,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'3737","reported_seq":3804,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.100845+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T17:17:22.100845+0000","last_peered":"2024-09-13T17:17:22.100845+0000","last_clean":"2024-09-13T17:17:22.100845+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T17:17:22.100845+0000","last_undegraded":"2024-09-13T17:17:22.100845+0000","last_fullsized":"2024-09-13T17:17:22.100845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3737,"log_dups_size":0,"ondisk_log_size":3737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20454,"num_write_kb":270405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.021643+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T17:17:14.021643+0000","last_peered":"2024-09-13T17:17:14.021643+0000","last_clean":"2024-09-13T17:17:14.021643+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T17:17:14.021643+0000","last_undegraded":"2024-09-13T17:17:14.021643+0000","last_fullsized":"2024-09-13T17:17:14.021643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"63'4331","reported_seq":4394,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.043747+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T17:17:14.043747+0000","last_peered":"2024-09-13T17:17:14.043747+0000","last_clean":"2024-09-13T17:17:14.043747+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T17:17:14.043747+0000","last_undegraded":"2024-09-13T17:17:14.043747+0000","last_fullsized":"2024-09-13T17:17:14.043747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":22875,"num_write_kb":325320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"63'23","reported_seq":91,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096300+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T17:17:22.096300+0000","last_peered":"2024-09-13T17:17:22.096300+0000","last_clean":"2024-09-13T17:17:22.096300+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T17:17:22.096300+0000","last_undegraded":"2024-09-13T17:17:22.096300+0000","last_fullsized":"2024-09-13T17:17:22.096300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"63'5559","reported_seq":5626,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.132606+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T17:17:22.132606+0000","last_peered":"2024-09-13T17:17:22.132606+0000","last_clean":"2024-09-13T17:17:22.132606+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T17:17:22.132606+0000","last_undegraded":"2024-09-13T17:17:22.132606+0000","last_fullsized":"2024-09-13T17:17:22.132606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5559,"log_dups_size":0,"ondisk_log_size":5559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30370,"num_write_kb":379865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"63'2","reported_seq":64,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:09.573144+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T17:17:09.573144+0000","last_peered":"2024-09-13T17:17:09.573144+0000","last_clean":"2024-09-13T17:17:09.573144+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T17:17:09.573144+0000","last_undegraded":"2024-09-13T17:17:09.573144+0000","last_fullsized":"2024-09-13T17:17:09.573144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"63'3581","reported_seq":3641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.986895+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T17:17:13.986895+0000","last_peered":"2024-09-13T17:17:13.986895+0000","last_clean":"2024-09-13T17:17:13.986895+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T17:17:13.986895+0000","last_undegraded":"2024-09-13T17:17:13.986895+0000","last_fullsized":"2024-09-13T17:17:13.986895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":268344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"63'40","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.099149+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T17:17:22.099149+0000","last_peered":"2024-09-13T17:17:22.099149+0000","last_clean":"2024-09-13T17:17:22.099149+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T17:17:22.099149+0000","last_undegraded":"2024-09-13T17:17:22.099149+0000","last_fullsized":"2024-09-13T17:17:22.099149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'3493","reported_seq":3559,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.063521+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T17:17:22.063521+0000","last_peered":"2024-09-13T17:17:22.063521+0000","last_clean":"2024-09-13T17:17:22.063521+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T17:17:22.063521+0000","last_undegraded":"2024-09-13T17:17:22.063521+0000","last_fullsized":"2024-09-13T17:17:22.063521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18921,"num_write_kb":256078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'726","reported_seq":812,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.996155+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T17:17:13.996155+0000","last_peered":"2024-09-13T17:17:13.996155+0000","last_clean":"2024-09-13T17:17:13.996155+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T17:17:13.996155+0000","last_undegraded":"2024-09-13T17:17:13.996155+0000","last_fullsized":"2024-09-13T17:17:13.996155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":75389,"num_write":714,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"63'3135","reported_seq":3195,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.003272+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T17:17:14.003272+0000","last_peered":"2024-09-13T17:17:14.003272+0000","last_clean":"2024-09-13T17:17:14.003272+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T17:17:14.003272+0000","last_undegraded":"2024-09-13T17:17:14.003272+0000","last_fullsized":"2024-09-13T17:17:14.003272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3135,"log_dups_size":0,"ondisk_log_size":3135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":232447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"63'5","reported_seq":106,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:16:38.283375+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T17:16:38.283375+0000","last_peered":"2024-09-13T17:16:38.283375+0000","last_clean":"2024-09-13T17:16:38.283375+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T17:16:38.283375+0000","last_undegraded":"2024-09-13T17:16:38.283375+0000","last_fullsized":"2024-09-13T17:16:38.283375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"63'5195","reported_seq":5259,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095099+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T17:17:22.095099+0000","last_peered":"2024-09-13T17:17:22.095099+0000","last_clean":"2024-09-13T17:17:22.095099+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T17:17:22.095099+0000","last_undegraded":"2024-09-13T17:17:22.095099+0000","last_fullsized":"2024-09-13T17:17:22.095099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29369,"num_write_kb":359470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"63'210","reported_seq":271,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:27.922744+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T17:17:27.922744+0000","last_peered":"2024-09-13T17:17:27.922744+0000","last_clean":"2024-09-13T17:17:27.922744+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T17:17:27.922744+0000","last_undegraded":"2024-09-13T17:17:27.922744+0000","last_fullsized":"2024-09-13T17:17:27.922744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153565,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"63'3570","reported_seq":3634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.050021+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T17:17:14.050021+0000","last_peered":"2024-09-13T17:17:14.050021+0000","last_clean":"2024-09-13T17:17:14.050021+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T17:17:14.050021+0000","last_undegraded":"2024-09-13T17:17:14.050021+0000","last_fullsized":"2024-09-13T17:17:14.050021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":19362,"num_write_kb":282506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"63'4274","reported_seq":4340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096597+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T17:17:22.096597+0000","last_peered":"2024-09-13T17:17:22.096597+0000","last_clean":"2024-09-13T17:17:22.096597+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T17:17:22.096597+0000","last_undegraded":"2024-09-13T17:17:22.096597+0000","last_fullsized":"2024-09-13T17:17:22.096597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23751,"num_write_kb":312338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"63'1418","reported_seq":1528,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.108614+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T17:17:22.108614+0000","last_peered":"2024-09-13T17:17:22.108614+0000","last_clean":"2024-09-13T17:17:22.108614+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T17:17:22.108614+0000","last_undegraded":"2024-09-13T17:17:22.108614+0000","last_fullsized":"2024-09-13T17:17:22.108614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1418,"log_dups_size":0,"ondisk_log_size":1418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":38147,"num_write":1396,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"63'6","reported_seq":90,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.030550+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T17:17:14.030550+0000","last_peered":"2024-09-13T17:17:14.030550+0000","last_clean":"2024-09-13T17:17:14.030550+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T17:17:14.030550+0000","last_undegraded":"2024-09-13T17:17:14.030550+0000","last_fullsized":"2024-09-13T17:17:14.030550+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"63'4212","reported_seq":4277,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.051215+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T17:17:14.051215+0000","last_peered":"2024-09-13T17:17:14.051215+0000","last_clean":"2024-09-13T17:17:14.051215+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T17:17:14.051215+0000","last_undegraded":"2024-09-13T17:17:14.051215+0000","last_fullsized":"2024-09-13T17:17:14.051215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":17,"num_write":23343,"num_write_kb":314959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"63'3296","reported_seq":3362,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.125915+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T17:17:22.125915+0000","last_peered":"2024-09-13T17:17:22.125915+0000","last_clean":"2024-09-13T17:17:22.125915+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T17:17:22.125915+0000","last_undegraded":"2024-09-13T17:17:22.125915+0000","last_fullsized":"2024-09-13T17:17:22.125915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3296,"log_dups_size":0,"ondisk_log_size":3296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17758,"num_write_kb":230990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'3","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.059668+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T17:17:22.059668+0000","last_peered":"2024-09-13T17:17:22.059668+0000","last_clean":"2024-09-13T17:17:22.059668+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T17:17:22.059668+0000","last_undegraded":"2024-09-13T17:17:22.059668+0000","last_fullsized":"2024-09-13T17:17:22.059668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'5","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.027550+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T17:17:14.027550+0000","last_peered":"2024-09-13T17:17:14.027550+0000","last_clean":"2024-09-13T17:17:14.027550+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T17:17:14.027550+0000","last_undegraded":"2024-09-13T17:17:14.027550+0000","last_fullsized":"2024-09-13T17:17:14.027550+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"63'3371","reported_seq":3434,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022230+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T17:17:14.022230+0000","last_peered":"2024-09-13T17:17:14.022230+0000","last_clean":"2024-09-13T17:17:14.022230+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T17:17:14.022230+0000","last_undegraded":"2024-09-13T17:17:14.022230+0000","last_fullsized":"2024-09-13T17:17:14.022230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3371,"log_dups_size":0,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18156,"num_write_kb":249012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"63'3752","reported_seq":3818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.984671+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T17:17:21.984671+0000","last_peered":"2024-09-13T17:17:21.984671+0000","last_clean":"2024-09-13T17:17:21.984671+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T17:17:21.984671+0000","last_undegraded":"2024-09-13T17:17:21.984671+0000","last_fullsized":"2024-09-13T17:17:21.984671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21127,"num_write_kb":268998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"63'2","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.056813+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T17:17:22.056813+0000","last_peered":"2024-09-13T17:17:22.056813+0000","last_clean":"2024-09-13T17:17:22.056813+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T17:17:22.056813+0000","last_undegraded":"2024-09-13T17:17:22.056813+0000","last_fullsized":"2024-09-13T17:17:22.056813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"63'4","reported_seq":88,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.029328+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T17:17:14.029328+0000","last_peered":"2024-09-13T17:17:14.029328+0000","last_clean":"2024-09-13T17:17:14.029328+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T17:17:14.029328+0000","last_undegraded":"2024-09-13T17:17:14.029328+0000","last_fullsized":"2024-09-13T17:17:14.029328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"63'4499","reported_seq":4565,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.023958+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T17:17:14.023958+0000","last_peered":"2024-09-13T17:17:14.023958+0000","last_clean":"2024-09-13T17:17:14.023958+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T17:17:14.023958+0000","last_undegraded":"2024-09-13T17:17:14.023958+0000","last_fullsized":"2024-09-13T17:17:14.023958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":24341,"num_write_kb":343418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"63'4043","reported_seq":4109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.908339+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T17:17:21.908339+0000","last_peered":"2024-09-13T17:17:21.908339+0000","last_clean":"2024-09-13T17:17:21.908339+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T17:17:21.908339+0000","last_undegraded":"2024-09-13T17:17:21.908339+0000","last_fullsized":"2024-09-13T17:17:21.908339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22005,"num_write_kb":288910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"63'5","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.010552+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T17:17:22.010552+0000","last_peered":"2024-09-13T17:17:22.010552+0000","last_clean":"2024-09-13T17:17:22.010552+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T17:17:22.010552+0000","last_undegraded":"2024-09-13T17:17:22.010552+0000","last_fullsized":"2024-09-13T17:17:22.010552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"63'7","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024599+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T17:17:14.024599+0000","last_peered":"2024-09-13T17:17:14.024599+0000","last_clean":"2024-09-13T17:17:14.024599+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T17:17:14.024599+0000","last_undegraded":"2024-09-13T17:17:14.024599+0000","last_fullsized":"2024-09-13T17:17:14.024599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"63'3470","reported_seq":3533,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.032577+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T17:17:14.032577+0000","last_peered":"2024-09-13T17:17:14.032577+0000","last_clean":"2024-09-13T17:17:14.032577+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T17:17:14.032577+0000","last_undegraded":"2024-09-13T17:17:14.032577+0000","last_fullsized":"2024-09-13T17:17:14.032577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19441,"num_write_kb":258288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'4844","reported_seq":4911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.060853+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T17:17:22.060853+0000","last_peered":"2024-09-13T17:17:22.060853+0000","last_clean":"2024-09-13T17:17:22.060853+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T17:17:22.060853+0000","last_undegraded":"2024-09-13T17:17:22.060853+0000","last_fullsized":"2024-09-13T17:17:22.060853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":26635,"num_write_kb":336207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"63'9","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.115612+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T17:17:22.115612+0000","last_peered":"2024-09-13T17:17:22.115612+0000","last_clean":"2024-09-13T17:17:22.115612+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T17:17:22.115612+0000","last_undegraded":"2024-09-13T17:17:22.115612+0000","last_fullsized":"2024-09-13T17:17:22.115612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"63'48","reported_seq":175,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:24.143768+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T17:17:24.143768+0000","last_peered":"2024-09-13T17:17:24.143768+0000","last_clean":"2024-09-13T17:17:24.143768+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T17:17:24.143768+0000","last_undegraded":"2024-09-13T17:17:24.143768+0000","last_fullsized":"2024-09-13T17:17:24.143768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92726,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":118,"num_write":43,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"63'3635","reported_seq":3699,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.001656+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T17:17:14.001656+0000","last_peered":"2024-09-13T17:17:14.001656+0000","last_clean":"2024-09-13T17:17:14.001656+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T17:17:14.001656+0000","last_undegraded":"2024-09-13T17:17:14.001656+0000","last_fullsized":"2024-09-13T17:17:14.001656+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19787,"num_write_kb":259620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'3639","reported_seq":3705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.121330+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T17:17:22.121330+0000","last_peered":"2024-09-13T17:17:22.121330+0000","last_clean":"2024-09-13T17:17:22.121330+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T17:17:22.121330+0000","last_undegraded":"2024-09-13T17:17:22.121330+0000","last_fullsized":"2024-09-13T17:17:22.121330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3639,"log_dups_size":0,"ondisk_log_size":3639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20066,"num_write_kb":240267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"63'265","reported_seq":350,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.104288+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T17:17:22.104288+0000","last_peered":"2024-09-13T17:17:22.104288+0000","last_clean":"2024-09-13T17:17:22.104288+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T17:17:22.104288+0000","last_undegraded":"2024-09-13T17:17:22.104288+0000","last_fullsized":"2024-09-13T17:17:22.104288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":46,"num_write":634,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"63'11","reported_seq":75,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.026249+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T17:17:14.026249+0000","last_peered":"2024-09-13T17:17:14.026249+0000","last_clean":"2024-09-13T17:17:14.026249+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T17:17:14.026249+0000","last_undegraded":"2024-09-13T17:17:14.026249+0000","last_fullsized":"2024-09-13T17:17:14.026249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"63'3190","reported_seq":3253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.040510+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T17:17:14.040510+0000","last_peered":"2024-09-13T17:17:14.040510+0000","last_clean":"2024-09-13T17:17:14.040510+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T17:17:14.040510+0000","last_undegraded":"2024-09-13T17:17:14.040510+0000","last_fullsized":"2024-09-13T17:17:14.040510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3190,"log_dups_size":0,"ondisk_log_size":3190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":254866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"63'4701","reported_seq":4768,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.071944+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T17:17:22.071944+0000","last_peered":"2024-09-13T17:17:22.071944+0000","last_clean":"2024-09-13T17:17:22.071944+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T17:17:22.071944+0000","last_undegraded":"2024-09-13T17:17:22.071944+0000","last_fullsized":"2024-09-13T17:17:22.071944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26171,"num_write_kb":338588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'2","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:19.802813+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T17:17:19.802813+0000","last_peered":"2024-09-13T17:17:19.802813+0000","last_clean":"2024-09-13T17:17:19.802813+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T17:17:19.802813+0000","last_undegraded":"2024-09-13T17:17:19.802813+0000","last_fullsized":"2024-09-13T17:17:19.802813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"63'3","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.988743+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T17:17:13.988743+0000","last_peered":"2024-09-13T17:17:13.988743+0000","last_clean":"2024-09-13T17:17:13.988743+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T17:17:13.988743+0000","last_undegraded":"2024-09-13T17:17:13.988743+0000","last_fullsized":"2024-09-13T17:17:13.988743+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"63'3859","reported_seq":3922,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.945566+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T17:17:13.945566+0000","last_peered":"2024-09-13T17:17:13.945566+0000","last_clean":"2024-09-13T17:17:13.945566+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T17:17:13.945566+0000","last_undegraded":"2024-09-13T17:17:13.945566+0000","last_fullsized":"2024-09-13T17:17:13.945566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21286,"num_write_kb":290222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"63'3876","reported_seq":3943,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.089484+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T17:17:22.089484+0000","last_peered":"2024-09-13T17:17:22.089484+0000","last_clean":"2024-09-13T17:17:22.089484+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T17:17:22.089484+0000","last_undegraded":"2024-09-13T17:17:22.089484+0000","last_fullsized":"2024-09-13T17:17:22.089484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21201,"num_write_kb":273354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"63'2","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.094910+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T17:17:22.094910+0000","last_peered":"2024-09-13T17:17:22.094910+0000","last_clean":"2024-09-13T17:17:22.094910+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T17:17:22.094910+0000","last_undegraded":"2024-09-13T17:17:22.094910+0000","last_fullsized":"2024-09-13T17:17:22.094910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'5","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.944152+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T17:17:12.944152+0000","last_peered":"2024-09-13T17:17:12.944152+0000","last_clean":"2024-09-13T17:17:12.944152+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T17:17:12.944152+0000","last_undegraded":"2024-09-13T17:17:12.944152+0000","last_fullsized":"2024-09-13T17:17:12.944152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"63'3422","reported_seq":3486,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058428+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T17:17:14.058428+0000","last_peered":"2024-09-13T17:17:14.058428+0000","last_clean":"2024-09-13T17:17:14.058428+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T17:17:14.058428+0000","last_undegraded":"2024-09-13T17:17:14.058428+0000","last_fullsized":"2024-09-13T17:17:14.058428+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":7,"num_write":19042,"num_write_kb":260722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"63'4802","reported_seq":4869,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.992037+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T17:17:21.992037+0000","last_peered":"2024-09-13T17:17:21.992037+0000","last_clean":"2024-09-13T17:17:21.992037+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T17:17:21.992037+0000","last_undegraded":"2024-09-13T17:17:21.992037+0000","last_fullsized":"2024-09-13T17:17:21.992037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26419,"num_write_kb":335470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"63'4","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.258399+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T17:17:14.258399+0000","last_peered":"2024-09-13T17:17:14.258399+0000","last_clean":"2024-09-13T17:17:14.258399+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T17:17:14.258399+0000","last_undegraded":"2024-09-13T17:17:14.258399+0000","last_fullsized":"2024-09-13T17:17:14.258399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"63'2","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.026552+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T17:17:14.026552+0000","last_peered":"2024-09-13T17:17:14.026552+0000","last_clean":"2024-09-13T17:17:14.026552+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T17:17:14.026552+0000","last_undegraded":"2024-09-13T17:17:14.026552+0000","last_fullsized":"2024-09-13T17:17:14.026552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"63'4215","reported_seq":4278,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.984815+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T17:17:13.984815+0000","last_peered":"2024-09-13T17:17:13.984815+0000","last_clean":"2024-09-13T17:17:13.984815+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T17:17:13.984815+0000","last_undegraded":"2024-09-13T17:17:13.984815+0000","last_fullsized":"2024-09-13T17:17:13.984815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":8,"num_write":22461,"num_write_kb":319331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'3790","reported_seq":3857,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107689+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T17:17:22.107689+0000","last_peered":"2024-09-13T17:17:22.107689+0000","last_clean":"2024-09-13T17:17:22.107689+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T17:17:22.107689+0000","last_undegraded":"2024-09-13T17:17:22.107689+0000","last_fullsized":"2024-09-13T17:17:22.107689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21077,"num_write_kb":260732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"63'40","reported_seq":162,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.062885+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T17:17:22.062885+0000","last_peered":"2024-09-13T17:17:22.062885+0000","last_clean":"2024-09-13T17:17:22.062885+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T17:17:22.062885+0000","last_undegraded":"2024-09-13T17:17:22.062885+0000","last_fullsized":"2024-09-13T17:17:22.062885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84545,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":54,"num_read_kb":105,"num_write":36,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"63'9","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.040704+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T17:17:14.040704+0000","last_peered":"2024-09-13T17:17:14.040704+0000","last_clean":"2024-09-13T17:17:14.040704+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T17:17:14.040704+0000","last_undegraded":"2024-09-13T17:17:14.040704+0000","last_fullsized":"2024-09-13T17:17:14.040704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"63'3128","reported_seq":3190,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.042016+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T17:17:14.042016+0000","last_peered":"2024-09-13T17:17:14.042016+0000","last_clean":"2024-09-13T17:17:14.042016+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T17:17:14.042016+0000","last_undegraded":"2024-09-13T17:17:14.042016+0000","last_fullsized":"2024-09-13T17:17:14.042016+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3128,"log_dups_size":0,"ondisk_log_size":3128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17776,"num_write_kb":243283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"63'3657","reported_seq":3724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.977555+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T17:17:21.977555+0000","last_peered":"2024-09-13T17:17:21.977555+0000","last_clean":"2024-09-13T17:17:21.977555+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T17:17:21.977555+0000","last_undegraded":"2024-09-13T17:17:21.977555+0000","last_fullsized":"2024-09-13T17:17:21.977555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19816,"num_write_kb":255512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'10","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.101605+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T17:17:22.101605+0000","last_peered":"2024-09-13T17:17:22.101605+0000","last_clean":"2024-09-13T17:17:22.101605+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T17:17:22.101605+0000","last_undegraded":"2024-09-13T17:17:22.101605+0000","last_fullsized":"2024-09-13T17:17:22.101605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'251","reported_seq":335,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.918664+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T17:17:22.918664+0000","last_peered":"2024-09-13T17:17:22.918664+0000","last_clean":"2024-09-13T17:17:22.918664+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T17:17:22.918664+0000","last_undegraded":"2024-09-13T17:17:22.918664+0000","last_fullsized":"2024-09-13T17:17:22.918664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":57,"num_write":597,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"63'4141","reported_seq":4205,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022107+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T17:17:14.022107+0000","last_peered":"2024-09-13T17:17:14.022107+0000","last_clean":"2024-09-13T17:17:14.022107+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T17:17:14.022107+0000","last_undegraded":"2024-09-13T17:17:14.022107+0000","last_fullsized":"2024-09-13T17:17:14.022107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22849,"num_write_kb":317008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'4103","reported_seq":4178,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.906868+0000","last_change":"2024-09-13T16:48:41.389774+0000","last_active":"2024-09-13T17:17:21.906868+0000","last_peered":"2024-09-13T17:17:21.906868+0000","last_clean":"2024-09-13T17:17:21.906868+0000","last_became_active":"2024-09-13T16:48:41.389507+0000","last_became_peered":"2024-09-13T16:48:41.389507+0000","last_unstale":"2024-09-13T17:17:21.906868+0000","last_undegraded":"2024-09-13T17:17:21.906868+0000","last_fullsized":"2024-09-13T17:17:21.906868+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:07:23.114010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22342,"num_write_kb":278194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"63'3","reported_seq":71,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058621+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T17:17:22.058621+0000","last_peered":"2024-09-13T17:17:22.058621+0000","last_clean":"2024-09-13T17:17:22.058621+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T17:17:22.058621+0000","last_undegraded":"2024-09-13T17:17:22.058621+0000","last_fullsized":"2024-09-13T17:17:22.058621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T17:16:04.972860+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T17:16:04.972860+0000","last_peered":"2024-09-13T17:16:04.972860+0000","last_clean":"2024-09-13T17:16:04.972860+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T17:16:04.972860+0000","last_undegraded":"2024-09-13T17:16:04.972860+0000","last_fullsized":"2024-09-13T17:16:04.972860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"63'2","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.946275+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T17:17:12.946275+0000","last_peered":"2024-09-13T17:17:12.946275+0000","last_clean":"2024-09-13T17:17:12.946275+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T17:17:12.946275+0000","last_undegraded":"2024-09-13T17:17:12.946275+0000","last_fullsized":"2024-09-13T17:17:12.946275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"63'2904","reported_seq":2967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058460+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T17:17:14.058460+0000","last_peered":"2024-09-13T17:17:14.058460+0000","last_clean":"2024-09-13T17:17:14.058460+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T17:17:14.058460+0000","last_undegraded":"2024-09-13T17:17:14.058460+0000","last_fullsized":"2024-09-13T17:17:14.058460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":2,"num_write":15607,"num_write_kb":214254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'4284","reported_seq":4350,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117903+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T17:17:22.117903+0000","last_peered":"2024-09-13T17:17:22.117903+0000","last_clean":"2024-09-13T17:17:22.117903+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T17:17:22.117903+0000","last_undegraded":"2024-09-13T17:17:22.117903+0000","last_fullsized":"2024-09-13T17:17:22.117903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23392,"num_write_kb":310963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"63'5","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096955+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T17:17:22.096955+0000","last_peered":"2024-09-13T17:17:22.096955+0000","last_clean":"2024-09-13T17:17:22.096955+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T17:17:22.096955+0000","last_undegraded":"2024-09-13T17:17:22.096955+0000","last_fullsized":"2024-09-13T17:17:22.096955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"63'3","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.991304+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T17:17:13.991304+0000","last_peered":"2024-09-13T17:17:13.991304+0000","last_clean":"2024-09-13T17:17:13.991304+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T17:17:13.991304+0000","last_undegraded":"2024-09-13T17:17:13.991304+0000","last_fullsized":"2024-09-13T17:17:13.991304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"63'4138","reported_seq":4200,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.019219+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T17:17:14.019219+0000","last_peered":"2024-09-13T17:17:14.019219+0000","last_clean":"2024-09-13T17:17:14.019219+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T17:17:14.019219+0000","last_undegraded":"2024-09-13T17:17:14.019219+0000","last_fullsized":"2024-09-13T17:17:14.019219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":309086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"63'4584","reported_seq":4643,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.128999+0000","last_change":"2024-09-13T16:48:40.767044+0000","last_active":"2024-09-13T17:17:22.128999+0000","last_peered":"2024-09-13T17:17:22.128999+0000","last_clean":"2024-09-13T17:17:22.128999+0000","last_became_active":"2024-09-13T16:48:40.765861+0000","last_became_peered":"2024-09-13T16:48:40.765861+0000","last_unstale":"2024-09-13T17:17:22.128999+0000","last_undegraded":"2024-09-13T17:17:22.128999+0000","last_fullsized":"2024-09-13T17:17:22.128999+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:39.408967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24690,"num_write_kb":320186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"63'6","reported_seq":91,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.999257+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T17:17:21.999257+0000","last_peered":"2024-09-13T17:17:21.999257+0000","last_clean":"2024-09-13T17:17:21.999257+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T17:17:21.999257+0000","last_undegraded":"2024-09-13T17:17:21.999257+0000","last_fullsized":"2024-09-13T17:17:21.999257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'1425","reported_seq":1536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.059648+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T17:17:14.059648+0000","last_peered":"2024-09-13T17:17:14.059648+0000","last_clean":"2024-09-13T17:17:14.059648+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T17:17:14.059648+0000","last_undegraded":"2024-09-13T17:17:14.059648+0000","last_fullsized":"2024-09-13T17:17:14.059648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":38459,"num_write":1400,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"63'3637","reported_seq":3700,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.967851+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T17:17:13.967851+0000","last_peered":"2024-09-13T17:17:13.967851+0000","last_clean":"2024-09-13T17:17:13.967851+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T17:17:13.967851+0000","last_undegraded":"2024-09-13T17:17:13.967851+0000","last_fullsized":"2024-09-13T17:17:13.967851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19635,"num_write_kb":259018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'5067","reported_seq":5126,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.041483+0000","last_change":"2024-09-13T16:48:40.831954+0000","last_active":"2024-09-13T17:17:22.041483+0000","last_peered":"2024-09-13T17:17:22.041483+0000","last_clean":"2024-09-13T17:17:22.041483+0000","last_became_active":"2024-09-13T16:48:40.831696+0000","last_became_peered":"2024-09-13T16:48:40.831696+0000","last_unstale":"2024-09-13T17:17:22.041483+0000","last_undegraded":"2024-09-13T17:17:22.041483+0000","last_fullsized":"2024-09-13T17:17:22.041483+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:53.452702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27028,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"63'7","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.064989+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T17:17:22.064989+0000","last_peered":"2024-09-13T17:17:22.064989+0000","last_clean":"2024-09-13T17:17:22.064989+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T17:17:22.064989+0000","last_undegraded":"2024-09-13T17:17:22.064989+0000","last_fullsized":"2024-09-13T17:17:22.064989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"63'5","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.951351+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T17:17:12.951351+0000","last_peered":"2024-09-13T17:17:12.951351+0000","last_clean":"2024-09-13T17:17:12.951351+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T17:17:12.951351+0000","last_undegraded":"2024-09-13T17:17:12.951351+0000","last_fullsized":"2024-09-13T17:17:12.951351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"63'2915","reported_seq":2977,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.931098+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T17:17:13.931098+0000","last_peered":"2024-09-13T17:17:13.931098+0000","last_clean":"2024-09-13T17:17:13.931098+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T17:17:13.931098+0000","last_undegraded":"2024-09-13T17:17:13.931098+0000","last_fullsized":"2024-09-13T17:17:13.931098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2915,"log_dups_size":0,"ondisk_log_size":2915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":234494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"63'3697","reported_seq":3763,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.091815+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T17:17:22.091815+0000","last_peered":"2024-09-13T17:17:22.091815+0000","last_clean":"2024-09-13T17:17:22.091815+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T17:17:22.091815+0000","last_undegraded":"2024-09-13T17:17:22.091815+0000","last_fullsized":"2024-09-13T17:17:22.091815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20206,"num_write_kb":253140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"63'4","reported_seq":89,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107701+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T17:17:22.107701+0000","last_peered":"2024-09-13T17:17:22.107701+0000","last_clean":"2024-09-13T17:17:22.107701+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T17:17:22.107701+0000","last_undegraded":"2024-09-13T17:17:22.107701+0000","last_fullsized":"2024-09-13T17:17:22.107701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"63'2","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.991145+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T17:17:13.991145+0000","last_peered":"2024-09-13T17:17:13.991145+0000","last_clean":"2024-09-13T17:17:13.991145+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T17:17:13.991145+0000","last_undegraded":"2024-09-13T17:17:13.991145+0000","last_fullsized":"2024-09-13T17:17:13.991145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"63'3522","reported_seq":3583,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.999295+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T17:17:13.999295+0000","last_peered":"2024-09-13T17:17:13.999295+0000","last_clean":"2024-09-13T17:17:13.999295+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T17:17:13.999295+0000","last_undegraded":"2024-09-13T17:17:13.999295+0000","last_fullsized":"2024-09-13T17:17:13.999295+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18828,"num_write_kb":272277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"63'3874","reported_seq":3941,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.067632+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T17:17:22.067632+0000","last_peered":"2024-09-13T17:17:22.067632+0000","last_clean":"2024-09-13T17:17:22.067632+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T17:17:22.067632+0000","last_undegraded":"2024-09-13T17:17:22.067632+0000","last_fullsized":"2024-09-13T17:17:22.067632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21413,"num_write_kb":278431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"63'14","reported_seq":170,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095666+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T17:17:22.095666+0000","last_peered":"2024-09-13T17:17:22.095666+0000","last_clean":"2024-09-13T17:17:22.095666+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T17:17:22.095666+0000","last_undegraded":"2024-09-13T17:17:22.095666+0000","last_fullsized":"2024-09-13T17:17:22.095666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":270,"num_read_kb":270,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036103+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T17:17:14.036103+0000","last_peered":"2024-09-13T17:17:14.036103+0000","last_clean":"2024-09-13T17:17:14.036103+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T17:17:14.036103+0000","last_undegraded":"2024-09-13T17:17:14.036103+0000","last_fullsized":"2024-09-13T17:17:14.036103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"63'3622","reported_seq":3686,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058427+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T17:17:14.058427+0000","last_peered":"2024-09-13T17:17:14.058427+0000","last_clean":"2024-09-13T17:17:14.058427+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T17:17:14.058427+0000","last_undegraded":"2024-09-13T17:17:14.058427+0000","last_fullsized":"2024-09-13T17:17:14.058427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":19996,"num_write_kb":293269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"63'3240","reported_seq":3305,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.013971+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T17:17:22.013971+0000","last_peered":"2024-09-13T17:17:22.013971+0000","last_clean":"2024-09-13T17:17:22.013971+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T17:17:22.013971+0000","last_undegraded":"2024-09-13T17:17:22.013971+0000","last_fullsized":"2024-09-13T17:17:22.013971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3240,"log_dups_size":0,"ondisk_log_size":3240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":237245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"63'5","reported_seq":71,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.098797+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T17:17:22.098797+0000","last_peered":"2024-09-13T17:17:22.098797+0000","last_clean":"2024-09-13T17:17:22.098797+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T17:17:22.098797+0000","last_undegraded":"2024-09-13T17:17:22.098797+0000","last_fullsized":"2024-09-13T17:17:22.098797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'5","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.992012+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T17:17:13.992012+0000","last_peered":"2024-09-13T17:17:13.992012+0000","last_clean":"2024-09-13T17:17:13.992012+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T17:17:13.992012+0000","last_undegraded":"2024-09-13T17:17:13.992012+0000","last_fullsized":"2024-09-13T17:17:13.992012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"63'3376","reported_seq":3437,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.779058+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T17:17:13.779058+0000","last_peered":"2024-09-13T17:17:13.779058+0000","last_clean":"2024-09-13T17:17:13.779058+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T17:17:13.779058+0000","last_undegraded":"2024-09-13T17:17:13.779058+0000","last_fullsized":"2024-09-13T17:17:13.779058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18879,"num_write_kb":251906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"63'4848","reported_seq":4913,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.099538+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T17:17:22.099538+0000","last_peered":"2024-09-13T17:17:22.099538+0000","last_clean":"2024-09-13T17:17:22.099538+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T17:17:22.099538+0000","last_undegraded":"2024-09-13T17:17:22.099538+0000","last_fullsized":"2024-09-13T17:17:22.099538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26963,"num_write_kb":321221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"63'3","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117357+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T17:17:22.117357+0000","last_peered":"2024-09-13T17:17:22.117357+0000","last_clean":"2024-09-13T17:17:22.117357+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T17:17:22.117357+0000","last_undegraded":"2024-09-13T17:17:22.117357+0000","last_fullsized":"2024-09-13T17:17:22.117357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"63'389","reported_seq":558,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:09.910889+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T17:17:09.910889+0000","last_peered":"2024-09-13T17:17:09.910889+0000","last_clean":"2024-09-13T17:17:09.910889+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T17:17:09.910889+0000","last_undegraded":"2024-09-13T17:17:09.910889+0000","last_fullsized":"2024-09-13T17:17:09.910889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":191,"num_read_kb":82094,"num_write":385,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'4856","reported_seq":4919,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.025593+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T17:17:14.025593+0000","last_peered":"2024-09-13T17:17:14.025593+0000","last_clean":"2024-09-13T17:17:14.025593+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T17:17:14.025593+0000","last_undegraded":"2024-09-13T17:17:14.025593+0000","last_fullsized":"2024-09-13T17:17:14.025593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":26453,"num_write_kb":368210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'4063","reported_seq":4128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.081187+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T17:17:22.081187+0000","last_peered":"2024-09-13T17:17:22.081187+0000","last_clean":"2024-09-13T17:17:22.081187+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T17:17:22.081187+0000","last_undegraded":"2024-09-13T17:17:22.081187+0000","last_fullsized":"2024-09-13T17:17:22.081187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4063,"log_dups_size":0,"ondisk_log_size":4063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21999,"num_write_kb":293018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"63'8","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.105763+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T17:17:22.105763+0000","last_peered":"2024-09-13T17:17:22.105763+0000","last_clean":"2024-09-13T17:17:22.105763+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T17:17:22.105763+0000","last_undegraded":"2024-09-13T17:17:22.105763+0000","last_fullsized":"2024-09-13T17:17:22.105763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":144,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'30","reported_seq":155,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.143808+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T17:17:14.143808+0000","last_peered":"2024-09-13T17:17:14.143808+0000","last_clean":"2024-09-13T17:17:14.143808+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T17:17:14.143808+0000","last_undegraded":"2024-09-13T17:17:14.143808+0000","last_fullsized":"2024-09-13T17:17:14.143808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":31,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"63'4239","reported_seq":4300,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.052220+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T17:17:14.052220+0000","last_peered":"2024-09-13T17:17:14.052220+0000","last_clean":"2024-09-13T17:17:14.052220+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T17:17:14.052220+0000","last_undegraded":"2024-09-13T17:17:14.052220+0000","last_fullsized":"2024-09-13T17:17:14.052220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23316,"num_write_kb":318961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'3718","reported_seq":3783,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.786167+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T17:17:21.786167+0000","last_peered":"2024-09-13T17:17:21.786167+0000","last_clean":"2024-09-13T17:17:21.786167+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T17:17:21.786167+0000","last_undegraded":"2024-09-13T17:17:21.786167+0000","last_fullsized":"2024-09-13T17:17:21.786167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3718,"log_dups_size":0,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20956,"num_write_kb":260218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"63'390","reported_seq":546,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:17.006685+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T17:17:17.006685+0000","last_peered":"2024-09-13T17:17:17.006685+0000","last_clean":"2024-09-13T17:17:17.006685+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T17:17:17.006685+0000","last_undegraded":"2024-09-13T17:17:17.006685+0000","last_fullsized":"2024-09-13T17:17:17.006685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":73872,"num_write":388,"num_write_kb":4310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.039214+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T17:17:14.039214+0000","last_peered":"2024-09-13T17:17:14.039214+0000","last_clean":"2024-09-13T17:17:14.039214+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T17:17:14.039214+0000","last_undegraded":"2024-09-13T17:17:14.039214+0000","last_fullsized":"2024-09-13T17:17:14.039214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"63'3522","reported_seq":3584,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.982810+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T17:17:13.982810+0000","last_peered":"2024-09-13T17:17:13.982810+0000","last_clean":"2024-09-13T17:17:13.982810+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T17:17:13.982810+0000","last_undegraded":"2024-09-13T17:17:13.982810+0000","last_fullsized":"2024-09-13T17:17:13.982810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":4,"num_write":19373,"num_write_kb":266222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"63'5138","reported_seq":5203,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095588+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T17:17:22.095588+0000","last_peered":"2024-09-13T17:17:22.095588+0000","last_clean":"2024-09-13T17:17:22.095588+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T17:17:22.095588+0000","last_undegraded":"2024-09-13T17:17:22.095588+0000","last_fullsized":"2024-09-13T17:17:22.095588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27863,"num_write_kb":353454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"63'23","reported_seq":125,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.131774+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T17:17:22.131774+0000","last_peered":"2024-09-13T17:17:22.131774+0000","last_clean":"2024-09-13T17:17:22.131774+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T17:17:22.131774+0000","last_undegraded":"2024-09-13T17:17:22.131774+0000","last_fullsized":"2024-09-13T17:17:22.131774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"63'7","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031857+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T17:17:14.031857+0000","last_peered":"2024-09-13T17:17:14.031857+0000","last_clean":"2024-09-13T17:17:14.031857+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T17:17:14.031857+0000","last_undegraded":"2024-09-13T17:17:14.031857+0000","last_fullsized":"2024-09-13T17:17:14.031857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":168,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"63'3602","reported_seq":3666,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.014764+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T17:17:14.014764+0000","last_peered":"2024-09-13T17:17:14.014764+0000","last_clean":"2024-09-13T17:17:14.014764+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T17:17:14.014764+0000","last_undegraded":"2024-09-13T17:17:14.014764+0000","last_fullsized":"2024-09-13T17:17:14.014764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":20290,"num_write_kb":278255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'3743","reported_seq":3808,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084809+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T17:17:22.084809+0000","last_peered":"2024-09-13T17:17:22.084809+0000","last_clean":"2024-09-13T17:17:22.084809+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T17:17:22.084809+0000","last_undegraded":"2024-09-13T17:17:22.084809+0000","last_fullsized":"2024-09-13T17:17:22.084809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20404,"num_write_kb":278673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"63'3","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.103865+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T17:17:22.103865+0000","last_peered":"2024-09-13T17:17:22.103865+0000","last_clean":"2024-09-13T17:17:22.103865+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T17:17:22.103865+0000","last_undegraded":"2024-09-13T17:17:22.103865+0000","last_fullsized":"2024-09-13T17:17:22.103865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"63'15","reported_seq":185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024454+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T17:17:14.024454+0000","last_peered":"2024-09-13T17:17:14.024454+0000","last_clean":"2024-09-13T17:17:14.024454+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T17:17:14.024454+0000","last_undegraded":"2024-09-13T17:17:14.024454+0000","last_fullsized":"2024-09-13T17:17:14.024454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":324,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"63'3189","reported_seq":3250,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.950668+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T17:17:13.950668+0000","last_peered":"2024-09-13T17:17:13.950668+0000","last_clean":"2024-09-13T17:17:13.950668+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T17:17:13.950668+0000","last_undegraded":"2024-09-13T17:17:13.950668+0000","last_fullsized":"2024-09-13T17:17:13.950668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17342,"num_write_kb":249797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"63'4223","reported_seq":4288,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117238+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T17:17:22.117238+0000","last_peered":"2024-09-13T17:17:22.117238+0000","last_clean":"2024-09-13T17:17:22.117238+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T17:17:22.117238+0000","last_undegraded":"2024-09-13T17:17:22.117238+0000","last_fullsized":"2024-09-13T17:17:22.117238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4223,"log_dups_size":0,"ondisk_log_size":4223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":295025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"63'4","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.059446+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T17:17:22.059446+0000","last_peered":"2024-09-13T17:17:22.059446+0000","last_clean":"2024-09-13T17:17:22.059446+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T17:17:22.059446+0000","last_undegraded":"2024-09-13T17:17:22.059446+0000","last_fullsized":"2024-09-13T17:17:22.059446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"63'5","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.021576+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T17:17:14.021576+0000","last_peered":"2024-09-13T17:17:14.021576+0000","last_clean":"2024-09-13T17:17:14.021576+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T17:17:14.021576+0000","last_undegraded":"2024-09-13T17:17:14.021576+0000","last_fullsized":"2024-09-13T17:17:14.021576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'5357","reported_seq":5423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058245+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T17:17:14.058245+0000","last_peered":"2024-09-13T17:17:14.058245+0000","last_clean":"2024-09-13T17:17:14.058245+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T17:17:14.058245+0000","last_undegraded":"2024-09-13T17:17:14.058245+0000","last_fullsized":"2024-09-13T17:17:14.058245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":39,"num_write":29637,"num_write_kb":393569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"63'26","reported_seq":90,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.989101+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T17:17:13.989101+0000","last_peered":"2024-09-13T17:17:13.989101+0000","last_clean":"2024-09-13T17:17:13.989101+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T17:17:13.989101+0000","last_undegraded":"2024-09-13T17:17:13.989101+0000","last_fullsized":"2024-09-13T17:17:13.989101+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"63'5122","reported_seq":5187,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.106963+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T17:17:22.106963+0000","last_peered":"2024-09-13T17:17:22.106963+0000","last_clean":"2024-09-13T17:17:22.106963+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T17:17:22.106963+0000","last_undegraded":"2024-09-13T17:17:22.106963+0000","last_fullsized":"2024-09-13T17:17:22.106963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":348704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"63'2","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:04.209346+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T17:17:04.209346+0000","last_peered":"2024-09-13T17:17:04.209346+0000","last_clean":"2024-09-13T17:17:04.209346+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T17:17:04.209346+0000","last_undegraded":"2024-09-13T17:17:04.209346+0000","last_fullsized":"2024-09-13T17:17:04.209346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"63'3554","reported_seq":3617,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036436+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T17:17:14.036436+0000","last_peered":"2024-09-13T17:17:14.036436+0000","last_clean":"2024-09-13T17:17:14.036436+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T17:17:14.036436+0000","last_undegraded":"2024-09-13T17:17:14.036436+0000","last_fullsized":"2024-09-13T17:17:14.036436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":268389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"63'1091","reported_seq":1205,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.012135+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T17:17:14.012135+0000","last_peered":"2024-09-13T17:17:14.012135+0000","last_clean":"2024-09-13T17:17:14.012135+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T17:17:14.012135+0000","last_undegraded":"2024-09-13T17:17:14.012135+0000","last_fullsized":"2024-09-13T17:17:14.012135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":9436,"num_write":1065,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"63'4714","reported_seq":4779,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.121909+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T17:17:22.121909+0000","last_peered":"2024-09-13T17:17:22.121909+0000","last_clean":"2024-09-13T17:17:22.121909+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T17:17:22.121909+0000","last_undegraded":"2024-09-13T17:17:22.121909+0000","last_fullsized":"2024-09-13T17:17:22.121909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25459,"num_write_kb":343662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"63'3","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.098646+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T17:17:22.098646+0000","last_peered":"2024-09-13T17:17:22.098646+0000","last_clean":"2024-09-13T17:17:22.098646+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T17:17:22.098646+0000","last_undegraded":"2024-09-13T17:17:22.098646+0000","last_fullsized":"2024-09-13T17:17:22.098646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"63'4328","reported_seq":4388,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.013223+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T17:17:14.013223+0000","last_peered":"2024-09-13T17:17:14.013223+0000","last_clean":"2024-09-13T17:17:14.013223+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T17:17:14.013223+0000","last_undegraded":"2024-09-13T17:17:14.013223+0000","last_fullsized":"2024-09-13T17:17:14.013223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24456,"num_write_kb":315374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"63'396","reported_seq":463,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:23.808734+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T17:17:23.808734+0000","last_peered":"2024-09-13T17:17:23.808734+0000","last_clean":"2024-09-13T17:17:23.808734+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T17:17:23.808734+0000","last_undegraded":"2024-09-13T17:17:23.808734+0000","last_fullsized":"2024-09-13T17:17:23.808734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2790242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":465,"num_write":372,"num_write_kb":2945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"63'4155","reported_seq":4219,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.033063+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T17:17:22.033063+0000","last_peered":"2024-09-13T17:17:22.033063+0000","last_clean":"2024-09-13T17:17:22.033063+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T17:17:22.033063+0000","last_undegraded":"2024-09-13T17:17:22.033063+0000","last_fullsized":"2024-09-13T17:17:22.033063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22247,"num_write_kb":285891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"63'5","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:16:34.958339+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T17:16:34.958339+0000","last_peered":"2024-09-13T17:16:34.958339+0000","last_clean":"2024-09-13T17:16:34.958339+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T17:16:34.958339+0000","last_undegraded":"2024-09-13T17:16:34.958339+0000","last_fullsized":"2024-09-13T17:16:34.958339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"63'3426","reported_seq":3489,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.986253+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T17:17:13.986253+0000","last_peered":"2024-09-13T17:17:13.986253+0000","last_clean":"2024-09-13T17:17:13.986253+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T17:17:13.986253+0000","last_undegraded":"2024-09-13T17:17:13.986253+0000","last_fullsized":"2024-09-13T17:17:13.986253+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3426,"log_dups_size":0,"ondisk_log_size":3426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":18510,"num_write_kb":250514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'712","reported_seq":799,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058400+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T17:17:22.058400+0000","last_peered":"2024-09-13T17:17:22.058400+0000","last_clean":"2024-09-13T17:17:22.058400+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T17:17:22.058400+0000","last_undegraded":"2024-09-13T17:17:22.058400+0000","last_fullsized":"2024-09-13T17:17:22.058400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":70578,"num_write":702,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"63'4316","reported_seq":4381,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.050313+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T17:17:22.050313+0000","last_peered":"2024-09-13T17:17:22.050313+0000","last_clean":"2024-09-13T17:17:22.050313+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T17:17:22.050313+0000","last_undegraded":"2024-09-13T17:17:22.050313+0000","last_fullsized":"2024-09-13T17:17:22.050313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22350,"num_write_kb":290228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'44","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031452+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T17:17:14.031452+0000","last_peered":"2024-09-13T17:17:14.031452+0000","last_clean":"2024-09-13T17:17:14.031452+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T17:17:14.031452+0000","last_undegraded":"2024-09-13T17:17:14.031452+0000","last_fullsized":"2024-09-13T17:17:14.031452+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"63'3129","reported_seq":3191,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031759+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T17:17:14.031759+0000","last_peered":"2024-09-13T17:17:14.031759+0000","last_clean":"2024-09-13T17:17:14.031759+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T17:17:14.031759+0000","last_undegraded":"2024-09-13T17:17:14.031759+0000","last_fullsized":"2024-09-13T17:17:14.031759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17279,"num_write_kb":257601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"63'4785","reported_seq":4852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.086557+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T17:17:22.086557+0000","last_peered":"2024-09-13T17:17:22.086557+0000","last_clean":"2024-09-13T17:17:22.086557+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T17:17:22.086557+0000","last_undegraded":"2024-09-13T17:17:22.086557+0000","last_fullsized":"2024-09-13T17:17:22.086557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26302,"num_write_kb":310249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"63'4231","reported_seq":4294,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.009044+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T17:17:14.009044+0000","last_peered":"2024-09-13T17:17:14.009044+0000","last_clean":"2024-09-13T17:17:14.009044+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T17:17:14.009044+0000","last_undegraded":"2024-09-13T17:17:14.009044+0000","last_fullsized":"2024-09-13T17:17:14.009044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22604,"num_write_kb":324065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"63'3874","reported_seq":3942,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.078497+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T17:17:22.078497+0000","last_peered":"2024-09-13T17:17:22.078497+0000","last_clean":"2024-09-13T17:17:22.078497+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T17:17:22.078497+0000","last_undegraded":"2024-09-13T17:17:22.078497+0000","last_fullsized":"2024-09-13T17:17:22.078497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20920,"num_write_kb":276387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"63'3865","reported_seq":3928,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.006270+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T17:17:14.006270+0000","last_peered":"2024-09-13T17:17:14.006270+0000","last_clean":"2024-09-13T17:17:14.006270+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T17:17:14.006270+0000","last_undegraded":"2024-09-13T17:17:14.006270+0000","last_fullsized":"2024-09-13T17:17:14.006270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21432,"num_write_kb":286796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"63'4552","reported_seq":4621,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.103214+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T17:17:22.103214+0000","last_peered":"2024-09-13T17:17:22.103214+0000","last_clean":"2024-09-13T17:17:22.103214+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T17:17:22.103214+0000","last_undegraded":"2024-09-13T17:17:22.103214+0000","last_fullsized":"2024-09-13T17:17:22.103214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":24031,"num_write_kb":313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"63'4071","reported_seq":4134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.013587+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T17:17:14.013587+0000","last_peered":"2024-09-13T17:17:14.013587+0000","last_clean":"2024-09-13T17:17:14.013587+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T17:17:14.013587+0000","last_undegraded":"2024-09-13T17:17:14.013587+0000","last_fullsized":"2024-09-13T17:17:14.013587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":22061,"num_write_kb":309650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"63'3506","reported_seq":3574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.087622+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T17:17:22.087622+0000","last_peered":"2024-09-13T17:17:22.087622+0000","last_clean":"2024-09-13T17:17:22.087622+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T17:17:22.087622+0000","last_undegraded":"2024-09-13T17:17:22.087622+0000","last_fullsized":"2024-09-13T17:17:22.087622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3506,"log_dups_size":0,"ondisk_log_size":3506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19493,"num_write_kb":254380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"63'4523","reported_seq":4586,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.039988+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T17:17:14.039988+0000","last_peered":"2024-09-13T17:17:14.039988+0000","last_clean":"2024-09-13T17:17:14.039988+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T17:17:14.039988+0000","last_undegraded":"2024-09-13T17:17:14.039988+0000","last_fullsized":"2024-09-13T17:17:14.039988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4523,"log_dups_size":0,"ondisk_log_size":4523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24221,"num_write_kb":338849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"63'4422","reported_seq":4489,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.072016+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T17:17:22.072016+0000","last_peered":"2024-09-13T17:17:22.072016+0000","last_clean":"2024-09-13T17:17:22.072016+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T17:17:22.072016+0000","last_undegraded":"2024-09-13T17:17:22.072016+0000","last_fullsized":"2024-09-13T17:17:22.072016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":24862,"num_write_kb":304752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"63'3771","reported_seq":3835,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022654+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T17:17:14.022654+0000","last_peered":"2024-09-13T17:17:14.022654+0000","last_clean":"2024-09-13T17:17:14.022654+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T17:17:14.022654+0000","last_undegraded":"2024-09-13T17:17:14.022654+0000","last_fullsized":"2024-09-13T17:17:14.022654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21135,"num_write_kb":285540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"63'4461","reported_seq":4527,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095053+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T17:17:22.095053+0000","last_peered":"2024-09-13T17:17:22.095053+0000","last_clean":"2024-09-13T17:17:22.095053+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T17:17:22.095053+0000","last_undegraded":"2024-09-13T17:17:22.095053+0000","last_fullsized":"2024-09-13T17:17:22.095053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25388,"num_write_kb":319094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"63'4193","reported_seq":4256,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.014710+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T17:17:14.014710+0000","last_peered":"2024-09-13T17:17:14.014710+0000","last_clean":"2024-09-13T17:17:14.014710+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T17:17:14.014710+0000","last_undegraded":"2024-09-13T17:17:14.014710+0000","last_fullsized":"2024-09-13T17:17:14.014710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4193,"log_dups_size":0,"ondisk_log_size":4193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23066,"num_write_kb":313713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"63'3978","reported_seq":4044,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.100942+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T17:17:22.100942+0000","last_peered":"2024-09-13T17:17:22.100942+0000","last_clean":"2024-09-13T17:17:22.100942+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T17:17:22.100942+0000","last_undegraded":"2024-09-13T17:17:22.100942+0000","last_fullsized":"2024-09-13T17:17:22.100942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22157,"num_write_kb":281285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"63'4050","reported_seq":4115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.048739+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T17:17:14.048739+0000","last_peered":"2024-09-13T17:17:14.048739+0000","last_clean":"2024-09-13T17:17:14.048739+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T17:17:14.048739+0000","last_undegraded":"2024-09-13T17:17:14.048739+0000","last_fullsized":"2024-09-13T17:17:14.048739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4050,"log_dups_size":0,"ondisk_log_size":4050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21731,"num_write_kb":291923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"63'5211","reported_seq":5272,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.046515+0000","last_change":"2024-09-13T16:48:40.831835+0000","last_active":"2024-09-13T17:17:22.046515+0000","last_peered":"2024-09-13T17:17:22.046515+0000","last_clean":"2024-09-13T17:17:22.046515+0000","last_became_active":"2024-09-13T16:48:40.831649+0000","last_became_peered":"2024-09-13T16:48:40.831649+0000","last_unstale":"2024-09-13T17:17:22.046515+0000","last_undegraded":"2024-09-13T17:17:22.046515+0000","last_fullsized":"2024-09-13T17:17:22.046515+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:54.247988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":28065,"num_write_kb":366501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"63'3806","reported_seq":3870,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024614+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T17:17:14.024614+0000","last_peered":"2024-09-13T17:17:14.024614+0000","last_clean":"2024-09-13T17:17:14.024614+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T17:17:14.024614+0000","last_undegraded":"2024-09-13T17:17:14.024614+0000","last_fullsized":"2024-09-13T17:17:14.024614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4,"num_write":20286,"num_write_kb":286237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"63'3659","reported_seq":3726,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126330+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T17:17:22.126330+0000","last_peered":"2024-09-13T17:17:22.126330+0000","last_clean":"2024-09-13T17:17:22.126330+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T17:17:22.126330+0000","last_undegraded":"2024-09-13T17:17:22.126330+0000","last_fullsized":"2024-09-13T17:17:22.126330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19936,"num_write_kb":264332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"63'5003","reported_seq":5065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.034159+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T17:17:14.034159+0000","last_peered":"2024-09-13T17:17:14.034159+0000","last_clean":"2024-09-13T17:17:14.034159+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T17:17:14.034159+0000","last_undegraded":"2024-09-13T17:17:14.034159+0000","last_fullsized":"2024-09-13T17:17:14.034159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27137,"num_write_kb":387189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"63'4068","reported_seq":4134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.106118+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T17:17:22.106118+0000","last_peered":"2024-09-13T17:17:22.106118+0000","last_clean":"2024-09-13T17:17:22.106118+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T17:17:22.106118+0000","last_undegraded":"2024-09-13T17:17:22.106118+0000","last_fullsized":"2024-09-13T17:17:22.106118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22149,"num_write_kb":293545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"63'3283","reported_seq":3345,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.057875+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T17:17:14.057875+0000","last_peered":"2024-09-13T17:17:14.057875+0000","last_clean":"2024-09-13T17:17:14.057875+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T17:17:14.057875+0000","last_undegraded":"2024-09-13T17:17:14.057875+0000","last_fullsized":"2024-09-13T17:17:14.057875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":248866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"63'4697","reported_seq":4763,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.910034+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T17:17:21.910034+0000","last_peered":"2024-09-13T17:17:21.910034+0000","last_clean":"2024-09-13T17:17:21.910034+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T17:17:21.910034+0000","last_undegraded":"2024-09-13T17:17:21.910034+0000","last_fullsized":"2024-09-13T17:17:21.910034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25800,"num_write_kb":320606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"63'3820","reported_seq":3882,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.997809+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T17:17:13.997809+0000","last_peered":"2024-09-13T17:17:13.997809+0000","last_clean":"2024-09-13T17:17:13.997809+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T17:17:13.997809+0000","last_undegraded":"2024-09-13T17:17:13.997809+0000","last_fullsized":"2024-09-13T17:17:13.997809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3820,"log_dups_size":0,"ondisk_log_size":3820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20813,"num_write_kb":300503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"63'4719","reported_seq":4787,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117962+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T17:17:22.117962+0000","last_peered":"2024-09-13T17:17:22.117962+0000","last_clean":"2024-09-13T17:17:22.117962+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T17:17:22.117962+0000","last_undegraded":"2024-09-13T17:17:22.117962+0000","last_fullsized":"2024-09-13T17:17:22.117962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":25263,"num_write_kb":330973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"63'3722","reported_seq":3784,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.001243+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T17:17:14.001243+0000","last_peered":"2024-09-13T17:17:14.001243+0000","last_clean":"2024-09-13T17:17:14.001243+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T17:17:14.001243+0000","last_undegraded":"2024-09-13T17:17:14.001243+0000","last_fullsized":"2024-09-13T17:17:14.001243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20022,"num_write_kb":289122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"63'3813","reported_seq":3879,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.064518+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T17:17:22.064518+0000","last_peered":"2024-09-13T17:17:22.064518+0000","last_clean":"2024-09-13T17:17:22.064518+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T17:17:22.064518+0000","last_undegraded":"2024-09-13T17:17:22.064518+0000","last_fullsized":"2024-09-13T17:17:22.064518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21202,"num_write_kb":265282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"63'4479","reported_seq":4545,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.060514+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T17:17:14.060514+0000","last_peered":"2024-09-13T17:17:14.060514+0000","last_clean":"2024-09-13T17:17:14.060514+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T17:17:14.060514+0000","last_undegraded":"2024-09-13T17:17:14.060514+0000","last_fullsized":"2024-09-13T17:17:14.060514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":24556,"num_write_kb":340437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"63'4549","reported_seq":4615,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058661+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T17:17:22.058661+0000","last_peered":"2024-09-13T17:17:22.058661+0000","last_clean":"2024-09-13T17:17:22.058661+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T17:17:22.058661+0000","last_undegraded":"2024-09-13T17:17:22.058661+0000","last_fullsized":"2024-09-13T17:17:22.058661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24549,"num_write_kb":319591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"63'3479","reported_seq":3542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.019701+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T17:17:14.019701+0000","last_peered":"2024-09-13T17:17:14.019701+0000","last_clean":"2024-09-13T17:17:14.019701+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T17:17:14.019701+0000","last_undegraded":"2024-09-13T17:17:14.019701+0000","last_fullsized":"2024-09-13T17:17:14.019701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3479,"log_dups_size":0,"ondisk_log_size":3479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18966,"num_write_kb":268432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"63'3573","reported_seq":3639,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084131+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T17:17:22.084131+0000","last_peered":"2024-09-13T17:17:22.084131+0000","last_clean":"2024-09-13T17:17:22.084131+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T17:17:22.084131+0000","last_undegraded":"2024-09-13T17:17:22.084131+0000","last_fullsized":"2024-09-13T17:17:22.084131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19348,"num_write_kb":270113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"63'4058","reported_seq":4120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036359+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T17:17:14.036359+0000","last_peered":"2024-09-13T17:17:14.036359+0000","last_clean":"2024-09-13T17:17:14.036359+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T17:17:14.036359+0000","last_undegraded":"2024-09-13T17:17:14.036359+0000","last_fullsized":"2024-09-13T17:17:14.036359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22331,"num_write_kb":302585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"63'5839","reported_seq":5904,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117408+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T17:17:22.117408+0000","last_peered":"2024-09-13T17:17:22.117408+0000","last_clean":"2024-09-13T17:17:22.117408+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T17:17:22.117408+0000","last_undegraded":"2024-09-13T17:17:22.117408+0000","last_fullsized":"2024-09-13T17:17:22.117408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31756,"num_write_kb":397521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"63'3835","reported_seq":3897,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.035326+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T17:17:14.035326+0000","last_peered":"2024-09-13T17:17:14.035326+0000","last_clean":"2024-09-13T17:17:14.035326+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T17:17:14.035326+0000","last_undegraded":"2024-09-13T17:17:14.035326+0000","last_fullsized":"2024-09-13T17:17:14.035326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":21185,"num_write_kb":306173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"63'4134","reported_seq":4199,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.079721+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T17:17:22.079721+0000","last_peered":"2024-09-13T17:17:22.079721+0000","last_clean":"2024-09-13T17:17:22.079721+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T17:17:22.079721+0000","last_undegraded":"2024-09-13T17:17:22.079721+0000","last_fullsized":"2024-09-13T17:17:22.079721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22268,"num_write_kb":281801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"63'5073","reported_seq":5136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058407+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T17:17:14.058407+0000","last_peered":"2024-09-13T17:17:14.058407+0000","last_clean":"2024-09-13T17:17:14.058407+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T17:17:14.058407+0000","last_undegraded":"2024-09-13T17:17:14.058407+0000","last_fullsized":"2024-09-13T17:17:14.058407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":28693,"num_write_kb":393724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"63'3822","reported_seq":3887,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107201+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T17:17:22.107201+0000","last_peered":"2024-09-13T17:17:22.107201+0000","last_clean":"2024-09-13T17:17:22.107201+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T17:17:22.107201+0000","last_undegraded":"2024-09-13T17:17:22.107201+0000","last_fullsized":"2024-09-13T17:17:22.107201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20823,"num_write_kb":275830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"63'2778","reported_seq":2842,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.048760+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T17:17:14.048760+0000","last_peered":"2024-09-13T17:17:14.048760+0000","last_clean":"2024-09-13T17:17:14.048760+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T17:17:14.048760+0000","last_undegraded":"2024-09-13T17:17:14.048760+0000","last_fullsized":"2024-09-13T17:17:14.048760+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":34,"num_write":15212,"num_write_kb":208573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"63'3975","reported_seq":4040,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.131512+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T17:17:22.131512+0000","last_peered":"2024-09-13T17:17:22.131512+0000","last_clean":"2024-09-13T17:17:22.131512+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T17:17:22.131512+0000","last_undegraded":"2024-09-13T17:17:22.131512+0000","last_fullsized":"2024-09-13T17:17:22.131512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3975,"log_dups_size":0,"ondisk_log_size":3975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21351,"num_write_kb":278200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"63'3844","reported_seq":3907,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.966555+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T17:17:13.966555+0000","last_peered":"2024-09-13T17:17:13.966555+0000","last_clean":"2024-09-13T17:17:13.966555+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T17:17:13.966555+0000","last_undegraded":"2024-09-13T17:17:13.966555+0000","last_fullsized":"2024-09-13T17:17:13.966555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21085,"num_write_kb":289508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"63'3324","reported_seq":3389,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.087163+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T17:17:22.087163+0000","last_peered":"2024-09-13T17:17:22.087163+0000","last_clean":"2024-09-13T17:17:22.087163+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T17:17:22.087163+0000","last_undegraded":"2024-09-13T17:17:22.087163+0000","last_fullsized":"2024-09-13T17:17:22.087163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3324,"log_dups_size":0,"ondisk_log_size":3324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18810,"num_write_kb":235312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"63'3743","reported_seq":3805,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.055690+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T17:17:14.055690+0000","last_peered":"2024-09-13T17:17:14.055690+0000","last_clean":"2024-09-13T17:17:14.055690+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T17:17:14.055690+0000","last_undegraded":"2024-09-13T17:17:14.055690+0000","last_fullsized":"2024-09-13T17:17:14.055690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20790,"num_write_kb":280180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"63'3173","reported_seq":3238,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.083735+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T17:17:22.083735+0000","last_peered":"2024-09-13T17:17:22.083735+0000","last_clean":"2024-09-13T17:17:22.083735+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T17:17:22.083735+0000","last_undegraded":"2024-09-13T17:17:22.083735+0000","last_fullsized":"2024-09-13T17:17:22.083735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17044,"num_write_kb":219853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"63'3460","reported_seq":3522,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.008254+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T17:17:14.008254+0000","last_peered":"2024-09-13T17:17:14.008254+0000","last_clean":"2024-09-13T17:17:14.008254+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T17:17:14.008254+0000","last_undegraded":"2024-09-13T17:17:14.008254+0000","last_fullsized":"2024-09-13T17:17:14.008254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":19788,"num_write_kb":263976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"63'3747","reported_seq":3812,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.008211+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T17:17:22.008211+0000","last_peered":"2024-09-13T17:17:22.008211+0000","last_clean":"2024-09-13T17:17:22.008211+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T17:17:22.008211+0000","last_undegraded":"2024-09-13T17:17:22.008211+0000","last_fullsized":"2024-09-13T17:17:22.008211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20231,"num_write_kb":269532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"63'3422","reported_seq":3483,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.003363+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T17:17:14.003363+0000","last_peered":"2024-09-13T17:17:14.003363+0000","last_clean":"2024-09-13T17:17:14.003363+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T17:17:14.003363+0000","last_undegraded":"2024-09-13T17:17:14.003363+0000","last_fullsized":"2024-09-13T17:17:14.003363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18722,"num_write_kb":267617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"63'4803","reported_seq":4868,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084790+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T17:17:22.084790+0000","last_peered":"2024-09-13T17:17:22.084790+0000","last_clean":"2024-09-13T17:17:22.084790+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T17:17:22.084790+0000","last_undegraded":"2024-09-13T17:17:22.084790+0000","last_fullsized":"2024-09-13T17:17:22.084790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26967,"num_write_kb":333429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"63'3840","reported_seq":3901,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.021162+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T17:17:14.021162+0000","last_peered":"2024-09-13T17:17:14.021162+0000","last_clean":"2024-09-13T17:17:14.021162+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T17:17:14.021162+0000","last_undegraded":"2024-09-13T17:17:14.021162+0000","last_fullsized":"2024-09-13T17:17:14.021162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21113,"num_write_kb":293770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"63'5433","reported_seq":5500,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.040459+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T17:17:22.040459+0000","last_peered":"2024-09-13T17:17:22.040459+0000","last_clean":"2024-09-13T17:17:22.040459+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T17:17:22.040459+0000","last_undegraded":"2024-09-13T17:17:22.040459+0000","last_fullsized":"2024-09-13T17:17:22.040459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30664,"num_write_kb":389336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"63'4741","reported_seq":4806,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.053779+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T17:17:14.053779+0000","last_peered":"2024-09-13T17:17:14.053779+0000","last_clean":"2024-09-13T17:17:14.053779+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T17:17:14.053779+0000","last_undegraded":"2024-09-13T17:17:14.053779+0000","last_fullsized":"2024-09-13T17:17:14.053779+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":15,"num_write":25588,"num_write_kb":340391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"63'4281","reported_seq":4345,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.981625+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T17:17:21.981625+0000","last_peered":"2024-09-13T17:17:21.981625+0000","last_clean":"2024-09-13T17:17:21.981625+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T17:17:21.981625+0000","last_undegraded":"2024-09-13T17:17:21.981625+0000","last_fullsized":"2024-09-13T17:17:21.981625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23373,"num_write_kb":302635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"63'3575","reported_seq":3641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.054561+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T17:17:14.054561+0000","last_peered":"2024-09-13T17:17:14.054561+0000","last_clean":"2024-09-13T17:17:14.054561+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T17:17:14.054561+0000","last_undegraded":"2024-09-13T17:17:14.054561+0000","last_fullsized":"2024-09-13T17:17:14.054561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":7,"num_write":20117,"num_write_kb":289677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"63'4404","reported_seq":4468,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.129331+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T17:17:22.129331+0000","last_peered":"2024-09-13T17:17:22.129331+0000","last_clean":"2024-09-13T17:17:22.129331+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T17:17:22.129331+0000","last_undegraded":"2024-09-13T17:17:22.129331+0000","last_fullsized":"2024-09-13T17:17:22.129331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24308,"num_write_kb":306423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"63'3318","reported_seq":3382,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.949311+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T17:17:13.949311+0000","last_peered":"2024-09-13T17:17:13.949311+0000","last_clean":"2024-09-13T17:17:13.949311+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T17:17:13.949311+0000","last_undegraded":"2024-09-13T17:17:13.949311+0000","last_fullsized":"2024-09-13T17:17:13.949311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3318,"log_dups_size":0,"ondisk_log_size":3318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18295,"num_write_kb":253537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"63'4013","reported_seq":4077,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.861950+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T17:17:21.861950+0000","last_peered":"2024-09-13T17:17:21.861950+0000","last_clean":"2024-09-13T17:17:21.861950+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T17:17:21.861950+0000","last_undegraded":"2024-09-13T17:17:21.861950+0000","last_fullsized":"2024-09-13T17:17:21.861950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21393,"num_write_kb":262042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"63'3658","reported_seq":3722,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.048939+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T17:17:14.048939+0000","last_peered":"2024-09-13T17:17:14.048939+0000","last_clean":"2024-09-13T17:17:14.048939+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T17:17:14.048939+0000","last_undegraded":"2024-09-13T17:17:14.048939+0000","last_fullsized":"2024-09-13T17:17:14.048939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19946,"num_write_kb":268446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"63'3910","reported_seq":3974,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.108541+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T17:17:22.108541+0000","last_peered":"2024-09-13T17:17:22.108541+0000","last_clean":"2024-09-13T17:17:22.108541+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T17:17:22.108541+0000","last_undegraded":"2024-09-13T17:17:22.108541+0000","last_fullsized":"2024-09-13T17:17:22.108541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3910,"log_dups_size":0,"ondisk_log_size":3910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":278651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"63'3074","reported_seq":3134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.837771+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T17:17:13.837771+0000","last_peered":"2024-09-13T17:17:13.837771+0000","last_clean":"2024-09-13T17:17:13.837771+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T17:17:13.837771+0000","last_undegraded":"2024-09-13T17:17:13.837771+0000","last_fullsized":"2024-09-13T17:17:13.837771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":246226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"63'3515","reported_seq":3583,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.005699+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T17:17:22.005699+0000","last_peered":"2024-09-13T17:17:22.005699+0000","last_clean":"2024-09-13T17:17:22.005699+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T17:17:22.005699+0000","last_undegraded":"2024-09-13T17:17:22.005699+0000","last_fullsized":"2024-09-13T17:17:22.005699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19512,"num_write_kb":258445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"63'3439","reported_seq":3500,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.038223+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T17:17:14.038223+0000","last_peered":"2024-09-13T17:17:14.038223+0000","last_clean":"2024-09-13T17:17:14.038223+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T17:17:14.038223+0000","last_undegraded":"2024-09-13T17:17:14.038223+0000","last_fullsized":"2024-09-13T17:17:14.038223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":17681,"num_write_kb":263603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"63'3892","reported_seq":3960,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.067970+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T17:17:22.067970+0000","last_peered":"2024-09-13T17:17:22.067970+0000","last_clean":"2024-09-13T17:17:22.067970+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T17:17:22.067970+0000","last_undegraded":"2024-09-13T17:17:22.067970+0000","last_fullsized":"2024-09-13T17:17:22.067970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21304,"num_write_kb":286190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"63'3622","reported_seq":3682,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.941100+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T17:17:13.941100+0000","last_peered":"2024-09-13T17:17:13.941100+0000","last_clean":"2024-09-13T17:17:13.941100+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T17:17:13.941100+0000","last_undegraded":"2024-09-13T17:17:13.941100+0000","last_fullsized":"2024-09-13T17:17:13.941100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19729,"num_write_kb":275691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"63'5036","reported_seq":5103,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.072290+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T17:17:22.072290+0000","last_peered":"2024-09-13T17:17:22.072290+0000","last_clean":"2024-09-13T17:17:22.072290+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T17:17:22.072290+0000","last_undegraded":"2024-09-13T17:17:22.072290+0000","last_fullsized":"2024-09-13T17:17:22.072290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27469,"num_write_kb":368239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"63'3689","reported_seq":3749,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.059558+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T17:17:14.059558+0000","last_peered":"2024-09-13T17:17:14.059558+0000","last_clean":"2024-09-13T17:17:14.059558+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T17:17:14.059558+0000","last_undegraded":"2024-09-13T17:17:14.059558+0000","last_fullsized":"2024-09-13T17:17:14.059558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3689,"log_dups_size":0,"ondisk_log_size":3689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20818,"num_write_kb":284187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"63'4047","reported_seq":4115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126604+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T17:17:22.126604+0000","last_peered":"2024-09-13T17:17:22.126604+0000","last_clean":"2024-09-13T17:17:22.126604+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T17:17:22.126604+0000","last_undegraded":"2024-09-13T17:17:22.126604+0000","last_fullsized":"2024-09-13T17:17:22.126604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22743,"num_write_kb":291925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":197,"num_write":1329762,"num_write_kb":18444058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242925,"ondisk_log_size":242925,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19024977,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1583,"num_read_kb":206738,"num_write":4472,"num_write_kb":20527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38109184,"data_stored":38049954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4352,"ondisk_log_size":4352,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":1479870,"num_write_kb":19043750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270762,"ondisk_log_size":270762,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":19653473,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1296,"num_read_kb":190456,"num_write":4501,"num_write_kb":21128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39370752,"data_stored":39306946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4332,"ondisk_log_size":4332,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":942768,"kb_used_data":13672,"kb_used_omap":1,"kb_used_meta":929086,"kb_avail":93429072,"statfs":{"total":96636764160,"available":95671369728,"internally_reserved":0,"allocated":14000128,"data_stored":13606602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":951384523},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":1.9890000000000001,"15min":1.212},"min":{"1min":0.24299999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":8.0960000000000001,"5min":16.571999999999999,"15min":16.571999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":2.3239999999999998,"5min":2.1040000000000001,"15min":1.3080000000000001},"min":{"1min":0.26600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":9.7799999999999994,"5min":16.523,"15min":16.523},"last":0.95399999999999996}]},{"osd":1,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":5.859,"5min":2.754,"15min":1.486},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":59.911999999999999,"5min":59.911999999999999,"15min":59.911999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":5.8680000000000003,"5min":2.8519999999999999,"15min":1.5660000000000001},"min":{"1min":0.28699999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":59.895000000000003,"5min":59.895000000000003,"15min":59.895000000000003},"last":1.1080000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.113,"5min":1.6659999999999999,"15min":1.1040000000000001},"min":{"1min":0.28999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.6869999999999994,"5min":16.023,"15min":16.023},"last":0.90000000000000002},{"interface":"front","average":{"1min":2.173,"5min":1.659,"15min":1.1559999999999999},"min":{"1min":0.28299999999999997,"5min":0.249,"15min":0.249},"max":{"1min":9.5619999999999994,"5min":10.654999999999999,"15min":10.654999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.2200000000000002,"5min":1.944,"15min":1.2210000000000001},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":9.7370000000000001,"5min":23.225000000000001,"15min":23.225000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":2.2959999999999998,"5min":1.944,"15min":1.2789999999999999},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.891,"5min":12.045999999999999,"15min":12.045999999999999},"last":1.0169999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":2.0659999999999998,"15min":1.274},"min":{"1min":0.25900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":14.209,"5min":18.004999999999999,"15min":18.004999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":3.9510000000000001,"5min":2.339,"15min":1.391},"min":{"1min":0.28299999999999997,"5min":0.245,"15min":0.245},"max":{"1min":22.315999999999999,"5min":22.315999999999999,"15min":22.315999999999999},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":4.9809999999999999,"5min":2.2959999999999998,"15min":1.419},"min":{"1min":0.32200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":37.302999999999997,"5min":37.302999999999997,"15min":37.302999999999997},"last":0.86599999999999999},{"interface":"front","average":{"1min":2.6560000000000001,"5min":1.8100000000000001,"15min":1.27},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":16.867999999999999,"5min":16.867999999999999,"15min":16.867999999999999},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":1.788,"15min":1.292},"min":{"1min":0.38100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":9.1660000000000004,"5min":17.968,"15min":17.968},"last":0.81200000000000006},{"interface":"front","average":{"1min":3.29,"5min":1.9910000000000001,"15min":1.3500000000000001},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.025,"5min":19.244,"15min":19.244},"last":0.99399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575208,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673180,"kb_used_data":11732,"kb_used_omap":1,"kb_used_meta":661438,"kb_avail":93698660,"statfs":{"total":96636764160,"available":95947427840,"internally_reserved":0,"allocated":12013568,"data_stored":11612788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":677312968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.97999999999999998,"15min":0.86399999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":2.3839999999999999,"5min":6.335,"15min":6.335},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.878,"5min":1.111,"15min":0.89300000000000002},"min":{"1min":0.24199999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.5299999999999998,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2130000000000001,"15min":0.95899999999999996},"min":{"1min":0.28899999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":6.3230000000000004,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":0.82599999999999996},{"interface":"front","average":{"1min":2.0379999999999998,"5min":1.5169999999999999,"15min":1.034},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":9.3629999999999995,"5min":10.111000000000001,"15min":10.111000000000001},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.98199999999999998,"15min":0.90700000000000003},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":6.2160000000000002,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.1850000000000001,"15min":1.006},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.1130000000000004,"5min":6.6079999999999997,"15min":6.6079999999999997},"last":1.1419999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5800000000000001,"15min":1.1220000000000001},"min":{"1min":0.308,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":15.068,"5min":15.247999999999999,"15min":15.247999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.782,"5min":1.5249999999999999,"15min":1.127},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":15.179,"5min":15.179,"15min":15.179},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":1.222,"15min":1.042},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4790000000000001,"5min":15.326000000000001,"15min":15.326000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.454,"15min":1.1259999999999999},"min":{"1min":0.24399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":11.48,"5min":15.026,"15min":15.026},"last":1.175}]},{"osd":5,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.5089999999999999,"15min":1.2},"min":{"1min":0.38200000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":9.6080000000000005,"5min":11.566000000000001,"15min":11.566000000000001},"last":1.256},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.4139999999999999,"15min":1.1639999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":10.223000000000001,"5min":10.223000000000001,"15min":10.223000000000001},"last":1.4159999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.583,"15min":1.2250000000000001},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":9.8960000000000008,"5min":13.112,"15min":13.112},"last":1.3149999999999999},{"interface":"front","average":{"1min":2.0470000000000002,"5min":1.7470000000000001,"15min":1.28},"min":{"1min":0.33200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":14.4,"5min":14.4,"15min":14.4},"last":1.4490000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575209,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":957772,"kb_used_data":4996,"kb_used_omap":1,"kb_used_meta":952766,"kb_avail":93414068,"statfs":{"total":96636764160,"available":95656005632,"internally_reserved":0,"allocated":5115904,"data_stored":4720442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":975632838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":3.762,"5min":1.978,"15min":1.1919999999999999},"min":{"1min":0.246,"5min":0.215,"15min":0.215},"max":{"1min":27.952999999999999,"5min":27.952999999999999,"15min":27.952999999999999},"last":1.042},{"interface":"front","average":{"1min":2.5059999999999998,"5min":1.629,"15min":1.121},"min":{"1min":0.28100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":23.619,"5min":23.619,"15min":23.619},"last":1.0980000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.8210000000000002,"5min":1.8109999999999999,"15min":1.1379999999999999},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":13.515000000000001,"5min":13.515000000000001,"15min":13.515000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.621,"5min":1.9299999999999999,"15min":1.1839999999999999},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":6.9290000000000003,"5min":35.186999999999998,"15min":35.186999999999998},"last":0.84099999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":2.306,"15min":1.3420000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.609,"5min":51.18,"15min":51.18},"last":0.873},{"interface":"front","average":{"1min":1.732,"5min":1.929,"15min":1.2949999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.8659999999999997,"5min":48.756,"15min":48.756},"last":0.99199999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.7480000000000002,"5min":2.0219999999999998,"15min":1.2789999999999999},"min":{"1min":0.39200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":15.946999999999999,"5min":19.800000000000001,"15min":19.800000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.7649999999999999,"15min":1.256},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":9.1020000000000003,"5min":9.4789999999999992,"15min":9.4789999999999992},"last":0.80000000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":4.2270000000000003,"5min":2.4260000000000002,"15min":1.379},"min":{"1min":0.40699999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":20.713999999999999,"5min":22.361999999999998,"15min":22.361999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":4.1710000000000003,"5min":2.9940000000000002,"15min":1.641},"min":{"1min":0.35299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":28.062999999999999,"5min":31.843,"15min":31.843},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.427,"15min":1.131},"min":{"1min":0.375,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.3300000000000001,"5min":8.1910000000000007,"15min":8.1910000000000007},"last":1.1220000000000001},{"interface":"front","average":{"1min":2.052,"5min":1.7649999999999999,"15min":1.26},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.1059999999999999,"5min":13.031000000000001,"15min":13.031000000000001},"last":0.92700000000000005}]},{"osd":7,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":2.1890000000000001,"15min":1.4139999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":11.239000000000001,"5min":35.124000000000002,"15min":35.124000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":2.0550000000000002,"5min":2.5089999999999999,"15min":1.546},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.99,"5min":32.328000000000003,"15min":32.328000000000003},"last":0.89300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575209,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1011316,"kb_used_data":13932,"kb_used_omap":1,"kb_used_meta":997374,"kb_avail":93360524,"statfs":{"total":96636764160,"available":95601176576,"internally_reserved":0,"allocated":14266368,"data_stored":13858021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1021311430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.4910000000000001,"15min":1.0369999999999999},"min":{"1min":0.25,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":13.18,"5min":13.18,"15min":13.18},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.829,"5min":1.575,"15min":1.141},"min":{"1min":0.25700000000000001,"5min":0.251,"15min":0.251},"max":{"1min":13.163,"5min":13.163,"15min":13.163},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":3.1299999999999999,"5min":1.73,"15min":1.1060000000000001},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":27.242000000000001,"5min":27.242000000000001,"15min":27.242000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":2.9119999999999999,"5min":1.978,"15min":1.232},"min":{"1min":0.34399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":26.545999999999999,"5min":26.545999999999999,"15min":26.545999999999999},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.577,"15min":1.1060000000000001},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":7.2619999999999996,"5min":11.803000000000001,"15min":11.803000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.573,"15min":1.131},"min":{"1min":0.35299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3319999999999999,"5min":8.6709999999999994,"15min":8.6709999999999994},"last":0.83699999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":2.1899999999999999,"15min":1.343},"min":{"1min":0.32000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":10.583,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":2.875,"5min":1.9770000000000001,"15min":1.298},"min":{"1min":0.39300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":20.468,"5min":20.468,"15min":20.468},"last":0.94599999999999995}]},{"osd":5,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.4079999999999999,"15min":1.087},"min":{"1min":0.314,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.109,"5min":13.146000000000001,"15min":13.146000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.823,"5min":1.8080000000000001,"15min":1.264},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":14.202,"5min":15.044,"15min":15.044},"last":0.99199999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4359999999999999,"15min":1.1279999999999999},"min":{"1min":0.35299999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":7.2030000000000003,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":1.232},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":1.234},"min":{"1min":0.38400000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.7320000000000002,"5min":7.5860000000000003,"15min":7.5860000000000003},"last":1.069}]},{"osd":7,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.8169999999999999,"15min":1.345},"min":{"1min":0.33500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":6.3449999999999998,"5min":24.518999999999998,"15min":24.518999999999998},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.595,"5min":1.7170000000000001,"15min":1.306},"min":{"1min":0.372,"5min":0.312,"15min":0.312},"max":{"1min":9.0519999999999996,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":1.139}]}]},{"osd":3,"up_from":13,"seq":55834575209,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":958468,"kb_used_data":19388,"kb_used_omap":1,"kb_used_meta":939070,"kb_avail":93413372,"statfs":{"total":96636764160,"available":95655292928,"internally_reserved":0,"allocated":19853312,"data_stored":19449342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":961608136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":2.702,"5min":1.4350000000000001,"15min":0.91700000000000004},"min":{"1min":0.19800000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":15.791,"5min":15.791,"15min":15.791},"last":0.36499999999999999},{"interface":"front","average":{"1min":4.133,"5min":1.508,"15min":0.96899999999999997},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":20.699999999999999,"5min":20.699999999999999,"15min":20.699999999999999},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.8999999999999999,"5min":1.391,"15min":0.91100000000000003},"min":{"1min":0.28499999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":10.933,"5min":16.100000000000001,"15min":16.100000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.377,"15min":0.98299999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":8.2919999999999998,"5min":10.417,"15min":10.417},"last":0.432}]},{"osd":2,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.196,"15min":0.93000000000000005},"min":{"1min":0.23200000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":5.1219999999999999,"5min":8.5950000000000006,"15min":8.5950000000000006},"last":0.40600000000000003},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2410000000000001,"15min":0.94599999999999995},"min":{"1min":0.24099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":10.737,"5min":15.433999999999999,"15min":15.433999999999999},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":2.2000000000000002,"5min":1.627,"15min":1.1799999999999999},"min":{"1min":0.312,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":10.835000000000001,"5min":10.835000000000001,"15min":10.835000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.9379999999999999,"5min":1.401,"15min":1.1240000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":7.0439999999999996,"5min":9.2759999999999998,"15min":9.2759999999999998},"last":0.64700000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":2.6880000000000002,"5min":1.841,"15min":1.3},"min":{"1min":0.42199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":31.187999999999999,"5min":31.187999999999999,"15min":31.187999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":2.508,"5min":1.802,"15min":1.29},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":17.466000000000001,"5min":17.466000000000001,"15min":17.466000000000001},"last":0.69999999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.764,"5min":1.768,"15min":1.286},"min":{"1min":0.34899999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":10.785,"5min":12.567,"15min":12.567},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.998,"5min":2.1499999999999999,"15min":1.464},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":10.872,"5min":15.882,"15min":15.882},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.6579999999999999,"15min":1.2929999999999999},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":9.2720000000000002,"5min":11.94,"15min":11.94},"last":0.625},{"interface":"front","average":{"1min":3.3420000000000001,"5min":2.1520000000000001,"15min":1.4339999999999999},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":25.652999999999999,"5min":25.652999999999999,"15min":25.652999999999999},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575209,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662268,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":661374,"kb_avail":93709572,"statfs":{"total":96636764160,"available":95958601728,"internally_reserved":0,"allocated":905216,"data_stored":517265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":677247430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.89500000000000002,"15min":0.70699999999999996},"min":{"1min":0.189,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":3.3319999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.4830000000000001,"15min":0.96999999999999997},"min":{"1min":0.218,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":9.2729999999999997,"5min":9.2729999999999997,"15min":9.2729999999999997},"last":0.85399999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.3069999999999999,"5min":1.766,"15min":1.0680000000000001},"min":{"1min":0.29499999999999998,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":11.319000000000001,"5min":33.579999999999998,"15min":33.579999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":2.4359999999999999,"5min":1.7509999999999999,"15min":1.089},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":14.624000000000001,"5min":23.965,"15min":23.965},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.7949999999999999,"5min":2.1779999999999999,"15min":1.266},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.245},"max":{"1min":11.837999999999999,"5min":22.760000000000002,"15min":22.760000000000002},"last":0.998},{"interface":"front","average":{"1min":2.5449999999999999,"5min":1.6499999999999999,"15min":1.093},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":10.372,"5min":10.372,"15min":10.372},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.1520000000000001,"5min":2.012,"15min":1.292},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":10.324999999999999,"5min":20.673999999999999,"15min":20.673999999999999},"last":1.024},{"interface":"front","average":{"1min":2.0289999999999999,"5min":1.7669999999999999,"15min":1.24},"min":{"1min":0.42199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":6.4059999999999997,"5min":16.5,"15min":16.5},"last":0.97199999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":4.601,"5min":2.5329999999999999,"15min":1.5149999999999999},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":21.722000000000001,"5min":21.722000000000001,"15min":21.722000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":3.2240000000000002,"5min":2.238,"15min":1.415},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":19.242999999999999,"5min":29.620999999999999,"15min":29.620999999999999},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.6739999999999999,"15min":1.254},"min":{"1min":0.44,"5min":0.38,"15min":0.38},"max":{"1min":5.3239999999999998,"5min":21.882999999999999,"15min":21.882999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":2.2669999999999999,"5min":2.1070000000000002,"15min":1.429},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.29,"5min":25.274999999999999,"15min":25.274999999999999},"last":0.94899999999999995}]},{"osd":7,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":5.9109999999999996,"5min":3.2639999999999998,"15min":1.825},"min":{"1min":0.45700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":28.094999999999999,"5min":43.649000000000001,"15min":43.649000000000001},"last":1.147},{"interface":"front","average":{"1min":6.6379999999999999,"5min":3.1709999999999998,"15min":1.8460000000000001},"min":{"1min":0.44600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":27.417999999999999,"5min":27.417999999999999,"15min":27.417999999999999},"last":1.1140000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575209,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798968,"kb_used_data":9008,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572872,"statfs":{"total":96636764160,"available":95818620928,"internally_reserved":0,"allocated":9224192,"data_stored":8822112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":808909254},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.8919999999999999,"5min":1.8089999999999999,"15min":1.04},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":9.3499999999999996,"5min":15.81,"15min":15.81},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.7050000000000001,"15min":1.056},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.8360000000000003,"5min":15.823,"15min":15.823},"last":0.627}]},{"osd":2,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":1.4139999999999999,"15min":0.91500000000000004},"min":{"1min":0.23699999999999999,"5min":0.193,"15min":0.193},"max":{"1min":9.4420000000000002,"5min":11.672000000000001,"15min":11.672000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.1919999999999999,"15min":0.90900000000000003},"min":{"1min":0.26000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.3440000000000003,"5min":11.722,"15min":11.722},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3719999999999999,"5min":1.901,"15min":1.171},"min":{"1min":0.26300000000000001,"5min":0.22,"15min":0.22},"max":{"1min":10.151999999999999,"5min":16.507999999999999,"15min":16.507999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.4790000000000001,"15min":1.0029999999999999},"min":{"1min":0.25600000000000001,"5min":0.218,"15min":0.218},"max":{"1min":5.024,"5min":12.047000000000001,"15min":12.047000000000001},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.6160000000000001,"5min":2.1040000000000001,"15min":1.2649999999999999},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":10.176,"5min":22.234000000000002,"15min":22.234000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":2.4510000000000001,"5min":2.2080000000000002,"15min":1.381},"min":{"1min":0.377,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":10.77,"5min":20.448,"15min":20.448},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.649,"15min":1.1910000000000001},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.7839999999999998,"5min":15.788,"15min":15.788},"last":0.58999999999999997},{"interface":"front","average":{"1min":2.105,"5min":1.8180000000000001,"15min":1.256},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":9.3000000000000007,"5min":11.731999999999999,"15min":11.731999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.4119999999999999,"15min":1.141},"min":{"1min":0.42499999999999999,"5min":0.255,"15min":0.255},"max":{"1min":9.1760000000000002,"5min":12.019,"15min":12.019},"last":0.72199999999999998},{"interface":"front","average":{"1min":2.1760000000000002,"5min":1.768,"15min":1.258},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":9.3339999999999996,"5min":11.628,"15min":11.628},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3809999999999998,"5min":1.9279999999999999,"15min":1.371},"min":{"1min":0.435,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":8.9689999999999994,"5min":21.161000000000001,"15min":21.161000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":4.2720000000000002,"5min":2.504,"15min":1.552},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":29.323,"5min":29.323,"15min":29.323},"last":0.78900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575209,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799540,"kb_used_data":9580,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572300,"statfs":{"total":96636764160,"available":95818035200,"internally_reserved":0,"allocated":9809920,"data_stored":9412264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":808909256},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":1.474,"15min":0.92700000000000005},"min":{"1min":0.23300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":2.5870000000000002,"5min":59.031999999999996,"15min":59.031999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":2.0249999999999999,"5min":2.222,"15min":1.196},"min":{"1min":0.20200000000000001,"5min":0.188,"15min":0.188},"max":{"1min":9.7110000000000003,"5min":57.186999999999998,"15min":57.186999999999998},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":0.93799999999999994,"15min":0.81200000000000006},"min":{"1min":0.28000000000000003,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":4.0919999999999996,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":2.0960000000000001,"5min":1.1899999999999999,"15min":0.84699999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":12.052,"5min":12.052,"15min":12.052},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.901,"5min":2.5739999999999998,"15min":1.353},"min":{"1min":0.26100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.6769999999999996,"5min":65.215000000000003,"15min":65.215000000000003},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.03,"5min":1.679,"15min":1.133},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.319,"5min":62.722999999999999,"15min":62.722999999999999},"last":0.84599999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":3.23,"5min":1.857,"15min":1.202},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":14.694000000000001,"5min":14.694000000000001,"15min":14.694000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.994,"5min":1.3360000000000001,"15min":1.095},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":9.3499999999999996,"5min":10.036,"15min":10.036},"last":0.92200000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":2.8300000000000001,"5min":1.748,"15min":1.1919999999999999},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":21.940999999999999,"5min":21.940999999999999,"15min":21.940999999999999},"last":1.038},{"interface":"front","average":{"1min":2.1440000000000001,"5min":1.962,"15min":1.3029999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.0620000000000003,"5min":9.3420000000000005,"15min":9.3420000000000005},"last":0.89900000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.7410000000000001,"5min":1.242,"15min":1.119},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":9.3840000000000003,"5min":9.3840000000000003,"15min":9.3840000000000003},"last":1.0229999999999999},{"interface":"front","average":{"1min":2.012,"5min":1.458,"15min":1.161},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.7060000000000004,"5min":8.157,"15min":8.157},"last":1.3560000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6140000000000001,"15min":1.2589999999999999},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":5.3209999999999997,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":1.2749999999999999},{"interface":"front","average":{"1min":2.601,"5min":2.677,"15min":1.5920000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40300000000000002},"max":{"1min":8.7210000000000001,"5min":32.069000000000003,"15min":32.069000000000003},"last":1.1910000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":84545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8482816,"data_stored":8473825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7000064,"data_stored":6984672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6995968,"data_stored":6985218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10542867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4296704,"data_stored":4287702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:17:44.033 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T17:17:44.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:17:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.185+0000 7f09e1747640 1 Processor -- start 2024-09-13T17:17:44.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.185+0000 7f09e1747640 1 -- start start 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.185+0000 7f09e1747640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09dc0611c0 0x7f09dc0bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.185+0000 7f09e1747640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09dc0c00b0 con 0x7f09dc06cbc0 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.185+0000 7f09e1747640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09dc153e70 con 0x7f09dc0c2db0 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.185+0000 7f09e1747640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09dc0c0300 con 0x7f09dc0611c0 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09da7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f09dc06cbc0 0x7f09dc1545e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40204/0 (socket says 172.21.15.39:40204) 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09da7fc640 1 -- 172.21.15.39:0/561485751 learned_addr learned my addr 172.21.15.39:0/561485751 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09db7fe640 1 --2- 172.21.15.39:0/561485751 >> v2:172.21.15.84:3300/0 conn(0x7f09dc0611c0 0x7f09dc0bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057208699 0 0) 0x7f09dc0c00b0 con 0x7f09dc06cbc0 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c4003530 con 0x7f09dc06cbc0 2024-09-13T17:17:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1320436516 0 0) 0x7f09c4003530 con 0x7f09dc06cbc0 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 >> v2:172.21.15.84:3300/0 conn(0x7f09dc0611c0 msgr2=0x7f09dc0bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 --2- 172.21.15.39:0/561485751 >> v2:172.21.15.84:3300/0 conn(0x7f09dc0611c0 0x7f09dc0bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 >> v1:172.21.15.84:6789/0 conn(0x7f09dc0c2db0 legacy=0x7f09dc1512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc0c05e0 con 0x7f09dc06cbc0 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f09d0001680 con 0x7f09dc06cbc0 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 >> v1:172.21.15.39:6789/0 conn(0x7f09dc06cbc0 legacy=0x7f09dc1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 --2- 172.21.15.39:0/561485751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c4003980 0x7f09c4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 --2- 172.21.15.39:0/561485751 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09c40047d0 0x7f09c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 --2- 172.21.15.39:0/561485751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c40052b0 0x7f09c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc0c0300 con 0x7f09c40052b0 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c400c180 con 0x7f09c4003980 2024-09-13T17:17:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c400c340 con 0x7f09c40047d0 2024-09-13T17:17:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09db7fe640 1 --2- 172.21.15.39:0/561485751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c40052b0 0x7f09c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09db7fe640 1 -- 172.21.15.39:0/561485751 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09c40047d0 msgr2=0x7f09c4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09db7fe640 1 --2- 172.21.15.39:0/561485751 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09c40047d0 0x7f09c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09db7fe640 1 -- 172.21.15.39:0/561485751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c4003980 msgr2=0x7f09c4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09db7fe640 1 --2- 172.21.15.39:0/561485751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09c4003980 0x7f09c4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09db7fe640 1 -- 172.21.15.39:0/561485751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c4003530 con 0x7f09c40052b0 2024-09-13T17:17:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09db7fe640 1 -- 172.21.15.39:0/561485751 reap_dead start 2024-09-13T17:17:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09db7fe640 1 --2- 172.21.15.39:0/561485751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c40052b0 0x7f09c400b990 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009ab0 tx=0x7f09d401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623b5477c1613e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4004020 con 0x7f09c40052b0 2024-09-13T17:17:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d4004180 con 0x7f09c40052b0 2024-09-13T17:17:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09d9ffb640 1 -- 172.21.15.39:0/561485751 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4003620 con 0x7f09c40052b0 2024-09-13T17:17:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09e1747640 1 -- 172.21.15.39:0/561485751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c40052b0 msgr2=0x7f09c400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.189+0000 7f09e1747640 1 --2- 172.21.15.39:0/561485751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c40052b0 0x7f09c400b990 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009ab0 tx=0x7f09d401ca90 comp rx=0 tx=0).stop 2024-09-13T17:17:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- 172.21.15.39:0/561485751 shutdown_connections 2024-09-13T17:17:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 --2- 172.21.15.39:0/561485751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c40052b0 0x7f09c400b990 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- 172.21.15.39:0/561485751 >> 172.21.15.39:0/561485751 conn(0x7f09dc069e10 msgr2=0x7f09dc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- 172.21.15.39:0/561485751 shutdown_connections 2024-09-13T17:17:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- 172.21.15.39:0/561485751 wait complete. 2024-09-13T17:17:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 Processor -- start 2024-09-13T17:17:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- start start 2024-09-13T17:17:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09c40052b0 0x7f09dc14fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09dc150230 0x7f09dc0c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09dc143cf0 0x7f09dc0c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc143b30 con 0x7f09dc150230 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc1ecd40 con 0x7f09dc143cf0 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc1ed1d0 con 0x7f09c40052b0 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09da7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09dc150230 0x7f09dc0c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09da7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09dc150230 0x7f09dc0c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40600/0 (socket says 172.21.15.39:40600) 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09da7fc640 1 -- 172.21.15.39:0/2702915835 learned_addr learned my addr 172.21.15.39:0/2702915835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09db7fe640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09dc143cf0 0x7f09dc0c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09daffd640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09c40052b0 0x7f09dc14fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09da7fc640 1 -- 172.21.15.39:0/2702915835 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09c40052b0 msgr2=0x7f09dc14fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09da7fc640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09c40052b0 0x7f09dc14fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09da7fc640 1 -- 172.21.15.39:0/2702915835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09dc143cf0 msgr2=0x7f09dc0c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09da7fc640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09dc143cf0 0x7f09dc0c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09da7fc640 1 -- 172.21.15.39:0/2702915835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc144260 con 0x7f09dc150230 2024-09-13T17:17:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09da7fc640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09dc150230 0x7f09dc0c5130 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f09d000c6c0 tx=0x7f09d000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09c37fe640 1 -- 172.21.15.39:0/2702915835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d00072d0 con 0x7f09dc150230 2024-09-13T17:17:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09c37fe640 1 -- 172.21.15.39:0/2702915835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d0007430 con 0x7f09dc150230 2024-09-13T17:17:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09c37fe640 1 -- 172.21.15.39:0/2702915835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d00047f0 con 0x7f09dc150230 2024-09-13T17:17:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc1ec410 con 0x7f09dc150230 2024-09-13T17:17:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.193+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc1ec9d0 con 0x7f09dc150230 2024-09-13T17:17:44.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.197+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc1ecb90 con 0x7f09dc150230 2024-09-13T17:17:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.197+0000 7f09c37fe640 1 -- 172.21.15.39:0/2702915835 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09d000a640 con 0x7f09dc150230 2024-09-13T17:17:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.197+0000 7f09c37fe640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09c80666c0 0x7f09c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.197+0000 7f09c37fe640 1 -- 172.21.15.39:0/2702915835 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7f09d00848e0 con 0x7f09dc150230 2024-09-13T17:17:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.197+0000 7f09daffd640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09c80666c0 0x7f09c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.197+0000 7f09daffd640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09c80666c0 0x7f09c8068b80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f09cc001cc0 tx=0x7f09cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:44.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.201+0000 7f09c37fe640 1 -- 172.21.15.39:0/2702915835 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09d0050bf0 con 0x7f09dc150230 2024-09-13T17:17:44.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.353+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09dc0c33a0 con 0x7f09c80666c0 2024-09-13T17:17:44.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.381+0000 7f09c37fe640 1 -- 172.21.15.39:0/2702915835 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528643 (secure 0 0 0) 0x7f09cc00d890 con 0x7f09c80666c0 2024-09-13T17:17:44.388 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:44.394 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T17:17:44.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09c80666c0 msgr2=0x7f09c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09c80666c0 0x7f09c8068b80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f09cc001cc0 tx=0x7f09cc009040 comp rx=0 tx=0).stop 2024-09-13T17:17:44.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09dc150230 msgr2=0x7f09dc0c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09dc150230 0x7f09dc0c5130 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f09d000c6c0 tx=0x7f09d000cb80 comp rx=0 tx=0).stop 2024-09-13T17:17:44.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 shutdown_connections 2024-09-13T17:17:44.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f09c80666c0 0x7f09c8068b80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09dc143cf0 0x7f09dc0c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09dc150230 0x7f09dc0c5130 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 --2- 172.21.15.39:0/2702915835 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f09c40052b0 0x7f09dc14fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 >> 172.21.15.39:0/2702915835 conn(0x7f09dc069e10 msgr2=0x7f09dc06cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:44.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 shutdown_connections 2024-09-13T17:17:44.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.389+0000 7f09e1747640 1 -- 172.21.15.39:0/2702915835 wait complete. 2024-09-13T17:17:44.409 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":954,"stamp":"2024-09-13T17:17:44.346474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39268818,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":3083,"num_read_kb":397491,"num_write":2818764,"num_write_kb":37531879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522488,"ondisk_log_size":522488,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6804280,"kb_used_data":83192,"kb_used_omap":12,"kb_used_meta":6721011,"kb_avail":748170440,"statfs":{"total":773094113280,"available":766126530560,"internally_reserved":0,"allocated":85188608,"data_stored":81998836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":6882315835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":429,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004228"},"pg_stats":[{"pgid":"5.18","version":"63'4385","reported_seq":4447,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058432+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T17:17:14.058432+0000","last_peered":"2024-09-13T17:17:14.058432+0000","last_clean":"2024-09-13T17:17:14.058432+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T17:17:14.058432+0000","last_undegraded":"2024-09-13T17:17:14.058432+0000","last_fullsized":"2024-09-13T17:17:14.058432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":23692,"num_write_kb":324330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"63'916","reported_seq":1025,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096259+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T17:17:22.096259+0000","last_peered":"2024-09-13T17:17:22.096259+0000","last_clean":"2024-09-13T17:17:22.096259+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T17:17:22.096259+0000","last_undegraded":"2024-09-13T17:17:22.096259+0000","last_fullsized":"2024-09-13T17:17:22.096259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":6289,"num_write":894,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'3737","reported_seq":3804,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.100845+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T17:17:22.100845+0000","last_peered":"2024-09-13T17:17:22.100845+0000","last_clean":"2024-09-13T17:17:22.100845+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T17:17:22.100845+0000","last_undegraded":"2024-09-13T17:17:22.100845+0000","last_fullsized":"2024-09-13T17:17:22.100845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3737,"log_dups_size":0,"ondisk_log_size":3737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20454,"num_write_kb":270405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.021643+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T17:17:14.021643+0000","last_peered":"2024-09-13T17:17:14.021643+0000","last_clean":"2024-09-13T17:17:14.021643+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T17:17:14.021643+0000","last_undegraded":"2024-09-13T17:17:14.021643+0000","last_fullsized":"2024-09-13T17:17:14.021643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"63'4331","reported_seq":4394,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.043747+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T17:17:14.043747+0000","last_peered":"2024-09-13T17:17:14.043747+0000","last_clean":"2024-09-13T17:17:14.043747+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T17:17:14.043747+0000","last_undegraded":"2024-09-13T17:17:14.043747+0000","last_fullsized":"2024-09-13T17:17:14.043747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":22875,"num_write_kb":325320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"63'23","reported_seq":91,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096300+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T17:17:22.096300+0000","last_peered":"2024-09-13T17:17:22.096300+0000","last_clean":"2024-09-13T17:17:22.096300+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T17:17:22.096300+0000","last_undegraded":"2024-09-13T17:17:22.096300+0000","last_fullsized":"2024-09-13T17:17:22.096300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"63'5559","reported_seq":5626,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.132606+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T17:17:22.132606+0000","last_peered":"2024-09-13T17:17:22.132606+0000","last_clean":"2024-09-13T17:17:22.132606+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T17:17:22.132606+0000","last_undegraded":"2024-09-13T17:17:22.132606+0000","last_fullsized":"2024-09-13T17:17:22.132606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5559,"log_dups_size":0,"ondisk_log_size":5559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30370,"num_write_kb":379865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"63'2","reported_seq":64,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:09.573144+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T17:17:09.573144+0000","last_peered":"2024-09-13T17:17:09.573144+0000","last_clean":"2024-09-13T17:17:09.573144+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T17:17:09.573144+0000","last_undegraded":"2024-09-13T17:17:09.573144+0000","last_fullsized":"2024-09-13T17:17:09.573144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"63'3581","reported_seq":3641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.986895+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T17:17:13.986895+0000","last_peered":"2024-09-13T17:17:13.986895+0000","last_clean":"2024-09-13T17:17:13.986895+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T17:17:13.986895+0000","last_undegraded":"2024-09-13T17:17:13.986895+0000","last_fullsized":"2024-09-13T17:17:13.986895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":268344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"63'40","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.099149+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T17:17:22.099149+0000","last_peered":"2024-09-13T17:17:22.099149+0000","last_clean":"2024-09-13T17:17:22.099149+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T17:17:22.099149+0000","last_undegraded":"2024-09-13T17:17:22.099149+0000","last_fullsized":"2024-09-13T17:17:22.099149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'3493","reported_seq":3559,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.063521+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T17:17:22.063521+0000","last_peered":"2024-09-13T17:17:22.063521+0000","last_clean":"2024-09-13T17:17:22.063521+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T17:17:22.063521+0000","last_undegraded":"2024-09-13T17:17:22.063521+0000","last_fullsized":"2024-09-13T17:17:22.063521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18921,"num_write_kb":256078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'726","reported_seq":812,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.996155+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T17:17:13.996155+0000","last_peered":"2024-09-13T17:17:13.996155+0000","last_clean":"2024-09-13T17:17:13.996155+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T17:17:13.996155+0000","last_undegraded":"2024-09-13T17:17:13.996155+0000","last_fullsized":"2024-09-13T17:17:13.996155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":75389,"num_write":714,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"63'3135","reported_seq":3195,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.003272+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T17:17:14.003272+0000","last_peered":"2024-09-13T17:17:14.003272+0000","last_clean":"2024-09-13T17:17:14.003272+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T17:17:14.003272+0000","last_undegraded":"2024-09-13T17:17:14.003272+0000","last_fullsized":"2024-09-13T17:17:14.003272+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3135,"log_dups_size":0,"ondisk_log_size":3135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":232447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"63'5","reported_seq":106,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:16:38.283375+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T17:16:38.283375+0000","last_peered":"2024-09-13T17:16:38.283375+0000","last_clean":"2024-09-13T17:16:38.283375+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T17:16:38.283375+0000","last_undegraded":"2024-09-13T17:16:38.283375+0000","last_fullsized":"2024-09-13T17:16:38.283375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"63'5195","reported_seq":5259,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095099+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T17:17:22.095099+0000","last_peered":"2024-09-13T17:17:22.095099+0000","last_clean":"2024-09-13T17:17:22.095099+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T17:17:22.095099+0000","last_undegraded":"2024-09-13T17:17:22.095099+0000","last_fullsized":"2024-09-13T17:17:22.095099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29369,"num_write_kb":359470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"63'210","reported_seq":271,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:27.922744+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T17:17:27.922744+0000","last_peered":"2024-09-13T17:17:27.922744+0000","last_clean":"2024-09-13T17:17:27.922744+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T17:17:27.922744+0000","last_undegraded":"2024-09-13T17:17:27.922744+0000","last_fullsized":"2024-09-13T17:17:27.922744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153565,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"63'3570","reported_seq":3634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.050021+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T17:17:14.050021+0000","last_peered":"2024-09-13T17:17:14.050021+0000","last_clean":"2024-09-13T17:17:14.050021+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T17:17:14.050021+0000","last_undegraded":"2024-09-13T17:17:14.050021+0000","last_fullsized":"2024-09-13T17:17:14.050021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":19362,"num_write_kb":282506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"63'4274","reported_seq":4340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096597+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T17:17:22.096597+0000","last_peered":"2024-09-13T17:17:22.096597+0000","last_clean":"2024-09-13T17:17:22.096597+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T17:17:22.096597+0000","last_undegraded":"2024-09-13T17:17:22.096597+0000","last_fullsized":"2024-09-13T17:17:22.096597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23751,"num_write_kb":312338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"63'1418","reported_seq":1528,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.108614+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T17:17:22.108614+0000","last_peered":"2024-09-13T17:17:22.108614+0000","last_clean":"2024-09-13T17:17:22.108614+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T17:17:22.108614+0000","last_undegraded":"2024-09-13T17:17:22.108614+0000","last_fullsized":"2024-09-13T17:17:22.108614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1418,"log_dups_size":0,"ondisk_log_size":1418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":38147,"num_write":1396,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"63'6","reported_seq":90,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.030550+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T17:17:14.030550+0000","last_peered":"2024-09-13T17:17:14.030550+0000","last_clean":"2024-09-13T17:17:14.030550+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T17:17:14.030550+0000","last_undegraded":"2024-09-13T17:17:14.030550+0000","last_fullsized":"2024-09-13T17:17:14.030550+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"63'4212","reported_seq":4277,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.051215+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T17:17:14.051215+0000","last_peered":"2024-09-13T17:17:14.051215+0000","last_clean":"2024-09-13T17:17:14.051215+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T17:17:14.051215+0000","last_undegraded":"2024-09-13T17:17:14.051215+0000","last_fullsized":"2024-09-13T17:17:14.051215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:25.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":17,"num_write":23343,"num_write_kb":314959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"63'3296","reported_seq":3362,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.125915+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T17:17:22.125915+0000","last_peered":"2024-09-13T17:17:22.125915+0000","last_clean":"2024-09-13T17:17:22.125915+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T17:17:22.125915+0000","last_undegraded":"2024-09-13T17:17:22.125915+0000","last_fullsized":"2024-09-13T17:17:22.125915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3296,"log_dups_size":0,"ondisk_log_size":3296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17758,"num_write_kb":230990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'3","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.059668+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T17:17:22.059668+0000","last_peered":"2024-09-13T17:17:22.059668+0000","last_clean":"2024-09-13T17:17:22.059668+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T17:17:22.059668+0000","last_undegraded":"2024-09-13T17:17:22.059668+0000","last_fullsized":"2024-09-13T17:17:22.059668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'5","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.027550+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T17:17:14.027550+0000","last_peered":"2024-09-13T17:17:14.027550+0000","last_clean":"2024-09-13T17:17:14.027550+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T17:17:14.027550+0000","last_undegraded":"2024-09-13T17:17:14.027550+0000","last_fullsized":"2024-09-13T17:17:14.027550+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"63'3371","reported_seq":3434,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022230+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T17:17:14.022230+0000","last_peered":"2024-09-13T17:17:14.022230+0000","last_clean":"2024-09-13T17:17:14.022230+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T17:17:14.022230+0000","last_undegraded":"2024-09-13T17:17:14.022230+0000","last_fullsized":"2024-09-13T17:17:14.022230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3371,"log_dups_size":0,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18156,"num_write_kb":249012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"63'3752","reported_seq":3818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.984671+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T17:17:21.984671+0000","last_peered":"2024-09-13T17:17:21.984671+0000","last_clean":"2024-09-13T17:17:21.984671+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T17:17:21.984671+0000","last_undegraded":"2024-09-13T17:17:21.984671+0000","last_fullsized":"2024-09-13T17:17:21.984671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21127,"num_write_kb":268998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"63'2","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.056813+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T17:17:22.056813+0000","last_peered":"2024-09-13T17:17:22.056813+0000","last_clean":"2024-09-13T17:17:22.056813+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T17:17:22.056813+0000","last_undegraded":"2024-09-13T17:17:22.056813+0000","last_fullsized":"2024-09-13T17:17:22.056813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"63'4","reported_seq":88,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.029328+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T17:17:14.029328+0000","last_peered":"2024-09-13T17:17:14.029328+0000","last_clean":"2024-09-13T17:17:14.029328+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T17:17:14.029328+0000","last_undegraded":"2024-09-13T17:17:14.029328+0000","last_fullsized":"2024-09-13T17:17:14.029328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"63'4499","reported_seq":4565,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.023958+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T17:17:14.023958+0000","last_peered":"2024-09-13T17:17:14.023958+0000","last_clean":"2024-09-13T17:17:14.023958+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T17:17:14.023958+0000","last_undegraded":"2024-09-13T17:17:14.023958+0000","last_fullsized":"2024-09-13T17:17:14.023958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":24341,"num_write_kb":343418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"63'4043","reported_seq":4109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.908339+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T17:17:21.908339+0000","last_peered":"2024-09-13T17:17:21.908339+0000","last_clean":"2024-09-13T17:17:21.908339+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T17:17:21.908339+0000","last_undegraded":"2024-09-13T17:17:21.908339+0000","last_fullsized":"2024-09-13T17:17:21.908339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22005,"num_write_kb":288910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"63'5","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.010552+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T17:17:22.010552+0000","last_peered":"2024-09-13T17:17:22.010552+0000","last_clean":"2024-09-13T17:17:22.010552+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T17:17:22.010552+0000","last_undegraded":"2024-09-13T17:17:22.010552+0000","last_fullsized":"2024-09-13T17:17:22.010552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"63'7","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024599+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T17:17:14.024599+0000","last_peered":"2024-09-13T17:17:14.024599+0000","last_clean":"2024-09-13T17:17:14.024599+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T17:17:14.024599+0000","last_undegraded":"2024-09-13T17:17:14.024599+0000","last_fullsized":"2024-09-13T17:17:14.024599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"63'3470","reported_seq":3533,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.032577+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T17:17:14.032577+0000","last_peered":"2024-09-13T17:17:14.032577+0000","last_clean":"2024-09-13T17:17:14.032577+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T17:17:14.032577+0000","last_undegraded":"2024-09-13T17:17:14.032577+0000","last_fullsized":"2024-09-13T17:17:14.032577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19441,"num_write_kb":258288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'4844","reported_seq":4911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.060853+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T17:17:22.060853+0000","last_peered":"2024-09-13T17:17:22.060853+0000","last_clean":"2024-09-13T17:17:22.060853+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T17:17:22.060853+0000","last_undegraded":"2024-09-13T17:17:22.060853+0000","last_fullsized":"2024-09-13T17:17:22.060853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":26635,"num_write_kb":336207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"63'9","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.115612+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T17:17:22.115612+0000","last_peered":"2024-09-13T17:17:22.115612+0000","last_clean":"2024-09-13T17:17:22.115612+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T17:17:22.115612+0000","last_undegraded":"2024-09-13T17:17:22.115612+0000","last_fullsized":"2024-09-13T17:17:22.115612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"63'48","reported_seq":175,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:24.143768+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T17:17:24.143768+0000","last_peered":"2024-09-13T17:17:24.143768+0000","last_clean":"2024-09-13T17:17:24.143768+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T17:17:24.143768+0000","last_undegraded":"2024-09-13T17:17:24.143768+0000","last_fullsized":"2024-09-13T17:17:24.143768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92726,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":118,"num_write":43,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"63'3635","reported_seq":3699,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.001656+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T17:17:14.001656+0000","last_peered":"2024-09-13T17:17:14.001656+0000","last_clean":"2024-09-13T17:17:14.001656+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T17:17:14.001656+0000","last_undegraded":"2024-09-13T17:17:14.001656+0000","last_fullsized":"2024-09-13T17:17:14.001656+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19787,"num_write_kb":259620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'3639","reported_seq":3705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.121330+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T17:17:22.121330+0000","last_peered":"2024-09-13T17:17:22.121330+0000","last_clean":"2024-09-13T17:17:22.121330+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T17:17:22.121330+0000","last_undegraded":"2024-09-13T17:17:22.121330+0000","last_fullsized":"2024-09-13T17:17:22.121330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3639,"log_dups_size":0,"ondisk_log_size":3639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20066,"num_write_kb":240267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"63'265","reported_seq":350,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.104288+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T17:17:22.104288+0000","last_peered":"2024-09-13T17:17:22.104288+0000","last_clean":"2024-09-13T17:17:22.104288+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T17:17:22.104288+0000","last_undegraded":"2024-09-13T17:17:22.104288+0000","last_fullsized":"2024-09-13T17:17:22.104288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":46,"num_write":634,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"63'11","reported_seq":75,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.026249+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T17:17:14.026249+0000","last_peered":"2024-09-13T17:17:14.026249+0000","last_clean":"2024-09-13T17:17:14.026249+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T17:17:14.026249+0000","last_undegraded":"2024-09-13T17:17:14.026249+0000","last_fullsized":"2024-09-13T17:17:14.026249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"63'3190","reported_seq":3253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.040510+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T17:17:14.040510+0000","last_peered":"2024-09-13T17:17:14.040510+0000","last_clean":"2024-09-13T17:17:14.040510+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T17:17:14.040510+0000","last_undegraded":"2024-09-13T17:17:14.040510+0000","last_fullsized":"2024-09-13T17:17:14.040510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3190,"log_dups_size":0,"ondisk_log_size":3190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":254866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"63'4701","reported_seq":4768,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.071944+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T17:17:22.071944+0000","last_peered":"2024-09-13T17:17:22.071944+0000","last_clean":"2024-09-13T17:17:22.071944+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T17:17:22.071944+0000","last_undegraded":"2024-09-13T17:17:22.071944+0000","last_fullsized":"2024-09-13T17:17:22.071944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26171,"num_write_kb":338588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'2","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:19.802813+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T17:17:19.802813+0000","last_peered":"2024-09-13T17:17:19.802813+0000","last_clean":"2024-09-13T17:17:19.802813+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T17:17:19.802813+0000","last_undegraded":"2024-09-13T17:17:19.802813+0000","last_fullsized":"2024-09-13T17:17:19.802813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"63'3","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.988743+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T17:17:13.988743+0000","last_peered":"2024-09-13T17:17:13.988743+0000","last_clean":"2024-09-13T17:17:13.988743+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T17:17:13.988743+0000","last_undegraded":"2024-09-13T17:17:13.988743+0000","last_fullsized":"2024-09-13T17:17:13.988743+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"63'3859","reported_seq":3922,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.945566+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T17:17:13.945566+0000","last_peered":"2024-09-13T17:17:13.945566+0000","last_clean":"2024-09-13T17:17:13.945566+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T17:17:13.945566+0000","last_undegraded":"2024-09-13T17:17:13.945566+0000","last_fullsized":"2024-09-13T17:17:13.945566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21286,"num_write_kb":290222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"63'3876","reported_seq":3943,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.089484+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T17:17:22.089484+0000","last_peered":"2024-09-13T17:17:22.089484+0000","last_clean":"2024-09-13T17:17:22.089484+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T17:17:22.089484+0000","last_undegraded":"2024-09-13T17:17:22.089484+0000","last_fullsized":"2024-09-13T17:17:22.089484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21201,"num_write_kb":273354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"63'2","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.094910+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T17:17:22.094910+0000","last_peered":"2024-09-13T17:17:22.094910+0000","last_clean":"2024-09-13T17:17:22.094910+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T17:17:22.094910+0000","last_undegraded":"2024-09-13T17:17:22.094910+0000","last_fullsized":"2024-09-13T17:17:22.094910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'5","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.944152+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T17:17:12.944152+0000","last_peered":"2024-09-13T17:17:12.944152+0000","last_clean":"2024-09-13T17:17:12.944152+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T17:17:12.944152+0000","last_undegraded":"2024-09-13T17:17:12.944152+0000","last_fullsized":"2024-09-13T17:17:12.944152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"63'3422","reported_seq":3486,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058428+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T17:17:14.058428+0000","last_peered":"2024-09-13T17:17:14.058428+0000","last_clean":"2024-09-13T17:17:14.058428+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T17:17:14.058428+0000","last_undegraded":"2024-09-13T17:17:14.058428+0000","last_fullsized":"2024-09-13T17:17:14.058428+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":7,"num_write":19042,"num_write_kb":260722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"63'4802","reported_seq":4869,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.992037+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T17:17:21.992037+0000","last_peered":"2024-09-13T17:17:21.992037+0000","last_clean":"2024-09-13T17:17:21.992037+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T17:17:21.992037+0000","last_undegraded":"2024-09-13T17:17:21.992037+0000","last_fullsized":"2024-09-13T17:17:21.992037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26419,"num_write_kb":335470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"63'4","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.258399+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T17:17:14.258399+0000","last_peered":"2024-09-13T17:17:14.258399+0000","last_clean":"2024-09-13T17:17:14.258399+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T17:17:14.258399+0000","last_undegraded":"2024-09-13T17:17:14.258399+0000","last_fullsized":"2024-09-13T17:17:14.258399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"63'2","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.026552+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T17:17:14.026552+0000","last_peered":"2024-09-13T17:17:14.026552+0000","last_clean":"2024-09-13T17:17:14.026552+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T17:17:14.026552+0000","last_undegraded":"2024-09-13T17:17:14.026552+0000","last_fullsized":"2024-09-13T17:17:14.026552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"63'4215","reported_seq":4278,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.984815+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T17:17:13.984815+0000","last_peered":"2024-09-13T17:17:13.984815+0000","last_clean":"2024-09-13T17:17:13.984815+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T17:17:13.984815+0000","last_undegraded":"2024-09-13T17:17:13.984815+0000","last_fullsized":"2024-09-13T17:17:13.984815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:06.188971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":8,"num_write":22461,"num_write_kb":319331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'3790","reported_seq":3857,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107689+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T17:17:22.107689+0000","last_peered":"2024-09-13T17:17:22.107689+0000","last_clean":"2024-09-13T17:17:22.107689+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T17:17:22.107689+0000","last_undegraded":"2024-09-13T17:17:22.107689+0000","last_fullsized":"2024-09-13T17:17:22.107689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21077,"num_write_kb":260732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"63'40","reported_seq":162,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.062885+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T17:17:22.062885+0000","last_peered":"2024-09-13T17:17:22.062885+0000","last_clean":"2024-09-13T17:17:22.062885+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T17:17:22.062885+0000","last_undegraded":"2024-09-13T17:17:22.062885+0000","last_fullsized":"2024-09-13T17:17:22.062885+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84545,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":54,"num_read_kb":105,"num_write":36,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"63'9","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.040704+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T17:17:14.040704+0000","last_peered":"2024-09-13T17:17:14.040704+0000","last_clean":"2024-09-13T17:17:14.040704+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T17:17:14.040704+0000","last_undegraded":"2024-09-13T17:17:14.040704+0000","last_fullsized":"2024-09-13T17:17:14.040704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"63'3128","reported_seq":3190,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.042016+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T17:17:14.042016+0000","last_peered":"2024-09-13T17:17:14.042016+0000","last_clean":"2024-09-13T17:17:14.042016+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T17:17:14.042016+0000","last_undegraded":"2024-09-13T17:17:14.042016+0000","last_fullsized":"2024-09-13T17:17:14.042016+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3128,"log_dups_size":0,"ondisk_log_size":3128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17776,"num_write_kb":243283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"63'3657","reported_seq":3724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.977555+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T17:17:21.977555+0000","last_peered":"2024-09-13T17:17:21.977555+0000","last_clean":"2024-09-13T17:17:21.977555+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T17:17:21.977555+0000","last_undegraded":"2024-09-13T17:17:21.977555+0000","last_fullsized":"2024-09-13T17:17:21.977555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19816,"num_write_kb":255512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'10","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.101605+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T17:17:22.101605+0000","last_peered":"2024-09-13T17:17:22.101605+0000","last_clean":"2024-09-13T17:17:22.101605+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T17:17:22.101605+0000","last_undegraded":"2024-09-13T17:17:22.101605+0000","last_fullsized":"2024-09-13T17:17:22.101605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'251","reported_seq":335,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.918664+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T17:17:22.918664+0000","last_peered":"2024-09-13T17:17:22.918664+0000","last_clean":"2024-09-13T17:17:22.918664+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T17:17:22.918664+0000","last_undegraded":"2024-09-13T17:17:22.918664+0000","last_fullsized":"2024-09-13T17:17:22.918664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":57,"num_write":597,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"63'4141","reported_seq":4205,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022107+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T17:17:14.022107+0000","last_peered":"2024-09-13T17:17:14.022107+0000","last_clean":"2024-09-13T17:17:14.022107+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T17:17:14.022107+0000","last_undegraded":"2024-09-13T17:17:14.022107+0000","last_fullsized":"2024-09-13T17:17:14.022107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22849,"num_write_kb":317008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'4103","reported_seq":4178,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.906868+0000","last_change":"2024-09-13T16:48:41.389774+0000","last_active":"2024-09-13T17:17:21.906868+0000","last_peered":"2024-09-13T17:17:21.906868+0000","last_clean":"2024-09-13T17:17:21.906868+0000","last_became_active":"2024-09-13T16:48:41.389507+0000","last_became_peered":"2024-09-13T16:48:41.389507+0000","last_unstale":"2024-09-13T17:17:21.906868+0000","last_undegraded":"2024-09-13T17:17:21.906868+0000","last_fullsized":"2024-09-13T17:17:21.906868+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:07:23.114010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22342,"num_write_kb":278194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"63'3","reported_seq":71,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058621+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T17:17:22.058621+0000","last_peered":"2024-09-13T17:17:22.058621+0000","last_clean":"2024-09-13T17:17:22.058621+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T17:17:22.058621+0000","last_undegraded":"2024-09-13T17:17:22.058621+0000","last_fullsized":"2024-09-13T17:17:22.058621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T17:16:04.972860+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T17:16:04.972860+0000","last_peered":"2024-09-13T17:16:04.972860+0000","last_clean":"2024-09-13T17:16:04.972860+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T17:16:04.972860+0000","last_undegraded":"2024-09-13T17:16:04.972860+0000","last_fullsized":"2024-09-13T17:16:04.972860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"63'2","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.946275+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T17:17:12.946275+0000","last_peered":"2024-09-13T17:17:12.946275+0000","last_clean":"2024-09-13T17:17:12.946275+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T17:17:12.946275+0000","last_undegraded":"2024-09-13T17:17:12.946275+0000","last_fullsized":"2024-09-13T17:17:12.946275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"63'2904","reported_seq":2967,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058460+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T17:17:14.058460+0000","last_peered":"2024-09-13T17:17:14.058460+0000","last_clean":"2024-09-13T17:17:14.058460+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T17:17:14.058460+0000","last_undegraded":"2024-09-13T17:17:14.058460+0000","last_fullsized":"2024-09-13T17:17:14.058460+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:32:00.189353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":2,"num_write":15607,"num_write_kb":214254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'4284","reported_seq":4350,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117903+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T17:17:22.117903+0000","last_peered":"2024-09-13T17:17:22.117903+0000","last_clean":"2024-09-13T17:17:22.117903+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T17:17:22.117903+0000","last_undegraded":"2024-09-13T17:17:22.117903+0000","last_fullsized":"2024-09-13T17:17:22.117903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23392,"num_write_kb":310963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"63'5","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096955+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T17:17:22.096955+0000","last_peered":"2024-09-13T17:17:22.096955+0000","last_clean":"2024-09-13T17:17:22.096955+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T17:17:22.096955+0000","last_undegraded":"2024-09-13T17:17:22.096955+0000","last_fullsized":"2024-09-13T17:17:22.096955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"63'3","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.991304+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T17:17:13.991304+0000","last_peered":"2024-09-13T17:17:13.991304+0000","last_clean":"2024-09-13T17:17:13.991304+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T17:17:13.991304+0000","last_undegraded":"2024-09-13T17:17:13.991304+0000","last_fullsized":"2024-09-13T17:17:13.991304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"63'4138","reported_seq":4200,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.019219+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T17:17:14.019219+0000","last_peered":"2024-09-13T17:17:14.019219+0000","last_clean":"2024-09-13T17:17:14.019219+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T17:17:14.019219+0000","last_undegraded":"2024-09-13T17:17:14.019219+0000","last_fullsized":"2024-09-13T17:17:14.019219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:29.309105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":309086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"63'4584","reported_seq":4643,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.128999+0000","last_change":"2024-09-13T16:48:40.767044+0000","last_active":"2024-09-13T17:17:22.128999+0000","last_peered":"2024-09-13T17:17:22.128999+0000","last_clean":"2024-09-13T17:17:22.128999+0000","last_became_active":"2024-09-13T16:48:40.765861+0000","last_became_peered":"2024-09-13T16:48:40.765861+0000","last_unstale":"2024-09-13T17:17:22.128999+0000","last_undegraded":"2024-09-13T17:17:22.128999+0000","last_fullsized":"2024-09-13T17:17:22.128999+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:39.408967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24690,"num_write_kb":320186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"63'6","reported_seq":91,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.999257+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T17:17:21.999257+0000","last_peered":"2024-09-13T17:17:21.999257+0000","last_clean":"2024-09-13T17:17:21.999257+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T17:17:21.999257+0000","last_undegraded":"2024-09-13T17:17:21.999257+0000","last_fullsized":"2024-09-13T17:17:21.999257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'1425","reported_seq":1536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.059648+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T17:17:14.059648+0000","last_peered":"2024-09-13T17:17:14.059648+0000","last_clean":"2024-09-13T17:17:14.059648+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T17:17:14.059648+0000","last_undegraded":"2024-09-13T17:17:14.059648+0000","last_fullsized":"2024-09-13T17:17:14.059648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":38459,"num_write":1400,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"63'3637","reported_seq":3700,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.967851+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T17:17:13.967851+0000","last_peered":"2024-09-13T17:17:13.967851+0000","last_clean":"2024-09-13T17:17:13.967851+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T17:17:13.967851+0000","last_undegraded":"2024-09-13T17:17:13.967851+0000","last_fullsized":"2024-09-13T17:17:13.967851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19635,"num_write_kb":259018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'5067","reported_seq":5126,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.041483+0000","last_change":"2024-09-13T16:48:40.831954+0000","last_active":"2024-09-13T17:17:22.041483+0000","last_peered":"2024-09-13T17:17:22.041483+0000","last_clean":"2024-09-13T17:17:22.041483+0000","last_became_active":"2024-09-13T16:48:40.831696+0000","last_became_peered":"2024-09-13T16:48:40.831696+0000","last_unstale":"2024-09-13T17:17:22.041483+0000","last_undegraded":"2024-09-13T17:17:22.041483+0000","last_fullsized":"2024-09-13T17:17:22.041483+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:53.452702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27028,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"63'7","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.064989+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T17:17:22.064989+0000","last_peered":"2024-09-13T17:17:22.064989+0000","last_clean":"2024-09-13T17:17:22.064989+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T17:17:22.064989+0000","last_undegraded":"2024-09-13T17:17:22.064989+0000","last_fullsized":"2024-09-13T17:17:22.064989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"63'5","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.951351+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T17:17:12.951351+0000","last_peered":"2024-09-13T17:17:12.951351+0000","last_clean":"2024-09-13T17:17:12.951351+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T17:17:12.951351+0000","last_undegraded":"2024-09-13T17:17:12.951351+0000","last_fullsized":"2024-09-13T17:17:12.951351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"63'2915","reported_seq":2977,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.931098+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T17:17:13.931098+0000","last_peered":"2024-09-13T17:17:13.931098+0000","last_clean":"2024-09-13T17:17:13.931098+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T17:17:13.931098+0000","last_undegraded":"2024-09-13T17:17:13.931098+0000","last_fullsized":"2024-09-13T17:17:13.931098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2915,"log_dups_size":0,"ondisk_log_size":2915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:17.259082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":234494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"63'3697","reported_seq":3763,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.091815+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T17:17:22.091815+0000","last_peered":"2024-09-13T17:17:22.091815+0000","last_clean":"2024-09-13T17:17:22.091815+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T17:17:22.091815+0000","last_undegraded":"2024-09-13T17:17:22.091815+0000","last_fullsized":"2024-09-13T17:17:22.091815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20206,"num_write_kb":253140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"63'4","reported_seq":89,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107701+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T17:17:22.107701+0000","last_peered":"2024-09-13T17:17:22.107701+0000","last_clean":"2024-09-13T17:17:22.107701+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T17:17:22.107701+0000","last_undegraded":"2024-09-13T17:17:22.107701+0000","last_fullsized":"2024-09-13T17:17:22.107701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"63'2","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.991145+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T17:17:13.991145+0000","last_peered":"2024-09-13T17:17:13.991145+0000","last_clean":"2024-09-13T17:17:13.991145+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T17:17:13.991145+0000","last_undegraded":"2024-09-13T17:17:13.991145+0000","last_fullsized":"2024-09-13T17:17:13.991145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"63'3522","reported_seq":3583,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.999295+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T17:17:13.999295+0000","last_peered":"2024-09-13T17:17:13.999295+0000","last_clean":"2024-09-13T17:17:13.999295+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T17:17:13.999295+0000","last_undegraded":"2024-09-13T17:17:13.999295+0000","last_fullsized":"2024-09-13T17:17:13.999295+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18828,"num_write_kb":272277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"63'3874","reported_seq":3941,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.067632+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T17:17:22.067632+0000","last_peered":"2024-09-13T17:17:22.067632+0000","last_clean":"2024-09-13T17:17:22.067632+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T17:17:22.067632+0000","last_undegraded":"2024-09-13T17:17:22.067632+0000","last_fullsized":"2024-09-13T17:17:22.067632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21413,"num_write_kb":278431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"63'14","reported_seq":170,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095666+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T17:17:22.095666+0000","last_peered":"2024-09-13T17:17:22.095666+0000","last_clean":"2024-09-13T17:17:22.095666+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T17:17:22.095666+0000","last_undegraded":"2024-09-13T17:17:22.095666+0000","last_fullsized":"2024-09-13T17:17:22.095666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":270,"num_read_kb":270,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036103+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T17:17:14.036103+0000","last_peered":"2024-09-13T17:17:14.036103+0000","last_clean":"2024-09-13T17:17:14.036103+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T17:17:14.036103+0000","last_undegraded":"2024-09-13T17:17:14.036103+0000","last_fullsized":"2024-09-13T17:17:14.036103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"63'3622","reported_seq":3686,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058427+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T17:17:14.058427+0000","last_peered":"2024-09-13T17:17:14.058427+0000","last_clean":"2024-09-13T17:17:14.058427+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T17:17:14.058427+0000","last_undegraded":"2024-09-13T17:17:14.058427+0000","last_fullsized":"2024-09-13T17:17:14.058427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":19996,"num_write_kb":293269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"63'3240","reported_seq":3305,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.013971+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T17:17:22.013971+0000","last_peered":"2024-09-13T17:17:22.013971+0000","last_clean":"2024-09-13T17:17:22.013971+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T17:17:22.013971+0000","last_undegraded":"2024-09-13T17:17:22.013971+0000","last_fullsized":"2024-09-13T17:17:22.013971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3240,"log_dups_size":0,"ondisk_log_size":3240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":237245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"63'5","reported_seq":71,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.098797+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T17:17:22.098797+0000","last_peered":"2024-09-13T17:17:22.098797+0000","last_clean":"2024-09-13T17:17:22.098797+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T17:17:22.098797+0000","last_undegraded":"2024-09-13T17:17:22.098797+0000","last_fullsized":"2024-09-13T17:17:22.098797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'5","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.992012+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T17:17:13.992012+0000","last_peered":"2024-09-13T17:17:13.992012+0000","last_clean":"2024-09-13T17:17:13.992012+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T17:17:13.992012+0000","last_undegraded":"2024-09-13T17:17:13.992012+0000","last_fullsized":"2024-09-13T17:17:13.992012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"63'3376","reported_seq":3437,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.779058+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T17:17:13.779058+0000","last_peered":"2024-09-13T17:17:13.779058+0000","last_clean":"2024-09-13T17:17:13.779058+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T17:17:13.779058+0000","last_undegraded":"2024-09-13T17:17:13.779058+0000","last_fullsized":"2024-09-13T17:17:13.779058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18879,"num_write_kb":251906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"63'4848","reported_seq":4913,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.099538+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T17:17:22.099538+0000","last_peered":"2024-09-13T17:17:22.099538+0000","last_clean":"2024-09-13T17:17:22.099538+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T17:17:22.099538+0000","last_undegraded":"2024-09-13T17:17:22.099538+0000","last_fullsized":"2024-09-13T17:17:22.099538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26963,"num_write_kb":321221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"63'3","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117357+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T17:17:22.117357+0000","last_peered":"2024-09-13T17:17:22.117357+0000","last_clean":"2024-09-13T17:17:22.117357+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T17:17:22.117357+0000","last_undegraded":"2024-09-13T17:17:22.117357+0000","last_fullsized":"2024-09-13T17:17:22.117357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"63'389","reported_seq":558,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:09.910889+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T17:17:09.910889+0000","last_peered":"2024-09-13T17:17:09.910889+0000","last_clean":"2024-09-13T17:17:09.910889+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T17:17:09.910889+0000","last_undegraded":"2024-09-13T17:17:09.910889+0000","last_fullsized":"2024-09-13T17:17:09.910889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":191,"num_read_kb":82094,"num_write":385,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'4856","reported_seq":4919,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.025593+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T17:17:14.025593+0000","last_peered":"2024-09-13T17:17:14.025593+0000","last_clean":"2024-09-13T17:17:14.025593+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T17:17:14.025593+0000","last_undegraded":"2024-09-13T17:17:14.025593+0000","last_fullsized":"2024-09-13T17:17:14.025593+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":26453,"num_write_kb":368210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'4063","reported_seq":4128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.081187+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T17:17:22.081187+0000","last_peered":"2024-09-13T17:17:22.081187+0000","last_clean":"2024-09-13T17:17:22.081187+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T17:17:22.081187+0000","last_undegraded":"2024-09-13T17:17:22.081187+0000","last_fullsized":"2024-09-13T17:17:22.081187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4063,"log_dups_size":0,"ondisk_log_size":4063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21999,"num_write_kb":293018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"63'8","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.105763+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T17:17:22.105763+0000","last_peered":"2024-09-13T17:17:22.105763+0000","last_clean":"2024-09-13T17:17:22.105763+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T17:17:22.105763+0000","last_undegraded":"2024-09-13T17:17:22.105763+0000","last_fullsized":"2024-09-13T17:17:22.105763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":144,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'30","reported_seq":155,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.143808+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T17:17:14.143808+0000","last_peered":"2024-09-13T17:17:14.143808+0000","last_clean":"2024-09-13T17:17:14.143808+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T17:17:14.143808+0000","last_undegraded":"2024-09-13T17:17:14.143808+0000","last_fullsized":"2024-09-13T17:17:14.143808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":31,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"63'4239","reported_seq":4300,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.052220+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T17:17:14.052220+0000","last_peered":"2024-09-13T17:17:14.052220+0000","last_clean":"2024-09-13T17:17:14.052220+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T17:17:14.052220+0000","last_undegraded":"2024-09-13T17:17:14.052220+0000","last_fullsized":"2024-09-13T17:17:14.052220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23316,"num_write_kb":318961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'3718","reported_seq":3783,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.786167+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T17:17:21.786167+0000","last_peered":"2024-09-13T17:17:21.786167+0000","last_clean":"2024-09-13T17:17:21.786167+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T17:17:21.786167+0000","last_undegraded":"2024-09-13T17:17:21.786167+0000","last_fullsized":"2024-09-13T17:17:21.786167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3718,"log_dups_size":0,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20956,"num_write_kb":260218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"63'390","reported_seq":546,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:17.006685+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T17:17:17.006685+0000","last_peered":"2024-09-13T17:17:17.006685+0000","last_clean":"2024-09-13T17:17:17.006685+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T17:17:17.006685+0000","last_undegraded":"2024-09-13T17:17:17.006685+0000","last_fullsized":"2024-09-13T17:17:17.006685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":73872,"num_write":388,"num_write_kb":4310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.039214+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T17:17:14.039214+0000","last_peered":"2024-09-13T17:17:14.039214+0000","last_clean":"2024-09-13T17:17:14.039214+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T17:17:14.039214+0000","last_undegraded":"2024-09-13T17:17:14.039214+0000","last_fullsized":"2024-09-13T17:17:14.039214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"63'3522","reported_seq":3584,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.982810+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T17:17:13.982810+0000","last_peered":"2024-09-13T17:17:13.982810+0000","last_clean":"2024-09-13T17:17:13.982810+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T17:17:13.982810+0000","last_undegraded":"2024-09-13T17:17:13.982810+0000","last_fullsized":"2024-09-13T17:17:13.982810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":4,"num_write":19373,"num_write_kb":266222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"63'5138","reported_seq":5203,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095588+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T17:17:22.095588+0000","last_peered":"2024-09-13T17:17:22.095588+0000","last_clean":"2024-09-13T17:17:22.095588+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T17:17:22.095588+0000","last_undegraded":"2024-09-13T17:17:22.095588+0000","last_fullsized":"2024-09-13T17:17:22.095588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27863,"num_write_kb":353454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"63'23","reported_seq":125,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.131774+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T17:17:22.131774+0000","last_peered":"2024-09-13T17:17:22.131774+0000","last_clean":"2024-09-13T17:17:22.131774+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T17:17:22.131774+0000","last_undegraded":"2024-09-13T17:17:22.131774+0000","last_fullsized":"2024-09-13T17:17:22.131774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"63'7","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031857+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T17:17:14.031857+0000","last_peered":"2024-09-13T17:17:14.031857+0000","last_clean":"2024-09-13T17:17:14.031857+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T17:17:14.031857+0000","last_undegraded":"2024-09-13T17:17:14.031857+0000","last_fullsized":"2024-09-13T17:17:14.031857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":168,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"63'3602","reported_seq":3666,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.014764+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T17:17:14.014764+0000","last_peered":"2024-09-13T17:17:14.014764+0000","last_clean":"2024-09-13T17:17:14.014764+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T17:17:14.014764+0000","last_undegraded":"2024-09-13T17:17:14.014764+0000","last_fullsized":"2024-09-13T17:17:14.014764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":20290,"num_write_kb":278255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'3743","reported_seq":3808,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084809+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T17:17:22.084809+0000","last_peered":"2024-09-13T17:17:22.084809+0000","last_clean":"2024-09-13T17:17:22.084809+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T17:17:22.084809+0000","last_undegraded":"2024-09-13T17:17:22.084809+0000","last_fullsized":"2024-09-13T17:17:22.084809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20404,"num_write_kb":278673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"63'3","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.103865+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T17:17:22.103865+0000","last_peered":"2024-09-13T17:17:22.103865+0000","last_clean":"2024-09-13T17:17:22.103865+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T17:17:22.103865+0000","last_undegraded":"2024-09-13T17:17:22.103865+0000","last_fullsized":"2024-09-13T17:17:22.103865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"63'15","reported_seq":185,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024454+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T17:17:14.024454+0000","last_peered":"2024-09-13T17:17:14.024454+0000","last_clean":"2024-09-13T17:17:14.024454+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T17:17:14.024454+0000","last_undegraded":"2024-09-13T17:17:14.024454+0000","last_fullsized":"2024-09-13T17:17:14.024454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":324,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"63'3189","reported_seq":3250,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.950668+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T17:17:13.950668+0000","last_peered":"2024-09-13T17:17:13.950668+0000","last_clean":"2024-09-13T17:17:13.950668+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T17:17:13.950668+0000","last_undegraded":"2024-09-13T17:17:13.950668+0000","last_fullsized":"2024-09-13T17:17:13.950668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17342,"num_write_kb":249797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"63'4223","reported_seq":4288,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117238+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T17:17:22.117238+0000","last_peered":"2024-09-13T17:17:22.117238+0000","last_clean":"2024-09-13T17:17:22.117238+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T17:17:22.117238+0000","last_undegraded":"2024-09-13T17:17:22.117238+0000","last_fullsized":"2024-09-13T17:17:22.117238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4223,"log_dups_size":0,"ondisk_log_size":4223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":295025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"63'4","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.059446+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T17:17:22.059446+0000","last_peered":"2024-09-13T17:17:22.059446+0000","last_clean":"2024-09-13T17:17:22.059446+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T17:17:22.059446+0000","last_undegraded":"2024-09-13T17:17:22.059446+0000","last_fullsized":"2024-09-13T17:17:22.059446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"63'5","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.021576+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T17:17:14.021576+0000","last_peered":"2024-09-13T17:17:14.021576+0000","last_clean":"2024-09-13T17:17:14.021576+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T17:17:14.021576+0000","last_undegraded":"2024-09-13T17:17:14.021576+0000","last_fullsized":"2024-09-13T17:17:14.021576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'5357","reported_seq":5423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058245+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T17:17:14.058245+0000","last_peered":"2024-09-13T17:17:14.058245+0000","last_clean":"2024-09-13T17:17:14.058245+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T17:17:14.058245+0000","last_undegraded":"2024-09-13T17:17:14.058245+0000","last_fullsized":"2024-09-13T17:17:14.058245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":39,"num_write":29637,"num_write_kb":393569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"63'26","reported_seq":90,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.989101+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T17:17:13.989101+0000","last_peered":"2024-09-13T17:17:13.989101+0000","last_clean":"2024-09-13T17:17:13.989101+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T17:17:13.989101+0000","last_undegraded":"2024-09-13T17:17:13.989101+0000","last_fullsized":"2024-09-13T17:17:13.989101+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"63'5122","reported_seq":5187,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.106963+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T17:17:22.106963+0000","last_peered":"2024-09-13T17:17:22.106963+0000","last_clean":"2024-09-13T17:17:22.106963+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T17:17:22.106963+0000","last_undegraded":"2024-09-13T17:17:22.106963+0000","last_fullsized":"2024-09-13T17:17:22.106963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":348704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"63'2","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:04.209346+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T17:17:04.209346+0000","last_peered":"2024-09-13T17:17:04.209346+0000","last_clean":"2024-09-13T17:17:04.209346+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T17:17:04.209346+0000","last_undegraded":"2024-09-13T17:17:04.209346+0000","last_fullsized":"2024-09-13T17:17:04.209346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"63'3554","reported_seq":3617,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036436+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T17:17:14.036436+0000","last_peered":"2024-09-13T17:17:14.036436+0000","last_clean":"2024-09-13T17:17:14.036436+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T17:17:14.036436+0000","last_undegraded":"2024-09-13T17:17:14.036436+0000","last_fullsized":"2024-09-13T17:17:14.036436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":268389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"63'1091","reported_seq":1205,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.012135+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T17:17:14.012135+0000","last_peered":"2024-09-13T17:17:14.012135+0000","last_clean":"2024-09-13T17:17:14.012135+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T17:17:14.012135+0000","last_undegraded":"2024-09-13T17:17:14.012135+0000","last_fullsized":"2024-09-13T17:17:14.012135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":9436,"num_write":1065,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"63'4714","reported_seq":4779,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.121909+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T17:17:22.121909+0000","last_peered":"2024-09-13T17:17:22.121909+0000","last_clean":"2024-09-13T17:17:22.121909+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T17:17:22.121909+0000","last_undegraded":"2024-09-13T17:17:22.121909+0000","last_fullsized":"2024-09-13T17:17:22.121909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25459,"num_write_kb":343662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"63'3","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.098646+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T17:17:22.098646+0000","last_peered":"2024-09-13T17:17:22.098646+0000","last_clean":"2024-09-13T17:17:22.098646+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T17:17:22.098646+0000","last_undegraded":"2024-09-13T17:17:22.098646+0000","last_fullsized":"2024-09-13T17:17:22.098646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"63'4328","reported_seq":4388,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.013223+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T17:17:14.013223+0000","last_peered":"2024-09-13T17:17:14.013223+0000","last_clean":"2024-09-13T17:17:14.013223+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T17:17:14.013223+0000","last_undegraded":"2024-09-13T17:17:14.013223+0000","last_fullsized":"2024-09-13T17:17:14.013223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24456,"num_write_kb":315374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"63'396","reported_seq":463,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:23.808734+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T17:17:23.808734+0000","last_peered":"2024-09-13T17:17:23.808734+0000","last_clean":"2024-09-13T17:17:23.808734+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T17:17:23.808734+0000","last_undegraded":"2024-09-13T17:17:23.808734+0000","last_fullsized":"2024-09-13T17:17:23.808734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2790242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":465,"num_write":372,"num_write_kb":2945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"63'4155","reported_seq":4219,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.033063+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T17:17:22.033063+0000","last_peered":"2024-09-13T17:17:22.033063+0000","last_clean":"2024-09-13T17:17:22.033063+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T17:17:22.033063+0000","last_undegraded":"2024-09-13T17:17:22.033063+0000","last_fullsized":"2024-09-13T17:17:22.033063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22247,"num_write_kb":285891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"63'5","reported_seq":108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:16:34.958339+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T17:16:34.958339+0000","last_peered":"2024-09-13T17:16:34.958339+0000","last_clean":"2024-09-13T17:16:34.958339+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T17:16:34.958339+0000","last_undegraded":"2024-09-13T17:16:34.958339+0000","last_fullsized":"2024-09-13T17:16:34.958339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:23.394258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"63'3426","reported_seq":3489,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.986253+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T17:17:13.986253+0000","last_peered":"2024-09-13T17:17:13.986253+0000","last_clean":"2024-09-13T17:17:13.986253+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T17:17:13.986253+0000","last_undegraded":"2024-09-13T17:17:13.986253+0000","last_fullsized":"2024-09-13T17:17:13.986253+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3426,"log_dups_size":0,"ondisk_log_size":3426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":18510,"num_write_kb":250514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'712","reported_seq":799,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058400+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T17:17:22.058400+0000","last_peered":"2024-09-13T17:17:22.058400+0000","last_clean":"2024-09-13T17:17:22.058400+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T17:17:22.058400+0000","last_undegraded":"2024-09-13T17:17:22.058400+0000","last_fullsized":"2024-09-13T17:17:22.058400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":70578,"num_write":702,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"63'4316","reported_seq":4381,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.050313+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T17:17:22.050313+0000","last_peered":"2024-09-13T17:17:22.050313+0000","last_clean":"2024-09-13T17:17:22.050313+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T17:17:22.050313+0000","last_undegraded":"2024-09-13T17:17:22.050313+0000","last_fullsized":"2024-09-13T17:17:22.050313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22350,"num_write_kb":290228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'44","reported_seq":147,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031452+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T17:17:14.031452+0000","last_peered":"2024-09-13T17:17:14.031452+0000","last_clean":"2024-09-13T17:17:14.031452+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T17:17:14.031452+0000","last_undegraded":"2024-09-13T17:17:14.031452+0000","last_fullsized":"2024-09-13T17:17:14.031452+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"63'3129","reported_seq":3191,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031759+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T17:17:14.031759+0000","last_peered":"2024-09-13T17:17:14.031759+0000","last_clean":"2024-09-13T17:17:14.031759+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T17:17:14.031759+0000","last_undegraded":"2024-09-13T17:17:14.031759+0000","last_fullsized":"2024-09-13T17:17:14.031759+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17279,"num_write_kb":257601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"63'4785","reported_seq":4852,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.086557+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T17:17:22.086557+0000","last_peered":"2024-09-13T17:17:22.086557+0000","last_clean":"2024-09-13T17:17:22.086557+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T17:17:22.086557+0000","last_undegraded":"2024-09-13T17:17:22.086557+0000","last_fullsized":"2024-09-13T17:17:22.086557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26302,"num_write_kb":310249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"63'4231","reported_seq":4294,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.009044+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T17:17:14.009044+0000","last_peered":"2024-09-13T17:17:14.009044+0000","last_clean":"2024-09-13T17:17:14.009044+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T17:17:14.009044+0000","last_undegraded":"2024-09-13T17:17:14.009044+0000","last_fullsized":"2024-09-13T17:17:14.009044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22604,"num_write_kb":324065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"63'3874","reported_seq":3942,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.078497+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T17:17:22.078497+0000","last_peered":"2024-09-13T17:17:22.078497+0000","last_clean":"2024-09-13T17:17:22.078497+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T17:17:22.078497+0000","last_undegraded":"2024-09-13T17:17:22.078497+0000","last_fullsized":"2024-09-13T17:17:22.078497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:59.590400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20920,"num_write_kb":276387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"63'3865","reported_seq":3928,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.006270+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T17:17:14.006270+0000","last_peered":"2024-09-13T17:17:14.006270+0000","last_clean":"2024-09-13T17:17:14.006270+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T17:17:14.006270+0000","last_undegraded":"2024-09-13T17:17:14.006270+0000","last_fullsized":"2024-09-13T17:17:14.006270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21432,"num_write_kb":286796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"63'4552","reported_seq":4621,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.103214+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T17:17:22.103214+0000","last_peered":"2024-09-13T17:17:22.103214+0000","last_clean":"2024-09-13T17:17:22.103214+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T17:17:22.103214+0000","last_undegraded":"2024-09-13T17:17:22.103214+0000","last_fullsized":"2024-09-13T17:17:22.103214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":24031,"num_write_kb":313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"63'4071","reported_seq":4134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.013587+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T17:17:14.013587+0000","last_peered":"2024-09-13T17:17:14.013587+0000","last_clean":"2024-09-13T17:17:14.013587+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T17:17:14.013587+0000","last_undegraded":"2024-09-13T17:17:14.013587+0000","last_fullsized":"2024-09-13T17:17:14.013587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":22061,"num_write_kb":309650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"63'3506","reported_seq":3574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.087622+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T17:17:22.087622+0000","last_peered":"2024-09-13T17:17:22.087622+0000","last_clean":"2024-09-13T17:17:22.087622+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T17:17:22.087622+0000","last_undegraded":"2024-09-13T17:17:22.087622+0000","last_fullsized":"2024-09-13T17:17:22.087622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3506,"log_dups_size":0,"ondisk_log_size":3506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19493,"num_write_kb":254380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"63'4523","reported_seq":4586,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.039988+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T17:17:14.039988+0000","last_peered":"2024-09-13T17:17:14.039988+0000","last_clean":"2024-09-13T17:17:14.039988+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T17:17:14.039988+0000","last_undegraded":"2024-09-13T17:17:14.039988+0000","last_fullsized":"2024-09-13T17:17:14.039988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4523,"log_dups_size":0,"ondisk_log_size":4523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24221,"num_write_kb":338849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"63'4422","reported_seq":4489,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.072016+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T17:17:22.072016+0000","last_peered":"2024-09-13T17:17:22.072016+0000","last_clean":"2024-09-13T17:17:22.072016+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T17:17:22.072016+0000","last_undegraded":"2024-09-13T17:17:22.072016+0000","last_fullsized":"2024-09-13T17:17:22.072016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":24862,"num_write_kb":304752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"63'3771","reported_seq":3835,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022654+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T17:17:14.022654+0000","last_peered":"2024-09-13T17:17:14.022654+0000","last_clean":"2024-09-13T17:17:14.022654+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T17:17:14.022654+0000","last_undegraded":"2024-09-13T17:17:14.022654+0000","last_fullsized":"2024-09-13T17:17:14.022654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21135,"num_write_kb":285540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"63'4461","reported_seq":4527,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095053+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T17:17:22.095053+0000","last_peered":"2024-09-13T17:17:22.095053+0000","last_clean":"2024-09-13T17:17:22.095053+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T17:17:22.095053+0000","last_undegraded":"2024-09-13T17:17:22.095053+0000","last_fullsized":"2024-09-13T17:17:22.095053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25388,"num_write_kb":319094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"63'4193","reported_seq":4256,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.014710+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T17:17:14.014710+0000","last_peered":"2024-09-13T17:17:14.014710+0000","last_clean":"2024-09-13T17:17:14.014710+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T17:17:14.014710+0000","last_undegraded":"2024-09-13T17:17:14.014710+0000","last_fullsized":"2024-09-13T17:17:14.014710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4193,"log_dups_size":0,"ondisk_log_size":4193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23066,"num_write_kb":313713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"63'3978","reported_seq":4044,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.100942+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T17:17:22.100942+0000","last_peered":"2024-09-13T17:17:22.100942+0000","last_clean":"2024-09-13T17:17:22.100942+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T17:17:22.100942+0000","last_undegraded":"2024-09-13T17:17:22.100942+0000","last_fullsized":"2024-09-13T17:17:22.100942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22157,"num_write_kb":281285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"63'4050","reported_seq":4115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.048739+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T17:17:14.048739+0000","last_peered":"2024-09-13T17:17:14.048739+0000","last_clean":"2024-09-13T17:17:14.048739+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T17:17:14.048739+0000","last_undegraded":"2024-09-13T17:17:14.048739+0000","last_fullsized":"2024-09-13T17:17:14.048739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4050,"log_dups_size":0,"ondisk_log_size":4050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21731,"num_write_kb":291923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"63'5211","reported_seq":5272,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.046515+0000","last_change":"2024-09-13T16:48:40.831835+0000","last_active":"2024-09-13T17:17:22.046515+0000","last_peered":"2024-09-13T17:17:22.046515+0000","last_clean":"2024-09-13T17:17:22.046515+0000","last_became_active":"2024-09-13T16:48:40.831649+0000","last_became_peered":"2024-09-13T16:48:40.831649+0000","last_unstale":"2024-09-13T17:17:22.046515+0000","last_undegraded":"2024-09-13T17:17:22.046515+0000","last_fullsized":"2024-09-13T17:17:22.046515+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:54.247988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":28065,"num_write_kb":366501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"63'3806","reported_seq":3870,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024614+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T17:17:14.024614+0000","last_peered":"2024-09-13T17:17:14.024614+0000","last_clean":"2024-09-13T17:17:14.024614+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T17:17:14.024614+0000","last_undegraded":"2024-09-13T17:17:14.024614+0000","last_fullsized":"2024-09-13T17:17:14.024614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4,"num_write":20286,"num_write_kb":286237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"63'3659","reported_seq":3726,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126330+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T17:17:22.126330+0000","last_peered":"2024-09-13T17:17:22.126330+0000","last_clean":"2024-09-13T17:17:22.126330+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T17:17:22.126330+0000","last_undegraded":"2024-09-13T17:17:22.126330+0000","last_fullsized":"2024-09-13T17:17:22.126330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19936,"num_write_kb":264332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"63'5003","reported_seq":5065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.034159+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T17:17:14.034159+0000","last_peered":"2024-09-13T17:17:14.034159+0000","last_clean":"2024-09-13T17:17:14.034159+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T17:17:14.034159+0000","last_undegraded":"2024-09-13T17:17:14.034159+0000","last_fullsized":"2024-09-13T17:17:14.034159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27137,"num_write_kb":387189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"63'4068","reported_seq":4134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.106118+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T17:17:22.106118+0000","last_peered":"2024-09-13T17:17:22.106118+0000","last_clean":"2024-09-13T17:17:22.106118+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T17:17:22.106118+0000","last_undegraded":"2024-09-13T17:17:22.106118+0000","last_fullsized":"2024-09-13T17:17:22.106118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22149,"num_write_kb":293545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"63'3283","reported_seq":3345,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.057875+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T17:17:14.057875+0000","last_peered":"2024-09-13T17:17:14.057875+0000","last_clean":"2024-09-13T17:17:14.057875+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T17:17:14.057875+0000","last_undegraded":"2024-09-13T17:17:14.057875+0000","last_fullsized":"2024-09-13T17:17:14.057875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:32.817409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":248866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"63'4697","reported_seq":4763,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.910034+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T17:17:21.910034+0000","last_peered":"2024-09-13T17:17:21.910034+0000","last_clean":"2024-09-13T17:17:21.910034+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T17:17:21.910034+0000","last_undegraded":"2024-09-13T17:17:21.910034+0000","last_fullsized":"2024-09-13T17:17:21.910034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25800,"num_write_kb":320606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"63'3820","reported_seq":3882,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.997809+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T17:17:13.997809+0000","last_peered":"2024-09-13T17:17:13.997809+0000","last_clean":"2024-09-13T17:17:13.997809+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T17:17:13.997809+0000","last_undegraded":"2024-09-13T17:17:13.997809+0000","last_fullsized":"2024-09-13T17:17:13.997809+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3820,"log_dups_size":0,"ondisk_log_size":3820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20813,"num_write_kb":300503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"63'4719","reported_seq":4787,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117962+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T17:17:22.117962+0000","last_peered":"2024-09-13T17:17:22.117962+0000","last_clean":"2024-09-13T17:17:22.117962+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T17:17:22.117962+0000","last_undegraded":"2024-09-13T17:17:22.117962+0000","last_fullsized":"2024-09-13T17:17:22.117962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":25263,"num_write_kb":330973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"63'3722","reported_seq":3784,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.001243+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T17:17:14.001243+0000","last_peered":"2024-09-13T17:17:14.001243+0000","last_clean":"2024-09-13T17:17:14.001243+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T17:17:14.001243+0000","last_undegraded":"2024-09-13T17:17:14.001243+0000","last_fullsized":"2024-09-13T17:17:14.001243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20022,"num_write_kb":289122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"63'3813","reported_seq":3879,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.064518+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T17:17:22.064518+0000","last_peered":"2024-09-13T17:17:22.064518+0000","last_clean":"2024-09-13T17:17:22.064518+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T17:17:22.064518+0000","last_undegraded":"2024-09-13T17:17:22.064518+0000","last_fullsized":"2024-09-13T17:17:22.064518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21202,"num_write_kb":265282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"63'4479","reported_seq":4545,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.060514+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T17:17:14.060514+0000","last_peered":"2024-09-13T17:17:14.060514+0000","last_clean":"2024-09-13T17:17:14.060514+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T17:17:14.060514+0000","last_undegraded":"2024-09-13T17:17:14.060514+0000","last_fullsized":"2024-09-13T17:17:14.060514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":24556,"num_write_kb":340437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"63'4549","reported_seq":4615,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058661+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T17:17:22.058661+0000","last_peered":"2024-09-13T17:17:22.058661+0000","last_clean":"2024-09-13T17:17:22.058661+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T17:17:22.058661+0000","last_undegraded":"2024-09-13T17:17:22.058661+0000","last_fullsized":"2024-09-13T17:17:22.058661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24549,"num_write_kb":319591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"63'3479","reported_seq":3542,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.019701+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T17:17:14.019701+0000","last_peered":"2024-09-13T17:17:14.019701+0000","last_clean":"2024-09-13T17:17:14.019701+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T17:17:14.019701+0000","last_undegraded":"2024-09-13T17:17:14.019701+0000","last_fullsized":"2024-09-13T17:17:14.019701+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3479,"log_dups_size":0,"ondisk_log_size":3479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18966,"num_write_kb":268432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"63'3573","reported_seq":3639,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084131+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T17:17:22.084131+0000","last_peered":"2024-09-13T17:17:22.084131+0000","last_clean":"2024-09-13T17:17:22.084131+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T17:17:22.084131+0000","last_undegraded":"2024-09-13T17:17:22.084131+0000","last_fullsized":"2024-09-13T17:17:22.084131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19348,"num_write_kb":270113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"63'4058","reported_seq":4120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036359+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T17:17:14.036359+0000","last_peered":"2024-09-13T17:17:14.036359+0000","last_clean":"2024-09-13T17:17:14.036359+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T17:17:14.036359+0000","last_undegraded":"2024-09-13T17:17:14.036359+0000","last_fullsized":"2024-09-13T17:17:14.036359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22331,"num_write_kb":302585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"63'5839","reported_seq":5904,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117408+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T17:17:22.117408+0000","last_peered":"2024-09-13T17:17:22.117408+0000","last_clean":"2024-09-13T17:17:22.117408+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T17:17:22.117408+0000","last_undegraded":"2024-09-13T17:17:22.117408+0000","last_fullsized":"2024-09-13T17:17:22.117408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31756,"num_write_kb":397521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"63'3835","reported_seq":3897,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.035326+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T17:17:14.035326+0000","last_peered":"2024-09-13T17:17:14.035326+0000","last_clean":"2024-09-13T17:17:14.035326+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T17:17:14.035326+0000","last_undegraded":"2024-09-13T17:17:14.035326+0000","last_fullsized":"2024-09-13T17:17:14.035326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":21185,"num_write_kb":306173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"63'4134","reported_seq":4199,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.079721+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T17:17:22.079721+0000","last_peered":"2024-09-13T17:17:22.079721+0000","last_clean":"2024-09-13T17:17:22.079721+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T17:17:22.079721+0000","last_undegraded":"2024-09-13T17:17:22.079721+0000","last_fullsized":"2024-09-13T17:17:22.079721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22268,"num_write_kb":281801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"63'5073","reported_seq":5136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058407+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T17:17:14.058407+0000","last_peered":"2024-09-13T17:17:14.058407+0000","last_clean":"2024-09-13T17:17:14.058407+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T17:17:14.058407+0000","last_undegraded":"2024-09-13T17:17:14.058407+0000","last_fullsized":"2024-09-13T17:17:14.058407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":28693,"num_write_kb":393724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"63'3822","reported_seq":3887,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107201+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T17:17:22.107201+0000","last_peered":"2024-09-13T17:17:22.107201+0000","last_clean":"2024-09-13T17:17:22.107201+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T17:17:22.107201+0000","last_undegraded":"2024-09-13T17:17:22.107201+0000","last_fullsized":"2024-09-13T17:17:22.107201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20823,"num_write_kb":275830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"63'2778","reported_seq":2842,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.048760+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T17:17:14.048760+0000","last_peered":"2024-09-13T17:17:14.048760+0000","last_clean":"2024-09-13T17:17:14.048760+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T17:17:14.048760+0000","last_undegraded":"2024-09-13T17:17:14.048760+0000","last_fullsized":"2024-09-13T17:17:14.048760+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":34,"num_write":15212,"num_write_kb":208573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"63'3975","reported_seq":4040,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.131512+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T17:17:22.131512+0000","last_peered":"2024-09-13T17:17:22.131512+0000","last_clean":"2024-09-13T17:17:22.131512+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T17:17:22.131512+0000","last_undegraded":"2024-09-13T17:17:22.131512+0000","last_fullsized":"2024-09-13T17:17:22.131512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3975,"log_dups_size":0,"ondisk_log_size":3975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21351,"num_write_kb":278200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"63'3844","reported_seq":3907,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.966555+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T17:17:13.966555+0000","last_peered":"2024-09-13T17:17:13.966555+0000","last_clean":"2024-09-13T17:17:13.966555+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T17:17:13.966555+0000","last_undegraded":"2024-09-13T17:17:13.966555+0000","last_fullsized":"2024-09-13T17:17:13.966555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21085,"num_write_kb":289508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"63'3324","reported_seq":3389,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.087163+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T17:17:22.087163+0000","last_peered":"2024-09-13T17:17:22.087163+0000","last_clean":"2024-09-13T17:17:22.087163+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T17:17:22.087163+0000","last_undegraded":"2024-09-13T17:17:22.087163+0000","last_fullsized":"2024-09-13T17:17:22.087163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3324,"log_dups_size":0,"ondisk_log_size":3324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18810,"num_write_kb":235312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"63'3743","reported_seq":3805,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.055690+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T17:17:14.055690+0000","last_peered":"2024-09-13T17:17:14.055690+0000","last_clean":"2024-09-13T17:17:14.055690+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T17:17:14.055690+0000","last_undegraded":"2024-09-13T17:17:14.055690+0000","last_fullsized":"2024-09-13T17:17:14.055690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20790,"num_write_kb":280180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"63'3173","reported_seq":3238,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.083735+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T17:17:22.083735+0000","last_peered":"2024-09-13T17:17:22.083735+0000","last_clean":"2024-09-13T17:17:22.083735+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T17:17:22.083735+0000","last_undegraded":"2024-09-13T17:17:22.083735+0000","last_fullsized":"2024-09-13T17:17:22.083735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17044,"num_write_kb":219853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"63'3460","reported_seq":3522,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.008254+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T17:17:14.008254+0000","last_peered":"2024-09-13T17:17:14.008254+0000","last_clean":"2024-09-13T17:17:14.008254+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T17:17:14.008254+0000","last_undegraded":"2024-09-13T17:17:14.008254+0000","last_fullsized":"2024-09-13T17:17:14.008254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:24.171305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":19788,"num_write_kb":263976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"63'3747","reported_seq":3812,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.008211+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T17:17:22.008211+0000","last_peered":"2024-09-13T17:17:22.008211+0000","last_clean":"2024-09-13T17:17:22.008211+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T17:17:22.008211+0000","last_undegraded":"2024-09-13T17:17:22.008211+0000","last_fullsized":"2024-09-13T17:17:22.008211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20231,"num_write_kb":269532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"63'3422","reported_seq":3483,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.003363+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T17:17:14.003363+0000","last_peered":"2024-09-13T17:17:14.003363+0000","last_clean":"2024-09-13T17:17:14.003363+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T17:17:14.003363+0000","last_undegraded":"2024-09-13T17:17:14.003363+0000","last_fullsized":"2024-09-13T17:17:14.003363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18722,"num_write_kb":267617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"63'4803","reported_seq":4868,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084790+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T17:17:22.084790+0000","last_peered":"2024-09-13T17:17:22.084790+0000","last_clean":"2024-09-13T17:17:22.084790+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T17:17:22.084790+0000","last_undegraded":"2024-09-13T17:17:22.084790+0000","last_fullsized":"2024-09-13T17:17:22.084790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26967,"num_write_kb":333429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"63'3840","reported_seq":3901,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.021162+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T17:17:14.021162+0000","last_peered":"2024-09-13T17:17:14.021162+0000","last_clean":"2024-09-13T17:17:14.021162+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T17:17:14.021162+0000","last_undegraded":"2024-09-13T17:17:14.021162+0000","last_fullsized":"2024-09-13T17:17:14.021162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21113,"num_write_kb":293770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"63'5433","reported_seq":5500,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.040459+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T17:17:22.040459+0000","last_peered":"2024-09-13T17:17:22.040459+0000","last_clean":"2024-09-13T17:17:22.040459+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T17:17:22.040459+0000","last_undegraded":"2024-09-13T17:17:22.040459+0000","last_fullsized":"2024-09-13T17:17:22.040459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30664,"num_write_kb":389336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"63'4741","reported_seq":4806,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.053779+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T17:17:14.053779+0000","last_peered":"2024-09-13T17:17:14.053779+0000","last_clean":"2024-09-13T17:17:14.053779+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T17:17:14.053779+0000","last_undegraded":"2024-09-13T17:17:14.053779+0000","last_fullsized":"2024-09-13T17:17:14.053779+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":15,"num_write":25588,"num_write_kb":340391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"63'4281","reported_seq":4345,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.981625+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T17:17:21.981625+0000","last_peered":"2024-09-13T17:17:21.981625+0000","last_clean":"2024-09-13T17:17:21.981625+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T17:17:21.981625+0000","last_undegraded":"2024-09-13T17:17:21.981625+0000","last_fullsized":"2024-09-13T17:17:21.981625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23373,"num_write_kb":302635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"63'3575","reported_seq":3641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.054561+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T17:17:14.054561+0000","last_peered":"2024-09-13T17:17:14.054561+0000","last_clean":"2024-09-13T17:17:14.054561+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T17:17:14.054561+0000","last_undegraded":"2024-09-13T17:17:14.054561+0000","last_fullsized":"2024-09-13T17:17:14.054561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":7,"num_write":20117,"num_write_kb":289677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"63'4404","reported_seq":4468,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.129331+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T17:17:22.129331+0000","last_peered":"2024-09-13T17:17:22.129331+0000","last_clean":"2024-09-13T17:17:22.129331+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T17:17:22.129331+0000","last_undegraded":"2024-09-13T17:17:22.129331+0000","last_fullsized":"2024-09-13T17:17:22.129331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24308,"num_write_kb":306423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"63'3318","reported_seq":3382,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.949311+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T17:17:13.949311+0000","last_peered":"2024-09-13T17:17:13.949311+0000","last_clean":"2024-09-13T17:17:13.949311+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T17:17:13.949311+0000","last_undegraded":"2024-09-13T17:17:13.949311+0000","last_fullsized":"2024-09-13T17:17:13.949311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3318,"log_dups_size":0,"ondisk_log_size":3318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18295,"num_write_kb":253537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"63'4013","reported_seq":4077,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.861950+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T17:17:21.861950+0000","last_peered":"2024-09-13T17:17:21.861950+0000","last_clean":"2024-09-13T17:17:21.861950+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T17:17:21.861950+0000","last_undegraded":"2024-09-13T17:17:21.861950+0000","last_fullsized":"2024-09-13T17:17:21.861950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21393,"num_write_kb":262042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"63'3658","reported_seq":3722,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.048939+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T17:17:14.048939+0000","last_peered":"2024-09-13T17:17:14.048939+0000","last_clean":"2024-09-13T17:17:14.048939+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T17:17:14.048939+0000","last_undegraded":"2024-09-13T17:17:14.048939+0000","last_fullsized":"2024-09-13T17:17:14.048939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19946,"num_write_kb":268446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"63'3910","reported_seq":3974,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.108541+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T17:17:22.108541+0000","last_peered":"2024-09-13T17:17:22.108541+0000","last_clean":"2024-09-13T17:17:22.108541+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T17:17:22.108541+0000","last_undegraded":"2024-09-13T17:17:22.108541+0000","last_fullsized":"2024-09-13T17:17:22.108541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3910,"log_dups_size":0,"ondisk_log_size":3910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":278651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"63'3074","reported_seq":3134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.837771+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T17:17:13.837771+0000","last_peered":"2024-09-13T17:17:13.837771+0000","last_clean":"2024-09-13T17:17:13.837771+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T17:17:13.837771+0000","last_undegraded":"2024-09-13T17:17:13.837771+0000","last_fullsized":"2024-09-13T17:17:13.837771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":246226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"63'3515","reported_seq":3583,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.005699+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T17:17:22.005699+0000","last_peered":"2024-09-13T17:17:22.005699+0000","last_clean":"2024-09-13T17:17:22.005699+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T17:17:22.005699+0000","last_undegraded":"2024-09-13T17:17:22.005699+0000","last_fullsized":"2024-09-13T17:17:22.005699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19512,"num_write_kb":258445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"63'3439","reported_seq":3500,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.038223+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T17:17:14.038223+0000","last_peered":"2024-09-13T17:17:14.038223+0000","last_clean":"2024-09-13T17:17:14.038223+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T17:17:14.038223+0000","last_undegraded":"2024-09-13T17:17:14.038223+0000","last_fullsized":"2024-09-13T17:17:14.038223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:05.489668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":17681,"num_write_kb":263603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"63'3892","reported_seq":3960,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.067970+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T17:17:22.067970+0000","last_peered":"2024-09-13T17:17:22.067970+0000","last_clean":"2024-09-13T17:17:22.067970+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T17:17:22.067970+0000","last_undegraded":"2024-09-13T17:17:22.067970+0000","last_fullsized":"2024-09-13T17:17:22.067970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21304,"num_write_kb":286190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"63'3622","reported_seq":3682,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.941100+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T17:17:13.941100+0000","last_peered":"2024-09-13T17:17:13.941100+0000","last_clean":"2024-09-13T17:17:13.941100+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T17:17:13.941100+0000","last_undegraded":"2024-09-13T17:17:13.941100+0000","last_fullsized":"2024-09-13T17:17:13.941100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19729,"num_write_kb":275691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"63'5036","reported_seq":5103,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.072290+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T17:17:22.072290+0000","last_peered":"2024-09-13T17:17:22.072290+0000","last_clean":"2024-09-13T17:17:22.072290+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T17:17:22.072290+0000","last_undegraded":"2024-09-13T17:17:22.072290+0000","last_fullsized":"2024-09-13T17:17:22.072290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27469,"num_write_kb":368239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"63'3689","reported_seq":3749,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.059558+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T17:17:14.059558+0000","last_peered":"2024-09-13T17:17:14.059558+0000","last_clean":"2024-09-13T17:17:14.059558+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T17:17:14.059558+0000","last_undegraded":"2024-09-13T17:17:14.059558+0000","last_fullsized":"2024-09-13T17:17:14.059558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3689,"log_dups_size":0,"ondisk_log_size":3689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20818,"num_write_kb":284187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"63'4047","reported_seq":4115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126604+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T17:17:22.126604+0000","last_peered":"2024-09-13T17:17:22.126604+0000","last_clean":"2024-09-13T17:17:22.126604+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T17:17:22.126604+0000","last_undegraded":"2024-09-13T17:17:22.126604+0000","last_fullsized":"2024-09-13T17:17:22.126604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22743,"num_write_kb":291925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":197,"num_write":1329762,"num_write_kb":18444058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242925,"ondisk_log_size":242925,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19024977,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1583,"num_read_kb":206738,"num_write":4472,"num_write_kb":20527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38109184,"data_stored":38049954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4352,"ondisk_log_size":4352,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":1479870,"num_write_kb":19043750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270762,"ondisk_log_size":270762,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":19653473,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1296,"num_read_kb":190456,"num_write":4501,"num_write_kb":21128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39370752,"data_stored":39306946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4332,"ondisk_log_size":4332,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":942768,"kb_used_data":13672,"kb_used_omap":1,"kb_used_meta":929086,"kb_avail":93429072,"statfs":{"total":96636764160,"available":95671369728,"internally_reserved":0,"allocated":14000128,"data_stored":13606602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":951384523},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":1.9890000000000001,"15min":1.212},"min":{"1min":0.24299999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":8.0960000000000001,"5min":16.571999999999999,"15min":16.571999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":2.3239999999999998,"5min":2.1040000000000001,"15min":1.3080000000000001},"min":{"1min":0.26600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":9.7799999999999994,"5min":16.523,"15min":16.523},"last":0.95399999999999996}]},{"osd":1,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":5.859,"5min":2.754,"15min":1.486},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":59.911999999999999,"5min":59.911999999999999,"15min":59.911999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":5.8680000000000003,"5min":2.8519999999999999,"15min":1.5660000000000001},"min":{"1min":0.28699999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":59.895000000000003,"5min":59.895000000000003,"15min":59.895000000000003},"last":1.1080000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.113,"5min":1.6659999999999999,"15min":1.1040000000000001},"min":{"1min":0.28999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.6869999999999994,"5min":16.023,"15min":16.023},"last":0.90000000000000002},{"interface":"front","average":{"1min":2.173,"5min":1.659,"15min":1.1559999999999999},"min":{"1min":0.28299999999999997,"5min":0.249,"15min":0.249},"max":{"1min":9.5619999999999994,"5min":10.654999999999999,"15min":10.654999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.2200000000000002,"5min":1.944,"15min":1.2210000000000001},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":9.7370000000000001,"5min":23.225000000000001,"15min":23.225000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":2.2959999999999998,"5min":1.944,"15min":1.2789999999999999},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.891,"5min":12.045999999999999,"15min":12.045999999999999},"last":1.0169999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":2.0659999999999998,"15min":1.274},"min":{"1min":0.25900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":14.209,"5min":18.004999999999999,"15min":18.004999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":3.9510000000000001,"5min":2.339,"15min":1.391},"min":{"1min":0.28299999999999997,"5min":0.245,"15min":0.245},"max":{"1min":22.315999999999999,"5min":22.315999999999999,"15min":22.315999999999999},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":4.9809999999999999,"5min":2.2959999999999998,"15min":1.419},"min":{"1min":0.32200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":37.302999999999997,"5min":37.302999999999997,"15min":37.302999999999997},"last":0.86599999999999999},{"interface":"front","average":{"1min":2.6560000000000001,"5min":1.8100000000000001,"15min":1.27},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":16.867999999999999,"5min":16.867999999999999,"15min":16.867999999999999},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":1.788,"15min":1.292},"min":{"1min":0.38100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":9.1660000000000004,"5min":17.968,"15min":17.968},"last":0.81200000000000006},{"interface":"front","average":{"1min":3.29,"5min":1.9910000000000001,"15min":1.3500000000000001},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.025,"5min":19.244,"15min":19.244},"last":0.99399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575209,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673180,"kb_used_data":11732,"kb_used_omap":1,"kb_used_meta":661438,"kb_avail":93698660,"statfs":{"total":96636764160,"available":95947427840,"internally_reserved":0,"allocated":12013568,"data_stored":11612788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":677312968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.97999999999999998,"15min":0.86399999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":2.3839999999999999,"5min":6.335,"15min":6.335},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.878,"5min":1.111,"15min":0.89300000000000002},"min":{"1min":0.24199999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.5299999999999998,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.96399999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2130000000000001,"15min":0.95899999999999996},"min":{"1min":0.28899999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":6.3230000000000004,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":2.0379999999999998,"5min":1.5169999999999999,"15min":1.034},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":9.3629999999999995,"5min":10.111000000000001,"15min":10.111000000000001},"last":0.81899999999999995}]},{"osd":2,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.98199999999999998,"15min":0.90700000000000003},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":6.2160000000000002,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.202,"5min":1.1850000000000001,"15min":1.006},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.1130000000000004,"5min":6.6079999999999997,"15min":6.6079999999999997},"last":0.92900000000000005}]},{"osd":3,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5800000000000001,"15min":1.1220000000000001},"min":{"1min":0.308,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":15.068,"5min":15.247999999999999,"15min":15.247999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.782,"5min":1.5249999999999999,"15min":1.127},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":15.179,"5min":15.179,"15min":15.179},"last":0.75}]},{"osd":4,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":1.222,"15min":1.042},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.4790000000000001,"5min":15.326000000000001,"15min":15.326000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.454,"15min":1.1259999999999999},"min":{"1min":0.24399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":11.48,"5min":15.026,"15min":15.026},"last":1.1000000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.5089999999999999,"15min":1.2},"min":{"1min":0.38200000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":9.6080000000000005,"5min":11.566000000000001,"15min":11.566000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.4139999999999999,"15min":1.1639999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":10.223000000000001,"5min":10.223000000000001,"15min":10.223000000000001},"last":1.028}]},{"osd":7,"last update":"Fri Sep 13 17:16:43 2024","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.583,"15min":1.2250000000000001},"min":{"1min":0.32900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":9.8960000000000008,"5min":13.112,"15min":13.112},"last":1.0489999999999999},{"interface":"front","average":{"1min":2.0470000000000002,"5min":1.7470000000000001,"15min":1.28},"min":{"1min":0.33200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":14.4,"5min":14.4,"15min":14.4},"last":0.90200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575210,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":957772,"kb_used_data":4996,"kb_used_omap":1,"kb_used_meta":952766,"kb_avail":93414068,"statfs":{"total":96636764160,"available":95656005632,"internally_reserved":0,"allocated":5115904,"data_stored":4720442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":975632838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":3.762,"5min":1.978,"15min":1.1919999999999999},"min":{"1min":0.246,"5min":0.215,"15min":0.215},"max":{"1min":27.952999999999999,"5min":27.952999999999999,"15min":27.952999999999999},"last":1.024},{"interface":"front","average":{"1min":2.5059999999999998,"5min":1.629,"15min":1.121},"min":{"1min":0.28100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":23.619,"5min":23.619,"15min":23.619},"last":0.84199999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.8210000000000002,"5min":1.8109999999999999,"15min":1.1379999999999999},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":13.515000000000001,"5min":13.515000000000001,"15min":13.515000000000001},"last":1.095},{"interface":"front","average":{"1min":1.621,"5min":1.9299999999999999,"15min":1.1839999999999999},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":6.9290000000000003,"5min":35.186999999999998,"15min":35.186999999999998},"last":0.94699999999999995}]},{"osd":2,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":2.306,"15min":1.3420000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.609,"5min":51.18,"15min":51.18},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.732,"5min":1.929,"15min":1.2949999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.8659999999999997,"5min":48.756,"15min":48.756},"last":0.97899999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.7480000000000002,"5min":2.0219999999999998,"15min":1.2789999999999999},"min":{"1min":0.39200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":15.946999999999999,"5min":19.800000000000001,"15min":19.800000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.7649999999999999,"15min":1.256},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":9.1020000000000003,"5min":9.4789999999999992,"15min":9.4789999999999992},"last":1.173}]},{"osd":4,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":4.2270000000000003,"5min":2.4260000000000002,"15min":1.379},"min":{"1min":0.40699999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":20.713999999999999,"5min":22.361999999999998,"15min":22.361999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":4.1710000000000003,"5min":2.9940000000000002,"15min":1.641},"min":{"1min":0.35299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":28.062999999999999,"5min":31.843,"15min":31.843},"last":1.2210000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.427,"15min":1.131},"min":{"1min":0.375,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.3300000000000001,"5min":8.1910000000000007,"15min":8.1910000000000007},"last":1.268},{"interface":"front","average":{"1min":2.052,"5min":1.7649999999999999,"15min":1.26},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.1059999999999999,"5min":13.031000000000001,"15min":13.031000000000001},"last":1.1299999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":2.1890000000000001,"15min":1.4139999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":11.239000000000001,"5min":35.124000000000002,"15min":35.124000000000002},"last":1.056},{"interface":"front","average":{"1min":2.0550000000000002,"5min":2.5089999999999999,"15min":1.546},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.99,"5min":32.328000000000003,"15min":32.328000000000003},"last":1.3200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575209,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1011316,"kb_used_data":13932,"kb_used_omap":1,"kb_used_meta":997374,"kb_avail":93360524,"statfs":{"total":96636764160,"available":95601176576,"internally_reserved":0,"allocated":14266368,"data_stored":13858021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1021311430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.4910000000000001,"15min":1.0369999999999999},"min":{"1min":0.25,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":13.18,"5min":13.18,"15min":13.18},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.829,"5min":1.575,"15min":1.141},"min":{"1min":0.25700000000000001,"5min":0.251,"15min":0.251},"max":{"1min":13.163,"5min":13.163,"15min":13.163},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":3.1299999999999999,"5min":1.73,"15min":1.1060000000000001},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":27.242000000000001,"5min":27.242000000000001,"15min":27.242000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":2.9119999999999999,"5min":1.978,"15min":1.232},"min":{"1min":0.34399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":26.545999999999999,"5min":26.545999999999999,"15min":26.545999999999999},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.577,"15min":1.1060000000000001},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":7.2619999999999996,"5min":11.803000000000001,"15min":11.803000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.573,"15min":1.131},"min":{"1min":0.35299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3319999999999999,"5min":8.6709999999999994,"15min":8.6709999999999994},"last":0.83699999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":2.1899999999999999,"15min":1.343},"min":{"1min":0.32000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":10.583,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":2.875,"5min":1.9770000000000001,"15min":1.298},"min":{"1min":0.39300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":20.468,"5min":20.468,"15min":20.468},"last":0.94599999999999995}]},{"osd":5,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.4079999999999999,"15min":1.087},"min":{"1min":0.314,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.109,"5min":13.146000000000001,"15min":13.146000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.823,"5min":1.8080000000000001,"15min":1.264},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":14.202,"5min":15.044,"15min":15.044},"last":0.99199999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4359999999999999,"15min":1.1279999999999999},"min":{"1min":0.35299999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":7.2030000000000003,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":1.232},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":1.234},"min":{"1min":0.38400000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.7320000000000002,"5min":7.5860000000000003,"15min":7.5860000000000003},"last":1.069}]},{"osd":7,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.8169999999999999,"15min":1.345},"min":{"1min":0.33500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":6.3449999999999998,"5min":24.518999999999998,"15min":24.518999999999998},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.595,"5min":1.7170000000000001,"15min":1.306},"min":{"1min":0.372,"5min":0.312,"15min":0.312},"max":{"1min":9.0519999999999996,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":1.139}]}]},{"osd":3,"up_from":13,"seq":55834575210,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":958468,"kb_used_data":19388,"kb_used_omap":1,"kb_used_meta":939070,"kb_avail":93413372,"statfs":{"total":96636764160,"available":95655292928,"internally_reserved":0,"allocated":19853312,"data_stored":19449342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":961608136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.5880000000000001,"15min":0.97999999999999998},"min":{"1min":0.19400000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":19.327999999999999,"5min":19.327999999999999,"15min":19.327999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.831,"5min":1.7549999999999999,"15min":1.0449999999999999},"min":{"1min":0.29699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":17.754000000000001,"5min":20.699999999999999,"15min":20.699999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":1.3720000000000001,"15min":0.90300000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.3340000000000001,"5min":16.100000000000001,"15min":16.100000000000001},"last":0.31},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.3260000000000001,"15min":0.97499999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.847,"5min":10.417,"15min":10.417},"last":0.5}]},{"osd":2,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3280000000000001,"15min":0.96699999999999997},"min":{"1min":0.38800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.6509999999999998,"5min":8.5950000000000006,"15min":8.5950000000000006},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.996,"5min":1.276,"15min":0.96499999999999997},"min":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.5380000000000003,"5min":15.433999999999999,"15min":15.433999999999999},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":3.4470000000000001,"5min":2.1099999999999999,"15min":1.3500000000000001},"min":{"1min":0.308,"5min":0.30599999999999999,"15min":0.29599999999999999},"max":{"1min":15.919,"5min":15.919,"15min":15.919},"last":0.69499999999999995},{"interface":"front","average":{"1min":2.3170000000000002,"5min":1.712,"15min":1.2110000000000001},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":9.3819999999999997,"5min":9.3819999999999997,"15min":9.3819999999999997},"last":0.83399999999999996}]},{"osd":5,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.9119999999999999,"15min":1.321},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":9.4030000000000005,"5min":31.187999999999999,"15min":31.187999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":2.427,"5min":2.121,"15min":1.383},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":22.951000000000001,"5min":22.951000000000001,"15min":22.951000000000001},"last":0.79400000000000004}]},{"osd":6,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.873,"15min":1.3120000000000001},"min":{"1min":0.44400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":4.202,"5min":12.567,"15min":12.567},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.226,"5min":2.1779999999999999,"15min":1.4730000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.7380000000000004,"5min":15.882,"15min":15.882},"last":0.77700000000000002}]},{"osd":7,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":4.8339999999999996,"5min":2.4390000000000001,"15min":1.538},"min":{"1min":0.48999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":33.969000000000001,"5min":33.969000000000001,"15min":33.969000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.4399999999999999,"5min":2.2480000000000002,"15min":1.46},"min":{"1min":0.47799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.4550000000000001,"5min":25.652999999999999,"15min":25.652999999999999},"last":0.80600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575210,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662268,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":661374,"kb_avail":93709572,"statfs":{"total":96636764160,"available":95958601728,"internally_reserved":0,"allocated":905216,"data_stored":517265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":677247430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.89500000000000002,"15min":0.70699999999999996},"min":{"1min":0.189,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":3.3319999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.628},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.4830000000000001,"15min":0.96999999999999997},"min":{"1min":0.218,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":9.2729999999999997,"5min":9.2729999999999997,"15min":9.2729999999999997},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.3069999999999999,"5min":1.766,"15min":1.0680000000000001},"min":{"1min":0.29499999999999998,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":11.319000000000001,"5min":33.579999999999998,"15min":33.579999999999998},"last":0.505},{"interface":"front","average":{"1min":2.4359999999999999,"5min":1.7509999999999999,"15min":1.089},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":14.624000000000001,"5min":23.965,"15min":23.965},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.7949999999999999,"5min":2.1779999999999999,"15min":1.266},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.245},"max":{"1min":11.837999999999999,"5min":22.760000000000002,"15min":22.760000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":2.5449999999999999,"5min":1.6499999999999999,"15min":1.093},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":10.372,"5min":10.372,"15min":10.372},"last":0.92900000000000005}]},{"osd":4,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.1520000000000001,"5min":2.012,"15min":1.292},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":10.324999999999999,"5min":20.673999999999999,"15min":20.673999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":2.0289999999999999,"5min":1.7669999999999999,"15min":1.24},"min":{"1min":0.42199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":6.4059999999999997,"5min":16.5,"15min":16.5},"last":0.97799999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":4.601,"5min":2.5329999999999999,"15min":1.5149999999999999},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":21.722000000000001,"5min":21.722000000000001,"15min":21.722000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":3.2240000000000002,"5min":2.238,"15min":1.415},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":19.242999999999999,"5min":29.620999999999999,"15min":29.620999999999999},"last":1.1839999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.6739999999999999,"15min":1.254},"min":{"1min":0.44,"5min":0.38,"15min":0.38},"max":{"1min":5.3239999999999998,"5min":21.882999999999999,"15min":21.882999999999999},"last":1.024},{"interface":"front","average":{"1min":2.2669999999999999,"5min":2.1070000000000002,"15min":1.429},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.29,"5min":25.274999999999999,"15min":25.274999999999999},"last":1.0600000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":5.9109999999999996,"5min":3.2639999999999998,"15min":1.825},"min":{"1min":0.45700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":28.094999999999999,"5min":43.649000000000001,"15min":43.649000000000001},"last":1.137},{"interface":"front","average":{"1min":6.6379999999999999,"5min":3.1709999999999998,"15min":1.8460000000000001},"min":{"1min":0.44600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":27.417999999999999,"5min":27.417999999999999,"15min":27.417999999999999},"last":1.1200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575210,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798968,"kb_used_data":9008,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572872,"statfs":{"total":96636764160,"available":95818620928,"internally_reserved":0,"allocated":9224192,"data_stored":8822112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":808909254},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.8919999999999999,"5min":1.8089999999999999,"15min":1.04},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":9.3499999999999996,"5min":15.81,"15min":15.81},"last":1.177},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.7050000000000001,"15min":1.056},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.8360000000000003,"5min":15.823,"15min":15.823},"last":1.0249999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":1.4139999999999999,"15min":0.91500000000000004},"min":{"1min":0.23699999999999999,"5min":0.193,"15min":0.193},"max":{"1min":9.4420000000000002,"5min":11.672000000000001,"15min":11.672000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.282,"5min":1.1919999999999999,"15min":0.90900000000000003},"min":{"1min":0.26000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.3440000000000003,"5min":11.722,"15min":11.722},"last":0.751}]},{"osd":3,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3719999999999999,"5min":1.901,"15min":1.171},"min":{"1min":0.26300000000000001,"5min":0.22,"15min":0.22},"max":{"1min":10.151999999999999,"5min":16.507999999999999,"15min":16.507999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.4790000000000001,"15min":1.0029999999999999},"min":{"1min":0.25600000000000001,"5min":0.218,"15min":0.218},"max":{"1min":5.024,"5min":12.047000000000001,"15min":12.047000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.6160000000000001,"5min":2.1040000000000001,"15min":1.2649999999999999},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":10.176,"5min":22.234000000000002,"15min":22.234000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":2.4510000000000001,"5min":2.2080000000000002,"15min":1.381},"min":{"1min":0.377,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":10.77,"5min":20.448,"15min":20.448},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.649,"15min":1.1910000000000001},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.7839999999999998,"5min":15.788,"15min":15.788},"last":0.82199999999999995},{"interface":"front","average":{"1min":2.105,"5min":1.8180000000000001,"15min":1.256},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":9.3000000000000007,"5min":11.731999999999999,"15min":11.731999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.4119999999999999,"15min":1.141},"min":{"1min":0.42499999999999999,"5min":0.255,"15min":0.255},"max":{"1min":9.1760000000000002,"5min":12.019,"15min":12.019},"last":1.0960000000000001},{"interface":"front","average":{"1min":2.1760000000000002,"5min":1.768,"15min":1.258},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":9.3339999999999996,"5min":11.628,"15min":11.628},"last":1.004}]},{"osd":7,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3809999999999998,"5min":1.9279999999999999,"15min":1.371},"min":{"1min":0.435,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":8.9689999999999994,"5min":21.161000000000001,"15min":21.161000000000001},"last":1.264},{"interface":"front","average":{"1min":4.2720000000000002,"5min":2.504,"15min":1.552},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":29.323,"5min":29.323,"15min":29.323},"last":1.1539999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575210,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799540,"kb_used_data":9580,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572300,"statfs":{"total":96636764160,"available":95818035200,"internally_reserved":0,"allocated":9809920,"data_stored":9412264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":808909256},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":1.474,"15min":0.92700000000000005},"min":{"1min":0.23300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":2.5870000000000002,"5min":59.031999999999996,"15min":59.031999999999996},"last":0.73499999999999999},{"interface":"front","average":{"1min":2.0249999999999999,"5min":2.222,"15min":1.196},"min":{"1min":0.20200000000000001,"5min":0.188,"15min":0.188},"max":{"1min":9.7110000000000003,"5min":57.186999999999998,"15min":57.186999999999998},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":0.93799999999999994,"15min":0.81200000000000006},"min":{"1min":0.28000000000000003,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":4.0919999999999996,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":0.90700000000000003},{"interface":"front","average":{"1min":2.0960000000000001,"5min":1.1899999999999999,"15min":0.84699999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":12.052,"5min":12.052,"15min":12.052},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.901,"5min":2.5739999999999998,"15min":1.353},"min":{"1min":0.26100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.6769999999999996,"5min":65.215000000000003,"15min":65.215000000000003},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.679,"15min":1.133},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.319,"5min":62.722999999999999,"15min":62.722999999999999},"last":1.002}]},{"osd":4,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":3.23,"5min":1.857,"15min":1.202},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":14.694000000000001,"5min":14.694000000000001,"15min":14.694000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.994,"5min":1.3360000000000001,"15min":1.095},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":9.3499999999999996,"5min":10.036,"15min":10.036},"last":1.0700000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":2.8300000000000001,"5min":1.748,"15min":1.1919999999999999},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":21.940999999999999,"5min":21.940999999999999,"15min":21.940999999999999},"last":1.163},{"interface":"front","average":{"1min":2.1440000000000001,"5min":1.962,"15min":1.3029999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.0620000000000003,"5min":9.3420000000000005,"15min":9.3420000000000005},"last":1.1319999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.7410000000000001,"5min":1.242,"15min":1.119},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":9.3840000000000003,"5min":9.3840000000000003,"15min":9.3840000000000003},"last":1.1990000000000001},{"interface":"front","average":{"1min":2.012,"5min":1.458,"15min":1.161},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.7060000000000004,"5min":8.157,"15min":8.157},"last":1.234}]},{"osd":7,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6140000000000001,"15min":1.2589999999999999},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":5.3209999999999997,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":1.274},{"interface":"front","average":{"1min":2.601,"5min":2.677,"15min":1.5920000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40300000000000002},"max":{"1min":8.7210000000000001,"5min":32.069000000000003,"15min":32.069000000000003},"last":1.3180000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":84545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8482816,"data_stored":8473825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7000064,"data_stored":6984672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6995968,"data_stored":6985218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10542867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4296704,"data_stored":4287702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:17:44.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:17:44.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb8f93640 1 Processor -- start 2024-09-13T17:17:44.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb8f93640 1 -- start start 2024-09-13T17:17:44.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb8f93640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdbb41526b0 0x7fdbb40c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb8f93640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdbb406c940 0x7fdbb40aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb8f93640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb414f790 con 0x7fdbb40ab100 2024-09-13T17:17:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb8f93640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb4152a90 con 0x7fdbb41526b0 2024-09-13T17:17:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb8f93640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb406ce40 con 0x7fdbb406c940 2024-09-13T17:17:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdbb41526b0 0x7fdbb40c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdbb41526b0 0x7fdbb40c12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40614/0 (socket says 172.21.15.39:40614) 2024-09-13T17:17:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb259c640 1 -- 172.21.15.39:0/95915547 learned_addr learned my addr 172.21.15.39:0/95915547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbabfff640 1 --2- 172.21.15.39:0/95915547 >> v2:172.21.15.84:3300/0 conn(0x7fdbb406c940 0x7fdbb40aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb259c640 1 -- 172.21.15.39:0/95915547 >> v2:172.21.15.84:3300/0 conn(0x7fdbb406c940 msgr2=0x7fdbb40aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb259c640 1 --2- 172.21.15.39:0/95915547 >> v2:172.21.15.84:3300/0 conn(0x7fdbb406c940 0x7fdbb40aabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb259c640 1 -- 172.21.15.39:0/95915547 >> v1:172.21.15.84:6789/0 conn(0x7fdbb40ab100 legacy=0x7fdbb41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb259c640 1 -- 172.21.15.39:0/95915547 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb406d120 con 0x7fdbb41526b0 2024-09-13T17:17:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.573+0000 7fdbb259c640 1 --2- 172.21.15.39:0/95915547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb41526b0 0x7fdbb40c12b0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40c33a0 tx=0x7fdba401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad43f5325c207704 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb1d9b640 1 -- 172.21.15.39:0/95915547 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40ab600 con 0x7fdbb41526b0 2024-09-13T17:17:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb1d9b640 1 -- 172.21.15.39:0/95915547 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba40a7050 con 0x7fdbb41526b0 2024-09-13T17:17:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb1d9b640 1 -- 172.21.15.39:0/95915547 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40ab980 con 0x7fdbb41526b0 2024-09-13T17:17:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- 172.21.15.39:0/95915547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb41526b0 msgr2=0x7fdbb40c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 --2- 172.21.15.39:0/95915547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb41526b0 0x7fdbb40c12b0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40c33a0 tx=0x7fdba401c4c0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- 172.21.15.39:0/95915547 shutdown_connections 2024-09-13T17:17:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 --2- 172.21.15.39:0/95915547 >> v2:172.21.15.84:3300/0 conn(0x7fdbb406c940 0x7fdbb40aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 --2- 172.21.15.39:0/95915547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb41526b0 0x7fdbb40c12b0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- 172.21.15.39:0/95915547 >> 172.21.15.39:0/95915547 conn(0x7fdbb4069e10 msgr2=0x7fdbb406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- 172.21.15.39:0/95915547 shutdown_connections 2024-09-13T17:17:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- 172.21.15.39:0/95915547 wait complete. 2024-09-13T17:17:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 Processor -- start 2024-09-13T17:17:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- start start 2024-09-13T17:17:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb406c940 0x7fdbb4144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdbb40ab100 0x7fdbb4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbb41526b0 0x7fdbb41ee190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb40a4fc0 con 0x7fdbb406c940 2024-09-13T17:17:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb40a8090 con 0x7fdbb41526b0 2024-09-13T17:17:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb40a7db0 con 0x7fdbb40ab100 2024-09-13T17:17:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb406c940 0x7fdbb4144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb406c940 0x7fdbb4144b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40630/0 (socket says 172.21.15.39:40630) 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb259c640 1 -- 172.21.15.39:0/463799229 learned_addr learned my addr 172.21.15.39:0/463799229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb2d9d640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbb41526b0 0x7fdbb41ee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb259c640 1 -- 172.21.15.39:0/463799229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdbb40ab100 msgr2=0x7fdbb4154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb259c640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdbb40ab100 0x7fdbb4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb259c640 1 -- 172.21.15.39:0/463799229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbb41526b0 msgr2=0x7fdbb41ee190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb259c640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbb41526b0 0x7fdbb41ee190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb259c640 1 -- 172.21.15.39:0/463799229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba40094d0 con 0x7fdbb406c940 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb259c640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb406c940 0x7fdbb4144b30 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fdba40ae040 tx=0x7fdba4019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbaaffd640 1 -- 172.21.15.39:0/463799229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40b4440 con 0x7fdbb406c940 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbaaffd640 1 -- 172.21.15.39:0/463799229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba40b45c0 con 0x7fdbb406c940 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb40a55c0 con 0x7fdbb406c940 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbaaffd640 1 -- 172.21.15.39:0/463799229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40b4970 con 0x7fdbb406c940 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.577+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb4154d90 con 0x7fdbb406c940 2024-09-13T17:17:44.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.581+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb40a5dc0 con 0x7fdbb406c940 2024-09-13T17:17:44.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.581+0000 7fdbaaffd640 1 -- 172.21.15.39:0/463799229 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdba40b2070 con 0x7fdbb406c940 2024-09-13T17:17:44.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.581+0000 7fdbaaffd640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb94066690 0x7fdb94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:44.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.581+0000 7fdbaaffd640 1 -- 172.21.15.39:0/463799229 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 9991+0+0 (secure 0 0 0) 0x7fdba411f100 con 0x7fdbb406c940 2024-09-13T17:17:44.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.581+0000 7fdbabfff640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb94066690 0x7fdb94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:44.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.581+0000 7fdbabfff640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb94066690 0x7fdb94068b50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdbb414f2a0 tx=0x7fdb9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.585+0000 7fdbaaffd640 1 -- 172.21.15.39:0/463799229 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdba40eb3f0 con 0x7fdbb406c940 2024-09-13T17:17:44.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.741+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbb40aabc0 con 0x7fdbb406c940 2024-09-13T17:17:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbaaffd640 1 -- 172.21.15.39:0/463799229 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+25425 (secure 0 0 0) 0x7fdba40eea10 con 0x7fdbb406c940 2024-09-13T17:17:44.748 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:44.748 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":63,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T17:16:34.950618+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6835/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6834/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.39:6834/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6837/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6835/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.84:6834/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.39:6836/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.39:6835/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.84:6835/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.39:6835/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.84:6834/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.39:6834/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.39:6834/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.39:6836/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.84:6832/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.84:6835/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.84:6832/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6836/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.84:6834/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.39:6835/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.84:6832/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6834/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6835/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.84:6834/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6835/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.39:6835/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6836/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6833/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.39:6835/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.39:6837/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6834/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6835/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6834/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.39:6835/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1750130579":"2024-09-14T17:09:29.828673+0000","172.21.15.39:6834/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.84:6832/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.39:6834/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.39:6837/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6836/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6835/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6834/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.84:6835/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.84:6833/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.84:6833/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6835/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.39:6834/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.84:6832/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6833/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6834/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6835/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6836/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6834/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6833/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6835/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6834/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.39:6834/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6835/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6836/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.39:6835/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.84:6833/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6836/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6837/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.84:6834/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.84:6834/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.84:6835/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.39:6837/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.39:6837/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6835/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.84:6832/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6835/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/1750130579":"2024-09-14T17:09:29.828673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:17:44.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb94066690 msgr2=0x7fdb94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb94066690 0x7fdb94068b50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdbb414f2a0 tx=0x7fdb9c008040 comp rx=0 tx=0).stop 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb406c940 msgr2=0x7fdbb4144b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb406c940 0x7fdbb4144b30 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fdba40ae040 tx=0x7fdba4019260 comp rx=0 tx=0).stop 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 shutdown_connections 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdb94066690 0x7fdb94068b50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbb41526b0 0x7fdbb41ee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fdbb40ab100 0x7fdbb4154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 --2- 172.21.15.39:0/463799229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb406c940 0x7fdbb4144b30 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 >> 172.21.15.39:0/463799229 conn(0x7fdbb4069e10 msgr2=0x7fdbb40c1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:44.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 shutdown_connections 2024-09-13T17:17:44.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:44.745+0000 7fdbb8f93640 1 -- 172.21.15.39:0/463799229 wait complete. 2024-09-13T17:17:45.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:45.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.261+0000 7f75ec593640 1 Processor -- start 2024-09-13T17:17:45.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75ec593640 1 -- start start 2024-09-13T17:17:45.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75ec593640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f75e406c870 0x7f75e40aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75ec593640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e40c40b0 0x7f75e415c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75ec593640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e4152a80 con 0x7f75e40aabc0 2024-09-13T17:17:45.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75ec593640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e40c30f0 con 0x7f75e406c870 2024-09-13T17:17:45.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75ec593640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e4151e00 con 0x7f75e40c40b0 2024-09-13T17:17:45.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75eab30640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e40c40b0 0x7f75e415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75ea32f640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f75e40aabc0 0x7f75e40c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:60846/0 (socket says 172.21.15.39:60846) 2024-09-13T17:17:45.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75ea32f640 1 -- 172.21.15.39:0/825476771 learned_addr learned my addr 172.21.15.39:0/825476771 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:45.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75e9b2e640 1 --2- 172.21.15.39:0/825476771 >> v2:172.21.15.84:3300/0 conn(0x7f75e406c870 0x7f75e40aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75eab30640 1 -- 172.21.15.39:0/825476771 >> v2:172.21.15.84:3300/0 conn(0x7f75e406c870 msgr2=0x7f75e40aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:45.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75eab30640 1 --2- 172.21.15.39:0/825476771 >> v2:172.21.15.84:3300/0 conn(0x7f75e406c870 0x7f75e40aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75eab30640 1 -- 172.21.15.39:0/825476771 >> v1:172.21.15.84:6789/0 conn(0x7f75e40aabc0 legacy=0x7f75e40c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75eab30640 1 -- 172.21.15.39:0/825476771 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e41520e0 con 0x7f75e40c40b0 2024-09-13T17:17:45.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.265+0000 7f75e9b2e640 1 --2- 172.21.15.39:0/825476771 >> v2:172.21.15.84:3300/0 conn(0x7f75e406c870 0x7f75e40aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:17:45.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75eab30640 1 --2- 172.21.15.39:0/825476771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e40c40b0 0x7f75e415c470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0097c0 tx=0x7f75dc01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f92658872466bf5f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75e932d640 1 -- 172.21.15.39:0/825476771 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0ac640 con 0x7f75e40c40b0 2024-09-13T17:17:45.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75e932d640 1 -- 172.21.15.39:0/825476771 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc0ac7a0 con 0x7f75e40c40b0 2024-09-13T17:17:45.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75e932d640 1 -- 172.21.15.39:0/825476771 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0acb30 con 0x7f75e40c40b0 2024-09-13T17:17:45.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 -- 172.21.15.39:0/825476771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e40c40b0 msgr2=0x7f75e415c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 --2- 172.21.15.39:0/825476771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e40c40b0 0x7f75e415c470 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0097c0 tx=0x7f75dc01c3e0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 -- 172.21.15.39:0/825476771 shutdown_connections 2024-09-13T17:17:45.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 --2- 172.21.15.39:0/825476771 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e40c40b0 0x7f75e415c470 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 --2- 172.21.15.39:0/825476771 >> v2:172.21.15.84:3300/0 conn(0x7f75e406c870 0x7f75e40aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 -- 172.21.15.39:0/825476771 >> 172.21.15.39:0/825476771 conn(0x7f75e4069e10 msgr2=0x7f75e406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:45.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 -- 172.21.15.39:0/825476771 shutdown_connections 2024-09-13T17:17:45.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 -- 172.21.15.39:0/825476771 wait complete. 2024-09-13T17:17:45.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 Processor -- start 2024-09-13T17:17:45.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 -- start start 2024-09-13T17:17:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e406c870 0x7f75e40bee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75e40aabc0 0x7f75e414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40c40b0 0x7f75e41ec530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40a7a10 con 0x7f75e40c40b0 2024-09-13T17:17:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40a7cf0 con 0x7f75e40aabc0 2024-09-13T17:17:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ec593640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e4071ab0 con 0x7f75e406c870 2024-09-13T17:17:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75eab30640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40c40b0 0x7f75e41ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75eab30640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40c40b0 0x7f75e41ec530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40644/0 (socket says 172.21.15.39:40644) 2024-09-13T17:17:45.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75eab30640 1 -- 172.21.15.39:0/2538091229 learned_addr learned my addr 172.21.15.39:0/2538091229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75e9b2e640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75e40aabc0 0x7f75e414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75ea32f640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e406c870 0x7f75e40bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75eab30640 1 -- 172.21.15.39:0/2538091229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e406c870 msgr2=0x7f75e40bee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75eab30640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e406c870 0x7f75e40bee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75eab30640 1 -- 172.21.15.39:0/2538091229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75e40aabc0 msgr2=0x7f75e414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75eab30640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75e40aabc0 0x7f75e414c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.269+0000 7f75eab30640 1 -- 172.21.15.39:0/2538091229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0009a70 con 0x7f75e40c40b0 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75eab30640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40c40b0 0x7f75e41ec530 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f75dc01a010 tx=0x7f75dc019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75d6ffd640 1 -- 172.21.15.39:0/2538091229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b5600 con 0x7f75e40c40b0 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc009420 con 0x7f75e40c40b0 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75d6ffd640 1 -- 172.21.15.39:0/2538091229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc0b5780 con 0x7f75e40c40b0 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75d6ffd640 1 -- 172.21.15.39:0/2538091229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b5b30 con 0x7f75e40c40b0 2024-09-13T17:17:45.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e4072030 con 0x7f75e40c40b0 2024-09-13T17:17:45.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e40a4fa0 con 0x7f75e40c40b0 2024-09-13T17:17:45.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75d6ffd640 1 -- 172.21.15.39:0/2538091229 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f75dc0b3040 con 0x7f75e40c40b0 2024-09-13T17:17:45.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75d6ffd640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75c8066690 0x7f75c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75d6ffd640 1 -- 172.21.15.39:0/2538091229 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f75dc11f5a0 con 0x7f75e40c40b0 2024-09-13T17:17:45.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75ea32f640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75c8066690 0x7f75c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.273+0000 7f75ea32f640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75c8066690 0x7f75c8068b50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f75d8005890 tx=0x7f75d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.277+0000 7f75d6ffd640 1 -- 172.21.15.39:0/2538091229 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f75dc0eb870 con 0x7f75e40c40b0 2024-09-13T17:17:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.473+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75e40a7e60 con 0x7f75e40c40b0 2024-09-13T17:17:45.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75d6ffd640 1 -- 172.21.15.39:0/2538091229 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+5181 (secure 0 0 0) 0x7f75dc0eee90 con 0x7f75e40c40b0 2024-09-13T17:17:45.479 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:45.480 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":203,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:17:44.964705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7555":{"gid":7555,"name":"d","rank":0,"incarnation":203,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 203 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75c8066690 msgr2=0x7f75c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75c8066690 0x7f75c8068b50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f75d8005890 tx=0x7f75d8009040 comp rx=0 tx=0).stop 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40c40b0 msgr2=0x7f75e41ec530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40c40b0 0x7f75e41ec530 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f75dc01a010 tx=0x7f75dc019180 comp rx=0 tx=0).stop 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 shutdown_connections 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f75c8066690 0x7f75c8068b50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75e40aabc0 0x7f75e414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40c40b0 0x7f75e41ec530 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 --2- 172.21.15.39:0/2538091229 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f75e406c870 0x7f75e40bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 >> 172.21.15.39:0/2538091229 conn(0x7f75e4069e10 msgr2=0x7f75e414ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 shutdown_connections 2024-09-13T17:17:45.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.477+0000 7f75ec593640 1 -- 172.21.15.39:0/2538091229 wait complete. 2024-09-13T17:17:45.499 INFO:tasks.mds_thrash.fs.[a]:mds.a down, removed from mdsmap 2024-09-13T17:17:45.500 INFO:tasks.mds_thrash.fs.[a]:waiting for mds cluster to stabilize... 2024-09-13T17:17:45.500 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:45.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8883a640 1 Processor -- start 2024-09-13T17:17:45.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8883a640 1 -- start start 2024-09-13T17:17:45.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8883a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f840a4e70 0x7f6f8406cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8883a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f840c2780 con 0x7f6f8406d300 2024-09-13T17:17:45.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8883a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f840c2a60 con 0x7f6f8414e0c0 2024-09-13T17:17:45.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8883a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f840c2d40 con 0x7f6f840a4e70 2024-09-13T17:17:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f82d9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f6f8406d300 0x7f6f840c4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40214/0 (socket says 172.21.15.39:40214) 2024-09-13T17:17:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f82d9d640 1 -- 172.21.15.39:0/4142364146 learned_addr learned my addr 172.21.15.39:0/4142364146 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f81d9b640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f840a4e70 0x7f6f8406cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873531822 0 0) 0x7f6f840c2780 con 0x7f6f8406d300 2024-09-13T17:17:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f6c003530 con 0x7f6f8406d300 2024-09-13T17:17:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2802407103 0 0) 0x7f6f6c003530 con 0x7f6f8406d300 2024-09-13T17:17:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f840a4e70 msgr2=0x7f6f8406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f840a4e70 0x7f6f8406cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 >> v1:172.21.15.84:6789/0 conn(0x7f6f8414e0c0 legacy=0x7f6f840c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840c3020 con 0x7f6f8406d300 2024-09-13T17:17:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f6f7c001a00 con 0x7f6f8406d300 2024-09-13T17:17:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 >> v1:172.21.15.39:6789/0 conn(0x7f6f8406d300 legacy=0x7f6f840c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f840a4e70 0x7f6f6c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f6c003ef0 0x7f6f6c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c004d10 0x7f6f6c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f840c2d40 con 0x7f6f6c003ef0 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c00bad0 con 0x7f6f6c004d10 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c00dea0 con 0x7f6f840a4e70 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f81d9b640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f6c003ef0 0x7f6f6c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f82d9d640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c004d10 0x7f6f6c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f8259c640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f840a4e70 0x7f6f6c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f81d9b640 1 -- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f840a4e70 msgr2=0x7f6f6c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f81d9b640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f840a4e70 0x7f6f6c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f81d9b640 1 -- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c004d10 msgr2=0x7f6f6c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f81d9b640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c004d10 0x7f6f6c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.613+0000 7f6f81d9b640 1 -- 172.21.15.39:0/4142364146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c003530 con 0x7f6f6c003ef0 2024-09-13T17:17:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f81d9b640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f6c003ef0 0x7f6f6c0047d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009ae0 tx=0x7f6f7801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9572b2a0649228bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f780a5660 con 0x7f6f6c003ef0 2024-09-13T17:17:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f780a57c0 con 0x7f6f6c003ef0 2024-09-13T17:17:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8159a640 1 -- 172.21.15.39:0/4142364146 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f780a5b70 con 0x7f6f6c003ef0 2024-09-13T17:17:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8883a640 1 -- 172.21.15.39:0/4142364146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f6c003ef0 msgr2=0x7f6f6c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8883a640 1 --2- 172.21.15.39:0/4142364146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f6c003ef0 0x7f6f6c0047d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009ae0 tx=0x7f6f7801ca70 comp rx=0 tx=0).stop 2024-09-13T17:17:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f82d9d640 1 -- 172.21.15.39:0/4142364146 reap_dead start 2024-09-13T17:17:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8883a640 1 -- 172.21.15.39:0/4142364146 shutdown_connections 2024-09-13T17:17:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8883a640 1 -- 172.21.15.39:0/4142364146 >> 172.21.15.39:0/4142364146 conn(0x7f6f84069e10 msgr2=0x7f6f8406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8883a640 1 -- 172.21.15.39:0/4142364146 shutdown_connections 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8883a640 1 -- 172.21.15.39:0/4142364146 wait complete. 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8883a640 1 Processor -- start 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.617+0000 7f6f8883a640 1 -- start start 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8883a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c003ef0 0x7f6f8414a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8883a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f841415b0 0x7f6f841ec920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8883a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f8413f930 0x7f6f841ece60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8883a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841ec460 con 0x7f6f8413f930 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8883a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f840bf690 con 0x7f6f6c003ef0 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8883a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841ed3a0 con 0x7f6f841415b0 2024-09-13T17:17:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f82d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f8413f930 0x7f6f841ece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f82d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f8413f930 0x7f6f841ece60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40668/0 (socket says 172.21.15.39:40668) 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f82d9d640 1 -- 172.21.15.39:0/3980154416 learned_addr learned my addr 172.21.15.39:0/3980154416 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f81d9b640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f841415b0 0x7f6f841ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8259c640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c003ef0 0x7f6f8414a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f82d9d640 1 -- 172.21.15.39:0/3980154416 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f841415b0 msgr2=0x7f6f841ec920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f82d9d640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f841415b0 0x7f6f841ec920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f82d9d640 1 -- 172.21.15.39:0/3980154416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c003ef0 msgr2=0x7f6f8414a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f82d9d640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c003ef0 0x7f6f8414a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f82d9d640 1 -- 172.21.15.39:0/3980154416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8414e620 con 0x7f6f8413f930 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8259c640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c003ef0 0x7f6f8414a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f82d9d640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f8413f930 0x7f6f841ece60 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6f840a7560 tx=0x7f6f7c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f72ffd640 1 -- 172.21.15.39:0/3980154416 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00ec10 con 0x7f6f8413f930 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f72ffd640 1 -- 172.21.15.39:0/3980154416 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c00ed70 con 0x7f6f8413f930 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f84140140 con 0x7f6f8413f930 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f72ffd640 1 -- 172.21.15.39:0/3980154416 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00eee0 con 0x7f6f8413f930 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f840bfdd0 con 0x7f6f8413f930 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f840c0c20 con 0x7f6f8413f930 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f72ffd640 1 -- 172.21.15.39:0/3980154416 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f7c014650 con 0x7f6f8413f930 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f72ffd640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6f640666c0 0x7f6f64068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f72ffd640 1 -- 172.21.15.39:0/3980154416 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f6f7c084e90 con 0x7f6f8413f930 2024-09-13T17:17:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8259c640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6f640666c0 0x7f6f64068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.621+0000 7f6f8259c640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6f640666c0 0x7f6f64068b80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f74000c10 tx=0x7f6f7400a9b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.625+0000 7f6f72ffd640 1 -- 172.21.15.39:0/3980154416 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6f840c0c20 con 0x7f6f8413f930 2024-09-13T17:17:45.767 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T17:17:45.767 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:17:45.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.817+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f840c0c20 con 0x7f6f8413f930 2024-09-13T17:17:45.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f72ffd640 1 -- 172.21.15.39:0/3980154416 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+5181 (secure 0 0 0) 0x7f6f840c0c20 con 0x7f6f8413f930 2024-09-13T17:17:45.823 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:45.823 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":203,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:17:44.964705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7555":{"gid":7555,"name":"d","rank":0,"incarnation":203,"state":"up:replay","state_seq":1,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:45.823 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 203 2024-09-13T17:17:45.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6f640666c0 msgr2=0x7f6f64068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6f640666c0 0x7f6f64068b80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f74000c10 tx=0x7f6f7400a9b0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f8413f930 msgr2=0x7f6f841ece60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f8413f930 0x7f6f841ece60 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6f840a7560 tx=0x7f6f7c00e530 comp rx=0 tx=0).stop 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 shutdown_connections 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6f640666c0 0x7f6f64068b80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f8413f930 0x7f6f841ece60 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6f841415b0 0x7f6f841ec920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 --2- 172.21.15.39:0/3980154416 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f6c003ef0 0x7f6f8414a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 >> 172.21.15.39:0/3980154416 conn(0x7f6f84069e10 msgr2=0x7f6f84143620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 shutdown_connections 2024-09-13T17:17:45.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.821+0000 7f6f8883a640 1 -- 172.21.15.39:0/3980154416 wait complete. 2024-09-13T17:17:45.842 INFO:tasks.mds_thrash.fs.[a]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 203, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:47:59.023080+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 203, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "a", "in": [ 0 ], "info": { "gid_7555": { "addr": "172.21.15.84:6835/3465334740", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6834", "nonce": 3465334740, "type": "v2" }, { "addr": "172.21.15.84:6835", "nonce": 3465334740, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7555, "incarnation": 203, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-09-13T17:17:44.964705+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7555 } } }, { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-09-13T16:48:08.135456+0000", "damaged": [], "data_pools": [ 5 ], "enabled": true, "epoch": 196, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "b", "in": [ 0 ], "info": { "gid_6638": { "addr": "172.21.15.39:6837/3255731814", "addrs": { "addrvec": [ { "addr": "172.21.15.39:6836", "nonce": 3255731814, "type": "v2" }, { "addr": "172.21.15.39:6837", "nonce": 3255731814, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6638, "incarnation": 193, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 62, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2024-09-13T17:16:08.963924+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6638 } } } ], "standbys": [ { "addr": "172.21.15.84:6833/641053781", "addrs": { "addrvec": [ { "addr": "172.21.15.84:6832", "nonce": 641053781, "type": "v2" }, { "addr": "172.21.15.84:6833", "nonce": 641053781, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 151, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6227, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-09-13T17:17:45.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb8530e8640 1 Processor -- start 2024-09-13T17:17:45.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb8530e8640 1 -- start start 2024-09-13T17:17:45.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb8530e8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb84c0bd020 0x7fb84c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb8530e8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c154420 0x7fb84c1548d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb8530e8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84c154e10 con 0x7fb84c151880 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb8530e8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84c0c5350 con 0x7fb84c0bd020 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb8530e8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c153f40 con 0x7fb84c154420 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb84bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb84c0bd020 0x7fb84c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb850e84640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb84c151880 0x7fb84c0c4310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40224/0 (socket says 172.21.15.39:40224) 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb850e84640 1 -- 172.21.15.39:0/1175845116 learned_addr learned my addr 172.21.15.39:0/1175845116 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb851685640 1 --2- 172.21.15.39:0/1175845116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c154420 0x7fb84c1548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb84b7fe640 1 -- 172.21.15.39:0/1175845116 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965518636 0 0) 0x7fb84c154e10 con 0x7fb84c151880 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb84b7fe640 1 -- 172.21.15.39:0/1175845116 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb834003530 con 0x7fb84c151880 2024-09-13T17:17:45.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb84bfff640 1 -- 172.21.15.39:0/1175845116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c154420 msgr2=0x7fb84c1548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:45.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb84bfff640 1 --2- 172.21.15.39:0/1175845116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c154420 0x7fb84c1548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb84bfff640 1 -- 172.21.15.39:0/1175845116 >> v1:172.21.15.39:6789/0 conn(0x7fb84c151880 legacy=0x7fb84c0c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb84bfff640 1 -- 172.21.15.39:0/1175845116 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c0ca120 con 0x7fb84c0bd020 2024-09-13T17:17:45.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.885+0000 7fb851685640 1 --2- 172.21.15.39:0/1175845116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c154420 0x7fb84c1548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:45.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb84bfff640 1 --2- 172.21.15.39:0/1175845116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c0bd020 0x7fb84c157e00 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb8400097c0 tx=0x7fb84001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a01181993a86d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb84b7fe640 1 -- 172.21.15.39:0/1175845116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8400ab600 con 0x7fb84c0bd020 2024-09-13T17:17:45.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb84b7fe640 1 -- 172.21.15.39:0/1175845116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8400a7050 con 0x7fb84c0bd020 2024-09-13T17:17:45.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb84b7fe640 1 -- 172.21.15.39:0/1175845116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8400ab980 con 0x7fb84c0bd020 2024-09-13T17:17:45.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- 172.21.15.39:0/1175845116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c0bd020 msgr2=0x7fb84c157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 --2- 172.21.15.39:0/1175845116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c0bd020 0x7fb84c157e00 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb8400097c0 tx=0x7fb84001c410 comp rx=0 tx=0).stop 2024-09-13T17:17:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- 172.21.15.39:0/1175845116 shutdown_connections 2024-09-13T17:17:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 --2- 172.21.15.39:0/1175845116 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c154420 0x7fb84c1548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 --2- 172.21.15.39:0/1175845116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c0bd020 0x7fb84c157e00 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- 172.21.15.39:0/1175845116 >> 172.21.15.39:0/1175845116 conn(0x7fb84c06a210 msgr2=0x7fb84c06a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- 172.21.15.39:0/1175845116 shutdown_connections 2024-09-13T17:17:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- 172.21.15.39:0/1175845116 wait complete. 2024-09-13T17:17:45.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 Processor -- start 2024-09-13T17:17:45.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- start start 2024-09-13T17:17:45.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb84c0bd020 0x7fb84c14ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c151880 0x7fb84c14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c154420 0x7fb84c1ec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0c0100 con 0x7fb84c154420 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c148d80 con 0x7fb84c0bd020 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c0bfa40 con 0x7fb84c151880 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb851685640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c154420 0x7fb84c1ec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb851685640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c154420 0x7fb84c1ec1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40692/0 (socket says 172.21.15.39:40692) 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb851685640 1 -- 172.21.15.39:0/2990492072 learned_addr learned my addr 172.21.15.39:0/2990492072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb850e84640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb84c0bd020 0x7fb84c14ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb84bfff640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c151880 0x7fb84c14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb851685640 1 -- 172.21.15.39:0/2990492072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c151880 msgr2=0x7fb84c14d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb851685640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb84c151880 0x7fb84c14d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb851685640 1 -- 172.21.15.39:0/2990492072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb84c0bd020 msgr2=0x7fb84c14ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb851685640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb84c0bd020 0x7fb84c14ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb851685640 1 -- 172.21.15.39:0/2990492072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840009420 con 0x7fb84c154420 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb850e84640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb84c0bd020 0x7fb84c14ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb851685640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c154420 0x7fb84c1ec1a0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb83c007df0 tx=0x7fb83c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8497fa640 1 -- 172.21.15.39:0/2990492072 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0039a0 con 0x7fb84c154420 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8497fa640 1 -- 172.21.15.39:0/2990492072 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c00f040 con 0x7fb84c154420 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84c0bfd20 con 0x7fb84c154420 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8497fa640 1 -- 172.21.15.39:0/2990492072 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c003d40 con 0x7fb84c154420 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.889+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c1ece70 con 0x7fb84c154420 2024-09-13T17:17:45.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb84c0710e0 con 0x7fb84c154420 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb8497fa640 1 -- 172.21.15.39:0/2990492072 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb83c004030 con 0x7fb84c154420 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb8497fa640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb838066690 0x7fb838068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb8497fa640 1 -- 172.21.15.39:0/2990492072 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7fb83c085550 con 0x7fb84c154420 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb850e84640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb838066690 0x7fb838068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb8497fa640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fb838073070 0x7fb8380754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb8497fa640 1 -- 172.21.15.39:0/2990492072 --> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb838075bc0 con 0x7fb838073070 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb8497fa640 1 -- 172.21.15.39:0/2990492072 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb83c087bb0 con 0x7fb84c154420 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb84bfff640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fb838073070 0x7fb8380754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb850e84640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb838066690 0x7fb838068b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb8440011f0 tx=0x7fb844009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb84bfff640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fb838073070 0x7fb8380754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:45.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.893+0000 7fb8497fa640 1 -- 172.21.15.39:0/2990492072 <== osd.0 v2:172.21.15.39:6800/3633417446 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb838075bc0 con 0x7fb838073070 2024-09-13T17:17:45.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.905+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 --> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb84c0bfbb0 con 0x7fb838073070 2024-09-13T17:17:45.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.905+0000 7fb8497fa640 1 -- 172.21.15.39:0/2990492072 <== osd.0 v2:172.21.15.39:6800/3633417446 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb84c0bfbb0 con 0x7fb838073070 2024-09-13T17:17:45.910 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T17:17:45.910 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:17:45.910 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T17:17:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.905+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fb838073070 msgr2=0x7fb8380754f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.905+0000 7fb8530e8640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6800/3633417446,v1:172.21.15.39:6801/3633417446] conn(0x7fb838073070 0x7fb8380754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.909+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb838066690 msgr2=0x7fb838068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.909+0000 7fb8530e8640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb838066690 0x7fb838068b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb8440011f0 tx=0x7fb844009040 comp rx=0 tx=0).stop 2024-09-13T17:17:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.909+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c154420 msgr2=0x7fb84c1ec1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.909+0000 7fb8530e8640 1 --2- 172.21.15.39:0/2990492072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb84c154420 0x7fb84c1ec1a0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb83c007df0 tx=0x7fb83c00d4a0 comp rx=0 tx=0).stop 2024-09-13T17:17:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.909+0000 7fb851685640 1 -- 172.21.15.39:0/2990492072 reap_dead start 2024-09-13T17:17:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.909+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 shutdown_connections 2024-09-13T17:17:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.909+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 >> 172.21.15.39:0/2990492072 conn(0x7fb84c06a210 msgr2=0x7fb84c1548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.909+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 shutdown_connections 2024-09-13T17:17:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:45.909+0000 7fb8530e8640 1 -- 172.21.15.39:0/2990492072 wait complete. 2024-09-13T17:17:45.926 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T17:17:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.033+0000 7f6de2f15640 1 Processor -- start 2024-09-13T17:17:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.033+0000 7f6de2f15640 1 -- start start 2024-09-13T17:17:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6ddc071dd0 0x7f6ddc154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc0c7800 0x7f6ddc1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ddc0c5a30 con 0x7f6ddc151500 2024-09-13T17:17:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ddc1576a0 con 0x7f6ddc071dd0 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc157980 con 0x7f6ddc0c7800 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6ddbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6ddc071dd0 0x7f6ddc154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de0cb1640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f6ddc151500 0x7f6ddc0c4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40236/0 (socket says 172.21.15.39:40236) 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de0cb1640 1 -- 172.21.15.39:0/463259038 learned_addr learned my addr 172.21.15.39:0/463259038 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de14b2640 1 --2- 172.21.15.39:0/463259038 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc0c7800 0x7f6ddc1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6ddbfff640 1 -- 172.21.15.39:0/463259038 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc0c7800 msgr2=0x7f6ddc1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6ddbfff640 1 --2- 172.21.15.39:0/463259038 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc0c7800 0x7f6ddc1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6ddbfff640 1 -- 172.21.15.39:0/463259038 >> v1:172.21.15.39:6789/0 conn(0x7f6ddc151500 legacy=0x7f6ddc0c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6ddbfff640 1 -- 172.21.15.39:0/463259038 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc157c60 con 0x7f6ddc071dd0 2024-09-13T17:17:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de14b2640 1 --2- 172.21.15.39:0/463259038 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc0c7800 0x7f6ddc1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6ddbfff640 1 --2- 172.21.15.39:0/463259038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc071dd0 0x7f6ddc154070 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00097f0 tx=0x7f6dd001ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795fbbcd12c7f500 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6ddb7fe640 1 -- 172.21.15.39:0/463259038 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00ab600 con 0x7f6ddc071dd0 2024-09-13T17:17:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6ddb7fe640 1 -- 172.21.15.39:0/463259038 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd00ab760 con 0x7f6ddc071dd0 2024-09-13T17:17:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6ddb7fe640 1 -- 172.21.15.39:0/463259038 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00abaf0 con 0x7f6ddc071dd0 2024-09-13T17:17:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 -- 172.21.15.39:0/463259038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc071dd0 msgr2=0x7f6ddc154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 --2- 172.21.15.39:0/463259038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc071dd0 0x7f6ddc154070 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00097f0 tx=0x7f6dd001ce40 comp rx=0 tx=0).stop 2024-09-13T17:17:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 -- 172.21.15.39:0/463259038 shutdown_connections 2024-09-13T17:17:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 --2- 172.21.15.39:0/463259038 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc0c7800 0x7f6ddc1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 --2- 172.21.15.39:0/463259038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc071dd0 0x7f6ddc154070 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 -- 172.21.15.39:0/463259038 >> 172.21.15.39:0/463259038 conn(0x7f6ddc0ab530 msgr2=0x7f6ddc0ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 -- 172.21.15.39:0/463259038 shutdown_connections 2024-09-13T17:17:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.037+0000 7f6de2f15640 1 -- 172.21.15.39:0/463259038 wait complete. 2024-09-13T17:17:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 Processor -- start 2024-09-13T17:17:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 -- start start 2024-09-13T17:17:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc071dd0 0x7f6ddc153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ddc0c7800 0x7f6ddc149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc151500 0x7f6ddc0aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0aad80 con 0x7f6ddc151500 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc14bab0 con 0x7f6ddc0c7800 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0aafd0 con 0x7f6ddc071dd0 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de14b2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc151500 0x7f6ddc0aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de14b2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc151500 0x7f6ddc0aa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40700/0 (socket says 172.21.15.39:40700) 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de14b2640 1 -- 172.21.15.39:0/1814370192 learned_addr learned my addr 172.21.15.39:0/1814370192 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6ddbfff640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ddc0c7800 0x7f6ddc149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de0cb1640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc071dd0 0x7f6ddc153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de14b2640 1 -- 172.21.15.39:0/1814370192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc071dd0 msgr2=0x7f6ddc153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de14b2640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc071dd0 0x7f6ddc153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de14b2640 1 -- 172.21.15.39:0/1814370192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ddc0c7800 msgr2=0x7f6ddc149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de14b2640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ddc0c7800 0x7f6ddc149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de14b2640 1 -- 172.21.15.39:0/1814370192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd0009420 con 0x7f6ddc151500 2024-09-13T17:17:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6ddbfff640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ddc0c7800 0x7f6ddc149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de14b2640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc151500 0x7f6ddc0aa350 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400cea0 tx=0x7f6dd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6dd97fa640 1 -- 172.21.15.39:0/1814370192 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4003ba0 con 0x7f6ddc151500 2024-09-13T17:17:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6dd97fa640 1 -- 172.21.15.39:0/1814370192 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd4003d20 con 0x7f6ddc151500 2024-09-13T17:17:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6dd97fa640 1 -- 172.21.15.39:0/1814370192 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400d880 con 0x7f6ddc151500 2024-09-13T17:17:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc14bd50 con 0x7f6ddc151500 2024-09-13T17:17:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc149ed0 con 0x7f6ddc151500 2024-09-13T17:17:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc14c200 con 0x7f6ddc151500 2024-09-13T17:17:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6dd97fa640 1 -- 172.21.15.39:0/1814370192 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6dd400d9e0 con 0x7f6ddc151500 2024-09-13T17:17:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.041+0000 7f6dd97fa640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6dc8066690 0x7f6dc8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.045+0000 7f6dd97fa640 1 -- 172.21.15.39:0/1814370192 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f6dd4085b70 con 0x7f6ddc151500 2024-09-13T17:17:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.045+0000 7f6de0cb1640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6dc8066690 0x7f6dc8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.045+0000 7f6de0cb1640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6dc8066690 0x7f6dc8068b50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000f70 tx=0x7f6dcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.049+0000 7f6dd97fa640 1 -- 172.21.15.39:0/1814370192 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6ddc14c200 con 0x7f6ddc151500 2024-09-13T17:17:46.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.201+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ddc1518e0 con 0x7f6dc8066690 2024-09-13T17:17:46.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.201+0000 7f6dd97fa640 1 -- 172.21.15.39:0/1814370192 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6dcc001340 con 0x7f6dc8066690 2024-09-13T17:17:46.206 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T17:17:46.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6dc8066690 msgr2=0x7f6dc8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6dc8066690 0x7f6dc8068b50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000f70 tx=0x7f6dcc009040 comp rx=0 tx=0).stop 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc151500 msgr2=0x7f6ddc0aa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc151500 0x7f6ddc0aa350 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400cea0 tx=0x7f6dd400cf90 comp rx=0 tx=0).stop 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 shutdown_connections 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f6dc8066690 0x7f6dc8068b50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6ddc0c7800 0x7f6ddc149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ddc151500 0x7f6ddc0aa350 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 --2- 172.21.15.39:0/1814370192 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6ddc071dd0 0x7f6ddc153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 >> 172.21.15.39:0/1814370192 conn(0x7f6ddc0ab530 msgr2=0x7f6ddc070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 shutdown_connections 2024-09-13T17:17:46.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.205+0000 7f6de2f15640 1 -- 172.21.15.39:0/1814370192 wait complete. 2024-09-13T17:17:46.224 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T17:17:46.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:17:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f283bd94640 1 Processor -- start 2024-09-13T17:17:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f283bd94640 1 -- start start 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f283bd94640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28340c4920 0x7f2834153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f283bd94640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28340c9c10 con 0x7f2834151460 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f283bd94640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2834158100 con 0x7f28340714b0 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f283bd94640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28340c9e60 con 0x7f28340c4920 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2839b30640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2834151460 0x7f28340c4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40252/0 (socket says 172.21.15.39:40252) 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2839b30640 1 -- 172.21.15.39:0/4208843566 learned_addr learned my addr 172.21.15.39:0/4208843566 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f283932f640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28340c4920 0x7f2834153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725886306 0 0) 0x7f28340c9c10 con 0x7f2834151460 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2820003530 con 0x7f2834151460 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252997112 0 0) 0x7f2820003530 con 0x7f2834151460 2024-09-13T17:17:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28340c4920 msgr2=0x7f2834153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28340c4920 0x7f2834153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 >> v1:172.21.15.84:6789/0 conn(0x7f28340714b0 legacy=0x7f2834154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28340ca140 con 0x7f2834151460 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f2830001680 con 0x7f2834151460 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 >> v1:172.21.15.39:6789/0 conn(0x7f2834151460 legacy=0x7f28340c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28200039b0 0x7f2820003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28340c4920 0x7f28200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2820004d10 0x7f2820005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28340c9e60 con 0x7f28200039b0 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282000baf0 con 0x7f2820004d10 2024-09-13T17:17:46.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282000bcb0 con 0x7f28340c4920 2024-09-13T17:17:46.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2839b30640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28200039b0 0x7f2820003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f283932f640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28340c4920 0x7f28200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f283a331640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2820004d10 0x7f2820005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2839b30640 1 -- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28340c4920 msgr2=0x7f28200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2839b30640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28340c4920 0x7f28200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2839b30640 1 -- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2820004d10 msgr2=0x7f2820005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2839b30640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2820004d10 0x7f2820005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.345+0000 7f2839b30640 1 -- 172.21.15.39:0/4208843566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820003530 con 0x7f28200039b0 2024-09-13T17:17:46.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f2839b30640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28200039b0 0x7f2820003dd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f283000c1c0 tx=0x7f283001cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b11a10df8836786 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830005ae0 con 0x7f28200039b0 2024-09-13T17:17:46.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2830005c40 con 0x7f28200039b0 2024-09-13T17:17:46.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f2838b2e640 1 -- 172.21.15.39:0/4208843566 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28300a5770 con 0x7f28200039b0 2024-09-13T17:17:46.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f283bd94640 1 -- 172.21.15.39:0/4208843566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28200039b0 msgr2=0x7f2820003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f283bd94640 1 --2- 172.21.15.39:0/4208843566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28200039b0 0x7f2820003dd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f283000c1c0 tx=0x7f283001cb80 comp rx=0 tx=0).stop 2024-09-13T17:17:46.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f283a331640 1 -- 172.21.15.39:0/4208843566 reap_dead start 2024-09-13T17:17:46.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f283bd94640 1 -- 172.21.15.39:0/4208843566 shutdown_connections 2024-09-13T17:17:46.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f283bd94640 1 -- 172.21.15.39:0/4208843566 >> 172.21.15.39:0/4208843566 conn(0x7f2834069fe0 msgr2=0x7f283406a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f283bd94640 1 -- 172.21.15.39:0/4208843566 shutdown_connections 2024-09-13T17:17:46.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.349+0000 7f283bd94640 1 -- 172.21.15.39:0/4208843566 wait complete. 2024-09-13T17:17:46.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 Processor -- start 2024-09-13T17:17:46.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 -- start start 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28200039b0 0x7f28341f0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28340ae820 0x7f28340aa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28340aac70 0x7f28340ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283415ca00 con 0x7f28340ae820 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28341f0fd0 con 0x7f28340aac70 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28341f0c80 con 0x7f28200039b0 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283932f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28340ae820 0x7f28340aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283932f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28340ae820 0x7f28340aa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40718/0 (socket says 172.21.15.39:40718) 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283932f640 1 -- 172.21.15.39:0/3197832149 learned_addr learned my addr 172.21.15.39:0/3197832149 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f2839b30640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28200039b0 0x7f28341f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283a331640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28340aac70 0x7f28340ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283932f640 1 -- 172.21.15.39:0/3197832149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28200039b0 msgr2=0x7f28341f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283932f640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f28200039b0 0x7f28341f0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283932f640 1 -- 172.21.15.39:0/3197832149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28340aac70 msgr2=0x7f28340ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283932f640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28340aac70 0x7f28340ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283932f640 1 -- 172.21.15.39:0/3197832149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283414cd70 con 0x7f28340ae820 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283932f640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28340ae820 0x7f28340aa730 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2824011590 tx=0x7f2824009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f282a7fc640 1 -- 172.21.15.39:0/3197832149 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824004500 con 0x7f28340ae820 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f282a7fc640 1 -- 172.21.15.39:0/3197832149 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2824003620 con 0x7f28340ae820 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f282a7fc640 1 -- 172.21.15.39:0/3197832149 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28240039d0 con 0x7f28340ae820 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283415c690 con 0x7f28340ae820 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28341f0890 con 0x7f28340ae820 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f283415fec0 con 0x7f28340ae820 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.353+0000 7f282a7fc640 1 -- 172.21.15.39:0/3197832149 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2824003b30 con 0x7f28340ae820 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f282a7fc640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2818066690 0x7f2818068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f282a7fc640 1 -- 172.21.15.39:0/3197832149 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f2824085490 con 0x7f28340ae820 2024-09-13T17:17:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f2839b30640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2818066690 0x7f2818068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f282a7fc640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f2818073070 0x7f28180754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f282a7fc640 1 -- 172.21.15.39:0/3197832149 --> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2818075bc0 con 0x7f2818073070 2024-09-13T17:17:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f282a7fc640 1 -- 172.21.15.39:0/3197832149 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2824087af0 con 0x7f28340ae820 2024-09-13T17:17:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f283a331640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f2818073070 0x7f28180754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f2839b30640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2818066690 0x7f2818068b50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f28300ab000 tx=0x7f28300a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f283a331640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f2818073070 0x7f28180754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.357+0000 7f282a7fc640 1 -- 172.21.15.39:0/3197832149 <== osd.1 v2:172.21.15.39:6808/2114478470 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2818075bc0 con 0x7f2818073070 2024-09-13T17:17:46.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 --> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f28340aa450 con 0x7f2818073070 2024-09-13T17:17:46.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f282a7fc640 1 -- 172.21.15.39:0/3197832149 <== osd.1 v2:172.21.15.39:6808/2114478470 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f28340aa450 con 0x7f2818073070 2024-09-13T17:17:46.373 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T17:17:46.373 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:17:46.373 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T17:17:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f2818073070 msgr2=0x7f28180754f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6808/2114478470,v1:172.21.15.39:6809/2114478470] conn(0x7f2818073070 0x7f28180754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2818066690 msgr2=0x7f2818068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f2818066690 0x7f2818068b50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f28300ab000 tx=0x7f28300a7070 comp rx=0 tx=0).stop 2024-09-13T17:17:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28340ae820 msgr2=0x7f28340aa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 --2- 172.21.15.39:0/3197832149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28340ae820 0x7f28340aa730 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2824011590 tx=0x7f2824009f90 comp rx=0 tx=0).stop 2024-09-13T17:17:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283a331640 1 -- 172.21.15.39:0/3197832149 reap_dead start 2024-09-13T17:17:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 shutdown_connections 2024-09-13T17:17:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 >> 172.21.15.39:0/3197832149 conn(0x7f2834069fe0 msgr2=0x7f2834150710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 shutdown_connections 2024-09-13T17:17:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.369+0000 7f283bd94640 1 -- 172.21.15.39:0/3197832149 wait complete. 2024-09-13T17:17:46.388 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T17:17:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d14a35640 1 Processor -- start 2024-09-13T17:17:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d14a35640 1 -- start start 2024-09-13T17:17:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d14a35640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7d100a98f0 0x7f7d10071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d14a35640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1014f8d0 con 0x7f7d100ab160 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d14a35640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d100aad40 con 0x7f7d100716e0 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d14a35640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d10151c40 con 0x7f7d100a98f0 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0ed9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7d100ab160 0x7f7d1015c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40264/0 (socket says 172.21.15.39:40264) 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0ed9d640 1 -- 172.21.15.39:0/4050616650 learned_addr learned my addr 172.21.15.39:0/4050616650 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0dd9b640 1 --2- 172.21.15.39:0/4050616650 >> v2:172.21.15.84:3300/0 conn(0x7f7d100a98f0 0x7f7d10071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788328473 0 0) 0x7f7d1014f8d0 con 0x7f7d100ab160 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf8003530 con 0x7f7d100ab160 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 606612060 0 0) 0x7f7cf8003530 con 0x7f7d100ab160 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 >> v2:172.21.15.84:3300/0 conn(0x7f7d100a98f0 msgr2=0x7f7d10071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 --2- 172.21.15.39:0/4050616650 >> v2:172.21.15.84:3300/0 conn(0x7f7d100a98f0 0x7f7d10071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 >> v1:172.21.15.84:6789/0 conn(0x7f7d100716e0 legacy=0x7f7d10152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1006dfd0 con 0x7f7d100ab160 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f7d08001a00 con 0x7f7d100ab160 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 >> v1:172.21.15.39:6789/0 conn(0x7f7d100ab160 legacy=0x7f7d1015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf8003980 0x7f7cf8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf80047d0 0x7f7cf8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7cf8005190 0x7f7cf800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf800c000 con 0x7f7cf8003980 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf800c170 con 0x7f7cf80047d0 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf800c330 con 0x7f7cf8005190 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0e59c640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf8003980 0x7f7cf8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0dd9b640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf80047d0 0x7f7cf8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0ed9d640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7cf8005190 0x7f7cf800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0e59c640 1 -- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7cf8005190 msgr2=0x7f7cf800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0e59c640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7cf8005190 0x7f7cf800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0e59c640 1 -- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf80047d0 msgr2=0x7f7cf8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0e59c640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf80047d0 0x7f7cf8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0e59c640 1 -- 172.21.15.39:0/4050616650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf8003530 con 0x7f7cf8003980 2024-09-13T17:17:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0ed9d640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7cf8005190 0x7f7cf800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0dd9b640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf80047d0 0x7f7cf8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:46.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.493+0000 7f7d0ed9d640 1 -- 172.21.15.39:0/4050616650 reap_dead start 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d0e59c640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf8003980 0x7f7cf8003da0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7d00001780 tx=0x7f7d0001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc4f88d3a23134d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000052d0 con 0x7f7cf8003980 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d000a7050 con 0x7f7cf8003980 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d0d59a640 1 -- 172.21.15.39:0/4050616650 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00003600 con 0x7f7cf8003980 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d14a35640 1 -- 172.21.15.39:0/4050616650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf8003980 msgr2=0x7f7cf8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d14a35640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf8003980 0x7f7cf8003da0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7d00001780 tx=0x7f7d0001c400 comp rx=0 tx=0).stop 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d14a35640 1 -- 172.21.15.39:0/4050616650 shutdown_connections 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d14a35640 1 --2- 172.21.15.39:0/4050616650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf8003980 0x7f7cf8003da0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d14a35640 1 -- 172.21.15.39:0/4050616650 >> 172.21.15.39:0/4050616650 conn(0x7f7d10069e10 msgr2=0x7f7d1006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d14a35640 1 -- 172.21.15.39:0/4050616650 shutdown_connections 2024-09-13T17:17:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d14a35640 1 -- 172.21.15.39:0/4050616650 wait complete. 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d14a35640 1 Processor -- start 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.497+0000 7f7d14a35640 1 -- start start 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d14a35640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf8003980 0x7f7d100c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d14a35640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d10149c60 0x7f7d101ee120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d14a35640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d101ee660 0x7f7d101eeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d14a35640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d101eab50 con 0x7f7d101ee660 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d14a35640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d101e0dc0 con 0x7f7cf8003980 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d14a35640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1014c200 con 0x7f7d10149c60 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d101ee660 0x7f7d101eeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d101ee660 0x7f7d101eeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40740/0 (socket says 172.21.15.39:40740) 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0ed9d640 1 -- 172.21.15.39:0/3912996491 learned_addr learned my addr 172.21.15.39:0/3912996491 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0e59c640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf8003980 0x7f7d100c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0dd9b640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d10149c60 0x7f7d101ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0ed9d640 1 -- 172.21.15.39:0/3912996491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d10149c60 msgr2=0x7f7d101ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0ed9d640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d10149c60 0x7f7d101ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0ed9d640 1 -- 172.21.15.39:0/3912996491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf8003980 msgr2=0x7f7d100c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0ed9d640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf8003980 0x7f7d100c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0ed9d640 1 -- 172.21.15.39:0/3912996491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1014c4e0 con 0x7f7d101ee660 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0ed9d640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d101ee660 0x7f7d101eeb10 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7d08001590 tx=0x7f7d0800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7cfeffd640 1 -- 172.21.15.39:0/3912996491 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08004d70 con 0x7f7d101ee660 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7cfeffd640 1 -- 172.21.15.39:0/3912996491 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0800baa0 con 0x7f7d101ee660 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1014c7c0 con 0x7f7d101ee660 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7cfeffd640 1 -- 172.21.15.39:0/3912996491 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0800be50 con 0x7f7d101ee660 2024-09-13T17:17:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d100c55b0 con 0x7f7d101ee660 2024-09-13T17:17:46.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d10155540 con 0x7f7d101ee660 2024-09-13T17:17:46.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7cfeffd640 1 -- 172.21.15.39:0/3912996491 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7d08007110 con 0x7f7d101ee660 2024-09-13T17:17:46.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7cfeffd640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7cf0066690 0x7f7cf0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7cfeffd640 1 -- 172.21.15.39:0/3912996491 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f7d08085480 con 0x7f7d101ee660 2024-09-13T17:17:46.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0e59c640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7cf0066690 0x7f7cf0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.501+0000 7f7d0e59c640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7cf0066690 0x7f7cf0068b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7d00004000 tx=0x7f7d000011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.505+0000 7f7cfeffd640 1 -- 172.21.15.39:0/3912996491 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7d08051720 con 0x7f7d101ee660 2024-09-13T17:17:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.669+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d1014c370 con 0x7f7cf0066690 2024-09-13T17:17:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.669+0000 7f7cfeffd640 1 -- 172.21.15.39:0/3912996491 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7d1014c370 con 0x7f7cf0066690 2024-09-13T17:17:46.673 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7cf0066690 msgr2=0x7f7cf0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7cf0066690 0x7f7cf0068b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7d00004000 tx=0x7f7d000011a0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d101ee660 msgr2=0x7f7d101eeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d101ee660 0x7f7d101eeb10 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7d08001590 tx=0x7f7d0800b4a0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 shutdown_connections 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7cf0066690 0x7f7cf0068b50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d101ee660 0x7f7d101eeb10 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7d10149c60 0x7f7d101ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 --2- 172.21.15.39:0/3912996491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cf8003980 0x7f7d100c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 >> 172.21.15.39:0/3912996491 conn(0x7f7d10069e10 msgr2=0x7f7d1006fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 shutdown_connections 2024-09-13T17:17:46.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.673+0000 7f7d14a35640 1 -- 172.21.15.39:0/3912996491 wait complete. 2024-09-13T17:17:46.693 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T17:17:46.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:17:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636fa48640 1 Processor -- start 2024-09-13T17:17:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636fa48640 1 -- start start 2024-09-13T17:17:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636fa48640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63680a4d60 0x7f636814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636fa48640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6368155d20 con 0x7f636814f4b0 2024-09-13T17:17:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636fa48640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63680c4750 con 0x7f63680aa2d0 2024-09-13T17:17:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636fa48640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6368155b30 con 0x7f63680a4d60 2024-09-13T17:17:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636dfe5640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f636814f4b0 0x7f6368151cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40266/0 (socket says 172.21.15.39:40266) 2024-09-13T17:17:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636dfe5640 1 -- 172.21.15.39:0/2264854041 learned_addr learned my addr 172.21.15.39:0/2264854041 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636cfe3640 1 --2- 172.21.15.39:0/2264854041 >> v2:172.21.15.84:3300/0 conn(0x7f63680a4d60 0x7f636814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123733314 0 0) 0x7f6368155d20 con 0x7f636814f4b0 2024-09-13T17:17:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6354003530 con 0x7f636814f4b0 2024-09-13T17:17:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2091997903 0 0) 0x7f6354003530 con 0x7f636814f4b0 2024-09-13T17:17:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 >> v2:172.21.15.84:3300/0 conn(0x7f63680a4d60 msgr2=0x7f636814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 --2- 172.21.15.39:0/2264854041 >> v2:172.21.15.84:3300/0 conn(0x7f63680a4d60 0x7f636814ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 >> v1:172.21.15.84:6789/0 conn(0x7f63680aa2d0 legacy=0x7f63680aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6368155fc0 con 0x7f636814f4b0 2024-09-13T17:17:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f6360001a00 con 0x7f636814f4b0 2024-09-13T17:17:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 >> v1:172.21.15.39:6789/0 conn(0x7f636814f4b0 legacy=0x7f6368151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6354003980 0x7f6354003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63540047d0 0x7f6354004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6354005190 0x7f635400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635400c100 con 0x7f6354005190 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635400c270 con 0x7f63540047d0 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635400c430 con 0x7f6354003980 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636dfe5640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6354005190 0x7f635400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636d7e4640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6354003980 0x7f6354003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636cfe3640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63540047d0 0x7f6354004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636dfe5640 1 -- 172.21.15.39:0/2264854041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6354003980 msgr2=0x7f6354003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636dfe5640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6354003980 0x7f6354003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636dfe5640 1 -- 172.21.15.39:0/2264854041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63540047d0 msgr2=0x7f6354004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636dfe5640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63540047d0 0x7f6354004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.817+0000 7f636dfe5640 1 -- 172.21.15.39:0/2264854041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354003530 con 0x7f6354005190 2024-09-13T17:17:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636dfe5640 1 -- 172.21.15.39:0/2264854041 reap_dead start 2024-09-13T17:17:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636dfe5640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6354005190 0x7f635400b990 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f636000a830 tx=0x7f636001fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f08eeb64abd2836 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63600aa6f0 con 0x7f6354005190 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63600aa890 con 0x7f6354005190 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f635ffff640 1 -- 172.21.15.39:0/2264854041 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63600aac40 con 0x7f6354005190 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636fa48640 1 -- 172.21.15.39:0/2264854041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6354005190 msgr2=0x7f635400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636fa48640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6354005190 0x7f635400b990 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f636000a830 tx=0x7f636001fac0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636fa48640 1 -- 172.21.15.39:0/2264854041 shutdown_connections 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636fa48640 1 --2- 172.21.15.39:0/2264854041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6354005190 0x7f635400b990 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636fa48640 1 -- 172.21.15.39:0/2264854041 >> 172.21.15.39:0/2264854041 conn(0x7f63680c5790 msgr2=0x7f63680c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636fa48640 1 -- 172.21.15.39:0/2264854041 shutdown_connections 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636fa48640 1 -- 172.21.15.39:0/2264854041 wait complete. 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636fa48640 1 Processor -- start 2024-09-13T17:17:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.821+0000 7f636fa48640 1 -- start start 2024-09-13T17:17:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636fa48640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6354005190 0x7f63680a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636fa48640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63680a7cf0 0x7f63680bee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636fa48640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636806c240 0x7f636806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636fa48640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63681e9a00 con 0x7f636806c240 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636fa48640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680a7700 con 0x7f63680a7cf0 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636fa48640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680c3cb0 con 0x7f6354005190 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636dfe5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636806c240 0x7f636806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636dfe5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636806c240 0x7f636806c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40766/0 (socket says 172.21.15.39:40766) 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636dfe5640 1 -- 172.21.15.39:0/1805810551 learned_addr learned my addr 172.21.15.39:0/1805810551 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636cfe3640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63680a7cf0 0x7f63680bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636d7e4640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6354005190 0x7f63680a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636dfe5640 1 -- 172.21.15.39:0/1805810551 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6354005190 msgr2=0x7f63680a71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636dfe5640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f6354005190 0x7f63680a71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636dfe5640 1 -- 172.21.15.39:0/1805810551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63680a7cf0 msgr2=0x7f63680bee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636dfe5640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63680a7cf0 0x7f63680bee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636dfe5640 1 -- 172.21.15.39:0/1805810551 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680c2e60 con 0x7f636806c240 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636dfe5640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636806c240 0x7f636806c6c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6360007040 tx=0x7f6354005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f635dffb640 1 -- 172.21.15.39:0/1805810551 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63600aa6f0 con 0x7f636806c240 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f635dffb640 1 -- 172.21.15.39:0/1805810551 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63600017c0 con 0x7f636806c240 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63680c3f50 con 0x7f636806c240 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f635dffb640 1 -- 172.21.15.39:0/1805810551 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360004b70 con 0x7f636806c240 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63680bf360 con 0x7f636806c240 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f63680bfe20 con 0x7f636806c240 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f635dffb640 1 -- 172.21.15.39:0/1805810551 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6360004cf0 con 0x7f636806c240 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f635dffb640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f634c0666c0 0x7f634c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f635dffb640 1 -- 172.21.15.39:0/1805810551 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f63601228f0 con 0x7f636806c240 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636d7e4640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f634c0666c0 0x7f634c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f635dffb640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7f634c0730a0 0x7f634c075520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f635dffb640 1 -- 172.21.15.39:0/1805810551 --> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f634c075bf0 con 0x7f634c0730a0 2024-09-13T17:17:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f635dffb640 1 -- 172.21.15.39:0/1805810551 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63601272a0 con 0x7f636806c240 2024-09-13T17:17:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636cfe3640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7f634c0730a0 0x7f634c075520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636d7e4640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f634c0666c0 0x7f634c068b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f63580053f0 tx=0x7f6358009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.825+0000 7f636cfe3640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7f634c0730a0 0x7f634c075520 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:46.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.829+0000 7f635dffb640 1 -- 172.21.15.39:0/1805810551 <== osd.2 v2:172.21.15.39:6816/3103155192 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f634c075bf0 con 0x7f634c0730a0 2024-09-13T17:17:46.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 --> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f63680c4cf0 con 0x7f634c0730a0 2024-09-13T17:17:46.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f635dffb640 1 -- 172.21.15.39:0/1805810551 <== osd.2 v2:172.21.15.39:6816/3103155192 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6364009af0 con 0x7f634c0730a0 2024-09-13T17:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T17:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:17:46.844 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T17:17:46.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7f634c0730a0 msgr2=0x7f634c075520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6816/3103155192,v1:172.21.15.39:6817/3103155192] conn(0x7f634c0730a0 0x7f634c075520 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f634c0666c0 msgr2=0x7f634c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f634c0666c0 0x7f634c068b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f63580053f0 tx=0x7f6358009040 comp rx=0 tx=0).stop 2024-09-13T17:17:46.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636806c240 msgr2=0x7f636806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 --2- 172.21.15.39:0/1805810551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f636806c240 0x7f636806c6c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6360007040 tx=0x7f6354005800 comp rx=0 tx=0).stop 2024-09-13T17:17:46.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636dfe5640 1 -- 172.21.15.39:0/1805810551 reap_dead start 2024-09-13T17:17:46.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 shutdown_connections 2024-09-13T17:17:46.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 >> 172.21.15.39:0/1805810551 conn(0x7f63680c5790 msgr2=0x7f63680c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:46.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 shutdown_connections 2024-09-13T17:17:46.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.841+0000 7f636fa48640 1 -- 172.21.15.39:0/1805810551 wait complete. 2024-09-13T17:17:46.859 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fde402640 1 Processor -- start 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fde402640 1 -- start start 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fde402640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd808a9c0 0x7f3fd80c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fde402640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd806d490 con 0x7f3fd80c4d50 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fde402640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd80c2cd0 con 0x7f3fd806b130 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fde402640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80c2220 con 0x7f3fd808a9c0 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fd7fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f3fd80c4d50 0x7f3fd81612a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40272/0 (socket says 172.21.15.39:40272) 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fd7fff640 1 -- 172.21.15.39:0/1948981700 learned_addr learned my addr 172.21.15.39:0/1948981700 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fdc99f640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd808a9c0 0x7f3fd80c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.969+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464457865 0 0) 0x7f3fd806d490 con 0x7f3fd80c4d50 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc4003450 con 0x7f3fd80c4d50 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 <== mon.1 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033377214 0 0) 0x7f3fd80c2cd0 con 0x7f3fd806b130 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd806d490 con 0x7f3fd806b130 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3009798831 0 0) 0x7f3fc4003450 con 0x7f3fd80c4d50 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd808a9c0 msgr2=0x7f3fd80c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd808a9c0 0x7f3fd80c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 >> v1:172.21.15.84:6789/0 conn(0x7f3fd806b130 legacy=0x7f3fd808a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd8160e10 con 0x7f3fd80c4d50 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f3fc8001680 con 0x7f3fd80c4d50 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 >> v1:172.21.15.39:6789/0 conn(0x7f3fd80c4d50 legacy=0x7f3fd81612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc4003d50 0x7f3fc4004150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fc4004810 0x7f3fc4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:46.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd808a9c0 0x7f3fc400c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80c2220 con 0x7f3fc4004810 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc400cc40 con 0x7f3fc4003d50 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc400ce00 con 0x7f3fd808a9c0 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fcffff640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fc4004810 0x7f3fc4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fd7fff640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc4003d50 0x7f3fc4004150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fdc99f640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd808a9c0 0x7f3fc400c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fcffff640 1 -- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd808a9c0 msgr2=0x7f3fc400c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fcffff640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd808a9c0 0x7f3fc400c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fcffff640 1 -- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc4003d50 msgr2=0x7f3fc4004150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fcffff640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc4003d50 0x7f3fc4004150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fcffff640 1 -- 172.21.15.39:0/1948981700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc4003450 con 0x7f3fc4004810 2024-09-13T17:17:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.993+0000 7f3fdc99f640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd808a9c0 0x7f3fc400c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fcffff640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fc4004810 0x7f3fc4004c90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00018d0 tx=0x7f3fc001c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b70a925952c459 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00052d0 con 0x7f3fc4004810 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc00a7050 con 0x7f3fc4004810 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fd77fe640 1 -- 172.21.15.39:0/1948981700 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0003600 con 0x7f3fc4004810 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 -- 172.21.15.39:0/1948981700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fc4004810 msgr2=0x7f3fc4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 --2- 172.21.15.39:0/1948981700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fc4004810 0x7f3fc4004c90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00018d0 tx=0x7f3fc001c580 comp rx=0 tx=0).stop 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fdc99f640 1 -- 172.21.15.39:0/1948981700 reap_dead start 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 -- 172.21.15.39:0/1948981700 shutdown_connections 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 -- 172.21.15.39:0/1948981700 >> 172.21.15.39:0/1948981700 conn(0x7f3fd80c5790 msgr2=0x7f3fd80c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 -- 172.21.15.39:0/1948981700 shutdown_connections 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 -- 172.21.15.39:0/1948981700 wait complete. 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 Processor -- start 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 -- start start 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc4004810 0x7f3fd81644c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd8088ca0 0x7f3fd8089120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:46.997+0000 7f3fde402640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81682a0 0x7f3fd8168730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fde402640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80a90c0 con 0x7f3fd81682a0 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fde402640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80a9ed0 con 0x7f3fc4004810 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fde402640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd81fc130 con 0x7f3fd8088ca0 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fdc99f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81682a0 0x7f3fd8168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fdc99f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81682a0 0x7f3fd8168730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40792/0 (socket says 172.21.15.39:40792) 2024-09-13T17:17:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fdc99f640 1 -- 172.21.15.39:0/3754232620 learned_addr learned my addr 172.21.15.39:0/3754232620 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fd7fff640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc4004810 0x7f3fd81644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fcffff640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd8088ca0 0x7f3fd8089120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fdc99f640 1 -- 172.21.15.39:0/3754232620 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd8088ca0 msgr2=0x7f3fd8089120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fdc99f640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd8088ca0 0x7f3fd8089120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fdc99f640 1 -- 172.21.15.39:0/3754232620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc4004810 msgr2=0x7f3fd81644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fdc99f640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc4004810 0x7f3fd81644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fdc99f640 1 -- 172.21.15.39:0/3754232620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80aa150 con 0x7f3fd81682a0 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fdc99f640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81682a0 0x7f3fd8168730 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3fd806b0c0 tx=0x7f3fd00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fd57fa640 1 -- 172.21.15.39:0/3754232620 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000e030 con 0x7f3fd81682a0 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fd57fa640 1 -- 172.21.15.39:0/3754232620 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd0016470 con 0x7f3fd81682a0 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fd57fa640 1 -- 172.21.15.39:0/3754232620 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0016820 con 0x7f3fd81682a0 2024-09-13T17:17:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd81fbbb0 con 0x7f3fd81682a0 2024-09-13T17:17:47.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd81fbe50 con 0x7f3fd81682a0 2024-09-13T17:17:47.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd80674c0 con 0x7f3fd81682a0 2024-09-13T17:17:47.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fd57fa640 1 -- 172.21.15.39:0/3754232620 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3fd0016980 con 0x7f3fd81682a0 2024-09-13T17:17:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fd57fa640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fb80666c0 0x7f3fb8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fd57fa640 1 -- 172.21.15.39:0/3754232620 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f3fd000f070 con 0x7f3fd81682a0 2024-09-13T17:17:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fd7fff640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fb80666c0 0x7f3fb8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.001+0000 7f3fd7fff640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fb80666c0 0x7f3fb8068b80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8072130 tx=0x7f3fc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.005+0000 7f3fd57fa640 1 -- 172.21.15.39:0/3754232620 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3fd008a070 con 0x7f3fd81682a0 2024-09-13T17:17:47.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.157+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fd80c29f0 con 0x7f3fb80666c0 2024-09-13T17:17:47.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fd57fa640 1 -- 172.21.15.39:0/3754232620 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3fc8007030 con 0x7f3fb80666c0 2024-09-13T17:17:47.163 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T17:17:47.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fb80666c0 msgr2=0x7f3fb8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fb80666c0 0x7f3fb8068b80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8072130 tx=0x7f3fc8009040 comp rx=0 tx=0).stop 2024-09-13T17:17:47.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81682a0 msgr2=0x7f3fd8168730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81682a0 0x7f3fd8168730 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3fd806b0c0 tx=0x7f3fd00115a0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 shutdown_connections 2024-09-13T17:17:47.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3fb80666c0 0x7f3fb8068b80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd81682a0 0x7f3fd8168730 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3fd8088ca0 0x7f3fd8089120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 --2- 172.21.15.39:0/3754232620 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3fc4004810 0x7f3fd81644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 >> 172.21.15.39:0/3754232620 conn(0x7f3fd80c5790 msgr2=0x7f3fd80c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:47.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 shutdown_connections 2024-09-13T17:17:47.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.161+0000 7f3fde402640 1 -- 172.21.15.39:0/3754232620 wait complete. 2024-09-13T17:17:47.182 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T17:17:47.182 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:17:47.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd0d9e2640 1 Processor -- start 2024-09-13T17:17:47.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd0d9e2640 1 -- start start 2024-09-13T17:17:47.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd0d9e2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd080a4d60 0x7fcd0814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd0d9e2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcd0814f4b0 0x7fcd08151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd0d9e2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd08155d20 con 0x7fcd080aa2d0 2024-09-13T17:17:47.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd0d9e2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd08155b30 con 0x7fcd0814f4b0 2024-09-13T17:17:47.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd0d9e2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080aadc0 con 0x7fcd080a4d60 2024-09-13T17:17:47.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd077fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcd0814f4b0 0x7fcd08151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd06ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcd080aa2d0 0x7fcd080aa6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40286/0 (socket says 172.21.15.39:40286) 2024-09-13T17:17:47.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd06ffd640 1 -- 172.21.15.39:0/396842386 learned_addr learned my addr 172.21.15.39:0/396842386 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:47.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd067fc640 1 --2- 172.21.15.39:0/396842386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd080a4d60 0x7fcd0814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd077fe640 1 -- 172.21.15.39:0/396842386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd080a4d60 msgr2=0x7fcd0814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:47.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd077fe640 1 --2- 172.21.15.39:0/396842386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd080a4d60 0x7fcd0814ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd077fe640 1 -- 172.21.15.39:0/396842386 >> v1:172.21.15.39:6789/0 conn(0x7fcd080aa2d0 legacy=0x7fcd080aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd077fe640 1 -- 172.21.15.39:0/396842386 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd080c4750 con 0x7fcd0814f4b0 2024-09-13T17:17:47.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd05ffb640 1 -- 172.21.15.39:0/396842386 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231329832 0 0) 0x7fcd08155d20 con 0x7fcd080aa2d0 2024-09-13T17:17:47.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.305+0000 7fcd067fc640 1 --2- 172.21.15.39:0/396842386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd080a4d60 0x7fcd0814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd077fe640 1 --2- 172.21.15.39:0/396842386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd0814f4b0 0x7fcd08151cf0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009dc0 tx=0x7fcd0001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788b29ff0eb9649a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd05ffb640 1 -- 172.21.15.39:0/396842386 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08152a40 con 0x7fcd0814f4b0 2024-09-13T17:17:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd05ffb640 1 -- 172.21.15.39:0/396842386 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd000a5860 con 0x7fcd0814f4b0 2024-09-13T17:17:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd05ffb640 1 -- 172.21.15.39:0/396842386 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000a5bf0 con 0x7fcd0814f4b0 2024-09-13T17:17:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/396842386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd0814f4b0 msgr2=0x7fcd08151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 --2- 172.21.15.39:0/396842386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd0814f4b0 0x7fcd08151cf0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009dc0 tx=0x7fcd0001c890 comp rx=0 tx=0).stop 2024-09-13T17:17:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/396842386 shutdown_connections 2024-09-13T17:17:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 --2- 172.21.15.39:0/396842386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd0814f4b0 0x7fcd08151cf0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 --2- 172.21.15.39:0/396842386 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd080a4d60 0x7fcd0814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/396842386 >> 172.21.15.39:0/396842386 conn(0x7fcd080c5790 msgr2=0x7fcd080c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/396842386 shutdown_connections 2024-09-13T17:17:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/396842386 wait complete. 2024-09-13T17:17:47.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 Processor -- start 2024-09-13T17:17:47.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.309+0000 7fcd0d9e2640 1 -- start start 2024-09-13T17:17:47.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd0d9e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd080a4d60 0x7fcd0806c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd0d9e2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd080aa2d0 0x7fcd08150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd0d9e2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd0814f4b0 0x7fcd08153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd0d9e2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0814cc10 con 0x7fcd080aa2d0 2024-09-13T17:17:47.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd0d9e2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0806cec0 con 0x7fcd080a4d60 2024-09-13T17:17:47.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd0d9e2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd081ef180 con 0x7fcd0814f4b0 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd067fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd080aa2d0 0x7fcd08150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd067fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd080aa2d0 0x7fcd08150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40806/0 (socket says 172.21.15.39:40806) 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd067fc640 1 -- 172.21.15.39:0/310325265 learned_addr learned my addr 172.21.15.39:0/310325265 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd06ffd640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd080a4d60 0x7fcd0806c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd077fe640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd0814f4b0 0x7fcd08153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd067fc640 1 -- 172.21.15.39:0/310325265 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd0814f4b0 msgr2=0x7fcd08153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd067fc640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcd0814f4b0 0x7fcd08153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd067fc640 1 -- 172.21.15.39:0/310325265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd080a4d60 msgr2=0x7fcd0806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd067fc640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd080a4d60 0x7fcd0806c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd067fc640 1 -- 172.21.15.39:0/310325265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00009a70 con 0x7fcd080aa2d0 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd067fc640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd080aa2d0 0x7fcd08150ff0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c7c0 tx=0x7fccfc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcceffff640 1 -- 172.21.15.39:0/310325265 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc007a30 con 0x7fcd080aa2d0 2024-09-13T17:17:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcceffff640 1 -- 172.21.15.39:0/310325265 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc007b90 con 0x7fcd080aa2d0 2024-09-13T17:17:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd081ef460 con 0x7fcd080aa2d0 2024-09-13T17:17:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcceffff640 1 -- 172.21.15.39:0/310325265 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00ce70 con 0x7fcd080aa2d0 2024-09-13T17:17:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0806d1a0 con 0x7fcd080aa2d0 2024-09-13T17:17:47.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd080bfe20 con 0x7fcd080aa2d0 2024-09-13T17:17:47.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcceffff640 1 -- 172.21.15.39:0/310325265 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccfc0113f0 con 0x7fcd080aa2d0 2024-09-13T17:17:47.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcceffff640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fccf4066690 0x7fccf4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcceffff640 1 -- 172.21.15.39:0/310325265 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7fccfc0859a0 con 0x7fcd080aa2d0 2024-09-13T17:17:47.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd06ffd640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fccf4066690 0x7fccf4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcceffff640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7fccf4073070 0x7fccf40754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcceffff640 1 -- 172.21.15.39:0/310325265 --> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccf4075bc0 con 0x7fccf4073070 2024-09-13T17:17:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcceffff640 1 -- 172.21.15.39:0/310325265 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccfc085db0 con 0x7fcd080aa2d0 2024-09-13T17:17:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd077fe640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7fccf4073070 0x7fccf40754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.313+0000 7fcd06ffd640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fccf4066690 0x7fccf4068b50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fccf8000f70 tx=0x7fccf8007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.317+0000 7fcd077fe640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7fccf4073070 0x7fccf40754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.317+0000 7fcceffff640 1 -- 172.21.15.39:0/310325265 <== osd.3 v2:172.21.15.39:6824/1984006749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fccf4075bc0 con 0x7fccf4073070 2024-09-13T17:17:47.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 --> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcd081ef980 con 0x7fccf4073070 2024-09-13T17:17:47.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcceffff640 1 -- 172.21.15.39:0/310325265 <== osd.3 v2:172.21.15.39:6824/1984006749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcd081ef980 con 0x7fccf4073070 2024-09-13T17:17:47.332 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T17:17:47.332 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:17:47.332 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T17:17:47.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7fccf4073070 msgr2=0x7fccf40754f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6824/1984006749,v1:172.21.15.39:6825/1984006749] conn(0x7fccf4073070 0x7fccf40754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fccf4066690 msgr2=0x7fccf4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fccf4066690 0x7fccf4068b50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fccf8000f70 tx=0x7fccf8007b60 comp rx=0 tx=0).stop 2024-09-13T17:17:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd080aa2d0 msgr2=0x7fcd08150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 --2- 172.21.15.39:0/310325265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd080aa2d0 0x7fcd08150ff0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c7c0 tx=0x7fccfc00cc80 comp rx=0 tx=0).stop 2024-09-13T17:17:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd077fe640 1 -- 172.21.15.39:0/310325265 reap_dead start 2024-09-13T17:17:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 shutdown_connections 2024-09-13T17:17:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 >> 172.21.15.39:0/310325265 conn(0x7fcd080c5790 msgr2=0x7fcd080aa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 shutdown_connections 2024-09-13T17:17:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.329+0000 7fcd0d9e2640 1 -- 172.21.15.39:0/310325265 wait complete. 2024-09-13T17:17:47.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T17:17:47.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce91868640 1 Processor -- start 2024-09-13T17:17:47.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce91868640 1 -- start start 2024-09-13T17:17:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce91868640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce8c1510a0 0x7fce8c0c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce91868640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 0x7fce8c0c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce91868640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c0c0f50 con 0x7fce8c0c08f0 2024-09-13T17:17:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce91868640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c06b6b0 con 0x7fce8c1510a0 2024-09-13T17:17:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce91868640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c06ca50 con 0x7fce8c06bad0 2024-09-13T17:17:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce8affd640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce8c1510a0 0x7fce8c0c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce8b7fe640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 0x7fce8c0c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce8a7fc640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fce8c0c08f0 0x7fce8c0c40a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:60908/0 (socket says 172.21.15.39:60908) 2024-09-13T17:17:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce8a7fc640 1 -- 172.21.15.39:0/1168882636 learned_addr learned my addr 172.21.15.39:0/1168882636 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce8affd640 1 -- 172.21.15.39:0/1168882636 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 msgr2=0x7fce8c0c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce8affd640 1 --2- 172.21.15.39:0/1168882636 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 0x7fce8c0c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce8affd640 1 -- 172.21.15.39:0/1168882636 >> v1:172.21.15.84:6789/0 conn(0x7fce8c0c08f0 legacy=0x7fce8c0c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.453+0000 7fce8affd640 1 -- 172.21.15.39:0/1168882636 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c0c18e0 con 0x7fce8c1510a0 2024-09-13T17:17:47.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce8affd640 1 --2- 172.21.15.39:0/1168882636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce8c1510a0 0x7fce8c0c03b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fce800097c0 tx=0x7fce8001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d5912e7b0a8e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce89ffb640 1 -- 172.21.15.39:0/1168882636 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce800ab600 con 0x7fce8c1510a0 2024-09-13T17:17:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce89ffb640 1 -- 172.21.15.39:0/1168882636 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce800a7050 con 0x7fce8c1510a0 2024-09-13T17:17:47.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce89ffb640 1 -- 172.21.15.39:0/1168882636 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce800ab980 con 0x7fce8c1510a0 2024-09-13T17:17:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 -- 172.21.15.39:0/1168882636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce8c1510a0 msgr2=0x7fce8c0c03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 --2- 172.21.15.39:0/1168882636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce8c1510a0 0x7fce8c0c03b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fce800097c0 tx=0x7fce8001c380 comp rx=0 tx=0).stop 2024-09-13T17:17:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 -- 172.21.15.39:0/1168882636 shutdown_connections 2024-09-13T17:17:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 --2- 172.21.15.39:0/1168882636 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 0x7fce8c0c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 --2- 172.21.15.39:0/1168882636 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce8c1510a0 0x7fce8c0c03b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 -- 172.21.15.39:0/1168882636 >> 172.21.15.39:0/1168882636 conn(0x7fce8c0c5790 msgr2=0x7fce8c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 -- 172.21.15.39:0/1168882636 shutdown_connections 2024-09-13T17:17:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 -- 172.21.15.39:0/1168882636 wait complete. 2024-09-13T17:17:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 Processor -- start 2024-09-13T17:17:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.457+0000 7fce91868640 1 -- start start 2024-09-13T17:17:47.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce91868640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 0x7fce8c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce91868640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce8c0c08f0 0x7fce8c152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce91868640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce8c1510a0 0x7fce8c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce91868640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c15e670 con 0x7fce8c0c08f0 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce91868640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c155970 con 0x7fce8c1510a0 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce91868640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c151ea0 con 0x7fce8c06bad0 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce8c0c08f0 0x7fce8c152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce8c0c08f0 0x7fce8c152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40810/0 (socket says 172.21.15.39:40810) 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8a7fc640 1 -- 172.21.15.39:0/583576755 learned_addr learned my addr 172.21.15.39:0/583576755 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8affd640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 0x7fce8c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8b7fe640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce8c1510a0 0x7fce8c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8a7fc640 1 -- 172.21.15.39:0/583576755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 msgr2=0x7fce8c15e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8a7fc640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 0x7fce8c15e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8a7fc640 1 -- 172.21.15.39:0/583576755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce8c1510a0 msgr2=0x7fce8c14e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8a7fc640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce8c1510a0 0x7fce8c14e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8a7fc640 1 -- 172.21.15.39:0/583576755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80009420 con 0x7fce8c0c08f0 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8a7fc640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce8c0c08f0 0x7fce8c152cd0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c530 tx=0x7fce7c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce73fff640 1 -- 172.21.15.39:0/583576755 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c005cf0 con 0x7fce8c0c08f0 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce73fff640 1 -- 172.21.15.39:0/583576755 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c005e50 con 0x7fce8c0c08f0 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c152180 con 0x7fce8c0c08f0 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce73fff640 1 -- 172.21.15.39:0/583576755 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c007270 con 0x7fce8c0c08f0 2024-09-13T17:17:47.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c15dc10 con 0x7fce8c0c08f0 2024-09-13T17:17:47.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c0674c0 con 0x7fce8c0c08f0 2024-09-13T17:17:47.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce73fff640 1 -- 172.21.15.39:0/583576755 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce7c0073d0 con 0x7fce8c0c08f0 2024-09-13T17:17:47.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce73fff640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce78066690 0x7fce78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce73fff640 1 -- 172.21.15.39:0/583576755 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7fce7c0846a0 con 0x7fce8c0c08f0 2024-09-13T17:17:47.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.461+0000 7fce8affd640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce78066690 0x7fce78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.465+0000 7fce8affd640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce78066690 0x7fce78068b50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fce800ad000 tx=0x7fce800a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.465+0000 7fce73fff640 1 -- 172.21.15.39:0/583576755 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce7c0509f0 con 0x7fce8c0c08f0 2024-09-13T17:17:47.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.625+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fce8c15d7b0 con 0x7fce78066690 2024-09-13T17:17:47.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.625+0000 7fce73fff640 1 -- 172.21.15.39:0/583576755 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fce800bb090 con 0x7fce78066690 2024-09-13T17:17:47.630 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce78066690 msgr2=0x7fce78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce78066690 0x7fce78068b50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fce800ad000 tx=0x7fce800a6000 comp rx=0 tx=0).stop 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce8c0c08f0 msgr2=0x7fce8c152cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce8c0c08f0 0x7fce8c152cd0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c530 tx=0x7fce7c00c9f0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 shutdown_connections 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fce78066690 0x7fce78068b50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce8c1510a0 0x7fce8c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce8c0c08f0 0x7fce8c152cd0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 --2- 172.21.15.39:0/583576755 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fce8c06bad0 0x7fce8c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 >> 172.21.15.39:0/583576755 conn(0x7fce8c0c5790 msgr2=0x7fce8c0a8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:47.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 shutdown_connections 2024-09-13T17:17:47.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.629+0000 7fce91868640 1 -- 172.21.15.39:0/583576755 wait complete. 2024-09-13T17:17:47.649 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T17:17:47.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:17:47.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.777+0000 7fe12515e640 1 Processor -- start 2024-09-13T17:17:47.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.777+0000 7fe12515e640 1 -- start start 2024-09-13T17:17:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.777+0000 7fe12515e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe1201514d0 0x7fe1200c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe120157370 0x7fe120154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe120155270 con 0x7fe1200bcf90 2024-09-13T17:17:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe120153d80 con 0x7fe1201514d0 2024-09-13T17:17:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe120157870 con 0x7fe120157370 2024-09-13T17:17:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11ed9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe1201514d0 0x7fe1200c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11ed9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe1201514d0 0x7fe1200c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40826/0 (socket says 172.21.15.39:40826) 2024-09-13T17:17:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11ed9d640 1 -- 172.21.15.39:0/2892012862 learned_addr learned my addr 172.21.15.39:0/2892012862 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11f59e640 1 --2- 172.21.15.39:0/2892012862 >> v2:172.21.15.84:3300/0 conn(0x7fe120157370 0x7fe120154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11ed9d640 1 -- 172.21.15.39:0/2892012862 >> v2:172.21.15.84:3300/0 conn(0x7fe120157370 msgr2=0x7fe120154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11ed9d640 1 --2- 172.21.15.39:0/2892012862 >> v2:172.21.15.84:3300/0 conn(0x7fe120157370 0x7fe120154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11ed9d640 1 -- 172.21.15.39:0/2892012862 >> v1:172.21.15.84:6789/0 conn(0x7fe1200bcf90 legacy=0x7fe120154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11ed9d640 1 -- 172.21.15.39:0/2892012862 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe120157b50 con 0x7fe1201514d0 2024-09-13T17:17:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11f59e640 1 --2- 172.21.15.39:0/2892012862 >> v2:172.21.15.84:3300/0 conn(0x7fe120157370 0x7fe120154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:47.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11ed9d640 1 --2- 172.21.15.39:0/2892012862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1201514d0 0x7fe1200c41f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe120061270 tx=0x7fe10801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9536f485410563b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11dd9b640 1 -- 172.21.15.39:0/2892012862 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080ab600 con 0x7fe1201514d0 2024-09-13T17:17:47.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11dd9b640 1 -- 172.21.15.39:0/2892012862 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1080a7050 con 0x7fe1201514d0 2024-09-13T17:17:47.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe11dd9b640 1 -- 172.21.15.39:0/2892012862 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080ab980 con 0x7fe1201514d0 2024-09-13T17:17:47.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 -- 172.21.15.39:0/2892012862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1201514d0 msgr2=0x7fe1200c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 --2- 172.21.15.39:0/2892012862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1201514d0 0x7fe1200c41f0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe120061270 tx=0x7fe10801c450 comp rx=0 tx=0).stop 2024-09-13T17:17:47.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 -- 172.21.15.39:0/2892012862 shutdown_connections 2024-09-13T17:17:47.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 --2- 172.21.15.39:0/2892012862 >> v2:172.21.15.84:3300/0 conn(0x7fe120157370 0x7fe120154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 --2- 172.21.15.39:0/2892012862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1201514d0 0x7fe1200c41f0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 -- 172.21.15.39:0/2892012862 >> 172.21.15.39:0/2892012862 conn(0x7fe12006a210 msgr2=0x7fe12006a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:47.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 -- 172.21.15.39:0/2892012862 shutdown_connections 2024-09-13T17:17:47.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.781+0000 7fe12515e640 1 -- 172.21.15.39:0/2892012862 wait complete. 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 Processor -- start 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 -- start start 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200bcf90 0x7fe120147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1201514d0 0x7fe1201504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe120157370 0x7fe1201445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12014f400 con 0x7fe1200bcf90 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe120147100 con 0x7fe120157370 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1200a9230 con 0x7fe1201514d0 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200bcf90 0x7fe120147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200bcf90 0x7fe120147c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40834/0 (socket says 172.21.15.39:40834) 2024-09-13T17:17:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11ed9d640 1 -- 172.21.15.39:0/202836094 learned_addr learned my addr 172.21.15.39:0/202836094 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11f59e640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe120157370 0x7fe1201445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11ed9d640 1 -- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1201514d0 msgr2=0x7fe1201504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11e59c640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1201514d0 0x7fe1201504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11ed9d640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe1201514d0 0x7fe1201504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11ed9d640 1 -- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe120157370 msgr2=0x7fe1201445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11ed9d640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe120157370 0x7fe1201445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11ed9d640 1 -- 172.21.15.39:0/202836094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080094d0 con 0x7fe1200bcf90 2024-09-13T17:17:47.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11f59e640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe120157370 0x7fe1201445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:47.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe11ed9d640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200bcf90 0x7fe120147c10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe10801ab00 tx=0x7fe1080191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe1177fe640 1 -- 172.21.15.39:0/202836094 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080b4420 con 0x7fe1200bcf90 2024-09-13T17:17:47.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe1177fe640 1 -- 172.21.15.39:0/202836094 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1080b4580 con 0x7fe1200bcf90 2024-09-13T17:17:47.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1201463a0 con 0x7fe1200bcf90 2024-09-13T17:17:47.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe1177fe640 1 -- 172.21.15.39:0/202836094 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080b48f0 con 0x7fe1200bcf90 2024-09-13T17:17:47.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1200c94a0 con 0x7fe1200bcf90 2024-09-13T17:17:47.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1200c3be0 con 0x7fe1200bcf90 2024-09-13T17:17:47.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.785+0000 7fe1177fe640 1 -- 172.21.15.39:0/202836094 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe1080b2070 con 0x7fe1200bcf90 2024-09-13T17:17:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe1177fe640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe100066690 0x7fe100068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe1177fe640 1 -- 172.21.15.39:0/202836094 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7fe10811f380 con 0x7fe1200bcf90 2024-09-13T17:17:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe11e59c640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe100066690 0x7fe100068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe1177fe640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7fe100073070 0x7fe1000754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe1177fe640 1 -- 172.21.15.39:0/202836094 --> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe100075bc0 con 0x7fe100073070 2024-09-13T17:17:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe1177fe640 1 -- 172.21.15.39:0/202836094 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe10811f790 con 0x7fe1200bcf90 2024-09-13T17:17:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe11e59c640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe100066690 0x7fe100068b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe120153d50 tx=0x7fe110008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe11f59e640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7fe100073070 0x7fe1000754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe11f59e640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7fe100073070 0x7fe1000754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.789+0000 7fe1177fe640 1 -- 172.21.15.39:0/202836094 <== osd.4 v2:172.21.15.84:6800/1588544227 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe100075bc0 con 0x7fe100073070 2024-09-13T17:17:47.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.801+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 --> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe12015de80 con 0x7fe100073070 2024-09-13T17:17:47.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe1177fe640 1 -- 172.21.15.39:0/202836094 <== osd.4 v2:172.21.15.84:6800/1588544227 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe12015de80 con 0x7fe100073070 2024-09-13T17:17:47.807 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T17:17:47.807 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:17:47.807 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T17:17:47.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7fe100073070 msgr2=0x7fe1000754f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.84:6800/1588544227,v1:172.21.15.84:6801/1588544227] conn(0x7fe100073070 0x7fe1000754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe100066690 msgr2=0x7fe100068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fe100066690 0x7fe100068b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe120153d50 tx=0x7fe110008040 comp rx=0 tx=0).stop 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200bcf90 msgr2=0x7fe120147c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 --2- 172.21.15.39:0/202836094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200bcf90 0x7fe120147c10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe10801ab00 tx=0x7fe1080191f0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe11f59e640 1 -- 172.21.15.39:0/202836094 reap_dead start 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 shutdown_connections 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 >> 172.21.15.39:0/202836094 conn(0x7fe12006a210 msgr2=0x7fe120155400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 shutdown_connections 2024-09-13T17:17:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.805+0000 7fe12515e640 1 -- 172.21.15.39:0/202836094 wait complete. 2024-09-13T17:17:47.825 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T17:17:47.844 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:47.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.933+0000 7f73d702d640 1 Processor -- start 2024-09-13T17:17:47.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.933+0000 7f73d702d640 1 -- start start 2024-09-13T17:17:47.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.933+0000 7f73d702d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f73d0071740 0x7f73d0151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.933+0000 7f73d702d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73d0072010 0x7f73d0157f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.933+0000 7f73d702d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d00c6490 con 0x7f73d00c5870 2024-09-13T17:17:47.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d0154dd0 con 0x7f73d0072010 2024-09-13T17:17:47.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d0154250 con 0x7f73d0071740 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d55ca640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73d0072010 0x7f73d0157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73c7fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f73d00c5870 0x7f73d0157830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40302/0 (socket says 172.21.15.39:40302) 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73c7fff640 1 -- 172.21.15.39:0/4106975724 learned_addr learned my addr 172.21.15.39:0/4106975724 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d4dc9640 1 --2- 172.21.15.39:0/4106975724 >> v2:172.21.15.84:3300/0 conn(0x7f73d0071740 0x7f73d0151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73c77fe640 1 -- 172.21.15.39:0/4106975724 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840937109 0 0) 0x7f73d00c6490 con 0x7f73d00c5870 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73c77fe640 1 -- 172.21.15.39:0/4106975724 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73b8003530 con 0x7f73d00c5870 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d55ca640 1 -- 172.21.15.39:0/4106975724 >> v2:172.21.15.84:3300/0 conn(0x7f73d0071740 msgr2=0x7f73d0151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d55ca640 1 --2- 172.21.15.39:0/4106975724 >> v2:172.21.15.84:3300/0 conn(0x7f73d0071740 0x7f73d0151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d55ca640 1 -- 172.21.15.39:0/4106975724 >> v1:172.21.15.39:6789/0 conn(0x7f73d00c5870 legacy=0x7f73d0157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d55ca640 1 -- 172.21.15.39:0/4106975724 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0154530 con 0x7f73d0072010 2024-09-13T17:17:47.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d55ca640 1 --2- 172.21.15.39:0/4106975724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d0072010 0x7f73d0157f40 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f73d00c54c0 tx=0x7f73c801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bbee513d46f108 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73c77fe640 1 -- 172.21.15.39:0/4106975724 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c80a58a0 con 0x7f73d0072010 2024-09-13T17:17:47.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73c77fe640 1 -- 172.21.15.39:0/4106975724 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c80a5a00 con 0x7f73d0072010 2024-09-13T17:17:47.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73c77fe640 1 -- 172.21.15.39:0/4106975724 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c80a5d90 con 0x7f73d0072010 2024-09-13T17:17:47.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 -- 172.21.15.39:0/4106975724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d0072010 msgr2=0x7f73d0157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 --2- 172.21.15.39:0/4106975724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d0072010 0x7f73d0157f40 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f73d00c54c0 tx=0x7f73c801c7e0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 -- 172.21.15.39:0/4106975724 shutdown_connections 2024-09-13T17:17:47.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 --2- 172.21.15.39:0/4106975724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d0072010 0x7f73d0157f40 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 --2- 172.21.15.39:0/4106975724 >> v2:172.21.15.84:3300/0 conn(0x7f73d0071740 0x7f73d0151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 -- 172.21.15.39:0/4106975724 >> 172.21.15.39:0/4106975724 conn(0x7f73d00a9720 msgr2=0x7f73d00a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:47.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 -- 172.21.15.39:0/4106975724 shutdown_connections 2024-09-13T17:17:47.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.937+0000 7f73d702d640 1 -- 172.21.15.39:0/4106975724 wait complete. 2024-09-13T17:17:47.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 Processor -- start 2024-09-13T17:17:47.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 -- start start 2024-09-13T17:17:47.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73d0071740 0x7f73d014cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d0072010 0x7f73d01536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d00c5870 0x7f73d0147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01534e0 con 0x7f73d00c5870 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d006a8b0 con 0x7f73d0072010 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0153200 con 0x7f73d0071740 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d55ca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d00c5870 0x7f73d0147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d55ca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d00c5870 0x7f73d0147bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40838/0 (socket says 172.21.15.39:40838) 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d55ca640 1 -- 172.21.15.39:0/1436643410 learned_addr learned my addr 172.21.15.39:0/1436643410 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d4dc9640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73d0071740 0x7f73d014cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73c7fff640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d0072010 0x7f73d01536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d55ca640 1 -- 172.21.15.39:0/1436643410 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73d0071740 msgr2=0x7f73d014cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d55ca640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73d0071740 0x7f73d014cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d55ca640 1 -- 172.21.15.39:0/1436643410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d0072010 msgr2=0x7f73d01536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d55ca640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d0072010 0x7f73d01536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:47.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d55ca640 1 -- 172.21.15.39:0/1436643410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c8009a70 con 0x7f73d00c5870 2024-09-13T17:17:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d55ca640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d00c5870 0x7f73d0147bb0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f73c80b0040 tx=0x7f73c8019580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73c57fa640 1 -- 172.21.15.39:0/1436643410 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c80b9650 con 0x7f73d00c5870 2024-09-13T17:17:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73c57fa640 1 -- 172.21.15.39:0/1436643410 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c80b97b0 con 0x7f73d00c5870 2024-09-13T17:17:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d01ecca0 con 0x7f73d00c5870 2024-09-13T17:17:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73c57fa640 1 -- 172.21.15.39:0/1436643410 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c80b9b80 con 0x7f73d00c5870 2024-09-13T17:17:47.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d014d0d0 con 0x7f73d00c5870 2024-09-13T17:17:47.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.941+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d006f6b0 con 0x7f73d00c5870 2024-09-13T17:17:47.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.945+0000 7f73c57fa640 1 -- 172.21.15.39:0/1436643410 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73c80b9ce0 con 0x7f73d00c5870 2024-09-13T17:17:47.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.945+0000 7f73c57fa640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73bc066690 0x7f73bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.945+0000 7f73c57fa640 1 -- 172.21.15.39:0/1436643410 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f73c81208b0 con 0x7f73d00c5870 2024-09-13T17:17:47.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.945+0000 7f73d4dc9640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73bc066690 0x7f73bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.945+0000 7f73d4dc9640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73bc066690 0x7f73bc068b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009a50 tx=0x7f73c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:47.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.949+0000 7f73c57fa640 1 -- 172.21.15.39:0/1436643410 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f73c80ecb70 con 0x7f73d00c5870 2024-09-13T17:17:47.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.993+0000 7f3a40236640 1 Processor -- start 2024-09-13T17:17:47.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.993+0000 7f3a40236640 1 -- start start 2024-09-13T17:17:47.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a40236640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a3806d2d0 0x7f3a3815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:47.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a40236640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a38156270 con 0x7f3a380abc50 2024-09-13T17:17:47.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a40236640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a381554b0 con 0x7f3a380ab340 2024-09-13T17:17:47.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a40236640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a38061530 con 0x7f3a3806d2d0 2024-09-13T17:17:47.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3dfd2640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f3a380abc50 0x7f3a3814f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40316/0 (socket says 172.21.15.39:40316) 2024-09-13T17:17:47.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3dfd2640 1 -- 172.21.15.39:0/2148189363 learned_addr learned my addr 172.21.15.39:0/2148189363 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:47.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3e7d3640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a3806d2d0 0x7f3a3815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:47.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737994407 0 0) 0x7f3a38156270 con 0x7f3a380abc50 2024-09-13T17:17:47.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a20003530 con 0x7f3a380abc50 2024-09-13T17:17:48.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2096584970 0 0) 0x7f3a20003530 con 0x7f3a380abc50 2024-09-13T17:17:48.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a3806d2d0 msgr2=0x7f3a3815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:48.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a3806d2d0 0x7f3a3815d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 >> v1:172.21.15.84:6789/0 conn(0x7f3a380ab340 legacy=0x7f3a3806cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a38155790 con 0x7f3a380abc50 2024-09-13T17:17:48.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f3a28001680 con 0x7f3a380abc50 2024-09-13T17:17:48.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 >> v1:172.21.15.39:6789/0 conn(0x7f3a380abc50 legacy=0x7f3a3814f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a200039b0 0x7f3a20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a200047d0 0x7f3a20004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a20005190 0x7f3a20014590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a38061530 con 0x7f3a200047d0 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a20014ad0 con 0x7f3a200039b0 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a20014c10 con 0x7f3a20005190 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3d7d1640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a200047d0 0x7f3a20004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3dfd2640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a200039b0 0x7f3a20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3d7d1640 1 -- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a20005190 msgr2=0x7f3a20014590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3d7d1640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a20005190 0x7f3a20014590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3d7d1640 1 -- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a200039b0 msgr2=0x7f3a20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3d7d1640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a200039b0 0x7f3a20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:47.997+0000 7f3a3d7d1640 1 -- 172.21.15.39:0/2148189363 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20003530 con 0x7f3a200047d0 2024-09-13T17:17:48.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3d7d1640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a200047d0 0x7f3a20004c50 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0017b0 tx=0x7f3a2c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaafa246770c0f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c0a4ea0 con 0x7f3a200047d0 2024-09-13T17:17:48.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2c0a7050 con 0x7f3a200047d0 2024-09-13T17:17:48.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3cfd0640 1 -- 172.21.15.39:0/2148189363 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c0052f0 con 0x7f3a200047d0 2024-09-13T17:17:48.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 -- 172.21.15.39:0/2148189363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a200047d0 msgr2=0x7f3a20004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 --2- 172.21.15.39:0/2148189363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a200047d0 0x7f3a20004c50 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0017b0 tx=0x7f3a2c01c420 comp rx=0 tx=0).stop 2024-09-13T17:17:48.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3e7d3640 1 -- 172.21.15.39:0/2148189363 reap_dead start 2024-09-13T17:17:48.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 -- 172.21.15.39:0/2148189363 shutdown_connections 2024-09-13T17:17:48.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 -- 172.21.15.39:0/2148189363 >> 172.21.15.39:0/2148189363 conn(0x7f3a38069e10 msgr2=0x7f3a3806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 -- 172.21.15.39:0/2148189363 shutdown_connections 2024-09-13T17:17:48.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 -- 172.21.15.39:0/2148189363 wait complete. 2024-09-13T17:17:48.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 Processor -- start 2024-09-13T17:17:48.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 -- start start 2024-09-13T17:17:48.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a3813f5e0 0x7f3a38144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a38146290 0x7f3a38146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a3813c1f0 0x7f3a3813c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a38146c50 con 0x7f3a3813f5e0 2024-09-13T17:17:48.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380c06c0 con 0x7f3a38146290 2024-09-13T17:17:48.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a40236640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380c03e0 con 0x7f3a3813c1f0 2024-09-13T17:17:48.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3dfd2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a3813f5e0 0x7f3a38144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3dfd2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a3813f5e0 0x7f3a38144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40848/0 (socket says 172.21.15.39:40848) 2024-09-13T17:17:48.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3dfd2640 1 -- 172.21.15.39:0/862454352 learned_addr learned my addr 172.21.15.39:0/862454352 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3e7d3640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a3813c1f0 0x7f3a3813c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.001+0000 7f3a3d7d1640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a38146290 0x7f3a38146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a3dfd2640 1 -- 172.21.15.39:0/862454352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a3813c1f0 msgr2=0x7f3a3813c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a3dfd2640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a3813c1f0 0x7f3a3813c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a3dfd2640 1 -- 172.21.15.39:0/862454352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a38146290 msgr2=0x7f3a38146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a3dfd2640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a38146290 0x7f3a38146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a3dfd2640 1 -- 172.21.15.39:0/862454352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380c5aa0 con 0x7f3a3813f5e0 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a3dfd2640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a3813f5e0 0x7f3a38144680 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800eaa0 tx=0x7f3a2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a367fc640 1 -- 172.21.15.39:0/862454352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800c8a0 con 0x7f3a3813f5e0 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a367fc640 1 -- 172.21.15.39:0/862454352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2800ca20 con 0x7f3a3813f5e0 2024-09-13T17:17:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a380c5d80 con 0x7f3a3813f5e0 2024-09-13T17:17:48.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a367fc640 1 -- 172.21.15.39:0/862454352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800cdd0 con 0x7f3a3813f5e0 2024-09-13T17:17:48.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a380c2a00 con 0x7f3a3813f5e0 2024-09-13T17:17:48.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a367fc640 1 -- 172.21.15.39:0/862454352 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a2801d460 con 0x7f3a3813f5e0 2024-09-13T17:17:48.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a367fc640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3a24066690 0x7f3a24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a38145fb0 con 0x7f3a3813f5e0 2024-09-13T17:17:48.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a367fc640 1 -- 172.21.15.39:0/862454352 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f3a28085270 con 0x7f3a3813f5e0 2024-09-13T17:17:48.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a3d7d1640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3a24066690 0x7f3a24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.005+0000 7f3a3d7d1640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3a24066690 0x7f3a24068b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c001780 tx=0x7f3a2c0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.009+0000 7f3a367fc640 1 -- 172.21.15.39:0/862454352 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3a38145fb0 con 0x7f3a3813f5e0 2024-09-13T17:17:48.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.125+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f73d006a5d0 con 0x7f73bc066690 2024-09-13T17:17:48.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.125+0000 7f73c57fa640 1 -- 172.21.15.39:0/1436643410 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f73d006a5d0 con 0x7f73bc066690 2024-09-13T17:17:48.129 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T17:17:48.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73bc066690 msgr2=0x7f73bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73bc066690 0x7f73bc068b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009a50 tx=0x7f73c0008040 comp rx=0 tx=0).stop 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d00c5870 msgr2=0x7f73d0147bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d00c5870 0x7f73d0147bb0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f73c80b0040 tx=0x7f73c8019580 comp rx=0 tx=0).stop 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 shutdown_connections 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f73bc066690 0x7f73bc068b50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f73d0072010 0x7f73d01536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73d00c5870 0x7f73d0147bb0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 --2- 172.21.15.39:0/1436643410 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f73d0071740 0x7f73d014cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 >> 172.21.15.39:0/1436643410 conn(0x7f73d00a9720 msgr2=0x7f73d014d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 shutdown_connections 2024-09-13T17:17:48.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.129+0000 7f73d702d640 1 -- 172.21.15.39:0/1436643410 wait complete. 2024-09-13T17:17:48.148 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T17:17:48.148 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:17:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.213+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a38145fb0 con 0x7f3a3813f5e0 2024-09-13T17:17:48.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.213+0000 7f3a367fc640 1 -- 172.21.15.39:0/862454352 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+5182 (secure 0 0 0) 0x7f3a38145fb0 con 0x7f3a3813f5e0 2024-09-13T17:17:48.218 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:48.218 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":206,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:17:47.987831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7555":{"gid":7555,"name":"d","rank":0,"incarnation":203,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:48.218 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 206 2024-09-13T17:17:48.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3a24066690 msgr2=0x7f3a24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3a24066690 0x7f3a24068b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c001780 tx=0x7f3a2c0011a0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a3813f5e0 msgr2=0x7f3a38144680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a3813f5e0 0x7f3a38144680 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800eaa0 tx=0x7f3a2800ef60 comp rx=0 tx=0).stop 2024-09-13T17:17:48.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 shutdown_connections 2024-09-13T17:17:48.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f3a24066690 0x7f3a24068b50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a38146290 0x7f3a38146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f3a3813c1f0 0x7f3a3813c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 --2- 172.21.15.39:0/862454352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a3813f5e0 0x7f3a38144680 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 >> 172.21.15.39:0/862454352 conn(0x7f3a38069e10 msgr2=0x7f3a38152ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 shutdown_connections 2024-09-13T17:17:48.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.217+0000 7f3a40236640 1 -- 172.21.15.39:0/862454352 wait complete. 2024-09-13T17:17:48.243 INFO:tasks.mds_thrash.fs.[a]:mds.d has gained rank=0, replacing gid=8319 2024-09-13T17:17:48.243 INFO:tasks.mds_thrash.fs.[a]:waiting for 1 secs before reviving mds.a 2024-09-13T17:17:48.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa571456640 1 Processor -- start 2024-09-13T17:17:48.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa571456640 1 -- start start 2024-09-13T17:17:48.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa571456640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa56c1514d0 0x7fa56c0c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa571456640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56c157370 0x7fa56c154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa571456640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c155270 con 0x7fa56c0bcf90 2024-09-13T17:17:48.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa571456640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c153d80 con 0x7fa56c1514d0 2024-09-13T17:17:48.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa571456640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c157870 con 0x7fa56c157370 2024-09-13T17:17:48.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa56affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa56c1514d0 0x7fa56c0c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa56a7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa56c0bcf90 0x7fa56c154240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40318/0 (socket says 172.21.15.39:40318) 2024-09-13T17:17:48.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa56a7fc640 1 -- 172.21.15.39:0/3386149607 learned_addr learned my addr 172.21.15.39:0/3386149607 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:48.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa56affd640 1 -- 172.21.15.39:0/3386149607 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56c157370 msgr2=0x7fa56c154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:48.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa56b7fe640 1 --2- 172.21.15.39:0/3386149607 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56c157370 0x7fa56c154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa56affd640 1 --2- 172.21.15.39:0/3386149607 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56c157370 0x7fa56c154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa56affd640 1 -- 172.21.15.39:0/3386149607 >> v1:172.21.15.39:6789/0 conn(0x7fa56c0bcf90 legacy=0x7fa56c154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa569ffb640 1 -- 172.21.15.39:0/3386149607 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704960478 0 0) 0x7fa56c155270 con 0x7fa56c0bcf90 2024-09-13T17:17:48.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.273+0000 7fa56affd640 1 -- 172.21.15.39:0/3386149607 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c157b50 con 0x7fa56c1514d0 2024-09-13T17:17:48.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa56affd640 1 --2- 172.21.15.39:0/3386149607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c1514d0 0x7fa56c0c41f0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0097c0 tx=0x7fa55c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867859be74bbd5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa569ffb640 1 -- 172.21.15.39:0/3386149607 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0ab600 con 0x7fa56c1514d0 2024-09-13T17:17:48.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa569ffb640 1 -- 172.21.15.39:0/3386149607 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c0a7050 con 0x7fa56c1514d0 2024-09-13T17:17:48.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa569ffb640 1 -- 172.21.15.39:0/3386149607 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0ab980 con 0x7fa56c1514d0 2024-09-13T17:17:48.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 -- 172.21.15.39:0/3386149607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c1514d0 msgr2=0x7fa56c0c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 --2- 172.21.15.39:0/3386149607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c1514d0 0x7fa56c0c41f0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0097c0 tx=0x7fa55c01c410 comp rx=0 tx=0).stop 2024-09-13T17:17:48.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 -- 172.21.15.39:0/3386149607 shutdown_connections 2024-09-13T17:17:48.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 --2- 172.21.15.39:0/3386149607 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56c157370 0x7fa56c154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 --2- 172.21.15.39:0/3386149607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c1514d0 0x7fa56c0c41f0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 -- 172.21.15.39:0/3386149607 >> 172.21.15.39:0/3386149607 conn(0x7fa56c06a210 msgr2=0x7fa56c06a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 -- 172.21.15.39:0/3386149607 shutdown_connections 2024-09-13T17:17:48.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 -- 172.21.15.39:0/3386149607 wait complete. 2024-09-13T17:17:48.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 Processor -- start 2024-09-13T17:17:48.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.277+0000 7fa571456640 1 -- start start 2024-09-13T17:17:48.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa571456640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56c0bcf90 0x7fa56c150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa571456640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa56c1514d0 0x7fa56c1f0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa571456640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c157370 0x7fa56c14efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa571456640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c1f0fb0 con 0x7fa56c157370 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa571456640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c148ba0 con 0x7fa56c1514d0 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa571456640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c148e80 con 0x7fa56c0bcf90 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c157370 0x7fa56c14efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c157370 0x7fa56c14efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40864/0 (socket says 172.21.15.39:40864) 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56b7fe640 1 -- 172.21.15.39:0/3143358374 learned_addr learned my addr 172.21.15.39:0/3143358374 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56a7fc640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa56c1514d0 0x7fa56c1f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56b7fe640 1 -- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56c0bcf90 msgr2=0x7fa56c150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56b7fe640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa56c0bcf90 0x7fa56c150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56b7fe640 1 -- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa56c1514d0 msgr2=0x7fa56c1f0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56b7fe640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa56c1514d0 0x7fa56c1f0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56b7fe640 1 -- 172.21.15.39:0/3143358374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c009420 con 0x7fa56c157370 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56b7fe640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c157370 0x7fa56c14efe0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa56400cd90 tx=0x7fa56400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa5637fe640 1 -- 172.21.15.39:0/3143358374 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564003ba0 con 0x7fa56c157370 2024-09-13T17:17:48.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa5637fe640 1 -- 172.21.15.39:0/3143358374 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564003d20 con 0x7fa56c157370 2024-09-13T17:17:48.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c150120 con 0x7fa56c157370 2024-09-13T17:17:48.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa5637fe640 1 -- 172.21.15.39:0/3143358374 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564004260 con 0x7fa56c157370 2024-09-13T17:17:48.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c0aae70 con 0x7fa56c157370 2024-09-13T17:17:48.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa56c1489e0 con 0x7fa56c157370 2024-09-13T17:17:48.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa5637fe640 1 -- 172.21.15.39:0/3143358374 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa5640043c0 con 0x7fa56c157370 2024-09-13T17:17:48.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa5637fe640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa54c066690 0x7fa54c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa5637fe640 1 -- 172.21.15.39:0/3143358374 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7fa564012070 con 0x7fa56c157370 2024-09-13T17:17:48.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56affd640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa54c066690 0x7fa54c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa5637fe640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7fa54c073070 0x7fa54c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa5637fe640 1 -- 172.21.15.39:0/3143358374 --> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa54c075bc0 con 0x7fa54c073070 2024-09-13T17:17:48.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa5637fe640 1 -- 172.21.15.39:0/3143358374 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5640864d0 con 0x7fa56c157370 2024-09-13T17:17:48.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56affd640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa54c066690 0x7fa54c068b50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0ad000 tx=0x7fa55c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56a7fc640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7fa54c073070 0x7fa54c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.281+0000 7fa56a7fc640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7fa54c073070 0x7fa54c0754f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.285+0000 7fa5637fe640 1 -- 172.21.15.39:0/3143358374 <== osd.5 v2:172.21.15.84:6808/2432365689 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa54c075bc0 con 0x7fa54c073070 2024-09-13T17:17:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 --> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa56c150360 con 0x7fa54c073070 2024-09-13T17:17:48.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa5637fe640 1 -- 172.21.15.39:0/3143358374 <== osd.5 v2:172.21.15.84:6808/2432365689 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa56c150360 con 0x7fa54c073070 2024-09-13T17:17:48.301 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T17:17:48.301 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:17:48.301 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T17:17:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7fa54c073070 msgr2=0x7fa54c0754f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.84:6808/2432365689,v1:172.21.15.84:6810/2432365689] conn(0x7fa54c073070 0x7fa54c0754f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa54c066690 msgr2=0x7fa54c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa54c066690 0x7fa54c068b50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0ad000 tx=0x7fa55c004740 comp rx=0 tx=0).stop 2024-09-13T17:17:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c157370 msgr2=0x7fa56c14efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 --2- 172.21.15.39:0/3143358374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56c157370 0x7fa56c14efe0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fa56400cd90 tx=0x7fa56400a660 comp rx=0 tx=0).stop 2024-09-13T17:17:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa56b7fe640 1 -- 172.21.15.39:0/3143358374 reap_dead start 2024-09-13T17:17:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 shutdown_connections 2024-09-13T17:17:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 >> 172.21.15.39:0/3143358374 conn(0x7fa56c06a210 msgr2=0x7fa56c154140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 shutdown_connections 2024-09-13T17:17:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.297+0000 7fa571456640 1 -- 172.21.15.39:0/3143358374 wait complete. 2024-09-13T17:17:48.316 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T17:17:48.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4cacf8f640 1 Processor -- start 2024-09-13T17:17:48.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4cacf8f640 1 -- start start 2024-09-13T17:17:48.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4cacf8f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4ca806b840 0x7f4ca815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4cacf8f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca806c740 con 0x7f4ca80c0b40 2024-09-13T17:17:48.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4cacf8f640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca80c0f20 con 0x7f4ca806c360 2024-09-13T17:17:48.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4cacf8f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca8151ab0 con 0x7f4ca806b840 2024-09-13T17:17:48.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4ca6d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4ca806b840 0x7f4ca815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4c9ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4ca80c0b40 0x7f4ca806b130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40332/0 (socket says 172.21.15.39:40332) 2024-09-13T17:17:48.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4c9ffff640 1 -- 172.21.15.39:0/4238552307 learned_addr learned my addr 172.21.15.39:0/4238552307 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:48.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4ca6d9d640 1 -- 172.21.15.39:0/4238552307 >> v1:172.21.15.84:6789/0 conn(0x7f4ca806c360 legacy=0x7f4ca80c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:48.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4ca6d9d640 1 -- 172.21.15.39:0/4238552307 >> v1:172.21.15.39:6789/0 conn(0x7f4ca80c0b40 legacy=0x7f4ca806b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4ca6d9d640 1 -- 172.21.15.39:0/4238552307 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80c21e0 con 0x7f4ca806b840 2024-09-13T17:17:48.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.469+0000 7f4ca5d9b640 1 -- 172.21.15.39:0/4238552307 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236288711 0 0) 0x7f4ca806c740 con 0x7f4ca80c0b40 2024-09-13T17:17:48.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4ca6d9d640 1 --2- 172.21.15.39:0/4238552307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806b840 0x7f4ca815c4d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009d90 tx=0x7f4ca001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f89ba929884430 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4ca5d9b640 1 -- 172.21.15.39:0/4238552307 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca00a57e0 con 0x7f4ca806b840 2024-09-13T17:17:48.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4ca5d9b640 1 -- 172.21.15.39:0/4238552307 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca00a5940 con 0x7f4ca806b840 2024-09-13T17:17:48.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4ca5d9b640 1 -- 172.21.15.39:0/4238552307 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca00a5cd0 con 0x7f4ca806b840 2024-09-13T17:17:48.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4cacf8f640 1 -- 172.21.15.39:0/4238552307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806b840 msgr2=0x7f4ca815c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4cacf8f640 1 --2- 172.21.15.39:0/4238552307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806b840 0x7f4ca815c4d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009d90 tx=0x7f4ca001c930 comp rx=0 tx=0).stop 2024-09-13T17:17:48.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4cacf8f640 1 -- 172.21.15.39:0/4238552307 shutdown_connections 2024-09-13T17:17:48.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4cacf8f640 1 --2- 172.21.15.39:0/4238552307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806b840 0x7f4ca815c4d0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4cacf8f640 1 -- 172.21.15.39:0/4238552307 >> 172.21.15.39:0/4238552307 conn(0x7f4ca80c5790 msgr2=0x7f4ca80c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4cacf8f640 1 -- 172.21.15.39:0/4238552307 shutdown_connections 2024-09-13T17:17:48.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4cacf8f640 1 -- 172.21.15.39:0/4238552307 wait complete. 2024-09-13T17:17:48.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4cacf8f640 1 Processor -- start 2024-09-13T17:17:48.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.473+0000 7f4cacf8f640 1 -- start start 2024-09-13T17:17:48.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4cacf8f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ca806b840 0x7f4ca814f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4cacf8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806c360 0x7f4ca815b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4cacf8f640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ca80c0b40 0x7f4ca815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4cacf8f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80bf3b0 con 0x7f4ca806c360 2024-09-13T17:17:48.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4cacf8f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca81f0350 con 0x7f4ca806b840 2024-09-13T17:17:48.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4cacf8f640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca81f0880 con 0x7f4ca80c0b40 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806c360 0x7f4ca815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806c360 0x7f4ca815b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40872/0 (socket says 172.21.15.39:40872) 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9ffff640 1 -- 172.21.15.39:0/3229756412 learned_addr learned my addr 172.21.15.39:0/3229756412 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4ca659c640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ca806b840 0x7f4ca814f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4ca6d9d640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ca80c0b40 0x7f4ca815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9ffff640 1 -- 172.21.15.39:0/3229756412 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ca80c0b40 msgr2=0x7f4ca815bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9ffff640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ca80c0b40 0x7f4ca815bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9ffff640 1 -- 172.21.15.39:0/3229756412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ca806b840 msgr2=0x7f4ca814f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9ffff640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ca806b840 0x7f4ca814f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9ffff640 1 -- 172.21.15.39:0/3229756412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca0009a70 con 0x7f4ca806c360 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4ca659c640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ca806b840 0x7f4ca814f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9ffff640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806c360 0x7f4ca815b650 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c4b0 tx=0x7f4c9000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9effd640 1 -- 172.21.15.39:0/3229756412 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c900074e0 con 0x7f4ca806c360 2024-09-13T17:17:48.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9effd640 1 -- 172.21.15.39:0/3229756412 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90007660 con 0x7f4ca806c360 2024-09-13T17:17:48.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca815c0d0 con 0x7f4ca806c360 2024-09-13T17:17:48.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9effd640 1 -- 172.21.15.39:0/3229756412 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90004a30 con 0x7f4ca806c360 2024-09-13T17:17:48.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca80becf0 con 0x7f4ca806c360 2024-09-13T17:17:48.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca80674c0 con 0x7f4ca806c360 2024-09-13T17:17:48.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9effd640 1 -- 172.21.15.39:0/3229756412 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c90004b90 con 0x7f4ca806c360 2024-09-13T17:17:48.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9effd640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c88066690 0x7f4c88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4c9effd640 1 -- 172.21.15.39:0/3229756412 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f4c900846f0 con 0x7f4ca806c360 2024-09-13T17:17:48.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4ca659c640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c88066690 0x7f4c88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.477+0000 7f4ca659c640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c88066690 0x7f4c88068b50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4c98003e40 tx=0x7f4c98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.481+0000 7f4c9effd640 1 -- 172.21.15.39:0/3229756412 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c90054420 con 0x7f4ca806c360 2024-09-13T17:17:48.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.633+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ca80c2fa0 con 0x7f4c88066690 2024-09-13T17:17:48.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4c9effd640 1 -- 172.21.15.39:0/3229756412 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4c98005420 con 0x7f4c88066690 2024-09-13T17:17:48.639 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c88066690 msgr2=0x7f4c88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c88066690 0x7f4c88068b50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4c98003e40 tx=0x7f4c98009040 comp rx=0 tx=0).stop 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806c360 msgr2=0x7f4ca815b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806c360 0x7f4ca815b650 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c4b0 tx=0x7f4c9000c970 comp rx=0 tx=0).stop 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 shutdown_connections 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4c88066690 0x7f4c88068b50 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4ca80c0b40 0x7f4ca815bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca806c360 0x7f4ca815b650 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 --2- 172.21.15.39:0/3229756412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ca806b840 0x7f4ca814f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 >> 172.21.15.39:0/3229756412 conn(0x7f4ca80c5790 msgr2=0x7f4ca80a69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 shutdown_connections 2024-09-13T17:17:48.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.637+0000 7f4cacf8f640 1 -- 172.21.15.39:0/3229756412 wait complete. 2024-09-13T17:17:48.656 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T17:17:48.657 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:17:48.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.769+0000 7f80e1a4e640 1 Processor -- start 2024-09-13T17:17:48.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80e1a4e640 1 -- start start 2024-09-13T17:17:48.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80e1a4e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80dc0aa2d0 0x7f80dc0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80e1a4e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f80dc0a4d60 0x7f80dc14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80e1a4e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f80dc14f4b0 0x7f80dc151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80e1a4e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80dc155b30 con 0x7f80dc14f4b0 2024-09-13T17:17:48.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80e1a4e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80dc155d20 con 0x7f80dc0a4d60 2024-09-13T17:17:48.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80e1a4e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc0c4750 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80db7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f80dc14f4b0 0x7f80dc151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80db7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f80dc14f4b0 0x7f80dc151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40878/0 (socket says 172.21.15.39:40878) 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80db7fe640 1 -- 172.21.15.39:0/2449421162 learned_addr learned my addr 172.21.15.39:0/2449421162 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80da7fc640 1 --2- 172.21.15.39:0/2449421162 >> v2:172.21.15.84:3300/0 conn(0x7f80dc0a4d60 0x7f80dc14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80db7fe640 1 -- 172.21.15.39:0/2449421162 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80dc0aa2d0 msgr2=0x7f80dc0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80db7fe640 1 --2- 172.21.15.39:0/2449421162 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80dc0aa2d0 0x7f80dc0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80db7fe640 1 -- 172.21.15.39:0/2449421162 >> v2:172.21.15.84:3300/0 conn(0x7f80dc0a4d60 msgr2=0x7f80dc14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80db7fe640 1 --2- 172.21.15.39:0/2449421162 >> v2:172.21.15.84:3300/0 conn(0x7f80dc0a4d60 0x7f80dc14ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80db7fe640 1 -- 172.21.15.39:0/2449421162 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc155fc0 con 0x7f80dc14f4b0 2024-09-13T17:17:48.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80da7fc640 1 --2- 172.21.15.39:0/2449421162 >> v2:172.21.15.84:3300/0 conn(0x7f80dc0a4d60 0x7f80dc14ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:48.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80db7fe640 1 --2- 172.21.15.39:0/2449421162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc14f4b0 0x7f80dc151cf0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0c67b0 tx=0x7f80d401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e08e33f7adbf1c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80d9ffb640 1 -- 172.21.15.39:0/2449421162 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc152a40 con 0x7f80dc14f4b0 2024-09-13T17:17:48.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80d9ffb640 1 -- 172.21.15.39:0/2449421162 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d40a56c0 con 0x7f80dc14f4b0 2024-09-13T17:17:48.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.773+0000 7f80d9ffb640 1 -- 172.21.15.39:0/2449421162 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d40a5a50 con 0x7f80dc14f4b0 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- 172.21.15.39:0/2449421162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc14f4b0 msgr2=0x7f80dc151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 --2- 172.21.15.39:0/2449421162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc14f4b0 0x7f80dc151cf0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0c67b0 tx=0x7f80d401c930 comp rx=0 tx=0).stop 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- 172.21.15.39:0/2449421162 shutdown_connections 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 --2- 172.21.15.39:0/2449421162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc14f4b0 0x7f80dc151cf0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 --2- 172.21.15.39:0/2449421162 >> v2:172.21.15.84:3300/0 conn(0x7f80dc0a4d60 0x7f80dc14ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 --2- 172.21.15.39:0/2449421162 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80dc0aa2d0 0x7f80dc0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- 172.21.15.39:0/2449421162 >> 172.21.15.39:0/2449421162 conn(0x7f80dc0c5790 msgr2=0x7f80dc0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- 172.21.15.39:0/2449421162 shutdown_connections 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- 172.21.15.39:0/2449421162 wait complete. 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 Processor -- start 2024-09-13T17:17:48.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- start start 2024-09-13T17:17:48.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80dc0a4d60 0x7f80dc1ed020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc0aa2d0 0x7f80dc0c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80dc14f4b0 0x7f80dc141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc072260 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc14ea40 con 0x7f80dc14f4b0 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc1ed970 con 0x7f80dc0a4d60 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80da7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc0aa2d0 0x7f80dc0c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80da7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc0aa2d0 0x7f80dc0c1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40892/0 (socket says 172.21.15.39:40892) 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80da7fc640 1 -- 172.21.15.39:0/17769886 learned_addr learned my addr 172.21.15.39:0/17769886 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80db7fe640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80dc14f4b0 0x7f80dc141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80da7fc640 1 -- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80dc0a4d60 msgr2=0x7f80dc1ed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80da7fc640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f80dc0a4d60 0x7f80dc1ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80da7fc640 1 -- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80dc14f4b0 msgr2=0x7f80dc141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80da7fc640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f80dc14f4b0 0x7f80dc141990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80da7fc640 1 -- 172.21.15.39:0/17769886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d4009b30 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80da7fc640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc0aa2d0 0x7f80dc0c1af0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c7f0 tx=0x7f80d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80c3fff640 1 -- 172.21.15.39:0/17769886 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d000cea0 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80c3fff640 1 -- 172.21.15.39:0/17769886 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0007a30 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80c3fff640 1 -- 172.21.15.39:0/17769886 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0007de0 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80dc071ac0 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80dc071f30 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.777+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80dc145b40 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80c3fff640 1 -- 172.21.15.39:0/17769886 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f80d0023420 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80c3fff640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c8066690 0x7f80c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80c3fff640 1 -- 172.21.15.39:0/17769886 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f80d0085450 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80daffd640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c8066690 0x7f80c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80c3fff640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f80c8073070 0x7f80c80754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80c3fff640 1 -- 172.21.15.39:0/17769886 --> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80c8075bc0 con 0x7f80c8073070 2024-09-13T17:17:48.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80c3fff640 1 -- 172.21.15.39:0/17769886 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80d0085860 con 0x7f80dc0aa2d0 2024-09-13T17:17:48.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80daffd640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c8066690 0x7f80c8068b50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009730 tx=0x7f80cc006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80db7fe640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f80c8073070 0x7f80c80754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80db7fe640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f80c8073070 0x7f80c80754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.781+0000 7f80c3fff640 1 -- 172.21.15.39:0/17769886 <== osd.6 v2:172.21.15.84:6809/2893476104 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f80c8075bc0 con 0x7f80c8073070 2024-09-13T17:17:48.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 --> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f80dc0c3690 con 0x7f80c8073070 2024-09-13T17:17:48.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80c3fff640 1 -- 172.21.15.39:0/17769886 <== osd.6 v2:172.21.15.84:6809/2893476104 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f80d4009210 con 0x7f80c8073070 2024-09-13T17:17:48.800 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T17:17:48.800 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:17:48.800 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f80c8073070 msgr2=0x7f80c80754f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.84:6809/2893476104,v1:172.21.15.84:6812/2893476104] conn(0x7f80c8073070 0x7f80c80754f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c8066690 msgr2=0x7f80c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f80c8066690 0x7f80c8068b50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009730 tx=0x7f80cc006ca0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc0aa2d0 msgr2=0x7f80dc0c1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 --2- 172.21.15.39:0/17769886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc0aa2d0 0x7f80dc0c1af0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c7f0 tx=0x7f80d000ccb0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80db7fe640 1 -- 172.21.15.39:0/17769886 reap_dead start 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 shutdown_connections 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 >> 172.21.15.39:0/17769886 conn(0x7f80dc0c5790 msgr2=0x7f80dc14ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 shutdown_connections 2024-09-13T17:17:48.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.797+0000 7f80e1a4e640 1 -- 172.21.15.39:0/17769886 wait complete. 2024-09-13T17:17:48.816 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T17:17:48.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f8be8640 1 Processor -- start 2024-09-13T17:17:48.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f8be8640 1 -- start start 2024-09-13T17:17:48.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f8be8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f23f40aa2d0 0x7f23f4154700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f8be8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f4153110 con 0x7f23f40aaaf0 2024-09-13T17:17:48.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f8be8640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f40c47f0 con 0x7f23f4155c20 2024-09-13T17:17:48.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f8be8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f4151c20 con 0x7f23f40aa2d0 2024-09-13T17:17:48.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f23f40aa2d0 0x7f23f4154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f23f40aa2d0 0x7f23f4154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40902/0 (socket says 172.21.15.39:40902) 2024-09-13T17:17:48.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f259c640 1 -- 172.21.15.39:0/395916462 learned_addr learned my addr 172.21.15.39:0/395916462 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:48.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f259c640 1 -- 172.21.15.39:0/395916462 >> v1:172.21.15.84:6789/0 conn(0x7f23f4155c20 legacy=0x7f23f4152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f259c640 1 -- 172.21.15.39:0/395916462 >> v1:172.21.15.39:6789/0 conn(0x7f23f40aaaf0 legacy=0x7f23f414b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f159a640 1 -- 172.21.15.39:0/395916462 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51293350 0 0) 0x7f23f4153110 con 0x7f23f40aaaf0 2024-09-13T17:17:48.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.929+0000 7f23f259c640 1 -- 172.21.15.39:0/395916462 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f40c4a60 con 0x7f23f40aa2d0 2024-09-13T17:17:48.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f259c640 1 --2- 172.21.15.39:0/395916462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aa2d0 0x7f23f4154700 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55bfd5e3f420429 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f159a640 1 -- 172.21.15.39:0/395916462 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e40aa620 con 0x7f23f40aa2d0 2024-09-13T17:17:48.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f159a640 1 -- 172.21.15.39:0/395916462 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e40aa780 con 0x7f23f40aa2d0 2024-09-13T17:17:48.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f159a640 1 -- 172.21.15.39:0/395916462 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e40aab10 con 0x7f23f40aa2d0 2024-09-13T17:17:48.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f8be8640 1 -- 172.21.15.39:0/395916462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aa2d0 msgr2=0x7f23f4154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f8be8640 1 --2- 172.21.15.39:0/395916462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aa2d0 0x7f23f4154700 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e401c4a0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f8be8640 1 -- 172.21.15.39:0/395916462 shutdown_connections 2024-09-13T17:17:48.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f8be8640 1 --2- 172.21.15.39:0/395916462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aa2d0 0x7f23f4154700 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f8be8640 1 -- 172.21.15.39:0/395916462 >> 172.21.15.39:0/395916462 conn(0x7f23f40c5790 msgr2=0x7f23f40c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:48.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f8be8640 1 -- 172.21.15.39:0/395916462 shutdown_connections 2024-09-13T17:17:48.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.933+0000 7f23f8be8640 1 -- 172.21.15.39:0/395916462 wait complete. 2024-09-13T17:17:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 Processor -- start 2024-09-13T17:17:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 -- start start 2024-09-13T17:17:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23f40aa2d0 0x7f23f41eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aaaf0 0x7f23f41ee420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23f4143c40 0x7f23f406f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f41ee960 con 0x7f23f40aaaf0 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f40714e0 con 0x7f23f40aa2d0 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4071ba0 con 0x7f23f4143c40 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aaaf0 0x7f23f41ee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f1d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aaaf0 0x7f23f41ee420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40918/0 (socket says 172.21.15.39:40918) 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f1d9b640 1 -- 172.21.15.39:0/2311169816 learned_addr learned my addr 172.21.15.39:0/2311169816 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f2d9d640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23f4143c40 0x7f23f406f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f259c640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23f40aa2d0 0x7f23f41eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f1d9b640 1 -- 172.21.15.39:0/2311169816 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23f4143c40 msgr2=0x7f23f406f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f1d9b640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23f4143c40 0x7f23f406f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f1d9b640 1 -- 172.21.15.39:0/2311169816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23f40aa2d0 msgr2=0x7f23f41eaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f1d9b640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23f40aa2d0 0x7f23f41eaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f1d9b640 1 -- 172.21.15.39:0/2311169816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e40094d0 con 0x7f23f40aaaf0 2024-09-13T17:17:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f1d9b640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aaaf0 0x7f23f41ee420 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f23e800c390 tx=0x7f23e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23e2ffd640 1 -- 172.21.15.39:0/2311169816 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e80052d0 con 0x7f23f40aaaf0 2024-09-13T17:17:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23e2ffd640 1 -- 172.21.15.39:0/2311169816 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e8005430 con 0x7f23f40aaaf0 2024-09-13T17:17:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23e2ffd640 1 -- 172.21.15.39:0/2311169816 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e8003820 con 0x7f23f40aaaf0 2024-09-13T17:17:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f4071e80 con 0x7f23f40aaaf0 2024-09-13T17:17:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f41ea7f0 con 0x7f23f40aaaf0 2024-09-13T17:17:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f40674c0 con 0x7f23f40aaaf0 2024-09-13T17:17:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23e2ffd640 1 -- 172.21.15.39:0/2311169816 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f23e8003980 con 0x7f23f40aaaf0 2024-09-13T17:17:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23e2ffd640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23d40666c0 0x7f23d4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23e2ffd640 1 -- 172.21.15.39:0/2311169816 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f23e80847f0 con 0x7f23f40aaaf0 2024-09-13T17:17:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.937+0000 7f23f259c640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23d40666c0 0x7f23d4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.941+0000 7f23f259c640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23d40666c0 0x7f23d4068b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ae000 tx=0x7f23e40b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:48.945+0000 7f23e2ffd640 1 -- 172.21.15.39:0/2311169816 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f23e8050a00 con 0x7f23f40aaaf0 2024-09-13T17:17:49.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.113+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f23f40aa900 con 0x7f23d40666c0 2024-09-13T17:17:49.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.113+0000 7f23e2ffd640 1 -- 172.21.15.39:0/2311169816 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f23e40aa970 con 0x7f23d40666c0 2024-09-13T17:17:49.117 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.113+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23d40666c0 msgr2=0x7f23d4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.113+0000 7f23f8be8640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23d40666c0 0x7f23d4068b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ae000 tx=0x7f23e40b5040 comp rx=0 tx=0).stop 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.113+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aaaf0 msgr2=0x7f23f41ee420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.113+0000 7f23f8be8640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aaaf0 0x7f23f41ee420 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f23e800c390 tx=0x7f23e800c850 comp rx=0 tx=0).stop 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.113+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 shutdown_connections 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.117+0000 7f23f8be8640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f23d40666c0 0x7f23d4068b80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.117+0000 7f23f8be8640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f23f4143c40 0x7f23f406f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.117+0000 7f23f8be8640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23f40aaaf0 0x7f23f41ee420 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.117+0000 7f23f8be8640 1 --2- 172.21.15.39:0/2311169816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23f40aa2d0 0x7f23f41eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.117+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 >> 172.21.15.39:0/2311169816 conn(0x7f23f40c5790 msgr2=0x7f23f4152070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.117+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 shutdown_connections 2024-09-13T17:17:49.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.117+0000 7f23f8be8640 1 -- 172.21.15.39:0/2311169816 wait complete. 2024-09-13T17:17:49.135 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T17:17:49.135 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:17:49.244 INFO:tasks.mds_thrash.fs.[a]:reviving mds.a 2024-09-13T17:17:49.244 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:17:49.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:17:49.247 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:17:49.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:49.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f453dc43640 1 Processor -- start 2024-09-13T17:17:49.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f453dc43640 1 -- start start 2024-09-13T17:17:49.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f453dc43640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f453814bf30 0x7f453806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f453dc43640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4538155160 0x7f45381520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f453dc43640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4538151bf0 con 0x7f453806b360 2024-09-13T17:17:49.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f453dc43640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4538155660 con 0x7f453814bf30 2024-09-13T17:17:49.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f453dc43640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4538155940 con 0x7f4538155160 2024-09-13T17:17:49.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f4536ffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4538155160 0x7f45381520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f45377fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f453814bf30 0x7f453806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f45377fe640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f453814bf30 0x7f453806ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:52766/0 (socket says 172.21.15.39:52766) 2024-09-13T17:17:49.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f45377fe640 1 -- 172.21.15.39:0/3663405274 learned_addr learned my addr 172.21.15.39:0/3663405274 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:49.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.249+0000 7f4537fff640 1 --1- 172.21.15.39:0/3663405274 >> v1:172.21.15.84:6789/0 conn(0x7f453806b360 0x7f4538152630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:60948/0 (socket says 172.21.15.39:60948) 2024-09-13T17:17:49.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f45377fe640 1 -- 172.21.15.39:0/3663405274 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4538155160 msgr2=0x7f45381520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:49.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f45377fe640 1 --2- 172.21.15.39:0/3663405274 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4538155160 0x7f45381520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f45377fe640 1 -- 172.21.15.39:0/3663405274 >> v1:172.21.15.84:6789/0 conn(0x7f453806b360 legacy=0x7f4538152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f45377fe640 1 -- 172.21.15.39:0/3663405274 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4538155c20 con 0x7f453814bf30 2024-09-13T17:17:49.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f45377fe640 1 --2- 172.21.15.39:0/3663405274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f453814bf30 0x7f453806ae20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f45280098a0 tx=0x7f452801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f39c5a416d7b754 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f45367fc640 1 -- 172.21.15.39:0/3663405274 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45280aa650 con 0x7f453814bf30 2024-09-13T17:17:49.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f45367fc640 1 -- 172.21.15.39:0/3663405274 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45280aa7b0 con 0x7f453814bf30 2024-09-13T17:17:49.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f45367fc640 1 -- 172.21.15.39:0/3663405274 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45280aab40 con 0x7f453814bf30 2024-09-13T17:17:49.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f453dc43640 1 -- 172.21.15.39:0/3663405274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f453814bf30 msgr2=0x7f453806ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f453dc43640 1 --2- 172.21.15.39:0/3663405274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f453814bf30 0x7f453806ae20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f45280098a0 tx=0x7f452801c510 comp rx=0 tx=0).stop 2024-09-13T17:17:49.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f453dc43640 1 -- 172.21.15.39:0/3663405274 shutdown_connections 2024-09-13T17:17:49.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f453dc43640 1 --2- 172.21.15.39:0/3663405274 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4538155160 0x7f45381520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f453dc43640 1 --2- 172.21.15.39:0/3663405274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f453814bf30 0x7f453806ae20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f453dc43640 1 -- 172.21.15.39:0/3663405274 >> 172.21.15.39:0/3663405274 conn(0x7f45380c32a0 msgr2=0x7f45380c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f453dc43640 1 -- 172.21.15.39:0/3663405274 shutdown_connections 2024-09-13T17:17:49.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.253+0000 7f453dc43640 1 -- 172.21.15.39:0/3663405274 wait complete. 2024-09-13T17:17:49.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 Processor -- start 2024-09-13T17:17:49.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 -- start start 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f453806b360 0x7f45381424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f453814bf30 0x7f4538147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4538155160 0x7f45381484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453814b270 con 0x7f4538155160 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453814b7a0 con 0x7f453814bf30 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453814a180 con 0x7f453806b360 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4537fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4538155160 0x7f45381484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4537fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4538155160 0x7f45381484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40930/0 (socket says 172.21.15.39:40930) 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4537fff640 1 -- 172.21.15.39:0/2796171525 learned_addr learned my addr 172.21.15.39:0/2796171525 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4536ffd640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f453814bf30 0x7f4538147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f45377fe640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f453806b360 0x7f45381424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4537fff640 1 -- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f453806b360 msgr2=0x7f45381424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4537fff640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f453806b360 0x7f45381424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4537fff640 1 -- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f453814bf30 msgr2=0x7f4538147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4537fff640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f453814bf30 0x7f4538147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4537fff640 1 -- 172.21.15.39:0/2796171525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280094d0 con 0x7f4538155160 2024-09-13T17:17:49.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f4537fff640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4538155160 0x7f45381484e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f453000c730 tx=0x7f453000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f451ffff640 1 -- 172.21.15.39:0/2796171525 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530004d70 con 0x7f4538155160 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f451ffff640 1 -- 172.21.15.39:0/2796171525 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453000a640 con 0x7f4538155160 2024-09-13T17:17:49.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453814b4c0 con 0x7f4538155160 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f451ffff640 1 -- 172.21.15.39:0/2796171525 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000a9d0 con 0x7f4538155160 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45381432e0 con 0x7f4538155160 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.257+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f453806e0c0 con 0x7f4538155160 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f451ffff640 1 -- 172.21.15.39:0/2796171525 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f453000ab30 con 0x7f4538155160 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f451ffff640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f45240666c0 0x7f4524068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f451ffff640 1 -- 172.21.15.39:0/2796171525 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f4530084830 con 0x7f4538155160 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f45377fe640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f45240666c0 0x7f4524068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f451ffff640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7f45240730a0 0x7f4524075520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f451ffff640 1 -- 172.21.15.39:0/2796171525 --> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4524075bf0 con 0x7f45240730a0 2024-09-13T17:17:49.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f451ffff640 1 -- 172.21.15.39:0/2796171525 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45300892a0 con 0x7f4538155160 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f4536ffd640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7f45240730a0 0x7f4524075520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f45377fe640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f45240666c0 0x7f4524068b80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f45280ae000 tx=0x7f45280a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.261+0000 7f4536ffd640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7f45240730a0 0x7f4524075520 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.265+0000 7f451ffff640 1 -- 172.21.15.39:0/2796171525 <== osd.7 v2:172.21.15.84:6824/413977365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4524075bf0 con 0x7f45240730a0 2024-09-13T17:17:49.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 --> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f453815ecb0 con 0x7f45240730a0 2024-09-13T17:17:49.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f451ffff640 1 -- 172.21.15.39:0/2796171525 <== osd.7 v2:172.21.15.84:6824/413977365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f452c009820 con 0x7f45240730a0 2024-09-13T17:17:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-13T17:17:49.280 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:17:49.280 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-13T17:17:49.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7f45240730a0 msgr2=0x7f4524075520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.84:6824/413977365,v1:172.21.15.84:6825/413977365] conn(0x7f45240730a0 0x7f4524075520 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f45240666c0 msgr2=0x7f4524068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f45240666c0 0x7f4524068b80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f45280ae000 tx=0x7f45280a7000 comp rx=0 tx=0).stop 2024-09-13T17:17:49.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4538155160 msgr2=0x7f45381484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 --2- 172.21.15.39:0/2796171525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4538155160 0x7f45381484e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f453000c730 tx=0x7f453000cbf0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f4537fff640 1 -- 172.21.15.39:0/2796171525 reap_dead start 2024-09-13T17:17:49.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 shutdown_connections 2024-09-13T17:17:49.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 >> 172.21.15.39:0/2796171525 conn(0x7f45380c32a0 msgr2=0x7f45381520f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 shutdown_connections 2024-09-13T17:17:49.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.277+0000 7f453dc43640 1 -- 172.21.15.39:0/2796171525 wait complete. 2024-09-13T17:17:49.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T17:17:49.309 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-13T17:17:49.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f1265f4e640 1 Processor -- start 2024-09-13T17:17:49.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f1265f4e640 1 -- start start 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f1265f4e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126014f1b0 0x7f1260151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f1265f4e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f126006e5e0 0x7f12600714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f1265f4e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126014f590 con 0x7f12600c7470 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f1265f4e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126006f660 con 0x7f126006e5e0 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f1265f4e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126006eae0 con 0x7f126014f1b0 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f125effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f126006e5e0 0x7f12600714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f125effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f126006e5e0 0x7f12600714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40954/0 (socket says 172.21.15.39:40954) 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f125effd640 1 -- 172.21.15.39:0/4211805738 learned_addr learned my addr 172.21.15.39:0/4211805738 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f125effd640 1 -- 172.21.15.39:0/4211805738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126014f1b0 msgr2=0x7f1260151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f125f7fe640 1 --2- 172.21.15.39:0/4211805738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126014f1b0 0x7f1260151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f125effd640 1 --2- 172.21.15.39:0/4211805738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126014f1b0 0x7f1260151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f125e7fc640 1 -- 172.21.15.39:0/4211805738 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493201615 0 0) 0x7f126014f590 con 0x7f12600c7470 2024-09-13T17:17:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f125effd640 1 -- 172.21.15.39:0/4211805738 >> v1:172.21.15.39:6789/0 conn(0x7f12600c7470 legacy=0x7f12600c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.353+0000 7f125effd640 1 -- 172.21.15.39:0/4211805738 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126006edc0 con 0x7f126006e5e0 2024-09-13T17:17:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f125effd640 1 --2- 172.21.15.39:0/4211805738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f126006e5e0 0x7f12600714e0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f1250009870 tx=0x7f125001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcea9cd9aa17b3fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f125e7fc640 1 -- 172.21.15.39:0/4211805738 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12500a95f0 con 0x7f126006e5e0 2024-09-13T17:17:49.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f125e7fc640 1 -- 172.21.15.39:0/4211805738 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12500a9750 con 0x7f126006e5e0 2024-09-13T17:17:49.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f125e7fc640 1 -- 172.21.15.39:0/4211805738 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12500a9ae0 con 0x7f126006e5e0 2024-09-13T17:17:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 -- 172.21.15.39:0/4211805738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f126006e5e0 msgr2=0x7f12600714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 --2- 172.21.15.39:0/4211805738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f126006e5e0 0x7f12600714e0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f1250009870 tx=0x7f125001c530 comp rx=0 tx=0).stop 2024-09-13T17:17:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 -- 172.21.15.39:0/4211805738 shutdown_connections 2024-09-13T17:17:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 --2- 172.21.15.39:0/4211805738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f126006e5e0 0x7f12600714e0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 --2- 172.21.15.39:0/4211805738 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126014f1b0 0x7f1260151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 -- 172.21.15.39:0/4211805738 >> 172.21.15.39:0/4211805738 conn(0x7f12600a9720 msgr2=0x7f12600a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 -- 172.21.15.39:0/4211805738 shutdown_connections 2024-09-13T17:17:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 -- 172.21.15.39:0/4211805738 wait complete. 2024-09-13T17:17:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 Processor -- start 2024-09-13T17:17:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 -- start start 2024-09-13T17:17:49.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126006e5e0 0x7f12600c1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.357+0000 7f1265f4e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12600c7470 0x7f126014bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1265f4e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f126014f1b0 0x7f126014c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1265f4e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126015c1c0 con 0x7f12600c7470 2024-09-13T17:17:49.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1265f4e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126014c5e0 con 0x7f126014f1b0 2024-09-13T17:17:49.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1265f4e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12600ada10 con 0x7f126006e5e0 2024-09-13T17:17:49.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12600c7470 0x7f126014bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12600c7470 0x7f126014bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40966/0 (socket says 172.21.15.39:40966) 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125effd640 1 -- 172.21.15.39:0/1421544868 learned_addr learned my addr 172.21.15.39:0/1421544868 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125ffff640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f126014f1b0 0x7f126014c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125f7fe640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126006e5e0 0x7f12600c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125effd640 1 -- 172.21.15.39:0/1421544868 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126006e5e0 msgr2=0x7f12600c1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125effd640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126006e5e0 0x7f12600c1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125effd640 1 -- 172.21.15.39:0/1421544868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f126014f1b0 msgr2=0x7f126014c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125effd640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f126014f1b0 0x7f126014c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125effd640 1 -- 172.21.15.39:0/1421544868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12500094d0 con 0x7f12600c7470 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125effd640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12600c7470 0x7f126014bb60 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f125001abe0 tx=0x7f125001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1247fff640 1 -- 172.21.15.39:0/1421544868 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12500b1800 con 0x7f12600c7470 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1247fff640 1 -- 172.21.15.39:0/1421544868 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12500b1980 con 0x7f12600c7470 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126015bf20 con 0x7f12600c7470 2024-09-13T17:17:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1247fff640 1 -- 172.21.15.39:0/1421544868 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12500b1d30 con 0x7f12600c7470 2024-09-13T17:17:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12600ab630 con 0x7f12600c7470 2024-09-13T17:17:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12600714e0 con 0x7f12600c7470 2024-09-13T17:17:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1247fff640 1 -- 172.21.15.39:0/1421544868 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f12500c53a0 con 0x7f12600c7470 2024-09-13T17:17:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1247fff640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f124c0666c0 0x7f124c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f1247fff640 1 -- 172.21.15.39:0/1421544868 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f1250120030 con 0x7f12600c7470 2024-09-13T17:17:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125f7fe640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f124c0666c0 0x7f124c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.361+0000 7f125f7fe640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f124c0666c0 0x7f124c068b80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f125400b7c0 tx=0x7f1254008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.365+0000 7f1247fff640 1 -- 172.21.15.39:0/1421544868 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f12500ec2d0 con 0x7f12600c7470 2024-09-13T17:17:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d9bc4640 1 Processor -- start 2024-09-13T17:17:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d9bc4640 1 -- start start 2024-09-13T17:17:49.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d9bc4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d414f1b0 0x7f57d4151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d9bc4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d414f590 con 0x7f57d406e5e0 2024-09-13T17:17:49.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d9bc4640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d406eae0 con 0x7f57d40c7470 2024-09-13T17:17:49.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d9bc4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d406f660 con 0x7f57d414f1b0 2024-09-13T17:17:49.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d2ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f57d406e5e0 0x7f57d40714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40352/0 (socket says 172.21.15.39:40352) 2024-09-13T17:17:49.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d2ffd640 1 -- 172.21.15.39:0/344846594 learned_addr learned my addr 172.21.15.39:0/344846594 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:49.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d37fe640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d414f1b0 0x7f57d4151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247992484 0 0) 0x7f57d414f590 con 0x7f57d406e5e0 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57c0003530 con 0x7f57d406e5e0 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903637789 0 0) 0x7f57c0003530 con 0x7f57d406e5e0 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d414f1b0 msgr2=0x7f57d4151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d414f1b0 0x7f57d4151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 >> v1:172.21.15.84:6789/0 conn(0x7f57d40c7470 legacy=0x7f57d40c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d406edc0 con 0x7f57d406e5e0 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f57bc001680 con 0x7f57d406e5e0 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 >> v1:172.21.15.39:6789/0 conn(0x7f57d406e5e0 legacy=0x7f57d40714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57c00047d0 0x7f57c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d414f1b0 0x7f57c0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d406f660 con 0x7f57c00039b0 2024-09-13T17:17:49.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c000ba40 con 0x7f57c00047d0 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.413+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c000bc00 con 0x7f57d414f1b0 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d37fe640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d2ffd640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57c00047d0 0x7f57c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d3fff640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d414f1b0 0x7f57c0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d37fe640 1 -- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d414f1b0 msgr2=0x7f57c0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d37fe640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d414f1b0 0x7f57c0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d37fe640 1 -- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57c00047d0 msgr2=0x7f57c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d37fe640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57c00047d0 0x7f57c0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d37fe640 1 -- 172.21.15.39:0/344846594 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c0003530 con 0x7f57c00039b0 2024-09-13T17:17:49.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d3fff640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d414f1b0 0x7f57c0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:49.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d37fe640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57c0003dd0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009ab0 tx=0x7f57cc01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2613bddb1c160df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0a64f0 con 0x7f57c00039b0 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc0a6650 con 0x7f57c00039b0 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d27fc640 1 -- 172.21.15.39:0/344846594 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0a6a00 con 0x7f57c00039b0 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d9bc4640 1 -- 172.21.15.39:0/344846594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 msgr2=0x7f57c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d9bc4640 1 --2- 172.21.15.39:0/344846594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57c0003dd0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009ab0 tx=0x7f57cc01ca00 comp rx=0 tx=0).stop 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d3fff640 1 -- 172.21.15.39:0/344846594 reap_dead start 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d9bc4640 1 -- 172.21.15.39:0/344846594 shutdown_connections 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d9bc4640 1 -- 172.21.15.39:0/344846594 >> 172.21.15.39:0/344846594 conn(0x7f57d40a9720 msgr2=0x7f57d40a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d9bc4640 1 -- 172.21.15.39:0/344846594 shutdown_connections 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.417+0000 7f57d9bc4640 1 -- 172.21.15.39:0/344846594 wait complete. 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 Processor -- start 2024-09-13T17:17:49.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 -- start start 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57d40ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d406bab0 0x7f57d4155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d40ab220 0x7f57d41efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d406b4d0 con 0x7f57c00039b0 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40ad080 con 0x7f57d40ab220 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40aca70 con 0x7f57d406bab0 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57d40ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57d40ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40988/0 (socket says 172.21.15.39:40988) 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d37fe640 1 -- 172.21.15.39:0/4223632006 learned_addr learned my addr 172.21.15.39:0/4223632006 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d2ffd640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d406bab0 0x7f57d4155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d3fff640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d40ab220 0x7f57d41efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d37fe640 1 -- 172.21.15.39:0/4223632006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d406bab0 msgr2=0x7f57d4155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d37fe640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d406bab0 0x7f57d4155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d37fe640 1 -- 172.21.15.39:0/4223632006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d40ab220 msgr2=0x7f57d41efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d37fe640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d40ab220 0x7f57d41efb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d37fe640 1 -- 172.21.15.39:0/4223632006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d406b6e0 con 0x7f57c00039b0 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d3fff640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d40ab220 0x7f57d41efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d37fe640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57d40ad510 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f57cc01c830 tx=0x7f57c0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57cbfff640 1 -- 172.21.15.39:0/4223632006 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0a69c0 con 0x7f57c00039b0 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57cbfff640 1 -- 172.21.15.39:0/4223632006 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc0a6b40 con 0x7f57c00039b0 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d40acd10 con 0x7f57c00039b0 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57cbfff640 1 -- 172.21.15.39:0/4223632006 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00f0c0 con 0x7f57c00039b0 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d4155eb0 con 0x7f57c00039b0 2024-09-13T17:17:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d40714e0 con 0x7f57c00039b0 2024-09-13T17:17:49.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57cbfff640 1 -- 172.21.15.39:0/4223632006 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57cc006040 con 0x7f57c00039b0 2024-09-13T17:17:49.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57cbfff640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57b4066690 0x7f57b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57cbfff640 1 -- 172.21.15.39:0/4223632006 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f57cc0aa070 con 0x7f57c00039b0 2024-09-13T17:17:49.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.421+0000 7f57d2ffd640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57b4066690 0x7f57b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.425+0000 7f57d2ffd640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57b4066690 0x7f57b4068b50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f57d41ea450 tx=0x7f57bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.425+0000 7f57cbfff640 1 -- 172.21.15.39:0/4223632006 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f57cc0ece90 con 0x7f57c00039b0 2024-09-13T17:17:49.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.565+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12601efba0 con 0x7f12600c7470 2024-09-13T17:17:49.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.565+0000 7f1247fff640 1 -- 172.21.15.39:0/1421544868 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+5182 (secure 0 0 0) 0x7f12500c1090 con 0x7f12600c7470 2024-09-13T17:17:49.569 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:49.569 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":206,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:17:47.987831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7555":{"gid":7555,"name":"d","rank":0,"incarnation":203,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:49.570 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 206 2024-09-13T17:17:49.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f124c0666c0 msgr2=0x7f124c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f124c0666c0 0x7f124c068b80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f125400b7c0 tx=0x7f1254008040 comp rx=0 tx=0).stop 2024-09-13T17:17:49.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12600c7470 msgr2=0x7f126014bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12600c7470 0x7f126014bb60 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f125001abe0 tx=0x7f125001ad70 comp rx=0 tx=0).stop 2024-09-13T17:17:49.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 shutdown_connections 2024-09-13T17:17:49.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f124c0666c0 0x7f124c068b80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f126014f1b0 0x7f126014c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12600c7470 0x7f126014bb60 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 --2- 172.21.15.39:0/1421544868 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f126006e5e0 0x7f12600c1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 >> 172.21.15.39:0/1421544868 conn(0x7f12600a9720 msgr2=0x7f1260071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 shutdown_connections 2024-09-13T17:17:49.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.569+0000 7f1265f4e640 1 -- 172.21.15.39:0/1421544868 wait complete. 2024-09-13T17:17:49.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f57d406b920 con 0x7f57b4066690 2024-09-13T17:17:49.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57cbfff640 1 -- 172.21.15.39:0/4223632006 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f57d406b920 con 0x7f57b4066690 2024-09-13T17:17:49.587 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T17:17:49.588 INFO:tasks.mds_thrash.fs.[a]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-09-13T17:17:49.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57b4066690 msgr2=0x7f57b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57b4066690 0x7f57b4068b50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f57d41ea450 tx=0x7f57bc009040 comp rx=0 tx=0).stop 2024-09-13T17:17:49.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 msgr2=0x7f57d40ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57d40ad510 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f57cc01c830 tx=0x7f57c0005800 comp rx=0 tx=0).stop 2024-09-13T17:17:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 shutdown_connections 2024-09-13T17:17:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f57b4066690 0x7f57b4068b50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57d40ab220 0x7f57d41efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f57d406bab0 0x7f57d4155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 --2- 172.21.15.39:0/4223632006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57c00039b0 0x7f57d40ad510 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 >> 172.21.15.39:0/4223632006 conn(0x7f57d40a9720 msgr2=0x7f57d40a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 shutdown_connections 2024-09-13T17:17:49.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.585+0000 7f57d9bc4640 1 -- 172.21.15.39:0/4223632006 wait complete. 2024-09-13T17:17:49.604 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:17:49.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d8e35c640 1 Processor -- start 2024-09-13T17:17:49.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d8e35c640 1 -- start start 2024-09-13T17:17:49.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d8e35c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9d88154bb0 0x7f9d880c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d8e35c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d88153e30 con 0x7f9d880c2370 2024-09-13T17:17:49.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d8e35c640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d880a7000 con 0x7f9d880c5870 2024-09-13T17:17:49.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d8e35c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d880720f0 con 0x7f9d88154bb0 2024-09-13T17:17:49.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d87fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9d880c2370 0x7f9d880c0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:40366/0 (socket says 172.21.15.39:40366) 2024-09-13T17:17:49.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d87fff640 1 -- 172.21.15.39:0/2350594090 learned_addr learned my addr 172.21.15.39:0/2350594090 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:49.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d877fe640 1 --2- 172.21.15.39:0/2350594090 >> v2:172.21.15.39:3300/0 conn(0x7f9d88154bb0 0x7f9d880c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d86ffd640 1 -- 172.21.15.39:0/2350594090 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363403288 0 0) 0x7f9d88153e30 con 0x7f9d880c2370 2024-09-13T17:17:49.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d86ffd640 1 -- 172.21.15.39:0/2350594090 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d70003530 con 0x7f9d880c2370 2024-09-13T17:17:49.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d877fe640 1 -- 172.21.15.39:0/2350594090 >> v1:172.21.15.84:6789/0 conn(0x7f9d880c5870 legacy=0x7f9d8815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:49.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d877fe640 1 -- 172.21.15.39:0/2350594090 >> v1:172.21.15.39:6789/0 conn(0x7f9d880c2370 legacy=0x7f9d880c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.765+0000 7f9d877fe640 1 -- 172.21.15.39:0/2350594090 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d881540d0 con 0x7f9d88154bb0 2024-09-13T17:17:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 --2- 172.21.15.39:0/2350594090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d88154bb0 0x7f9d880c2750 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0097c0 tx=0x7f9d7c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e61cab6f300f0c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d86ffd640 1 -- 172.21.15.39:0/2350594090 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0ab600 con 0x7f9d88154bb0 2024-09-13T17:17:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d86ffd640 1 -- 172.21.15.39:0/2350594090 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c0a7050 con 0x7f9d88154bb0 2024-09-13T17:17:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d86ffd640 1 -- 172.21.15.39:0/2350594090 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0ab980 con 0x7f9d88154bb0 2024-09-13T17:17:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- 172.21.15.39:0/2350594090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d88154bb0 msgr2=0x7f9d880c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 --2- 172.21.15.39:0/2350594090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d88154bb0 0x7f9d880c2750 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0097c0 tx=0x7f9d7c01c3d0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- 172.21.15.39:0/2350594090 shutdown_connections 2024-09-13T17:17:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 --2- 172.21.15.39:0/2350594090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d88154bb0 0x7f9d880c2750 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- 172.21.15.39:0/2350594090 >> 172.21.15.39:0/2350594090 conn(0x7f9d880a9720 msgr2=0x7f9d880a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- 172.21.15.39:0/2350594090 shutdown_connections 2024-09-13T17:17:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- 172.21.15.39:0/2350594090 wait complete. 2024-09-13T17:17:49.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 Processor -- start 2024-09-13T17:17:49.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- start start 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d880c2370 0x7f9d880c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d880c5870 0x7f9d88145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d88154bb0 0x7f9d88146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d881446d0 con 0x7f9d880c5870 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d881440a0 con 0x7f9d880c2370 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d88146940 con 0x7f9d88154bb0 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d880c5870 0x7f9d88145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d880c5870 0x7f9d88145ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41010/0 (socket says 172.21.15.39:41010) 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 -- 172.21.15.39:0/312923644 learned_addr learned my addr 172.21.15.39:0/312923644 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d87fff640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d880c2370 0x7f9d880c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 -- 172.21.15.39:0/312923644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d88154bb0 msgr2=0x7f9d88146400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8c8f9640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d88154bb0 0x7f9d88146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d88154bb0 0x7f9d88146400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 -- 172.21.15.39:0/312923644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d880c2370 msgr2=0x7f9d880c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d880c2370 0x7f9d880c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 -- 172.21.15.39:0/312923644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c009420 con 0x7f9d880c5870 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d87fff640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d880c2370 0x7f9d880c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d877fe640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d880c5870 0x7f9d88145ec0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0097c0 tx=0x7f9d7c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d84ff9640 1 -- 172.21.15.39:0/312923644 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0b4430 con 0x7f9d880c5870 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d84ff9640 1 -- 172.21.15.39:0/312923644 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c0b4590 con 0x7f9d880c5870 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8814a4d0 con 0x7f9d880c5870 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d84ff9640 1 -- 172.21.15.39:0/312923644 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0b4940 con 0x7f9d880c5870 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.769+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d881442f0 con 0x7f9d880c5870 2024-09-13T17:17:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.773+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d880c2750 con 0x7f9d880c5870 2024-09-13T17:17:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.773+0000 7f9d84ff9640 1 -- 172.21.15.39:0/312923644 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d7c0b2040 con 0x7f9d880c5870 2024-09-13T17:17:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.773+0000 7f9d84ff9640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d740666c0 0x7f9d74068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.773+0000 7f9d84ff9640 1 -- 172.21.15.39:0/312923644 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f9d7c11ef10 con 0x7f9d880c5870 2024-09-13T17:17:49.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.773+0000 7f9d87fff640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d740666c0 0x7f9d74068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:49.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.773+0000 7f9d87fff640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d740666c0 0x7f9d74068b80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9d78001570 tx=0x7f9d78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:49.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.777+0000 7f9d84ff9640 1 -- 172.21.15.39:0/312923644 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9d7c0eb1b0 con 0x7f9d880c5870 2024-09-13T17:17:49.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.929+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9d8806bb20 con 0x7f9d740666c0 2024-09-13T17:17:49.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.969+0000 7f9d84ff9640 1 -- 172.21.15.39:0/312923644 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+527960 (secure 0 0 0) 0x7f9d78001ef0 con 0x7f9d740666c0 2024-09-13T17:17:49.977 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:49.981 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T17:17:49.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d740666c0 msgr2=0x7f9d74068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d740666c0 0x7f9d74068b80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9d78001570 tx=0x7f9d78009040 comp rx=0 tx=0).stop 2024-09-13T17:17:49.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d880c5870 msgr2=0x7f9d88145ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:49.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d880c5870 0x7f9d88145ec0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0097c0 tx=0x7f9d7c019170 comp rx=0 tx=0).stop 2024-09-13T17:17:49.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 shutdown_connections 2024-09-13T17:17:49.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9d740666c0 0x7f9d74068b80 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9d88154bb0 0x7f9d88146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d880c5870 0x7f9d88145ec0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 --2- 172.21.15.39:0/312923644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d880c2370 0x7f9d880c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:49.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 >> 172.21.15.39:0/312923644 conn(0x7f9d880a9720 msgr2=0x7f9d881498e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:49.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 shutdown_connections 2024-09-13T17:17:49.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:49.973+0000 7f9d8e35c640 1 -- 172.21.15.39:0/312923644 wait complete. 2024-09-13T17:17:49.994 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":957,"stamp":"2024-09-13T17:17:48.348074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39268919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":3109,"num_read_kb":408429,"num_write":2818765,"num_write_kb":37531880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522493,"ondisk_log_size":522493,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6804360,"kb_used_data":83256,"kb_used_omap":12,"kb_used_meta":6721011,"kb_avail":748170360,"statfs":{"total":773094113280,"available":766126448640,"internally_reserved":0,"allocated":85254144,"data_stored":82040288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":6882315835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":530,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":10938,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004016"},"pg_stats":[{"pgid":"5.18","version":"63'4385","reported_seq":4447,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058432+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T17:17:14.058432+0000","last_peered":"2024-09-13T17:17:14.058432+0000","last_clean":"2024-09-13T17:17:14.058432+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T17:17:14.058432+0000","last_undegraded":"2024-09-13T17:17:14.058432+0000","last_fullsized":"2024-09-13T17:17:14.058432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":23692,"num_write_kb":324330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"63'916","reported_seq":1025,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096259+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T17:17:22.096259+0000","last_peered":"2024-09-13T17:17:22.096259+0000","last_clean":"2024-09-13T17:17:22.096259+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T17:17:22.096259+0000","last_undegraded":"2024-09-13T17:17:22.096259+0000","last_fullsized":"2024-09-13T17:17:22.096259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:05:32.598608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":6289,"num_write":894,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'3737","reported_seq":3805,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.101092+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T17:17:22.101092+0000","last_peered":"2024-09-13T17:17:22.101092+0000","last_clean":"2024-09-13T17:17:22.101092+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T17:17:22.101092+0000","last_undegraded":"2024-09-13T17:17:22.101092+0000","last_fullsized":"2024-09-13T17:17:22.101092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3737,"log_dups_size":0,"ondisk_log_size":3737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20454,"num_write_kb":270405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"63'3","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.026123+0000","last_change":"2024-09-13T16:48:07.776331+0000","last_active":"2024-09-13T17:17:14.026123+0000","last_peered":"2024-09-13T17:17:14.026123+0000","last_clean":"2024-09-13T17:17:14.026123+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T17:17:14.026123+0000","last_undegraded":"2024-09-13T17:17:14.026123+0000","last_fullsized":"2024-09-13T17:17:14.026123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:04.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"63'4331","reported_seq":4394,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.043747+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T17:17:14.043747+0000","last_peered":"2024-09-13T17:17:14.043747+0000","last_clean":"2024-09-13T17:17:14.043747+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T17:17:14.043747+0000","last_undegraded":"2024-09-13T17:17:14.043747+0000","last_fullsized":"2024-09-13T17:17:14.043747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":22875,"num_write_kb":325320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"64'24","reported_seq":93,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.977005+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T17:17:44.977005+0000","last_peered":"2024-09-13T17:17:44.977005+0000","last_clean":"2024-09-13T17:17:44.977005+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T17:17:44.977005+0000","last_undegraded":"2024-09-13T17:17:44.977005+0000","last_fullsized":"2024-09-13T17:17:44.977005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:29.550611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"63'5559","reported_seq":5626,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.132606+0000","last_change":"2024-09-13T16:47:59.055557+0000","last_active":"2024-09-13T17:17:22.132606+0000","last_peered":"2024-09-13T17:17:22.132606+0000","last_clean":"2024-09-13T17:17:22.132606+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T17:17:22.132606+0000","last_undegraded":"2024-09-13T17:17:22.132606+0000","last_fullsized":"2024-09-13T17:17:22.132606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5559,"log_dups_size":0,"ondisk_log_size":5559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30370,"num_write_kb":379865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"63'2","reported_seq":64,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:09.573144+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T17:17:09.573144+0000","last_peered":"2024-09-13T17:17:09.573144+0000","last_clean":"2024-09-13T17:17:09.573144+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T17:17:09.573144+0000","last_undegraded":"2024-09-13T17:17:09.573144+0000","last_fullsized":"2024-09-13T17:17:09.573144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"63'3581","reported_seq":3641,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.986895+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T17:17:13.986895+0000","last_peered":"2024-09-13T17:17:13.986895+0000","last_clean":"2024-09-13T17:17:13.986895+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T17:17:13.986895+0000","last_undegraded":"2024-09-13T17:17:13.986895+0000","last_fullsized":"2024-09-13T17:17:13.986895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":268344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"63'40","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.099149+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T17:17:22.099149+0000","last_peered":"2024-09-13T17:17:22.099149+0000","last_clean":"2024-09-13T17:17:22.099149+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T17:17:22.099149+0000","last_undegraded":"2024-09-13T17:17:22.099149+0000","last_fullsized":"2024-09-13T17:17:22.099149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'3493","reported_seq":3560,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.086084+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T17:17:22.086084+0000","last_peered":"2024-09-13T17:17:22.086084+0000","last_clean":"2024-09-13T17:17:22.086084+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T17:17:22.086084+0000","last_undegraded":"2024-09-13T17:17:22.086084+0000","last_fullsized":"2024-09-13T17:17:22.086084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18921,"num_write_kb":256078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"63'726","reported_seq":812,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.996155+0000","last_change":"2024-09-13T16:48:07.784187+0000","last_active":"2024-09-13T17:17:13.996155+0000","last_peered":"2024-09-13T17:17:13.996155+0000","last_clean":"2024-09-13T17:17:13.996155+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T17:17:13.996155+0000","last_undegraded":"2024-09-13T17:17:13.996155+0000","last_fullsized":"2024-09-13T17:17:13.996155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:08.036966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":75389,"num_write":714,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"63'3135","reported_seq":3196,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036153+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T17:17:14.036153+0000","last_peered":"2024-09-13T17:17:14.036153+0000","last_clean":"2024-09-13T17:17:14.036153+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T17:17:14.036153+0000","last_undegraded":"2024-09-13T17:17:14.036153+0000","last_fullsized":"2024-09-13T17:17:14.036153+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3135,"log_dups_size":0,"ondisk_log_size":3135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":232447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"63'5","reported_seq":106,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:16:38.283375+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T17:16:38.283375+0000","last_peered":"2024-09-13T17:16:38.283375+0000","last_clean":"2024-09-13T17:16:38.283375+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T17:16:38.283375+0000","last_undegraded":"2024-09-13T17:16:38.283375+0000","last_fullsized":"2024-09-13T17:16:38.283375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"63'5195","reported_seq":5260,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.134068+0000","last_change":"2024-09-13T16:47:59.045602+0000","last_active":"2024-09-13T17:17:22.134068+0000","last_peered":"2024-09-13T17:17:22.134068+0000","last_clean":"2024-09-13T17:17:22.134068+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T17:17:22.134068+0000","last_undegraded":"2024-09-13T17:17:22.134068+0000","last_fullsized":"2024-09-13T17:17:22.134068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29369,"num_write_kb":359470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"63'210","reported_seq":271,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:27.922744+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T17:17:27.922744+0000","last_peered":"2024-09-13T17:17:27.922744+0000","last_clean":"2024-09-13T17:17:27.922744+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T17:17:27.922744+0000","last_undegraded":"2024-09-13T17:17:27.922744+0000","last_fullsized":"2024-09-13T17:17:27.922744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:58.009900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153565,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"63'3570","reported_seq":3634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.050021+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T17:17:14.050021+0000","last_peered":"2024-09-13T17:17:14.050021+0000","last_clean":"2024-09-13T17:17:14.050021+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T17:17:14.050021+0000","last_undegraded":"2024-09-13T17:17:14.050021+0000","last_fullsized":"2024-09-13T17:17:14.050021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":19362,"num_write_kb":282506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"63'4274","reported_seq":4341,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.124505+0000","last_change":"2024-09-13T16:47:59.051577+0000","last_active":"2024-09-13T17:17:22.124505+0000","last_peered":"2024-09-13T17:17:22.124505+0000","last_clean":"2024-09-13T17:17:22.124505+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T17:17:22.124505+0000","last_undegraded":"2024-09-13T17:17:22.124505+0000","last_fullsized":"2024-09-13T17:17:22.124505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23751,"num_write_kb":312338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"63'1418","reported_seq":1528,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.108614+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T17:17:22.108614+0000","last_peered":"2024-09-13T17:17:22.108614+0000","last_clean":"2024-09-13T17:17:22.108614+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T17:17:22.108614+0000","last_undegraded":"2024-09-13T17:17:22.108614+0000","last_fullsized":"2024-09-13T17:17:22.108614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1418,"log_dups_size":0,"ondisk_log_size":1418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":38147,"num_write":1396,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"63'6","reported_seq":90,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.030550+0000","last_change":"2024-09-13T16:48:07.776824+0000","last_active":"2024-09-13T17:17:14.030550+0000","last_peered":"2024-09-13T17:17:14.030550+0000","last_clean":"2024-09-13T17:17:14.030550+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T17:17:14.030550+0000","last_undegraded":"2024-09-13T17:17:14.030550+0000","last_fullsized":"2024-09-13T17:17:14.030550+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:28.320227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"63'4212","reported_seq":4279,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007878+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T17:17:45.007878+0000","last_peered":"2024-09-13T17:17:45.007878+0000","last_clean":"2024-09-13T17:17:45.007878+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T17:17:45.007878+0000","last_undegraded":"2024-09-13T17:17:45.007878+0000","last_fullsized":"2024-09-13T17:17:45.007878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":17,"num_write":23343,"num_write_kb":314959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"63'3296","reported_seq":3363,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126064+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T17:17:22.126064+0000","last_peered":"2024-09-13T17:17:22.126064+0000","last_clean":"2024-09-13T17:17:22.126064+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T17:17:22.126064+0000","last_undegraded":"2024-09-13T17:17:22.126064+0000","last_fullsized":"2024-09-13T17:17:22.126064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3296,"log_dups_size":0,"ondisk_log_size":3296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17758,"num_write_kb":230990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'3","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.059668+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T17:17:22.059668+0000","last_peered":"2024-09-13T17:17:22.059668+0000","last_clean":"2024-09-13T17:17:22.059668+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T17:17:22.059668+0000","last_undegraded":"2024-09-13T17:17:22.059668+0000","last_fullsized":"2024-09-13T17:17:22.059668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'5","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.027550+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T17:17:14.027550+0000","last_peered":"2024-09-13T17:17:14.027550+0000","last_clean":"2024-09-13T17:17:14.027550+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T17:17:14.027550+0000","last_undegraded":"2024-09-13T17:17:14.027550+0000","last_fullsized":"2024-09-13T17:17:14.027550+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:28.295676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"63'3371","reported_seq":3434,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022230+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T17:17:14.022230+0000","last_peered":"2024-09-13T17:17:14.022230+0000","last_clean":"2024-09-13T17:17:14.022230+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T17:17:14.022230+0000","last_undegraded":"2024-09-13T17:17:14.022230+0000","last_fullsized":"2024-09-13T17:17:14.022230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3371,"log_dups_size":0,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18156,"num_write_kb":249012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"63'3752","reported_seq":3819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.066911+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T17:17:22.066911+0000","last_peered":"2024-09-13T17:17:22.066911+0000","last_clean":"2024-09-13T17:17:22.066911+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T17:17:22.066911+0000","last_undegraded":"2024-09-13T17:17:22.066911+0000","last_fullsized":"2024-09-13T17:17:22.066911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21127,"num_write_kb":268998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"63'2","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.056813+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T17:17:22.056813+0000","last_peered":"2024-09-13T17:17:22.056813+0000","last_clean":"2024-09-13T17:17:22.056813+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T17:17:22.056813+0000","last_undegraded":"2024-09-13T17:17:22.056813+0000","last_fullsized":"2024-09-13T17:17:22.056813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"63'4","reported_seq":88,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.029328+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T17:17:14.029328+0000","last_peered":"2024-09-13T17:17:14.029328+0000","last_clean":"2024-09-13T17:17:14.029328+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T17:17:14.029328+0000","last_undegraded":"2024-09-13T17:17:14.029328+0000","last_fullsized":"2024-09-13T17:17:14.029328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"63'4499","reported_seq":4566,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:34.270352+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T17:17:34.270352+0000","last_peered":"2024-09-13T17:17:34.270352+0000","last_clean":"2024-09-13T17:17:34.270352+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T17:17:34.270352+0000","last_undegraded":"2024-09-13T17:17:34.270352+0000","last_fullsized":"2024-09-13T17:17:34.270352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":24341,"num_write_kb":343418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"63'4043","reported_seq":4109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.908339+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T17:17:21.908339+0000","last_peered":"2024-09-13T17:17:21.908339+0000","last_clean":"2024-09-13T17:17:21.908339+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T17:17:21.908339+0000","last_undegraded":"2024-09-13T17:17:21.908339+0000","last_fullsized":"2024-09-13T17:17:21.908339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:29.510665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22005,"num_write_kb":288910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"63'5","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.010552+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T17:17:22.010552+0000","last_peered":"2024-09-13T17:17:22.010552+0000","last_clean":"2024-09-13T17:17:22.010552+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T17:17:22.010552+0000","last_undegraded":"2024-09-13T17:17:22.010552+0000","last_fullsized":"2024-09-13T17:17:22.010552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"63'7","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.024599+0000","last_change":"2024-09-13T16:48:07.771097+0000","last_active":"2024-09-13T17:17:14.024599+0000","last_peered":"2024-09-13T17:17:14.024599+0000","last_clean":"2024-09-13T17:17:14.024599+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T17:17:14.024599+0000","last_undegraded":"2024-09-13T17:17:14.024599+0000","last_fullsized":"2024-09-13T17:17:14.024599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"63'3470","reported_seq":3534,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.047830+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T17:17:14.047830+0000","last_peered":"2024-09-13T17:17:14.047830+0000","last_clean":"2024-09-13T17:17:14.047830+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T17:17:14.047830+0000","last_undegraded":"2024-09-13T17:17:14.047830+0000","last_fullsized":"2024-09-13T17:17:14.047830+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19441,"num_write_kb":258288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'4844","reported_seq":4911,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.060853+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T17:17:22.060853+0000","last_peered":"2024-09-13T17:17:22.060853+0000","last_clean":"2024-09-13T17:17:22.060853+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T17:17:22.060853+0000","last_undegraded":"2024-09-13T17:17:22.060853+0000","last_fullsized":"2024-09-13T17:17:22.060853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:00:20.669843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":26635,"num_write_kb":336207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"63'9","reported_seq":113,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.128964+0000","last_change":"2024-09-13T16:47:58.693050+0000","last_active":"2024-09-13T17:17:22.128964+0000","last_peered":"2024-09-13T17:17:22.128964+0000","last_clean":"2024-09-13T17:17:22.128964+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T17:17:22.128964+0000","last_undegraded":"2024-09-13T17:17:22.128964+0000","last_fullsized":"2024-09-13T17:17:22.128964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"63'48","reported_seq":175,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:24.143768+0000","last_change":"2024-09-13T16:48:07.773328+0000","last_active":"2024-09-13T17:17:24.143768+0000","last_peered":"2024-09-13T17:17:24.143768+0000","last_clean":"2024-09-13T17:17:24.143768+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T17:17:24.143768+0000","last_undegraded":"2024-09-13T17:17:24.143768+0000","last_fullsized":"2024-09-13T17:17:24.143768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:09.713920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92726,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":118,"num_write":43,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"63'3635","reported_seq":3699,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.001656+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T17:17:14.001656+0000","last_peered":"2024-09-13T17:17:14.001656+0000","last_clean":"2024-09-13T17:17:14.001656+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T17:17:14.001656+0000","last_undegraded":"2024-09-13T17:17:14.001656+0000","last_fullsized":"2024-09-13T17:17:14.001656+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19787,"num_write_kb":259620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'3639","reported_seq":3706,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.125150+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T17:17:22.125150+0000","last_peered":"2024-09-13T17:17:22.125150+0000","last_clean":"2024-09-13T17:17:22.125150+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T17:17:22.125150+0000","last_undegraded":"2024-09-13T17:17:22.125150+0000","last_fullsized":"2024-09-13T17:17:22.125150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3639,"log_dups_size":0,"ondisk_log_size":3639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20066,"num_write_kb":240267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"63'265","reported_seq":350,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.104288+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T17:17:22.104288+0000","last_peered":"2024-09-13T17:17:22.104288+0000","last_clean":"2024-09-13T17:17:22.104288+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T17:17:22.104288+0000","last_undegraded":"2024-09-13T17:17:22.104288+0000","last_fullsized":"2024-09-13T17:17:22.104288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:13.178229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":36,"num_read_kb":46,"num_write":634,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"63'11","reported_seq":76,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.053090+0000","last_change":"2024-09-13T16:48:07.790342+0000","last_active":"2024-09-13T17:17:14.053090+0000","last_peered":"2024-09-13T17:17:14.053090+0000","last_clean":"2024-09-13T17:17:14.053090+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T17:17:14.053090+0000","last_undegraded":"2024-09-13T17:17:14.053090+0000","last_fullsized":"2024-09-13T17:17:14.053090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:48.023335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"63'3190","reported_seq":3254,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:23.119014+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T17:17:23.119014+0000","last_peered":"2024-09-13T17:17:23.119014+0000","last_clean":"2024-09-13T17:17:23.119014+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T17:17:23.119014+0000","last_undegraded":"2024-09-13T17:17:23.119014+0000","last_fullsized":"2024-09-13T17:17:23.119014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3190,"log_dups_size":0,"ondisk_log_size":3190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":254866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"63'4701","reported_seq":4768,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.071944+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T17:17:22.071944+0000","last_peered":"2024-09-13T17:17:22.071944+0000","last_clean":"2024-09-13T17:17:22.071944+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T17:17:22.071944+0000","last_undegraded":"2024-09-13T17:17:22.071944+0000","last_fullsized":"2024-09-13T17:17:22.071944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26171,"num_write_kb":338588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'2","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:19.802813+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T17:17:19.802813+0000","last_peered":"2024-09-13T17:17:19.802813+0000","last_clean":"2024-09-13T17:17:19.802813+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T17:17:19.802813+0000","last_undegraded":"2024-09-13T17:17:19.802813+0000","last_fullsized":"2024-09-13T17:17:19.802813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"63'3","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.988743+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T17:17:13.988743+0000","last_peered":"2024-09-13T17:17:13.988743+0000","last_clean":"2024-09-13T17:17:13.988743+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T17:17:13.988743+0000","last_undegraded":"2024-09-13T17:17:13.988743+0000","last_fullsized":"2024-09-13T17:17:13.988743+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"63'3859","reported_seq":3922,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.945566+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T17:17:13.945566+0000","last_peered":"2024-09-13T17:17:13.945566+0000","last_clean":"2024-09-13T17:17:13.945566+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T17:17:13.945566+0000","last_undegraded":"2024-09-13T17:17:13.945566+0000","last_fullsized":"2024-09-13T17:17:13.945566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:54.504578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21286,"num_write_kb":290222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"63'3876","reported_seq":3944,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117761+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T17:17:22.117761+0000","last_peered":"2024-09-13T17:17:22.117761+0000","last_clean":"2024-09-13T17:17:22.117761+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T17:17:22.117761+0000","last_undegraded":"2024-09-13T17:17:22.117761+0000","last_fullsized":"2024-09-13T17:17:22.117761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21201,"num_write_kb":273354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"63'2","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.094910+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T17:17:22.094910+0000","last_peered":"2024-09-13T17:17:22.094910+0000","last_clean":"2024-09-13T17:17:22.094910+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T17:17:22.094910+0000","last_undegraded":"2024-09-13T17:17:22.094910+0000","last_fullsized":"2024-09-13T17:17:22.094910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'5","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:12.944152+0000","last_change":"2024-09-13T16:48:07.780995+0000","last_active":"2024-09-13T17:17:12.944152+0000","last_peered":"2024-09-13T17:17:12.944152+0000","last_clean":"2024-09-13T17:17:12.944152+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T17:17:12.944152+0000","last_undegraded":"2024-09-13T17:17:12.944152+0000","last_fullsized":"2024-09-13T17:17:12.944152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"63'3422","reported_seq":3486,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058428+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T17:17:14.058428+0000","last_peered":"2024-09-13T17:17:14.058428+0000","last_clean":"2024-09-13T17:17:14.058428+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T17:17:14.058428+0000","last_undegraded":"2024-09-13T17:17:14.058428+0000","last_fullsized":"2024-09-13T17:17:14.058428+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":7,"num_write":19042,"num_write_kb":260722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"63'4802","reported_seq":4869,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.992037+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T17:17:21.992037+0000","last_peered":"2024-09-13T17:17:21.992037+0000","last_clean":"2024-09-13T17:17:21.992037+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T17:17:21.992037+0000","last_undegraded":"2024-09-13T17:17:21.992037+0000","last_fullsized":"2024-09-13T17:17:21.992037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26419,"num_write_kb":335470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"63'4","reported_seq":73,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.005204+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T17:17:22.005204+0000","last_peered":"2024-09-13T17:17:22.005204+0000","last_clean":"2024-09-13T17:17:22.005204+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T17:17:22.005204+0000","last_undegraded":"2024-09-13T17:17:22.005204+0000","last_fullsized":"2024-09-13T17:17:22.005204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"63'2","reported_seq":66,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.026552+0000","last_change":"2024-09-13T16:48:07.770442+0000","last_active":"2024-09-13T17:17:14.026552+0000","last_peered":"2024-09-13T17:17:14.026552+0000","last_clean":"2024-09-13T17:17:14.026552+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T17:17:14.026552+0000","last_undegraded":"2024-09-13T17:17:14.026552+0000","last_fullsized":"2024-09-13T17:17:14.026552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"63'4215","reported_seq":4280,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009307+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T17:17:45.009307+0000","last_peered":"2024-09-13T17:17:45.009307+0000","last_clean":"2024-09-13T17:17:45.009307+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T17:17:45.009307+0000","last_undegraded":"2024-09-13T17:17:45.009307+0000","last_fullsized":"2024-09-13T17:17:45.009307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":8,"num_write":22461,"num_write_kb":319331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'3790","reported_seq":3857,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107689+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T17:17:22.107689+0000","last_peered":"2024-09-13T17:17:22.107689+0000","last_clean":"2024-09-13T17:17:22.107689+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T17:17:22.107689+0000","last_undegraded":"2024-09-13T17:17:22.107689+0000","last_fullsized":"2024-09-13T17:17:22.107689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21077,"num_write_kb":260732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'42","reported_seq":168,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.005437+0000","last_change":"2024-09-13T16:47:58.663710+0000","last_active":"2024-09-13T17:17:48.005437+0000","last_peered":"2024-09-13T17:17:48.005437+0000","last_clean":"2024-09-13T17:17:48.005437+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T17:17:48.005437+0000","last_undegraded":"2024-09-13T17:17:48.005437+0000","last_fullsized":"2024-09-13T17:17:48.005437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84646,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":106,"num_write":37,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"63'9","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.040704+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T17:17:14.040704+0000","last_peered":"2024-09-13T17:17:14.040704+0000","last_clean":"2024-09-13T17:17:14.040704+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T17:17:14.040704+0000","last_undegraded":"2024-09-13T17:17:14.040704+0000","last_fullsized":"2024-09-13T17:17:14.040704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"63'3128","reported_seq":3191,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.060092+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T17:17:14.060092+0000","last_peered":"2024-09-13T17:17:14.060092+0000","last_clean":"2024-09-13T17:17:14.060092+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T17:17:14.060092+0000","last_undegraded":"2024-09-13T17:17:14.060092+0000","last_fullsized":"2024-09-13T17:17:14.060092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3128,"log_dups_size":0,"ondisk_log_size":3128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17776,"num_write_kb":243283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"63'3657","reported_seq":3724,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.977555+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T17:17:21.977555+0000","last_peered":"2024-09-13T17:17:21.977555+0000","last_clean":"2024-09-13T17:17:21.977555+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T17:17:21.977555+0000","last_undegraded":"2024-09-13T17:17:21.977555+0000","last_fullsized":"2024-09-13T17:17:21.977555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19816,"num_write_kb":255512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"63'10","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.101605+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T17:17:22.101605+0000","last_peered":"2024-09-13T17:17:22.101605+0000","last_clean":"2024-09-13T17:17:22.101605+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T17:17:22.101605+0000","last_undegraded":"2024-09-13T17:17:22.101605+0000","last_fullsized":"2024-09-13T17:17:22.101605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'251","reported_seq":335,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.918664+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T17:17:22.918664+0000","last_peered":"2024-09-13T17:17:22.918664+0000","last_clean":"2024-09-13T17:17:22.918664+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T17:17:22.918664+0000","last_undegraded":"2024-09-13T17:17:22.918664+0000","last_fullsized":"2024-09-13T17:17:22.918664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:31.803980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":57,"num_write":597,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"63'4141","reported_seq":4205,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022107+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T17:17:14.022107+0000","last_peered":"2024-09-13T17:17:14.022107+0000","last_clean":"2024-09-13T17:17:14.022107+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T17:17:14.022107+0000","last_undegraded":"2024-09-13T17:17:14.022107+0000","last_fullsized":"2024-09-13T17:17:14.022107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:21.770668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22849,"num_write_kb":317008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'4103","reported_seq":4179,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126990+0000","last_change":"2024-09-13T16:48:41.389774+0000","last_active":"2024-09-13T17:17:22.126990+0000","last_peered":"2024-09-13T17:17:22.126990+0000","last_clean":"2024-09-13T17:17:22.126990+0000","last_became_active":"2024-09-13T16:48:41.389507+0000","last_became_peered":"2024-09-13T16:48:41.389507+0000","last_unstale":"2024-09-13T17:17:22.126990+0000","last_undegraded":"2024-09-13T17:17:22.126990+0000","last_fullsized":"2024-09-13T17:17:22.126990+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:07:23.114010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22342,"num_write_kb":278194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"63'3","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.059314+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T17:17:22.059314+0000","last_peered":"2024-09-13T17:17:22.059314+0000","last_clean":"2024-09-13T17:17:22.059314+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T17:17:22.059314+0000","last_undegraded":"2024-09-13T17:17:22.059314+0000","last_fullsized":"2024-09-13T17:17:22.059314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":221,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T17:16:04.972860+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T17:16:04.972860+0000","last_peered":"2024-09-13T17:16:04.972860+0000","last_clean":"2024-09-13T17:16:04.972860+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T17:16:04.972860+0000","last_undegraded":"2024-09-13T17:16:04.972860+0000","last_fullsized":"2024-09-13T17:16:04.972860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:32:09.420575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"63'2","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:23.119120+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T17:17:23.119120+0000","last_peered":"2024-09-13T17:17:23.119120+0000","last_clean":"2024-09-13T17:17:23.119120+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T17:17:23.119120+0000","last_undegraded":"2024-09-13T17:17:23.119120+0000","last_fullsized":"2024-09-13T17:17:23.119120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"63'2904","reported_seq":2969,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007931+0000","last_change":"2024-09-13T16:48:08.171323+0000","last_active":"2024-09-13T17:17:45.007931+0000","last_peered":"2024-09-13T17:17:45.007931+0000","last_clean":"2024-09-13T17:17:45.007931+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T17:17:45.007931+0000","last_undegraded":"2024-09-13T17:17:45.007931+0000","last_fullsized":"2024-09-13T17:17:45.007931+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":2,"num_write":15607,"num_write_kb":214254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'4284","reported_seq":4350,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117903+0000","last_change":"2024-09-13T16:47:59.062248+0000","last_active":"2024-09-13T17:17:22.117903+0000","last_peered":"2024-09-13T17:17:22.117903+0000","last_clean":"2024-09-13T17:17:22.117903+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T17:17:22.117903+0000","last_undegraded":"2024-09-13T17:17:22.117903+0000","last_fullsized":"2024-09-13T17:17:22.117903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:01.630853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23392,"num_write_kb":310963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"63'5","reported_seq":74,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009009+0000","last_change":"2024-09-13T16:47:58.676483+0000","last_active":"2024-09-13T17:17:45.009009+0000","last_peered":"2024-09-13T17:17:45.009009+0000","last_clean":"2024-09-13T17:17:45.009009+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T17:17:45.009009+0000","last_undegraded":"2024-09-13T17:17:45.009009+0000","last_fullsized":"2024-09-13T17:17:45.009009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"63'3","reported_seq":67,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.043703+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T17:17:14.043703+0000","last_peered":"2024-09-13T17:17:14.043703+0000","last_clean":"2024-09-13T17:17:14.043703+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T17:17:14.043703+0000","last_undegraded":"2024-09-13T17:17:14.043703+0000","last_fullsized":"2024-09-13T17:17:14.043703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:16.245653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"63'4138","reported_seq":4202,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.006357+0000","last_change":"2024-09-13T16:48:08.178326+0000","last_active":"2024-09-13T17:17:45.006357+0000","last_peered":"2024-09-13T17:17:45.006357+0000","last_clean":"2024-09-13T17:17:45.006357+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T17:17:45.006357+0000","last_undegraded":"2024-09-13T17:17:45.006357+0000","last_fullsized":"2024-09-13T17:17:45.006357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":309086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"63'4584","reported_seq":4645,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007933+0000","last_change":"2024-09-13T16:48:40.767044+0000","last_active":"2024-09-13T17:17:45.007933+0000","last_peered":"2024-09-13T17:17:45.007933+0000","last_clean":"2024-09-13T17:17:45.007933+0000","last_became_active":"2024-09-13T16:48:40.765861+0000","last_became_peered":"2024-09-13T16:48:40.765861+0000","last_unstale":"2024-09-13T17:17:45.007933+0000","last_undegraded":"2024-09-13T17:17:45.007933+0000","last_fullsized":"2024-09-13T17:17:45.007933+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24690,"num_write_kb":320186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"63'6","reported_seq":91,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.999257+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T17:17:21.999257+0000","last_peered":"2024-09-13T17:17:21.999257+0000","last_clean":"2024-09-13T17:17:21.999257+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T17:17:21.999257+0000","last_undegraded":"2024-09-13T17:17:21.999257+0000","last_fullsized":"2024-09-13T17:17:21.999257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:15.602255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'1425","reported_seq":1536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.059648+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T17:17:14.059648+0000","last_peered":"2024-09-13T17:17:14.059648+0000","last_clean":"2024-09-13T17:17:14.059648+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T17:17:14.059648+0000","last_undegraded":"2024-09-13T17:17:14.059648+0000","last_fullsized":"2024-09-13T17:17:14.059648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:04.476239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":38459,"num_write":1400,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"63'3637","reported_seq":3700,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.967851+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T17:17:13.967851+0000","last_peered":"2024-09-13T17:17:13.967851+0000","last_clean":"2024-09-13T17:17:13.967851+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T17:17:13.967851+0000","last_undegraded":"2024-09-13T17:17:13.967851+0000","last_fullsized":"2024-09-13T17:17:13.967851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:14:46.926174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19635,"num_write_kb":259018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'5067","reported_seq":5128,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009400+0000","last_change":"2024-09-13T16:48:40.831954+0000","last_active":"2024-09-13T17:17:45.009400+0000","last_peered":"2024-09-13T17:17:45.009400+0000","last_clean":"2024-09-13T17:17:45.009400+0000","last_became_active":"2024-09-13T16:48:40.831696+0000","last_became_peered":"2024-09-13T16:48:40.831696+0000","last_unstale":"2024-09-13T17:17:45.009400+0000","last_undegraded":"2024-09-13T17:17:45.009400+0000","last_fullsized":"2024-09-13T17:17:45.009400+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27028,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"63'7","reported_seq":114,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.996966+0000","last_change":"2024-09-13T16:47:58.662211+0000","last_active":"2024-09-13T17:17:47.996966+0000","last_peered":"2024-09-13T17:17:47.996966+0000","last_clean":"2024-09-13T17:17:47.996966+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T17:17:47.996966+0000","last_undegraded":"2024-09-13T17:17:47.996966+0000","last_fullsized":"2024-09-13T17:17:47.996966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"63'5","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.028663+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T17:17:14.028663+0000","last_peered":"2024-09-13T17:17:14.028663+0000","last_clean":"2024-09-13T17:17:14.028663+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T17:17:14.028663+0000","last_undegraded":"2024-09-13T17:17:14.028663+0000","last_fullsized":"2024-09-13T17:17:14.028663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:20.286580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"63'2915","reported_seq":2979,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.006202+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T17:17:45.006202+0000","last_peered":"2024-09-13T17:17:45.006202+0000","last_clean":"2024-09-13T17:17:45.006202+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T17:17:45.006202+0000","last_undegraded":"2024-09-13T17:17:45.006202+0000","last_fullsized":"2024-09-13T17:17:45.006202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2915,"log_dups_size":0,"ondisk_log_size":2915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":234494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"63'3697","reported_seq":3763,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.091815+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T17:17:22.091815+0000","last_peered":"2024-09-13T17:17:22.091815+0000","last_clean":"2024-09-13T17:17:22.091815+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T17:17:22.091815+0000","last_undegraded":"2024-09-13T17:17:22.091815+0000","last_fullsized":"2024-09-13T17:17:22.091815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20206,"num_write_kb":253140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"63'4","reported_seq":89,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107701+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T17:17:22.107701+0000","last_peered":"2024-09-13T17:17:22.107701+0000","last_clean":"2024-09-13T17:17:22.107701+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T17:17:22.107701+0000","last_undegraded":"2024-09-13T17:17:22.107701+0000","last_fullsized":"2024-09-13T17:17:22.107701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"63'2","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.991145+0000","last_change":"2024-09-13T16:48:07.773400+0000","last_active":"2024-09-13T17:17:13.991145+0000","last_peered":"2024-09-13T17:17:13.991145+0000","last_clean":"2024-09-13T17:17:13.991145+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T17:17:13.991145+0000","last_undegraded":"2024-09-13T17:17:13.991145+0000","last_fullsized":"2024-09-13T17:17:13.991145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:50.403511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"63'3522","reported_seq":3584,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.052108+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T17:17:14.052108+0000","last_peered":"2024-09-13T17:17:14.052108+0000","last_clean":"2024-09-13T17:17:14.052108+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T17:17:14.052108+0000","last_undegraded":"2024-09-13T17:17:14.052108+0000","last_fullsized":"2024-09-13T17:17:14.052108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18828,"num_write_kb":272277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"63'3874","reported_seq":3941,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.067632+0000","last_change":"2024-09-13T16:47:59.057877+0000","last_active":"2024-09-13T17:17:22.067632+0000","last_peered":"2024-09-13T17:17:22.067632+0000","last_clean":"2024-09-13T17:17:22.067632+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T17:17:22.067632+0000","last_undegraded":"2024-09-13T17:17:22.067632+0000","last_fullsized":"2024-09-13T17:17:22.067632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21413,"num_write_kb":278431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'15","reported_seq":177,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.001665+0000","last_change":"2024-09-13T16:47:58.657701+0000","last_active":"2024-09-13T17:17:48.001665+0000","last_peered":"2024-09-13T17:17:48.001665+0000","last_clean":"2024-09-13T17:17:48.001665+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T17:17:48.001665+0000","last_undegraded":"2024-09-13T17:17:48.001665+0000","last_fullsized":"2024-09-13T17:17:48.001665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036103+0000","last_change":"2024-09-13T16:48:07.780088+0000","last_active":"2024-09-13T17:17:14.036103+0000","last_peered":"2024-09-13T17:17:14.036103+0000","last_clean":"2024-09-13T17:17:14.036103+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T17:17:14.036103+0000","last_undegraded":"2024-09-13T17:17:14.036103+0000","last_fullsized":"2024-09-13T17:17:14.036103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:23.157876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"63'3622","reported_seq":3687,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.258791+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T17:17:14.258791+0000","last_peered":"2024-09-13T17:17:14.258791+0000","last_clean":"2024-09-13T17:17:14.258791+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T17:17:14.258791+0000","last_undegraded":"2024-09-13T17:17:14.258791+0000","last_fullsized":"2024-09-13T17:17:14.258791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":19996,"num_write_kb":293269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"63'3240","reported_seq":3306,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.115212+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T17:17:22.115212+0000","last_peered":"2024-09-13T17:17:22.115212+0000","last_clean":"2024-09-13T17:17:22.115212+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T17:17:22.115212+0000","last_undegraded":"2024-09-13T17:17:22.115212+0000","last_fullsized":"2024-09-13T17:17:22.115212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3240,"log_dups_size":0,"ondisk_log_size":3240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:18:47.356536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":237245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"63'5","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.102680+0000","last_change":"2024-09-13T16:47:58.689009+0000","last_active":"2024-09-13T17:17:22.102680+0000","last_peered":"2024-09-13T17:17:22.102680+0000","last_clean":"2024-09-13T17:17:22.102680+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T17:17:22.102680+0000","last_undegraded":"2024-09-13T17:17:22.102680+0000","last_fullsized":"2024-09-13T17:17:22.102680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'5","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.028898+0000","last_change":"2024-09-13T16:48:07.788639+0000","last_active":"2024-09-13T17:17:14.028898+0000","last_peered":"2024-09-13T17:17:14.028898+0000","last_clean":"2024-09-13T17:17:14.028898+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T17:17:14.028898+0000","last_undegraded":"2024-09-13T17:17:14.028898+0000","last_fullsized":"2024-09-13T17:17:14.028898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:53.078363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"63'3376","reported_seq":3438,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.779280+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T17:17:13.779280+0000","last_peered":"2024-09-13T17:17:13.779280+0000","last_clean":"2024-09-13T17:17:13.779280+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T17:17:13.779280+0000","last_undegraded":"2024-09-13T17:17:13.779280+0000","last_fullsized":"2024-09-13T17:17:13.779280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18879,"num_write_kb":251906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"63'4848","reported_seq":4913,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.099538+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T17:17:22.099538+0000","last_peered":"2024-09-13T17:17:22.099538+0000","last_clean":"2024-09-13T17:17:22.099538+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T17:17:22.099538+0000","last_undegraded":"2024-09-13T17:17:22.099538+0000","last_fullsized":"2024-09-13T17:17:22.099538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26963,"num_write_kb":321221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"63'3","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117465+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T17:17:22.117465+0000","last_peered":"2024-09-13T17:17:22.117465+0000","last_clean":"2024-09-13T17:17:22.117465+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T17:17:22.117465+0000","last_undegraded":"2024-09-13T17:17:22.117465+0000","last_fullsized":"2024-09-13T17:17:22.117465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"63'389","reported_seq":559,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:27.920919+0000","last_change":"2024-09-13T16:48:07.777911+0000","last_active":"2024-09-13T17:17:27.920919+0000","last_peered":"2024-09-13T17:17:27.920919+0000","last_clean":"2024-09-13T17:17:27.920919+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T17:17:27.920919+0000","last_undegraded":"2024-09-13T17:17:27.920919+0000","last_fullsized":"2024-09-13T17:17:27.920919+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":191,"num_read_kb":82094,"num_write":385,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'4856","reported_seq":4920,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.046257+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T17:17:14.046257+0000","last_peered":"2024-09-13T17:17:14.046257+0000","last_clean":"2024-09-13T17:17:14.046257+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T17:17:14.046257+0000","last_undegraded":"2024-09-13T17:17:14.046257+0000","last_fullsized":"2024-09-13T17:17:14.046257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:58.733539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":26453,"num_write_kb":368210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'4063","reported_seq":4128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.081187+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T17:17:22.081187+0000","last_peered":"2024-09-13T17:17:22.081187+0000","last_clean":"2024-09-13T17:17:22.081187+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T17:17:22.081187+0000","last_undegraded":"2024-09-13T17:17:22.081187+0000","last_fullsized":"2024-09-13T17:17:22.081187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4063,"log_dups_size":0,"ondisk_log_size":4063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:41.090453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21999,"num_write_kb":293018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"63'8","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.105763+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T17:17:22.105763+0000","last_peered":"2024-09-13T17:17:22.105763+0000","last_clean":"2024-09-13T17:17:22.105763+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T17:17:22.105763+0000","last_undegraded":"2024-09-13T17:17:22.105763+0000","last_fullsized":"2024-09-13T17:17:22.105763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:19:19.039572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":144,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"63'30","reported_seq":156,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:24.150602+0000","last_change":"2024-09-13T16:48:07.786095+0000","last_active":"2024-09-13T17:17:24.150602+0000","last_peered":"2024-09-13T17:17:24.150602+0000","last_clean":"2024-09-13T17:17:24.150602+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T17:17:24.150602+0000","last_undegraded":"2024-09-13T17:17:24.150602+0000","last_fullsized":"2024-09-13T17:17:24.150602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:53.491149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":31,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"63'4239","reported_seq":4301,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.056405+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T17:17:14.056405+0000","last_peered":"2024-09-13T17:17:14.056405+0000","last_clean":"2024-09-13T17:17:14.056405+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T17:17:14.056405+0000","last_undegraded":"2024-09-13T17:17:14.056405+0000","last_fullsized":"2024-09-13T17:17:14.056405+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.675519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23316,"num_write_kb":318961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'3718","reported_seq":3783,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:21.786167+0000","last_change":"2024-09-13T16:47:59.061237+0000","last_active":"2024-09-13T17:17:21.786167+0000","last_peered":"2024-09-13T17:17:21.786167+0000","last_clean":"2024-09-13T17:17:21.786167+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T17:17:21.786167+0000","last_undegraded":"2024-09-13T17:17:21.786167+0000","last_fullsized":"2024-09-13T17:17:21.786167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3718,"log_dups_size":0,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20956,"num_write_kb":260218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"63'390","reported_seq":550,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.008859+0000","last_change":"2024-09-13T16:47:58.689230+0000","last_active":"2024-09-13T17:17:45.008859+0000","last_peered":"2024-09-13T17:17:45.008859+0000","last_clean":"2024-09-13T17:17:45.008859+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T17:17:45.008859+0000","last_undegraded":"2024-09-13T17:17:45.008859+0000","last_fullsized":"2024-09-13T17:17:45.008859+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:53.081544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":165,"num_read_kb":77970,"num_write":388,"num_write_kb":4310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"63'3","reported_seq":65,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.039214+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T17:17:14.039214+0000","last_peered":"2024-09-13T17:17:14.039214+0000","last_clean":"2024-09-13T17:17:14.039214+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T17:17:14.039214+0000","last_undegraded":"2024-09-13T17:17:14.039214+0000","last_fullsized":"2024-09-13T17:17:14.039214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:14.264036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"63'3522","reported_seq":3584,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.982810+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T17:17:13.982810+0000","last_peered":"2024-09-13T17:17:13.982810+0000","last_clean":"2024-09-13T17:17:13.982810+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T17:17:13.982810+0000","last_undegraded":"2024-09-13T17:17:13.982810+0000","last_fullsized":"2024-09-13T17:17:13.982810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":4,"num_write":19373,"num_write_kb":266222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"63'5138","reported_seq":5203,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095588+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T17:17:22.095588+0000","last_peered":"2024-09-13T17:17:22.095588+0000","last_clean":"2024-09-13T17:17:22.095588+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T17:17:22.095588+0000","last_undegraded":"2024-09-13T17:17:22.095588+0000","last_fullsized":"2024-09-13T17:17:22.095588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:16.612771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27863,"num_write_kb":353454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"63'23","reported_seq":125,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.131774+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T17:17:22.131774+0000","last_peered":"2024-09-13T17:17:22.131774+0000","last_clean":"2024-09-13T17:17:22.131774+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T17:17:22.131774+0000","last_undegraded":"2024-09-13T17:17:22.131774+0000","last_fullsized":"2024-09-13T17:17:22.131774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:02.920217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":24,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"63'7","reported_seq":111,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031857+0000","last_change":"2024-09-13T16:48:07.776337+0000","last_active":"2024-09-13T17:17:14.031857+0000","last_peered":"2024-09-13T17:17:14.031857+0000","last_clean":"2024-09-13T17:17:14.031857+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T17:17:14.031857+0000","last_undegraded":"2024-09-13T17:17:14.031857+0000","last_fullsized":"2024-09-13T17:17:14.031857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:49:05.175542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":168,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"63'3602","reported_seq":3667,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.045822+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T17:17:14.045822+0000","last_peered":"2024-09-13T17:17:14.045822+0000","last_clean":"2024-09-13T17:17:14.045822+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T17:17:14.045822+0000","last_undegraded":"2024-09-13T17:17:14.045822+0000","last_fullsized":"2024-09-13T17:17:14.045822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":20290,"num_write_kb":278255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'3743","reported_seq":3808,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084809+0000","last_change":"2024-09-13T16:47:59.055134+0000","last_active":"2024-09-13T17:17:22.084809+0000","last_peered":"2024-09-13T17:17:22.084809+0000","last_clean":"2024-09-13T17:17:22.084809+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T17:17:22.084809+0000","last_undegraded":"2024-09-13T17:17:22.084809+0000","last_fullsized":"2024-09-13T17:17:22.084809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20404,"num_write_kb":278673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"63'3","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.132312+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T17:17:22.132312+0000","last_peered":"2024-09-13T17:17:22.132312+0000","last_clean":"2024-09-13T17:17:22.132312+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T17:17:22.132312+0000","last_undegraded":"2024-09-13T17:17:22.132312+0000","last_fullsized":"2024-09-13T17:17:22.132312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:28.500149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"63'15","reported_seq":186,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:24.145040+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T17:17:24.145040+0000","last_peered":"2024-09-13T17:17:24.145040+0000","last_clean":"2024-09-13T17:17:24.145040+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T17:17:24.145040+0000","last_undegraded":"2024-09-13T17:17:24.145040+0000","last_fullsized":"2024-09-13T17:17:24.145040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:21.516308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":324,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"63'3189","reported_seq":3250,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.950668+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T17:17:13.950668+0000","last_peered":"2024-09-13T17:17:13.950668+0000","last_clean":"2024-09-13T17:17:13.950668+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T17:17:13.950668+0000","last_undegraded":"2024-09-13T17:17:13.950668+0000","last_fullsized":"2024-09-13T17:17:13.950668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17342,"num_write_kb":249797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"63'4223","reported_seq":4289,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.120733+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T17:17:22.120733+0000","last_peered":"2024-09-13T17:17:22.120733+0000","last_clean":"2024-09-13T17:17:22.120733+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T17:17:22.120733+0000","last_undegraded":"2024-09-13T17:17:22.120733+0000","last_fullsized":"2024-09-13T17:17:22.120733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4223,"log_dups_size":0,"ondisk_log_size":4223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:58.817509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":295025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"63'4","reported_seq":71,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.102508+0000","last_change":"2024-09-13T16:47:58.686671+0000","last_active":"2024-09-13T17:17:22.102508+0000","last_peered":"2024-09-13T17:17:22.102508+0000","last_clean":"2024-09-13T17:17:22.102508+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T17:17:22.102508+0000","last_undegraded":"2024-09-13T17:17:22.102508+0000","last_fullsized":"2024-09-13T17:17:22.102508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:05.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"63'5","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.025851+0000","last_change":"2024-09-13T16:48:07.788767+0000","last_active":"2024-09-13T17:17:14.025851+0000","last_peered":"2024-09-13T17:17:14.025851+0000","last_clean":"2024-09-13T17:17:14.025851+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T17:17:14.025851+0000","last_undegraded":"2024-09-13T17:17:14.025851+0000","last_fullsized":"2024-09-13T17:17:14.025851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:14.698391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'5357","reported_seq":5424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:23.119212+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T17:17:23.119212+0000","last_peered":"2024-09-13T17:17:23.119212+0000","last_clean":"2024-09-13T17:17:23.119212+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T17:17:23.119212+0000","last_undegraded":"2024-09-13T17:17:23.119212+0000","last_fullsized":"2024-09-13T17:17:23.119212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":39,"num_write":29637,"num_write_kb":393569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"63'26","reported_seq":90,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.989101+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T17:17:13.989101+0000","last_peered":"2024-09-13T17:17:13.989101+0000","last_clean":"2024-09-13T17:17:13.989101+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T17:17:13.989101+0000","last_undegraded":"2024-09-13T17:17:13.989101+0000","last_fullsized":"2024-09-13T17:17:13.989101+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:05:24.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"63'5122","reported_seq":5187,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.106963+0000","last_change":"2024-09-13T16:47:59.053497+0000","last_active":"2024-09-13T17:17:22.106963+0000","last_peered":"2024-09-13T17:17:22.106963+0000","last_clean":"2024-09-13T17:17:22.106963+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T17:17:22.106963+0000","last_undegraded":"2024-09-13T17:17:22.106963+0000","last_fullsized":"2024-09-13T17:17:22.106963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":348704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"63'2","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.001468+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T17:17:22.001468+0000","last_peered":"2024-09-13T17:17:22.001468+0000","last_clean":"2024-09-13T17:17:22.001468+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T17:17:22.001468+0000","last_undegraded":"2024-09-13T17:17:22.001468+0000","last_fullsized":"2024-09-13T17:17:22.001468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"63'3554","reported_seq":3617,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036436+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T17:17:14.036436+0000","last_peered":"2024-09-13T17:17:14.036436+0000","last_clean":"2024-09-13T17:17:14.036436+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T17:17:14.036436+0000","last_undegraded":"2024-09-13T17:17:14.036436+0000","last_fullsized":"2024-09-13T17:17:14.036436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:15.711820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":268389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"63'1091","reported_seq":1206,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:18.118214+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T17:17:18.118214+0000","last_peered":"2024-09-13T17:17:18.118214+0000","last_clean":"2024-09-13T17:17:18.118214+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T17:17:18.118214+0000","last_undegraded":"2024-09-13T17:17:18.118214+0000","last_fullsized":"2024-09-13T17:17:18.118214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:59.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":9436,"num_write":1065,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"63'4714","reported_seq":4780,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.130044+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T17:17:22.130044+0000","last_peered":"2024-09-13T17:17:22.130044+0000","last_clean":"2024-09-13T17:17:22.130044+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T17:17:22.130044+0000","last_undegraded":"2024-09-13T17:17:22.130044+0000","last_fullsized":"2024-09-13T17:17:22.130044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25459,"num_write_kb":343662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"63'3","reported_seq":70,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.105910+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T17:17:22.105910+0000","last_peered":"2024-09-13T17:17:22.105910+0000","last_clean":"2024-09-13T17:17:22.105910+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T17:17:22.105910+0000","last_undegraded":"2024-09-13T17:17:22.105910+0000","last_fullsized":"2024-09-13T17:17:22.105910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:39.013594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"63'4328","reported_seq":4388,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.013223+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T17:17:14.013223+0000","last_peered":"2024-09-13T17:17:14.013223+0000","last_clean":"2024-09-13T17:17:14.013223+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T17:17:14.013223+0000","last_undegraded":"2024-09-13T17:17:14.013223+0000","last_fullsized":"2024-09-13T17:17:14.013223+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24456,"num_write_kb":315374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"64'397","reported_seq":467,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.020483+0000","last_change":"2024-09-13T16:47:58.686384+0000","last_active":"2024-09-13T17:17:45.020483+0000","last_peered":"2024-09-13T17:17:45.020483+0000","last_clean":"2024-09-13T17:17:45.020483+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T17:17:45.020483+0000","last_undegraded":"2024-09-13T17:17:45.020483+0000","last_fullsized":"2024-09-13T17:17:45.020483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:27.905677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2790242,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3190,"num_write":372,"num_write_kb":2945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"63'4155","reported_seq":4219,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.033063+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T17:17:22.033063+0000","last_peered":"2024-09-13T17:17:22.033063+0000","last_clean":"2024-09-13T17:17:22.033063+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T17:17:22.033063+0000","last_undegraded":"2024-09-13T17:17:22.033063+0000","last_fullsized":"2024-09-13T17:17:22.033063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22247,"num_write_kb":285891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"63'5","reported_seq":110,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.008704+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T17:17:45.008704+0000","last_peered":"2024-09-13T17:17:45.008704+0000","last_clean":"2024-09-13T17:17:45.008704+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T17:17:45.008704+0000","last_undegraded":"2024-09-13T17:17:45.008704+0000","last_fullsized":"2024-09-13T17:17:45.008704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"63'3426","reported_seq":3489,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.986253+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T17:17:13.986253+0000","last_peered":"2024-09-13T17:17:13.986253+0000","last_clean":"2024-09-13T17:17:13.986253+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T17:17:13.986253+0000","last_undegraded":"2024-09-13T17:17:13.986253+0000","last_fullsized":"2024-09-13T17:17:13.986253+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3426,"log_dups_size":0,"ondisk_log_size":3426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:58.940000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":18510,"num_write_kb":250514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'712","reported_seq":801,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975371+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T17:17:44.975371+0000","last_peered":"2024-09-13T17:17:44.975371+0000","last_clean":"2024-09-13T17:17:44.975371+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T17:17:44.975371+0000","last_undegraded":"2024-09-13T17:17:44.975371+0000","last_fullsized":"2024-09-13T17:17:44.975371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:15.253376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":74674,"num_write":702,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"63'4316","reported_seq":4381,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.050313+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T17:17:22.050313+0000","last_peered":"2024-09-13T17:17:22.050313+0000","last_clean":"2024-09-13T17:17:22.050313+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T17:17:22.050313+0000","last_undegraded":"2024-09-13T17:17:22.050313+0000","last_fullsized":"2024-09-13T17:17:22.050313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:06.839125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22350,"num_write_kb":290228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"63'44","reported_seq":148,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.031545+0000","last_change":"2024-09-13T16:48:07.783187+0000","last_active":"2024-09-13T17:17:14.031545+0000","last_peered":"2024-09-13T17:17:14.031545+0000","last_clean":"2024-09-13T17:17:14.031545+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T17:17:14.031545+0000","last_undegraded":"2024-09-13T17:17:14.031545+0000","last_fullsized":"2024-09-13T17:17:14.031545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:46.157263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"63'3129","reported_seq":3192,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.032027+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T17:17:14.032027+0000","last_peered":"2024-09-13T17:17:14.032027+0000","last_clean":"2024-09-13T17:17:14.032027+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T17:17:14.032027+0000","last_undegraded":"2024-09-13T17:17:14.032027+0000","last_fullsized":"2024-09-13T17:17:14.032027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:57.716813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17279,"num_write_kb":257601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"63'4785","reported_seq":4853,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.115233+0000","last_change":"2024-09-13T16:47:59.048199+0000","last_active":"2024-09-13T17:17:22.115233+0000","last_peered":"2024-09-13T17:17:22.115233+0000","last_clean":"2024-09-13T17:17:22.115233+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T17:17:22.115233+0000","last_undegraded":"2024-09-13T17:17:22.115233+0000","last_fullsized":"2024-09-13T17:17:22.115233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:33.989462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26302,"num_write_kb":310249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"63'4231","reported_seq":4294,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.009044+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T17:17:14.009044+0000","last_peered":"2024-09-13T17:17:14.009044+0000","last_clean":"2024-09-13T17:17:14.009044+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T17:17:14.009044+0000","last_undegraded":"2024-09-13T17:17:14.009044+0000","last_fullsized":"2024-09-13T17:17:14.009044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:12.688566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22604,"num_write_kb":324065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"63'3874","reported_seq":3944,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.006277+0000","last_change":"2024-09-13T16:47:59.046650+0000","last_active":"2024-09-13T17:17:45.006277+0000","last_peered":"2024-09-13T17:17:45.006277+0000","last_clean":"2024-09-13T17:17:45.006277+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T17:17:45.006277+0000","last_undegraded":"2024-09-13T17:17:45.006277+0000","last_fullsized":"2024-09-13T17:17:45.006277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20920,"num_write_kb":276387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"63'3865","reported_seq":3929,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.055420+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T17:17:14.055420+0000","last_peered":"2024-09-13T17:17:14.055420+0000","last_clean":"2024-09-13T17:17:14.055420+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T17:17:14.055420+0000","last_undegraded":"2024-09-13T17:17:14.055420+0000","last_fullsized":"2024-09-13T17:17:14.055420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21432,"num_write_kb":286796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"63'4552","reported_seq":4622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.111004+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T17:17:22.111004+0000","last_peered":"2024-09-13T17:17:22.111004+0000","last_clean":"2024-09-13T17:17:22.111004+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T17:17:22.111004+0000","last_undegraded":"2024-09-13T17:17:22.111004+0000","last_fullsized":"2024-09-13T17:17:22.111004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:42.320445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":24031,"num_write_kb":313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"63'4071","reported_seq":4134,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.013587+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T17:17:14.013587+0000","last_peered":"2024-09-13T17:17:14.013587+0000","last_clean":"2024-09-13T17:17:14.013587+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T17:17:14.013587+0000","last_undegraded":"2024-09-13T17:17:14.013587+0000","last_fullsized":"2024-09-13T17:17:14.013587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:36.264073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":22061,"num_write_kb":309650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"63'3506","reported_seq":3574,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.087622+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T17:17:22.087622+0000","last_peered":"2024-09-13T17:17:22.087622+0000","last_clean":"2024-09-13T17:17:22.087622+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T17:17:22.087622+0000","last_undegraded":"2024-09-13T17:17:22.087622+0000","last_fullsized":"2024-09-13T17:17:22.087622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3506,"log_dups_size":0,"ondisk_log_size":3506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19493,"num_write_kb":254380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"63'4523","reported_seq":4586,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.039988+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T17:17:14.039988+0000","last_peered":"2024-09-13T17:17:14.039988+0000","last_clean":"2024-09-13T17:17:14.039988+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T17:17:14.039988+0000","last_undegraded":"2024-09-13T17:17:14.039988+0000","last_fullsized":"2024-09-13T17:17:14.039988+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4523,"log_dups_size":0,"ondisk_log_size":4523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:18.161872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24221,"num_write_kb":338849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"63'4422","reported_seq":4490,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.133017+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T17:17:22.133017+0000","last_peered":"2024-09-13T17:17:22.133017+0000","last_clean":"2024-09-13T17:17:22.133017+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T17:17:22.133017+0000","last_undegraded":"2024-09-13T17:17:22.133017+0000","last_fullsized":"2024-09-13T17:17:22.133017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":24862,"num_write_kb":304752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"63'3771","reported_seq":3835,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.022654+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T17:17:14.022654+0000","last_peered":"2024-09-13T17:17:14.022654+0000","last_clean":"2024-09-13T17:17:14.022654+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T17:17:14.022654+0000","last_undegraded":"2024-09-13T17:17:14.022654+0000","last_fullsized":"2024-09-13T17:17:14.022654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21135,"num_write_kb":285540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"63'4461","reported_seq":4527,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.095053+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T17:17:22.095053+0000","last_peered":"2024-09-13T17:17:22.095053+0000","last_clean":"2024-09-13T17:17:22.095053+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T17:17:22.095053+0000","last_undegraded":"2024-09-13T17:17:22.095053+0000","last_fullsized":"2024-09-13T17:17:22.095053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25388,"num_write_kb":319094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"63'4193","reported_seq":4256,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.014710+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T17:17:14.014710+0000","last_peered":"2024-09-13T17:17:14.014710+0000","last_clean":"2024-09-13T17:17:14.014710+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T17:17:14.014710+0000","last_undegraded":"2024-09-13T17:17:14.014710+0000","last_fullsized":"2024-09-13T17:17:14.014710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4193,"log_dups_size":0,"ondisk_log_size":4193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23066,"num_write_kb":313713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"63'3978","reported_seq":4045,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.112801+0000","last_change":"2024-09-13T16:47:59.051802+0000","last_active":"2024-09-13T17:17:22.112801+0000","last_peered":"2024-09-13T17:17:22.112801+0000","last_clean":"2024-09-13T17:17:22.112801+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T17:17:22.112801+0000","last_undegraded":"2024-09-13T17:17:22.112801+0000","last_fullsized":"2024-09-13T17:17:22.112801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22157,"num_write_kb":281285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"63'4050","reported_seq":4116,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:23.118957+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T17:17:23.118957+0000","last_peered":"2024-09-13T17:17:23.118957+0000","last_clean":"2024-09-13T17:17:23.118957+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T17:17:23.118957+0000","last_undegraded":"2024-09-13T17:17:23.118957+0000","last_fullsized":"2024-09-13T17:17:23.118957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4050,"log_dups_size":0,"ondisk_log_size":4050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:49.036764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21731,"num_write_kb":291923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"63'5211","reported_seq":5274,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009487+0000","last_change":"2024-09-13T16:48:40.831835+0000","last_active":"2024-09-13T17:17:45.009487+0000","last_peered":"2024-09-13T17:17:45.009487+0000","last_clean":"2024-09-13T17:17:45.009487+0000","last_became_active":"2024-09-13T16:48:40.831649+0000","last_became_peered":"2024-09-13T16:48:40.831649+0000","last_unstale":"2024-09-13T17:17:45.009487+0000","last_undegraded":"2024-09-13T17:17:45.009487+0000","last_fullsized":"2024-09-13T17:17:45.009487+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":28065,"num_write_kb":366501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"63'3806","reported_seq":3871,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.053855+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T17:17:14.053855+0000","last_peered":"2024-09-13T17:17:14.053855+0000","last_clean":"2024-09-13T17:17:14.053855+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T17:17:14.053855+0000","last_undegraded":"2024-09-13T17:17:14.053855+0000","last_fullsized":"2024-09-13T17:17:14.053855+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:32.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4,"num_write":20286,"num_write_kb":286237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"63'3659","reported_seq":3727,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126457+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T17:17:22.126457+0000","last_peered":"2024-09-13T17:17:22.126457+0000","last_clean":"2024-09-13T17:17:22.126457+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T17:17:22.126457+0000","last_undegraded":"2024-09-13T17:17:22.126457+0000","last_fullsized":"2024-09-13T17:17:22.126457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19936,"num_write_kb":264332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"63'5003","reported_seq":5065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.034159+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T17:17:14.034159+0000","last_peered":"2024-09-13T17:17:14.034159+0000","last_clean":"2024-09-13T17:17:14.034159+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T17:17:14.034159+0000","last_undegraded":"2024-09-13T17:17:14.034159+0000","last_fullsized":"2024-09-13T17:17:14.034159+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27137,"num_write_kb":387189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"63'4068","reported_seq":4135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.114101+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T17:17:22.114101+0000","last_peered":"2024-09-13T17:17:22.114101+0000","last_clean":"2024-09-13T17:17:22.114101+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T17:17:22.114101+0000","last_undegraded":"2024-09-13T17:17:22.114101+0000","last_fullsized":"2024-09-13T17:17:22.114101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22149,"num_write_kb":293545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"63'3283","reported_seq":3347,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007911+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T17:17:45.007911+0000","last_peered":"2024-09-13T17:17:45.007911+0000","last_clean":"2024-09-13T17:17:45.007911+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T17:17:45.007911+0000","last_undegraded":"2024-09-13T17:17:45.007911+0000","last_fullsized":"2024-09-13T17:17:45.007911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":248866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"63'4697","reported_seq":4764,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.065666+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T17:17:22.065666+0000","last_peered":"2024-09-13T17:17:22.065666+0000","last_clean":"2024-09-13T17:17:22.065666+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T17:17:22.065666+0000","last_undegraded":"2024-09-13T17:17:22.065666+0000","last_fullsized":"2024-09-13T17:17:22.065666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:48:20.237160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25800,"num_write_kb":320606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"63'3820","reported_seq":3883,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:34.004243+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T17:17:34.004243+0000","last_peered":"2024-09-13T17:17:34.004243+0000","last_clean":"2024-09-13T17:17:34.004243+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T17:17:34.004243+0000","last_undegraded":"2024-09-13T17:17:34.004243+0000","last_fullsized":"2024-09-13T17:17:34.004243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3820,"log_dups_size":0,"ondisk_log_size":3820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:00.970805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20813,"num_write_kb":300503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"63'4719","reported_seq":4787,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.117962+0000","last_change":"2024-09-13T16:47:59.049460+0000","last_active":"2024-09-13T17:17:22.117962+0000","last_peered":"2024-09-13T17:17:22.117962+0000","last_clean":"2024-09-13T17:17:22.117962+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T17:17:22.117962+0000","last_undegraded":"2024-09-13T17:17:22.117962+0000","last_fullsized":"2024-09-13T17:17:22.117962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:19.837659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":25263,"num_write_kb":330973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"63'3722","reported_seq":3784,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.001243+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T17:17:14.001243+0000","last_peered":"2024-09-13T17:17:14.001243+0000","last_clean":"2024-09-13T17:17:14.001243+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T17:17:14.001243+0000","last_undegraded":"2024-09-13T17:17:14.001243+0000","last_fullsized":"2024-09-13T17:17:14.001243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:54.091792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20022,"num_write_kb":289122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"63'3813","reported_seq":3880,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.128440+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T17:17:22.128440+0000","last_peered":"2024-09-13T17:17:22.128440+0000","last_clean":"2024-09-13T17:17:22.128440+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T17:17:22.128440+0000","last_undegraded":"2024-09-13T17:17:22.128440+0000","last_fullsized":"2024-09-13T17:17:22.128440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:59:26.367172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21202,"num_write_kb":265282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"63'4479","reported_seq":4545,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.060514+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T17:17:14.060514+0000","last_peered":"2024-09-13T17:17:14.060514+0000","last_clean":"2024-09-13T17:17:14.060514+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T17:17:14.060514+0000","last_undegraded":"2024-09-13T17:17:14.060514+0000","last_fullsized":"2024-09-13T17:17:14.060514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":24556,"num_write_kb":340437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"63'4549","reported_seq":4615,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058661+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T17:17:22.058661+0000","last_peered":"2024-09-13T17:17:22.058661+0000","last_clean":"2024-09-13T17:17:22.058661+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T17:17:22.058661+0000","last_undegraded":"2024-09-13T17:17:22.058661+0000","last_fullsized":"2024-09-13T17:17:22.058661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:35.954510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24549,"num_write_kb":319591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"63'3479","reported_seq":3543,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.033581+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T17:17:14.033581+0000","last_peered":"2024-09-13T17:17:14.033581+0000","last_clean":"2024-09-13T17:17:14.033581+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T17:17:14.033581+0000","last_undegraded":"2024-09-13T17:17:14.033581+0000","last_fullsized":"2024-09-13T17:17:14.033581+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3479,"log_dups_size":0,"ondisk_log_size":3479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18966,"num_write_kb":268432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"63'3573","reported_seq":3641,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009368+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T17:17:45.009368+0000","last_peered":"2024-09-13T17:17:45.009368+0000","last_clean":"2024-09-13T17:17:45.009368+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T17:17:45.009368+0000","last_undegraded":"2024-09-13T17:17:45.009368+0000","last_fullsized":"2024-09-13T17:17:45.009368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19348,"num_write_kb":270113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"63'4058","reported_seq":4120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.036359+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T17:17:14.036359+0000","last_peered":"2024-09-13T17:17:14.036359+0000","last_clean":"2024-09-13T17:17:14.036359+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T17:17:14.036359+0000","last_undegraded":"2024-09-13T17:17:14.036359+0000","last_fullsized":"2024-09-13T17:17:14.036359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:05:22.529737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22331,"num_write_kb":302585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"63'5839","reported_seq":5905,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.136431+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T17:17:22.136431+0000","last_peered":"2024-09-13T17:17:22.136431+0000","last_clean":"2024-09-13T17:17:22.136431+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T17:17:22.136431+0000","last_undegraded":"2024-09-13T17:17:22.136431+0000","last_fullsized":"2024-09-13T17:17:22.136431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31756,"num_write_kb":397521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"63'3835","reported_seq":3897,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.035326+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T17:17:14.035326+0000","last_peered":"2024-09-13T17:17:14.035326+0000","last_clean":"2024-09-13T17:17:14.035326+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T17:17:14.035326+0000","last_undegraded":"2024-09-13T17:17:14.035326+0000","last_fullsized":"2024-09-13T17:17:14.035326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:55.629779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":21185,"num_write_kb":306173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"63'4134","reported_seq":4200,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.096728+0000","last_change":"2024-09-13T16:47:59.047097+0000","last_active":"2024-09-13T17:17:22.096728+0000","last_peered":"2024-09-13T17:17:22.096728+0000","last_clean":"2024-09-13T17:17:22.096728+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T17:17:22.096728+0000","last_undegraded":"2024-09-13T17:17:22.096728+0000","last_fullsized":"2024-09-13T17:17:22.096728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:30.192493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22268,"num_write_kb":281801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"63'5073","reported_seq":5136,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058407+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T17:17:14.058407+0000","last_peered":"2024-09-13T17:17:14.058407+0000","last_clean":"2024-09-13T17:17:14.058407+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T17:17:14.058407+0000","last_undegraded":"2024-09-13T17:17:14.058407+0000","last_fullsized":"2024-09-13T17:17:14.058407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:25.676241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":28693,"num_write_kb":393724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"63'3822","reported_seq":3887,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.107201+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T17:17:22.107201+0000","last_peered":"2024-09-13T17:17:22.107201+0000","last_clean":"2024-09-13T17:17:22.107201+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T17:17:22.107201+0000","last_undegraded":"2024-09-13T17:17:22.107201+0000","last_fullsized":"2024-09-13T17:17:22.107201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:03.478606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20823,"num_write_kb":275830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"63'2778","reported_seq":2843,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.258529+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T17:17:14.258529+0000","last_peered":"2024-09-13T17:17:14.258529+0000","last_clean":"2024-09-13T17:17:14.258529+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T17:17:14.258529+0000","last_undegraded":"2024-09-13T17:17:14.258529+0000","last_fullsized":"2024-09-13T17:17:14.258529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":34,"num_write":15212,"num_write_kb":208573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"63'3975","reported_seq":4040,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.131512+0000","last_change":"2024-09-13T16:47:59.049072+0000","last_active":"2024-09-13T17:17:22.131512+0000","last_peered":"2024-09-13T17:17:22.131512+0000","last_clean":"2024-09-13T17:17:22.131512+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T17:17:22.131512+0000","last_undegraded":"2024-09-13T17:17:22.131512+0000","last_fullsized":"2024-09-13T17:17:22.131512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3975,"log_dups_size":0,"ondisk_log_size":3975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:58.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21351,"num_write_kb":278200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"63'3844","reported_seq":3907,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.966555+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T17:17:13.966555+0000","last_peered":"2024-09-13T17:17:13.966555+0000","last_clean":"2024-09-13T17:17:13.966555+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T17:17:13.966555+0000","last_undegraded":"2024-09-13T17:17:13.966555+0000","last_fullsized":"2024-09-13T17:17:13.966555+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:17.974800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21085,"num_write_kb":289508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"63'3324","reported_seq":3390,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.115125+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T17:17:22.115125+0000","last_peered":"2024-09-13T17:17:22.115125+0000","last_clean":"2024-09-13T17:17:22.115125+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T17:17:22.115125+0000","last_undegraded":"2024-09-13T17:17:22.115125+0000","last_fullsized":"2024-09-13T17:17:22.115125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3324,"log_dups_size":0,"ondisk_log_size":3324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:15.311191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18810,"num_write_kb":235312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"63'3743","reported_seq":3806,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.057831+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T17:17:14.057831+0000","last_peered":"2024-09-13T17:17:14.057831+0000","last_clean":"2024-09-13T17:17:14.057831+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T17:17:14.057831+0000","last_undegraded":"2024-09-13T17:17:14.057831+0000","last_fullsized":"2024-09-13T17:17:14.057831+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:39.768635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20790,"num_write_kb":280180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"63'3173","reported_seq":3238,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.083735+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T17:17:22.083735+0000","last_peered":"2024-09-13T17:17:22.083735+0000","last_clean":"2024-09-13T17:17:22.083735+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T17:17:22.083735+0000","last_undegraded":"2024-09-13T17:17:22.083735+0000","last_fullsized":"2024-09-13T17:17:22.083735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:03.930733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17044,"num_write_kb":219853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"63'3460","reported_seq":3524,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007456+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T17:17:45.007456+0000","last_peered":"2024-09-13T17:17:45.007456+0000","last_clean":"2024-09-13T17:17:45.007456+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T17:17:45.007456+0000","last_undegraded":"2024-09-13T17:17:45.007456+0000","last_fullsized":"2024-09-13T17:17:45.007456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":19788,"num_write_kb":263976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"63'3747","reported_seq":3813,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.058102+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T17:17:22.058102+0000","last_peered":"2024-09-13T17:17:22.058102+0000","last_clean":"2024-09-13T17:17:22.058102+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T17:17:22.058102+0000","last_undegraded":"2024-09-13T17:17:22.058102+0000","last_fullsized":"2024-09-13T17:17:22.058102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20231,"num_write_kb":269532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"63'3422","reported_seq":3483,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.003363+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T17:17:14.003363+0000","last_peered":"2024-09-13T17:17:14.003363+0000","last_clean":"2024-09-13T17:17:14.003363+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T17:17:14.003363+0000","last_undegraded":"2024-09-13T17:17:14.003363+0000","last_fullsized":"2024-09-13T17:17:14.003363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18722,"num_write_kb":267617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"63'4803","reported_seq":4869,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.084927+0000","last_change":"2024-09-13T16:47:59.054527+0000","last_active":"2024-09-13T17:17:22.084927+0000","last_peered":"2024-09-13T17:17:22.084927+0000","last_clean":"2024-09-13T17:17:22.084927+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T17:17:22.084927+0000","last_undegraded":"2024-09-13T17:17:22.084927+0000","last_fullsized":"2024-09-13T17:17:22.084927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.607144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26967,"num_write_kb":333429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"63'3840","reported_seq":3902,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.058273+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T17:17:14.058273+0000","last_peered":"2024-09-13T17:17:14.058273+0000","last_clean":"2024-09-13T17:17:14.058273+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T17:17:14.058273+0000","last_undegraded":"2024-09-13T17:17:14.058273+0000","last_fullsized":"2024-09-13T17:17:14.058273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:44:41.231128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21113,"num_write_kb":293770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"63'5433","reported_seq":5500,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.040459+0000","last_change":"2024-09-13T16:47:59.062325+0000","last_active":"2024-09-13T17:17:22.040459+0000","last_peered":"2024-09-13T17:17:22.040459+0000","last_clean":"2024-09-13T17:17:22.040459+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T17:17:22.040459+0000","last_undegraded":"2024-09-13T17:17:22.040459+0000","last_fullsized":"2024-09-13T17:17:22.040459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30664,"num_write_kb":389336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"63'4741","reported_seq":4807,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:24.002397+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T17:17:24.002397+0000","last_peered":"2024-09-13T17:17:24.002397+0000","last_clean":"2024-09-13T17:17:24.002397+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T17:17:24.002397+0000","last_undegraded":"2024-09-13T17:17:24.002397+0000","last_fullsized":"2024-09-13T17:17:24.002397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":15,"num_write":25588,"num_write_kb":340391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"63'4281","reported_seq":4346,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.111327+0000","last_change":"2024-09-13T16:47:59.062633+0000","last_active":"2024-09-13T17:17:22.111327+0000","last_peered":"2024-09-13T17:17:22.111327+0000","last_clean":"2024-09-13T17:17:22.111327+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T17:17:22.111327+0000","last_undegraded":"2024-09-13T17:17:22.111327+0000","last_fullsized":"2024-09-13T17:17:22.111327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23373,"num_write_kb":302635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"63'3575","reported_seq":3642,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:23.119184+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T17:17:23.119184+0000","last_peered":"2024-09-13T17:17:23.119184+0000","last_clean":"2024-09-13T17:17:23.119184+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T17:17:23.119184+0000","last_undegraded":"2024-09-13T17:17:23.119184+0000","last_fullsized":"2024-09-13T17:17:23.119184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.300009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":7,"num_write":20117,"num_write_kb":289677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"63'4404","reported_seq":4468,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.129331+0000","last_change":"2024-09-13T16:47:59.049813+0000","last_active":"2024-09-13T17:17:22.129331+0000","last_peered":"2024-09-13T17:17:22.129331+0000","last_clean":"2024-09-13T17:17:22.129331+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T17:17:22.129331+0000","last_undegraded":"2024-09-13T17:17:22.129331+0000","last_fullsized":"2024-09-13T17:17:22.129331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:07.341759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24308,"num_write_kb":306423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"63'3318","reported_seq":3382,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.949311+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T17:17:13.949311+0000","last_peered":"2024-09-13T17:17:13.949311+0000","last_clean":"2024-09-13T17:17:13.949311+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T17:17:13.949311+0000","last_undegraded":"2024-09-13T17:17:13.949311+0000","last_fullsized":"2024-09-13T17:17:13.949311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3318,"log_dups_size":0,"ondisk_log_size":3318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:38.080640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18295,"num_write_kb":253537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"63'4013","reported_seq":4078,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.136986+0000","last_change":"2024-09-13T16:47:59.049023+0000","last_active":"2024-09-13T17:17:22.136986+0000","last_peered":"2024-09-13T17:17:22.136986+0000","last_clean":"2024-09-13T17:17:22.136986+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T17:17:22.136986+0000","last_undegraded":"2024-09-13T17:17:22.136986+0000","last_fullsized":"2024-09-13T17:17:22.136986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:17.996124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21393,"num_write_kb":262042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"63'3658","reported_seq":3723,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:19.266963+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T17:17:19.266963+0000","last_peered":"2024-09-13T17:17:19.266963+0000","last_clean":"2024-09-13T17:17:19.266963+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T17:17:19.266963+0000","last_undegraded":"2024-09-13T17:17:19.266963+0000","last_fullsized":"2024-09-13T17:17:19.266963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:42.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19946,"num_write_kb":268446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"63'3910","reported_seq":3974,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.108541+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T17:17:22.108541+0000","last_peered":"2024-09-13T17:17:22.108541+0000","last_clean":"2024-09-13T17:17:22.108541+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T17:17:22.108541+0000","last_undegraded":"2024-09-13T17:17:22.108541+0000","last_fullsized":"2024-09-13T17:17:22.108541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3910,"log_dups_size":0,"ondisk_log_size":3910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:50.928997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":278651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"63'3074","reported_seq":3135,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.040436+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T17:17:14.040436+0000","last_peered":"2024-09-13T17:17:14.040436+0000","last_clean":"2024-09-13T17:17:14.040436+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T17:17:14.040436+0000","last_undegraded":"2024-09-13T17:17:14.040436+0000","last_fullsized":"2024-09-13T17:17:14.040436+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:08.508386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":246226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"63'3515","reported_seq":3584,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.072919+0000","last_change":"2024-09-13T16:47:59.060992+0000","last_active":"2024-09-13T17:17:22.072919+0000","last_peered":"2024-09-13T17:17:22.072919+0000","last_clean":"2024-09-13T17:17:22.072919+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T17:17:22.072919+0000","last_undegraded":"2024-09-13T17:17:22.072919+0000","last_fullsized":"2024-09-13T17:17:22.072919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:49.926833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19512,"num_write_kb":258445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"63'3439","reported_seq":3502,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.006251+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T17:17:45.006251+0000","last_peered":"2024-09-13T17:17:45.006251+0000","last_clean":"2024-09-13T17:17:45.006251+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T17:17:45.006251+0000","last_undegraded":"2024-09-13T17:17:45.006251+0000","last_fullsized":"2024-09-13T17:17:45.006251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":17681,"num_write_kb":263603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"63'3892","reported_seq":3960,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.067970+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T17:17:22.067970+0000","last_peered":"2024-09-13T17:17:22.067970+0000","last_clean":"2024-09-13T17:17:22.067970+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T17:17:22.067970+0000","last_undegraded":"2024-09-13T17:17:22.067970+0000","last_fullsized":"2024-09-13T17:17:22.067970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:14.188745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21304,"num_write_kb":286190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"63'3622","reported_seq":3682,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:13.941100+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T17:17:13.941100+0000","last_peered":"2024-09-13T17:17:13.941100+0000","last_clean":"2024-09-13T17:17:13.941100+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T17:17:13.941100+0000","last_undegraded":"2024-09-13T17:17:13.941100+0000","last_fullsized":"2024-09-13T17:17:13.941100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:04:23.094129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19729,"num_write_kb":275691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"63'5036","reported_seq":5103,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.072290+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T17:17:22.072290+0000","last_peered":"2024-09-13T17:17:22.072290+0000","last_clean":"2024-09-13T17:17:22.072290+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T17:17:22.072290+0000","last_undegraded":"2024-09-13T17:17:22.072290+0000","last_fullsized":"2024-09-13T17:17:22.072290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:35.269895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27469,"num_write_kb":368239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"63'3689","reported_seq":3749,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:14.059558+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T17:17:14.059558+0000","last_peered":"2024-09-13T17:17:14.059558+0000","last_clean":"2024-09-13T17:17:14.059558+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T17:17:14.059558+0000","last_undegraded":"2024-09-13T17:17:14.059558+0000","last_fullsized":"2024-09-13T17:17:14.059558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3689,"log_dups_size":0,"ondisk_log_size":3689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:29.369320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20818,"num_write_kb":284187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"63'4047","reported_seq":4115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T17:17:22.126604+0000","last_change":"2024-09-13T16:47:59.055866+0000","last_active":"2024-09-13T17:17:22.126604+0000","last_peered":"2024-09-13T17:17:22.126604+0000","last_clean":"2024-09-13T17:17:22.126604+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T17:17:22.126604+0000","last_undegraded":"2024-09-13T17:17:22.126604+0000","last_fullsized":"2024-09-13T17:17:22.126604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:18:59.953899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22743,"num_write_kb":291925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":197,"num_write":1329762,"num_write_kb":18444058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242925,"ondisk_log_size":242925,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19024977,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1583,"num_read_kb":206738,"num_write":4472,"num_write_kb":20527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38109184,"data_stored":38049954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4352,"ondisk_log_size":4352,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":1479870,"num_write_kb":19043750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270762,"ondisk_log_size":270762,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":19653574,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1322,"num_read_kb":201394,"num_write":4502,"num_write_kb":21129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39370752,"data_stored":39307047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4337,"ondisk_log_size":4337,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575212,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":942768,"kb_used_data":13672,"kb_used_omap":1,"kb_used_meta":929086,"kb_avail":93429072,"statfs":{"total":96636764160,"available":95671369728,"internally_reserved":0,"allocated":14000128,"data_stored":13606602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":951384523},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.1059999999999999,"5min":1.9890000000000001,"15min":1.212},"min":{"1min":0.24299999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":8.0960000000000001,"5min":16.571999999999999,"15min":16.571999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":2.3239999999999998,"5min":2.1040000000000001,"15min":1.3080000000000001},"min":{"1min":0.26600000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":9.7799999999999994,"5min":16.523,"15min":16.523},"last":1.1359999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":5.859,"5min":2.754,"15min":1.486},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":59.911999999999999,"5min":59.911999999999999,"15min":59.911999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":5.8680000000000003,"5min":2.8519999999999999,"15min":1.5660000000000001},"min":{"1min":0.28699999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":59.895000000000003,"5min":59.895000000000003,"15min":59.895000000000003},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.113,"5min":1.6659999999999999,"15min":1.1040000000000001},"min":{"1min":0.28999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.6869999999999994,"5min":16.023,"15min":16.023},"last":0.93999999999999995},{"interface":"front","average":{"1min":2.173,"5min":1.659,"15min":1.1559999999999999},"min":{"1min":0.28299999999999997,"5min":0.249,"15min":0.249},"max":{"1min":9.5619999999999994,"5min":10.654999999999999,"15min":10.654999999999999},"last":0.82199999999999995}]},{"osd":3,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.2200000000000002,"5min":1.944,"15min":1.2210000000000001},"min":{"1min":0.30199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":9.7370000000000001,"5min":23.225000000000001,"15min":23.225000000000001},"last":1.093},{"interface":"front","average":{"1min":2.2959999999999998,"5min":1.944,"15min":1.2789999999999999},"min":{"1min":0.35699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":8.891,"5min":12.045999999999999,"15min":12.045999999999999},"last":0.98799999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":2.0659999999999998,"15min":1.274},"min":{"1min":0.25900000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":14.209,"5min":18.004999999999999,"15min":18.004999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":3.9510000000000001,"5min":2.339,"15min":1.391},"min":{"1min":0.28299999999999997,"5min":0.245,"15min":0.245},"max":{"1min":22.315999999999999,"5min":22.315999999999999,"15min":22.315999999999999},"last":1.1919999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":4.9809999999999999,"5min":2.2959999999999998,"15min":1.419},"min":{"1min":0.32200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":37.302999999999997,"5min":37.302999999999997,"15min":37.302999999999997},"last":1.0620000000000001},{"interface":"front","average":{"1min":2.6560000000000001,"5min":1.8100000000000001,"15min":1.27},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.316},"max":{"1min":16.867999999999999,"5min":16.867999999999999,"15min":16.867999999999999},"last":1.2290000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:16:45 2024","interfaces":[{"interface":"back","average":{"1min":2.2610000000000001,"5min":1.788,"15min":1.292},"min":{"1min":0.38100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":9.1660000000000004,"5min":17.968,"15min":17.968},"last":1.2989999999999999},{"interface":"front","average":{"1min":3.29,"5min":1.9910000000000001,"15min":1.3500000000000001},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.025,"5min":19.244,"15min":19.244},"last":1.2749999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575210,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673200,"kb_used_data":11748,"kb_used_omap":1,"kb_used_meta":661438,"kb_avail":93698640,"statfs":{"total":96636764160,"available":95947407360,"internally_reserved":0,"allocated":12029952,"data_stored":11623151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":677312968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.0700000000000001,"15min":0.89000000000000001},"min":{"1min":0.27500000000000002,"5min":0.222,"15min":0.222},"max":{"1min":6.0780000000000003,"5min":6.335,"15min":6.335},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.052,"5min":1.133,"15min":0.91200000000000003},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":4.907,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.80500000000000005}]},{"osd":1,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":1.198,"15min":0.95099999999999996},"min":{"1min":0.24299999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.474,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":1.736,"5min":1.6850000000000001,"15min":1.0980000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":7.3600000000000003,"5min":10.111000000000001,"15min":10.111000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.234,"15min":0.97799999999999998},"min":{"1min":0.432,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":15.638,"5min":15.638,"15min":15.638},"last":0.76400000000000001},{"interface":"front","average":{"1min":2.1960000000000002,"5min":1.4450000000000001,"15min":1.091},"min":{"1min":0.41999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":19.442,"5min":19.442,"15min":19.442},"last":0.83199999999999996}]},{"osd":3,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.5840000000000001,"15min":1.1259999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":3.7389999999999999,"5min":15.247999999999999,"15min":15.247999999999999},"last":0.873},{"interface":"front","average":{"1min":1.413,"5min":1.6399999999999999,"15min":1.161},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.1379999999999999,"5min":15.179,"15min":15.179},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":1.4810000000000001,"15min":1.1160000000000001},"min":{"1min":0.35299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":14.114000000000001,"5min":15.326000000000001,"15min":15.326000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.8799999999999999,"5min":1.6619999999999999,"15min":1.1930000000000001},"min":{"1min":0.27300000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":14.057,"5min":15.026,"15min":15.026},"last":0.95799999999999996}]},{"osd":5,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.6659999999999999,"15min":1.242},"min":{"1min":0.42699999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.2690000000000001,"5min":11.566000000000001,"15min":11.566000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.4570000000000001,"15min":1.1879999999999999},"min":{"1min":0.36199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":6.2789999999999999,"5min":10.223000000000001,"15min":10.223000000000001},"last":1.012}]},{"osd":7,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.6499999999999999,"15min":1.238},"min":{"1min":0.318,"5min":0.311,"15min":0.311},"max":{"1min":4.0979999999999999,"5min":13.112,"15min":13.112},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.782,"15min":1.274},"min":{"1min":0.39700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.6679999999999999,"5min":14.4,"15min":14.4},"last":1.0389999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575211,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":957792,"kb_used_data":5012,"kb_used_omap":1,"kb_used_meta":952766,"kb_avail":93414048,"statfs":{"total":96636764160,"available":95655985152,"internally_reserved":0,"allocated":5132288,"data_stored":4730805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":975632838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":3.762,"5min":1.978,"15min":1.1919999999999999},"min":{"1min":0.246,"5min":0.215,"15min":0.215},"max":{"1min":27.952999999999999,"5min":27.952999999999999,"15min":27.952999999999999},"last":1.024},{"interface":"front","average":{"1min":2.5059999999999998,"5min":1.629,"15min":1.121},"min":{"1min":0.28100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":23.619,"5min":23.619,"15min":23.619},"last":0.84199999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.8210000000000002,"5min":1.8109999999999999,"15min":1.1379999999999999},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":13.515000000000001,"5min":13.515000000000001,"15min":13.515000000000001},"last":1.095},{"interface":"front","average":{"1min":1.621,"5min":1.9299999999999999,"15min":1.1839999999999999},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":6.9290000000000003,"5min":35.186999999999998,"15min":35.186999999999998},"last":0.94699999999999995}]},{"osd":2,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":2.306,"15min":1.3420000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.609,"5min":51.18,"15min":51.18},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.732,"5min":1.929,"15min":1.2949999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.8659999999999997,"5min":48.756,"15min":48.756},"last":0.97899999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.7480000000000002,"5min":2.0219999999999998,"15min":1.2789999999999999},"min":{"1min":0.39200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":15.946999999999999,"5min":19.800000000000001,"15min":19.800000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.7649999999999999,"15min":1.256},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":9.1020000000000003,"5min":9.4789999999999992,"15min":9.4789999999999992},"last":1.173}]},{"osd":4,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":4.2270000000000003,"5min":2.4260000000000002,"15min":1.379},"min":{"1min":0.40699999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":20.713999999999999,"5min":22.361999999999998,"15min":22.361999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":4.1710000000000003,"5min":2.9940000000000002,"15min":1.641},"min":{"1min":0.35299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":28.062999999999999,"5min":31.843,"15min":31.843},"last":1.2210000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.427,"15min":1.131},"min":{"1min":0.375,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.3300000000000001,"5min":8.1910000000000007,"15min":8.1910000000000007},"last":1.268},{"interface":"front","average":{"1min":2.052,"5min":1.7649999999999999,"15min":1.26},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.1059999999999999,"5min":13.031000000000001,"15min":13.031000000000001},"last":1.1299999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":2.1890000000000001,"15min":1.4139999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":11.239000000000001,"5min":35.124000000000002,"15min":35.124000000000002},"last":1.056},{"interface":"front","average":{"1min":2.0550000000000002,"5min":2.5089999999999999,"15min":1.546},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.99,"5min":32.328000000000003,"15min":32.328000000000003},"last":1.3200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575210,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1011316,"kb_used_data":13932,"kb_used_omap":1,"kb_used_meta":997374,"kb_avail":93360524,"statfs":{"total":96636764160,"available":95601176576,"internally_reserved":0,"allocated":14266368,"data_stored":13858021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1021311430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.4910000000000001,"15min":1.0369999999999999},"min":{"1min":0.25,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":13.18,"5min":13.18,"15min":13.18},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.829,"5min":1.575,"15min":1.141},"min":{"1min":0.25700000000000001,"5min":0.251,"15min":0.251},"max":{"1min":13.163,"5min":13.163,"15min":13.163},"last":0.97999999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":3.1299999999999999,"5min":1.73,"15min":1.1060000000000001},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":27.242000000000001,"5min":27.242000000000001,"15min":27.242000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":2.9119999999999999,"5min":1.978,"15min":1.232},"min":{"1min":0.34399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":26.545999999999999,"5min":26.545999999999999,"15min":26.545999999999999},"last":0.84199999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.577,"15min":1.1060000000000001},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":7.2619999999999996,"5min":11.803000000000001,"15min":11.803000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.573,"15min":1.131},"min":{"1min":0.35299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3319999999999999,"5min":8.6709999999999994,"15min":8.6709999999999994},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.9079999999999999,"5min":2.1899999999999999,"15min":1.343},"min":{"1min":0.32000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":10.583,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":2.875,"5min":1.9770000000000001,"15min":1.298},"min":{"1min":0.39300000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":20.468,"5min":20.468,"15min":20.468},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.4079999999999999,"15min":1.087},"min":{"1min":0.314,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.109,"5min":13.146000000000001,"15min":13.146000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.823,"5min":1.8080000000000001,"15min":1.264},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":14.202,"5min":15.044,"15min":15.044},"last":1.1950000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4359999999999999,"15min":1.1279999999999999},"min":{"1min":0.35299999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":7.2030000000000003,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.6160000000000001,"15min":1.234},"min":{"1min":0.38400000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.7320000000000002,"5min":7.5860000000000003,"15min":7.5860000000000003},"last":1.1040000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:16:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.8169999999999999,"15min":1.345},"min":{"1min":0.33500000000000002,"5min":0.313,"15min":0.313},"max":{"1min":6.3449999999999998,"5min":24.518999999999998,"15min":24.518999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.7170000000000001,"15min":1.306},"min":{"1min":0.372,"5min":0.312,"15min":0.312},"max":{"1min":9.0519999999999996,"5min":9.5950000000000006,"15min":9.5950000000000006},"last":1.1639999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575211,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":958488,"kb_used_data":19404,"kb_used_omap":1,"kb_used_meta":939070,"kb_avail":93413352,"statfs":{"total":96636764160,"available":95655272448,"internally_reserved":0,"allocated":19869696,"data_stored":19459705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":961608136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.5880000000000001,"15min":0.97999999999999998},"min":{"1min":0.19400000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":19.327999999999999,"5min":19.327999999999999,"15min":19.327999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.831,"5min":1.7549999999999999,"15min":1.0449999999999999},"min":{"1min":0.29699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":17.754000000000001,"5min":20.699999999999999,"15min":20.699999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":1.3720000000000001,"15min":0.90300000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.3340000000000001,"5min":16.100000000000001,"15min":16.100000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.3260000000000001,"15min":0.97499999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.847,"5min":10.417,"15min":10.417},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3280000000000001,"15min":0.96699999999999997},"min":{"1min":0.38800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.6509999999999998,"5min":8.5950000000000006,"15min":8.5950000000000006},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.996,"5min":1.276,"15min":0.96499999999999997},"min":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.5380000000000003,"5min":15.433999999999999,"15min":15.433999999999999},"last":0.81799999999999995}]},{"osd":4,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":3.4470000000000001,"5min":2.1099999999999999,"15min":1.3500000000000001},"min":{"1min":0.308,"5min":0.30599999999999999,"15min":0.29599999999999999},"max":{"1min":15.919,"5min":15.919,"15min":15.919},"last":0.98599999999999999},{"interface":"front","average":{"1min":2.3170000000000002,"5min":1.712,"15min":1.2110000000000001},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":9.3819999999999997,"5min":9.3819999999999997,"15min":9.3819999999999997},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.9119999999999999,"15min":1.321},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":9.4030000000000005,"5min":31.187999999999999,"15min":31.187999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":2.427,"5min":2.121,"15min":1.383},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":22.951000000000001,"5min":22.951000000000001,"15min":22.951000000000001},"last":1.1399999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.873,"15min":1.3120000000000001},"min":{"1min":0.44400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":4.202,"5min":12.567,"15min":12.567},"last":1.165},{"interface":"front","average":{"1min":1.226,"5min":2.1779999999999999,"15min":1.4730000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.7380000000000004,"5min":15.882,"15min":15.882},"last":1.008}]},{"osd":7,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":4.8339999999999996,"5min":2.4390000000000001,"15min":1.538},"min":{"1min":0.48999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":33.969000000000001,"5min":33.969000000000001,"15min":33.969000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":2.2480000000000002,"15min":1.46},"min":{"1min":0.47799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.4550000000000001,"5min":25.652999999999999,"15min":25.652999999999999},"last":1.1200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575211,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662288,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":661374,"kb_avail":93709552,"statfs":{"total":96636764160,"available":95958581248,"internally_reserved":0,"allocated":921600,"data_stored":527628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":677247430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.89500000000000002,"15min":0.70699999999999996},"min":{"1min":0.189,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":3.3319999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.4830000000000001,"15min":0.96999999999999997},"min":{"1min":0.218,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":9.2729999999999997,"5min":9.2729999999999997,"15min":9.2729999999999997},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.3069999999999999,"5min":1.766,"15min":1.0680000000000001},"min":{"1min":0.29499999999999998,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":11.319000000000001,"5min":33.579999999999998,"15min":33.579999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":2.4359999999999999,"5min":1.7509999999999999,"15min":1.089},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":14.624000000000001,"5min":23.965,"15min":23.965},"last":0.624}]},{"osd":3,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.7949999999999999,"5min":2.1779999999999999,"15min":1.266},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.245},"max":{"1min":11.837999999999999,"5min":22.760000000000002,"15min":22.760000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":2.5449999999999999,"5min":1.6499999999999999,"15min":1.093},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":10.372,"5min":10.372,"15min":10.372},"last":0.83399999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":2.1520000000000001,"5min":2.012,"15min":1.292},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":10.324999999999999,"5min":20.673999999999999,"15min":20.673999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":2.0289999999999999,"5min":1.7669999999999999,"15min":1.24},"min":{"1min":0.42199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":6.4059999999999997,"5min":16.5,"15min":16.5},"last":1.089}]},{"osd":5,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":4.601,"5min":2.5329999999999999,"15min":1.5149999999999999},"min":{"1min":0.31,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":21.722000000000001,"5min":21.722000000000001,"15min":21.722000000000001},"last":1.073},{"interface":"front","average":{"1min":3.2240000000000002,"5min":2.238,"15min":1.415},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":19.242999999999999,"5min":29.620999999999999,"15min":29.620999999999999},"last":1.1539999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.6739999999999999,"15min":1.254},"min":{"1min":0.44,"5min":0.38,"15min":0.38},"max":{"1min":5.3239999999999998,"5min":21.882999999999999,"15min":21.882999999999999},"last":1.306},{"interface":"front","average":{"1min":2.2669999999999999,"5min":2.1070000000000002,"15min":1.429},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.29,"5min":25.274999999999999,"15min":25.274999999999999},"last":1.1879999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:16:54 2024","interfaces":[{"interface":"back","average":{"1min":5.9109999999999996,"5min":3.2639999999999998,"15min":1.825},"min":{"1min":0.45700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":28.094999999999999,"5min":43.649000000000001,"15min":43.649000000000001},"last":1.056},{"interface":"front","average":{"1min":6.6379999999999999,"5min":3.1709999999999998,"15min":1.8460000000000001},"min":{"1min":0.44600000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":27.417999999999999,"5min":27.417999999999999,"15min":27.417999999999999},"last":1.1339999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575210,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798968,"kb_used_data":9008,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572872,"statfs":{"total":96636764160,"available":95818620928,"internally_reserved":0,"allocated":9224192,"data_stored":8822112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":808909254},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.8919999999999999,"5min":1.8089999999999999,"15min":1.04},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":9.3499999999999996,"5min":15.81,"15min":15.81},"last":1.177},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.7050000000000001,"15min":1.056},"min":{"1min":0.19900000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.8360000000000003,"5min":15.823,"15min":15.823},"last":1.0249999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":1.4139999999999999,"15min":0.91500000000000004},"min":{"1min":0.23699999999999999,"5min":0.193,"15min":0.193},"max":{"1min":9.4420000000000002,"5min":11.672000000000001,"15min":11.672000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.282,"5min":1.1919999999999999,"15min":0.90900000000000003},"min":{"1min":0.26000000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.3440000000000003,"5min":11.722,"15min":11.722},"last":0.751}]},{"osd":3,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3719999999999999,"5min":1.901,"15min":1.171},"min":{"1min":0.26300000000000001,"5min":0.22,"15min":0.22},"max":{"1min":10.151999999999999,"5min":16.507999999999999,"15min":16.507999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.4790000000000001,"15min":1.0029999999999999},"min":{"1min":0.25600000000000001,"5min":0.218,"15min":0.218},"max":{"1min":5.024,"5min":12.047000000000001,"15min":12.047000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.6160000000000001,"5min":2.1040000000000001,"15min":1.2649999999999999},"min":{"1min":0.28399999999999997,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":10.176,"5min":22.234000000000002,"15min":22.234000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":2.4510000000000001,"5min":2.2080000000000002,"15min":1.381},"min":{"1min":0.377,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":10.77,"5min":20.448,"15min":20.448},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.649,"15min":1.1910000000000001},"min":{"1min":0.32900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.7839999999999998,"5min":15.788,"15min":15.788},"last":0.82199999999999995},{"interface":"front","average":{"1min":2.105,"5min":1.8180000000000001,"15min":1.256},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":9.3000000000000007,"5min":11.731999999999999,"15min":11.731999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":1.7090000000000001,"5min":1.4119999999999999,"15min":1.141},"min":{"1min":0.42499999999999999,"5min":0.255,"15min":0.255},"max":{"1min":9.1760000000000002,"5min":12.019,"15min":12.019},"last":1.0960000000000001},{"interface":"front","average":{"1min":2.1760000000000002,"5min":1.768,"15min":1.258},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":9.3339999999999996,"5min":11.628,"15min":11.628},"last":1.004}]},{"osd":7,"last update":"Fri Sep 13 17:16:50 2024","interfaces":[{"interface":"back","average":{"1min":2.3809999999999998,"5min":1.9279999999999999,"15min":1.371},"min":{"1min":0.435,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":8.9689999999999994,"5min":21.161000000000001,"15min":21.161000000000001},"last":1.264},{"interface":"front","average":{"1min":4.2720000000000002,"5min":2.504,"15min":1.552},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":29.323,"5min":29.323,"15min":29.323},"last":1.1539999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575210,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799540,"kb_used_data":9580,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572300,"statfs":{"total":96636764160,"available":95818035200,"internally_reserved":0,"allocated":9809920,"data_stored":9412264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":808909256},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":1.474,"15min":0.92700000000000005},"min":{"1min":0.23300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":2.5870000000000002,"5min":59.031999999999996,"15min":59.031999999999996},"last":0.73499999999999999},{"interface":"front","average":{"1min":2.0249999999999999,"5min":2.222,"15min":1.196},"min":{"1min":0.20200000000000001,"5min":0.188,"15min":0.188},"max":{"1min":9.7110000000000003,"5min":57.186999999999998,"15min":57.186999999999998},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":0.93799999999999994,"15min":0.81200000000000006},"min":{"1min":0.28000000000000003,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":4.0919999999999996,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":0.90700000000000003},{"interface":"front","average":{"1min":2.0960000000000001,"5min":1.1899999999999999,"15min":0.84699999999999998},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":12.052,"5min":12.052,"15min":12.052},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.901,"5min":2.5739999999999998,"15min":1.353},"min":{"1min":0.26100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.6769999999999996,"5min":65.215000000000003,"15min":65.215000000000003},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.679,"15min":1.133},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.319,"5min":62.722999999999999,"15min":62.722999999999999},"last":1.002}]},{"osd":4,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":3.23,"5min":1.857,"15min":1.202},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":14.694000000000001,"5min":14.694000000000001,"15min":14.694000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.994,"5min":1.3360000000000001,"15min":1.095},"min":{"1min":0.32400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":9.3499999999999996,"5min":10.036,"15min":10.036},"last":1.0700000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":2.8300000000000001,"5min":1.748,"15min":1.1919999999999999},"min":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":21.940999999999999,"5min":21.940999999999999,"15min":21.940999999999999},"last":1.163},{"interface":"front","average":{"1min":2.1440000000000001,"5min":1.962,"15min":1.3029999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":7.0620000000000003,"5min":9.3420000000000005,"15min":9.3420000000000005},"last":1.1319999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.7410000000000001,"5min":1.242,"15min":1.119},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":9.3840000000000003,"5min":9.3840000000000003,"15min":9.3840000000000003},"last":1.1990000000000001},{"interface":"front","average":{"1min":2.012,"5min":1.458,"15min":1.161},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.7060000000000004,"5min":8.157,"15min":8.157},"last":1.234}]},{"osd":7,"last update":"Fri Sep 13 17:16:44 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6140000000000001,"15min":1.2589999999999999},"min":{"1min":0.40500000000000003,"5min":0.373,"15min":0.373},"max":{"1min":5.3209999999999997,"5min":8.1430000000000007,"15min":8.1430000000000007},"last":1.274},{"interface":"front","average":{"1min":2.601,"5min":2.677,"15min":1.5920000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40300000000000002},"max":{"1min":8.7210000000000001,"5min":32.069000000000003,"15min":32.069000000000003},"last":1.3180000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":84646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8482816,"data_stored":8473825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7000064,"data_stored":6984672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6995968,"data_stored":6985218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10542867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4296704,"data_stored":4287702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:17:50.006 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.007 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.008 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.009 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.010 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.010 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.010 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.010 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.010 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.010 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.010 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.010 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.010 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.011 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.011 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.011 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.011 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.011 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.011 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.011 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.011 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.011 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.012 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.013 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.013 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.013 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.013 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.013 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T16:47:47.706385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=47, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.013 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.013 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.014 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.014 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.014 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.014 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.014 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.014 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.014 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.014 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.014 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.015 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.015 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.015 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.015 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.015 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.015 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.015 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.015 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.015 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.016 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.016 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.016 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.016 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.016 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.016 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.016 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.016 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.016 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.017 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.017 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.017 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.017 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.017 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.017 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.017 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.017 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.017 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.018 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.018 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.018 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.018 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.018 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.018 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.018 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.018 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.019 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.019 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.019 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.019 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.019 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.019 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.019 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.019 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.019 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.020 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.020 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.020 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.020 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.020 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.020 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.020 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.020 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.020 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.021 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.021 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.021 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.021 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.021 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.021 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.021 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.021 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.021 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.022 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.022 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.022 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.022 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.022 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.022 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.022 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.022 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.023 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.023 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.023 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.023 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.023 INFO:tasks.ceph:pgid 5.2c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.023 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.023 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.023 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.023 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.024 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.024 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.024 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.024 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.024 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.024 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.024 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.024 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.025 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.025 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.025 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.025 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.025 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.025 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.025 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.025 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.025 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.026 INFO:tasks.ceph:pgid 5.33 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.026 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.026 INFO:tasks.ceph:pgid 5.30 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.026 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.026 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.026 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.026 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.026 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.026 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.027 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.027 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.027 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.027 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.027 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.027 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.027 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.027 INFO:tasks.ceph:pgid 5.3b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.027 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.028 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.028 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.028 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.028 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:17:50.028 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T17:17:51.324 INFO:tasks.ceph.mds.d.smithi084.stderr:2024-09-13T17:17:51.320+0000 7f7f5de86640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:17:51.589 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:17:51.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.721+0000 7fc57f01a640 1 Processor -- start 2024-09-13T17:17:51.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.721+0000 7fc57f01a640 1 -- start start 2024-09-13T17:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57f01a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5780c1790 0x7fc578157f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57f01a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc578157370 con 0x7fc57806d370 2024-09-13T17:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57f01a640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc578153df0 con 0x7fc5780c20d0 2024-09-13T17:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57f01a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780c04a0 con 0x7fc5780c1790 2024-09-13T17:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57d5b7640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc57806d370 0x7fc57815e700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:48364/0 (socket says 172.21.15.39:48364) 2024-09-13T17:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57d5b7640 1 -- 172.21.15.39:0/2552736127 learned_addr learned my addr 172.21.15.39:0/2552736127 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc577fff640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5780c1790 0x7fc578157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133080853 0 0) 0x7fc578157370 con 0x7fc57806d370 2024-09-13T17:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc560003530 con 0x7fc57806d370 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1893966573 0 0) 0x7fc560003530 con 0x7fc57806d370 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5780c1790 msgr2=0x7fc578157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5780c1790 0x7fc578157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 >> v1:172.21.15.84:6789/0 conn(0x7fc5780c20d0 legacy=0x7fc578157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57806c700 con 0x7fc57806d370 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fc570001a00 con 0x7fc57806d370 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 >> v1:172.21.15.39:6789/0 conn(0x7fc57806d370 legacy=0x7fc57815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5600039b0 0x7fc560003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5780c1790 0x7fc5600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc560004d10 0x7fc560005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56000ba40 con 0x7fc5600039b0 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56000bb80 con 0x7fc560004d10 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56000bd40 con 0x7fc5780c1790 2024-09-13T17:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57cdb6640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5600039b0 0x7fc560003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc577fff640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5780c1790 0x7fc5600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57d5b7640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc560004d10 0x7fc560005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57cdb6640 1 -- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5780c1790 msgr2=0x7fc5600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57cdb6640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5780c1790 0x7fc5600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57cdb6640 1 -- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc560004d10 msgr2=0x7fc560005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57cdb6640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc560004d10 0x7fc560005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.725+0000 7fc57cdb6640 1 -- 172.21.15.39:0/2552736127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc560003530 con 0x7fc5600039b0 2024-09-13T17:17:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57cdb6640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5600039b0 0x7fc560003dd0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc568001750 tx=0x7fc56801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2dcde1bf993d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5680052d0 con 0x7fc5600039b0 2024-09-13T17:17:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5680a7050 con 0x7fc5600039b0 2024-09-13T17:17:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc5777fe640 1 -- 172.21.15.39:0/2552736127 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568003600 con 0x7fc5600039b0 2024-09-13T17:17:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 -- 172.21.15.39:0/2552736127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5600039b0 msgr2=0x7fc560003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 --2- 172.21.15.39:0/2552736127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5600039b0 0x7fc560003dd0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc568001750 tx=0x7fc56801c3b0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57d5b7640 1 -- 172.21.15.39:0/2552736127 reap_dead start 2024-09-13T17:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 -- 172.21.15.39:0/2552736127 shutdown_connections 2024-09-13T17:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 -- 172.21.15.39:0/2552736127 >> 172.21.15.39:0/2552736127 conn(0x7fc578069e10 msgr2=0x7fc57806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 -- 172.21.15.39:0/2552736127 shutdown_connections 2024-09-13T17:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 -- 172.21.15.39:0/2552736127 wait complete. 2024-09-13T17:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 Processor -- start 2024-09-13T17:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 -- start start 2024-09-13T17:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5600039b0 0x7fc5781ed300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57815cb90 0x7fc578150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5781558e0 0x7fc578155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57815ebe0 con 0x7fc57815cb90 2024-09-13T17:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57815ef30 con 0x7fc5600039b0 2024-09-13T17:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57f01a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5781ed840 con 0x7fc5781558e0 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc577fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57815cb90 0x7fc578150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc577fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57815cb90 0x7fc578150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46578/0 (socket says 172.21.15.39:46578) 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc577fff640 1 -- 172.21.15.39:0/1776040541 learned_addr learned my addr 172.21.15.39:0/1776040541 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57cdb6640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5600039b0 0x7fc5781ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc57d5b7640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5781558e0 0x7fc578155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc577fff640 1 -- 172.21.15.39:0/1776040541 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5781558e0 msgr2=0x7fc578155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc577fff640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5781558e0 0x7fc578155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc577fff640 1 -- 172.21.15.39:0/1776040541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5600039b0 msgr2=0x7fc5781ed300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc577fff640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5600039b0 0x7fc5781ed300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.729+0000 7fc577fff640 1 -- 172.21.15.39:0/1776040541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc578155170 con 0x7fc57815cb90 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc577fff640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57815cb90 0x7fc578150780 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc56c011630 tx=0x7fc56c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc5757fa640 1 -- 172.21.15.39:0/1776040541 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c015870 con 0x7fc57815cb90 2024-09-13T17:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc5757fa640 1 -- 172.21.15.39:0/1776040541 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c0159d0 con 0x7fc57815cb90 2024-09-13T17:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc578155450 con 0x7fc57815cb90 2024-09-13T17:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc5757fa640 1 -- 172.21.15.39:0/1776040541 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c015d80 con 0x7fc57815cb90 2024-09-13T17:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc578156a60 con 0x7fc57815cb90 2024-09-13T17:17:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5781555e0 con 0x7fc57815cb90 2024-09-13T17:17:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc5757fa640 1 -- 172.21.15.39:0/1776040541 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc56c00f450 con 0x7fc57815cb90 2024-09-13T17:17:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc5757fa640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5640666c0 0x7fc564068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:17:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc5757fa640 1 -- 172.21.15.39:0/1776040541 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7fc56c0858e0 con 0x7fc57815cb90 2024-09-13T17:17:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc57cdb6640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5640666c0 0x7fc564068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:17:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.733+0000 7fc57cdb6640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5640666c0 0x7fc564068b80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc5600057b0 tx=0x7fc5680011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:17:51.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.741+0000 7fc5757fa640 1 -- 172.21.15.39:0/1776040541 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc56c051b80 con 0x7fc57815cb90 2024-09-13T17:17:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.937+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc578157800 con 0x7fc57815cb90 2024-09-13T17:17:51.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc5757fa640 1 -- 172.21.15.39:0/1776040541 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) v1 ==== 95+0+5937 (secure 0 0 0) 0x7fc56c0551a0 con 0x7fc57815cb90 2024-09-13T17:17:51.943 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:17:51.943 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":207,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428801917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428801917},{"type":"v1","addr":"172.21.15.39:6835","nonce":428801917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":207}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:17:47.987831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7555":{"gid":7555,"name":"d","rank":0,"incarnation":203,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:17:51.944 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 207 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5640666c0 msgr2=0x7fc564068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5640666c0 0x7fc564068b80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc5600057b0 tx=0x7fc5680011a0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57815cb90 msgr2=0x7fc578150780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57815cb90 0x7fc578150780 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc56c011630 tx=0x7fc56c011af0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 shutdown_connections 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fc5640666c0 0x7fc564068b80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fc5781558e0 0x7fc578155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57815cb90 0x7fc578150780 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 --2- 172.21.15.39:0/1776040541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5600039b0 0x7fc5781ed300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:17:51.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 >> 172.21.15.39:0/1776040541 conn(0x7fc578069e10 msgr2=0x7fc57806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:17:51.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 shutdown_connections 2024-09-13T17:17:51.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:17:51.941+0000 7fc57f01a640 1 -- 172.21.15.39:0/1776040541 wait complete. 2024-09-13T17:17:51.964 INFO:tasks.mds_thrash.fs.[a]:mds.a reported in up:standby state 2024-09-13T17:17:51.964 INFO:tasks.mds_thrash.fs.[a]:waiting for 112 secs before thrashing 2024-09-13T17:18:06.779 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:18:06.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:18:10.029 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:18:10.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480ec38640 1 Processor -- start 2024-09-13T17:18:10.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480ec38640 1 -- start start 2024-09-13T17:18:10.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480ec38640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f48080c7960 0x7f4808155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:10.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480ec38640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f480806fa50 0x7f4808155f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480ec38640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48080c5e50 con 0x7f48080c6000 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480ec38640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480806ff50 con 0x7f480806fa50 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480ec38640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4808072000 con 0x7f48080c7960 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480d1d5640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f48080c6000 0x7f48080c2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:52348/0 (socket says 172.21.15.39:52348) 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480d1d5640 1 -- 172.21.15.39:0/2278142427 learned_addr learned my addr 172.21.15.39:0/2278142427 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480c9d4640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f48080c7960 0x7f4808155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f4807fff640 1 --2- 172.21.15.39:0/2278142427 >> v2:172.21.15.84:3300/0 conn(0x7f480806fa50 0x7f4808155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12917411 0 0) 0x7f48080c5e50 con 0x7f48080c6000 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47f8003530 con 0x7f48080c6000 2024-09-13T17:18:10.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3747785770 0 0) 0x7f47f8003530 con 0x7f48080c6000 2024-09-13T17:18:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f48080c7960 msgr2=0x7f4808155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:18:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f48080c7960 0x7f4808155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 >> v2:172.21.15.84:3300/0 conn(0x7f480806fa50 msgr2=0x7f4808155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:18:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 --2- 172.21.15.39:0/2278142427 >> v2:172.21.15.84:3300/0 conn(0x7f480806fa50 0x7f4808155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080722e0 con 0x7f48080c6000 2024-09-13T17:18:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f4800001a00 con 0x7f48080c6000 2024-09-13T17:18:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 >> v1:172.21.15.39:6789/0 conn(0x7f48080c6000 legacy=0x7f48080c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:10.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f48080c7960 0x7f47f8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47f8003f00 0x7f47f80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8004d10 0x7f47f8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480806ff50 con 0x7f47f8004d10 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4808072000 con 0x7f47f8003f00 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f800b9e0 con 0x7f48080c7960 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480d1d5640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8004d10 0x7f47f8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480c9d4640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f48080c7960 0x7f47f8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f4807fff640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47f8003f00 0x7f47f80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480d1d5640 1 -- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f48080c7960 msgr2=0x7f47f8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480d1d5640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f48080c7960 0x7f47f8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480d1d5640 1 -- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47f8003f00 msgr2=0x7f47f80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:18:10.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480d1d5640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47f8003f00 0x7f47f80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f480d1d5640 1 -- 172.21.15.39:0/2278142427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8003530 con 0x7f47f8004d10 2024-09-13T17:18:10.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.161+0000 7f4807fff640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47f8003f00 0x7f47f80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:18:10.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480d1d5640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8004d10 0x7f47f8005190 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f480000a900 tx=0x7f480001fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b75930dda3d6c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:10.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000aa6e0 con 0x7f47f8004d10 2024-09-13T17:18:10.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48000aa840 con 0x7f47f8004d10 2024-09-13T17:18:10.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f48077fe640 1 -- 172.21.15.39:0/2278142427 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000aabf0 con 0x7f47f8004d10 2024-09-13T17:18:10.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480ec38640 1 -- 172.21.15.39:0/2278142427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8004d10 msgr2=0x7f47f8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:10.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480ec38640 1 --2- 172.21.15.39:0/2278142427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47f8004d10 0x7f47f8005190 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f480000a900 tx=0x7f480001fb40 comp rx=0 tx=0).stop 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480d1d5640 1 -- 172.21.15.39:0/2278142427 reap_dead start 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480ec38640 1 -- 172.21.15.39:0/2278142427 shutdown_connections 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480ec38640 1 -- 172.21.15.39:0/2278142427 >> 172.21.15.39:0/2278142427 conn(0x7f480806c020 msgr2=0x7f480806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480ec38640 1 -- 172.21.15.39:0/2278142427 shutdown_connections 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480ec38640 1 -- 172.21.15.39:0/2278142427 wait complete. 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480ec38640 1 Processor -- start 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.165+0000 7f480ec38640 1 -- start start 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480ec38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480814c520 0x7f480806b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480ec38640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f480806b590 0x7f4808149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480ec38640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f480814a0a0 0x7f480814ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480ec38640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080ac590 con 0x7f480806b590 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480ec38640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480814cfd0 con 0x7f480814c520 2024-09-13T17:18:10.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480ec38640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48081ed370 con 0x7f480814a0a0 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f4807fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f480806b590 0x7f4808149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f4807fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f480806b590 0x7f4808149b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49088/0 (socket says 172.21.15.39:49088) 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f4807fff640 1 -- 172.21.15.39:0/2154906946 learned_addr learned my addr 172.21.15.39:0/2154906946 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480c9d4640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480814c520 0x7f480806b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480d1d5640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f480814a0a0 0x7f480814ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f4807fff640 1 -- 172.21.15.39:0/2154906946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f480814a0a0 msgr2=0x7f480814ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f4807fff640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f480814a0a0 0x7f480814ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f4807fff640 1 -- 172.21.15.39:0/2154906946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480814c520 msgr2=0x7f480806b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f4807fff640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480814c520 0x7f480806b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f4807fff640 1 -- 172.21.15.39:0/2154906946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48081ed8a0 con 0x7f480806b590 2024-09-13T17:18:10.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480c9d4640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480814c520 0x7f480806b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f4807fff640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f480806b590 0x7f4808149b60 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f47f4011590 tx=0x7f47f4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f48057fa640 1 -- 172.21.15.39:0/2154906946 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4004220 con 0x7f480806b590 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f48057fa640 1 -- 172.21.15.39:0/2154906946 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4004380 con 0x7f480806b590 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48080abaf0 con 0x7f480806b590 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f48057fa640 1 -- 172.21.15.39:0/2154906946 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4015870 con 0x7f480806b590 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48081ed5c0 con 0x7f480806b590 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48080c7d40 con 0x7f480806b590 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f48057fa640 1 -- 172.21.15.39:0/2154906946 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47f40159d0 con 0x7f480806b590 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f48057fa640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47ec0666c0 0x7f47ec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f48057fa640 1 -- 172.21.15.39:0/2154906946 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f47f4085c20 con 0x7f480806b590 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.169+0000 7f480c9d4640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47ec0666c0 0x7f47ec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:10.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.173+0000 7f480c9d4640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47ec0666c0 0x7f47ec068b80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009be0 tx=0x7f47fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:10.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.173+0000 7f48057fa640 1 -- 172.21.15.39:0/2154906946 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f47f4051eb0 con 0x7f480806b590 2024-09-13T17:18:10.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.333+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48081564a0 con 0x7f47ec0666c0 2024-09-13T17:18:10.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.353+0000 7f48057fa640 1 -- 172.21.15.39:0/2154906946 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+524440 (secure 0 0 0) 0x7f47fc014bb0 con 0x7f47ec0666c0 2024-09-13T17:18:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:18:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T17:18:10.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47ec0666c0 msgr2=0x7f47ec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:10.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47ec0666c0 0x7f47ec068b80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009be0 tx=0x7f47fc008040 comp rx=0 tx=0).stop 2024-09-13T17:18:10.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f480806b590 msgr2=0x7f4808149b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f480806b590 0x7f4808149b60 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f47f4011590 tx=0x7f47f4009f90 comp rx=0 tx=0).stop 2024-09-13T17:18:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 shutdown_connections 2024-09-13T17:18:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f47ec0666c0 0x7f47ec068b80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f480814a0a0 0x7f480814ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f480806b590 0x7f4808149b60 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 --2- 172.21.15.39:0/2154906946 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f480814c520 0x7f480806b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 >> 172.21.15.39:0/2154906946 conn(0x7f480806c020 msgr2=0x7f48080c2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 shutdown_connections 2024-09-13T17:18:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:10.361+0000 7f480ec38640 1 -- 172.21.15.39:0/2154906946 wait complete. 2024-09-13T17:18:10.385 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":967,"stamp":"2024-09-13T17:18:08.357133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39291023,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":3153,"num_read_kb":416667,"num_write":2818784,"num_write_kb":37531913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522541,"ondisk_log_size":522541,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6804448,"kb_used_data":83360,"kb_used_omap":12,"kb_used_meta":6721011,"kb_avail":748170272,"statfs":{"total":773094113280,"available":766126358528,"internally_reserved":0,"allocated":85360640,"data_stored":82126150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":6882315835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005617"},"pg_stats":[{"pgid":"5.18","version":"63'4385","reported_seq":4449,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974458+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T17:17:44.974458+0000","last_peered":"2024-09-13T17:17:44.974458+0000","last_clean":"2024-09-13T17:17:44.974458+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T17:17:44.974458+0000","last_undegraded":"2024-09-13T17:17:44.974458+0000","last_fullsized":"2024-09-13T17:17:44.974458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":23692,"num_write_kb":324330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"63'916","reported_seq":1030,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.207086+0000","last_change":"2024-09-13T16:47:58.668248+0000","last_active":"2024-09-13T17:17:46.207086+0000","last_peered":"2024-09-13T17:17:46.207086+0000","last_clean":"2024-09-13T17:17:46.207086+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T17:17:46.207086+0000","last_undegraded":"2024-09-13T17:17:46.207086+0000","last_fullsized":"2024-09-13T17:17:46.207086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":10389,"num_write":894,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'3737","reported_seq":3806,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976547+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T17:17:44.976547+0000","last_peered":"2024-09-13T17:17:44.976547+0000","last_clean":"2024-09-13T17:17:44.976547+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T17:17:44.976547+0000","last_undegraded":"2024-09-13T17:17:44.976547+0000","last_fullsized":"2024-09-13T17:17:44.976547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3737,"log_dups_size":0,"ondisk_log_size":3737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20454,"num_write_kb":270405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.082725+0000","last_change":"2024-09-13T17:17:52.082969+0000","last_active":"2024-09-13T17:17:52.082725+0000","last_peered":"2024-09-13T17:17:52.082725+0000","last_clean":"2024-09-13T17:17:52.082725+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T17:17:52.082725+0000","last_undegraded":"2024-09-13T17:17:52.082725+0000","last_fullsized":"2024-09-13T17:17:52.082725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.082672+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.023294+0000","scrub_duration":0.0034166320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"63'4331","reported_seq":4396,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992385+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T17:17:44.992385+0000","last_peered":"2024-09-13T17:17:44.992385+0000","last_clean":"2024-09-13T17:17:44.992385+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T17:17:44.992385+0000","last_undegraded":"2024-09-13T17:17:44.992385+0000","last_fullsized":"2024-09-13T17:17:44.992385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":22875,"num_write_kb":325320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"64'24","reported_seq":94,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.529478+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T17:17:45.529478+0000","last_peered":"2024-09-13T17:17:45.529478+0000","last_clean":"2024-09-13T17:17:45.529478+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T17:17:45.529478+0000","last_undegraded":"2024-09-13T17:17:45.529478+0000","last_fullsized":"2024-09-13T17:17:45.529478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"63'5559","reported_seq":5639,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:55.943224+0000","last_change":"2024-09-13T17:17:55.943470+0000","last_active":"2024-09-13T17:17:55.943224+0000","last_peered":"2024-09-13T17:17:55.943224+0000","last_clean":"2024-09-13T17:17:55.943224+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T17:17:55.943224+0000","last_undegraded":"2024-09-13T17:17:55.943224+0000","last_fullsized":"2024-09-13T17:17:55.943224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5559","last_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_deep_scrub":"63'5559","last_deep_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_clean_scrub_stamp":"2024-09-13T17:17:55.943168+0000","objects_scrubbed":0,"log_size":5559,"log_dups_size":0,"ondisk_log_size":5559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:10.227939+0000","scrub_duration":0.037150519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30370,"num_write_kb":379865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"63'2","reported_seq":66,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974552+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T17:17:44.974552+0000","last_peered":"2024-09-13T17:17:44.974552+0000","last_clean":"2024-09-13T17:17:44.974552+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T17:17:44.974552+0000","last_undegraded":"2024-09-13T17:17:44.974552+0000","last_fullsized":"2024-09-13T17:17:44.974552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"63'3581","reported_seq":3643,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992805+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T17:17:44.992805+0000","last_peered":"2024-09-13T17:17:44.992805+0000","last_clean":"2024-09-13T17:17:44.992805+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T17:17:44.992805+0000","last_undegraded":"2024-09-13T17:17:44.992805+0000","last_fullsized":"2024-09-13T17:17:44.992805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":268344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"64'43","reported_seq":149,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591097+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T17:17:49.591097+0000","last_peered":"2024-09-13T17:17:49.591097+0000","last_clean":"2024-09-13T17:17:49.591097+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T17:17:49.591097+0000","last_undegraded":"2024-09-13T17:17:49.591097+0000","last_fullsized":"2024-09-13T17:17:49.591097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'3493","reported_seq":3562,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.642622+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T17:17:48.642622+0000","last_peered":"2024-09-13T17:17:48.642622+0000","last_clean":"2024-09-13T17:17:48.642622+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T17:17:48.642622+0000","last_undegraded":"2024-09-13T17:17:48.642622+0000","last_fullsized":"2024-09-13T17:17:48.642622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18921,"num_write_kb":256078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'727","reported_seq":826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.186390+0000","last_change":"2024-09-13T17:17:57.186917+0000","last_active":"2024-09-13T17:17:57.186390+0000","last_peered":"2024-09-13T17:17:57.186390+0000","last_clean":"2024-09-13T17:17:57.186390+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T17:17:57.186390+0000","last_undegraded":"2024-09-13T17:17:57.186390+0000","last_fullsized":"2024-09-13T17:17:57.186390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'727","last_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_deep_scrub":"64'727","last_deep_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.186269+0000","objects_scrubbed":1,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:00.928968+0000","scrub_duration":0.026043038000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":75389,"num_write":714,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"63'3135","reported_seq":3197,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980133+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T17:17:44.980133+0000","last_peered":"2024-09-13T17:17:44.980133+0000","last_clean":"2024-09-13T17:17:44.980133+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T17:17:44.980133+0000","last_undegraded":"2024-09-13T17:17:44.980133+0000","last_fullsized":"2024-09-13T17:17:44.980133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3135,"log_dups_size":0,"ondisk_log_size":3135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":232447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"63'5","reported_seq":111,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.590779+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T17:17:49.590779+0000","last_peered":"2024-09-13T17:17:49.590779+0000","last_clean":"2024-09-13T17:17:49.590779+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T17:17:49.590779+0000","last_undegraded":"2024-09-13T17:17:49.590779+0000","last_fullsized":"2024-09-13T17:17:49.590779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"63'5195","reported_seq":5273,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.511870+0000","last_change":"2024-09-13T17:17:49.511870+0000","last_active":"2024-09-13T17:17:49.511870+0000","last_peered":"2024-09-13T17:17:49.511870+0000","last_clean":"2024-09-13T17:17:49.511870+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T17:17:49.511870+0000","last_undegraded":"2024-09-13T17:17:49.511870+0000","last_fullsized":"2024-09-13T17:17:49.511870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5195","last_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_deep_scrub":"63'5195","last_deep_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.511449+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:30.737466+0000","scrub_duration":0.041661353999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29369,"num_write_kb":359470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"63'210","reported_seq":274,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.132787+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T17:17:48.132787+0000","last_peered":"2024-09-13T17:17:48.132787+0000","last_clean":"2024-09-13T17:17:48.132787+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T17:17:48.132787+0000","last_undegraded":"2024-09-13T17:17:48.132787+0000","last_fullsized":"2024-09-13T17:17:48.132787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153565,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"63'3570","reported_seq":3636,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996947+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T17:17:44.996947+0000","last_peered":"2024-09-13T17:17:44.996947+0000","last_clean":"2024-09-13T17:17:44.996947+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T17:17:44.996947+0000","last_undegraded":"2024-09-13T17:17:44.996947+0000","last_fullsized":"2024-09-13T17:17:44.996947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":19362,"num_write_kb":282506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"63'4274","reported_seq":4353,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.521159+0000","last_change":"2024-09-13T17:18:01.521451+0000","last_active":"2024-09-13T17:18:01.521159+0000","last_peered":"2024-09-13T17:18:01.521159+0000","last_clean":"2024-09-13T17:18:01.521159+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T17:18:01.521159+0000","last_undegraded":"2024-09-13T17:18:01.521159+0000","last_fullsized":"2024-09-13T17:18:01.521159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4274","last_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_deep_scrub":"63'4274","last_deep_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.521104+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:51.017545+0000","scrub_duration":0.027199207999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23751,"num_write_kb":312338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"64'1419","reported_seq":1533,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.007429+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T17:17:48.007429+0000","last_peered":"2024-09-13T17:17:48.007429+0000","last_clean":"2024-09-13T17:17:48.007429+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T17:17:48.007429+0000","last_undegraded":"2024-09-13T17:17:48.007429+0000","last_fullsized":"2024-09-13T17:17:48.007429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1419,"log_dups_size":0,"ondisk_log_size":1419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":42245,"num_write":1396,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'7","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.140201+0000","last_change":"2024-09-13T17:18:01.140415+0000","last_active":"2024-09-13T17:18:01.140201+0000","last_peered":"2024-09-13T17:18:01.140201+0000","last_clean":"2024-09-13T17:18:01.140201+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T17:18:01.140201+0000","last_undegraded":"2024-09-13T17:18:01.140201+0000","last_fullsized":"2024-09-13T17:18:01.140201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'7","last_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_deep_scrub":"64'7","last_deep_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.140142+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:50.636583+0000","scrub_duration":0.0071129380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"63'4212","reported_seq":4279,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007878+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T17:17:45.007878+0000","last_peered":"2024-09-13T17:17:45.007878+0000","last_clean":"2024-09-13T17:17:45.007878+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T17:17:45.007878+0000","last_undegraded":"2024-09-13T17:17:45.007878+0000","last_fullsized":"2024-09-13T17:17:45.007878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":17,"num_write":23343,"num_write_kb":314959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"63'3296","reported_seq":3364,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975110+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T17:17:44.975110+0000","last_peered":"2024-09-13T17:17:44.975110+0000","last_clean":"2024-09-13T17:17:44.975110+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T17:17:44.975110+0000","last_undegraded":"2024-09-13T17:17:44.975110+0000","last_fullsized":"2024-09-13T17:17:44.975110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3296,"log_dups_size":0,"ondisk_log_size":3296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17758,"num_write_kb":230990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'3","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996274+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T17:17:44.996274+0000","last_peered":"2024-09-13T17:17:44.996274+0000","last_clean":"2024-09-13T17:17:44.996274+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T17:17:44.996274+0000","last_undegraded":"2024-09-13T17:17:44.996274+0000","last_fullsized":"2024-09-13T17:17:44.996274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'5","reported_seq":70,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978376+0000","last_change":"2024-09-13T16:48:07.780141+0000","last_active":"2024-09-13T17:17:44.978376+0000","last_peered":"2024-09-13T17:17:44.978376+0000","last_clean":"2024-09-13T17:17:44.978376+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T17:17:44.978376+0000","last_undegraded":"2024-09-13T17:17:44.978376+0000","last_fullsized":"2024-09-13T17:17:44.978376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"63'3371","reported_seq":3436,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980902+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T17:17:44.980902+0000","last_peered":"2024-09-13T17:17:44.980902+0000","last_clean":"2024-09-13T17:17:44.980902+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T17:17:44.980902+0000","last_undegraded":"2024-09-13T17:17:44.980902+0000","last_fullsized":"2024-09-13T17:17:44.980902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3371,"log_dups_size":0,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18156,"num_write_kb":249012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"63'3752","reported_seq":3820,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.972754+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T17:17:44.972754+0000","last_peered":"2024-09-13T17:17:44.972754+0000","last_clean":"2024-09-13T17:17:44.972754+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T17:17:44.972754+0000","last_undegraded":"2024-09-13T17:17:44.972754+0000","last_fullsized":"2024-09-13T17:17:44.972754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21127,"num_write_kb":268998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"63'2","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996182+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T17:17:44.996182+0000","last_peered":"2024-09-13T17:17:44.996182+0000","last_clean":"2024-09-13T17:17:44.996182+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T17:17:44.996182+0000","last_undegraded":"2024-09-13T17:17:44.996182+0000","last_fullsized":"2024-09-13T17:17:44.996182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"63'4","reported_seq":90,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.997489+0000","last_change":"2024-09-13T16:48:07.778348+0000","last_active":"2024-09-13T17:17:44.997489+0000","last_peered":"2024-09-13T17:17:44.997489+0000","last_clean":"2024-09-13T17:17:44.997489+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T17:17:44.997489+0000","last_undegraded":"2024-09-13T17:17:44.997489+0000","last_fullsized":"2024-09-13T17:17:44.997489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"63'4499","reported_seq":4567,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.987188+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T17:17:44.987188+0000","last_peered":"2024-09-13T17:17:44.987188+0000","last_clean":"2024-09-13T17:17:44.987188+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T17:17:44.987188+0000","last_undegraded":"2024-09-13T17:17:44.987188+0000","last_fullsized":"2024-09-13T17:17:44.987188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":24341,"num_write_kb":343418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"63'4043","reported_seq":4111,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.977039+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T17:17:44.977039+0000","last_peered":"2024-09-13T17:17:44.977039+0000","last_clean":"2024-09-13T17:17:44.977039+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T17:17:44.977039+0000","last_undegraded":"2024-09-13T17:17:44.977039+0000","last_fullsized":"2024-09-13T17:17:44.977039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22005,"num_write_kb":288910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"63'5","reported_seq":74,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992499+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T17:17:44.992499+0000","last_peered":"2024-09-13T17:17:44.992499+0000","last_clean":"2024-09-13T17:17:44.992499+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T17:17:44.992499+0000","last_undegraded":"2024-09-13T17:17:44.992499+0000","last_fullsized":"2024-09-13T17:17:44.992499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.171804+0000","last_change":"2024-09-13T17:18:04.172104+0000","last_active":"2024-09-13T17:18:04.171804+0000","last_peered":"2024-09-13T17:18:04.171804+0000","last_clean":"2024-09-13T17:18:04.171804+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T17:18:04.171804+0000","last_undegraded":"2024-09-13T17:18:04.171804+0000","last_fullsized":"2024-09-13T17:18:04.171804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.171747+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:58.830350+0000","scrub_duration":0.0065161840000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"63'3470","reported_seq":3535,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.971072+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T17:17:44.971072+0000","last_peered":"2024-09-13T17:17:44.971072+0000","last_clean":"2024-09-13T17:17:44.971072+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T17:17:44.971072+0000","last_undegraded":"2024-09-13T17:17:44.971072+0000","last_fullsized":"2024-09-13T17:17:44.971072+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19441,"num_write_kb":258288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'4844","reported_seq":4913,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.999011+0000","last_change":"2024-09-13T16:47:59.046076+0000","last_active":"2024-09-13T17:17:44.999011+0000","last_peered":"2024-09-13T17:17:44.999011+0000","last_clean":"2024-09-13T17:17:44.999011+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T17:17:44.999011+0000","last_undegraded":"2024-09-13T17:17:44.999011+0000","last_fullsized":"2024-09-13T17:17:44.999011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":26635,"num_write_kb":336207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'10","reported_seq":128,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.498236+0000","last_change":"2024-09-13T17:18:05.498454+0000","last_active":"2024-09-13T17:18:05.498236+0000","last_peered":"2024-09-13T17:18:05.498236+0000","last_clean":"2024-09-13T17:18:05.498236+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T17:18:05.498236+0000","last_undegraded":"2024-09-13T17:18:05.498236+0000","last_fullsized":"2024-09-13T17:18:05.498236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_deep_scrub":"64'10","last_deep_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.498184+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:15.438806+0000","scrub_duration":0.0057060160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'49","reported_seq":189,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.166590+0000","last_change":"2024-09-13T17:17:58.166876+0000","last_active":"2024-09-13T17:17:58.166590+0000","last_peered":"2024-09-13T17:17:58.166590+0000","last_clean":"2024-09-13T17:17:58.166590+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T17:17:58.166590+0000","last_undegraded":"2024-09-13T17:17:58.166590+0000","last_fullsized":"2024-09-13T17:17:58.166590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'49","last_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_deep_scrub":"64'49","last_deep_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.166530+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:05.921781+0000","scrub_duration":0.0073340999999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92726,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":118,"num_write":43,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"63'3635","reported_seq":3701,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974442+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T17:17:44.974442+0000","last_peered":"2024-09-13T17:17:44.974442+0000","last_clean":"2024-09-13T17:17:44.974442+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T17:17:44.974442+0000","last_undegraded":"2024-09-13T17:17:44.974442+0000","last_fullsized":"2024-09-13T17:17:44.974442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19787,"num_write_kb":259620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'3639","reported_seq":3707,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976538+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T17:17:44.976538+0000","last_peered":"2024-09-13T17:17:44.976538+0000","last_clean":"2024-09-13T17:17:44.976538+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T17:17:44.976538+0000","last_undegraded":"2024-09-13T17:17:44.976538+0000","last_fullsized":"2024-09-13T17:17:44.976538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3639,"log_dups_size":0,"ondisk_log_size":3639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20066,"num_write_kb":240267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"64'267","reported_seq":355,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.003230+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T17:17:48.003230+0000","last_peered":"2024-09-13T17:17:48.003230+0000","last_clean":"2024-09-13T17:17:48.003230+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T17:17:48.003230+0000","last_undegraded":"2024-09-13T17:17:48.003230+0000","last_fullsized":"2024-09-13T17:17:48.003230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":38,"num_read_kb":48,"num_write":636,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"63'11","reported_seq":88,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.102648+0000","last_change":"2024-09-13T17:17:50.102762+0000","last_active":"2024-09-13T17:17:50.102648+0000","last_peered":"2024-09-13T17:17:50.102648+0000","last_clean":"2024-09-13T17:17:50.102648+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T17:17:50.102648+0000","last_undegraded":"2024-09-13T17:17:50.102648+0000","last_fullsized":"2024-09-13T17:17:50.102648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_deep_scrub":"63'11","last_deep_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.102616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.736833+0000","scrub_duration":0.0039927280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"63'3190","reported_seq":3255,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993610+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T17:17:44.993610+0000","last_peered":"2024-09-13T17:17:44.993610+0000","last_clean":"2024-09-13T17:17:44.993610+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T17:17:44.993610+0000","last_undegraded":"2024-09-13T17:17:44.993610+0000","last_fullsized":"2024-09-13T17:17:44.993610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3190,"log_dups_size":0,"ondisk_log_size":3190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":254866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"63'4701","reported_seq":4770,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.998464+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T17:17:44.998464+0000","last_peered":"2024-09-13T17:17:44.998464+0000","last_clean":"2024-09-13T17:17:44.998464+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T17:17:44.998464+0000","last_undegraded":"2024-09-13T17:17:44.998464+0000","last_fullsized":"2024-09-13T17:17:44.998464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26171,"num_write_kb":338588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'2","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996573+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T17:17:44.996573+0000","last_peered":"2024-09-13T17:17:44.996573+0000","last_clean":"2024-09-13T17:17:44.996573+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T17:17:44.996573+0000","last_undegraded":"2024-09-13T17:17:44.996573+0000","last_fullsized":"2024-09-13T17:17:44.996573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"63'3","reported_seq":69,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.002286+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T17:17:45.002286+0000","last_peered":"2024-09-13T17:17:45.002286+0000","last_clean":"2024-09-13T17:17:45.002286+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T17:17:45.002286+0000","last_undegraded":"2024-09-13T17:17:45.002286+0000","last_fullsized":"2024-09-13T17:17:45.002286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"63'3859","reported_seq":3924,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.003745+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T17:17:45.003745+0000","last_peered":"2024-09-13T17:17:45.003745+0000","last_clean":"2024-09-13T17:17:45.003745+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T17:17:45.003745+0000","last_undegraded":"2024-09-13T17:17:45.003745+0000","last_fullsized":"2024-09-13T17:17:45.003745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21286,"num_write_kb":290222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"63'3876","reported_seq":3945,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.985116+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T17:17:44.985116+0000","last_peered":"2024-09-13T17:17:44.985116+0000","last_clean":"2024-09-13T17:17:44.985116+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T17:17:44.985116+0000","last_undegraded":"2024-09-13T17:17:44.985116+0000","last_fullsized":"2024-09-13T17:17:44.985116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21201,"num_write_kb":273354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"63'2","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.999079+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T17:17:44.999079+0000","last_peered":"2024-09-13T17:17:44.999079+0000","last_clean":"2024-09-13T17:17:44.999079+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T17:17:44.999079+0000","last_undegraded":"2024-09-13T17:17:44.999079+0000","last_fullsized":"2024-09-13T17:17:44.999079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'5","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.226738+0000","last_change":"2024-09-13T17:17:57.227606+0000","last_active":"2024-09-13T17:17:57.226738+0000","last_peered":"2024-09-13T17:17:57.226738+0000","last_clean":"2024-09-13T17:17:57.226738+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T17:17:57.226738+0000","last_undegraded":"2024-09-13T17:17:57.226738+0000","last_fullsized":"2024-09-13T17:17:57.226738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226662+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:46.723103+0000","scrub_duration":0.0036245560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"63'3422","reported_seq":3488,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993940+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T17:17:44.993940+0000","last_peered":"2024-09-13T17:17:44.993940+0000","last_clean":"2024-09-13T17:17:44.993940+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T17:17:44.993940+0000","last_undegraded":"2024-09-13T17:17:44.993940+0000","last_fullsized":"2024-09-13T17:17:44.993940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":7,"num_write":19042,"num_write_kb":260722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"63'4802","reported_seq":4871,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.979511+0000","last_change":"2024-09-13T16:47:59.049100+0000","last_active":"2024-09-13T17:17:44.979511+0000","last_peered":"2024-09-13T17:17:44.979511+0000","last_clean":"2024-09-13T17:17:44.979511+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T17:17:44.979511+0000","last_undegraded":"2024-09-13T17:17:44.979511+0000","last_fullsized":"2024-09-13T17:17:44.979511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26419,"num_write_kb":335470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"63'4","reported_seq":74,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978957+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T17:17:44.978957+0000","last_peered":"2024-09-13T17:17:44.978957+0000","last_clean":"2024-09-13T17:17:44.978957+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T17:17:44.978957+0000","last_undegraded":"2024-09-13T17:17:44.978957+0000","last_fullsized":"2024-09-13T17:17:44.978957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:54.181019+0000","last_change":"2024-09-13T17:17:54.181249+0000","last_active":"2024-09-13T17:17:54.181019+0000","last_peered":"2024-09-13T17:17:54.181019+0000","last_clean":"2024-09-13T17:17:54.181019+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T17:17:54.181019+0000","last_undegraded":"2024-09-13T17:17:54.181019+0000","last_fullsized":"2024-09-13T17:17:54.181019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_clean_scrub_stamp":"2024-09-13T17:17:54.180966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:37.317744+0000","scrub_duration":0.0037272709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"63'4215","reported_seq":4280,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009307+0000","last_change":"2024-09-13T16:48:08.172909+0000","last_active":"2024-09-13T17:17:45.009307+0000","last_peered":"2024-09-13T17:17:45.009307+0000","last_clean":"2024-09-13T17:17:45.009307+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T17:17:45.009307+0000","last_undegraded":"2024-09-13T17:17:45.009307+0000","last_fullsized":"2024-09-13T17:17:45.009307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":8,"num_write":22461,"num_write_kb":319331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'3790","reported_seq":3859,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992494+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T17:17:44.992494+0000","last_peered":"2024-09-13T17:17:44.992494+0000","last_clean":"2024-09-13T17:17:44.992494+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T17:17:44.992494+0000","last_undegraded":"2024-09-13T17:17:44.992494+0000","last_fullsized":"2024-09-13T17:17:44.992494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21077,"num_write_kb":260732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'42","reported_seq":179,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.602904+0000","last_change":"2024-09-13T17:18:03.603126+0000","last_active":"2024-09-13T17:18:03.602904+0000","last_peered":"2024-09-13T17:18:03.602904+0000","last_clean":"2024-09-13T17:18:03.602904+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T17:18:03.602904+0000","last_undegraded":"2024-09-13T17:18:03.602904+0000","last_fullsized":"2024-09-13T17:18:03.602904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'42","last_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_deep_scrub":"64'42","last_deep_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.602845+0000","objects_scrubbed":1,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:03.563076+0000","scrub_duration":0.004957576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84646,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":106,"num_write":37,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"63'9","reported_seq":116,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.997784+0000","last_change":"2024-09-13T16:48:07.769420+0000","last_active":"2024-09-13T17:17:44.997784+0000","last_peered":"2024-09-13T17:17:44.997784+0000","last_clean":"2024-09-13T17:17:44.997784+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T17:17:44.997784+0000","last_undegraded":"2024-09-13T17:17:44.997784+0000","last_fullsized":"2024-09-13T17:17:44.997784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"63'3128","reported_seq":3192,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.987242+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T17:17:44.987242+0000","last_peered":"2024-09-13T17:17:44.987242+0000","last_clean":"2024-09-13T17:17:44.987242+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T17:17:44.987242+0000","last_undegraded":"2024-09-13T17:17:44.987242+0000","last_fullsized":"2024-09-13T17:17:44.987242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3128,"log_dups_size":0,"ondisk_log_size":3128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17776,"num_write_kb":243283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"63'3657","reported_seq":3726,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991994+0000","last_change":"2024-09-13T16:47:59.056514+0000","last_active":"2024-09-13T17:17:44.991994+0000","last_peered":"2024-09-13T17:17:44.991994+0000","last_clean":"2024-09-13T17:17:44.991994+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T17:17:44.991994+0000","last_undegraded":"2024-09-13T17:17:44.991994+0000","last_fullsized":"2024-09-13T17:17:44.991994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19816,"num_write_kb":255512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'11","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.531660+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T17:17:45.531660+0000","last_peered":"2024-09-13T17:17:45.531660+0000","last_clean":"2024-09-13T17:17:45.531660+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T17:17:45.531660+0000","last_undegraded":"2024-09-13T17:17:45.531660+0000","last_fullsized":"2024-09-13T17:17:45.531660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'251","reported_seq":337,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.997995+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T17:17:44.997995+0000","last_peered":"2024-09-13T17:17:44.997995+0000","last_clean":"2024-09-13T17:17:44.997995+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T17:17:44.997995+0000","last_undegraded":"2024-09-13T17:17:44.997995+0000","last_fullsized":"2024-09-13T17:17:44.997995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":57,"num_write":597,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"63'4141","reported_seq":4207,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992340+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T17:17:44.992340+0000","last_peered":"2024-09-13T17:17:44.992340+0000","last_clean":"2024-09-13T17:17:44.992340+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T17:17:44.992340+0000","last_undegraded":"2024-09-13T17:17:44.992340+0000","last_fullsized":"2024-09-13T17:17:44.992340+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22849,"num_write_kb":317008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'4103","reported_seq":4180,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.972983+0000","last_change":"2024-09-13T16:48:41.389774+0000","last_active":"2024-09-13T17:17:44.972983+0000","last_peered":"2024-09-13T17:17:44.972983+0000","last_clean":"2024-09-13T17:17:44.972983+0000","last_became_active":"2024-09-13T16:48:41.389507+0000","last_became_peered":"2024-09-13T16:48:41.389507+0000","last_unstale":"2024-09-13T17:17:44.972983+0000","last_undegraded":"2024-09-13T17:17:44.972983+0000","last_fullsized":"2024-09-13T17:17:44.972983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22342,"num_write_kb":278194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"63'3","reported_seq":73,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980918+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T17:17:44.980918+0000","last_peered":"2024-09-13T17:17:44.980918+0000","last_clean":"2024-09-13T17:17:44.980918+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T17:17:44.980918+0000","last_undegraded":"2024-09-13T17:17:44.980918+0000","last_fullsized":"2024-09-13T17:17:44.980918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.590577+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T17:17:49.590577+0000","last_peered":"2024-09-13T17:17:49.590577+0000","last_clean":"2024-09-13T17:17:49.590577+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T17:17:49.590577+0000","last_undegraded":"2024-09-13T17:17:49.590577+0000","last_fullsized":"2024-09-13T17:17:49.590577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"63'2","reported_seq":68,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994652+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T17:17:44.994652+0000","last_peered":"2024-09-13T17:17:44.994652+0000","last_clean":"2024-09-13T17:17:44.994652+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T17:17:44.994652+0000","last_undegraded":"2024-09-13T17:17:44.994652+0000","last_fullsized":"2024-09-13T17:17:44.994652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"63'2904","reported_seq":2980,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.337844+0000","last_change":"2024-09-13T17:17:48.337931+0000","last_active":"2024-09-13T17:17:48.337844+0000","last_peered":"2024-09-13T17:17:48.337844+0000","last_clean":"2024-09-13T17:17:48.337844+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T17:17:48.337844+0000","last_undegraded":"2024-09-13T17:17:48.337844+0000","last_fullsized":"2024-09-13T17:17:48.337844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2904","last_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_deep_scrub":"63'2904","last_deep_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_clean_scrub_stamp":"2024-09-13T17:17:48.337825+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:15.694145+0000","scrub_duration":0.0051743259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":2,"num_write":15607,"num_write_kb":214254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'4284","reported_seq":4363,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.341991+0000","last_change":"2024-09-13T17:17:52.342307+0000","last_active":"2024-09-13T17:17:52.341991+0000","last_peered":"2024-09-13T17:17:52.341991+0000","last_clean":"2024-09-13T17:17:52.341991+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T17:17:52.341991+0000","last_undegraded":"2024-09-13T17:17:52.341991+0000","last_fullsized":"2024-09-13T17:17:52.341991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4284","last_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_deep_scrub":"63'4284","last_deep_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.341937+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.282559+0000","scrub_duration":0.043379468999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23392,"num_write_kb":310963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"63'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.502278+0000","last_change":"2024-09-13T17:17:53.502523+0000","last_active":"2024-09-13T17:17:53.502278+0000","last_peered":"2024-09-13T17:17:53.502278+0000","last_clean":"2024-09-13T17:17:53.502278+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T17:17:53.502278+0000","last_undegraded":"2024-09-13T17:17:53.502278+0000","last_fullsized":"2024-09-13T17:17:53.502278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.502221+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:42.741599+0000","scrub_duration":0.0040979670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"63'3","reported_seq":68,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994051+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T17:17:44.994051+0000","last_peered":"2024-09-13T17:17:44.994051+0000","last_clean":"2024-09-13T17:17:44.994051+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T17:17:44.994051+0000","last_undegraded":"2024-09-13T17:17:44.994051+0000","last_fullsized":"2024-09-13T17:17:44.994051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"63'4138","reported_seq":4213,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.398557+0000","last_change":"2024-09-13T17:17:49.398680+0000","last_active":"2024-09-13T17:17:49.398557+0000","last_peered":"2024-09-13T17:17:49.398557+0000","last_clean":"2024-09-13T17:17:49.398557+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T17:17:49.398557+0000","last_undegraded":"2024-09-13T17:17:49.398557+0000","last_fullsized":"2024-09-13T17:17:49.398557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4138","last_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_deep_scrub":"63'4138","last_deep_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.398536+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:51.412144+0000","scrub_duration":0.020992730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":309086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"63'4584","reported_seq":4646,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.164508+0000","last_change":"2024-09-13T16:48:40.767044+0000","last_active":"2024-09-13T17:17:47.164508+0000","last_peered":"2024-09-13T17:17:47.164508+0000","last_clean":"2024-09-13T17:17:47.164508+0000","last_became_active":"2024-09-13T16:48:40.765861+0000","last_became_peered":"2024-09-13T16:48:40.765861+0000","last_unstale":"2024-09-13T17:17:47.164508+0000","last_undegraded":"2024-09-13T17:17:47.164508+0000","last_fullsized":"2024-09-13T17:17:47.164508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24690,"num_write_kb":320186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"63'6","reported_seq":94,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.995320+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T17:17:44.995320+0000","last_peered":"2024-09-13T17:17:44.995320+0000","last_clean":"2024-09-13T17:17:44.995320+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T17:17:44.995320+0000","last_undegraded":"2024-09-13T17:17:44.995320+0000","last_fullsized":"2024-09-13T17:17:44.995320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'1425","reported_seq":1538,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.001931+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T17:17:45.001931+0000","last_peered":"2024-09-13T17:17:45.001931+0000","last_clean":"2024-09-13T17:17:45.001931+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T17:17:45.001931+0000","last_undegraded":"2024-09-13T17:17:45.001931+0000","last_fullsized":"2024-09-13T17:17:45.001931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":38459,"num_write":1400,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"63'3637","reported_seq":3702,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991867+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T17:17:44.991867+0000","last_peered":"2024-09-13T17:17:44.991867+0000","last_clean":"2024-09-13T17:17:44.991867+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T17:17:44.991867+0000","last_undegraded":"2024-09-13T17:17:44.991867+0000","last_fullsized":"2024-09-13T17:17:44.991867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19635,"num_write_kb":259018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'5067","reported_seq":5129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.164365+0000","last_change":"2024-09-13T16:48:40.831954+0000","last_active":"2024-09-13T17:17:47.164365+0000","last_peered":"2024-09-13T17:17:47.164365+0000","last_clean":"2024-09-13T17:17:47.164365+0000","last_became_active":"2024-09-13T16:48:40.831696+0000","last_became_peered":"2024-09-13T16:48:40.831696+0000","last_unstale":"2024-09-13T17:17:47.164365+0000","last_undegraded":"2024-09-13T17:17:47.164365+0000","last_fullsized":"2024-09-13T17:17:47.164365+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27028,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.556099+0000","last_change":"2024-09-13T17:18:02.556314+0000","last_active":"2024-09-13T17:18:02.556099+0000","last_peered":"2024-09-13T17:18:02.556099+0000","last_clean":"2024-09-13T17:18:02.556099+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T17:18:02.556099+0000","last_undegraded":"2024-09-13T17:18:02.556099+0000","last_fullsized":"2024-09-13T17:18:02.556099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.556044+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:52.052485+0000","scrub_duration":0.0060730330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"63'5","reported_seq":70,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978443+0000","last_change":"2024-09-13T16:48:07.791000+0000","last_active":"2024-09-13T17:17:44.978443+0000","last_peered":"2024-09-13T17:17:44.978443+0000","last_clean":"2024-09-13T17:17:44.978443+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T17:17:44.978443+0000","last_undegraded":"2024-09-13T17:17:44.978443+0000","last_fullsized":"2024-09-13T17:17:44.978443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"63'2915","reported_seq":2979,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.006202+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T17:17:45.006202+0000","last_peered":"2024-09-13T17:17:45.006202+0000","last_clean":"2024-09-13T17:17:45.006202+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T17:17:45.006202+0000","last_undegraded":"2024-09-13T17:17:45.006202+0000","last_fullsized":"2024-09-13T17:17:45.006202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2915,"log_dups_size":0,"ondisk_log_size":2915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":234494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"63'3697","reported_seq":3765,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993206+0000","last_change":"2024-09-13T16:47:59.057384+0000","last_active":"2024-09-13T17:17:44.993206+0000","last_peered":"2024-09-13T17:17:44.993206+0000","last_clean":"2024-09-13T17:17:44.993206+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T17:17:44.993206+0000","last_undegraded":"2024-09-13T17:17:44.993206+0000","last_fullsized":"2024-09-13T17:17:44.993206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20206,"num_write_kb":253140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"63'4","reported_seq":92,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.989601+0000","last_change":"2024-09-13T16:47:58.662727+0000","last_active":"2024-09-13T17:17:44.989601+0000","last_peered":"2024-09-13T17:17:44.989601+0000","last_clean":"2024-09-13T17:17:44.989601+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T17:17:44.989601+0000","last_undegraded":"2024-09-13T17:17:44.989601+0000","last_fullsized":"2024-09-13T17:17:44.989601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"63'2","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.158270+0000","last_change":"2024-09-13T17:18:02.158515+0000","last_active":"2024-09-13T17:18:02.158270+0000","last_peered":"2024-09-13T17:18:02.158270+0000","last_clean":"2024-09-13T17:18:02.158270+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T17:18:02.158270+0000","last_undegraded":"2024-09-13T17:18:02.158270+0000","last_fullsized":"2024-09-13T17:18:02.158270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.158216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.118447+0000","scrub_duration":0.003800346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"63'3522","reported_seq":3585,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975808+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T17:17:44.975808+0000","last_peered":"2024-09-13T17:17:44.975808+0000","last_clean":"2024-09-13T17:17:44.975808+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T17:17:44.975808+0000","last_undegraded":"2024-09-13T17:17:44.975808+0000","last_fullsized":"2024-09-13T17:17:44.975808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18828,"num_write_kb":272277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"63'3874","reported_seq":3954,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.213202+0000","last_change":"2024-09-13T17:17:59.213453+0000","last_active":"2024-09-13T17:17:59.213202+0000","last_peered":"2024-09-13T17:17:59.213202+0000","last_clean":"2024-09-13T17:17:59.213202+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T17:17:59.213202+0000","last_undegraded":"2024-09-13T17:17:59.213202+0000","last_fullsized":"2024-09-13T17:17:59.213202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.213146+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.190638+0000","scrub_duration":0.033789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21413,"num_write_kb":278431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'19","reported_seq":192,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.533537+0000","last_change":"2024-09-13T17:17:59.533761+0000","last_active":"2024-09-13T17:17:59.533537+0000","last_peered":"2024-09-13T17:17:59.533537+0000","last_clean":"2024-09-13T17:17:59.533537+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T17:17:59.533537+0000","last_undegraded":"2024-09-13T17:17:59.533537+0000","last_fullsized":"2024-09-13T17:17:59.533537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_deep_scrub":"64'19","last_deep_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.533477+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:42.670255+0000","scrub_duration":0.0092684789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:00.848306+0000","last_change":"2024-09-13T17:18:00.848525+0000","last_active":"2024-09-13T17:18:00.848306+0000","last_peered":"2024-09-13T17:18:00.848306+0000","last_clean":"2024-09-13T17:18:00.848306+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T17:18:00.848306+0000","last_undegraded":"2024-09-13T17:18:00.848306+0000","last_fullsized":"2024-09-13T17:18:00.848306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_clean_scrub_stamp":"2024-09-13T17:18:00.848253+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:51.294142+0000","scrub_duration":0.0034249229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"63'3622","reported_seq":3688,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.984518+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T17:17:44.984518+0000","last_peered":"2024-09-13T17:17:44.984518+0000","last_clean":"2024-09-13T17:17:44.984518+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T17:17:44.984518+0000","last_undegraded":"2024-09-13T17:17:44.984518+0000","last_fullsized":"2024-09-13T17:17:44.984518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":19996,"num_write_kb":293269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"63'3240","reported_seq":3307,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994131+0000","last_change":"2024-09-13T16:47:59.048032+0000","last_active":"2024-09-13T17:17:44.994131+0000","last_peered":"2024-09-13T17:17:44.994131+0000","last_clean":"2024-09-13T17:17:44.994131+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T17:17:44.994131+0000","last_undegraded":"2024-09-13T17:17:44.994131+0000","last_fullsized":"2024-09-13T17:17:44.994131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3240,"log_dups_size":0,"ondisk_log_size":3240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":237245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"63'5","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.449588+0000","last_change":"2024-09-13T17:18:02.449798+0000","last_active":"2024-09-13T17:18:02.449588+0000","last_peered":"2024-09-13T17:18:02.449588+0000","last_clean":"2024-09-13T17:18:02.449588+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T17:18:02.449588+0000","last_undegraded":"2024-09-13T17:18:02.449588+0000","last_fullsized":"2024-09-13T17:18:02.449588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.449537+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.409768+0000","scrub_duration":0.0032633279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'5","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.060756+0000","last_change":"2024-09-13T17:17:51.060978+0000","last_active":"2024-09-13T17:17:51.060756+0000","last_peered":"2024-09-13T17:17:51.060756+0000","last_clean":"2024-09-13T17:17:51.060756+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T17:17:51.060756+0000","last_undegraded":"2024-09-13T17:17:51.060756+0000","last_fullsized":"2024-09-13T17:17:51.060756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.060702+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:45.719305+0000","scrub_duration":0.0041164770000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"63'3376","reported_seq":3439,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.981690+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T17:17:44.981690+0000","last_peered":"2024-09-13T17:17:44.981690+0000","last_clean":"2024-09-13T17:17:44.981690+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T17:17:44.981690+0000","last_undegraded":"2024-09-13T17:17:44.981690+0000","last_fullsized":"2024-09-13T17:17:44.981690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18879,"num_write_kb":251906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"63'4848","reported_seq":4915,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993627+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T17:17:44.993627+0000","last_peered":"2024-09-13T17:17:44.993627+0000","last_clean":"2024-09-13T17:17:44.993627+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T17:17:44.993627+0000","last_undegraded":"2024-09-13T17:17:44.993627+0000","last_fullsized":"2024-09-13T17:17:44.993627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26963,"num_write_kb":321221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"63'3","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978813+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T17:17:44.978813+0000","last_peered":"2024-09-13T17:17:44.978813+0000","last_clean":"2024-09-13T17:17:44.978813+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T17:17:44.978813+0000","last_undegraded":"2024-09-13T17:17:44.978813+0000","last_fullsized":"2024-09-13T17:17:44.978813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'392","reported_seq":574,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.116288+0000","last_change":"2024-09-13T17:17:49.116420+0000","last_active":"2024-09-13T17:17:49.116288+0000","last_peered":"2024-09-13T17:17:49.116288+0000","last_clean":"2024-09-13T17:17:49.116288+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T17:17:49.116288+0000","last_undegraded":"2024-09-13T17:17:49.116288+0000","last_fullsized":"2024-09-13T17:17:49.116288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'392","last_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_deep_scrub":"64'392","last_deep_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.116249+0000","objects_scrubbed":4,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:49.076480+0000","scrub_duration":0.026108916999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":191,"num_read_kb":82094,"num_write":385,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'4856","reported_seq":4921,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974382+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T17:17:44.974382+0000","last_peered":"2024-09-13T17:17:44.974382+0000","last_clean":"2024-09-13T17:17:44.974382+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T17:17:44.974382+0000","last_undegraded":"2024-09-13T17:17:44.974382+0000","last_fullsized":"2024-09-13T17:17:44.974382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":26453,"num_write_kb":368210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'4063","reported_seq":4130,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.001087+0000","last_change":"2024-09-13T16:47:59.054006+0000","last_active":"2024-09-13T17:17:45.001087+0000","last_peered":"2024-09-13T17:17:45.001087+0000","last_clean":"2024-09-13T17:17:45.001087+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T17:17:45.001087+0000","last_undegraded":"2024-09-13T17:17:45.001087+0000","last_fullsized":"2024-09-13T17:17:45.001087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4063,"log_dups_size":0,"ondisk_log_size":4063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21999,"num_write_kb":293018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"64'9","reported_seq":115,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.206956+0000","last_change":"2024-09-13T16:47:58.661604+0000","last_active":"2024-09-13T17:17:46.206956+0000","last_peered":"2024-09-13T17:17:46.206956+0000","last_clean":"2024-09-13T17:17:46.206956+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T17:17:46.206956+0000","last_undegraded":"2024-09-13T17:17:46.206956+0000","last_fullsized":"2024-09-13T17:17:46.206956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":152,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'32","reported_seq":170,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.557513+0000","last_change":"2024-09-13T17:17:59.557788+0000","last_active":"2024-09-13T17:17:59.557513+0000","last_peered":"2024-09-13T17:17:59.557513+0000","last_clean":"2024-09-13T17:17:59.557513+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T17:17:59.557513+0000","last_undegraded":"2024-09-13T17:17:59.557513+0000","last_fullsized":"2024-09-13T17:17:59.557513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'32","last_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_deep_scrub":"64'32","last_deep_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.557461+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.534953+0000","scrub_duration":0.0076444640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":31,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"63'4239","reported_seq":4302,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.973983+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T17:17:44.973983+0000","last_peered":"2024-09-13T17:17:44.973983+0000","last_clean":"2024-09-13T17:17:44.973983+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T17:17:44.973983+0000","last_undegraded":"2024-09-13T17:17:44.973983+0000","last_fullsized":"2024-09-13T17:17:44.973983+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23316,"num_write_kb":318961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'3718","reported_seq":3796,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.214013+0000","last_change":"2024-09-13T17:17:57.214254+0000","last_active":"2024-09-13T17:17:57.214013+0000","last_peered":"2024-09-13T17:17:57.214013+0000","last_clean":"2024-09-13T17:17:57.214013+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T17:17:57.214013+0000","last_undegraded":"2024-09-13T17:17:57.214013+0000","last_fullsized":"2024-09-13T17:17:57.214013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3718","last_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_deep_scrub":"63'3718","last_deep_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.213958+0000","objects_scrubbed":0,"log_size":3718,"log_dups_size":0,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.719666+0000","scrub_duration":0.032519578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20956,"num_write_kb":260218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"64'395","reported_seq":569,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.567234+0000","last_change":"2024-09-13T17:18:04.567435+0000","last_active":"2024-09-13T17:18:04.567234+0000","last_peered":"2024-09-13T17:18:04.567234+0000","last_clean":"2024-09-13T17:18:04.567234+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T17:18:04.567234+0000","last_undegraded":"2024-09-13T17:18:04.567234+0000","last_fullsized":"2024-09-13T17:18:04.567234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'395","last_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_deep_scrub":"64'395","last_deep_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.567179+0000","objects_scrubbed":4,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:59.225783+0000","scrub_duration":0.027913209000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":171,"num_read_kb":77976,"num_write":389,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"63'3","reported_seq":67,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994321+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T17:17:44.994321+0000","last_peered":"2024-09-13T17:17:44.994321+0000","last_clean":"2024-09-13T17:17:44.994321+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T17:17:44.994321+0000","last_undegraded":"2024-09-13T17:17:44.994321+0000","last_fullsized":"2024-09-13T17:17:44.994321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"63'3522","reported_seq":3586,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994900+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T17:17:44.994900+0000","last_peered":"2024-09-13T17:17:44.994900+0000","last_clean":"2024-09-13T17:17:44.994900+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T17:17:44.994900+0000","last_undegraded":"2024-09-13T17:17:44.994900+0000","last_fullsized":"2024-09-13T17:17:44.994900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":4,"num_write":19373,"num_write_kb":266222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"63'5138","reported_seq":5205,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.979167+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T17:17:44.979167+0000","last_peered":"2024-09-13T17:17:44.979167+0000","last_clean":"2024-09-13T17:17:44.979167+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T17:17:44.979167+0000","last_undegraded":"2024-09-13T17:17:44.979167+0000","last_fullsized":"2024-09-13T17:17:44.979167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27863,"num_write_kb":353454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'25","reported_seq":131,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.003205+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T17:17:48.003205+0000","last_peered":"2024-09-13T17:17:48.003205+0000","last_clean":"2024-09-13T17:17:48.003205+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T17:17:48.003205+0000","last_undegraded":"2024-09-13T17:17:48.003205+0000","last_fullsized":"2024-09-13T17:17:48.003205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'9","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.154568+0000","last_change":"2024-09-13T17:18:03.154830+0000","last_active":"2024-09-13T17:18:03.154568+0000","last_peered":"2024-09-13T17:18:03.154568+0000","last_clean":"2024-09-13T17:18:03.154568+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T17:18:03.154568+0000","last_undegraded":"2024-09-13T17:18:03.154568+0000","last_fullsized":"2024-09-13T17:18:03.154568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_deep_scrub":"64'9","last_deep_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.154514+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:05.788731+0000","scrub_duration":0.0080602239999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":168,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"63'3602","reported_seq":3668,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993944+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T17:17:44.993944+0000","last_peered":"2024-09-13T17:17:44.993944+0000","last_clean":"2024-09-13T17:17:44.993944+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T17:17:44.993944+0000","last_undegraded":"2024-09-13T17:17:44.993944+0000","last_fullsized":"2024-09-13T17:17:44.993944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":20290,"num_write_kb":278255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'3743","reported_seq":3821,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.841035+0000","last_change":"2024-09-13T17:17:57.841262+0000","last_active":"2024-09-13T17:17:57.841035+0000","last_peered":"2024-09-13T17:17:57.841035+0000","last_clean":"2024-09-13T17:17:57.841035+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T17:17:57.841035+0000","last_undegraded":"2024-09-13T17:17:57.841035+0000","last_fullsized":"2024-09-13T17:17:57.841035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3743","last_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_deep_scrub":"63'3743","last_deep_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.840983+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:01.952379+0000","scrub_duration":0.026384685000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20404,"num_write_kb":278673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"63'3","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.986857+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T17:17:44.986857+0000","last_peered":"2024-09-13T17:17:44.986857+0000","last_clean":"2024-09-13T17:17:44.986857+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T17:17:44.986857+0000","last_undegraded":"2024-09-13T17:17:44.986857+0000","last_fullsized":"2024-09-13T17:17:44.986857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"63'15","reported_seq":187,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.971211+0000","last_change":"2024-09-13T16:48:07.776452+0000","last_active":"2024-09-13T17:17:44.971211+0000","last_peered":"2024-09-13T17:17:44.971211+0000","last_clean":"2024-09-13T17:17:44.971211+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T17:17:44.971211+0000","last_undegraded":"2024-09-13T17:17:44.971211+0000","last_fullsized":"2024-09-13T17:17:44.971211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":324,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"63'3189","reported_seq":3252,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992384+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T17:17:44.992384+0000","last_peered":"2024-09-13T17:17:44.992384+0000","last_clean":"2024-09-13T17:17:44.992384+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T17:17:44.992384+0000","last_undegraded":"2024-09-13T17:17:44.992384+0000","last_fullsized":"2024-09-13T17:17:44.992384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17342,"num_write_kb":249797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"63'4223","reported_seq":4290,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.985904+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T17:17:44.985904+0000","last_peered":"2024-09-13T17:17:44.985904+0000","last_clean":"2024-09-13T17:17:44.985904+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T17:17:44.985904+0000","last_undegraded":"2024-09-13T17:17:44.985904+0000","last_fullsized":"2024-09-13T17:17:44.985904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4223,"log_dups_size":0,"ondisk_log_size":4223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":295025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"64'5","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:06.450397+0000","last_change":"2024-09-13T17:18:06.450600+0000","last_active":"2024-09-13T17:18:06.450397+0000","last_peered":"2024-09-13T17:18:06.450397+0000","last_clean":"2024-09-13T17:18:06.450397+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T17:18:06.450397+0000","last_undegraded":"2024-09-13T17:18:06.450397+0000","last_fullsized":"2024-09-13T17:18:06.450397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_clean_scrub_stamp":"2024-09-13T17:18:06.450341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:29.338436+0000","scrub_duration":0.003290758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"63'5","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.123526+0000","last_change":"2024-09-13T17:17:53.123757+0000","last_active":"2024-09-13T17:17:53.123526+0000","last_peered":"2024-09-13T17:17:53.123526+0000","last_clean":"2024-09-13T17:17:53.123526+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T17:17:53.123526+0000","last_undegraded":"2024-09-13T17:17:53.123526+0000","last_fullsized":"2024-09-13T17:17:53.123526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.123466+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.011562+0000","scrub_duration":0.0038056650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'5357","reported_seq":5425,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.995114+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T17:17:44.995114+0000","last_peered":"2024-09-13T17:17:44.995114+0000","last_clean":"2024-09-13T17:17:44.995114+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T17:17:44.995114+0000","last_undegraded":"2024-09-13T17:17:44.995114+0000","last_fullsized":"2024-09-13T17:17:44.995114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":39,"num_write":29637,"num_write_kb":393569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"63'26","reported_seq":92,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.997682+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T17:17:44.997682+0000","last_peered":"2024-09-13T17:17:44.997682+0000","last_clean":"2024-09-13T17:17:44.997682+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T17:17:44.997682+0000","last_undegraded":"2024-09-13T17:17:44.997682+0000","last_fullsized":"2024-09-13T17:17:44.997682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"63'5122","reported_seq":5200,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.190845+0000","last_change":"2024-09-13T17:18:01.191064+0000","last_active":"2024-09-13T17:18:01.190845+0000","last_peered":"2024-09-13T17:18:01.190845+0000","last_clean":"2024-09-13T17:18:01.190845+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T17:18:01.190845+0000","last_undegraded":"2024-09-13T17:18:01.190845+0000","last_fullsized":"2024-09-13T17:18:01.190845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5122","last_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_deep_scrub":"63'5122","last_deep_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.190787+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:03.825004+0000","scrub_duration":0.032904946999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":348704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"63'2","reported_seq":70,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980774+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T17:17:44.980774+0000","last_peered":"2024-09-13T17:17:44.980774+0000","last_clean":"2024-09-13T17:17:44.980774+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T17:17:44.980774+0000","last_undegraded":"2024-09-13T17:17:44.980774+0000","last_fullsized":"2024-09-13T17:17:44.980774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"63'3554","reported_seq":3619,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996585+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T17:17:44.996585+0000","last_peered":"2024-09-13T17:17:44.996585+0000","last_clean":"2024-09-13T17:17:44.996585+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T17:17:44.996585+0000","last_undegraded":"2024-09-13T17:17:44.996585+0000","last_fullsized":"2024-09-13T17:17:44.996585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":268389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"63'1091","reported_seq":1207,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.995556+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T17:17:44.995556+0000","last_peered":"2024-09-13T17:17:44.995556+0000","last_clean":"2024-09-13T17:17:44.995556+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T17:17:44.995556+0000","last_undegraded":"2024-09-13T17:17:44.995556+0000","last_fullsized":"2024-09-13T17:17:44.995556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":9436,"num_write":1065,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"63'4714","reported_seq":4781,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.972738+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T17:17:44.972738+0000","last_peered":"2024-09-13T17:17:44.972738+0000","last_clean":"2024-09-13T17:17:44.972738+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T17:17:44.972738+0000","last_undegraded":"2024-09-13T17:17:44.972738+0000","last_fullsized":"2024-09-13T17:17:44.972738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25459,"num_write_kb":343662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"63'3","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.973656+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T17:17:44.973656+0000","last_peered":"2024-09-13T17:17:44.973656+0000","last_clean":"2024-09-13T17:17:44.973656+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T17:17:44.973656+0000","last_undegraded":"2024-09-13T17:17:44.973656+0000","last_fullsized":"2024-09-13T17:17:44.973656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"63'4328","reported_seq":4390,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993187+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T17:17:44.993187+0000","last_peered":"2024-09-13T17:17:44.993187+0000","last_clean":"2024-09-13T17:17:44.993187+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T17:17:44.993187+0000","last_undegraded":"2024-09-13T17:17:44.993187+0000","last_fullsized":"2024-09-13T17:17:44.993187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24456,"num_write_kb":315374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"64'411","reported_seq":494,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.515483+0000","last_change":"2024-09-13T17:18:03.515735+0000","last_active":"2024-09-13T17:18:03.515483+0000","last_peered":"2024-09-13T17:18:03.515483+0000","last_clean":"2024-09-13T17:18:03.515483+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T17:18:03.515483+0000","last_undegraded":"2024-09-13T17:18:03.515483+0000","last_fullsized":"2024-09-13T17:18:03.515483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'411","last_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_deep_scrub":"64'411","last_deep_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.515426+0000","objects_scrubbed":1,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:06.149643+0000","scrub_duration":0.019312658999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2812346,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3190,"num_write":386,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"63'4155","reported_seq":4222,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.207177+0000","last_change":"2024-09-13T16:47:59.057031+0000","last_active":"2024-09-13T17:17:46.207177+0000","last_peered":"2024-09-13T17:17:46.207177+0000","last_clean":"2024-09-13T17:17:46.207177+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T17:17:46.207177+0000","last_undegraded":"2024-09-13T17:17:46.207177+0000","last_fullsized":"2024-09-13T17:17:46.207177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22247,"num_write_kb":285891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"63'5","reported_seq":111,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.164557+0000","last_change":"2024-09-13T16:48:07.772124+0000","last_active":"2024-09-13T17:17:47.164557+0000","last_peered":"2024-09-13T17:17:47.164557+0000","last_clean":"2024-09-13T17:17:47.164557+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T17:17:47.164557+0000","last_undegraded":"2024-09-13T17:17:47.164557+0000","last_fullsized":"2024-09-13T17:17:47.164557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"63'3426","reported_seq":3491,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.977841+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T17:17:44.977841+0000","last_peered":"2024-09-13T17:17:44.977841+0000","last_clean":"2024-09-13T17:17:44.977841+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T17:17:44.977841+0000","last_undegraded":"2024-09-13T17:17:44.977841+0000","last_fullsized":"2024-09-13T17:17:44.977841+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3426,"log_dups_size":0,"ondisk_log_size":3426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":18510,"num_write_kb":250514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'712","reported_seq":803,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631599+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T17:17:47.631599+0000","last_peered":"2024-09-13T17:17:47.631599+0000","last_clean":"2024-09-13T17:17:47.631599+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T17:17:47.631599+0000","last_undegraded":"2024-09-13T17:17:47.631599+0000","last_fullsized":"2024-09-13T17:17:47.631599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":74674,"num_write":702,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"63'4316","reported_seq":4384,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.132275+0000","last_change":"2024-09-13T16:47:59.039135+0000","last_active":"2024-09-13T17:17:48.132275+0000","last_peered":"2024-09-13T17:17:48.132275+0000","last_clean":"2024-09-13T17:17:48.132275+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T17:17:48.132275+0000","last_undegraded":"2024-09-13T17:17:48.132275+0000","last_fullsized":"2024-09-13T17:17:48.132275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22350,"num_write_kb":290228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"64'45","reported_seq":162,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.128882+0000","last_change":"2024-09-13T17:17:59.128882+0000","last_active":"2024-09-13T17:17:59.128882+0000","last_peered":"2024-09-13T17:17:59.128882+0000","last_clean":"2024-09-13T17:17:59.128882+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T17:17:59.128882+0000","last_undegraded":"2024-09-13T17:17:59.128882+0000","last_fullsized":"2024-09-13T17:17:59.128882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'45","last_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_deep_scrub":"64'45","last_deep_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.128560+0000","objects_scrubbed":1,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.106052+0000","scrub_duration":0.0062496970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"63'3129","reported_seq":3193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975860+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T17:17:44.975860+0000","last_peered":"2024-09-13T17:17:44.975860+0000","last_clean":"2024-09-13T17:17:44.975860+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T17:17:44.975860+0000","last_undegraded":"2024-09-13T17:17:44.975860+0000","last_fullsized":"2024-09-13T17:17:44.975860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17279,"num_write_kb":257601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"63'4785","reported_seq":4865,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.565656+0000","last_change":"2024-09-13T17:17:49.565883+0000","last_active":"2024-09-13T17:17:49.565656+0000","last_peered":"2024-09-13T17:17:49.565656+0000","last_clean":"2024-09-13T17:17:49.565656+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T17:17:49.565656+0000","last_undegraded":"2024-09-13T17:17:49.565656+0000","last_fullsized":"2024-09-13T17:17:49.565656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4785","last_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_deep_scrub":"63'4785","last_deep_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.565599+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.011489+0000","scrub_duration":0.012593717000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26302,"num_write_kb":310249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"63'4231","reported_seq":4296,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996163+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T17:17:44.996163+0000","last_peered":"2024-09-13T17:17:44.996163+0000","last_clean":"2024-09-13T17:17:44.996163+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T17:17:44.996163+0000","last_undegraded":"2024-09-13T17:17:44.996163+0000","last_fullsized":"2024-09-13T17:17:44.996163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22604,"num_write_kb":324065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"63'3874","reported_seq":3955,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.662237+0000","last_change":"2024-09-13T17:18:04.662513+0000","last_active":"2024-09-13T17:18:04.662237+0000","last_peered":"2024-09-13T17:18:04.662237+0000","last_clean":"2024-09-13T17:18:04.662237+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T17:18:04.662237+0000","last_undegraded":"2024-09-13T17:18:04.662237+0000","last_fullsized":"2024-09-13T17:18:04.662237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.662182+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:07.296399+0000","scrub_duration":0.021443912999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20920,"num_write_kb":276387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"63'3865","reported_seq":3930,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976014+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T17:17:44.976014+0000","last_peered":"2024-09-13T17:17:44.976014+0000","last_clean":"2024-09-13T17:17:44.976014+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T17:17:44.976014+0000","last_undegraded":"2024-09-13T17:17:44.976014+0000","last_fullsized":"2024-09-13T17:17:44.976014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21432,"num_write_kb":286796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"63'4552","reported_seq":4623,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978243+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T17:17:44.978243+0000","last_peered":"2024-09-13T17:17:44.978243+0000","last_clean":"2024-09-13T17:17:44.978243+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T17:17:44.978243+0000","last_undegraded":"2024-09-13T17:17:44.978243+0000","last_fullsized":"2024-09-13T17:17:44.978243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":24031,"num_write_kb":313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"63'4071","reported_seq":4136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992984+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T17:17:44.992984+0000","last_peered":"2024-09-13T17:17:44.992984+0000","last_clean":"2024-09-13T17:17:44.992984+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T17:17:44.992984+0000","last_undegraded":"2024-09-13T17:17:44.992984+0000","last_fullsized":"2024-09-13T17:17:44.992984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":22061,"num_write_kb":309650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"63'3506","reported_seq":3576,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.004143+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T17:17:45.004143+0000","last_peered":"2024-09-13T17:17:45.004143+0000","last_clean":"2024-09-13T17:17:45.004143+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T17:17:45.004143+0000","last_undegraded":"2024-09-13T17:17:45.004143+0000","last_fullsized":"2024-09-13T17:17:45.004143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3506,"log_dups_size":0,"ondisk_log_size":3506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19493,"num_write_kb":254380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"63'4523","reported_seq":4588,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992102+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T17:17:44.992102+0000","last_peered":"2024-09-13T17:17:44.992102+0000","last_clean":"2024-09-13T17:17:44.992102+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T17:17:44.992102+0000","last_undegraded":"2024-09-13T17:17:44.992102+0000","last_fullsized":"2024-09-13T17:17:44.992102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4523,"log_dups_size":0,"ondisk_log_size":4523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24221,"num_write_kb":338849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"63'4422","reported_seq":4491,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976237+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T17:17:44.976237+0000","last_peered":"2024-09-13T17:17:44.976237+0000","last_clean":"2024-09-13T17:17:44.976237+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T17:17:44.976237+0000","last_undegraded":"2024-09-13T17:17:44.976237+0000","last_fullsized":"2024-09-13T17:17:44.976237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":24862,"num_write_kb":304752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"63'3771","reported_seq":3837,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976919+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T17:17:44.976919+0000","last_peered":"2024-09-13T17:17:44.976919+0000","last_clean":"2024-09-13T17:17:44.976919+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T17:17:44.976919+0000","last_undegraded":"2024-09-13T17:17:44.976919+0000","last_fullsized":"2024-09-13T17:17:44.976919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21135,"num_write_kb":285540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"63'4461","reported_seq":4529,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994366+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T17:17:44.994366+0000","last_peered":"2024-09-13T17:17:44.994366+0000","last_clean":"2024-09-13T17:17:44.994366+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T17:17:44.994366+0000","last_undegraded":"2024-09-13T17:17:44.994366+0000","last_fullsized":"2024-09-13T17:17:44.994366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25388,"num_write_kb":319094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"63'4193","reported_seq":4258,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991839+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T17:17:44.991839+0000","last_peered":"2024-09-13T17:17:44.991839+0000","last_clean":"2024-09-13T17:17:44.991839+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T17:17:44.991839+0000","last_undegraded":"2024-09-13T17:17:44.991839+0000","last_fullsized":"2024-09-13T17:17:44.991839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4193,"log_dups_size":0,"ondisk_log_size":4193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23066,"num_write_kb":313713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"63'3978","reported_seq":4058,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.616395+0000","last_change":"2024-09-13T17:17:57.616624+0000","last_active":"2024-09-13T17:17:57.616395+0000","last_peered":"2024-09-13T17:17:57.616395+0000","last_clean":"2024-09-13T17:17:57.616395+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T17:17:57.616395+0000","last_undegraded":"2024-09-13T17:17:57.616395+0000","last_fullsized":"2024-09-13T17:17:57.616395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3978","last_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_deep_scrub":"63'3978","last_deep_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.616344+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:01.359043+0000","scrub_duration":0.024757745000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22157,"num_write_kb":281285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"63'4050","reported_seq":4117,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993755+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T17:17:44.993755+0000","last_peered":"2024-09-13T17:17:44.993755+0000","last_clean":"2024-09-13T17:17:44.993755+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T17:17:44.993755+0000","last_undegraded":"2024-09-13T17:17:44.993755+0000","last_fullsized":"2024-09-13T17:17:44.993755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4050,"log_dups_size":0,"ondisk_log_size":4050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21731,"num_write_kb":291923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"63'5211","reported_seq":5274,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009487+0000","last_change":"2024-09-13T16:48:40.831835+0000","last_active":"2024-09-13T17:17:45.009487+0000","last_peered":"2024-09-13T17:17:45.009487+0000","last_clean":"2024-09-13T17:17:45.009487+0000","last_became_active":"2024-09-13T16:48:40.831649+0000","last_became_peered":"2024-09-13T16:48:40.831649+0000","last_unstale":"2024-09-13T17:17:45.009487+0000","last_undegraded":"2024-09-13T17:17:45.009487+0000","last_fullsized":"2024-09-13T17:17:45.009487+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":28065,"num_write_kb":366501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"63'3806","reported_seq":3872,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975332+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T17:17:44.975332+0000","last_peered":"2024-09-13T17:17:44.975332+0000","last_clean":"2024-09-13T17:17:44.975332+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T17:17:44.975332+0000","last_undegraded":"2024-09-13T17:17:44.975332+0000","last_fullsized":"2024-09-13T17:17:44.975332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4,"num_write":20286,"num_write_kb":286237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"63'3659","reported_seq":3728,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.979776+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T17:17:44.979776+0000","last_peered":"2024-09-13T17:17:44.979776+0000","last_clean":"2024-09-13T17:17:44.979776+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T17:17:44.979776+0000","last_undegraded":"2024-09-13T17:17:44.979776+0000","last_fullsized":"2024-09-13T17:17:44.979776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19936,"num_write_kb":264332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"63'5003","reported_seq":5067,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993722+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T17:17:44.993722+0000","last_peered":"2024-09-13T17:17:44.993722+0000","last_clean":"2024-09-13T17:17:44.993722+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T17:17:44.993722+0000","last_undegraded":"2024-09-13T17:17:44.993722+0000","last_fullsized":"2024-09-13T17:17:44.993722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27137,"num_write_kb":387189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"63'4068","reported_seq":4136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.972641+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T17:17:44.972641+0000","last_peered":"2024-09-13T17:17:44.972641+0000","last_clean":"2024-09-13T17:17:44.972641+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T17:17:44.972641+0000","last_undegraded":"2024-09-13T17:17:44.972641+0000","last_fullsized":"2024-09-13T17:17:44.972641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22149,"num_write_kb":293545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"63'3283","reported_seq":3347,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007911+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T17:17:45.007911+0000","last_peered":"2024-09-13T17:17:45.007911+0000","last_clean":"2024-09-13T17:17:45.007911+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T17:17:45.007911+0000","last_undegraded":"2024-09-13T17:17:45.007911+0000","last_fullsized":"2024-09-13T17:17:45.007911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":248866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"63'4697","reported_seq":4765,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.977325+0000","last_change":"2024-09-13T16:47:59.053243+0000","last_active":"2024-09-13T17:17:44.977325+0000","last_peered":"2024-09-13T17:17:44.977325+0000","last_clean":"2024-09-13T17:17:44.977325+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T17:17:44.977325+0000","last_undegraded":"2024-09-13T17:17:44.977325+0000","last_fullsized":"2024-09-13T17:17:44.977325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25800,"num_write_kb":320606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"63'3820","reported_seq":3884,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974366+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T17:17:44.974366+0000","last_peered":"2024-09-13T17:17:44.974366+0000","last_clean":"2024-09-13T17:17:44.974366+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T17:17:44.974366+0000","last_undegraded":"2024-09-13T17:17:44.974366+0000","last_fullsized":"2024-09-13T17:17:44.974366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3820,"log_dups_size":0,"ondisk_log_size":3820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20813,"num_write_kb":300503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"63'4719","reported_seq":4800,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:56.892882+0000","last_change":"2024-09-13T17:17:56.893148+0000","last_active":"2024-09-13T17:17:56.892882+0000","last_peered":"2024-09-13T17:17:56.892882+0000","last_clean":"2024-09-13T17:17:56.892882+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T17:17:56.892882+0000","last_undegraded":"2024-09-13T17:17:56.892882+0000","last_fullsized":"2024-09-13T17:17:56.892882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4719","last_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_deep_scrub":"63'4719","last_deep_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_clean_scrub_stamp":"2024-09-13T17:17:56.892826+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:26.656787+0000","scrub_duration":0.032626751000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":25263,"num_write_kb":330973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"63'3722","reported_seq":3786,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975189+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T17:17:44.975189+0000","last_peered":"2024-09-13T17:17:44.975189+0000","last_clean":"2024-09-13T17:17:44.975189+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T17:17:44.975189+0000","last_undegraded":"2024-09-13T17:17:44.975189+0000","last_fullsized":"2024-09-13T17:17:44.975189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20022,"num_write_kb":289122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"63'3813","reported_seq":3881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996656+0000","last_change":"2024-09-13T16:47:59.052917+0000","last_active":"2024-09-13T17:17:44.996656+0000","last_peered":"2024-09-13T17:17:44.996656+0000","last_clean":"2024-09-13T17:17:44.996656+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T17:17:44.996656+0000","last_undegraded":"2024-09-13T17:17:44.996656+0000","last_fullsized":"2024-09-13T17:17:44.996656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21202,"num_write_kb":265282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"63'4479","reported_seq":4547,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976201+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T17:17:44.976201+0000","last_peered":"2024-09-13T17:17:44.976201+0000","last_clean":"2024-09-13T17:17:44.976201+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T17:17:44.976201+0000","last_undegraded":"2024-09-13T17:17:44.976201+0000","last_fullsized":"2024-09-13T17:17:44.976201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":24556,"num_write_kb":340437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"63'4549","reported_seq":4617,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996096+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T17:17:44.996096+0000","last_peered":"2024-09-13T17:17:44.996096+0000","last_clean":"2024-09-13T17:17:44.996096+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T17:17:44.996096+0000","last_undegraded":"2024-09-13T17:17:44.996096+0000","last_fullsized":"2024-09-13T17:17:44.996096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24549,"num_write_kb":319591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"63'3479","reported_seq":3544,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.977167+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T17:17:44.977167+0000","last_peered":"2024-09-13T17:17:44.977167+0000","last_clean":"2024-09-13T17:17:44.977167+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T17:17:44.977167+0000","last_undegraded":"2024-09-13T17:17:44.977167+0000","last_fullsized":"2024-09-13T17:17:44.977167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3479,"log_dups_size":0,"ondisk_log_size":3479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18966,"num_write_kb":268432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"63'3573","reported_seq":3641,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009368+0000","last_change":"2024-09-13T16:47:59.047402+0000","last_active":"2024-09-13T17:17:45.009368+0000","last_peered":"2024-09-13T17:17:45.009368+0000","last_clean":"2024-09-13T17:17:45.009368+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T17:17:45.009368+0000","last_undegraded":"2024-09-13T17:17:45.009368+0000","last_fullsized":"2024-09-13T17:17:45.009368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19348,"num_write_kb":270113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"63'4058","reported_seq":4122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991645+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T17:17:44.991645+0000","last_peered":"2024-09-13T17:17:44.991645+0000","last_clean":"2024-09-13T17:17:44.991645+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T17:17:44.991645+0000","last_undegraded":"2024-09-13T17:17:44.991645+0000","last_fullsized":"2024-09-13T17:17:44.991645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22331,"num_write_kb":302585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"63'5839","reported_seq":5906,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.987605+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T17:17:44.987605+0000","last_peered":"2024-09-13T17:17:44.987605+0000","last_clean":"2024-09-13T17:17:44.987605+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T17:17:44.987605+0000","last_undegraded":"2024-09-13T17:17:44.987605+0000","last_fullsized":"2024-09-13T17:17:44.987605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31756,"num_write_kb":397521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"63'3835","reported_seq":3899,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992992+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T17:17:44.992992+0000","last_peered":"2024-09-13T17:17:44.992992+0000","last_clean":"2024-09-13T17:17:44.992992+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T17:17:44.992992+0000","last_undegraded":"2024-09-13T17:17:44.992992+0000","last_fullsized":"2024-09-13T17:17:44.992992+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":21185,"num_write_kb":306173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"63'4134","reported_seq":4212,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.595557+0000","last_change":"2024-09-13T17:17:57.595782+0000","last_active":"2024-09-13T17:17:57.595557+0000","last_peered":"2024-09-13T17:17:57.595557+0000","last_clean":"2024-09-13T17:17:57.595557+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T17:17:57.595557+0000","last_undegraded":"2024-09-13T17:17:57.595557+0000","last_fullsized":"2024-09-13T17:17:57.595557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4134","last_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_deep_scrub":"63'4134","last_deep_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.595505+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:50.101213+0000","scrub_duration":0.024248147000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22268,"num_write_kb":281801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"63'5073","reported_seq":5138,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.981849+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T17:17:44.981849+0000","last_peered":"2024-09-13T17:17:44.981849+0000","last_clean":"2024-09-13T17:17:44.981849+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T17:17:44.981849+0000","last_undegraded":"2024-09-13T17:17:44.981849+0000","last_fullsized":"2024-09-13T17:17:44.981849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":28693,"num_write_kb":393724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"63'3822","reported_seq":3889,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994849+0000","last_change":"2024-09-13T16:47:59.039035+0000","last_active":"2024-09-13T17:17:44.994849+0000","last_peered":"2024-09-13T17:17:44.994849+0000","last_clean":"2024-09-13T17:17:44.994849+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T17:17:44.994849+0000","last_undegraded":"2024-09-13T17:17:44.994849+0000","last_fullsized":"2024-09-13T17:17:44.994849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20823,"num_write_kb":275830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"63'2778","reported_seq":2844,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975133+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T17:17:44.975133+0000","last_peered":"2024-09-13T17:17:44.975133+0000","last_clean":"2024-09-13T17:17:44.975133+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T17:17:44.975133+0000","last_undegraded":"2024-09-13T17:17:44.975133+0000","last_fullsized":"2024-09-13T17:17:44.975133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":34,"num_write":15212,"num_write_kb":208573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"63'3975","reported_seq":4053,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.856701+0000","last_change":"2024-09-13T17:18:01.856922+0000","last_active":"2024-09-13T17:18:01.856701+0000","last_peered":"2024-09-13T17:18:01.856701+0000","last_clean":"2024-09-13T17:18:01.856701+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T17:18:01.856701+0000","last_undegraded":"2024-09-13T17:18:01.856701+0000","last_fullsized":"2024-09-13T17:18:01.856701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3975","last_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_deep_scrub":"63'3975","last_deep_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.856645+0000","objects_scrubbed":0,"log_size":3975,"log_dups_size":0,"ondisk_log_size":3975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:51.096023+0000","scrub_duration":0.021351690999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21351,"num_write_kb":278200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"63'3844","reported_seq":3909,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994868+0000","last_change":"2024-09-13T16:48:08.195038+0000","last_active":"2024-09-13T17:17:44.994868+0000","last_peered":"2024-09-13T17:17:44.994868+0000","last_clean":"2024-09-13T17:17:44.994868+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T17:17:44.994868+0000","last_undegraded":"2024-09-13T17:17:44.994868+0000","last_fullsized":"2024-09-13T17:17:44.994868+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21085,"num_write_kb":289508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"63'3324","reported_seq":3391,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974142+0000","last_change":"2024-09-13T16:47:59.059644+0000","last_active":"2024-09-13T17:17:44.974142+0000","last_peered":"2024-09-13T17:17:44.974142+0000","last_clean":"2024-09-13T17:17:44.974142+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T17:17:44.974142+0000","last_undegraded":"2024-09-13T17:17:44.974142+0000","last_fullsized":"2024-09-13T17:17:44.974142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3324,"log_dups_size":0,"ondisk_log_size":3324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18810,"num_write_kb":235312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"63'3743","reported_seq":3807,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.972429+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T17:17:44.972429+0000","last_peered":"2024-09-13T17:17:44.972429+0000","last_clean":"2024-09-13T17:17:44.972429+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T17:17:44.972429+0000","last_undegraded":"2024-09-13T17:17:44.972429+0000","last_fullsized":"2024-09-13T17:17:44.972429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20790,"num_write_kb":280180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"63'3173","reported_seq":3240,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978903+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T17:17:44.978903+0000","last_peered":"2024-09-13T17:17:44.978903+0000","last_clean":"2024-09-13T17:17:44.978903+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T17:17:44.978903+0000","last_undegraded":"2024-09-13T17:17:44.978903+0000","last_fullsized":"2024-09-13T17:17:44.978903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17044,"num_write_kb":219853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"63'3460","reported_seq":3524,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007456+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T17:17:45.007456+0000","last_peered":"2024-09-13T17:17:45.007456+0000","last_clean":"2024-09-13T17:17:45.007456+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T17:17:45.007456+0000","last_undegraded":"2024-09-13T17:17:45.007456+0000","last_fullsized":"2024-09-13T17:17:45.007456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":19788,"num_write_kb":263976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"63'3747","reported_seq":3814,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994889+0000","last_change":"2024-09-13T16:47:59.054642+0000","last_active":"2024-09-13T17:17:44.994889+0000","last_peered":"2024-09-13T17:17:44.994889+0000","last_clean":"2024-09-13T17:17:44.994889+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T17:17:44.994889+0000","last_undegraded":"2024-09-13T17:17:44.994889+0000","last_fullsized":"2024-09-13T17:17:44.994889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20231,"num_write_kb":269532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"63'3422","reported_seq":3485,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992876+0000","last_change":"2024-09-13T16:48:08.192709+0000","last_active":"2024-09-13T17:17:44.992876+0000","last_peered":"2024-09-13T17:17:44.992876+0000","last_clean":"2024-09-13T17:17:44.992876+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T17:17:44.992876+0000","last_undegraded":"2024-09-13T17:17:44.992876+0000","last_fullsized":"2024-09-13T17:17:44.992876+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18722,"num_write_kb":267617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"63'4803","reported_seq":4881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.621276+0000","last_change":"2024-09-13T17:17:58.621508+0000","last_active":"2024-09-13T17:17:58.621276+0000","last_peered":"2024-09-13T17:17:58.621276+0000","last_clean":"2024-09-13T17:17:58.621276+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T17:17:58.621276+0000","last_undegraded":"2024-09-13T17:17:58.621276+0000","last_fullsized":"2024-09-13T17:17:58.621276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4803","last_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_deep_scrub":"63'4803","last_deep_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.621222+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:06.376472+0000","scrub_duration":0.023354388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26967,"num_write_kb":333429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"63'3840","reported_seq":3903,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980170+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T17:17:44.980170+0000","last_peered":"2024-09-13T17:17:44.980170+0000","last_clean":"2024-09-13T17:17:44.980170+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T17:17:44.980170+0000","last_undegraded":"2024-09-13T17:17:44.980170+0000","last_fullsized":"2024-09-13T17:17:44.980170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21113,"num_write_kb":293770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"63'5433","reported_seq":5513,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.303582+0000","last_change":"2024-09-13T17:17:53.303824+0000","last_active":"2024-09-13T17:17:53.303582+0000","last_peered":"2024-09-13T17:17:53.303582+0000","last_clean":"2024-09-13T17:17:53.303582+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T17:17:53.303582+0000","last_undegraded":"2024-09-13T17:17:53.303582+0000","last_fullsized":"2024-09-13T17:17:53.303582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5433","last_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_deep_scrub":"63'5433","last_deep_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.303524+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.191620+0000","scrub_duration":0.041011686999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30664,"num_write_kb":389336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"63'4741","reported_seq":4808,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974867+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T17:17:44.974867+0000","last_peered":"2024-09-13T17:17:44.974867+0000","last_clean":"2024-09-13T17:17:44.974867+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T17:17:44.974867+0000","last_undegraded":"2024-09-13T17:17:44.974867+0000","last_fullsized":"2024-09-13T17:17:44.974867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":15,"num_write":25588,"num_write_kb":340391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"63'4281","reported_seq":4359,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.227117+0000","last_change":"2024-09-13T17:17:57.227117+0000","last_active":"2024-09-13T17:17:57.227117+0000","last_peered":"2024-09-13T17:17:57.227117+0000","last_clean":"2024-09-13T17:17:57.227117+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T17:17:57.227117+0000","last_undegraded":"2024-09-13T17:17:57.227117+0000","last_fullsized":"2024-09-13T17:17:57.227117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4281","last_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_deep_scrub":"63'4281","last_deep_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226795+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.732503+0000","scrub_duration":0.034883109000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23373,"num_write_kb":302635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"63'3575","reported_seq":3643,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994905+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T17:17:44.994905+0000","last_peered":"2024-09-13T17:17:44.994905+0000","last_clean":"2024-09-13T17:17:44.994905+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T17:17:44.994905+0000","last_undegraded":"2024-09-13T17:17:44.994905+0000","last_fullsized":"2024-09-13T17:17:44.994905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":7,"num_write":20117,"num_write_kb":289677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"63'4404","reported_seq":4482,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.337858+0000","last_change":"2024-09-13T17:17:50.337858+0000","last_active":"2024-09-13T17:17:50.337858+0000","last_peered":"2024-09-13T17:17:50.337858+0000","last_clean":"2024-09-13T17:17:50.337858+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T17:17:50.337858+0000","last_undegraded":"2024-09-13T17:17:50.337858+0000","last_fullsized":"2024-09-13T17:17:50.337858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4404","last_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_deep_scrub":"63'4404","last_deep_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.337585+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.971802+0000","scrub_duration":0.031410212999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24308,"num_write_kb":306423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"63'3318","reported_seq":3384,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.995979+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T17:17:44.995979+0000","last_peered":"2024-09-13T17:17:44.995979+0000","last_clean":"2024-09-13T17:17:44.995979+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T17:17:44.995979+0000","last_undegraded":"2024-09-13T17:17:44.995979+0000","last_fullsized":"2024-09-13T17:17:44.995979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3318,"log_dups_size":0,"ondisk_log_size":3318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18295,"num_write_kb":253537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"63'4013","reported_seq":4091,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.598539+0000","last_change":"2024-09-13T17:17:53.598539+0000","last_active":"2024-09-13T17:17:53.598539+0000","last_peered":"2024-09-13T17:17:53.598539+0000","last_clean":"2024-09-13T17:17:53.598539+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T17:17:53.598539+0000","last_undegraded":"2024-09-13T17:17:53.598539+0000","last_fullsized":"2024-09-13T17:17:53.598539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4013","last_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_deep_scrub":"63'4013","last_deep_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.598242+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.486338+0000","scrub_duration":0.028004864000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21393,"num_write_kb":262042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"63'3658","reported_seq":3724,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.982869+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T17:17:44.982869+0000","last_peered":"2024-09-13T17:17:44.982869+0000","last_clean":"2024-09-13T17:17:44.982869+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T17:17:44.982869+0000","last_undegraded":"2024-09-13T17:17:44.982869+0000","last_fullsized":"2024-09-13T17:17:44.982869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19946,"num_write_kb":268446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"63'3910","reported_seq":3977,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591116+0000","last_change":"2024-09-13T16:47:59.052281+0000","last_active":"2024-09-13T17:17:49.591116+0000","last_peered":"2024-09-13T17:17:49.591116+0000","last_clean":"2024-09-13T17:17:49.591116+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T17:17:49.591116+0000","last_undegraded":"2024-09-13T17:17:49.591116+0000","last_fullsized":"2024-09-13T17:17:49.591116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3910,"log_dups_size":0,"ondisk_log_size":3910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":278651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"63'3074","reported_seq":3136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.971975+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T17:17:44.971975+0000","last_peered":"2024-09-13T17:17:44.971975+0000","last_clean":"2024-09-13T17:17:44.971975+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T17:17:44.971975+0000","last_undegraded":"2024-09-13T17:17:44.971975+0000","last_fullsized":"2024-09-13T17:17:44.971975+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":246226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"63'3515","reported_seq":3596,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.560145+0000","last_change":"2024-09-13T17:17:52.560367+0000","last_active":"2024-09-13T17:17:52.560145+0000","last_peered":"2024-09-13T17:17:52.560145+0000","last_clean":"2024-09-13T17:17:52.560145+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T17:17:52.560145+0000","last_undegraded":"2024-09-13T17:17:52.560145+0000","last_fullsized":"2024-09-13T17:17:52.560145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3515","last_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_deep_scrub":"63'3515","last_deep_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.560093+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.500715+0000","scrub_duration":0.026529094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19512,"num_write_kb":258445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"63'3439","reported_seq":3502,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.006251+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T17:17:45.006251+0000","last_peered":"2024-09-13T17:17:45.006251+0000","last_clean":"2024-09-13T17:17:45.006251+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T17:17:45.006251+0000","last_undegraded":"2024-09-13T17:17:45.006251+0000","last_fullsized":"2024-09-13T17:17:45.006251+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":17681,"num_write_kb":263603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"63'3892","reported_seq":3962,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.995220+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T17:17:44.995220+0000","last_peered":"2024-09-13T17:17:44.995220+0000","last_clean":"2024-09-13T17:17:44.995220+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T17:17:44.995220+0000","last_undegraded":"2024-09-13T17:17:44.995220+0000","last_fullsized":"2024-09-13T17:17:44.995220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21304,"num_write_kb":286190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"63'3622","reported_seq":3684,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991777+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T17:17:44.991777+0000","last_peered":"2024-09-13T17:17:44.991777+0000","last_clean":"2024-09-13T17:17:44.991777+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T17:17:44.991777+0000","last_undegraded":"2024-09-13T17:17:44.991777+0000","last_fullsized":"2024-09-13T17:17:44.991777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19729,"num_write_kb":275691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"63'5036","reported_seq":5105,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994403+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T17:17:44.994403+0000","last_peered":"2024-09-13T17:17:44.994403+0000","last_clean":"2024-09-13T17:17:44.994403+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T17:17:44.994403+0000","last_undegraded":"2024-09-13T17:17:44.994403+0000","last_fullsized":"2024-09-13T17:17:44.994403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27469,"num_write_kb":368239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"63'3689","reported_seq":3751,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991999+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T17:17:44.991999+0000","last_peered":"2024-09-13T17:17:44.991999+0000","last_clean":"2024-09-13T17:17:44.991999+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T17:17:44.991999+0000","last_undegraded":"2024-09-13T17:17:44.991999+0000","last_fullsized":"2024-09-13T17:17:44.991999+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3689,"log_dups_size":0,"ondisk_log_size":3689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20818,"num_write_kb":284187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"63'4047","reported_seq":4128,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.364122+0000","last_change":"2024-09-13T17:17:51.364373+0000","last_active":"2024-09-13T17:17:51.364122+0000","last_peered":"2024-09-13T17:17:51.364122+0000","last_clean":"2024-09-13T17:17:51.364122+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T17:17:51.364122+0000","last_undegraded":"2024-09-13T17:17:51.364122+0000","last_fullsized":"2024-09-13T17:17:51.364122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4047","last_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_deep_scrub":"63'4047","last_deep_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.364064+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:46.022668+0000","scrub_duration":0.028243739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22743,"num_write_kb":291925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":197,"num_write":1329762,"num_write_kb":18444058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242925,"ondisk_log_size":242925,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19024977,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1583,"num_read_kb":206738,"num_write":4472,"num_write_kb":20527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38109184,"data_stored":38049954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4364,"ondisk_log_size":4364,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":1479870,"num_write_kb":19043750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270762,"ondisk_log_size":270762,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":19675678,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1366,"num_read_kb":209632,"num_write":4521,"num_write_kb":21162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39411712,"data_stored":39351356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4373,"ondisk_log_size":4373,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575216,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":942784,"kb_used_data":13688,"kb_used_omap":1,"kb_used_meta":929086,"kb_avail":93429056,"statfs":{"total":96636764160,"available":95671353344,"internally_reserved":0,"allocated":14016512,"data_stored":13616965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":951384523},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":2.1549999999999998,"15min":1.284},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":15.93,"5min":16.571999999999999,"15min":16.571999999999999},"last":0.754},{"interface":"front","average":{"1min":2.089,"5min":2.2909999999999999,"15min":1.3859999999999999},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.20899999999999999},"max":{"1min":14.792,"5min":16.523,"15min":16.523},"last":0.97199999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":2.927,"15min":1.5589999999999999},"min":{"1min":0.27800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":15.965999999999999,"5min":59.911999999999999,"15min":59.911999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":2.48,"5min":3.1080000000000001,"15min":1.6699999999999999},"min":{"1min":0.38400000000000001,"5min":0.28399999999999997,"15min":0.23000000000000001},"max":{"1min":14.823,"5min":59.895000000000003,"15min":59.895000000000003},"last":1.1060000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.798,"15min":1.153},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.25700000000000001},"max":{"1min":6.0039999999999996,"5min":16.023,"15min":16.023},"last":0.88500000000000001},{"interface":"front","average":{"1min":2.5670000000000002,"5min":2.0030000000000001,"15min":1.2669999999999999},"min":{"1min":0.34300000000000003,"5min":0.249,"15min":0.249},"max":{"1min":18.734000000000002,"5min":18.734000000000002,"15min":18.734000000000002},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.1640000000000001,"5min":2.1989999999999998,"15min":1.306},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.28299999999999997},"max":{"1min":13.859999999999999,"5min":23.225000000000001,"15min":23.225000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":3.673,"5min":2.4900000000000002,"15min":1.46},"min":{"1min":0.48799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":19.709,"5min":19.709,"15min":19.709},"last":0.91800000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.7679999999999998,"5min":2.4390000000000001,"15min":1.399},"min":{"1min":0.41799999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":16.555,"5min":18.004999999999999,"15min":18.004999999999999},"last":1.056},{"interface":"front","average":{"1min":2.294,"5min":2.5939999999999999,"15min":1.484},"min":{"1min":0.309,"5min":0.245,"15min":0.245},"max":{"1min":16.684000000000001,"5min":22.315999999999999,"15min":22.315999999999999},"last":1.014}]},{"osd":5,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.657,"5min":2.6459999999999999,"15min":1.5289999999999999},"min":{"1min":0.47799999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":17.321000000000002,"5min":37.302999999999997,"15min":37.302999999999997},"last":1.0409999999999999},{"interface":"front","average":{"1min":2.5619999999999998,"5min":2.0800000000000001,"15min":1.3700000000000001},"min":{"1min":0.42899999999999999,"5min":0.32500000000000001,"15min":0.316},"max":{"1min":14.991,"5min":16.867999999999999,"15min":16.867999999999999},"last":1.21}]},{"osd":6,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.4529999999999998,"5min":2.0979999999999999,"15min":1.3859999999999999},"min":{"1min":0.496,"5min":0.28000000000000003,"15min":0.23000000000000001},"max":{"1min":18.335000000000001,"5min":18.335000000000001,"15min":18.335000000000001},"last":1.147},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1549999999999998,"15min":1.397},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.28100000000000003},"max":{"1min":9.0739999999999998,"5min":19.244,"15min":19.244},"last":1.085}]}]},{"osd":6,"up_from":13,"seq":55834575214,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673216,"kb_used_data":11768,"kb_used_omap":1,"kb_used_meta":661438,"kb_avail":93698624,"statfs":{"total":96636764160,"available":95947390976,"internally_reserved":0,"allocated":12050432,"data_stored":11645255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":677312968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.0700000000000001,"15min":0.89000000000000001},"min":{"1min":0.27500000000000002,"5min":0.222,"15min":0.222},"max":{"1min":6.0780000000000003,"5min":6.335,"15min":6.335},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.052,"5min":1.133,"15min":0.91200000000000003},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":4.907,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":1.198,"15min":0.95099999999999996},"min":{"1min":0.24299999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.474,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.736,"5min":1.6850000000000001,"15min":1.0980000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":7.3600000000000003,"5min":10.111000000000001,"15min":10.111000000000001},"last":0.77600000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.234,"15min":0.97799999999999998},"min":{"1min":0.432,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":15.638,"5min":15.638,"15min":15.638},"last":0.68600000000000005},{"interface":"front","average":{"1min":2.1960000000000002,"5min":1.4450000000000001,"15min":1.091},"min":{"1min":0.41999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":19.442,"5min":19.442,"15min":19.442},"last":1.0649999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.5840000000000001,"15min":1.1259999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":3.7389999999999999,"5min":15.247999999999999,"15min":15.247999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.6399999999999999,"15min":1.161},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.1379999999999999,"5min":15.179,"15min":15.179},"last":1.0009999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":1.4810000000000001,"15min":1.1160000000000001},"min":{"1min":0.35299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":14.114000000000001,"5min":15.326000000000001,"15min":15.326000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.8799999999999999,"5min":1.6619999999999999,"15min":1.1930000000000001},"min":{"1min":0.27300000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":14.057,"5min":15.026,"15min":15.026},"last":1.038}]},{"osd":5,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.6659999999999999,"15min":1.242},"min":{"1min":0.42699999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.2690000000000001,"5min":11.566000000000001,"15min":11.566000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.4570000000000001,"15min":1.1879999999999999},"min":{"1min":0.36199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":6.2789999999999999,"5min":10.223000000000001,"15min":10.223000000000001},"last":1.1319999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.6499999999999999,"15min":1.238},"min":{"1min":0.318,"5min":0.311,"15min":0.311},"max":{"1min":4.0979999999999999,"5min":13.112,"15min":13.112},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.782,"15min":1.274},"min":{"1min":0.39700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.6679999999999999,"5min":14.4,"15min":14.4},"last":0.90300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575215,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":957788,"kb_used_data":5012,"kb_used_omap":1,"kb_used_meta":952766,"kb_avail":93414052,"statfs":{"total":96636764160,"available":95655989248,"internally_reserved":0,"allocated":5132288,"data_stored":4730805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":975632838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":3.762,"5min":1.978,"15min":1.1919999999999999},"min":{"1min":0.246,"5min":0.215,"15min":0.215},"max":{"1min":27.952999999999999,"5min":27.952999999999999,"15min":27.952999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":2.5059999999999998,"5min":1.629,"15min":1.121},"min":{"1min":0.28100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":23.619,"5min":23.619,"15min":23.619},"last":1.0229999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.8210000000000002,"5min":1.8109999999999999,"15min":1.1379999999999999},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":13.515000000000001,"5min":13.515000000000001,"15min":13.515000000000001},"last":1.091},{"interface":"front","average":{"1min":1.621,"5min":1.9299999999999999,"15min":1.1839999999999999},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":6.9290000000000003,"5min":35.186999999999998,"15min":35.186999999999998},"last":0.73899999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":2.306,"15min":1.3420000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.609,"5min":51.18,"15min":51.18},"last":1.119},{"interface":"front","average":{"1min":1.732,"5min":1.929,"15min":1.2949999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.8659999999999997,"5min":48.756,"15min":48.756},"last":1.1559999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.7480000000000002,"5min":2.0219999999999998,"15min":1.2789999999999999},"min":{"1min":0.39200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":15.946999999999999,"5min":19.800000000000001,"15min":19.800000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.7649999999999999,"15min":1.256},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":9.1020000000000003,"5min":9.4789999999999992,"15min":9.4789999999999992},"last":1.202}]},{"osd":4,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":4.2270000000000003,"5min":2.4260000000000002,"15min":1.379},"min":{"1min":0.40699999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":20.713999999999999,"5min":22.361999999999998,"15min":22.361999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":4.1710000000000003,"5min":2.9940000000000002,"15min":1.641},"min":{"1min":0.35299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":28.062999999999999,"5min":31.843,"15min":31.843},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.427,"15min":1.131},"min":{"1min":0.375,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.3300000000000001,"5min":8.1910000000000007,"15min":8.1910000000000007},"last":0.94099999999999995},{"interface":"front","average":{"1min":2.052,"5min":1.7649999999999999,"15min":1.26},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.1059999999999999,"5min":13.031000000000001,"15min":13.031000000000001},"last":0.875}]},{"osd":7,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":2.1890000000000001,"15min":1.4139999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":11.239000000000001,"5min":35.124000000000002,"15min":35.124000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":2.0550000000000002,"5min":2.5089999999999999,"15min":1.546},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.99,"5min":32.328000000000003,"15min":32.328000000000003},"last":1.2450000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575214,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1011352,"kb_used_data":13968,"kb_used_omap":1,"kb_used_meta":997374,"kb_avail":93360488,"statfs":{"total":96636764160,"available":95601139712,"internally_reserved":0,"allocated":14303232,"data_stored":13890488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1021311430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":2.4870000000000001,"5min":1.8520000000000001,"15min":1.155},"min":{"1min":0.47399999999999998,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":18.841000000000001,"5min":18.841000000000001,"15min":18.841000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":2.6619999999999999,"5min":1.9570000000000001,"15min":1.2609999999999999},"min":{"1min":0.33800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":20.312000000000001,"5min":20.312000000000001,"15min":20.312000000000001},"last":0.80700000000000005}]},{"osd":1,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":1.7030000000000001,"15min":1.1020000000000001},"min":{"1min":0.27100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0840000000000001,"5min":27.242000000000001,"15min":27.242000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":2.1850000000000001,"5min":2.2829999999999999,"15min":1.3220000000000001},"min":{"1min":0.39100000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.103999999999999,"5min":26.545999999999999,"15min":26.545999999999999},"last":1.099}]},{"osd":2,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.5880000000000001,"15min":1.1080000000000001},"min":{"1min":0.29199999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8839999999999999,"5min":11.803000000000001,"15min":11.803000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.825,"15min":1.2210000000000001},"min":{"1min":0.55300000000000005,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":13.972,"5min":13.972,"15min":13.972},"last":1.1659999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":2.3969999999999998,"15min":1.409},"min":{"1min":0.39900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":10.763,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.874,"5min":1.9550000000000001,"15min":1.2909999999999999},"min":{"1min":0.33200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2,"5min":20.468,"15min":20.468},"last":1.0329999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":2.645,"5min":1.796,"15min":1.2010000000000001},"min":{"1min":0.42299999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":14.031000000000001,"5min":14.031000000000001,"15min":14.031000000000001},"last":1.214},{"interface":"front","average":{"1min":2.4180000000000001,"5min":2.1339999999999999,"15min":1.363},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":14.205,"5min":15.044,"15min":15.044},"last":1.071}]},{"osd":6,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.546,"15min":1.155},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.7160000000000002,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":0.94099999999999995},{"interface":"front","average":{"1min":2.2970000000000002,"5min":1.911,"15min":1.3169999999999999},"min":{"1min":0.45100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":14.131,"5min":14.131,"15min":14.131},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":3.056,"5min":2.2269999999999999,"15min":1.478},"min":{"1min":0.36899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":20.154,"5min":24.518999999999998,"15min":24.518999999999998},"last":1},{"interface":"front","average":{"1min":1.984,"5min":1.851,"15min":1.3640000000000001},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":15.209,"5min":15.209,"15min":15.209},"last":1.1339999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575215,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":958484,"kb_used_data":19404,"kb_used_omap":1,"kb_used_meta":939070,"kb_avail":93413356,"statfs":{"total":96636764160,"available":95655276544,"internally_reserved":0,"allocated":19869696,"data_stored":19459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":961608136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.5880000000000001,"15min":0.97999999999999998},"min":{"1min":0.19400000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":19.327999999999999,"5min":19.327999999999999,"15min":19.327999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":1.831,"5min":1.7549999999999999,"15min":1.0449999999999999},"min":{"1min":0.29699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":17.754000000000001,"5min":20.699999999999999,"15min":20.699999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":1.3720000000000001,"15min":0.90300000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.3340000000000001,"5min":16.100000000000001,"15min":16.100000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.3260000000000001,"15min":0.97499999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.847,"5min":10.417,"15min":10.417},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3280000000000001,"15min":0.96699999999999997},"min":{"1min":0.38800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.6509999999999998,"5min":8.5950000000000006,"15min":8.5950000000000006},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.996,"5min":1.276,"15min":0.96499999999999997},"min":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.5380000000000003,"5min":15.433999999999999,"15min":15.433999999999999},"last":1.02}]},{"osd":4,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":3.4470000000000001,"5min":2.1099999999999999,"15min":1.3500000000000001},"min":{"1min":0.308,"5min":0.30599999999999999,"15min":0.29599999999999999},"max":{"1min":15.919,"5min":15.919,"15min":15.919},"last":1.069},{"interface":"front","average":{"1min":2.3170000000000002,"5min":1.712,"15min":1.2110000000000001},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":9.3819999999999997,"5min":9.3819999999999997,"15min":9.3819999999999997},"last":0.99199999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.9119999999999999,"15min":1.321},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":9.4030000000000005,"5min":31.187999999999999,"15min":31.187999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":2.427,"5min":2.121,"15min":1.383},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":22.951000000000001,"5min":22.951000000000001,"15min":22.951000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.873,"15min":1.3120000000000001},"min":{"1min":0.44400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":4.202,"5min":12.567,"15min":12.567},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.226,"5min":2.1779999999999999,"15min":1.4730000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.7380000000000004,"5min":15.882,"15min":15.882},"last":1.1679999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":4.8339999999999996,"5min":2.4390000000000001,"15min":1.538},"min":{"1min":0.48999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":33.969000000000001,"5min":33.969000000000001,"15min":33.969000000000001},"last":1.198},{"interface":"front","average":{"1min":1.4399999999999999,"5min":2.2480000000000002,"15min":1.46},"min":{"1min":0.47799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.4550000000000001,"5min":25.652999999999999,"15min":25.652999999999999},"last":1.3009999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575215,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662284,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":661374,"kb_avail":93709556,"statfs":{"total":96636764160,"available":95958585344,"internally_reserved":0,"allocated":921600,"data_stored":527729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":677247430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.90800000000000003,"15min":0.72499999999999998},"min":{"1min":0.28499999999999998,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":3.6509999999999998,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.516,"15min":1.0329999999999999},"min":{"1min":0.26300000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":6.6539999999999999,"5min":9.2729999999999997,"15min":9.2729999999999997},"last":0.80400000000000005}]},{"osd":1,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":1.7110000000000001,"15min":1.0700000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20000000000000001},"max":{"1min":2.7450000000000001,"5min":33.579999999999998,"15min":33.579999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.7310000000000001,"15min":1.119},"min":{"1min":0.35199999999999998,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":3.7029999999999998,"5min":23.965,"15min":23.965},"last":0.97199999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":2.2069999999999999,"15min":1.3049999999999999},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.245},"max":{"1min":10.137,"5min":22.760000000000002,"15min":22.760000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.738,"15min":1.127},"min":{"1min":0.28799999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":8.4849999999999994,"5min":10.372,"15min":10.372},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.9850000000000001,"15min":1.2909999999999999},"min":{"1min":0.35799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.6400000000000001,"5min":20.673999999999999,"15min":20.673999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.395,"5min":1.7410000000000001,"15min":1.2709999999999999},"min":{"1min":0.54400000000000004,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.6699999999999999,"5min":16.5,"15min":16.5},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":2.484,"15min":1.528},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.798,"5min":21.722000000000001,"15min":21.722000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":2.222,"15min":1.427},"min":{"1min":0.53800000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":4.9189999999999996,"5min":29.620999999999999,"15min":29.620999999999999},"last":1.0329999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.702,"15min":1.2889999999999999},"min":{"1min":0.51300000000000001,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":6.5880000000000001,"5min":21.882999999999999,"15min":21.882999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.748,"5min":2.2210000000000001,"15min":1.4710000000000001},"min":{"1min":0.34799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.407,"5min":25.274999999999999,"15min":25.274999999999999},"last":1.145}]},{"osd":7,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":3.2799999999999998,"15min":1.859},"min":{"1min":0.61199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":7.0119999999999996,"5min":43.649000000000001,"15min":43.649000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":3.1589999999999998,"15min":1.855},"min":{"1min":0.40200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.6949999999999998,"5min":27.417999999999999,"15min":27.417999999999999},"last":1.093}]}]},{"osd":1,"up_from":13,"seq":55834575214,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798984,"kb_used_data":9024,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572856,"statfs":{"total":96636764160,"available":95818604544,"internally_reserved":0,"allocated":9240576,"data_stored":8832475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":808909254},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.8620000000000001,"15min":1.0800000000000001},"min":{"1min":0.185,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":4.851,"5min":15.81,"15min":15.81},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.681,"15min":1.0609999999999999},"min":{"1min":0.188,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":2.8319999999999999,"5min":15.823,"15min":15.823},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.411,"15min":0.92200000000000004},"min":{"1min":0.215,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":3.8119999999999998,"5min":11.672000000000001,"15min":11.672000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.2929999999999999,"15min":0.95299999999999996},"min":{"1min":0.32200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.6890000000000001,"5min":11.722,"15min":11.722},"last":0.86599999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":2.024,"15min":1.2190000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.22},"max":{"1min":6.6379999999999999,"5min":16.507999999999999,"15min":16.507999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.016,"5min":1.482,"15min":1.0169999999999999},"min":{"1min":0.34799999999999998,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":4.3440000000000003,"5min":12.047000000000001,"15min":12.047000000000001},"last":0.67800000000000005}]},{"osd":4,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":2.1299999999999999,"15min":1.27},"min":{"1min":0.35899999999999999,"5min":0.27800000000000002,"15min":0.24399999999999999},"max":{"1min":3.7530000000000001,"5min":22.234000000000002,"15min":22.234000000000002},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.431,"5min":2.258,"15min":1.403},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":8.1240000000000006,"5min":20.448,"15min":20.448},"last":1.056}]},{"osd":5,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.679,"15min":1.2010000000000001},"min":{"1min":0.44900000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":4.3090000000000002,"5min":15.788,"15min":15.788},"last":0.93700000000000006},{"interface":"front","average":{"1min":2.5659999999999998,"5min":2.1709999999999998,"15min":1.359},"min":{"1min":0.33400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":24.763999999999999,"5min":24.763999999999999,"15min":24.763999999999999},"last":0.97499999999999998}]},{"osd":6,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.3879999999999999,"15min":1.139},"min":{"1min":0.43099999999999999,"5min":0.255,"15min":0.255},"max":{"1min":2.7200000000000002,"5min":12.019,"15min":12.019},"last":1.02},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.857,"15min":1.272},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.7970000000000002,"5min":11.628,"15min":11.628},"last":1.034}]},{"osd":7,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":2.02,"15min":1.405},"min":{"1min":0.50600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":6.9850000000000003,"5min":21.161000000000001,"15min":21.161000000000001},"last":1},{"interface":"front","average":{"1min":1.2230000000000001,"5min":2.4969999999999999,"15min":1.554},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.1980000000000004,"5min":29.323,"15min":29.323},"last":1.0720000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575214,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799556,"kb_used_data":9596,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572284,"statfs":{"total":96636764160,"available":95818018816,"internally_reserved":0,"allocated":9826304,"data_stored":9422627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":808909256},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.4610000000000001,"15min":0.90800000000000003},"min":{"1min":0.22,"5min":0.20000000000000001,"15min":0.189},"max":{"1min":1.677,"5min":59.031999999999996,"15min":59.031999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":2.0870000000000002,"5min":2.351,"15min":1.2869999999999999},"min":{"1min":0.23499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":9.8800000000000008,"5min":57.186999999999998,"15min":57.186999999999998},"last":0.79700000000000004}]},{"osd":2,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.96,"5min":1.155,"15min":0.871},"min":{"1min":0.26700000000000002,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":12.971,"5min":12.971,"15min":12.971},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.9990000000000001,"5min":1.371,"15min":0.93400000000000005},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":14.526,"5min":14.526,"15min":14.526},"last":0.75700000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.101,"5min":2.6800000000000002,"15min":1.4370000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":23.495999999999999,"5min":65.215000000000003,"15min":65.215000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.8169999999999999,"15min":1.1479999999999999},"min":{"1min":0.435,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":6.3479999999999999,"5min":62.722999999999999,"15min":62.722999999999999},"last":0.89500000000000002}]},{"osd":4,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.74,"5min":1.972,"15min":1.2569999999999999},"min":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":14.464,"5min":14.694000000000001,"15min":14.694000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.3320000000000001,"15min":1.069},"min":{"1min":0.29699999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.698,"5min":10.036,"15min":10.036},"last":0.96399999999999997}]},{"osd":5,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.145,"5min":1.9450000000000001,"15min":1.2689999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":10.769,"5min":21.940999999999999,"15min":21.940999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.889,"5min":1.996,"15min":1.3620000000000001},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":9.8200000000000003,"5min":9.8200000000000003,"15min":9.8200000000000003},"last":0.98699999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":1.4410000000000001,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.819000000000001,"5min":10.819000000000001,"15min":10.819000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.5900000000000001,"15min":1.2130000000000001},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":10.741,"5min":10.741,"15min":10.741},"last":1.0669999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":3.0329999999999999,"5min":1.9139999999999999,"15min":1.3600000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":32.975999999999999,"5min":32.975999999999999,"15min":32.975999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":2.468,"5min":2.7930000000000001,"15min":1.6830000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.40300000000000002},"max":{"1min":17.484000000000002,"5min":32.069000000000003,"15min":32.069000000000003},"last":0.93500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":84646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8482816,"data_stored":8473926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7020544,"data_stored":7006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7016448,"data_stored":7007322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10542867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4296704,"data_stored":4287702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:18:10.395 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.395 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.395 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.396 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.396 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.396 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.396 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.396 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.396 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.396 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.397 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.397 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.397 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.398 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.398 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.398 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.398 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.398 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.398 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.398 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.398 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.399 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.399 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.399 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.399 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.399 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.399 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.399 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.399 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.399 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.400 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.400 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.400 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.400 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.400 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.400 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.400 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.400 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.400 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.401 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.401 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.401 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.401 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.401 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.401 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.401 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.401 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.402 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.402 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.402 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.402 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T16:47:47.706385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=47, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.402 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.402 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.402 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.402 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.403 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.403 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.403 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.403 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.403 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.403 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.403 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.403 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.404 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.404 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.404 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.404 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.404 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.404 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.404 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.404 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.404 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.405 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.405 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.405 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.405 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.405 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.405 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.405 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.405 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.406 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.406 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.406 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.406 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.406 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.406 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.406 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.406 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.406 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.407 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.407 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.407 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.407 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.407 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.407 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.407 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.407 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.407 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.408 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.408 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.408 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.408 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.408 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.408 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.408 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.408 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 5.2c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.409 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.410 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.410 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.410 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.410 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.410 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.410 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.410 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.410 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.411 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.411 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.411 INFO:tasks.ceph:pgid 5.35 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.411 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.411 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.411 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.411 INFO:tasks.ceph:pgid 5.33 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.411 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.411 INFO:tasks.ceph:pgid 5.30 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.412 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.412 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.412 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.412 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.412 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.412 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.412 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.412 INFO:tasks.ceph:pgid 5.3b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.413 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.413 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.413 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.413 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:10.413 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T17:18:30.415 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:18:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d292a640 1 Processor -- start 2024-09-13T17:18:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d292a640 1 -- start start 2024-09-13T17:18:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d292a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42cc14e0c0 0x7f42cc0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d292a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc0c1f50 0x7f42cc06cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d292a640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f42cc06d300 0x7f42cc0c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d292a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42cc0c2780 con 0x7f42cc14e0c0 2024-09-13T17:18:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d292a640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42cc0c2a60 con 0x7f42cc06d300 2024-09-13T17:18:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d292a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0c2d40 con 0x7f42cc0c1f50 2024-09-13T17:18:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42cbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42cc14e0c0 0x7f42cc0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42cbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42cc14e0c0 0x7f42cc0a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43946/0 (socket says 172.21.15.39:43946) 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42cbfff640 1 -- 172.21.15.39:0/2152665445 learned_addr learned my addr 172.21.15.39:0/2152665445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d0ec7640 1 --2- 172.21.15.39:0/2152665445 >> v2:172.21.15.84:3300/0 conn(0x7f42cc06d300 0x7f42cc0c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42cb7fe640 1 --2- 172.21.15.39:0/2152665445 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc0c1f50 0x7f42cc06cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42cbfff640 1 -- 172.21.15.39:0/2152665445 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc0c1f50 msgr2=0x7f42cc06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42cbfff640 1 --2- 172.21.15.39:0/2152665445 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc0c1f50 0x7f42cc06cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42cbfff640 1 -- 172.21.15.39:0/2152665445 >> v2:172.21.15.84:3300/0 conn(0x7f42cc06d300 msgr2=0x7f42cc0c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42cbfff640 1 --2- 172.21.15.39:0/2152665445 >> v2:172.21.15.84:3300/0 conn(0x7f42cc06d300 0x7f42cc0c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42cbfff640 1 -- 172.21.15.39:0/2152665445 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc0c3020 con 0x7f42cc14e0c0 2024-09-13T17:18:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.548+0000 7f42d0ec7640 1 --2- 172.21.15.39:0/2152665445 >> v2:172.21.15.84:3300/0 conn(0x7f42cc06d300 0x7f42cc0c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:18:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42cbfff640 1 --2- 172.21.15.39:0/2152665445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc14e0c0 0x7f42cc0a4e70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0c3f40 tx=0x7f42bc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210ddf5c274f49cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42caffd640 1 -- 172.21.15.39:0/2152665445 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0c5490 con 0x7f42cc14e0c0 2024-09-13T17:18:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42caffd640 1 -- 172.21.15.39:0/2152665445 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc0a7050 con 0x7f42cc14e0c0 2024-09-13T17:18:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42caffd640 1 -- 172.21.15.39:0/2152665445 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0ab820 con 0x7f42cc14e0c0 2024-09-13T17:18:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 -- 172.21.15.39:0/2152665445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc14e0c0 msgr2=0x7f42cc0a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 --2- 172.21.15.39:0/2152665445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc14e0c0 0x7f42cc0a4e70 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0c3f40 tx=0x7f42bc01c460 comp rx=0 tx=0).stop 2024-09-13T17:18:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 -- 172.21.15.39:0/2152665445 shutdown_connections 2024-09-13T17:18:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 --2- 172.21.15.39:0/2152665445 >> v2:172.21.15.84:3300/0 conn(0x7f42cc06d300 0x7f42cc0c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 --2- 172.21.15.39:0/2152665445 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc0c1f50 0x7f42cc06cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 --2- 172.21.15.39:0/2152665445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc14e0c0 0x7f42cc0a4e70 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 -- 172.21.15.39:0/2152665445 >> 172.21.15.39:0/2152665445 conn(0x7f42cc069e10 msgr2=0x7f42cc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:30.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 -- 172.21.15.39:0/2152665445 shutdown_connections 2024-09-13T17:18:30.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 -- 172.21.15.39:0/2152665445 wait complete. 2024-09-13T17:18:30.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 Processor -- start 2024-09-13T17:18:30.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 -- start start 2024-09-13T17:18:30.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc06d300 0x7f42cc14ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42cc0c1f50 0x7f42cc145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc14e0c0 0x7f42cc1457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc147ab0 con 0x7f42cc06d300 2024-09-13T17:18:30.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc144df0 con 0x7f42cc0c1f50 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42d292a640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc1ecdf0 con 0x7f42cc14e0c0 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42cbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc06d300 0x7f42cc14ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42cbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc06d300 0x7f42cc14ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43950/0 (socket says 172.21.15.39:43950) 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.552+0000 7f42cbfff640 1 -- 172.21.15.39:0/1746462015 learned_addr learned my addr 172.21.15.39:0/1746462015 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42cb7fe640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42cc0c1f50 0x7f42cc145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42d0ec7640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc14e0c0 0x7f42cc1457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42cbfff640 1 -- 172.21.15.39:0/1746462015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc14e0c0 msgr2=0x7f42cc1457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42cbfff640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc14e0c0 0x7f42cc1457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42cbfff640 1 -- 172.21.15.39:0/1746462015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42cc0c1f50 msgr2=0x7f42cc145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42cbfff640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42cc0c1f50 0x7f42cc145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42cbfff640 1 -- 172.21.15.39:0/1746462015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc0094d0 con 0x7f42cc06d300 2024-09-13T17:18:30.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42cbfff640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc06d300 0x7f42cc14ceb0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f42bc01a090 tx=0x7f42bc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42c8ff9640 1 -- 172.21.15.39:0/1746462015 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0b6620 con 0x7f42cc06d300 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42c8ff9640 1 -- 172.21.15.39:0/1746462015 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc0abe90 con 0x7f42cc06d300 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42c8ff9640 1 -- 172.21.15.39:0/1746462015 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0b69e0 con 0x7f42cc06d300 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42cc1474f0 con 0x7f42cc06d300 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc1ec960 con 0x7f42cc06d300 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42cc144b90 con 0x7f42cc06d300 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42c8ff9640 1 -- 172.21.15.39:0/1746462015 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f42bc0b2040 con 0x7f42cc06d300 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42c8ff9640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42b8066690 0x7f42b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42c8ff9640 1 -- 172.21.15.39:0/1746462015 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f42bc11f900 con 0x7f42cc06d300 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42cb7fe640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42b8066690 0x7f42b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:30.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.556+0000 7f42cb7fe640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42b8066690 0x7f42b8068b50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f42cc146ba0 tx=0x7f42c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:30.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.560+0000 7f42c8ff9640 1 -- 172.21.15.39:0/1746462015 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f42bc0ebcc0 con 0x7f42cc06d300 2024-09-13T17:18:30.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.720+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42cc0c2540 con 0x7f42b8066690 2024-09-13T17:18:30.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.756+0000 7f42c8ff9640 1 -- 172.21.15.39:0/1746462015 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526102 (secure 0 0 0) 0x7f42c000c050 con 0x7f42b8066690 2024-09-13T17:18:30.766 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:18:30.770 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42b8066690 msgr2=0x7f42b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42b8066690 0x7f42b8068b50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f42cc146ba0 tx=0x7f42c0008040 comp rx=0 tx=0).stop 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc06d300 msgr2=0x7f42cc14ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc06d300 0x7f42cc14ceb0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f42bc01a090 tx=0x7f42bc01aca0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 shutdown_connections 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f42b8066690 0x7f42b8068b50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42cc0c1f50 0x7f42cc145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f42cc14e0c0 0x7f42cc1457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 --2- 172.21.15.39:0/1746462015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42cc06d300 0x7f42cc14ceb0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 >> 172.21.15.39:0/1746462015 conn(0x7f42cc069e10 msgr2=0x7f42cc143cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.760+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 shutdown_connections 2024-09-13T17:18:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:30.764+0000 7f42d292a640 1 -- 172.21.15.39:0/1746462015 wait complete. 2024-09-13T17:18:30.786 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":978,"stamp":"2024-09-13T17:18:30.368033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39291023,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":3153,"num_read_kb":416667,"num_write":2818784,"num_write_kb":37531913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522554,"ondisk_log_size":522554,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6804448,"kb_used_data":83360,"kb_used_omap":12,"kb_used_meta":6721011,"kb_avail":748170272,"statfs":{"total":773094113280,"available":766126358528,"internally_reserved":0,"allocated":85360640,"data_stored":82126150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":6882315835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006170"},"pg_stats":[{"pgid":"5.18","version":"63'4385","reported_seq":4449,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974458+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T17:17:44.974458+0000","last_peered":"2024-09-13T17:17:44.974458+0000","last_clean":"2024-09-13T17:17:44.974458+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T17:17:44.974458+0000","last_undegraded":"2024-09-13T17:17:44.974458+0000","last_fullsized":"2024-09-13T17:17:44.974458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":23692,"num_write_kb":324330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"64'919","reported_seq":1044,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.775073+0000","last_change":"2024-09-13T17:18:10.775401+0000","last_active":"2024-09-13T17:18:10.775073+0000","last_peered":"2024-09-13T17:18:10.775073+0000","last_clean":"2024-09-13T17:18:10.775073+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T17:18:10.775073+0000","last_undegraded":"2024-09-13T17:18:10.775073+0000","last_fullsized":"2024-09-13T17:18:10.775073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'919","last_scrub_stamp":"2024-09-13T17:18:10.775003+0000","last_deep_scrub":"64'919","last_deep_scrub_stamp":"2024-09-13T17:18:10.775003+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.775003+0000","objects_scrubbed":3,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:00.271443+0000","scrub_duration":0.030271704,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":10389,"num_write":894,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'3737","reported_seq":3807,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631607+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T17:17:47.631607+0000","last_peered":"2024-09-13T17:17:47.631607+0000","last_clean":"2024-09-13T17:17:47.631607+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T17:17:47.631607+0000","last_undegraded":"2024-09-13T17:17:47.631607+0000","last_fullsized":"2024-09-13T17:17:47.631607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3737,"log_dups_size":0,"ondisk_log_size":3737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20454,"num_write_kb":270405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.082725+0000","last_change":"2024-09-13T17:17:52.082969+0000","last_active":"2024-09-13T17:17:52.082725+0000","last_peered":"2024-09-13T17:17:52.082725+0000","last_clean":"2024-09-13T17:17:52.082725+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T17:17:52.082725+0000","last_undegraded":"2024-09-13T17:17:52.082725+0000","last_fullsized":"2024-09-13T17:17:52.082725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.082672+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.023294+0000","scrub_duration":0.0034166320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"63'4331","reported_seq":4396,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992385+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T17:17:44.992385+0000","last_peered":"2024-09-13T17:17:44.992385+0000","last_clean":"2024-09-13T17:17:44.992385+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T17:17:44.992385+0000","last_undegraded":"2024-09-13T17:17:44.992385+0000","last_fullsized":"2024-09-13T17:17:44.992385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":22875,"num_write_kb":325320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"64'24","reported_seq":95,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631458+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T17:17:47.631458+0000","last_peered":"2024-09-13T17:17:47.631458+0000","last_clean":"2024-09-13T17:17:47.631458+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T17:17:47.631458+0000","last_undegraded":"2024-09-13T17:17:47.631458+0000","last_fullsized":"2024-09-13T17:17:47.631458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"63'5559","reported_seq":5639,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:55.943224+0000","last_change":"2024-09-13T17:17:55.943470+0000","last_active":"2024-09-13T17:17:55.943224+0000","last_peered":"2024-09-13T17:17:55.943224+0000","last_clean":"2024-09-13T17:17:55.943224+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T17:17:55.943224+0000","last_undegraded":"2024-09-13T17:17:55.943224+0000","last_fullsized":"2024-09-13T17:17:55.943224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5559","last_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_deep_scrub":"63'5559","last_deep_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_clean_scrub_stamp":"2024-09-13T17:17:55.943168+0000","objects_scrubbed":0,"log_size":5559,"log_dups_size":0,"ondisk_log_size":5559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:10.227939+0000","scrub_duration":0.037150519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30370,"num_write_kb":379865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"63'2","reported_seq":66,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974552+0000","last_change":"2024-09-13T16:48:07.772469+0000","last_active":"2024-09-13T17:17:44.974552+0000","last_peered":"2024-09-13T17:17:44.974552+0000","last_clean":"2024-09-13T17:17:44.974552+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T17:17:44.974552+0000","last_undegraded":"2024-09-13T17:17:44.974552+0000","last_fullsized":"2024-09-13T17:17:44.974552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"63'3581","reported_seq":3644,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591311+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T17:17:49.591311+0000","last_peered":"2024-09-13T17:17:49.591311+0000","last_clean":"2024-09-13T17:17:49.591311+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T17:17:49.591311+0000","last_undegraded":"2024-09-13T17:17:49.591311+0000","last_fullsized":"2024-09-13T17:17:49.591311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":268344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"64'43","reported_seq":149,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591097+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T17:17:49.591097+0000","last_peered":"2024-09-13T17:17:49.591097+0000","last_clean":"2024-09-13T17:17:49.591097+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T17:17:49.591097+0000","last_undegraded":"2024-09-13T17:17:49.591097+0000","last_fullsized":"2024-09-13T17:17:49.591097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'3493","reported_seq":3562,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.642622+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T17:17:48.642622+0000","last_peered":"2024-09-13T17:17:48.642622+0000","last_clean":"2024-09-13T17:17:48.642622+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T17:17:48.642622+0000","last_undegraded":"2024-09-13T17:17:48.642622+0000","last_fullsized":"2024-09-13T17:17:48.642622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18921,"num_write_kb":256078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'727","reported_seq":827,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.186917+0000","last_change":"2024-09-13T17:17:57.186917+0000","last_active":"2024-09-13T17:17:57.186917+0000","last_peered":"2024-09-13T17:17:57.186917+0000","last_clean":"2024-09-13T17:17:57.186917+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T17:17:57.186917+0000","last_undegraded":"2024-09-13T17:17:57.186917+0000","last_fullsized":"2024-09-13T17:17:57.186917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'727","last_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_deep_scrub":"64'727","last_deep_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.186269+0000","objects_scrubbed":1,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:00.928968+0000","scrub_duration":0.026043038000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":75389,"num_write":714,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"63'3135","reported_seq":3198,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.643188+0000","last_change":"2024-09-13T16:48:08.173009+0000","last_active":"2024-09-13T17:17:48.643188+0000","last_peered":"2024-09-13T17:17:48.643188+0000","last_clean":"2024-09-13T17:17:48.643188+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T17:17:48.643188+0000","last_undegraded":"2024-09-13T17:17:48.643188+0000","last_fullsized":"2024-09-13T17:17:48.643188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3135,"log_dups_size":0,"ondisk_log_size":3135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":232447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"63'5","reported_seq":111,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.590779+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T17:17:49.590779+0000","last_peered":"2024-09-13T17:17:49.590779+0000","last_clean":"2024-09-13T17:17:49.590779+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T17:17:49.590779+0000","last_undegraded":"2024-09-13T17:17:49.590779+0000","last_fullsized":"2024-09-13T17:17:49.590779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"63'5195","reported_seq":5273,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.511870+0000","last_change":"2024-09-13T17:17:49.511870+0000","last_active":"2024-09-13T17:17:49.511870+0000","last_peered":"2024-09-13T17:17:49.511870+0000","last_clean":"2024-09-13T17:17:49.511870+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T17:17:49.511870+0000","last_undegraded":"2024-09-13T17:17:49.511870+0000","last_fullsized":"2024-09-13T17:17:49.511870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5195","last_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_deep_scrub":"63'5195","last_deep_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.511449+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:30.737466+0000","scrub_duration":0.041661353999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29369,"num_write_kb":359470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"63'210","reported_seq":274,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.132787+0000","last_change":"2024-09-13T16:48:07.780083+0000","last_active":"2024-09-13T17:17:48.132787+0000","last_peered":"2024-09-13T17:17:48.132787+0000","last_clean":"2024-09-13T17:17:48.132787+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T17:17:48.132787+0000","last_undegraded":"2024-09-13T17:17:48.132787+0000","last_fullsized":"2024-09-13T17:17:48.132787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153565,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"63'3570","reported_seq":3636,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996947+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T17:17:44.996947+0000","last_peered":"2024-09-13T17:17:44.996947+0000","last_clean":"2024-09-13T17:17:44.996947+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T17:17:44.996947+0000","last_undegraded":"2024-09-13T17:17:44.996947+0000","last_fullsized":"2024-09-13T17:17:44.996947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":19362,"num_write_kb":282506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"63'4274","reported_seq":4353,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.521159+0000","last_change":"2024-09-13T17:18:01.521451+0000","last_active":"2024-09-13T17:18:01.521159+0000","last_peered":"2024-09-13T17:18:01.521159+0000","last_clean":"2024-09-13T17:18:01.521159+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T17:18:01.521159+0000","last_undegraded":"2024-09-13T17:18:01.521159+0000","last_fullsized":"2024-09-13T17:18:01.521159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4274","last_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_deep_scrub":"63'4274","last_deep_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.521104+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:51.017545+0000","scrub_duration":0.027199207999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23751,"num_write_kb":312338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"64'1419","reported_seq":1533,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.007429+0000","last_change":"2024-09-13T16:47:58.683464+0000","last_active":"2024-09-13T17:17:48.007429+0000","last_peered":"2024-09-13T17:17:48.007429+0000","last_clean":"2024-09-13T17:17:48.007429+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T17:17:48.007429+0000","last_undegraded":"2024-09-13T17:17:48.007429+0000","last_fullsized":"2024-09-13T17:17:48.007429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":1419,"log_dups_size":0,"ondisk_log_size":1419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":42245,"num_write":1396,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'7","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.140201+0000","last_change":"2024-09-13T17:18:01.140415+0000","last_active":"2024-09-13T17:18:01.140201+0000","last_peered":"2024-09-13T17:18:01.140201+0000","last_clean":"2024-09-13T17:18:01.140201+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T17:18:01.140201+0000","last_undegraded":"2024-09-13T17:18:01.140201+0000","last_fullsized":"2024-09-13T17:18:01.140201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'7","last_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_deep_scrub":"64'7","last_deep_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.140142+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:50.636583+0000","scrub_duration":0.0071129380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"63'4212","reported_seq":4279,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007878+0000","last_change":"2024-09-13T16:48:08.180595+0000","last_active":"2024-09-13T17:17:45.007878+0000","last_peered":"2024-09-13T17:17:45.007878+0000","last_clean":"2024-09-13T17:17:45.007878+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T17:17:45.007878+0000","last_undegraded":"2024-09-13T17:17:45.007878+0000","last_fullsized":"2024-09-13T17:17:45.007878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":17,"num_write":23343,"num_write_kb":314959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"63'3296","reported_seq":3364,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975110+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T17:17:44.975110+0000","last_peered":"2024-09-13T17:17:44.975110+0000","last_clean":"2024-09-13T17:17:44.975110+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T17:17:44.975110+0000","last_undegraded":"2024-09-13T17:17:44.975110+0000","last_fullsized":"2024-09-13T17:17:44.975110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3296,"log_dups_size":0,"ondisk_log_size":3296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17758,"num_write_kb":230990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'3","reported_seq":73,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.674499+0000","last_change":"2024-09-13T16:47:58.662093+0000","last_active":"2024-09-13T17:17:46.674499+0000","last_peered":"2024-09-13T17:17:46.674499+0000","last_clean":"2024-09-13T17:17:46.674499+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T17:17:46.674499+0000","last_undegraded":"2024-09-13T17:17:46.674499+0000","last_fullsized":"2024-09-13T17:17:46.674499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.139202+0000","last_change":"2024-09-13T17:18:05.139417+0000","last_active":"2024-09-13T17:18:05.139202+0000","last_peered":"2024-09-13T17:18:05.139202+0000","last_clean":"2024-09-13T17:18:05.139202+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T17:18:05.139202+0000","last_undegraded":"2024-09-13T17:18:05.139202+0000","last_fullsized":"2024-09-13T17:18:05.139202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:05.139147+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:05.139147+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.139147+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:15.079769+0000","scrub_duration":0.0035900110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"63'3371","reported_seq":3436,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980902+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T17:17:44.980902+0000","last_peered":"2024-09-13T17:17:44.980902+0000","last_clean":"2024-09-13T17:17:44.980902+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T17:17:44.980902+0000","last_undegraded":"2024-09-13T17:17:44.980902+0000","last_fullsized":"2024-09-13T17:17:44.980902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3371,"log_dups_size":0,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18156,"num_write_kb":249012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"63'3752","reported_seq":3821,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631388+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T17:17:47.631388+0000","last_peered":"2024-09-13T17:17:47.631388+0000","last_clean":"2024-09-13T17:17:47.631388+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T17:17:47.631388+0000","last_undegraded":"2024-09-13T17:17:47.631388+0000","last_fullsized":"2024-09-13T17:17:47.631388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21127,"num_write_kb":268998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"63'2","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.674520+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T17:17:46.674520+0000","last_peered":"2024-09-13T17:17:46.674520+0000","last_clean":"2024-09-13T17:17:46.674520+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T17:17:46.674520+0000","last_undegraded":"2024-09-13T17:17:46.674520+0000","last_fullsized":"2024-09-13T17:17:46.674520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"64'5","reported_seq":102,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:11.257475+0000","last_change":"2024-09-13T17:18:11.257689+0000","last_active":"2024-09-13T17:18:11.257475+0000","last_peered":"2024-09-13T17:18:11.257475+0000","last_clean":"2024-09-13T17:18:11.257475+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T17:18:11.257475+0000","last_undegraded":"2024-09-13T17:18:11.257475+0000","last_fullsized":"2024-09-13T17:18:11.257475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:11.257421+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:11.257421+0000","last_clean_scrub_stamp":"2024-09-13T17:18:11.257421+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:03.763129+0000","scrub_duration":0.0063405759999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"63'4499","reported_seq":4567,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.987188+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T17:17:44.987188+0000","last_peered":"2024-09-13T17:17:44.987188+0000","last_clean":"2024-09-13T17:17:44.987188+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T17:17:44.987188+0000","last_undegraded":"2024-09-13T17:17:44.987188+0000","last_fullsized":"2024-09-13T17:17:44.987188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":24341,"num_write_kb":343418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"63'4043","reported_seq":4112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.132253+0000","last_change":"2024-09-13T16:47:59.038612+0000","last_active":"2024-09-13T17:17:48.132253+0000","last_peered":"2024-09-13T17:17:48.132253+0000","last_clean":"2024-09-13T17:17:48.132253+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T17:17:48.132253+0000","last_undegraded":"2024-09-13T17:17:48.132253+0000","last_fullsized":"2024-09-13T17:17:48.132253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22005,"num_write_kb":288910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"63'5","reported_seq":75,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.590791+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T17:17:49.590791+0000","last_peered":"2024-09-13T17:17:49.590791+0000","last_clean":"2024-09-13T17:17:49.590791+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T17:17:49.590791+0000","last_undegraded":"2024-09-13T17:17:49.590791+0000","last_fullsized":"2024-09-13T17:17:49.590791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.171804+0000","last_change":"2024-09-13T17:18:04.172104+0000","last_active":"2024-09-13T17:18:04.171804+0000","last_peered":"2024-09-13T17:18:04.171804+0000","last_clean":"2024-09-13T17:18:04.171804+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T17:18:04.171804+0000","last_undegraded":"2024-09-13T17:18:04.171804+0000","last_fullsized":"2024-09-13T17:18:04.171804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.171747+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:58.830350+0000","scrub_duration":0.0065161840000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"63'3470","reported_seq":3536,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632011+0000","last_change":"2024-09-13T16:48:08.164777+0000","last_active":"2024-09-13T17:17:47.632011+0000","last_peered":"2024-09-13T17:17:47.632011+0000","last_clean":"2024-09-13T17:17:47.632011+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T17:17:47.632011+0000","last_undegraded":"2024-09-13T17:17:47.632011+0000","last_fullsized":"2024-09-13T17:17:47.632011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19441,"num_write_kb":258288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'4844","reported_seq":4925,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:18.899218+0000","last_change":"2024-09-13T17:18:18.899218+0000","last_active":"2024-09-13T17:18:18.899218+0000","last_peered":"2024-09-13T17:18:18.899218+0000","last_clean":"2024-09-13T17:18:18.899218+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T17:18:18.899218+0000","last_undegraded":"2024-09-13T17:18:18.899218+0000","last_fullsized":"2024-09-13T17:18:18.899218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4844","last_scrub_stamp":"2024-09-13T17:18:18.898914+0000","last_deep_scrub":"63'4844","last_deep_scrub_stamp":"2024-09-13T17:18:18.898914+0000","last_clean_scrub_stamp":"2024-09-13T17:18:18.898914+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:18.212032+0000","scrub_duration":0.026998708999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":26635,"num_write_kb":336207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'10","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.498454+0000","last_change":"2024-09-13T17:18:05.498454+0000","last_active":"2024-09-13T17:18:05.498454+0000","last_peered":"2024-09-13T17:18:05.498454+0000","last_clean":"2024-09-13T17:18:05.498454+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T17:18:05.498454+0000","last_undegraded":"2024-09-13T17:18:05.498454+0000","last_fullsized":"2024-09-13T17:18:05.498454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_deep_scrub":"64'10","last_deep_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.498184+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:15.438806+0000","scrub_duration":0.0057060160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'49","reported_seq":189,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.166590+0000","last_change":"2024-09-13T17:17:58.166876+0000","last_active":"2024-09-13T17:17:58.166590+0000","last_peered":"2024-09-13T17:17:58.166590+0000","last_clean":"2024-09-13T17:17:58.166590+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T17:17:58.166590+0000","last_undegraded":"2024-09-13T17:17:58.166590+0000","last_fullsized":"2024-09-13T17:17:58.166590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'49","last_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_deep_scrub":"64'49","last_deep_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.166530+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:05.921781+0000","scrub_duration":0.0073340999999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92726,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":118,"num_write":43,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"63'3635","reported_seq":3701,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974442+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T17:17:44.974442+0000","last_peered":"2024-09-13T17:17:44.974442+0000","last_clean":"2024-09-13T17:17:44.974442+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T17:17:44.974442+0000","last_undegraded":"2024-09-13T17:17:44.974442+0000","last_fullsized":"2024-09-13T17:17:44.974442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19787,"num_write_kb":259620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'3639","reported_seq":3708,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631524+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T17:17:47.631524+0000","last_peered":"2024-09-13T17:17:47.631524+0000","last_clean":"2024-09-13T17:17:47.631524+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T17:17:47.631524+0000","last_undegraded":"2024-09-13T17:17:47.631524+0000","last_fullsized":"2024-09-13T17:17:47.631524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3639,"log_dups_size":0,"ondisk_log_size":3639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20066,"num_write_kb":240267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"64'267","reported_seq":356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.589241+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T17:17:50.589241+0000","last_peered":"2024-09-13T17:17:50.589241+0000","last_clean":"2024-09-13T17:17:50.589241+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T17:17:50.589241+0000","last_undegraded":"2024-09-13T17:17:50.589241+0000","last_fullsized":"2024-09-13T17:17:50.589241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":38,"num_read_kb":48,"num_write":636,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"63'11","reported_seq":89,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.102762+0000","last_change":"2024-09-13T17:17:50.102762+0000","last_active":"2024-09-13T17:17:50.102762+0000","last_peered":"2024-09-13T17:17:50.102762+0000","last_clean":"2024-09-13T17:17:50.102762+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T17:17:50.102762+0000","last_undegraded":"2024-09-13T17:17:50.102762+0000","last_fullsized":"2024-09-13T17:17:50.102762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_deep_scrub":"63'11","last_deep_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.102616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.736833+0000","scrub_duration":0.0039927280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"63'3190","reported_seq":3255,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993610+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T17:17:44.993610+0000","last_peered":"2024-09-13T17:17:44.993610+0000","last_clean":"2024-09-13T17:17:44.993610+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T17:17:44.993610+0000","last_undegraded":"2024-09-13T17:17:44.993610+0000","last_fullsized":"2024-09-13T17:17:44.993610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3190,"log_dups_size":0,"ondisk_log_size":3190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":254866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"63'4701","reported_seq":4770,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.998464+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T17:17:44.998464+0000","last_peered":"2024-09-13T17:17:44.998464+0000","last_clean":"2024-09-13T17:17:44.998464+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T17:17:44.998464+0000","last_undegraded":"2024-09-13T17:17:44.998464+0000","last_fullsized":"2024-09-13T17:17:44.998464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26171,"num_write_kb":338588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'2","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996573+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T17:17:44.996573+0000","last_peered":"2024-09-13T17:17:44.996573+0000","last_clean":"2024-09-13T17:17:44.996573+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T17:17:44.996573+0000","last_undegraded":"2024-09-13T17:17:44.996573+0000","last_fullsized":"2024-09-13T17:17:44.996573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"63'3","reported_seq":69,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.002286+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T17:17:45.002286+0000","last_peered":"2024-09-13T17:17:45.002286+0000","last_clean":"2024-09-13T17:17:45.002286+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T17:17:45.002286+0000","last_undegraded":"2024-09-13T17:17:45.002286+0000","last_fullsized":"2024-09-13T17:17:45.002286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"63'3859","reported_seq":3924,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.003745+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T17:17:45.003745+0000","last_peered":"2024-09-13T17:17:45.003745+0000","last_clean":"2024-09-13T17:17:45.003745+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T17:17:45.003745+0000","last_undegraded":"2024-09-13T17:17:45.003745+0000","last_fullsized":"2024-09-13T17:17:45.003745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21286,"num_write_kb":290222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"63'3876","reported_seq":3945,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.985116+0000","last_change":"2024-09-13T16:47:59.058844+0000","last_active":"2024-09-13T17:17:44.985116+0000","last_peered":"2024-09-13T17:17:44.985116+0000","last_clean":"2024-09-13T17:17:44.985116+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T17:17:44.985116+0000","last_undegraded":"2024-09-13T17:17:44.985116+0000","last_fullsized":"2024-09-13T17:17:44.985116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21201,"num_write_kb":273354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"63'2","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.999079+0000","last_change":"2024-09-13T16:47:58.655374+0000","last_active":"2024-09-13T17:17:44.999079+0000","last_peered":"2024-09-13T17:17:44.999079+0000","last_clean":"2024-09-13T17:17:44.999079+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T17:17:44.999079+0000","last_undegraded":"2024-09-13T17:17:44.999079+0000","last_fullsized":"2024-09-13T17:17:44.999079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'5","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.226738+0000","last_change":"2024-09-13T17:17:57.227606+0000","last_active":"2024-09-13T17:17:57.226738+0000","last_peered":"2024-09-13T17:17:57.226738+0000","last_clean":"2024-09-13T17:17:57.226738+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T17:17:57.226738+0000","last_undegraded":"2024-09-13T17:17:57.226738+0000","last_fullsized":"2024-09-13T17:17:57.226738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226662+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:46.723103+0000","scrub_duration":0.0036245560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"63'3422","reported_seq":3488,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993940+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T17:17:44.993940+0000","last_peered":"2024-09-13T17:17:44.993940+0000","last_clean":"2024-09-13T17:17:44.993940+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T17:17:44.993940+0000","last_undegraded":"2024-09-13T17:17:44.993940+0000","last_fullsized":"2024-09-13T17:17:44.993940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":7,"num_write":19042,"num_write_kb":260722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"63'4802","reported_seq":4882,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:12.260957+0000","last_change":"2024-09-13T17:18:12.261210+0000","last_active":"2024-09-13T17:18:12.260957+0000","last_peered":"2024-09-13T17:18:12.260957+0000","last_clean":"2024-09-13T17:18:12.260957+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T17:18:12.260957+0000","last_undegraded":"2024-09-13T17:18:12.260957+0000","last_fullsized":"2024-09-13T17:18:12.260957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4802","last_scrub_stamp":"2024-09-13T17:18:12.260903+0000","last_deep_scrub":"63'4802","last_deep_scrub_stamp":"2024-09-13T17:18:12.260903+0000","last_clean_scrub_stamp":"2024-09-13T17:18:12.260903+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:25.238396+0000","scrub_duration":0.040392404999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26419,"num_write_kb":335470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"63'4","reported_seq":74,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978957+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T17:17:44.978957+0000","last_peered":"2024-09-13T17:17:44.978957+0000","last_clean":"2024-09-13T17:17:44.978957+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T17:17:44.978957+0000","last_undegraded":"2024-09-13T17:17:44.978957+0000","last_fullsized":"2024-09-13T17:17:44.978957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:54.181019+0000","last_change":"2024-09-13T17:17:54.181249+0000","last_active":"2024-09-13T17:17:54.181019+0000","last_peered":"2024-09-13T17:17:54.181019+0000","last_clean":"2024-09-13T17:17:54.181019+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T17:17:54.181019+0000","last_undegraded":"2024-09-13T17:17:54.181019+0000","last_fullsized":"2024-09-13T17:17:54.181019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_clean_scrub_stamp":"2024-09-13T17:17:54.180966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:37.317744+0000","scrub_duration":0.0037272709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"63'4215","reported_seq":4291,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:14.596322+0000","last_change":"2024-09-13T17:18:14.596560+0000","last_active":"2024-09-13T17:18:14.596322+0000","last_peered":"2024-09-13T17:18:14.596322+0000","last_clean":"2024-09-13T17:18:14.596322+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T17:18:14.596322+0000","last_undegraded":"2024-09-13T17:18:14.596322+0000","last_fullsized":"2024-09-13T17:18:14.596322+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4215","last_scrub_stamp":"2024-09-13T17:18:14.596269+0000","last_deep_scrub":"63'4215","last_deep_scrub_stamp":"2024-09-13T17:18:14.596269+0000","last_clean_scrub_stamp":"2024-09-13T17:18:14.596269+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:07.101977+0000","scrub_duration":0.024198185000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":8,"num_write":22461,"num_write_kb":319331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'3790","reported_seq":3860,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.674564+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T17:17:46.674564+0000","last_peered":"2024-09-13T17:17:46.674564+0000","last_clean":"2024-09-13T17:17:46.674564+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T17:17:46.674564+0000","last_undegraded":"2024-09-13T17:17:46.674564+0000","last_fullsized":"2024-09-13T17:17:46.674564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21077,"num_write_kb":260732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'42","reported_seq":179,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.602904+0000","last_change":"2024-09-13T17:18:03.603126+0000","last_active":"2024-09-13T17:18:03.602904+0000","last_peered":"2024-09-13T17:18:03.602904+0000","last_clean":"2024-09-13T17:18:03.602904+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T17:18:03.602904+0000","last_undegraded":"2024-09-13T17:18:03.602904+0000","last_fullsized":"2024-09-13T17:18:03.602904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'42","last_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_deep_scrub":"64'42","last_deep_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.602845+0000","objects_scrubbed":1,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:03.563076+0000","scrub_duration":0.004957576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84646,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":106,"num_write":37,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'10","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.102806+0000","last_change":"2024-09-13T17:18:05.102806+0000","last_active":"2024-09-13T17:18:05.102806+0000","last_peered":"2024-09-13T17:18:05.102806+0000","last_clean":"2024-09-13T17:18:05.102806+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T17:18:05.102806+0000","last_undegraded":"2024-09-13T17:18:05.102806+0000","last_fullsized":"2024-09-13T17:18:05.102806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-13T17:18:05.101908+0000","last_deep_scrub":"64'10","last_deep_scrub_stamp":"2024-09-13T17:18:05.101908+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.101908+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:04.415025+0000","scrub_duration":0.0065388449999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"63'3128","reported_seq":3192,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.987242+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T17:17:44.987242+0000","last_peered":"2024-09-13T17:17:44.987242+0000","last_clean":"2024-09-13T17:17:44.987242+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T17:17:44.987242+0000","last_undegraded":"2024-09-13T17:17:44.987242+0000","last_fullsized":"2024-09-13T17:17:44.987242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3128,"log_dups_size":0,"ondisk_log_size":3128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17776,"num_write_kb":243283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"63'3657","reported_seq":3737,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:14.878066+0000","last_change":"2024-09-13T17:18:14.878283+0000","last_active":"2024-09-13T17:18:14.878066+0000","last_peered":"2024-09-13T17:18:14.878066+0000","last_clean":"2024-09-13T17:18:14.878066+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T17:18:14.878066+0000","last_undegraded":"2024-09-13T17:18:14.878066+0000","last_fullsized":"2024-09-13T17:18:14.878066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3657","last_scrub_stamp":"2024-09-13T17:18:14.878012+0000","last_deep_scrub":"63'3657","last_deep_scrub_stamp":"2024-09-13T17:18:14.878012+0000","last_clean_scrub_stamp":"2024-09-13T17:18:14.878012+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:17.512229+0000","scrub_duration":0.026162280999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19816,"num_write_kb":255512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'11","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.590655+0000","last_change":"2024-09-13T16:47:58.685547+0000","last_active":"2024-09-13T17:17:49.590655+0000","last_peered":"2024-09-13T17:17:49.590655+0000","last_clean":"2024-09-13T17:17:49.590655+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T17:17:49.590655+0000","last_undegraded":"2024-09-13T17:17:49.590655+0000","last_fullsized":"2024-09-13T17:17:49.590655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'251","reported_seq":338,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.207140+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T17:17:46.207140+0000","last_peered":"2024-09-13T17:17:46.207140+0000","last_clean":"2024-09-13T17:17:46.207140+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T17:17:46.207140+0000","last_undegraded":"2024-09-13T17:17:46.207140+0000","last_fullsized":"2024-09-13T17:17:46.207140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":57,"num_write":597,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"63'4141","reported_seq":4207,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992340+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T17:17:44.992340+0000","last_peered":"2024-09-13T17:17:44.992340+0000","last_clean":"2024-09-13T17:17:44.992340+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T17:17:44.992340+0000","last_undegraded":"2024-09-13T17:17:44.992340+0000","last_fullsized":"2024-09-13T17:17:44.992340+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22849,"num_write_kb":317008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'4103","reported_seq":4180,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.972983+0000","last_change":"2024-09-13T16:48:41.389774+0000","last_active":"2024-09-13T17:17:44.972983+0000","last_peered":"2024-09-13T17:17:44.972983+0000","last_clean":"2024-09-13T17:17:44.972983+0000","last_became_active":"2024-09-13T16:48:41.389507+0000","last_became_peered":"2024-09-13T16:48:41.389507+0000","last_unstale":"2024-09-13T17:17:44.972983+0000","last_undegraded":"2024-09-13T17:17:44.972983+0000","last_fullsized":"2024-09-13T17:17:44.972983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22342,"num_write_kb":278194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"63'3","reported_seq":73,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980918+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T17:17:44.980918+0000","last_peered":"2024-09-13T17:17:44.980918+0000","last_clean":"2024-09-13T17:17:44.980918+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T17:17:44.980918+0000","last_undegraded":"2024-09-13T17:17:44.980918+0000","last_fullsized":"2024-09-13T17:17:44.980918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.590577+0000","last_change":"2024-09-13T16:47:48.775461+0000","last_active":"2024-09-13T17:17:49.590577+0000","last_peered":"2024-09-13T17:17:49.590577+0000","last_clean":"2024-09-13T17:17:49.590577+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T17:17:49.590577+0000","last_undegraded":"2024-09-13T17:17:49.590577+0000","last_fullsized":"2024-09-13T17:17:49.590577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:47.706385+0000","last_clean_scrub_stamp":"2024-09-13T16:47:47.706385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"63'2","reported_seq":68,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994652+0000","last_change":"2024-09-13T16:48:07.785511+0000","last_active":"2024-09-13T17:17:44.994652+0000","last_peered":"2024-09-13T17:17:44.994652+0000","last_clean":"2024-09-13T17:17:44.994652+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T17:17:44.994652+0000","last_undegraded":"2024-09-13T17:17:44.994652+0000","last_fullsized":"2024-09-13T17:17:44.994652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"63'2904","reported_seq":2980,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.337844+0000","last_change":"2024-09-13T17:17:48.337931+0000","last_active":"2024-09-13T17:17:48.337844+0000","last_peered":"2024-09-13T17:17:48.337844+0000","last_clean":"2024-09-13T17:17:48.337844+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T17:17:48.337844+0000","last_undegraded":"2024-09-13T17:17:48.337844+0000","last_fullsized":"2024-09-13T17:17:48.337844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2904","last_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_deep_scrub":"63'2904","last_deep_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_clean_scrub_stamp":"2024-09-13T17:17:48.337825+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:15.694145+0000","scrub_duration":0.0051743259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":2,"num_write":15607,"num_write_kb":214254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'4284","reported_seq":4363,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.341991+0000","last_change":"2024-09-13T17:17:52.342307+0000","last_active":"2024-09-13T17:17:52.341991+0000","last_peered":"2024-09-13T17:17:52.341991+0000","last_clean":"2024-09-13T17:17:52.341991+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T17:17:52.341991+0000","last_undegraded":"2024-09-13T17:17:52.341991+0000","last_fullsized":"2024-09-13T17:17:52.341991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4284","last_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_deep_scrub":"63'4284","last_deep_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.341937+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.282559+0000","scrub_duration":0.043379468999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23392,"num_write_kb":310963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"63'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.502278+0000","last_change":"2024-09-13T17:17:53.502523+0000","last_active":"2024-09-13T17:17:53.502278+0000","last_peered":"2024-09-13T17:17:53.502278+0000","last_clean":"2024-09-13T17:17:53.502278+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T17:17:53.502278+0000","last_undegraded":"2024-09-13T17:17:53.502278+0000","last_fullsized":"2024-09-13T17:17:53.502278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.502221+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:42.741599+0000","scrub_duration":0.0040979670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"63'3","reported_seq":68,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994051+0000","last_change":"2024-09-13T16:48:07.777053+0000","last_active":"2024-09-13T17:17:44.994051+0000","last_peered":"2024-09-13T17:17:44.994051+0000","last_clean":"2024-09-13T17:17:44.994051+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T17:17:44.994051+0000","last_undegraded":"2024-09-13T17:17:44.994051+0000","last_fullsized":"2024-09-13T17:17:44.994051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"63'4138","reported_seq":4213,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.398557+0000","last_change":"2024-09-13T17:17:49.398680+0000","last_active":"2024-09-13T17:17:49.398557+0000","last_peered":"2024-09-13T17:17:49.398557+0000","last_clean":"2024-09-13T17:17:49.398557+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T17:17:49.398557+0000","last_undegraded":"2024-09-13T17:17:49.398557+0000","last_fullsized":"2024-09-13T17:17:49.398557+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4138","last_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_deep_scrub":"63'4138","last_deep_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.398536+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:51.412144+0000","scrub_duration":0.020992730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":309086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"63'4584","reported_seq":4646,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.164508+0000","last_change":"2024-09-13T16:48:40.767044+0000","last_active":"2024-09-13T17:17:47.164508+0000","last_peered":"2024-09-13T17:17:47.164508+0000","last_clean":"2024-09-13T17:17:47.164508+0000","last_became_active":"2024-09-13T16:48:40.765861+0000","last_became_peered":"2024-09-13T16:48:40.765861+0000","last_unstale":"2024-09-13T17:17:47.164508+0000","last_undegraded":"2024-09-13T17:17:47.164508+0000","last_fullsized":"2024-09-13T17:17:47.164508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24690,"num_write_kb":320186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"63'6","reported_seq":94,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.995320+0000","last_change":"2024-09-13T16:47:58.661776+0000","last_active":"2024-09-13T17:17:44.995320+0000","last_peered":"2024-09-13T17:17:44.995320+0000","last_clean":"2024-09-13T17:17:44.995320+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T17:17:44.995320+0000","last_undegraded":"2024-09-13T17:17:44.995320+0000","last_fullsized":"2024-09-13T17:17:44.995320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'1425","reported_seq":1538,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.001931+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T17:17:45.001931+0000","last_peered":"2024-09-13T17:17:45.001931+0000","last_clean":"2024-09-13T17:17:45.001931+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T17:17:45.001931+0000","last_undegraded":"2024-09-13T17:17:45.001931+0000","last_fullsized":"2024-09-13T17:17:45.001931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":38459,"num_write":1400,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"63'3637","reported_seq":3702,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991867+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T17:17:44.991867+0000","last_peered":"2024-09-13T17:17:44.991867+0000","last_clean":"2024-09-13T17:17:44.991867+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T17:17:44.991867+0000","last_undegraded":"2024-09-13T17:17:44.991867+0000","last_fullsized":"2024-09-13T17:17:44.991867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19635,"num_write_kb":259018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'5067","reported_seq":5129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.164365+0000","last_change":"2024-09-13T16:48:40.831954+0000","last_active":"2024-09-13T17:17:47.164365+0000","last_peered":"2024-09-13T17:17:47.164365+0000","last_clean":"2024-09-13T17:17:47.164365+0000","last_became_active":"2024-09-13T16:48:40.831696+0000","last_became_peered":"2024-09-13T16:48:40.831696+0000","last_unstale":"2024-09-13T17:17:47.164365+0000","last_undegraded":"2024-09-13T17:17:47.164365+0000","last_fullsized":"2024-09-13T17:17:47.164365+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27028,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.556099+0000","last_change":"2024-09-13T17:18:02.556314+0000","last_active":"2024-09-13T17:18:02.556099+0000","last_peered":"2024-09-13T17:18:02.556099+0000","last_clean":"2024-09-13T17:18:02.556099+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T17:18:02.556099+0000","last_undegraded":"2024-09-13T17:18:02.556099+0000","last_fullsized":"2024-09-13T17:18:02.556099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.556044+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:52.052485+0000","scrub_duration":0.0060730330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"63'5","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:16.546615+0000","last_change":"2024-09-13T17:18:16.546615+0000","last_active":"2024-09-13T17:18:16.546615+0000","last_peered":"2024-09-13T17:18:16.546615+0000","last_clean":"2024-09-13T17:18:16.546615+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T17:18:16.546615+0000","last_undegraded":"2024-09-13T17:18:16.546615+0000","last_fullsized":"2024-09-13T17:18:16.546615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:16.546281+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:16.546281+0000","last_clean_scrub_stamp":"2024-09-13T17:18:16.546281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:15.818163+0000","scrub_duration":0.0035841639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"63'2915","reported_seq":2979,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.006202+0000","last_change":"2024-09-13T16:48:08.168968+0000","last_active":"2024-09-13T17:17:45.006202+0000","last_peered":"2024-09-13T17:17:45.006202+0000","last_clean":"2024-09-13T17:17:45.006202+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T17:17:45.006202+0000","last_undegraded":"2024-09-13T17:17:45.006202+0000","last_fullsized":"2024-09-13T17:17:45.006202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2915,"log_dups_size":0,"ondisk_log_size":2915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":234494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"63'3697","reported_seq":3776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.226187+0000","last_change":"2024-09-13T17:18:29.226417+0000","last_active":"2024-09-13T17:18:29.226187+0000","last_peered":"2024-09-13T17:18:29.226187+0000","last_clean":"2024-09-13T17:18:29.226187+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T17:18:29.226187+0000","last_undegraded":"2024-09-13T17:18:29.226187+0000","last_fullsized":"2024-09-13T17:18:29.226187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3697","last_scrub_stamp":"2024-09-13T17:18:29.226132+0000","last_deep_scrub":"63'3697","last_deep_scrub_stamp":"2024-09-13T17:18:29.226132+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.226132+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:46.199826+0000","scrub_duration":0.032222396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20206,"num_write_kb":253140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"64'5","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:06.185518+0000","last_change":"2024-09-13T17:18:06.186368+0000","last_active":"2024-09-13T17:18:06.185518+0000","last_peered":"2024-09-13T17:18:06.185518+0000","last_clean":"2024-09-13T17:18:06.185518+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T17:18:06.185518+0000","last_undegraded":"2024-09-13T17:18:06.185518+0000","last_fullsized":"2024-09-13T17:18:06.185518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:06.185462+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:06.185462+0000","last_clean_scrub_stamp":"2024-09-13T17:18:06.185462+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:29.073557+0000","scrub_duration":0.0065555370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.158515+0000","last_change":"2024-09-13T17:18:02.158515+0000","last_active":"2024-09-13T17:18:02.158515+0000","last_peered":"2024-09-13T17:18:02.158515+0000","last_clean":"2024-09-13T17:18:02.158515+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T17:18:02.158515+0000","last_undegraded":"2024-09-13T17:18:02.158515+0000","last_fullsized":"2024-09-13T17:18:02.158515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.158216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.118447+0000","scrub_duration":0.003800346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"63'3522","reported_seq":3585,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975808+0000","last_change":"2024-09-13T16:48:08.165318+0000","last_active":"2024-09-13T17:17:44.975808+0000","last_peered":"2024-09-13T17:17:44.975808+0000","last_clean":"2024-09-13T17:17:44.975808+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T17:17:44.975808+0000","last_undegraded":"2024-09-13T17:17:44.975808+0000","last_fullsized":"2024-09-13T17:17:44.975808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18828,"num_write_kb":272277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"63'3874","reported_seq":3955,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.213453+0000","last_change":"2024-09-13T17:17:59.213453+0000","last_active":"2024-09-13T17:17:59.213453+0000","last_peered":"2024-09-13T17:17:59.213453+0000","last_clean":"2024-09-13T17:17:59.213453+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T17:17:59.213453+0000","last_undegraded":"2024-09-13T17:17:59.213453+0000","last_fullsized":"2024-09-13T17:17:59.213453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.213146+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.190638+0000","scrub_duration":0.033789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21413,"num_write_kb":278431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'19","reported_seq":193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.533761+0000","last_change":"2024-09-13T17:17:59.533761+0000","last_active":"2024-09-13T17:17:59.533761+0000","last_peered":"2024-09-13T17:17:59.533761+0000","last_clean":"2024-09-13T17:17:59.533761+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T17:17:59.533761+0000","last_undegraded":"2024-09-13T17:17:59.533761+0000","last_fullsized":"2024-09-13T17:17:59.533761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_deep_scrub":"64'19","last_deep_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.533477+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:42.670255+0000","scrub_duration":0.0092684789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:00.848306+0000","last_change":"2024-09-13T17:18:00.848525+0000","last_active":"2024-09-13T17:18:00.848306+0000","last_peered":"2024-09-13T17:18:00.848306+0000","last_clean":"2024-09-13T17:18:00.848306+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T17:18:00.848306+0000","last_undegraded":"2024-09-13T17:18:00.848306+0000","last_fullsized":"2024-09-13T17:18:00.848306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_clean_scrub_stamp":"2024-09-13T17:18:00.848253+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:51.294142+0000","scrub_duration":0.0034249229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"63'3622","reported_seq":3688,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.984518+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T17:17:44.984518+0000","last_peered":"2024-09-13T17:17:44.984518+0000","last_clean":"2024-09-13T17:17:44.984518+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T17:17:44.984518+0000","last_undegraded":"2024-09-13T17:17:44.984518+0000","last_fullsized":"2024-09-13T17:17:44.984518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":19996,"num_write_kb":293269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"63'3240","reported_seq":3319,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.494730+0000","last_change":"2024-09-13T17:18:10.494730+0000","last_active":"2024-09-13T17:18:10.494730+0000","last_peered":"2024-09-13T17:18:10.494730+0000","last_clean":"2024-09-13T17:18:10.494730+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T17:18:10.494730+0000","last_undegraded":"2024-09-13T17:18:10.494730+0000","last_fullsized":"2024-09-13T17:18:10.494730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3240","last_scrub_stamp":"2024-09-13T17:18:10.494430+0000","last_deep_scrub":"63'3240","last_deep_scrub_stamp":"2024-09-13T17:18:10.494430+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.494430+0000","objects_scrubbed":0,"log_size":3240,"log_dups_size":0,"ondisk_log_size":3240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:03.000138+0000","scrub_duration":0.014460961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":237245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"63'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.449798+0000","last_change":"2024-09-13T17:18:02.449798+0000","last_active":"2024-09-13T17:18:02.449798+0000","last_peered":"2024-09-13T17:18:02.449798+0000","last_clean":"2024-09-13T17:18:02.449798+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T17:18:02.449798+0000","last_undegraded":"2024-09-13T17:18:02.449798+0000","last_fullsized":"2024-09-13T17:18:02.449798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.449537+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.409768+0000","scrub_duration":0.0032633279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'5","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.060756+0000","last_change":"2024-09-13T17:17:51.060978+0000","last_active":"2024-09-13T17:17:51.060756+0000","last_peered":"2024-09-13T17:17:51.060756+0000","last_clean":"2024-09-13T17:17:51.060756+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T17:17:51.060756+0000","last_undegraded":"2024-09-13T17:17:51.060756+0000","last_fullsized":"2024-09-13T17:17:51.060756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.060702+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:45.719305+0000","scrub_duration":0.0041164770000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"63'3376","reported_seq":3439,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.981690+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T17:17:44.981690+0000","last_peered":"2024-09-13T17:17:44.981690+0000","last_clean":"2024-09-13T17:17:44.981690+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T17:17:44.981690+0000","last_undegraded":"2024-09-13T17:17:44.981690+0000","last_fullsized":"2024-09-13T17:17:44.981690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18879,"num_write_kb":251906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"63'4848","reported_seq":4916,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.590986+0000","last_change":"2024-09-13T16:47:59.057847+0000","last_active":"2024-09-13T17:17:49.590986+0000","last_peered":"2024-09-13T17:17:49.590986+0000","last_clean":"2024-09-13T17:17:49.590986+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T17:17:49.590986+0000","last_undegraded":"2024-09-13T17:17:49.590986+0000","last_fullsized":"2024-09-13T17:17:49.590986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26963,"num_write_kb":321221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"63'3","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.642511+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T17:17:48.642511+0000","last_peered":"2024-09-13T17:17:48.642511+0000","last_clean":"2024-09-13T17:17:48.642511+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T17:17:48.642511+0000","last_undegraded":"2024-09-13T17:17:48.642511+0000","last_fullsized":"2024-09-13T17:17:48.642511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'392","reported_seq":574,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.116288+0000","last_change":"2024-09-13T17:17:49.116420+0000","last_active":"2024-09-13T17:17:49.116288+0000","last_peered":"2024-09-13T17:17:49.116288+0000","last_clean":"2024-09-13T17:17:49.116288+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T17:17:49.116288+0000","last_undegraded":"2024-09-13T17:17:49.116288+0000","last_fullsized":"2024-09-13T17:17:49.116288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'392","last_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_deep_scrub":"64'392","last_deep_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.116249+0000","objects_scrubbed":4,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:49.076480+0000","scrub_duration":0.026108916999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":191,"num_read_kb":82094,"num_write":385,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'4856","reported_seq":4921,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974382+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T17:17:44.974382+0000","last_peered":"2024-09-13T17:17:44.974382+0000","last_clean":"2024-09-13T17:17:44.974382+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T17:17:44.974382+0000","last_undegraded":"2024-09-13T17:17:44.974382+0000","last_fullsized":"2024-09-13T17:17:44.974382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":26453,"num_write_kb":368210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'4063","reported_seq":4142,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:24.994002+0000","last_change":"2024-09-13T17:18:24.994290+0000","last_active":"2024-09-13T17:18:24.994002+0000","last_peered":"2024-09-13T17:18:24.994002+0000","last_clean":"2024-09-13T17:18:24.994002+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T17:18:24.994002+0000","last_undegraded":"2024-09-13T17:18:24.994002+0000","last_fullsized":"2024-09-13T17:18:24.994002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4063","last_scrub_stamp":"2024-09-13T17:18:24.993948+0000","last_deep_scrub":"63'4063","last_deep_scrub_stamp":"2024-09-13T17:18:24.993948+0000","last_clean_scrub_stamp":"2024-09-13T17:18:24.993948+0000","objects_scrubbed":0,"log_size":4063,"log_dups_size":0,"ondisk_log_size":4063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:17.499656+0000","scrub_duration":0.029339008999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21999,"num_write_kb":293018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"64'11","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:07.796310+0000","last_change":"2024-09-13T17:18:07.796310+0000","last_active":"2024-09-13T17:18:07.796310+0000","last_peered":"2024-09-13T17:18:07.796310+0000","last_clean":"2024-09-13T17:18:07.796310+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T17:18:07.796310+0000","last_undegraded":"2024-09-13T17:18:07.796310+0000","last_fullsized":"2024-09-13T17:18:07.796310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-13T17:18:07.796034+0000","last_deep_scrub":"64'11","last_deep_scrub_stamp":"2024-09-13T17:18:07.796034+0000","last_clean_scrub_stamp":"2024-09-13T17:18:07.796034+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:50.932812+0000","scrub_duration":0.0076063959999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":152,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'32","reported_seq":170,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.557513+0000","last_change":"2024-09-13T17:17:59.557788+0000","last_active":"2024-09-13T17:17:59.557513+0000","last_peered":"2024-09-13T17:17:59.557513+0000","last_clean":"2024-09-13T17:17:59.557513+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T17:17:59.557513+0000","last_undegraded":"2024-09-13T17:17:59.557513+0000","last_fullsized":"2024-09-13T17:17:59.557513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'32","last_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_deep_scrub":"64'32","last_deep_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.557461+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.534953+0000","scrub_duration":0.0076444640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":31,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"63'4239","reported_seq":4302,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.973983+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T17:17:44.973983+0000","last_peered":"2024-09-13T17:17:44.973983+0000","last_clean":"2024-09-13T17:17:44.973983+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T17:17:44.973983+0000","last_undegraded":"2024-09-13T17:17:44.973983+0000","last_fullsized":"2024-09-13T17:17:44.973983+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23316,"num_write_kb":318961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'3718","reported_seq":3796,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.214013+0000","last_change":"2024-09-13T17:17:57.214254+0000","last_active":"2024-09-13T17:17:57.214013+0000","last_peered":"2024-09-13T17:17:57.214013+0000","last_clean":"2024-09-13T17:17:57.214013+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T17:17:57.214013+0000","last_undegraded":"2024-09-13T17:17:57.214013+0000","last_fullsized":"2024-09-13T17:17:57.214013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3718","last_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_deep_scrub":"63'3718","last_deep_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.213958+0000","objects_scrubbed":0,"log_size":3718,"log_dups_size":0,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.719666+0000","scrub_duration":0.032519578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20956,"num_write_kb":260218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"64'395","reported_seq":569,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.567234+0000","last_change":"2024-09-13T17:18:04.567435+0000","last_active":"2024-09-13T17:18:04.567234+0000","last_peered":"2024-09-13T17:18:04.567234+0000","last_clean":"2024-09-13T17:18:04.567234+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T17:18:04.567234+0000","last_undegraded":"2024-09-13T17:18:04.567234+0000","last_fullsized":"2024-09-13T17:18:04.567234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'395","last_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_deep_scrub":"64'395","last_deep_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.567179+0000","objects_scrubbed":4,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:59.225783+0000","scrub_duration":0.027913209000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":171,"num_read_kb":77976,"num_write":389,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"63'3","reported_seq":68,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.674906+0000","last_change":"2024-09-13T16:48:07.785427+0000","last_active":"2024-09-13T17:17:46.674906+0000","last_peered":"2024-09-13T17:17:46.674906+0000","last_clean":"2024-09-13T17:17:46.674906+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T17:17:46.674906+0000","last_undegraded":"2024-09-13T17:17:46.674906+0000","last_fullsized":"2024-09-13T17:17:46.674906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"63'3522","reported_seq":3586,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994900+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T17:17:44.994900+0000","last_peered":"2024-09-13T17:17:44.994900+0000","last_clean":"2024-09-13T17:17:44.994900+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T17:17:44.994900+0000","last_undegraded":"2024-09-13T17:17:44.994900+0000","last_fullsized":"2024-09-13T17:17:44.994900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":4,"num_write":19373,"num_write_kb":266222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"63'5138","reported_seq":5206,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.132234+0000","last_change":"2024-09-13T16:47:59.048780+0000","last_active":"2024-09-13T17:17:48.132234+0000","last_peered":"2024-09-13T17:17:48.132234+0000","last_clean":"2024-09-13T17:17:48.132234+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T17:17:48.132234+0000","last_undegraded":"2024-09-13T17:17:48.132234+0000","last_fullsized":"2024-09-13T17:17:48.132234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27863,"num_write_kb":353454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'25","reported_seq":131,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.003205+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T17:17:48.003205+0000","last_peered":"2024-09-13T17:17:48.003205+0000","last_clean":"2024-09-13T17:17:48.003205+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T17:17:48.003205+0000","last_undegraded":"2024-09-13T17:17:48.003205+0000","last_fullsized":"2024-09-13T17:17:48.003205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'9","reported_seq":127,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.154830+0000","last_change":"2024-09-13T17:18:03.154830+0000","last_active":"2024-09-13T17:18:03.154830+0000","last_peered":"2024-09-13T17:18:03.154830+0000","last_clean":"2024-09-13T17:18:03.154830+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T17:18:03.154830+0000","last_undegraded":"2024-09-13T17:18:03.154830+0000","last_fullsized":"2024-09-13T17:18:03.154830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_deep_scrub":"64'9","last_deep_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.154514+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:05.788731+0000","scrub_duration":0.0080602239999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":168,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"63'3602","reported_seq":3668,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993944+0000","last_change":"2024-09-13T16:48:08.201191+0000","last_active":"2024-09-13T17:17:44.993944+0000","last_peered":"2024-09-13T17:17:44.993944+0000","last_clean":"2024-09-13T17:17:44.993944+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T17:17:44.993944+0000","last_undegraded":"2024-09-13T17:17:44.993944+0000","last_fullsized":"2024-09-13T17:17:44.993944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":20290,"num_write_kb":278255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'3743","reported_seq":3821,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.841035+0000","last_change":"2024-09-13T17:17:57.841262+0000","last_active":"2024-09-13T17:17:57.841035+0000","last_peered":"2024-09-13T17:17:57.841035+0000","last_clean":"2024-09-13T17:17:57.841035+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T17:17:57.841035+0000","last_undegraded":"2024-09-13T17:17:57.841035+0000","last_fullsized":"2024-09-13T17:17:57.841035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3743","last_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_deep_scrub":"63'3743","last_deep_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.840983+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:01.952379+0000","scrub_duration":0.026384685000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20404,"num_write_kb":278673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"63'3","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.986857+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T17:17:44.986857+0000","last_peered":"2024-09-13T17:17:44.986857+0000","last_clean":"2024-09-13T17:17:44.986857+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T17:17:44.986857+0000","last_undegraded":"2024-09-13T17:17:44.986857+0000","last_fullsized":"2024-09-13T17:17:44.986857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'19","reported_seq":203,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:20.179269+0000","last_change":"2024-09-13T17:18:20.179269+0000","last_active":"2024-09-13T17:18:20.179269+0000","last_peered":"2024-09-13T17:18:20.179269+0000","last_clean":"2024-09-13T17:18:20.179269+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T17:18:20.179269+0000","last_undegraded":"2024-09-13T17:18:20.179269+0000","last_fullsized":"2024-09-13T17:18:20.179269+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-13T17:18:20.178917+0000","last_deep_scrub":"64'19","last_deep_scrub_stamp":"2024-09-13T17:18:20.178917+0000","last_clean_scrub_stamp":"2024-09-13T17:18:20.178917+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:19.604304+0000","scrub_duration":0.010053862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":324,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"63'3189","reported_seq":3252,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992384+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T17:17:44.992384+0000","last_peered":"2024-09-13T17:17:44.992384+0000","last_clean":"2024-09-13T17:17:44.992384+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T17:17:44.992384+0000","last_undegraded":"2024-09-13T17:17:44.992384+0000","last_fullsized":"2024-09-13T17:17:44.992384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17342,"num_write_kb":249797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"63'4223","reported_seq":4290,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.985904+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T17:17:44.985904+0000","last_peered":"2024-09-13T17:17:44.985904+0000","last_clean":"2024-09-13T17:17:44.985904+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T17:17:44.985904+0000","last_undegraded":"2024-09-13T17:17:44.985904+0000","last_fullsized":"2024-09-13T17:17:44.985904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4223,"log_dups_size":0,"ondisk_log_size":4223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":295025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"64'5","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:06.450397+0000","last_change":"2024-09-13T17:18:06.450600+0000","last_active":"2024-09-13T17:18:06.450397+0000","last_peered":"2024-09-13T17:18:06.450397+0000","last_clean":"2024-09-13T17:18:06.450397+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T17:18:06.450397+0000","last_undegraded":"2024-09-13T17:18:06.450397+0000","last_fullsized":"2024-09-13T17:18:06.450397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_clean_scrub_stamp":"2024-09-13T17:18:06.450341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:29.338436+0000","scrub_duration":0.003290758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.123757+0000","last_change":"2024-09-13T17:17:53.123757+0000","last_active":"2024-09-13T17:17:53.123757+0000","last_peered":"2024-09-13T17:17:53.123757+0000","last_clean":"2024-09-13T17:17:53.123757+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T17:17:53.123757+0000","last_undegraded":"2024-09-13T17:17:53.123757+0000","last_fullsized":"2024-09-13T17:17:53.123757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.123466+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.011562+0000","scrub_duration":0.0038056650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'5357","reported_seq":5425,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.995114+0000","last_change":"2024-09-13T16:48:08.195549+0000","last_active":"2024-09-13T17:17:44.995114+0000","last_peered":"2024-09-13T17:17:44.995114+0000","last_clean":"2024-09-13T17:17:44.995114+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T17:17:44.995114+0000","last_undegraded":"2024-09-13T17:17:44.995114+0000","last_fullsized":"2024-09-13T17:17:44.995114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":39,"num_write":29637,"num_write_kb":393569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"63'26","reported_seq":93,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.207278+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T17:17:46.207278+0000","last_peered":"2024-09-13T17:17:46.207278+0000","last_clean":"2024-09-13T17:17:46.207278+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T17:17:46.207278+0000","last_undegraded":"2024-09-13T17:17:46.207278+0000","last_fullsized":"2024-09-13T17:17:46.207278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"63'5122","reported_seq":5200,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.190845+0000","last_change":"2024-09-13T17:18:01.191064+0000","last_active":"2024-09-13T17:18:01.190845+0000","last_peered":"2024-09-13T17:18:01.190845+0000","last_clean":"2024-09-13T17:18:01.190845+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T17:18:01.190845+0000","last_undegraded":"2024-09-13T17:18:01.190845+0000","last_fullsized":"2024-09-13T17:18:01.190845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5122","last_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_deep_scrub":"63'5122","last_deep_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.190787+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:03.825004+0000","scrub_duration":0.032904946999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":348704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"63'2","reported_seq":70,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980774+0000","last_change":"2024-09-13T16:47:58.660017+0000","last_active":"2024-09-13T17:17:44.980774+0000","last_peered":"2024-09-13T17:17:44.980774+0000","last_clean":"2024-09-13T17:17:44.980774+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T17:17:44.980774+0000","last_undegraded":"2024-09-13T17:17:44.980774+0000","last_fullsized":"2024-09-13T17:17:44.980774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"63'3554","reported_seq":3620,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.207332+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T17:17:46.207332+0000","last_peered":"2024-09-13T17:17:46.207332+0000","last_clean":"2024-09-13T17:17:46.207332+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T17:17:46.207332+0000","last_undegraded":"2024-09-13T17:17:46.207332+0000","last_fullsized":"2024-09-13T17:17:46.207332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":268389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"63'1091","reported_seq":1207,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.995556+0000","last_change":"2024-09-13T16:48:07.786906+0000","last_active":"2024-09-13T17:17:44.995556+0000","last_peered":"2024-09-13T17:17:44.995556+0000","last_clean":"2024-09-13T17:17:44.995556+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T17:17:44.995556+0000","last_undegraded":"2024-09-13T17:17:44.995556+0000","last_fullsized":"2024-09-13T17:17:44.995556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":9436,"num_write":1065,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"63'4714","reported_seq":4781,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.972738+0000","last_change":"2024-09-13T16:47:59.057328+0000","last_active":"2024-09-13T17:17:44.972738+0000","last_peered":"2024-09-13T17:17:44.972738+0000","last_clean":"2024-09-13T17:17:44.972738+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T17:17:44.972738+0000","last_undegraded":"2024-09-13T17:17:44.972738+0000","last_fullsized":"2024-09-13T17:17:44.972738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25459,"num_write_kb":343662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"63'3","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.973656+0000","last_change":"2024-09-13T16:47:58.683361+0000","last_active":"2024-09-13T17:17:44.973656+0000","last_peered":"2024-09-13T17:17:44.973656+0000","last_clean":"2024-09-13T17:17:44.973656+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T17:17:44.973656+0000","last_undegraded":"2024-09-13T17:17:44.973656+0000","last_fullsized":"2024-09-13T17:17:44.973656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"63'4328","reported_seq":4391,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675065+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T17:17:46.675065+0000","last_peered":"2024-09-13T17:17:46.675065+0000","last_clean":"2024-09-13T17:17:46.675065+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T17:17:46.675065+0000","last_undegraded":"2024-09-13T17:17:46.675065+0000","last_fullsized":"2024-09-13T17:17:46.675065+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24456,"num_write_kb":315374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"64'411","reported_seq":494,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.515483+0000","last_change":"2024-09-13T17:18:03.515735+0000","last_active":"2024-09-13T17:18:03.515483+0000","last_peered":"2024-09-13T17:18:03.515483+0000","last_clean":"2024-09-13T17:18:03.515483+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T17:18:03.515483+0000","last_undegraded":"2024-09-13T17:18:03.515483+0000","last_fullsized":"2024-09-13T17:18:03.515483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'411","last_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_deep_scrub":"64'411","last_deep_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.515426+0000","objects_scrubbed":1,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:06.149643+0000","scrub_duration":0.019312658999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2812346,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3190,"num_write":386,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"63'4155","reported_seq":4233,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:20.944870+0000","last_change":"2024-09-13T17:18:20.945097+0000","last_active":"2024-09-13T17:18:20.944870+0000","last_peered":"2024-09-13T17:18:20.944870+0000","last_clean":"2024-09-13T17:18:20.944870+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T17:18:20.944870+0000","last_undegraded":"2024-09-13T17:18:20.944870+0000","last_fullsized":"2024-09-13T17:18:20.944870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4155","last_scrub_stamp":"2024-09-13T17:18:20.944817+0000","last_deep_scrub":"63'4155","last_deep_scrub_stamp":"2024-09-13T17:18:20.944817+0000","last_clean_scrub_stamp":"2024-09-13T17:18:20.944817+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:43.832912+0000","scrub_duration":0.029833493999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22247,"num_write_kb":285891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'6","reported_seq":123,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.503784+0000","last_change":"2024-09-13T17:18:10.504013+0000","last_active":"2024-09-13T17:18:10.503784+0000","last_peered":"2024-09-13T17:18:10.503784+0000","last_clean":"2024-09-13T17:18:10.503784+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T17:18:10.503784+0000","last_undegraded":"2024-09-13T17:18:10.503784+0000","last_fullsized":"2024-09-13T17:18:10.503784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-09-13T17:18:10.503729+0000","last_deep_scrub":"64'6","last_deep_scrub_stamp":"2024-09-13T17:18:10.503729+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.503729+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:33.391824+0000","scrub_duration":0.0063226599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"63'3426","reported_seq":3492,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.132960+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T17:17:48.132960+0000","last_peered":"2024-09-13T17:17:48.132960+0000","last_clean":"2024-09-13T17:17:48.132960+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T17:17:48.132960+0000","last_undegraded":"2024-09-13T17:17:48.132960+0000","last_fullsized":"2024-09-13T17:17:48.132960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3426,"log_dups_size":0,"ondisk_log_size":3426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":18510,"num_write_kb":250514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'712","reported_seq":803,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631599+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T17:17:47.631599+0000","last_peered":"2024-09-13T17:17:47.631599+0000","last_clean":"2024-09-13T17:17:47.631599+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T17:17:47.631599+0000","last_undegraded":"2024-09-13T17:17:47.631599+0000","last_fullsized":"2024-09-13T17:17:47.631599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":74674,"num_write":702,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"63'4316","reported_seq":4395,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.250446+0000","last_change":"2024-09-13T17:18:10.250685+0000","last_active":"2024-09-13T17:18:10.250446+0000","last_peered":"2024-09-13T17:18:10.250446+0000","last_clean":"2024-09-13T17:18:10.250446+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T17:18:10.250446+0000","last_undegraded":"2024-09-13T17:18:10.250446+0000","last_fullsized":"2024-09-13T17:18:10.250446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4316","last_scrub_stamp":"2024-09-13T17:18:10.250390+0000","last_deep_scrub":"63'4316","last_deep_scrub_stamp":"2024-09-13T17:18:10.250390+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.250390+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:02.756098+0000","scrub_duration":0.033652449000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22350,"num_write_kb":290228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"64'45","reported_seq":162,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.128882+0000","last_change":"2024-09-13T17:17:59.128882+0000","last_active":"2024-09-13T17:17:59.128882+0000","last_peered":"2024-09-13T17:17:59.128882+0000","last_clean":"2024-09-13T17:17:59.128882+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T17:17:59.128882+0000","last_undegraded":"2024-09-13T17:17:59.128882+0000","last_fullsized":"2024-09-13T17:17:59.128882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'45","last_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_deep_scrub":"64'45","last_deep_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.128560+0000","objects_scrubbed":1,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.106052+0000","scrub_duration":0.0062496970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"63'3129","reported_seq":3193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975860+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T17:17:44.975860+0000","last_peered":"2024-09-13T17:17:44.975860+0000","last_clean":"2024-09-13T17:17:44.975860+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T17:17:44.975860+0000","last_undegraded":"2024-09-13T17:17:44.975860+0000","last_fullsized":"2024-09-13T17:17:44.975860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17279,"num_write_kb":257601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"63'4785","reported_seq":4865,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.565656+0000","last_change":"2024-09-13T17:17:49.565883+0000","last_active":"2024-09-13T17:17:49.565656+0000","last_peered":"2024-09-13T17:17:49.565656+0000","last_clean":"2024-09-13T17:17:49.565656+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T17:17:49.565656+0000","last_undegraded":"2024-09-13T17:17:49.565656+0000","last_fullsized":"2024-09-13T17:17:49.565656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4785","last_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_deep_scrub":"63'4785","last_deep_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.565599+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.011489+0000","scrub_duration":0.012593717000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26302,"num_write_kb":310249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"63'4231","reported_seq":4296,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996163+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T17:17:44.996163+0000","last_peered":"2024-09-13T17:17:44.996163+0000","last_clean":"2024-09-13T17:17:44.996163+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T17:17:44.996163+0000","last_undegraded":"2024-09-13T17:17:44.996163+0000","last_fullsized":"2024-09-13T17:17:44.996163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22604,"num_write_kb":324065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"63'3874","reported_seq":3955,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.662237+0000","last_change":"2024-09-13T17:18:04.662513+0000","last_active":"2024-09-13T17:18:04.662237+0000","last_peered":"2024-09-13T17:18:04.662237+0000","last_clean":"2024-09-13T17:18:04.662237+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T17:18:04.662237+0000","last_undegraded":"2024-09-13T17:18:04.662237+0000","last_fullsized":"2024-09-13T17:18:04.662237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.662182+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:07.296399+0000","scrub_duration":0.021443912999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20920,"num_write_kb":276387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"63'3865","reported_seq":3930,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976014+0000","last_change":"2024-09-13T16:48:08.177324+0000","last_active":"2024-09-13T17:17:44.976014+0000","last_peered":"2024-09-13T17:17:44.976014+0000","last_clean":"2024-09-13T17:17:44.976014+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T17:17:44.976014+0000","last_undegraded":"2024-09-13T17:17:44.976014+0000","last_fullsized":"2024-09-13T17:17:44.976014+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21432,"num_write_kb":286796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"63'4552","reported_seq":4623,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978243+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T17:17:44.978243+0000","last_peered":"2024-09-13T17:17:44.978243+0000","last_clean":"2024-09-13T17:17:44.978243+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T17:17:44.978243+0000","last_undegraded":"2024-09-13T17:17:44.978243+0000","last_fullsized":"2024-09-13T17:17:44.978243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":24031,"num_write_kb":313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"63'4071","reported_seq":4136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992984+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T17:17:44.992984+0000","last_peered":"2024-09-13T17:17:44.992984+0000","last_clean":"2024-09-13T17:17:44.992984+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T17:17:44.992984+0000","last_undegraded":"2024-09-13T17:17:44.992984+0000","last_fullsized":"2024-09-13T17:17:44.992984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":22061,"num_write_kb":309650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"63'3506","reported_seq":3576,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.004143+0000","last_change":"2024-09-13T16:47:59.057918+0000","last_active":"2024-09-13T17:17:45.004143+0000","last_peered":"2024-09-13T17:17:45.004143+0000","last_clean":"2024-09-13T17:17:45.004143+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T17:17:45.004143+0000","last_undegraded":"2024-09-13T17:17:45.004143+0000","last_fullsized":"2024-09-13T17:17:45.004143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3506,"log_dups_size":0,"ondisk_log_size":3506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19493,"num_write_kb":254380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"63'4523","reported_seq":4588,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992102+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T17:17:44.992102+0000","last_peered":"2024-09-13T17:17:44.992102+0000","last_clean":"2024-09-13T17:17:44.992102+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T17:17:44.992102+0000","last_undegraded":"2024-09-13T17:17:44.992102+0000","last_fullsized":"2024-09-13T17:17:44.992102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4523,"log_dups_size":0,"ondisk_log_size":4523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24221,"num_write_kb":338849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"63'4422","reported_seq":4492,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631627+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T17:17:47.631627+0000","last_peered":"2024-09-13T17:17:47.631627+0000","last_clean":"2024-09-13T17:17:47.631627+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T17:17:47.631627+0000","last_undegraded":"2024-09-13T17:17:47.631627+0000","last_fullsized":"2024-09-13T17:17:47.631627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":24862,"num_write_kb":304752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"63'3771","reported_seq":3837,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976919+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T17:17:44.976919+0000","last_peered":"2024-09-13T17:17:44.976919+0000","last_clean":"2024-09-13T17:17:44.976919+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T17:17:44.976919+0000","last_undegraded":"2024-09-13T17:17:44.976919+0000","last_fullsized":"2024-09-13T17:17:44.976919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21135,"num_write_kb":285540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"63'4461","reported_seq":4530,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.674960+0000","last_change":"2024-09-13T16:47:59.060910+0000","last_active":"2024-09-13T17:17:46.674960+0000","last_peered":"2024-09-13T17:17:46.674960+0000","last_clean":"2024-09-13T17:17:46.674960+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T17:17:46.674960+0000","last_undegraded":"2024-09-13T17:17:46.674960+0000","last_fullsized":"2024-09-13T17:17:46.674960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25388,"num_write_kb":319094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"63'4193","reported_seq":4258,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991839+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T17:17:44.991839+0000","last_peered":"2024-09-13T17:17:44.991839+0000","last_clean":"2024-09-13T17:17:44.991839+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T17:17:44.991839+0000","last_undegraded":"2024-09-13T17:17:44.991839+0000","last_fullsized":"2024-09-13T17:17:44.991839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4193,"log_dups_size":0,"ondisk_log_size":4193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23066,"num_write_kb":313713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"63'3978","reported_seq":4058,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.616395+0000","last_change":"2024-09-13T17:17:57.616624+0000","last_active":"2024-09-13T17:17:57.616395+0000","last_peered":"2024-09-13T17:17:57.616395+0000","last_clean":"2024-09-13T17:17:57.616395+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T17:17:57.616395+0000","last_undegraded":"2024-09-13T17:17:57.616395+0000","last_fullsized":"2024-09-13T17:17:57.616395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3978","last_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_deep_scrub":"63'3978","last_deep_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.616344+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:01.359043+0000","scrub_duration":0.024757745000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22157,"num_write_kb":281285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"63'4050","reported_seq":4117,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993755+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T17:17:44.993755+0000","last_peered":"2024-09-13T17:17:44.993755+0000","last_clean":"2024-09-13T17:17:44.993755+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T17:17:44.993755+0000","last_undegraded":"2024-09-13T17:17:44.993755+0000","last_fullsized":"2024-09-13T17:17:44.993755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4050,"log_dups_size":0,"ondisk_log_size":4050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21731,"num_write_kb":291923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"63'5211","reported_seq":5274,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009487+0000","last_change":"2024-09-13T16:48:40.831835+0000","last_active":"2024-09-13T17:17:45.009487+0000","last_peered":"2024-09-13T17:17:45.009487+0000","last_clean":"2024-09-13T17:17:45.009487+0000","last_became_active":"2024-09-13T16:48:40.831649+0000","last_became_peered":"2024-09-13T16:48:40.831649+0000","last_unstale":"2024-09-13T17:17:45.009487+0000","last_undegraded":"2024-09-13T17:17:45.009487+0000","last_fullsized":"2024-09-13T17:17:45.009487+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":28065,"num_write_kb":366501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"63'3806","reported_seq":3872,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975332+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T17:17:44.975332+0000","last_peered":"2024-09-13T17:17:44.975332+0000","last_clean":"2024-09-13T17:17:44.975332+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T17:17:44.975332+0000","last_undegraded":"2024-09-13T17:17:44.975332+0000","last_fullsized":"2024-09-13T17:17:44.975332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4,"num_write":20286,"num_write_kb":286237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"63'3659","reported_seq":3728,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.979776+0000","last_change":"2024-09-13T16:47:59.055658+0000","last_active":"2024-09-13T17:17:44.979776+0000","last_peered":"2024-09-13T17:17:44.979776+0000","last_clean":"2024-09-13T17:17:44.979776+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T17:17:44.979776+0000","last_undegraded":"2024-09-13T17:17:44.979776+0000","last_fullsized":"2024-09-13T17:17:44.979776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19936,"num_write_kb":264332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"63'5003","reported_seq":5067,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993722+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T17:17:44.993722+0000","last_peered":"2024-09-13T17:17:44.993722+0000","last_clean":"2024-09-13T17:17:44.993722+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T17:17:44.993722+0000","last_undegraded":"2024-09-13T17:17:44.993722+0000","last_fullsized":"2024-09-13T17:17:44.993722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27137,"num_write_kb":387189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"63'4068","reported_seq":4136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.972641+0000","last_change":"2024-09-13T16:47:59.051386+0000","last_active":"2024-09-13T17:17:44.972641+0000","last_peered":"2024-09-13T17:17:44.972641+0000","last_clean":"2024-09-13T17:17:44.972641+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T17:17:44.972641+0000","last_undegraded":"2024-09-13T17:17:44.972641+0000","last_fullsized":"2024-09-13T17:17:44.972641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22149,"num_write_kb":293545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"63'3283","reported_seq":3347,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007911+0000","last_change":"2024-09-13T16:48:08.183347+0000","last_active":"2024-09-13T17:17:45.007911+0000","last_peered":"2024-09-13T17:17:45.007911+0000","last_clean":"2024-09-13T17:17:45.007911+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T17:17:45.007911+0000","last_undegraded":"2024-09-13T17:17:45.007911+0000","last_fullsized":"2024-09-13T17:17:45.007911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":248866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"63'4697","reported_seq":4776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:13.654346+0000","last_change":"2024-09-13T17:18:13.654578+0000","last_active":"2024-09-13T17:18:13.654346+0000","last_peered":"2024-09-13T17:18:13.654346+0000","last_clean":"2024-09-13T17:18:13.654346+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T17:18:13.654346+0000","last_undegraded":"2024-09-13T17:18:13.654346+0000","last_fullsized":"2024-09-13T17:18:13.654346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4697","last_scrub_stamp":"2024-09-13T17:18:13.654295+0000","last_deep_scrub":"63'4697","last_deep_scrub_stamp":"2024-09-13T17:18:13.654295+0000","last_clean_scrub_stamp":"2024-09-13T17:18:13.654295+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:58.717333+0000","scrub_duration":0.035392587000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25800,"num_write_kb":320606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"63'3820","reported_seq":3884,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974366+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T17:17:44.974366+0000","last_peered":"2024-09-13T17:17:44.974366+0000","last_clean":"2024-09-13T17:17:44.974366+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T17:17:44.974366+0000","last_undegraded":"2024-09-13T17:17:44.974366+0000","last_fullsized":"2024-09-13T17:17:44.974366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3820,"log_dups_size":0,"ondisk_log_size":3820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20813,"num_write_kb":300503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"63'4719","reported_seq":4800,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:56.892882+0000","last_change":"2024-09-13T17:17:56.893148+0000","last_active":"2024-09-13T17:17:56.892882+0000","last_peered":"2024-09-13T17:17:56.892882+0000","last_clean":"2024-09-13T17:17:56.892882+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T17:17:56.892882+0000","last_undegraded":"2024-09-13T17:17:56.892882+0000","last_fullsized":"2024-09-13T17:17:56.892882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4719","last_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_deep_scrub":"63'4719","last_deep_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_clean_scrub_stamp":"2024-09-13T17:17:56.892826+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:26.656787+0000","scrub_duration":0.032626751000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":25263,"num_write_kb":330973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"63'3722","reported_seq":3786,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975189+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T17:17:44.975189+0000","last_peered":"2024-09-13T17:17:44.975189+0000","last_clean":"2024-09-13T17:17:44.975189+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T17:17:44.975189+0000","last_undegraded":"2024-09-13T17:17:44.975189+0000","last_fullsized":"2024-09-13T17:17:44.975189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20022,"num_write_kb":289122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"63'3813","reported_seq":3892,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:26.209831+0000","last_change":"2024-09-13T17:18:26.210107+0000","last_active":"2024-09-13T17:18:26.209831+0000","last_peered":"2024-09-13T17:18:26.209831+0000","last_clean":"2024-09-13T17:18:26.209831+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T17:18:26.209831+0000","last_undegraded":"2024-09-13T17:18:26.209831+0000","last_fullsized":"2024-09-13T17:18:26.209831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3813","last_scrub_stamp":"2024-09-13T17:18:26.209775+0000","last_deep_scrub":"63'3813","last_deep_scrub_stamp":"2024-09-13T17:18:26.209775+0000","last_clean_scrub_stamp":"2024-09-13T17:18:26.209775+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:11.272813+0000","scrub_duration":0.022253663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21202,"num_write_kb":265282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"63'4479","reported_seq":4547,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976201+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T17:17:44.976201+0000","last_peered":"2024-09-13T17:17:44.976201+0000","last_clean":"2024-09-13T17:17:44.976201+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T17:17:44.976201+0000","last_undegraded":"2024-09-13T17:17:44.976201+0000","last_fullsized":"2024-09-13T17:17:44.976201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":24556,"num_write_kb":340437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"63'4549","reported_seq":4617,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996096+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T17:17:44.996096+0000","last_peered":"2024-09-13T17:17:44.996096+0000","last_clean":"2024-09-13T17:17:44.996096+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T17:17:44.996096+0000","last_undegraded":"2024-09-13T17:17:44.996096+0000","last_fullsized":"2024-09-13T17:17:44.996096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24549,"num_write_kb":319591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"63'3479","reported_seq":3544,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.977167+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T17:17:44.977167+0000","last_peered":"2024-09-13T17:17:44.977167+0000","last_clean":"2024-09-13T17:17:44.977167+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T17:17:44.977167+0000","last_undegraded":"2024-09-13T17:17:44.977167+0000","last_fullsized":"2024-09-13T17:17:44.977167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3479,"log_dups_size":0,"ondisk_log_size":3479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18966,"num_write_kb":268432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"63'3573","reported_seq":3653,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:08.568217+0000","last_change":"2024-09-13T17:18:08.568217+0000","last_active":"2024-09-13T17:18:08.568217+0000","last_peered":"2024-09-13T17:18:08.568217+0000","last_clean":"2024-09-13T17:18:08.568217+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T17:18:08.568217+0000","last_undegraded":"2024-09-13T17:18:08.568217+0000","last_fullsized":"2024-09-13T17:18:08.568217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3573","last_scrub_stamp":"2024-09-13T17:18:08.567856+0000","last_deep_scrub":"63'3573","last_deep_scrub_stamp":"2024-09-13T17:18:08.567856+0000","last_clean_scrub_stamp":"2024-09-13T17:18:08.567856+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:07.880974+0000","scrub_duration":0.01354239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19348,"num_write_kb":270113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"63'4058","reported_seq":4122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991645+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T17:17:44.991645+0000","last_peered":"2024-09-13T17:17:44.991645+0000","last_clean":"2024-09-13T17:17:44.991645+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T17:17:44.991645+0000","last_undegraded":"2024-09-13T17:17:44.991645+0000","last_fullsized":"2024-09-13T17:17:44.991645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22331,"num_write_kb":302585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"63'5839","reported_seq":5907,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.642632+0000","last_change":"2024-09-13T16:47:59.054512+0000","last_active":"2024-09-13T17:17:48.642632+0000","last_peered":"2024-09-13T17:17:48.642632+0000","last_clean":"2024-09-13T17:17:48.642632+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T17:17:48.642632+0000","last_undegraded":"2024-09-13T17:17:48.642632+0000","last_fullsized":"2024-09-13T17:17:48.642632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31756,"num_write_kb":397521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"63'3835","reported_seq":3899,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992992+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T17:17:44.992992+0000","last_peered":"2024-09-13T17:17:44.992992+0000","last_clean":"2024-09-13T17:17:44.992992+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T17:17:44.992992+0000","last_undegraded":"2024-09-13T17:17:44.992992+0000","last_fullsized":"2024-09-13T17:17:44.992992+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":21185,"num_write_kb":306173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"63'4134","reported_seq":4213,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.595782+0000","last_change":"2024-09-13T17:17:57.595782+0000","last_active":"2024-09-13T17:17:57.595782+0000","last_peered":"2024-09-13T17:17:57.595782+0000","last_clean":"2024-09-13T17:17:57.595782+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T17:17:57.595782+0000","last_undegraded":"2024-09-13T17:17:57.595782+0000","last_fullsized":"2024-09-13T17:17:57.595782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4134","last_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_deep_scrub":"63'4134","last_deep_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.595505+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:50.101213+0000","scrub_duration":0.024248147000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22268,"num_write_kb":281801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"63'5073","reported_seq":5138,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.981849+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T17:17:44.981849+0000","last_peered":"2024-09-13T17:17:44.981849+0000","last_clean":"2024-09-13T17:17:44.981849+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T17:17:44.981849+0000","last_undegraded":"2024-09-13T17:17:44.981849+0000","last_fullsized":"2024-09-13T17:17:44.981849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":28693,"num_write_kb":393724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"63'3822","reported_seq":3901,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:07.186357+0000","last_change":"2024-09-13T17:18:07.186357+0000","last_active":"2024-09-13T17:18:07.186357+0000","last_peered":"2024-09-13T17:18:07.186357+0000","last_clean":"2024-09-13T17:18:07.186357+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T17:18:07.186357+0000","last_undegraded":"2024-09-13T17:18:07.186357+0000","last_fullsized":"2024-09-13T17:18:07.186357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3822","last_scrub_stamp":"2024-09-13T17:18:07.186043+0000","last_deep_scrub":"63'3822","last_deep_scrub_stamp":"2024-09-13T17:18:07.186043+0000","last_clean_scrub_stamp":"2024-09-13T17:18:07.186043+0000","objects_scrubbed":0,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:30.074139+0000","scrub_duration":0.030687262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20823,"num_write_kb":275830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"63'2778","reported_seq":2844,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975133+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T17:17:44.975133+0000","last_peered":"2024-09-13T17:17:44.975133+0000","last_clean":"2024-09-13T17:17:44.975133+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T17:17:44.975133+0000","last_undegraded":"2024-09-13T17:17:44.975133+0000","last_fullsized":"2024-09-13T17:17:44.975133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":34,"num_write":15212,"num_write_kb":208573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"63'3975","reported_seq":4054,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.856922+0000","last_change":"2024-09-13T17:18:01.856922+0000","last_active":"2024-09-13T17:18:01.856922+0000","last_peered":"2024-09-13T17:18:01.856922+0000","last_clean":"2024-09-13T17:18:01.856922+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T17:18:01.856922+0000","last_undegraded":"2024-09-13T17:18:01.856922+0000","last_fullsized":"2024-09-13T17:18:01.856922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3975","last_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_deep_scrub":"63'3975","last_deep_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.856645+0000","objects_scrubbed":0,"log_size":3975,"log_dups_size":0,"ondisk_log_size":3975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:51.096023+0000","scrub_duration":0.021351690999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21351,"num_write_kb":278200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"63'3844","reported_seq":3920,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:17.247940+0000","last_change":"2024-09-13T17:18:17.248778+0000","last_active":"2024-09-13T17:18:17.247940+0000","last_peered":"2024-09-13T17:18:17.247940+0000","last_clean":"2024-09-13T17:18:17.247940+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T17:18:17.247940+0000","last_undegraded":"2024-09-13T17:18:17.247940+0000","last_fullsized":"2024-09-13T17:18:17.247940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3844","last_scrub_stamp":"2024-09-13T17:18:17.247886+0000","last_deep_scrub":"63'3844","last_deep_scrub_stamp":"2024-09-13T17:18:17.247886+0000","last_clean_scrub_stamp":"2024-09-13T17:18:17.247886+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:16.519768+0000","scrub_duration":0.029367863000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21085,"num_write_kb":289508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"63'3324","reported_seq":3402,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:18.177496+0000","last_change":"2024-09-13T17:18:18.177724+0000","last_active":"2024-09-13T17:18:18.177496+0000","last_peered":"2024-09-13T17:18:18.177496+0000","last_clean":"2024-09-13T17:18:18.177496+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T17:18:18.177496+0000","last_undegraded":"2024-09-13T17:18:18.177496+0000","last_fullsized":"2024-09-13T17:18:18.177496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3324","last_scrub_stamp":"2024-09-13T17:18:18.177437+0000","last_deep_scrub":"63'3324","last_deep_scrub_stamp":"2024-09-13T17:18:18.177437+0000","last_clean_scrub_stamp":"2024-09-13T17:18:18.177437+0000","objects_scrubbed":0,"log_size":3324,"log_dups_size":0,"ondisk_log_size":3324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:17.449318+0000","scrub_duration":0.024197943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18810,"num_write_kb":235312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"63'3743","reported_seq":3807,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.972429+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T17:17:44.972429+0000","last_peered":"2024-09-13T17:17:44.972429+0000","last_clean":"2024-09-13T17:17:44.972429+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T17:17:44.972429+0000","last_undegraded":"2024-09-13T17:17:44.972429+0000","last_fullsized":"2024-09-13T17:17:44.972429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20790,"num_write_kb":280180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"63'3173","reported_seq":3240,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978903+0000","last_change":"2024-09-13T16:47:59.047332+0000","last_active":"2024-09-13T17:17:44.978903+0000","last_peered":"2024-09-13T17:17:44.978903+0000","last_clean":"2024-09-13T17:17:44.978903+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T17:17:44.978903+0000","last_undegraded":"2024-09-13T17:17:44.978903+0000","last_fullsized":"2024-09-13T17:17:44.978903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17044,"num_write_kb":219853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"63'3460","reported_seq":3524,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.007456+0000","last_change":"2024-09-13T16:48:08.169314+0000","last_active":"2024-09-13T17:17:45.007456+0000","last_peered":"2024-09-13T17:17:45.007456+0000","last_clean":"2024-09-13T17:17:45.007456+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T17:17:45.007456+0000","last_undegraded":"2024-09-13T17:17:45.007456+0000","last_fullsized":"2024-09-13T17:17:45.007456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":19788,"num_write_kb":263976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"63'3747","reported_seq":3825,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:12.464112+0000","last_change":"2024-09-13T17:18:12.464344+0000","last_active":"2024-09-13T17:18:12.464112+0000","last_peered":"2024-09-13T17:18:12.464112+0000","last_clean":"2024-09-13T17:18:12.464112+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T17:18:12.464112+0000","last_undegraded":"2024-09-13T17:18:12.464112+0000","last_fullsized":"2024-09-13T17:18:12.464112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3747","last_scrub_stamp":"2024-09-13T17:18:12.464055+0000","last_deep_scrub":"63'3747","last_deep_scrub_stamp":"2024-09-13T17:18:12.464055+0000","last_clean_scrub_stamp":"2024-09-13T17:18:12.464055+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:25.441547+0000","scrub_duration":0.021103311999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20231,"num_write_kb":269532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"63'3422","reported_seq":3496,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:27.174800+0000","last_change":"2024-09-13T17:18:27.175049+0000","last_active":"2024-09-13T17:18:27.174800+0000","last_peered":"2024-09-13T17:18:27.174800+0000","last_clean":"2024-09-13T17:18:27.174800+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T17:18:27.174800+0000","last_undegraded":"2024-09-13T17:18:27.174800+0000","last_fullsized":"2024-09-13T17:18:27.174800+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3422","last_scrub_stamp":"2024-09-13T17:18:27.174746+0000","last_deep_scrub":"63'3422","last_deep_scrub_stamp":"2024-09-13T17:18:27.174746+0000","last_clean_scrub_stamp":"2024-09-13T17:18:27.174746+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:39.964382+0000","scrub_duration":0.029236364000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18722,"num_write_kb":267617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"63'4803","reported_seq":4881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.621276+0000","last_change":"2024-09-13T17:17:58.621508+0000","last_active":"2024-09-13T17:17:58.621276+0000","last_peered":"2024-09-13T17:17:58.621276+0000","last_clean":"2024-09-13T17:17:58.621276+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T17:17:58.621276+0000","last_undegraded":"2024-09-13T17:17:58.621276+0000","last_fullsized":"2024-09-13T17:17:58.621276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4803","last_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_deep_scrub":"63'4803","last_deep_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.621222+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:06.376472+0000","scrub_duration":0.023354388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26967,"num_write_kb":333429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"63'3840","reported_seq":3903,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980170+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T17:17:44.980170+0000","last_peered":"2024-09-13T17:17:44.980170+0000","last_clean":"2024-09-13T17:17:44.980170+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T17:17:44.980170+0000","last_undegraded":"2024-09-13T17:17:44.980170+0000","last_fullsized":"2024-09-13T17:17:44.980170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21113,"num_write_kb":293770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"63'5433","reported_seq":5513,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.303582+0000","last_change":"2024-09-13T17:17:53.303824+0000","last_active":"2024-09-13T17:17:53.303582+0000","last_peered":"2024-09-13T17:17:53.303582+0000","last_clean":"2024-09-13T17:17:53.303582+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T17:17:53.303582+0000","last_undegraded":"2024-09-13T17:17:53.303582+0000","last_fullsized":"2024-09-13T17:17:53.303582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5433","last_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_deep_scrub":"63'5433","last_deep_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.303524+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.191620+0000","scrub_duration":0.041011686999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30664,"num_write_kb":389336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"63'4741","reported_seq":4808,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974867+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T17:17:44.974867+0000","last_peered":"2024-09-13T17:17:44.974867+0000","last_clean":"2024-09-13T17:17:44.974867+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T17:17:44.974867+0000","last_undegraded":"2024-09-13T17:17:44.974867+0000","last_fullsized":"2024-09-13T17:17:44.974867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":15,"num_write":25588,"num_write_kb":340391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"63'4281","reported_seq":4359,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.227117+0000","last_change":"2024-09-13T17:17:57.227117+0000","last_active":"2024-09-13T17:17:57.227117+0000","last_peered":"2024-09-13T17:17:57.227117+0000","last_clean":"2024-09-13T17:17:57.227117+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T17:17:57.227117+0000","last_undegraded":"2024-09-13T17:17:57.227117+0000","last_fullsized":"2024-09-13T17:17:57.227117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4281","last_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_deep_scrub":"63'4281","last_deep_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226795+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.732503+0000","scrub_duration":0.034883109000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23373,"num_write_kb":302635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"63'3575","reported_seq":3643,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994905+0000","last_change":"2024-09-13T16:48:08.178954+0000","last_active":"2024-09-13T17:17:44.994905+0000","last_peered":"2024-09-13T17:17:44.994905+0000","last_clean":"2024-09-13T17:17:44.994905+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T17:17:44.994905+0000","last_undegraded":"2024-09-13T17:17:44.994905+0000","last_fullsized":"2024-09-13T17:17:44.994905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":7,"num_write":20117,"num_write_kb":289677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"63'4404","reported_seq":4482,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.337858+0000","last_change":"2024-09-13T17:17:50.337858+0000","last_active":"2024-09-13T17:17:50.337858+0000","last_peered":"2024-09-13T17:17:50.337858+0000","last_clean":"2024-09-13T17:17:50.337858+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T17:17:50.337858+0000","last_undegraded":"2024-09-13T17:17:50.337858+0000","last_fullsized":"2024-09-13T17:17:50.337858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4404","last_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_deep_scrub":"63'4404","last_deep_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.337585+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.971802+0000","scrub_duration":0.031410212999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24308,"num_write_kb":306423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"63'3318","reported_seq":3385,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675177+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T17:17:46.675177+0000","last_peered":"2024-09-13T17:17:46.675177+0000","last_clean":"2024-09-13T17:17:46.675177+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T17:17:46.675177+0000","last_undegraded":"2024-09-13T17:17:46.675177+0000","last_fullsized":"2024-09-13T17:17:46.675177+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3318,"log_dups_size":0,"ondisk_log_size":3318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18295,"num_write_kb":253537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"63'4013","reported_seq":4091,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.598539+0000","last_change":"2024-09-13T17:17:53.598539+0000","last_active":"2024-09-13T17:17:53.598539+0000","last_peered":"2024-09-13T17:17:53.598539+0000","last_clean":"2024-09-13T17:17:53.598539+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T17:17:53.598539+0000","last_undegraded":"2024-09-13T17:17:53.598539+0000","last_fullsized":"2024-09-13T17:17:53.598539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4013","last_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_deep_scrub":"63'4013","last_deep_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.598242+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.486338+0000","scrub_duration":0.028004864000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21393,"num_write_kb":262042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"63'3658","reported_seq":3724,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.982869+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T17:17:44.982869+0000","last_peered":"2024-09-13T17:17:44.982869+0000","last_clean":"2024-09-13T17:17:44.982869+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T17:17:44.982869+0000","last_undegraded":"2024-09-13T17:17:44.982869+0000","last_fullsized":"2024-09-13T17:17:44.982869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19946,"num_write_kb":268446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"63'3910","reported_seq":3988,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:19.247862+0000","last_change":"2024-09-13T17:18:19.248105+0000","last_active":"2024-09-13T17:18:19.247862+0000","last_peered":"2024-09-13T17:18:19.247862+0000","last_clean":"2024-09-13T17:18:19.247862+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T17:18:19.247862+0000","last_undegraded":"2024-09-13T17:18:19.247862+0000","last_fullsized":"2024-09-13T17:18:19.247862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3910","last_scrub_stamp":"2024-09-13T17:18:19.247809+0000","last_deep_scrub":"63'3910","last_deep_scrub_stamp":"2024-09-13T17:18:19.247809+0000","last_clean_scrub_stamp":"2024-09-13T17:18:19.247809+0000","objects_scrubbed":0,"log_size":3910,"log_dups_size":0,"ondisk_log_size":3910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:18.673196+0000","scrub_duration":0.029454408000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":278651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"63'3074","reported_seq":3137,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632259+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T17:17:47.632259+0000","last_peered":"2024-09-13T17:17:47.632259+0000","last_clean":"2024-09-13T17:17:47.632259+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T17:17:47.632259+0000","last_undegraded":"2024-09-13T17:17:47.632259+0000","last_fullsized":"2024-09-13T17:17:47.632259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":246226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"63'3515","reported_seq":3597,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.560367+0000","last_change":"2024-09-13T17:17:52.560367+0000","last_active":"2024-09-13T17:17:52.560367+0000","last_peered":"2024-09-13T17:17:52.560367+0000","last_clean":"2024-09-13T17:17:52.560367+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T17:17:52.560367+0000","last_undegraded":"2024-09-13T17:17:52.560367+0000","last_fullsized":"2024-09-13T17:17:52.560367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3515","last_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_deep_scrub":"63'3515","last_deep_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.560093+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.500715+0000","scrub_duration":0.026529094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19512,"num_write_kb":258445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"63'3439","reported_seq":3503,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.164818+0000","last_change":"2024-09-13T16:48:08.164437+0000","last_active":"2024-09-13T17:17:47.164818+0000","last_peered":"2024-09-13T17:17:47.164818+0000","last_clean":"2024-09-13T17:17:47.164818+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T17:17:47.164818+0000","last_undegraded":"2024-09-13T17:17:47.164818+0000","last_fullsized":"2024-09-13T17:17:47.164818+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":17681,"num_write_kb":263603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"63'3892","reported_seq":3962,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.995220+0000","last_change":"2024-09-13T16:47:59.047736+0000","last_active":"2024-09-13T17:17:44.995220+0000","last_peered":"2024-09-13T17:17:44.995220+0000","last_clean":"2024-09-13T17:17:44.995220+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T17:17:44.995220+0000","last_undegraded":"2024-09-13T17:17:44.995220+0000","last_fullsized":"2024-09-13T17:17:44.995220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21304,"num_write_kb":286190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"63'3622","reported_seq":3685,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591670+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T17:17:49.591670+0000","last_peered":"2024-09-13T17:17:49.591670+0000","last_clean":"2024-09-13T17:17:49.591670+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T17:17:49.591670+0000","last_undegraded":"2024-09-13T17:17:49.591670+0000","last_fullsized":"2024-09-13T17:17:49.591670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19729,"num_write_kb":275691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"63'5036","reported_seq":5105,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.994403+0000","last_change":"2024-09-13T16:47:59.048483+0000","last_active":"2024-09-13T17:17:44.994403+0000","last_peered":"2024-09-13T17:17:44.994403+0000","last_clean":"2024-09-13T17:17:44.994403+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T17:17:44.994403+0000","last_undegraded":"2024-09-13T17:17:44.994403+0000","last_fullsized":"2024-09-13T17:17:44.994403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27469,"num_write_kb":368239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"63'3689","reported_seq":3752,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591554+0000","last_change":"2024-09-13T16:48:08.193644+0000","last_active":"2024-09-13T17:17:49.591554+0000","last_peered":"2024-09-13T17:17:49.591554+0000","last_clean":"2024-09-13T17:17:49.591554+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T17:17:49.591554+0000","last_undegraded":"2024-09-13T17:17:49.591554+0000","last_fullsized":"2024-09-13T17:17:49.591554+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3689,"log_dups_size":0,"ondisk_log_size":3689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20818,"num_write_kb":284187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"63'4047","reported_seq":4128,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.364122+0000","last_change":"2024-09-13T17:17:51.364373+0000","last_active":"2024-09-13T17:17:51.364122+0000","last_peered":"2024-09-13T17:17:51.364122+0000","last_clean":"2024-09-13T17:17:51.364122+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T17:17:51.364122+0000","last_undegraded":"2024-09-13T17:17:51.364122+0000","last_fullsized":"2024-09-13T17:17:51.364122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4047","last_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_deep_scrub":"63'4047","last_deep_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.364064+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:46.022668+0000","scrub_duration":0.028243739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22743,"num_write_kb":291925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":197,"num_write":1329762,"num_write_kb":18444058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242925,"ondisk_log_size":242925,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19024977,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1583,"num_read_kb":206738,"num_write":4472,"num_write_kb":20527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38109184,"data_stored":38049954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4371,"ondisk_log_size":4371,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":1479870,"num_write_kb":19043750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270762,"ondisk_log_size":270762,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":19675678,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1366,"num_read_kb":209632,"num_write":4521,"num_write_kb":21162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39411712,"data_stored":39351356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4379,"ondisk_log_size":4379,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575221,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":942784,"kb_used_data":13688,"kb_used_omap":1,"kb_used_meta":929086,"kb_avail":93429056,"statfs":{"total":96636764160,"available":95671353344,"internally_reserved":0,"allocated":14016512,"data_stored":13616965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":951384523},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.853,"5min":2.1549999999999998,"15min":1.284},"min":{"1min":0.26900000000000002,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":15.93,"5min":16.571999999999999,"15min":16.571999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":2.089,"5min":2.2909999999999999,"15min":1.3859999999999999},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.20899999999999999},"max":{"1min":14.792,"5min":16.523,"15min":16.523},"last":1.028}]},{"osd":1,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":2.927,"15min":1.5589999999999999},"min":{"1min":0.27800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":15.965999999999999,"5min":59.911999999999999,"15min":59.911999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":2.48,"5min":3.1080000000000001,"15min":1.6699999999999999},"min":{"1min":0.38400000000000001,"5min":0.28399999999999997,"15min":0.23000000000000001},"max":{"1min":14.823,"5min":59.895000000000003,"15min":59.895000000000003},"last":0.94799999999999995}]},{"osd":2,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.798,"15min":1.153},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.25700000000000001},"max":{"1min":6.0039999999999996,"5min":16.023,"15min":16.023},"last":0.97099999999999997},{"interface":"front","average":{"1min":2.5670000000000002,"5min":2.0030000000000001,"15min":1.2669999999999999},"min":{"1min":0.34300000000000003,"5min":0.249,"15min":0.249},"max":{"1min":18.734000000000002,"5min":18.734000000000002,"15min":18.734000000000002},"last":1.1319999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.1640000000000001,"5min":2.1989999999999998,"15min":1.306},"min":{"1min":0.38900000000000001,"5min":0.29499999999999998,"15min":0.28299999999999997},"max":{"1min":13.859999999999999,"5min":23.225000000000001,"15min":23.225000000000001},"last":1.052},{"interface":"front","average":{"1min":3.673,"5min":2.4900000000000002,"15min":1.46},"min":{"1min":0.48799999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":19.709,"5min":19.709,"15min":19.709},"last":0.91900000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.7679999999999998,"5min":2.4390000000000001,"15min":1.399},"min":{"1min":0.41799999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":16.555,"5min":18.004999999999999,"15min":18.004999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":2.294,"5min":2.5939999999999999,"15min":1.484},"min":{"1min":0.309,"5min":0.245,"15min":0.245},"max":{"1min":16.684000000000001,"5min":22.315999999999999,"15min":22.315999999999999},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.657,"5min":2.6459999999999999,"15min":1.5289999999999999},"min":{"1min":0.47799999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":17.321000000000002,"5min":37.302999999999997,"15min":37.302999999999997},"last":0.997},{"interface":"front","average":{"1min":2.5619999999999998,"5min":2.0800000000000001,"15min":1.3700000000000001},"min":{"1min":0.42899999999999999,"5min":0.32500000000000001,"15min":0.316},"max":{"1min":14.991,"5min":16.867999999999999,"15min":16.867999999999999},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.4529999999999998,"5min":2.0979999999999999,"15min":1.3859999999999999},"min":{"1min":0.496,"5min":0.28000000000000003,"15min":0.23000000000000001},"max":{"1min":18.335000000000001,"5min":18.335000000000001,"15min":18.335000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1549999999999998,"15min":1.397},"min":{"1min":0.41699999999999998,"5min":0.33200000000000002,"15min":0.28100000000000003},"max":{"1min":9.0739999999999998,"5min":19.244,"15min":19.244},"last":0.88500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575218,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673216,"kb_used_data":11768,"kb_used_omap":1,"kb_used_meta":661438,"kb_avail":93698624,"statfs":{"total":96636764160,"available":95947390976,"internally_reserved":0,"allocated":12050432,"data_stored":11645255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":677312968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.0700000000000001,"15min":0.89000000000000001},"min":{"1min":0.27500000000000002,"5min":0.222,"15min":0.222},"max":{"1min":6.0780000000000003,"5min":6.335,"15min":6.335},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.052,"5min":1.133,"15min":0.91200000000000003},"min":{"1min":0.247,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":4.907,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":1.198,"15min":0.95099999999999996},"min":{"1min":0.24299999999999999,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.474,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.736,"5min":1.6850000000000001,"15min":1.0980000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":7.3600000000000003,"5min":10.111000000000001,"15min":10.111000000000001},"last":0.93500000000000005}]},{"osd":2,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.234,"15min":0.97799999999999998},"min":{"1min":0.432,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":15.638,"5min":15.638,"15min":15.638},"last":0.76900000000000002},{"interface":"front","average":{"1min":2.1960000000000002,"5min":1.4450000000000001,"15min":1.091},"min":{"1min":0.41999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":19.442,"5min":19.442,"15min":19.442},"last":1.0780000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.5840000000000001,"15min":1.1259999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":3.7389999999999999,"5min":15.247999999999999,"15min":15.247999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.413,"5min":1.6399999999999999,"15min":1.161},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":6.1379999999999999,"5min":15.179,"15min":15.179},"last":1.0009999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":1.4810000000000001,"15min":1.1160000000000001},"min":{"1min":0.35299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":14.114000000000001,"5min":15.326000000000001,"15min":15.326000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.8799999999999999,"5min":1.6619999999999999,"15min":1.1930000000000001},"min":{"1min":0.27300000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":14.057,"5min":15.026,"15min":15.026},"last":1.0289999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.6659999999999999,"15min":1.242},"min":{"1min":0.42699999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":6.2690000000000001,"5min":11.566000000000001,"15min":11.566000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.4570000000000001,"15min":1.1879999999999999},"min":{"1min":0.36199999999999999,"5min":0.308,"15min":0.308},"max":{"1min":6.2789999999999999,"5min":10.223000000000001,"15min":10.223000000000001},"last":0.98399999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.6499999999999999,"15min":1.238},"min":{"1min":0.318,"5min":0.311,"15min":0.311},"max":{"1min":4.0979999999999999,"5min":13.112,"15min":13.112},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.782,"15min":1.274},"min":{"1min":0.39700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.6679999999999999,"5min":14.4,"15min":14.4},"last":1.0569999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575219,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":957788,"kb_used_data":5012,"kb_used_omap":1,"kb_used_meta":952766,"kb_avail":93414052,"statfs":{"total":96636764160,"available":95655989248,"internally_reserved":0,"allocated":5132288,"data_stored":4730805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":975632838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":3.762,"5min":1.978,"15min":1.1919999999999999},"min":{"1min":0.246,"5min":0.215,"15min":0.215},"max":{"1min":27.952999999999999,"5min":27.952999999999999,"15min":27.952999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":2.5059999999999998,"5min":1.629,"15min":1.121},"min":{"1min":0.28100000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":23.619,"5min":23.619,"15min":23.619},"last":1.0660000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.8210000000000002,"5min":1.8109999999999999,"15min":1.1379999999999999},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":13.515000000000001,"5min":13.515000000000001,"15min":13.515000000000001},"last":1.107},{"interface":"front","average":{"1min":1.621,"5min":1.9299999999999999,"15min":1.1839999999999999},"min":{"1min":0.26700000000000002,"5min":0.248,"15min":0.248},"max":{"1min":6.9290000000000003,"5min":35.186999999999998,"15min":35.186999999999998},"last":0.629}]},{"osd":2,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":2.306,"15min":1.3420000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":11.609,"5min":51.18,"15min":51.18},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.732,"5min":1.929,"15min":1.2949999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.8659999999999997,"5min":48.756,"15min":48.756},"last":1.1759999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.7480000000000002,"5min":2.0219999999999998,"15min":1.2789999999999999},"min":{"1min":0.39200000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":15.946999999999999,"5min":19.800000000000001,"15min":19.800000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.7649999999999999,"15min":1.256},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":9.1020000000000003,"5min":9.4789999999999992,"15min":9.4789999999999992},"last":0.91900000000000004}]},{"osd":4,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":4.2270000000000003,"5min":2.4260000000000002,"15min":1.379},"min":{"1min":0.40699999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":20.713999999999999,"5min":22.361999999999998,"15min":22.361999999999998},"last":0.80500000000000005},{"interface":"front","average":{"1min":4.1710000000000003,"5min":2.9940000000000002,"15min":1.641},"min":{"1min":0.35299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":28.062999999999999,"5min":31.843,"15min":31.843},"last":1.1279999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.427,"15min":1.131},"min":{"1min":0.375,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":3.3300000000000001,"5min":8.1910000000000007,"15min":8.1910000000000007},"last":1.028},{"interface":"front","average":{"1min":2.052,"5min":1.7649999999999999,"15min":1.26},"min":{"1min":0.375,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.1059999999999999,"5min":13.031000000000001,"15min":13.031000000000001},"last":0.95099999999999996}]},{"osd":7,"last update":"Fri Sep 13 17:17:37 2024","interfaces":[{"interface":"back","average":{"1min":2.1920000000000002,"5min":2.1890000000000001,"15min":1.4139999999999999},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":11.239000000000001,"5min":35.124000000000002,"15min":35.124000000000002},"last":1.2150000000000001},{"interface":"front","average":{"1min":2.0550000000000002,"5min":2.5089999999999999,"15min":1.546},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.99,"5min":32.328000000000003,"15min":32.328000000000003},"last":1.2410000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575219,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1011352,"kb_used_data":13968,"kb_used_omap":1,"kb_used_meta":997374,"kb_avail":93360488,"statfs":{"total":96636764160,"available":95601139712,"internally_reserved":0,"allocated":14303232,"data_stored":13890488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1021311430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":2.4870000000000001,"5min":1.8520000000000001,"15min":1.155},"min":{"1min":0.47399999999999998,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":18.841000000000001,"5min":18.841000000000001,"15min":18.841000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":2.6619999999999999,"5min":1.9570000000000001,"15min":1.2609999999999999},"min":{"1min":0.33800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":20.312000000000001,"5min":20.312000000000001,"15min":20.312000000000001},"last":0.95699999999999996}]},{"osd":1,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":1.7030000000000001,"15min":1.1020000000000001},"min":{"1min":0.27100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0840000000000001,"5min":27.242000000000001,"15min":27.242000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":2.1850000000000001,"5min":2.2829999999999999,"15min":1.3220000000000001},"min":{"1min":0.39100000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.103999999999999,"5min":26.545999999999999,"15min":26.545999999999999},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.5880000000000001,"15min":1.1080000000000001},"min":{"1min":0.29199999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8839999999999999,"5min":11.803000000000001,"15min":11.803000000000001},"last":0.751},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.825,"15min":1.2210000000000001},"min":{"1min":0.55300000000000005,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":13.972,"5min":13.972,"15min":13.972},"last":1.1759999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":2.3969999999999998,"15min":1.409},"min":{"1min":0.39900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":10.763,"5min":15.085000000000001,"15min":15.085000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.874,"5min":1.9550000000000001,"15min":1.2909999999999999},"min":{"1min":0.33200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2,"5min":20.468,"15min":20.468},"last":1.107}]},{"osd":5,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":2.645,"5min":1.796,"15min":1.2010000000000001},"min":{"1min":0.42299999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":14.031000000000001,"5min":14.031000000000001,"15min":14.031000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":2.4180000000000001,"5min":2.1339999999999999,"15min":1.363},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":14.205,"5min":15.044,"15min":15.044},"last":1.0669999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.546,"15min":1.155},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.7160000000000002,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":0.99399999999999999},{"interface":"front","average":{"1min":2.2970000000000002,"5min":1.911,"15min":1.3169999999999999},"min":{"1min":0.45100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":14.131,"5min":14.131,"15min":14.131},"last":1.2589999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":3.056,"5min":2.2269999999999999,"15min":1.478},"min":{"1min":0.36899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":20.154,"5min":24.518999999999998,"15min":24.518999999999998},"last":1.296},{"interface":"front","average":{"1min":1.984,"5min":1.851,"15min":1.3640000000000001},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":15.209,"5min":15.209,"15min":15.209},"last":1.222}]}]},{"osd":3,"up_from":13,"seq":55834575219,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":958484,"kb_used_data":19404,"kb_used_omap":1,"kb_used_meta":939070,"kb_avail":93413356,"statfs":{"total":96636764160,"available":95655276544,"internally_reserved":0,"allocated":19869696,"data_stored":19459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":961608136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.5880000000000001,"15min":0.97999999999999998},"min":{"1min":0.19400000000000001,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":19.327999999999999,"5min":19.327999999999999,"15min":19.327999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.831,"5min":1.7549999999999999,"15min":1.0449999999999999},"min":{"1min":0.29699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":17.754000000000001,"5min":20.699999999999999,"15min":20.699999999999999},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":1.3720000000000001,"15min":0.90300000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.3340000000000001,"5min":16.100000000000001,"15min":16.100000000000001},"last":0.622},{"interface":"front","average":{"1min":0.70999999999999996,"5min":1.3260000000000001,"15min":0.97499999999999998},"min":{"1min":0.247,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.847,"5min":10.417,"15min":10.417},"last":0.95299999999999996}]},{"osd":2,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3280000000000001,"15min":0.96699999999999997},"min":{"1min":0.38800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.6509999999999998,"5min":8.5950000000000006,"15min":8.5950000000000006},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.276,"15min":0.96499999999999997},"min":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":4.5380000000000003,"5min":15.433999999999999,"15min":15.433999999999999},"last":0.85199999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":3.4470000000000001,"5min":2.1099999999999999,"15min":1.3500000000000001},"min":{"1min":0.308,"5min":0.30599999999999999,"15min":0.29599999999999999},"max":{"1min":15.919,"5min":15.919,"15min":15.919},"last":1.0740000000000001},{"interface":"front","average":{"1min":2.3170000000000002,"5min":1.712,"15min":1.2110000000000001},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":9.3819999999999997,"5min":9.3819999999999997,"15min":9.3819999999999997},"last":1.1180000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.9119999999999999,"15min":1.321},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":9.4030000000000005,"5min":31.187999999999999,"15min":31.187999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":2.427,"5min":2.121,"15min":1.383},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":22.951000000000001,"5min":22.951000000000001,"15min":22.951000000000001},"last":0.98599999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.873,"15min":1.3120000000000001},"min":{"1min":0.44400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":4.202,"5min":12.567,"15min":12.567},"last":1.093},{"interface":"front","average":{"1min":1.226,"5min":2.1779999999999999,"15min":1.4730000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.7380000000000004,"5min":15.882,"15min":15.882},"last":1.2629999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:43 2024","interfaces":[{"interface":"back","average":{"1min":4.8339999999999996,"5min":2.4390000000000001,"15min":1.538},"min":{"1min":0.48999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":33.969000000000001,"5min":33.969000000000001,"15min":33.969000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":2.2480000000000002,"15min":1.46},"min":{"1min":0.47799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.4550000000000001,"5min":25.652999999999999,"15min":25.652999999999999},"last":1.173}]}]},{"osd":2,"up_from":13,"seq":55834575219,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662284,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":661374,"kb_avail":93709556,"statfs":{"total":96636764160,"available":95958585344,"internally_reserved":0,"allocated":921600,"data_stored":527729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":677247430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.90800000000000003,"15min":0.72499999999999998},"min":{"1min":0.28499999999999998,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":3.6509999999999998,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.516,"15min":1.0329999999999999},"min":{"1min":0.26300000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":6.6539999999999999,"5min":9.2729999999999997,"15min":9.2729999999999997},"last":0.91400000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":1.7110000000000001,"15min":1.0700000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20000000000000001},"max":{"1min":2.7450000000000001,"5min":33.579999999999998,"15min":33.579999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.7310000000000001,"15min":1.119},"min":{"1min":0.35199999999999998,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":3.7029999999999998,"5min":23.965,"15min":23.965},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":2.2069999999999999,"15min":1.3049999999999999},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.245},"max":{"1min":10.137,"5min":22.760000000000002,"15min":22.760000000000002},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.738,"15min":1.127},"min":{"1min":0.28799999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":8.4849999999999994,"5min":10.372,"15min":10.372},"last":0.84199999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.9850000000000001,"15min":1.2909999999999999},"min":{"1min":0.35799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.6400000000000001,"5min":20.673999999999999,"15min":20.673999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.395,"5min":1.7410000000000001,"15min":1.2709999999999999},"min":{"1min":0.54400000000000004,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.6699999999999999,"5min":16.5,"15min":16.5},"last":1.0669999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":2.484,"15min":1.528},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.798,"5min":21.722000000000001,"15min":21.722000000000001},"last":1.425},{"interface":"front","average":{"1min":1.2050000000000001,"5min":2.222,"15min":1.427},"min":{"1min":0.53800000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":4.9189999999999996,"5min":29.620999999999999,"15min":29.620999999999999},"last":1.1060000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.702,"15min":1.2889999999999999},"min":{"1min":0.51300000000000001,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":6.5880000000000001,"5min":21.882999999999999,"15min":21.882999999999999},"last":1.175},{"interface":"front","average":{"1min":1.748,"5min":2.2210000000000001,"15min":1.4710000000000001},"min":{"1min":0.34799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.407,"5min":25.274999999999999,"15min":25.274999999999999},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":3.2799999999999998,"15min":1.859},"min":{"1min":0.61199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":7.0119999999999996,"5min":43.649000000000001,"15min":43.649000000000001},"last":1.26},{"interface":"front","average":{"1min":1.3400000000000001,"5min":3.1589999999999998,"15min":1.855},"min":{"1min":0.40200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.6949999999999998,"5min":27.417999999999999,"15min":27.417999999999999},"last":1.4730000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575219,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798984,"kb_used_data":9024,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572856,"statfs":{"total":96636764160,"available":95818604544,"internally_reserved":0,"allocated":9240576,"data_stored":8832475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":808909254},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.8620000000000001,"15min":1.0800000000000001},"min":{"1min":0.185,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":4.851,"5min":15.81,"15min":15.81},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.681,"15min":1.0609999999999999},"min":{"1min":0.188,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":2.8319999999999999,"5min":15.823,"15min":15.823},"last":0.85699999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.411,"15min":0.92200000000000004},"min":{"1min":0.215,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":3.8119999999999998,"5min":11.672000000000001,"15min":11.672000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.2929999999999999,"15min":0.95299999999999996},"min":{"1min":0.32200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.6890000000000001,"5min":11.722,"15min":11.722},"last":0.94199999999999995}]},{"osd":3,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":2.024,"15min":1.2190000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.22},"max":{"1min":6.6379999999999999,"5min":16.507999999999999,"15min":16.507999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.016,"5min":1.482,"15min":1.0169999999999999},"min":{"1min":0.34799999999999998,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":4.3440000000000003,"5min":12.047000000000001,"15min":12.047000000000001},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":2.1299999999999999,"15min":1.27},"min":{"1min":0.35899999999999999,"5min":0.27800000000000002,"15min":0.24399999999999999},"max":{"1min":3.7530000000000001,"5min":22.234000000000002,"15min":22.234000000000002},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.431,"5min":2.258,"15min":1.403},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":8.1240000000000006,"5min":20.448,"15min":20.448},"last":1.127}]},{"osd":5,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.679,"15min":1.2010000000000001},"min":{"1min":0.44900000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":4.3090000000000002,"5min":15.788,"15min":15.788},"last":0.97299999999999998},{"interface":"front","average":{"1min":2.5659999999999998,"5min":2.1709999999999998,"15min":1.359},"min":{"1min":0.33400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":24.763999999999999,"5min":24.763999999999999,"15min":24.763999999999999},"last":1.0009999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.3879999999999999,"15min":1.139},"min":{"1min":0.43099999999999999,"5min":0.255,"15min":0.255},"max":{"1min":2.7200000000000002,"5min":12.019,"15min":12.019},"last":1.153},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.857,"15min":1.272},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.7970000000000002,"5min":11.628,"15min":11.628},"last":1.1859999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":2.02,"15min":1.405},"min":{"1min":0.50600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":6.9850000000000003,"5min":21.161000000000001,"15min":21.161000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":2.4969999999999999,"15min":1.554},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.1980000000000004,"5min":29.323,"15min":29.323},"last":1.2470000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575219,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799556,"kb_used_data":9596,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572284,"statfs":{"total":96636764160,"available":95818018816,"internally_reserved":0,"allocated":9826304,"data_stored":9422627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":808909256},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.4610000000000001,"15min":0.90800000000000003},"min":{"1min":0.22,"5min":0.20000000000000001,"15min":0.189},"max":{"1min":1.677,"5min":59.031999999999996,"15min":59.031999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":2.0870000000000002,"5min":2.351,"15min":1.2869999999999999},"min":{"1min":0.23499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":9.8800000000000008,"5min":57.186999999999998,"15min":57.186999999999998},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.96,"5min":1.155,"15min":0.871},"min":{"1min":0.26700000000000002,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":12.971,"5min":12.971,"15min":12.971},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.9990000000000001,"5min":1.371,"15min":0.93400000000000005},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":14.526,"5min":14.526,"15min":14.526},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.101,"5min":2.6800000000000002,"15min":1.4370000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":23.495999999999999,"5min":65.215000000000003,"15min":65.215000000000003},"last":1.085},{"interface":"front","average":{"1min":1.373,"5min":1.8169999999999999,"15min":1.1479999999999999},"min":{"1min":0.435,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":6.3479999999999999,"5min":62.722999999999999,"15min":62.722999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.74,"5min":1.972,"15min":1.2569999999999999},"min":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":14.464,"5min":14.694000000000001,"15min":14.694000000000001},"last":1.133},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.3320000000000001,"15min":1.069},"min":{"1min":0.29699999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.698,"5min":10.036,"15min":10.036},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.145,"5min":1.9450000000000001,"15min":1.2689999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":10.769,"5min":21.940999999999999,"15min":21.940999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.889,"5min":1.996,"15min":1.3620000000000001},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":9.8200000000000003,"5min":9.8200000000000003,"15min":9.8200000000000003},"last":1.1759999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":1.4410000000000001,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.819000000000001,"5min":10.819000000000001,"15min":10.819000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.5900000000000001,"15min":1.2130000000000001},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":10.741,"5min":10.741,"15min":10.741},"last":1.054}]},{"osd":7,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":3.0329999999999999,"5min":1.9139999999999999,"15min":1.3600000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":32.975999999999999,"5min":32.975999999999999,"15min":32.975999999999999},"last":1.21},{"interface":"front","average":{"1min":2.468,"5min":2.7930000000000001,"15min":1.6830000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.40300000000000002},"max":{"1min":17.484000000000002,"5min":32.069000000000003,"15min":32.069000000000003},"last":1.3089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":84646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8482816,"data_stored":8473926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7020544,"data_stored":7006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7016448,"data_stored":7007322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10542867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4296704,"data_stored":4287702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:18:30.796 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.796 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.796 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.796 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.796 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.797 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.797 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.797 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.797 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.797 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.797 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.797 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.797 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.798 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.798 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.798 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.798 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.798 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.798 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.798 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.798 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.798 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.799 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.799 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.799 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.799 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.799 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.799 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.799 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.799 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.800 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.800 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.800 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.800 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.800 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.800 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.800 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.801 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.801 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.801 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.801 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.801 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.801 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T16:47:47.706385+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=47, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.801 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.801 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.802 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.802 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.802 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.802 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.802 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.802 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.802 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.802 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.803 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.803 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.803 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.803 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.803 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.803 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.803 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.803 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.804 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.804 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.804 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.804 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.804 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.804 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.804 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.804 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.805 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.805 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.805 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.805 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.805 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.805 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.805 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.805 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.806 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.806 INFO:tasks.ceph:pgid 5.24 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.806 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.806 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.806 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.806 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.806 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.806 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.807 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.807 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.807 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.807 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.807 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.807 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.807 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.807 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.808 INFO:tasks.ceph:pgid 5.2c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.808 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.808 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.808 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.808 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.808 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.808 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.808 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.809 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.809 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.809 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.809 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.809 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.809 INFO:tasks.ceph:pgid 5.33 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.809 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.809 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.810 INFO:tasks.ceph:pgid 5.3f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.810 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.810 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.810 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.810 INFO:tasks.ceph:pgid 5.3b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.810 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.810 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.810 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.811 INFO:tasks.ceph:pgid 5.39 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:30.811 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T17:18:36.893 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:18:36.896 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:18:50.811 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:18:50.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.944+0000 7f9be4ca2640 1 Processor -- start 2024-09-13T17:18:50.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.944+0000 7f9be4ca2640 1 -- start start 2024-09-13T17:18:50.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9be4ca2640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9be006c940 0x7f9be00c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:50.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9be4ca2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01555f0 0x7f9be0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:50.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9be4ca2640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be014f790 con 0x7f9be01526b0 2024-09-13T17:18:50.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9be4ca2640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be0152a90 con 0x7f9be006c940 2024-09-13T17:18:50.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9be4ca2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be006ce40 con 0x7f9be01555f0 2024-09-13T17:18:50.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bded9d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01555f0 0x7f9be0155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:50.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bddd9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9be006c940 0x7f9be00c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:50.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bded9d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01555f0 0x7f9be0155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:38774/0 (socket says 172.21.15.39:38774) 2024-09-13T17:18:50.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bddd9b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9be006c940 0x7f9be00c12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.39:59170/0 (socket says 172.21.15.39:59170) 2024-09-13T17:18:50.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bded9d640 1 -- 172.21.15.39:0/1761112071 learned_addr learned my addr 172.21.15.39:0/1761112071 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:18:50.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bde59c640 1 --1- 172.21.15.39:0/1761112071 >> v1:172.21.15.84:6789/0 conn(0x7f9be01526b0 0x7f9be00aabc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:59472/0 (socket says 172.21.15.39:59472) 2024-09-13T17:18:50.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bded9d640 1 -- 172.21.15.39:0/1761112071 >> v2:172.21.15.84:3300/0 conn(0x7f9be006c940 msgr2=0x7f9be00c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:18:50.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bded9d640 1 --2- 172.21.15.39:0/1761112071 >> v2:172.21.15.84:3300/0 conn(0x7f9be006c940 0x7f9be00c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:50.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bded9d640 1 -- 172.21.15.39:0/1761112071 >> v1:172.21.15.84:6789/0 conn(0x7f9be01526b0 legacy=0x7f9be00aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:50.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bded9d640 1 -- 172.21.15.39:0/1761112071 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be006d120 con 0x7f9be01555f0 2024-09-13T17:18:50.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.948+0000 7f9bddd9b640 1 --2- 172.21.15.39:0/1761112071 >> v2:172.21.15.84:3300/0 conn(0x7f9be006c940 0x7f9be00c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:18:50.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9bded9d640 1 --2- 172.21.15.39:0/1761112071 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01555f0 0x7f9be0155a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009e50 tx=0x7f9bd801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f51af75da667be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:50.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9bdd59a640 1 -- 172.21.15.39:0/1761112071 <== mon.2 v2:172.21.15.84:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a57f0 con 0x7f9be01555f0 2024-09-13T17:18:50.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9bdd59a640 1 -- 172.21.15.39:0/1761112071 <== mon.2 v2:172.21.15.84:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd80a5950 con 0x7f9be01555f0 2024-09-13T17:18:50.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9bdd59a640 1 -- 172.21.15.39:0/1761112071 <== mon.2 v2:172.21.15.84:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a5ce0 con 0x7f9be01555f0 2024-09-13T17:18:50.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 -- 172.21.15.39:0/1761112071 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01555f0 msgr2=0x7f9be0155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:50.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 --2- 172.21.15.39:0/1761112071 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01555f0 0x7f9be0155a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009e50 tx=0x7f9bd801ca30 comp rx=0 tx=0).stop 2024-09-13T17:18:50.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 -- 172.21.15.39:0/1761112071 shutdown_connections 2024-09-13T17:18:50.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 --2- 172.21.15.39:0/1761112071 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01555f0 0x7f9be0155a60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:50.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 --2- 172.21.15.39:0/1761112071 >> v2:172.21.15.84:3300/0 conn(0x7f9be006c940 0x7f9be00c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:50.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 -- 172.21.15.39:0/1761112071 >> 172.21.15.39:0/1761112071 conn(0x7f9be0069e10 msgr2=0x7f9be006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:50.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 -- 172.21.15.39:0/1761112071 shutdown_connections 2024-09-13T17:18:50.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 -- 172.21.15.39:0/1761112071 wait complete. 2024-09-13T17:18:50.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 Processor -- start 2024-09-13T17:18:50.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.952+0000 7f9be4ca2640 1 -- start start 2024-09-13T17:18:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9be4ca2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be006c940 0x7f9be0149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9be4ca2640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01526b0 0x7f9be01462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9be4ca2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9be01555f0 0x7f9be0146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9be4ca2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be014f500 con 0x7f9be006c940 2024-09-13T17:18:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9be4ca2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00a62e0 con 0x7f9be01555f0 2024-09-13T17:18:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9be4ca2640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00a5e50 con 0x7f9be01526b0 2024-09-13T17:18:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be006c940 0x7f9be0149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bde59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be006c940 0x7f9be0149330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33034/0 (socket says 172.21.15.39:33034) 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bde59c640 1 -- 172.21.15.39:0/587259493 learned_addr learned my addr 172.21.15.39:0/587259493 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bded9d640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9be01555f0 0x7f9be0146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bddd9b640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01526b0 0x7f9be01462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bde59c640 1 -- 172.21.15.39:0/587259493 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01526b0 msgr2=0x7f9be01462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bde59c640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01526b0 0x7f9be01462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bde59c640 1 -- 172.21.15.39:0/587259493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9be01555f0 msgr2=0x7f9be0146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bde59c640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9be01555f0 0x7f9be0146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bde59c640 1 -- 172.21.15.39:0/587259493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4009420 con 0x7f9be006c940 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bde59c640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be006c940 0x7f9be0149330 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000c490 tx=0x7f9bd000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bceffd640 1 -- 172.21.15.39:0/587259493 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0007480 con 0x7f9be006c940 2024-09-13T17:18:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bceffd640 1 -- 172.21.15.39:0/587259493 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd0007600 con 0x7f9be006c940 2024-09-13T17:18:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bceffd640 1 -- 172.21.15.39:0/587259493 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0004a30 con 0x7f9be006c940 2024-09-13T17:18:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd8009b30 con 0x7f9be006c940 2024-09-13T17:18:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be014c380 con 0x7f9be006c940 2024-09-13T17:18:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be0151ac0 con 0x7f9be006c940 2024-09-13T17:18:50.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bceffd640 1 -- 172.21.15.39:0/587259493 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9bd0004b90 con 0x7f9be006c940 2024-09-13T17:18:50.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bceffd640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9bc0066690 0x7f9bc0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:18:50.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bceffd640 1 -- 172.21.15.39:0/587259493 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f9bd0084350 con 0x7f9be006c940 2024-09-13T17:18:50.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.956+0000 7f9bddd9b640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9bc0066690 0x7f9bc0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:18:50.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.960+0000 7f9bddd9b640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9bc0066690 0x7f9bc0068b50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9be015b760 tx=0x7f9bd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:18:50.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:50.960+0000 7f9bceffd640 1 -- 172.21.15.39:0/587259493 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9be0151ac0 con 0x7f9be006c940 2024-09-13T17:18:51.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.128+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9be0155fa0 con 0x7f9bc0066690 2024-09-13T17:18:51.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.144+0000 7f9bceffd640 1 -- 172.21.15.39:0/587259493 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528572 (secure 0 0 0) 0x7f9bd4011e60 con 0x7f9bc0066690 2024-09-13T17:18:51.152 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:18:51.156 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T17:18:51.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9bc0066690 msgr2=0x7f9bc0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:51.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9bc0066690 0x7f9bc0068b50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9be015b760 tx=0x7f9bd4008040 comp rx=0 tx=0).stop 2024-09-13T17:18:51.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be006c940 msgr2=0x7f9be0149330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:18:51.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be006c940 0x7f9be0149330 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000c490 tx=0x7f9bd000c950 comp rx=0 tx=0).stop 2024-09-13T17:18:51.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 shutdown_connections 2024-09-13T17:18:51.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9bc0066690 0x7f9bc0068b50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:51.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9be01555f0 0x7f9be0146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:51.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9be01526b0 0x7f9be01462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:51.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 --2- 172.21.15.39:0/587259493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be006c940 0x7f9be0149330 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:18:51.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 >> 172.21.15.39:0/587259493 conn(0x7f9be0069e10 msgr2=0x7f9be00c1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:18:51.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 shutdown_connections 2024-09-13T17:18:51.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:18:51.148+0000 7f9be4ca2640 1 -- 172.21.15.39:0/587259493 wait complete. 2024-09-13T17:18:51.171 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":988,"stamp":"2024-09-13T17:18:50.378463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39291023,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":3153,"num_read_kb":416667,"num_write":2818784,"num_write_kb":37531913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522563,"ondisk_log_size":522563,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6804448,"kb_used_data":83360,"kb_used_omap":12,"kb_used_meta":6721011,"kb_avail":748170272,"statfs":{"total":773094113280,"available":766126358528,"internally_reserved":0,"allocated":85360640,"data_stored":82126150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":6882315835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006558"},"pg_stats":[{"pgid":"5.18","version":"63'4385","reported_seq":4449,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974458+0000","last_change":"2024-09-13T16:48:08.185978+0000","last_active":"2024-09-13T17:17:44.974458+0000","last_peered":"2024-09-13T17:17:44.974458+0000","last_clean":"2024-09-13T17:17:44.974458+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T17:17:44.974458+0000","last_undegraded":"2024-09-13T17:17:44.974458+0000","last_fullsized":"2024-09-13T17:17:44.974458+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":23692,"num_write_kb":324330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"64'919","reported_seq":1044,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.775073+0000","last_change":"2024-09-13T17:18:10.775401+0000","last_active":"2024-09-13T17:18:10.775073+0000","last_peered":"2024-09-13T17:18:10.775073+0000","last_clean":"2024-09-13T17:18:10.775073+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T17:18:10.775073+0000","last_undegraded":"2024-09-13T17:18:10.775073+0000","last_fullsized":"2024-09-13T17:18:10.775073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'919","last_scrub_stamp":"2024-09-13T17:18:10.775003+0000","last_deep_scrub":"64'919","last_deep_scrub_stamp":"2024-09-13T17:18:10.775003+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.775003+0000","objects_scrubbed":3,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:00.271443+0000","scrub_duration":0.030271704,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":10389,"num_write":894,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'3737","reported_seq":3807,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631607+0000","last_change":"2024-09-13T16:47:59.059823+0000","last_active":"2024-09-13T17:17:47.631607+0000","last_peered":"2024-09-13T17:17:47.631607+0000","last_clean":"2024-09-13T17:17:47.631607+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T17:17:47.631607+0000","last_undegraded":"2024-09-13T17:17:47.631607+0000","last_fullsized":"2024-09-13T17:17:47.631607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3737,"log_dups_size":0,"ondisk_log_size":3737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20454,"num_write_kb":270405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.082725+0000","last_change":"2024-09-13T17:17:52.082969+0000","last_active":"2024-09-13T17:17:52.082725+0000","last_peered":"2024-09-13T17:17:52.082725+0000","last_clean":"2024-09-13T17:17:52.082725+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T17:17:52.082725+0000","last_undegraded":"2024-09-13T17:17:52.082725+0000","last_fullsized":"2024-09-13T17:17:52.082725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.082672+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.023294+0000","scrub_duration":0.0034166320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"63'4331","reported_seq":4396,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992385+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T17:17:44.992385+0000","last_peered":"2024-09-13T17:17:44.992385+0000","last_clean":"2024-09-13T17:17:44.992385+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T17:17:44.992385+0000","last_undegraded":"2024-09-13T17:17:44.992385+0000","last_fullsized":"2024-09-13T17:17:44.992385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":22875,"num_write_kb":325320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"64'24","reported_seq":95,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631458+0000","last_change":"2024-09-13T16:47:58.661277+0000","last_active":"2024-09-13T17:17:47.631458+0000","last_peered":"2024-09-13T17:17:47.631458+0000","last_clean":"2024-09-13T17:17:47.631458+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T17:17:47.631458+0000","last_undegraded":"2024-09-13T17:17:47.631458+0000","last_fullsized":"2024-09-13T17:17:47.631458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"63'5559","reported_seq":5639,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:55.943224+0000","last_change":"2024-09-13T17:17:55.943470+0000","last_active":"2024-09-13T17:17:55.943224+0000","last_peered":"2024-09-13T17:17:55.943224+0000","last_clean":"2024-09-13T17:17:55.943224+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T17:17:55.943224+0000","last_undegraded":"2024-09-13T17:17:55.943224+0000","last_fullsized":"2024-09-13T17:17:55.943224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5559","last_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_deep_scrub":"63'5559","last_deep_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_clean_scrub_stamp":"2024-09-13T17:17:55.943168+0000","objects_scrubbed":0,"log_size":5559,"log_dups_size":0,"ondisk_log_size":5559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:10.227939+0000","scrub_duration":0.037150519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30370,"num_write_kb":379865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"63'2","reported_seq":77,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.149583+0000","last_change":"2024-09-13T17:18:46.149840+0000","last_active":"2024-09-13T17:18:46.149583+0000","last_peered":"2024-09-13T17:18:46.149583+0000","last_clean":"2024-09-13T17:18:46.149583+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T17:18:46.149583+0000","last_undegraded":"2024-09-13T17:18:46.149583+0000","last_fullsized":"2024-09-13T17:18:46.149583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:46.149528+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:46.149528+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.149528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:41.450773+0000","scrub_duration":0.003414533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"63'3581","reported_seq":3644,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591311+0000","last_change":"2024-09-13T16:48:08.205778+0000","last_active":"2024-09-13T17:17:49.591311+0000","last_peered":"2024-09-13T17:17:49.591311+0000","last_clean":"2024-09-13T17:17:49.591311+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T17:17:49.591311+0000","last_undegraded":"2024-09-13T17:17:49.591311+0000","last_fullsized":"2024-09-13T17:17:49.591311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":268344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"64'43","reported_seq":149,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591097+0000","last_change":"2024-09-13T16:47:58.692866+0000","last_active":"2024-09-13T17:17:49.591097+0000","last_peered":"2024-09-13T17:17:49.591097+0000","last_clean":"2024-09-13T17:17:49.591097+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T17:17:49.591097+0000","last_undegraded":"2024-09-13T17:17:49.591097+0000","last_fullsized":"2024-09-13T17:17:49.591097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'3493","reported_seq":3562,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.642622+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T17:17:48.642622+0000","last_peered":"2024-09-13T17:17:48.642622+0000","last_clean":"2024-09-13T17:17:48.642622+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T17:17:48.642622+0000","last_undegraded":"2024-09-13T17:17:48.642622+0000","last_fullsized":"2024-09-13T17:17:48.642622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18921,"num_write_kb":256078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'727","reported_seq":827,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.186917+0000","last_change":"2024-09-13T17:17:57.186917+0000","last_active":"2024-09-13T17:17:57.186917+0000","last_peered":"2024-09-13T17:17:57.186917+0000","last_clean":"2024-09-13T17:17:57.186917+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T17:17:57.186917+0000","last_undegraded":"2024-09-13T17:17:57.186917+0000","last_fullsized":"2024-09-13T17:17:57.186917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'727","last_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_deep_scrub":"64'727","last_deep_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.186269+0000","objects_scrubbed":1,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:00.928968+0000","scrub_duration":0.026043038000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":75389,"num_write":714,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"63'3135","reported_seq":3209,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:42.600862+0000","last_change":"2024-09-13T17:18:42.601105+0000","last_active":"2024-09-13T17:18:42.600862+0000","last_peered":"2024-09-13T17:18:42.600862+0000","last_clean":"2024-09-13T17:18:42.600862+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T17:18:42.600862+0000","last_undegraded":"2024-09-13T17:18:42.600862+0000","last_fullsized":"2024-09-13T17:18:42.600862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3135","last_scrub_stamp":"2024-09-13T17:18:42.600803+0000","last_deep_scrub":"63'3135","last_deep_scrub_stamp":"2024-09-13T17:18:42.600803+0000","last_clean_scrub_stamp":"2024-09-13T17:18:42.600803+0000","objects_scrubbed":0,"log_size":3135,"log_dups_size":0,"ondisk_log_size":3135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:53.145621+0000","scrub_duration":0.022581403999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":232447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"63'5","reported_seq":111,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.590779+0000","last_change":"2024-09-13T16:47:58.684292+0000","last_active":"2024-09-13T17:17:49.590779+0000","last_peered":"2024-09-13T17:17:49.590779+0000","last_clean":"2024-09-13T17:17:49.590779+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T17:17:49.590779+0000","last_undegraded":"2024-09-13T17:17:49.590779+0000","last_fullsized":"2024-09-13T17:17:49.590779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"63'5195","reported_seq":5273,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.511870+0000","last_change":"2024-09-13T17:17:49.511870+0000","last_active":"2024-09-13T17:17:49.511870+0000","last_peered":"2024-09-13T17:17:49.511870+0000","last_clean":"2024-09-13T17:17:49.511870+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T17:17:49.511870+0000","last_undegraded":"2024-09-13T17:17:49.511870+0000","last_fullsized":"2024-09-13T17:17:49.511870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5195","last_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_deep_scrub":"63'5195","last_deep_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.511449+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:30.737466+0000","scrub_duration":0.041661353999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29369,"num_write_kb":359470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"64'211","reported_seq":286,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.156875+0000","last_change":"2024-09-13T17:18:44.157099+0000","last_active":"2024-09-13T17:18:44.156875+0000","last_peered":"2024-09-13T17:18:44.156875+0000","last_clean":"2024-09-13T17:18:44.156875+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T17:18:44.156875+0000","last_undegraded":"2024-09-13T17:18:44.156875+0000","last_fullsized":"2024-09-13T17:18:44.156875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'211","last_scrub_stamp":"2024-09-13T17:18:44.156820+0000","last_deep_scrub":"64'211","last_deep_scrub_stamp":"2024-09-13T17:18:44.156820+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.156820+0000","objects_scrubbed":1,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:21.506931+0000","scrub_duration":0.018253704999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153565,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"63'3570","reported_seq":3636,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996947+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T17:17:44.996947+0000","last_peered":"2024-09-13T17:17:44.996947+0000","last_clean":"2024-09-13T17:17:44.996947+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T17:17:44.996947+0000","last_undegraded":"2024-09-13T17:17:44.996947+0000","last_fullsized":"2024-09-13T17:17:44.996947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":19362,"num_write_kb":282506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"63'4274","reported_seq":4354,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.521451+0000","last_change":"2024-09-13T17:18:01.521451+0000","last_active":"2024-09-13T17:18:01.521451+0000","last_peered":"2024-09-13T17:18:01.521451+0000","last_clean":"2024-09-13T17:18:01.521451+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T17:18:01.521451+0000","last_undegraded":"2024-09-13T17:18:01.521451+0000","last_fullsized":"2024-09-13T17:18:01.521451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4274","last_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_deep_scrub":"63'4274","last_deep_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.521104+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:51.017545+0000","scrub_duration":0.027199207999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23751,"num_write_kb":312338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"64'1421","reported_seq":1546,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.346755+0000","last_change":"2024-09-13T17:18:46.347010+0000","last_active":"2024-09-13T17:18:46.346755+0000","last_peered":"2024-09-13T17:18:46.346755+0000","last_clean":"2024-09-13T17:18:46.346755+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T17:18:46.346755+0000","last_undegraded":"2024-09-13T17:18:46.346755+0000","last_fullsized":"2024-09-13T17:18:46.346755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1421","last_scrub_stamp":"2024-09-13T17:18:46.346701+0000","last_deep_scrub":"64'1421","last_deep_scrub_stamp":"2024-09-13T17:18:46.346701+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.346701+0000","objects_scrubbed":2,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:47.745552+0000","scrub_duration":0.031375147999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":42245,"num_write":1396,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'7","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.140201+0000","last_change":"2024-09-13T17:18:01.140415+0000","last_active":"2024-09-13T17:18:01.140201+0000","last_peered":"2024-09-13T17:18:01.140201+0000","last_clean":"2024-09-13T17:18:01.140201+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T17:18:01.140201+0000","last_undegraded":"2024-09-13T17:18:01.140201+0000","last_fullsized":"2024-09-13T17:18:01.140201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'7","last_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_deep_scrub":"64'7","last_deep_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.140142+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:50.636583+0000","scrub_duration":0.0071129380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"63'4212","reported_seq":4290,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:32.577261+0000","last_change":"2024-09-13T17:18:32.577494+0000","last_active":"2024-09-13T17:18:32.577261+0000","last_peered":"2024-09-13T17:18:32.577261+0000","last_clean":"2024-09-13T17:18:32.577261+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T17:18:32.577261+0000","last_undegraded":"2024-09-13T17:18:32.577261+0000","last_fullsized":"2024-09-13T17:18:32.577261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4212","last_scrub_stamp":"2024-09-13T17:18:32.577202+0000","last_deep_scrub":"63'4212","last_deep_scrub_stamp":"2024-09-13T17:18:32.577202+0000","last_clean_scrub_stamp":"2024-09-13T17:18:32.577202+0000","objects_scrubbed":0,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:31.849084+0000","scrub_duration":0.024339607999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":17,"num_write":23343,"num_write_kb":314959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"63'3296","reported_seq":3364,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975110+0000","last_change":"2024-09-13T16:47:59.060081+0000","last_active":"2024-09-13T17:17:44.975110+0000","last_peered":"2024-09-13T17:17:44.975110+0000","last_clean":"2024-09-13T17:17:44.975110+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T17:17:44.975110+0000","last_undegraded":"2024-09-13T17:17:44.975110+0000","last_fullsized":"2024-09-13T17:17:44.975110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3296,"log_dups_size":0,"ondisk_log_size":3296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17758,"num_write_kb":230990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'3","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.272530+0000","last_change":"2024-09-13T17:18:44.272747+0000","last_active":"2024-09-13T17:18:44.272530+0000","last_peered":"2024-09-13T17:18:44.272530+0000","last_clean":"2024-09-13T17:18:44.272530+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T17:18:44.272530+0000","last_undegraded":"2024-09-13T17:18:44.272530+0000","last_fullsized":"2024-09-13T17:18:44.272530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:44.272476+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:44.272476+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.272476+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:21.622587+0000","scrub_duration":0.0033040719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.139202+0000","last_change":"2024-09-13T17:18:05.139417+0000","last_active":"2024-09-13T17:18:05.139202+0000","last_peered":"2024-09-13T17:18:05.139202+0000","last_clean":"2024-09-13T17:18:05.139202+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T17:18:05.139202+0000","last_undegraded":"2024-09-13T17:18:05.139202+0000","last_fullsized":"2024-09-13T17:18:05.139202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:05.139147+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:05.139147+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.139147+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:15.079769+0000","scrub_duration":0.0035900110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"63'3371","reported_seq":3436,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980902+0000","last_change":"2024-09-13T16:48:08.203642+0000","last_active":"2024-09-13T17:17:44.980902+0000","last_peered":"2024-09-13T17:17:44.980902+0000","last_clean":"2024-09-13T17:17:44.980902+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T17:17:44.980902+0000","last_undegraded":"2024-09-13T17:17:44.980902+0000","last_fullsized":"2024-09-13T17:17:44.980902+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3371,"log_dups_size":0,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18156,"num_write_kb":249012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"63'3752","reported_seq":3821,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631388+0000","last_change":"2024-09-13T16:47:59.055081+0000","last_active":"2024-09-13T17:17:47.631388+0000","last_peered":"2024-09-13T17:17:47.631388+0000","last_clean":"2024-09-13T17:17:47.631388+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T17:17:47.631388+0000","last_undegraded":"2024-09-13T17:17:47.631388+0000","last_fullsized":"2024-09-13T17:17:47.631388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21127,"num_write_kb":268998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"63'2","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.674520+0000","last_change":"2024-09-13T16:47:58.666822+0000","last_active":"2024-09-13T17:17:46.674520+0000","last_peered":"2024-09-13T17:17:46.674520+0000","last_clean":"2024-09-13T17:17:46.674520+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T17:17:46.674520+0000","last_undegraded":"2024-09-13T17:17:46.674520+0000","last_fullsized":"2024-09-13T17:17:46.674520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"64'5","reported_seq":102,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:11.257475+0000","last_change":"2024-09-13T17:18:11.257689+0000","last_active":"2024-09-13T17:18:11.257475+0000","last_peered":"2024-09-13T17:18:11.257475+0000","last_clean":"2024-09-13T17:18:11.257475+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T17:18:11.257475+0000","last_undegraded":"2024-09-13T17:18:11.257475+0000","last_fullsized":"2024-09-13T17:18:11.257475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:11.257421+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:11.257421+0000","last_clean_scrub_stamp":"2024-09-13T17:18:11.257421+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:03.763129+0000","scrub_duration":0.0063405759999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"63'4499","reported_seq":4567,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.987188+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T17:17:44.987188+0000","last_peered":"2024-09-13T17:17:44.987188+0000","last_clean":"2024-09-13T17:17:44.987188+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T17:17:44.987188+0000","last_undegraded":"2024-09-13T17:17:44.987188+0000","last_fullsized":"2024-09-13T17:17:44.987188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":24341,"num_write_kb":343418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"63'4043","reported_seq":4123,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:31.227364+0000","last_change":"2024-09-13T17:18:31.227589+0000","last_active":"2024-09-13T17:18:31.227364+0000","last_peered":"2024-09-13T17:18:31.227364+0000","last_clean":"2024-09-13T17:18:31.227364+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T17:18:31.227364+0000","last_undegraded":"2024-09-13T17:18:31.227364+0000","last_fullsized":"2024-09-13T17:18:31.227364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4043","last_scrub_stamp":"2024-09-13T17:18:31.227305+0000","last_deep_scrub":"63'4043","last_deep_scrub_stamp":"2024-09-13T17:18:31.227305+0000","last_clean_scrub_stamp":"2024-09-13T17:18:31.227305+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:30.499187+0000","scrub_duration":0.032759703000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22005,"num_write_kb":288910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"63'5","reported_seq":75,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.590791+0000","last_change":"2024-09-13T16:47:58.685003+0000","last_active":"2024-09-13T17:17:49.590791+0000","last_peered":"2024-09-13T17:17:49.590791+0000","last_clean":"2024-09-13T17:17:49.590791+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T17:17:49.590791+0000","last_undegraded":"2024-09-13T17:17:49.590791+0000","last_fullsized":"2024-09-13T17:17:49.590791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.171804+0000","last_change":"2024-09-13T17:18:04.172104+0000","last_active":"2024-09-13T17:18:04.171804+0000","last_peered":"2024-09-13T17:18:04.171804+0000","last_clean":"2024-09-13T17:18:04.171804+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T17:18:04.171804+0000","last_undegraded":"2024-09-13T17:18:04.171804+0000","last_fullsized":"2024-09-13T17:18:04.171804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.171747+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:58.830350+0000","scrub_duration":0.0065161840000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"63'3470","reported_seq":3547,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:30.976569+0000","last_change":"2024-09-13T17:18:30.976875+0000","last_active":"2024-09-13T17:18:30.976569+0000","last_peered":"2024-09-13T17:18:30.976569+0000","last_clean":"2024-09-13T17:18:30.976569+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T17:18:30.976569+0000","last_undegraded":"2024-09-13T17:18:30.976569+0000","last_fullsized":"2024-09-13T17:18:30.976569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3470","last_scrub_stamp":"2024-09-13T17:18:30.976507+0000","last_deep_scrub":"63'3470","last_deep_scrub_stamp":"2024-09-13T17:18:30.976507+0000","last_clean_scrub_stamp":"2024-09-13T17:18:30.976507+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:08.326618+0000","scrub_duration":0.024724408999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19441,"num_write_kb":258288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'4844","reported_seq":4925,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:18.899218+0000","last_change":"2024-09-13T17:18:18.899218+0000","last_active":"2024-09-13T17:18:18.899218+0000","last_peered":"2024-09-13T17:18:18.899218+0000","last_clean":"2024-09-13T17:18:18.899218+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T17:18:18.899218+0000","last_undegraded":"2024-09-13T17:18:18.899218+0000","last_fullsized":"2024-09-13T17:18:18.899218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4844","last_scrub_stamp":"2024-09-13T17:18:18.898914+0000","last_deep_scrub":"63'4844","last_deep_scrub_stamp":"2024-09-13T17:18:18.898914+0000","last_clean_scrub_stamp":"2024-09-13T17:18:18.898914+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:18.212032+0000","scrub_duration":0.026998708999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":26635,"num_write_kb":336207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'10","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.498454+0000","last_change":"2024-09-13T17:18:05.498454+0000","last_active":"2024-09-13T17:18:05.498454+0000","last_peered":"2024-09-13T17:18:05.498454+0000","last_clean":"2024-09-13T17:18:05.498454+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T17:18:05.498454+0000","last_undegraded":"2024-09-13T17:18:05.498454+0000","last_fullsized":"2024-09-13T17:18:05.498454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_deep_scrub":"64'10","last_deep_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.498184+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:15.438806+0000","scrub_duration":0.0057060160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'49","reported_seq":189,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.166590+0000","last_change":"2024-09-13T17:17:58.166876+0000","last_active":"2024-09-13T17:17:58.166590+0000","last_peered":"2024-09-13T17:17:58.166590+0000","last_clean":"2024-09-13T17:17:58.166590+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T17:17:58.166590+0000","last_undegraded":"2024-09-13T17:17:58.166590+0000","last_fullsized":"2024-09-13T17:17:58.166590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'49","last_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_deep_scrub":"64'49","last_deep_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.166530+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:05.921781+0000","scrub_duration":0.0073340999999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92726,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":118,"num_write":43,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"63'3635","reported_seq":3701,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974442+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T17:17:44.974442+0000","last_peered":"2024-09-13T17:17:44.974442+0000","last_clean":"2024-09-13T17:17:44.974442+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T17:17:44.974442+0000","last_undegraded":"2024-09-13T17:17:44.974442+0000","last_fullsized":"2024-09-13T17:17:44.974442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19787,"num_write_kb":259620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'3639","reported_seq":3708,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631524+0000","last_change":"2024-09-13T16:47:59.063230+0000","last_active":"2024-09-13T17:17:47.631524+0000","last_peered":"2024-09-13T17:17:47.631524+0000","last_clean":"2024-09-13T17:17:47.631524+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T17:17:47.631524+0000","last_undegraded":"2024-09-13T17:17:47.631524+0000","last_fullsized":"2024-09-13T17:17:47.631524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3639,"log_dups_size":0,"ondisk_log_size":3639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20066,"num_write_kb":240267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"64'267","reported_seq":356,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.589241+0000","last_change":"2024-09-13T16:47:58.682399+0000","last_active":"2024-09-13T17:17:50.589241+0000","last_peered":"2024-09-13T17:17:50.589241+0000","last_clean":"2024-09-13T17:17:50.589241+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T17:17:50.589241+0000","last_undegraded":"2024-09-13T17:17:50.589241+0000","last_fullsized":"2024-09-13T17:17:50.589241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":38,"num_read_kb":48,"num_write":636,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"63'11","reported_seq":89,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.102762+0000","last_change":"2024-09-13T17:17:50.102762+0000","last_active":"2024-09-13T17:17:50.102762+0000","last_peered":"2024-09-13T17:17:50.102762+0000","last_clean":"2024-09-13T17:17:50.102762+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T17:17:50.102762+0000","last_undegraded":"2024-09-13T17:17:50.102762+0000","last_fullsized":"2024-09-13T17:17:50.102762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_deep_scrub":"63'11","last_deep_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.102616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.736833+0000","scrub_duration":0.0039927280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"63'3190","reported_seq":3255,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993610+0000","last_change":"2024-09-13T16:48:08.169871+0000","last_active":"2024-09-13T17:17:44.993610+0000","last_peered":"2024-09-13T17:17:44.993610+0000","last_clean":"2024-09-13T17:17:44.993610+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T17:17:44.993610+0000","last_undegraded":"2024-09-13T17:17:44.993610+0000","last_fullsized":"2024-09-13T17:17:44.993610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3190,"log_dups_size":0,"ondisk_log_size":3190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":254866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"63'4701","reported_seq":4770,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.998464+0000","last_change":"2024-09-13T16:47:59.038265+0000","last_active":"2024-09-13T17:17:44.998464+0000","last_peered":"2024-09-13T17:17:44.998464+0000","last_clean":"2024-09-13T17:17:44.998464+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T17:17:44.998464+0000","last_undegraded":"2024-09-13T17:17:44.998464+0000","last_fullsized":"2024-09-13T17:17:44.998464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26171,"num_write_kb":338588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'2","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996573+0000","last_change":"2024-09-13T16:47:58.685957+0000","last_active":"2024-09-13T17:17:44.996573+0000","last_peered":"2024-09-13T17:17:44.996573+0000","last_clean":"2024-09-13T17:17:44.996573+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T17:17:44.996573+0000","last_undegraded":"2024-09-13T17:17:44.996573+0000","last_fullsized":"2024-09-13T17:17:44.996573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"63'3","reported_seq":69,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.002286+0000","last_change":"2024-09-13T16:48:07.774834+0000","last_active":"2024-09-13T17:17:45.002286+0000","last_peered":"2024-09-13T17:17:45.002286+0000","last_clean":"2024-09-13T17:17:45.002286+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T17:17:45.002286+0000","last_undegraded":"2024-09-13T17:17:45.002286+0000","last_fullsized":"2024-09-13T17:17:45.002286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"63'3859","reported_seq":3924,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.003745+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T17:17:45.003745+0000","last_peered":"2024-09-13T17:17:45.003745+0000","last_clean":"2024-09-13T17:17:45.003745+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T17:17:45.003745+0000","last_undegraded":"2024-09-13T17:17:45.003745+0000","last_fullsized":"2024-09-13T17:17:45.003745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21286,"num_write_kb":290222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"63'3876","reported_seq":3956,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:31.476414+0000","last_change":"2024-09-13T17:18:31.476681+0000","last_active":"2024-09-13T17:18:31.476414+0000","last_peered":"2024-09-13T17:18:31.476414+0000","last_clean":"2024-09-13T17:18:31.476414+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T17:18:31.476414+0000","last_undegraded":"2024-09-13T17:18:31.476414+0000","last_fullsized":"2024-09-13T17:18:31.476414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3876","last_scrub_stamp":"2024-09-13T17:18:31.476360+0000","last_deep_scrub":"63'3876","last_deep_scrub_stamp":"2024-09-13T17:18:31.476360+0000","last_clean_scrub_stamp":"2024-09-13T17:18:31.476360+0000","objects_scrubbed":0,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:48.450054+0000","scrub_duration":0.030063065999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21201,"num_write_kb":273354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"63'2","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:42.323131+0000","last_change":"2024-09-13T17:18:42.323409+0000","last_active":"2024-09-13T17:18:42.323131+0000","last_peered":"2024-09-13T17:18:42.323131+0000","last_clean":"2024-09-13T17:18:42.323131+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T17:18:42.323131+0000","last_undegraded":"2024-09-13T17:18:42.323131+0000","last_fullsized":"2024-09-13T17:18:42.323131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:42.323076+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:42.323076+0000","last_clean_scrub_stamp":"2024-09-13T17:18:42.323076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:59.296770+0000","scrub_duration":0.003371626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'5","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.226738+0000","last_change":"2024-09-13T17:17:57.227606+0000","last_active":"2024-09-13T17:17:57.226738+0000","last_peered":"2024-09-13T17:17:57.226738+0000","last_clean":"2024-09-13T17:17:57.226738+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T17:17:57.226738+0000","last_undegraded":"2024-09-13T17:17:57.226738+0000","last_fullsized":"2024-09-13T17:17:57.226738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226662+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:46.723103+0000","scrub_duration":0.0036245560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"63'3422","reported_seq":3488,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993940+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T17:17:44.993940+0000","last_peered":"2024-09-13T17:17:44.993940+0000","last_clean":"2024-09-13T17:17:44.993940+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T17:17:44.993940+0000","last_undegraded":"2024-09-13T17:17:44.993940+0000","last_fullsized":"2024-09-13T17:17:44.993940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":7,"num_write":19042,"num_write_kb":260722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"63'4802","reported_seq":4882,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:12.260957+0000","last_change":"2024-09-13T17:18:12.261210+0000","last_active":"2024-09-13T17:18:12.260957+0000","last_peered":"2024-09-13T17:18:12.260957+0000","last_clean":"2024-09-13T17:18:12.260957+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T17:18:12.260957+0000","last_undegraded":"2024-09-13T17:18:12.260957+0000","last_fullsized":"2024-09-13T17:18:12.260957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4802","last_scrub_stamp":"2024-09-13T17:18:12.260903+0000","last_deep_scrub":"63'4802","last_deep_scrub_stamp":"2024-09-13T17:18:12.260903+0000","last_clean_scrub_stamp":"2024-09-13T17:18:12.260903+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:25.238396+0000","scrub_duration":0.040392404999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26419,"num_write_kb":335470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"63'4","reported_seq":74,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978957+0000","last_change":"2024-09-13T16:47:58.663800+0000","last_active":"2024-09-13T17:17:44.978957+0000","last_peered":"2024-09-13T17:17:44.978957+0000","last_clean":"2024-09-13T17:17:44.978957+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T17:17:44.978957+0000","last_undegraded":"2024-09-13T17:17:44.978957+0000","last_fullsized":"2024-09-13T17:17:44.978957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:54.181019+0000","last_change":"2024-09-13T17:17:54.181249+0000","last_active":"2024-09-13T17:17:54.181019+0000","last_peered":"2024-09-13T17:17:54.181019+0000","last_clean":"2024-09-13T17:17:54.181019+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T17:17:54.181019+0000","last_undegraded":"2024-09-13T17:17:54.181019+0000","last_fullsized":"2024-09-13T17:17:54.181019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_clean_scrub_stamp":"2024-09-13T17:17:54.180966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:37.317744+0000","scrub_duration":0.0037272709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"63'4215","reported_seq":4292,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:14.596560+0000","last_change":"2024-09-13T17:18:14.596560+0000","last_active":"2024-09-13T17:18:14.596560+0000","last_peered":"2024-09-13T17:18:14.596560+0000","last_clean":"2024-09-13T17:18:14.596560+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T17:18:14.596560+0000","last_undegraded":"2024-09-13T17:18:14.596560+0000","last_fullsized":"2024-09-13T17:18:14.596560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4215","last_scrub_stamp":"2024-09-13T17:18:14.596269+0000","last_deep_scrub":"63'4215","last_deep_scrub_stamp":"2024-09-13T17:18:14.596269+0000","last_clean_scrub_stamp":"2024-09-13T17:18:14.596269+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:07.101977+0000","scrub_duration":0.024198185000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":8,"num_write":22461,"num_write_kb":319331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'3790","reported_seq":3860,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.674564+0000","last_change":"2024-09-13T16:47:59.037136+0000","last_active":"2024-09-13T17:17:46.674564+0000","last_peered":"2024-09-13T17:17:46.674564+0000","last_clean":"2024-09-13T17:17:46.674564+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T17:17:46.674564+0000","last_undegraded":"2024-09-13T17:17:46.674564+0000","last_fullsized":"2024-09-13T17:17:46.674564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21077,"num_write_kb":260732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'42","reported_seq":179,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.602904+0000","last_change":"2024-09-13T17:18:03.603126+0000","last_active":"2024-09-13T17:18:03.602904+0000","last_peered":"2024-09-13T17:18:03.602904+0000","last_clean":"2024-09-13T17:18:03.602904+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T17:18:03.602904+0000","last_undegraded":"2024-09-13T17:18:03.602904+0000","last_fullsized":"2024-09-13T17:18:03.602904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'42","last_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_deep_scrub":"64'42","last_deep_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.602845+0000","objects_scrubbed":1,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:03.563076+0000","scrub_duration":0.004957576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84646,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":106,"num_write":37,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'10","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.102806+0000","last_change":"2024-09-13T17:18:05.102806+0000","last_active":"2024-09-13T17:18:05.102806+0000","last_peered":"2024-09-13T17:18:05.102806+0000","last_clean":"2024-09-13T17:18:05.102806+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T17:18:05.102806+0000","last_undegraded":"2024-09-13T17:18:05.102806+0000","last_fullsized":"2024-09-13T17:18:05.102806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-13T17:18:05.101908+0000","last_deep_scrub":"64'10","last_deep_scrub_stamp":"2024-09-13T17:18:05.101908+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.101908+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:04.415025+0000","scrub_duration":0.0065388449999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"63'3128","reported_seq":3193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.642943+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T17:17:48.642943+0000","last_peered":"2024-09-13T17:17:48.642943+0000","last_clean":"2024-09-13T17:17:48.642943+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T17:17:48.642943+0000","last_undegraded":"2024-09-13T17:17:48.642943+0000","last_fullsized":"2024-09-13T17:17:48.642943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3128,"log_dups_size":0,"ondisk_log_size":3128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17776,"num_write_kb":243283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"63'3657","reported_seq":3737,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:14.878066+0000","last_change":"2024-09-13T17:18:14.878283+0000","last_active":"2024-09-13T17:18:14.878066+0000","last_peered":"2024-09-13T17:18:14.878066+0000","last_clean":"2024-09-13T17:18:14.878066+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T17:18:14.878066+0000","last_undegraded":"2024-09-13T17:18:14.878066+0000","last_fullsized":"2024-09-13T17:18:14.878066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3657","last_scrub_stamp":"2024-09-13T17:18:14.878012+0000","last_deep_scrub":"63'3657","last_deep_scrub_stamp":"2024-09-13T17:18:14.878012+0000","last_clean_scrub_stamp":"2024-09-13T17:18:14.878012+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:17.512229+0000","scrub_duration":0.026162280999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19816,"num_write_kb":255512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'11","reported_seq":93,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.280947+0000","last_change":"2024-09-13T17:18:44.281154+0000","last_active":"2024-09-13T17:18:44.280947+0000","last_peered":"2024-09-13T17:18:44.280947+0000","last_clean":"2024-09-13T17:18:44.280947+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T17:18:44.280947+0000","last_undegraded":"2024-09-13T17:18:44.280947+0000","last_fullsized":"2024-09-13T17:18:44.280947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-13T17:18:44.280891+0000","last_deep_scrub":"64'11","last_deep_scrub_stamp":"2024-09-13T17:18:44.280891+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.280891+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:24.706976+0000","scrub_duration":0.0031866279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"63'251","reported_seq":338,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.207140+0000","last_change":"2024-09-13T16:48:07.779590+0000","last_active":"2024-09-13T17:17:46.207140+0000","last_peered":"2024-09-13T17:17:46.207140+0000","last_clean":"2024-09-13T17:17:46.207140+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T17:17:46.207140+0000","last_undegraded":"2024-09-13T17:17:46.207140+0000","last_fullsized":"2024-09-13T17:17:46.207140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":57,"num_write":597,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"63'4141","reported_seq":4207,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992340+0000","last_change":"2024-09-13T16:48:08.192995+0000","last_active":"2024-09-13T17:17:44.992340+0000","last_peered":"2024-09-13T17:17:44.992340+0000","last_clean":"2024-09-13T17:17:44.992340+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T17:17:44.992340+0000","last_undegraded":"2024-09-13T17:17:44.992340+0000","last_fullsized":"2024-09-13T17:17:44.992340+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22849,"num_write_kb":317008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'4103","reported_seq":4191,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:42.981381+0000","last_change":"2024-09-13T17:18:42.981633+0000","last_active":"2024-09-13T17:18:42.981381+0000","last_peered":"2024-09-13T17:18:42.981381+0000","last_clean":"2024-09-13T17:18:42.981381+0000","last_became_active":"2024-09-13T16:48:41.389507+0000","last_became_peered":"2024-09-13T16:48:41.389507+0000","last_unstale":"2024-09-13T17:18:42.981381+0000","last_undegraded":"2024-09-13T17:18:42.981381+0000","last_fullsized":"2024-09-13T17:18:42.981381+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4103","last_scrub_stamp":"2024-09-13T17:18:42.981318+0000","last_deep_scrub":"63'4103","last_deep_scrub_stamp":"2024-09-13T17:18:42.981318+0000","last_clean_scrub_stamp":"2024-09-13T17:18:42.981318+0000","objects_scrubbed":0,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:15.675900+0000","scrub_duration":0.025114515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22342,"num_write_kb":278194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"63'3","reported_seq":73,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.980918+0000","last_change":"2024-09-13T16:47:58.657082+0000","last_active":"2024-09-13T17:17:44.980918+0000","last_peered":"2024-09-13T17:17:44.980918+0000","last_clean":"2024-09-13T17:17:44.980918+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T17:17:44.980918+0000","last_undegraded":"2024-09-13T17:17:44.980918+0000","last_fullsized":"2024-09-13T17:17:44.980918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'119","reported_seq":237,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.251938+0000","last_change":"2024-09-13T17:18:43.252212+0000","last_active":"2024-09-13T17:18:43.251938+0000","last_peered":"2024-09-13T17:18:43.251938+0000","last_clean":"2024-09-13T17:18:43.251938+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T17:18:43.251938+0000","last_undegraded":"2024-09-13T17:18:43.251938+0000","last_fullsized":"2024-09-13T17:18:43.251938+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'119","last_scrub_stamp":"2024-09-13T17:18:43.251885+0000","last_deep_scrub":"64'119","last_deep_scrub_stamp":"2024-09-13T17:18:43.251885+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.251885+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:32.643640+0000","scrub_duration":0.010406291999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:40.136105+0000","last_change":"2024-09-13T17:18:40.136327+0000","last_active":"2024-09-13T17:18:40.136105+0000","last_peered":"2024-09-13T17:18:40.136105+0000","last_clean":"2024-09-13T17:18:40.136105+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T17:18:40.136105+0000","last_undegraded":"2024-09-13T17:18:40.136105+0000","last_fullsized":"2024-09-13T17:18:40.136105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:40.136049+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:40.136049+0000","last_clean_scrub_stamp":"2024-09-13T17:18:40.136049+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:52.925685+0000","scrub_duration":0.0033099650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"63'2904","reported_seq":2980,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.337844+0000","last_change":"2024-09-13T17:17:48.337931+0000","last_active":"2024-09-13T17:17:48.337844+0000","last_peered":"2024-09-13T17:17:48.337844+0000","last_clean":"2024-09-13T17:17:48.337844+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T17:17:48.337844+0000","last_undegraded":"2024-09-13T17:17:48.337844+0000","last_fullsized":"2024-09-13T17:17:48.337844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2904","last_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_deep_scrub":"63'2904","last_deep_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_clean_scrub_stamp":"2024-09-13T17:17:48.337825+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:15.694145+0000","scrub_duration":0.0051743259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":2,"num_write":15607,"num_write_kb":214254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'4284","reported_seq":4363,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.341991+0000","last_change":"2024-09-13T17:17:52.342307+0000","last_active":"2024-09-13T17:17:52.341991+0000","last_peered":"2024-09-13T17:17:52.341991+0000","last_clean":"2024-09-13T17:17:52.341991+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T17:17:52.341991+0000","last_undegraded":"2024-09-13T17:17:52.341991+0000","last_fullsized":"2024-09-13T17:17:52.341991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4284","last_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_deep_scrub":"63'4284","last_deep_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.341937+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.282559+0000","scrub_duration":0.043379468999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23392,"num_write_kb":310963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"63'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.502278+0000","last_change":"2024-09-13T17:17:53.502523+0000","last_active":"2024-09-13T17:17:53.502278+0000","last_peered":"2024-09-13T17:17:53.502278+0000","last_clean":"2024-09-13T17:17:53.502278+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T17:17:53.502278+0000","last_undegraded":"2024-09-13T17:17:53.502278+0000","last_fullsized":"2024-09-13T17:17:53.502278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.502221+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:42.741599+0000","scrub_duration":0.0040979670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"63'3","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:41.132756+0000","last_change":"2024-09-13T17:18:41.133001+0000","last_active":"2024-09-13T17:18:41.132756+0000","last_peered":"2024-09-13T17:18:41.132756+0000","last_clean":"2024-09-13T17:18:41.132756+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T17:18:41.132756+0000","last_undegraded":"2024-09-13T17:18:41.132756+0000","last_fullsized":"2024-09-13T17:18:41.132756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:41.132703+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:41.132703+0000","last_clean_scrub_stamp":"2024-09-13T17:18:41.132703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:58.106397+0000","scrub_duration":0.0037951880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"63'4138","reported_seq":4214,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.398680+0000","last_change":"2024-09-13T17:17:49.398680+0000","last_active":"2024-09-13T17:17:49.398680+0000","last_peered":"2024-09-13T17:17:49.398680+0000","last_clean":"2024-09-13T17:17:49.398680+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T17:17:49.398680+0000","last_undegraded":"2024-09-13T17:17:49.398680+0000","last_fullsized":"2024-09-13T17:17:49.398680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4138","last_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_deep_scrub":"63'4138","last_deep_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.398536+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:51.412144+0000","scrub_duration":0.020992730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":309086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"63'4584","reported_seq":4646,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.164508+0000","last_change":"2024-09-13T16:48:40.767044+0000","last_active":"2024-09-13T17:17:47.164508+0000","last_peered":"2024-09-13T17:17:47.164508+0000","last_clean":"2024-09-13T17:17:47.164508+0000","last_became_active":"2024-09-13T16:48:40.765861+0000","last_became_peered":"2024-09-13T16:48:40.765861+0000","last_unstale":"2024-09-13T17:17:47.164508+0000","last_undegraded":"2024-09-13T17:17:47.164508+0000","last_fullsized":"2024-09-13T17:17:47.164508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24690,"num_write_kb":320186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"64'7","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.308611+0000","last_change":"2024-09-13T17:18:46.308867+0000","last_active":"2024-09-13T17:18:46.308611+0000","last_peered":"2024-09-13T17:18:46.308611+0000","last_clean":"2024-09-13T17:18:46.308611+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T17:18:46.308611+0000","last_undegraded":"2024-09-13T17:18:46.308611+0000","last_fullsized":"2024-09-13T17:18:46.308611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'7","last_scrub_stamp":"2024-09-13T17:18:46.308558+0000","last_deep_scrub":"64'7","last_deep_scrub_stamp":"2024-09-13T17:18:46.308558+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.308558+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:56.445404+0000","scrub_duration":0.0073412219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"63'1425","reported_seq":1538,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.001931+0000","last_change":"2024-09-13T16:48:07.776943+0000","last_active":"2024-09-13T17:17:45.001931+0000","last_peered":"2024-09-13T17:17:45.001931+0000","last_clean":"2024-09-13T17:17:45.001931+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T17:17:45.001931+0000","last_undegraded":"2024-09-13T17:17:45.001931+0000","last_fullsized":"2024-09-13T17:17:45.001931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":38459,"num_write":1400,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"63'3637","reported_seq":3703,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591154+0000","last_change":"2024-09-13T16:48:08.189112+0000","last_active":"2024-09-13T17:17:49.591154+0000","last_peered":"2024-09-13T17:17:49.591154+0000","last_clean":"2024-09-13T17:17:49.591154+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T17:17:49.591154+0000","last_undegraded":"2024-09-13T17:17:49.591154+0000","last_fullsized":"2024-09-13T17:17:49.591154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19635,"num_write_kb":259018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'5067","reported_seq":5129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.164365+0000","last_change":"2024-09-13T16:48:40.831954+0000","last_active":"2024-09-13T17:17:47.164365+0000","last_peered":"2024-09-13T17:17:47.164365+0000","last_clean":"2024-09-13T17:17:47.164365+0000","last_became_active":"2024-09-13T16:48:40.831696+0000","last_became_peered":"2024-09-13T16:48:40.831696+0000","last_unstale":"2024-09-13T17:17:47.164365+0000","last_undegraded":"2024-09-13T17:17:47.164365+0000","last_fullsized":"2024-09-13T17:17:47.164365+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27028,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.556099+0000","last_change":"2024-09-13T17:18:02.556314+0000","last_active":"2024-09-13T17:18:02.556099+0000","last_peered":"2024-09-13T17:18:02.556099+0000","last_clean":"2024-09-13T17:18:02.556099+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T17:18:02.556099+0000","last_undegraded":"2024-09-13T17:18:02.556099+0000","last_fullsized":"2024-09-13T17:18:02.556099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.556044+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:52.052485+0000","scrub_duration":0.0060730330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"63'5","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:16.546615+0000","last_change":"2024-09-13T17:18:16.546615+0000","last_active":"2024-09-13T17:18:16.546615+0000","last_peered":"2024-09-13T17:18:16.546615+0000","last_clean":"2024-09-13T17:18:16.546615+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T17:18:16.546615+0000","last_undegraded":"2024-09-13T17:18:16.546615+0000","last_fullsized":"2024-09-13T17:18:16.546615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:16.546281+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:16.546281+0000","last_clean_scrub_stamp":"2024-09-13T17:18:16.546281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:15.818163+0000","scrub_duration":0.0035841639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"63'2915","reported_seq":2991,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.588368+0000","last_change":"2024-09-13T17:18:29.588368+0000","last_active":"2024-09-13T17:18:29.588368+0000","last_peered":"2024-09-13T17:18:29.588368+0000","last_clean":"2024-09-13T17:18:29.588368+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T17:18:29.588368+0000","last_undegraded":"2024-09-13T17:18:29.588368+0000","last_fullsized":"2024-09-13T17:18:29.588368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2915","last_scrub_stamp":"2024-09-13T17:18:29.588063+0000","last_deep_scrub":"63'2915","last_deep_scrub_stamp":"2024-09-13T17:18:29.588063+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.588063+0000","objects_scrubbed":0,"log_size":2915,"log_dups_size":0,"ondisk_log_size":2915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:14.651101+0000","scrub_duration":0.01592476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":234494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"63'3697","reported_seq":3776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.226187+0000","last_change":"2024-09-13T17:18:29.226417+0000","last_active":"2024-09-13T17:18:29.226187+0000","last_peered":"2024-09-13T17:18:29.226187+0000","last_clean":"2024-09-13T17:18:29.226187+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T17:18:29.226187+0000","last_undegraded":"2024-09-13T17:18:29.226187+0000","last_fullsized":"2024-09-13T17:18:29.226187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3697","last_scrub_stamp":"2024-09-13T17:18:29.226132+0000","last_deep_scrub":"63'3697","last_deep_scrub_stamp":"2024-09-13T17:18:29.226132+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.226132+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:46.199826+0000","scrub_duration":0.032222396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20206,"num_write_kb":253140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"64'5","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:06.185518+0000","last_change":"2024-09-13T17:18:06.186368+0000","last_active":"2024-09-13T17:18:06.185518+0000","last_peered":"2024-09-13T17:18:06.185518+0000","last_clean":"2024-09-13T17:18:06.185518+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T17:18:06.185518+0000","last_undegraded":"2024-09-13T17:18:06.185518+0000","last_fullsized":"2024-09-13T17:18:06.185518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:06.185462+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:06.185462+0000","last_clean_scrub_stamp":"2024-09-13T17:18:06.185462+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:29.073557+0000","scrub_duration":0.0065555370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.158515+0000","last_change":"2024-09-13T17:18:02.158515+0000","last_active":"2024-09-13T17:18:02.158515+0000","last_peered":"2024-09-13T17:18:02.158515+0000","last_clean":"2024-09-13T17:18:02.158515+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T17:18:02.158515+0000","last_undegraded":"2024-09-13T17:18:02.158515+0000","last_fullsized":"2024-09-13T17:18:02.158515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.158216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.118447+0000","scrub_duration":0.003800346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"63'3522","reported_seq":3596,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:40.984683+0000","last_change":"2024-09-13T17:18:40.984936+0000","last_active":"2024-09-13T17:18:40.984683+0000","last_peered":"2024-09-13T17:18:40.984683+0000","last_clean":"2024-09-13T17:18:40.984683+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T17:18:40.984683+0000","last_undegraded":"2024-09-13T17:18:40.984683+0000","last_fullsized":"2024-09-13T17:18:40.984683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3522","last_scrub_stamp":"2024-09-13T17:18:40.984629+0000","last_deep_scrub":"63'3522","last_deep_scrub_stamp":"2024-09-13T17:18:40.984629+0000","last_clean_scrub_stamp":"2024-09-13T17:18:40.984629+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:51.529447+0000","scrub_duration":0.025504836999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18828,"num_write_kb":272277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"63'3874","reported_seq":3955,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.213453+0000","last_change":"2024-09-13T17:17:59.213453+0000","last_active":"2024-09-13T17:17:59.213453+0000","last_peered":"2024-09-13T17:17:59.213453+0000","last_clean":"2024-09-13T17:17:59.213453+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T17:17:59.213453+0000","last_undegraded":"2024-09-13T17:17:59.213453+0000","last_fullsized":"2024-09-13T17:17:59.213453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.213146+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.190638+0000","scrub_duration":0.033789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21413,"num_write_kb":278431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'19","reported_seq":193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.533761+0000","last_change":"2024-09-13T17:17:59.533761+0000","last_active":"2024-09-13T17:17:59.533761+0000","last_peered":"2024-09-13T17:17:59.533761+0000","last_clean":"2024-09-13T17:17:59.533761+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T17:17:59.533761+0000","last_undegraded":"2024-09-13T17:17:59.533761+0000","last_fullsized":"2024-09-13T17:17:59.533761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_deep_scrub":"64'19","last_deep_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.533477+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:42.670255+0000","scrub_duration":0.0092684789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:00.848306+0000","last_change":"2024-09-13T17:18:00.848525+0000","last_active":"2024-09-13T17:18:00.848306+0000","last_peered":"2024-09-13T17:18:00.848306+0000","last_clean":"2024-09-13T17:18:00.848306+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T17:18:00.848306+0000","last_undegraded":"2024-09-13T17:18:00.848306+0000","last_fullsized":"2024-09-13T17:18:00.848306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_clean_scrub_stamp":"2024-09-13T17:18:00.848253+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:51.294142+0000","scrub_duration":0.0034249229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"63'3622","reported_seq":3688,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.984518+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T17:17:44.984518+0000","last_peered":"2024-09-13T17:17:44.984518+0000","last_clean":"2024-09-13T17:17:44.984518+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T17:17:44.984518+0000","last_undegraded":"2024-09-13T17:17:44.984518+0000","last_fullsized":"2024-09-13T17:17:44.984518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":19996,"num_write_kb":293269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"63'3240","reported_seq":3319,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.494730+0000","last_change":"2024-09-13T17:18:10.494730+0000","last_active":"2024-09-13T17:18:10.494730+0000","last_peered":"2024-09-13T17:18:10.494730+0000","last_clean":"2024-09-13T17:18:10.494730+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T17:18:10.494730+0000","last_undegraded":"2024-09-13T17:18:10.494730+0000","last_fullsized":"2024-09-13T17:18:10.494730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3240","last_scrub_stamp":"2024-09-13T17:18:10.494430+0000","last_deep_scrub":"63'3240","last_deep_scrub_stamp":"2024-09-13T17:18:10.494430+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.494430+0000","objects_scrubbed":0,"log_size":3240,"log_dups_size":0,"ondisk_log_size":3240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:03.000138+0000","scrub_duration":0.014460961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":237245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"63'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.449798+0000","last_change":"2024-09-13T17:18:02.449798+0000","last_active":"2024-09-13T17:18:02.449798+0000","last_peered":"2024-09-13T17:18:02.449798+0000","last_clean":"2024-09-13T17:18:02.449798+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T17:18:02.449798+0000","last_undegraded":"2024-09-13T17:18:02.449798+0000","last_fullsized":"2024-09-13T17:18:02.449798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.449537+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.409768+0000","scrub_duration":0.0032633279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'5","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.060756+0000","last_change":"2024-09-13T17:17:51.060978+0000","last_active":"2024-09-13T17:17:51.060756+0000","last_peered":"2024-09-13T17:17:51.060756+0000","last_clean":"2024-09-13T17:17:51.060756+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T17:17:51.060756+0000","last_undegraded":"2024-09-13T17:17:51.060756+0000","last_fullsized":"2024-09-13T17:17:51.060756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.060702+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:45.719305+0000","scrub_duration":0.0041164770000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"63'3376","reported_seq":3439,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.981690+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T17:17:44.981690+0000","last_peered":"2024-09-13T17:17:44.981690+0000","last_clean":"2024-09-13T17:17:44.981690+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T17:17:44.981690+0000","last_undegraded":"2024-09-13T17:17:44.981690+0000","last_fullsized":"2024-09-13T17:17:44.981690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18879,"num_write_kb":251906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"63'4848","reported_seq":4927,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:41.225569+0000","last_change":"2024-09-13T17:18:41.225810+0000","last_active":"2024-09-13T17:18:41.225569+0000","last_peered":"2024-09-13T17:18:41.225569+0000","last_clean":"2024-09-13T17:18:41.225569+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T17:18:41.225569+0000","last_undegraded":"2024-09-13T17:18:41.225569+0000","last_fullsized":"2024-09-13T17:18:41.225569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4848","last_scrub_stamp":"2024-09-13T17:18:41.225514+0000","last_deep_scrub":"63'4848","last_deep_scrub_stamp":"2024-09-13T17:18:41.225514+0000","last_clean_scrub_stamp":"2024-09-13T17:18:41.225514+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:51.770332+0000","scrub_duration":0.049129424999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26963,"num_write_kb":321221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"63'3","reported_seq":72,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.642511+0000","last_change":"2024-09-13T16:47:58.678365+0000","last_active":"2024-09-13T17:17:48.642511+0000","last_peered":"2024-09-13T17:17:48.642511+0000","last_clean":"2024-09-13T17:17:48.642511+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T17:17:48.642511+0000","last_undegraded":"2024-09-13T17:17:48.642511+0000","last_fullsized":"2024-09-13T17:17:48.642511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'392","reported_seq":575,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.116420+0000","last_change":"2024-09-13T17:17:49.116420+0000","last_active":"2024-09-13T17:17:49.116420+0000","last_peered":"2024-09-13T17:17:49.116420+0000","last_clean":"2024-09-13T17:17:49.116420+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T17:17:49.116420+0000","last_undegraded":"2024-09-13T17:17:49.116420+0000","last_fullsized":"2024-09-13T17:17:49.116420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'392","last_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_deep_scrub":"64'392","last_deep_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.116249+0000","objects_scrubbed":4,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:49.076480+0000","scrub_duration":0.026108916999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":191,"num_read_kb":82094,"num_write":385,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'4856","reported_seq":4922,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632005+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T17:17:47.632005+0000","last_peered":"2024-09-13T17:17:47.632005+0000","last_clean":"2024-09-13T17:17:47.632005+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T17:17:47.632005+0000","last_undegraded":"2024-09-13T17:17:47.632005+0000","last_fullsized":"2024-09-13T17:17:47.632005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":26453,"num_write_kb":368210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'4063","reported_seq":4142,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:24.994002+0000","last_change":"2024-09-13T17:18:24.994290+0000","last_active":"2024-09-13T17:18:24.994002+0000","last_peered":"2024-09-13T17:18:24.994002+0000","last_clean":"2024-09-13T17:18:24.994002+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T17:18:24.994002+0000","last_undegraded":"2024-09-13T17:18:24.994002+0000","last_fullsized":"2024-09-13T17:18:24.994002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4063","last_scrub_stamp":"2024-09-13T17:18:24.993948+0000","last_deep_scrub":"63'4063","last_deep_scrub_stamp":"2024-09-13T17:18:24.993948+0000","last_clean_scrub_stamp":"2024-09-13T17:18:24.993948+0000","objects_scrubbed":0,"log_size":4063,"log_dups_size":0,"ondisk_log_size":4063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:17.499656+0000","scrub_duration":0.029339008999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21999,"num_write_kb":293018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"64'11","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:07.796310+0000","last_change":"2024-09-13T17:18:07.796310+0000","last_active":"2024-09-13T17:18:07.796310+0000","last_peered":"2024-09-13T17:18:07.796310+0000","last_clean":"2024-09-13T17:18:07.796310+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T17:18:07.796310+0000","last_undegraded":"2024-09-13T17:18:07.796310+0000","last_fullsized":"2024-09-13T17:18:07.796310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-13T17:18:07.796034+0000","last_deep_scrub":"64'11","last_deep_scrub_stamp":"2024-09-13T17:18:07.796034+0000","last_clean_scrub_stamp":"2024-09-13T17:18:07.796034+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:50.932812+0000","scrub_duration":0.0076063959999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":152,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'32","reported_seq":170,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.557513+0000","last_change":"2024-09-13T17:17:59.557788+0000","last_active":"2024-09-13T17:17:59.557513+0000","last_peered":"2024-09-13T17:17:59.557513+0000","last_clean":"2024-09-13T17:17:59.557513+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T17:17:59.557513+0000","last_undegraded":"2024-09-13T17:17:59.557513+0000","last_fullsized":"2024-09-13T17:17:59.557513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'32","last_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_deep_scrub":"64'32","last_deep_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.557461+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.534953+0000","scrub_duration":0.0076444640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":31,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"63'4239","reported_seq":4303,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631955+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T17:17:47.631955+0000","last_peered":"2024-09-13T17:17:47.631955+0000","last_clean":"2024-09-13T17:17:47.631955+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T17:17:47.631955+0000","last_undegraded":"2024-09-13T17:17:47.631955+0000","last_fullsized":"2024-09-13T17:17:47.631955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23316,"num_write_kb":318961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'3718","reported_seq":3796,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.214013+0000","last_change":"2024-09-13T17:17:57.214254+0000","last_active":"2024-09-13T17:17:57.214013+0000","last_peered":"2024-09-13T17:17:57.214013+0000","last_clean":"2024-09-13T17:17:57.214013+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T17:17:57.214013+0000","last_undegraded":"2024-09-13T17:17:57.214013+0000","last_fullsized":"2024-09-13T17:17:57.214013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3718","last_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_deep_scrub":"63'3718","last_deep_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.213958+0000","objects_scrubbed":0,"log_size":3718,"log_dups_size":0,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.719666+0000","scrub_duration":0.032519578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20956,"num_write_kb":260218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"64'395","reported_seq":570,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.567435+0000","last_change":"2024-09-13T17:18:04.567435+0000","last_active":"2024-09-13T17:18:04.567435+0000","last_peered":"2024-09-13T17:18:04.567435+0000","last_clean":"2024-09-13T17:18:04.567435+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T17:18:04.567435+0000","last_undegraded":"2024-09-13T17:18:04.567435+0000","last_fullsized":"2024-09-13T17:18:04.567435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'395","last_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_deep_scrub":"64'395","last_deep_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.567179+0000","objects_scrubbed":4,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:59.225783+0000","scrub_duration":0.027913209000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":171,"num_read_kb":77976,"num_write":389,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"63'3","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:33.246155+0000","last_change":"2024-09-13T17:18:33.246380+0000","last_active":"2024-09-13T17:18:33.246155+0000","last_peered":"2024-09-13T17:18:33.246155+0000","last_clean":"2024-09-13T17:18:33.246155+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T17:18:33.246155+0000","last_undegraded":"2024-09-13T17:18:33.246155+0000","last_fullsized":"2024-09-13T17:18:33.246155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:33.246101+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:33.246101+0000","last_clean_scrub_stamp":"2024-09-13T17:18:33.246101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:32.671487+0000","scrub_duration":0.0034645399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"63'3522","reported_seq":3587,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.674980+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T17:17:46.674980+0000","last_peered":"2024-09-13T17:17:46.674980+0000","last_clean":"2024-09-13T17:17:46.674980+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T17:17:46.674980+0000","last_undegraded":"2024-09-13T17:17:46.674980+0000","last_fullsized":"2024-09-13T17:17:46.674980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":4,"num_write":19373,"num_write_kb":266222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"63'5138","reported_seq":5217,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:33.196310+0000","last_change":"2024-09-13T17:18:33.196535+0000","last_active":"2024-09-13T17:18:33.196310+0000","last_peered":"2024-09-13T17:18:33.196310+0000","last_clean":"2024-09-13T17:18:33.196310+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T17:18:33.196310+0000","last_undegraded":"2024-09-13T17:18:33.196310+0000","last_fullsized":"2024-09-13T17:18:33.196310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5138","last_scrub_stamp":"2024-09-13T17:18:33.196260+0000","last_deep_scrub":"63'5138","last_deep_scrub_stamp":"2024-09-13T17:18:33.196260+0000","last_clean_scrub_stamp":"2024-09-13T17:18:33.196260+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:32.621647+0000","scrub_duration":0.048334911000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27863,"num_write_kb":353454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'25","reported_seq":131,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.003205+0000","last_change":"2024-09-13T16:47:58.678346+0000","last_active":"2024-09-13T17:17:48.003205+0000","last_peered":"2024-09-13T17:17:48.003205+0000","last_clean":"2024-09-13T17:17:48.003205+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T17:17:48.003205+0000","last_undegraded":"2024-09-13T17:17:48.003205+0000","last_fullsized":"2024-09-13T17:17:48.003205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'9","reported_seq":127,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.154830+0000","last_change":"2024-09-13T17:18:03.154830+0000","last_active":"2024-09-13T17:18:03.154830+0000","last_peered":"2024-09-13T17:18:03.154830+0000","last_clean":"2024-09-13T17:18:03.154830+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T17:18:03.154830+0000","last_undegraded":"2024-09-13T17:18:03.154830+0000","last_fullsized":"2024-09-13T17:18:03.154830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_deep_scrub":"64'9","last_deep_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.154514+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:05.788731+0000","scrub_duration":0.0080602239999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":168,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"63'3602","reported_seq":3680,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.062966+0000","last_change":"2024-09-13T17:18:46.063201+0000","last_active":"2024-09-13T17:18:46.062966+0000","last_peered":"2024-09-13T17:18:46.062966+0000","last_clean":"2024-09-13T17:18:46.062966+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T17:18:46.062966+0000","last_undegraded":"2024-09-13T17:18:46.062966+0000","last_fullsized":"2024-09-13T17:18:46.062966+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3602","last_scrub_stamp":"2024-09-13T17:18:46.062913+0000","last_deep_scrub":"63'3602","last_deep_scrub_stamp":"2024-09-13T17:18:46.062913+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.062913+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:43.137455+0000","scrub_duration":0.025752721999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":20290,"num_write_kb":278255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'3743","reported_seq":3821,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.841035+0000","last_change":"2024-09-13T17:17:57.841262+0000","last_active":"2024-09-13T17:17:57.841035+0000","last_peered":"2024-09-13T17:17:57.841035+0000","last_clean":"2024-09-13T17:17:57.841035+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T17:17:57.841035+0000","last_undegraded":"2024-09-13T17:17:57.841035+0000","last_fullsized":"2024-09-13T17:17:57.841035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3743","last_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_deep_scrub":"63'3743","last_deep_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.840983+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:01.952379+0000","scrub_duration":0.026384685000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20404,"num_write_kb":278673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"63'3","reported_seq":71,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.986857+0000","last_change":"2024-09-13T16:47:58.690936+0000","last_active":"2024-09-13T17:17:44.986857+0000","last_peered":"2024-09-13T17:17:44.986857+0000","last_clean":"2024-09-13T17:17:44.986857+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T17:17:44.986857+0000","last_undegraded":"2024-09-13T17:17:44.986857+0000","last_fullsized":"2024-09-13T17:17:44.986857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'19","reported_seq":203,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:20.179269+0000","last_change":"2024-09-13T17:18:20.179269+0000","last_active":"2024-09-13T17:18:20.179269+0000","last_peered":"2024-09-13T17:18:20.179269+0000","last_clean":"2024-09-13T17:18:20.179269+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T17:18:20.179269+0000","last_undegraded":"2024-09-13T17:18:20.179269+0000","last_fullsized":"2024-09-13T17:18:20.179269+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-13T17:18:20.178917+0000","last_deep_scrub":"64'19","last_deep_scrub_stamp":"2024-09-13T17:18:20.178917+0000","last_clean_scrub_stamp":"2024-09-13T17:18:20.178917+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:19.604304+0000","scrub_duration":0.010053862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":324,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"63'3189","reported_seq":3253,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591477+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T17:17:49.591477+0000","last_peered":"2024-09-13T17:17:49.591477+0000","last_clean":"2024-09-13T17:17:49.591477+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T17:17:49.591477+0000","last_undegraded":"2024-09-13T17:17:49.591477+0000","last_fullsized":"2024-09-13T17:17:49.591477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17342,"num_write_kb":249797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"63'4223","reported_seq":4290,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.985904+0000","last_change":"2024-09-13T16:47:59.046646+0000","last_active":"2024-09-13T17:17:44.985904+0000","last_peered":"2024-09-13T17:17:44.985904+0000","last_clean":"2024-09-13T17:17:44.985904+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T17:17:44.985904+0000","last_undegraded":"2024-09-13T17:17:44.985904+0000","last_fullsized":"2024-09-13T17:17:44.985904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4223,"log_dups_size":0,"ondisk_log_size":4223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":295025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"64'5","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:06.450397+0000","last_change":"2024-09-13T17:18:06.450600+0000","last_active":"2024-09-13T17:18:06.450397+0000","last_peered":"2024-09-13T17:18:06.450397+0000","last_clean":"2024-09-13T17:18:06.450397+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T17:18:06.450397+0000","last_undegraded":"2024-09-13T17:18:06.450397+0000","last_fullsized":"2024-09-13T17:18:06.450397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_clean_scrub_stamp":"2024-09-13T17:18:06.450341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:29.338436+0000","scrub_duration":0.003290758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.123757+0000","last_change":"2024-09-13T17:17:53.123757+0000","last_active":"2024-09-13T17:17:53.123757+0000","last_peered":"2024-09-13T17:17:53.123757+0000","last_clean":"2024-09-13T17:17:53.123757+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T17:17:53.123757+0000","last_undegraded":"2024-09-13T17:17:53.123757+0000","last_fullsized":"2024-09-13T17:17:53.123757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.123466+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.011562+0000","scrub_duration":0.0038056650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'5357","reported_seq":5436,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.115658+0000","last_change":"2024-09-13T17:18:43.115905+0000","last_active":"2024-09-13T17:18:43.115658+0000","last_peered":"2024-09-13T17:18:43.115658+0000","last_clean":"2024-09-13T17:18:43.115658+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T17:18:43.115658+0000","last_undegraded":"2024-09-13T17:18:43.115658+0000","last_fullsized":"2024-09-13T17:18:43.115658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5357","last_scrub_stamp":"2024-09-13T17:18:43.115582+0000","last_deep_scrub":"63'5357","last_deep_scrub_stamp":"2024-09-13T17:18:43.115582+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.115582+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:20.465692+0000","scrub_duration":0.033536497999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":39,"num_write":29637,"num_write_kb":393569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"63'26","reported_seq":93,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.207278+0000","last_change":"2024-09-13T16:48:07.785480+0000","last_active":"2024-09-13T17:17:46.207278+0000","last_peered":"2024-09-13T17:17:46.207278+0000","last_clean":"2024-09-13T17:17:46.207278+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T17:17:46.207278+0000","last_undegraded":"2024-09-13T17:17:46.207278+0000","last_fullsized":"2024-09-13T17:17:46.207278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:06.713484+0000","last_clean_scrub_stamp":"2024-09-13T16:48:06.713484+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"63'5122","reported_seq":5200,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.190845+0000","last_change":"2024-09-13T17:18:01.191064+0000","last_active":"2024-09-13T17:18:01.190845+0000","last_peered":"2024-09-13T17:18:01.190845+0000","last_clean":"2024-09-13T17:18:01.190845+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T17:18:01.190845+0000","last_undegraded":"2024-09-13T17:18:01.190845+0000","last_fullsized":"2024-09-13T17:18:01.190845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5122","last_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_deep_scrub":"63'5122","last_deep_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.190787+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:03.825004+0000","scrub_duration":0.032904946999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":348704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"63'2","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.623848+0000","last_change":"2024-09-13T17:18:45.624092+0000","last_active":"2024-09-13T17:18:45.623848+0000","last_peered":"2024-09-13T17:18:45.623848+0000","last_clean":"2024-09-13T17:18:45.623848+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T17:18:45.623848+0000","last_undegraded":"2024-09-13T17:18:45.623848+0000","last_fullsized":"2024-09-13T17:18:45.623848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:45.623791+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:45.623791+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.623791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:26.049876+0000","scrub_duration":0.003142663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"63'3554","reported_seq":3620,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.207332+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T17:17:46.207332+0000","last_peered":"2024-09-13T17:17:46.207332+0000","last_clean":"2024-09-13T17:17:46.207332+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T17:17:46.207332+0000","last_undegraded":"2024-09-13T17:17:46.207332+0000","last_fullsized":"2024-09-13T17:17:46.207332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":268389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"64'1094","reported_seq":1221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.174321+0000","last_change":"2024-09-13T17:18:29.174543+0000","last_active":"2024-09-13T17:18:29.174321+0000","last_peered":"2024-09-13T17:18:29.174321+0000","last_clean":"2024-09-13T17:18:29.174321+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T17:18:29.174321+0000","last_undegraded":"2024-09-13T17:18:29.174321+0000","last_fullsized":"2024-09-13T17:18:29.174321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1094","last_scrub_stamp":"2024-09-13T17:18:29.174267+0000","last_deep_scrub":"64'1094","last_deep_scrub_stamp":"2024-09-13T17:18:29.174267+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.174267+0000","objects_scrubbed":3,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:28.446149+0000","scrub_duration":0.031395304999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":9436,"num_write":1065,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"63'4714","reported_seq":4792,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:32.980839+0000","last_change":"2024-09-13T17:18:32.981101+0000","last_active":"2024-09-13T17:18:32.980839+0000","last_peered":"2024-09-13T17:18:32.980839+0000","last_clean":"2024-09-13T17:18:32.980839+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T17:18:32.980839+0000","last_undegraded":"2024-09-13T17:18:32.980839+0000","last_fullsized":"2024-09-13T17:18:32.980839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4714","last_scrub_stamp":"2024-09-13T17:18:32.980782+0000","last_deep_scrub":"63'4714","last_deep_scrub_stamp":"2024-09-13T17:18:32.980782+0000","last_clean_scrub_stamp":"2024-09-13T17:18:32.980782+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:43.117628+0000","scrub_duration":0.031918914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25459,"num_write_kb":343662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"63'3","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.965455+0000","last_change":"2024-09-13T17:18:43.965730+0000","last_active":"2024-09-13T17:18:43.965455+0000","last_peered":"2024-09-13T17:18:43.965455+0000","last_clean":"2024-09-13T17:18:43.965455+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T17:18:43.965455+0000","last_undegraded":"2024-09-13T17:18:43.965455+0000","last_fullsized":"2024-09-13T17:18:43.965455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:43.965393+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:43.965393+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.965393+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:24.391478+0000","scrub_duration":0.0038411180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"63'4328","reported_seq":4391,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675065+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T17:17:46.675065+0000","last_peered":"2024-09-13T17:17:46.675065+0000","last_clean":"2024-09-13T17:17:46.675065+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T17:17:46.675065+0000","last_undegraded":"2024-09-13T17:17:46.675065+0000","last_fullsized":"2024-09-13T17:17:46.675065+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24456,"num_write_kb":315374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"64'411","reported_seq":494,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.515483+0000","last_change":"2024-09-13T17:18:03.515735+0000","last_active":"2024-09-13T17:18:03.515483+0000","last_peered":"2024-09-13T17:18:03.515483+0000","last_clean":"2024-09-13T17:18:03.515483+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T17:18:03.515483+0000","last_undegraded":"2024-09-13T17:18:03.515483+0000","last_fullsized":"2024-09-13T17:18:03.515483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'411","last_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_deep_scrub":"64'411","last_deep_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.515426+0000","objects_scrubbed":1,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:06.149643+0000","scrub_duration":0.019312658999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2812346,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3190,"num_write":386,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"63'4155","reported_seq":4233,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:20.944870+0000","last_change":"2024-09-13T17:18:20.945097+0000","last_active":"2024-09-13T17:18:20.944870+0000","last_peered":"2024-09-13T17:18:20.944870+0000","last_clean":"2024-09-13T17:18:20.944870+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T17:18:20.944870+0000","last_undegraded":"2024-09-13T17:18:20.944870+0000","last_fullsized":"2024-09-13T17:18:20.944870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4155","last_scrub_stamp":"2024-09-13T17:18:20.944817+0000","last_deep_scrub":"63'4155","last_deep_scrub_stamp":"2024-09-13T17:18:20.944817+0000","last_clean_scrub_stamp":"2024-09-13T17:18:20.944817+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:43.832912+0000","scrub_duration":0.029833493999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22247,"num_write_kb":285891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'6","reported_seq":123,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.503784+0000","last_change":"2024-09-13T17:18:10.504013+0000","last_active":"2024-09-13T17:18:10.503784+0000","last_peered":"2024-09-13T17:18:10.503784+0000","last_clean":"2024-09-13T17:18:10.503784+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T17:18:10.503784+0000","last_undegraded":"2024-09-13T17:18:10.503784+0000","last_fullsized":"2024-09-13T17:18:10.503784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-09-13T17:18:10.503729+0000","last_deep_scrub":"64'6","last_deep_scrub_stamp":"2024-09-13T17:18:10.503729+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.503729+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:33.391824+0000","scrub_duration":0.0063226599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"63'3426","reported_seq":3492,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.132960+0000","last_change":"2024-09-13T16:48:08.204237+0000","last_active":"2024-09-13T17:17:48.132960+0000","last_peered":"2024-09-13T17:17:48.132960+0000","last_clean":"2024-09-13T17:17:48.132960+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T17:17:48.132960+0000","last_undegraded":"2024-09-13T17:17:48.132960+0000","last_fullsized":"2024-09-13T17:17:48.132960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3426,"log_dups_size":0,"ondisk_log_size":3426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":18510,"num_write_kb":250514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'712","reported_seq":803,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631599+0000","last_change":"2024-09-13T16:47:58.674520+0000","last_active":"2024-09-13T17:17:47.631599+0000","last_peered":"2024-09-13T17:17:47.631599+0000","last_clean":"2024-09-13T17:17:47.631599+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T17:17:47.631599+0000","last_undegraded":"2024-09-13T17:17:47.631599+0000","last_fullsized":"2024-09-13T17:17:47.631599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:57.619902+0000","last_clean_scrub_stamp":"2024-09-13T16:47:57.619902+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":74674,"num_write":702,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"63'4316","reported_seq":4395,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.250446+0000","last_change":"2024-09-13T17:18:10.250685+0000","last_active":"2024-09-13T17:18:10.250446+0000","last_peered":"2024-09-13T17:18:10.250446+0000","last_clean":"2024-09-13T17:18:10.250446+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T17:18:10.250446+0000","last_undegraded":"2024-09-13T17:18:10.250446+0000","last_fullsized":"2024-09-13T17:18:10.250446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4316","last_scrub_stamp":"2024-09-13T17:18:10.250390+0000","last_deep_scrub":"63'4316","last_deep_scrub_stamp":"2024-09-13T17:18:10.250390+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.250390+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:02.756098+0000","scrub_duration":0.033652449000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22350,"num_write_kb":290228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"64'45","reported_seq":162,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.128882+0000","last_change":"2024-09-13T17:17:59.128882+0000","last_active":"2024-09-13T17:17:59.128882+0000","last_peered":"2024-09-13T17:17:59.128882+0000","last_clean":"2024-09-13T17:17:59.128882+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T17:17:59.128882+0000","last_undegraded":"2024-09-13T17:17:59.128882+0000","last_fullsized":"2024-09-13T17:17:59.128882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'45","last_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_deep_scrub":"64'45","last_deep_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.128560+0000","objects_scrubbed":1,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.106052+0000","scrub_duration":0.0062496970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"63'3129","reported_seq":3194,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632026+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T17:17:47.632026+0000","last_peered":"2024-09-13T17:17:47.632026+0000","last_clean":"2024-09-13T17:17:47.632026+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T17:17:47.632026+0000","last_undegraded":"2024-09-13T17:17:47.632026+0000","last_fullsized":"2024-09-13T17:17:47.632026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17279,"num_write_kb":257601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"63'4785","reported_seq":4865,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.565656+0000","last_change":"2024-09-13T17:17:49.565883+0000","last_active":"2024-09-13T17:17:49.565656+0000","last_peered":"2024-09-13T17:17:49.565656+0000","last_clean":"2024-09-13T17:17:49.565656+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T17:17:49.565656+0000","last_undegraded":"2024-09-13T17:17:49.565656+0000","last_fullsized":"2024-09-13T17:17:49.565656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4785","last_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_deep_scrub":"63'4785","last_deep_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.565599+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.011489+0000","scrub_duration":0.012593717000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26302,"num_write_kb":310249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"63'4231","reported_seq":4297,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675102+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T17:17:46.675102+0000","last_peered":"2024-09-13T17:17:46.675102+0000","last_clean":"2024-09-13T17:17:46.675102+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T17:17:46.675102+0000","last_undegraded":"2024-09-13T17:17:46.675102+0000","last_fullsized":"2024-09-13T17:17:46.675102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22604,"num_write_kb":324065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"63'3874","reported_seq":3955,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.662237+0000","last_change":"2024-09-13T17:18:04.662513+0000","last_active":"2024-09-13T17:18:04.662237+0000","last_peered":"2024-09-13T17:18:04.662237+0000","last_clean":"2024-09-13T17:18:04.662237+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T17:18:04.662237+0000","last_undegraded":"2024-09-13T17:18:04.662237+0000","last_fullsized":"2024-09-13T17:18:04.662237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.662182+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:07.296399+0000","scrub_duration":0.021443912999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20920,"num_write_kb":276387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"63'3865","reported_seq":3942,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:32.467133+0000","last_change":"2024-09-13T17:18:32.467133+0000","last_active":"2024-09-13T17:18:32.467133+0000","last_peered":"2024-09-13T17:18:32.467133+0000","last_clean":"2024-09-13T17:18:32.467133+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T17:18:32.467133+0000","last_undegraded":"2024-09-13T17:18:32.467133+0000","last_fullsized":"2024-09-13T17:18:32.467133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3865","last_scrub_stamp":"2024-09-13T17:18:32.466850+0000","last_deep_scrub":"63'3865","last_deep_scrub_stamp":"2024-09-13T17:18:32.466850+0000","last_clean_scrub_stamp":"2024-09-13T17:18:32.466850+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:09.816960+0000","scrub_duration":0.029037197000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21432,"num_write_kb":286796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"63'4552","reported_seq":4623,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.978243+0000","last_change":"2024-09-13T16:47:59.059277+0000","last_active":"2024-09-13T17:17:44.978243+0000","last_peered":"2024-09-13T17:17:44.978243+0000","last_clean":"2024-09-13T17:17:44.978243+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T17:17:44.978243+0000","last_undegraded":"2024-09-13T17:17:44.978243+0000","last_fullsized":"2024-09-13T17:17:44.978243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":24031,"num_write_kb":313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"63'4071","reported_seq":4137,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675131+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T17:17:46.675131+0000","last_peered":"2024-09-13T17:17:46.675131+0000","last_clean":"2024-09-13T17:17:46.675131+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T17:17:46.675131+0000","last_undegraded":"2024-09-13T17:17:46.675131+0000","last_fullsized":"2024-09-13T17:17:46.675131+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":22061,"num_write_kb":309650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"63'3506","reported_seq":3587,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:47.181003+0000","last_change":"2024-09-13T17:18:47.181238+0000","last_active":"2024-09-13T17:18:47.181003+0000","last_peered":"2024-09-13T17:18:47.181003+0000","last_clean":"2024-09-13T17:18:47.181003+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T17:18:47.181003+0000","last_undegraded":"2024-09-13T17:18:47.181003+0000","last_fullsized":"2024-09-13T17:18:47.181003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3506","last_scrub_stamp":"2024-09-13T17:18:47.180948+0000","last_deep_scrub":"63'3506","last_deep_scrub_stamp":"2024-09-13T17:18:47.180948+0000","last_clean_scrub_stamp":"2024-09-13T17:18:47.180948+0000","objects_scrubbed":0,"log_size":3506,"log_dups_size":0,"ondisk_log_size":3506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:46.606335+0000","scrub_duration":0.025510459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19493,"num_write_kb":254380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"63'4523","reported_seq":4588,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992102+0000","last_change":"2024-09-13T16:48:08.187264+0000","last_active":"2024-09-13T17:17:44.992102+0000","last_peered":"2024-09-13T17:17:44.992102+0000","last_clean":"2024-09-13T17:17:44.992102+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T17:17:44.992102+0000","last_undegraded":"2024-09-13T17:17:44.992102+0000","last_fullsized":"2024-09-13T17:17:44.992102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4523,"log_dups_size":0,"ondisk_log_size":4523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24221,"num_write_kb":338849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"63'4422","reported_seq":4492,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631627+0000","last_change":"2024-09-13T16:47:59.059554+0000","last_active":"2024-09-13T17:17:47.631627+0000","last_peered":"2024-09-13T17:17:47.631627+0000","last_clean":"2024-09-13T17:17:47.631627+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T17:17:47.631627+0000","last_undegraded":"2024-09-13T17:17:47.631627+0000","last_fullsized":"2024-09-13T17:17:47.631627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":24862,"num_write_kb":304752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"63'3771","reported_seq":3837,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976919+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T17:17:44.976919+0000","last_peered":"2024-09-13T17:17:44.976919+0000","last_clean":"2024-09-13T17:17:44.976919+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T17:17:44.976919+0000","last_undegraded":"2024-09-13T17:17:44.976919+0000","last_fullsized":"2024-09-13T17:17:44.976919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21135,"num_write_kb":285540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"63'4461","reported_seq":4541,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:31.181911+0000","last_change":"2024-09-13T17:18:31.182225+0000","last_active":"2024-09-13T17:18:31.181911+0000","last_peered":"2024-09-13T17:18:31.181911+0000","last_clean":"2024-09-13T17:18:31.181911+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T17:18:31.181911+0000","last_undegraded":"2024-09-13T17:18:31.181911+0000","last_fullsized":"2024-09-13T17:18:31.181911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4461","last_scrub_stamp":"2024-09-13T17:18:31.181856+0000","last_deep_scrub":"63'4461","last_deep_scrub_stamp":"2024-09-13T17:18:31.181856+0000","last_clean_scrub_stamp":"2024-09-13T17:18:31.181856+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:30.453738+0000","scrub_duration":0.032702224000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25388,"num_write_kb":319094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"63'4193","reported_seq":4258,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991839+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T17:17:44.991839+0000","last_peered":"2024-09-13T17:17:44.991839+0000","last_clean":"2024-09-13T17:17:44.991839+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T17:17:44.991839+0000","last_undegraded":"2024-09-13T17:17:44.991839+0000","last_fullsized":"2024-09-13T17:17:44.991839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4193,"log_dups_size":0,"ondisk_log_size":4193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23066,"num_write_kb":313713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"63'3978","reported_seq":4058,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.616395+0000","last_change":"2024-09-13T17:17:57.616624+0000","last_active":"2024-09-13T17:17:57.616395+0000","last_peered":"2024-09-13T17:17:57.616395+0000","last_clean":"2024-09-13T17:17:57.616395+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T17:17:57.616395+0000","last_undegraded":"2024-09-13T17:17:57.616395+0000","last_fullsized":"2024-09-13T17:17:57.616395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3978","last_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_deep_scrub":"63'3978","last_deep_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.616344+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:01.359043+0000","scrub_duration":0.024757745000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22157,"num_write_kb":281285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"63'4050","reported_seq":4117,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993755+0000","last_change":"2024-09-13T16:48:08.164325+0000","last_active":"2024-09-13T17:17:44.993755+0000","last_peered":"2024-09-13T17:17:44.993755+0000","last_clean":"2024-09-13T17:17:44.993755+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T17:17:44.993755+0000","last_undegraded":"2024-09-13T17:17:44.993755+0000","last_fullsized":"2024-09-13T17:17:44.993755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4050,"log_dups_size":0,"ondisk_log_size":4050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21731,"num_write_kb":291923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"63'5211","reported_seq":5274,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.009487+0000","last_change":"2024-09-13T16:48:40.831835+0000","last_active":"2024-09-13T17:17:45.009487+0000","last_peered":"2024-09-13T17:17:45.009487+0000","last_clean":"2024-09-13T17:17:45.009487+0000","last_became_active":"2024-09-13T16:48:40.831649+0000","last_became_peered":"2024-09-13T16:48:40.831649+0000","last_unstale":"2024-09-13T17:17:45.009487+0000","last_undegraded":"2024-09-13T17:17:45.009487+0000","last_fullsized":"2024-09-13T17:17:45.009487+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":28065,"num_write_kb":366501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"63'3806","reported_seq":3873,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632082+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T17:17:47.632082+0000","last_peered":"2024-09-13T17:17:47.632082+0000","last_clean":"2024-09-13T17:17:47.632082+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T17:17:47.632082+0000","last_undegraded":"2024-09-13T17:17:47.632082+0000","last_fullsized":"2024-09-13T17:17:47.632082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4,"num_write":20286,"num_write_kb":286237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"63'3659","reported_seq":3740,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:34.530920+0000","last_change":"2024-09-13T17:18:34.531140+0000","last_active":"2024-09-13T17:18:34.530920+0000","last_peered":"2024-09-13T17:18:34.530920+0000","last_clean":"2024-09-13T17:18:34.530920+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T17:18:34.530920+0000","last_undegraded":"2024-09-13T17:18:34.530920+0000","last_fullsized":"2024-09-13T17:18:34.530920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3659","last_scrub_stamp":"2024-09-13T17:18:34.530868+0000","last_deep_scrub":"63'3659","last_deep_scrub_stamp":"2024-09-13T17:18:34.530868+0000","last_clean_scrub_stamp":"2024-09-13T17:18:34.530868+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:29.832112+0000","scrub_duration":0.027670581999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19936,"num_write_kb":264332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"63'5003","reported_seq":5068,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591294+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T17:17:49.591294+0000","last_peered":"2024-09-13T17:17:49.591294+0000","last_clean":"2024-09-13T17:17:49.591294+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T17:17:49.591294+0000","last_undegraded":"2024-09-13T17:17:49.591294+0000","last_fullsized":"2024-09-13T17:17:49.591294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27137,"num_write_kb":387189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"63'4068","reported_seq":4147,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:30.029523+0000","last_change":"2024-09-13T17:18:30.029753+0000","last_active":"2024-09-13T17:18:30.029523+0000","last_peered":"2024-09-13T17:18:30.029523+0000","last_clean":"2024-09-13T17:18:30.029523+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T17:18:30.029523+0000","last_undegraded":"2024-09-13T17:18:30.029523+0000","last_fullsized":"2024-09-13T17:18:30.029523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4068","last_scrub_stamp":"2024-09-13T17:18:30.029466+0000","last_deep_scrub":"63'4068","last_deep_scrub_stamp":"2024-09-13T17:18:30.029466+0000","last_clean_scrub_stamp":"2024-09-13T17:18:30.029466+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:47.003161+0000","scrub_duration":0.033158017999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22149,"num_write_kb":293545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"63'3283","reported_seq":3358,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:47.724019+0000","last_change":"2024-09-13T17:18:47.724256+0000","last_active":"2024-09-13T17:18:47.724019+0000","last_peered":"2024-09-13T17:18:47.724019+0000","last_clean":"2024-09-13T17:18:47.724019+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T17:18:47.724019+0000","last_undegraded":"2024-09-13T17:18:47.724019+0000","last_fullsized":"2024-09-13T17:18:47.724019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3283","last_scrub_stamp":"2024-09-13T17:18:47.723967+0000","last_deep_scrub":"63'3283","last_deep_scrub_stamp":"2024-09-13T17:18:47.723967+0000","last_clean_scrub_stamp":"2024-09-13T17:18:47.723967+0000","objects_scrubbed":0,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:25.074078+0000","scrub_duration":0.018868821000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":248866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"63'4697","reported_seq":4776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:13.654346+0000","last_change":"2024-09-13T17:18:13.654578+0000","last_active":"2024-09-13T17:18:13.654346+0000","last_peered":"2024-09-13T17:18:13.654346+0000","last_clean":"2024-09-13T17:18:13.654346+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T17:18:13.654346+0000","last_undegraded":"2024-09-13T17:18:13.654346+0000","last_fullsized":"2024-09-13T17:18:13.654346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4697","last_scrub_stamp":"2024-09-13T17:18:13.654295+0000","last_deep_scrub":"63'4697","last_deep_scrub_stamp":"2024-09-13T17:18:13.654295+0000","last_clean_scrub_stamp":"2024-09-13T17:18:13.654295+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:58.717333+0000","scrub_duration":0.035392587000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25800,"num_write_kb":320606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"63'3820","reported_seq":3884,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974366+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T17:17:44.974366+0000","last_peered":"2024-09-13T17:17:44.974366+0000","last_clean":"2024-09-13T17:17:44.974366+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T17:17:44.974366+0000","last_undegraded":"2024-09-13T17:17:44.974366+0000","last_fullsized":"2024-09-13T17:17:44.974366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3820,"log_dups_size":0,"ondisk_log_size":3820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20813,"num_write_kb":300503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"63'4719","reported_seq":4800,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:56.892882+0000","last_change":"2024-09-13T17:17:56.893148+0000","last_active":"2024-09-13T17:17:56.892882+0000","last_peered":"2024-09-13T17:17:56.892882+0000","last_clean":"2024-09-13T17:17:56.892882+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T17:17:56.892882+0000","last_undegraded":"2024-09-13T17:17:56.892882+0000","last_fullsized":"2024-09-13T17:17:56.892882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4719","last_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_deep_scrub":"63'4719","last_deep_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_clean_scrub_stamp":"2024-09-13T17:17:56.892826+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:26.656787+0000","scrub_duration":0.032626751000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":25263,"num_write_kb":330973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"63'3722","reported_seq":3786,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975189+0000","last_change":"2024-09-13T16:48:08.192061+0000","last_active":"2024-09-13T17:17:44.975189+0000","last_peered":"2024-09-13T17:17:44.975189+0000","last_clean":"2024-09-13T17:17:44.975189+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T17:17:44.975189+0000","last_undegraded":"2024-09-13T17:17:44.975189+0000","last_fullsized":"2024-09-13T17:17:44.975189+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20022,"num_write_kb":289122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"63'3813","reported_seq":3892,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:26.209831+0000","last_change":"2024-09-13T17:18:26.210107+0000","last_active":"2024-09-13T17:18:26.209831+0000","last_peered":"2024-09-13T17:18:26.209831+0000","last_clean":"2024-09-13T17:18:26.209831+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T17:18:26.209831+0000","last_undegraded":"2024-09-13T17:18:26.209831+0000","last_fullsized":"2024-09-13T17:18:26.209831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3813","last_scrub_stamp":"2024-09-13T17:18:26.209775+0000","last_deep_scrub":"63'3813","last_deep_scrub_stamp":"2024-09-13T17:18:26.209775+0000","last_clean_scrub_stamp":"2024-09-13T17:18:26.209775+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:11.272813+0000","scrub_duration":0.022253663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21202,"num_write_kb":265282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"63'4479","reported_seq":4547,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976201+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T17:17:44.976201+0000","last_peered":"2024-09-13T17:17:44.976201+0000","last_clean":"2024-09-13T17:17:44.976201+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T17:17:44.976201+0000","last_undegraded":"2024-09-13T17:17:44.976201+0000","last_fullsized":"2024-09-13T17:17:44.976201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":24556,"num_write_kb":340437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"63'4549","reported_seq":4617,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996096+0000","last_change":"2024-09-13T16:47:59.057630+0000","last_active":"2024-09-13T17:17:44.996096+0000","last_peered":"2024-09-13T17:17:44.996096+0000","last_clean":"2024-09-13T17:17:44.996096+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T17:17:44.996096+0000","last_undegraded":"2024-09-13T17:17:44.996096+0000","last_fullsized":"2024-09-13T17:17:44.996096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24549,"num_write_kb":319591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"63'3479","reported_seq":3545,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.643279+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T17:17:48.643279+0000","last_peered":"2024-09-13T17:17:48.643279+0000","last_clean":"2024-09-13T17:17:48.643279+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T17:17:48.643279+0000","last_undegraded":"2024-09-13T17:17:48.643279+0000","last_fullsized":"2024-09-13T17:17:48.643279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3479,"log_dups_size":0,"ondisk_log_size":3479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18966,"num_write_kb":268432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"63'3573","reported_seq":3653,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:08.568217+0000","last_change":"2024-09-13T17:18:08.568217+0000","last_active":"2024-09-13T17:18:08.568217+0000","last_peered":"2024-09-13T17:18:08.568217+0000","last_clean":"2024-09-13T17:18:08.568217+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T17:18:08.568217+0000","last_undegraded":"2024-09-13T17:18:08.568217+0000","last_fullsized":"2024-09-13T17:18:08.568217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3573","last_scrub_stamp":"2024-09-13T17:18:08.567856+0000","last_deep_scrub":"63'3573","last_deep_scrub_stamp":"2024-09-13T17:18:08.567856+0000","last_clean_scrub_stamp":"2024-09-13T17:18:08.567856+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:07.880974+0000","scrub_duration":0.01354239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19348,"num_write_kb":270113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"63'4058","reported_seq":4122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991645+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T17:17:44.991645+0000","last_peered":"2024-09-13T17:17:44.991645+0000","last_clean":"2024-09-13T17:17:44.991645+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T17:17:44.991645+0000","last_undegraded":"2024-09-13T17:17:44.991645+0000","last_fullsized":"2024-09-13T17:17:44.991645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22331,"num_write_kb":302585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"63'5839","reported_seq":5918,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.655103+0000","last_change":"2024-09-13T17:18:44.655336+0000","last_active":"2024-09-13T17:18:44.655103+0000","last_peered":"2024-09-13T17:18:44.655103+0000","last_clean":"2024-09-13T17:18:44.655103+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T17:18:44.655103+0000","last_undegraded":"2024-09-13T17:18:44.655103+0000","last_fullsized":"2024-09-13T17:18:44.655103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5839","last_scrub_stamp":"2024-09-13T17:18:44.655047+0000","last_deep_scrub":"63'5839","last_deep_scrub_stamp":"2024-09-13T17:18:44.655047+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.655047+0000","objects_scrubbed":0,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:17.349629+0000","scrub_duration":0.042557057000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31756,"num_write_kb":397521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"63'3835","reported_seq":3899,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992992+0000","last_change":"2024-09-13T16:48:08.167735+0000","last_active":"2024-09-13T17:17:44.992992+0000","last_peered":"2024-09-13T17:17:44.992992+0000","last_clean":"2024-09-13T17:17:44.992992+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T17:17:44.992992+0000","last_undegraded":"2024-09-13T17:17:44.992992+0000","last_fullsized":"2024-09-13T17:17:44.992992+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":21185,"num_write_kb":306173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"63'4134","reported_seq":4213,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.595782+0000","last_change":"2024-09-13T17:17:57.595782+0000","last_active":"2024-09-13T17:17:57.595782+0000","last_peered":"2024-09-13T17:17:57.595782+0000","last_clean":"2024-09-13T17:17:57.595782+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T17:17:57.595782+0000","last_undegraded":"2024-09-13T17:17:57.595782+0000","last_fullsized":"2024-09-13T17:17:57.595782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4134","last_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_deep_scrub":"63'4134","last_deep_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.595505+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:50.101213+0000","scrub_duration":0.024248147000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22268,"num_write_kb":281801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"63'5073","reported_seq":5138,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.981849+0000","last_change":"2024-09-13T16:48:08.204209+0000","last_active":"2024-09-13T17:17:44.981849+0000","last_peered":"2024-09-13T17:17:44.981849+0000","last_clean":"2024-09-13T17:17:44.981849+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T17:17:44.981849+0000","last_undegraded":"2024-09-13T17:17:44.981849+0000","last_fullsized":"2024-09-13T17:17:44.981849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":28693,"num_write_kb":393724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"63'3822","reported_seq":3901,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:07.186357+0000","last_change":"2024-09-13T17:18:07.186357+0000","last_active":"2024-09-13T17:18:07.186357+0000","last_peered":"2024-09-13T17:18:07.186357+0000","last_clean":"2024-09-13T17:18:07.186357+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T17:18:07.186357+0000","last_undegraded":"2024-09-13T17:18:07.186357+0000","last_fullsized":"2024-09-13T17:18:07.186357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3822","last_scrub_stamp":"2024-09-13T17:18:07.186043+0000","last_deep_scrub":"63'3822","last_deep_scrub_stamp":"2024-09-13T17:18:07.186043+0000","last_clean_scrub_stamp":"2024-09-13T17:18:07.186043+0000","objects_scrubbed":0,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:30.074139+0000","scrub_duration":0.030687262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20823,"num_write_kb":275830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"63'2778","reported_seq":2844,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975133+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T17:17:44.975133+0000","last_peered":"2024-09-13T17:17:44.975133+0000","last_clean":"2024-09-13T17:17:44.975133+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T17:17:44.975133+0000","last_undegraded":"2024-09-13T17:17:44.975133+0000","last_fullsized":"2024-09-13T17:17:44.975133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":34,"num_write":15212,"num_write_kb":208573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"63'3975","reported_seq":4054,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.856922+0000","last_change":"2024-09-13T17:18:01.856922+0000","last_active":"2024-09-13T17:18:01.856922+0000","last_peered":"2024-09-13T17:18:01.856922+0000","last_clean":"2024-09-13T17:18:01.856922+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T17:18:01.856922+0000","last_undegraded":"2024-09-13T17:18:01.856922+0000","last_fullsized":"2024-09-13T17:18:01.856922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3975","last_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_deep_scrub":"63'3975","last_deep_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.856645+0000","objects_scrubbed":0,"log_size":3975,"log_dups_size":0,"ondisk_log_size":3975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:51.096023+0000","scrub_duration":0.021351690999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21351,"num_write_kb":278200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"63'3844","reported_seq":3920,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:17.247940+0000","last_change":"2024-09-13T17:18:17.248778+0000","last_active":"2024-09-13T17:18:17.247940+0000","last_peered":"2024-09-13T17:18:17.247940+0000","last_clean":"2024-09-13T17:18:17.247940+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T17:18:17.247940+0000","last_undegraded":"2024-09-13T17:18:17.247940+0000","last_fullsized":"2024-09-13T17:18:17.247940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3844","last_scrub_stamp":"2024-09-13T17:18:17.247886+0000","last_deep_scrub":"63'3844","last_deep_scrub_stamp":"2024-09-13T17:18:17.247886+0000","last_clean_scrub_stamp":"2024-09-13T17:18:17.247886+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:16.519768+0000","scrub_duration":0.029367863000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21085,"num_write_kb":289508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"63'3324","reported_seq":3402,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:18.177496+0000","last_change":"2024-09-13T17:18:18.177724+0000","last_active":"2024-09-13T17:18:18.177496+0000","last_peered":"2024-09-13T17:18:18.177496+0000","last_clean":"2024-09-13T17:18:18.177496+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T17:18:18.177496+0000","last_undegraded":"2024-09-13T17:18:18.177496+0000","last_fullsized":"2024-09-13T17:18:18.177496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3324","last_scrub_stamp":"2024-09-13T17:18:18.177437+0000","last_deep_scrub":"63'3324","last_deep_scrub_stamp":"2024-09-13T17:18:18.177437+0000","last_clean_scrub_stamp":"2024-09-13T17:18:18.177437+0000","objects_scrubbed":0,"log_size":3324,"log_dups_size":0,"ondisk_log_size":3324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:17.449318+0000","scrub_duration":0.024197943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18810,"num_write_kb":235312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"63'3743","reported_seq":3808,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632187+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T17:17:47.632187+0000","last_peered":"2024-09-13T17:17:47.632187+0000","last_clean":"2024-09-13T17:17:47.632187+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T17:17:47.632187+0000","last_undegraded":"2024-09-13T17:17:47.632187+0000","last_fullsized":"2024-09-13T17:17:47.632187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20790,"num_write_kb":280180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"63'3173","reported_seq":3251,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.207709+0000","last_change":"2024-09-13T17:18:43.207942+0000","last_active":"2024-09-13T17:18:43.207709+0000","last_peered":"2024-09-13T17:18:43.207709+0000","last_clean":"2024-09-13T17:18:43.207709+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T17:18:43.207709+0000","last_undegraded":"2024-09-13T17:18:43.207709+0000","last_fullsized":"2024-09-13T17:18:43.207709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3173","last_scrub_stamp":"2024-09-13T17:18:43.207650+0000","last_deep_scrub":"63'3173","last_deep_scrub_stamp":"2024-09-13T17:18:43.207650+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.207650+0000","objects_scrubbed":0,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:00.181344+0000","scrub_duration":0.024785841999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17044,"num_write_kb":219853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"63'3460","reported_seq":3535,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.695358+0000","last_change":"2024-09-13T17:18:44.695678+0000","last_active":"2024-09-13T17:18:44.695358+0000","last_peered":"2024-09-13T17:18:44.695358+0000","last_clean":"2024-09-13T17:18:44.695358+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T17:18:44.695358+0000","last_undegraded":"2024-09-13T17:18:44.695358+0000","last_fullsized":"2024-09-13T17:18:44.695358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3460","last_scrub_stamp":"2024-09-13T17:18:44.695305+0000","last_deep_scrub":"63'3460","last_deep_scrub_stamp":"2024-09-13T17:18:44.695305+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.695305+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:57.484941+0000","scrub_duration":0.017970386000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":19788,"num_write_kb":263976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"63'3747","reported_seq":3825,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:12.464112+0000","last_change":"2024-09-13T17:18:12.464344+0000","last_active":"2024-09-13T17:18:12.464112+0000","last_peered":"2024-09-13T17:18:12.464112+0000","last_clean":"2024-09-13T17:18:12.464112+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T17:18:12.464112+0000","last_undegraded":"2024-09-13T17:18:12.464112+0000","last_fullsized":"2024-09-13T17:18:12.464112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3747","last_scrub_stamp":"2024-09-13T17:18:12.464055+0000","last_deep_scrub":"63'3747","last_deep_scrub_stamp":"2024-09-13T17:18:12.464055+0000","last_clean_scrub_stamp":"2024-09-13T17:18:12.464055+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:25.441547+0000","scrub_duration":0.021103311999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20231,"num_write_kb":269532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"63'3422","reported_seq":3497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:27.175049+0000","last_change":"2024-09-13T17:18:27.175049+0000","last_active":"2024-09-13T17:18:27.175049+0000","last_peered":"2024-09-13T17:18:27.175049+0000","last_clean":"2024-09-13T17:18:27.175049+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T17:18:27.175049+0000","last_undegraded":"2024-09-13T17:18:27.175049+0000","last_fullsized":"2024-09-13T17:18:27.175049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3422","last_scrub_stamp":"2024-09-13T17:18:27.174746+0000","last_deep_scrub":"63'3422","last_deep_scrub_stamp":"2024-09-13T17:18:27.174746+0000","last_clean_scrub_stamp":"2024-09-13T17:18:27.174746+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:39.964382+0000","scrub_duration":0.029236364000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18722,"num_write_kb":267617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"63'4803","reported_seq":4881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.621276+0000","last_change":"2024-09-13T17:17:58.621508+0000","last_active":"2024-09-13T17:17:58.621276+0000","last_peered":"2024-09-13T17:17:58.621276+0000","last_clean":"2024-09-13T17:17:58.621276+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T17:17:58.621276+0000","last_undegraded":"2024-09-13T17:17:58.621276+0000","last_fullsized":"2024-09-13T17:17:58.621276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4803","last_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_deep_scrub":"63'4803","last_deep_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.621222+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:06.376472+0000","scrub_duration":0.023354388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26967,"num_write_kb":333429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"63'3840","reported_seq":3904,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.643285+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T17:17:48.643285+0000","last_peered":"2024-09-13T17:17:48.643285+0000","last_clean":"2024-09-13T17:17:48.643285+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T17:17:48.643285+0000","last_undegraded":"2024-09-13T17:17:48.643285+0000","last_fullsized":"2024-09-13T17:17:48.643285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21113,"num_write_kb":293770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"63'5433","reported_seq":5513,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.303582+0000","last_change":"2024-09-13T17:17:53.303824+0000","last_active":"2024-09-13T17:17:53.303582+0000","last_peered":"2024-09-13T17:17:53.303582+0000","last_clean":"2024-09-13T17:17:53.303582+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T17:17:53.303582+0000","last_undegraded":"2024-09-13T17:17:53.303582+0000","last_fullsized":"2024-09-13T17:17:53.303582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5433","last_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_deep_scrub":"63'5433","last_deep_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.303524+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.191620+0000","scrub_duration":0.041011686999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30664,"num_write_kb":389336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"63'4741","reported_seq":4808,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974867+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T17:17:44.974867+0000","last_peered":"2024-09-13T17:17:44.974867+0000","last_clean":"2024-09-13T17:17:44.974867+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T17:17:44.974867+0000","last_undegraded":"2024-09-13T17:17:44.974867+0000","last_fullsized":"2024-09-13T17:17:44.974867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":15,"num_write":25588,"num_write_kb":340391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"63'4281","reported_seq":4359,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.227117+0000","last_change":"2024-09-13T17:17:57.227117+0000","last_active":"2024-09-13T17:17:57.227117+0000","last_peered":"2024-09-13T17:17:57.227117+0000","last_clean":"2024-09-13T17:17:57.227117+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T17:17:57.227117+0000","last_undegraded":"2024-09-13T17:17:57.227117+0000","last_fullsized":"2024-09-13T17:17:57.227117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4281","last_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_deep_scrub":"63'4281","last_deep_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226795+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.732503+0000","scrub_duration":0.034883109000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23373,"num_write_kb":302635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"63'3575","reported_seq":3654,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.020735+0000","last_change":"2024-09-13T17:18:45.020990+0000","last_active":"2024-09-13T17:18:45.020735+0000","last_peered":"2024-09-13T17:18:45.020735+0000","last_clean":"2024-09-13T17:18:45.020735+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T17:18:45.020735+0000","last_undegraded":"2024-09-13T17:18:45.020735+0000","last_fullsized":"2024-09-13T17:18:45.020735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3575","last_scrub_stamp":"2024-09-13T17:18:45.020682+0000","last_deep_scrub":"63'3575","last_deep_scrub_stamp":"2024-09-13T17:18:45.020682+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.020682+0000","objects_scrubbed":0,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:55.157528+0000","scrub_duration":0.013036646000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":7,"num_write":20117,"num_write_kb":289677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"63'4404","reported_seq":4482,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.337858+0000","last_change":"2024-09-13T17:17:50.337858+0000","last_active":"2024-09-13T17:17:50.337858+0000","last_peered":"2024-09-13T17:17:50.337858+0000","last_clean":"2024-09-13T17:17:50.337858+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T17:17:50.337858+0000","last_undegraded":"2024-09-13T17:17:50.337858+0000","last_fullsized":"2024-09-13T17:17:50.337858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4404","last_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_deep_scrub":"63'4404","last_deep_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.337585+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.971802+0000","scrub_duration":0.031410212999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24308,"num_write_kb":306423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"63'3318","reported_seq":3385,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675177+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T17:17:46.675177+0000","last_peered":"2024-09-13T17:17:46.675177+0000","last_clean":"2024-09-13T17:17:46.675177+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T17:17:46.675177+0000","last_undegraded":"2024-09-13T17:17:46.675177+0000","last_fullsized":"2024-09-13T17:17:46.675177+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3318,"log_dups_size":0,"ondisk_log_size":3318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18295,"num_write_kb":253537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"63'4013","reported_seq":4091,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.598539+0000","last_change":"2024-09-13T17:17:53.598539+0000","last_active":"2024-09-13T17:17:53.598539+0000","last_peered":"2024-09-13T17:17:53.598539+0000","last_clean":"2024-09-13T17:17:53.598539+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T17:17:53.598539+0000","last_undegraded":"2024-09-13T17:17:53.598539+0000","last_fullsized":"2024-09-13T17:17:53.598539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4013","last_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_deep_scrub":"63'4013","last_deep_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.598242+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.486338+0000","scrub_duration":0.028004864000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21393,"num_write_kb":262042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"63'3658","reported_seq":3724,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.982869+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T17:17:44.982869+0000","last_peered":"2024-09-13T17:17:44.982869+0000","last_clean":"2024-09-13T17:17:44.982869+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T17:17:44.982869+0000","last_undegraded":"2024-09-13T17:17:44.982869+0000","last_fullsized":"2024-09-13T17:17:44.982869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19946,"num_write_kb":268446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"63'3910","reported_seq":3988,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:19.247862+0000","last_change":"2024-09-13T17:18:19.248105+0000","last_active":"2024-09-13T17:18:19.247862+0000","last_peered":"2024-09-13T17:18:19.247862+0000","last_clean":"2024-09-13T17:18:19.247862+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T17:18:19.247862+0000","last_undegraded":"2024-09-13T17:18:19.247862+0000","last_fullsized":"2024-09-13T17:18:19.247862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3910","last_scrub_stamp":"2024-09-13T17:18:19.247809+0000","last_deep_scrub":"63'3910","last_deep_scrub_stamp":"2024-09-13T17:18:19.247809+0000","last_clean_scrub_stamp":"2024-09-13T17:18:19.247809+0000","objects_scrubbed":0,"log_size":3910,"log_dups_size":0,"ondisk_log_size":3910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:18.673196+0000","scrub_duration":0.029454408000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":278651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"63'3074","reported_seq":3137,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632259+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T17:17:47.632259+0000","last_peered":"2024-09-13T17:17:47.632259+0000","last_clean":"2024-09-13T17:17:47.632259+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T17:17:47.632259+0000","last_undegraded":"2024-09-13T17:17:47.632259+0000","last_fullsized":"2024-09-13T17:17:47.632259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":246226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"63'3515","reported_seq":3597,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.560367+0000","last_change":"2024-09-13T17:17:52.560367+0000","last_active":"2024-09-13T17:17:52.560367+0000","last_peered":"2024-09-13T17:17:52.560367+0000","last_clean":"2024-09-13T17:17:52.560367+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T17:17:52.560367+0000","last_undegraded":"2024-09-13T17:17:52.560367+0000","last_fullsized":"2024-09-13T17:17:52.560367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3515","last_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_deep_scrub":"63'3515","last_deep_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.560093+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.500715+0000","scrub_duration":0.026529094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19512,"num_write_kb":258445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"63'3439","reported_seq":3514,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.727356+0000","last_change":"2024-09-13T17:18:45.727616+0000","last_active":"2024-09-13T17:18:45.727356+0000","last_peered":"2024-09-13T17:18:45.727356+0000","last_clean":"2024-09-13T17:18:45.727356+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T17:18:45.727356+0000","last_undegraded":"2024-09-13T17:18:45.727356+0000","last_fullsized":"2024-09-13T17:18:45.727356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3439","last_scrub_stamp":"2024-09-13T17:18:45.727303+0000","last_deep_scrub":"63'3439","last_deep_scrub_stamp":"2024-09-13T17:18:45.727303+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.727303+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:02.700997+0000","scrub_duration":0.020628344999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":17681,"num_write_kb":263603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"63'3892","reported_seq":3973,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.152704+0000","last_change":"2024-09-13T17:18:45.152938+0000","last_active":"2024-09-13T17:18:45.152704+0000","last_peered":"2024-09-13T17:18:45.152704+0000","last_clean":"2024-09-13T17:18:45.152704+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T17:18:45.152704+0000","last_undegraded":"2024-09-13T17:18:45.152704+0000","last_fullsized":"2024-09-13T17:18:45.152704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3892","last_scrub_stamp":"2024-09-13T17:18:45.152646+0000","last_deep_scrub":"63'3892","last_deep_scrub_stamp":"2024-09-13T17:18:45.152646+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.152646+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:44.424528+0000","scrub_duration":0.018925824000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21304,"num_write_kb":286190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"63'3622","reported_seq":3685,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591670+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T17:17:49.591670+0000","last_peered":"2024-09-13T17:17:49.591670+0000","last_clean":"2024-09-13T17:17:49.591670+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T17:17:49.591670+0000","last_undegraded":"2024-09-13T17:17:49.591670+0000","last_fullsized":"2024-09-13T17:17:49.591670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19729,"num_write_kb":275691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"63'5036","reported_seq":5116,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:41.341661+0000","last_change":"2024-09-13T17:18:41.341900+0000","last_active":"2024-09-13T17:18:41.341661+0000","last_peered":"2024-09-13T17:18:41.341661+0000","last_clean":"2024-09-13T17:18:41.341661+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T17:18:41.341661+0000","last_undegraded":"2024-09-13T17:18:41.341661+0000","last_fullsized":"2024-09-13T17:18:41.341661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5036","last_scrub_stamp":"2024-09-13T17:18:41.341603+0000","last_deep_scrub":"63'5036","last_deep_scrub_stamp":"2024-09-13T17:18:41.341603+0000","last_clean_scrub_stamp":"2024-09-13T17:18:41.341603+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:54.131238+0000","scrub_duration":0.03025245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27469,"num_write_kb":368239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"63'3689","reported_seq":3763,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:34.266698+0000","last_change":"2024-09-13T17:18:34.266933+0000","last_active":"2024-09-13T17:18:34.266698+0000","last_peered":"2024-09-13T17:18:34.266698+0000","last_clean":"2024-09-13T17:18:34.266698+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T17:18:34.266698+0000","last_undegraded":"2024-09-13T17:18:34.266698+0000","last_fullsized":"2024-09-13T17:18:34.266698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3689","last_scrub_stamp":"2024-09-13T17:18:34.266642+0000","last_deep_scrub":"63'3689","last_deep_scrub_stamp":"2024-09-13T17:18:34.266642+0000","last_clean_scrub_stamp":"2024-09-13T17:18:34.266642+0000","objects_scrubbed":0,"log_size":3689,"log_dups_size":0,"ondisk_log_size":3689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:44.403488+0000","scrub_duration":0.024276446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20818,"num_write_kb":284187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"63'4047","reported_seq":4128,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.364122+0000","last_change":"2024-09-13T17:17:51.364373+0000","last_active":"2024-09-13T17:17:51.364122+0000","last_peered":"2024-09-13T17:17:51.364122+0000","last_clean":"2024-09-13T17:17:51.364122+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T17:17:51.364122+0000","last_undegraded":"2024-09-13T17:17:51.364122+0000","last_fullsized":"2024-09-13T17:17:51.364122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4047","last_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_deep_scrub":"63'4047","last_deep_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.364064+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:46.022668+0000","scrub_duration":0.028243739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22743,"num_write_kb":291925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":197,"num_write":1329762,"num_write_kb":18444058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242925,"ondisk_log_size":242925,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19024977,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1583,"num_read_kb":206738,"num_write":4472,"num_write_kb":20527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38109184,"data_stored":38049954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4375,"ondisk_log_size":4375,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":1479870,"num_write_kb":19043750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270762,"ondisk_log_size":270762,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":19675678,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1366,"num_read_kb":209632,"num_write":4521,"num_write_kb":21162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39411712,"data_stored":39351356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4382,"ondisk_log_size":4382,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575225,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":942784,"kb_used_data":13688,"kb_used_omap":1,"kb_used_meta":929086,"kb_avail":93429056,"statfs":{"total":96636764160,"available":95671353344,"internally_reserved":0,"allocated":14016512,"data_stored":13616965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":951384523},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":2.0699999999999998,"15min":1.2929999999999999},"min":{"1min":0.61499999999999999,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":1.3169999999999999,"5min":16.571999999999999,"15min":16.571999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":2.1110000000000002,"15min":1.385},"min":{"1min":0.60699999999999998,"5min":0.26600000000000001,"15min":0.20899999999999999},"max":{"1min":1.177,"5min":16.523,"15min":16.523},"last":0.80900000000000005}]},{"osd":1,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":2.6899999999999999,"15min":1.5640000000000001},"min":{"1min":0.56799999999999995,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.294,"5min":59.911999999999999,"15min":59.911999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.004,"5min":2.7850000000000001,"15min":1.671},"min":{"1min":0.69699999999999995,"5min":0.28399999999999997,"15min":0.23000000000000001},"max":{"1min":1.2110000000000001,"5min":59.895000000000003,"15min":59.895000000000003},"last":1.014}]},{"osd":2,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.718,"15min":1.153},"min":{"1min":0.64100000000000001,"5min":0.28999999999999998,"15min":0.25700000000000001},"max":{"1min":1.143,"5min":16.023,"15min":16.023},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.851,"15min":1.2669999999999999},"min":{"1min":0.63300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.397,"5min":18.734000000000002,"15min":18.734000000000002},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":2.0779999999999998,"15min":1.3120000000000001},"min":{"1min":0.69899999999999995,"5min":0.30199999999999999,"15min":0.28299999999999997},"max":{"1min":1.1359999999999999,"5min":23.225000000000001,"15min":23.225000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":2.319,"15min":1.4650000000000001},"min":{"1min":0.81399999999999995,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.47,"5min":19.709,"15min":19.709},"last":0.96799999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":2.133,"15min":1.3959999999999999},"min":{"1min":0.58799999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.2769999999999999,"5min":18.004999999999999,"15min":18.004999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":2.3690000000000002,"15min":1.48},"min":{"1min":0.66300000000000003,"5min":0.27300000000000002,"15min":0.245},"max":{"1min":1.1439999999999999,"5min":22.315999999999999,"15min":22.315999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":2.5169999999999999,"15min":1.5329999999999999},"min":{"1min":0.72499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.5429999999999999,"5min":37.302999999999997,"15min":37.302999999999997},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.8759999999999999,"15min":1.369},"min":{"1min":0.66300000000000003,"5min":0.33000000000000002,"15min":0.316},"max":{"1min":1.4330000000000001,"5min":16.867999999999999,"15min":16.867999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.909,"15min":1.383},"min":{"1min":0.73499999999999999,"5min":0.28000000000000003,"15min":0.23000000000000001},"max":{"1min":1.355,"5min":18.335000000000001,"15min":18.335000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.089,"5min":2.085,"15min":1.397},"min":{"1min":0.76000000000000001,"5min":0.33200000000000002,"15min":0.28100000000000003},"max":{"1min":1.268,"5min":19.244,"15min":19.244},"last":1.054}]}]},{"osd":6,"up_from":13,"seq":55834575222,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673216,"kb_used_data":11768,"kb_used_omap":1,"kb_used_meta":661438,"kb_avail":93698624,"statfs":{"total":96636764160,"available":95947390976,"internally_reserved":0,"allocated":12050432,"data_stored":11645255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":677312968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0329999999999999,"15min":0.89300000000000002},"min":{"1min":0.66500000000000004,"5min":0.222,"15min":0.222},"max":{"1min":1.1910000000000001,"5min":6.335,"15min":6.335},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.873,"5min":1.044,"15min":0.91700000000000004},"min":{"1min":0.61299999999999999,"5min":0.24199999999999999,"15min":0.20799999999999999},"max":{"1min":1.1619999999999999,"5min":4.907,"15min":5.9459999999999997},"last":1.024}]},{"osd":1,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.149,"15min":0.95499999999999996},"min":{"1min":0.59599999999999997,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.24,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.679,"15min":1.0960000000000001},"min":{"1min":0.66500000000000004,"5min":0.248,"15min":0.248},"max":{"1min":1.069,"5min":10.111000000000001,"15min":10.111000000000001},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.2270000000000001,"15min":0.98299999999999998},"min":{"1min":0.68600000000000005,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.248,"5min":15.638,"15min":15.638},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3680000000000001,"15min":1.095},"min":{"1min":0.77800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":1.1719999999999999,"5min":19.442,"15min":19.442},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.3759999999999999,"15min":1.1319999999999999},"min":{"1min":0.70499999999999996,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":1.216,"5min":15.247999999999999,"15min":15.247999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.03,"5min":1.4730000000000001,"15min":1.167},"min":{"1min":0.79900000000000004,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1759999999999999,"5min":15.179,"15min":15.179},"last":1.0680000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.4410000000000001,"15min":1.109},"min":{"1min":0.66200000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.1220000000000001,"5min":15.326000000000001,"15min":15.326000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.5229999999999999,"15min":1.1970000000000001},"min":{"1min":0.72399999999999998,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":1.278,"5min":15.026,"15min":15.026},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.494,"15min":1.246},"min":{"1min":0.88200000000000001,"5min":0.32500000000000001,"15min":0.28899999999999998},"max":{"1min":1.256,"5min":9.6080000000000005,"15min":11.566000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.325,"15min":1.1879999999999999},"min":{"1min":0.65500000000000003,"5min":0.308,"15min":0.308},"max":{"1min":1.2889999999999999,"5min":10.223000000000001,"15min":10.223000000000001},"last":1.1879999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.462,"15min":1.238},"min":{"1min":0.84499999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.266,"5min":9.8960000000000008,"15min":13.112},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.5840000000000001,"15min":1.274},"min":{"1min":0.80800000000000005,"5min":0.33200000000000002,"15min":0.245},"max":{"1min":1.331,"5min":14.4,"15min":14.4},"last":1.1599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575223,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":957788,"kb_used_data":5012,"kb_used_omap":1,"kb_used_meta":952766,"kb_avail":93414052,"statfs":{"total":96636764160,"available":95655989248,"internally_reserved":0,"allocated":5132288,"data_stored":4730805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":975632838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.99,"15min":1.2030000000000001},"min":{"1min":0.65900000000000003,"5min":0.215,"15min":0.215},"max":{"1min":1.129,"5min":27.952999999999999,"15min":27.952999999999999},"last":1},{"interface":"front","average":{"1min":1.052,"5min":1.617,"15min":1.1339999999999999},"min":{"1min":0.84199999999999997,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.319,"5min":23.619,"15min":23.619},"last":0.96699999999999997}]},{"osd":1,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.7749999999999999,"15min":1.1399999999999999},"min":{"1min":0.63800000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.1479999999999999,"5min":13.515000000000001,"15min":13.515000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":1.875,"15min":1.1890000000000001},"min":{"1min":0.628,"5min":0.25700000000000001,"15min":0.248},"max":{"1min":1.1020000000000001,"5min":35.186999999999998,"15min":35.186999999999998},"last":0.89000000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.694,"15min":1.3440000000000001},"min":{"1min":0.70099999999999996,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2310000000000001,"5min":11.609,"15min":51.18},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.395,"15min":1.296},"min":{"1min":0.61499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1759999999999999,"5min":8.282,"15min":48.756},"last":1.0289999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.8740000000000001,"15min":1.2809999999999999},"min":{"1min":0.69399999999999995,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1419999999999999,"5min":19.800000000000001,"15min":19.800000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.6679999999999999,"15min":1.258},"min":{"1min":0.76300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2270000000000001,"5min":9.4789999999999992,"15min":9.4789999999999992},"last":0.93300000000000005}]},{"osd":4,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":2.4159999999999999,"15min":1.3839999999999999},"min":{"1min":0.56399999999999995,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1879999999999999,"5min":22.361999999999998,"15min":22.361999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.999,"5min":2.6480000000000001,"15min":1.6439999999999999},"min":{"1min":0.753,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2669999999999999,"5min":28.062999999999999,"15min":31.843},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.403,"15min":1.1299999999999999},"min":{"1min":0.65000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.383,"5min":8.1910000000000007,"15min":8.1910000000000007},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.7310000000000001,"15min":1.2689999999999999},"min":{"1min":0.68799999999999994,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.3,"5min":13.031000000000001,"15min":13.031000000000001},"last":1.0800000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.98,"15min":1.4219999999999999},"min":{"1min":0.73299999999999998,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":1.345,"5min":35.124000000000002,"15min":35.124000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.181,"5min":2.2730000000000001,"15min":1.552},"min":{"1min":0.91800000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.373,"5min":32.328000000000003,"15min":32.328000000000003},"last":1.1060000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575223,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1011352,"kb_used_data":13968,"kb_used_omap":1,"kb_used_meta":997374,"kb_avail":93360488,"statfs":{"total":96636764160,"available":95601139712,"internally_reserved":0,"allocated":14303232,"data_stored":13890488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1021311430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":2.4870000000000001,"5min":1.8520000000000001,"15min":1.155},"min":{"1min":0.47399999999999998,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":18.841000000000001,"5min":18.841000000000001,"15min":18.841000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":2.6619999999999999,"5min":1.9570000000000001,"15min":1.2609999999999999},"min":{"1min":0.33800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":20.312000000000001,"5min":20.312000000000001,"15min":20.312000000000001},"last":0.97999999999999998}]},{"osd":1,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":1.7030000000000001,"15min":1.1020000000000001},"min":{"1min":0.27100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.0840000000000001,"5min":27.242000000000001,"15min":27.242000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":2.1850000000000001,"5min":2.2829999999999999,"15min":1.3220000000000001},"min":{"1min":0.39100000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":14.103999999999999,"5min":26.545999999999999,"15min":26.545999999999999},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.5880000000000001,"15min":1.1080000000000001},"min":{"1min":0.29199999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.8839999999999999,"5min":11.803000000000001,"15min":11.803000000000001},"last":1.137},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.825,"15min":1.2210000000000001},"min":{"1min":0.55300000000000005,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":13.972,"5min":13.972,"15min":13.972},"last":1}]},{"osd":3,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.917,"5min":2.3969999999999998,"15min":1.409},"min":{"1min":0.39900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":10.763,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.874,"5min":1.9550000000000001,"15min":1.2909999999999999},"min":{"1min":0.33200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.2,"5min":20.468,"15min":20.468},"last":0.94899999999999995}]},{"osd":5,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":2.645,"5min":1.796,"15min":1.2010000000000001},"min":{"1min":0.42299999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":14.031000000000001,"5min":14.031000000000001,"15min":14.031000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":2.4180000000000001,"5min":2.1339999999999999,"15min":1.363},"min":{"1min":0.41899999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":14.205,"5min":15.044,"15min":15.044},"last":1.0580000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.546,"15min":1.155},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.7160000000000002,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":0.79500000000000004},{"interface":"front","average":{"1min":2.2970000000000002,"5min":1.911,"15min":1.3169999999999999},"min":{"1min":0.45100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":14.131,"5min":14.131,"15min":14.131},"last":1.196}]},{"osd":7,"last update":"Fri Sep 13 17:17:56 2024","interfaces":[{"interface":"back","average":{"1min":3.056,"5min":2.2269999999999999,"15min":1.478},"min":{"1min":0.36899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":20.154,"5min":24.518999999999998,"15min":24.518999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.984,"5min":1.851,"15min":1.3640000000000001},"min":{"1min":0.32600000000000001,"5min":0.312,"15min":0.312},"max":{"1min":15.209,"5min":15.209,"15min":15.209},"last":1.0309999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575223,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":958484,"kb_used_data":19404,"kb_used_omap":1,"kb_used_meta":939070,"kb_avail":93413356,"statfs":{"total":96636764160,"available":95655276544,"internally_reserved":0,"allocated":19869696,"data_stored":19459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":961608136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.5509999999999999,"15min":0.96299999999999997},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.17599999999999999},"max":{"1min":0.91700000000000004,"5min":19.327999999999999,"15min":19.327999999999999},"last":0.753},{"interface":"front","average":{"1min":0.72699999999999998,"5min":1.7809999999999999,"15min":1.044},"min":{"1min":0.45600000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1,"5min":20.699999999999999,"15min":20.699999999999999},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":1.2949999999999999,"15min":0.90000000000000002},"min":{"1min":0.32200000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.048,"5min":16.100000000000001,"15min":16.100000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":1.2509999999999999,"15min":0.95499999999999996},"min":{"1min":0.32700000000000001,"5min":0.245,"15min":0.22900000000000001},"max":{"1min":1.121,"5min":10.417,"15min":10.417},"last":0.95299999999999996}]},{"osd":2,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":1.2909999999999999,"15min":0.96799999999999997},"min":{"1min":0.40799999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.387,"5min":8.5950000000000006,"15min":8.5950000000000006},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.1339999999999999,"15min":0.96399999999999997},"min":{"1min":0.39000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.274,"5min":10.737,"15min":15.433999999999999},"last":0.99399999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":2.028,"15min":1.3320000000000001},"min":{"1min":0.56699999999999995,"5min":0.308,"15min":0.29599999999999999},"max":{"1min":1.3140000000000001,"5min":15.919,"15min":15.919},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.6559999999999999,"15min":1.21},"min":{"1min":0.60899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5029999999999999,"5min":9.3819999999999997,"15min":9.3819999999999997},"last":1.034}]},{"osd":5,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.802,"15min":1.3009999999999999},"min":{"1min":0.67800000000000005,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.5900000000000001,"5min":31.187999999999999,"15min":31.187999999999999},"last":1.405},{"interface":"front","average":{"1min":1.097,"5min":2.077,"15min":1.383},"min":{"1min":0.63700000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.5580000000000001,"5min":22.951000000000001,"15min":22.951000000000001},"last":1.1890000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.849,"15min":1.3100000000000001},"min":{"1min":0.71599999999999997,"5min":0.34899999999999998,"15min":0.31900000000000001},"max":{"1min":1.619,"5min":12.567,"15min":12.567},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.95,"15min":1.4550000000000001},"min":{"1min":0.61799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5109999999999999,"5min":15.882,"15min":15.882},"last":1.4590000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":2.2589999999999999,"15min":1.5389999999999999},"min":{"1min":0.73699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6080000000000001,"5min":33.969000000000001,"15min":33.969000000000001},"last":1.246},{"interface":"front","average":{"1min":1.1619999999999999,"5min":2.0510000000000002,"15min":1.458},"min":{"1min":0.63700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.5489999999999999,"5min":25.652999999999999,"15min":25.652999999999999},"last":1.27}]}]},{"osd":2,"up_from":13,"seq":55834575223,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662284,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":661374,"kb_avail":93709556,"statfs":{"total":96636764160,"available":95958585344,"internally_reserved":0,"allocated":921600,"data_stored":527729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":677247430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.90800000000000003,"15min":0.72499999999999998},"min":{"1min":0.28499999999999998,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":3.6509999999999998,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.516,"15min":1.0329999999999999},"min":{"1min":0.26300000000000001,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":6.6539999999999999,"5min":9.2729999999999997,"15min":9.2729999999999997},"last":0.79100000000000004}]},{"osd":1,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":1.7110000000000001,"15min":1.0700000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20000000000000001},"max":{"1min":2.7450000000000001,"5min":33.579999999999998,"15min":33.579999999999998},"last":0.625},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.7310000000000001,"15min":1.119},"min":{"1min":0.35199999999999998,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":3.7029999999999998,"5min":23.965,"15min":23.965},"last":0.93899999999999995}]},{"osd":3,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":2.2069999999999999,"15min":1.3049999999999999},"min":{"1min":0.26100000000000001,"5min":0.247,"15min":0.245},"max":{"1min":10.137,"5min":22.760000000000002,"15min":22.760000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.738,"15min":1.127},"min":{"1min":0.28799999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":8.4849999999999994,"5min":10.372,"15min":10.372},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.9850000000000001,"15min":1.2909999999999999},"min":{"1min":0.35799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":2.6400000000000001,"5min":20.673999999999999,"15min":20.673999999999999},"last":1.004},{"interface":"front","average":{"1min":1.395,"5min":1.7410000000000001,"15min":1.2709999999999999},"min":{"1min":0.54400000000000004,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.6699999999999999,"5min":16.5,"15min":16.5},"last":1.179}]},{"osd":5,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":2.484,"15min":1.528},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.798,"5min":21.722000000000001,"15min":21.722000000000001},"last":1.05},{"interface":"front","average":{"1min":1.2050000000000001,"5min":2.222,"15min":1.427},"min":{"1min":0.53800000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":4.9189999999999996,"5min":29.620999999999999,"15min":29.620999999999999},"last":1.0780000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.702,"15min":1.2889999999999999},"min":{"1min":0.51300000000000001,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":6.5880000000000001,"5min":21.882999999999999,"15min":21.882999999999999},"last":1.3},{"interface":"front","average":{"1min":1.748,"5min":2.2210000000000001,"15min":1.4710000000000001},"min":{"1min":0.34799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.407,"5min":25.274999999999999,"15min":25.274999999999999},"last":1.2290000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:17:57 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":3.2799999999999998,"15min":1.859},"min":{"1min":0.61199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":7.0119999999999996,"5min":43.649000000000001,"15min":43.649000000000001},"last":1.387},{"interface":"front","average":{"1min":1.3400000000000001,"5min":3.1589999999999998,"15min":1.855},"min":{"1min":0.40200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":3.6949999999999998,"5min":27.417999999999999,"15min":27.417999999999999},"last":1.1439999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575223,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798984,"kb_used_data":9024,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572856,"statfs":{"total":96636764160,"available":95818604544,"internally_reserved":0,"allocated":9240576,"data_stored":8832475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":808909254},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.8620000000000001,"15min":1.0800000000000001},"min":{"1min":0.185,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":4.851,"5min":15.81,"15min":15.81},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":1.681,"15min":1.0609999999999999},"min":{"1min":0.188,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":2.8319999999999999,"5min":15.823,"15min":15.823},"last":0.90800000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.411,"15min":0.92200000000000004},"min":{"1min":0.215,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":3.8119999999999998,"5min":11.672000000000001,"15min":11.672000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.2929999999999999,"15min":0.95299999999999996},"min":{"1min":0.32200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":4.6890000000000001,"5min":11.722,"15min":11.722},"last":1.2}]},{"osd":3,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":2.024,"15min":1.2190000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.22},"max":{"1min":6.6379999999999999,"5min":16.507999999999999,"15min":16.507999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.482,"15min":1.0169999999999999},"min":{"1min":0.34799999999999998,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":4.3440000000000003,"5min":12.047000000000001,"15min":12.047000000000001},"last":1.0289999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":2.1299999999999999,"15min":1.27},"min":{"1min":0.35899999999999999,"5min":0.27800000000000002,"15min":0.24399999999999999},"max":{"1min":3.7530000000000001,"5min":22.234000000000002,"15min":22.234000000000002},"last":1.657},{"interface":"front","average":{"1min":1.431,"5min":2.258,"15min":1.403},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":8.1240000000000006,"5min":20.448,"15min":20.448},"last":1.4450000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.679,"15min":1.2010000000000001},"min":{"1min":0.44900000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":4.3090000000000002,"5min":15.788,"15min":15.788},"last":1.522},{"interface":"front","average":{"1min":2.5659999999999998,"5min":2.1709999999999998,"15min":1.359},"min":{"1min":0.33400000000000002,"5min":0.312,"15min":0.312},"max":{"1min":24.763999999999999,"5min":24.763999999999999,"15min":24.763999999999999},"last":1.403}]},{"osd":6,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.3879999999999999,"15min":1.139},"min":{"1min":0.43099999999999999,"5min":0.255,"15min":0.255},"max":{"1min":2.7200000000000002,"5min":12.019,"15min":12.019},"last":1.629},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.857,"15min":1.272},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":3.7970000000000002,"5min":11.628,"15min":11.628},"last":1.4730000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:17:52 2024","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":2.02,"15min":1.405},"min":{"1min":0.50600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":6.9850000000000003,"5min":21.161000000000001,"15min":21.161000000000001},"last":1.375},{"interface":"front","average":{"1min":1.2230000000000001,"5min":2.4969999999999999,"15min":1.554},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.1980000000000004,"5min":29.323,"15min":29.323},"last":1.6870000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575223,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799556,"kb_used_data":9596,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572284,"statfs":{"total":96636764160,"available":95818018816,"internally_reserved":0,"allocated":9826304,"data_stored":9422627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":808909256},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.4610000000000001,"15min":0.90800000000000003},"min":{"1min":0.22,"5min":0.20000000000000001,"15min":0.189},"max":{"1min":1.677,"5min":59.031999999999996,"15min":59.031999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":2.0870000000000002,"5min":2.351,"15min":1.2869999999999999},"min":{"1min":0.23499999999999999,"5min":0.188,"15min":0.188},"max":{"1min":9.8800000000000008,"5min":57.186999999999998,"15min":57.186999999999998},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.96,"5min":1.155,"15min":0.871},"min":{"1min":0.26700000000000002,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":12.971,"5min":12.971,"15min":12.971},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.9990000000000001,"5min":1.371,"15min":0.93400000000000005},"min":{"1min":0.253,"5min":0.246,"15min":0.246},"max":{"1min":14.526,"5min":14.526,"15min":14.526},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.101,"5min":2.6800000000000002,"15min":1.4370000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":23.495999999999999,"5min":65.215000000000003,"15min":65.215000000000003},"last":1.02},{"interface":"front","average":{"1min":1.373,"5min":1.8169999999999999,"15min":1.1479999999999999},"min":{"1min":0.435,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":6.3479999999999999,"5min":62.722999999999999,"15min":62.722999999999999},"last":0.83999999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.74,"5min":1.972,"15min":1.2569999999999999},"min":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":14.464,"5min":14.694000000000001,"15min":14.694000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.3320000000000001,"15min":1.069},"min":{"1min":0.29699999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.698,"5min":10.036,"15min":10.036},"last":1.048}]},{"osd":5,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":2.145,"5min":1.9450000000000001,"15min":1.2689999999999999},"min":{"1min":0.36699999999999999,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":10.769,"5min":21.940999999999999,"15min":21.940999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.889,"5min":1.996,"15min":1.3620000000000001},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":9.8200000000000003,"5min":9.8200000000000003,"15min":9.8200000000000003},"last":1.093}]},{"osd":6,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":1.4410000000000001,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":10.819000000000001,"5min":10.819000000000001,"15min":10.819000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.8600000000000001,"5min":1.5900000000000001,"15min":1.2130000000000001},"min":{"1min":0.40400000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":10.741,"5min":10.741,"15min":10.741},"last":0.89000000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:17:46 2024","interfaces":[{"interface":"back","average":{"1min":3.0329999999999999,"5min":1.9139999999999999,"15min":1.3600000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":32.975999999999999,"5min":32.975999999999999,"15min":32.975999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":2.468,"5min":2.7930000000000001,"15min":1.6830000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.40300000000000002},"max":{"1min":17.484000000000002,"5min":32.069000000000003,"15min":32.069000000000003},"last":0.93500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":84646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8482816,"data_stored":8473926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7020544,"data_stored":7006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7016448,"data_stored":7007322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10542867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4296704,"data_stored":4287702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:18:51.180 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.181 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.181 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.181 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.181 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.181 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.181 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.182 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.182 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.182 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.182 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.182 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.182 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.182 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.182 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.183 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.183 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.183 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.183 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.183 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.183 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.183 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.183 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.184 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.184 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.184 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.184 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.184 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.184 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.184 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.185 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.185 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.185 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.185 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.185 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.185 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.185 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.186 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.186 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.186 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.186 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.186 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.186 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.186 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.187 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T16:48:06.713484+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=6, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.187 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.187 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.187 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.188 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-13T16:47:57.619902+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=57, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.188 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.188 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.188 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.188 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.188 INFO:tasks.ceph:pgid 5.22 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.188 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.189 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.189 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.189 INFO:tasks.ceph:pgid 5.21 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.189 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.189 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.189 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.189 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.189 INFO:tasks.ceph:pgid 5.2a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.190 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.190 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.190 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.190 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.190 INFO:tasks.ceph:pgid 5.36 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.191 INFO:tasks.ceph:pgid 5.37 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.191 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.191 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.191 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.191 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.192 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.192 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.192 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.192 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:18:51.192 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T17:19:06.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:19:06.991 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:19:11.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:19:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.328+0000 7f13e7ce9640 1 Processor -- start 2024-09-13T17:19:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.328+0000 7f13e7ce9640 1 -- start start 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e7ce9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f13e006c870 0x7f13e00aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e7ce9640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f13e00c40b0 0x7f13e015c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e7ce9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e0152a80 con 0x7f13e00aabc0 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e7ce9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e00c30f0 con 0x7f13e006c870 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e7ce9640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e0151e00 con 0x7f13e00c40b0 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e5284640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f13e006c870 0x7f13e00aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e5a85640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f13e00aabc0 0x7f13e00c39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:57854/0 (socket says 172.21.15.39:57854) 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e5a85640 1 -- 172.21.15.39:0/3517546638 learned_addr learned my addr 172.21.15.39:0/3517546638 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e6286640 1 --2- 172.21.15.39:0/3517546638 >> v2:172.21.15.84:3300/0 conn(0x7f13e00c40b0 0x7f13e015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e5284640 1 -- 172.21.15.39:0/3517546638 >> v2:172.21.15.84:3300/0 conn(0x7f13e00c40b0 msgr2=0x7f13e015c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:11.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e5284640 1 --2- 172.21.15.39:0/3517546638 >> v2:172.21.15.84:3300/0 conn(0x7f13e00c40b0 0x7f13e015c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e5284640 1 -- 172.21.15.39:0/3517546638 >> v1:172.21.15.39:6789/0 conn(0x7f13e00aabc0 legacy=0x7f13e00c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:11.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e5284640 1 -- 172.21.15.39:0/3517546638 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e01520e0 con 0x7f13e006c870 2024-09-13T17:19:11.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e4a83640 1 -- 172.21.15.39:0/3517546638 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754127994 0 0) 0x7f13d00014a0 con 0x7f13e00aabc0 2024-09-13T17:19:11.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e6286640 1 --2- 172.21.15.39:0/3517546638 >> v2:172.21.15.84:3300/0 conn(0x7f13e00c40b0 0x7f13e015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:19:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e5284640 1 --2- 172.21.15.39:0/3517546638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e006c870 0x7f13e00aafa0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f13e006f220 tx=0x7f13d401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c6fd57932c5f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e4a83640 1 -- 172.21.15.39:0/3517546638 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d40ac620 con 0x7f13e006c870 2024-09-13T17:19:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e4a83640 1 -- 172.21.15.39:0/3517546638 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d40ac780 con 0x7f13e006c870 2024-09-13T17:19:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.332+0000 7f13e4a83640 1 -- 172.21.15.39:0/3517546638 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d40acb10 con 0x7f13e006c870 2024-09-13T17:19:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- 172.21.15.39:0/3517546638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e006c870 msgr2=0x7f13e00aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 --2- 172.21.15.39:0/3517546638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e006c870 0x7f13e00aafa0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f13e006f220 tx=0x7f13d401c3c0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- 172.21.15.39:0/3517546638 shutdown_connections 2024-09-13T17:19:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 --2- 172.21.15.39:0/3517546638 >> v2:172.21.15.84:3300/0 conn(0x7f13e00c40b0 0x7f13e015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 --2- 172.21.15.39:0/3517546638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e006c870 0x7f13e00aafa0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- 172.21.15.39:0/3517546638 >> 172.21.15.39:0/3517546638 conn(0x7f13e0069e10 msgr2=0x7f13e006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- 172.21.15.39:0/3517546638 shutdown_connections 2024-09-13T17:19:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- 172.21.15.39:0/3517546638 wait complete. 2024-09-13T17:19:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 Processor -- start 2024-09-13T17:19:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- start start 2024-09-13T17:19:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e006c870 0x7f13e00a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e00aabc0 0x7f13e0150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13e00c40b0 0x7f13e014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0149800 con 0x7f13e00aabc0 2024-09-13T17:19:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0149ac0 con 0x7f13e006c870 2024-09-13T17:19:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e0154f60 con 0x7f13e00c40b0 2024-09-13T17:19:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5284640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e00aabc0 0x7f13e0150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5284640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e00aabc0 0x7f13e0150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58388/0 (socket says 172.21.15.39:58388) 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5284640 1 -- 172.21.15.39:0/876100486 learned_addr learned my addr 172.21.15.39:0/876100486 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5a85640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e006c870 0x7f13e00a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e6286640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13e00c40b0 0x7f13e014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5284640 1 -- 172.21.15.39:0/876100486 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13e00c40b0 msgr2=0x7f13e014b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5284640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13e00c40b0 0x7f13e014b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5284640 1 -- 172.21.15.39:0/876100486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e006c870 msgr2=0x7f13e00a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5284640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e006c870 0x7f13e00a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5284640 1 -- 172.21.15.39:0/876100486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4009420 con 0x7f13e00aabc0 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e5284640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e00aabc0 0x7f13e0150d40 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f13d40af040 tx=0x7f13d4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13de7fc640 1 -- 172.21.15.39:0/876100486 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d40b5610 con 0x7f13e00aabc0 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13de7fc640 1 -- 172.21.15.39:0/876100486 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d40b5770 con 0x7f13e00aabc0 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e014a0a0 con 0x7f13e00aabc0 2024-09-13T17:19:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13de7fc640 1 -- 172.21.15.39:0/876100486 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d40b5b20 con 0x7f13e00aabc0 2024-09-13T17:19:11.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.336+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e0151280 con 0x7f13e00aabc0 2024-09-13T17:19:11.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.340+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e00a6760 con 0x7f13e00aabc0 2024-09-13T17:19:11.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.340+0000 7f13de7fc640 1 -- 172.21.15.39:0/876100486 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f13d40b3040 con 0x7f13e00aabc0 2024-09-13T17:19:11.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.340+0000 7f13de7fc640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13cc066690 0x7f13cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:11.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.340+0000 7f13de7fc640 1 -- 172.21.15.39:0/876100486 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f13d411f640 con 0x7f13e00aabc0 2024-09-13T17:19:11.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.340+0000 7f13e5a85640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13cc066690 0x7f13cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:11.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.340+0000 7f13e5a85640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13cc066690 0x7f13cc068b50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f13d00012f0 tx=0x7f13d0007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:11.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.344+0000 7f13de7fc640 1 -- 172.21.15.39:0/876100486 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f13e00a6760 con 0x7f13e00aabc0 2024-09-13T17:19:11.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.504+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13e00a4fe0 con 0x7f13cc066690 2024-09-13T17:19:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.540+0000 7f13de7fc640 1 -- 172.21.15.39:0/876100486 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+530986 (secure 0 0 0) 0x7f13d0004b70 con 0x7f13cc066690 2024-09-13T17:19:11.550 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:19:11.554 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T17:19:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13cc066690 msgr2=0x7f13cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13cc066690 0x7f13cc068b50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f13d00012f0 tx=0x7f13d0007d40 comp rx=0 tx=0).stop 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e00aabc0 msgr2=0x7f13e0150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e00aabc0 0x7f13e0150d40 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f13d40af040 tx=0x7f13d4019160 comp rx=0 tx=0).stop 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 shutdown_connections 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f13cc066690 0x7f13cc068b50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f13e00c40b0 0x7f13e014b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13e00aabc0 0x7f13e0150d40 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 --2- 172.21.15.39:0/876100486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e006c870 0x7f13e00a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 >> 172.21.15.39:0/876100486 conn(0x7f13e0069e10 msgr2=0x7f13e014ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 shutdown_connections 2024-09-13T17:19:11.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:11.548+0000 7f13e7ce9640 1 -- 172.21.15.39:0/876100486 wait complete. 2024-09-13T17:19:11.575 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":998,"stamp":"2024-09-13T17:19:10.388178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39291023,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":3153,"num_read_kb":416667,"num_write":2818784,"num_write_kb":37531913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522572,"ondisk_log_size":522572,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6804448,"kb_used_data":83360,"kb_used_omap":12,"kb_used_meta":6721011,"kb_avail":748170272,"statfs":{"total":773094113280,"available":766126358528,"internally_reserved":0,"allocated":85360640,"data_stored":82126150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":6882315835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005977"},"pg_stats":[{"pgid":"5.18","version":"63'4385","reported_seq":4460,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:56.211163+0000","last_change":"2024-09-13T17:18:56.211391+0000","last_active":"2024-09-13T17:18:56.211163+0000","last_peered":"2024-09-13T17:18:56.211163+0000","last_clean":"2024-09-13T17:18:56.211163+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T17:18:56.211163+0000","last_undegraded":"2024-09-13T17:18:56.211163+0000","last_fullsized":"2024-09-13T17:18:56.211163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4385","last_scrub_stamp":"2024-09-13T17:18:56.211105+0000","last_deep_scrub":"63'4385","last_deep_scrub_stamp":"2024-09-13T17:18:56.211105+0000","last_clean_scrub_stamp":"2024-09-13T17:18:56.211105+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:28.905686+0000","scrub_duration":0.034523294000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":23692,"num_write_kb":324330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"64'919","reported_seq":1044,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.775073+0000","last_change":"2024-09-13T17:18:10.775401+0000","last_active":"2024-09-13T17:18:10.775073+0000","last_peered":"2024-09-13T17:18:10.775073+0000","last_clean":"2024-09-13T17:18:10.775073+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T17:18:10.775073+0000","last_undegraded":"2024-09-13T17:18:10.775073+0000","last_fullsized":"2024-09-13T17:18:10.775073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'919","last_scrub_stamp":"2024-09-13T17:18:10.775003+0000","last_deep_scrub":"64'919","last_deep_scrub_stamp":"2024-09-13T17:18:10.775003+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.775003+0000","objects_scrubbed":3,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:00.271443+0000","scrub_duration":0.030271704,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":10389,"num_write":894,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'3737","reported_seq":3818,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.006249+0000","last_change":"2024-09-13T17:19:05.006483+0000","last_active":"2024-09-13T17:19:05.006249+0000","last_peered":"2024-09-13T17:19:05.006249+0000","last_clean":"2024-09-13T17:19:05.006249+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T17:19:05.006249+0000","last_undegraded":"2024-09-13T17:19:05.006249+0000","last_fullsized":"2024-09-13T17:19:05.006249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3737","last_scrub_stamp":"2024-09-13T17:19:05.006193+0000","last_deep_scrub":"63'3737","last_deep_scrub_stamp":"2024-09-13T17:19:05.006193+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.006193+0000","objects_scrubbed":0,"log_size":3737,"log_dups_size":0,"ondisk_log_size":3737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:00.849618+0000","scrub_duration":0.027405445,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20454,"num_write_kb":270405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.082725+0000","last_change":"2024-09-13T17:17:52.082969+0000","last_active":"2024-09-13T17:17:52.082725+0000","last_peered":"2024-09-13T17:17:52.082725+0000","last_clean":"2024-09-13T17:17:52.082725+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T17:17:52.082725+0000","last_undegraded":"2024-09-13T17:17:52.082725+0000","last_fullsized":"2024-09-13T17:17:52.082725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.082672+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.023294+0000","scrub_duration":0.0034166320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"63'4331","reported_seq":4396,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.992385+0000","last_change":"2024-09-13T16:48:08.179079+0000","last_active":"2024-09-13T17:17:44.992385+0000","last_peered":"2024-09-13T17:17:44.992385+0000","last_clean":"2024-09-13T17:17:44.992385+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T17:17:44.992385+0000","last_undegraded":"2024-09-13T17:17:44.992385+0000","last_fullsized":"2024-09-13T17:17:44.992385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":22875,"num_write_kb":325320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"64'24","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:49.931421+0000","last_change":"2024-09-13T17:18:49.931654+0000","last_active":"2024-09-13T17:18:49.931421+0000","last_peered":"2024-09-13T17:18:49.931421+0000","last_clean":"2024-09-13T17:18:49.931421+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T17:18:49.931421+0000","last_undegraded":"2024-09-13T17:18:49.931421+0000","last_fullsized":"2024-09-13T17:18:49.931421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-09-13T17:18:49.931368+0000","last_deep_scrub":"64'24","last_deep_scrub_stamp":"2024-09-13T17:18:49.931368+0000","last_clean_scrub_stamp":"2024-09-13T17:18:49.931368+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:06.061913+0000","scrub_duration":0.0033432700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"63'5559","reported_seq":5639,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:55.943224+0000","last_change":"2024-09-13T17:17:55.943470+0000","last_active":"2024-09-13T17:17:55.943224+0000","last_peered":"2024-09-13T17:17:55.943224+0000","last_clean":"2024-09-13T17:17:55.943224+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T17:17:55.943224+0000","last_undegraded":"2024-09-13T17:17:55.943224+0000","last_fullsized":"2024-09-13T17:17:55.943224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5559","last_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_deep_scrub":"63'5559","last_deep_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_clean_scrub_stamp":"2024-09-13T17:17:55.943168+0000","objects_scrubbed":0,"log_size":5559,"log_dups_size":0,"ondisk_log_size":5559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:10.227939+0000","scrub_duration":0.037150519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30370,"num_write_kb":379865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"63'2","reported_seq":77,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.149583+0000","last_change":"2024-09-13T17:18:46.149840+0000","last_active":"2024-09-13T17:18:46.149583+0000","last_peered":"2024-09-13T17:18:46.149583+0000","last_clean":"2024-09-13T17:18:46.149583+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T17:18:46.149583+0000","last_undegraded":"2024-09-13T17:18:46.149583+0000","last_fullsized":"2024-09-13T17:18:46.149583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:46.149528+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:46.149528+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.149528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:41.450773+0000","scrub_duration":0.003414533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"63'3581","reported_seq":3655,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:04.446149+0000","last_change":"2024-09-13T17:19:04.446383+0000","last_active":"2024-09-13T17:19:04.446149+0000","last_peered":"2024-09-13T17:19:04.446149+0000","last_clean":"2024-09-13T17:19:04.446149+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T17:19:04.446149+0000","last_undegraded":"2024-09-13T17:19:04.446149+0000","last_fullsized":"2024-09-13T17:19:04.446149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3581","last_scrub_stamp":"2024-09-13T17:19:04.446098+0000","last_deep_scrub":"63'3581","last_deep_scrub_stamp":"2024-09-13T17:19:04.446098+0000","last_clean_scrub_stamp":"2024-09-13T17:19:04.446098+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:00.289523+0000","scrub_duration":0.033645664999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":268344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"64'44","reported_seq":161,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:59.331452+0000","last_change":"2024-09-13T17:18:59.331766+0000","last_active":"2024-09-13T17:18:59.331452+0000","last_peered":"2024-09-13T17:18:59.331452+0000","last_clean":"2024-09-13T17:18:59.331452+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T17:18:59.331452+0000","last_undegraded":"2024-09-13T17:18:59.331452+0000","last_fullsized":"2024-09-13T17:18:59.331452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'44","last_scrub_stamp":"2024-09-13T17:18:59.331399+0000","last_deep_scrub":"64'44","last_deep_scrub_stamp":"2024-09-13T17:18:59.331399+0000","last_clean_scrub_stamp":"2024-09-13T17:18:59.331399+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:00.438654+0000","scrub_duration":0.0059461519999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'3493","reported_seq":3562,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.642622+0000","last_change":"2024-09-13T16:47:59.051924+0000","last_active":"2024-09-13T17:17:48.642622+0000","last_peered":"2024-09-13T17:17:48.642622+0000","last_clean":"2024-09-13T17:17:48.642622+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T17:17:48.642622+0000","last_undegraded":"2024-09-13T17:17:48.642622+0000","last_fullsized":"2024-09-13T17:17:48.642622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:47:58.630418+0000","last_clean_scrub_stamp":"2024-09-13T16:47:58.630418+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18921,"num_write_kb":256078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'727","reported_seq":827,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.186917+0000","last_change":"2024-09-13T17:17:57.186917+0000","last_active":"2024-09-13T17:17:57.186917+0000","last_peered":"2024-09-13T17:17:57.186917+0000","last_clean":"2024-09-13T17:17:57.186917+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T17:17:57.186917+0000","last_undegraded":"2024-09-13T17:17:57.186917+0000","last_fullsized":"2024-09-13T17:17:57.186917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'727","last_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_deep_scrub":"64'727","last_deep_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.186269+0000","objects_scrubbed":1,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:00.928968+0000","scrub_duration":0.026043038000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":75389,"num_write":714,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"63'3135","reported_seq":3209,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:42.600862+0000","last_change":"2024-09-13T17:18:42.601105+0000","last_active":"2024-09-13T17:18:42.600862+0000","last_peered":"2024-09-13T17:18:42.600862+0000","last_clean":"2024-09-13T17:18:42.600862+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T17:18:42.600862+0000","last_undegraded":"2024-09-13T17:18:42.600862+0000","last_fullsized":"2024-09-13T17:18:42.600862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3135","last_scrub_stamp":"2024-09-13T17:18:42.600803+0000","last_deep_scrub":"63'3135","last_deep_scrub_stamp":"2024-09-13T17:18:42.600803+0000","last_clean_scrub_stamp":"2024-09-13T17:18:42.600803+0000","objects_scrubbed":0,"log_size":3135,"log_dups_size":0,"ondisk_log_size":3135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:53.145621+0000","scrub_duration":0.022581403999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":232447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"64'6","reported_seq":123,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:50.338514+0000","last_change":"2024-09-13T17:18:50.338743+0000","last_active":"2024-09-13T17:18:50.338514+0000","last_peered":"2024-09-13T17:18:50.338514+0000","last_clean":"2024-09-13T17:18:50.338514+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T17:18:50.338514+0000","last_undegraded":"2024-09-13T17:18:50.338514+0000","last_fullsized":"2024-09-13T17:18:50.338514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-09-13T17:18:50.338459+0000","last_deep_scrub":"64'6","last_deep_scrub_stamp":"2024-09-13T17:18:50.338459+0000","last_clean_scrub_stamp":"2024-09-13T17:18:50.338459+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:06.469004+0000","scrub_duration":0.0063925359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"63'5195","reported_seq":5273,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.511870+0000","last_change":"2024-09-13T17:17:49.511870+0000","last_active":"2024-09-13T17:17:49.511870+0000","last_peered":"2024-09-13T17:17:49.511870+0000","last_clean":"2024-09-13T17:17:49.511870+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T17:17:49.511870+0000","last_undegraded":"2024-09-13T17:17:49.511870+0000","last_fullsized":"2024-09-13T17:17:49.511870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5195","last_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_deep_scrub":"63'5195","last_deep_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.511449+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:30.737466+0000","scrub_duration":0.041661353999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29369,"num_write_kb":359470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"64'211","reported_seq":286,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.156875+0000","last_change":"2024-09-13T17:18:44.157099+0000","last_active":"2024-09-13T17:18:44.156875+0000","last_peered":"2024-09-13T17:18:44.156875+0000","last_clean":"2024-09-13T17:18:44.156875+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T17:18:44.156875+0000","last_undegraded":"2024-09-13T17:18:44.156875+0000","last_fullsized":"2024-09-13T17:18:44.156875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'211","last_scrub_stamp":"2024-09-13T17:18:44.156820+0000","last_deep_scrub":"64'211","last_deep_scrub_stamp":"2024-09-13T17:18:44.156820+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.156820+0000","objects_scrubbed":1,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:21.506931+0000","scrub_duration":0.018253704999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153565,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"63'3570","reported_seq":3636,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.996947+0000","last_change":"2024-09-13T16:48:08.200159+0000","last_active":"2024-09-13T17:17:44.996947+0000","last_peered":"2024-09-13T17:17:44.996947+0000","last_clean":"2024-09-13T17:17:44.996947+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T17:17:44.996947+0000","last_undegraded":"2024-09-13T17:17:44.996947+0000","last_fullsized":"2024-09-13T17:17:44.996947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":19362,"num_write_kb":282506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"63'4274","reported_seq":4354,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.521451+0000","last_change":"2024-09-13T17:18:01.521451+0000","last_active":"2024-09-13T17:18:01.521451+0000","last_peered":"2024-09-13T17:18:01.521451+0000","last_clean":"2024-09-13T17:18:01.521451+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T17:18:01.521451+0000","last_undegraded":"2024-09-13T17:18:01.521451+0000","last_fullsized":"2024-09-13T17:18:01.521451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4274","last_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_deep_scrub":"63'4274","last_deep_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.521104+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:51.017545+0000","scrub_duration":0.027199207999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23751,"num_write_kb":312338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"64'1421","reported_seq":1546,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.346755+0000","last_change":"2024-09-13T17:18:46.347010+0000","last_active":"2024-09-13T17:18:46.346755+0000","last_peered":"2024-09-13T17:18:46.346755+0000","last_clean":"2024-09-13T17:18:46.346755+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T17:18:46.346755+0000","last_undegraded":"2024-09-13T17:18:46.346755+0000","last_fullsized":"2024-09-13T17:18:46.346755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1421","last_scrub_stamp":"2024-09-13T17:18:46.346701+0000","last_deep_scrub":"64'1421","last_deep_scrub_stamp":"2024-09-13T17:18:46.346701+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.346701+0000","objects_scrubbed":2,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:47.745552+0000","scrub_duration":0.031375147999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":42245,"num_write":1396,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'7","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.140201+0000","last_change":"2024-09-13T17:18:01.140415+0000","last_active":"2024-09-13T17:18:01.140201+0000","last_peered":"2024-09-13T17:18:01.140201+0000","last_clean":"2024-09-13T17:18:01.140201+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T17:18:01.140201+0000","last_undegraded":"2024-09-13T17:18:01.140201+0000","last_fullsized":"2024-09-13T17:18:01.140201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'7","last_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_deep_scrub":"64'7","last_deep_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.140142+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:50.636583+0000","scrub_duration":0.0071129380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"63'4212","reported_seq":4290,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:32.577261+0000","last_change":"2024-09-13T17:18:32.577494+0000","last_active":"2024-09-13T17:18:32.577261+0000","last_peered":"2024-09-13T17:18:32.577261+0000","last_clean":"2024-09-13T17:18:32.577261+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T17:18:32.577261+0000","last_undegraded":"2024-09-13T17:18:32.577261+0000","last_fullsized":"2024-09-13T17:18:32.577261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4212","last_scrub_stamp":"2024-09-13T17:18:32.577202+0000","last_deep_scrub":"63'4212","last_deep_scrub_stamp":"2024-09-13T17:18:32.577202+0000","last_clean_scrub_stamp":"2024-09-13T17:18:32.577202+0000","objects_scrubbed":0,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:31.849084+0000","scrub_duration":0.024339607999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":17,"num_write":23343,"num_write_kb":314959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"63'3296","reported_seq":3375,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.980218+0000","last_change":"2024-09-13T17:19:05.980458+0000","last_active":"2024-09-13T17:19:05.980218+0000","last_peered":"2024-09-13T17:19:05.980218+0000","last_clean":"2024-09-13T17:19:05.980218+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T17:19:05.980218+0000","last_undegraded":"2024-09-13T17:19:05.980218+0000","last_fullsized":"2024-09-13T17:19:05.980218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3296","last_scrub_stamp":"2024-09-13T17:19:05.980161+0000","last_deep_scrub":"63'3296","last_deep_scrub_stamp":"2024-09-13T17:19:05.980161+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.980161+0000","objects_scrubbed":0,"log_size":3296,"log_dups_size":0,"ondisk_log_size":3296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:26.138684+0000","scrub_duration":0.023800585999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17758,"num_write_kb":230990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'3","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.272530+0000","last_change":"2024-09-13T17:18:44.272747+0000","last_active":"2024-09-13T17:18:44.272530+0000","last_peered":"2024-09-13T17:18:44.272530+0000","last_clean":"2024-09-13T17:18:44.272530+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T17:18:44.272530+0000","last_undegraded":"2024-09-13T17:18:44.272530+0000","last_fullsized":"2024-09-13T17:18:44.272530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:44.272476+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:44.272476+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.272476+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:21.622587+0000","scrub_duration":0.0033040719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.139202+0000","last_change":"2024-09-13T17:18:05.139417+0000","last_active":"2024-09-13T17:18:05.139202+0000","last_peered":"2024-09-13T17:18:05.139202+0000","last_clean":"2024-09-13T17:18:05.139202+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T17:18:05.139202+0000","last_undegraded":"2024-09-13T17:18:05.139202+0000","last_fullsized":"2024-09-13T17:18:05.139202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:05.139147+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:05.139147+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.139147+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:15.079769+0000","scrub_duration":0.0035900110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"63'3371","reported_seq":3447,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:54.182862+0000","last_change":"2024-09-13T17:18:54.183094+0000","last_active":"2024-09-13T17:18:54.182862+0000","last_peered":"2024-09-13T17:18:54.182862+0000","last_clean":"2024-09-13T17:18:54.182862+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T17:18:54.182862+0000","last_undegraded":"2024-09-13T17:18:54.182862+0000","last_fullsized":"2024-09-13T17:18:54.182862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3371","last_scrub_stamp":"2024-09-13T17:18:54.182803+0000","last_deep_scrub":"63'3371","last_deep_scrub_stamp":"2024-09-13T17:18:54.182803+0000","last_clean_scrub_stamp":"2024-09-13T17:18:54.182803+0000","objects_scrubbed":0,"log_size":3371,"log_dups_size":0,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:04.727620+0000","scrub_duration":0.031207789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18156,"num_write_kb":249012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"63'3752","reported_seq":3832,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:59.895440+0000","last_change":"2024-09-13T17:18:59.895722+0000","last_active":"2024-09-13T17:18:59.895440+0000","last_peered":"2024-09-13T17:18:59.895440+0000","last_clean":"2024-09-13T17:18:59.895440+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T17:18:59.895440+0000","last_undegraded":"2024-09-13T17:18:59.895440+0000","last_fullsized":"2024-09-13T17:18:59.895440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3752","last_scrub_stamp":"2024-09-13T17:18:59.895370+0000","last_deep_scrub":"63'3752","last_deep_scrub_stamp":"2024-09-13T17:18:59.895370+0000","last_clean_scrub_stamp":"2024-09-13T17:18:59.895370+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:01.002625+0000","scrub_duration":0.023944989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21127,"num_write_kb":268998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"63'2","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:56.315394+0000","last_change":"2024-09-13T17:18:56.315620+0000","last_active":"2024-09-13T17:18:56.315394+0000","last_peered":"2024-09-13T17:18:56.315394+0000","last_clean":"2024-09-13T17:18:56.315394+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T17:18:56.315394+0000","last_undegraded":"2024-09-13T17:18:56.315394+0000","last_fullsized":"2024-09-13T17:18:56.315394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:56.315340+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:56.315340+0000","last_clean_scrub_stamp":"2024-09-13T17:18:56.315340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:29.009922+0000","scrub_duration":0.003501695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"64'5","reported_seq":102,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:11.257475+0000","last_change":"2024-09-13T17:18:11.257689+0000","last_active":"2024-09-13T17:18:11.257475+0000","last_peered":"2024-09-13T17:18:11.257475+0000","last_clean":"2024-09-13T17:18:11.257475+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T17:18:11.257475+0000","last_undegraded":"2024-09-13T17:18:11.257475+0000","last_fullsized":"2024-09-13T17:18:11.257475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:11.257421+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:11.257421+0000","last_clean_scrub_stamp":"2024-09-13T17:18:11.257421+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:03.763129+0000","scrub_duration":0.0063405759999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"63'4499","reported_seq":4567,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.987188+0000","last_change":"2024-09-13T16:48:08.200334+0000","last_active":"2024-09-13T17:17:44.987188+0000","last_peered":"2024-09-13T17:17:44.987188+0000","last_clean":"2024-09-13T17:17:44.987188+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T17:17:44.987188+0000","last_undegraded":"2024-09-13T17:17:44.987188+0000","last_fullsized":"2024-09-13T17:17:44.987188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":24341,"num_write_kb":343418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"63'4043","reported_seq":4123,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:31.227364+0000","last_change":"2024-09-13T17:18:31.227589+0000","last_active":"2024-09-13T17:18:31.227364+0000","last_peered":"2024-09-13T17:18:31.227364+0000","last_clean":"2024-09-13T17:18:31.227364+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T17:18:31.227364+0000","last_undegraded":"2024-09-13T17:18:31.227364+0000","last_fullsized":"2024-09-13T17:18:31.227364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4043","last_scrub_stamp":"2024-09-13T17:18:31.227305+0000","last_deep_scrub":"63'4043","last_deep_scrub_stamp":"2024-09-13T17:18:31.227305+0000","last_clean_scrub_stamp":"2024-09-13T17:18:31.227305+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:30.499187+0000","scrub_duration":0.032759703000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22005,"num_write_kb":288910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"63'5","reported_seq":86,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:52.326458+0000","last_change":"2024-09-13T17:18:52.326686+0000","last_active":"2024-09-13T17:18:52.326458+0000","last_peered":"2024-09-13T17:18:52.326458+0000","last_clean":"2024-09-13T17:18:52.326458+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T17:18:52.326458+0000","last_undegraded":"2024-09-13T17:18:52.326458+0000","last_fullsized":"2024-09-13T17:18:52.326458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:52.326404+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:52.326404+0000","last_clean_scrub_stamp":"2024-09-13T17:18:52.326404+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:28.371100+0000","scrub_duration":0.0033383689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.171804+0000","last_change":"2024-09-13T17:18:04.172104+0000","last_active":"2024-09-13T17:18:04.171804+0000","last_peered":"2024-09-13T17:18:04.171804+0000","last_clean":"2024-09-13T17:18:04.171804+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T17:18:04.171804+0000","last_undegraded":"2024-09-13T17:18:04.171804+0000","last_fullsized":"2024-09-13T17:18:04.171804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.171747+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:58.830350+0000","scrub_duration":0.0065161840000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"63'3470","reported_seq":3547,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:30.976569+0000","last_change":"2024-09-13T17:18:30.976875+0000","last_active":"2024-09-13T17:18:30.976569+0000","last_peered":"2024-09-13T17:18:30.976569+0000","last_clean":"2024-09-13T17:18:30.976569+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T17:18:30.976569+0000","last_undegraded":"2024-09-13T17:18:30.976569+0000","last_fullsized":"2024-09-13T17:18:30.976569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3470","last_scrub_stamp":"2024-09-13T17:18:30.976507+0000","last_deep_scrub":"63'3470","last_deep_scrub_stamp":"2024-09-13T17:18:30.976507+0000","last_clean_scrub_stamp":"2024-09-13T17:18:30.976507+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:08.326618+0000","scrub_duration":0.024724408999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19441,"num_write_kb":258288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'4844","reported_seq":4925,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:18.899218+0000","last_change":"2024-09-13T17:18:18.899218+0000","last_active":"2024-09-13T17:18:18.899218+0000","last_peered":"2024-09-13T17:18:18.899218+0000","last_clean":"2024-09-13T17:18:18.899218+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T17:18:18.899218+0000","last_undegraded":"2024-09-13T17:18:18.899218+0000","last_fullsized":"2024-09-13T17:18:18.899218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4844","last_scrub_stamp":"2024-09-13T17:18:18.898914+0000","last_deep_scrub":"63'4844","last_deep_scrub_stamp":"2024-09-13T17:18:18.898914+0000","last_clean_scrub_stamp":"2024-09-13T17:18:18.898914+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:18.212032+0000","scrub_duration":0.026998708999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":26635,"num_write_kb":336207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'10","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.498454+0000","last_change":"2024-09-13T17:18:05.498454+0000","last_active":"2024-09-13T17:18:05.498454+0000","last_peered":"2024-09-13T17:18:05.498454+0000","last_clean":"2024-09-13T17:18:05.498454+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T17:18:05.498454+0000","last_undegraded":"2024-09-13T17:18:05.498454+0000","last_fullsized":"2024-09-13T17:18:05.498454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_deep_scrub":"64'10","last_deep_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.498184+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:15.438806+0000","scrub_duration":0.0057060160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'49","reported_seq":189,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.166590+0000","last_change":"2024-09-13T17:17:58.166876+0000","last_active":"2024-09-13T17:17:58.166590+0000","last_peered":"2024-09-13T17:17:58.166590+0000","last_clean":"2024-09-13T17:17:58.166590+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T17:17:58.166590+0000","last_undegraded":"2024-09-13T17:17:58.166590+0000","last_fullsized":"2024-09-13T17:17:58.166590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'49","last_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_deep_scrub":"64'49","last_deep_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.166530+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:05.921781+0000","scrub_duration":0.0073340999999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92726,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":118,"num_write":43,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"63'3635","reported_seq":3701,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974442+0000","last_change":"2024-09-13T16:48:08.192026+0000","last_active":"2024-09-13T17:17:44.974442+0000","last_peered":"2024-09-13T17:17:44.974442+0000","last_clean":"2024-09-13T17:17:44.974442+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T17:17:44.974442+0000","last_undegraded":"2024-09-13T17:17:44.974442+0000","last_fullsized":"2024-09-13T17:17:44.974442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19787,"num_write_kb":259620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'3639","reported_seq":3719,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:03.984961+0000","last_change":"2024-09-13T17:19:03.985202+0000","last_active":"2024-09-13T17:19:03.984961+0000","last_peered":"2024-09-13T17:19:03.984961+0000","last_clean":"2024-09-13T17:19:03.984961+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T17:19:03.984961+0000","last_undegraded":"2024-09-13T17:19:03.984961+0000","last_fullsized":"2024-09-13T17:19:03.984961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3639","last_scrub_stamp":"2024-09-13T17:19:03.984902+0000","last_deep_scrub":"63'3639","last_deep_scrub_stamp":"2024-09-13T17:19:03.984902+0000","last_clean_scrub_stamp":"2024-09-13T17:19:03.984902+0000","objects_scrubbed":0,"log_size":3639,"log_dups_size":0,"ondisk_log_size":3639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:32.587792+0000","scrub_duration":0.028565929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20066,"num_write_kb":240267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"64'268","reported_seq":368,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:54.361866+0000","last_change":"2024-09-13T17:18:54.362093+0000","last_active":"2024-09-13T17:18:54.361866+0000","last_peered":"2024-09-13T17:18:54.361866+0000","last_clean":"2024-09-13T17:18:54.361866+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T17:18:54.361866+0000","last_undegraded":"2024-09-13T17:18:54.361866+0000","last_fullsized":"2024-09-13T17:18:54.361866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'268","last_scrub_stamp":"2024-09-13T17:18:54.361813+0000","last_deep_scrub":"64'268","last_deep_scrub_stamp":"2024-09-13T17:18:54.361813+0000","last_clean_scrub_stamp":"2024-09-13T17:18:54.361813+0000","objects_scrubbed":1,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:04.906631+0000","scrub_duration":0.0081052720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":38,"num_read_kb":48,"num_write":636,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"63'11","reported_seq":89,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.102762+0000","last_change":"2024-09-13T17:17:50.102762+0000","last_active":"2024-09-13T17:17:50.102762+0000","last_peered":"2024-09-13T17:17:50.102762+0000","last_clean":"2024-09-13T17:17:50.102762+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T17:17:50.102762+0000","last_undegraded":"2024-09-13T17:17:50.102762+0000","last_fullsized":"2024-09-13T17:17:50.102762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_deep_scrub":"63'11","last_deep_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.102616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.736833+0000","scrub_duration":0.0039927280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"63'3190","reported_seq":3266,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:52.074374+0000","last_change":"2024-09-13T17:18:52.074598+0000","last_active":"2024-09-13T17:18:52.074374+0000","last_peered":"2024-09-13T17:18:52.074374+0000","last_clean":"2024-09-13T17:18:52.074374+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T17:18:52.074374+0000","last_undegraded":"2024-09-13T17:18:52.074374+0000","last_fullsized":"2024-09-13T17:18:52.074374+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3190","last_scrub_stamp":"2024-09-13T17:18:52.074315+0000","last_deep_scrub":"63'3190","last_deep_scrub_stamp":"2024-09-13T17:18:52.074315+0000","last_clean_scrub_stamp":"2024-09-13T17:18:52.074315+0000","objects_scrubbed":0,"log_size":3190,"log_dups_size":0,"ondisk_log_size":3190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:02.619132+0000","scrub_duration":0.01789174,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":254866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"63'4701","reported_seq":4781,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:03.191095+0000","last_change":"2024-09-13T17:19:03.191365+0000","last_active":"2024-09-13T17:19:03.191095+0000","last_peered":"2024-09-13T17:19:03.191095+0000","last_clean":"2024-09-13T17:19:03.191095+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T17:19:03.191095+0000","last_undegraded":"2024-09-13T17:19:03.191095+0000","last_fullsized":"2024-09-13T17:19:03.191095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4701","last_scrub_stamp":"2024-09-13T17:19:03.191038+0000","last_deep_scrub":"63'4701","last_deep_scrub_stamp":"2024-09-13T17:19:03.191038+0000","last_clean_scrub_stamp":"2024-09-13T17:19:03.191038+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:19.321583+0000","scrub_duration":0.022656645999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26171,"num_write_kb":338588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'2","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:58.283498+0000","last_change":"2024-09-13T17:18:58.283758+0000","last_active":"2024-09-13T17:18:58.283498+0000","last_peered":"2024-09-13T17:18:58.283498+0000","last_clean":"2024-09-13T17:18:58.283498+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T17:18:58.283498+0000","last_undegraded":"2024-09-13T17:18:58.283498+0000","last_fullsized":"2024-09-13T17:18:58.283498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:58.283441+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:58.283441+0000","last_clean_scrub_stamp":"2024-09-13T17:18:58.283441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:09.998943+0000","scrub_duration":0.0033138870000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"63'3","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:57.112081+0000","last_change":"2024-09-13T17:18:57.112301+0000","last_active":"2024-09-13T17:18:57.112081+0000","last_peered":"2024-09-13T17:18:57.112081+0000","last_clean":"2024-09-13T17:18:57.112081+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T17:18:57.112081+0000","last_undegraded":"2024-09-13T17:18:57.112081+0000","last_fullsized":"2024-09-13T17:18:57.112081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:57.112022+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:57.112022+0000","last_clean_scrub_stamp":"2024-09-13T17:18:57.112022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:14.085716+0000","scrub_duration":0.0037361640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"63'3859","reported_seq":3924,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:45.003745+0000","last_change":"2024-09-13T16:48:08.177916+0000","last_active":"2024-09-13T17:17:45.003745+0000","last_peered":"2024-09-13T17:17:45.003745+0000","last_clean":"2024-09-13T17:17:45.003745+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T17:17:45.003745+0000","last_undegraded":"2024-09-13T17:17:45.003745+0000","last_fullsized":"2024-09-13T17:17:45.003745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21286,"num_write_kb":290222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"63'3876","reported_seq":3956,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:31.476414+0000","last_change":"2024-09-13T17:18:31.476681+0000","last_active":"2024-09-13T17:18:31.476414+0000","last_peered":"2024-09-13T17:18:31.476414+0000","last_clean":"2024-09-13T17:18:31.476414+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T17:18:31.476414+0000","last_undegraded":"2024-09-13T17:18:31.476414+0000","last_fullsized":"2024-09-13T17:18:31.476414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3876","last_scrub_stamp":"2024-09-13T17:18:31.476360+0000","last_deep_scrub":"63'3876","last_deep_scrub_stamp":"2024-09-13T17:18:31.476360+0000","last_clean_scrub_stamp":"2024-09-13T17:18:31.476360+0000","objects_scrubbed":0,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:48.450054+0000","scrub_duration":0.030063065999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21201,"num_write_kb":273354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"63'2","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:42.323131+0000","last_change":"2024-09-13T17:18:42.323409+0000","last_active":"2024-09-13T17:18:42.323131+0000","last_peered":"2024-09-13T17:18:42.323131+0000","last_clean":"2024-09-13T17:18:42.323131+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T17:18:42.323131+0000","last_undegraded":"2024-09-13T17:18:42.323131+0000","last_fullsized":"2024-09-13T17:18:42.323131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:42.323076+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:42.323076+0000","last_clean_scrub_stamp":"2024-09-13T17:18:42.323076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:59.296770+0000","scrub_duration":0.003371626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'5","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.226738+0000","last_change":"2024-09-13T17:17:57.227606+0000","last_active":"2024-09-13T17:17:57.226738+0000","last_peered":"2024-09-13T17:17:57.226738+0000","last_clean":"2024-09-13T17:17:57.226738+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T17:17:57.226738+0000","last_undegraded":"2024-09-13T17:17:57.226738+0000","last_fullsized":"2024-09-13T17:17:57.226738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226662+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:46.723103+0000","scrub_duration":0.0036245560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"63'3422","reported_seq":3488,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.993940+0000","last_change":"2024-09-13T16:48:08.174894+0000","last_active":"2024-09-13T17:17:44.993940+0000","last_peered":"2024-09-13T17:17:44.993940+0000","last_clean":"2024-09-13T17:17:44.993940+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T17:17:44.993940+0000","last_undegraded":"2024-09-13T17:17:44.993940+0000","last_fullsized":"2024-09-13T17:17:44.993940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":7,"num_write":19042,"num_write_kb":260722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"63'4802","reported_seq":4882,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:12.260957+0000","last_change":"2024-09-13T17:18:12.261210+0000","last_active":"2024-09-13T17:18:12.260957+0000","last_peered":"2024-09-13T17:18:12.260957+0000","last_clean":"2024-09-13T17:18:12.260957+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T17:18:12.260957+0000","last_undegraded":"2024-09-13T17:18:12.260957+0000","last_fullsized":"2024-09-13T17:18:12.260957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4802","last_scrub_stamp":"2024-09-13T17:18:12.260903+0000","last_deep_scrub":"63'4802","last_deep_scrub_stamp":"2024-09-13T17:18:12.260903+0000","last_clean_scrub_stamp":"2024-09-13T17:18:12.260903+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:25.238396+0000","scrub_duration":0.040392404999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26419,"num_write_kb":335470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"63'4","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:54.699579+0000","last_change":"2024-09-13T17:18:54.699803+0000","last_active":"2024-09-13T17:18:54.699579+0000","last_peered":"2024-09-13T17:18:54.699579+0000","last_clean":"2024-09-13T17:18:54.699579+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T17:18:54.699579+0000","last_undegraded":"2024-09-13T17:18:54.699579+0000","last_fullsized":"2024-09-13T17:18:54.699579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-13T17:18:54.699526+0000","last_deep_scrub":"63'4","last_deep_scrub_stamp":"2024-09-13T17:18:54.699526+0000","last_clean_scrub_stamp":"2024-09-13T17:18:54.699526+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:30.744221+0000","scrub_duration":0.003592694,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:54.181019+0000","last_change":"2024-09-13T17:17:54.181249+0000","last_active":"2024-09-13T17:17:54.181019+0000","last_peered":"2024-09-13T17:17:54.181019+0000","last_clean":"2024-09-13T17:17:54.181019+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T17:17:54.181019+0000","last_undegraded":"2024-09-13T17:17:54.181019+0000","last_fullsized":"2024-09-13T17:17:54.181019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_clean_scrub_stamp":"2024-09-13T17:17:54.180966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:37.317744+0000","scrub_duration":0.0037272709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"63'4215","reported_seq":4292,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:14.596560+0000","last_change":"2024-09-13T17:18:14.596560+0000","last_active":"2024-09-13T17:18:14.596560+0000","last_peered":"2024-09-13T17:18:14.596560+0000","last_clean":"2024-09-13T17:18:14.596560+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T17:18:14.596560+0000","last_undegraded":"2024-09-13T17:18:14.596560+0000","last_fullsized":"2024-09-13T17:18:14.596560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4215","last_scrub_stamp":"2024-09-13T17:18:14.596269+0000","last_deep_scrub":"63'4215","last_deep_scrub_stamp":"2024-09-13T17:18:14.596269+0000","last_clean_scrub_stamp":"2024-09-13T17:18:14.596269+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:07.101977+0000","scrub_duration":0.024198185000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":8,"num_write":22461,"num_write_kb":319331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'3790","reported_seq":3871,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:59.270895+0000","last_change":"2024-09-13T17:18:59.271188+0000","last_active":"2024-09-13T17:18:59.270895+0000","last_peered":"2024-09-13T17:18:59.270895+0000","last_clean":"2024-09-13T17:18:59.270895+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T17:18:59.270895+0000","last_undegraded":"2024-09-13T17:18:59.270895+0000","last_fullsized":"2024-09-13T17:18:59.270895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3790","last_scrub_stamp":"2024-09-13T17:18:59.270842+0000","last_deep_scrub":"63'3790","last_deep_scrub_stamp":"2024-09-13T17:18:59.270842+0000","last_clean_scrub_stamp":"2024-09-13T17:18:59.270842+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:00.669692+0000","scrub_duration":0.017889702,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21077,"num_write_kb":260732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'42","reported_seq":179,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.602904+0000","last_change":"2024-09-13T17:18:03.603126+0000","last_active":"2024-09-13T17:18:03.602904+0000","last_peered":"2024-09-13T17:18:03.602904+0000","last_clean":"2024-09-13T17:18:03.602904+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T17:18:03.602904+0000","last_undegraded":"2024-09-13T17:18:03.602904+0000","last_fullsized":"2024-09-13T17:18:03.602904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'42","last_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_deep_scrub":"64'42","last_deep_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.602845+0000","objects_scrubbed":1,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:03.563076+0000","scrub_duration":0.004957576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84646,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":106,"num_write":37,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'10","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.102806+0000","last_change":"2024-09-13T17:18:05.102806+0000","last_active":"2024-09-13T17:18:05.102806+0000","last_peered":"2024-09-13T17:18:05.102806+0000","last_clean":"2024-09-13T17:18:05.102806+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T17:18:05.102806+0000","last_undegraded":"2024-09-13T17:18:05.102806+0000","last_fullsized":"2024-09-13T17:18:05.102806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-13T17:18:05.101908+0000","last_deep_scrub":"64'10","last_deep_scrub_stamp":"2024-09-13T17:18:05.101908+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.101908+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:04.415025+0000","scrub_duration":0.0065388449999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"63'3128","reported_seq":3193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.642943+0000","last_change":"2024-09-13T16:48:08.176109+0000","last_active":"2024-09-13T17:17:48.642943+0000","last_peered":"2024-09-13T17:17:48.642943+0000","last_clean":"2024-09-13T17:17:48.642943+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T17:17:48.642943+0000","last_undegraded":"2024-09-13T17:17:48.642943+0000","last_fullsized":"2024-09-13T17:17:48.642943+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3128,"log_dups_size":0,"ondisk_log_size":3128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17776,"num_write_kb":243283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"63'3657","reported_seq":3737,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:14.878066+0000","last_change":"2024-09-13T17:18:14.878283+0000","last_active":"2024-09-13T17:18:14.878066+0000","last_peered":"2024-09-13T17:18:14.878066+0000","last_clean":"2024-09-13T17:18:14.878066+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T17:18:14.878066+0000","last_undegraded":"2024-09-13T17:18:14.878066+0000","last_fullsized":"2024-09-13T17:18:14.878066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3657","last_scrub_stamp":"2024-09-13T17:18:14.878012+0000","last_deep_scrub":"63'3657","last_deep_scrub_stamp":"2024-09-13T17:18:14.878012+0000","last_clean_scrub_stamp":"2024-09-13T17:18:14.878012+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:17.512229+0000","scrub_duration":0.026162280999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19816,"num_write_kb":255512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'11","reported_seq":93,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.280947+0000","last_change":"2024-09-13T17:18:44.281154+0000","last_active":"2024-09-13T17:18:44.280947+0000","last_peered":"2024-09-13T17:18:44.280947+0000","last_clean":"2024-09-13T17:18:44.280947+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T17:18:44.280947+0000","last_undegraded":"2024-09-13T17:18:44.280947+0000","last_fullsized":"2024-09-13T17:18:44.280947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-13T17:18:44.280891+0000","last_deep_scrub":"64'11","last_deep_scrub_stamp":"2024-09-13T17:18:44.280891+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.280891+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:24.706976+0000","scrub_duration":0.0031866279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'252","reported_seq":350,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:00.134660+0000","last_change":"2024-09-13T17:19:00.134862+0000","last_active":"2024-09-13T17:19:00.134660+0000","last_peered":"2024-09-13T17:19:00.134660+0000","last_clean":"2024-09-13T17:19:00.134660+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T17:19:00.134660+0000","last_undegraded":"2024-09-13T17:19:00.134660+0000","last_fullsized":"2024-09-13T17:19:00.134660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'252","last_scrub_stamp":"2024-09-13T17:19:00.134608+0000","last_deep_scrub":"64'252","last_deep_scrub_stamp":"2024-09-13T17:19:00.134608+0000","last_clean_scrub_stamp":"2024-09-13T17:19:00.134608+0000","objects_scrubbed":1,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:55.435852+0000","scrub_duration":0.0081424600000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":57,"num_write":597,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"63'4141","reported_seq":4218,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.424281+0000","last_change":"2024-09-13T17:19:05.424513+0000","last_active":"2024-09-13T17:19:05.424281+0000","last_peered":"2024-09-13T17:19:05.424281+0000","last_clean":"2024-09-13T17:19:05.424281+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T17:19:05.424281+0000","last_undegraded":"2024-09-13T17:19:05.424281+0000","last_fullsized":"2024-09-13T17:19:05.424281+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4141","last_scrub_stamp":"2024-09-13T17:19:05.424228+0000","last_deep_scrub":"63'4141","last_deep_scrub_stamp":"2024-09-13T17:19:05.424228+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.424228+0000","objects_scrubbed":0,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:25.582751+0000","scrub_duration":0.029955119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22849,"num_write_kb":317008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'4103","reported_seq":4191,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:42.981381+0000","last_change":"2024-09-13T17:18:42.981633+0000","last_active":"2024-09-13T17:18:42.981381+0000","last_peered":"2024-09-13T17:18:42.981381+0000","last_clean":"2024-09-13T17:18:42.981381+0000","last_became_active":"2024-09-13T16:48:41.389507+0000","last_became_peered":"2024-09-13T16:48:41.389507+0000","last_unstale":"2024-09-13T17:18:42.981381+0000","last_undegraded":"2024-09-13T17:18:42.981381+0000","last_fullsized":"2024-09-13T17:18:42.981381+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4103","last_scrub_stamp":"2024-09-13T17:18:42.981318+0000","last_deep_scrub":"63'4103","last_deep_scrub_stamp":"2024-09-13T17:18:42.981318+0000","last_clean_scrub_stamp":"2024-09-13T17:18:42.981318+0000","objects_scrubbed":0,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:15.675900+0000","scrub_duration":0.025114515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22342,"num_write_kb":278194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"63'3","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:51.588538+0000","last_change":"2024-09-13T17:18:51.588758+0000","last_active":"2024-09-13T17:18:51.588538+0000","last_peered":"2024-09-13T17:18:51.588538+0000","last_clean":"2024-09-13T17:18:51.588538+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T17:18:51.588538+0000","last_undegraded":"2024-09-13T17:18:51.588538+0000","last_fullsized":"2024-09-13T17:18:51.588538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:51.588486+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:51.588486+0000","last_clean_scrub_stamp":"2024-09-13T17:18:51.588486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:07.719031+0000","scrub_duration":0.0034308670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'119","reported_seq":237,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.251938+0000","last_change":"2024-09-13T17:18:43.252212+0000","last_active":"2024-09-13T17:18:43.251938+0000","last_peered":"2024-09-13T17:18:43.251938+0000","last_clean":"2024-09-13T17:18:43.251938+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T17:18:43.251938+0000","last_undegraded":"2024-09-13T17:18:43.251938+0000","last_fullsized":"2024-09-13T17:18:43.251938+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'119","last_scrub_stamp":"2024-09-13T17:18:43.251885+0000","last_deep_scrub":"64'119","last_deep_scrub_stamp":"2024-09-13T17:18:43.251885+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.251885+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:32.643640+0000","scrub_duration":0.010406291999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:40.136105+0000","last_change":"2024-09-13T17:18:40.136327+0000","last_active":"2024-09-13T17:18:40.136105+0000","last_peered":"2024-09-13T17:18:40.136105+0000","last_clean":"2024-09-13T17:18:40.136105+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T17:18:40.136105+0000","last_undegraded":"2024-09-13T17:18:40.136105+0000","last_fullsized":"2024-09-13T17:18:40.136105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:40.136049+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:40.136049+0000","last_clean_scrub_stamp":"2024-09-13T17:18:40.136049+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:52.925685+0000","scrub_duration":0.0033099650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"63'2904","reported_seq":2980,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.337844+0000","last_change":"2024-09-13T17:17:48.337931+0000","last_active":"2024-09-13T17:17:48.337844+0000","last_peered":"2024-09-13T17:17:48.337844+0000","last_clean":"2024-09-13T17:17:48.337844+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T17:17:48.337844+0000","last_undegraded":"2024-09-13T17:17:48.337844+0000","last_fullsized":"2024-09-13T17:17:48.337844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2904","last_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_deep_scrub":"63'2904","last_deep_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_clean_scrub_stamp":"2024-09-13T17:17:48.337825+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:15.694145+0000","scrub_duration":0.0051743259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":2,"num_write":15607,"num_write_kb":214254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'4284","reported_seq":4363,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.341991+0000","last_change":"2024-09-13T17:17:52.342307+0000","last_active":"2024-09-13T17:17:52.341991+0000","last_peered":"2024-09-13T17:17:52.341991+0000","last_clean":"2024-09-13T17:17:52.341991+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T17:17:52.341991+0000","last_undegraded":"2024-09-13T17:17:52.341991+0000","last_fullsized":"2024-09-13T17:17:52.341991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4284","last_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_deep_scrub":"63'4284","last_deep_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.341937+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.282559+0000","scrub_duration":0.043379468999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23392,"num_write_kb":310963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"63'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.502278+0000","last_change":"2024-09-13T17:17:53.502523+0000","last_active":"2024-09-13T17:17:53.502278+0000","last_peered":"2024-09-13T17:17:53.502278+0000","last_clean":"2024-09-13T17:17:53.502278+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T17:17:53.502278+0000","last_undegraded":"2024-09-13T17:17:53.502278+0000","last_fullsized":"2024-09-13T17:17:53.502278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.502221+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:42.741599+0000","scrub_duration":0.0040979670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"63'3","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:41.132756+0000","last_change":"2024-09-13T17:18:41.133001+0000","last_active":"2024-09-13T17:18:41.132756+0000","last_peered":"2024-09-13T17:18:41.132756+0000","last_clean":"2024-09-13T17:18:41.132756+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T17:18:41.132756+0000","last_undegraded":"2024-09-13T17:18:41.132756+0000","last_fullsized":"2024-09-13T17:18:41.132756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:41.132703+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:41.132703+0000","last_clean_scrub_stamp":"2024-09-13T17:18:41.132703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:58.106397+0000","scrub_duration":0.0037951880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"63'4138","reported_seq":4214,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.398680+0000","last_change":"2024-09-13T17:17:49.398680+0000","last_active":"2024-09-13T17:17:49.398680+0000","last_peered":"2024-09-13T17:17:49.398680+0000","last_clean":"2024-09-13T17:17:49.398680+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T17:17:49.398680+0000","last_undegraded":"2024-09-13T17:17:49.398680+0000","last_fullsized":"2024-09-13T17:17:49.398680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4138","last_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_deep_scrub":"63'4138","last_deep_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.398536+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:51.412144+0000","scrub_duration":0.020992730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":309086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"63'4584","reported_seq":4657,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:49.729734+0000","last_change":"2024-09-13T17:18:49.729975+0000","last_active":"2024-09-13T17:18:49.729734+0000","last_peered":"2024-09-13T17:18:49.729734+0000","last_clean":"2024-09-13T17:18:49.729734+0000","last_became_active":"2024-09-13T16:48:40.765861+0000","last_became_peered":"2024-09-13T16:48:40.765861+0000","last_unstale":"2024-09-13T17:18:49.729734+0000","last_undegraded":"2024-09-13T17:18:49.729734+0000","last_fullsized":"2024-09-13T17:18:49.729734+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4584","last_scrub_stamp":"2024-09-13T17:18:49.729674+0000","last_deep_scrub":"63'4584","last_deep_scrub_stamp":"2024-09-13T17:18:49.729674+0000","last_clean_scrub_stamp":"2024-09-13T17:18:49.729674+0000","objects_scrubbed":0,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:59.866520+0000","scrub_duration":0.023671662,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24690,"num_write_kb":320186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"64'7","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.308611+0000","last_change":"2024-09-13T17:18:46.308867+0000","last_active":"2024-09-13T17:18:46.308611+0000","last_peered":"2024-09-13T17:18:46.308611+0000","last_clean":"2024-09-13T17:18:46.308611+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T17:18:46.308611+0000","last_undegraded":"2024-09-13T17:18:46.308611+0000","last_fullsized":"2024-09-13T17:18:46.308611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'7","last_scrub_stamp":"2024-09-13T17:18:46.308558+0000","last_deep_scrub":"64'7","last_deep_scrub_stamp":"2024-09-13T17:18:46.308558+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.308558+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:56.445404+0000","scrub_duration":0.0073412219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'1427","reported_seq":1551,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:58.168946+0000","last_change":"2024-09-13T17:18:58.169177+0000","last_active":"2024-09-13T17:18:58.168946+0000","last_peered":"2024-09-13T17:18:58.168946+0000","last_clean":"2024-09-13T17:18:58.168946+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T17:18:58.168946+0000","last_undegraded":"2024-09-13T17:18:58.168946+0000","last_fullsized":"2024-09-13T17:18:58.168946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1427","last_scrub_stamp":"2024-09-13T17:18:58.168889+0000","last_deep_scrub":"64'1427","last_deep_scrub_stamp":"2024-09-13T17:18:58.168889+0000","last_clean_scrub_stamp":"2024-09-13T17:18:58.168889+0000","objects_scrubbed":2,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:35.519000+0000","scrub_duration":0.031253693999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":38459,"num_write":1400,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"63'3637","reported_seq":3714,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:03.412582+0000","last_change":"2024-09-13T17:19:03.412806+0000","last_active":"2024-09-13T17:19:03.412582+0000","last_peered":"2024-09-13T17:19:03.412582+0000","last_clean":"2024-09-13T17:19:03.412582+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T17:19:03.412582+0000","last_undegraded":"2024-09-13T17:19:03.412582+0000","last_fullsized":"2024-09-13T17:19:03.412582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3637","last_scrub_stamp":"2024-09-13T17:19:03.412530+0000","last_deep_scrub":"63'3637","last_deep_scrub_stamp":"2024-09-13T17:19:03.412530+0000","last_clean_scrub_stamp":"2024-09-13T17:19:03.412530+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:32.015420+0000","scrub_duration":0.030553734999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19635,"num_write_kb":259018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'5067","reported_seq":5140,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:59.638860+0000","last_change":"2024-09-13T17:18:59.639093+0000","last_active":"2024-09-13T17:18:59.638860+0000","last_peered":"2024-09-13T17:18:59.638860+0000","last_clean":"2024-09-13T17:18:59.638860+0000","last_became_active":"2024-09-13T16:48:40.831696+0000","last_became_peered":"2024-09-13T16:48:40.831696+0000","last_unstale":"2024-09-13T17:18:59.638860+0000","last_undegraded":"2024-09-13T17:18:59.638860+0000","last_fullsized":"2024-09-13T17:18:59.638860+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5067","last_scrub_stamp":"2024-09-13T17:18:59.638799+0000","last_deep_scrub":"63'5067","last_deep_scrub_stamp":"2024-09-13T17:18:59.638799+0000","last_clean_scrub_stamp":"2024-09-13T17:18:59.638799+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:32.333381+0000","scrub_duration":0.031396859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27028,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.556099+0000","last_change":"2024-09-13T17:18:02.556314+0000","last_active":"2024-09-13T17:18:02.556099+0000","last_peered":"2024-09-13T17:18:02.556099+0000","last_clean":"2024-09-13T17:18:02.556099+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T17:18:02.556099+0000","last_undegraded":"2024-09-13T17:18:02.556099+0000","last_fullsized":"2024-09-13T17:18:02.556099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.556044+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:52.052485+0000","scrub_duration":0.0060730330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"63'5","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:16.546615+0000","last_change":"2024-09-13T17:18:16.546615+0000","last_active":"2024-09-13T17:18:16.546615+0000","last_peered":"2024-09-13T17:18:16.546615+0000","last_clean":"2024-09-13T17:18:16.546615+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T17:18:16.546615+0000","last_undegraded":"2024-09-13T17:18:16.546615+0000","last_fullsized":"2024-09-13T17:18:16.546615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:16.546281+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:16.546281+0000","last_clean_scrub_stamp":"2024-09-13T17:18:16.546281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:15.818163+0000","scrub_duration":0.0035841639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"63'2915","reported_seq":2991,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.588368+0000","last_change":"2024-09-13T17:18:29.588368+0000","last_active":"2024-09-13T17:18:29.588368+0000","last_peered":"2024-09-13T17:18:29.588368+0000","last_clean":"2024-09-13T17:18:29.588368+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T17:18:29.588368+0000","last_undegraded":"2024-09-13T17:18:29.588368+0000","last_fullsized":"2024-09-13T17:18:29.588368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2915","last_scrub_stamp":"2024-09-13T17:18:29.588063+0000","last_deep_scrub":"63'2915","last_deep_scrub_stamp":"2024-09-13T17:18:29.588063+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.588063+0000","objects_scrubbed":0,"log_size":2915,"log_dups_size":0,"ondisk_log_size":2915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:14.651101+0000","scrub_duration":0.01592476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":234494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"63'3697","reported_seq":3776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.226187+0000","last_change":"2024-09-13T17:18:29.226417+0000","last_active":"2024-09-13T17:18:29.226187+0000","last_peered":"2024-09-13T17:18:29.226187+0000","last_clean":"2024-09-13T17:18:29.226187+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T17:18:29.226187+0000","last_undegraded":"2024-09-13T17:18:29.226187+0000","last_fullsized":"2024-09-13T17:18:29.226187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3697","last_scrub_stamp":"2024-09-13T17:18:29.226132+0000","last_deep_scrub":"63'3697","last_deep_scrub_stamp":"2024-09-13T17:18:29.226132+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.226132+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:46.199826+0000","scrub_duration":0.032222396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20206,"num_write_kb":253140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"64'5","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:06.185518+0000","last_change":"2024-09-13T17:18:06.186368+0000","last_active":"2024-09-13T17:18:06.185518+0000","last_peered":"2024-09-13T17:18:06.185518+0000","last_clean":"2024-09-13T17:18:06.185518+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T17:18:06.185518+0000","last_undegraded":"2024-09-13T17:18:06.185518+0000","last_fullsized":"2024-09-13T17:18:06.185518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:06.185462+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:06.185462+0000","last_clean_scrub_stamp":"2024-09-13T17:18:06.185462+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:29.073557+0000","scrub_duration":0.0065555370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.158515+0000","last_change":"2024-09-13T17:18:02.158515+0000","last_active":"2024-09-13T17:18:02.158515+0000","last_peered":"2024-09-13T17:18:02.158515+0000","last_clean":"2024-09-13T17:18:02.158515+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T17:18:02.158515+0000","last_undegraded":"2024-09-13T17:18:02.158515+0000","last_fullsized":"2024-09-13T17:18:02.158515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.158216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.118447+0000","scrub_duration":0.003800346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"63'3522","reported_seq":3596,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:40.984683+0000","last_change":"2024-09-13T17:18:40.984936+0000","last_active":"2024-09-13T17:18:40.984683+0000","last_peered":"2024-09-13T17:18:40.984683+0000","last_clean":"2024-09-13T17:18:40.984683+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T17:18:40.984683+0000","last_undegraded":"2024-09-13T17:18:40.984683+0000","last_fullsized":"2024-09-13T17:18:40.984683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3522","last_scrub_stamp":"2024-09-13T17:18:40.984629+0000","last_deep_scrub":"63'3522","last_deep_scrub_stamp":"2024-09-13T17:18:40.984629+0000","last_clean_scrub_stamp":"2024-09-13T17:18:40.984629+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:51.529447+0000","scrub_duration":0.025504836999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18828,"num_write_kb":272277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"63'3874","reported_seq":3955,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.213453+0000","last_change":"2024-09-13T17:17:59.213453+0000","last_active":"2024-09-13T17:17:59.213453+0000","last_peered":"2024-09-13T17:17:59.213453+0000","last_clean":"2024-09-13T17:17:59.213453+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T17:17:59.213453+0000","last_undegraded":"2024-09-13T17:17:59.213453+0000","last_fullsized":"2024-09-13T17:17:59.213453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.213146+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.190638+0000","scrub_duration":0.033789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21413,"num_write_kb":278431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'19","reported_seq":193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.533761+0000","last_change":"2024-09-13T17:17:59.533761+0000","last_active":"2024-09-13T17:17:59.533761+0000","last_peered":"2024-09-13T17:17:59.533761+0000","last_clean":"2024-09-13T17:17:59.533761+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T17:17:59.533761+0000","last_undegraded":"2024-09-13T17:17:59.533761+0000","last_fullsized":"2024-09-13T17:17:59.533761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_deep_scrub":"64'19","last_deep_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.533477+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:42.670255+0000","scrub_duration":0.0092684789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:00.848306+0000","last_change":"2024-09-13T17:18:00.848525+0000","last_active":"2024-09-13T17:18:00.848306+0000","last_peered":"2024-09-13T17:18:00.848306+0000","last_clean":"2024-09-13T17:18:00.848306+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T17:18:00.848306+0000","last_undegraded":"2024-09-13T17:18:00.848306+0000","last_fullsized":"2024-09-13T17:18:00.848306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_clean_scrub_stamp":"2024-09-13T17:18:00.848253+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:51.294142+0000","scrub_duration":0.0034249229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"63'3622","reported_seq":3688,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.984518+0000","last_change":"2024-09-13T16:48:08.171857+0000","last_active":"2024-09-13T17:17:44.984518+0000","last_peered":"2024-09-13T17:17:44.984518+0000","last_clean":"2024-09-13T17:17:44.984518+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T17:17:44.984518+0000","last_undegraded":"2024-09-13T17:17:44.984518+0000","last_fullsized":"2024-09-13T17:17:44.984518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":19996,"num_write_kb":293269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"63'3240","reported_seq":3319,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.494730+0000","last_change":"2024-09-13T17:18:10.494730+0000","last_active":"2024-09-13T17:18:10.494730+0000","last_peered":"2024-09-13T17:18:10.494730+0000","last_clean":"2024-09-13T17:18:10.494730+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T17:18:10.494730+0000","last_undegraded":"2024-09-13T17:18:10.494730+0000","last_fullsized":"2024-09-13T17:18:10.494730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3240","last_scrub_stamp":"2024-09-13T17:18:10.494430+0000","last_deep_scrub":"63'3240","last_deep_scrub_stamp":"2024-09-13T17:18:10.494430+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.494430+0000","objects_scrubbed":0,"log_size":3240,"log_dups_size":0,"ondisk_log_size":3240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:03.000138+0000","scrub_duration":0.014460961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":237245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"63'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.449798+0000","last_change":"2024-09-13T17:18:02.449798+0000","last_active":"2024-09-13T17:18:02.449798+0000","last_peered":"2024-09-13T17:18:02.449798+0000","last_clean":"2024-09-13T17:18:02.449798+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T17:18:02.449798+0000","last_undegraded":"2024-09-13T17:18:02.449798+0000","last_fullsized":"2024-09-13T17:18:02.449798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.449537+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.409768+0000","scrub_duration":0.0032633279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'5","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.060756+0000","last_change":"2024-09-13T17:17:51.060978+0000","last_active":"2024-09-13T17:17:51.060756+0000","last_peered":"2024-09-13T17:17:51.060756+0000","last_clean":"2024-09-13T17:17:51.060756+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T17:17:51.060756+0000","last_undegraded":"2024-09-13T17:17:51.060756+0000","last_fullsized":"2024-09-13T17:17:51.060756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.060702+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:45.719305+0000","scrub_duration":0.0041164770000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"63'3376","reported_seq":3439,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.981690+0000","last_change":"2024-09-13T16:48:08.173904+0000","last_active":"2024-09-13T17:17:44.981690+0000","last_peered":"2024-09-13T17:17:44.981690+0000","last_clean":"2024-09-13T17:17:44.981690+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T17:17:44.981690+0000","last_undegraded":"2024-09-13T17:17:44.981690+0000","last_fullsized":"2024-09-13T17:17:44.981690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18879,"num_write_kb":251906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"63'4848","reported_seq":4927,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:41.225569+0000","last_change":"2024-09-13T17:18:41.225810+0000","last_active":"2024-09-13T17:18:41.225569+0000","last_peered":"2024-09-13T17:18:41.225569+0000","last_clean":"2024-09-13T17:18:41.225569+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T17:18:41.225569+0000","last_undegraded":"2024-09-13T17:18:41.225569+0000","last_fullsized":"2024-09-13T17:18:41.225569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4848","last_scrub_stamp":"2024-09-13T17:18:41.225514+0000","last_deep_scrub":"63'4848","last_deep_scrub_stamp":"2024-09-13T17:18:41.225514+0000","last_clean_scrub_stamp":"2024-09-13T17:18:41.225514+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:51.770332+0000","scrub_duration":0.049129424999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26963,"num_write_kb":321221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"63'3","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:02.632405+0000","last_change":"2024-09-13T17:19:02.632627+0000","last_active":"2024-09-13T17:19:02.632405+0000","last_peered":"2024-09-13T17:19:02.632405+0000","last_clean":"2024-09-13T17:19:02.632405+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T17:19:02.632405+0000","last_undegraded":"2024-09-13T17:19:02.632405+0000","last_fullsized":"2024-09-13T17:19:02.632405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:19:02.632354+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:19:02.632354+0000","last_clean_scrub_stamp":"2024-09-13T17:19:02.632354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:44.165675+0000","scrub_duration":0.0032785549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'392","reported_seq":575,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.116420+0000","last_change":"2024-09-13T17:17:49.116420+0000","last_active":"2024-09-13T17:17:49.116420+0000","last_peered":"2024-09-13T17:17:49.116420+0000","last_clean":"2024-09-13T17:17:49.116420+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T17:17:49.116420+0000","last_undegraded":"2024-09-13T17:17:49.116420+0000","last_fullsized":"2024-09-13T17:17:49.116420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'392","last_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_deep_scrub":"64'392","last_deep_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.116249+0000","objects_scrubbed":4,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:49.076480+0000","scrub_duration":0.026108916999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":191,"num_read_kb":82094,"num_write":385,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'4856","reported_seq":4922,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632005+0000","last_change":"2024-09-13T16:48:08.165612+0000","last_active":"2024-09-13T17:17:47.632005+0000","last_peered":"2024-09-13T17:17:47.632005+0000","last_clean":"2024-09-13T17:17:47.632005+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T17:17:47.632005+0000","last_undegraded":"2024-09-13T17:17:47.632005+0000","last_fullsized":"2024-09-13T17:17:47.632005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":26453,"num_write_kb":368210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'4063","reported_seq":4142,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:24.994002+0000","last_change":"2024-09-13T17:18:24.994290+0000","last_active":"2024-09-13T17:18:24.994002+0000","last_peered":"2024-09-13T17:18:24.994002+0000","last_clean":"2024-09-13T17:18:24.994002+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T17:18:24.994002+0000","last_undegraded":"2024-09-13T17:18:24.994002+0000","last_fullsized":"2024-09-13T17:18:24.994002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4063","last_scrub_stamp":"2024-09-13T17:18:24.993948+0000","last_deep_scrub":"63'4063","last_deep_scrub_stamp":"2024-09-13T17:18:24.993948+0000","last_clean_scrub_stamp":"2024-09-13T17:18:24.993948+0000","objects_scrubbed":0,"log_size":4063,"log_dups_size":0,"ondisk_log_size":4063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:17.499656+0000","scrub_duration":0.029339008999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21999,"num_write_kb":293018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"64'11","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:07.796310+0000","last_change":"2024-09-13T17:18:07.796310+0000","last_active":"2024-09-13T17:18:07.796310+0000","last_peered":"2024-09-13T17:18:07.796310+0000","last_clean":"2024-09-13T17:18:07.796310+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T17:18:07.796310+0000","last_undegraded":"2024-09-13T17:18:07.796310+0000","last_fullsized":"2024-09-13T17:18:07.796310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-13T17:18:07.796034+0000","last_deep_scrub":"64'11","last_deep_scrub_stamp":"2024-09-13T17:18:07.796034+0000","last_clean_scrub_stamp":"2024-09-13T17:18:07.796034+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:50.932812+0000","scrub_duration":0.0076063959999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":152,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'32","reported_seq":170,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.557513+0000","last_change":"2024-09-13T17:17:59.557788+0000","last_active":"2024-09-13T17:17:59.557513+0000","last_peered":"2024-09-13T17:17:59.557513+0000","last_clean":"2024-09-13T17:17:59.557513+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T17:17:59.557513+0000","last_undegraded":"2024-09-13T17:17:59.557513+0000","last_fullsized":"2024-09-13T17:17:59.557513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'32","last_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_deep_scrub":"64'32","last_deep_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.557461+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.534953+0000","scrub_duration":0.0076444640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":31,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"63'4239","reported_seq":4303,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.631955+0000","last_change":"2024-09-13T16:48:08.171581+0000","last_active":"2024-09-13T17:17:47.631955+0000","last_peered":"2024-09-13T17:17:47.631955+0000","last_clean":"2024-09-13T17:17:47.631955+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T17:17:47.631955+0000","last_undegraded":"2024-09-13T17:17:47.631955+0000","last_fullsized":"2024-09-13T17:17:47.631955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23316,"num_write_kb":318961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'3718","reported_seq":3796,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.214013+0000","last_change":"2024-09-13T17:17:57.214254+0000","last_active":"2024-09-13T17:17:57.214013+0000","last_peered":"2024-09-13T17:17:57.214013+0000","last_clean":"2024-09-13T17:17:57.214013+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T17:17:57.214013+0000","last_undegraded":"2024-09-13T17:17:57.214013+0000","last_fullsized":"2024-09-13T17:17:57.214013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3718","last_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_deep_scrub":"63'3718","last_deep_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.213958+0000","objects_scrubbed":0,"log_size":3718,"log_dups_size":0,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.719666+0000","scrub_duration":0.032519578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20956,"num_write_kb":260218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"64'395","reported_seq":570,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.567435+0000","last_change":"2024-09-13T17:18:04.567435+0000","last_active":"2024-09-13T17:18:04.567435+0000","last_peered":"2024-09-13T17:18:04.567435+0000","last_clean":"2024-09-13T17:18:04.567435+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T17:18:04.567435+0000","last_undegraded":"2024-09-13T17:18:04.567435+0000","last_fullsized":"2024-09-13T17:18:04.567435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'395","last_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_deep_scrub":"64'395","last_deep_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.567179+0000","objects_scrubbed":4,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:59.225783+0000","scrub_duration":0.027913209000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":171,"num_read_kb":77976,"num_write":389,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"63'3","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:33.246155+0000","last_change":"2024-09-13T17:18:33.246380+0000","last_active":"2024-09-13T17:18:33.246155+0000","last_peered":"2024-09-13T17:18:33.246155+0000","last_clean":"2024-09-13T17:18:33.246155+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T17:18:33.246155+0000","last_undegraded":"2024-09-13T17:18:33.246155+0000","last_fullsized":"2024-09-13T17:18:33.246155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:33.246101+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:33.246101+0000","last_clean_scrub_stamp":"2024-09-13T17:18:33.246101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:32.671487+0000","scrub_duration":0.0034645399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"63'3522","reported_seq":3587,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.674980+0000","last_change":"2024-09-13T16:48:08.173797+0000","last_active":"2024-09-13T17:17:46.674980+0000","last_peered":"2024-09-13T17:17:46.674980+0000","last_clean":"2024-09-13T17:17:46.674980+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T17:17:46.674980+0000","last_undegraded":"2024-09-13T17:17:46.674980+0000","last_fullsized":"2024-09-13T17:17:46.674980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":4,"num_write":19373,"num_write_kb":266222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"63'5138","reported_seq":5217,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:33.196310+0000","last_change":"2024-09-13T17:18:33.196535+0000","last_active":"2024-09-13T17:18:33.196310+0000","last_peered":"2024-09-13T17:18:33.196310+0000","last_clean":"2024-09-13T17:18:33.196310+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T17:18:33.196310+0000","last_undegraded":"2024-09-13T17:18:33.196310+0000","last_fullsized":"2024-09-13T17:18:33.196310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5138","last_scrub_stamp":"2024-09-13T17:18:33.196260+0000","last_deep_scrub":"63'5138","last_deep_scrub_stamp":"2024-09-13T17:18:33.196260+0000","last_clean_scrub_stamp":"2024-09-13T17:18:33.196260+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:32.621647+0000","scrub_duration":0.048334911000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27863,"num_write_kb":353454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'27","reported_seq":144,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:57.282270+0000","last_change":"2024-09-13T17:18:57.282500+0000","last_active":"2024-09-13T17:18:57.282270+0000","last_peered":"2024-09-13T17:18:57.282270+0000","last_clean":"2024-09-13T17:18:57.282270+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T17:18:57.282270+0000","last_undegraded":"2024-09-13T17:18:57.282270+0000","last_fullsized":"2024-09-13T17:18:57.282270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-09-13T17:18:57.282214+0000","last_deep_scrub":"64'27","last_deep_scrub_stamp":"2024-09-13T17:18:57.282214+0000","last_clean_scrub_stamp":"2024-09-13T17:18:57.282214+0000","objects_scrubbed":2,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:37.708299+0000","scrub_duration":0.0070313399999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'9","reported_seq":127,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.154830+0000","last_change":"2024-09-13T17:18:03.154830+0000","last_active":"2024-09-13T17:18:03.154830+0000","last_peered":"2024-09-13T17:18:03.154830+0000","last_clean":"2024-09-13T17:18:03.154830+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T17:18:03.154830+0000","last_undegraded":"2024-09-13T17:18:03.154830+0000","last_fullsized":"2024-09-13T17:18:03.154830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_deep_scrub":"64'9","last_deep_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.154514+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:05.788731+0000","scrub_duration":0.0080602239999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":168,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"63'3602","reported_seq":3680,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.062966+0000","last_change":"2024-09-13T17:18:46.063201+0000","last_active":"2024-09-13T17:18:46.062966+0000","last_peered":"2024-09-13T17:18:46.062966+0000","last_clean":"2024-09-13T17:18:46.062966+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T17:18:46.062966+0000","last_undegraded":"2024-09-13T17:18:46.062966+0000","last_fullsized":"2024-09-13T17:18:46.062966+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3602","last_scrub_stamp":"2024-09-13T17:18:46.062913+0000","last_deep_scrub":"63'3602","last_deep_scrub_stamp":"2024-09-13T17:18:46.062913+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.062913+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:43.137455+0000","scrub_duration":0.025752721999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":20290,"num_write_kb":278255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'3743","reported_seq":3821,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.841035+0000","last_change":"2024-09-13T17:17:57.841262+0000","last_active":"2024-09-13T17:17:57.841035+0000","last_peered":"2024-09-13T17:17:57.841035+0000","last_clean":"2024-09-13T17:17:57.841035+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T17:17:57.841035+0000","last_undegraded":"2024-09-13T17:17:57.841035+0000","last_fullsized":"2024-09-13T17:17:57.841035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3743","last_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_deep_scrub":"63'3743","last_deep_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.840983+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:01.952379+0000","scrub_duration":0.026384685000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20404,"num_write_kb":278673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"63'3","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:01.646133+0000","last_change":"2024-09-13T17:19:01.646372+0000","last_active":"2024-09-13T17:19:01.646133+0000","last_peered":"2024-09-13T17:19:01.646133+0000","last_clean":"2024-09-13T17:19:01.646133+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T17:19:01.646133+0000","last_undegraded":"2024-09-13T17:19:01.646133+0000","last_fullsized":"2024-09-13T17:19:01.646133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:19:01.646081+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:19:01.646081+0000","last_clean_scrub_stamp":"2024-09-13T17:19:01.646081+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:02.753337+0000","scrub_duration":0.0033163099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'19","reported_seq":203,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:20.179269+0000","last_change":"2024-09-13T17:18:20.179269+0000","last_active":"2024-09-13T17:18:20.179269+0000","last_peered":"2024-09-13T17:18:20.179269+0000","last_clean":"2024-09-13T17:18:20.179269+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T17:18:20.179269+0000","last_undegraded":"2024-09-13T17:18:20.179269+0000","last_fullsized":"2024-09-13T17:18:20.179269+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-13T17:18:20.178917+0000","last_deep_scrub":"64'19","last_deep_scrub_stamp":"2024-09-13T17:18:20.178917+0000","last_clean_scrub_stamp":"2024-09-13T17:18:20.178917+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:19.604304+0000","scrub_duration":0.010053862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":324,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"63'3189","reported_seq":3253,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591477+0000","last_change":"2024-09-13T16:48:08.202913+0000","last_active":"2024-09-13T17:17:49.591477+0000","last_peered":"2024-09-13T17:17:49.591477+0000","last_clean":"2024-09-13T17:17:49.591477+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T17:17:49.591477+0000","last_undegraded":"2024-09-13T17:17:49.591477+0000","last_fullsized":"2024-09-13T17:17:49.591477+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17342,"num_write_kb":249797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"63'4223","reported_seq":4301,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:06.854959+0000","last_change":"2024-09-13T17:19:06.855194+0000","last_active":"2024-09-13T17:19:06.854959+0000","last_peered":"2024-09-13T17:19:06.854959+0000","last_clean":"2024-09-13T17:19:06.854959+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T17:19:06.854959+0000","last_undegraded":"2024-09-13T17:19:06.854959+0000","last_fullsized":"2024-09-13T17:19:06.854959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4223","last_scrub_stamp":"2024-09-13T17:19:06.854904+0000","last_deep_scrub":"63'4223","last_deep_scrub_stamp":"2024-09-13T17:19:06.854904+0000","last_clean_scrub_stamp":"2024-09-13T17:19:06.854904+0000","objects_scrubbed":0,"log_size":4223,"log_dups_size":0,"ondisk_log_size":4223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:02.698329+0000","scrub_duration":0.242061791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":295025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"64'5","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:06.450397+0000","last_change":"2024-09-13T17:18:06.450600+0000","last_active":"2024-09-13T17:18:06.450397+0000","last_peered":"2024-09-13T17:18:06.450397+0000","last_clean":"2024-09-13T17:18:06.450397+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T17:18:06.450397+0000","last_undegraded":"2024-09-13T17:18:06.450397+0000","last_fullsized":"2024-09-13T17:18:06.450397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_clean_scrub_stamp":"2024-09-13T17:18:06.450341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:29.338436+0000","scrub_duration":0.003290758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.123757+0000","last_change":"2024-09-13T17:17:53.123757+0000","last_active":"2024-09-13T17:17:53.123757+0000","last_peered":"2024-09-13T17:17:53.123757+0000","last_clean":"2024-09-13T17:17:53.123757+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T17:17:53.123757+0000","last_undegraded":"2024-09-13T17:17:53.123757+0000","last_fullsized":"2024-09-13T17:17:53.123757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.123466+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.011562+0000","scrub_duration":0.0038056650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'5357","reported_seq":5436,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.115658+0000","last_change":"2024-09-13T17:18:43.115905+0000","last_active":"2024-09-13T17:18:43.115658+0000","last_peered":"2024-09-13T17:18:43.115658+0000","last_clean":"2024-09-13T17:18:43.115658+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T17:18:43.115658+0000","last_undegraded":"2024-09-13T17:18:43.115658+0000","last_fullsized":"2024-09-13T17:18:43.115658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5357","last_scrub_stamp":"2024-09-13T17:18:43.115582+0000","last_deep_scrub":"63'5357","last_deep_scrub_stamp":"2024-09-13T17:18:43.115582+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.115582+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:20.465692+0000","scrub_duration":0.033536497999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":39,"num_write":29637,"num_write_kb":393569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"63'26","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:08.229713+0000","last_change":"2024-09-13T17:19:08.229943+0000","last_active":"2024-09-13T17:19:08.229713+0000","last_peered":"2024-09-13T17:19:08.229713+0000","last_clean":"2024-09-13T17:19:08.229713+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T17:19:08.229713+0000","last_undegraded":"2024-09-13T17:19:08.229713+0000","last_fullsized":"2024-09-13T17:19:08.229713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'26","last_scrub_stamp":"2024-09-13T17:19:08.229660+0000","last_deep_scrub":"63'26","last_deep_scrub_stamp":"2024-09-13T17:19:08.229660+0000","last_clean_scrub_stamp":"2024-09-13T17:19:08.229660+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:18.774478+0000","scrub_duration":0.0036406939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"63'5122","reported_seq":5200,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.190845+0000","last_change":"2024-09-13T17:18:01.191064+0000","last_active":"2024-09-13T17:18:01.190845+0000","last_peered":"2024-09-13T17:18:01.190845+0000","last_clean":"2024-09-13T17:18:01.190845+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T17:18:01.190845+0000","last_undegraded":"2024-09-13T17:18:01.190845+0000","last_fullsized":"2024-09-13T17:18:01.190845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5122","last_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_deep_scrub":"63'5122","last_deep_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.190787+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:03.825004+0000","scrub_duration":0.032904946999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":348704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"63'2","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.623848+0000","last_change":"2024-09-13T17:18:45.624092+0000","last_active":"2024-09-13T17:18:45.623848+0000","last_peered":"2024-09-13T17:18:45.623848+0000","last_clean":"2024-09-13T17:18:45.623848+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T17:18:45.623848+0000","last_undegraded":"2024-09-13T17:18:45.623848+0000","last_fullsized":"2024-09-13T17:18:45.623848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:45.623791+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:45.623791+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.623791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:26.049876+0000","scrub_duration":0.003142663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"63'3554","reported_seq":3620,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.207332+0000","last_change":"2024-09-13T16:48:08.187103+0000","last_active":"2024-09-13T17:17:46.207332+0000","last_peered":"2024-09-13T17:17:46.207332+0000","last_clean":"2024-09-13T17:17:46.207332+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T17:17:46.207332+0000","last_undegraded":"2024-09-13T17:17:46.207332+0000","last_fullsized":"2024-09-13T17:17:46.207332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":268389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"64'1094","reported_seq":1221,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.174321+0000","last_change":"2024-09-13T17:18:29.174543+0000","last_active":"2024-09-13T17:18:29.174321+0000","last_peered":"2024-09-13T17:18:29.174321+0000","last_clean":"2024-09-13T17:18:29.174321+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T17:18:29.174321+0000","last_undegraded":"2024-09-13T17:18:29.174321+0000","last_fullsized":"2024-09-13T17:18:29.174321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1094","last_scrub_stamp":"2024-09-13T17:18:29.174267+0000","last_deep_scrub":"64'1094","last_deep_scrub_stamp":"2024-09-13T17:18:29.174267+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.174267+0000","objects_scrubbed":3,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:28.446149+0000","scrub_duration":0.031395304999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":9436,"num_write":1065,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"63'4714","reported_seq":4792,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:32.980839+0000","last_change":"2024-09-13T17:18:32.981101+0000","last_active":"2024-09-13T17:18:32.980839+0000","last_peered":"2024-09-13T17:18:32.980839+0000","last_clean":"2024-09-13T17:18:32.980839+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T17:18:32.980839+0000","last_undegraded":"2024-09-13T17:18:32.980839+0000","last_fullsized":"2024-09-13T17:18:32.980839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4714","last_scrub_stamp":"2024-09-13T17:18:32.980782+0000","last_deep_scrub":"63'4714","last_deep_scrub_stamp":"2024-09-13T17:18:32.980782+0000","last_clean_scrub_stamp":"2024-09-13T17:18:32.980782+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:43.117628+0000","scrub_duration":0.031918914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25459,"num_write_kb":343662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"63'3","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.965455+0000","last_change":"2024-09-13T17:18:43.965730+0000","last_active":"2024-09-13T17:18:43.965455+0000","last_peered":"2024-09-13T17:18:43.965455+0000","last_clean":"2024-09-13T17:18:43.965455+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T17:18:43.965455+0000","last_undegraded":"2024-09-13T17:18:43.965455+0000","last_fullsized":"2024-09-13T17:18:43.965455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:43.965393+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:43.965393+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.965393+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:24.391478+0000","scrub_duration":0.0038411180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"63'4328","reported_seq":4391,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675065+0000","last_change":"2024-09-13T16:48:08.205178+0000","last_active":"2024-09-13T17:17:46.675065+0000","last_peered":"2024-09-13T17:17:46.675065+0000","last_clean":"2024-09-13T17:17:46.675065+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T17:17:46.675065+0000","last_undegraded":"2024-09-13T17:17:46.675065+0000","last_fullsized":"2024-09-13T17:17:46.675065+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24456,"num_write_kb":315374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"64'411","reported_seq":494,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.515483+0000","last_change":"2024-09-13T17:18:03.515735+0000","last_active":"2024-09-13T17:18:03.515483+0000","last_peered":"2024-09-13T17:18:03.515483+0000","last_clean":"2024-09-13T17:18:03.515483+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T17:18:03.515483+0000","last_undegraded":"2024-09-13T17:18:03.515483+0000","last_fullsized":"2024-09-13T17:18:03.515483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'411","last_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_deep_scrub":"64'411","last_deep_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.515426+0000","objects_scrubbed":1,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:06.149643+0000","scrub_duration":0.019312658999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2812346,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3190,"num_write":386,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"63'4155","reported_seq":4233,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:20.944870+0000","last_change":"2024-09-13T17:18:20.945097+0000","last_active":"2024-09-13T17:18:20.944870+0000","last_peered":"2024-09-13T17:18:20.944870+0000","last_clean":"2024-09-13T17:18:20.944870+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T17:18:20.944870+0000","last_undegraded":"2024-09-13T17:18:20.944870+0000","last_fullsized":"2024-09-13T17:18:20.944870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4155","last_scrub_stamp":"2024-09-13T17:18:20.944817+0000","last_deep_scrub":"63'4155","last_deep_scrub_stamp":"2024-09-13T17:18:20.944817+0000","last_clean_scrub_stamp":"2024-09-13T17:18:20.944817+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:43.832912+0000","scrub_duration":0.029833493999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22247,"num_write_kb":285891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'6","reported_seq":123,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.503784+0000","last_change":"2024-09-13T17:18:10.504013+0000","last_active":"2024-09-13T17:18:10.503784+0000","last_peered":"2024-09-13T17:18:10.503784+0000","last_clean":"2024-09-13T17:18:10.503784+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T17:18:10.503784+0000","last_undegraded":"2024-09-13T17:18:10.503784+0000","last_fullsized":"2024-09-13T17:18:10.503784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-09-13T17:18:10.503729+0000","last_deep_scrub":"64'6","last_deep_scrub_stamp":"2024-09-13T17:18:10.503729+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.503729+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:33.391824+0000","scrub_duration":0.0063226599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"63'3426","reported_seq":3503,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:57.209418+0000","last_change":"2024-09-13T17:18:57.209707+0000","last_active":"2024-09-13T17:18:57.209418+0000","last_peered":"2024-09-13T17:18:57.209418+0000","last_clean":"2024-09-13T17:18:57.209418+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T17:18:57.209418+0000","last_undegraded":"2024-09-13T17:18:57.209418+0000","last_fullsized":"2024-09-13T17:18:57.209418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3426","last_scrub_stamp":"2024-09-13T17:18:57.209365+0000","last_deep_scrub":"63'3426","last_deep_scrub_stamp":"2024-09-13T17:18:57.209365+0000","last_clean_scrub_stamp":"2024-09-13T17:18:57.209365+0000","objects_scrubbed":0,"log_size":3426,"log_dups_size":0,"ondisk_log_size":3426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:37.635450+0000","scrub_duration":0.029481411999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":18510,"num_write_kb":250514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"64'713","reported_seq":815,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:52.922610+0000","last_change":"2024-09-13T17:18:52.922907+0000","last_active":"2024-09-13T17:18:52.922610+0000","last_peered":"2024-09-13T17:18:52.922610+0000","last_clean":"2024-09-13T17:18:52.922610+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T17:18:52.922610+0000","last_undegraded":"2024-09-13T17:18:52.922610+0000","last_fullsized":"2024-09-13T17:18:52.922610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'713","last_scrub_stamp":"2024-09-13T17:18:52.922549+0000","last_deep_scrub":"64'713","last_deep_scrub_stamp":"2024-09-13T17:18:52.922549+0000","last_clean_scrub_stamp":"2024-09-13T17:18:52.922549+0000","objects_scrubbed":1,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:28.967244+0000","scrub_duration":0.025846372999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":74674,"num_write":702,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"63'4316","reported_seq":4395,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.250446+0000","last_change":"2024-09-13T17:18:10.250685+0000","last_active":"2024-09-13T17:18:10.250446+0000","last_peered":"2024-09-13T17:18:10.250446+0000","last_clean":"2024-09-13T17:18:10.250446+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T17:18:10.250446+0000","last_undegraded":"2024-09-13T17:18:10.250446+0000","last_fullsized":"2024-09-13T17:18:10.250446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4316","last_scrub_stamp":"2024-09-13T17:18:10.250390+0000","last_deep_scrub":"63'4316","last_deep_scrub_stamp":"2024-09-13T17:18:10.250390+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.250390+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:02.756098+0000","scrub_duration":0.033652449000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22350,"num_write_kb":290228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"64'45","reported_seq":162,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.128882+0000","last_change":"2024-09-13T17:17:59.128882+0000","last_active":"2024-09-13T17:17:59.128882+0000","last_peered":"2024-09-13T17:17:59.128882+0000","last_clean":"2024-09-13T17:17:59.128882+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T17:17:59.128882+0000","last_undegraded":"2024-09-13T17:17:59.128882+0000","last_fullsized":"2024-09-13T17:17:59.128882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'45","last_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_deep_scrub":"64'45","last_deep_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.128560+0000","objects_scrubbed":1,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.106052+0000","scrub_duration":0.0062496970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"63'3129","reported_seq":3194,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632026+0000","last_change":"2024-09-13T16:48:08.188266+0000","last_active":"2024-09-13T17:17:47.632026+0000","last_peered":"2024-09-13T17:17:47.632026+0000","last_clean":"2024-09-13T17:17:47.632026+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T17:17:47.632026+0000","last_undegraded":"2024-09-13T17:17:47.632026+0000","last_fullsized":"2024-09-13T17:17:47.632026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17279,"num_write_kb":257601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"63'4785","reported_seq":4865,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.565656+0000","last_change":"2024-09-13T17:17:49.565883+0000","last_active":"2024-09-13T17:17:49.565656+0000","last_peered":"2024-09-13T17:17:49.565656+0000","last_clean":"2024-09-13T17:17:49.565656+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T17:17:49.565656+0000","last_undegraded":"2024-09-13T17:17:49.565656+0000","last_fullsized":"2024-09-13T17:17:49.565656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4785","last_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_deep_scrub":"63'4785","last_deep_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.565599+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.011489+0000","scrub_duration":0.012593717000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26302,"num_write_kb":310249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"63'4231","reported_seq":4297,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675102+0000","last_change":"2024-09-13T16:48:08.181470+0000","last_active":"2024-09-13T17:17:46.675102+0000","last_peered":"2024-09-13T17:17:46.675102+0000","last_clean":"2024-09-13T17:17:46.675102+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T17:17:46.675102+0000","last_undegraded":"2024-09-13T17:17:46.675102+0000","last_fullsized":"2024-09-13T17:17:46.675102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22604,"num_write_kb":324065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"63'3874","reported_seq":3955,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.662237+0000","last_change":"2024-09-13T17:18:04.662513+0000","last_active":"2024-09-13T17:18:04.662237+0000","last_peered":"2024-09-13T17:18:04.662237+0000","last_clean":"2024-09-13T17:18:04.662237+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T17:18:04.662237+0000","last_undegraded":"2024-09-13T17:18:04.662237+0000","last_fullsized":"2024-09-13T17:18:04.662237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.662182+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:07.296399+0000","scrub_duration":0.021443912999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20920,"num_write_kb":276387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"63'3865","reported_seq":3942,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:32.467133+0000","last_change":"2024-09-13T17:18:32.467133+0000","last_active":"2024-09-13T17:18:32.467133+0000","last_peered":"2024-09-13T17:18:32.467133+0000","last_clean":"2024-09-13T17:18:32.467133+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T17:18:32.467133+0000","last_undegraded":"2024-09-13T17:18:32.467133+0000","last_fullsized":"2024-09-13T17:18:32.467133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3865","last_scrub_stamp":"2024-09-13T17:18:32.466850+0000","last_deep_scrub":"63'3865","last_deep_scrub_stamp":"2024-09-13T17:18:32.466850+0000","last_clean_scrub_stamp":"2024-09-13T17:18:32.466850+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:09.816960+0000","scrub_duration":0.029037197000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21432,"num_write_kb":286796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"63'4552","reported_seq":4634,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.654626+0000","last_change":"2024-09-13T17:19:05.654868+0000","last_active":"2024-09-13T17:19:05.654626+0000","last_peered":"2024-09-13T17:19:05.654626+0000","last_clean":"2024-09-13T17:19:05.654626+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T17:19:05.654626+0000","last_undegraded":"2024-09-13T17:19:05.654626+0000","last_fullsized":"2024-09-13T17:19:05.654626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4552","last_scrub_stamp":"2024-09-13T17:19:05.654571+0000","last_deep_scrub":"63'4552","last_deep_scrub_stamp":"2024-09-13T17:19:05.654571+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.654571+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:34.257461+0000","scrub_duration":0.029954933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":24031,"num_write_kb":313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"63'4071","reported_seq":4137,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675131+0000","last_change":"2024-09-13T16:48:08.205486+0000","last_active":"2024-09-13T17:17:46.675131+0000","last_peered":"2024-09-13T17:17:46.675131+0000","last_clean":"2024-09-13T17:17:46.675131+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T17:17:46.675131+0000","last_undegraded":"2024-09-13T17:17:46.675131+0000","last_fullsized":"2024-09-13T17:17:46.675131+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":22061,"num_write_kb":309650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"63'3506","reported_seq":3587,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:47.181003+0000","last_change":"2024-09-13T17:18:47.181238+0000","last_active":"2024-09-13T17:18:47.181003+0000","last_peered":"2024-09-13T17:18:47.181003+0000","last_clean":"2024-09-13T17:18:47.181003+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T17:18:47.181003+0000","last_undegraded":"2024-09-13T17:18:47.181003+0000","last_fullsized":"2024-09-13T17:18:47.181003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3506","last_scrub_stamp":"2024-09-13T17:18:47.180948+0000","last_deep_scrub":"63'3506","last_deep_scrub_stamp":"2024-09-13T17:18:47.180948+0000","last_clean_scrub_stamp":"2024-09-13T17:18:47.180948+0000","objects_scrubbed":0,"log_size":3506,"log_dups_size":0,"ondisk_log_size":3506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:46.606335+0000","scrub_duration":0.025510459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19493,"num_write_kb":254380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"63'4523","reported_seq":4599,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:09.440008+0000","last_change":"2024-09-13T17:19:09.440708+0000","last_active":"2024-09-13T17:19:09.440008+0000","last_peered":"2024-09-13T17:19:09.440008+0000","last_clean":"2024-09-13T17:19:09.440008+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T17:19:09.440008+0000","last_undegraded":"2024-09-13T17:19:09.440008+0000","last_fullsized":"2024-09-13T17:19:09.440008+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4523","last_scrub_stamp":"2024-09-13T17:19:09.439955+0000","last_deep_scrub":"63'4523","last_deep_scrub_stamp":"2024-09-13T17:19:09.439955+0000","last_clean_scrub_stamp":"2024-09-13T17:19:09.439955+0000","objects_scrubbed":0,"log_size":4523,"log_dups_size":0,"ondisk_log_size":4523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:50.148915+0000","scrub_duration":0.035549714000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24221,"num_write_kb":338849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"63'4422","reported_seq":4503,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:00.945847+0000","last_change":"2024-09-13T17:19:00.946067+0000","last_active":"2024-09-13T17:19:00.945847+0000","last_peered":"2024-09-13T17:19:00.945847+0000","last_clean":"2024-09-13T17:19:00.945847+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T17:19:00.945847+0000","last_undegraded":"2024-09-13T17:19:00.945847+0000","last_fullsized":"2024-09-13T17:19:00.945847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4422","last_scrub_stamp":"2024-09-13T17:19:00.945792+0000","last_deep_scrub":"63'4422","last_deep_scrub_stamp":"2024-09-13T17:19:00.945792+0000","last_clean_scrub_stamp":"2024-09-13T17:19:00.945792+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:42.479112+0000","scrub_duration":0.032777075000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":24862,"num_write_kb":304752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"63'3771","reported_seq":3837,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976919+0000","last_change":"2024-09-13T16:48:08.175884+0000","last_active":"2024-09-13T17:17:44.976919+0000","last_peered":"2024-09-13T17:17:44.976919+0000","last_clean":"2024-09-13T17:17:44.976919+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T17:17:44.976919+0000","last_undegraded":"2024-09-13T17:17:44.976919+0000","last_fullsized":"2024-09-13T17:17:44.976919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21135,"num_write_kb":285540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"63'4461","reported_seq":4541,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:31.181911+0000","last_change":"2024-09-13T17:18:31.182225+0000","last_active":"2024-09-13T17:18:31.181911+0000","last_peered":"2024-09-13T17:18:31.181911+0000","last_clean":"2024-09-13T17:18:31.181911+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T17:18:31.181911+0000","last_undegraded":"2024-09-13T17:18:31.181911+0000","last_fullsized":"2024-09-13T17:18:31.181911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4461","last_scrub_stamp":"2024-09-13T17:18:31.181856+0000","last_deep_scrub":"63'4461","last_deep_scrub_stamp":"2024-09-13T17:18:31.181856+0000","last_clean_scrub_stamp":"2024-09-13T17:18:31.181856+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:30.453738+0000","scrub_duration":0.032702224000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25388,"num_write_kb":319094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"63'4193","reported_seq":4258,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991839+0000","last_change":"2024-09-13T16:48:08.201046+0000","last_active":"2024-09-13T17:17:44.991839+0000","last_peered":"2024-09-13T17:17:44.991839+0000","last_clean":"2024-09-13T17:17:44.991839+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T17:17:44.991839+0000","last_undegraded":"2024-09-13T17:17:44.991839+0000","last_fullsized":"2024-09-13T17:17:44.991839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4193,"log_dups_size":0,"ondisk_log_size":4193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23066,"num_write_kb":313713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"63'3978","reported_seq":4058,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.616395+0000","last_change":"2024-09-13T17:17:57.616624+0000","last_active":"2024-09-13T17:17:57.616395+0000","last_peered":"2024-09-13T17:17:57.616395+0000","last_clean":"2024-09-13T17:17:57.616395+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T17:17:57.616395+0000","last_undegraded":"2024-09-13T17:17:57.616395+0000","last_fullsized":"2024-09-13T17:17:57.616395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3978","last_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_deep_scrub":"63'3978","last_deep_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.616344+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:01.359043+0000","scrub_duration":0.024757745000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22157,"num_write_kb":281285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"63'4050","reported_seq":4128,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:54.092899+0000","last_change":"2024-09-13T17:18:54.093155+0000","last_active":"2024-09-13T17:18:54.092899+0000","last_peered":"2024-09-13T17:18:54.092899+0000","last_clean":"2024-09-13T17:18:54.092899+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T17:18:54.092899+0000","last_undegraded":"2024-09-13T17:18:54.092899+0000","last_fullsized":"2024-09-13T17:18:54.092899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4050","last_scrub_stamp":"2024-09-13T17:18:54.092845+0000","last_deep_scrub":"63'4050","last_deep_scrub_stamp":"2024-09-13T17:18:54.092845+0000","last_clean_scrub_stamp":"2024-09-13T17:18:54.092845+0000","objects_scrubbed":0,"log_size":4050,"log_dups_size":0,"ondisk_log_size":4050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:43.484599+0000","scrub_duration":0.012829612000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21731,"num_write_kb":291923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"63'5211","reported_seq":5286,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:57.601433+0000","last_change":"2024-09-13T17:18:57.601433+0000","last_active":"2024-09-13T17:18:57.601433+0000","last_peered":"2024-09-13T17:18:57.601433+0000","last_clean":"2024-09-13T17:18:57.601433+0000","last_became_active":"2024-09-13T16:48:40.831649+0000","last_became_peered":"2024-09-13T16:48:40.831649+0000","last_unstale":"2024-09-13T17:18:57.601433+0000","last_undegraded":"2024-09-13T17:18:57.601433+0000","last_fullsized":"2024-09-13T17:18:57.601433+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5211","last_scrub_stamp":"2024-09-13T17:18:57.601109+0000","last_deep_scrub":"63'5211","last_deep_scrub_stamp":"2024-09-13T17:18:57.601109+0000","last_clean_scrub_stamp":"2024-09-13T17:18:57.601109+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:08.145927+0000","scrub_duration":0.03195572,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":28065,"num_write_kb":366501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"63'3806","reported_seq":3873,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632082+0000","last_change":"2024-09-13T16:48:08.166249+0000","last_active":"2024-09-13T17:17:47.632082+0000","last_peered":"2024-09-13T17:17:47.632082+0000","last_clean":"2024-09-13T17:17:47.632082+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T17:17:47.632082+0000","last_undegraded":"2024-09-13T17:17:47.632082+0000","last_fullsized":"2024-09-13T17:17:47.632082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4,"num_write":20286,"num_write_kb":286237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"63'3659","reported_seq":3740,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:34.530920+0000","last_change":"2024-09-13T17:18:34.531140+0000","last_active":"2024-09-13T17:18:34.530920+0000","last_peered":"2024-09-13T17:18:34.530920+0000","last_clean":"2024-09-13T17:18:34.530920+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T17:18:34.530920+0000","last_undegraded":"2024-09-13T17:18:34.530920+0000","last_fullsized":"2024-09-13T17:18:34.530920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3659","last_scrub_stamp":"2024-09-13T17:18:34.530868+0000","last_deep_scrub":"63'3659","last_deep_scrub_stamp":"2024-09-13T17:18:34.530868+0000","last_clean_scrub_stamp":"2024-09-13T17:18:34.530868+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:29.832112+0000","scrub_duration":0.027670581999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19936,"num_write_kb":264332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"63'5003","reported_seq":5068,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591294+0000","last_change":"2024-09-13T16:48:08.195018+0000","last_active":"2024-09-13T17:17:49.591294+0000","last_peered":"2024-09-13T17:17:49.591294+0000","last_clean":"2024-09-13T17:17:49.591294+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T17:17:49.591294+0000","last_undegraded":"2024-09-13T17:17:49.591294+0000","last_fullsized":"2024-09-13T17:17:49.591294+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27137,"num_write_kb":387189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"63'4068","reported_seq":4147,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:30.029523+0000","last_change":"2024-09-13T17:18:30.029753+0000","last_active":"2024-09-13T17:18:30.029523+0000","last_peered":"2024-09-13T17:18:30.029523+0000","last_clean":"2024-09-13T17:18:30.029523+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T17:18:30.029523+0000","last_undegraded":"2024-09-13T17:18:30.029523+0000","last_fullsized":"2024-09-13T17:18:30.029523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4068","last_scrub_stamp":"2024-09-13T17:18:30.029466+0000","last_deep_scrub":"63'4068","last_deep_scrub_stamp":"2024-09-13T17:18:30.029466+0000","last_clean_scrub_stamp":"2024-09-13T17:18:30.029466+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:47.003161+0000","scrub_duration":0.033158017999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22149,"num_write_kb":293545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"63'3283","reported_seq":3358,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:47.724019+0000","last_change":"2024-09-13T17:18:47.724256+0000","last_active":"2024-09-13T17:18:47.724019+0000","last_peered":"2024-09-13T17:18:47.724019+0000","last_clean":"2024-09-13T17:18:47.724019+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T17:18:47.724019+0000","last_undegraded":"2024-09-13T17:18:47.724019+0000","last_fullsized":"2024-09-13T17:18:47.724019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3283","last_scrub_stamp":"2024-09-13T17:18:47.723967+0000","last_deep_scrub":"63'3283","last_deep_scrub_stamp":"2024-09-13T17:18:47.723967+0000","last_clean_scrub_stamp":"2024-09-13T17:18:47.723967+0000","objects_scrubbed":0,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:25.074078+0000","scrub_duration":0.018868821000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":248866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"63'4697","reported_seq":4776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:13.654346+0000","last_change":"2024-09-13T17:18:13.654578+0000","last_active":"2024-09-13T17:18:13.654346+0000","last_peered":"2024-09-13T17:18:13.654346+0000","last_clean":"2024-09-13T17:18:13.654346+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T17:18:13.654346+0000","last_undegraded":"2024-09-13T17:18:13.654346+0000","last_fullsized":"2024-09-13T17:18:13.654346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4697","last_scrub_stamp":"2024-09-13T17:18:13.654295+0000","last_deep_scrub":"63'4697","last_deep_scrub_stamp":"2024-09-13T17:18:13.654295+0000","last_clean_scrub_stamp":"2024-09-13T17:18:13.654295+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:58.717333+0000","scrub_duration":0.035392587000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25800,"num_write_kb":320606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"63'3820","reported_seq":3884,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974366+0000","last_change":"2024-09-13T16:48:08.164698+0000","last_active":"2024-09-13T17:17:44.974366+0000","last_peered":"2024-09-13T17:17:44.974366+0000","last_clean":"2024-09-13T17:17:44.974366+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T17:17:44.974366+0000","last_undegraded":"2024-09-13T17:17:44.974366+0000","last_fullsized":"2024-09-13T17:17:44.974366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3820,"log_dups_size":0,"ondisk_log_size":3820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20813,"num_write_kb":300503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"63'4719","reported_seq":4800,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:56.892882+0000","last_change":"2024-09-13T17:17:56.893148+0000","last_active":"2024-09-13T17:17:56.892882+0000","last_peered":"2024-09-13T17:17:56.892882+0000","last_clean":"2024-09-13T17:17:56.892882+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T17:17:56.892882+0000","last_undegraded":"2024-09-13T17:17:56.892882+0000","last_fullsized":"2024-09-13T17:17:56.892882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4719","last_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_deep_scrub":"63'4719","last_deep_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_clean_scrub_stamp":"2024-09-13T17:17:56.892826+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:26.656787+0000","scrub_duration":0.032626751000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":25263,"num_write_kb":330973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"63'3722","reported_seq":3797,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:03.285307+0000","last_change":"2024-09-13T17:19:03.285533+0000","last_active":"2024-09-13T17:19:03.285307+0000","last_peered":"2024-09-13T17:19:03.285307+0000","last_clean":"2024-09-13T17:19:03.285307+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T17:19:03.285307+0000","last_undegraded":"2024-09-13T17:19:03.285307+0000","last_fullsized":"2024-09-13T17:19:03.285307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3722","last_scrub_stamp":"2024-09-13T17:19:03.285252+0000","last_deep_scrub":"63'3722","last_deep_scrub_stamp":"2024-09-13T17:19:03.285252+0000","last_clean_scrub_stamp":"2024-09-13T17:19:03.285252+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:19.415797+0000","scrub_duration":0.032938005999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20022,"num_write_kb":289122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"63'3813","reported_seq":3892,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:26.209831+0000","last_change":"2024-09-13T17:18:26.210107+0000","last_active":"2024-09-13T17:18:26.209831+0000","last_peered":"2024-09-13T17:18:26.209831+0000","last_clean":"2024-09-13T17:18:26.209831+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T17:18:26.209831+0000","last_undegraded":"2024-09-13T17:18:26.209831+0000","last_fullsized":"2024-09-13T17:18:26.209831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3813","last_scrub_stamp":"2024-09-13T17:18:26.209775+0000","last_deep_scrub":"63'3813","last_deep_scrub_stamp":"2024-09-13T17:18:26.209775+0000","last_clean_scrub_stamp":"2024-09-13T17:18:26.209775+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:11.272813+0000","scrub_duration":0.022253663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21202,"num_write_kb":265282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"63'4479","reported_seq":4547,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.976201+0000","last_change":"2024-09-13T16:48:08.186344+0000","last_active":"2024-09-13T17:17:44.976201+0000","last_peered":"2024-09-13T17:17:44.976201+0000","last_clean":"2024-09-13T17:17:44.976201+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T17:17:44.976201+0000","last_undegraded":"2024-09-13T17:17:44.976201+0000","last_fullsized":"2024-09-13T17:17:44.976201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":24556,"num_write_kb":340437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"63'4549","reported_seq":4628,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:00.399885+0000","last_change":"2024-09-13T17:19:00.400112+0000","last_active":"2024-09-13T17:19:00.399885+0000","last_peered":"2024-09-13T17:19:00.399885+0000","last_clean":"2024-09-13T17:19:00.399885+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T17:19:00.399885+0000","last_undegraded":"2024-09-13T17:19:00.399885+0000","last_fullsized":"2024-09-13T17:19:00.399885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4549","last_scrub_stamp":"2024-09-13T17:19:00.399832+0000","last_deep_scrub":"63'4549","last_deep_scrub_stamp":"2024-09-13T17:19:00.399832+0000","last_clean_scrub_stamp":"2024-09-13T17:19:00.399832+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:41.933152+0000","scrub_duration":0.029616984999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24549,"num_write_kb":319591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"63'3479","reported_seq":3545,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.643279+0000","last_change":"2024-09-13T16:48:08.180859+0000","last_active":"2024-09-13T17:17:48.643279+0000","last_peered":"2024-09-13T17:17:48.643279+0000","last_clean":"2024-09-13T17:17:48.643279+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T17:17:48.643279+0000","last_undegraded":"2024-09-13T17:17:48.643279+0000","last_fullsized":"2024-09-13T17:17:48.643279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3479,"log_dups_size":0,"ondisk_log_size":3479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18966,"num_write_kb":268432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"63'3573","reported_seq":3653,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:08.568217+0000","last_change":"2024-09-13T17:18:08.568217+0000","last_active":"2024-09-13T17:18:08.568217+0000","last_peered":"2024-09-13T17:18:08.568217+0000","last_clean":"2024-09-13T17:18:08.568217+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T17:18:08.568217+0000","last_undegraded":"2024-09-13T17:18:08.568217+0000","last_fullsized":"2024-09-13T17:18:08.568217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3573","last_scrub_stamp":"2024-09-13T17:18:08.567856+0000","last_deep_scrub":"63'3573","last_deep_scrub_stamp":"2024-09-13T17:18:08.567856+0000","last_clean_scrub_stamp":"2024-09-13T17:18:08.567856+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:07.880974+0000","scrub_duration":0.01354239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19348,"num_write_kb":270113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"63'4058","reported_seq":4122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.991645+0000","last_change":"2024-09-13T16:48:08.194914+0000","last_active":"2024-09-13T17:17:44.991645+0000","last_peered":"2024-09-13T17:17:44.991645+0000","last_clean":"2024-09-13T17:17:44.991645+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T17:17:44.991645+0000","last_undegraded":"2024-09-13T17:17:44.991645+0000","last_fullsized":"2024-09-13T17:17:44.991645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22331,"num_write_kb":302585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"63'5839","reported_seq":5918,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.655103+0000","last_change":"2024-09-13T17:18:44.655336+0000","last_active":"2024-09-13T17:18:44.655103+0000","last_peered":"2024-09-13T17:18:44.655103+0000","last_clean":"2024-09-13T17:18:44.655103+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T17:18:44.655103+0000","last_undegraded":"2024-09-13T17:18:44.655103+0000","last_fullsized":"2024-09-13T17:18:44.655103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5839","last_scrub_stamp":"2024-09-13T17:18:44.655047+0000","last_deep_scrub":"63'5839","last_deep_scrub_stamp":"2024-09-13T17:18:44.655047+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.655047+0000","objects_scrubbed":0,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:17.349629+0000","scrub_duration":0.042557057000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31756,"num_write_kb":397521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"63'3835","reported_seq":3910,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.213984+0000","last_change":"2024-09-13T17:19:05.214237+0000","last_active":"2024-09-13T17:19:05.213984+0000","last_peered":"2024-09-13T17:19:05.213984+0000","last_clean":"2024-09-13T17:19:05.213984+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T17:19:05.213984+0000","last_undegraded":"2024-09-13T17:19:05.213984+0000","last_fullsized":"2024-09-13T17:19:05.213984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3835","last_scrub_stamp":"2024-09-13T17:19:05.213932+0000","last_deep_scrub":"63'3835","last_deep_scrub_stamp":"2024-09-13T17:19:05.213932+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.213932+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:23:44.268101+0000","scrub_duration":0.028321004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":21185,"num_write_kb":306173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"63'4134","reported_seq":4213,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.595782+0000","last_change":"2024-09-13T17:17:57.595782+0000","last_active":"2024-09-13T17:17:57.595782+0000","last_peered":"2024-09-13T17:17:57.595782+0000","last_clean":"2024-09-13T17:17:57.595782+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T17:17:57.595782+0000","last_undegraded":"2024-09-13T17:17:57.595782+0000","last_fullsized":"2024-09-13T17:17:57.595782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4134","last_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_deep_scrub":"63'4134","last_deep_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.595505+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:50.101213+0000","scrub_duration":0.024248147000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22268,"num_write_kb":281801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"63'5073","reported_seq":5149,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:06.320617+0000","last_change":"2024-09-13T17:19:06.320883+0000","last_active":"2024-09-13T17:19:06.320617+0000","last_peered":"2024-09-13T17:19:06.320617+0000","last_clean":"2024-09-13T17:19:06.320617+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T17:19:06.320617+0000","last_undegraded":"2024-09-13T17:19:06.320617+0000","last_fullsized":"2024-09-13T17:19:06.320617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5073","last_scrub_stamp":"2024-09-13T17:19:06.320563+0000","last_deep_scrub":"63'5073","last_deep_scrub_stamp":"2024-09-13T17:19:06.320563+0000","last_clean_scrub_stamp":"2024-09-13T17:19:06.320563+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:42.365259+0000","scrub_duration":0.043951086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":28693,"num_write_kb":393724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"63'3822","reported_seq":3901,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:07.186357+0000","last_change":"2024-09-13T17:18:07.186357+0000","last_active":"2024-09-13T17:18:07.186357+0000","last_peered":"2024-09-13T17:18:07.186357+0000","last_clean":"2024-09-13T17:18:07.186357+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T17:18:07.186357+0000","last_undegraded":"2024-09-13T17:18:07.186357+0000","last_fullsized":"2024-09-13T17:18:07.186357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3822","last_scrub_stamp":"2024-09-13T17:18:07.186043+0000","last_deep_scrub":"63'3822","last_deep_scrub_stamp":"2024-09-13T17:18:07.186043+0000","last_clean_scrub_stamp":"2024-09-13T17:18:07.186043+0000","objects_scrubbed":0,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:30.074139+0000","scrub_duration":0.030687262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20823,"num_write_kb":275830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"63'2778","reported_seq":2844,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.975133+0000","last_change":"2024-09-13T16:48:08.182095+0000","last_active":"2024-09-13T17:17:44.975133+0000","last_peered":"2024-09-13T17:17:44.975133+0000","last_clean":"2024-09-13T17:17:44.975133+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T17:17:44.975133+0000","last_undegraded":"2024-09-13T17:17:44.975133+0000","last_fullsized":"2024-09-13T17:17:44.975133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":34,"num_write":15212,"num_write_kb":208573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"63'3975","reported_seq":4054,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.856922+0000","last_change":"2024-09-13T17:18:01.856922+0000","last_active":"2024-09-13T17:18:01.856922+0000","last_peered":"2024-09-13T17:18:01.856922+0000","last_clean":"2024-09-13T17:18:01.856922+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T17:18:01.856922+0000","last_undegraded":"2024-09-13T17:18:01.856922+0000","last_fullsized":"2024-09-13T17:18:01.856922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3975","last_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_deep_scrub":"63'3975","last_deep_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.856645+0000","objects_scrubbed":0,"log_size":3975,"log_dups_size":0,"ondisk_log_size":3975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:51.096023+0000","scrub_duration":0.021351690999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21351,"num_write_kb":278200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"63'3844","reported_seq":3920,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:17.247940+0000","last_change":"2024-09-13T17:18:17.248778+0000","last_active":"2024-09-13T17:18:17.247940+0000","last_peered":"2024-09-13T17:18:17.247940+0000","last_clean":"2024-09-13T17:18:17.247940+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T17:18:17.247940+0000","last_undegraded":"2024-09-13T17:18:17.247940+0000","last_fullsized":"2024-09-13T17:18:17.247940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3844","last_scrub_stamp":"2024-09-13T17:18:17.247886+0000","last_deep_scrub":"63'3844","last_deep_scrub_stamp":"2024-09-13T17:18:17.247886+0000","last_clean_scrub_stamp":"2024-09-13T17:18:17.247886+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:16.519768+0000","scrub_duration":0.029367863000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21085,"num_write_kb":289508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"63'3324","reported_seq":3402,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:18.177496+0000","last_change":"2024-09-13T17:18:18.177724+0000","last_active":"2024-09-13T17:18:18.177496+0000","last_peered":"2024-09-13T17:18:18.177496+0000","last_clean":"2024-09-13T17:18:18.177496+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T17:18:18.177496+0000","last_undegraded":"2024-09-13T17:18:18.177496+0000","last_fullsized":"2024-09-13T17:18:18.177496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3324","last_scrub_stamp":"2024-09-13T17:18:18.177437+0000","last_deep_scrub":"63'3324","last_deep_scrub_stamp":"2024-09-13T17:18:18.177437+0000","last_clean_scrub_stamp":"2024-09-13T17:18:18.177437+0000","objects_scrubbed":0,"log_size":3324,"log_dups_size":0,"ondisk_log_size":3324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:17.449318+0000","scrub_duration":0.024197943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18810,"num_write_kb":235312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"63'3743","reported_seq":3808,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632187+0000","last_change":"2024-09-13T16:48:08.170473+0000","last_active":"2024-09-13T17:17:47.632187+0000","last_peered":"2024-09-13T17:17:47.632187+0000","last_clean":"2024-09-13T17:17:47.632187+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T17:17:47.632187+0000","last_undegraded":"2024-09-13T17:17:47.632187+0000","last_fullsized":"2024-09-13T17:17:47.632187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20790,"num_write_kb":280180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"63'3173","reported_seq":3251,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.207709+0000","last_change":"2024-09-13T17:18:43.207942+0000","last_active":"2024-09-13T17:18:43.207709+0000","last_peered":"2024-09-13T17:18:43.207709+0000","last_clean":"2024-09-13T17:18:43.207709+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T17:18:43.207709+0000","last_undegraded":"2024-09-13T17:18:43.207709+0000","last_fullsized":"2024-09-13T17:18:43.207709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3173","last_scrub_stamp":"2024-09-13T17:18:43.207650+0000","last_deep_scrub":"63'3173","last_deep_scrub_stamp":"2024-09-13T17:18:43.207650+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.207650+0000","objects_scrubbed":0,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:00.181344+0000","scrub_duration":0.024785841999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17044,"num_write_kb":219853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"63'3460","reported_seq":3535,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.695358+0000","last_change":"2024-09-13T17:18:44.695678+0000","last_active":"2024-09-13T17:18:44.695358+0000","last_peered":"2024-09-13T17:18:44.695358+0000","last_clean":"2024-09-13T17:18:44.695358+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T17:18:44.695358+0000","last_undegraded":"2024-09-13T17:18:44.695358+0000","last_fullsized":"2024-09-13T17:18:44.695358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3460","last_scrub_stamp":"2024-09-13T17:18:44.695305+0000","last_deep_scrub":"63'3460","last_deep_scrub_stamp":"2024-09-13T17:18:44.695305+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.695305+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:57.484941+0000","scrub_duration":0.017970386000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":19788,"num_write_kb":263976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"63'3747","reported_seq":3825,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:12.464112+0000","last_change":"2024-09-13T17:18:12.464344+0000","last_active":"2024-09-13T17:18:12.464112+0000","last_peered":"2024-09-13T17:18:12.464112+0000","last_clean":"2024-09-13T17:18:12.464112+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T17:18:12.464112+0000","last_undegraded":"2024-09-13T17:18:12.464112+0000","last_fullsized":"2024-09-13T17:18:12.464112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3747","last_scrub_stamp":"2024-09-13T17:18:12.464055+0000","last_deep_scrub":"63'3747","last_deep_scrub_stamp":"2024-09-13T17:18:12.464055+0000","last_clean_scrub_stamp":"2024-09-13T17:18:12.464055+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:25.441547+0000","scrub_duration":0.021103311999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20231,"num_write_kb":269532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"63'3422","reported_seq":3497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:27.175049+0000","last_change":"2024-09-13T17:18:27.175049+0000","last_active":"2024-09-13T17:18:27.175049+0000","last_peered":"2024-09-13T17:18:27.175049+0000","last_clean":"2024-09-13T17:18:27.175049+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T17:18:27.175049+0000","last_undegraded":"2024-09-13T17:18:27.175049+0000","last_fullsized":"2024-09-13T17:18:27.175049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3422","last_scrub_stamp":"2024-09-13T17:18:27.174746+0000","last_deep_scrub":"63'3422","last_deep_scrub_stamp":"2024-09-13T17:18:27.174746+0000","last_clean_scrub_stamp":"2024-09-13T17:18:27.174746+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:39.964382+0000","scrub_duration":0.029236364000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18722,"num_write_kb":267617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"63'4803","reported_seq":4881,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.621276+0000","last_change":"2024-09-13T17:17:58.621508+0000","last_active":"2024-09-13T17:17:58.621276+0000","last_peered":"2024-09-13T17:17:58.621276+0000","last_clean":"2024-09-13T17:17:58.621276+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T17:17:58.621276+0000","last_undegraded":"2024-09-13T17:17:58.621276+0000","last_fullsized":"2024-09-13T17:17:58.621276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4803","last_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_deep_scrub":"63'4803","last_deep_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.621222+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:06.376472+0000","scrub_duration":0.023354388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26967,"num_write_kb":333429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"63'3840","reported_seq":3904,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.643285+0000","last_change":"2024-09-13T16:48:08.175188+0000","last_active":"2024-09-13T17:17:48.643285+0000","last_peered":"2024-09-13T17:17:48.643285+0000","last_clean":"2024-09-13T17:17:48.643285+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T17:17:48.643285+0000","last_undegraded":"2024-09-13T17:17:48.643285+0000","last_fullsized":"2024-09-13T17:17:48.643285+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21113,"num_write_kb":293770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"63'5433","reported_seq":5513,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.303582+0000","last_change":"2024-09-13T17:17:53.303824+0000","last_active":"2024-09-13T17:17:53.303582+0000","last_peered":"2024-09-13T17:17:53.303582+0000","last_clean":"2024-09-13T17:17:53.303582+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T17:17:53.303582+0000","last_undegraded":"2024-09-13T17:17:53.303582+0000","last_fullsized":"2024-09-13T17:17:53.303582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5433","last_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_deep_scrub":"63'5433","last_deep_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.303524+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.191620+0000","scrub_duration":0.041011686999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30664,"num_write_kb":389336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"63'4741","reported_seq":4808,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.974867+0000","last_change":"2024-09-13T16:48:08.169204+0000","last_active":"2024-09-13T17:17:44.974867+0000","last_peered":"2024-09-13T17:17:44.974867+0000","last_clean":"2024-09-13T17:17:44.974867+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T17:17:44.974867+0000","last_undegraded":"2024-09-13T17:17:44.974867+0000","last_fullsized":"2024-09-13T17:17:44.974867+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":15,"num_write":25588,"num_write_kb":340391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"63'4281","reported_seq":4359,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.227117+0000","last_change":"2024-09-13T17:17:57.227117+0000","last_active":"2024-09-13T17:17:57.227117+0000","last_peered":"2024-09-13T17:17:57.227117+0000","last_clean":"2024-09-13T17:17:57.227117+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T17:17:57.227117+0000","last_undegraded":"2024-09-13T17:17:57.227117+0000","last_fullsized":"2024-09-13T17:17:57.227117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4281","last_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_deep_scrub":"63'4281","last_deep_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226795+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.732503+0000","scrub_duration":0.034883109000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23373,"num_write_kb":302635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"63'3575","reported_seq":3654,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.020735+0000","last_change":"2024-09-13T17:18:45.020990+0000","last_active":"2024-09-13T17:18:45.020735+0000","last_peered":"2024-09-13T17:18:45.020735+0000","last_clean":"2024-09-13T17:18:45.020735+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T17:18:45.020735+0000","last_undegraded":"2024-09-13T17:18:45.020735+0000","last_fullsized":"2024-09-13T17:18:45.020735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3575","last_scrub_stamp":"2024-09-13T17:18:45.020682+0000","last_deep_scrub":"63'3575","last_deep_scrub_stamp":"2024-09-13T17:18:45.020682+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.020682+0000","objects_scrubbed":0,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:55.157528+0000","scrub_duration":0.013036646000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":7,"num_write":20117,"num_write_kb":289677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"63'4404","reported_seq":4482,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.337858+0000","last_change":"2024-09-13T17:17:50.337858+0000","last_active":"2024-09-13T17:17:50.337858+0000","last_peered":"2024-09-13T17:17:50.337858+0000","last_clean":"2024-09-13T17:17:50.337858+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T17:17:50.337858+0000","last_undegraded":"2024-09-13T17:17:50.337858+0000","last_fullsized":"2024-09-13T17:17:50.337858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4404","last_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_deep_scrub":"63'4404","last_deep_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.337585+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.971802+0000","scrub_duration":0.031410212999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24308,"num_write_kb":306423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"63'3318","reported_seq":3385,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:46.675177+0000","last_change":"2024-09-13T16:48:08.185306+0000","last_active":"2024-09-13T17:17:46.675177+0000","last_peered":"2024-09-13T17:17:46.675177+0000","last_clean":"2024-09-13T17:17:46.675177+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T17:17:46.675177+0000","last_undegraded":"2024-09-13T17:17:46.675177+0000","last_fullsized":"2024-09-13T17:17:46.675177+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3318,"log_dups_size":0,"ondisk_log_size":3318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18295,"num_write_kb":253537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"63'4013","reported_seq":4091,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.598539+0000","last_change":"2024-09-13T17:17:53.598539+0000","last_active":"2024-09-13T17:17:53.598539+0000","last_peered":"2024-09-13T17:17:53.598539+0000","last_clean":"2024-09-13T17:17:53.598539+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T17:17:53.598539+0000","last_undegraded":"2024-09-13T17:17:53.598539+0000","last_fullsized":"2024-09-13T17:17:53.598539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4013","last_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_deep_scrub":"63'4013","last_deep_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.598242+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.486338+0000","scrub_duration":0.028004864000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21393,"num_write_kb":262042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"63'3658","reported_seq":3724,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:44.982869+0000","last_change":"2024-09-13T16:48:08.172330+0000","last_active":"2024-09-13T17:17:44.982869+0000","last_peered":"2024-09-13T17:17:44.982869+0000","last_clean":"2024-09-13T17:17:44.982869+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T17:17:44.982869+0000","last_undegraded":"2024-09-13T17:17:44.982869+0000","last_fullsized":"2024-09-13T17:17:44.982869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19946,"num_write_kb":268446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"63'3910","reported_seq":3988,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:19.247862+0000","last_change":"2024-09-13T17:18:19.248105+0000","last_active":"2024-09-13T17:18:19.247862+0000","last_peered":"2024-09-13T17:18:19.247862+0000","last_clean":"2024-09-13T17:18:19.247862+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T17:18:19.247862+0000","last_undegraded":"2024-09-13T17:18:19.247862+0000","last_fullsized":"2024-09-13T17:18:19.247862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3910","last_scrub_stamp":"2024-09-13T17:18:19.247809+0000","last_deep_scrub":"63'3910","last_deep_scrub_stamp":"2024-09-13T17:18:19.247809+0000","last_clean_scrub_stamp":"2024-09-13T17:18:19.247809+0000","objects_scrubbed":0,"log_size":3910,"log_dups_size":0,"ondisk_log_size":3910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:18.673196+0000","scrub_duration":0.029454408000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":278651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"63'3074","reported_seq":3137,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:47.632259+0000","last_change":"2024-09-13T16:48:08.173133+0000","last_active":"2024-09-13T17:17:47.632259+0000","last_peered":"2024-09-13T17:17:47.632259+0000","last_clean":"2024-09-13T17:17:47.632259+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T17:17:47.632259+0000","last_undegraded":"2024-09-13T17:17:47.632259+0000","last_fullsized":"2024-09-13T17:17:47.632259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":246226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"63'3515","reported_seq":3597,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.560367+0000","last_change":"2024-09-13T17:17:52.560367+0000","last_active":"2024-09-13T17:17:52.560367+0000","last_peered":"2024-09-13T17:17:52.560367+0000","last_clean":"2024-09-13T17:17:52.560367+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T17:17:52.560367+0000","last_undegraded":"2024-09-13T17:17:52.560367+0000","last_fullsized":"2024-09-13T17:17:52.560367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3515","last_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_deep_scrub":"63'3515","last_deep_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.560093+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.500715+0000","scrub_duration":0.026529094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19512,"num_write_kb":258445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"63'3439","reported_seq":3514,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.727356+0000","last_change":"2024-09-13T17:18:45.727616+0000","last_active":"2024-09-13T17:18:45.727356+0000","last_peered":"2024-09-13T17:18:45.727356+0000","last_clean":"2024-09-13T17:18:45.727356+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T17:18:45.727356+0000","last_undegraded":"2024-09-13T17:18:45.727356+0000","last_fullsized":"2024-09-13T17:18:45.727356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3439","last_scrub_stamp":"2024-09-13T17:18:45.727303+0000","last_deep_scrub":"63'3439","last_deep_scrub_stamp":"2024-09-13T17:18:45.727303+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.727303+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:02.700997+0000","scrub_duration":0.020628344999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":17681,"num_write_kb":263603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"63'3892","reported_seq":3973,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.152704+0000","last_change":"2024-09-13T17:18:45.152938+0000","last_active":"2024-09-13T17:18:45.152704+0000","last_peered":"2024-09-13T17:18:45.152704+0000","last_clean":"2024-09-13T17:18:45.152704+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T17:18:45.152704+0000","last_undegraded":"2024-09-13T17:18:45.152704+0000","last_fullsized":"2024-09-13T17:18:45.152704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3892","last_scrub_stamp":"2024-09-13T17:18:45.152646+0000","last_deep_scrub":"63'3892","last_deep_scrub_stamp":"2024-09-13T17:18:45.152646+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.152646+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:44.424528+0000","scrub_duration":0.018925824000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21304,"num_write_kb":286190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"63'3622","reported_seq":3685,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.591670+0000","last_change":"2024-09-13T16:48:08.186882+0000","last_active":"2024-09-13T17:17:49.591670+0000","last_peered":"2024-09-13T17:17:49.591670+0000","last_clean":"2024-09-13T17:17:49.591670+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T17:17:49.591670+0000","last_undegraded":"2024-09-13T17:17:49.591670+0000","last_fullsized":"2024-09-13T17:17:49.591670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T16:48:07.726913+0000","last_clean_scrub_stamp":"2024-09-13T16:48:07.726913+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19729,"num_write_kb":275691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"63'5036","reported_seq":5116,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:41.341661+0000","last_change":"2024-09-13T17:18:41.341900+0000","last_active":"2024-09-13T17:18:41.341661+0000","last_peered":"2024-09-13T17:18:41.341661+0000","last_clean":"2024-09-13T17:18:41.341661+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T17:18:41.341661+0000","last_undegraded":"2024-09-13T17:18:41.341661+0000","last_fullsized":"2024-09-13T17:18:41.341661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5036","last_scrub_stamp":"2024-09-13T17:18:41.341603+0000","last_deep_scrub":"63'5036","last_deep_scrub_stamp":"2024-09-13T17:18:41.341603+0000","last_clean_scrub_stamp":"2024-09-13T17:18:41.341603+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:54.131238+0000","scrub_duration":0.03025245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27469,"num_write_kb":368239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"63'3689","reported_seq":3763,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:34.266698+0000","last_change":"2024-09-13T17:18:34.266933+0000","last_active":"2024-09-13T17:18:34.266698+0000","last_peered":"2024-09-13T17:18:34.266698+0000","last_clean":"2024-09-13T17:18:34.266698+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T17:18:34.266698+0000","last_undegraded":"2024-09-13T17:18:34.266698+0000","last_fullsized":"2024-09-13T17:18:34.266698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3689","last_scrub_stamp":"2024-09-13T17:18:34.266642+0000","last_deep_scrub":"63'3689","last_deep_scrub_stamp":"2024-09-13T17:18:34.266642+0000","last_clean_scrub_stamp":"2024-09-13T17:18:34.266642+0000","objects_scrubbed":0,"log_size":3689,"log_dups_size":0,"ondisk_log_size":3689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:44.403488+0000","scrub_duration":0.024276446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20818,"num_write_kb":284187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"63'4047","reported_seq":4128,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.364122+0000","last_change":"2024-09-13T17:17:51.364373+0000","last_active":"2024-09-13T17:17:51.364122+0000","last_peered":"2024-09-13T17:17:51.364122+0000","last_clean":"2024-09-13T17:17:51.364122+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T17:17:51.364122+0000","last_undegraded":"2024-09-13T17:17:51.364122+0000","last_fullsized":"2024-09-13T17:17:51.364122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4047","last_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_deep_scrub":"63'4047","last_deep_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.364064+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:46.022668+0000","scrub_duration":0.028243739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22743,"num_write_kb":291925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":197,"num_write":1329762,"num_write_kb":18444058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242925,"ondisk_log_size":242925,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19024977,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1583,"num_read_kb":206738,"num_write":4472,"num_write_kb":20527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38109184,"data_stored":38049954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4378,"ondisk_log_size":4378,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":1479870,"num_write_kb":19043750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270762,"ondisk_log_size":270762,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":19675678,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1366,"num_read_kb":209632,"num_write":4521,"num_write_kb":21162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39411712,"data_stored":39351356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4388,"ondisk_log_size":4388,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575229,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":942784,"kb_used_data":13688,"kb_used_omap":1,"kb_used_meta":929086,"kb_avail":93429056,"statfs":{"total":96636764160,"available":95671353344,"internally_reserved":0,"allocated":14016512,"data_stored":13616965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":951384523},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":2.0699999999999998,"15min":1.2929999999999999},"min":{"1min":0.61499999999999999,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":1.3169999999999999,"5min":16.571999999999999,"15min":16.571999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":2.1110000000000002,"15min":1.385},"min":{"1min":0.60699999999999998,"5min":0.26600000000000001,"15min":0.20899999999999999},"max":{"1min":1.177,"5min":16.523,"15min":16.523},"last":1.1339999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":2.6899999999999999,"15min":1.5640000000000001},"min":{"1min":0.56799999999999995,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.294,"5min":59.911999999999999,"15min":59.911999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.004,"5min":2.7850000000000001,"15min":1.671},"min":{"1min":0.69699999999999995,"5min":0.28399999999999997,"15min":0.23000000000000001},"max":{"1min":1.2110000000000001,"5min":59.895000000000003,"15min":59.895000000000003},"last":1.087}]},{"osd":2,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.718,"15min":1.153},"min":{"1min":0.64100000000000001,"5min":0.28999999999999998,"15min":0.25700000000000001},"max":{"1min":1.143,"5min":16.023,"15min":16.023},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.851,"15min":1.2669999999999999},"min":{"1min":0.63300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.397,"5min":18.734000000000002,"15min":18.734000000000002},"last":0.85999999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":2.0779999999999998,"15min":1.3120000000000001},"min":{"1min":0.69899999999999995,"5min":0.30199999999999999,"15min":0.28299999999999997},"max":{"1min":1.1359999999999999,"5min":23.225000000000001,"15min":23.225000000000001},"last":1},{"interface":"front","average":{"1min":1.0549999999999999,"5min":2.319,"15min":1.4650000000000001},"min":{"1min":0.81399999999999995,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.47,"5min":19.709,"15min":19.709},"last":0.93000000000000005}]},{"osd":4,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":2.133,"15min":1.3959999999999999},"min":{"1min":0.58799999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.2769999999999999,"5min":18.004999999999999,"15min":18.004999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":2.3690000000000002,"15min":1.48},"min":{"1min":0.66300000000000003,"5min":0.27300000000000002,"15min":0.245},"max":{"1min":1.1439999999999999,"5min":22.315999999999999,"15min":22.315999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":2.5169999999999999,"15min":1.5329999999999999},"min":{"1min":0.72499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.5429999999999999,"5min":37.302999999999997,"15min":37.302999999999997},"last":0.745},{"interface":"front","average":{"1min":1.02,"5min":1.8759999999999999,"15min":1.369},"min":{"1min":0.66300000000000003,"5min":0.33000000000000002,"15min":0.316},"max":{"1min":1.4330000000000001,"5min":16.867999999999999,"15min":16.867999999999999},"last":1.1779999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.909,"15min":1.383},"min":{"1min":0.73499999999999999,"5min":0.28000000000000003,"15min":0.23000000000000001},"max":{"1min":1.355,"5min":18.335000000000001,"15min":18.335000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.089,"5min":2.085,"15min":1.397},"min":{"1min":0.76000000000000001,"5min":0.33200000000000002,"15min":0.28100000000000003},"max":{"1min":1.268,"5min":19.244,"15min":19.244},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575226,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673216,"kb_used_data":11768,"kb_used_omap":1,"kb_used_meta":661438,"kb_avail":93698624,"statfs":{"total":96636764160,"available":95947390976,"internally_reserved":0,"allocated":12050432,"data_stored":11645255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":677312968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0329999999999999,"15min":0.89300000000000002},"min":{"1min":0.66500000000000004,"5min":0.222,"15min":0.222},"max":{"1min":1.1910000000000001,"5min":6.335,"15min":6.335},"last":1.071},{"interface":"front","average":{"1min":0.873,"5min":1.044,"15min":0.91700000000000004},"min":{"1min":0.61299999999999999,"5min":0.24199999999999999,"15min":0.20799999999999999},"max":{"1min":1.1619999999999999,"5min":4.907,"15min":5.9459999999999997},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.149,"15min":0.95499999999999996},"min":{"1min":0.59599999999999997,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.24,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.679,"15min":1.0960000000000001},"min":{"1min":0.66500000000000004,"5min":0.248,"15min":0.248},"max":{"1min":1.069,"5min":10.111000000000001,"15min":10.111000000000001},"last":0.72499999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.2270000000000001,"15min":0.98299999999999998},"min":{"1min":0.68600000000000005,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.248,"5min":15.638,"15min":15.638},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3680000000000001,"15min":1.095},"min":{"1min":0.77800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":1.1719999999999999,"5min":19.442,"15min":19.442},"last":0.88800000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.3759999999999999,"15min":1.1319999999999999},"min":{"1min":0.70499999999999996,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":1.216,"5min":15.247999999999999,"15min":15.247999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.03,"5min":1.4730000000000001,"15min":1.167},"min":{"1min":0.79900000000000004,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1759999999999999,"5min":15.179,"15min":15.179},"last":0.85199999999999998}]},{"osd":4,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.4410000000000001,"15min":1.109},"min":{"1min":0.66200000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.1220000000000001,"5min":15.326000000000001,"15min":15.326000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.5229999999999999,"15min":1.1970000000000001},"min":{"1min":0.72399999999999998,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":1.278,"5min":15.026,"15min":15.026},"last":0.80300000000000005}]},{"osd":5,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.494,"15min":1.246},"min":{"1min":0.88200000000000001,"5min":0.32500000000000001,"15min":0.28899999999999998},"max":{"1min":1.256,"5min":9.6080000000000005,"15min":11.566000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.325,"15min":1.1879999999999999},"min":{"1min":0.65500000000000003,"5min":0.308,"15min":0.308},"max":{"1min":1.2889999999999999,"5min":10.223000000000001,"15min":10.223000000000001},"last":1}]},{"osd":7,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.462,"15min":1.238},"min":{"1min":0.84499999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.266,"5min":9.8960000000000008,"15min":13.112},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.5840000000000001,"15min":1.274},"min":{"1min":0.80800000000000005,"5min":0.33200000000000002,"15min":0.245},"max":{"1min":1.331,"5min":14.4,"15min":14.4},"last":1.1830000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575227,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":957788,"kb_used_data":5012,"kb_used_omap":1,"kb_used_meta":952766,"kb_avail":93414052,"statfs":{"total":96636764160,"available":95655989248,"internally_reserved":0,"allocated":5132288,"data_stored":4730805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":975632838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.99,"15min":1.2030000000000001},"min":{"1min":0.65900000000000003,"5min":0.215,"15min":0.215},"max":{"1min":1.129,"5min":27.952999999999999,"15min":27.952999999999999},"last":0.746},{"interface":"front","average":{"1min":1.052,"5min":1.617,"15min":1.1339999999999999},"min":{"1min":0.84199999999999997,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.319,"5min":23.619,"15min":23.619},"last":1.113}]},{"osd":1,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.7749999999999999,"15min":1.1399999999999999},"min":{"1min":0.63800000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.1479999999999999,"5min":13.515000000000001,"15min":13.515000000000001},"last":1.075},{"interface":"front","average":{"1min":0.84899999999999998,"5min":1.875,"15min":1.1890000000000001},"min":{"1min":0.628,"5min":0.25700000000000001,"15min":0.248},"max":{"1min":1.1020000000000001,"5min":35.186999999999998,"15min":35.186999999999998},"last":0.877}]},{"osd":2,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.694,"15min":1.3440000000000001},"min":{"1min":0.70099999999999996,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2310000000000001,"5min":11.609,"15min":51.18},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.395,"15min":1.296},"min":{"1min":0.61499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1759999999999999,"5min":8.282,"15min":48.756},"last":0.93500000000000005}]},{"osd":3,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.8740000000000001,"15min":1.2809999999999999},"min":{"1min":0.69399999999999995,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1419999999999999,"5min":19.800000000000001,"15min":19.800000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.024,"5min":1.6679999999999999,"15min":1.258},"min":{"1min":0.76300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2270000000000001,"5min":9.4789999999999992,"15min":9.4789999999999992},"last":1.153}]},{"osd":4,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":2.4159999999999999,"15min":1.3839999999999999},"min":{"1min":0.56399999999999995,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1879999999999999,"5min":22.361999999999998,"15min":22.361999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.999,"5min":2.6480000000000001,"15min":1.6439999999999999},"min":{"1min":0.753,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2669999999999999,"5min":28.062999999999999,"15min":31.843},"last":1.0389999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.403,"15min":1.1299999999999999},"min":{"1min":0.65000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.383,"5min":8.1910000000000007,"15min":8.1910000000000007},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.7310000000000001,"15min":1.2689999999999999},"min":{"1min":0.68799999999999994,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.3,"5min":13.031000000000001,"15min":13.031000000000001},"last":1.2030000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.98,"15min":1.4219999999999999},"min":{"1min":0.73299999999999998,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":1.345,"5min":35.124000000000002,"15min":35.124000000000002},"last":1.177},{"interface":"front","average":{"1min":1.181,"5min":2.2730000000000001,"15min":1.552},"min":{"1min":0.91800000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.373,"5min":32.328000000000003,"15min":32.328000000000003},"last":1.2250000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575227,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1011352,"kb_used_data":13968,"kb_used_omap":1,"kb_used_meta":997374,"kb_avail":93360488,"statfs":{"total":96636764160,"available":95601139712,"internally_reserved":0,"allocated":14303232,"data_stored":13890488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1021311430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.6970000000000001,"15min":1.1659999999999999},"min":{"1min":0.60999999999999999,"5min":0.23899999999999999,"15min":0.20399999999999999},"max":{"1min":1.2290000000000001,"5min":18.841000000000001,"15min":18.841000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.694,"15min":1.2729999999999999},"min":{"1min":0.80700000000000005,"5min":0.25700000000000001,"15min":0.251},"max":{"1min":1.2290000000000001,"5min":20.312000000000001,"15min":20.312000000000001},"last":0.98799999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.623,"15min":1.1180000000000001},"min":{"1min":0.69299999999999995,"5min":0.27100000000000002,"15min":0.26100000000000001},"max":{"1min":1.262,"5min":27.242000000000001,"15min":27.242000000000001},"last":1.2729999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":2.1899999999999999,"15min":1.3220000000000001},"min":{"1min":0.66400000000000003,"5min":0.32200000000000001,"15min":0.26000000000000001},"max":{"1min":1.153,"5min":26.545999999999999,"15min":26.545999999999999},"last":1.02}]},{"osd":2,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.4510000000000001,"15min":1.105},"min":{"1min":0.73999999999999999,"5min":0.29199999999999998,"15min":0.29099999999999998},"max":{"1min":1.1559999999999999,"5min":10.205,"15min":11.803000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.534,"15min":1.228},"min":{"1min":0.748,"5min":0.29399999999999998,"15min":0.26200000000000001},"max":{"1min":1.24,"5min":13.972,"15min":13.972},"last":1.2490000000000001}]},{"osd":3,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":2.052,"15min":1.419},"min":{"1min":0.69199999999999995,"5min":0.32000000000000001,"15min":0.30499999999999999},"max":{"1min":1.1579999999999999,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.012,"5min":1.728,"15min":1.2909999999999999},"min":{"1min":0.73999999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.268,"5min":20.468,"15min":20.468},"last":1.141}]},{"osd":5,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.6919999999999999,"15min":1.1930000000000001},"min":{"1min":0.67800000000000005,"5min":0.27200000000000002,"15min":0.23499999999999999},"max":{"1min":1.214,"5min":14.031000000000001,"15min":14.031000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.819,"15min":1.3640000000000001},"min":{"1min":0.69899999999999995,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.2330000000000001,"5min":15.044,"15min":15.044},"last":0.93200000000000005}]},{"osd":6,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.464,"15min":1.1519999999999999},"min":{"1min":0.77500000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.3540000000000001,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.8140000000000001,"15min":1.3160000000000001},"min":{"1min":0.70699999999999996,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.262,"5min":14.131,"15min":14.131},"last":1.103}]},{"osd":7,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.661,"15min":1.4790000000000001},"min":{"1min":0.78500000000000003,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":1.296,"5min":20.154,"15min":24.518999999999998},"last":1.042},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.577,"15min":1.3600000000000001},"min":{"1min":0.73799999999999999,"5min":0.32600000000000001,"15min":0.312},"max":{"1min":1.427,"5min":15.209,"15min":15.209},"last":1.2150000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575227,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":958484,"kb_used_data":19404,"kb_used_omap":1,"kb_used_meta":939070,"kb_avail":93413356,"statfs":{"total":96636764160,"available":95655276544,"internally_reserved":0,"allocated":19869696,"data_stored":19459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":961608136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.5509999999999999,"15min":0.96299999999999997},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.17599999999999999},"max":{"1min":0.91700000000000004,"5min":19.327999999999999,"15min":19.327999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":1.7809999999999999,"15min":1.044},"min":{"1min":0.45600000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1,"5min":20.699999999999999,"15min":20.699999999999999},"last":0.94799999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":1.2949999999999999,"15min":0.90000000000000002},"min":{"1min":0.32200000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.048,"5min":16.100000000000001,"15min":16.100000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":1.2509999999999999,"15min":0.95499999999999996},"min":{"1min":0.32700000000000001,"5min":0.245,"15min":0.22900000000000001},"max":{"1min":1.121,"5min":10.417,"15min":10.417},"last":0.91400000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":1.2909999999999999,"15min":0.96799999999999997},"min":{"1min":0.40799999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.387,"5min":8.5950000000000006,"15min":8.5950000000000006},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.1339999999999999,"15min":0.96399999999999997},"min":{"1min":0.39000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.274,"5min":10.737,"15min":15.433999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":2.028,"15min":1.3320000000000001},"min":{"1min":0.56699999999999995,"5min":0.308,"15min":0.29599999999999999},"max":{"1min":1.3140000000000001,"5min":15.919,"15min":15.919},"last":1.228},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.6559999999999999,"15min":1.21},"min":{"1min":0.60899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5029999999999999,"5min":9.3819999999999997,"15min":9.3819999999999997},"last":1.1870000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.802,"15min":1.3009999999999999},"min":{"1min":0.67800000000000005,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.5900000000000001,"5min":31.187999999999999,"15min":31.187999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.097,"5min":2.077,"15min":1.383},"min":{"1min":0.63700000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.5580000000000001,"5min":22.951000000000001,"15min":22.951000000000001},"last":1.0429999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.849,"15min":1.3100000000000001},"min":{"1min":0.71599999999999997,"5min":0.34899999999999998,"15min":0.31900000000000001},"max":{"1min":1.619,"5min":12.567,"15min":12.567},"last":1.419},{"interface":"front","average":{"1min":1.141,"5min":1.95,"15min":1.4550000000000001},"min":{"1min":0.61799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5109999999999999,"5min":15.882,"15min":15.882},"last":1.28}]},{"osd":7,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":2.2589999999999999,"15min":1.5389999999999999},"min":{"1min":0.73699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6080000000000001,"5min":33.969000000000001,"15min":33.969000000000001},"last":1.252},{"interface":"front","average":{"1min":1.1619999999999999,"5min":2.0510000000000002,"15min":1.458},"min":{"1min":0.63700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.5489999999999999,"5min":25.652999999999999,"15min":25.652999999999999},"last":1.367}]}]},{"osd":2,"up_from":13,"seq":55834575227,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662284,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":661374,"kb_avail":93709556,"statfs":{"total":96636764160,"available":95958585344,"internally_reserved":0,"allocated":921600,"data_stored":527729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":677247430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.89700000000000002,"15min":0.72599999999999998},"min":{"1min":0.53300000000000003,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.91500000000000004,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.472,"15min":1.042},"min":{"1min":0.57699999999999996,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.1990000000000001,"5min":9.2729999999999997,"15min":9.2729999999999997},"last":1.2370000000000001}]},{"osd":1,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":1.6899999999999999,"15min":1.069},"min":{"1min":0.58499999999999996,"5min":0.20200000000000001,"15min":0.20000000000000001},"max":{"1min":1.042,"5min":33.579999999999998,"15min":33.579999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.6459999999999999,"15min":1.127},"min":{"1min":0.63700000000000001,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":1.032,"5min":23.965,"15min":23.965},"last":0.86599999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":1.8520000000000001,"15min":1.3120000000000001},"min":{"1min":0.625,"5min":0.247,"15min":0.245},"max":{"1min":1.1499999999999999,"5min":11.837999999999999,"15min":22.760000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.605,"15min":1.1379999999999999},"min":{"1min":0.70499999999999996,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.252,"5min":10.372,"15min":10.372},"last":0.91200000000000003}]},{"osd":4,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.7569999999999999,"15min":1.298},"min":{"1min":0.79100000000000004,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.3069999999999999,"5min":20.673999999999999,"15min":20.673999999999999},"last":1.044},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.6579999999999999,"15min":1.282},"min":{"1min":0.86699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.506,"5min":16.5,"15min":16.5},"last":1.0069999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":2.3399999999999999,"15min":1.54},"min":{"1min":0.82099999999999995,"5min":0.31,"15min":0.27900000000000003},"max":{"1min":1.4299999999999999,"5min":21.722000000000001,"15min":21.722000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":2.097,"15min":1.4339999999999999},"min":{"1min":0.873,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4470000000000001,"5min":29.620999999999999,"15min":29.620999999999999},"last":1.1990000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.6519999999999999,"15min":1.296},"min":{"1min":0.878,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":1.4870000000000001,"5min":21.882999999999999,"15min":21.882999999999999},"last":1.123},{"interface":"front","average":{"1min":1.1759999999999999,"5min":2.0299999999999998,"15min":1.478},"min":{"1min":0.83899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.478,"5min":25.274999999999999,"15min":25.274999999999999},"last":1.087}]},{"osd":7,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":3.1190000000000002,"15min":1.871},"min":{"1min":0.98899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.587,"5min":43.649000000000001,"15min":43.649000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.232,"5min":3.0990000000000002,"15min":1.8580000000000001},"min":{"1min":1.054,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":1.4730000000000001,"5min":27.417999999999999,"15min":27.417999999999999},"last":1.1679999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575227,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798984,"kb_used_data":9024,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572856,"statfs":{"total":96636764160,"available":95818604544,"internally_reserved":0,"allocated":9240576,"data_stored":8832475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":808909254},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":1.7709999999999999,"15min":1.085},"min":{"1min":0.52200000000000002,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":1.155,"5min":15.81,"15min":15.81},"last":0.87},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.6419999999999999,"15min":1.069},"min":{"1min":0.66600000000000004,"5min":0.188,"15min":0.17599999999999999},"max":{"1min":1.073,"5min":15.823,"15min":15.823},"last":0.81499999999999995}]},{"osd":2,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.3069999999999999,"15min":0.92300000000000004},"min":{"1min":0.51400000000000001,"5min":0.215,"15min":0.193},"max":{"1min":1.093,"5min":11.672000000000001,"15min":11.672000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.2729999999999999,"15min":0.95999999999999996},"min":{"1min":0.55200000000000005,"5min":0.23699999999999999,"15min":0.22600000000000001},"max":{"1min":1.272,"5min":11.722,"15min":11.722},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.871,"15min":1.226},"min":{"1min":0.63100000000000001,"5min":0.24299999999999999,"15min":0.22},"max":{"1min":1.0840000000000001,"5min":16.507999999999999,"15min":16.507999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":1.3680000000000001,"15min":1.016},"min":{"1min":0.56799999999999995,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":1.073,"5min":12.047000000000001,"15min":12.047000000000001},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.925,"15min":1.2769999999999999},"min":{"1min":0.76600000000000001,"5min":0.27800000000000002,"15min":0.24399999999999999},"max":{"1min":1.657,"5min":22.234000000000002,"15min":22.234000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.0580000000000001,"5min":2.0819999999999999,"15min":1.407},"min":{"1min":0.89200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4450000000000001,"5min":20.448,"15min":20.448},"last":1.1000000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.6180000000000001,"15min":1.2010000000000001},"min":{"1min":0.76500000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":1.522,"5min":15.788,"15min":15.788},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.01,"5min":2.0299999999999998,"15min":1.365},"min":{"1min":0.83999999999999997,"5min":0.312,"15min":0.312},"max":{"1min":1.403,"5min":24.763999999999999,"15min":24.763999999999999},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.3959999999999999,"15min":1.1459999999999999},"min":{"1min":0.81899999999999995,"5min":0.255,"15min":0.255},"max":{"1min":1.629,"5min":12.019,"15min":12.019},"last":1.054},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.8129999999999999,"15min":1.2769999999999999},"min":{"1min":0.84499999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4730000000000001,"5min":11.628,"15min":11.628},"last":0.83899999999999997}]},{"osd":7,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.7769999999999999,"15min":1.409},"min":{"1min":0.81899999999999995,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.466,"5min":10.667,"15min":21.161000000000001},"last":1.179},{"interface":"front","average":{"1min":1.163,"5min":2.2989999999999999,"15min":1.5589999999999999},"min":{"1min":0.97499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.6870000000000001,"5min":29.323,"15min":29.323},"last":1.1539999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575227,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799556,"kb_used_data":9596,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572284,"statfs":{"total":96636764160,"available":95818018816,"internally_reserved":0,"allocated":9826304,"data_stored":9422627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":808909256},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":1.377,"15min":0.91100000000000003},"min":{"1min":0.317,"5min":0.20000000000000001,"15min":0.189},"max":{"1min":0.89800000000000002,"5min":59.031999999999996,"15min":59.031999999999996},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":2.3100000000000001,"15min":1.286},"min":{"1min":0.437,"5min":0.19700000000000001,"15min":0.188},"max":{"1min":0.97199999999999998,"5min":57.186999999999998,"15min":57.186999999999998},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.123,"15min":0.877},"min":{"1min":0.371,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":1.1910000000000001,"5min":12.971,"15min":12.971},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":1.3500000000000001,"15min":0.93700000000000006},"min":{"1min":0.35999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":1.353,"5min":14.526,"15min":14.526},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":2.5830000000000002,"15min":1.4359999999999999},"min":{"1min":0.38500000000000001,"5min":0.221,"15min":0.221},"max":{"1min":1.085,"5min":65.215000000000003,"15min":65.215000000000003},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.92900000000000005,"5min":1.827,"15min":1.153},"min":{"1min":0.41199999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3089999999999999,"5min":62.722999999999999,"15min":62.722999999999999},"last":1.1970000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.889,"15min":1.2649999999999999},"min":{"1min":0.64100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.177,"5min":14.694000000000001,"15min":14.694000000000001},"last":1.052},{"interface":"front","average":{"1min":1.026,"5min":1.3149999999999999,"15min":1.073},"min":{"1min":0.77200000000000002,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":1.2949999999999999,"5min":10.036,"15min":10.036},"last":1.129}]},{"osd":5,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.8009999999999999,"15min":1.2709999999999999},"min":{"1min":0.71599999999999997,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":1.2989999999999999,"5min":21.940999999999999,"15min":21.940999999999999},"last":0.753},{"interface":"front","average":{"1min":1.107,"5min":1.7969999999999999,"15min":1.3620000000000001},"min":{"1min":0.624,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.391,"5min":9.8200000000000003,"15min":9.8200000000000003},"last":1.0069999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.427,"15min":1.1599999999999999},"min":{"1min":0.60799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.4370000000000001,"5min":10.819000000000001,"15min":10.819000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.4990000000000001,"15min":1.2190000000000001},"min":{"1min":0.73499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4419999999999999,"5min":10.741,"15min":10.741},"last":1.232}]},{"osd":7,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.772,"15min":1.367},"min":{"1min":0.67600000000000005,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":1.4710000000000001,"5min":32.975999999999999,"15min":32.975999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":2.5390000000000001,"15min":1.6819999999999999},"min":{"1min":0.754,"5min":0.41299999999999998,"15min":0.40300000000000002},"max":{"1min":1.399,"5min":32.069000000000003,"15min":32.069000000000003},"last":1.0980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":84646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8482816,"data_stored":8473926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7020544,"data_stored":7006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7016448,"data_stored":7007322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10542867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4296704,"data_stored":4287702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:19:11.584 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.585 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T16:47:58.630418+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=47, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.585 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.585 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.585 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.586 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.586 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.586 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.587 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.587 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.587 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.587 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.587 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.588 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.588 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.588 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.588 INFO:tasks.ceph:pgid 5.26 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.588 INFO:tasks.ceph:pgid 5.27 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.588 INFO:tasks.ceph:pgid 5.25 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.589 INFO:tasks.ceph:pgid 5.23 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.589 INFO:tasks.ceph:pgid 5.20 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.589 INFO:tasks.ceph:pgid 5.2e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.589 INFO:tasks.ceph:pgid 5.2f last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.589 INFO:tasks.ceph:pgid 5.2d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.589 INFO:tasks.ceph:pgid 5.2b last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.589 INFO:tasks.ceph:pgid 5.28 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.590 INFO:tasks.ceph:pgid 5.29 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.590 INFO:tasks.ceph:pgid 5.34 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.590 INFO:tasks.ceph:pgid 5.32 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.590 INFO:tasks.ceph:pgid 5.31 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.590 INFO:tasks.ceph:pgid 5.3e last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.591 INFO:tasks.ceph:pgid 5.3c last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.591 INFO:tasks.ceph:pgid 5.3d last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.591 INFO:tasks.ceph:pgid 5.3a last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.591 INFO:tasks.ceph:pgid 5.38 last_scrub_stamp 2024-09-13T16:48:07.726913+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=16, tm_min=48, tm_sec=7, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=17, tm_sec=44, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:19:11.591 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T17:19:31.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:19:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4ce13b0640 1 Processor -- start 2024-09-13T17:19:31.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4ce13b0640 1 -- start start 2024-09-13T17:19:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4ce13b0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 0x7f4cdc071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4ce13b0640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4cdc0c7180 0x7f4cdc06f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:31.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4ce13b0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4cdc0c26c0 0x7f4cdc06f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4ce13b0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cdc06e410 con 0x7f4cdc0c26c0 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4ce13b0640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cdc0a7000 con 0x7f4cdc0c7180 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4ce13b0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc06d750 con 0x7f4cdc14f340 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cdb7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4cdc0c26c0 0x7f4cdc06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cdaffd640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 0x7f4cdc071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cdb7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4cdc0c26c0 0x7f4cdc06f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39380/0 (socket says 172.21.15.39:39380) 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cdb7fe640 1 -- 172.21.15.39:0/917728894 learned_addr learned my addr 172.21.15.39:0/917728894 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cda7fc640 1 --2- 172.21.15.39:0/917728894 >> v2:172.21.15.84:3300/0 conn(0x7f4cdc0c7180 0x7f4cdc06f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cdb7fe640 1 -- 172.21.15.39:0/917728894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 msgr2=0x7f4cdc071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cdb7fe640 1 --2- 172.21.15.39:0/917728894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 0x7f4cdc071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cdb7fe640 1 -- 172.21.15.39:0/917728894 >> v2:172.21.15.84:3300/0 conn(0x7f4cdc0c7180 msgr2=0x7f4cdc06f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cdb7fe640 1 --2- 172.21.15.39:0/917728894 >> v2:172.21.15.84:3300/0 conn(0x7f4cdc0c7180 0x7f4cdc06f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.723+0000 7f4cdb7fe640 1 -- 172.21.15.39:0/917728894 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc155ce0 con 0x7f4cdc0c26c0 2024-09-13T17:19:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4cdb7fe640 1 --2- 172.21.15.39:0/917728894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c26c0 0x7f4cdc06f590 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc071f90 tx=0x7f4cd401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe779e10c47a3b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:31.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4cd9ffb640 1 -- 172.21.15.39:0/917728894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40ab600 con 0x7f4cdc0c26c0 2024-09-13T17:19:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4cd9ffb640 1 -- 172.21.15.39:0/917728894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd40a7050 con 0x7f4cdc0c26c0 2024-09-13T17:19:31.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4cd9ffb640 1 -- 172.21.15.39:0/917728894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40ab980 con 0x7f4cdc0c26c0 2024-09-13T17:19:31.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4ce13b0640 1 -- 172.21.15.39:0/917728894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c26c0 msgr2=0x7f4cdc06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:31.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/917728894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c26c0 0x7f4cdc06f590 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc071f90 tx=0x7f4cd401c490 comp rx=0 tx=0).stop 2024-09-13T17:19:31.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4ce13b0640 1 -- 172.21.15.39:0/917728894 shutdown_connections 2024-09-13T17:19:31.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/917728894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c26c0 0x7f4cdc06f590 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/917728894 >> v2:172.21.15.84:3300/0 conn(0x7f4cdc0c7180 0x7f4cdc06f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/917728894 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 0x7f4cdc071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4ce13b0640 1 -- 172.21.15.39:0/917728894 >> 172.21.15.39:0/917728894 conn(0x7f4cdc0a96b0 msgr2=0x7f4cdc0a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:31.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4ce13b0640 1 -- 172.21.15.39:0/917728894 shutdown_connections 2024-09-13T17:19:31.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.727+0000 7f4ce13b0640 1 -- 172.21.15.39:0/917728894 wait complete. 2024-09-13T17:19:31.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 Processor -- start 2024-09-13T17:19:31.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 -- start start 2024-09-13T17:19:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0c26c0 0x7f4cdc15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c7180 0x7f4cdc1ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 0x7f4cdc1ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc14df60 con 0x7f4cdc0c7180 2024-09-13T17:19:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc151270 con 0x7f4cdc0c26c0 2024-09-13T17:19:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1f09c0 con 0x7f4cdc14f340 2024-09-13T17:19:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cda7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c7180 0x7f4cdc1ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:31.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cda7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c7180 0x7f4cdc1ecba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39388/0 (socket says 172.21.15.39:39388) 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cda7fc640 1 -- 172.21.15.39:0/177185183 learned_addr learned my addr 172.21.15.39:0/177185183 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cdaffd640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0c26c0 0x7f4cdc15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cdb7fe640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 0x7f4cdc1ed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cda7fc640 1 -- 172.21.15.39:0/177185183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 msgr2=0x7f4cdc1ed0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cda7fc640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 0x7f4cdc1ed0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cda7fc640 1 -- 172.21.15.39:0/177185183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0c26c0 msgr2=0x7f4cdc15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cda7fc640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0c26c0 0x7f4cdc15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cda7fc640 1 -- 172.21.15.39:0/177185183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd40094d0 con 0x7f4cdc0c7180 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cdb7fe640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 0x7f4cdc1ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:19:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cda7fc640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c7180 0x7f4cdc1ecba0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00ef00 tx=0x7f4ccc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cd37fe640 1 -- 172.21.15.39:0/177185183 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc010040 con 0x7f4cdc0c7180 2024-09-13T17:19:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cd37fe640 1 -- 172.21.15.39:0/177185183 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc009e40 con 0x7f4cdc0c7180 2024-09-13T17:19:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc1f0c50 con 0x7f4cdc0c7180 2024-09-13T17:19:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cd37fe640 1 -- 172.21.15.39:0/177185183 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc014620 con 0x7f4cdc0c7180 2024-09-13T17:19:31.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc15ae30 con 0x7f4cdc0c7180 2024-09-13T17:19:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc0c7650 con 0x7f4cdc0c7180 2024-09-13T17:19:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.731+0000 7f4cd37fe640 1 -- 172.21.15.39:0/177185183 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ccc014780 con 0x7f4cdc0c7180 2024-09-13T17:19:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.735+0000 7f4cd37fe640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cbc066690 0x7f4cbc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.735+0000 7f4cd37fe640 1 -- 172.21.15.39:0/177185183 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f4ccc085d30 con 0x7f4cdc0c7180 2024-09-13T17:19:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.735+0000 7f4cdaffd640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cbc066690 0x7f4cbc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.735+0000 7f4cdaffd640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cbc066690 0x7f4cbc068b50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009a50 tx=0x7f4cc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.739+0000 7f4cd37fe640 1 -- 172.21.15.39:0/177185183 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4ccc051f40 con 0x7f4cdc0c7180 2024-09-13T17:19:31.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.891+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 --> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4cdc0c6560 con 0x7f4cbc066690 2024-09-13T17:19:31.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.931+0000 7f4cd37fe640 1 -- 172.21.15.39:0/177185183 <== mgr.4101 v2:172.21.15.39:6832/1271482023 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+533129 (secure 0 0 0) 0x7f4cdc0c6560 con 0x7f4cbc066690 2024-09-13T17:19:31.942 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:19:31.947 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-13T17:19:31.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cbc066690 msgr2=0x7f4cbc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:31.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cbc066690 0x7f4cbc068b50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009a50 tx=0x7f4cc4008040 comp rx=0 tx=0).stop 2024-09-13T17:19:31.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c7180 msgr2=0x7f4cdc1ecba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:31.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c7180 0x7f4cdc1ecba0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00ef00 tx=0x7f4ccc00b480 comp rx=0 tx=0).stop 2024-09-13T17:19:31.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 shutdown_connections 2024-09-13T17:19:31.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f4cbc066690 0x7f4cbc068b50 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f4cdc14f340 0x7f4cdc1ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cdc0c7180 0x7f4cdc1ecba0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 --2- 172.21.15.39:0/177185183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4cdc0c26c0 0x7f4cdc15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:31.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 >> 172.21.15.39:0/177185183 conn(0x7f4cdc0a96b0 msgr2=0x7f4cdc0a8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:31.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 shutdown_connections 2024-09-13T17:19:31.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:31.939+0000 7f4ce13b0640 1 -- 172.21.15.39:0/177185183 wait complete. 2024-09-13T17:19:31.960 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":1008,"stamp":"2024-09-13T17:19:30.397778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39291023,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":3153,"num_read_kb":416667,"num_write":2818784,"num_write_kb":37531913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522572,"ondisk_log_size":522572,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6804448,"kb_used_data":83360,"kb_used_omap":12,"kb_used_meta":6721011,"kb_avail":748170272,"statfs":{"total":773094113280,"available":766126358528,"internally_reserved":0,"allocated":85360640,"data_stored":82126150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":6882315835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005769"},"pg_stats":[{"pgid":"5.18","version":"63'4385","reported_seq":4460,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:56.211163+0000","last_change":"2024-09-13T17:18:56.211391+0000","last_active":"2024-09-13T17:18:56.211163+0000","last_peered":"2024-09-13T17:18:56.211163+0000","last_clean":"2024-09-13T17:18:56.211163+0000","last_became_active":"2024-09-13T16:48:08.185233+0000","last_became_peered":"2024-09-13T16:48:08.185233+0000","last_unstale":"2024-09-13T17:18:56.211163+0000","last_undegraded":"2024-09-13T17:18:56.211163+0000","last_fullsized":"2024-09-13T17:18:56.211163+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4385","last_scrub_stamp":"2024-09-13T17:18:56.211105+0000","last_deep_scrub":"63'4385","last_deep_scrub_stamp":"2024-09-13T17:18:56.211105+0000","last_clean_scrub_stamp":"2024-09-13T17:18:56.211105+0000","objects_scrubbed":0,"log_size":4385,"log_dups_size":0,"ondisk_log_size":4385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:28.905686+0000","scrub_duration":0.034523294000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":23692,"num_write_kb":324330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"64'919","reported_seq":1045,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.775401+0000","last_change":"2024-09-13T17:18:10.775401+0000","last_active":"2024-09-13T17:18:10.775401+0000","last_peered":"2024-09-13T17:18:10.775401+0000","last_clean":"2024-09-13T17:18:10.775401+0000","last_became_active":"2024-09-13T16:47:58.667962+0000","last_became_peered":"2024-09-13T16:47:58.667962+0000","last_unstale":"2024-09-13T17:18:10.775401+0000","last_undegraded":"2024-09-13T17:18:10.775401+0000","last_fullsized":"2024-09-13T17:18:10.775401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'919","last_scrub_stamp":"2024-09-13T17:18:10.775003+0000","last_deep_scrub":"64'919","last_deep_scrub_stamp":"2024-09-13T17:18:10.775003+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.775003+0000","objects_scrubbed":3,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:00.271443+0000","scrub_duration":0.030271704,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":10389,"num_write":894,"num_write_kb":4597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"63'3737","reported_seq":3818,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.006249+0000","last_change":"2024-09-13T17:19:05.006483+0000","last_active":"2024-09-13T17:19:05.006249+0000","last_peered":"2024-09-13T17:19:05.006249+0000","last_clean":"2024-09-13T17:19:05.006249+0000","last_became_active":"2024-09-13T16:47:59.059172+0000","last_became_peered":"2024-09-13T16:47:59.059172+0000","last_unstale":"2024-09-13T17:19:05.006249+0000","last_undegraded":"2024-09-13T17:19:05.006249+0000","last_fullsized":"2024-09-13T17:19:05.006249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3737","last_scrub_stamp":"2024-09-13T17:19:05.006193+0000","last_deep_scrub":"63'3737","last_deep_scrub_stamp":"2024-09-13T17:19:05.006193+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.006193+0000","objects_scrubbed":0,"log_size":3737,"log_dups_size":0,"ondisk_log_size":3737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:00.849618+0000","scrub_duration":0.027405445,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20454,"num_write_kb":270405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.082725+0000","last_change":"2024-09-13T17:17:52.082969+0000","last_active":"2024-09-13T17:17:52.082725+0000","last_peered":"2024-09-13T17:17:52.082725+0000","last_clean":"2024-09-13T17:17:52.082725+0000","last_became_active":"2024-09-13T16:48:07.775785+0000","last_became_peered":"2024-09-13T16:48:07.775785+0000","last_unstale":"2024-09-13T17:17:52.082725+0000","last_undegraded":"2024-09-13T17:17:52.082725+0000","last_fullsized":"2024-09-13T17:17:52.082725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:17:52.082672+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.082672+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.023294+0000","scrub_duration":0.0034166320000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"63'4331","reported_seq":4407,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:09.288822+0000","last_change":"2024-09-13T17:19:09.289058+0000","last_active":"2024-09-13T17:19:09.288822+0000","last_peered":"2024-09-13T17:19:09.288822+0000","last_clean":"2024-09-13T17:19:09.288822+0000","last_became_active":"2024-09-13T16:48:08.178690+0000","last_became_peered":"2024-09-13T16:48:08.178690+0000","last_unstale":"2024-09-13T17:19:09.288822+0000","last_undegraded":"2024-09-13T17:19:09.288822+0000","last_fullsized":"2024-09-13T17:19:09.288822+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4331","last_scrub_stamp":"2024-09-13T17:19:09.288764+0000","last_deep_scrub":"63'4331","last_deep_scrub_stamp":"2024-09-13T17:19:09.288764+0000","last_clean_scrub_stamp":"2024-09-13T17:19:09.288764+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:21.004265+0000","scrub_duration":0.031338309000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":22875,"num_write_kb":325320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"64'24","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:49.931421+0000","last_change":"2024-09-13T17:18:49.931654+0000","last_active":"2024-09-13T17:18:49.931421+0000","last_peered":"2024-09-13T17:18:49.931421+0000","last_clean":"2024-09-13T17:18:49.931421+0000","last_became_active":"2024-09-13T16:47:58.660925+0000","last_became_peered":"2024-09-13T16:47:58.660925+0000","last_unstale":"2024-09-13T17:18:49.931421+0000","last_undegraded":"2024-09-13T17:18:49.931421+0000","last_fullsized":"2024-09-13T17:18:49.931421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-09-13T17:18:49.931368+0000","last_deep_scrub":"64'24","last_deep_scrub_stamp":"2024-09-13T17:18:49.931368+0000","last_clean_scrub_stamp":"2024-09-13T17:18:49.931368+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:06.061913+0000","scrub_duration":0.0033432700000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"63'5559","reported_seq":5640,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:55.943470+0000","last_change":"2024-09-13T17:17:55.943470+0000","last_active":"2024-09-13T17:17:55.943470+0000","last_peered":"2024-09-13T17:17:55.943470+0000","last_clean":"2024-09-13T17:17:55.943470+0000","last_became_active":"2024-09-13T16:47:59.054943+0000","last_became_peered":"2024-09-13T16:47:59.054943+0000","last_unstale":"2024-09-13T17:17:55.943470+0000","last_undegraded":"2024-09-13T17:17:55.943470+0000","last_fullsized":"2024-09-13T17:17:55.943470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5559","last_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_deep_scrub":"63'5559","last_deep_scrub_stamp":"2024-09-13T17:17:55.943168+0000","last_clean_scrub_stamp":"2024-09-13T17:17:55.943168+0000","objects_scrubbed":0,"log_size":5559,"log_dups_size":0,"ondisk_log_size":5559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:10.227939+0000","scrub_duration":0.037150519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30370,"num_write_kb":379865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"63'2","reported_seq":77,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.149583+0000","last_change":"2024-09-13T17:18:46.149840+0000","last_active":"2024-09-13T17:18:46.149583+0000","last_peered":"2024-09-13T17:18:46.149583+0000","last_clean":"2024-09-13T17:18:46.149583+0000","last_became_active":"2024-09-13T16:48:07.771779+0000","last_became_peered":"2024-09-13T16:48:07.771779+0000","last_unstale":"2024-09-13T17:18:46.149583+0000","last_undegraded":"2024-09-13T17:18:46.149583+0000","last_fullsized":"2024-09-13T17:18:46.149583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:46.149528+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:46.149528+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.149528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:41.450773+0000","scrub_duration":0.003414533,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"63'3581","reported_seq":3655,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:04.446149+0000","last_change":"2024-09-13T17:19:04.446383+0000","last_active":"2024-09-13T17:19:04.446149+0000","last_peered":"2024-09-13T17:19:04.446149+0000","last_clean":"2024-09-13T17:19:04.446149+0000","last_became_active":"2024-09-13T16:48:08.205149+0000","last_became_peered":"2024-09-13T16:48:08.205149+0000","last_unstale":"2024-09-13T17:19:04.446149+0000","last_undegraded":"2024-09-13T17:19:04.446149+0000","last_fullsized":"2024-09-13T17:19:04.446149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3581","last_scrub_stamp":"2024-09-13T17:19:04.446098+0000","last_deep_scrub":"63'3581","last_deep_scrub_stamp":"2024-09-13T17:19:04.446098+0000","last_clean_scrub_stamp":"2024-09-13T17:19:04.446098+0000","objects_scrubbed":0,"log_size":3581,"log_dups_size":0,"ondisk_log_size":3581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:00.289523+0000","scrub_duration":0.033645664999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18910,"num_write_kb":268344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"64'44","reported_seq":161,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:59.331452+0000","last_change":"2024-09-13T17:18:59.331766+0000","last_active":"2024-09-13T17:18:59.331452+0000","last_peered":"2024-09-13T17:18:59.331452+0000","last_clean":"2024-09-13T17:18:59.331452+0000","last_became_active":"2024-09-13T16:47:58.692222+0000","last_became_peered":"2024-09-13T16:47:58.692222+0000","last_unstale":"2024-09-13T17:18:59.331452+0000","last_undegraded":"2024-09-13T17:18:59.331452+0000","last_fullsized":"2024-09-13T17:18:59.331452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'44","last_scrub_stamp":"2024-09-13T17:18:59.331399+0000","last_deep_scrub":"64'44","last_deep_scrub_stamp":"2024-09-13T17:18:59.331399+0000","last_clean_scrub_stamp":"2024-09-13T17:18:59.331399+0000","objects_scrubbed":1,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:00.438654+0000","scrub_duration":0.0059461519999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'3493","reported_seq":3573,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:07.660231+0000","last_change":"2024-09-13T17:19:07.660463+0000","last_active":"2024-09-13T17:19:07.660231+0000","last_peered":"2024-09-13T17:19:07.660231+0000","last_clean":"2024-09-13T17:19:07.660231+0000","last_became_active":"2024-09-13T16:47:59.051557+0000","last_became_peered":"2024-09-13T16:47:59.051557+0000","last_unstale":"2024-09-13T17:19:07.660231+0000","last_undegraded":"2024-09-13T17:19:07.660231+0000","last_fullsized":"2024-09-13T17:19:07.660231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3493","last_scrub_stamp":"2024-09-13T17:19:07.660176+0000","last_deep_scrub":"63'3493","last_deep_scrub_stamp":"2024-09-13T17:19:07.660176+0000","last_clean_scrub_stamp":"2024-09-13T17:19:07.660176+0000","objects_scrubbed":0,"log_size":3493,"log_dups_size":0,"ondisk_log_size":3493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:27.818699+0000","scrub_duration":0.029529214000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18921,"num_write_kb":256078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"64'727","reported_seq":827,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.186917+0000","last_change":"2024-09-13T17:17:57.186917+0000","last_active":"2024-09-13T17:17:57.186917+0000","last_peered":"2024-09-13T17:17:57.186917+0000","last_clean":"2024-09-13T17:17:57.186917+0000","last_became_active":"2024-09-13T16:48:07.783477+0000","last_became_peered":"2024-09-13T16:48:07.783477+0000","last_unstale":"2024-09-13T17:17:57.186917+0000","last_undegraded":"2024-09-13T17:17:57.186917+0000","last_fullsized":"2024-09-13T17:17:57.186917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'727","last_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_deep_scrub":"64'727","last_deep_scrub_stamp":"2024-09-13T17:17:57.186269+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.186269+0000","objects_scrubbed":1,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:00.928968+0000","scrub_duration":0.026043038000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":75389,"num_write":714,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"63'3135","reported_seq":3209,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:42.600862+0000","last_change":"2024-09-13T17:18:42.601105+0000","last_active":"2024-09-13T17:18:42.600862+0000","last_peered":"2024-09-13T17:18:42.600862+0000","last_clean":"2024-09-13T17:18:42.600862+0000","last_became_active":"2024-09-13T16:48:08.172188+0000","last_became_peered":"2024-09-13T16:48:08.172188+0000","last_unstale":"2024-09-13T17:18:42.600862+0000","last_undegraded":"2024-09-13T17:18:42.600862+0000","last_fullsized":"2024-09-13T17:18:42.600862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3135","last_scrub_stamp":"2024-09-13T17:18:42.600803+0000","last_deep_scrub":"63'3135","last_deep_scrub_stamp":"2024-09-13T17:18:42.600803+0000","last_clean_scrub_stamp":"2024-09-13T17:18:42.600803+0000","objects_scrubbed":0,"log_size":3135,"log_dups_size":0,"ondisk_log_size":3135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:53.145621+0000","scrub_duration":0.022581403999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":232447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"64'6","reported_seq":123,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:50.338514+0000","last_change":"2024-09-13T17:18:50.338743+0000","last_active":"2024-09-13T17:18:50.338514+0000","last_peered":"2024-09-13T17:18:50.338514+0000","last_clean":"2024-09-13T17:18:50.338514+0000","last_became_active":"2024-09-13T16:47:58.683524+0000","last_became_peered":"2024-09-13T16:47:58.683524+0000","last_unstale":"2024-09-13T17:18:50.338514+0000","last_undegraded":"2024-09-13T17:18:50.338514+0000","last_fullsized":"2024-09-13T17:18:50.338514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-09-13T17:18:50.338459+0000","last_deep_scrub":"64'6","last_deep_scrub_stamp":"2024-09-13T17:18:50.338459+0000","last_clean_scrub_stamp":"2024-09-13T17:18:50.338459+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:06.469004+0000","scrub_duration":0.0063925359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"63'5195","reported_seq":5273,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.511870+0000","last_change":"2024-09-13T17:17:49.511870+0000","last_active":"2024-09-13T17:17:49.511870+0000","last_peered":"2024-09-13T17:17:49.511870+0000","last_clean":"2024-09-13T17:17:49.511870+0000","last_became_active":"2024-09-13T16:47:59.045272+0000","last_became_peered":"2024-09-13T16:47:59.045272+0000","last_unstale":"2024-09-13T17:17:49.511870+0000","last_undegraded":"2024-09-13T17:17:49.511870+0000","last_fullsized":"2024-09-13T17:17:49.511870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5195","last_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_deep_scrub":"63'5195","last_deep_scrub_stamp":"2024-09-13T17:17:49.511449+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.511449+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:30.737466+0000","scrub_duration":0.041661353999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29369,"num_write_kb":359470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"64'211","reported_seq":286,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.156875+0000","last_change":"2024-09-13T17:18:44.157099+0000","last_active":"2024-09-13T17:18:44.156875+0000","last_peered":"2024-09-13T17:18:44.156875+0000","last_clean":"2024-09-13T17:18:44.156875+0000","last_became_active":"2024-09-13T16:48:07.778061+0000","last_became_peered":"2024-09-13T16:48:07.778061+0000","last_unstale":"2024-09-13T17:18:44.156875+0000","last_undegraded":"2024-09-13T17:18:44.156875+0000","last_fullsized":"2024-09-13T17:18:44.156875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'211","last_scrub_stamp":"2024-09-13T17:18:44.156820+0000","last_deep_scrub":"64'211","last_deep_scrub_stamp":"2024-09-13T17:18:44.156820+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.156820+0000","objects_scrubbed":1,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:21.506931+0000","scrub_duration":0.018253704999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2153565,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":2214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"63'3570","reported_seq":3647,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:10.301577+0000","last_change":"2024-09-13T17:19:10.301813+0000","last_active":"2024-09-13T17:19:10.301577+0000","last_peered":"2024-09-13T17:19:10.301577+0000","last_clean":"2024-09-13T17:19:10.301577+0000","last_became_active":"2024-09-13T16:48:08.199898+0000","last_became_peered":"2024-09-13T16:48:08.199898+0000","last_unstale":"2024-09-13T17:19:10.301577+0000","last_undegraded":"2024-09-13T17:19:10.301577+0000","last_fullsized":"2024-09-13T17:19:10.301577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3570","last_scrub_stamp":"2024-09-13T17:19:10.301525+0000","last_deep_scrub":"63'3570","last_deep_scrub_stamp":"2024-09-13T17:19:10.301525+0000","last_clean_scrub_stamp":"2024-09-13T17:19:10.301525+0000","objects_scrubbed":0,"log_size":3570,"log_dups_size":0,"ondisk_log_size":3570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:11.408781+0000","scrub_duration":0.028621875000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4,"num_write":19362,"num_write_kb":282506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"63'4274","reported_seq":4354,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.521451+0000","last_change":"2024-09-13T17:18:01.521451+0000","last_active":"2024-09-13T17:18:01.521451+0000","last_peered":"2024-09-13T17:18:01.521451+0000","last_clean":"2024-09-13T17:18:01.521451+0000","last_became_active":"2024-09-13T16:47:59.051303+0000","last_became_peered":"2024-09-13T16:47:59.051303+0000","last_unstale":"2024-09-13T17:18:01.521451+0000","last_undegraded":"2024-09-13T17:18:01.521451+0000","last_fullsized":"2024-09-13T17:18:01.521451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4274","last_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_deep_scrub":"63'4274","last_deep_scrub_stamp":"2024-09-13T17:18:01.521104+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.521104+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:51.017545+0000","scrub_duration":0.027199207999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23751,"num_write_kb":312338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"64'1421","reported_seq":1547,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.347010+0000","last_change":"2024-09-13T17:18:46.347010+0000","last_active":"2024-09-13T17:18:46.347010+0000","last_peered":"2024-09-13T17:18:46.347010+0000","last_clean":"2024-09-13T17:18:46.347010+0000","last_became_active":"2024-09-13T16:47:58.682829+0000","last_became_peered":"2024-09-13T16:47:58.682829+0000","last_unstale":"2024-09-13T17:18:46.347010+0000","last_undegraded":"2024-09-13T17:18:46.347010+0000","last_fullsized":"2024-09-13T17:18:46.347010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1421","last_scrub_stamp":"2024-09-13T17:18:46.346701+0000","last_deep_scrub":"64'1421","last_deep_scrub_stamp":"2024-09-13T17:18:46.346701+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.346701+0000","objects_scrubbed":2,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:47.745552+0000","scrub_duration":0.031375147999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":42245,"num_write":1396,"num_write_kb":4491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'7","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.140201+0000","last_change":"2024-09-13T17:18:01.140415+0000","last_active":"2024-09-13T17:18:01.140201+0000","last_peered":"2024-09-13T17:18:01.140201+0000","last_clean":"2024-09-13T17:18:01.140201+0000","last_became_active":"2024-09-13T16:48:07.770124+0000","last_became_peered":"2024-09-13T16:48:07.770124+0000","last_unstale":"2024-09-13T17:18:01.140201+0000","last_undegraded":"2024-09-13T17:18:01.140201+0000","last_fullsized":"2024-09-13T17:18:01.140201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'7","last_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_deep_scrub":"64'7","last_deep_scrub_stamp":"2024-09-13T17:18:01.140142+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.140142+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:50.636583+0000","scrub_duration":0.0071129380000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"63'4212","reported_seq":4290,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:32.577261+0000","last_change":"2024-09-13T17:18:32.577494+0000","last_active":"2024-09-13T17:18:32.577261+0000","last_peered":"2024-09-13T17:18:32.577261+0000","last_clean":"2024-09-13T17:18:32.577261+0000","last_became_active":"2024-09-13T16:48:08.180368+0000","last_became_peered":"2024-09-13T16:48:08.180368+0000","last_unstale":"2024-09-13T17:18:32.577261+0000","last_undegraded":"2024-09-13T17:18:32.577261+0000","last_fullsized":"2024-09-13T17:18:32.577261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4212","last_scrub_stamp":"2024-09-13T17:18:32.577202+0000","last_deep_scrub":"63'4212","last_deep_scrub_stamp":"2024-09-13T17:18:32.577202+0000","last_clean_scrub_stamp":"2024-09-13T17:18:32.577202+0000","objects_scrubbed":0,"log_size":4212,"log_dups_size":0,"ondisk_log_size":4212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:31.849084+0000","scrub_duration":0.024339607999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":17,"num_write":23343,"num_write_kb":314959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"63'3296","reported_seq":3375,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.980218+0000","last_change":"2024-09-13T17:19:05.980458+0000","last_active":"2024-09-13T17:19:05.980218+0000","last_peered":"2024-09-13T17:19:05.980218+0000","last_clean":"2024-09-13T17:19:05.980218+0000","last_became_active":"2024-09-13T16:47:59.059814+0000","last_became_peered":"2024-09-13T16:47:59.059814+0000","last_unstale":"2024-09-13T17:19:05.980218+0000","last_undegraded":"2024-09-13T17:19:05.980218+0000","last_fullsized":"2024-09-13T17:19:05.980218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3296","last_scrub_stamp":"2024-09-13T17:19:05.980161+0000","last_deep_scrub":"63'3296","last_deep_scrub_stamp":"2024-09-13T17:19:05.980161+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.980161+0000","objects_scrubbed":0,"log_size":3296,"log_dups_size":0,"ondisk_log_size":3296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:26.138684+0000","scrub_duration":0.023800585999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17758,"num_write_kb":230990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'3","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.272530+0000","last_change":"2024-09-13T17:18:44.272747+0000","last_active":"2024-09-13T17:18:44.272530+0000","last_peered":"2024-09-13T17:18:44.272530+0000","last_clean":"2024-09-13T17:18:44.272530+0000","last_became_active":"2024-09-13T16:47:58.661818+0000","last_became_peered":"2024-09-13T16:47:58.661818+0000","last_unstale":"2024-09-13T17:18:44.272530+0000","last_undegraded":"2024-09-13T17:18:44.272530+0000","last_fullsized":"2024-09-13T17:18:44.272530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:44.272476+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:44.272476+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.272476+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:21.622587+0000","scrub_duration":0.0033040719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.139202+0000","last_change":"2024-09-13T17:18:05.139417+0000","last_active":"2024-09-13T17:18:05.139202+0000","last_peered":"2024-09-13T17:18:05.139202+0000","last_clean":"2024-09-13T17:18:05.139202+0000","last_became_active":"2024-09-13T16:48:07.778244+0000","last_became_peered":"2024-09-13T16:48:07.778244+0000","last_unstale":"2024-09-13T17:18:05.139202+0000","last_undegraded":"2024-09-13T17:18:05.139202+0000","last_fullsized":"2024-09-13T17:18:05.139202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:05.139147+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:05.139147+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.139147+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:15.079769+0000","scrub_duration":0.0035900110000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"63'3371","reported_seq":3447,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:54.182862+0000","last_change":"2024-09-13T17:18:54.183094+0000","last_active":"2024-09-13T17:18:54.182862+0000","last_peered":"2024-09-13T17:18:54.182862+0000","last_clean":"2024-09-13T17:18:54.182862+0000","last_became_active":"2024-09-13T16:48:08.193626+0000","last_became_peered":"2024-09-13T16:48:08.193626+0000","last_unstale":"2024-09-13T17:18:54.182862+0000","last_undegraded":"2024-09-13T17:18:54.182862+0000","last_fullsized":"2024-09-13T17:18:54.182862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3371","last_scrub_stamp":"2024-09-13T17:18:54.182803+0000","last_deep_scrub":"63'3371","last_deep_scrub_stamp":"2024-09-13T17:18:54.182803+0000","last_clean_scrub_stamp":"2024-09-13T17:18:54.182803+0000","objects_scrubbed":0,"log_size":3371,"log_dups_size":0,"ondisk_log_size":3371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:04.727620+0000","scrub_duration":0.031207789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18156,"num_write_kb":249012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"63'3752","reported_seq":3832,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:59.895440+0000","last_change":"2024-09-13T17:18:59.895722+0000","last_active":"2024-09-13T17:18:59.895440+0000","last_peered":"2024-09-13T17:18:59.895440+0000","last_clean":"2024-09-13T17:18:59.895440+0000","last_became_active":"2024-09-13T16:47:59.051946+0000","last_became_peered":"2024-09-13T16:47:59.051946+0000","last_unstale":"2024-09-13T17:18:59.895440+0000","last_undegraded":"2024-09-13T17:18:59.895440+0000","last_fullsized":"2024-09-13T17:18:59.895440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3752","last_scrub_stamp":"2024-09-13T17:18:59.895370+0000","last_deep_scrub":"63'3752","last_deep_scrub_stamp":"2024-09-13T17:18:59.895370+0000","last_clean_scrub_stamp":"2024-09-13T17:18:59.895370+0000","objects_scrubbed":0,"log_size":3752,"log_dups_size":0,"ondisk_log_size":3752,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:01.002625+0000","scrub_duration":0.023944989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21127,"num_write_kb":268998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"63'2","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:56.315394+0000","last_change":"2024-09-13T17:18:56.315620+0000","last_active":"2024-09-13T17:18:56.315394+0000","last_peered":"2024-09-13T17:18:56.315394+0000","last_clean":"2024-09-13T17:18:56.315394+0000","last_became_active":"2024-09-13T16:47:58.666465+0000","last_became_peered":"2024-09-13T16:47:58.666465+0000","last_unstale":"2024-09-13T17:18:56.315394+0000","last_undegraded":"2024-09-13T17:18:56.315394+0000","last_fullsized":"2024-09-13T17:18:56.315394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:56.315340+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:56.315340+0000","last_clean_scrub_stamp":"2024-09-13T17:18:56.315340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:29.009922+0000","scrub_duration":0.003501695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"64'5","reported_seq":102,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:11.257475+0000","last_change":"2024-09-13T17:18:11.257689+0000","last_active":"2024-09-13T17:18:11.257475+0000","last_peered":"2024-09-13T17:18:11.257475+0000","last_clean":"2024-09-13T17:18:11.257475+0000","last_became_active":"2024-09-13T16:48:07.778074+0000","last_became_peered":"2024-09-13T16:48:07.778074+0000","last_unstale":"2024-09-13T17:18:11.257475+0000","last_undegraded":"2024-09-13T17:18:11.257475+0000","last_fullsized":"2024-09-13T17:18:11.257475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:11.257421+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:11.257421+0000","last_clean_scrub_stamp":"2024-09-13T17:18:11.257421+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:03.763129+0000","scrub_duration":0.0063405759999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"63'4499","reported_seq":4578,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:17.572279+0000","last_change":"2024-09-13T17:19:17.572534+0000","last_active":"2024-09-13T17:19:17.572279+0000","last_peered":"2024-09-13T17:19:17.572279+0000","last_clean":"2024-09-13T17:19:17.572279+0000","last_became_active":"2024-09-13T16:48:08.199667+0000","last_became_peered":"2024-09-13T16:48:08.199667+0000","last_unstale":"2024-09-13T17:19:17.572279+0000","last_undegraded":"2024-09-13T17:19:17.572279+0000","last_fullsized":"2024-09-13T17:19:17.572279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4499","last_scrub_stamp":"2024-09-13T17:19:17.572227+0000","last_deep_scrub":"63'4499","last_deep_scrub_stamp":"2024-09-13T17:19:17.572227+0000","last_clean_scrub_stamp":"2024-09-13T17:19:17.572227+0000","objects_scrubbed":0,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:17.551077+0000","scrub_duration":0.038787393000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":7,"num_write":24341,"num_write_kb":343418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"63'4043","reported_seq":4123,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:31.227364+0000","last_change":"2024-09-13T17:18:31.227589+0000","last_active":"2024-09-13T17:18:31.227364+0000","last_peered":"2024-09-13T17:18:31.227364+0000","last_clean":"2024-09-13T17:18:31.227364+0000","last_became_active":"2024-09-13T16:47:59.038255+0000","last_became_peered":"2024-09-13T16:47:59.038255+0000","last_unstale":"2024-09-13T17:18:31.227364+0000","last_undegraded":"2024-09-13T17:18:31.227364+0000","last_fullsized":"2024-09-13T17:18:31.227364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4043","last_scrub_stamp":"2024-09-13T17:18:31.227305+0000","last_deep_scrub":"63'4043","last_deep_scrub_stamp":"2024-09-13T17:18:31.227305+0000","last_clean_scrub_stamp":"2024-09-13T17:18:31.227305+0000","objects_scrubbed":0,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:30.499187+0000","scrub_duration":0.032759703000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22005,"num_write_kb":288910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"63'5","reported_seq":86,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:52.326458+0000","last_change":"2024-09-13T17:18:52.326686+0000","last_active":"2024-09-13T17:18:52.326458+0000","last_peered":"2024-09-13T17:18:52.326458+0000","last_clean":"2024-09-13T17:18:52.326458+0000","last_became_active":"2024-09-13T16:47:58.683614+0000","last_became_peered":"2024-09-13T16:47:58.683614+0000","last_unstale":"2024-09-13T17:18:52.326458+0000","last_undegraded":"2024-09-13T17:18:52.326458+0000","last_fullsized":"2024-09-13T17:18:52.326458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:52.326404+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:52.326404+0000","last_clean_scrub_stamp":"2024-09-13T17:18:52.326404+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:28.371100+0000","scrub_duration":0.0033383689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.171804+0000","last_change":"2024-09-13T17:18:04.172104+0000","last_active":"2024-09-13T17:18:04.171804+0000","last_peered":"2024-09-13T17:18:04.171804+0000","last_clean":"2024-09-13T17:18:04.171804+0000","last_became_active":"2024-09-13T16:48:07.770417+0000","last_became_peered":"2024-09-13T16:48:07.770417+0000","last_unstale":"2024-09-13T17:18:04.171804+0000","last_undegraded":"2024-09-13T17:18:04.171804+0000","last_fullsized":"2024-09-13T17:18:04.171804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:04.171747+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.171747+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:58.830350+0000","scrub_duration":0.0065161840000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"63'3470","reported_seq":3547,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:30.976569+0000","last_change":"2024-09-13T17:18:30.976875+0000","last_active":"2024-09-13T17:18:30.976569+0000","last_peered":"2024-09-13T17:18:30.976569+0000","last_clean":"2024-09-13T17:18:30.976569+0000","last_became_active":"2024-09-13T16:48:08.164404+0000","last_became_peered":"2024-09-13T16:48:08.164404+0000","last_unstale":"2024-09-13T17:18:30.976569+0000","last_undegraded":"2024-09-13T17:18:30.976569+0000","last_fullsized":"2024-09-13T17:18:30.976569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3470","last_scrub_stamp":"2024-09-13T17:18:30.976507+0000","last_deep_scrub":"63'3470","last_deep_scrub_stamp":"2024-09-13T17:18:30.976507+0000","last_clean_scrub_stamp":"2024-09-13T17:18:30.976507+0000","objects_scrubbed":0,"log_size":3470,"log_dups_size":0,"ondisk_log_size":3470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:08.326618+0000","scrub_duration":0.024724408999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19441,"num_write_kb":258288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'4844","reported_seq":4925,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:18.899218+0000","last_change":"2024-09-13T17:18:18.899218+0000","last_active":"2024-09-13T17:18:18.899218+0000","last_peered":"2024-09-13T17:18:18.899218+0000","last_clean":"2024-09-13T17:18:18.899218+0000","last_became_active":"2024-09-13T16:47:59.045800+0000","last_became_peered":"2024-09-13T16:47:59.045800+0000","last_unstale":"2024-09-13T17:18:18.899218+0000","last_undegraded":"2024-09-13T17:18:18.899218+0000","last_fullsized":"2024-09-13T17:18:18.899218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4844","last_scrub_stamp":"2024-09-13T17:18:18.898914+0000","last_deep_scrub":"63'4844","last_deep_scrub_stamp":"2024-09-13T17:18:18.898914+0000","last_clean_scrub_stamp":"2024-09-13T17:18:18.898914+0000","objects_scrubbed":0,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:18.212032+0000","scrub_duration":0.026998708999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":26635,"num_write_kb":336207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"64'10","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.498454+0000","last_change":"2024-09-13T17:18:05.498454+0000","last_active":"2024-09-13T17:18:05.498454+0000","last_peered":"2024-09-13T17:18:05.498454+0000","last_clean":"2024-09-13T17:18:05.498454+0000","last_became_active":"2024-09-13T16:47:58.692379+0000","last_became_peered":"2024-09-13T16:47:58.692379+0000","last_unstale":"2024-09-13T17:18:05.498454+0000","last_undegraded":"2024-09-13T17:18:05.498454+0000","last_fullsized":"2024-09-13T17:18:05.498454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_deep_scrub":"64'10","last_deep_scrub_stamp":"2024-09-13T17:18:05.498184+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.498184+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:15.438806+0000","scrub_duration":0.0057060160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'49","reported_seq":190,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.166876+0000","last_change":"2024-09-13T17:17:58.166876+0000","last_active":"2024-09-13T17:17:58.166876+0000","last_peered":"2024-09-13T17:17:58.166876+0000","last_clean":"2024-09-13T17:17:58.166876+0000","last_became_active":"2024-09-13T16:48:07.772343+0000","last_became_peered":"2024-09-13T16:48:07.772343+0000","last_unstale":"2024-09-13T17:17:58.166876+0000","last_undegraded":"2024-09-13T17:17:58.166876+0000","last_fullsized":"2024-09-13T17:17:58.166876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'49","last_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_deep_scrub":"64'49","last_deep_scrub_stamp":"2024-09-13T17:17:58.166530+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.166530+0000","objects_scrubbed":1,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:05.921781+0000","scrub_duration":0.0073340999999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92726,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":118,"num_write":43,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"63'3635","reported_seq":3713,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:11.235091+0000","last_change":"2024-09-13T17:19:11.235091+0000","last_active":"2024-09-13T17:19:11.235091+0000","last_peered":"2024-09-13T17:19:11.235091+0000","last_clean":"2024-09-13T17:19:11.235091+0000","last_became_active":"2024-09-13T16:48:08.191241+0000","last_became_peered":"2024-09-13T16:48:08.191241+0000","last_unstale":"2024-09-13T17:19:11.235091+0000","last_undegraded":"2024-09-13T17:19:11.235091+0000","last_fullsized":"2024-09-13T17:19:11.235091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3635","last_scrub_stamp":"2024-09-13T17:19:11.234785+0000","last_deep_scrub":"63'3635","last_deep_scrub_stamp":"2024-09-13T17:19:11.234785+0000","last_clean_scrub_stamp":"2024-09-13T17:19:11.234785+0000","objects_scrubbed":0,"log_size":3635,"log_dups_size":0,"ondisk_log_size":3635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:22.950287+0000","scrub_duration":0.030795928,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19787,"num_write_kb":259620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'3639","reported_seq":3719,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:03.984961+0000","last_change":"2024-09-13T17:19:03.985202+0000","last_active":"2024-09-13T17:19:03.984961+0000","last_peered":"2024-09-13T17:19:03.984961+0000","last_clean":"2024-09-13T17:19:03.984961+0000","last_became_active":"2024-09-13T16:47:59.062828+0000","last_became_peered":"2024-09-13T16:47:59.062828+0000","last_unstale":"2024-09-13T17:19:03.984961+0000","last_undegraded":"2024-09-13T17:19:03.984961+0000","last_fullsized":"2024-09-13T17:19:03.984961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3639","last_scrub_stamp":"2024-09-13T17:19:03.984902+0000","last_deep_scrub":"63'3639","last_deep_scrub_stamp":"2024-09-13T17:19:03.984902+0000","last_clean_scrub_stamp":"2024-09-13T17:19:03.984902+0000","objects_scrubbed":0,"log_size":3639,"log_dups_size":0,"ondisk_log_size":3639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:32.587792+0000","scrub_duration":0.028565929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20066,"num_write_kb":240267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"64'268","reported_seq":368,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:54.361866+0000","last_change":"2024-09-13T17:18:54.362093+0000","last_active":"2024-09-13T17:18:54.361866+0000","last_peered":"2024-09-13T17:18:54.361866+0000","last_clean":"2024-09-13T17:18:54.361866+0000","last_became_active":"2024-09-13T16:47:58.682076+0000","last_became_peered":"2024-09-13T16:47:58.682076+0000","last_unstale":"2024-09-13T17:18:54.361866+0000","last_undegraded":"2024-09-13T17:18:54.361866+0000","last_fullsized":"2024-09-13T17:18:54.361866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'268","last_scrub_stamp":"2024-09-13T17:18:54.361813+0000","last_deep_scrub":"64'268","last_deep_scrub_stamp":"2024-09-13T17:18:54.361813+0000","last_clean_scrub_stamp":"2024-09-13T17:18:54.361813+0000","objects_scrubbed":1,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:04.906631+0000","scrub_duration":0.0081052720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":38,"num_read_kb":48,"num_write":636,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"63'11","reported_seq":89,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.102762+0000","last_change":"2024-09-13T17:17:50.102762+0000","last_active":"2024-09-13T17:17:50.102762+0000","last_peered":"2024-09-13T17:17:50.102762+0000","last_clean":"2024-09-13T17:17:50.102762+0000","last_became_active":"2024-09-13T16:48:07.789992+0000","last_became_peered":"2024-09-13T16:48:07.789992+0000","last_unstale":"2024-09-13T17:17:50.102762+0000","last_undegraded":"2024-09-13T17:17:50.102762+0000","last_fullsized":"2024-09-13T17:17:50.102762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'11","last_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_deep_scrub":"63'11","last_deep_scrub_stamp":"2024-09-13T17:17:50.102616+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.102616+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.736833+0000","scrub_duration":0.0039927280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"63'3190","reported_seq":3267,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:52.074598+0000","last_change":"2024-09-13T17:18:52.074598+0000","last_active":"2024-09-13T17:18:52.074598+0000","last_peered":"2024-09-13T17:18:52.074598+0000","last_clean":"2024-09-13T17:18:52.074598+0000","last_became_active":"2024-09-13T16:48:08.169162+0000","last_became_peered":"2024-09-13T16:48:08.169162+0000","last_unstale":"2024-09-13T17:18:52.074598+0000","last_undegraded":"2024-09-13T17:18:52.074598+0000","last_fullsized":"2024-09-13T17:18:52.074598+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3190","last_scrub_stamp":"2024-09-13T17:18:52.074315+0000","last_deep_scrub":"63'3190","last_deep_scrub_stamp":"2024-09-13T17:18:52.074315+0000","last_clean_scrub_stamp":"2024-09-13T17:18:52.074315+0000","objects_scrubbed":0,"log_size":3190,"log_dups_size":0,"ondisk_log_size":3190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:02.619132+0000","scrub_duration":0.01789174,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17591,"num_write_kb":254866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"63'4701","reported_seq":4782,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:03.191365+0000","last_change":"2024-09-13T17:19:03.191365+0000","last_active":"2024-09-13T17:19:03.191365+0000","last_peered":"2024-09-13T17:19:03.191365+0000","last_clean":"2024-09-13T17:19:03.191365+0000","last_became_active":"2024-09-13T16:47:59.037570+0000","last_became_peered":"2024-09-13T16:47:59.037570+0000","last_unstale":"2024-09-13T17:19:03.191365+0000","last_undegraded":"2024-09-13T17:19:03.191365+0000","last_fullsized":"2024-09-13T17:19:03.191365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4701","last_scrub_stamp":"2024-09-13T17:19:03.191038+0000","last_deep_scrub":"63'4701","last_deep_scrub_stamp":"2024-09-13T17:19:03.191038+0000","last_clean_scrub_stamp":"2024-09-13T17:19:03.191038+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:19.321583+0000","scrub_duration":0.022656645999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26171,"num_write_kb":338588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"63'2","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:58.283758+0000","last_change":"2024-09-13T17:18:58.283758+0000","last_active":"2024-09-13T17:18:58.283758+0000","last_peered":"2024-09-13T17:18:58.283758+0000","last_clean":"2024-09-13T17:18:58.283758+0000","last_became_active":"2024-09-13T16:47:58.683977+0000","last_became_peered":"2024-09-13T16:47:58.683977+0000","last_unstale":"2024-09-13T17:18:58.283758+0000","last_undegraded":"2024-09-13T17:18:58.283758+0000","last_fullsized":"2024-09-13T17:18:58.283758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:58.283441+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:58.283441+0000","last_clean_scrub_stamp":"2024-09-13T17:18:58.283441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:09.998943+0000","scrub_duration":0.0033138870000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"63'3","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:57.112301+0000","last_change":"2024-09-13T17:18:57.112301+0000","last_active":"2024-09-13T17:18:57.112301+0000","last_peered":"2024-09-13T17:18:57.112301+0000","last_clean":"2024-09-13T17:18:57.112301+0000","last_became_active":"2024-09-13T16:48:07.771754+0000","last_became_peered":"2024-09-13T16:48:07.771754+0000","last_unstale":"2024-09-13T17:18:57.112301+0000","last_undegraded":"2024-09-13T17:18:57.112301+0000","last_fullsized":"2024-09-13T17:18:57.112301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:57.112022+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:57.112022+0000","last_clean_scrub_stamp":"2024-09-13T17:18:57.112022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:14.085716+0000","scrub_duration":0.0037361640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"63'3859","reported_seq":3936,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:11.287834+0000","last_change":"2024-09-13T17:19:11.287834+0000","last_active":"2024-09-13T17:19:11.287834+0000","last_peered":"2024-09-13T17:19:11.287834+0000","last_clean":"2024-09-13T17:19:11.287834+0000","last_became_active":"2024-09-13T16:48:08.177434+0000","last_became_peered":"2024-09-13T16:48:08.177434+0000","last_unstale":"2024-09-13T17:19:11.287834+0000","last_undegraded":"2024-09-13T17:19:11.287834+0000","last_fullsized":"2024-09-13T17:19:11.287834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3859","last_scrub_stamp":"2024-09-13T17:19:11.287545+0000","last_deep_scrub":"63'3859","last_deep_scrub_stamp":"2024-09-13T17:19:11.287545+0000","last_clean_scrub_stamp":"2024-09-13T17:19:11.287545+0000","objects_scrubbed":0,"log_size":3859,"log_dups_size":0,"ondisk_log_size":3859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:51.713630+0000","scrub_duration":0.019334607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21286,"num_write_kb":290222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"63'3876","reported_seq":3957,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:31.476681+0000","last_change":"2024-09-13T17:18:31.476681+0000","last_active":"2024-09-13T17:18:31.476681+0000","last_peered":"2024-09-13T17:18:31.476681+0000","last_clean":"2024-09-13T17:18:31.476681+0000","last_became_active":"2024-09-13T16:47:59.056320+0000","last_became_peered":"2024-09-13T16:47:59.056320+0000","last_unstale":"2024-09-13T17:18:31.476681+0000","last_undegraded":"2024-09-13T17:18:31.476681+0000","last_fullsized":"2024-09-13T17:18:31.476681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3876","last_scrub_stamp":"2024-09-13T17:18:31.476360+0000","last_deep_scrub":"63'3876","last_deep_scrub_stamp":"2024-09-13T17:18:31.476360+0000","last_clean_scrub_stamp":"2024-09-13T17:18:31.476360+0000","objects_scrubbed":0,"log_size":3876,"log_dups_size":0,"ondisk_log_size":3876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:48.450054+0000","scrub_duration":0.030063065999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21201,"num_write_kb":273354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"63'2","reported_seq":84,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:42.323409+0000","last_change":"2024-09-13T17:18:42.323409+0000","last_active":"2024-09-13T17:18:42.323409+0000","last_peered":"2024-09-13T17:18:42.323409+0000","last_clean":"2024-09-13T17:18:42.323409+0000","last_became_active":"2024-09-13T16:47:58.654967+0000","last_became_peered":"2024-09-13T16:47:58.654967+0000","last_unstale":"2024-09-13T17:18:42.323409+0000","last_undegraded":"2024-09-13T17:18:42.323409+0000","last_fullsized":"2024-09-13T17:18:42.323409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:42.323076+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:42.323076+0000","last_clean_scrub_stamp":"2024-09-13T17:18:42.323076+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:59.296770+0000","scrub_duration":0.003371626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"63'5","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.227606+0000","last_change":"2024-09-13T17:17:57.227606+0000","last_active":"2024-09-13T17:17:57.227606+0000","last_peered":"2024-09-13T17:17:57.227606+0000","last_clean":"2024-09-13T17:17:57.227606+0000","last_became_active":"2024-09-13T16:48:07.780747+0000","last_became_peered":"2024-09-13T16:48:07.780747+0000","last_unstale":"2024-09-13T17:17:57.227606+0000","last_undegraded":"2024-09-13T17:17:57.227606+0000","last_fullsized":"2024-09-13T17:17:57.227606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:57.226662+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226662+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:46.723103+0000","scrub_duration":0.0036245560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"63'3422","reported_seq":3499,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:10.259125+0000","last_change":"2024-09-13T17:19:10.259344+0000","last_active":"2024-09-13T17:19:10.259125+0000","last_peered":"2024-09-13T17:19:10.259125+0000","last_clean":"2024-09-13T17:19:10.259125+0000","last_became_active":"2024-09-13T16:48:08.174515+0000","last_became_peered":"2024-09-13T16:48:08.174515+0000","last_unstale":"2024-09-13T17:19:10.259125+0000","last_undegraded":"2024-09-13T17:19:10.259125+0000","last_fullsized":"2024-09-13T17:19:10.259125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3422","last_scrub_stamp":"2024-09-13T17:19:10.259072+0000","last_deep_scrub":"63'3422","last_deep_scrub_stamp":"2024-09-13T17:19:10.259072+0000","last_clean_scrub_stamp":"2024-09-13T17:19:10.259072+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:42.953654+0000","scrub_duration":0.020414504,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":7,"num_write":19042,"num_write_kb":260722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"63'4802","reported_seq":4883,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:12.261210+0000","last_change":"2024-09-13T17:18:12.261210+0000","last_active":"2024-09-13T17:18:12.261210+0000","last_peered":"2024-09-13T17:18:12.261210+0000","last_clean":"2024-09-13T17:18:12.261210+0000","last_became_active":"2024-09-13T16:47:59.048838+0000","last_became_peered":"2024-09-13T16:47:59.048838+0000","last_unstale":"2024-09-13T17:18:12.261210+0000","last_undegraded":"2024-09-13T17:18:12.261210+0000","last_fullsized":"2024-09-13T17:18:12.261210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4802","last_scrub_stamp":"2024-09-13T17:18:12.260903+0000","last_deep_scrub":"63'4802","last_deep_scrub_stamp":"2024-09-13T17:18:12.260903+0000","last_clean_scrub_stamp":"2024-09-13T17:18:12.260903+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:25.238396+0000","scrub_duration":0.040392404999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26419,"num_write_kb":335470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"63'4","reported_seq":86,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:54.699803+0000","last_change":"2024-09-13T17:18:54.699803+0000","last_active":"2024-09-13T17:18:54.699803+0000","last_peered":"2024-09-13T17:18:54.699803+0000","last_clean":"2024-09-13T17:18:54.699803+0000","last_became_active":"2024-09-13T16:47:58.662199+0000","last_became_peered":"2024-09-13T16:47:58.662199+0000","last_unstale":"2024-09-13T17:18:54.699803+0000","last_undegraded":"2024-09-13T17:18:54.699803+0000","last_fullsized":"2024-09-13T17:18:54.699803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4","last_scrub_stamp":"2024-09-13T17:18:54.699526+0000","last_deep_scrub":"63'4","last_deep_scrub_stamp":"2024-09-13T17:18:54.699526+0000","last_clean_scrub_stamp":"2024-09-13T17:18:54.699526+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:30.744221+0000","scrub_duration":0.003592694,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"63'2","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:54.181249+0000","last_change":"2024-09-13T17:17:54.181249+0000","last_active":"2024-09-13T17:17:54.181249+0000","last_peered":"2024-09-13T17:17:54.181249+0000","last_clean":"2024-09-13T17:17:54.181249+0000","last_became_active":"2024-09-13T16:48:07.767901+0000","last_became_peered":"2024-09-13T16:48:07.767901+0000","last_unstale":"2024-09-13T17:17:54.181249+0000","last_undegraded":"2024-09-13T17:17:54.181249+0000","last_fullsized":"2024-09-13T17:17:54.181249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:17:54.180966+0000","last_clean_scrub_stamp":"2024-09-13T17:17:54.180966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:37.317744+0000","scrub_duration":0.0037272709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"63'4215","reported_seq":4292,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:14.596560+0000","last_change":"2024-09-13T17:18:14.596560+0000","last_active":"2024-09-13T17:18:14.596560+0000","last_peered":"2024-09-13T17:18:14.596560+0000","last_clean":"2024-09-13T17:18:14.596560+0000","last_became_active":"2024-09-13T16:48:08.172621+0000","last_became_peered":"2024-09-13T16:48:08.172621+0000","last_unstale":"2024-09-13T17:18:14.596560+0000","last_undegraded":"2024-09-13T17:18:14.596560+0000","last_fullsized":"2024-09-13T17:18:14.596560+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4215","last_scrub_stamp":"2024-09-13T17:18:14.596269+0000","last_deep_scrub":"63'4215","last_deep_scrub_stamp":"2024-09-13T17:18:14.596269+0000","last_clean_scrub_stamp":"2024-09-13T17:18:14.596269+0000","objects_scrubbed":0,"log_size":4215,"log_dups_size":0,"ondisk_log_size":4215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:07.101977+0000","scrub_duration":0.024198185000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":8,"num_write":22461,"num_write_kb":319331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'3790","reported_seq":3871,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:59.270895+0000","last_change":"2024-09-13T17:18:59.271188+0000","last_active":"2024-09-13T17:18:59.270895+0000","last_peered":"2024-09-13T17:18:59.270895+0000","last_clean":"2024-09-13T17:18:59.270895+0000","last_became_active":"2024-09-13T16:47:59.036901+0000","last_became_peered":"2024-09-13T16:47:59.036901+0000","last_unstale":"2024-09-13T17:18:59.270895+0000","last_undegraded":"2024-09-13T17:18:59.270895+0000","last_fullsized":"2024-09-13T17:18:59.270895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3790","last_scrub_stamp":"2024-09-13T17:18:59.270842+0000","last_deep_scrub":"63'3790","last_deep_scrub_stamp":"2024-09-13T17:18:59.270842+0000","last_clean_scrub_stamp":"2024-09-13T17:18:59.270842+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:28:00.669692+0000","scrub_duration":0.017889702,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21077,"num_write_kb":260732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'42","reported_seq":180,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.603126+0000","last_change":"2024-09-13T17:18:03.603126+0000","last_active":"2024-09-13T17:18:03.603126+0000","last_peered":"2024-09-13T17:18:03.603126+0000","last_clean":"2024-09-13T17:18:03.603126+0000","last_became_active":"2024-09-13T16:47:58.663345+0000","last_became_peered":"2024-09-13T16:47:58.663345+0000","last_unstale":"2024-09-13T17:18:03.603126+0000","last_undegraded":"2024-09-13T17:18:03.603126+0000","last_fullsized":"2024-09-13T17:18:03.603126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'42","last_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_deep_scrub":"64'42","last_deep_scrub_stamp":"2024-09-13T17:18:03.602845+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.602845+0000","objects_scrubbed":1,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:03.563076+0000","scrub_duration":0.004957576,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84646,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":106,"num_write":37,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'10","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:05.102806+0000","last_change":"2024-09-13T17:18:05.102806+0000","last_active":"2024-09-13T17:18:05.102806+0000","last_peered":"2024-09-13T17:18:05.102806+0000","last_clean":"2024-09-13T17:18:05.102806+0000","last_became_active":"2024-09-13T16:48:07.767896+0000","last_became_peered":"2024-09-13T16:48:07.767896+0000","last_unstale":"2024-09-13T17:18:05.102806+0000","last_undegraded":"2024-09-13T17:18:05.102806+0000","last_fullsized":"2024-09-13T17:18:05.102806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-09-13T17:18:05.101908+0000","last_deep_scrub":"64'10","last_deep_scrub_stamp":"2024-09-13T17:18:05.101908+0000","last_clean_scrub_stamp":"2024-09-13T17:18:05.101908+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:04.415025+0000","scrub_duration":0.0065388449999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":8,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"63'3128","reported_seq":3204,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:11.594777+0000","last_change":"2024-09-13T17:19:11.594986+0000","last_active":"2024-09-13T17:19:11.594777+0000","last_peered":"2024-09-13T17:19:11.594777+0000","last_clean":"2024-09-13T17:19:11.594777+0000","last_became_active":"2024-09-13T16:48:08.175362+0000","last_became_peered":"2024-09-13T16:48:08.175362+0000","last_unstale":"2024-09-13T17:19:11.594777+0000","last_undegraded":"2024-09-13T17:19:11.594777+0000","last_fullsized":"2024-09-13T17:19:11.594777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3128","last_scrub_stamp":"2024-09-13T17:19:11.594726+0000","last_deep_scrub":"63'3128","last_deep_scrub_stamp":"2024-09-13T17:19:11.594726+0000","last_clean_scrub_stamp":"2024-09-13T17:19:11.594726+0000","objects_scrubbed":0,"log_size":3128,"log_dups_size":0,"ondisk_log_size":3128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:52.303686+0000","scrub_duration":0.023210991,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17776,"num_write_kb":243283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"63'3657","reported_seq":3738,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:14.878283+0000","last_change":"2024-09-13T17:18:14.878283+0000","last_active":"2024-09-13T17:18:14.878283+0000","last_peered":"2024-09-13T17:18:14.878283+0000","last_clean":"2024-09-13T17:18:14.878283+0000","last_became_active":"2024-09-13T16:47:59.056268+0000","last_became_peered":"2024-09-13T16:47:59.056268+0000","last_unstale":"2024-09-13T17:18:14.878283+0000","last_undegraded":"2024-09-13T17:18:14.878283+0000","last_fullsized":"2024-09-13T17:18:14.878283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3657","last_scrub_stamp":"2024-09-13T17:18:14.878012+0000","last_deep_scrub":"63'3657","last_deep_scrub_stamp":"2024-09-13T17:18:14.878012+0000","last_clean_scrub_stamp":"2024-09-13T17:18:14.878012+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:17.512229+0000","scrub_duration":0.026162280999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19816,"num_write_kb":255512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"64'11","reported_seq":93,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.280947+0000","last_change":"2024-09-13T17:18:44.281154+0000","last_active":"2024-09-13T17:18:44.280947+0000","last_peered":"2024-09-13T17:18:44.280947+0000","last_clean":"2024-09-13T17:18:44.280947+0000","last_became_active":"2024-09-13T16:47:58.684366+0000","last_became_peered":"2024-09-13T16:47:58.684366+0000","last_unstale":"2024-09-13T17:18:44.280947+0000","last_undegraded":"2024-09-13T17:18:44.280947+0000","last_fullsized":"2024-09-13T17:18:44.280947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-13T17:18:44.280891+0000","last_deep_scrub":"64'11","last_deep_scrub_stamp":"2024-09-13T17:18:44.280891+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.280891+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:24.706976+0000","scrub_duration":0.0031866279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'252","reported_seq":350,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:00.134660+0000","last_change":"2024-09-13T17:19:00.134862+0000","last_active":"2024-09-13T17:19:00.134660+0000","last_peered":"2024-09-13T17:19:00.134660+0000","last_clean":"2024-09-13T17:19:00.134660+0000","last_became_active":"2024-09-13T16:48:07.778530+0000","last_became_peered":"2024-09-13T16:48:07.778530+0000","last_unstale":"2024-09-13T17:19:00.134660+0000","last_undegraded":"2024-09-13T17:19:00.134660+0000","last_fullsized":"2024-09-13T17:19:00.134660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'252","last_scrub_stamp":"2024-09-13T17:19:00.134608+0000","last_deep_scrub":"64'252","last_deep_scrub_stamp":"2024-09-13T17:19:00.134608+0000","last_clean_scrub_stamp":"2024-09-13T17:19:00.134608+0000","objects_scrubbed":1,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:55.435852+0000","scrub_duration":0.0081424600000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":57,"num_write":597,"num_write_kb":218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"63'4141","reported_seq":4219,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.424513+0000","last_change":"2024-09-13T17:19:05.424513+0000","last_active":"2024-09-13T17:19:05.424513+0000","last_peered":"2024-09-13T17:19:05.424513+0000","last_clean":"2024-09-13T17:19:05.424513+0000","last_became_active":"2024-09-13T16:48:08.187836+0000","last_became_peered":"2024-09-13T16:48:08.187836+0000","last_unstale":"2024-09-13T17:19:05.424513+0000","last_undegraded":"2024-09-13T17:19:05.424513+0000","last_fullsized":"2024-09-13T17:19:05.424513+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4141","last_scrub_stamp":"2024-09-13T17:19:05.424228+0000","last_deep_scrub":"63'4141","last_deep_scrub_stamp":"2024-09-13T17:19:05.424228+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.424228+0000","objects_scrubbed":0,"log_size":4141,"log_dups_size":0,"ondisk_log_size":4141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:25.582751+0000","scrub_duration":0.029955119999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22849,"num_write_kb":317008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'4103","reported_seq":4192,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:42.981633+0000","last_change":"2024-09-13T17:18:42.981633+0000","last_active":"2024-09-13T17:18:42.981633+0000","last_peered":"2024-09-13T17:18:42.981633+0000","last_clean":"2024-09-13T17:18:42.981633+0000","last_became_active":"2024-09-13T16:48:41.389507+0000","last_became_peered":"2024-09-13T16:48:41.389507+0000","last_unstale":"2024-09-13T17:18:42.981633+0000","last_undegraded":"2024-09-13T17:18:42.981633+0000","last_fullsized":"2024-09-13T17:18:42.981633+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4103","last_scrub_stamp":"2024-09-13T17:18:42.981318+0000","last_deep_scrub":"63'4103","last_deep_scrub_stamp":"2024-09-13T17:18:42.981318+0000","last_clean_scrub_stamp":"2024-09-13T17:18:42.981318+0000","objects_scrubbed":0,"log_size":4103,"log_dups_size":0,"ondisk_log_size":4103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:15.675900+0000","scrub_duration":0.025114515,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22342,"num_write_kb":278194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"63'3","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:51.588758+0000","last_change":"2024-09-13T17:18:51.588758+0000","last_active":"2024-09-13T17:18:51.588758+0000","last_peered":"2024-09-13T17:18:51.588758+0000","last_clean":"2024-09-13T17:18:51.588758+0000","last_became_active":"2024-09-13T16:47:58.655592+0000","last_became_peered":"2024-09-13T16:47:58.655592+0000","last_unstale":"2024-09-13T17:18:51.588758+0000","last_undegraded":"2024-09-13T17:18:51.588758+0000","last_fullsized":"2024-09-13T17:18:51.588758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:51.588486+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:51.588486+0000","last_clean_scrub_stamp":"2024-09-13T17:18:51.588486+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:07.719031+0000","scrub_duration":0.0034308670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"64'119","reported_seq":237,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.251938+0000","last_change":"2024-09-13T17:18:43.252212+0000","last_active":"2024-09-13T17:18:43.251938+0000","last_peered":"2024-09-13T17:18:43.251938+0000","last_clean":"2024-09-13T17:18:43.251938+0000","last_became_active":"2024-09-13T16:47:48.773333+0000","last_became_peered":"2024-09-13T16:47:48.773333+0000","last_unstale":"2024-09-13T17:18:43.251938+0000","last_undegraded":"2024-09-13T17:18:43.251938+0000","last_fullsized":"2024-09-13T17:18:43.251938+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'119","last_scrub_stamp":"2024-09-13T17:18:43.251885+0000","last_deep_scrub":"64'119","last_deep_scrub_stamp":"2024-09-13T17:18:43.251885+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.251885+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:32.643640+0000","scrub_duration":0.010406291999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:40.136105+0000","last_change":"2024-09-13T17:18:40.136327+0000","last_active":"2024-09-13T17:18:40.136105+0000","last_peered":"2024-09-13T17:18:40.136105+0000","last_clean":"2024-09-13T17:18:40.136105+0000","last_became_active":"2024-09-13T16:48:07.784680+0000","last_became_peered":"2024-09-13T16:48:07.784680+0000","last_unstale":"2024-09-13T17:18:40.136105+0000","last_undegraded":"2024-09-13T17:18:40.136105+0000","last_fullsized":"2024-09-13T17:18:40.136105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:40.136049+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:40.136049+0000","last_clean_scrub_stamp":"2024-09-13T17:18:40.136049+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:52.925685+0000","scrub_duration":0.0033099650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.b","version":"63'2904","reported_seq":2980,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:48.337844+0000","last_change":"2024-09-13T17:17:48.337931+0000","last_active":"2024-09-13T17:17:48.337844+0000","last_peered":"2024-09-13T17:17:48.337844+0000","last_clean":"2024-09-13T17:17:48.337844+0000","last_became_active":"2024-09-13T16:48:08.171069+0000","last_became_peered":"2024-09-13T16:48:08.171069+0000","last_unstale":"2024-09-13T17:17:48.337844+0000","last_undegraded":"2024-09-13T17:17:48.337844+0000","last_fullsized":"2024-09-13T17:17:48.337844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2904","last_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_deep_scrub":"63'2904","last_deep_scrub_stamp":"2024-09-13T17:17:48.337825+0000","last_clean_scrub_stamp":"2024-09-13T17:17:48.337825+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:15.694145+0000","scrub_duration":0.0051743259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":2,"num_write":15607,"num_write_kb":214254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"63'4284","reported_seq":4363,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.341991+0000","last_change":"2024-09-13T17:17:52.342307+0000","last_active":"2024-09-13T17:17:52.341991+0000","last_peered":"2024-09-13T17:17:52.341991+0000","last_clean":"2024-09-13T17:17:52.341991+0000","last_became_active":"2024-09-13T16:47:59.061972+0000","last_became_peered":"2024-09-13T16:47:59.061972+0000","last_unstale":"2024-09-13T17:17:52.341991+0000","last_undegraded":"2024-09-13T17:17:52.341991+0000","last_fullsized":"2024-09-13T17:17:52.341991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4284","last_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_deep_scrub":"63'4284","last_deep_scrub_stamp":"2024-09-13T17:17:52.341937+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.341937+0000","objects_scrubbed":0,"log_size":4284,"log_dups_size":0,"ondisk_log_size":4284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.282559+0000","scrub_duration":0.043379468999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23392,"num_write_kb":310963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"63'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.502278+0000","last_change":"2024-09-13T17:17:53.502523+0000","last_active":"2024-09-13T17:17:53.502278+0000","last_peered":"2024-09-13T17:17:53.502278+0000","last_clean":"2024-09-13T17:17:53.502278+0000","last_became_active":"2024-09-13T16:47:58.676085+0000","last_became_peered":"2024-09-13T16:47:58.676085+0000","last_unstale":"2024-09-13T17:17:53.502278+0000","last_undegraded":"2024-09-13T17:17:53.502278+0000","last_fullsized":"2024-09-13T17:17:53.502278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.502221+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.502221+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:42.741599+0000","scrub_duration":0.0040979670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"63'3","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:41.132756+0000","last_change":"2024-09-13T17:18:41.133001+0000","last_active":"2024-09-13T17:18:41.132756+0000","last_peered":"2024-09-13T17:18:41.132756+0000","last_clean":"2024-09-13T17:18:41.132756+0000","last_became_active":"2024-09-13T16:48:07.776136+0000","last_became_peered":"2024-09-13T16:48:07.776136+0000","last_unstale":"2024-09-13T17:18:41.132756+0000","last_undegraded":"2024-09-13T17:18:41.132756+0000","last_fullsized":"2024-09-13T17:18:41.132756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:41.132703+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:41.132703+0000","last_clean_scrub_stamp":"2024-09-13T17:18:41.132703+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:58.106397+0000","scrub_duration":0.0037951880000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"63'4138","reported_seq":4214,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.398680+0000","last_change":"2024-09-13T17:17:49.398680+0000","last_active":"2024-09-13T17:17:49.398680+0000","last_peered":"2024-09-13T17:17:49.398680+0000","last_clean":"2024-09-13T17:17:49.398680+0000","last_became_active":"2024-09-13T16:48:08.177919+0000","last_became_peered":"2024-09-13T16:48:08.177919+0000","last_unstale":"2024-09-13T17:17:49.398680+0000","last_undegraded":"2024-09-13T17:17:49.398680+0000","last_fullsized":"2024-09-13T17:17:49.398680+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4138","last_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_deep_scrub":"63'4138","last_deep_scrub_stamp":"2024-09-13T17:17:49.398536+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.398536+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:51.412144+0000","scrub_duration":0.020992730000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22906,"num_write_kb":309086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"63'4584","reported_seq":4657,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:49.729734+0000","last_change":"2024-09-13T17:18:49.729975+0000","last_active":"2024-09-13T17:18:49.729734+0000","last_peered":"2024-09-13T17:18:49.729734+0000","last_clean":"2024-09-13T17:18:49.729734+0000","last_became_active":"2024-09-13T16:48:40.765861+0000","last_became_peered":"2024-09-13T16:48:40.765861+0000","last_unstale":"2024-09-13T17:18:49.729734+0000","last_undegraded":"2024-09-13T17:18:49.729734+0000","last_fullsized":"2024-09-13T17:18:49.729734+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4584","last_scrub_stamp":"2024-09-13T17:18:49.729674+0000","last_deep_scrub":"63'4584","last_deep_scrub_stamp":"2024-09-13T17:18:49.729674+0000","last_clean_scrub_stamp":"2024-09-13T17:18:49.729674+0000","objects_scrubbed":0,"log_size":4584,"log_dups_size":0,"ondisk_log_size":4584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:59.866520+0000","scrub_duration":0.023671662,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24690,"num_write_kb":320186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"64'7","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.308611+0000","last_change":"2024-09-13T17:18:46.308867+0000","last_active":"2024-09-13T17:18:46.308611+0000","last_peered":"2024-09-13T17:18:46.308611+0000","last_clean":"2024-09-13T17:18:46.308611+0000","last_became_active":"2024-09-13T16:47:58.661482+0000","last_became_peered":"2024-09-13T16:47:58.661482+0000","last_unstale":"2024-09-13T17:18:46.308611+0000","last_undegraded":"2024-09-13T17:18:46.308611+0000","last_fullsized":"2024-09-13T17:18:46.308611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'7","last_scrub_stamp":"2024-09-13T17:18:46.308558+0000","last_deep_scrub":"64'7","last_deep_scrub_stamp":"2024-09-13T17:18:46.308558+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.308558+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:56.445404+0000","scrub_duration":0.0073412219999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'1427","reported_seq":1551,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:58.168946+0000","last_change":"2024-09-13T17:18:58.169177+0000","last_active":"2024-09-13T17:18:58.168946+0000","last_peered":"2024-09-13T17:18:58.168946+0000","last_clean":"2024-09-13T17:18:58.168946+0000","last_became_active":"2024-09-13T16:48:07.776277+0000","last_became_peered":"2024-09-13T16:48:07.776277+0000","last_unstale":"2024-09-13T17:18:58.168946+0000","last_undegraded":"2024-09-13T17:18:58.168946+0000","last_fullsized":"2024-09-13T17:18:58.168946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1427","last_scrub_stamp":"2024-09-13T17:18:58.168889+0000","last_deep_scrub":"64'1427","last_deep_scrub_stamp":"2024-09-13T17:18:58.168889+0000","last_clean_scrub_stamp":"2024-09-13T17:18:58.168889+0000","objects_scrubbed":2,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:35.519000+0000","scrub_duration":0.031253693999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":38459,"num_write":1400,"num_write_kb":4490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"63'3637","reported_seq":3714,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:03.412582+0000","last_change":"2024-09-13T17:19:03.412806+0000","last_active":"2024-09-13T17:19:03.412582+0000","last_peered":"2024-09-13T17:19:03.412582+0000","last_clean":"2024-09-13T17:19:03.412582+0000","last_became_active":"2024-09-13T16:48:08.187790+0000","last_became_peered":"2024-09-13T16:48:08.187790+0000","last_unstale":"2024-09-13T17:19:03.412582+0000","last_undegraded":"2024-09-13T17:19:03.412582+0000","last_fullsized":"2024-09-13T17:19:03.412582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3637","last_scrub_stamp":"2024-09-13T17:19:03.412530+0000","last_deep_scrub":"63'3637","last_deep_scrub_stamp":"2024-09-13T17:19:03.412530+0000","last_clean_scrub_stamp":"2024-09-13T17:19:03.412530+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:32.015420+0000","scrub_duration":0.030553734999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19635,"num_write_kb":259018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'5067","reported_seq":5140,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:59.638860+0000","last_change":"2024-09-13T17:18:59.639093+0000","last_active":"2024-09-13T17:18:59.638860+0000","last_peered":"2024-09-13T17:18:59.638860+0000","last_clean":"2024-09-13T17:18:59.638860+0000","last_became_active":"2024-09-13T16:48:40.831696+0000","last_became_peered":"2024-09-13T16:48:40.831696+0000","last_unstale":"2024-09-13T17:18:59.638860+0000","last_undegraded":"2024-09-13T17:18:59.638860+0000","last_fullsized":"2024-09-13T17:18:59.638860+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5067","last_scrub_stamp":"2024-09-13T17:18:59.638799+0000","last_deep_scrub":"63'5067","last_deep_scrub_stamp":"2024-09-13T17:18:59.638799+0000","last_clean_scrub_stamp":"2024-09-13T17:18:59.638799+0000","objects_scrubbed":0,"log_size":5067,"log_dups_size":0,"ondisk_log_size":5067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:32.333381+0000","scrub_duration":0.031396859999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27028,"num_write_kb":362438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"64'8","reported_seq":126,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.556099+0000","last_change":"2024-09-13T17:18:02.556314+0000","last_active":"2024-09-13T17:18:02.556099+0000","last_peered":"2024-09-13T17:18:02.556099+0000","last_clean":"2024-09-13T17:18:02.556099+0000","last_became_active":"2024-09-13T16:47:58.661928+0000","last_became_peered":"2024-09-13T16:47:58.661928+0000","last_unstale":"2024-09-13T17:18:02.556099+0000","last_undegraded":"2024-09-13T17:18:02.556099+0000","last_fullsized":"2024-09-13T17:18:02.556099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_deep_scrub":"64'8","last_deep_scrub_stamp":"2024-09-13T17:18:02.556044+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.556044+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:09:52.052485+0000","scrub_duration":0.0060730330000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"63'5","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:16.546615+0000","last_change":"2024-09-13T17:18:16.546615+0000","last_active":"2024-09-13T17:18:16.546615+0000","last_peered":"2024-09-13T17:18:16.546615+0000","last_clean":"2024-09-13T17:18:16.546615+0000","last_became_active":"2024-09-13T16:48:07.790261+0000","last_became_peered":"2024-09-13T16:48:07.790261+0000","last_unstale":"2024-09-13T17:18:16.546615+0000","last_undegraded":"2024-09-13T17:18:16.546615+0000","last_fullsized":"2024-09-13T17:18:16.546615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:16.546281+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:16.546281+0000","last_clean_scrub_stamp":"2024-09-13T17:18:16.546281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:15.818163+0000","scrub_duration":0.0035841639999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"63'2915","reported_seq":2991,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.588368+0000","last_change":"2024-09-13T17:18:29.588368+0000","last_active":"2024-09-13T17:18:29.588368+0000","last_peered":"2024-09-13T17:18:29.588368+0000","last_clean":"2024-09-13T17:18:29.588368+0000","last_became_active":"2024-09-13T16:48:08.168696+0000","last_became_peered":"2024-09-13T16:48:08.168696+0000","last_unstale":"2024-09-13T17:18:29.588368+0000","last_undegraded":"2024-09-13T17:18:29.588368+0000","last_fullsized":"2024-09-13T17:18:29.588368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2915","last_scrub_stamp":"2024-09-13T17:18:29.588063+0000","last_deep_scrub":"63'2915","last_deep_scrub_stamp":"2024-09-13T17:18:29.588063+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.588063+0000","objects_scrubbed":0,"log_size":2915,"log_dups_size":0,"ondisk_log_size":2915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:14.651101+0000","scrub_duration":0.01592476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16596,"num_write_kb":234494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"63'3697","reported_seq":3776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.226187+0000","last_change":"2024-09-13T17:18:29.226417+0000","last_active":"2024-09-13T17:18:29.226187+0000","last_peered":"2024-09-13T17:18:29.226187+0000","last_clean":"2024-09-13T17:18:29.226187+0000","last_became_active":"2024-09-13T16:47:59.056819+0000","last_became_peered":"2024-09-13T16:47:59.056819+0000","last_unstale":"2024-09-13T17:18:29.226187+0000","last_undegraded":"2024-09-13T17:18:29.226187+0000","last_fullsized":"2024-09-13T17:18:29.226187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3697","last_scrub_stamp":"2024-09-13T17:18:29.226132+0000","last_deep_scrub":"63'3697","last_deep_scrub_stamp":"2024-09-13T17:18:29.226132+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.226132+0000","objects_scrubbed":0,"log_size":3697,"log_dups_size":0,"ondisk_log_size":3697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:46.199826+0000","scrub_duration":0.032222396,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20206,"num_write_kb":253140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"64'5","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:06.185518+0000","last_change":"2024-09-13T17:18:06.186368+0000","last_active":"2024-09-13T17:18:06.185518+0000","last_peered":"2024-09-13T17:18:06.185518+0000","last_clean":"2024-09-13T17:18:06.185518+0000","last_became_active":"2024-09-13T16:47:58.661369+0000","last_became_peered":"2024-09-13T16:47:58.661369+0000","last_unstale":"2024-09-13T17:18:06.185518+0000","last_undegraded":"2024-09-13T17:18:06.185518+0000","last_fullsized":"2024-09-13T17:18:06.185518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:06.185462+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:06.185462+0000","last_clean_scrub_stamp":"2024-09-13T17:18:06.185462+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:29.073557+0000","scrub_duration":0.0065555370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"63'2","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.158515+0000","last_change":"2024-09-13T17:18:02.158515+0000","last_active":"2024-09-13T17:18:02.158515+0000","last_peered":"2024-09-13T17:18:02.158515+0000","last_clean":"2024-09-13T17:18:02.158515+0000","last_became_active":"2024-09-13T16:48:07.772473+0000","last_became_peered":"2024-09-13T16:48:07.772473+0000","last_unstale":"2024-09-13T17:18:02.158515+0000","last_undegraded":"2024-09-13T17:18:02.158515+0000","last_fullsized":"2024-09-13T17:18:02.158515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:02.158216+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.158216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.118447+0000","scrub_duration":0.003800346,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"63'3522","reported_seq":3596,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:40.984683+0000","last_change":"2024-09-13T17:18:40.984936+0000","last_active":"2024-09-13T17:18:40.984683+0000","last_peered":"2024-09-13T17:18:40.984683+0000","last_clean":"2024-09-13T17:18:40.984683+0000","last_became_active":"2024-09-13T16:48:08.164955+0000","last_became_peered":"2024-09-13T16:48:08.164955+0000","last_unstale":"2024-09-13T17:18:40.984683+0000","last_undegraded":"2024-09-13T17:18:40.984683+0000","last_fullsized":"2024-09-13T17:18:40.984683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3522","last_scrub_stamp":"2024-09-13T17:18:40.984629+0000","last_deep_scrub":"63'3522","last_deep_scrub_stamp":"2024-09-13T17:18:40.984629+0000","last_clean_scrub_stamp":"2024-09-13T17:18:40.984629+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:51.529447+0000","scrub_duration":0.025504836999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18828,"num_write_kb":272277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"63'3874","reported_seq":3955,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.213453+0000","last_change":"2024-09-13T17:17:59.213453+0000","last_active":"2024-09-13T17:17:59.213453+0000","last_peered":"2024-09-13T17:17:59.213453+0000","last_clean":"2024-09-13T17:17:59.213453+0000","last_became_active":"2024-09-13T16:47:59.056962+0000","last_became_peered":"2024-09-13T16:47:59.056962+0000","last_unstale":"2024-09-13T17:17:59.213453+0000","last_undegraded":"2024-09-13T17:17:59.213453+0000","last_fullsized":"2024-09-13T17:17:59.213453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:17:59.213146+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.213146+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.190638+0000","scrub_duration":0.033789,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21413,"num_write_kb":278431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"64'19","reported_seq":193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.533761+0000","last_change":"2024-09-13T17:17:59.533761+0000","last_active":"2024-09-13T17:17:59.533761+0000","last_peered":"2024-09-13T17:17:59.533761+0000","last_clean":"2024-09-13T17:17:59.533761+0000","last_became_active":"2024-09-13T16:47:58.657273+0000","last_became_peered":"2024-09-13T16:47:58.657273+0000","last_unstale":"2024-09-13T17:17:59.533761+0000","last_undegraded":"2024-09-13T17:17:59.533761+0000","last_fullsized":"2024-09-13T17:17:59.533761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_deep_scrub":"64'19","last_deep_scrub_stamp":"2024-09-13T17:17:59.533477+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.533477+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:42.670255+0000","scrub_duration":0.0092684789999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"63'3","reported_seq":78,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:00.848306+0000","last_change":"2024-09-13T17:18:00.848525+0000","last_active":"2024-09-13T17:18:00.848306+0000","last_peered":"2024-09-13T17:18:00.848306+0000","last_clean":"2024-09-13T17:18:00.848306+0000","last_became_active":"2024-09-13T16:48:07.779623+0000","last_became_peered":"2024-09-13T16:48:07.779623+0000","last_unstale":"2024-09-13T17:18:00.848306+0000","last_undegraded":"2024-09-13T17:18:00.848306+0000","last_fullsized":"2024-09-13T17:18:00.848306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:00.848253+0000","last_clean_scrub_stamp":"2024-09-13T17:18:00.848253+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:51.294142+0000","scrub_duration":0.0034249229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"63'3622","reported_seq":3699,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:14.509662+0000","last_change":"2024-09-13T17:19:14.509907+0000","last_active":"2024-09-13T17:19:14.509662+0000","last_peered":"2024-09-13T17:19:14.509662+0000","last_clean":"2024-09-13T17:19:14.509662+0000","last_became_active":"2024-09-13T16:48:08.171080+0000","last_became_peered":"2024-09-13T16:48:08.171080+0000","last_unstale":"2024-09-13T17:19:14.509662+0000","last_undegraded":"2024-09-13T17:19:14.509662+0000","last_fullsized":"2024-09-13T17:19:14.509662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3622","last_scrub_stamp":"2024-09-13T17:19:14.509608+0000","last_deep_scrub":"63'3622","last_deep_scrub_stamp":"2024-09-13T17:19:14.509608+0000","last_clean_scrub_stamp":"2024-09-13T17:19:14.509608+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:18.416684+0000","scrub_duration":0.032222925999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":16,"num_write":19996,"num_write_kb":293269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"63'3240","reported_seq":3319,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.494730+0000","last_change":"2024-09-13T17:18:10.494730+0000","last_active":"2024-09-13T17:18:10.494730+0000","last_peered":"2024-09-13T17:18:10.494730+0000","last_clean":"2024-09-13T17:18:10.494730+0000","last_became_active":"2024-09-13T16:47:59.047743+0000","last_became_peered":"2024-09-13T16:47:59.047743+0000","last_unstale":"2024-09-13T17:18:10.494730+0000","last_undegraded":"2024-09-13T17:18:10.494730+0000","last_fullsized":"2024-09-13T17:18:10.494730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3240","last_scrub_stamp":"2024-09-13T17:18:10.494430+0000","last_deep_scrub":"63'3240","last_deep_scrub_stamp":"2024-09-13T17:18:10.494430+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.494430+0000","objects_scrubbed":0,"log_size":3240,"log_dups_size":0,"ondisk_log_size":3240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:03.000138+0000","scrub_duration":0.014460961,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17303,"num_write_kb":237245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"63'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:02.449798+0000","last_change":"2024-09-13T17:18:02.449798+0000","last_active":"2024-09-13T17:18:02.449798+0000","last_peered":"2024-09-13T17:18:02.449798+0000","last_clean":"2024-09-13T17:18:02.449798+0000","last_became_active":"2024-09-13T16:47:58.686232+0000","last_became_peered":"2024-09-13T16:47:58.686232+0000","last_unstale":"2024-09-13T17:18:02.449798+0000","last_undegraded":"2024-09-13T17:18:02.449798+0000","last_fullsized":"2024-09-13T17:18:02.449798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:18:02.449537+0000","last_clean_scrub_stamp":"2024-09-13T17:18:02.449537+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:02.409768+0000","scrub_duration":0.0032633279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"63'5","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.060756+0000","last_change":"2024-09-13T17:17:51.060978+0000","last_active":"2024-09-13T17:17:51.060756+0000","last_peered":"2024-09-13T17:17:51.060756+0000","last_clean":"2024-09-13T17:17:51.060756+0000","last_became_active":"2024-09-13T16:48:07.788333+0000","last_became_peered":"2024-09-13T16:48:07.788333+0000","last_unstale":"2024-09-13T17:17:51.060756+0000","last_undegraded":"2024-09-13T17:17:51.060756+0000","last_fullsized":"2024-09-13T17:17:51.060756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:51.060702+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.060702+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:45.719305+0000","scrub_duration":0.0041164770000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"63'3376","reported_seq":3450,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:25.695117+0000","last_change":"2024-09-13T17:19:25.695335+0000","last_active":"2024-09-13T17:19:25.695117+0000","last_peered":"2024-09-13T17:19:25.695117+0000","last_clean":"2024-09-13T17:19:25.695117+0000","last_became_active":"2024-09-13T16:48:08.173074+0000","last_became_peered":"2024-09-13T16:48:08.173074+0000","last_unstale":"2024-09-13T17:19:25.695117+0000","last_undegraded":"2024-09-13T17:19:25.695117+0000","last_fullsized":"2024-09-13T17:19:25.695117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3376","last_scrub_stamp":"2024-09-13T17:19:25.695065+0000","last_deep_scrub":"63'3376","last_deep_scrub_stamp":"2024-09-13T17:19:25.695065+0000","last_clean_scrub_stamp":"2024-09-13T17:19:25.695065+0000","objects_scrubbed":0,"log_size":3376,"log_dups_size":0,"ondisk_log_size":3376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:01.898683+0000","scrub_duration":0.025235779,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18879,"num_write_kb":251906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"63'4848","reported_seq":4927,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:41.225569+0000","last_change":"2024-09-13T17:18:41.225810+0000","last_active":"2024-09-13T17:18:41.225569+0000","last_peered":"2024-09-13T17:18:41.225569+0000","last_clean":"2024-09-13T17:18:41.225569+0000","last_became_active":"2024-09-13T16:47:59.057593+0000","last_became_peered":"2024-09-13T16:47:59.057593+0000","last_unstale":"2024-09-13T17:18:41.225569+0000","last_undegraded":"2024-09-13T17:18:41.225569+0000","last_fullsized":"2024-09-13T17:18:41.225569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4848","last_scrub_stamp":"2024-09-13T17:18:41.225514+0000","last_deep_scrub":"63'4848","last_deep_scrub_stamp":"2024-09-13T17:18:41.225514+0000","last_clean_scrub_stamp":"2024-09-13T17:18:41.225514+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:51.770332+0000","scrub_duration":0.049129424999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26963,"num_write_kb":321221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"63'3","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:02.632405+0000","last_change":"2024-09-13T17:19:02.632627+0000","last_active":"2024-09-13T17:19:02.632405+0000","last_peered":"2024-09-13T17:19:02.632405+0000","last_clean":"2024-09-13T17:19:02.632405+0000","last_became_active":"2024-09-13T16:47:58.677781+0000","last_became_peered":"2024-09-13T16:47:58.677781+0000","last_unstale":"2024-09-13T17:19:02.632405+0000","last_undegraded":"2024-09-13T17:19:02.632405+0000","last_fullsized":"2024-09-13T17:19:02.632405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:19:02.632354+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:19:02.632354+0000","last_clean_scrub_stamp":"2024-09-13T17:19:02.632354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:44.165675+0000","scrub_duration":0.0032785549999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'392","reported_seq":575,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.116420+0000","last_change":"2024-09-13T17:17:49.116420+0000","last_active":"2024-09-13T17:17:49.116420+0000","last_peered":"2024-09-13T17:17:49.116420+0000","last_clean":"2024-09-13T17:17:49.116420+0000","last_became_active":"2024-09-13T16:48:07.777628+0000","last_became_peered":"2024-09-13T16:48:07.777628+0000","last_unstale":"2024-09-13T17:17:49.116420+0000","last_undegraded":"2024-09-13T17:17:49.116420+0000","last_fullsized":"2024-09-13T17:17:49.116420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'392","last_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_deep_scrub":"64'392","last_deep_scrub_stamp":"2024-09-13T17:17:49.116249+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.116249+0000","objects_scrubbed":4,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:49.076480+0000","scrub_duration":0.026108916999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":191,"num_read_kb":82094,"num_write":385,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"63'4856","reported_seq":4933,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:16.980231+0000","last_change":"2024-09-13T17:19:16.980510+0000","last_active":"2024-09-13T17:19:16.980231+0000","last_peered":"2024-09-13T17:19:16.980231+0000","last_clean":"2024-09-13T17:19:16.980231+0000","last_became_active":"2024-09-13T16:48:08.165440+0000","last_became_peered":"2024-09-13T16:48:08.165440+0000","last_unstale":"2024-09-13T17:19:16.980231+0000","last_undegraded":"2024-09-13T17:19:16.980231+0000","last_fullsized":"2024-09-13T17:19:16.980231+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4856","last_scrub_stamp":"2024-09-13T17:19:16.980173+0000","last_deep_scrub":"63'4856","last_deep_scrub_stamp":"2024-09-13T17:19:16.980173+0000","last_clean_scrub_stamp":"2024-09-13T17:19:16.980173+0000","objects_scrubbed":0,"log_size":4856,"log_dups_size":0,"ondisk_log_size":4856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:42.486741+0000","scrub_duration":0.056636632999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":26453,"num_write_kb":368210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'4063","reported_seq":4142,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:24.994002+0000","last_change":"2024-09-13T17:18:24.994290+0000","last_active":"2024-09-13T17:18:24.994002+0000","last_peered":"2024-09-13T17:18:24.994002+0000","last_clean":"2024-09-13T17:18:24.994002+0000","last_became_active":"2024-09-13T16:47:59.053730+0000","last_became_peered":"2024-09-13T16:47:59.053730+0000","last_unstale":"2024-09-13T17:18:24.994002+0000","last_undegraded":"2024-09-13T17:18:24.994002+0000","last_fullsized":"2024-09-13T17:18:24.994002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4063","last_scrub_stamp":"2024-09-13T17:18:24.993948+0000","last_deep_scrub":"63'4063","last_deep_scrub_stamp":"2024-09-13T17:18:24.993948+0000","last_clean_scrub_stamp":"2024-09-13T17:18:24.993948+0000","objects_scrubbed":0,"log_size":4063,"log_dups_size":0,"ondisk_log_size":4063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:17.499656+0000","scrub_duration":0.029339008999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21999,"num_write_kb":293018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"64'11","reported_seq":129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:07.796310+0000","last_change":"2024-09-13T17:18:07.796310+0000","last_active":"2024-09-13T17:18:07.796310+0000","last_peered":"2024-09-13T17:18:07.796310+0000","last_clean":"2024-09-13T17:18:07.796310+0000","last_became_active":"2024-09-13T16:47:58.661082+0000","last_became_peered":"2024-09-13T16:47:58.661082+0000","last_unstale":"2024-09-13T17:18:07.796310+0000","last_undegraded":"2024-09-13T17:18:07.796310+0000","last_fullsized":"2024-09-13T17:18:07.796310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-09-13T17:18:07.796034+0000","last_deep_scrub":"64'11","last_deep_scrub_stamp":"2024-09-13T17:18:07.796034+0000","last_clean_scrub_stamp":"2024-09-13T17:18:07.796034+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:49:50.932812+0000","scrub_duration":0.0076063959999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":152,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'32","reported_seq":171,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.557788+0000","last_change":"2024-09-13T17:17:59.557788+0000","last_active":"2024-09-13T17:17:59.557788+0000","last_peered":"2024-09-13T17:17:59.557788+0000","last_clean":"2024-09-13T17:17:59.557788+0000","last_became_active":"2024-09-13T16:48:07.785294+0000","last_became_peered":"2024-09-13T16:48:07.785294+0000","last_unstale":"2024-09-13T17:17:59.557788+0000","last_undegraded":"2024-09-13T17:17:59.557788+0000","last_fullsized":"2024-09-13T17:17:59.557788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'32","last_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_deep_scrub":"64'32","last_deep_scrub_stamp":"2024-09-13T17:17:59.557461+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.557461+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.534953+0000","scrub_duration":0.0076444640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":31,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.13","version":"63'4239","reported_seq":4314,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:24.978834+0000","last_change":"2024-09-13T17:19:24.979072+0000","last_active":"2024-09-13T17:19:24.978834+0000","last_peered":"2024-09-13T17:19:24.978834+0000","last_clean":"2024-09-13T17:19:24.978834+0000","last_became_active":"2024-09-13T16:48:08.171243+0000","last_became_peered":"2024-09-13T16:48:08.171243+0000","last_unstale":"2024-09-13T17:19:24.978834+0000","last_undegraded":"2024-09-13T17:19:24.978834+0000","last_fullsized":"2024-09-13T17:19:24.978834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4239","last_scrub_stamp":"2024-09-13T17:19:24.978779+0000","last_deep_scrub":"63'4239","last_deep_scrub_stamp":"2024-09-13T17:19:24.978779+0000","last_clean_scrub_stamp":"2024-09-13T17:19:24.978779+0000","objects_scrubbed":0,"log_size":4239,"log_dups_size":0,"ondisk_log_size":4239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:01.182397+0000","scrub_duration":0.033318397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23316,"num_write_kb":318961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'3718","reported_seq":3797,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.214254+0000","last_change":"2024-09-13T17:17:57.214254+0000","last_active":"2024-09-13T17:17:57.214254+0000","last_peered":"2024-09-13T17:17:57.214254+0000","last_clean":"2024-09-13T17:17:57.214254+0000","last_became_active":"2024-09-13T16:47:59.060974+0000","last_became_peered":"2024-09-13T16:47:59.060974+0000","last_unstale":"2024-09-13T17:17:57.214254+0000","last_undegraded":"2024-09-13T17:17:57.214254+0000","last_fullsized":"2024-09-13T17:17:57.214254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3718","last_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_deep_scrub":"63'3718","last_deep_scrub_stamp":"2024-09-13T17:17:57.213958+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.213958+0000","objects_scrubbed":0,"log_size":3718,"log_dups_size":0,"ondisk_log_size":3718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.719666+0000","scrub_duration":0.032519578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20956,"num_write_kb":260218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"64'395","reported_seq":570,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.567435+0000","last_change":"2024-09-13T17:18:04.567435+0000","last_active":"2024-09-13T17:18:04.567435+0000","last_peered":"2024-09-13T17:18:04.567435+0000","last_clean":"2024-09-13T17:18:04.567435+0000","last_became_active":"2024-09-13T16:47:58.687116+0000","last_became_peered":"2024-09-13T16:47:58.687116+0000","last_unstale":"2024-09-13T17:18:04.567435+0000","last_undegraded":"2024-09-13T17:18:04.567435+0000","last_fullsized":"2024-09-13T17:18:04.567435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'395","last_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_deep_scrub":"64'395","last_deep_scrub_stamp":"2024-09-13T17:18:04.567179+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.567179+0000","objects_scrubbed":4,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:59.225783+0000","scrub_duration":0.027913209000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":171,"num_read_kb":77976,"num_write":389,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"63'3","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:33.246155+0000","last_change":"2024-09-13T17:18:33.246380+0000","last_active":"2024-09-13T17:18:33.246155+0000","last_peered":"2024-09-13T17:18:33.246155+0000","last_clean":"2024-09-13T17:18:33.246155+0000","last_became_active":"2024-09-13T16:48:07.785168+0000","last_became_peered":"2024-09-13T16:48:07.785168+0000","last_unstale":"2024-09-13T17:18:33.246155+0000","last_undegraded":"2024-09-13T17:18:33.246155+0000","last_fullsized":"2024-09-13T17:18:33.246155+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:33.246101+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:33.246101+0000","last_clean_scrub_stamp":"2024-09-13T17:18:33.246101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:32.671487+0000","scrub_duration":0.0034645399999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"63'3522","reported_seq":3598,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:11.288817+0000","last_change":"2024-09-13T17:19:11.288965+0000","last_active":"2024-09-13T17:19:11.288817+0000","last_peered":"2024-09-13T17:19:11.288817+0000","last_clean":"2024-09-13T17:19:11.288817+0000","last_became_active":"2024-09-13T16:48:08.173494+0000","last_became_peered":"2024-09-13T16:48:08.173494+0000","last_unstale":"2024-09-13T17:19:11.288817+0000","last_undegraded":"2024-09-13T17:19:11.288817+0000","last_fullsized":"2024-09-13T17:19:11.288817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3522","last_scrub_stamp":"2024-09-13T17:19:11.288777+0000","last_deep_scrub":"63'3522","last_deep_scrub_stamp":"2024-09-13T17:19:11.288777+0000","last_clean_scrub_stamp":"2024-09-13T17:19:11.288777+0000","objects_scrubbed":0,"log_size":3522,"log_dups_size":0,"ondisk_log_size":3522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:52.822097+0000","scrub_duration":0.021914875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":4,"num_write":19373,"num_write_kb":266222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"63'5138","reported_seq":5217,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:33.196310+0000","last_change":"2024-09-13T17:18:33.196535+0000","last_active":"2024-09-13T17:18:33.196310+0000","last_peered":"2024-09-13T17:18:33.196310+0000","last_clean":"2024-09-13T17:18:33.196310+0000","last_became_active":"2024-09-13T16:47:59.048518+0000","last_became_peered":"2024-09-13T16:47:59.048518+0000","last_unstale":"2024-09-13T17:18:33.196310+0000","last_undegraded":"2024-09-13T17:18:33.196310+0000","last_fullsized":"2024-09-13T17:18:33.196310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5138","last_scrub_stamp":"2024-09-13T17:18:33.196260+0000","last_deep_scrub":"63'5138","last_deep_scrub_stamp":"2024-09-13T17:18:33.196260+0000","last_clean_scrub_stamp":"2024-09-13T17:18:33.196260+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:32.621647+0000","scrub_duration":0.048334911000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27863,"num_write_kb":353454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"64'27","reported_seq":145,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:57.282500+0000","last_change":"2024-09-13T17:18:57.282500+0000","last_active":"2024-09-13T17:18:57.282500+0000","last_peered":"2024-09-13T17:18:57.282500+0000","last_clean":"2024-09-13T17:18:57.282500+0000","last_became_active":"2024-09-13T16:47:58.678071+0000","last_became_peered":"2024-09-13T16:47:58.678071+0000","last_unstale":"2024-09-13T17:18:57.282500+0000","last_undegraded":"2024-09-13T17:18:57.282500+0000","last_fullsized":"2024-09-13T17:18:57.282500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-09-13T17:18:57.282214+0000","last_deep_scrub":"64'27","last_deep_scrub_stamp":"2024-09-13T17:18:57.282214+0000","last_clean_scrub_stamp":"2024-09-13T17:18:57.282214+0000","objects_scrubbed":2,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:37.708299+0000","scrub_duration":0.0070313399999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":26,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'9","reported_seq":127,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.154830+0000","last_change":"2024-09-13T17:18:03.154830+0000","last_active":"2024-09-13T17:18:03.154830+0000","last_peered":"2024-09-13T17:18:03.154830+0000","last_clean":"2024-09-13T17:18:03.154830+0000","last_became_active":"2024-09-13T16:48:07.775552+0000","last_became_peered":"2024-09-13T16:48:07.775552+0000","last_unstale":"2024-09-13T17:18:03.154830+0000","last_undegraded":"2024-09-13T17:18:03.154830+0000","last_fullsized":"2024-09-13T17:18:03.154830+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_deep_scrub":"64'9","last_deep_scrub_stamp":"2024-09-13T17:18:03.154514+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.154514+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:05.788731+0000","scrub_duration":0.0080602239999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":168,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"63'3602","reported_seq":3680,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:46.062966+0000","last_change":"2024-09-13T17:18:46.063201+0000","last_active":"2024-09-13T17:18:46.062966+0000","last_peered":"2024-09-13T17:18:46.062966+0000","last_clean":"2024-09-13T17:18:46.062966+0000","last_became_active":"2024-09-13T16:48:08.200691+0000","last_became_peered":"2024-09-13T16:48:08.200691+0000","last_unstale":"2024-09-13T17:18:46.062966+0000","last_undegraded":"2024-09-13T17:18:46.062966+0000","last_fullsized":"2024-09-13T17:18:46.062966+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3602","last_scrub_stamp":"2024-09-13T17:18:46.062913+0000","last_deep_scrub":"63'3602","last_deep_scrub_stamp":"2024-09-13T17:18:46.062913+0000","last_clean_scrub_stamp":"2024-09-13T17:18:46.062913+0000","objects_scrubbed":0,"log_size":3602,"log_dups_size":0,"ondisk_log_size":3602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:43.137455+0000","scrub_duration":0.025752721999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":20290,"num_write_kb":278255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"63'3743","reported_seq":3822,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.841262+0000","last_change":"2024-09-13T17:17:57.841262+0000","last_active":"2024-09-13T17:17:57.841262+0000","last_peered":"2024-09-13T17:17:57.841262+0000","last_clean":"2024-09-13T17:17:57.841262+0000","last_became_active":"2024-09-13T16:47:59.054472+0000","last_became_peered":"2024-09-13T16:47:59.054472+0000","last_unstale":"2024-09-13T17:17:57.841262+0000","last_undegraded":"2024-09-13T17:17:57.841262+0000","last_fullsized":"2024-09-13T17:17:57.841262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3743","last_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_deep_scrub":"63'3743","last_deep_scrub_stamp":"2024-09-13T17:17:57.840983+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.840983+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:01.952379+0000","scrub_duration":0.026384685000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20404,"num_write_kb":278673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"63'3","reported_seq":83,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:01.646372+0000","last_change":"2024-09-13T17:19:01.646372+0000","last_active":"2024-09-13T17:19:01.646372+0000","last_peered":"2024-09-13T17:19:01.646372+0000","last_clean":"2024-09-13T17:19:01.646372+0000","last_became_active":"2024-09-13T16:47:58.690275+0000","last_became_peered":"2024-09-13T16:47:58.690275+0000","last_unstale":"2024-09-13T17:19:01.646372+0000","last_undegraded":"2024-09-13T17:19:01.646372+0000","last_fullsized":"2024-09-13T17:19:01.646372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:19:01.646081+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:19:01.646081+0000","last_clean_scrub_stamp":"2024-09-13T17:19:01.646081+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:02.753337+0000","scrub_duration":0.0033163099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'19","reported_seq":203,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:20.179269+0000","last_change":"2024-09-13T17:18:20.179269+0000","last_active":"2024-09-13T17:18:20.179269+0000","last_peered":"2024-09-13T17:18:20.179269+0000","last_clean":"2024-09-13T17:18:20.179269+0000","last_became_active":"2024-09-13T16:48:07.773996+0000","last_became_peered":"2024-09-13T16:48:07.773996+0000","last_unstale":"2024-09-13T17:18:20.179269+0000","last_undegraded":"2024-09-13T17:18:20.179269+0000","last_fullsized":"2024-09-13T17:18:20.179269+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-09-13T17:18:20.178917+0000","last_deep_scrub":"64'19","last_deep_scrub_stamp":"2024-09-13T17:18:20.178917+0000","last_clean_scrub_stamp":"2024-09-13T17:18:20.178917+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:19.604304+0000","scrub_duration":0.010053862,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":324,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"63'3189","reported_seq":3264,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:12.446440+0000","last_change":"2024-09-13T17:19:12.446768+0000","last_active":"2024-09-13T17:19:12.446440+0000","last_peered":"2024-09-13T17:19:12.446440+0000","last_clean":"2024-09-13T17:19:12.446440+0000","last_became_active":"2024-09-13T16:48:08.202189+0000","last_became_peered":"2024-09-13T16:48:08.202189+0000","last_unstale":"2024-09-13T17:19:12.446440+0000","last_undegraded":"2024-09-13T17:19:12.446440+0000","last_fullsized":"2024-09-13T17:19:12.446440+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3189","last_scrub_stamp":"2024-09-13T17:19:12.446385+0000","last_deep_scrub":"63'3189","last_deep_scrub_stamp":"2024-09-13T17:19:12.446385+0000","last_clean_scrub_stamp":"2024-09-13T17:19:12.446385+0000","objects_scrubbed":0,"log_size":3189,"log_dups_size":0,"ondisk_log_size":3189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:16.353461+0000","scrub_duration":0.031394140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17342,"num_write_kb":249797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"63'4223","reported_seq":4302,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:06.855194+0000","last_change":"2024-09-13T17:19:06.855194+0000","last_active":"2024-09-13T17:19:06.855194+0000","last_peered":"2024-09-13T17:19:06.855194+0000","last_clean":"2024-09-13T17:19:06.855194+0000","last_became_active":"2024-09-13T16:47:59.046388+0000","last_became_peered":"2024-09-13T16:47:59.046388+0000","last_unstale":"2024-09-13T17:19:06.855194+0000","last_undegraded":"2024-09-13T17:19:06.855194+0000","last_fullsized":"2024-09-13T17:19:06.855194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4223","last_scrub_stamp":"2024-09-13T17:19:06.854904+0000","last_deep_scrub":"63'4223","last_deep_scrub_stamp":"2024-09-13T17:19:06.854904+0000","last_clean_scrub_stamp":"2024-09-13T17:19:06.854904+0000","objects_scrubbed":0,"log_size":4223,"log_dups_size":0,"ondisk_log_size":4223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:02.698329+0000","scrub_duration":0.242061791,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22517,"num_write_kb":295025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"64'5","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:06.450600+0000","last_change":"2024-09-13T17:18:06.450600+0000","last_active":"2024-09-13T17:18:06.450600+0000","last_peered":"2024-09-13T17:18:06.450600+0000","last_clean":"2024-09-13T17:18:06.450600+0000","last_became_active":"2024-09-13T16:47:58.685500+0000","last_became_peered":"2024-09-13T16:47:58.685500+0000","last_unstale":"2024-09-13T17:18:06.450600+0000","last_undegraded":"2024-09-13T17:18:06.450600+0000","last_fullsized":"2024-09-13T17:18:06.450600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'5","last_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_deep_scrub":"64'5","last_deep_scrub_stamp":"2024-09-13T17:18:06.450341+0000","last_clean_scrub_stamp":"2024-09-13T17:18:06.450341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:29.338436+0000","scrub_duration":0.003290758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"63'5","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.123757+0000","last_change":"2024-09-13T17:17:53.123757+0000","last_active":"2024-09-13T17:17:53.123757+0000","last_peered":"2024-09-13T17:17:53.123757+0000","last_clean":"2024-09-13T17:17:53.123757+0000","last_became_active":"2024-09-13T16:48:07.788505+0000","last_became_peered":"2024-09-13T16:48:07.788505+0000","last_unstale":"2024-09-13T17:17:53.123757+0000","last_undegraded":"2024-09-13T17:17:53.123757+0000","last_fullsized":"2024-09-13T17:17:53.123757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5","last_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_deep_scrub":"63'5","last_deep_scrub_stamp":"2024-09-13T17:17:53.123466+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.123466+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.011562+0000","scrub_duration":0.0038056650000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"63'5357","reported_seq":5437,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.115905+0000","last_change":"2024-09-13T17:18:43.115905+0000","last_active":"2024-09-13T17:18:43.115905+0000","last_peered":"2024-09-13T17:18:43.115905+0000","last_clean":"2024-09-13T17:18:43.115905+0000","last_became_active":"2024-09-13T16:48:08.194794+0000","last_became_peered":"2024-09-13T16:48:08.194794+0000","last_unstale":"2024-09-13T17:18:43.115905+0000","last_undegraded":"2024-09-13T17:18:43.115905+0000","last_fullsized":"2024-09-13T17:18:43.115905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5357","last_scrub_stamp":"2024-09-13T17:18:43.115582+0000","last_deep_scrub":"63'5357","last_deep_scrub_stamp":"2024-09-13T17:18:43.115582+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.115582+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:20.465692+0000","scrub_duration":0.033536497999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":39,"num_write":29637,"num_write_kb":393569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"63'26","reported_seq":104,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:08.229713+0000","last_change":"2024-09-13T17:19:08.229943+0000","last_active":"2024-09-13T17:19:08.229713+0000","last_peered":"2024-09-13T17:19:08.229713+0000","last_clean":"2024-09-13T17:19:08.229713+0000","last_became_active":"2024-09-13T16:48:07.785151+0000","last_became_peered":"2024-09-13T16:48:07.785151+0000","last_unstale":"2024-09-13T17:19:08.229713+0000","last_undegraded":"2024-09-13T17:19:08.229713+0000","last_fullsized":"2024-09-13T17:19:08.229713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'26","last_scrub_stamp":"2024-09-13T17:19:08.229660+0000","last_deep_scrub":"63'26","last_deep_scrub_stamp":"2024-09-13T17:19:08.229660+0000","last_clean_scrub_stamp":"2024-09-13T17:19:08.229660+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:18.774478+0000","scrub_duration":0.0036406939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"63'5122","reported_seq":5200,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.190845+0000","last_change":"2024-09-13T17:18:01.191064+0000","last_active":"2024-09-13T17:18:01.190845+0000","last_peered":"2024-09-13T17:18:01.190845+0000","last_clean":"2024-09-13T17:18:01.190845+0000","last_became_active":"2024-09-13T16:47:59.053044+0000","last_became_peered":"2024-09-13T16:47:59.053044+0000","last_unstale":"2024-09-13T17:18:01.190845+0000","last_undegraded":"2024-09-13T17:18:01.190845+0000","last_fullsized":"2024-09-13T17:18:01.190845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5122","last_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_deep_scrub":"63'5122","last_deep_scrub_stamp":"2024-09-13T17:18:01.190787+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.190787+0000","objects_scrubbed":0,"log_size":5122,"log_dups_size":0,"ondisk_log_size":5122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:03.825004+0000","scrub_duration":0.032904946999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":348704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"63'2","reported_seq":81,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.623848+0000","last_change":"2024-09-13T17:18:45.624092+0000","last_active":"2024-09-13T17:18:45.623848+0000","last_peered":"2024-09-13T17:18:45.623848+0000","last_clean":"2024-09-13T17:18:45.623848+0000","last_became_active":"2024-09-13T16:47:58.656673+0000","last_became_peered":"2024-09-13T16:47:58.656673+0000","last_unstale":"2024-09-13T17:18:45.623848+0000","last_undegraded":"2024-09-13T17:18:45.623848+0000","last_fullsized":"2024-09-13T17:18:45.623848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-13T17:18:45.623791+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-13T17:18:45.623791+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.623791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:26.049876+0000","scrub_duration":0.003142663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1e","version":"63'3554","reported_seq":3631,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:17.324739+0000","last_change":"2024-09-13T17:19:17.325002+0000","last_active":"2024-09-13T17:19:17.324739+0000","last_peered":"2024-09-13T17:19:17.324739+0000","last_clean":"2024-09-13T17:19:17.324739+0000","last_became_active":"2024-09-13T16:48:08.186824+0000","last_became_peered":"2024-09-13T16:48:08.186824+0000","last_unstale":"2024-09-13T17:19:17.324739+0000","last_undegraded":"2024-09-13T17:19:17.324739+0000","last_fullsized":"2024-09-13T17:19:17.324739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3554","last_scrub_stamp":"2024-09-13T17:19:17.324685+0000","last_deep_scrub":"63'3554","last_deep_scrub_stamp":"2024-09-13T17:19:17.324685+0000","last_clean_scrub_stamp":"2024-09-13T17:19:17.324685+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:33.455230+0000","scrub_duration":0.027523696,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19915,"num_write_kb":268389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"64'1094","reported_seq":1222,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:29.174543+0000","last_change":"2024-09-13T17:18:29.174543+0000","last_active":"2024-09-13T17:18:29.174543+0000","last_peered":"2024-09-13T17:18:29.174543+0000","last_clean":"2024-09-13T17:18:29.174543+0000","last_became_active":"2024-09-13T16:48:07.786256+0000","last_became_peered":"2024-09-13T16:48:07.786256+0000","last_unstale":"2024-09-13T17:18:29.174543+0000","last_undegraded":"2024-09-13T17:18:29.174543+0000","last_fullsized":"2024-09-13T17:18:29.174543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1094","last_scrub_stamp":"2024-09-13T17:18:29.174267+0000","last_deep_scrub":"64'1094","last_deep_scrub_stamp":"2024-09-13T17:18:29.174267+0000","last_clean_scrub_stamp":"2024-09-13T17:18:29.174267+0000","objects_scrubbed":3,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:28.446149+0000","scrub_duration":0.031395304999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":9436,"num_write":1065,"num_write_kb":4730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"63'4714","reported_seq":4792,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:32.980839+0000","last_change":"2024-09-13T17:18:32.981101+0000","last_active":"2024-09-13T17:18:32.980839+0000","last_peered":"2024-09-13T17:18:32.980839+0000","last_clean":"2024-09-13T17:18:32.980839+0000","last_became_active":"2024-09-13T16:47:59.056898+0000","last_became_peered":"2024-09-13T16:47:59.056898+0000","last_unstale":"2024-09-13T17:18:32.980839+0000","last_undegraded":"2024-09-13T17:18:32.980839+0000","last_fullsized":"2024-09-13T17:18:32.980839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4714","last_scrub_stamp":"2024-09-13T17:18:32.980782+0000","last_deep_scrub":"63'4714","last_deep_scrub_stamp":"2024-09-13T17:18:32.980782+0000","last_clean_scrub_stamp":"2024-09-13T17:18:32.980782+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:43.117628+0000","scrub_duration":0.031918914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25459,"num_write_kb":343662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"63'3","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.965455+0000","last_change":"2024-09-13T17:18:43.965730+0000","last_active":"2024-09-13T17:18:43.965455+0000","last_peered":"2024-09-13T17:18:43.965455+0000","last_clean":"2024-09-13T17:18:43.965455+0000","last_became_active":"2024-09-13T16:47:58.683030+0000","last_became_peered":"2024-09-13T16:47:58.683030+0000","last_unstale":"2024-09-13T17:18:43.965455+0000","last_undegraded":"2024-09-13T17:18:43.965455+0000","last_fullsized":"2024-09-13T17:18:43.965455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-13T17:18:43.965393+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-13T17:18:43.965393+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.965393+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:24.391478+0000","scrub_duration":0.0038411180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"63'4328","reported_seq":4402,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:22.354194+0000","last_change":"2024-09-13T17:19:22.354426+0000","last_active":"2024-09-13T17:19:22.354194+0000","last_peered":"2024-09-13T17:19:22.354194+0000","last_clean":"2024-09-13T17:19:22.354194+0000","last_became_active":"2024-09-13T16:48:08.204928+0000","last_became_peered":"2024-09-13T16:48:08.204928+0000","last_unstale":"2024-09-13T17:19:22.354194+0000","last_undegraded":"2024-09-13T17:19:22.354194+0000","last_fullsized":"2024-09-13T17:19:22.354194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4328","last_scrub_stamp":"2024-09-13T17:19:22.354138+0000","last_deep_scrub":"63'4328","last_deep_scrub_stamp":"2024-09-13T17:19:22.354138+0000","last_clean_scrub_stamp":"2024-09-13T17:19:22.354138+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:03.063098+0000","scrub_duration":0.035116857000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24456,"num_write_kb":315374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"64'411","reported_seq":494,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:03.515483+0000","last_change":"2024-09-13T17:18:03.515735+0000","last_active":"2024-09-13T17:18:03.515483+0000","last_peered":"2024-09-13T17:18:03.515483+0000","last_clean":"2024-09-13T17:18:03.515483+0000","last_became_active":"2024-09-13T16:47:58.685512+0000","last_became_peered":"2024-09-13T16:47:58.685512+0000","last_unstale":"2024-09-13T17:18:03.515483+0000","last_undegraded":"2024-09-13T17:18:03.515483+0000","last_fullsized":"2024-09-13T17:18:03.515483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'411","last_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_deep_scrub":"64'411","last_deep_scrub_stamp":"2024-09-13T17:18:03.515426+0000","last_clean_scrub_stamp":"2024-09-13T17:18:03.515426+0000","objects_scrubbed":1,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:06.149643+0000","scrub_duration":0.019312658999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2812346,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3190,"num_write":386,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"63'4155","reported_seq":4233,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:20.944870+0000","last_change":"2024-09-13T17:18:20.945097+0000","last_active":"2024-09-13T17:18:20.944870+0000","last_peered":"2024-09-13T17:18:20.944870+0000","last_clean":"2024-09-13T17:18:20.944870+0000","last_became_active":"2024-09-13T16:47:59.056792+0000","last_became_peered":"2024-09-13T16:47:59.056792+0000","last_unstale":"2024-09-13T17:18:20.944870+0000","last_undegraded":"2024-09-13T17:18:20.944870+0000","last_fullsized":"2024-09-13T17:18:20.944870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4155","last_scrub_stamp":"2024-09-13T17:18:20.944817+0000","last_deep_scrub":"63'4155","last_deep_scrub_stamp":"2024-09-13T17:18:20.944817+0000","last_clean_scrub_stamp":"2024-09-13T17:18:20.944817+0000","objects_scrubbed":0,"log_size":4155,"log_dups_size":0,"ondisk_log_size":4155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:43.832912+0000","scrub_duration":0.029833493999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22247,"num_write_kb":285891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"64'6","reported_seq":123,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.503784+0000","last_change":"2024-09-13T17:18:10.504013+0000","last_active":"2024-09-13T17:18:10.503784+0000","last_peered":"2024-09-13T17:18:10.503784+0000","last_clean":"2024-09-13T17:18:10.503784+0000","last_became_active":"2024-09-13T16:48:07.771611+0000","last_became_peered":"2024-09-13T16:48:07.771611+0000","last_unstale":"2024-09-13T17:18:10.503784+0000","last_undegraded":"2024-09-13T17:18:10.503784+0000","last_fullsized":"2024-09-13T17:18:10.503784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'6","last_scrub_stamp":"2024-09-13T17:18:10.503729+0000","last_deep_scrub":"64'6","last_deep_scrub_stamp":"2024-09-13T17:18:10.503729+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.503729+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:33.391824+0000","scrub_duration":0.0063226599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"63'3426","reported_seq":3503,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:57.209418+0000","last_change":"2024-09-13T17:18:57.209707+0000","last_active":"2024-09-13T17:18:57.209418+0000","last_peered":"2024-09-13T17:18:57.209418+0000","last_clean":"2024-09-13T17:18:57.209418+0000","last_became_active":"2024-09-13T16:48:08.200333+0000","last_became_peered":"2024-09-13T16:48:08.200333+0000","last_unstale":"2024-09-13T17:18:57.209418+0000","last_undegraded":"2024-09-13T17:18:57.209418+0000","last_fullsized":"2024-09-13T17:18:57.209418+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3426","last_scrub_stamp":"2024-09-13T17:18:57.209365+0000","last_deep_scrub":"63'3426","last_deep_scrub_stamp":"2024-09-13T17:18:57.209365+0000","last_clean_scrub_stamp":"2024-09-13T17:18:57.209365+0000","objects_scrubbed":0,"log_size":3426,"log_dups_size":0,"ondisk_log_size":3426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:37.635450+0000","scrub_duration":0.029481411999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5,"num_write":18510,"num_write_kb":250514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"64'713","reported_seq":815,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:52.922610+0000","last_change":"2024-09-13T17:18:52.922907+0000","last_active":"2024-09-13T17:18:52.922610+0000","last_peered":"2024-09-13T17:18:52.922610+0000","last_clean":"2024-09-13T17:18:52.922610+0000","last_became_active":"2024-09-13T16:47:58.674177+0000","last_became_peered":"2024-09-13T16:47:58.674177+0000","last_unstale":"2024-09-13T17:18:52.922610+0000","last_undegraded":"2024-09-13T17:18:52.922610+0000","last_fullsized":"2024-09-13T17:18:52.922610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'713","last_scrub_stamp":"2024-09-13T17:18:52.922549+0000","last_deep_scrub":"64'713","last_deep_scrub_stamp":"2024-09-13T17:18:52.922549+0000","last_clean_scrub_stamp":"2024-09-13T17:18:52.922549+0000","objects_scrubbed":1,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:28.967244+0000","scrub_duration":0.025846372999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":74674,"num_write":702,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"63'4316","reported_seq":4395,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:10.250446+0000","last_change":"2024-09-13T17:18:10.250685+0000","last_active":"2024-09-13T17:18:10.250446+0000","last_peered":"2024-09-13T17:18:10.250446+0000","last_clean":"2024-09-13T17:18:10.250446+0000","last_became_active":"2024-09-13T16:47:59.037721+0000","last_became_peered":"2024-09-13T16:47:59.037721+0000","last_unstale":"2024-09-13T17:18:10.250446+0000","last_undegraded":"2024-09-13T17:18:10.250446+0000","last_fullsized":"2024-09-13T17:18:10.250446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4316","last_scrub_stamp":"2024-09-13T17:18:10.250390+0000","last_deep_scrub":"63'4316","last_deep_scrub_stamp":"2024-09-13T17:18:10.250390+0000","last_clean_scrub_stamp":"2024-09-13T17:18:10.250390+0000","objects_scrubbed":0,"log_size":4316,"log_dups_size":0,"ondisk_log_size":4316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:02.756098+0000","scrub_duration":0.033652449000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22350,"num_write_kb":290228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"64'45","reported_seq":162,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:59.128882+0000","last_change":"2024-09-13T17:17:59.128882+0000","last_active":"2024-09-13T17:17:59.128882+0000","last_peered":"2024-09-13T17:17:59.128882+0000","last_clean":"2024-09-13T17:17:59.128882+0000","last_became_active":"2024-09-13T16:48:07.782130+0000","last_became_peered":"2024-09-13T16:48:07.782130+0000","last_unstale":"2024-09-13T17:17:59.128882+0000","last_undegraded":"2024-09-13T17:17:59.128882+0000","last_fullsized":"2024-09-13T17:17:59.128882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'45","last_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_deep_scrub":"64'45","last_deep_scrub_stamp":"2024-09-13T17:17:59.128560+0000","last_clean_scrub_stamp":"2024-09-13T17:17:59.128560+0000","objects_scrubbed":1,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:12.106052+0000","scrub_duration":0.0062496970000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.26","version":"63'3129","reported_seq":3205,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:10.980559+0000","last_change":"2024-09-13T17:19:10.980805+0000","last_active":"2024-09-13T17:19:10.980559+0000","last_peered":"2024-09-13T17:19:10.980559+0000","last_clean":"2024-09-13T17:19:10.980559+0000","last_became_active":"2024-09-13T16:48:08.188042+0000","last_became_peered":"2024-09-13T16:48:08.188042+0000","last_unstale":"2024-09-13T17:19:10.980559+0000","last_undegraded":"2024-09-13T17:19:10.980559+0000","last_fullsized":"2024-09-13T17:19:10.980559+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3129","last_scrub_stamp":"2024-09-13T17:19:10.980506+0000","last_deep_scrub":"63'3129","last_deep_scrub_stamp":"2024-09-13T17:19:10.980506+0000","last_clean_scrub_stamp":"2024-09-13T17:19:10.980506+0000","objects_scrubbed":0,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:33.930171+0000","scrub_duration":0.028211038000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17279,"num_write_kb":257601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"63'4785","reported_seq":4866,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:49.565883+0000","last_change":"2024-09-13T17:17:49.565883+0000","last_active":"2024-09-13T17:17:49.565883+0000","last_peered":"2024-09-13T17:17:49.565883+0000","last_clean":"2024-09-13T17:17:49.565883+0000","last_became_active":"2024-09-13T16:47:59.047857+0000","last_became_peered":"2024-09-13T16:47:59.047857+0000","last_unstale":"2024-09-13T17:17:49.565883+0000","last_undegraded":"2024-09-13T17:17:49.565883+0000","last_fullsized":"2024-09-13T17:17:49.565883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4785","last_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_deep_scrub":"63'4785","last_deep_scrub_stamp":"2024-09-13T17:17:49.565599+0000","last_clean_scrub_stamp":"2024-09-13T17:17:49.565599+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.011489+0000","scrub_duration":0.012593717000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26302,"num_write_kb":310249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.27","version":"63'4231","reported_seq":4308,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:15.309579+0000","last_change":"2024-09-13T17:19:15.309819+0000","last_active":"2024-09-13T17:19:15.309579+0000","last_peered":"2024-09-13T17:19:15.309579+0000","last_clean":"2024-09-13T17:19:15.309579+0000","last_became_active":"2024-09-13T16:48:08.181245+0000","last_became_peered":"2024-09-13T16:48:08.181245+0000","last_unstale":"2024-09-13T17:19:15.309579+0000","last_undegraded":"2024-09-13T17:19:15.309579+0000","last_fullsized":"2024-09-13T17:19:15.309579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4231","last_scrub_stamp":"2024-09-13T17:19:15.309524+0000","last_deep_scrub":"63'4231","last_deep_scrub_stamp":"2024-09-13T17:19:15.309524+0000","last_clean_scrub_stamp":"2024-09-13T17:19:15.309524+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:43.912414+0000","scrub_duration":0.032821093000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22604,"num_write_kb":324065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"63'3874","reported_seq":3956,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:04.662513+0000","last_change":"2024-09-13T17:18:04.662513+0000","last_active":"2024-09-13T17:18:04.662513+0000","last_peered":"2024-09-13T17:18:04.662513+0000","last_clean":"2024-09-13T17:18:04.662513+0000","last_became_active":"2024-09-13T16:47:59.046165+0000","last_became_peered":"2024-09-13T16:47:59.046165+0000","last_unstale":"2024-09-13T17:18:04.662513+0000","last_undegraded":"2024-09-13T17:18:04.662513+0000","last_fullsized":"2024-09-13T17:18:04.662513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3874","last_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_deep_scrub":"63'3874","last_deep_scrub_stamp":"2024-09-13T17:18:04.662182+0000","last_clean_scrub_stamp":"2024-09-13T17:18:04.662182+0000","objects_scrubbed":0,"log_size":3874,"log_dups_size":0,"ondisk_log_size":3874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:07.296399+0000","scrub_duration":0.021443912999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20920,"num_write_kb":276387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"63'3865","reported_seq":3942,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:32.467133+0000","last_change":"2024-09-13T17:18:32.467133+0000","last_active":"2024-09-13T17:18:32.467133+0000","last_peered":"2024-09-13T17:18:32.467133+0000","last_clean":"2024-09-13T17:18:32.467133+0000","last_became_active":"2024-09-13T16:48:08.176613+0000","last_became_peered":"2024-09-13T16:48:08.176613+0000","last_unstale":"2024-09-13T17:18:32.467133+0000","last_undegraded":"2024-09-13T17:18:32.467133+0000","last_fullsized":"2024-09-13T17:18:32.467133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3865","last_scrub_stamp":"2024-09-13T17:18:32.466850+0000","last_deep_scrub":"63'3865","last_deep_scrub_stamp":"2024-09-13T17:18:32.466850+0000","last_clean_scrub_stamp":"2024-09-13T17:18:32.466850+0000","objects_scrubbed":0,"log_size":3865,"log_dups_size":0,"ondisk_log_size":3865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:09.816960+0000","scrub_duration":0.029037197000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21432,"num_write_kb":286796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"63'4552","reported_seq":4635,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.654868+0000","last_change":"2024-09-13T17:19:05.654868+0000","last_active":"2024-09-13T17:19:05.654868+0000","last_peered":"2024-09-13T17:19:05.654868+0000","last_clean":"2024-09-13T17:19:05.654868+0000","last_became_active":"2024-09-13T16:47:59.058948+0000","last_became_peered":"2024-09-13T16:47:59.058948+0000","last_unstale":"2024-09-13T17:19:05.654868+0000","last_undegraded":"2024-09-13T17:19:05.654868+0000","last_fullsized":"2024-09-13T17:19:05.654868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4552","last_scrub_stamp":"2024-09-13T17:19:05.654571+0000","last_deep_scrub":"63'4552","last_deep_scrub_stamp":"2024-09-13T17:19:05.654571+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.654571+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:34.257461+0000","scrub_duration":0.029954933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":24031,"num_write_kb":313421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.25","version":"63'4071","reported_seq":4148,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:23.352399+0000","last_change":"2024-09-13T17:19:23.352625+0000","last_active":"2024-09-13T17:19:23.352399+0000","last_peered":"2024-09-13T17:19:23.352399+0000","last_clean":"2024-09-13T17:19:23.352399+0000","last_became_active":"2024-09-13T16:48:08.205154+0000","last_became_peered":"2024-09-13T16:48:08.205154+0000","last_unstale":"2024-09-13T17:19:23.352399+0000","last_undegraded":"2024-09-13T17:19:23.352399+0000","last_fullsized":"2024-09-13T17:19:23.352399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4071","last_scrub_stamp":"2024-09-13T17:19:23.352340+0000","last_deep_scrub":"63'4071","last_deep_scrub_stamp":"2024-09-13T17:19:23.352340+0000","last_clean_scrub_stamp":"2024-09-13T17:19:23.352340+0000","objects_scrubbed":0,"log_size":4071,"log_dups_size":0,"ondisk_log_size":4071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:46.302005+0000","scrub_duration":0.032147508999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":22061,"num_write_kb":309650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"63'3506","reported_seq":3588,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:47.181238+0000","last_change":"2024-09-13T17:18:47.181238+0000","last_active":"2024-09-13T17:18:47.181238+0000","last_peered":"2024-09-13T17:18:47.181238+0000","last_clean":"2024-09-13T17:18:47.181238+0000","last_became_active":"2024-09-13T16:47:59.057352+0000","last_became_peered":"2024-09-13T16:47:59.057352+0000","last_unstale":"2024-09-13T17:18:47.181238+0000","last_undegraded":"2024-09-13T17:18:47.181238+0000","last_fullsized":"2024-09-13T17:18:47.181238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3506","last_scrub_stamp":"2024-09-13T17:18:47.180948+0000","last_deep_scrub":"63'3506","last_deep_scrub_stamp":"2024-09-13T17:18:47.180948+0000","last_clean_scrub_stamp":"2024-09-13T17:18:47.180948+0000","objects_scrubbed":0,"log_size":3506,"log_dups_size":0,"ondisk_log_size":3506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:46.606335+0000","scrub_duration":0.025510459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19493,"num_write_kb":254380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"63'4523","reported_seq":4600,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:09.440708+0000","last_change":"2024-09-13T17:19:09.440708+0000","last_active":"2024-09-13T17:19:09.440708+0000","last_peered":"2024-09-13T17:19:09.440708+0000","last_clean":"2024-09-13T17:19:09.440708+0000","last_became_active":"2024-09-13T16:48:08.185887+0000","last_became_peered":"2024-09-13T16:48:08.185887+0000","last_unstale":"2024-09-13T17:19:09.440708+0000","last_undegraded":"2024-09-13T17:19:09.440708+0000","last_fullsized":"2024-09-13T17:19:09.440708+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4523","last_scrub_stamp":"2024-09-13T17:19:09.439955+0000","last_deep_scrub":"63'4523","last_deep_scrub_stamp":"2024-09-13T17:19:09.439955+0000","last_clean_scrub_stamp":"2024-09-13T17:19:09.439955+0000","objects_scrubbed":0,"log_size":4523,"log_dups_size":0,"ondisk_log_size":4523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:50.148915+0000","scrub_duration":0.035549714000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24221,"num_write_kb":338849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.24","version":"63'4422","reported_seq":4503,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:00.945847+0000","last_change":"2024-09-13T17:19:00.946067+0000","last_active":"2024-09-13T17:19:00.945847+0000","last_peered":"2024-09-13T17:19:00.945847+0000","last_clean":"2024-09-13T17:19:00.945847+0000","last_became_active":"2024-09-13T16:47:59.056455+0000","last_became_peered":"2024-09-13T16:47:59.056455+0000","last_unstale":"2024-09-13T17:19:00.945847+0000","last_undegraded":"2024-09-13T17:19:00.945847+0000","last_fullsized":"2024-09-13T17:19:00.945847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4422","last_scrub_stamp":"2024-09-13T17:19:00.945792+0000","last_deep_scrub":"63'4422","last_deep_scrub_stamp":"2024-09-13T17:19:00.945792+0000","last_clean_scrub_stamp":"2024-09-13T17:19:00.945792+0000","objects_scrubbed":0,"log_size":4422,"log_dups_size":0,"ondisk_log_size":4422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:42.479112+0000","scrub_duration":0.032777075000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":24862,"num_write_kb":304752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.23","version":"63'3771","reported_seq":3848,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:12.262217+0000","last_change":"2024-09-13T17:19:12.262564+0000","last_active":"2024-09-13T17:19:12.262217+0000","last_peered":"2024-09-13T17:19:12.262217+0000","last_clean":"2024-09-13T17:19:12.262217+0000","last_became_active":"2024-09-13T16:48:08.175094+0000","last_became_peered":"2024-09-13T16:48:08.175094+0000","last_unstale":"2024-09-13T17:19:12.262217+0000","last_undegraded":"2024-09-13T17:19:12.262217+0000","last_fullsized":"2024-09-13T17:19:12.262217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3771","last_scrub_stamp":"2024-09-13T17:19:12.262079+0000","last_deep_scrub":"63'3771","last_deep_scrub_stamp":"2024-09-13T17:19:12.262079+0000","last_clean_scrub_stamp":"2024-09-13T17:19:12.262079+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:13.369335+0000","scrub_duration":0.033166157000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21135,"num_write_kb":285540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"63'4461","reported_seq":4541,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:31.181911+0000","last_change":"2024-09-13T17:18:31.182225+0000","last_active":"2024-09-13T17:18:31.181911+0000","last_peered":"2024-09-13T17:18:31.181911+0000","last_clean":"2024-09-13T17:18:31.181911+0000","last_became_active":"2024-09-13T16:47:59.060660+0000","last_became_peered":"2024-09-13T16:47:59.060660+0000","last_unstale":"2024-09-13T17:18:31.181911+0000","last_undegraded":"2024-09-13T17:18:31.181911+0000","last_fullsized":"2024-09-13T17:18:31.181911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4461","last_scrub_stamp":"2024-09-13T17:18:31.181856+0000","last_deep_scrub":"63'4461","last_deep_scrub_stamp":"2024-09-13T17:18:31.181856+0000","last_clean_scrub_stamp":"2024-09-13T17:18:31.181856+0000","objects_scrubbed":0,"log_size":4461,"log_dups_size":0,"ondisk_log_size":4461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:30.453738+0000","scrub_duration":0.032702224000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25388,"num_write_kb":319094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.20","version":"63'4193","reported_seq":4270,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:15.373287+0000","last_change":"2024-09-13T17:19:15.373287+0000","last_active":"2024-09-13T17:19:15.373287+0000","last_peered":"2024-09-13T17:19:15.373287+0000","last_clean":"2024-09-13T17:19:15.373287+0000","last_became_active":"2024-09-13T16:48:08.200430+0000","last_became_peered":"2024-09-13T16:48:08.200430+0000","last_unstale":"2024-09-13T17:19:15.373287+0000","last_undegraded":"2024-09-13T17:19:15.373287+0000","last_fullsized":"2024-09-13T17:19:15.373287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4193","last_scrub_stamp":"2024-09-13T17:19:15.373009+0000","last_deep_scrub":"63'4193","last_deep_scrub_stamp":"2024-09-13T17:19:15.373009+0000","last_clean_scrub_stamp":"2024-09-13T17:19:15.373009+0000","objects_scrubbed":0,"log_size":4193,"log_dups_size":0,"ondisk_log_size":4193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:58:15.351859+0000","scrub_duration":0.037197431000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23066,"num_write_kb":313713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.26","version":"63'3978","reported_seq":4058,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.616395+0000","last_change":"2024-09-13T17:17:57.616624+0000","last_active":"2024-09-13T17:17:57.616395+0000","last_peered":"2024-09-13T17:17:57.616395+0000","last_clean":"2024-09-13T17:17:57.616395+0000","last_became_active":"2024-09-13T16:47:59.051437+0000","last_became_peered":"2024-09-13T16:47:59.051437+0000","last_unstale":"2024-09-13T17:17:57.616395+0000","last_undegraded":"2024-09-13T17:17:57.616395+0000","last_fullsized":"2024-09-13T17:17:57.616395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3978","last_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_deep_scrub":"63'3978","last_deep_scrub_stamp":"2024-09-13T17:17:57.616344+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.616344+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:01.359043+0000","scrub_duration":0.024757745000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22157,"num_write_kb":281285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"63'4050","reported_seq":4129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:54.093155+0000","last_change":"2024-09-13T17:18:54.093155+0000","last_active":"2024-09-13T17:18:54.093155+0000","last_peered":"2024-09-13T17:18:54.093155+0000","last_clean":"2024-09-13T17:18:54.093155+0000","last_became_active":"2024-09-13T16:48:08.163515+0000","last_became_peered":"2024-09-13T16:48:08.163515+0000","last_unstale":"2024-09-13T17:18:54.093155+0000","last_undegraded":"2024-09-13T17:18:54.093155+0000","last_fullsized":"2024-09-13T17:18:54.093155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4050","last_scrub_stamp":"2024-09-13T17:18:54.092845+0000","last_deep_scrub":"63'4050","last_deep_scrub_stamp":"2024-09-13T17:18:54.092845+0000","last_clean_scrub_stamp":"2024-09-13T17:18:54.092845+0000","objects_scrubbed":0,"log_size":4050,"log_dups_size":0,"ondisk_log_size":4050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:43.484599+0000","scrub_duration":0.012829612000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21731,"num_write_kb":291923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"63'5211","reported_seq":5286,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:57.601433+0000","last_change":"2024-09-13T17:18:57.601433+0000","last_active":"2024-09-13T17:18:57.601433+0000","last_peered":"2024-09-13T17:18:57.601433+0000","last_clean":"2024-09-13T17:18:57.601433+0000","last_became_active":"2024-09-13T16:48:40.831649+0000","last_became_peered":"2024-09-13T16:48:40.831649+0000","last_unstale":"2024-09-13T17:18:57.601433+0000","last_undegraded":"2024-09-13T17:18:57.601433+0000","last_fullsized":"2024-09-13T17:18:57.601433+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5211","last_scrub_stamp":"2024-09-13T17:18:57.601109+0000","last_deep_scrub":"63'5211","last_deep_scrub_stamp":"2024-09-13T17:18:57.601109+0000","last_clean_scrub_stamp":"2024-09-13T17:18:57.601109+0000","objects_scrubbed":0,"log_size":5211,"log_dups_size":0,"ondisk_log_size":5211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:08.145927+0000","scrub_duration":0.03195572,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":28065,"num_write_kb":366501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"63'3806","reported_seq":3884,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:19.985345+0000","last_change":"2024-09-13T17:19:19.985587+0000","last_active":"2024-09-13T17:19:19.985345+0000","last_peered":"2024-09-13T17:19:19.985345+0000","last_clean":"2024-09-13T17:19:19.985345+0000","last_became_active":"2024-09-13T16:48:08.165795+0000","last_became_peered":"2024-09-13T16:48:08.165795+0000","last_unstale":"2024-09-13T17:19:19.985345+0000","last_undegraded":"2024-09-13T17:19:19.985345+0000","last_fullsized":"2024-09-13T17:19:19.985345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3806","last_scrub_stamp":"2024-09-13T17:19:19.985285+0000","last_deep_scrub":"63'3806","last_deep_scrub_stamp":"2024-09-13T17:19:19.985285+0000","last_clean_scrub_stamp":"2024-09-13T17:19:19.985285+0000","objects_scrubbed":0,"log_size":3806,"log_dups_size":0,"ondisk_log_size":3806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:38.514003+0000","scrub_duration":0.026919018999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4,"num_write":20286,"num_write_kb":286237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"63'3659","reported_seq":3740,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:34.530920+0000","last_change":"2024-09-13T17:18:34.531140+0000","last_active":"2024-09-13T17:18:34.530920+0000","last_peered":"2024-09-13T17:18:34.530920+0000","last_clean":"2024-09-13T17:18:34.530920+0000","last_became_active":"2024-09-13T16:47:59.055279+0000","last_became_peered":"2024-09-13T16:47:59.055279+0000","last_unstale":"2024-09-13T17:18:34.530920+0000","last_undegraded":"2024-09-13T17:18:34.530920+0000","last_fullsized":"2024-09-13T17:18:34.530920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3659","last_scrub_stamp":"2024-09-13T17:18:34.530868+0000","last_deep_scrub":"63'3659","last_deep_scrub_stamp":"2024-09-13T17:18:34.530868+0000","last_clean_scrub_stamp":"2024-09-13T17:18:34.530868+0000","objects_scrubbed":0,"log_size":3659,"log_dups_size":0,"ondisk_log_size":3659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:29.832112+0000","scrub_duration":0.027670581999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19936,"num_write_kb":264332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2f","version":"63'5003","reported_seq":5079,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:11.422993+0000","last_change":"2024-09-13T17:19:11.423904+0000","last_active":"2024-09-13T17:19:11.422993+0000","last_peered":"2024-09-13T17:19:11.422993+0000","last_clean":"2024-09-13T17:19:11.422993+0000","last_became_active":"2024-09-13T16:48:08.193032+0000","last_became_peered":"2024-09-13T16:48:08.193032+0000","last_unstale":"2024-09-13T17:19:11.422993+0000","last_undegraded":"2024-09-13T17:19:11.422993+0000","last_fullsized":"2024-09-13T17:19:11.422993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5003","last_scrub_stamp":"2024-09-13T17:19:11.422941+0000","last_deep_scrub":"63'5003","last_deep_scrub_stamp":"2024-09-13T17:19:11.422941+0000","last_clean_scrub_stamp":"2024-09-13T17:19:11.422941+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:34.372606+0000","scrub_duration":0.039294029000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":27137,"num_write_kb":387189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"63'4068","reported_seq":4147,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:30.029523+0000","last_change":"2024-09-13T17:18:30.029753+0000","last_active":"2024-09-13T17:18:30.029523+0000","last_peered":"2024-09-13T17:18:30.029523+0000","last_clean":"2024-09-13T17:18:30.029523+0000","last_became_active":"2024-09-13T16:47:59.051118+0000","last_became_peered":"2024-09-13T16:47:59.051118+0000","last_unstale":"2024-09-13T17:18:30.029523+0000","last_undegraded":"2024-09-13T17:18:30.029523+0000","last_fullsized":"2024-09-13T17:18:30.029523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4068","last_scrub_stamp":"2024-09-13T17:18:30.029466+0000","last_deep_scrub":"63'4068","last_deep_scrub_stamp":"2024-09-13T17:18:30.029466+0000","last_clean_scrub_stamp":"2024-09-13T17:18:30.029466+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:47.003161+0000","scrub_duration":0.033158017999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22149,"num_write_kb":293545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"63'3283","reported_seq":3358,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:47.724019+0000","last_change":"2024-09-13T17:18:47.724256+0000","last_active":"2024-09-13T17:18:47.724019+0000","last_peered":"2024-09-13T17:18:47.724019+0000","last_clean":"2024-09-13T17:18:47.724019+0000","last_became_active":"2024-09-13T16:48:08.182866+0000","last_became_peered":"2024-09-13T16:48:08.182866+0000","last_unstale":"2024-09-13T17:18:47.724019+0000","last_undegraded":"2024-09-13T17:18:47.724019+0000","last_fullsized":"2024-09-13T17:18:47.724019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3283","last_scrub_stamp":"2024-09-13T17:18:47.723967+0000","last_deep_scrub":"63'3283","last_deep_scrub_stamp":"2024-09-13T17:18:47.723967+0000","last_clean_scrub_stamp":"2024-09-13T17:18:47.723967+0000","objects_scrubbed":0,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:25.074078+0000","scrub_duration":0.018868821000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18126,"num_write_kb":248866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"63'4697","reported_seq":4776,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:13.654346+0000","last_change":"2024-09-13T17:18:13.654578+0000","last_active":"2024-09-13T17:18:13.654346+0000","last_peered":"2024-09-13T17:18:13.654346+0000","last_clean":"2024-09-13T17:18:13.654346+0000","last_became_active":"2024-09-13T16:47:59.052900+0000","last_became_peered":"2024-09-13T16:47:59.052900+0000","last_unstale":"2024-09-13T17:18:13.654346+0000","last_undegraded":"2024-09-13T17:18:13.654346+0000","last_fullsized":"2024-09-13T17:18:13.654346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4697","last_scrub_stamp":"2024-09-13T17:18:13.654295+0000","last_deep_scrub":"63'4697","last_deep_scrub_stamp":"2024-09-13T17:18:13.654295+0000","last_clean_scrub_stamp":"2024-09-13T17:18:13.654295+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:58.717333+0000","scrub_duration":0.035392587000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25800,"num_write_kb":320606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2d","version":"63'3820","reported_seq":3895,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:18.967795+0000","last_change":"2024-09-13T17:19:18.968024+0000","last_active":"2024-09-13T17:19:18.967795+0000","last_peered":"2024-09-13T17:19:18.967795+0000","last_clean":"2024-09-13T17:19:18.967795+0000","last_became_active":"2024-09-13T16:48:08.164435+0000","last_became_peered":"2024-09-13T16:48:08.164435+0000","last_unstale":"2024-09-13T17:19:18.967795+0000","last_undegraded":"2024-09-13T17:19:18.967795+0000","last_fullsized":"2024-09-13T17:19:18.967795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3820","last_scrub_stamp":"2024-09-13T17:19:18.967743+0000","last_deep_scrub":"63'3820","last_deep_scrub_stamp":"2024-09-13T17:19:18.967743+0000","last_clean_scrub_stamp":"2024-09-13T17:19:18.967743+0000","objects_scrubbed":0,"log_size":3820,"log_dups_size":0,"ondisk_log_size":3820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:16.750711+0000","scrub_duration":0.030586519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20813,"num_write_kb":300503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2b","version":"63'4719","reported_seq":4800,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:56.892882+0000","last_change":"2024-09-13T17:17:56.893148+0000","last_active":"2024-09-13T17:17:56.892882+0000","last_peered":"2024-09-13T17:17:56.892882+0000","last_clean":"2024-09-13T17:17:56.892882+0000","last_became_active":"2024-09-13T16:47:59.048126+0000","last_became_peered":"2024-09-13T16:47:59.048126+0000","last_unstale":"2024-09-13T17:17:56.892882+0000","last_undegraded":"2024-09-13T17:17:56.892882+0000","last_fullsized":"2024-09-13T17:17:56.892882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4719","last_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_deep_scrub":"63'4719","last_deep_scrub_stamp":"2024-09-13T17:17:56.892826+0000","last_clean_scrub_stamp":"2024-09-13T17:17:56.892826+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:26.656787+0000","scrub_duration":0.032626751000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":25263,"num_write_kb":330973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"63'3722","reported_seq":3797,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:03.285307+0000","last_change":"2024-09-13T17:19:03.285533+0000","last_active":"2024-09-13T17:19:03.285307+0000","last_peered":"2024-09-13T17:19:03.285307+0000","last_clean":"2024-09-13T17:19:03.285307+0000","last_became_active":"2024-09-13T16:48:08.191357+0000","last_became_peered":"2024-09-13T16:48:08.191357+0000","last_unstale":"2024-09-13T17:19:03.285307+0000","last_undegraded":"2024-09-13T17:19:03.285307+0000","last_fullsized":"2024-09-13T17:19:03.285307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3722","last_scrub_stamp":"2024-09-13T17:19:03.285252+0000","last_deep_scrub":"63'3722","last_deep_scrub_stamp":"2024-09-13T17:19:03.285252+0000","last_clean_scrub_stamp":"2024-09-13T17:19:03.285252+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:19.415797+0000","scrub_duration":0.032938005999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20022,"num_write_kb":289122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"63'3813","reported_seq":3892,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:26.209831+0000","last_change":"2024-09-13T17:18:26.210107+0000","last_active":"2024-09-13T17:18:26.209831+0000","last_peered":"2024-09-13T17:18:26.209831+0000","last_clean":"2024-09-13T17:18:26.209831+0000","last_became_active":"2024-09-13T16:47:59.052633+0000","last_became_peered":"2024-09-13T16:47:59.052633+0000","last_unstale":"2024-09-13T17:18:26.209831+0000","last_undegraded":"2024-09-13T17:18:26.209831+0000","last_fullsized":"2024-09-13T17:18:26.209831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3813","last_scrub_stamp":"2024-09-13T17:18:26.209775+0000","last_deep_scrub":"63'3813","last_deep_scrub_stamp":"2024-09-13T17:18:26.209775+0000","last_clean_scrub_stamp":"2024-09-13T17:18:26.209775+0000","objects_scrubbed":0,"log_size":3813,"log_dups_size":0,"ondisk_log_size":3813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:11.272813+0000","scrub_duration":0.022253663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21202,"num_write_kb":265282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"63'4479","reported_seq":4558,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:13.263907+0000","last_change":"2024-09-13T17:19:13.264140+0000","last_active":"2024-09-13T17:19:13.263907+0000","last_peered":"2024-09-13T17:19:13.263907+0000","last_clean":"2024-09-13T17:19:13.263907+0000","last_became_active":"2024-09-13T16:48:08.184167+0000","last_became_peered":"2024-09-13T16:48:08.184167+0000","last_unstale":"2024-09-13T17:19:13.263907+0000","last_undegraded":"2024-09-13T17:19:13.263907+0000","last_fullsized":"2024-09-13T17:19:13.263907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4479","last_scrub_stamp":"2024-09-13T17:19:13.263853+0000","last_deep_scrub":"63'4479","last_deep_scrub_stamp":"2024-09-13T17:19:13.263853+0000","last_clean_scrub_stamp":"2024-09-13T17:19:13.263853+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:54.797173+0000","scrub_duration":0.036340245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":24556,"num_write_kb":340437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2d","version":"63'4549","reported_seq":4628,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:00.399885+0000","last_change":"2024-09-13T17:19:00.400112+0000","last_active":"2024-09-13T17:19:00.399885+0000","last_peered":"2024-09-13T17:19:00.399885+0000","last_clean":"2024-09-13T17:19:00.399885+0000","last_became_active":"2024-09-13T16:47:59.055237+0000","last_became_peered":"2024-09-13T16:47:59.055237+0000","last_unstale":"2024-09-13T17:19:00.399885+0000","last_undegraded":"2024-09-13T17:19:00.399885+0000","last_fullsized":"2024-09-13T17:19:00.399885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4549","last_scrub_stamp":"2024-09-13T17:19:00.399832+0000","last_deep_scrub":"63'4549","last_deep_scrub_stamp":"2024-09-13T17:19:00.399832+0000","last_clean_scrub_stamp":"2024-09-13T17:19:00.399832+0000","objects_scrubbed":0,"log_size":4549,"log_dups_size":0,"ondisk_log_size":4549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:41.933152+0000","scrub_duration":0.029616984999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24549,"num_write_kb":319591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.28","version":"63'3479","reported_seq":3556,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:19.565610+0000","last_change":"2024-09-13T17:19:19.565844+0000","last_active":"2024-09-13T17:19:19.565610+0000","last_peered":"2024-09-13T17:19:19.565610+0000","last_clean":"2024-09-13T17:19:19.565610+0000","last_became_active":"2024-09-13T16:48:08.170317+0000","last_became_peered":"2024-09-13T16:48:08.170317+0000","last_unstale":"2024-09-13T17:19:19.565610+0000","last_undegraded":"2024-09-13T17:19:19.565610+0000","last_fullsized":"2024-09-13T17:19:19.565610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3479","last_scrub_stamp":"2024-09-13T17:19:19.565555+0000","last_deep_scrub":"63'3479","last_deep_scrub_stamp":"2024-09-13T17:19:19.565555+0000","last_clean_scrub_stamp":"2024-09-13T17:19:19.565555+0000","objects_scrubbed":0,"log_size":3479,"log_dups_size":0,"ondisk_log_size":3479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:44:31.283146+0000","scrub_duration":0.027455219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18966,"num_write_kb":268432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"63'3573","reported_seq":3653,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:08.568217+0000","last_change":"2024-09-13T17:18:08.568217+0000","last_active":"2024-09-13T17:18:08.568217+0000","last_peered":"2024-09-13T17:18:08.568217+0000","last_clean":"2024-09-13T17:18:08.568217+0000","last_became_active":"2024-09-13T16:47:59.047018+0000","last_became_peered":"2024-09-13T16:47:59.047018+0000","last_unstale":"2024-09-13T17:18:08.568217+0000","last_undegraded":"2024-09-13T17:18:08.568217+0000","last_fullsized":"2024-09-13T17:18:08.568217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3573","last_scrub_stamp":"2024-09-13T17:18:08.567856+0000","last_deep_scrub":"63'3573","last_deep_scrub_stamp":"2024-09-13T17:18:08.567856+0000","last_clean_scrub_stamp":"2024-09-13T17:18:08.567856+0000","objects_scrubbed":0,"log_size":3573,"log_dups_size":0,"ondisk_log_size":3573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:07.880974+0000","scrub_duration":0.01354239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19348,"num_write_kb":270113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"63'4058","reported_seq":4133,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:20.300792+0000","last_change":"2024-09-13T17:19:20.301033+0000","last_active":"2024-09-13T17:19:20.300792+0000","last_peered":"2024-09-13T17:19:20.300792+0000","last_clean":"2024-09-13T17:19:20.300792+0000","last_became_active":"2024-09-13T16:48:08.194540+0000","last_became_peered":"2024-09-13T16:48:08.194540+0000","last_unstale":"2024-09-13T17:19:20.300792+0000","last_undegraded":"2024-09-13T17:19:20.300792+0000","last_fullsized":"2024-09-13T17:19:20.300792+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4058","last_scrub_stamp":"2024-09-13T17:19:20.300738+0000","last_deep_scrub":"63'4058","last_deep_scrub_stamp":"2024-09-13T17:19:20.300738+0000","last_clean_scrub_stamp":"2024-09-13T17:19:20.300738+0000","objects_scrubbed":0,"log_size":4058,"log_dups_size":0,"ondisk_log_size":4058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:56.345433+0000","scrub_duration":0.035220950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22331,"num_write_kb":302585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"63'5839","reported_seq":5918,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.655103+0000","last_change":"2024-09-13T17:18:44.655336+0000","last_active":"2024-09-13T17:18:44.655103+0000","last_peered":"2024-09-13T17:18:44.655103+0000","last_clean":"2024-09-13T17:18:44.655103+0000","last_became_active":"2024-09-13T16:47:59.054240+0000","last_became_peered":"2024-09-13T16:47:59.054240+0000","last_unstale":"2024-09-13T17:18:44.655103+0000","last_undegraded":"2024-09-13T17:18:44.655103+0000","last_fullsized":"2024-09-13T17:18:44.655103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5839","last_scrub_stamp":"2024-09-13T17:18:44.655047+0000","last_deep_scrub":"63'5839","last_deep_scrub_stamp":"2024-09-13T17:18:44.655047+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.655047+0000","objects_scrubbed":0,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:17.349629+0000","scrub_duration":0.042557057000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31756,"num_write_kb":397521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"63'3835","reported_seq":3910,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:05.213984+0000","last_change":"2024-09-13T17:19:05.214237+0000","last_active":"2024-09-13T17:19:05.213984+0000","last_peered":"2024-09-13T17:19:05.213984+0000","last_clean":"2024-09-13T17:19:05.213984+0000","last_became_active":"2024-09-13T16:48:08.167345+0000","last_became_peered":"2024-09-13T16:48:08.167345+0000","last_unstale":"2024-09-13T17:19:05.213984+0000","last_undegraded":"2024-09-13T17:19:05.213984+0000","last_fullsized":"2024-09-13T17:19:05.213984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3835","last_scrub_stamp":"2024-09-13T17:19:05.213932+0000","last_deep_scrub":"63'3835","last_deep_scrub_stamp":"2024-09-13T17:19:05.213932+0000","last_clean_scrub_stamp":"2024-09-13T17:19:05.213932+0000","objects_scrubbed":0,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:23:44.268101+0000","scrub_duration":0.028321004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":6,"num_write":21185,"num_write_kb":306173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"63'4134","reported_seq":4213,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.595782+0000","last_change":"2024-09-13T17:17:57.595782+0000","last_active":"2024-09-13T17:17:57.595782+0000","last_peered":"2024-09-13T17:17:57.595782+0000","last_clean":"2024-09-13T17:17:57.595782+0000","last_became_active":"2024-09-13T16:47:59.046791+0000","last_became_peered":"2024-09-13T16:47:59.046791+0000","last_unstale":"2024-09-13T17:17:57.595782+0000","last_undegraded":"2024-09-13T17:17:57.595782+0000","last_fullsized":"2024-09-13T17:17:57.595782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4134","last_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_deep_scrub":"63'4134","last_deep_scrub_stamp":"2024-09-13T17:17:57.595505+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.595505+0000","objects_scrubbed":0,"log_size":4134,"log_dups_size":0,"ondisk_log_size":4134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:50.101213+0000","scrub_duration":0.024248147000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22268,"num_write_kb":281801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.37","version":"63'5073","reported_seq":5149,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:06.320617+0000","last_change":"2024-09-13T17:19:06.320883+0000","last_active":"2024-09-13T17:19:06.320617+0000","last_peered":"2024-09-13T17:19:06.320617+0000","last_clean":"2024-09-13T17:19:06.320617+0000","last_became_active":"2024-09-13T16:48:08.203703+0000","last_became_peered":"2024-09-13T16:48:08.203703+0000","last_unstale":"2024-09-13T17:19:06.320617+0000","last_undegraded":"2024-09-13T17:19:06.320617+0000","last_fullsized":"2024-09-13T17:19:06.320617+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5073","last_scrub_stamp":"2024-09-13T17:19:06.320563+0000","last_deep_scrub":"63'5073","last_deep_scrub_stamp":"2024-09-13T17:19:06.320563+0000","last_clean_scrub_stamp":"2024-09-13T17:19:06.320563+0000","objects_scrubbed":0,"log_size":5073,"log_dups_size":0,"ondisk_log_size":5073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:33:42.365259+0000","scrub_duration":0.043951086,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":28693,"num_write_kb":393724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.31","version":"63'3822","reported_seq":3901,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:07.186357+0000","last_change":"2024-09-13T17:18:07.186357+0000","last_active":"2024-09-13T17:18:07.186357+0000","last_peered":"2024-09-13T17:18:07.186357+0000","last_clean":"2024-09-13T17:18:07.186357+0000","last_became_active":"2024-09-13T16:47:59.038786+0000","last_became_peered":"2024-09-13T16:47:59.038786+0000","last_unstale":"2024-09-13T17:18:07.186357+0000","last_undegraded":"2024-09-13T17:18:07.186357+0000","last_fullsized":"2024-09-13T17:18:07.186357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3822","last_scrub_stamp":"2024-09-13T17:18:07.186043+0000","last_deep_scrub":"63'3822","last_deep_scrub_stamp":"2024-09-13T17:18:07.186043+0000","last_clean_scrub_stamp":"2024-09-13T17:18:07.186043+0000","objects_scrubbed":0,"log_size":3822,"log_dups_size":0,"ondisk_log_size":3822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:30.074139+0000","scrub_duration":0.030687262,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20823,"num_write_kb":275830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.34","version":"63'2778","reported_seq":2855,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:13.552399+0000","last_change":"2024-09-13T17:19:13.552625+0000","last_active":"2024-09-13T17:19:13.552399+0000","last_peered":"2024-09-13T17:19:13.552399+0000","last_clean":"2024-09-13T17:19:13.552399+0000","last_became_active":"2024-09-13T16:48:08.181290+0000","last_became_peered":"2024-09-13T16:48:08.181290+0000","last_unstale":"2024-09-13T17:19:13.552399+0000","last_undegraded":"2024-09-13T17:19:13.552399+0000","last_fullsized":"2024-09-13T17:19:13.552399+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2778","last_scrub_stamp":"2024-09-13T17:19:13.552346+0000","last_deep_scrub":"63'2778","last_deep_scrub_stamp":"2024-09-13T17:19:13.552346+0000","last_clean_scrub_stamp":"2024-09-13T17:19:13.552346+0000","objects_scrubbed":0,"log_size":2778,"log_dups_size":0,"ondisk_log_size":2778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.699051+0000","scrub_duration":0.025646242,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":34,"num_write":15212,"num_write_kb":208573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"63'3975","reported_seq":4054,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:01.856922+0000","last_change":"2024-09-13T17:18:01.856922+0000","last_active":"2024-09-13T17:18:01.856922+0000","last_peered":"2024-09-13T17:18:01.856922+0000","last_clean":"2024-09-13T17:18:01.856922+0000","last_became_active":"2024-09-13T16:47:59.048847+0000","last_became_peered":"2024-09-13T16:47:59.048847+0000","last_unstale":"2024-09-13T17:18:01.856922+0000","last_undegraded":"2024-09-13T17:18:01.856922+0000","last_fullsized":"2024-09-13T17:18:01.856922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3975","last_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_deep_scrub":"63'3975","last_deep_scrub_stamp":"2024-09-13T17:18:01.856645+0000","last_clean_scrub_stamp":"2024-09-13T17:18:01.856645+0000","objects_scrubbed":0,"log_size":3975,"log_dups_size":0,"ondisk_log_size":3975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:51.096023+0000","scrub_duration":0.021351690999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21351,"num_write_kb":278200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"63'3844","reported_seq":3920,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:17.247940+0000","last_change":"2024-09-13T17:18:17.248778+0000","last_active":"2024-09-13T17:18:17.247940+0000","last_peered":"2024-09-13T17:18:17.247940+0000","last_clean":"2024-09-13T17:18:17.247940+0000","last_became_active":"2024-09-13T16:48:08.193560+0000","last_became_peered":"2024-09-13T16:48:08.193560+0000","last_unstale":"2024-09-13T17:18:17.247940+0000","last_undegraded":"2024-09-13T17:18:17.247940+0000","last_fullsized":"2024-09-13T17:18:17.247940+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3844","last_scrub_stamp":"2024-09-13T17:18:17.247886+0000","last_deep_scrub":"63'3844","last_deep_scrub_stamp":"2024-09-13T17:18:17.247886+0000","last_clean_scrub_stamp":"2024-09-13T17:18:17.247886+0000","objects_scrubbed":0,"log_size":3844,"log_dups_size":0,"ondisk_log_size":3844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:16.519768+0000","scrub_duration":0.029367863000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":21085,"num_write_kb":289508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.33","version":"63'3324","reported_seq":3403,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:18.177724+0000","last_change":"2024-09-13T17:18:18.177724+0000","last_active":"2024-09-13T17:18:18.177724+0000","last_peered":"2024-09-13T17:18:18.177724+0000","last_clean":"2024-09-13T17:18:18.177724+0000","last_became_active":"2024-09-13T16:47:59.059418+0000","last_became_peered":"2024-09-13T16:47:59.059418+0000","last_unstale":"2024-09-13T17:18:18.177724+0000","last_undegraded":"2024-09-13T17:18:18.177724+0000","last_fullsized":"2024-09-13T17:18:18.177724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3324","last_scrub_stamp":"2024-09-13T17:18:18.177437+0000","last_deep_scrub":"63'3324","last_deep_scrub_stamp":"2024-09-13T17:18:18.177437+0000","last_clean_scrub_stamp":"2024-09-13T17:18:18.177437+0000","objects_scrubbed":0,"log_size":3324,"log_dups_size":0,"ondisk_log_size":3324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:17.449318+0000","scrub_duration":0.024197943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18810,"num_write_kb":235312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.32","version":"63'3743","reported_seq":3819,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:10.005164+0000","last_change":"2024-09-13T17:19:10.005406+0000","last_active":"2024-09-13T17:19:10.005164+0000","last_peered":"2024-09-13T17:19:10.005164+0000","last_clean":"2024-09-13T17:19:10.005164+0000","last_became_active":"2024-09-13T16:48:08.169937+0000","last_became_peered":"2024-09-13T16:48:08.169937+0000","last_unstale":"2024-09-13T17:19:10.005164+0000","last_undegraded":"2024-09-13T17:19:10.005164+0000","last_fullsized":"2024-09-13T17:19:10.005164+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3743","last_scrub_stamp":"2024-09-13T17:19:10.005105+0000","last_deep_scrub":"63'3743","last_deep_scrub_stamp":"2024-09-13T17:19:10.005105+0000","last_clean_scrub_stamp":"2024-09-13T17:19:10.005105+0000","objects_scrubbed":0,"log_size":3743,"log_dups_size":0,"ondisk_log_size":3743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:50.714066+0000","scrub_duration":0.028772444000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20790,"num_write_kb":280180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"63'3173","reported_seq":3252,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:43.207942+0000","last_change":"2024-09-13T17:18:43.207942+0000","last_active":"2024-09-13T17:18:43.207942+0000","last_peered":"2024-09-13T17:18:43.207942+0000","last_clean":"2024-09-13T17:18:43.207942+0000","last_became_active":"2024-09-13T16:47:59.047030+0000","last_became_peered":"2024-09-13T16:47:59.047030+0000","last_unstale":"2024-09-13T17:18:43.207942+0000","last_undegraded":"2024-09-13T17:18:43.207942+0000","last_fullsized":"2024-09-13T17:18:43.207942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3173","last_scrub_stamp":"2024-09-13T17:18:43.207650+0000","last_deep_scrub":"63'3173","last_deep_scrub_stamp":"2024-09-13T17:18:43.207650+0000","last_clean_scrub_stamp":"2024-09-13T17:18:43.207650+0000","objects_scrubbed":0,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:00.181344+0000","scrub_duration":0.024785841999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17044,"num_write_kb":219853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.33","version":"63'3460","reported_seq":3535,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:44.695358+0000","last_change":"2024-09-13T17:18:44.695678+0000","last_active":"2024-09-13T17:18:44.695358+0000","last_peered":"2024-09-13T17:18:44.695358+0000","last_clean":"2024-09-13T17:18:44.695358+0000","last_became_active":"2024-09-13T16:48:08.168860+0000","last_became_peered":"2024-09-13T16:48:08.168860+0000","last_unstale":"2024-09-13T17:18:44.695358+0000","last_undegraded":"2024-09-13T17:18:44.695358+0000","last_fullsized":"2024-09-13T17:18:44.695358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3460","last_scrub_stamp":"2024-09-13T17:18:44.695305+0000","last_deep_scrub":"63'3460","last_deep_scrub_stamp":"2024-09-13T17:18:44.695305+0000","last_clean_scrub_stamp":"2024-09-13T17:18:44.695305+0000","objects_scrubbed":0,"log_size":3460,"log_dups_size":0,"ondisk_log_size":3460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:57.484941+0000","scrub_duration":0.017970386000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":19788,"num_write_kb":263976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.35","version":"63'3747","reported_seq":3826,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:12.464344+0000","last_change":"2024-09-13T17:18:12.464344+0000","last_active":"2024-09-13T17:18:12.464344+0000","last_peered":"2024-09-13T17:18:12.464344+0000","last_clean":"2024-09-13T17:18:12.464344+0000","last_became_active":"2024-09-13T16:47:59.054386+0000","last_became_peered":"2024-09-13T16:47:59.054386+0000","last_unstale":"2024-09-13T17:18:12.464344+0000","last_undegraded":"2024-09-13T17:18:12.464344+0000","last_fullsized":"2024-09-13T17:18:12.464344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3747","last_scrub_stamp":"2024-09-13T17:18:12.464055+0000","last_deep_scrub":"63'3747","last_deep_scrub_stamp":"2024-09-13T17:18:12.464055+0000","last_clean_scrub_stamp":"2024-09-13T17:18:12.464055+0000","objects_scrubbed":0,"log_size":3747,"log_dups_size":0,"ondisk_log_size":3747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:25.441547+0000","scrub_duration":0.021103311999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20231,"num_write_kb":269532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.30","version":"63'3422","reported_seq":3497,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:27.175049+0000","last_change":"2024-09-13T17:18:27.175049+0000","last_active":"2024-09-13T17:18:27.175049+0000","last_peered":"2024-09-13T17:18:27.175049+0000","last_clean":"2024-09-13T17:18:27.175049+0000","last_became_active":"2024-09-13T16:48:08.192058+0000","last_became_peered":"2024-09-13T16:48:08.192058+0000","last_unstale":"2024-09-13T17:18:27.175049+0000","last_undegraded":"2024-09-13T17:18:27.175049+0000","last_fullsized":"2024-09-13T17:18:27.175049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3422","last_scrub_stamp":"2024-09-13T17:18:27.174746+0000","last_deep_scrub":"63'3422","last_deep_scrub_stamp":"2024-09-13T17:18:27.174746+0000","last_clean_scrub_stamp":"2024-09-13T17:18:27.174746+0000","objects_scrubbed":0,"log_size":3422,"log_dups_size":0,"ondisk_log_size":3422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:39.964382+0000","scrub_duration":0.029236364000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18722,"num_write_kb":267617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"63'4803","reported_seq":4882,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:58.621508+0000","last_change":"2024-09-13T17:17:58.621508+0000","last_active":"2024-09-13T17:17:58.621508+0000","last_peered":"2024-09-13T17:17:58.621508+0000","last_clean":"2024-09-13T17:17:58.621508+0000","last_became_active":"2024-09-13T16:47:59.054212+0000","last_became_peered":"2024-09-13T16:47:59.054212+0000","last_unstale":"2024-09-13T17:17:58.621508+0000","last_undegraded":"2024-09-13T17:17:58.621508+0000","last_fullsized":"2024-09-13T17:17:58.621508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4803","last_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_deep_scrub":"63'4803","last_deep_scrub_stamp":"2024-09-13T17:17:58.621222+0000","last_clean_scrub_stamp":"2024-09-13T17:17:58.621222+0000","objects_scrubbed":0,"log_size":4803,"log_dups_size":0,"ondisk_log_size":4803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:06.376472+0000","scrub_duration":0.023354388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26967,"num_write_kb":333429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.31","version":"63'3840","reported_seq":3915,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:18.515080+0000","last_change":"2024-09-13T17:19:18.515315+0000","last_active":"2024-09-13T17:19:18.515080+0000","last_peered":"2024-09-13T17:19:18.515080+0000","last_clean":"2024-09-13T17:19:18.515080+0000","last_became_active":"2024-09-13T16:48:08.174466+0000","last_became_peered":"2024-09-13T16:48:08.174466+0000","last_unstale":"2024-09-13T17:19:18.515080+0000","last_undegraded":"2024-09-13T17:19:18.515080+0000","last_fullsized":"2024-09-13T17:19:18.515080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3840","last_scrub_stamp":"2024-09-13T17:19:18.515027+0000","last_deep_scrub":"63'3840","last_deep_scrub_stamp":"2024-09-13T17:19:18.515027+0000","last_clean_scrub_stamp":"2024-09-13T17:19:18.515027+0000","objects_scrubbed":0,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:16.297994+0000","scrub_duration":0.026649774000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21113,"num_write_kb":293770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"63'5433","reported_seq":5513,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.303582+0000","last_change":"2024-09-13T17:17:53.303824+0000","last_active":"2024-09-13T17:17:53.303582+0000","last_peered":"2024-09-13T17:17:53.303582+0000","last_clean":"2024-09-13T17:17:53.303582+0000","last_became_active":"2024-09-13T16:47:59.061983+0000","last_became_peered":"2024-09-13T16:47:59.061983+0000","last_unstale":"2024-09-13T17:17:53.303582+0000","last_undegraded":"2024-09-13T17:17:53.303582+0000","last_fullsized":"2024-09-13T17:17:53.303582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5433","last_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_deep_scrub":"63'5433","last_deep_scrub_stamp":"2024-09-13T17:17:53.303524+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.303524+0000","objects_scrubbed":0,"log_size":5433,"log_dups_size":0,"ondisk_log_size":5433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.191620+0000","scrub_duration":0.041011686999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30664,"num_write_kb":389336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3e","version":"63'4741","reported_seq":4820,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:21.014261+0000","last_change":"2024-09-13T17:19:21.014261+0000","last_active":"2024-09-13T17:19:21.014261+0000","last_peered":"2024-09-13T17:19:21.014261+0000","last_clean":"2024-09-13T17:19:21.014261+0000","last_became_active":"2024-09-13T16:48:08.168970+0000","last_became_peered":"2024-09-13T16:48:08.168970+0000","last_unstale":"2024-09-13T17:19:21.014261+0000","last_undegraded":"2024-09-13T17:19:21.014261+0000","last_fullsized":"2024-09-13T17:19:21.014261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4741","last_scrub_stamp":"2024-09-13T17:19:21.013945+0000","last_deep_scrub":"63'4741","last_deep_scrub_stamp":"2024-09-13T17:19:21.013945+0000","last_clean_scrub_stamp":"2024-09-13T17:19:21.013945+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:27.328550+0000","scrub_duration":0.032848018999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":15,"num_write":25588,"num_write_kb":340391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.38","version":"63'4281","reported_seq":4359,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:57.227117+0000","last_change":"2024-09-13T17:17:57.227117+0000","last_active":"2024-09-13T17:17:57.227117+0000","last_peered":"2024-09-13T17:17:57.227117+0000","last_clean":"2024-09-13T17:17:57.227117+0000","last_became_active":"2024-09-13T16:47:59.059415+0000","last_became_peered":"2024-09-13T16:47:59.059415+0000","last_unstale":"2024-09-13T17:17:57.227117+0000","last_undegraded":"2024-09-13T17:17:57.227117+0000","last_fullsized":"2024-09-13T17:17:57.227117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4281","last_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_deep_scrub":"63'4281","last_deep_scrub_stamp":"2024-09-13T17:17:57.226795+0000","last_clean_scrub_stamp":"2024-09-13T17:17:57.226795+0000","objects_scrubbed":0,"log_size":4281,"log_dups_size":0,"ondisk_log_size":4281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:49.732503+0000","scrub_duration":0.034883109000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23373,"num_write_kb":302635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3f","version":"63'3575","reported_seq":3655,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.020990+0000","last_change":"2024-09-13T17:18:45.020990+0000","last_active":"2024-09-13T17:18:45.020990+0000","last_peered":"2024-09-13T17:18:45.020990+0000","last_clean":"2024-09-13T17:18:45.020990+0000","last_became_active":"2024-09-13T16:48:08.178160+0000","last_became_peered":"2024-09-13T16:48:08.178160+0000","last_unstale":"2024-09-13T17:18:45.020990+0000","last_undegraded":"2024-09-13T17:18:45.020990+0000","last_fullsized":"2024-09-13T17:18:45.020990+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3575","last_scrub_stamp":"2024-09-13T17:18:45.020682+0000","last_deep_scrub":"63'3575","last_deep_scrub_stamp":"2024-09-13T17:18:45.020682+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.020682+0000","objects_scrubbed":0,"log_size":3575,"log_dups_size":0,"ondisk_log_size":3575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:55.157528+0000","scrub_duration":0.013036646000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":7,"num_write":20117,"num_write_kb":289677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.39","version":"63'4404","reported_seq":4482,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:50.337858+0000","last_change":"2024-09-13T17:17:50.337858+0000","last_active":"2024-09-13T17:17:50.337858+0000","last_peered":"2024-09-13T17:17:50.337858+0000","last_clean":"2024-09-13T17:17:50.337858+0000","last_became_active":"2024-09-13T16:47:59.047783+0000","last_became_peered":"2024-09-13T16:47:59.047783+0000","last_unstale":"2024-09-13T17:17:50.337858+0000","last_undegraded":"2024-09-13T17:17:50.337858+0000","last_fullsized":"2024-09-13T17:17:50.337858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4404","last_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_deep_scrub":"63'4404","last_deep_scrub_stamp":"2024-09-13T17:17:50.337585+0000","last_clean_scrub_stamp":"2024-09-13T17:17:50.337585+0000","objects_scrubbed":0,"log_size":4404,"log_dups_size":0,"ondisk_log_size":4404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:03:52.971802+0000","scrub_duration":0.031410212999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24308,"num_write_kb":306423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3c","version":"63'3318","reported_seq":3396,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:16.263174+0000","last_change":"2024-09-13T17:19:16.263410+0000","last_active":"2024-09-13T17:19:16.263174+0000","last_peered":"2024-09-13T17:19:16.263174+0000","last_clean":"2024-09-13T17:19:16.263174+0000","last_became_active":"2024-09-13T16:48:08.185087+0000","last_became_peered":"2024-09-13T16:48:08.185087+0000","last_unstale":"2024-09-13T17:19:16.263174+0000","last_undegraded":"2024-09-13T17:19:16.263174+0000","last_fullsized":"2024-09-13T17:19:16.263174+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3318","last_scrub_stamp":"2024-09-13T17:19:16.263115+0000","last_deep_scrub":"63'3318","last_deep_scrub_stamp":"2024-09-13T17:19:16.263115+0000","last_clean_scrub_stamp":"2024-09-13T17:19:16.263115+0000","objects_scrubbed":0,"log_size":3318,"log_dups_size":0,"ondisk_log_size":3318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:33:30.197758+0000","scrub_duration":0.026865445000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":18295,"num_write_kb":253537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"63'4013","reported_seq":4091,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:53.598539+0000","last_change":"2024-09-13T17:17:53.598539+0000","last_active":"2024-09-13T17:17:53.598539+0000","last_peered":"2024-09-13T17:17:53.598539+0000","last_clean":"2024-09-13T17:17:53.598539+0000","last_became_active":"2024-09-13T16:47:59.048670+0000","last_became_peered":"2024-09-13T16:47:59.048670+0000","last_unstale":"2024-09-13T17:17:53.598539+0000","last_undegraded":"2024-09-13T17:17:53.598539+0000","last_fullsized":"2024-09-13T17:17:53.598539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4013","last_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_deep_scrub":"63'4013","last_deep_scrub_stamp":"2024-09-13T17:17:53.598242+0000","last_clean_scrub_stamp":"2024-09-13T17:17:53.598242+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:16.486338+0000","scrub_duration":0.028004864000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21393,"num_write_kb":262042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"63'3658","reported_seq":3736,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:21.532611+0000","last_change":"2024-09-13T17:19:21.532611+0000","last_active":"2024-09-13T17:19:21.532611+0000","last_peered":"2024-09-13T17:19:21.532611+0000","last_clean":"2024-09-13T17:19:21.532611+0000","last_became_active":"2024-09-13T16:48:08.171437+0000","last_became_peered":"2024-09-13T16:48:08.171437+0000","last_unstale":"2024-09-13T17:19:21.532611+0000","last_undegraded":"2024-09-13T17:19:21.532611+0000","last_fullsized":"2024-09-13T17:19:21.532611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3658","last_scrub_stamp":"2024-09-13T17:19:21.532336+0000","last_deep_scrub":"63'3658","last_deep_scrub_stamp":"2024-09-13T17:19:21.532336+0000","last_clean_scrub_stamp":"2024-09-13T17:19:21.532336+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:27.846941+0000","scrub_duration":0.024452629,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19946,"num_write_kb":268446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"63'3910","reported_seq":3988,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:19.247862+0000","last_change":"2024-09-13T17:18:19.248105+0000","last_active":"2024-09-13T17:18:19.247862+0000","last_peered":"2024-09-13T17:18:19.247862+0000","last_clean":"2024-09-13T17:18:19.247862+0000","last_became_active":"2024-09-13T16:47:59.052022+0000","last_became_peered":"2024-09-13T16:47:59.052022+0000","last_unstale":"2024-09-13T17:18:19.247862+0000","last_undegraded":"2024-09-13T17:18:19.247862+0000","last_fullsized":"2024-09-13T17:18:19.247862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3910","last_scrub_stamp":"2024-09-13T17:18:19.247809+0000","last_deep_scrub":"63'3910","last_deep_scrub_stamp":"2024-09-13T17:18:19.247809+0000","last_clean_scrub_stamp":"2024-09-13T17:18:19.247809+0000","objects_scrubbed":0,"log_size":3910,"log_dups_size":0,"ondisk_log_size":3910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:18.673196+0000","scrub_duration":0.029454408000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20421,"num_write_kb":278651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3a","version":"63'3074","reported_seq":3148,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:12.976040+0000","last_change":"2024-09-13T17:19:12.976284+0000","last_active":"2024-09-13T17:19:12.976040+0000","last_peered":"2024-09-13T17:19:12.976040+0000","last_clean":"2024-09-13T17:19:12.976040+0000","last_became_active":"2024-09-13T16:48:08.172856+0000","last_became_peered":"2024-09-13T16:48:08.172856+0000","last_unstale":"2024-09-13T17:19:12.976040+0000","last_undegraded":"2024-09-13T17:19:12.976040+0000","last_fullsized":"2024-09-13T17:19:12.976040+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3074","last_scrub_stamp":"2024-09-13T17:19:12.975988+0000","last_deep_scrub":"63'3074","last_deep_scrub_stamp":"2024-09-13T17:19:12.975988+0000","last_clean_scrub_stamp":"2024-09-13T17:19:12.975988+0000","objects_scrubbed":0,"log_size":3074,"log_dups_size":0,"ondisk_log_size":3074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:04:53.229968+0000","scrub_duration":0.026782991999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":246226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"63'3515","reported_seq":3597,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:52.560367+0000","last_change":"2024-09-13T17:17:52.560367+0000","last_active":"2024-09-13T17:17:52.560367+0000","last_peered":"2024-09-13T17:17:52.560367+0000","last_clean":"2024-09-13T17:17:52.560367+0000","last_became_active":"2024-09-13T16:47:59.060646+0000","last_became_peered":"2024-09-13T16:47:59.060646+0000","last_unstale":"2024-09-13T17:17:52.560367+0000","last_undegraded":"2024-09-13T17:17:52.560367+0000","last_fullsized":"2024-09-13T17:17:52.560367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3515","last_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_deep_scrub":"63'3515","last_deep_scrub_stamp":"2024-09-13T17:17:52.560093+0000","last_clean_scrub_stamp":"2024-09-13T17:17:52.560093+0000","objects_scrubbed":0,"log_size":3515,"log_dups_size":0,"ondisk_log_size":3515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:02.500715+0000","scrub_duration":0.026529094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":19512,"num_write_kb":258445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3b","version":"63'3439","reported_seq":3514,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.727356+0000","last_change":"2024-09-13T17:18:45.727616+0000","last_active":"2024-09-13T17:18:45.727356+0000","last_peered":"2024-09-13T17:18:45.727356+0000","last_clean":"2024-09-13T17:18:45.727356+0000","last_became_active":"2024-09-13T16:48:08.163950+0000","last_became_peered":"2024-09-13T16:48:08.163950+0000","last_unstale":"2024-09-13T17:18:45.727356+0000","last_undegraded":"2024-09-13T17:18:45.727356+0000","last_fullsized":"2024-09-13T17:18:45.727356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3439","last_scrub_stamp":"2024-09-13T17:18:45.727303+0000","last_deep_scrub":"63'3439","last_deep_scrub_stamp":"2024-09-13T17:18:45.727303+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.727303+0000","objects_scrubbed":0,"log_size":3439,"log_dups_size":0,"ondisk_log_size":3439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:45:02.700997+0000","scrub_duration":0.020628344999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":3,"num_write":17681,"num_write_kb":263603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"63'3892","reported_seq":3974,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:45.152938+0000","last_change":"2024-09-13T17:18:45.152938+0000","last_active":"2024-09-13T17:18:45.152938+0000","last_peered":"2024-09-13T17:18:45.152938+0000","last_clean":"2024-09-13T17:18:45.152938+0000","last_became_active":"2024-09-13T16:47:59.047467+0000","last_became_peered":"2024-09-13T16:47:59.047467+0000","last_unstale":"2024-09-13T17:18:45.152938+0000","last_undegraded":"2024-09-13T17:18:45.152938+0000","last_fullsized":"2024-09-13T17:18:45.152938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3892","last_scrub_stamp":"2024-09-13T17:18:45.152646+0000","last_deep_scrub":"63'3892","last_deep_scrub_stamp":"2024-09-13T17:18:45.152646+0000","last_clean_scrub_stamp":"2024-09-13T17:18:45.152646+0000","objects_scrubbed":0,"log_size":3892,"log_dups_size":0,"ondisk_log_size":3892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:38:44.424528+0000","scrub_duration":0.018925824000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":21304,"num_write_kb":286190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"63'3622","reported_seq":3696,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:19:16.343537+0000","last_change":"2024-09-13T17:19:16.343898+0000","last_active":"2024-09-13T17:19:16.343537+0000","last_peered":"2024-09-13T17:19:16.343537+0000","last_clean":"2024-09-13T17:19:16.343537+0000","last_became_active":"2024-09-13T16:48:08.185839+0000","last_became_peered":"2024-09-13T16:48:08.185839+0000","last_unstale":"2024-09-13T17:19:16.343537+0000","last_undegraded":"2024-09-13T17:19:16.343537+0000","last_fullsized":"2024-09-13T17:19:16.343537+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3622","last_scrub_stamp":"2024-09-13T17:19:16.343426+0000","last_deep_scrub":"63'3622","last_deep_scrub_stamp":"2024-09-13T17:19:16.343426+0000","last_clean_scrub_stamp":"2024-09-13T17:19:16.343426+0000","objects_scrubbed":0,"log_size":3622,"log_dups_size":0,"ondisk_log_size":3622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:30:14.126393+0000","scrub_duration":0.028067912,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19729,"num_write_kb":275691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"63'5036","reported_seq":5117,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:41.341900+0000","last_change":"2024-09-13T17:18:41.341900+0000","last_active":"2024-09-13T17:18:41.341900+0000","last_peered":"2024-09-13T17:18:41.341900+0000","last_clean":"2024-09-13T17:18:41.341900+0000","last_became_active":"2024-09-13T16:47:59.048065+0000","last_became_peered":"2024-09-13T16:47:59.048065+0000","last_unstale":"2024-09-13T17:18:41.341900+0000","last_undegraded":"2024-09-13T17:18:41.341900+0000","last_fullsized":"2024-09-13T17:18:41.341900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'5036","last_scrub_stamp":"2024-09-13T17:18:41.341603+0000","last_deep_scrub":"63'5036","last_deep_scrub_stamp":"2024-09-13T17:18:41.341603+0000","last_clean_scrub_stamp":"2024-09-13T17:18:41.341603+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:45:54.131238+0000","scrub_duration":0.03025245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27469,"num_write_kb":368239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"63'3689","reported_seq":3763,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:18:34.266698+0000","last_change":"2024-09-13T17:18:34.266933+0000","last_active":"2024-09-13T17:18:34.266698+0000","last_peered":"2024-09-13T17:18:34.266698+0000","last_clean":"2024-09-13T17:18:34.266698+0000","last_became_active":"2024-09-13T16:48:08.188804+0000","last_became_peered":"2024-09-13T16:48:08.188804+0000","last_unstale":"2024-09-13T17:18:34.266698+0000","last_undegraded":"2024-09-13T17:18:34.266698+0000","last_fullsized":"2024-09-13T17:18:34.266698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3689","last_scrub_stamp":"2024-09-13T17:18:34.266642+0000","last_deep_scrub":"63'3689","last_deep_scrub_stamp":"2024-09-13T17:18:34.266642+0000","last_clean_scrub_stamp":"2024-09-13T17:18:34.266642+0000","objects_scrubbed":0,"log_size":3689,"log_dups_size":0,"ondisk_log_size":3689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:44.403488+0000","scrub_duration":0.024276446,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20818,"num_write_kb":284187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"63'4047","reported_seq":4129,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T17:17:51.364373+0000","last_change":"2024-09-13T17:17:51.364373+0000","last_active":"2024-09-13T17:17:51.364373+0000","last_peered":"2024-09-13T17:17:51.364373+0000","last_clean":"2024-09-13T17:17:51.364373+0000","last_became_active":"2024-09-13T16:47:59.055561+0000","last_became_peered":"2024-09-13T16:47:59.055561+0000","last_unstale":"2024-09-13T17:17:51.364373+0000","last_undegraded":"2024-09-13T17:17:51.364373+0000","last_fullsized":"2024-09-13T17:17:51.364373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'4047","last_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_deep_scrub":"63'4047","last_deep_scrub_stamp":"2024-09-13T17:17:51.364064+0000","last_clean_scrub_stamp":"2024-09-13T17:17:51.364064+0000","objects_scrubbed":0,"log_size":4047,"log_dups_size":0,"ondisk_log_size":4047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:35:46.022668+0000","scrub_duration":0.028243739,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22743,"num_write_kb":291925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":197,"num_write":1329762,"num_write_kb":18444058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242925,"ondisk_log_size":242925,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19024977,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1583,"num_read_kb":206738,"num_write":4472,"num_write_kb":20527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38109184,"data_stored":38049954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4378,"ondisk_log_size":4378,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":1479870,"num_write_kb":19043750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270762,"ondisk_log_size":270762,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":19675678,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1366,"num_read_kb":209632,"num_write":4521,"num_write_kb":21162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":39411712,"data_stored":39351356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4388,"ondisk_log_size":4388,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575233,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":942784,"kb_used_data":13688,"kb_used_omap":1,"kb_used_meta":929086,"kb_avail":93429056,"statfs":{"total":96636764160,"available":95671353344,"internally_reserved":0,"allocated":14016512,"data_stored":13616965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":951384523},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":2.0699999999999998,"15min":1.2929999999999999},"min":{"1min":0.61499999999999999,"5min":0.23200000000000001,"15min":0.23100000000000001},"max":{"1min":1.3169999999999999,"5min":16.571999999999999,"15min":16.571999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":2.1110000000000002,"15min":1.385},"min":{"1min":0.60699999999999998,"5min":0.26600000000000001,"15min":0.20899999999999999},"max":{"1min":1.177,"5min":16.523,"15min":16.523},"last":0.999}]},{"osd":1,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":2.6899999999999999,"15min":1.5640000000000001},"min":{"1min":0.56799999999999995,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.294,"5min":59.911999999999999,"15min":59.911999999999999},"last":1.028},{"interface":"front","average":{"1min":1.004,"5min":2.7850000000000001,"15min":1.671},"min":{"1min":0.69699999999999995,"5min":0.28399999999999997,"15min":0.23000000000000001},"max":{"1min":1.2110000000000001,"5min":59.895000000000003,"15min":59.895000000000003},"last":1.0509999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.718,"15min":1.153},"min":{"1min":0.64100000000000001,"5min":0.28999999999999998,"15min":0.25700000000000001},"max":{"1min":1.143,"5min":16.023,"15min":16.023},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.851,"15min":1.2669999999999999},"min":{"1min":0.63300000000000001,"5min":0.249,"15min":0.249},"max":{"1min":1.397,"5min":18.734000000000002,"15min":18.734000000000002},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":2.0779999999999998,"15min":1.3120000000000001},"min":{"1min":0.69899999999999995,"5min":0.30199999999999999,"15min":0.28299999999999997},"max":{"1min":1.1359999999999999,"5min":23.225000000000001,"15min":23.225000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0549999999999999,"5min":2.319,"15min":1.4650000000000001},"min":{"1min":0.81399999999999995,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.47,"5min":19.709,"15min":19.709},"last":1.077}]},{"osd":4,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":2.133,"15min":1.3959999999999999},"min":{"1min":0.58799999999999997,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.2769999999999999,"5min":18.004999999999999,"15min":18.004999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":2.3690000000000002,"15min":1.48},"min":{"1min":0.66300000000000003,"5min":0.27300000000000002,"15min":0.245},"max":{"1min":1.1439999999999999,"5min":22.315999999999999,"15min":22.315999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":2.5169999999999999,"15min":1.5329999999999999},"min":{"1min":0.72499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.5429999999999999,"5min":37.302999999999997,"15min":37.302999999999997},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.02,"5min":1.8759999999999999,"15min":1.369},"min":{"1min":0.66300000000000003,"5min":0.33000000000000002,"15min":0.316},"max":{"1min":1.4330000000000001,"5min":16.867999999999999,"15min":16.867999999999999},"last":0.80400000000000005}]},{"osd":6,"last update":"Fri Sep 13 17:18:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.909,"15min":1.383},"min":{"1min":0.73499999999999999,"5min":0.28000000000000003,"15min":0.23000000000000001},"max":{"1min":1.355,"5min":18.335000000000001,"15min":18.335000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.089,"5min":2.085,"15min":1.397},"min":{"1min":0.76000000000000001,"5min":0.33200000000000002,"15min":0.28100000000000003},"max":{"1min":1.268,"5min":19.244,"15min":19.244},"last":0.97399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575230,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673216,"kb_used_data":11768,"kb_used_omap":1,"kb_used_meta":661438,"kb_avail":93698624,"statfs":{"total":96636764160,"available":95947390976,"internally_reserved":0,"allocated":12050432,"data_stored":11645255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":677312968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0329999999999999,"15min":0.89300000000000002},"min":{"1min":0.66500000000000004,"5min":0.222,"15min":0.222},"max":{"1min":1.1910000000000001,"5min":6.335,"15min":6.335},"last":1.095},{"interface":"front","average":{"1min":0.873,"5min":1.044,"15min":0.91700000000000004},"min":{"1min":0.61299999999999999,"5min":0.24199999999999999,"15min":0.20799999999999999},"max":{"1min":1.1619999999999999,"5min":4.907,"15min":5.9459999999999997},"last":1.0249999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.149,"15min":0.95499999999999996},"min":{"1min":0.59599999999999997,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.24,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":0.88},{"interface":"front","average":{"1min":0.85999999999999999,"5min":1.679,"15min":1.0960000000000001},"min":{"1min":0.66500000000000004,"5min":0.248,"15min":0.248},"max":{"1min":1.069,"5min":10.111000000000001,"15min":10.111000000000001},"last":1.1439999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.2270000000000001,"15min":0.98299999999999998},"min":{"1min":0.68600000000000005,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.248,"5min":15.638,"15min":15.638},"last":0.996},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.3680000000000001,"15min":1.095},"min":{"1min":0.77800000000000002,"5min":0.254,"15min":0.254},"max":{"1min":1.1719999999999999,"5min":19.442,"15min":19.442},"last":1.048}]},{"osd":3,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.3759999999999999,"15min":1.1319999999999999},"min":{"1min":0.70499999999999996,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":1.216,"5min":15.247999999999999,"15min":15.247999999999999},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.03,"5min":1.4730000000000001,"15min":1.167},"min":{"1min":0.79900000000000004,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.1759999999999999,"5min":15.179,"15min":15.179},"last":1.075}]},{"osd":4,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":1.4410000000000001,"15min":1.109},"min":{"1min":0.66200000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.1220000000000001,"5min":15.326000000000001,"15min":15.326000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.016,"5min":1.5229999999999999,"15min":1.1970000000000001},"min":{"1min":0.72399999999999998,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":1.278,"5min":15.026,"15min":15.026},"last":1.173}]},{"osd":5,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.494,"15min":1.246},"min":{"1min":0.88200000000000001,"5min":0.32500000000000001,"15min":0.28899999999999998},"max":{"1min":1.256,"5min":9.6080000000000005,"15min":11.566000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.325,"15min":1.1879999999999999},"min":{"1min":0.65500000000000003,"5min":0.308,"15min":0.308},"max":{"1min":1.2889999999999999,"5min":10.223000000000001,"15min":10.223000000000001},"last":0.97199999999999998}]},{"osd":7,"last update":"Fri Sep 13 17:18:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.462,"15min":1.238},"min":{"1min":0.84499999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.266,"5min":9.8960000000000008,"15min":13.112},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.5840000000000001,"15min":1.274},"min":{"1min":0.80800000000000005,"5min":0.33200000000000002,"15min":0.245},"max":{"1min":1.331,"5min":14.4,"15min":14.4},"last":0.89500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575231,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":957788,"kb_used_data":5012,"kb_used_omap":1,"kb_used_meta":952766,"kb_avail":93414052,"statfs":{"total":96636764160,"available":95655989248,"internally_reserved":0,"allocated":5132288,"data_stored":4730805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":975632838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1.99,"15min":1.2030000000000001},"min":{"1min":0.65900000000000003,"5min":0.215,"15min":0.215},"max":{"1min":1.129,"5min":27.952999999999999,"15min":27.952999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.052,"5min":1.617,"15min":1.1339999999999999},"min":{"1min":0.84199999999999997,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.319,"5min":23.619,"15min":23.619},"last":1.099}]},{"osd":1,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1.7749999999999999,"15min":1.1399999999999999},"min":{"1min":0.63800000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.1479999999999999,"5min":13.515000000000001,"15min":13.515000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":1.875,"15min":1.1890000000000001},"min":{"1min":0.628,"5min":0.25700000000000001,"15min":0.248},"max":{"1min":1.1020000000000001,"5min":35.186999999999998,"15min":35.186999999999998},"last":0.99299999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.694,"15min":1.3440000000000001},"min":{"1min":0.70099999999999996,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2310000000000001,"5min":11.609,"15min":51.18},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.395,"15min":1.296},"min":{"1min":0.61499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1759999999999999,"5min":8.282,"15min":48.756},"last":0.96599999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.8740000000000001,"15min":1.2809999999999999},"min":{"1min":0.69399999999999995,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1419999999999999,"5min":19.800000000000001,"15min":19.800000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.6679999999999999,"15min":1.258},"min":{"1min":0.76300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2270000000000001,"5min":9.4789999999999992,"15min":9.4789999999999992},"last":1.073}]},{"osd":4,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":2.4159999999999999,"15min":1.3839999999999999},"min":{"1min":0.56399999999999995,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.1879999999999999,"5min":22.361999999999998,"15min":22.361999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.999,"5min":2.6480000000000001,"15min":1.6439999999999999},"min":{"1min":0.753,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2669999999999999,"5min":28.062999999999999,"15min":31.843},"last":1.022}]},{"osd":6,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.403,"15min":1.1299999999999999},"min":{"1min":0.65000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.383,"5min":8.1910000000000007,"15min":8.1910000000000007},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.7310000000000001,"15min":1.2689999999999999},"min":{"1min":0.68799999999999994,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.3,"5min":13.031000000000001,"15min":13.031000000000001},"last":1.0509999999999999}]},{"osd":7,"last update":"Fri Sep 13 17:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.98,"15min":1.4219999999999999},"min":{"1min":0.73299999999999998,"5min":0.29499999999999998,"15min":0.28899999999999998},"max":{"1min":1.345,"5min":35.124000000000002,"15min":35.124000000000002},"last":1.159},{"interface":"front","average":{"1min":1.181,"5min":2.2730000000000001,"15min":1.552},"min":{"1min":0.91800000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.373,"5min":32.328000000000003,"15min":32.328000000000003},"last":1.1399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575231,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1011352,"kb_used_data":13968,"kb_used_omap":1,"kb_used_meta":997374,"kb_avail":93360488,"statfs":{"total":96636764160,"available":95601139712,"internally_reserved":0,"allocated":14303232,"data_stored":13890488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1021311430},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":1.6970000000000001,"15min":1.1659999999999999},"min":{"1min":0.60999999999999999,"5min":0.23899999999999999,"15min":0.20399999999999999},"max":{"1min":1.2290000000000001,"5min":18.841000000000001,"15min":18.841000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.694,"15min":1.2729999999999999},"min":{"1min":0.80700000000000005,"5min":0.25700000000000001,"15min":0.251},"max":{"1min":1.2290000000000001,"5min":20.312000000000001,"15min":20.312000000000001},"last":1.1319999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.623,"15min":1.1180000000000001},"min":{"1min":0.69299999999999995,"5min":0.27100000000000002,"15min":0.26100000000000001},"max":{"1min":1.262,"5min":27.242000000000001,"15min":27.242000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":2.1899999999999999,"15min":1.3220000000000001},"min":{"1min":0.66400000000000003,"5min":0.32200000000000001,"15min":0.26000000000000001},"max":{"1min":1.153,"5min":26.545999999999999,"15min":26.545999999999999},"last":0.97699999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.4510000000000001,"15min":1.105},"min":{"1min":0.73999999999999999,"5min":0.29199999999999998,"15min":0.29099999999999998},"max":{"1min":1.1559999999999999,"5min":10.205,"15min":11.803000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.022,"5min":1.534,"15min":1.228},"min":{"1min":0.748,"5min":0.29399999999999998,"15min":0.26200000000000001},"max":{"1min":1.24,"5min":13.972,"15min":13.972},"last":1.2589999999999999}]},{"osd":3,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":2.052,"15min":1.419},"min":{"1min":0.69199999999999995,"5min":0.32000000000000001,"15min":0.30499999999999999},"max":{"1min":1.1579999999999999,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.012,"5min":1.728,"15min":1.2909999999999999},"min":{"1min":0.73999999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.268,"5min":20.468,"15min":20.468},"last":1.0600000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.6919999999999999,"15min":1.1930000000000001},"min":{"1min":0.67800000000000005,"5min":0.27200000000000002,"15min":0.23499999999999999},"max":{"1min":1.214,"5min":14.031000000000001,"15min":14.031000000000001},"last":1.105},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.819,"15min":1.3640000000000001},"min":{"1min":0.69899999999999995,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.2330000000000001,"5min":15.044,"15min":15.044},"last":1.0249999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.464,"15min":1.1519999999999999},"min":{"1min":0.77500000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.3540000000000001,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":1.151},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.8140000000000001,"15min":1.3160000000000001},"min":{"1min":0.70699999999999996,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.262,"5min":14.131,"15min":14.131},"last":1.306}]},{"osd":7,"last update":"Fri Sep 13 17:18:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.661,"15min":1.4790000000000001},"min":{"1min":0.78500000000000003,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":1.296,"5min":20.154,"15min":24.518999999999998},"last":1.206},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.577,"15min":1.3600000000000001},"min":{"1min":0.73799999999999999,"5min":0.32600000000000001,"15min":0.312},"max":{"1min":1.427,"5min":15.209,"15min":15.209},"last":1.228}]}]},{"osd":3,"up_from":13,"seq":55834575231,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":958484,"kb_used_data":19404,"kb_used_omap":1,"kb_used_meta":939070,"kb_avail":93413356,"statfs":{"total":96636764160,"available":95655276544,"internally_reserved":0,"allocated":19869696,"data_stored":19459806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":961608136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":1.5509999999999999,"15min":0.96299999999999997},"min":{"1min":0.28999999999999998,"5min":0.191,"15min":0.17599999999999999},"max":{"1min":0.91700000000000004,"5min":19.327999999999999,"15min":19.327999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":1.7809999999999999,"15min":1.044},"min":{"1min":0.45600000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1,"5min":20.699999999999999,"15min":20.699999999999999},"last":1.0569999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":1.2949999999999999,"15min":0.90000000000000002},"min":{"1min":0.32200000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":1.048,"5min":16.100000000000001,"15min":16.100000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":1.2509999999999999,"15min":0.95499999999999996},"min":{"1min":0.32700000000000001,"5min":0.245,"15min":0.22900000000000001},"max":{"1min":1.121,"5min":10.417,"15min":10.417},"last":0.89200000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":1.2909999999999999,"15min":0.96799999999999997},"min":{"1min":0.40799999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.387,"5min":8.5950000000000006,"15min":8.5950000000000006},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.1339999999999999,"15min":0.96399999999999997},"min":{"1min":0.39000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.274,"5min":10.737,"15min":15.433999999999999},"last":0.95399999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":2.028,"15min":1.3320000000000001},"min":{"1min":0.56699999999999995,"5min":0.308,"15min":0.29599999999999999},"max":{"1min":1.3140000000000001,"5min":15.919,"15min":15.919},"last":1.202},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.6559999999999999,"15min":1.21},"min":{"1min":0.60899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5029999999999999,"5min":9.3819999999999997,"15min":9.3819999999999997},"last":0.84799999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.802,"15min":1.3009999999999999},"min":{"1min":0.67800000000000005,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.5900000000000001,"5min":31.187999999999999,"15min":31.187999999999999},"last":1.113},{"interface":"front","average":{"1min":1.097,"5min":2.077,"15min":1.383},"min":{"1min":0.63700000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.5580000000000001,"5min":22.951000000000001,"15min":22.951000000000001},"last":1.1699999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.849,"15min":1.3100000000000001},"min":{"1min":0.71599999999999997,"5min":0.34899999999999998,"15min":0.31900000000000001},"max":{"1min":1.619,"5min":12.567,"15min":12.567},"last":1.24},{"interface":"front","average":{"1min":1.141,"5min":1.95,"15min":1.4550000000000001},"min":{"1min":0.61799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5109999999999999,"5min":15.882,"15min":15.882},"last":1.3220000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:18:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":2.2589999999999999,"15min":1.5389999999999999},"min":{"1min":0.73699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6080000000000001,"5min":33.969000000000001,"15min":33.969000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":2.0510000000000002,"15min":1.458},"min":{"1min":0.63700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.5489999999999999,"5min":25.652999999999999,"15min":25.652999999999999},"last":1.3600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575231,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662284,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":661374,"kb_avail":93709556,"statfs":{"total":96636764160,"available":95958585344,"internally_reserved":0,"allocated":921600,"data_stored":527729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":677247430},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.89700000000000002,"15min":0.72599999999999998},"min":{"1min":0.53300000000000003,"5min":0.189,"15min":0.17399999999999999},"max":{"1min":0.91500000000000004,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.472,"15min":1.042},"min":{"1min":0.57699999999999996,"5min":0.17899999999999999,"15min":0.17899999999999999},"max":{"1min":1.1990000000000001,"5min":9.2729999999999997,"15min":9.2729999999999997},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":1.6899999999999999,"15min":1.069},"min":{"1min":0.58499999999999996,"5min":0.20200000000000001,"15min":0.20000000000000001},"max":{"1min":1.042,"5min":33.579999999999998,"15min":33.579999999999998},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.6459999999999999,"15min":1.127},"min":{"1min":0.63700000000000001,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":1.032,"5min":23.965,"15min":23.965},"last":1.127}]},{"osd":3,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":1.8520000000000001,"15min":1.3120000000000001},"min":{"1min":0.625,"5min":0.247,"15min":0.245},"max":{"1min":1.1499999999999999,"5min":11.837999999999999,"15min":22.760000000000002},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.89000000000000001,"5min":1.605,"15min":1.1379999999999999},"min":{"1min":0.70499999999999996,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":1.252,"5min":10.372,"15min":10.372},"last":0.97099999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.7569999999999999,"15min":1.298},"min":{"1min":0.79100000000000004,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.3069999999999999,"5min":20.673999999999999,"15min":20.673999999999999},"last":1.073},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.6579999999999999,"15min":1.282},"min":{"1min":0.86699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.506,"5min":16.5,"15min":16.5},"last":1.232}]},{"osd":5,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":2.3399999999999999,"15min":1.54},"min":{"1min":0.82099999999999995,"5min":0.31,"15min":0.27900000000000003},"max":{"1min":1.4299999999999999,"5min":21.722000000000001,"15min":21.722000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":2.097,"15min":1.4339999999999999},"min":{"1min":0.873,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.4470000000000001,"5min":29.620999999999999,"15min":29.620999999999999},"last":1.163}]},{"osd":6,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.6519999999999999,"15min":1.296},"min":{"1min":0.878,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":1.4870000000000001,"5min":21.882999999999999,"15min":21.882999999999999},"last":1.294},{"interface":"front","average":{"1min":1.1759999999999999,"5min":2.0299999999999998,"15min":1.478},"min":{"1min":0.83899999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.478,"5min":25.274999999999999,"15min":25.274999999999999},"last":1.3500000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:19:00 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":3.1190000000000002,"15min":1.871},"min":{"1min":0.98899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.587,"5min":43.649000000000001,"15min":43.649000000000001},"last":1.375},{"interface":"front","average":{"1min":1.232,"5min":3.0990000000000002,"15min":1.8580000000000001},"min":{"1min":1.054,"5min":0.40200000000000002,"15min":0.35899999999999999},"max":{"1min":1.4730000000000001,"5min":27.417999999999999,"15min":27.417999999999999},"last":1.4410000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575231,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798984,"kb_used_data":9024,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572856,"statfs":{"total":96636764160,"available":95818604544,"internally_reserved":0,"allocated":9240576,"data_stored":8832475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":808909254},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":1.7709999999999999,"15min":1.085},"min":{"1min":0.52200000000000002,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":1.155,"5min":15.81,"15min":15.81},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.6419999999999999,"15min":1.069},"min":{"1min":0.66600000000000004,"5min":0.188,"15min":0.17599999999999999},"max":{"1min":1.073,"5min":15.823,"15min":15.823},"last":0.70999999999999996}]},{"osd":2,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":1.3069999999999999,"15min":0.92300000000000004},"min":{"1min":0.51400000000000001,"5min":0.215,"15min":0.193},"max":{"1min":1.093,"5min":11.672000000000001,"15min":11.672000000000001},"last":0.627},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.2729999999999999,"15min":0.95999999999999996},"min":{"1min":0.55200000000000005,"5min":0.23699999999999999,"15min":0.22600000000000001},"max":{"1min":1.272,"5min":11.722,"15min":11.722},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.871,"15min":1.226},"min":{"1min":0.63100000000000001,"5min":0.24299999999999999,"15min":0.22},"max":{"1min":1.0840000000000001,"5min":16.507999999999999,"15min":16.507999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":1.3680000000000001,"15min":1.016},"min":{"1min":0.56799999999999995,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":1.073,"5min":12.047000000000001,"15min":12.047000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.925,"15min":1.2769999999999999},"min":{"1min":0.76600000000000001,"5min":0.27800000000000002,"15min":0.24399999999999999},"max":{"1min":1.657,"5min":22.234000000000002,"15min":22.234000000000002},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":2.0819999999999999,"15min":1.407},"min":{"1min":0.89200000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4450000000000001,"5min":20.448,"15min":20.448},"last":0.88600000000000001}]},{"osd":5,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.6180000000000001,"15min":1.2010000000000001},"min":{"1min":0.76500000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":1.522,"5min":15.788,"15min":15.788},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.01,"5min":2.0299999999999998,"15min":1.365},"min":{"1min":0.83999999999999997,"5min":0.312,"15min":0.312},"max":{"1min":1.403,"5min":24.763999999999999,"15min":24.763999999999999},"last":0.80400000000000005}]},{"osd":6,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.3959999999999999,"15min":1.1459999999999999},"min":{"1min":0.81899999999999995,"5min":0.255,"15min":0.255},"max":{"1min":1.629,"5min":12.019,"15min":12.019},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.8129999999999999,"15min":1.2769999999999999},"min":{"1min":0.84499999999999997,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4730000000000001,"5min":11.628,"15min":11.628},"last":0.90800000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:18:57 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.7769999999999999,"15min":1.409},"min":{"1min":0.81899999999999995,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.466,"5min":10.667,"15min":21.161000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.163,"5min":2.2989999999999999,"15min":1.5589999999999999},"min":{"1min":0.97499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.6870000000000001,"5min":29.323,"15min":29.323},"last":0.97399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575231,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799556,"kb_used_data":9596,"kb_used_omap":1,"kb_used_meta":789950,"kb_avail":93572284,"statfs":{"total":96636764160,"available":95818018816,"internally_reserved":0,"allocated":9826304,"data_stored":9422627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":808909256},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":1.377,"15min":0.91100000000000003},"min":{"1min":0.317,"5min":0.20000000000000001,"15min":0.189},"max":{"1min":0.89800000000000002,"5min":59.031999999999996,"15min":59.031999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":2.3100000000000001,"15min":1.286},"min":{"1min":0.437,"5min":0.19700000000000001,"15min":0.188},"max":{"1min":0.97199999999999998,"5min":57.186999999999998,"15min":57.186999999999998},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.123,"15min":0.877},"min":{"1min":0.371,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":1.1910000000000001,"5min":12.971,"15min":12.971},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":1.3500000000000001,"15min":0.93700000000000006},"min":{"1min":0.35999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":1.353,"5min":14.526,"15min":14.526},"last":0.64300000000000002}]},{"osd":3,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":2.5830000000000002,"15min":1.4359999999999999},"min":{"1min":0.38500000000000001,"5min":0.221,"15min":0.221},"max":{"1min":1.085,"5min":65.215000000000003,"15min":65.215000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.92900000000000005,"5min":1.827,"15min":1.153},"min":{"1min":0.41199999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.3089999999999999,"5min":62.722999999999999,"15min":62.722999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.889,"15min":1.2649999999999999},"min":{"1min":0.64100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.177,"5min":14.694000000000001,"15min":14.694000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.026,"5min":1.3149999999999999,"15min":1.073},"min":{"1min":0.77200000000000002,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":1.2949999999999999,"5min":10.036,"15min":10.036},"last":1.0109999999999999}]},{"osd":5,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.8009999999999999,"15min":1.2709999999999999},"min":{"1min":0.71599999999999997,"5min":0.34899999999999998,"15min":0.34599999999999997},"max":{"1min":1.2989999999999999,"5min":21.940999999999999,"15min":21.940999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.107,"5min":1.7969999999999999,"15min":1.3620000000000001},"min":{"1min":0.624,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.391,"5min":9.8200000000000003,"15min":9.8200000000000003},"last":1.145}]},{"osd":6,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.427,"15min":1.1599999999999999},"min":{"1min":0.60799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.4370000000000001,"5min":10.819000000000001,"15min":10.819000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.4990000000000001,"15min":1.2190000000000001},"min":{"1min":0.73499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4419999999999999,"5min":10.741,"15min":10.741},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Sep 13 17:18:49 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.772,"15min":1.367},"min":{"1min":0.67600000000000005,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":1.4710000000000001,"5min":32.975999999999999,"15min":32.975999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":2.5390000000000001,"15min":1.6819999999999999},"min":{"1min":0.754,"5min":0.41299999999999998,"15min":0.40300000000000002},"max":{"1min":1.399,"5min":32.069000000000003,"15min":32.069000000000003},"last":1.26}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4194942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":84646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8482816,"data_stored":8473926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7020544,"data_stored":7006776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7016448,"data_stored":7007322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":12583494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8400896,"data_stored":8389302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10555392,"data_stored":10542867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6440699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4296704,"data_stored":4287702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:19:31.973 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T17:19:32.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a2977b640 1 Processor -- start 2024-09-13T17:19:32.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a2977b640 1 -- start start 2024-09-13T17:19:32.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a2977b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7a2406b080 0x7f7a24151170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a2977b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7a241549f0 0x7f7a241516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a2977b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a24157910 con 0x7f7a240c46a0 2024-09-13T17:19:32.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a2977b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a24151bf0 con 0x7f7a241549f0 2024-09-13T17:19:32.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a2977b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a24153ed0 con 0x7f7a2406b080 2024-09-13T17:19:32.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a237fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7a241549f0 0x7f7a241516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a237fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7a241549f0 0x7f7a241516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39396/0 (socket says 172.21.15.39:39396) 2024-09-13T17:19:32.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a237fe640 1 -- 172.21.15.39:0/3186237055 learned_addr learned my addr 172.21.15.39:0/3186237055 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:32.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a227fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7a240c46a0 0x7f7a241542e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49452/0 (socket says 172.21.15.39:49452) 2024-09-13T17:19:32.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a22ffd640 1 --2- 172.21.15.39:0/3186237055 >> v2:172.21.15.84:3300/0 conn(0x7f7a2406b080 0x7f7a24151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a237fe640 1 -- 172.21.15.39:0/3186237055 >> v2:172.21.15.84:3300/0 conn(0x7f7a2406b080 msgr2=0x7f7a24151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:32.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a237fe640 1 --2- 172.21.15.39:0/3186237055 >> v2:172.21.15.84:3300/0 conn(0x7f7a2406b080 0x7f7a24151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a237fe640 1 -- 172.21.15.39:0/3186237055 >> v1:172.21.15.39:6789/0 conn(0x7f7a240c46a0 legacy=0x7f7a241542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a237fe640 1 -- 172.21.15.39:0/3186237055 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a24157bf0 con 0x7f7a241549f0 2024-09-13T17:19:32.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.127+0000 7f7a21ffb640 1 -- 172.21.15.39:0/3186237055 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792234399 0 0) 0x7f7a24157910 con 0x7f7a240c46a0 2024-09-13T17:19:32.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a237fe640 1 --2- 172.21.15.39:0/3186237055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 0x7f7a241516b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7a240c6680 tx=0x7f7a1c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5b8e070a286aae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:32.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a21ffb640 1 -- 172.21.15.39:0/3186237055 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0a5620 con 0x7f7a241549f0 2024-09-13T17:19:32.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a21ffb640 1 -- 172.21.15.39:0/3186237055 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c0a5780 con 0x7f7a241549f0 2024-09-13T17:19:32.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a21ffb640 1 -- 172.21.15.39:0/3186237055 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0a5b10 con 0x7f7a241549f0 2024-09-13T17:19:32.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 -- 172.21.15.39:0/3186237055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 msgr2=0x7f7a241516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 --2- 172.21.15.39:0/3186237055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 0x7f7a241516b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7a240c6680 tx=0x7f7a1c01c960 comp rx=0 tx=0).stop 2024-09-13T17:19:32.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 -- 172.21.15.39:0/3186237055 shutdown_connections 2024-09-13T17:19:32.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 --2- 172.21.15.39:0/3186237055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 0x7f7a241516b0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 --2- 172.21.15.39:0/3186237055 >> v2:172.21.15.84:3300/0 conn(0x7f7a2406b080 0x7f7a24151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 -- 172.21.15.39:0/3186237055 >> 172.21.15.39:0/3186237055 conn(0x7f7a240c32a0 msgr2=0x7f7a240c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:32.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 -- 172.21.15.39:0/3186237055 shutdown_connections 2024-09-13T17:19:32.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 -- 172.21.15.39:0/3186237055 wait complete. 2024-09-13T17:19:32.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 Processor -- start 2024-09-13T17:19:32.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 -- start start 2024-09-13T17:19:32.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a2406b080 0x7f7a240c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a240c46a0 0x7f7a240715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 0x7f7a241ef950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a240c1ca0 con 0x7f7a241549f0 2024-09-13T17:19:32.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a241f0380 con 0x7f7a2406b080 2024-09-13T17:19:32.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a2977b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a240c1770 con 0x7f7a240c46a0 2024-09-13T17:19:32.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a237fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 0x7f7a241ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a237fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 0x7f7a241ef950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39412/0 (socket says 172.21.15.39:39412) 2024-09-13T17:19:32.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a237fe640 1 -- 172.21.15.39:0/2622872666 learned_addr learned my addr 172.21.15.39:0/2622872666 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:32.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a22ffd640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a2406b080 0x7f7a240c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a227fc640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a240c46a0 0x7f7a240715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a237fe640 1 -- 172.21.15.39:0/2622872666 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a240c46a0 msgr2=0x7f7a240715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a237fe640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a240c46a0 0x7f7a240715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a237fe640 1 -- 172.21.15.39:0/2622872666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a2406b080 msgr2=0x7f7a240c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a237fe640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a2406b080 0x7f7a240c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.131+0000 7f7a237fe640 1 -- 172.21.15.39:0/2622872666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c009b30 con 0x7f7a241549f0 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a237fe640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 0x7f7a241ef950 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0af040 tx=0x7f7a1c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a0b7fe640 1 -- 172.21.15.39:0/2622872666 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c009510 con 0x7f7a241549f0 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a0b7fe640 1 -- 172.21.15.39:0/2622872666 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c0039a0 con 0x7f7a241549f0 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a0b7fe640 1 -- 172.21.15.39:0/2622872666 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c003d10 con 0x7f7a241549f0 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a240c29f0 con 0x7f7a241549f0 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a240c19c0 con 0x7f7a241549f0 2024-09-13T17:19:32.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a241f0530 con 0x7f7a241549f0 2024-09-13T17:19:32.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a0b7fe640 1 -- 172.21.15.39:0/2622872666 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a1c003e70 con 0x7f7a241549f0 2024-09-13T17:19:32.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a0b7fe640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a100666c0 0x7f7a10068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a0b7fe640 1 -- 172.21.15.39:0/2622872666 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f7a1c1200e0 con 0x7f7a241549f0 2024-09-13T17:19:32.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a22ffd640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a100666c0 0x7f7a10068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.135+0000 7f7a22ffd640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a100666c0 0x7f7a10068b80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7a2414e420 tx=0x7f7a18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:32.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.139+0000 7f7a0b7fe640 1 -- 172.21.15.39:0/2622872666 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7a241f0530 con 0x7f7a241549f0 2024-09-13T17:19:32.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.299+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f7a241f0530 con 0x7f7a241549f0 2024-09-13T17:19:32.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.307+0000 7f7a0b7fe640 1 -- 172.21.15.39:0/2622872666 <== mon.0 v2:172.21.15.39:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7a1c0ec300 con 0x7f7a241549f0 2024-09-13T17:19:32.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.307+0000 7f7a0b7fe640 1 -- 172.21.15.39:0/2622872666 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f7a241f0530 con 0x7f7a241549f0 2024-09-13T17:19:32.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a100666c0 msgr2=0x7f7a10068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a100666c0 0x7f7a10068b80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7a2414e420 tx=0x7f7a18008040 comp rx=0 tx=0).stop 2024-09-13T17:19:32.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 msgr2=0x7f7a241ef950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 0x7f7a241ef950 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0af040 tx=0x7f7a1c019700 comp rx=0 tx=0).stop 2024-09-13T17:19:32.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 shutdown_connections 2024-09-13T17:19:32.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f7a100666c0 0x7f7a10068b80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a241549f0 0x7f7a241ef950 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f7a240c46a0 0x7f7a240715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 --2- 172.21.15.39:0/2622872666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a2406b080 0x7f7a240c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 >> 172.21.15.39:0/2622872666 conn(0x7f7a240c32a0 msgr2=0x7f7a2406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:32.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 shutdown_connections 2024-09-13T17:19:32.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.311+0000 7f7a2977b640 1 -- 172.21.15.39:0/2622872666 wait complete. 2024-09-13T17:19:32.332 INFO:tasks.cephfs.filesystem:Destroying file system a and related pools 2024-09-13T17:19:32.332 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:19:32.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff952c5640 1 Processor -- start 2024-09-13T17:19:32.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff952c5640 1 -- start start 2024-09-13T17:19:32.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff952c5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff90151500 0x7eff900c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff952c5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff900c5a30 con 0x7eff900c47c0 2024-09-13T17:19:32.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff952c5640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff901576a0 con 0x7eff90071dd0 2024-09-13T17:19:32.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff952c5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff90157980 con 0x7eff90151500 2024-09-13T17:19:32.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7eff900c47c0 0x7eff90154780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49466/0 (socket says 172.21.15.39:49466) 2024-09-13T17:19:32.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8f7fe640 1 -- 172.21.15.39:0/1815427347 learned_addr learned my addr 172.21.15.39:0/1815427347 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:32.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8effd640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff90151500 0x7eff900c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477264426 0 0) 0x7eff900c5a30 con 0x7eff900c47c0 2024-09-13T17:19:32.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff70003530 con 0x7eff900c47c0 2024-09-13T17:19:32.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1689566112 0 0) 0x7eff70003530 con 0x7eff900c47c0 2024-09-13T17:19:32.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff90151500 msgr2=0x7eff900c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:32.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff90151500 0x7eff900c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 >> v1:172.21.15.84:6789/0 conn(0x7eff90071dd0 legacy=0x7eff90154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff90157c60 con 0x7eff900c47c0 2024-09-13T17:19:32.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7eff80001680 con 0x7eff900c47c0 2024-09-13T17:19:32.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 >> v1:172.21.15.39:6789/0 conn(0x7eff900c47c0 legacy=0x7eff90154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700039b0 0x7eff70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff90151500 0x7eff700047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff70004d10 0x7eff70005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff90157980 con 0x7eff70004d10 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7000bac0 con 0x7eff700039b0 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7000bc80 con 0x7eff90151500 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8f7fe640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff70004d10 0x7eff70005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8effd640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700039b0 0x7eff70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8f7fe640 1 -- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff90151500 msgr2=0x7eff700047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8f7fe640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff90151500 0x7eff700047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8f7fe640 1 -- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700039b0 msgr2=0x7eff70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8f7fe640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff700039b0 0x7eff70003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.443+0000 7eff8f7fe640 1 -- 172.21.15.39:0/1815427347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70003530 con 0x7eff70004d10 2024-09-13T17:19:32.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff8f7fe640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff70004d10 0x7eff70005190 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7eff8000c240 tx=0x7eff8001f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a122a59d4eca869 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:32.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80005ae0 con 0x7eff70004d10 2024-09-13T17:19:32.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7eff800ab070 con 0x7eff70004d10 2024-09-13T17:19:32.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff8dffb640 1 -- 172.21.15.39:0/1815427347 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80007070 con 0x7eff70004d10 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff952c5640 1 -- 172.21.15.39:0/1815427347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff70004d10 msgr2=0x7eff70005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff952c5640 1 --2- 172.21.15.39:0/1815427347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff70004d10 0x7eff70005190 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7eff8000c240 tx=0x7eff8001f3d0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff8f7fe640 1 -- 172.21.15.39:0/1815427347 reap_dead start 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff952c5640 1 -- 172.21.15.39:0/1815427347 shutdown_connections 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff952c5640 1 -- 172.21.15.39:0/1815427347 >> 172.21.15.39:0/1815427347 conn(0x7eff900ab530 msgr2=0x7eff900ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff952c5640 1 -- 172.21.15.39:0/1815427347 shutdown_connections 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff952c5640 1 -- 172.21.15.39:0/1815427347 wait complete. 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff952c5640 1 Processor -- start 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.447+0000 7eff952c5640 1 -- start start 2024-09-13T17:19:32.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff952c5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff70004d10 0x7eff90143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff952c5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff901507e0 0x7eff901421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff952c5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff9014eda0 0x7eff901426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff952c5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff90142c30 con 0x7eff901507e0 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff952c5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff90150d00 con 0x7eff70004d10 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff952c5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff901ec7a0 con 0x7eff9014eda0 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff901507e0 0x7eff901421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8e7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff901507e0 0x7eff901421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39440/0 (socket says 172.21.15.39:39440) 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8e7fc640 1 -- 172.21.15.39:0/120304491 learned_addr learned my addr 172.21.15.39:0/120304491 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8f7fe640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff9014eda0 0x7eff901426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8effd640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff70004d10 0x7eff90143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8e7fc640 1 -- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff9014eda0 msgr2=0x7eff901426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8e7fc640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff9014eda0 0x7eff901426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8e7fc640 1 -- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff70004d10 msgr2=0x7eff90143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8e7fc640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff70004d10 0x7eff90143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8e7fc640 1 -- 172.21.15.39:0/120304491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff901eca80 con 0x7eff901507e0 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8f7fe640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff9014eda0 0x7eff901426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8effd640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff70004d10 0x7eff90143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8e7fc640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff901507e0 0x7eff901421b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7eff8800c7a0 tx=0x7eff8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff877fe640 1 -- 172.21.15.39:0/120304491 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88003b80 con 0x7eff901507e0 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff877fe640 1 -- 172.21.15.39:0/120304491 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7eff88003d00 con 0x7eff901507e0 2024-09-13T17:19:32.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff877fe640 1 -- 172.21.15.39:0/120304491 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8800aab0 con 0x7eff901507e0 2024-09-13T17:19:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff901ecd60 con 0x7eff901507e0 2024-09-13T17:19:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff90150100 con 0x7eff901507e0 2024-09-13T17:19:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff900c7800 con 0x7eff901507e0 2024-09-13T17:19:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff877fe640 1 -- 172.21.15.39:0/120304491 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7eff8800ac10 con 0x7eff901507e0 2024-09-13T17:19:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff877fe640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7eff740666c0 0x7eff74068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff877fe640 1 -- 172.21.15.39:0/120304491 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7eff88085230 con 0x7eff901507e0 2024-09-13T17:19:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8effd640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7eff740666c0 0x7eff74068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.451+0000 7eff8effd640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7eff740666c0 0x7eff74068b80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7eff78009a20 tx=0x7eff78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:32.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.455+0000 7eff877fe640 1 -- 172.21.15.39:0/120304491 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7eff880513b0 con 0x7eff901507e0 2024-09-13T17:19:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.639+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff90071bb0 con 0x7eff901507e0 2024-09-13T17:19:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff877fe640 1 -- 172.21.15.39:0/120304491 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) v1 ==== 95+0+5937 (secure 0 0 0) 0x7eff880549d0 con 0x7eff901507e0 2024-09-13T17:19:32.648 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:19:32.648 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":207,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428801917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428801917},{"type":"v1","addr":"172.21.15.39:6835","nonce":428801917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":207}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:17:47.987831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7555":{"gid":7555,"name":"d","rank":0,"incarnation":203,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:19:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 207 2024-09-13T17:19:32.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7eff740666c0 msgr2=0x7eff74068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7eff740666c0 0x7eff74068b80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7eff78009a20 tx=0x7eff78008040 comp rx=0 tx=0).stop 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff901507e0 msgr2=0x7eff901421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff901507e0 0x7eff901421b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7eff8800c7a0 tx=0x7eff8800cc60 comp rx=0 tx=0).stop 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 shutdown_connections 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7eff740666c0 0x7eff74068b80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7eff9014eda0 0x7eff901426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff901507e0 0x7eff901421b0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 --2- 172.21.15.39:0/120304491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff70004d10 0x7eff90143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 >> 172.21.15.39:0/120304491 conn(0x7eff900ab530 msgr2=0x7eff90153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 shutdown_connections 2024-09-13T17:19:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.643+0000 7eff952c5640 1 -- 172.21.15.39:0/120304491 wait complete. 2024-09-13T17:19:32.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:19:32.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2c63a3640 1 Processor -- start 2024-09-13T17:19:32.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2c63a3640 1 -- start start 2024-09-13T17:19:32.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2c63a3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa2c00a4e70 0x7fa2c006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2c63a3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00c2780 con 0x7fa2c014e0c0 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2c63a3640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00c2a60 con 0x7fa2c006d300 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2c63a3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c00c2d40 con 0x7fa2c00a4e70 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2bffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa2c014e0c0 0x7fa2c00c1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49478/0 (socket says 172.21.15.39:49478) 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2bffff640 1 -- 172.21.15.39:0/3417121982 learned_addr learned my addr 172.21.15.39:0/3417121982 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2b7fff640 1 --2- 172.21.15.39:0/3417121982 >> v2:172.21.15.39:3300/0 conn(0x7fa2c00a4e70 0x7fa2c006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2bf7fe640 1 -- 172.21.15.39:0/3417121982 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138582200 0 0) 0x7fa2c00c2780 con 0x7fa2c014e0c0 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2bf7fe640 1 -- 172.21.15.39:0/3417121982 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac003530 con 0x7fa2c014e0c0 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2b7fff640 1 -- 172.21.15.39:0/3417121982 >> v1:172.21.15.84:6789/0 conn(0x7fa2c006d300 legacy=0x7fa2c00c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2bf7fe640 1 -- 172.21.15.39:0/3417121982 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3309835003 0 0) 0x7fa2ac003530 con 0x7fa2c014e0c0 2024-09-13T17:19:32.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2b7fff640 1 -- 172.21.15.39:0/3417121982 >> v1:172.21.15.39:6789/0 conn(0x7fa2c014e0c0 legacy=0x7fa2c00c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.779+0000 7fa2b7fff640 1 -- 172.21.15.39:0/3417121982 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00c3020 con 0x7fa2c00a4e70 2024-09-13T17:19:32.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2b7fff640 1 --2- 172.21.15.39:0/3417121982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c00a4e70 0x7fa2c006cdc0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097c0 tx=0x7fa2a801c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f8df7cac5002d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:32.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2bf7fe640 1 -- 172.21.15.39:0/3417121982 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0001680 con 0x7fa2c00a4e70 2024-09-13T17:19:32.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2bf7fe640 1 -- 172.21.15.39:0/3417121982 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa2a80a7050 con 0x7fa2c00a4e70 2024-09-13T17:19:32.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2bf7fe640 1 -- 172.21.15.39:0/3417121982 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80ab600 con 0x7fa2c00a4e70 2024-09-13T17:19:32.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2c63a3640 1 -- 172.21.15.39:0/3417121982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c00a4e70 msgr2=0x7fa2c006cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2c63a3640 1 --2- 172.21.15.39:0/3417121982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c00a4e70 0x7fa2c006cdc0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097c0 tx=0x7fa2a801c1e0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2c63a3640 1 -- 172.21.15.39:0/3417121982 shutdown_connections 2024-09-13T17:19:32.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2c63a3640 1 --2- 172.21.15.39:0/3417121982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c00a4e70 0x7fa2c006cdc0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2c63a3640 1 -- 172.21.15.39:0/3417121982 >> 172.21.15.39:0/3417121982 conn(0x7fa2c0069e10 msgr2=0x7fa2c006a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:32.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2c63a3640 1 -- 172.21.15.39:0/3417121982 shutdown_connections 2024-09-13T17:19:32.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.783+0000 7fa2c63a3640 1 -- 172.21.15.39:0/3417121982 wait complete. 2024-09-13T17:19:32.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 Processor -- start 2024-09-13T17:19:32.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 -- start start 2024-09-13T17:19:32.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2c006d300 0x7fa2c01543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c014e0c0 0x7fa2c00c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c01478a0 0x7fa2c01461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0072280 con 0x7fa2c01478a0 2024-09-13T17:19:32.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0145440 con 0x7fa2c014e0c0 2024-09-13T17:19:32.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0071b30 con 0x7fa2c006d300 2024-09-13T17:19:32.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c4940640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c01478a0 0x7fa2c01461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c4940640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c01478a0 0x7fa2c01461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39448/0 (socket says 172.21.15.39:39448) 2024-09-13T17:19:32.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c4940640 1 -- 172.21.15.39:0/2985209780 learned_addr learned my addr 172.21.15.39:0/2985209780 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:32.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2b7fff640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c014e0c0 0x7fa2c00c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2bffff640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2c006d300 0x7fa2c01543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c4940640 1 -- 172.21.15.39:0/2985209780 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2c006d300 msgr2=0x7fa2c01543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c4940640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2c006d300 0x7fa2c01543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c4940640 1 -- 172.21.15.39:0/2985209780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c014e0c0 msgr2=0x7fa2c00c13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c4940640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c014e0c0 0x7fa2c00c13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c4940640 1 -- 172.21.15.39:0/2985209780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a8009420 con 0x7fa2c01478a0 2024-09-13T17:19:32.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c4940640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c01478a0 0x7fa2c01461c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c8d0 tx=0x7fa2b800cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:32.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2bd7fa640 1 -- 172.21.15.39:0/2985209780 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003db0 con 0x7fa2c01478a0 2024-09-13T17:19:32.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2bd7fa640 1 -- 172.21.15.39:0/2985209780 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa2b8006b40 con 0x7fa2c01478a0 2024-09-13T17:19:32.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c0071df0 con 0x7fa2c01478a0 2024-09-13T17:19:32.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2bd7fa640 1 -- 172.21.15.39:0/2985209780 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8005070 con 0x7fa2c01478a0 2024-09-13T17:19:32.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c00c1fd0 con 0x7fa2c01478a0 2024-09-13T17:19:32.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c0071f80 con 0x7fa2c01478a0 2024-09-13T17:19:32.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.787+0000 7fa2bd7fa640 1 -- 172.21.15.39:0/2985209780 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa2b80051d0 con 0x7fa2c01478a0 2024-09-13T17:19:32.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.791+0000 7fa2bd7fa640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2a0066690 0x7fa2a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:32.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.791+0000 7fa2bd7fa640 1 -- 172.21.15.39:0/2985209780 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7fa2b8084940 con 0x7fa2c01478a0 2024-09-13T17:19:32.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.791+0000 7fa2bffff640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2a0066690 0x7fa2a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:32.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.791+0000 7fa2bffff640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2a0066690 0x7fa2a0068b50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00c2650 tx=0x7fa2b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:32.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.795+0000 7fa2bd7fa640 1 -- 172.21.15.39:0/2985209780 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa2b8050ac0 con 0x7fa2c01478a0 2024-09-13T17:19:32.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.987+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2c006cdc0 con 0x7fa2c01478a0 2024-09-13T17:19:32.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.987+0000 7fa2bd7fa640 1 -- 172.21.15.39:0/2985209780 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) v1 ==== 95+0+5937 (secure 0 0 0) 0x7fa2b8018030 con 0x7fa2c01478a0 2024-09-13T17:19:32.993 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:19:32.994 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":207,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428801917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428801917},{"type":"v1","addr":"172.21.15.39:6835","nonce":428801917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":207}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:47:59.023080+0000","modified":"2024-09-13T17:17:47.987831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7555},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7555":{"gid":7555,"name":"d","rank":0,"incarnation":203,"state":"up:active","state_seq":18,"addr":"172.21.15.84:6835/3465334740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3465334740},{"type":"v1","addr":"172.21.15.84:6835","nonce":3465334740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"a","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1},{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:19:32.994 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 207 2024-09-13T17:19:32.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2a0066690 msgr2=0x7fa2a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2a0066690 0x7fa2a0068b50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00c2650 tx=0x7fa2b0009040 comp rx=0 tx=0).stop 2024-09-13T17:19:32.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c01478a0 msgr2=0x7fa2c01461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:32.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c01478a0 0x7fa2c01461c0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c8d0 tx=0x7fa2b800cd90 comp rx=0 tx=0).stop 2024-09-13T17:19:32.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 shutdown_connections 2024-09-13T17:19:32.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fa2a0066690 0x7fa2a0068b50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c014e0c0 0x7fa2c00c13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2c01478a0 0x7fa2c01461c0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 --2- 172.21.15.39:0/2985209780 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fa2c006d300 0x7fa2c01543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:32.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 >> 172.21.15.39:0/2985209780 conn(0x7fa2c0069e10 msgr2=0x7fa2c014a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:32.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 shutdown_connections 2024-09-13T17:19:32.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:32.991+0000 7fa2c63a3640 1 -- 172.21.15.39:0/2985209780 wait complete. 2024-09-13T17:19:33.014 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:19:33.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbea3d640 1 Processor -- start 2024-09-13T17:19:33.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbea3d640 1 -- start start 2024-09-13T17:19:33.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbea3d640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9fb80a4e70 0x7f9fb806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbea3d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb806d300 0x7f9fb80c4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbea3d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb80c2780 con 0x7f9fb814e0c0 2024-09-13T17:19:33.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbea3d640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb80c2a60 con 0x7f9fb80a4e70 2024-09-13T17:19:33.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbea3d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c2d40 con 0x7f9fb806d300 2024-09-13T17:19:33.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb7fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9fb814e0c0 0x7f9fb80c1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49480/0 (socket says 172.21.15.39:49480) 2024-09-13T17:19:33.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb7fff640 1 -- 172.21.15.39:0/1640506830 learned_addr learned my addr 172.21.15.39:0/1640506830 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:33.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb77fe640 1 --2- 172.21.15.39:0/1640506830 >> v2:172.21.15.84:3300/0 conn(0x7f9fb80a4e70 0x7f9fb806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbcfda640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb806d300 0x7f9fb80c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960125827 0 0) 0x7f9fb80c2780 con 0x7f9fb814e0c0 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa0003530 con 0x7f9fb814e0c0 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2048311492 0 0) 0x7f9fa0003530 con 0x7f9fb814e0c0 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb806d300 msgr2=0x7f9fb80c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb806d300 0x7f9fb80c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 >> v2:172.21.15.84:3300/0 conn(0x7f9fb80a4e70 msgr2=0x7f9fb806cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 --2- 172.21.15.39:0/1640506830 >> v2:172.21.15.84:3300/0 conn(0x7f9fb80a4e70 0x7f9fb806cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb80c3020 con 0x7f9fb814e0c0 2024-09-13T17:19:33.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f9fa8001680 con 0x7f9fb814e0c0 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 >> v1:172.21.15.39:6789/0 conn(0x7f9fb814e0c0 legacy=0x7f9fb80c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb806d300 0x7f9fa0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa0003f00 0x7f9fa00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fa0004d10 0x7f9fa0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c2a60 con 0x7f9fa0004d10 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c2d40 con 0x7f9fa0003f00 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa000b9e0 con 0x7f9fb806d300 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbcfda640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fa0004d10 0x7f9fa0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb7fff640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb806d300 0x7f9fa0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbcfda640 1 -- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb806d300 msgr2=0x7f9fa0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:33.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbcfda640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb806d300 0x7f9fa0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbcfda640 1 -- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa0003f00 msgr2=0x7f9fa00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:33.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbcfda640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fa0003f00 0x7f9fa00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fbcfda640 1 -- 172.21.15.39:0/1640506830 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa0003530 con 0x7f9fa0004d10 2024-09-13T17:19:33.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.127+0000 7f9fb7fff640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb806d300 0x7f9fa0003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:19:33.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbcfda640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fa0004d10 0x7f9fa0005190 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009bd0 tx=0x7f9fb001ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b486a741168d13a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:33.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00a7e80 con 0x7f9fa0004d10 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9fb000f040 con 0x7f9fa0004d10 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fb6ffd640 1 -- 172.21.15.39:0/1640506830 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00af680 con 0x7f9fa0004d10 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1640506830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fa0004d10 msgr2=0x7f9fa0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbea3d640 1 --2- 172.21.15.39:0/1640506830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fa0004d10 0x7f9fa0005190 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009bd0 tx=0x7f9fb001ee70 comp rx=0 tx=0).stop 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbcfda640 1 -- 172.21.15.39:0/1640506830 reap_dead start 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1640506830 shutdown_connections 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1640506830 >> 172.21.15.39:0/1640506830 conn(0x7f9fb8069e10 msgr2=0x7f9fb806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1640506830 shutdown_connections 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1640506830 wait complete. 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbea3d640 1 Processor -- start 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.131+0000 7f9fbea3d640 1 -- start start 2024-09-13T17:19:33.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbea3d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fb80ab270 0x7f9fb813de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbea3d640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb80c12c0 0x7f9fb813e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbea3d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb813e8c0 0x7f9fb813cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbea3d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb813eeb0 con 0x7f9fb80ab270 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbea3d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb813c890 con 0x7f9fb813e8c0 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbea3d640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c0100 con 0x7f9fb80c12c0 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fb80ab270 0x7f9fb813de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fb80ab270 0x7f9fb813de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39468/0 (socket says 172.21.15.39:39468) 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb7fff640 1 -- 172.21.15.39:0/1523340421 learned_addr learned my addr 172.21.15.39:0/1523340421 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbcfda640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb813e8c0 0x7f9fb813cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb77fe640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb80c12c0 0x7f9fb813e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb7fff640 1 -- 172.21.15.39:0/1523340421 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb80c12c0 msgr2=0x7f9fb813e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb7fff640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb80c12c0 0x7f9fb813e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb7fff640 1 -- 172.21.15.39:0/1523340421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb813e8c0 msgr2=0x7f9fb813cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb7fff640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb813e8c0 0x7f9fb813cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb7fff640 1 -- 172.21.15.39:0/1523340421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb814e620 con 0x7f9fb80ab270 2024-09-13T17:19:33.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb7fff640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fb80ab270 0x7f9fb813de40 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8011860 tx=0x7f9fa8011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:33.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb4ff9640 1 -- 172.21.15.39:0/1523340421 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800fb60 con 0x7f9fb80ab270 2024-09-13T17:19:33.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb4ff9640 1 -- 172.21.15.39:0/1523340421 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9fa800fcc0 con 0x7f9fb80ab270 2024-09-13T17:19:33.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb4ff9640 1 -- 172.21.15.39:0/1523340421 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8015670 con 0x7f9fb80ab270 2024-09-13T17:19:33.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb80c0350 con 0x7f9fb80ab270 2024-09-13T17:19:33.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb80c06e0 con 0x7f9fb80ab270 2024-09-13T17:19:33.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb80c2780 con 0x7f9fb80ab270 2024-09-13T17:19:33.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb4ff9640 1 -- 172.21.15.39:0/1523340421 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9fa80157d0 con 0x7f9fb80ab270 2024-09-13T17:19:33.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb4ff9640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa40666c0 0x7f9fa4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb4ff9640 1 -- 172.21.15.39:0/1523340421 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f9fa80891e0 con 0x7f9fb80ab270 2024-09-13T17:19:33.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb77fe640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa40666c0 0x7f9fa4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.135+0000 7f9fb77fe640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa40666c0 0x7f9fa4068b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0047c0 tx=0x7f9fac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:33.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.139+0000 7f9fb4ff9640 1 -- 172.21.15.39:0/1523340421 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9fa80553e0 con 0x7f9fb80ab270 2024-09-13T17:19:33.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.299+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fb80c2440 con 0x7f9fb80ab270 2024-09-13T17:19:33.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.303+0000 7f9fb4ff9640 1 -- 172.21.15.39:0/1523340421 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+25553 (secure 0 0 0) 0x7f9fa8058a00 con 0x7f9fb80ab270 2024-09-13T17:19:33.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:19:33.309 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":64,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T17:17:44.963769+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"a_metadata","create_time":"2024-09-13T16:47:57.579177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"a"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"a_data","create_time":"2024-09-13T16:47:58.000542+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"a"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.27","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6835/980326904":"2024-09-14T17:17:44.963677+0000","172.21.15.39:6834/980326904":"2024-09-14T17:17:44.963677+0000","172.21.15.84:6835/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6834/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.39:6834/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6837/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6835/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.84:6834/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.39:6836/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.39:6835/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.84:6835/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.39:6835/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.84:6834/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.39:6834/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.39:6834/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.39:6836/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.84:6832/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.84:6835/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.84:6832/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6836/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.84:6834/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.39:6835/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.84:6832/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6834/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6835/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.84:6834/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6835/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.39:6835/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6836/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6833/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.39:6835/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.39:6837/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6834/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6835/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6834/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.39:6835/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1750130579":"2024-09-14T17:09:29.828673+0000","172.21.15.39:6834/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.84:6832/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.39:6834/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.39:6837/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6836/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6835/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6834/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.84:6835/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.84:6833/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.84:6833/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6835/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.39:6834/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.84:6832/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6833/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6834/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6835/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6836/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6834/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6833/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6835/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6834/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.39:6834/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6835/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6836/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.39:6835/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.84:6833/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6836/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6837/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.84:6834/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.84:6834/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.84:6835/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.39:6837/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.39:6837/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6835/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.84:6832/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6835/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/1750130579":"2024-09-14T17:09:29.828673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:19:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa40666c0 msgr2=0x7f9fa4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa40666c0 0x7f9fa4068b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0047c0 tx=0x7f9fac008040 comp rx=0 tx=0).stop 2024-09-13T17:19:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fb80ab270 msgr2=0x7f9fb813de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fb80ab270 0x7f9fb813de40 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8011860 tx=0x7f9fa8011d20 comp rx=0 tx=0).stop 2024-09-13T17:19:33.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 shutdown_connections 2024-09-13T17:19:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f9fa40666c0 0x7f9fa4068b80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fb813e8c0 0x7f9fb813cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f9fb80c12c0 0x7f9fb813e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 --2- 172.21.15.39:0/1523340421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fb80ab270 0x7f9fb813de40 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 >> 172.21.15.39:0/1523340421 conn(0x7f9fb8069e10 msgr2=0x7f9fb806bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 shutdown_connections 2024-09-13T17:19:33.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.307+0000 7f9fbea3d640 1 -- 172.21.15.39:0/1523340421 wait complete. 2024-09-13T17:19:33.329 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail a 2024-09-13T17:19:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02bce3b640 1 Processor -- start 2024-09-13T17:19:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02bce3b640 1 -- start start 2024-09-13T17:19:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02bce3b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f02b8151500 0x7f02b80c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02bce3b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8071dd0 0x7f02b8154070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02bce3b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f02b80c47c0 0x7f02b81545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02bce3b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b80c5a30 con 0x7f02b8151500 2024-09-13T17:19:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02bce3b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b81576a0 con 0x7f02b80c47c0 2024-09-13T17:19:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02bce3b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b8157980 con 0x7f02b8071dd0 2024-09-13T17:19:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b659c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f02b8151500 0x7f02b80c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b5d9b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8071dd0 0x7f02b8154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b659c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f02b8151500 0x7f02b80c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39478/0 (socket says 172.21.15.39:39478) 2024-09-13T17:19:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b659c640 1 -- 172.21.15.39:0/1508287765 learned_addr learned my addr 172.21.15.39:0/1508287765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b6d9d640 1 --2- 172.21.15.39:0/1508287765 >> v2:172.21.15.84:3300/0 conn(0x7f02b80c47c0 0x7f02b81545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b659c640 1 -- 172.21.15.39:0/1508287765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8071dd0 msgr2=0x7f02b8154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:33.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b659c640 1 --2- 172.21.15.39:0/1508287765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8071dd0 0x7f02b8154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b659c640 1 -- 172.21.15.39:0/1508287765 >> v2:172.21.15.84:3300/0 conn(0x7f02b80c47c0 msgr2=0x7f02b81545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:33.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b659c640 1 --2- 172.21.15.39:0/1508287765 >> v2:172.21.15.84:3300/0 conn(0x7f02b80c47c0 0x7f02b81545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.443+0000 7f02b659c640 1 -- 172.21.15.39:0/1508287765 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b8157c60 con 0x7f02b8151500 2024-09-13T17:19:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02b659c640 1 --2- 172.21.15.39:0/1508287765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8151500 0x7f02b80c4280 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f02b80c6260 tx=0x7f02a801cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac343bda81d329f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02b559a640 1 -- 172.21.15.39:0/1508287765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a80ab640 con 0x7f02b8151500 2024-09-13T17:19:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02b559a640 1 -- 172.21.15.39:0/1508287765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f02a80a7070 con 0x7f02b8151500 2024-09-13T17:19:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02b559a640 1 -- 172.21.15.39:0/1508287765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a80ab9e0 con 0x7f02b8151500 2024-09-13T17:19:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02bce3b640 1 -- 172.21.15.39:0/1508287765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8151500 msgr2=0x7f02b80c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02bce3b640 1 --2- 172.21.15.39:0/1508287765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8151500 0x7f02b80c4280 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f02b80c6260 tx=0x7f02a801cec0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02bce3b640 1 -- 172.21.15.39:0/1508287765 shutdown_connections 2024-09-13T17:19:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02bce3b640 1 --2- 172.21.15.39:0/1508287765 >> v2:172.21.15.84:3300/0 conn(0x7f02b80c47c0 0x7f02b81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02bce3b640 1 --2- 172.21.15.39:0/1508287765 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8071dd0 0x7f02b8154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02bce3b640 1 --2- 172.21.15.39:0/1508287765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b8151500 0x7f02b80c4280 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02bce3b640 1 -- 172.21.15.39:0/1508287765 >> 172.21.15.39:0/1508287765 conn(0x7f02b80ab530 msgr2=0x7f02b80ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02bce3b640 1 -- 172.21.15.39:0/1508287765 shutdown_connections 2024-09-13T17:19:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.447+0000 7f02bce3b640 1 -- 172.21.15.39:0/1508287765 wait complete. 2024-09-13T17:19:33.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 Processor -- start 2024-09-13T17:19:33.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 -- start start 2024-09-13T17:19:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02b8071dd0 0x7f02b81561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b80c47c0 0x7f02b81ecc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8151500 0x7f02b81ed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b81571e0 con 0x7f02b80c47c0 2024-09-13T17:19:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b80c0af0 con 0x7f02b8071dd0 2024-09-13T17:19:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b81ed700 con 0x7f02b8151500 2024-09-13T17:19:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b80c47c0 0x7f02b81ecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b5d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b80c47c0 0x7f02b81ecc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39492/0 (socket says 172.21.15.39:39492) 2024-09-13T17:19:33.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b5d9b640 1 -- 172.21.15.39:0/2366569139 learned_addr learned my addr 172.21.15.39:0/2366569139 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b6d9d640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8151500 0x7f02b81ed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b659c640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02b8071dd0 0x7f02b81561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b5d9b640 1 -- 172.21.15.39:0/2366569139 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8151500 msgr2=0x7f02b81ed1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b5d9b640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8151500 0x7f02b81ed1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b5d9b640 1 -- 172.21.15.39:0/2366569139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02b8071dd0 msgr2=0x7f02b81561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b5d9b640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02b8071dd0 0x7f02b81561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b5d9b640 1 -- 172.21.15.39:0/2366569139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a80094d0 con 0x7f02b80c47c0 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b6d9d640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8151500 0x7f02b81ed1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02b5d9b640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b80c47c0 0x7f02b81ecc80 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00e8b0 tx=0x7f02ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02a6ffd640 1 -- 172.21.15.39:0/2366569139 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00c940 con 0x7f02b80c47c0 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02a6ffd640 1 -- 172.21.15.39:0/2366569139 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f02ac00caa0 con 0x7f02b80c47c0 2024-09-13T17:19:33.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02a6ffd640 1 -- 172.21.15.39:0/2366569139 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00ce50 con 0x7f02b80c47c0 2024-09-13T17:19:33.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b81ec780 con 0x7f02b80c47c0 2024-09-13T17:19:33.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b815d910 con 0x7f02b80c47c0 2024-09-13T17:19:33.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b80c4280 con 0x7f02b80c47c0 2024-09-13T17:19:33.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.451+0000 7f02a6ffd640 1 -- 172.21.15.39:0/2366569139 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f02ac013470 con 0x7f02b80c47c0 2024-09-13T17:19:33.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.455+0000 7f02a6ffd640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0298066690 0x7f0298068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.455+0000 7f02a6ffd640 1 -- 172.21.15.39:0/2366569139 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 10077+0+0 (secure 0 0 0) 0x7f02ac085530 con 0x7f02b80c47c0 2024-09-13T17:19:33.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.455+0000 7f02b659c640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0298066690 0x7f0298068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.455+0000 7f02b659c640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0298066690 0x7f0298068b50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f02a80af000 tx=0x7f02a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:33.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.459+0000 7f02a6ffd640 1 -- 172.21.15.39:0/2366569139 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f02ac051740 con 0x7f02b80c47c0 2024-09-13T17:19:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.643+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "a"} v 0) v1 -- 0x7f02b8071bb0 con 0x7f02b80c47c0 2024-09-13T17:19:33.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.659+0000 7f02a6ffd640 1 -- 172.21.15.39:0/2366569139 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "a"}]=0 a marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v208) v1 ==== 151+0+0 (secure 0 0 0) 0x7f02ac021080 con 0x7f02b80c47c0 2024-09-13T17:19:33.665 INFO:teuthology.orchestra.run.smithi039.stderr:a marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:19:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0298066690 msgr2=0x7f0298068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0298066690 0x7f0298068b50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f02a80af000 tx=0x7f02a8008040 comp rx=0 tx=0).stop 2024-09-13T17:19:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b80c47c0 msgr2=0x7f02b81ecc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b80c47c0 0x7f02b81ecc80 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00e8b0 tx=0x7f02ac00ed70 comp rx=0 tx=0).stop 2024-09-13T17:19:33.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 shutdown_connections 2024-09-13T17:19:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f0298066690 0x7f0298068b50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f02b8151500 0x7f02b81ed1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02b80c47c0 0x7f02b81ecc80 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 --2- 172.21.15.39:0/2366569139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02b8071dd0 0x7f02b81561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 >> 172.21.15.39:0/2366569139 conn(0x7f02b80ab530 msgr2=0x7f02b814ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 shutdown_connections 2024-09-13T17:19:33.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.663+0000 7f02bce3b640 1 -- 172.21.15.39:0/2366569139 wait complete. 2024-09-13T17:19:33.684 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm a --yes-i-really-mean-it 2024-09-13T17:19:33.698 INFO:tasks.ceph.mds.d.smithi084.stderr: -2569> 2024-09-13T17:17:51.320+0000 7f7f5de86640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:19:33.740 INFO:tasks.ceph.mds.d.smithi084.stdout:starting mds.d at 2024-09-13T17:19:33.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.791+0000 7f72d9481640 1 Processor -- start 2024-09-13T17:19:33.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.791+0000 7f72d9481640 1 -- start start 2024-09-13T17:19:33.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f72d41516d0 0x7f72d40c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d406ddd0 0x7f72d4153e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f72d40c7cf0 0x7f72d41543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d4071d80 con 0x7f72d40c7cf0 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d406ec80 con 0x7f72d41516d0 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40c0e60 con 0x7f72d406ddd0 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f72d40c7cf0 0x7f72d41543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f72d40c7cf0 0x7f72d41543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39494/0 (socket says 172.21.15.39:39494) 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d37fe640 1 -- 172.21.15.39:0/2113190316 learned_addr learned my addr 172.21.15.39:0/2113190316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72cbfff640 1 --2- 172.21.15.39:0/2113190316 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d406ddd0 0x7f72d4153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d2ffd640 1 --2- 172.21.15.39:0/2113190316 >> v2:172.21.15.84:3300/0 conn(0x7f72d41516d0 0x7f72d40c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d37fe640 1 -- 172.21.15.39:0/2113190316 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d406ddd0 msgr2=0x7f72d4153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d37fe640 1 --2- 172.21.15.39:0/2113190316 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d406ddd0 0x7f72d4153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d37fe640 1 -- 172.21.15.39:0/2113190316 >> v2:172.21.15.84:3300/0 conn(0x7f72d41516d0 msgr2=0x7f72d40c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:33.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d37fe640 1 --2- 172.21.15.39:0/2113190316 >> v2:172.21.15.84:3300/0 conn(0x7f72d41516d0 0x7f72d40c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d37fe640 1 -- 172.21.15.39:0/2113190316 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40c1100 con 0x7f72d40c7cf0 2024-09-13T17:19:33.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d2ffd640 1 --2- 172.21.15.39:0/2113190316 >> v2:172.21.15.84:3300/0 conn(0x7f72d41516d0 0x7f72d40c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:19:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d37fe640 1 --2- 172.21.15.39:0/2113190316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d40c7cf0 0x7f72d41543d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f72d40c53b0 tx=0x7f72cc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124aa31dfb0bd516 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d27fc640 1 -- 172.21.15.39:0/2113190316 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0a56d0 con 0x7f72d40c7cf0 2024-09-13T17:19:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d27fc640 1 -- 172.21.15.39:0/2113190316 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f72cc0a5830 con 0x7f72d40c7cf0 2024-09-13T17:19:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d27fc640 1 -- 172.21.15.39:0/2113190316 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0a5be0 con 0x7f72d40c7cf0 2024-09-13T17:19:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 -- 172.21.15.39:0/2113190316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d40c7cf0 msgr2=0x7f72d41543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 --2- 172.21.15.39:0/2113190316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d40c7cf0 0x7f72d41543d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f72d40c53b0 tx=0x7f72cc01c960 comp rx=0 tx=0).stop 2024-09-13T17:19:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 -- 172.21.15.39:0/2113190316 shutdown_connections 2024-09-13T17:19:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 --2- 172.21.15.39:0/2113190316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d40c7cf0 0x7f72d41543d0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 --2- 172.21.15.39:0/2113190316 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d406ddd0 0x7f72d4153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 --2- 172.21.15.39:0/2113190316 >> v2:172.21.15.84:3300/0 conn(0x7f72d41516d0 0x7f72d40c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 -- 172.21.15.39:0/2113190316 >> 172.21.15.39:0/2113190316 conn(0x7f72d406c020 msgr2=0x7f72d406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 -- 172.21.15.39:0/2113190316 shutdown_connections 2024-09-13T17:19:33.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.795+0000 7f72d9481640 1 -- 172.21.15.39:0/2113190316 wait complete. 2024-09-13T17:19:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 Processor -- start 2024-09-13T17:19:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 -- start start 2024-09-13T17:19:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d406ddd0 0x7f72d4155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d40c7cf0 0x7f72d406b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72d41516d0 0x7f72d4156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d4155950 con 0x7f72d406ddd0 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d4152f70 con 0x7f72d41516d0 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d4153250 con 0x7f72d40c7cf0 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d406ddd0 0x7f72d4155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d2ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d406ddd0 0x7f72d4155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39502/0 (socket says 172.21.15.39:39502) 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d2ffd640 1 -- 172.21.15.39:0/625705498 learned_addr learned my addr 172.21.15.39:0/625705498 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72cbfff640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d40c7cf0 0x7f72d406b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d2ffd640 1 -- 172.21.15.39:0/625705498 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d40c7cf0 msgr2=0x7f72d406b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d2ffd640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d40c7cf0 0x7f72d406b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d37fe640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72d41516d0 0x7f72d4156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d2ffd640 1 -- 172.21.15.39:0/625705498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72d41516d0 msgr2=0x7f72d4156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d2ffd640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72d41516d0 0x7f72d4156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d2ffd640 1 -- 172.21.15.39:0/625705498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc009b30 con 0x7f72d406ddd0 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d37fe640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72d41516d0 0x7f72d4156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d2ffd640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d406ddd0 0x7f72d4155410 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f72c400d800 tx=0x7f72c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72cb7fe640 1 -- 172.21.15.39:0/625705498 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c40096c0 con 0x7f72d406ddd0 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72cb7fe640 1 -- 172.21.15.39:0/625705498 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f72c4009820 con 0x7f72d406ddd0 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d406bcb0 con 0x7f72d406ddd0 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72cb7fe640 1 -- 172.21.15.39:0/625705498 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4009bf0 con 0x7f72d406ddd0 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d4145f90 con 0x7f72d406ddd0 2024-09-13T17:19:33.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d40c5410 con 0x7f72d406ddd0 2024-09-13T17:19:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72cb7fe640 1 -- 172.21.15.39:0/625705498 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72c4009d50 con 0x7f72d406ddd0 2024-09-13T17:19:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.799+0000 7f72cb7fe640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f72b4066690 0x7f72b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.803+0000 7f72cb7fe640 1 -- 172.21.15.39:0/625705498 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 10163+0+0 (secure 0 0 0) 0x7f72c4085a70 con 0x7f72d406ddd0 2024-09-13T17:19:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.803+0000 7f72cbfff640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f72b4066690 0x7f72b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:33.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.803+0000 7f72cbfff640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f72b4066690 0x7f72b4068b50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f72d4156ea0 tx=0x7f72bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:33.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:33.807+0000 7f72cb7fe640 1 -- 172.21.15.39:0/625705498 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f72c4051c20 con 0x7f72d406ddd0 2024-09-13T17:19:34.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.003+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "a", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f72d40a71c0 con 0x7f72d406ddd0 2024-09-13T17:19:34.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.667+0000 7f72cb7fe640 1 -- 172.21.15.39:0/625705498 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "a", "yes_i_really_mean_it": true}]=0 v209) v1 ==== 99+0+0 (secure 0 0 0) 0x7f72c4055240 con 0x7f72d406ddd0 2024-09-13T17:19:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f72b4066690 msgr2=0x7f72b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f72b4066690 0x7f72b4068b50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f72d4156ea0 tx=0x7f72bc008040 comp rx=0 tx=0).stop 2024-09-13T17:19:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d406ddd0 msgr2=0x7f72d4155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d406ddd0 0x7f72d4155410 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f72c400d800 tx=0x7f72c400dcc0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 shutdown_connections 2024-09-13T17:19:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f72b4066690 0x7f72b4068b50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72d41516d0 0x7f72d4156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f72d40c7cf0 0x7f72d406b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 --2- 172.21.15.39:0/625705498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72d406ddd0 0x7f72d4155410 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 >> 172.21.15.39:0/625705498 conn(0x7f72d406c020 msgr2=0x7f72d414af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 shutdown_connections 2024-09-13T17:19:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.671+0000 7f72d9481640 1 -- 172.21.15.39:0/625705498 wait complete. 2024-09-13T17:19:34.700 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm a_metadata a_metadata --yes-i-really-really-mean-it 2024-09-13T17:19:34.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.835+0000 7f02a36f9640 1 Processor -- start 2024-09-13T17:19:34.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.835+0000 7f02a36f9640 1 -- start start 2024-09-13T17:19:34.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.835+0000 7f02a36f9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f029c0c7190 0x7f029c0c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:34.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.835+0000 7f02a36f9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029c06de90 con 0x7f029c071f00 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.835+0000 7f02a36f9640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029c06f2f0 con 0x7f029c06edf0 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a36f9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0c6170 con 0x7f029c0c7190 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a1495640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f029c071f00 0x7f029c0c4090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49494/0 (socket says 172.21.15.39:49494) 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a1495640 1 -- 172.21.15.39:0/3722864565 learned_addr learned my addr 172.21.15.39:0/3722864565 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a1c96640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f029c0c7190 0x7f029c0c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170499142 0 0) 0x7f029c06de90 con 0x7f029c071f00 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0288003530 con 0x7f029c071f00 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855242064 0 0) 0x7f0288003530 con 0x7f029c071f00 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f029c0c7190 msgr2=0x7f029c0c2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f029c0c7190 0x7f029c0c2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 >> v1:172.21.15.84:6789/0 conn(0x7f029c06edf0 legacy=0x7f029c0c2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c0c5e20 con 0x7f029c071f00 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7f0298001680 con 0x7f029c071f00 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 >> v1:172.21.15.39:6789/0 conn(0x7f029c071f00 legacy=0x7f029c0c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f029c0c7190 0x7f02880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0288003f30 0x7f02880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0288004d10 0x7f0288005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0c6170 con 0x7f0288003f30 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028800bb20 con 0x7f0288004d10 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028800bce0 con 0x7f029c0c7190 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a0c94640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0288003f30 0x7f02880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:34.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a1495640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f029c0c7190 0x7f02880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:34.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a1c96640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0288004d10 0x7f0288005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:34.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a0c94640 1 -- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f029c0c7190 msgr2=0x7f02880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:34.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a0c94640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f029c0c7190 0x7f02880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a0c94640 1 -- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0288004d10 msgr2=0x7f0288005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:34.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a0c94640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0288004d10 0x7f0288005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a0c94640 1 -- 172.21.15.39:0/3722864565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0288003530 con 0x7f0288003f30 2024-09-13T17:19:34.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f02a0c94640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0288003f30 0x7f02880047d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f028c001750 tx=0x7f028c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74c92748b166a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:34.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c0052d0 con 0x7f0288003f30 2024-09-13T17:19:34.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f028c0a7050 con 0x7f0288003f30 2024-09-13T17:19:34.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.839+0000 7f0293fff640 1 -- 172.21.15.39:0/3722864565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c003600 con 0x7f0288003f30 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- 172.21.15.39:0/3722864565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0288003f30 msgr2=0x7f02880047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 --2- 172.21.15.39:0/3722864565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0288003f30 0x7f02880047d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f028c001750 tx=0x7f028c01c3e0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 -- 172.21.15.39:0/3722864565 reap_dead start 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- 172.21.15.39:0/3722864565 shutdown_connections 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- 172.21.15.39:0/3722864565 >> 172.21.15.39:0/3722864565 conn(0x7f029c06c020 msgr2=0x7f029c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- 172.21.15.39:0/3722864565 shutdown_connections 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- 172.21.15.39:0/3722864565 wait complete. 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 Processor -- start 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- start start 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0288003f30 0x7f029c06a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:34.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f029c14df10 0x7f029c1524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f029c153a70 0x7f029c14d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c069e10 con 0x7f029c153a70 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c06ada0 con 0x7f029c14df10 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029c154ae0 con 0x7f0288003f30 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f029c153a70 0x7f029c14d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f029c153a70 0x7f029c14d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39516/0 (socket says 172.21.15.39:39516) 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 -- 172.21.15.39:0/2320258692 learned_addr learned my addr 172.21.15.39:0/2320258692 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1495640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0288003f30 0x7f029c06a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a0c94640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f029c14df10 0x7f029c1524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 -- 172.21.15.39:0/2320258692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0288003f30 msgr2=0x7f029c06a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:34.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0288003f30 0x7f029c06a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 -- 172.21.15.39:0/2320258692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f029c14df10 msgr2=0x7f029c1524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f029c14df10 0x7f029c1524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 -- 172.21.15.39:0/2320258692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c06b320 con 0x7f029c153a70 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1495640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0288003f30 0x7f029c06a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a1c96640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f029c153a70 0x7f029c14d210 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f029c06f5d0 tx=0x7f02940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f0291ffb640 1 -- 172.21.15.39:0/2320258692 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400e070 con 0x7f029c153a70 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f0291ffb640 1 -- 172.21.15.39:0/2320258692 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0294011e10 con 0x7f029c153a70 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f0291ffb640 1 -- 172.21.15.39:0/2320258692 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940039a0 con 0x7f029c153a70 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c06b600 con 0x7f029c153a70 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c06a340 con 0x7f029c153a70 2024-09-13T17:19:34.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.843+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029c0ace40 con 0x7f029c153a70 2024-09-13T17:19:34.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.847+0000 7f0291ffb640 1 -- 172.21.15.39:0/2320258692 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0294003b00 con 0x7f029c153a70 2024-09-13T17:19:34.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.847+0000 7f0291ffb640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f02800666c0 0x7f0280068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:34.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.847+0000 7f0291ffb640 1 -- 172.21.15.39:0/2320258692 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 10163+0+0 (secure 0 0 0) 0x7f0294085970 con 0x7f029c153a70 2024-09-13T17:19:34.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.847+0000 7f02a1495640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f02800666c0 0x7f0280068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:34.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.847+0000 7f02a1495640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f02800666c0 0x7f0280068b80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0298001ca0 tx=0x7f0298009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:34.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:34.851+0000 7f0291ffb640 1 -- 172.21.15.39:0/2320258692 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0294051b20 con 0x7f029c153a70 2024-09-13T17:19:35.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.015+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "a_metadata", "pool2": "a_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f029c06b7f0 con 0x7f029c153a70 2024-09-13T17:19:35.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.671+0000 7f0291ffb640 1 -- 172.21.15.39:0/2320258692 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "a_metadata", "pool2": "a_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'a_metadata' removed v66) v1 ==== 166+0+0 (secure 0 0 0) 0x7f0294055140 con 0x7f029c153a70 2024-09-13T17:19:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'a_metadata' removed 2024-09-13T17:19:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.675+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f02800666c0 msgr2=0x7f0280068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.675+0000 7f02a36f9640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f02800666c0 0x7f0280068b80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0298001ca0 tx=0x7f0298009040 comp rx=0 tx=0).stop 2024-09-13T17:19:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.675+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f029c153a70 msgr2=0x7f029c14d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.675+0000 7f02a36f9640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f029c153a70 0x7f029c14d210 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f029c06f5d0 tx=0x7f02940115a0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.679+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 shutdown_connections 2024-09-13T17:19:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.679+0000 7f02a36f9640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f02800666c0 0x7f0280068b80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.679+0000 7f02a36f9640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f029c14df10 0x7f029c1524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.679+0000 7f02a36f9640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f029c153a70 0x7f029c14d210 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.679+0000 7f02a36f9640 1 --2- 172.21.15.39:0/2320258692 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f0288003f30 0x7f029c06a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.679+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 >> 172.21.15.39:0/2320258692 conn(0x7f029c06c020 msgr2=0x7f029c0a97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.679+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 shutdown_connections 2024-09-13T17:19:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.679+0000 7f02a36f9640 1 -- 172.21.15.39:0/2320258692 wait complete. 2024-09-13T17:19:35.706 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm a_data a_data --yes-i-really-really-mean-it 2024-09-13T17:19:35.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfd19f0640 1 Processor -- start 2024-09-13T17:19:35.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfd19f0640 1 -- start start 2024-09-13T17:19:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfd19f0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcfcc1514d0 0x7fcfcc0c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfd19f0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc157370 0x7fcfcc154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfd19f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc155270 con 0x7fcfcc0bcf90 2024-09-13T17:19:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfd19f0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc153d80 con 0x7fcfcc1514d0 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfd19f0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc157870 con 0x7fcfcc157370 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfca7fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcfcc0bcf90 0x7fcfcc154240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49504/0 (socket says 172.21.15.39:49504) 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfca7fc640 1 -- 172.21.15.39:0/1842034077 learned_addr learned my addr 172.21.15.39:0/1842034077 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/1842034077 >> v2:172.21.15.39:3300/0 conn(0x7fcfcc1514d0 0x7fcfcc0c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfcb7fe640 1 --2- 172.21.15.39:0/1842034077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc157370 0x7fcfcc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfc9ffb640 1 -- 172.21.15.39:0/1842034077 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725392290 0 0) 0x7fcfcc155270 con 0x7fcfcc0bcf90 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfc9ffb640 1 -- 172.21.15.39:0/1842034077 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb4003530 con 0x7fcfcc0bcf90 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfcaffd640 1 -- 172.21.15.39:0/1842034077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc157370 msgr2=0x7fcfcc154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/1842034077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc157370 0x7fcfcc154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.831+0000 7fcfcaffd640 1 -- 172.21.15.39:0/1842034077 >> v1:172.21.15.39:6789/0 conn(0x7fcfcc0bcf90 legacy=0x7fcfcc154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:35.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfcaffd640 1 -- 172.21.15.39:0/1842034077 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc157b50 con 0x7fcfcc1514d0 2024-09-13T17:19:35.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/1842034077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc1514d0 0x7fcfcc0c41f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009d90 tx=0x7fcfbc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a1f60f8bc057a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:35.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfc9ffb640 1 -- 172.21.15.39:0/1842034077 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0a5890 con 0x7fcfcc1514d0 2024-09-13T17:19:35.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfc9ffb640 1 -- 172.21.15.39:0/1842034077 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcfbc0a59f0 con 0x7fcfcc1514d0 2024-09-13T17:19:35.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfc9ffb640 1 -- 172.21.15.39:0/1842034077 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0a5da0 con 0x7fcfcc1514d0 2024-09-13T17:19:35.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfd19f0640 1 -- 172.21.15.39:0/1842034077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc1514d0 msgr2=0x7fcfcc0c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:35.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfd19f0640 1 --2- 172.21.15.39:0/1842034077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc1514d0 0x7fcfcc0c41f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009d90 tx=0x7fcfbc01c8e0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfd19f0640 1 -- 172.21.15.39:0/1842034077 shutdown_connections 2024-09-13T17:19:35.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfd19f0640 1 --2- 172.21.15.39:0/1842034077 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc157370 0x7fcfcc154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfd19f0640 1 --2- 172.21.15.39:0/1842034077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc1514d0 0x7fcfcc0c41f0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfd19f0640 1 -- 172.21.15.39:0/1842034077 >> 172.21.15.39:0/1842034077 conn(0x7fcfcc06a210 msgr2=0x7fcfcc06a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:35.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfd19f0640 1 -- 172.21.15.39:0/1842034077 shutdown_connections 2024-09-13T17:19:35.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.835+0000 7fcfd19f0640 1 -- 172.21.15.39:0/1842034077 wait complete. 2024-09-13T17:19:35.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 Processor -- start 2024-09-13T17:19:35.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 -- start start 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc0bcf90 0x7fcfcc0be210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc1514d0 0x7fcfcc1ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc157370 0x7fcfcc0bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc1eca60 con 0x7fcfcc157370 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc1ec750 con 0x7fcfcc0bcf90 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0af450 con 0x7fcfcc1514d0 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc157370 0x7fcfcc0bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc157370 0x7fcfcc0bebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39542/0 (socket says 172.21.15.39:39542) 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcb7fe640 1 -- 172.21.15.39:0/203350876 learned_addr learned my addr 172.21.15.39:0/203350876 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc0bcf90 0x7fcfcc0be210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfca7fc640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc1514d0 0x7fcfcc1ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:35.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcb7fe640 1 -- 172.21.15.39:0/203350876 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc1514d0 msgr2=0x7fcfcc1ecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcb7fe640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc1514d0 0x7fcfcc1ecef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcb7fe640 1 -- 172.21.15.39:0/203350876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc0bcf90 msgr2=0x7fcfcc0be210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcb7fe640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc0bcf90 0x7fcfcc0be210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcb7fe640 1 -- 172.21.15.39:0/203350876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc009a70 con 0x7fcfcc157370 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfca7fc640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc1514d0 0x7fcfcc1ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcb7fe640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc157370 0x7fcfcc0bebe0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400d5d0 tx=0x7fcfc400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfb3fff640 1 -- 172.21.15.39:0/203350876 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40125f0 con 0x7fcfcc157370 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfb3fff640 1 -- 172.21.15.39:0/203350876 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcfc4012770 con 0x7fcfcc157370 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc0bfee0 con 0x7fcfcc157370 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfb3fff640 1 -- 172.21.15.39:0/203350876 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4012b40 con 0x7fcfcc157370 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc0bd580 con 0x7fcfcc157370 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc0c0390 con 0x7fcfcc157370 2024-09-13T17:19:35.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfb3fff640 1 -- 172.21.15.39:0/203350876 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcfc4012ca0 con 0x7fcfcc157370 2024-09-13T17:19:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfb3fff640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb80666c0 0x7fcfb8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfb3fff640 1 -- 172.21.15.39:0/203350876 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 9735+0+0 (secure 0 0 0) 0x7fcfc40855e0 con 0x7fcfcc157370 2024-09-13T17:19:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.839+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb80666c0 0x7fcfb8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:35.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.843+0000 7fcfcaffd640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb80666c0 0x7fcfb8068b80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0af000 tx=0x7fcfbc0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:35.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:35.843+0000 7fcfb3fff640 1 -- 172.21.15.39:0/203350876 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcfc40519d0 con 0x7fcfcc157370 2024-09-13T17:19:36.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.011+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "a_data", "pool2": "a_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcfcc1ec890 con 0x7fcfcc157370 2024-09-13T17:19:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.703+0000 7fcfb3fff640 1 -- 172.21.15.39:0/203350876 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "a_data", "pool2": "a_data", "yes_i_really_really_mean_it": true}]=0 pool 'a_data' removed v67) v1 ==== 154+0+0 (secure 0 0 0) 0x7fcfc4054ff0 con 0x7fcfcc157370 2024-09-13T17:19:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'a_data' removed 2024-09-13T17:19:36.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb80666c0 msgr2=0x7fcfb8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:36.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb80666c0 0x7fcfb8068b80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0af000 tx=0x7fcfbc0a7070 comp rx=0 tx=0).stop 2024-09-13T17:19:36.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc157370 msgr2=0x7fcfcc0bebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:36.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc157370 0x7fcfcc0bebe0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400d5d0 tx=0x7fcfc400da90 comp rx=0 tx=0).stop 2024-09-13T17:19:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 shutdown_connections 2024-09-13T17:19:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fcfb80666c0 0x7fcfb8068b80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfcc157370 0x7fcfcc0bebe0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fcfcc1514d0 0x7fcfcc1ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 --2- 172.21.15.39:0/203350876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfcc0bcf90 0x7fcfcc0be210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 >> 172.21.15.39:0/203350876 conn(0x7fcfcc06a210 msgr2=0x7fcfcc152090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 shutdown_connections 2024-09-13T17:19:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.707+0000 7fcfd19f0640 1 -- 172.21.15.39:0/203350876 wait complete. 2024-09-13T17:19:36.747 INFO:tasks.cephfs.filesystem:Destroying file system b and related pools 2024-09-13T17:19:36.748 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:19:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.867+0000 7fb5f439b640 1 Processor -- start 2024-09-13T17:19:36.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.867+0000 7fb5f439b640 1 -- start start 2024-09-13T17:19:36.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f439b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5ec14e0c0 0x7fb5ec0a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:36.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f439b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb5ec0c4740 0x7fb5ec0c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:36.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f439b640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec0c2780 con 0x7fb5ec0c1f50 2024-09-13T17:19:36.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f439b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c2a60 con 0x7fb5ec0c4740 2024-09-13T17:19:36.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f439b640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c2d40 con 0x7fb5ec14e0c0 2024-09-13T17:19:36.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f2938640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb5ec0c4740 0x7fb5ec0c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:36.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f2938640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb5ec0c4740 0x7fb5ec0c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39552/0 (socket says 172.21.15.39:39552) 2024-09-13T17:19:36.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f2938640 1 -- 172.21.15.39:0/4135713689 learned_addr learned my addr 172.21.15.39:0/4135713689 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:36.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f2137640 1 --2- 172.21.15.39:0/4135713689 >> v2:172.21.15.84:3300/0 conn(0x7fb5ec14e0c0 0x7fb5ec0a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:36.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f2938640 1 -- 172.21.15.39:0/4135713689 >> v2:172.21.15.84:3300/0 conn(0x7fb5ec14e0c0 msgr2=0x7fb5ec0a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:36.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f2938640 1 --2- 172.21.15.39:0/4135713689 >> v2:172.21.15.84:3300/0 conn(0x7fb5ec14e0c0 0x7fb5ec0a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:36.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f2938640 1 -- 172.21.15.39:0/4135713689 >> v1:172.21.15.84:6789/0 conn(0x7fb5ec0c1f50 legacy=0x7fb5ec06cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:36.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f2938640 1 -- 172.21.15.39:0/4135713689 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0c3020 con 0x7fb5ec0c4740 2024-09-13T17:19:36.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f2938640 1 --2- 172.21.15.39:0/4135713689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 0x7fb5ec0c4b90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0c3370 tx=0x7fb5dc01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411546b9b358a77a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:36.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f1135640 1 -- 172.21.15.39:0/4135713689 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0a7a10 con 0x7fb5ec0c4740 2024-09-13T17:19:36.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f1135640 1 -- 172.21.15.39:0/4135713689 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb5dc0a7b70 con 0x7fb5ec0c4740 2024-09-13T17:19:36.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f1135640 1 -- 172.21.15.39:0/4135713689 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0af440 con 0x7fb5ec0c4740 2024-09-13T17:19:36.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f439b640 1 -- 172.21.15.39:0/4135713689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 msgr2=0x7fb5ec0c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:36.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.871+0000 7fb5f439b640 1 --2- 172.21.15.39:0/4135713689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 0x7fb5ec0c4b90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec0c3370 tx=0x7fb5dc01eb30 comp rx=0 tx=0).stop 2024-09-13T17:19:36.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- 172.21.15.39:0/4135713689 shutdown_connections 2024-09-13T17:19:36.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 --2- 172.21.15.39:0/4135713689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 0x7fb5ec0c4b90 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:36.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 --2- 172.21.15.39:0/4135713689 >> v2:172.21.15.84:3300/0 conn(0x7fb5ec14e0c0 0x7fb5ec0a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:36.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- 172.21.15.39:0/4135713689 >> 172.21.15.39:0/4135713689 conn(0x7fb5ec069e10 msgr2=0x7fb5ec06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:36.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- 172.21.15.39:0/4135713689 shutdown_connections 2024-09-13T17:19:36.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- 172.21.15.39:0/4135713689 wait complete. 2024-09-13T17:19:36.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 Processor -- start 2024-09-13T17:19:36.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- start start 2024-09-13T17:19:36.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5ec0c1f50 0x7fb5ec148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 0x7fb5ec0bee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5ec14e0c0 0x7fb5ec154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec14b3b0 con 0x7fb5ec0c4740 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec14adf0 con 0x7fb5ec14e0c0 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec14b0d0 con 0x7fb5ec0c1f50 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f1936640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 0x7fb5ec0bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f1936640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 0x7fb5ec0bee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39568/0 (socket says 172.21.15.39:39568) 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f1936640 1 -- 172.21.15.39:0/2612281748 learned_addr learned my addr 172.21.15.39:0/2612281748 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f2938640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5ec14e0c0 0x7fb5ec154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f2137640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5ec0c1f50 0x7fb5ec148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f1936640 1 -- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5ec0c1f50 msgr2=0x7fb5ec148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:36.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f1936640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5ec0c1f50 0x7fb5ec148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f1936640 1 -- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5ec14e0c0 msgr2=0x7fb5ec154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f1936640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5ec14e0c0 0x7fb5ec154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f1936640 1 -- 172.21.15.39:0/2612281748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc00b770 con 0x7fb5ec0c4740 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f2938640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5ec14e0c0 0x7fb5ec154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f2137640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5ec0c1f50 0x7fb5ec148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f1936640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 0x7fb5ec0bee90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400c430 tx=0x7fb5e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5eaffd640 1 -- 172.21.15.39:0/2612281748 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40052d0 con 0x7fb5ec0c4740 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5eaffd640 1 -- 172.21.15.39:0/2612281748 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb5e4005430 con 0x7fb5ec0c4740 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec14aaf0 con 0x7fb5ec0c4740 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5eaffd640 1 -- 172.21.15.39:0/2612281748 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4007280 con 0x7fb5ec0c4740 2024-09-13T17:19:36.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec0bf3d0 con 0x7fb5ec0c4740 2024-09-13T17:19:36.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.875+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ec06c8b0 con 0x7fb5ec0c4740 2024-09-13T17:19:36.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.879+0000 7fb5eaffd640 1 -- 172.21.15.39:0/2612281748 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5e40073e0 con 0x7fb5ec0c4740 2024-09-13T17:19:36.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.879+0000 7fb5eaffd640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5d8066690 0x7fb5d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:36.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.879+0000 7fb5f2137640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5d8066690 0x7fb5d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:36.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.879+0000 7fb5eaffd640 1 -- 172.21.15.39:0/2612281748 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 9215+0+0 (secure 0 0 0) 0x7fb5e40844e0 con 0x7fb5ec0c4740 2024-09-13T17:19:36.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.879+0000 7fb5f2137640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5d8066690 0x7fb5d8068b50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009fd0 tx=0x7fb5e000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:36.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:36.883+0000 7fb5eaffd640 1 -- 172.21.15.39:0/2612281748 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb5e4050930 con 0x7fb5ec0c4740 2024-09-13T17:19:37.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:19:37.051 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:19:37.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.083+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5ec151270 con 0x7fb5ec0c4740 2024-09-13T17:19:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.083+0000 7fb5eaffd640 1 -- 172.21.15.39:0/2612281748 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) v1 ==== 95+0+4732 (secure 0 0 0) 0x7fb5e4007690 con 0x7fb5ec0c4740 2024-09-13T17:19:37.089 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:19:37.089 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":209,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6932,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1521872458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1521872458},{"type":"v1","addr":"172.21.15.84:6835","nonce":1521872458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428801917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428801917},{"type":"v1","addr":"172.21.15.39:6835","nonce":428801917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":207}],"filesystems":[{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:19:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 209 2024-09-13T17:19:37.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5d8066690 msgr2=0x7fb5d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5d8066690 0x7fb5d8068b50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009fd0 tx=0x7fb5e000b040 comp rx=0 tx=0).stop 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 msgr2=0x7fb5ec0bee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 0x7fb5ec0bee90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400c430 tx=0x7fb5e400c8f0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 shutdown_connections 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb5d8066690 0x7fb5d8068b50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5ec14e0c0 0x7fb5ec154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5ec0c4740 0x7fb5ec0bee90 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 --2- 172.21.15.39:0/2612281748 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb5ec0c1f50 0x7fb5ec148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 >> 172.21.15.39:0/2612281748 conn(0x7fb5ec069e10 msgr2=0x7fb5ec14f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 shutdown_connections 2024-09-13T17:19:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.087+0000 7fb5f439b640 1 -- 172.21.15.39:0/2612281748 wait complete. 2024-09-13T17:19:37.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:19:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe017b7e640 1 Processor -- start 2024-09-13T17:19:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe017b7e640 1 -- start start 2024-09-13T17:19:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe017b7e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe010151500 0x7fe0100c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe017b7e640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe0100c47c0 0x7fe010154780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe017b7e640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0100c5a30 con 0x7fe010071dd0 2024-09-13T17:19:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe017b7e640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0101576a0 con 0x7fe0100c47c0 2024-09-13T17:19:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe017b7e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe010157980 con 0x7fe010151500 2024-09-13T17:19:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe01591a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe010151500 0x7fe0100c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe01611b640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe0100c47c0 0x7fe010154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe015119640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe010071dd0 0x7fe010154070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:47054/0 (socket says 172.21.15.39:47054) 2024-09-13T17:19:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe01591a640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe010151500 0x7fe0100c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3301/0 says I am v2:172.21.15.39:47798/0 (socket says 172.21.15.39:47798) 2024-09-13T17:19:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe015119640 1 -- 172.21.15.39:0/1440677989 learned_addr learned my addr 172.21.15.39:0/1440677989 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe01611b640 1 -- 172.21.15.39:0/1440677989 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe010151500 msgr2=0x7fe0100c4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe01611b640 1 --2- 172.21.15.39:0/1440677989 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe010151500 0x7fe0100c4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe01611b640 1 -- 172.21.15.39:0/1440677989 >> v1:172.21.15.84:6789/0 conn(0x7fe010071dd0 legacy=0x7fe010154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe01611b640 1 -- 172.21.15.39:0/1440677989 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe010157c60 con 0x7fe0100c47c0 2024-09-13T17:19:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.267+0000 7fe01591a640 1 --2- 172.21.15.39:0/1440677989 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe010151500 0x7fe0100c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:19:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe01611b640 1 --2- 172.21.15.39:0/1440677989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0100c47c0 0x7fe010154780 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe008009d90 tx=0x7fe00801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb6cf11b8d13208 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe014918640 1 -- 172.21.15.39:0/1440677989 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080a5990 con 0x7fe0100c47c0 2024-09-13T17:19:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe014918640 1 -- 172.21.15.39:0/1440677989 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe0080a5af0 con 0x7fe0100c47c0 2024-09-13T17:19:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe014918640 1 -- 172.21.15.39:0/1440677989 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080a5ea0 con 0x7fe0100c47c0 2024-09-13T17:19:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 -- 172.21.15.39:0/1440677989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0100c47c0 msgr2=0x7fe010154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 --2- 172.21.15.39:0/1440677989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0100c47c0 0x7fe010154780 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe008009d90 tx=0x7fe00801c8b0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 -- 172.21.15.39:0/1440677989 shutdown_connections 2024-09-13T17:19:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 --2- 172.21.15.39:0/1440677989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0100c47c0 0x7fe010154780 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 --2- 172.21.15.39:0/1440677989 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe010151500 0x7fe0100c4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 -- 172.21.15.39:0/1440677989 >> 172.21.15.39:0/1440677989 conn(0x7fe0100ab530 msgr2=0x7fe0100ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 -- 172.21.15.39:0/1440677989 shutdown_connections 2024-09-13T17:19:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 -- 172.21.15.39:0/1440677989 wait complete. 2024-09-13T17:19:37.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 Processor -- start 2024-09-13T17:19:37.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.271+0000 7fe017b7e640 1 -- start start 2024-09-13T17:19:37.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe017b7e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe010071dd0 0x7fe01014b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe017b7e640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0100c47c0 0x7fe0101492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe017b7e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe010151500 0x7fe010146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe017b7e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010147810 con 0x7fe010151500 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe017b7e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01014e8d0 con 0x7fe010071dd0 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe017b7e640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01014bf10 con 0x7fe0100c47c0 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01611b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe010151500 0x7fe010146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01611b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe010151500 0x7fe010146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39578/0 (socket says 172.21.15.39:39578) 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01611b640 1 -- 172.21.15.39:0/7400428 learned_addr learned my addr 172.21.15.39:0/7400428 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01591a640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe010071dd0 0x7fe01014b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe015119640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0100c47c0 0x7fe0101492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01611b640 1 -- 172.21.15.39:0/7400428 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0100c47c0 msgr2=0x7fe0101492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01611b640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0100c47c0 0x7fe0101492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01611b640 1 -- 172.21.15.39:0/7400428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe010071dd0 msgr2=0x7fe01014b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01611b640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe010071dd0 0x7fe01014b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01611b640 1 -- 172.21.15.39:0/7400428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c009420 con 0x7fe010151500 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01591a640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe010071dd0 0x7fe01014b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01611b640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe010151500 0x7fe010146b70 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fe00801af60 tx=0x7fe008019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe0067fc640 1 -- 172.21.15.39:0/7400428 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080add90 con 0x7fe010151500 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe0067fc640 1 -- 172.21.15.39:0/7400428 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe008009450 con 0x7fe010151500 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe008009a70 con 0x7fe010151500 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe0067fc640 1 -- 172.21.15.39:0/7400428 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080b9670 con 0x7fe010151500 2024-09-13T17:19:37.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01014a190 con 0x7fe010151500 2024-09-13T17:19:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0100c4280 con 0x7fe010151500 2024-09-13T17:19:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe0067fc640 1 -- 172.21.15.39:0/7400428 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe0080b97d0 con 0x7fe010151500 2024-09-13T17:19:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe0067fc640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdff4066690 0x7fdff4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe0067fc640 1 -- 172.21.15.39:0/7400428 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 9215+0+0 (secure 0 0 0) 0x7fe008120820 con 0x7fe010151500 2024-09-13T17:19:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.275+0000 7fe01591a640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdff4066690 0x7fdff4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.279+0000 7fe01591a640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdff4066690 0x7fdff4068b50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe0100c90c0 tx=0x7fe00c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.283+0000 7fe0067fc640 1 -- 172.21.15.39:0/7400428 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe0080ece20 con 0x7fe010151500 2024-09-13T17:19:37.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe010071bb0 con 0x7fe010151500 2024-09-13T17:19:37.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe0067fc640 1 -- 172.21.15.39:0/7400428 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) v1 ==== 95+0+4732 (secure 0 0 0) 0x7fe0080b2090 con 0x7fe010151500 2024-09-13T17:19:37.488 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:19:37.488 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":209,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6932,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1521872458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1521872458},{"type":"v1","addr":"172.21.15.84:6835","nonce":1521872458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428801917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428801917},{"type":"v1","addr":"172.21.15.39:6835","nonce":428801917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":207}],"filesystems":[{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T16:48:08.135456+0000","modified":"2024-09-13T17:16:08.963924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6638":{"gid":6638,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":9,"addr":"172.21.15.39:6837/3255731814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":3255731814},{"type":"v1","addr":"172.21.15.39:6837","nonce":3255731814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:19:37.488 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 209 2024-09-13T17:19:37.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdff4066690 msgr2=0x7fdff4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdff4066690 0x7fdff4068b50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe0100c90c0 tx=0x7fe00c008040 comp rx=0 tx=0).stop 2024-09-13T17:19:37.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe010151500 msgr2=0x7fe010146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe010151500 0x7fe010146b70 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fe00801af60 tx=0x7fe008019650 comp rx=0 tx=0).stop 2024-09-13T17:19:37.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 shutdown_connections 2024-09-13T17:19:37.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fdff4066690 0x7fdff4068b50 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe010151500 0x7fe010146b70 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fe0100c47c0 0x7fe0101492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 --2- 172.21.15.39:0/7400428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe010071dd0 0x7fe01014b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 >> 172.21.15.39:0/7400428 conn(0x7fe0100ab530 msgr2=0x7fe0100ab910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:37.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 shutdown_connections 2024-09-13T17:19:37.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.483+0000 7fe017b7e640 1 -- 172.21.15.39:0/7400428 wait complete. 2024-09-13T17:19:37.508 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:19:37.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbecccdb640 1 Processor -- start 2024-09-13T17:19:37.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbecccdb640 1 -- start start 2024-09-13T17:19:37.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbecccdb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c7470 0x7fbec8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbecccdb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec814f590 con 0x7fbec806e5e0 2024-09-13T17:19:37.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbecccdb640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec806eae0 con 0x7fbec814f1b0 2024-09-13T17:19:37.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbecccdb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec806f660 con 0x7fbec80c7470 2024-09-13T17:19:37.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbebffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbec806e5e0 0x7fbec80c5c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49520/0 (socket says 172.21.15.39:49520) 2024-09-13T17:19:37.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbebffff640 1 -- 172.21.15.39:0/3021274511 learned_addr learned my addr 172.21.15.39:0/3021274511 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:37.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec6d9d640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c7470 0x7fbec8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894926551 0 0) 0x7fbec814f590 con 0x7fbec806e5e0 2024-09-13T17:19:37.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb4003530 con 0x7fbec806e5e0 2024-09-13T17:19:37.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574201532 0 0) 0x7fbeb4003530 con 0x7fbec806e5e0 2024-09-13T17:19:37.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c7470 msgr2=0x7fbec8151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:37.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c7470 0x7fbec8151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 >> v1:172.21.15.84:6789/0 conn(0x7fbec814f1b0 legacy=0x7fbec80714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec806edc0 con 0x7fbec806e5e0 2024-09-13T17:19:37.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2620178932 0 0) 0x7fbeb0001680 con 0x7fbec806e5e0 2024-09-13T17:19:37.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 >> v1:172.21.15.39:6789/0 conn(0x7fbec806e5e0 legacy=0x7fbec80c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c7470 0x7fbeb40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4003ef0 0x7fbeb40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb4004d10 0x7fbeb4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec806f660 con 0x7fbeb4003ef0 2024-09-13T17:19:37.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb400b9f0 con 0x7fbeb4004d10 2024-09-13T17:19:37.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb400bbb0 con 0x7fbec80c7470 2024-09-13T17:19:37.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbebffff640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4003ef0 0x7fbeb40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec659c640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c7470 0x7fbeb40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbec6d9d640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb4004d10 0x7fbeb4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbebffff640 1 -- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c7470 msgr2=0x7fbeb40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:37.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbebffff640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80c7470 0x7fbeb40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbebffff640 1 -- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb4004d10 msgr2=0x7fbeb4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:37.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbebffff640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbeb4004d10 0x7fbeb4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.623+0000 7fbebffff640 1 -- 172.21.15.39:0/3021274511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb4003530 con 0x7fbeb4003ef0 2024-09-13T17:19:37.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbebffff640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4003ef0 0x7fbeb40047d0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fbec806ea10 tx=0x7fbeb001cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23bfc2b77648f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:37.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0005ae0 con 0x7fbeb4003ef0 2024-09-13T17:19:37.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbeb0005c60 con 0x7fbeb4003ef0 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbec5d9b640 1 -- 172.21.15.39:0/3021274511 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb00a5ac0 con 0x7fbeb4003ef0 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbecccdb640 1 -- 172.21.15.39:0/3021274511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4003ef0 msgr2=0x7fbeb40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbecccdb640 1 --2- 172.21.15.39:0/3021274511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4003ef0 0x7fbeb40047d0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fbec806ea10 tx=0x7fbeb001cb60 comp rx=0 tx=0).stop 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbec6d9d640 1 -- 172.21.15.39:0/3021274511 reap_dead start 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbecccdb640 1 -- 172.21.15.39:0/3021274511 shutdown_connections 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbecccdb640 1 -- 172.21.15.39:0/3021274511 >> 172.21.15.39:0/3021274511 conn(0x7fbec80a9720 msgr2=0x7fbec80a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbecccdb640 1 -- 172.21.15.39:0/3021274511 shutdown_connections 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbecccdb640 1 -- 172.21.15.39:0/3021274511 wait complete. 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbecccdb640 1 Processor -- start 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.627+0000 7fbecccdb640 1 -- start start 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbecccdb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80ac070 0x7fbec8154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbecccdb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec806d5d0 0x7fbec8154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbecccdb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8155c70 0x7fbec8153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbecccdb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec80ab540 con 0x7fbec8155c70 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbecccdb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec8154cb0 con 0x7fbec806d5d0 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbecccdb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec81553d0 con 0x7fbec80ac070 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8155c70 0x7fbec8153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec6d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8155c70 0x7fbec8153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39608/0 (socket says 172.21.15.39:39608) 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec6d9d640 1 -- 172.21.15.39:0/122364687 learned_addr learned my addr 172.21.15.39:0/122364687 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec659c640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80ac070 0x7fbec8154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbebffff640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec806d5d0 0x7fbec8154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec6d9d640 1 -- 172.21.15.39:0/122364687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80ac070 msgr2=0x7fbec8154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec6d9d640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80ac070 0x7fbec8154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec6d9d640 1 -- 172.21.15.39:0/122364687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec806d5d0 msgr2=0x7fbec8154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec6d9d640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec806d5d0 0x7fbec8154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec6d9d640 1 -- 172.21.15.39:0/122364687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec81556b0 con 0x7fbec8155c70 2024-09-13T17:19:37.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec6d9d640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8155c70 0x7fbec8153150 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fbec806cfc0 tx=0x7fbec00070e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:37.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbebeffd640 1 -- 172.21.15.39:0/122364687 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec000e040 con 0x7fbec8155c70 2024-09-13T17:19:37.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbebeffd640 1 -- 172.21.15.39:0/122364687 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbec000f040 con 0x7fbec8155c70 2024-09-13T17:19:37.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec80ab750 con 0x7fbec8155c70 2024-09-13T17:19:37.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbebeffd640 1 -- 172.21.15.39:0/122364687 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0015430 con 0x7fbec8155c70 2024-09-13T17:19:37.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec80adc50 con 0x7fbec8155c70 2024-09-13T17:19:37.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec80abc00 con 0x7fbec8155c70 2024-09-13T17:19:37.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbebeffd640 1 -- 172.21.15.39:0/122364687 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbec0015590 con 0x7fbec8155c70 2024-09-13T17:19:37.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbebeffd640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbea80666c0 0x7fbea8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:37.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbebeffd640 1 -- 172.21.15.39:0/122364687 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 9215+0+0 (secure 0 0 0) 0x7fbec0085650 con 0x7fbec8155c70 2024-09-13T17:19:37.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec659c640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbea80666c0 0x7fbea8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:37.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.631+0000 7fbec659c640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbea80666c0 0x7fbea8068b80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8000f70 tx=0x7fbeb8000ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:37.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.635+0000 7fbebeffd640 1 -- 172.21.15.39:0/122364687 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbec80abc00 con 0x7fbec8155c70 2024-09-13T17:19:37.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.867+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbec80abc00 con 0x7fbec8155c70 2024-09-13T17:19:37.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.867+0000 7fbebeffd640 1 -- 172.21.15.39:0/122364687 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+21808 (secure 0 0 0) 0x7fbec80abc00 con 0x7fbec8155c70 2024-09-13T17:19:37.873 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:19:37.873 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":67,"fsid":"b9195ac8-73ed-4db1-a594-cde2ff5ccb0f","created":"2024-09-13T16:47:34.823850+0000","modified":"2024-09-13T17:19:36.685076+0000","last_up_change":"2024-09-13T16:47:46.703091+0000","last_in_change":"2024-09-13T16:47:37.169194+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T16:47:47.637217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"b_metadata","create_time":"2024-09-13T16:48:06.063601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"b"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"b_data","create_time":"2024-09-13T16:48:07.109684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"b"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e58e45d3-a9d5-45da-bd6c-9bcf3fce54d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6801","nonce":3633417446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6803","nonce":3633417446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6807","nonce":3633417446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3633417446},{"type":"v1","addr":"172.21.15.39:6805","nonce":3633417446}]},"public_addr":"172.21.15.39:6801/3633417446","cluster_addr":"172.21.15.39:6803/3633417446","heartbeat_back_addr":"172.21.15.39:6807/3633417446","heartbeat_front_addr":"172.21.15.39:6805/3633417446","state":["exists","up"]},{"osd":1,"uuid":"069d6f12-1c43-468a-871e-4f53f7755e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6809","nonce":2114478470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6811","nonce":2114478470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6815","nonce":2114478470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":2114478470},{"type":"v1","addr":"172.21.15.39:6813","nonce":2114478470}]},"public_addr":"172.21.15.39:6809/2114478470","cluster_addr":"172.21.15.39:6811/2114478470","heartbeat_back_addr":"172.21.15.39:6815/2114478470","heartbeat_front_addr":"172.21.15.39:6813/2114478470","state":["exists","up"]},{"osd":2,"uuid":"cb3c693e-e427-46c1-97a3-fd7723d4993b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6817","nonce":3103155192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6819","nonce":3103155192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6823","nonce":3103155192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3103155192},{"type":"v1","addr":"172.21.15.39:6821","nonce":3103155192}]},"public_addr":"172.21.15.39:6817/3103155192","cluster_addr":"172.21.15.39:6819/3103155192","heartbeat_back_addr":"172.21.15.39:6823/3103155192","heartbeat_front_addr":"172.21.15.39:6821/3103155192","state":["exists","up"]},{"osd":3,"uuid":"441b35f8-eab6-409d-a03c-6dca2d9f5671","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6825","nonce":1984006749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6827","nonce":1984006749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6831","nonce":1984006749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1984006749},{"type":"v1","addr":"172.21.15.39:6829","nonce":1984006749}]},"public_addr":"172.21.15.39:6825/1984006749","cluster_addr":"172.21.15.39:6827/1984006749","heartbeat_back_addr":"172.21.15.39:6831/1984006749","heartbeat_front_addr":"172.21.15.39:6829/1984006749","state":["exists","up"]},{"osd":4,"uuid":"7387ca7b-a2ad-4477-aead-26e0ee232e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6801","nonce":1588544227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6803","nonce":1588544227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6807","nonce":1588544227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":1588544227},{"type":"v1","addr":"172.21.15.84:6805","nonce":1588544227}]},"public_addr":"172.21.15.84:6801/1588544227","cluster_addr":"172.21.15.84:6803/1588544227","heartbeat_back_addr":"172.21.15.84:6807/1588544227","heartbeat_front_addr":"172.21.15.84:6805/1588544227","state":["exists","up"]},{"osd":5,"uuid":"68838edd-ff7e-4147-9d5d-8f2103121f21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6810","nonce":2432365689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6811","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6813","nonce":2432365689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6820","nonce":2432365689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6815","nonce":2432365689},{"type":"v1","addr":"172.21.15.84:6816","nonce":2432365689}]},"public_addr":"172.21.15.84:6810/2432365689","cluster_addr":"172.21.15.84:6813/2432365689","heartbeat_back_addr":"172.21.15.84:6820/2432365689","heartbeat_front_addr":"172.21.15.84:6816/2432365689","state":["exists","up"]},{"osd":6,"uuid":"f67525eb-b919-4f97-b0bf-9f45ffe8f709","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6812","nonce":2893476104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6817","nonce":2893476104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6823","nonce":2893476104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6819","nonce":2893476104},{"type":"v1","addr":"172.21.15.84:6821","nonce":2893476104}]},"public_addr":"172.21.15.84:6812/2893476104","cluster_addr":"172.21.15.84:6817/2893476104","heartbeat_back_addr":"172.21.15.84:6823/2893476104","heartbeat_front_addr":"172.21.15.84:6821/2893476104","state":["exists","up"]},{"osd":7,"uuid":"d525e6b6-f481-4193-9f63-888925559396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6825","nonce":413977365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6827","nonce":413977365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6831","nonce":413977365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":413977365},{"type":"v1","addr":"172.21.15.84:6829","nonce":413977365}]},"public_addr":"172.21.15.84:6825/413977365","cluster_addr":"172.21.15.84:6827/413977365","heartbeat_back_addr":"172.21.15.84:6831/413977365","heartbeat_front_addr":"172.21.15.84:6829/413977365","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.884888+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.763805+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.720027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.656258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.735161+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.805677+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.723322+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T16:47:42.839116+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6835/3465334740":"2024-09-14T17:19:33.649573+0000","172.21.15.84:6834/3465334740":"2024-09-14T17:19:33.649573+0000","172.21.15.39:6835/980326904":"2024-09-14T17:17:44.963677+0000","172.21.15.39:6834/980326904":"2024-09-14T17:17:44.963677+0000","172.21.15.84:6835/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6834/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.39:6834/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6837/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6835/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.84:6834/397460031":"2024-09-14T17:12:19.869778+0000","172.21.15.39:6836/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.39:6835/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.84:6835/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.39:6835/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.84:6834/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.39:6834/1334276441":"2024-09-14T16:59:04.677958+0000","172.21.15.39:6834/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.39:6836/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.84:6832/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.84:6835/1326467723":"2024-09-14T17:00:02.030979+0000","172.21.15.84:6832/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6836/1355102409":"2024-09-14T17:15:29.935985+0000","172.21.15.84:6834/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.39:6835/3721635324":"2024-09-14T17:16:04.944309+0000","172.21.15.39:6837/2351253009":"2024-09-14T17:11:09.852639+0000","172.21.15.84:6832/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6834/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6835/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.84:6834/380008541":"2024-09-14T17:16:34.950555+0000","172.21.15.39:6835/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.39:6835/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6836/282472862":"2024-09-14T17:14:04.899010+0000","172.21.15.84:6833/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/3334565883":"2024-09-14T17:11:59.865711+0000","172.21.15.39:6835/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.39:6837/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6834/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6835/4094657757":"2024-09-14T16:56:04.630964+0000","172.21.15.84:6834/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.39:6835/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1750130579":"2024-09-14T17:09:29.828673+0000","172.21.15.39:6834/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.84:6832/1355376480":"2024-09-14T16:54:09.605050+0000","172.21.15.39:6834/2065196144":"2024-09-14T16:53:14.593565+0000","172.21.15.84:6832/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.39:6834/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/3122796453":"2024-09-14T16:59:54.690333+0000","172.21.15.39:6837/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6836/2521680730":"2024-09-14T17:09:29.828780+0000","172.21.15.84:6835/661906662":"2024-09-14T16:57:59.663485+0000","172.21.15.39:6834/1572373831":"2024-09-14T16:51:14.567493+0000","172.21.15.84:6835/1969975539":"2024-09-14T16:52:59.589819+0000","172.21.15.84:6833/1392588044":"2024-09-14T16:50:49.562750+0000","172.21.15.84:6833/2144264453":"2024-09-14T16:52:24.581694+0000","172.21.15.39:6835/3163580917":"2024-09-14T16:49:44.548920+0000","172.21.15.39:6834/3941977343":"2024-09-14T17:08:49.820117+0000","172.21.15.84:6832/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6833/2740003107":"2024-09-14T16:57:14.654506+0000","172.21.15.39:6834/4246622083":"2024-09-14T17:01:24.709561+0000","172.21.15.39:6835/305129652":"2024-09-14T17:14:49.920017+0000","172.21.15.39:6836/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6834/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6833/2638959497":"2024-09-14T17:01:24.709473+0000","172.21.15.84:6835/828722052":"2024-09-14T17:03:19.734465+0000","172.21.15.84:6834/3600322972":"2024-09-14T16:55:14.618334+0000","172.21.15.39:6834/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6835/611669592":"2024-09-14T17:04:24.751785+0000","172.21.15.39:6836/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.39:6835/4200926406":"2024-09-14T17:05:59.776545+0000","172.21.15.84:6833/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6836/3105501456":"2024-09-14T17:03:49.742996+0000","172.21.15.39:6837/1590039204":"2024-09-14T17:05:09.765210+0000","172.21.15.84:6834/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.84:6834/3923043380":"2024-09-14T17:09:59.836463+0000","172.21.15.84:6835/263561599":"2024-09-14T17:05:54.774622+0000","172.21.15.39:6837/1392182763":"2024-09-14T17:07:19.797311+0000","172.21.15.39:6837/835951775":"2024-09-14T17:02:59.730250+0000","172.21.15.84:6835/3540656099":"2024-09-14T16:50:14.555020+0000","172.21.15.84:6832/440028459":"2024-09-14T17:07:34.801438+0000","172.21.15.39:6835/1034701751":"2024-09-14T17:10:39.845194+0000","172.21.15.84:6833/1750130579":"2024-09-14T17:09:29.828673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:19:37.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbea80666c0 msgr2=0x7fbea8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbea80666c0 0x7fbea8068b80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8000f70 tx=0x7fbeb8000ee0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8155c70 msgr2=0x7fbec8153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:37.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8155c70 0x7fbec8153150 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fbec806cfc0 tx=0x7fbec00070e0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 shutdown_connections 2024-09-13T17:19:37.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fbea80666c0 0x7fbea8068b80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbec806d5d0 0x7fbec8154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec8155c70 0x7fbec8153150 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 --2- 172.21.15.39:0/122364687 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fbec80ac070 0x7fbec8154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:37.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.875+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 >> 172.21.15.39:0/122364687 conn(0x7fbec80a9720 msgr2=0x7fbec80a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:37.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.879+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 shutdown_connections 2024-09-13T17:19:37.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:37.879+0000 7fbecccdb640 1 -- 172.21.15.39:0/122364687 wait complete. 2024-09-13T17:19:37.933 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail b 2024-09-13T17:19:38.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb2a2901640 1 Processor -- start 2024-09-13T17:19:38.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb2a2901640 1 -- start start 2024-09-13T17:19:38.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb2a2901640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c06c360 0x7fb29c0c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb2a2901640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb29c0c47a0 0x7fb29c0c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb2a2901640 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29c06c740 con 0x7fb29c06b130 2024-09-13T17:19:38.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb2a2901640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29c151ab0 con 0x7fb29c0c47a0 2024-09-13T17:19:38.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb2a2901640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0c21e0 con 0x7fb29c06c360 2024-09-13T17:19:38.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb29bfff640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c06c360 0x7fb29c0c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb293fff640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb29c0c47a0 0x7fb29c0c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb2a0e9e640 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb29c06b130 0x7fb29c06b5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.39:47062/0 (socket says 172.21.15.39:47062) 2024-09-13T17:19:38.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb2a0e9e640 1 -- 172.21.15.39:0/2163464682 learned_addr learned my addr 172.21.15.39:0/2163464682 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:38.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb293fff640 1 -- 172.21.15.39:0/2163464682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c06c360 msgr2=0x7fb29c0c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:38.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb293fff640 1 --2- 172.21.15.39:0/2163464682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c06c360 0x7fb29c0c0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb293fff640 1 -- 172.21.15.39:0/2163464682 >> v1:172.21.15.84:6789/0 conn(0x7fb29c06b130 legacy=0x7fb29c06b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:38.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb293fff640 1 -- 172.21.15.39:0/2163464682 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0c24c0 con 0x7fb29c0c47a0 2024-09-13T17:19:38.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.063+0000 7fb29bfff640 1 --2- 172.21.15.39:0/2163464682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c06c360 0x7fb29c0c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:19:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb293fff640 1 --2- 172.21.15.39:0/2163464682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29c0c47a0 0x7fb29c0c4b80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb2840097c0 tx=0x7fb28401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e9aaaf55e336949 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb29b7fe640 1 -- 172.21.15.39:0/2163464682 <== mon.1 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2840ad640 con 0x7fb29c0c47a0 2024-09-13T17:19:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb29b7fe640 1 -- 172.21.15.39:0/2163464682 <== mon.1 v2:172.21.15.84:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb2840ad7a0 con 0x7fb29c0c47a0 2024-09-13T17:19:38.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb29b7fe640 1 -- 172.21.15.39:0/2163464682 <== mon.1 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2840adb50 con 0x7fb29c0c47a0 2024-09-13T17:19:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 -- 172.21.15.39:0/2163464682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29c0c47a0 msgr2=0x7fb29c0c4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 --2- 172.21.15.39:0/2163464682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29c0c47a0 0x7fb29c0c4b80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb2840097c0 tx=0x7fb28401c4a0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 -- 172.21.15.39:0/2163464682 shutdown_connections 2024-09-13T17:19:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 --2- 172.21.15.39:0/2163464682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29c0c47a0 0x7fb29c0c4b80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 --2- 172.21.15.39:0/2163464682 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c06c360 0x7fb29c0c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 -- 172.21.15.39:0/2163464682 >> 172.21.15.39:0/2163464682 conn(0x7fb29c0c5790 msgr2=0x7fb29c0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 -- 172.21.15.39:0/2163464682 shutdown_connections 2024-09-13T17:19:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 -- 172.21.15.39:0/2163464682 wait complete. 2024-09-13T17:19:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 Processor -- start 2024-09-13T17:19:38.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 -- start start 2024-09-13T17:19:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb29c06b130 0x7fb29c1ede50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29c06c360 0x7fb29c1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c14e1c0 0x7fb29c159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0bfba0 con 0x7fb29c06b130 2024-09-13T17:19:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c1ee390 con 0x7fb29c06c360 2024-09-13T17:19:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb2a2901640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0bf5e0 con 0x7fb29c14e1c0 2024-09-13T17:19:38.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb29bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb29c06b130 0x7fb29c1ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb29bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb29c06b130 0x7fb29c1ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39622/0 (socket says 172.21.15.39:39622) 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.067+0000 7fb29bfff640 1 -- 172.21.15.39:0/1264813032 learned_addr learned my addr 172.21.15.39:0/1264813032 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2a0e9e640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c14e1c0 0x7fb29c159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb293fff640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29c06c360 0x7fb29c1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb29bfff640 1 -- 172.21.15.39:0/1264813032 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c14e1c0 msgr2=0x7fb29c159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb29bfff640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c14e1c0 0x7fb29c159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb29bfff640 1 -- 172.21.15.39:0/1264813032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29c06c360 msgr2=0x7fb29c1500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb29bfff640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29c06c360 0x7fb29c1500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb29bfff640 1 -- 172.21.15.39:0/1264813032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c009420 con 0x7fb29c06b130 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb29bfff640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb29c06b130 0x7fb29c1ede50 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00e980 tx=0x7fb28c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2997fa640 1 -- 172.21.15.39:0/1264813032 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00ca00 con 0x7fb29c06b130 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2997fa640 1 -- 172.21.15.39:0/1264813032 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb28c00cb60 con 0x7fb29c06b130 2024-09-13T17:19:38.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb284009420 con 0x7fb29c06b130 2024-09-13T17:19:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2997fa640 1 -- 172.21.15.39:0/1264813032 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c013470 con 0x7fb29c06b130 2024-09-13T17:19:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c0be590 con 0x7fb29c06b130 2024-09-13T17:19:38.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29c0674c0 con 0x7fb29c06b130 2024-09-13T17:19:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2997fa640 1 -- 172.21.15.39:0/1264813032 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb28c0135d0 con 0x7fb29c06b130 2024-09-13T17:19:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2997fa640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb27c066690 0x7fb27c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb2997fa640 1 -- 172.21.15.39:0/1264813032 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 9215+0+0 (secure 0 0 0) 0x7fb28c085150 con 0x7fb29c06b130 2024-09-13T17:19:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb293fff640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb27c066690 0x7fb27c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.071+0000 7fb293fff640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb27c066690 0x7fb27c068b50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb2840af000 tx=0x7fb284008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:38.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.075+0000 7fb2997fa640 1 -- 172.21.15.39:0/1264813032 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb28c051800 con 0x7fb29c06b130 2024-09-13T17:19:38.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.263+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "b"} v 0) v1 -- 0x7fb29c0c0b40 con 0x7fb29c06b130 2024-09-13T17:19:38.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.283+0000 7fb2997fa640 1 -- 172.21.15.39:0/1264813032 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "b"}]=0 b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v210) v1 ==== 151+0+0 (secure 0 0 0) 0x7fb28c054e20 con 0x7fb29c06b130 2024-09-13T17:19:38.288 INFO:teuthology.orchestra.run.smithi039.stderr:b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:19:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb27c066690 msgr2=0x7fb27c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb27c066690 0x7fb27c068b50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb2840af000 tx=0x7fb284008040 comp rx=0 tx=0).stop 2024-09-13T17:19:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb29c06b130 msgr2=0x7fb29c1ede50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb29c06b130 0x7fb29c1ede50 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00e980 tx=0x7fb28c00ee40 comp rx=0 tx=0).stop 2024-09-13T17:19:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 shutdown_connections 2024-09-13T17:19:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fb27c066690 0x7fb27c068b50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb29c06c360 0x7fb29c1500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fb29c14e1c0 0x7fb29c159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 --2- 172.21.15.39:0/1264813032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb29c06b130 0x7fb29c1ede50 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 >> 172.21.15.39:0/1264813032 conn(0x7fb29c0c5790 msgr2=0x7fb29c06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 shutdown_connections 2024-09-13T17:19:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.287+0000 7fb2a2901640 1 -- 172.21.15.39:0/1264813032 wait complete. 2024-09-13T17:19:38.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm b --yes-i-really-mean-it 2024-09-13T17:19:38.334 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-13T17:19:38.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5ac0a5640 1 Processor -- start 2024-09-13T17:19:38.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5ac0a5640 1 -- start start 2024-09-13T17:19:38.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5ac0a5640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5a40aa2d0 0x7fd5a40aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5ac0a5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd5a40a4d60 0x7fd5a414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5ac0a5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a414f4b0 0x7fd5a4151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5ac0a5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a4155b30 con 0x7fd5a40a4d60 2024-09-13T17:19:38.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5ac0a5640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a4155d20 con 0x7fd5a40aa2d0 2024-09-13T17:19:38.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5ac0a5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40c4750 con 0x7fd5a414f4b0 2024-09-13T17:19:38.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5a9640640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd5a40a4d60 0x7fd5a414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5a9640640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd5a40a4d60 0x7fd5a414ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39648/0 (socket says 172.21.15.39:39648) 2024-09-13T17:19:38.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5a9640640 1 -- 172.21.15.39:0/3431713208 learned_addr learned my addr 172.21.15.39:0/3431713208 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:38.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5aa642640 1 --2- 172.21.15.39:0/3431713208 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a414f4b0 0x7fd5a4151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5a9e41640 1 --2- 172.21.15.39:0/3431713208 >> v2:172.21.15.84:3300/0 conn(0x7fd5a40aa2d0 0x7fd5a40aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5a9640640 1 -- 172.21.15.39:0/3431713208 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a414f4b0 msgr2=0x7fd5a4151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:38.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5a9640640 1 --2- 172.21.15.39:0/3431713208 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a414f4b0 0x7fd5a4151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5a9640640 1 -- 172.21.15.39:0/3431713208 >> v2:172.21.15.84:3300/0 conn(0x7fd5a40aa2d0 msgr2=0x7fd5a40aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:38.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5a9640640 1 --2- 172.21.15.39:0/3431713208 >> v2:172.21.15.84:3300/0 conn(0x7fd5a40aa2d0 0x7fd5a40aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.423+0000 7fd5a9640640 1 -- 172.21.15.39:0/3431713208 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a4155fc0 con 0x7fd5a40a4d60 2024-09-13T17:19:38.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9640640 1 --2- 172.21.15.39:0/3431713208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 0x7fd5a414ef70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd5a406d4d0 tx=0x7fd59401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c903b27b7f6a28e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:38.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a8e3f640 1 -- 172.21.15.39:0/3431713208 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4152a40 con 0x7fd5a40a4d60 2024-09-13T17:19:38.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a8e3f640 1 -- 172.21.15.39:0/3431713208 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd5940a7050 con 0x7fd5a40a4d60 2024-09-13T17:19:38.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a8e3f640 1 -- 172.21.15.39:0/3431713208 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940ab840 con 0x7fd5a40a4d60 2024-09-13T17:19:38.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/3431713208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 msgr2=0x7fd5a414ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:38.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/3431713208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 0x7fd5a414ef70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd5a406d4d0 tx=0x7fd59401c460 comp rx=0 tx=0).stop 2024-09-13T17:19:38.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/3431713208 shutdown_connections 2024-09-13T17:19:38.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/3431713208 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a414f4b0 0x7fd5a4151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/3431713208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 0x7fd5a414ef70 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/3431713208 >> v2:172.21.15.84:3300/0 conn(0x7fd5a40aa2d0 0x7fd5a40aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/3431713208 >> 172.21.15.39:0/3431713208 conn(0x7fd5a40c5790 msgr2=0x7fd5a40c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:38.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/3431713208 shutdown_connections 2024-09-13T17:19:38.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/3431713208 wait complete. 2024-09-13T17:19:38.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 Processor -- start 2024-09-13T17:19:38.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 -- start start 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 0x7fd5a40bed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a40aa2d0 0x7fd5a41eff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a414f4b0 0x7fd5a41f0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a4155400 con 0x7fd5a40a4d60 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a4071fc0 con 0x7fd5a414f4b0 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5ac0a5640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a4071ce0 con 0x7fd5a40aa2d0 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9e41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 0x7fd5a40bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9e41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 0x7fd5a40bed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39662/0 (socket says 172.21.15.39:39662) 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9e41640 1 -- 172.21.15.39:0/4064621883 learned_addr learned my addr 172.21.15.39:0/4064621883 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5aa642640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a414f4b0 0x7fd5a41f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9e41640 1 -- 172.21.15.39:0/4064621883 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a40aa2d0 msgr2=0x7fd5a41eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:38.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9640640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a40aa2d0 0x7fd5a41eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9e41640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a40aa2d0 0x7fd5a41eff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9e41640 1 -- 172.21.15.39:0/4064621883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a414f4b0 msgr2=0x7fd5a41f0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9e41640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a414f4b0 0x7fd5a41f0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9e41640 1 -- 172.21.15.39:0/4064621883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5940094d0 con 0x7fd5a40a4d60 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.427+0000 7fd5a9e41640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 0x7fd5a40bed90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000c7f0 tx=0x7fd5a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd59a7fc640 1 -- 172.21.15.39:0/4064621883 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0004220 con 0x7fd5a40a4d60 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd59a7fc640 1 -- 172.21.15.39:0/4064621883 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd5a0004380 con 0x7fd5a40a4d60 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a40c0130 con 0x7fd5a40a4d60 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd59a7fc640 1 -- 172.21.15.39:0/4064621883 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0003880 con 0x7fd5a40a4d60 2024-09-13T17:19:38.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a40c06f0 con 0x7fd5a40a4d60 2024-09-13T17:19:38.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd59a7fc640 1 -- 172.21.15.39:0/4064621883 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd5a0003bf0 con 0x7fd5a40a4d60 2024-09-13T17:19:38.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a40aa6b0 con 0x7fd5a40a4d60 2024-09-13T17:19:38.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd59a7fc640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd588066690 0x7fd588068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:38.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd5a9640640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd588066690 0x7fd588068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:38.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd59a7fc640 1 -- 172.21.15.39:0/4064621883 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 9301+0+0 (secure 0 0 0) 0x7fd5a00850c0 con 0x7fd5a40a4d60 2024-09-13T17:19:38.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.431+0000 7fd5a9640640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd588066690 0x7fd588068b50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd5940ad000 tx=0x7fd594008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:38.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.435+0000 7fd59a7fc640 1 -- 172.21.15.39:0/4064621883 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd5a00514b0 con 0x7fd5a40a4d60 2024-09-13T17:19:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:38.623+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "b", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd5a415c2e0 con 0x7fd5a40a4d60 2024-09-13T17:19:39.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.287+0000 7fd59a7fc640 1 -- 172.21.15.39:0/4064621883 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "b", "yes_i_really_mean_it": true}]=0 v211) v1 ==== 99+0+0 (secure 0 0 0) 0x7fd5a00533e0 con 0x7fd5a40a4d60 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd588066690 msgr2=0x7fd588068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd588066690 0x7fd588068b50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd5940ad000 tx=0x7fd594008040 comp rx=0 tx=0).stop 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 msgr2=0x7fd5a40bed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 0x7fd5a40bed90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000c7f0 tx=0x7fd5a000ccb0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 shutdown_connections 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7fd588066690 0x7fd588068b50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5a414f4b0 0x7fd5a41f0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7fd5a40aa2d0 0x7fd5a41eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 --2- 172.21.15.39:0/4064621883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a40a4d60 0x7fd5a40bed90 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 >> 172.21.15.39:0/4064621883 conn(0x7fd5a40c5790 msgr2=0x7fd5a40c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:39.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 shutdown_connections 2024-09-13T17:19:39.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.291+0000 7fd5ac0a5640 1 -- 172.21.15.39:0/4064621883 wait complete. 2024-09-13T17:19:39.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm b_metadata b_metadata --yes-i-really-really-mean-it 2024-09-13T17:19:39.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c3cf7640 1 Processor -- start 2024-09-13T17:19:39.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c3cf7640 1 -- start start 2024-09-13T17:19:39.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c3cf7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc0c4920 0x7f07bc153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c3cf7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f07bc0714b0 0x7f07bc154510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c3cf7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07bc0c9c10 con 0x7f07bc151460 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c3cf7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc158100 con 0x7f07bc0714b0 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c3cf7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc0c9e60 con 0x7f07bc0c4920 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c2294640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f07bc0714b0 0x7f07bc154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c1a93640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f07bc151460 0x7f07bc0c4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:49542/0 (socket says 172.21.15.39:49542) 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c1a93640 1 -- 172.21.15.39:0/3633629153 learned_addr learned my addr 172.21.15.39:0/3633629153 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c1292640 1 --2- 172.21.15.39:0/3633629153 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc0c4920 0x7f07bc153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c2294640 1 -- 172.21.15.39:0/3633629153 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc0c4920 msgr2=0x7f07bc153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c2294640 1 --2- 172.21.15.39:0/3633629153 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc0c4920 0x7f07bc153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c2294640 1 -- 172.21.15.39:0/3633629153 >> v1:172.21.15.39:6789/0 conn(0x7f07bc151460 legacy=0x7f07bc0c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:39.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.483+0000 7f07c2294640 1 -- 172.21.15.39:0/3633629153 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc0ca140 con 0x7f07bc0714b0 2024-09-13T17:19:39.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c2294640 1 --2- 172.21.15.39:0/3633629153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0714b0 0x7f07bc154510 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009d90 tx=0x7f07b401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b7065d1b2b2cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:39.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c0a91640 1 -- 172.21.15.39:0/3633629153 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b40a58b0 con 0x7f07bc0714b0 2024-09-13T17:19:39.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c0a91640 1 -- 172.21.15.39:0/3633629153 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f07b40a5a10 con 0x7f07bc0714b0 2024-09-13T17:19:39.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c0a91640 1 -- 172.21.15.39:0/3633629153 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b40a5dc0 con 0x7f07bc0714b0 2024-09-13T17:19:39.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 -- 172.21.15.39:0/3633629153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0714b0 msgr2=0x7f07bc154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:39.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 --2- 172.21.15.39:0/3633629153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0714b0 0x7f07bc154510 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009d90 tx=0x7f07b401c8e0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 -- 172.21.15.39:0/3633629153 shutdown_connections 2024-09-13T17:19:39.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 --2- 172.21.15.39:0/3633629153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0714b0 0x7f07bc154510 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 --2- 172.21.15.39:0/3633629153 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc0c4920 0x7f07bc153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 -- 172.21.15.39:0/3633629153 >> 172.21.15.39:0/3633629153 conn(0x7f07bc069fe0 msgr2=0x7f07bc06a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:39.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 -- 172.21.15.39:0/3633629153 shutdown_connections 2024-09-13T17:19:39.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 -- 172.21.15.39:0/3633629153 wait complete. 2024-09-13T17:19:39.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 Processor -- start 2024-09-13T17:19:39.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.487+0000 7f07c3cf7640 1 -- start start 2024-09-13T17:19:39.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c3cf7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07bc0714b0 0x7f07bc1f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c3cf7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0c4920 0x7f07bc0bdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c3cf7640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc151460 0x7f07bc1f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c3cf7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc14dc70 con 0x7f07bc0c4920 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c3cf7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc0ac2a0 con 0x7f07bc0714b0 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c3cf7640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc14e180 con 0x7f07bc151460 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1292640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0c4920 0x7f07bc0bdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1292640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0c4920 0x7f07bc0bdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39674/0 (socket says 172.21.15.39:39674) 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1292640 1 -- 172.21.15.39:0/1193465875 learned_addr learned my addr 172.21.15.39:0/1193465875 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1a93640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07bc0714b0 0x7f07bc1f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c2294640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc151460 0x7f07bc1f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1292640 1 -- 172.21.15.39:0/1193465875 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc151460 msgr2=0x7f07bc1f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1292640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc151460 0x7f07bc1f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1292640 1 -- 172.21.15.39:0/1193465875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07bc0714b0 msgr2=0x7f07bc1f05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:39.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1292640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07bc0714b0 0x7f07bc1f05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1292640 1 -- 172.21.15.39:0/1193465875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b4009a70 con 0x7f07bc0c4920 2024-09-13T17:19:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1292640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0c4920 0x7f07bc0bdda0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f07b800d800 tx=0x7f07b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07b27fc640 1 -- 172.21.15.39:0/1193465875 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8004030 con 0x7f07bc0c4920 2024-09-13T17:19:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07b27fc640 1 -- 172.21.15.39:0/1193465875 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f07b8004190 con 0x7f07bc0c4920 2024-09-13T17:19:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07bc14dea0 con 0x7f07bc0c4920 2024-09-13T17:19:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07b27fc640 1 -- 172.21.15.39:0/1193465875 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800b6c0 con 0x7f07bc0c4920 2024-09-13T17:19:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc0bf790 con 0x7f07bc0c4920 2024-09-13T17:19:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc0bf920 con 0x7f07bc0c4920 2024-09-13T17:19:39.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07b27fc640 1 -- 172.21.15.39:0/1193465875 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f07b800b820 con 0x7f07bc0c4920 2024-09-13T17:19:39.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07b27fc640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f07a0066690 0x7f07a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:39.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07b27fc640 1 -- 172.21.15.39:0/1193465875 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 9301+0+0 (secure 0 0 0) 0x7f07b8085200 con 0x7f07bc0c4920 2024-09-13T17:19:39.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1a93640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f07a0066690 0x7f07a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:39.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.491+0000 7f07c1a93640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f07a0066690 0x7f07a0068b50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005890 tx=0x7f07ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:39.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.495+0000 7f07b27fc640 1 -- 172.21.15.39:0/1193465875 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f07b8051710 con 0x7f07bc0c4920 2024-09-13T17:19:39.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:39.659+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "b_metadata", "pool2": "b_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f07bc0c4210 con 0x7f07bc0c4920 2024-09-13T17:19:40.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.299+0000 7f07b27fc640 1 -- 172.21.15.39:0/1193465875 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "b_metadata", "pool2": "b_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'b_metadata' removed v69) v1 ==== 166+0+0 (secure 0 0 0) 0x7f07b8054d30 con 0x7f07bc0c4920 2024-09-13T17:19:40.304 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'b_metadata' removed 2024-09-13T17:19:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f07a0066690 msgr2=0x7f07a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f07a0066690 0x7f07a0068b50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005890 tx=0x7f07ac009040 comp rx=0 tx=0).stop 2024-09-13T17:19:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0c4920 msgr2=0x7f07bc0bdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0c4920 0x7f07bc0bdda0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f07b800d800 tx=0x7f07b800dcc0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 shutdown_connections 2024-09-13T17:19:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f07a0066690 0x7f07a0068b50 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f07bc151460 0x7f07bc1f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07bc0c4920 0x7f07bc0bdda0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 --2- 172.21.15.39:0/1193465875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07bc0714b0 0x7f07bc1f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 >> 172.21.15.39:0/1193465875 conn(0x7f07bc069fe0 msgr2=0x7f07bc14f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 shutdown_connections 2024-09-13T17:19:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.303+0000 7f07c3cf7640 1 -- 172.21.15.39:0/1193465875 wait complete. 2024-09-13T17:19:40.329 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm b_data b_data --yes-i-really-really-mean-it 2024-09-13T17:19:40.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58b1459640 1 Processor -- start 2024-09-13T17:19:40.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58b1459640 1 -- start start 2024-09-13T17:19:40.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58b1459640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f58ac0aa2d0 0x7f58ac0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:40.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58b1459640 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58ac0a4d60 0x7f58ac14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58b1459640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac155d20 con 0x7f58ac14f4b0 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58b1459640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac155b30 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58b1459640 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac0c4750 con 0x7f58ac0a4d60 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58aaffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f58ac0aa2d0 0x7f58ac0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58ab7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f58ac14f4b0 0x7f58ac151cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:35408/0 (socket says 172.21.15.39:35408) 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58aaffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f58ac0aa2d0 0x7f58ac0aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46000/0 (socket says 172.21.15.39:46000) 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58ab7fe640 1 -- 172.21.15.39:0/2059127852 learned_addr learned my addr 172.21.15.39:0/2059127852 (peer_addr_for_me v1:172.21.15.39:0/0) 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58aaffd640 1 -- 172.21.15.39:0/2059127852 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0a4d60 msgr2=0x7f58ac14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58aa7fc640 1 --2- 172.21.15.39:0/2059127852 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0a4d60 0x7f58ac14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58aaffd640 1 --2- 172.21.15.39:0/2059127852 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0a4d60 0x7f58ac14ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58aaffd640 1 -- 172.21.15.39:0/2059127852 >> v1:172.21.15.39:6789/0 conn(0x7f58ac14f4b0 legacy=0x7f58ac151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58a9ffb640 1 -- 172.21.15.39:0/2059127852 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165769780 0 0) 0x7f58ac155d20 con 0x7f58ac14f4b0 2024-09-13T17:19:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.451+0000 7f58aaffd640 1 -- 172.21.15.39:0/2059127852 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac155fc0 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58aaffd640 1 --2- 172.21.15.39:0/2059127852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 0x7f58ac0aa6b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0c5340 tx=0x7f589401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef86ae322815b848 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58a9ffb640 1 -- 172.21.15.39:0/2059127852 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58940ab600 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58a9ffb640 1 -- 172.21.15.39:0/2059127852 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f58940a7050 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58a9ffb640 1 -- 172.21.15.39:0/2059127852 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58940ab9a0 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 -- 172.21.15.39:0/2059127852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 msgr2=0x7f58ac0aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 --2- 172.21.15.39:0/2059127852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 0x7f58ac0aa6b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0c5340 tx=0x7f589401c3c0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 -- 172.21.15.39:0/2059127852 shutdown_connections 2024-09-13T17:19:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 --2- 172.21.15.39:0/2059127852 >> v2:172.21.15.84:3300/0 conn(0x7f58ac0a4d60 0x7f58ac14ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 --2- 172.21.15.39:0/2059127852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 0x7f58ac0aa6b0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 -- 172.21.15.39:0/2059127852 >> 172.21.15.39:0/2059127852 conn(0x7f58ac0c5790 msgr2=0x7f58ac0c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 -- 172.21.15.39:0/2059127852 shutdown_connections 2024-09-13T17:19:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 -- 172.21.15.39:0/2059127852 wait complete. 2024-09-13T17:19:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 Processor -- start 2024-09-13T17:19:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.455+0000 7f58b1459640 1 -- start start 2024-09-13T17:19:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58b1459640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac0a4d60 0x7f58ac0c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58b1459640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 0x7f58ac0bee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58b1459640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58ac14f4b0 0x7f58ac1ec240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58b1459640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0c0410 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58b1459640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0c0570 con 0x7f58ac14f4b0 2024-09-13T17:19:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58b1459640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0c2cf0 con 0x7f58ac0a4d60 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 0x7f58ac0bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aa7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 0x7f58ac0bee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46004/0 (socket says 172.21.15.39:46004) 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aa7fc640 1 -- 172.21.15.39:0/1063376947 learned_addr learned my addr 172.21.15.39:0/1063376947 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58ab7fe640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58ac14f4b0 0x7f58ac1ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aaffd640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac0a4d60 0x7f58ac0c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aa7fc640 1 -- 172.21.15.39:0/1063376947 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac0a4d60 msgr2=0x7f58ac0c0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aa7fc640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac0a4d60 0x7f58ac0c0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aa7fc640 1 -- 172.21.15.39:0/1063376947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58ac14f4b0 msgr2=0x7f58ac1ec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aa7fc640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58ac14f4b0 0x7f58ac1ec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aa7fc640 1 -- 172.21.15.39:0/1063376947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5894009420 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aa7fc640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 0x7f58ac0bee20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c7f0 tx=0x7f589c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58a37fe640 1 -- 172.21.15.39:0/1063376947 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c004220 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58a37fe640 1 -- 172.21.15.39:0/1063376947 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f589c0043a0 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac0c2f70 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58a37fe640 1 -- 172.21.15.39:0/1063376947 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c004510 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac06cf70 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac06cdd0 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58a37fe640 1 -- 172.21.15.39:0/1063376947 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f589c003880 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58a37fe640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f588c066690 0x7f588c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58a37fe640 1 -- 172.21.15.39:0/1063376947 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 8873+0+0 (secure 0 0 0) 0x7f589c084bf0 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.459+0000 7f58aaffd640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f588c066690 0x7f588c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.463+0000 7f58aaffd640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f588c066690 0x7f588c068b50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f58940ad000 tx=0x7f58940a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.463+0000 7f58a37fe640 1 -- 172.21.15.39:0/1063376947 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f58ac06cdd0 con 0x7f58ac0aa2d0 2024-09-13T17:19:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:40.623+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "b_data", "pool2": "b_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f58ac06cdd0 con 0x7f58ac0aa2d0 2024-09-13T17:19:41.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.319+0000 7f58a37fe640 1 -- 172.21.15.39:0/1063376947 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "b_data", "pool2": "b_data", "yes_i_really_really_mean_it": true}]=0 pool 'b_data' removed v70) v1 ==== 154+0+0 (secure 0 0 0) 0x7f58ac06cdd0 con 0x7f58ac0aa2d0 2024-09-13T17:19:41.326 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'b_data' removed 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f588c066690 msgr2=0x7f588c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f588c066690 0x7f588c068b50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f58940ad000 tx=0x7f58940a6000 comp rx=0 tx=0).stop 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 msgr2=0x7f58ac0bee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 0x7f58ac0bee20 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c7f0 tx=0x7f589c00ccb0 comp rx=0 tx=0).stop 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 shutdown_connections 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f588c066690 0x7f588c068b50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58ac14f4b0 0x7f58ac1ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58ac0aa2d0 0x7f58ac0bee20 secure :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c7f0 tx=0x7f589c00ccb0 comp rx=0 tx=0).stop 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 --2- 172.21.15.39:0/1063376947 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f58ac0a4d60 0x7f58ac0c0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:41.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 >> 172.21.15.39:0/1063376947 conn(0x7f58ac0c5790 msgr2=0x7f58ac1510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:41.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 shutdown_connections 2024-09-13T17:19:41.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:41.327+0000 7f58b1459640 1 -- 172.21.15.39:0/1063376947 wait complete. 2024-09-13T17:19:41.365 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T17:19:41.365 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:19:41.365 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:19:41.379 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:19:41.379 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T17:19:41.379 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:19:41.436 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T17:19:41.436 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:19:41.436 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:19:41.448 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:19:41.448 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T17:19:41.449 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:19:41.501 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T17:19:41.501 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T17:19:41.502 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T17:19:41.502 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:19:43.965 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:19:44.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bd1deb640 1 Processor -- start 2024-09-13T17:19:44.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bd1deb640 1 -- start start 2024-09-13T17:19:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bd1deb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8bcc0c21d0 0x7f8bcc0aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bd1deb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 0x7f8bcc14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bd1deb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0c3ae0 con 0x7f8bcc06aac0 2024-09-13T17:19:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bd1deb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bcc0ab340 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bd1deb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0abb30 con 0x7f8bcc14b850 2024-09-13T17:19:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bcb7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8bcc0c21d0 0x7f8bcc0aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bcb7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8bcc0c21d0 0x7f8bcc0aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46012/0 (socket says 172.21.15.39:46012) 2024-09-13T17:19:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bcb7fe640 1 -- 172.21.15.39:0/4172029792 learned_addr learned my addr 172.21.15.39:0/4172029792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bcaffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8bcc06aac0 0x7f8bcc151c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:35416/0 (socket says 172.21.15.39:35416) 2024-09-13T17:19:44.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bcbfff640 1 --2- 172.21.15.39:0/4172029792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 0x7f8bcc14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:44.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bcb7fe640 1 -- 172.21.15.39:0/4172029792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 msgr2=0x7f8bcc14bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:19:44.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bcb7fe640 1 --2- 172.21.15.39:0/4172029792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 0x7f8bcc14bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bcb7fe640 1 -- 172.21.15.39:0/4172029792 >> v1:172.21.15.39:6789/0 conn(0x7f8bcc06aac0 legacy=0x7f8bcc151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:44.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bca7fc640 1 -- 172.21.15.39:0/4172029792 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378405915 0 0) 0x7f8bcc0c3ae0 con 0x7f8bcc06aac0 2024-09-13T17:19:44.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.095+0000 7f8bcb7fe640 1 -- 172.21.15.39:0/4172029792 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc0abe10 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bcb7fe640 1 --2- 172.21.15.39:0/4172029792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 0x7f8bcc0aae00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009870 tx=0x7f8bbc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88203199d2b23a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:44.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bca7fc640 1 -- 172.21.15.39:0/4172029792 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0a6040 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bca7fc640 1 -- 172.21.15.39:0/4172029792 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8bbc0aa600 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bca7fc640 1 -- 172.21.15.39:0/4172029792 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0aa9b0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 -- 172.21.15.39:0/4172029792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 msgr2=0x7f8bcc0aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:44.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 --2- 172.21.15.39:0/4172029792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 0x7f8bcc0aae00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009870 tx=0x7f8bbc01c4a0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 -- 172.21.15.39:0/4172029792 shutdown_connections 2024-09-13T17:19:44.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 --2- 172.21.15.39:0/4172029792 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 0x7f8bcc14bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 --2- 172.21.15.39:0/4172029792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 0x7f8bcc0aae00 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 -- 172.21.15.39:0/4172029792 >> 172.21.15.39:0/4172029792 conn(0x7f8bcc069e10 msgr2=0x7f8bcc06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:44.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 -- 172.21.15.39:0/4172029792 shutdown_connections 2024-09-13T17:19:44.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 -- 172.21.15.39:0/4172029792 wait complete. 2024-09-13T17:19:44.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 Processor -- start 2024-09-13T17:19:44.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 -- start start 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bcc06aac0 0x7f8bcc14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 0x7f8bcc145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 --2- >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 0x7f8bcc145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc15add0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc1ed9b0 con 0x7f8bcc06aac0 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bd1deb640 1 -- --> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc1eddd0 con 0x7f8bcc14b850 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bcaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 0x7f8bcc145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bcaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 0x7f8bcc145a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46014/0 (socket says 172.21.15.39:46014) 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.099+0000 7f8bcaffd640 1 -- 172.21.15.39:0/3055802820 learned_addr learned my addr 172.21.15.39:0/3055802820 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcb7fe640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bcc06aac0 0x7f8bcc14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:44.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcbfff640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 0x7f8bcc145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcaffd640 1 -- 172.21.15.39:0/3055802820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 msgr2=0x7f8bcc145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcaffd640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 0x7f8bcc145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcaffd640 1 -- 172.21.15.39:0/3055802820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bcc06aac0 msgr2=0x7f8bcc14ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcaffd640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bcc06aac0 0x7f8bcc14ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcaffd640 1 -- 172.21.15.39:0/3055802820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc0094d0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcbfff640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 0x7f8bcc145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcaffd640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 0x7f8bcc145a00 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c4b0 tx=0x7f8bb400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bc3fff640 1 -- 172.21.15.39:0/3055802820 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40074e0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bc3fff640 1 -- 172.21.15.39:0/3055802820 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8bb4007660 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc14e140 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bc3fff640 1 -- 172.21.15.39:0/3055802820 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4004a50 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc1ed2f0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc1edaf0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bc3fff640 1 -- 172.21.15.39:0/3055802820 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8bb4004bb0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bc3fff640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8bac066690 0x7f8bac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:19:44.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bc3fff640 1 -- 172.21.15.39:0/3055802820 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8bb4083d40 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcb7fe640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8bac066690 0x7f8bac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:19:44.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.103+0000 7f8bcb7fe640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8bac066690 0x7f8bac068b50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0ae000 tx=0x7f8bbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:19:44.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.107+0000 7f8bc3fff640 1 -- 172.21.15.39:0/3055802820 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8bcc1edaf0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.307+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bcc1edaf0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.307+0000 7f8bc3fff640 1 -- 172.21.15.39:0/3055802820 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+3528 (secure 0 0 0) 0x7f8bcc1edaf0 con 0x7f8bcc0c21d0 2024-09-13T17:19:44.314 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-13T17:19:44.314 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":211,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6227,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6833/641053781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":641053781},{"type":"v1","addr":"172.21.15.84:6833","nonce":641053781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":151},{"gid":6932,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1521872458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1521872458},{"type":"v1","addr":"172.21.15.84:6835","nonce":1521872458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428801917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428801917},{"type":"v1","addr":"172.21.15.39:6835","nonce":428801917}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":207},{"gid":8916,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/4045456601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":4045456601},{"type":"v1","addr":"172.21.15.39:6837","nonce":4045456601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":211}],"filesystems":[]} 2024-09-13T17:19:44.314 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 211 2024-09-13T17:19:44.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8bac066690 msgr2=0x7f8bac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:44.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8bac066690 0x7f8bac068b50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0ae000 tx=0x7f8bbc008040 comp rx=0 tx=0).stop 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 msgr2=0x7f8bcc145a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 0x7f8bcc145a00 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c4b0 tx=0x7f8bb400c970 comp rx=0 tx=0).stop 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 shutdown_connections 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:6832/1271482023,v1:172.21.15.39:6833/1271482023] conn(0x7f8bac066690 0x7f8bac068b50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.84:3301/0,v1:172.21.15.84:6790/0] conn(0x7f8bcc14b850 0x7f8bcc145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bcc0c21d0 0x7f8bcc145a00 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 --2- 172.21.15.39:0/3055802820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bcc06aac0 0x7f8bcc14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 >> 172.21.15.39:0/3055802820 conn(0x7f8bcc069e10 msgr2=0x7f8bcc0c4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 shutdown_connections 2024-09-13T17:19:44.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-13T17:19:44.311+0000 7f8bd1deb640 1 -- 172.21.15.39:0/3055802820 wait complete. 2024-09-13T17:19:44.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-13T17:19:47.604 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T17:19:47.604 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T17:19:47.604 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:19:50.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-13T17:19:50.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-13T17:19:53.706 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T17:19:53.707 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T17:19:53.707 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:19:53.778 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T17:19:53.778 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T17:19:53.778 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:19:56.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T17:19:56.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-13T17:19:59.880 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T17:19:59.881 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T17:19:59.881 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:19:59.980 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T17:19:59.980 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T17:19:59.981 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:20:02.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-13T17:20:02.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T17:20:06.083 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T17:20:06.083 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T17:20:06.083 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:20:06.169 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T17:20:06.170 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T17:20:06.170 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:20:07.161 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:20:07.163 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:20:07.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-13T17:20:07.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-13T17:20:12.271 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T17:20:12.272 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T17:20:12.272 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T17:20:12.272 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:20:12.311 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T17:20:12.311 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T17:20:12.311 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:20:12.346 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T17:20:12.347 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T17:20:12.347 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T17:20:12.347 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:20:12.376 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T17:20:12.377 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T17:20:12.377 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:20:12.409 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T17:20:12.410 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T17:20:12.410 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:20:12.471 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T17:20:12.472 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T17:20:12.472 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v 'Replacing daemon mds' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-13T17:20:12.548 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T17:20:12.548 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T17:20:12.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-13T17:20:12.828 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T17:20:12.829 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T17:20:13.025 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T17:20:13.025 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T17:20:13.211 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T17:20:13.211 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T17:20:13.408 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T17:20:13.408 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T17:20:13.696 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T17:20:13.696 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T17:20:13.930 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T17:20:13.931 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T17:20:14.139 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T17:20:14.140 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T17:20:14.364 INFO:tasks.ceph:Archiving mon data... 2024-09-13T17:20:14.366 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830/data/mon.a.tgz 2024-09-13T17:20:14.366 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-13T17:20:14.386 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.6f9lKc4Whi 2024-09-13T17:20:14.386 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.6f9lKc4Whi 2024-09-13T17:20:14.556 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.6f9lKc4Whi 2024-09-13T17:20:14.659 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.6f9lKc4Whi is 455KB 2024-09-13T17:20:14.720 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.6f9lKc4Whi 2024-09-13T17:20:14.726 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830/data/mon.b.tgz 2024-09-13T17:20:14.726 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-09-13T17:20:14.742 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.HFHom1jbij 2024-09-13T17:20:14.742 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.HFHom1jbij 2024-09-13T17:20:14.908 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.HFHom1jbij 2024-09-13T17:20:15.003 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.HFHom1jbij is 459KB 2024-09-13T17:20:15.063 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.HFHom1jbij 2024-09-13T17:20:15.069 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830/data/mon.c.tgz 2024-09-13T17:20:15.069 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-09-13T17:20:15.118 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.TMp9RBSq0W 2024-09-13T17:20:15.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.TMp9RBSq0W 2024-09-13T17:20:15.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.TMp9RBSq0W 2024-09-13T17:20:15.330 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.TMp9RBSq0W is 459KB 2024-09-13T17:20:15.391 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.TMp9RBSq0W 2024-09-13T17:20:15.397 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T17:20:15.398 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T17:20:15.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T17:20:15.454 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T17:20:15.454 DEBUG:teuthology.orchestra.run.smithi039:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T17:20:15.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T17:20:15.484 INFO:teuthology.orchestra.run.smithi039.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T17:20:15.530 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T17:20:15.533 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T17:20:15.535 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830/remote/smithi039/crash 2024-09-13T17:20:15.536 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T17:20:15.549 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830/remote/smithi084/crash 2024-09-13T17:20:15.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T17:20:15.587 INFO:tasks.ceph:Shutting down logrotate 2024-09-13T17:20:15.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:20:15.596 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:20:15.672 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T17:20:15.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T17:20:15.725 INFO:tasks.ceph:Compressing logs... 2024-09-13T17:20:15.725 DEBUG:teuthology.orchestra.run.smithi039:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:20:15.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:25:15.406 INFO:tasks.ceph:Archiving logs... 2024-09-13T17:25:15.407 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830/remote/smithi039/log 2024-09-13T17:25:15.408 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:25:38.357 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830/remote/smithi084/log 2024-09-13T17:25:38.358 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:26:01.022 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T17:26:01.034 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T17:26:01.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:26:01.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:26:01.129 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T17:26:01.137 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T17:26:01.138 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:26:01.140 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:26:01.186 INFO:teuthology.orchestra.run.smithi039.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T17:26:01.186 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================== 2024-09-13T17:26:01.186 INFO:teuthology.orchestra.run.smithi039.stdout:*hv01.front.sepi 67.205.162.81 3 u 20 64 377 0.108 -1.125 0.197 2024-09-13T17:26:01.186 INFO:teuthology.orchestra.run.smithi039.stdout:+hv02.front.sepi 63.231.80.2 3 u 6 64 377 0.073 -2.343 0.270 2024-09-13T17:26:01.186 INFO:teuthology.orchestra.run.smithi039.stdout:+hv03.front.sepi 74.6.168.72 3 u 5 64 377 0.103 -0.862 0.317 2024-09-13T17:26:01.186 INFO:teuthology.orchestra.run.smithi039.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-09-13T17:26:01.239 INFO:teuthology.orchestra.run.smithi084.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T17:26:01.240 INFO:teuthology.orchestra.run.smithi084.stdout:============================================================================== 2024-09-13T17:26:01.240 INFO:teuthology.orchestra.run.smithi084.stdout:*hv01.front.sepi 67.205.162.81 3 u 63 64 377 0.084 -1.114 1.329 2024-09-13T17:26:01.240 INFO:teuthology.orchestra.run.smithi084.stdout:+hv02.front.sepi 63.231.80.2 3 u 63 64 377 0.082 -3.397 0.584 2024-09-13T17:26:01.240 INFO:teuthology.orchestra.run.smithi084.stdout:+hv03.front.sepi 74.6.168.72 3 u 1 64 377 0.104 -1.101 0.684 2024-09-13T17:26:01.240 INFO:teuthology.orchestra.run.smithi084.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-09-13T17:26:01.241 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T17:26:01.253 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T17:26:01.253 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T17:26:01.267 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T17:26:01.278 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T17:26:01.292 INFO:teuthology.task.internal:Duration was 2965.248540 seconds 2024-09-13T17:26:01.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T17:26:01.308 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T17:26:01.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:26:01.313 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:26:01.474 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T17:26:01.475 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-09-13T17:26:01.475 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:26:01.486 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2024-09-13T17:26:01.486 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:26:01.497 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T17:26:01.497 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:26:01.535 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:26:01.580 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T17:26:01.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:26:01.619 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:26:01.807 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T17:26:01.817 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T17:26:01.817 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:26:01.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:26:01.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T17:26:01.892 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:26:01.894 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:26:01.916 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-09-13T17:26:01.945 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2024-09-13T17:26:01.970 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:26:01.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:26:01.978 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:26:02.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:26:02.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T17:26:02.028 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T17:26:02.030 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830/remote/smithi039 2024-09-13T17:26:02.030 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:26:02.052 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903830/remote/smithi084 2024-09-13T17:26:02.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:26:02.083 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T17:26:02.083 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:26:02.091 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:26:02.131 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T17:26:02.142 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T17:26:02.142 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T17:26:02.209 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T17:26:02.210 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:26:02.212 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:26:02.217 INFO:teuthology.orchestra.run.smithi039.stdout: 658464 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:26 /home/ubuntu/cephtest 2024-09-13T17:26:02.219 INFO:teuthology.orchestra.run.smithi084.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:26 /home/ubuntu/cephtest 2024-09-13T17:26:02.220 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T17:26:02.230 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T17:26:02.261 INFO:teuthology.run:Summary data: description: fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} duration: 2965.248539686203 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T17:26:02.261 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:26:02.371 INFO:teuthology.run:pass